Goodhart’s Law holds that when a measure becomes a target, it ceases to be a good measure. The economist Charles Goodhart made the original observation about monetary policy in 1975. The version most people quote is Marilyn Strathern’s 1997 phrasing, which is snappier and has outlived the original.
A metric is a proxy. It stands in for something you actually care about but cannot observe directly.
The proxy holds as long as nobody is optimizing it. The moment it carries consequences, people optimize the proxy rather than the thing behind it, and the two come apart. This is not usually cynical. It is what happens when you tell a competent team what you will judge them on.
Set a target on mean time to close alerts, and closure times improve. Some of that is better triage. Some of it is alerts closed as benign that deserved another ten minutes.
Set a target on phishing click rates, and click rates fall. Some of that is a workforce that learned something. Some of it is simulations quietly made easier, or a template that half the office warned each other about on Slack.
Set a target on the count of open critical findings, and the count drops. Some of that is remediation. Some of it is severity reclassification.
In each case the number improves and the underlying condition may not have moved at all.
Pair every target with a counter-metric that would degrade if the first one were being gamed. Closure time pairs with reopened case rate. Phishing click rate pairs with report rate. Open critical findings pairs with severity downgrade volume.
Rotate what you put a target on, and keep measuring the rest without targets attached.
Keep a full record of how each number was produced. Most gaming shows up as a change in method rather than a change in the world, and it is invisible unless the method is recorded alongside the value. See definition drift.
Treat a suspiciously clean trend as a question, not an achievement.
From the blog