Metric drift is the gradual movement of a metric’s value over time. Unlike definition drift, where the formula changes and the world may not, metric drift is the value moving while the formula stays fixed.
The distinction matters because the two look identical on a chart and demand completely different responses.
The world changed. The estate grew, the threat environment shifted, a control degraded. This is the case where the metric is doing its job.
The population changed. An acquisition, a divestment, a reclassification, a new region coming into scope. The formula held, the underlying set did not, and the movement is partly arithmetic. See baseline.
The data changed. A collector started failing silently. A source system began truncating a field. An integration broke in a way that returns fewer rows rather than an error. The world and the population are unchanged and the number moved anyway.
The third is the dangerous one, because it presents as good news roughly half the time and nobody investigates good news.
You cannot, from the value alone. This is the whole argument for retaining source records rather than only computed results.
With the underlying records kept, a step in a series can be checked directly. Did the denominator population change size. Did one source contribute far fewer records than usual. Did the formula version change on that date.
Without them, the investigation is a reconstruction from memory and export files, and it usually ends in a plausible story rather than an answer.
Monitor record volume per source alongside the metric itself. A metric that improves while its inputs thin out is a data problem until proven otherwise.
Flag population size as its own series. Coverage ratios move when the denominator moves, and the denominator moving is often the real story.
Treat a step change as a question by default. Gradual movement is usually the world. Steps are usually the measurement.
From the blog