Measurement maturity describes how far an organization has progressed from ad hoc counting toward measurement it can defend, repeat and steer by. It is not the same as security maturity. A well-defended organization can have poor measurement, and a poorly defended one can measure itself accurately.
At the first stage, numbers are produced on request. Somebody asks, an analyst pulls an export, a figure appears in a slide. Nothing is retained, and the next request produces a slightly different number.
At the second stage, a set of metrics is defined and reported on a cycle. Definitions exist, often in a document nobody opens. Production is manual and takes days.
At the third stage, the numbers are reproducible. Definitions are versioned, computation is automated where the data allows, and a figure from two years ago can be explained.
At the fourth stage, measurement drives decisions before they are forced. Thresholds trigger action, targets are agreed with the business, and the reporting pack is drawn from a record that already exists.
Almost everyone stalls between two and three, and the reason is consistent. Stage two can be reached with spreadsheets and a diligent analyst. Stage three cannot.
Reproducibility requires that the source records be kept, the formula be versioned, and the computation be repeatable without the person who invented it. A spreadsheet fails all three the first time it is copied.
So the programme looks functional for years. Reports go out on time. Then a regulator asks for the basis of a figure from eighteen months ago, or a vendor gets replaced and the trend breaks, and it becomes clear the second stage was never a foundation.
Can you produce a figure from two years ago and explain how it was calculated?
If the analyst who owns the reporting pack left tomorrow, what would break?
When your last security tool was replaced, what happened to the metrics that depended on it?
How long does the quarterly pack take to assemble, and how much of that is gathering rather than thinking?
Slow, uncomfortable answers to these are the actual maturity assessment.
From the blog