Patch Compliance Is 94 Percent. How Do You Know?

Every security leader has lived through this moment. The CISO stands up at the quarterly review, presents a clean slide that says patch compliance is at 94 percent, and the audit committee nods politely. Then someone asks the question that breaks the room: how do you know? Not whether the number is high enough, but whether it is the same number the team would produce tomorrow, or next week, or after the analyst who built the query has moved on. Most reports cannot survive that question. They were never built to.

Deterministic vs. Best-Effort: What the Words Actually Mean

The distinction we want every security team to internalize is the one between deterministic and best-effort metrics. A deterministic metric is computed the same way every time, from the same underlying data, by the same documented logic. Feed it the same inputs in January or in July, run it on a laptop in Berlin or a server in São Paulo, and it returns the same answer. A best-effort metric is whatever the current analyst’s query produced against whatever copy of the data they happened to pull on a Tuesday morning. The slide looks identical. The defensibility is not.

Why This Question Has Gotten Harder

The reason this matters now, and did not matter as much five years ago, is that the audience for security metrics has changed. Boards used to receive narrative updates. Regulators used to accept attestation. Today, both want evidence that the measurement environment is disciplined and that the measurements themselves are governed. DORA in the EU, the SEC cyber disclosure rules in the US, and NIS2 across the broader European market, along with the maturing expectations under frameworks like NIST CSF 2.0, all converge on the same idea: if a number is material to a decision, the process that produced it must be reproducible. “Trust us, we ran it” is no longer a defensible answer. Reproducibility is.

The Gap in Practice

Consider the practical gap. A best-effort mean time to remediate is often a SQL query saved in someone’s notebook, joining a vulnerability scanner export against a ticketing system, with manual exclusions for assets the analyst decided were out of scope that week. Six months later, that analyst has been promoted, the scanner has changed its CVSS scoring, and the ticket schema has a new field. The query still runs. It just no longer means what it meant. The chart on the slide is unchanged. The underlying assertion has quietly drifted into fiction.

Treating Computation as Code

A deterministic metric closes that gap by treating computation as code rather than craft. The data sources are versioned. The transformation logic lives in a reviewed, version-managed pipeline. The exclusions, thresholds, and definitions are encoded as parameters, not as the unwritten judgment of whoever happened to build it. When the regulator asks how the number was produced, the answer is not a person but a pipeline, and that pipeline can be re-executed against the same inputs to produce the same output, on demand, in front of anyone.

The Vocabulary of Defensible Reporting

This is also where the vocabulary becomes important in governance conversations. Words like baseline, definition, lineage, and re-derivation are no longer the territory of data engineering teams. They belong in the security leader’s working vocabulary because they are the language auditors are already using. When a security KPI cannot be re-derived by a stranger six months from now, working only from documentation and source data, it is not a metric. It is a guess with a chart on top. The faster a team accepts that framing, the faster their CISO reporting graduates from theatre to evidence. This is precisely why security needs a measurement layer of its own: not another viewer, but a layer that owns the definitions, the cadence, and the audit trail.

We built Metric Maestro because we believe security numbers should survive board questions, audit questions, and the inevitable change of personnel. If your reports produce different answers depending on who runs them, that is the conversation we want to have. Reach out, and let us show you what reproducible security measurement looks like.