Backfill is the recomputation of historical metric values, usually after a correction, a definition change, or the arrival of data that was missing at the time.
A collector was broken for three weeks and the gap has now been filled from the source.
A definition was corrected and the history needs to reflect the corrected formula.
A new source came online holding historical data that improves earlier periods.
An error in normalization was found and the affected records were remapped.
The source records for the affected period, retained. This is the constraint that decides whether backfill is possible at all.
If only the computed results were stored, there is nothing to recompute from. The history is fixed at whatever it was when it was first calculated, correct or not.
A defined horizon. How far back recomputation goes should be a policy rather than a decision made under pressure. Common practice is to allow unrestricted backfill inside a recent window, and to treat anything beyond it as a restatement requiring approval.
A backfilled value must be distinguishable from an originally computed one.
Silently overwriting history is how a measurement record loses its evidential value. Someone reported a figure to a committee in March. If the stored value for March later changes and nothing records that it changed, the pack and the record disagree, and there is no way to explain why.
Store the original value, the corrected value, the reason and the date. Reporting can show the corrected series. The audit trail keeps both.
These are the ones to be careful with. Recomputing history under a new definition produces a clean comparable series, which is genuinely useful, and also silently changes numbers that people have already acted on.
Do it, but mark the transition on the series and note it in the accompanying commentary. A board that sees a figure move between two meetings without explanation will ask, and the answer needs to be ready rather than researched.
From the blog