Control testing is the procedure used to establish whether a control is operating as designed. The method chosen determines how much the result is worth, and the weakest methods are the most common.
Inquiry means asking the control owner whether the control is operating. It is the cheapest and proves nothing on its own.
Observation means watching the control being performed. Better, and subject to the obvious problem that people perform differently when observed.
Inspection means examining evidence that the control operated. This is where most audit testing sits.
Reperformance means independently carrying out the control and comparing results. It is the strongest and the least used, because it costs the most.
Automated testing against the full population is a different category, and where it is available it beats all four.
A sample of twenty-five from a population of forty thousand supports a statement about the sample. Extending it to the population requires assumptions about how failures are distributed, and failures are rarely distributed randomly.
Controls tend to fail systematically. At month end. For one region. For a system class that was onboarded without the standard process. A random sample of twenty-five is unlikely to catch any of those patterns, and if it does catch one instance, the pattern is still invisible.
This is the strongest argument for continuous testing, independent of the efficiency case.
What was tested, against what population, by what method, on what date, by whom.
The result, and the evidence supporting it, retained.
Any exceptions found, whether they were remediated, and by when.
Without the population and the method recorded, a pass result cannot be interpreted later, and it will need to be.
From the blog