Ten separate defects in one product build had the same shape. A failed query, a blocking policy, or a truncating limit rendered as a plausible empty state or a confident number. Every one of them passed code review. Every one surfaced only by looking at a screen or calling an endpoint directly.
This is an article about that shape, because it is not a bug class specific to one codebase. It is the default behaviour of almost every measurement system ever built, and in security it is worse than useless, it is a dashboard that reports a broken sensor as good news.
The industry principle underneath it
Security and assurance methods already depend on sufficient, relevant and reliable evidence. NIST continuous-monitoring guidance, control-assessment practice and ISO performance evaluation all assume that collection quality constrains the conclusion. IO's distinctive implementation is to make the unknown state first-class in the data model, not merely a footnote on a report.
This is also why zero, none observed, not applicable, not collected, collection failed and insufficient sample must never share one value. They lead to different decisions.
The four mechanisms
Converting an error into a value. The pattern is a single line: on failure, return zero. It is written to keep a page from crashing, and it succeeds, the page renders, the tile shows a number, and an access failure has become a factual claim about the world.
A guard that filters instead of raising. An authorisation predicate that returns null under some execution context matches no rows. The query succeeds. Zero rows come back. Nothing errors anywhere. An unauthorised caller and a caller with genuinely no data receive identical responses, and the system has no way to tell them apart.
A limit that starves a subset. A global cap applied for performance reasons truncates the result set. Whichever records fall past the cap simply do not exist as far as every downstream computation is concerned.
An empty object on undefined. A component receives no data and renders its zero state rather than its unknown state, because zero states are what get designed and unknown states are what get forgotten.
All four produce the same output: a number where there should be a question.
Why this matters more in security than elsewhere
If a retail dashboard under-reports revenue, someone notices, because revenue is checked against a second source that exists for other reasons.
Security measurement has no such second source. If a control coverage figure reads 94% and the true figure is unknown because a connector silently stopped three weeks ago, there is nothing to check it against. The number is the only artefact. It gets carried into a board pack, and it becomes the basis of a decision not to invest in the thing that has quietly stopped working.
The failure is not that the number is wrong. It is that the number is confident and wrong, and confidence is the whole product.
Three states, not two
The fix is architectural rather than a matter of care.
Every measurement surface must distinguish three states: measured and good, measured and bad, and not measured. The third must be visually and structurally distinct from the first two, must never sit on the same scale, and must always carry a reason.
Applied consistently that means:
- residual risk reads not yet measured, never inheriting from inherent
- appetite has a third state, cannot be assessed, held apart from within and breaching
- maturity has not yet assessable, held strictly apart from Level 0
- a domain dimension with a zero denominator does not band, score, or contribute
- a forecast lens with insufficient history is skipped, not projected
The rule for guards
A guard raises. It never filters.
An unauthorised caller gets an error. A caller with no data gets zero. Those two states must be distinguishable at every layer, and any code path where they converge is a defect regardless of what it renders.
This is more work than filtering, and it produces uglier failure modes, errors where there used to be quiet empty pages. That is the correct trade. An error is a question. A zero is an answer.
What this costs
The honest version of this discipline is uncomfortable to ship.
One register built this way reports that roughly three-quarters of its risks cannot be assessed against appetite. That is a far worse-looking number than the alternative, which is to report those risks as within appetite because they sit below a ceiling nobody measured them against.
It is also the correct number, and it is the most credible thing in the product. A buyer who understands what they are looking at will trust the remaining quarter far more than they would have trusted a full register with no coverage figure attached.
How to find these in your own systems
Not by reading code. Every one of the ten was invisible in review.
Look at a screen where you know there should be nothing, and check whether it says nothing or says zero. Call the endpoint directly rather than through the interface. Ask for the actual output pasted back rather than a confirmation that it works, "confirm X is correct" gets you a yes, "paste the four headlines" gets you the bug.
Check the output, not the report.
What this does not mean
Absence of evidence is not automatically evidence of absence, but neither is the reverse universally true. A complete, well-designed observation process may legitimately observe zero qualifying events. The point is to preserve the collection conditions that make zero defensible, rather than banning zero as a result.