FACT: A sample can pass while the unobserved corpus contains unmapped fields, withheld records, or skipped page families. INFERENCE: The denominator is part of the claim, and a zero rate over a partial denominator is not full coverage. RECOMMENDATION: Separate sample diagnostics from full-corpus acceptance, enumerate the actual denominator, and make selector or family coverage non-vacuous before reporting completeness. LIMITATION: Full-corpus evidence proves the named snapshot only; freshness and source correctness remain separate questions.
Problem details
- Observed symptom
- A coverage report says zero unmapped fields or zero missing routes even though a larger capture, skipped selector, or unvisited family contains failures.
- Context
- Large static projections often use samples, capped responses, and per-family selectors to keep tests fast.
- Environment
- Unknown · not established
- Symptom signature
- Component
- coverage verification
- Operation
- report completeness
- Protocol
- captured static corpus
- Literal source
- Not supplied
- Expected behavior
- Not supplied
Known approaches
solution · Revision 1
Name and verify the full denominator before claiming completeness
FACT: A reliable coverage gate records the expected corpus, visited members, withheld items, skipped families, and every unmapped path; it also proves that required selectors matched real targets. RECOMMENDATION: Run the full capture or an explicitly complete manifest for acceptance, keep sample metrics labeled as samples, and fail on zero-target or unexplained denominator drift. LIMITATION: The full denominator must come from an attested snapshot rather than a guessed total.
- Problem id
- 8ebe7614-f364-4ab9-9db1-a5c73178b633
- Proposed action
- Build a pre-render materialization plan and full field ledger, compare both directions against the rendered artifact, and reject vacuous zero findings.
- Applicability
- State
- known
- Text
- Use for API field ledgers, route inventories, rendered-page audits, selector-based tests, and corpus-wide release checks.
- Limitations
- State
- known
- Text
- A complete scan cannot prove fields absent from the source or future snapshots; those remain explicit unknowns.
- Success criteria
- The report exposes its denominator, all required families have non-zero exercised targets, full capture and rendered destinations reconcile, and a deliberately skipped family cannot produce a false zero.
- Risk notes
- Text
- Do not turn a sample, capped page, or selector miss into a claim about the complete artifact.
- Lifecycle
- active
Page 1 · 1 children total
Sources and related records
No source relations recorded.