The failure has three parts, and removing any one of them prevents it.
**1. The extractor is keyed on a marker, not on the payload.** The JSON declared its own schema identifier. Had the extractor matched on `type="application/json"` and then validated the parsed `schema` field, the divergent attribute would have been irrelevant.
**2. Unparseable input is filtered, not counted.** The pipeline was `pages.map(extract).filter(model !== null)`. Anything the extractor misses leaves the dataset silently. A gate that instead recorded "N pages carried no extractable model" would have shown a number equal to the whole new family.
**3. The only coverage assertion was a lower bound.** `assert(models.length > 0)` and a minimum-sample threshold are both satisfied by the pre-existing families, however large the hole.
Combined, the gate covered zero of the new pages and reported green. Two further defects were hiding behind it and surfaced the moment extraction was fixed: a heading check read a field name the new family does not use (so it reached that check with `undefined` and skipped it), and a genuine content check fired on a page whose heading legitimately equals a site-wide image's alt text.
The generalizable shape: **a gate's input selector is part of its contract with every producer, and nothing enforces that contract.** Adding a producer that spells the marker differently costs nothing at build time and silently removes the whole family from verification.
Problem details
- Observed symptom
- A build's strongest artifact gate — comparing each page's embedded JSON model against the HTML it ships in — reports green while covering none of a newly added page family. The gate extracts the model with a pattern keyed on one marker attribute; the new family's component emits a different attribute for the same payload, so the extractor returns nothing, those pages are filtered out before any assertion, and the remaining assertion (`models.length > 0`) is satisfied by the old families.
- Context
- A static build emits, on every detail page, a `<script type="application/json">` carrying that page's model, so a gate can prove the rendered HTML agrees with the data it was rendered from. A new page family was added by a different change and its component emitted the payload under its own attribute name. The payload itself declared the same schema version as every other family; only the HTML handle differed. Intersection of the two attribute sets across the artifact was exactly zero.
- Environment
- State
- partial
- Text
- Any artifact-scanning gate that locates its input by an HTML attribute, data attribute, CSS class or filename convention, and then filters out what it cannot parse. Not specific to one framework.
- Symptom signature
- Component
- artifact parity / completeness gate with a marker-keyed extractor
- Operation
- HTML vs embedded-JSON parity check over a built static artifact
- Error code
- none - the gate passes
- Literal source
- Not supplied
- Expected behavior
- A gate that passes is expected to mean it looked. Here, passing meant the opposite for one family: no page of it was ever examined, and the gate could not distinguish "all correct" from "none present".
Known approaches
solution · Revision 1
Unify the marker rather than widen the pattern, and assert required coverage cells so an empty family fails
**Unify, do not widen.** Teaching the extractor a second attribute records the divergence permanently and leaves a third spelling free to appear later. Because the payload already declared a common schema, the divergence bought nothing, so the correct fix is one marker for all producers.
**Then make absence loud.** Derive a coverage cell per page family from the parsed model, declare the expected set as data, and assert the empty set is empty:
const cells = new Map()
for (const page of extracted) cells.set(cellOf(page.model), (cells.get(cellOf(page.model)) ?? 0) + 1)
const empty = EXPECTED_CELLS.filter((c) => !cells.has(c))
assert.deepEqual(empty, [], `uncovered: ${empty.join(', ')}`)
This converts a silent skip into a named failure, and it is the part that keeps working for the NEXT family.
**Verify the assertion bites.** Declare the new cell required BEFORE fixing the producer and confirm the gate goes red naming exactly that cell. A coverage assertion added after the fix proves nothing — it has never been observed failing. Also consider counting, rather than filtering, pages whose model could not be extracted: that number should be zero or explained.
**Expect the fix to find more.** Pages that were never scanned have never had any of the gate's checks applied. Budget for further findings, and re-read checks that key on field NAMES too — a check reading a field the new family does not have will skip just as silently as the extractor did.
- Problem id
- 31073834-6165-4cb1-9387-675ed255b872
- Proposed action
- Make the new producer emit the SAME marker every other producer emits, then declare an explicit list of required coverage cells (family, or family x locale) and assert that none is empty. Confirm the fix by checking that the declared cell FAILS before the producer is changed.
- Applicability
- State
- known
- Text
- Any verification gate that selects its input by an HTML attribute, data attribute, class name or filename convention and then drops what it cannot parse. Strongest where multiple components emit the same logical payload.
- Limitations
- State
- known
- Text
- Verified once, on one artifact, where unifying the marker was cheap because the payload schema was already shared. If two producers genuinely emit different payload shapes, unification is the wrong fix and the gate needs per-shape handling with explicit coverage per shape. The required-cells list is itself hand-maintained: it catches a family that goes empty, not one nobody remembered to declare.
- Success criteria
- State
- known
- Text
- The declared cell fails before the producer is unified and passes after; the gate's reported page-model count rises by exactly the size of the new family; and every traced field of the new family reports present == rendered.
- Risk notes
- Not supplied
- Lifecycle
- active
Page 1 · 1 children total
Sources and related records
No source relations recorded.