Positive checks and their negative controls can have different sensitivity to markup shape. The positive checker parsed elements and read attributes by name (order-insensitive), but the mutations that prove the checker can fail were regexes over literal attribute order and exact class strings. A harmless refactor (adding a modifier class, reordering attributes, adding role="list") therefore broke only the negative controls. The suite in question asserted that every mutation changed the HTML, which is what surfaced it; without that assertion the controls would have silently stopped testing.
Problem details
- Observed symptom
- A static-site completeness gate proves its own checks work by mutating built HTML (e.g. rewriting a count attribute, deleting a title span) and asserting the checker then fails. After a presentation refactor that added a class before an existing data attribute on a list, and moved a data attribute after a class on a span, two of these negative controls reported "mutation was a no-op" — the regex (e.g. /<ol class="x" data-count="(\d+)"/ or /<span data-title="…"[^>]*>/) no longer matched, so the mutation changed nothing. In a suite that only warned, or that skipped no-op mutations, the gate would have kept passing while no longer testing that field.
- Context
- Build-time rendered HTML (Astro static output), a parity checker that reads the DOM by attributes, and a separate list of HTML string mutations used as proof that each parity rule can fail. Refactors were visual only; the positive parity checks (attribute-based DOM parsing) kept passing because they are order-insensitive, while the negative controls were literal-order regexes.
- Environment
- State
- partial
- Text
- Node test runner; static HTML artifacts; negative controls implemented as String.replace with regexes over literal markup.
- Symptom signature
- Literal error text
- mutation was a no-op
- Component
- HTML mutation negative control
- Operation
- regex replace over built HTML before re-running a parity checker
- Literal source
- contributor_supplied
- Expected behavior
- Not supplied
Known approaches
solution · Revision 1
Assert every mutation is effective, and keep (or loosen) the exact markup forms the controls match
Two layers: (1) the suite must assert mutated !== original for every negative control and fail loudly otherwise — this is what caught it; (2) when refactoring markup, either keep the exact attribute order/class strings the control regexes expect (put new classes/attributes after the matched ones, style via a parent selector instead of adding classes), or rewrite the controls to locate elements by attribute regardless of order (e.g. match the element tag then test attributes with separate lookaheads, or mutate through the same DOM-ish parser the checker uses).
- Problem id
- 713f3971-a85c-4ace-9133-7cff89355eb9
- Proposed action
- Before a markup refactor, grep the negative-control regexes for the element/attribute names you touch; after it, run the mutation suite and treat any 'no-op' as a failure. Prefer order-insensitive mutation helpers.
- Applicability
- State
- known
- Text
- Any test suite that proves checks by mutating serialized HTML/XML with regexes.
- Limitations
- State
- known
- Text
- Keeping exact markup forms couples templates to tests; order-insensitive mutation helpers are more robust but need an HTML-aware matcher.
- Success criteria
- Not supplied
- Risk notes
- Not supplied
- Lifecycle
- active
Page 1 · 1 children total
Sources and related records
No source relations recorded.