docs(901): pin the artifact whole — a predicate over an unimplemented grammar #920
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: timothy/ersatztv#920
Reference in New Issue
Block a user
fixes #901
What this decides
New decision record
docs/decisions/records/testing/guard-pins-the-artifact-not-a-shape.md(
testing.guard-pins-the-artifact-not-a-shape): a predicate over an artifact whose grammar it doesnot implement pins the artifact WHOLE by default — the whole file, or a whole named span, compared
for equality under a stated normalisation. Matching a shape inside it is the exception, and must
carry a four-part argument (grammar + parser, closed input space, measured fail direction, what it
buys over the pin).
The two open questions #901 left, both answered:
(nobody reads it), while a pin's failure is a false RED (a human reads the diff). The defeat a
"write the matcher, pin it later" policy waits for therefore arrives from a reviewer or an
incident, not from the guard — "not defeated yet" measures who has looked, not what the input
space is. Both worked incidents (#887, #891) never withdrew because a count got high.
author either wrote or didn't, so a reviewer can refuse one: (1) the grammar named and the parser
that implements it, (2) the input space as a CLOSED enumeration with the reason it's closed —
never just "the spellings I thought of", (3) the failure direction MEASURED via an executed
mutation (
testing.mutation-claims-are-executed), not asserted, (4) what it buys over the pin,priced against a cost the pin actually charges.
Rejected alternatives (both explicit in the record): a NUMERIC bar ("survives N adversarial
spellings") — measures the reviewer's imagination, not the input space, and would have licensed
#887's parser through most of its nine defects; and a REVIEWER SIGN-OFF bar — makes the exception
depend on the very signal the threshold says arrives late.
Two riders, both measured: (1) a pin assumes it's pinning the artifact that still DECIDES — ask
what else could decide this (another file that outranks it, another occurrence, another workflow, a
hook the pinned command invokes) and whether the pin would still match. (2) widening a clause retires
the proofs calibrated against the narrower one — a canary asserting a finer mutation still SURVIVES
has as its precondition that the guard is deliberately coarse there; re-derive every such proof in
the same commit that widens the clause.
The threshold for withdrawing an already-shipped shape-matcher: not a spelling count, but the
moment the NEXT spelling is found by the reviewer rather than the author — evidence the author's
model of the input space is not the input space.
The record also documents two worked pins in
scripts/tests/test_image_build_delegates_the_spa_suite.py(three differently-normalised pins) and tightens
scripts/tests/test_hook_fire_log.py's existingpreamble pin's comments to state what its SELECTION cannot see (a runtime-composed/
evaledreassignment of the pinned variable is outside the pin's reach) rather than claiming whole-file
byte-identity.
Docs updated in this PR
docs/README.md— task-signal map entry for "Proposing a new guard / CI check / regression testconvention" now points to this record for the shape of the predicate itself.
docs/guard-inventory.md— cross-references the record before editing a pinned-artifact guard.docs/decisions/README.md— catalog regenerated to include the new record.Gate results (this session, worktree
wt-901, no rebase needed — origin/main had not moved)PYTHONPATH=. python3 -m pytest scripts/tests -q -p no:cacheprovider: first run stalled underload (PID confirmed near-zero CPU growth over 10+ minutes, its child step.sh already exited) —
killed and reran once: 1599 passed, 3 skipped in 798.63s
PYTHONPATH=. python3 scripts/decisions_validate.py: decisions-validate: OK (advisory-only60-line-ceiling warnings for 50 unrelated records, non-blocking)
PYTHONPATH=. python3 scripts/build_decisions_catalog.py --check: catalog up to datepython3 scripts/check-doc-narrative.py --diff origin/main: 0 advisory warning(s) (non-blocking)ruff check scripts/tests/test_hook_fire_log.py: All checks passed!No dotnet, no web build/lint gate required — this change is docs + one Python test-comment edit.
(The husky pre-push hook itself does run web lint/build/typecheck since this worktree lacked
web/node_modules; that was resolved by copying it from the shared checkout per runbook, not byskipping the hook.)
<<REVIEW HISTORY
Three review runs, nine rounds total, across the life of this branch. Every blocker was a prose
absolute or an uncalibrated claim in the decision record itself — never a code defect, since the
only non-doc file touched is a test-comment tightening. Each round was answered by subtraction:
removing an unsupported absolute, an uncited count, or a duplicated clause, rather than by adding
qualification on top.
Commits, in order (
git log --oneline origin/main..HEAD, oldest first):0f4552d09— pin the artifact whole is the DEFAULT for a predicate over a grammar it does not implement87d90f330— the hook-preamble fault does not name a constant — correct the mechanics claim4c2ceccaf— carry both rejected alternatives in the rule, not only in the body78593ace9— a quotation that resolves to nothing, and a universal over three pins that holds for two32e857843— the one line a guard author reads names two of the three rules17c65005f— the hook pin is a pin WITHIN a selection, and the selection has a measured escapec67b33cdc— a mutation OUTCOME is a manifest entry or it is not written — state the selector instead0977f0417— the respelling-immunity thesis is bounded by the pin's REACH, and the neighbour doc declines the rule it was cited for761a71278— pay for the two corrections by tightening the prose around them11f92761a— the whitespace collapse belongs to the PRIMITIVE, so it belongs to both text pins868ab657d— the record was wrapped narrower than its own corpus, so 65 lines was a WRAP width, not a word countdc3f158bc— #885 merged, so the sentence the record's structural fact refutes is corrected at its source20875b0ca— answer round seven by SUBTRACTION — the thesis loses both absolutes, the list loses two copies81358dec5— the acceptance hole belongs to the REJECTED splitter, so the clause claiming a second one is deleted022a05acb— the third count had no in-repo source, so it becomes what both cited artifacts sayThe last run's three commits (
20875b0ca,81358dec5,022a05acb) closed round seven through nine: eachanswered a blocker by removing prose rather than adding it — an unsupported absolute, a clause
claiming a second acceptance hole that in fact belongs to the rejected alternative, and a count with
no in-repo source, replaced by what the two cited artifacts actually say. Last round: nits only.
REVIEW HISTORY>>
Cross-family review: not required (routine risk class under process.independent-review-rubric).
Deferred: none.
🤖 Generated with Claude Code
https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiV
The record's own rule ends: a pin also declares its NORMALISATION and what the normalisation cannot see, because "pinned whole" invites a reader to assume byte equality. The `mechanics:` field did that for the image-build pin (it names `_normalise_lines` and what it drops) and not for the hook pin, which it described only as pinning two lines byte for byte. The comparison at scripts/tests/test_hook_fire_log.py:161-167 is over a SELECTED set — lines containing the literal `ETV_HOOK_FIRE_LIB` that are not comment-led — so byte-identity holds within the selection and says nothing about a writer of that variable spelled without the literal. MEASURED 2026-09-05 on this branch against `.claude/hooks/decisions-guard.sh`: appending eval "$(printf %s%s=/dev/null ETV_HOOK_FIRE _LIB)" after the canonical assignment leaves `instrumentation_faults(text, 'decisions-guard')` == `[]`, byte-identical to the unmutated baseline `[]`, while running those two lines under bash prints `final=/dev/null` — the sink is repointed and the checker is silent. A false GREEN, which is the failure direction this record exists to argue about. The residual is #891 code and is not introduced here; no code changes. What changes is that a record citing this file as one of its two worked pins now states the residual instead of implying byte equality over the file. Refs #901, #891 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiVThe two corrections added four prose lines to a record already one line over the 60-line advisory ceiling. Recover what can be recovered without losing substance: reflow the paragraphs, drop the padding ("and it fails silently" → ", silently"; "the count rises" → "and rises"), and cut one restatement. It lands at 65 lines, not 60. That is a deliberate stop: the remaining candidates are the *why* behind non-obvious choices — which the repo's own docs rule says to keep — and `decisions_validate` reports the constant itself as drifted from the distribution it is supposed to mark the tail of (p90=104, p95=142, 50 of 222 records over it). The validator passes. Refs #901 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiV`mechanics:` opened with a universal — "a residual read off one of them does not transfer to the others" — and then attributed the collapse-whitespace-inside-a-string residual to `_normalise_lines` alone. It does transfer, between exactly the two pins that sentence separates: `_normalise_lines` is `_normalise` applied per line (`scripts/tests/test_image_build_delegates_the_spa_suite.py:327`), and the stage-command pin calls `_normalise` directly (lines 386/389, compared at line 586), so it cannot see a whitespace change inside a quoted shell string either. For SHELL text that is the more consequential of the two residuals, which is the opposite of what the old ordering implied. The sentence now says NEED NOT transfer, names the one that does, and puts the string-literal blindness on `_normalise` where it originates. The vite-only fact that survives is the blank-line drop, and the claim that the test STATES its residual is narrowed to the vite test, which is the only one of the two that does. The hole is dormant rather than live — no entry in `PINNED_STAGE_COMMANDS` carries a quote character — but the defect was the prose universal, which the record's own rule ("a pin also declares its NORMALISATION and what the normalisation cannot see") is what this paragraph exists to demonstrate. This is the third finding read off this one sentence: `b8dc321af` corrected its fault-message half and `4d5bd0dbb` its outcome-claim half. TWO OBLIGATIONS ARE OWED to the closing record, both blocked on #885 (open, so this branch does not touch `scripts/tests/`), and both freed together when it closes: 1. The `CLAIMS` entry in `scripts/tests/mutation_manifest.py` for the hook-preamble selector — target `.claude/hooks/decisions-guard.sh`, clause = the canonical sink assignment, replacement = that line plus the `eval` spelling, proof = a `test_hook_fire_log.py` node, outcome=GREEN with the mandatory `reach_replacement`/`reach_expect`. Until then `mechanics:` states the SELECTOR as a structural fact and makes no outcome claim (`4d5bd0dbb`). 2. The comment at `scripts/tests/test_hook_fire_log.py:153-154`, which ends "this arm's errors are refusals, never acceptances". The structural fact this record ships — the compared set is the non-comment-led lines containing the literal `ETV_HOOK_FIRE_LIB`, so a reassignment that never spells the literal is outside the selection — is an ACCEPTANCE by that arm, and the same comment block concedes the class five lines later ("so it is an acceptance hole, not only a stricter refusal"). The sentence is owed a correction; the record documents the residual beside it in the meantime. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiV`798e5ee40` stopped at 65 prose lines and gave a reason: the remaining candidates for removal were the *why* behind non-obvious choices, which `docs.no-session-narrative` says to keep. That reason is sound about the WORDS and wrong about the LINE COUNT, because the two are not the same quantity. `decisions_validate.record_prose_lines` counts PHYSICAL lines, and this record was the narrowest thing in the set being measured: max body width, 222 active records: median 106; 28 at <=100, 154 at 101-120, 40 >120 this record: 100. The sibling it cites by key, process.hook-resolves-inputs-from-repo-root: 116. The 60-line ceiling was derived at #620 from that distribution, so measuring a 100-column record against it charges the record for a wrap width the corpus does not use. Re-wrapping the seven body paragraphs at 116 — the exact width of the neighbour record — takes the body from 66 physical lines to 58, and the validator now reports 58, off the over-ceiling list (50 records over -> 49, and the key no longer appears). The reflow removes NOTHING: the script asserted word count equal before and after (893) and whitespace-normalised body text byte-identical, and refused to write otherwise. What it buys is that the issue's `## Done-when` box "The record is under the 60-line advisory prose ceiling" is satisfiable as written, so `pretooluse-merge-consent.sh` is not asked to derive consent from a box ticked falsely or left standing. The advisory itself was never breached — the ceiling is a `::warning::`, the validator exits 0, and 49 of 222 records are over it inside the 2-25% CEILING_MINORITY band. Also: the file was the only one of 222 records with no final newline. Fixed in the same commit; `record_prose_lines` is `splitlines()`, so it does not move the count. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiV`scripts/tests/test_hook_fire_log.py:154` ended "this arm's errors are refusals, never acceptances". That is false for the SELECTION, which is the half this branch's record is about: `mentions` is chosen by the literal `ETV_HOOK_FIRE_LIB`, so a writer of that variable which never spells the literal is outside the compared set altogether, and the checker never sees it. The comment already conceded an acceptance hole of the same kind six lines later, about `split("\n")` versus `splitlines()`, so the universal was contradicted inside its own block. The correction stays STRUCTURAL — it reads the selector one line below and says what the selector reaches — and makes no claim about what a mutated hook would return. An outcome claim would be a `CLAIMS` entry in `scripts/tests/mutation_manifest.py` or nothing, per `testing.mutation-claims-are-executed` as amended by #881, which is the same reason `mechanics:` states the selector rather than a measured result (`4d5bd0dbb`). WHY THIS LANDS NOW: #885 closed while this branch was in review (merged as #919, `c30847204`), which frees `scripts/tests/`. Of the two obligations `47619320e` recorded as owed, that message is superseded here: - DISCHARGED: this one, inline, above. - NOT OWED, and the reason is not the blocker: the `CLAIMS` entry for the hook-preamble selector. `4d5bd0dbb` removed the outcome claim from `mechanics:` rather than binding it, so the record asserts no mutation outcome and the rule it invokes has nothing to bind. Re-adding a claim in order to bind it would reverse a review-mandated change; an executed GREEN entry (target `.claude/hooks/decisions-guard.sh`, clause = the canonical sink assignment, replacement = that line plus the `eval` spelling, plus the mandatory `reach_replacement`/`reach_expect`) remains available as an ENRICHMENT of the structural fact, and belongs to whoever wants the fact executed rather than argued. Rebased onto `366a0f904..c30847204` on the way: the `docs/guard-inventory.md` conflict is two rows, resolved by taking #885's newer `test_workflow_persist_credentials.py` row (it gained a second invariant) and this branch's `test_image_build_delegates_the_spa_suite.py` row (it dates the vite `DEFAULT_CONFIG_FILES` reading to 8.1.3 and to the lockfile). `docs/decisions/README.md` regenerated, not merged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiVRound seven's blocker was the record's own thesis sentence, and the fix is to delete rather than to re-argue. Six rounds each replaced a refuted absolute with a fresh one; this one states only what was measured. THE THESIS (body). Struck: "a pin cannot be defeated by a respelling of what it COMPARES" and "nothing [stays exposed] for a whole-file pin". Both are false for the record's own worked whole-file pin — measured against the guard's own primitives: _normalise_lines("const flag = '--run --reporter=x';") == _normalise_lines("const flag = '--run --reporter=x';") -> True _normalise('RUN npm ci && echo "a b"') == _normalise('RUN npm ci && echo "a b"') -> True Both pairs differ in bytes and both are respellings of what the pin compares. `PINNED_VITE_CONFIG` — the pin the record calls "pinned WHOLE" — is compared through `_normalise_lines`, so its normalisation is a second exposure axis beside the selector's. The record already refuted itself twice: `rule:` ends "A pin also declares its NORMALISATION and what the normalisation cannot see", and `mechanics:` says the whitespace collapse "including inside a QUOTED STRING" belongs to both TEXT pins. What replaces the sentence is the fail DIRECTION alone — a pin's is a false RED, a shape-matcher's a false GREEN — plus the declaration obligation the rule already carries. No new universal is written in its place. MECHANICS. "named once so the checker, the mutation proofs and the hooks cannot come to mean different strings" is deleted, not repaired: the assignment string is written out at THREE sites in `test_hook_fire_log.py` (measured by walking the AST and comparing each assembled string to `CANONICAL_SINK_ASSIGNMENT` — the constant, and the `current` local of `test_an_ENV_VAR_resolved_sink_path_is_DETECTED` and of `test_the_NEXT_env_var_to_be_invented_is_DETECTED`). The source comment making the same claim is corrected in place, and its correction is STRUCTURAL: it names the three sites and the `current in text` assertion each proof carries, and asserts no mutation outcome, because "an edit here faults loudly there" would be a `CLAIMS` entry under `testing.mutation-claims-are-executed` — wherever it is written — or it is not written. Same reason `4d5bd0dbb` removed the outcome claim from `mechanics:` rather than binding it. THE QUALIFYING-GRAMMAR LIST now exists once, in the record's `rule:`. `docs/README.md` and `docs/guard-inventory.md` state the operative test — an artifact with a grammar the predicate does not implement — and point at the record. The copies had already disagreed inside the commit that wrote them: `docs/README.md` carried five of the six members, omitting JSON5, which is the member the issue's own correction comment names as the one the narrow "shell or config TEXT" framing would have let through (`dont-keep-a-copy-of-a-set`, #869). Rider 1 drops its copy of the vite `DEFAULT_CONFIG_FILES` ordering the same way, deferring to the dated reading in the `test_image_build_delegates_the_spa_suite.py` row. Gate: `pytest scripts/tests -q` 1599 passed, 3 skipped (all pre-existing by-design skips); `decisions_validate.py` OK with the record off the >60-line list; catalog regenerated with no diff; ruff clean; `check-doc-narrative --diff origin/main` 0 warnings. Refs #901 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiV`dc3f158bc` corrected a false universal ("this arm's errors are refusals, never acceptances") and, in the same sentence, appended a second unmeasured fail-direction claim: "and the `split(\"\\n\")` residual below is the second acceptance of the same kind". That clause inverts the paragraph it points at. MEASURED, by loading the module and calling its own primitives with `CANONICAL_SINK_ASSIGNMENT + \x0c + "rm -rf /tmp/nothing"` on one physical line followed by `CANONICAL_SINK_SOURCE`: instrumentation_faults(...) -> "the sink preamble is not the canonical two lines" splitlines() selection == [CANON, SOURCE] -> True (ACCEPTS) split("\n") selection == [CANON, SOURCE] -> False (REFUSES) The shipped `split("\n")` refuses exactly where the rejected `splitlines()` accepts, which is what the pre-existing paragraph six lines below already says. There is no second acceptance below; the acceptance belongs to the alternative that was NOT shipped. `\x0c` is not a line terminator for bash either, so `split("\n")` matches the shell the checker models and has no hole of this kind. The FIRST half of the sentence stands and was re-measured: a hook carrying the two canonical lines plus `v="ETV_HOOK_FIRE""_LIB=/tmp/evil.sh"; eval "$v"` produces no preamble fault, so the selector really does accept a writer that never spells the literal. Answered by SUBTRACTION per the brief: the sentence ends at "not all refusals." and no replacement prose is written. `dc3f158bc`'s message argues from the same inversion ("the comment already conceded an acceptance hole of the same kind six lines later"); that half of its reasoning is withdrawn here. Its correction of the universal is unaffected — the selector escape it names is structural and independently measured, above. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiVReview-verdict: MERGEABLE @
022a05aOrchestrated-session workflow: nine cold review rounds across three runs (correctness Opus high in its own worktree, conformance sonnet high); each round's blocker was a prose absolute in the new record, answered in the end by subtraction; final round nits only. Full Python suite 1599 passed / 3 skipped on the pushed head; decisions validator OK; catalog byte-stable. Cross-family review: not required (routine risk class; docs + one test comment). Verdict on the pushed head; patch unchanged since review.