Sixth cold review (a different reviewer, in-repo, worktree-isolated after the cross-family runs wedged twice on their sandbox). One High, one Medium, two Low, two Nit. All fixed. HIGH, and it is the third time this population has been wrong. `rglob` is recursive, so it also enumerated `.husky/_/` — 17 husky shims generated by `npm ci` via web/package.json's `prepare`, gitignored and untracked. The guard therefore derived 76 files against a 59-row table and was RED on every checkout that has run `npm ci`, while staying GREEN in CI, whose `script-tests` job checks out and pip-installs but never runs `npm ci`. A guard that fails everywhere except where it runs is the fastest possible route to "that test is always broken, ignore it" — on the artifact whose entire thesis is population correctness. Reproduced, then fixed at the source rather than with a fourth traversal patch: the population now comes from `git ls-files`. The index is authoritative, identical for CI and every checkout, and excludes untracked build output by construction instead of by an exclusion list someone must maintain. That is what this PR's own record says to do; the first three attempts each derived from whatever happened to be on disk. Three tests go red against the rglob predecessor. MEDIUM — twin-missed, in the fix from the previous round. Round 4 re-read the base before the branch-protection lookup, inside the scheduled branch only, leaving the #632 retarget DETECTION still reading the top-of-hook snapshot. The reviewer demonstrated it with this PR's own fixture: scheduled+retarget denied while immediate+retarget AUTO-GRANTED. The re-read is now hoisted above every base-dependent consumer, so one read serves both paths, and the duplicate is gone. Note for the record: the hoist is the load-bearing part — once `live_base` is fresh, #632's own comparison catches the retarget too, so the explicit deny only bites when no verdict records a base. The tests are scoped to exactly that case, because as first written they passed under mutation. LOW — a 404 from `branch_protections/<ref>` does not prove the branch is unprotected. Gitea keys that endpoint on the RULE name, so a base covered by a glob rule 404s while being fully protected, and an unencoded ref containing `/` (`release/26.4`) 404s because the path is malformed. Both produced a hard deny stating a specific, false cause — and a deny blocks outright rather than prompting. The ref is percent-encoded, and a 404 now consults the rule list before denying; an unreadable list asks. LOW/NIT — the scope prose attached the extension restriction to `scripts/` alone while the guard applied it everywhere (a `.py` hook would have joined the described scope and acquired no row); `.yaml` workflows are now in scope too. The `PINNED` definition required re-validation, which two legitimately-pinned rows do not do because their check and use are one step over an immutable event-payload sha. Row ordering restored. And once more, the recurring one: adding a scope TABLE to the doc made three prose rows parse as inventory sites — the parser reading its own documentation as data, the same defect as the UNSAFE-KNOWN check that once parsed the paragraph defining UNSAFE-KNOWN. Row parsing is now bounded to the inventory section explicitly. refs #778 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
19 KiB
Guard inventory (ersatztv#774 / #775)
Every executable guard file in this repo, what it blocks, and whether it ships a proof it can go
red. scripts/tests/test_guard_inventory.py derives the population from the filesystem and the
workflow/hook call sites and asserts set equality against the Guard column, so a new guard
cannot be added without acquiring a row here, and a row cannot name a proof that does not exist.
Read docs/decisions/records/testing/guard-derives-population-from-source.md and
…/guard-ships-with-mutation-proof.md before editing a guard or adding a row.
Columns
- Kind —
GUARD(it can block a commit, a push, a tool call, a merge or a CI job),TOOLING(it does work but asserts nothing; listed so its absence from the guard set is a recorded decision, not an oversight), orPROOF(ascripts/tests/file whose job is to prove another guard).PROOFexists to stop a regress: once test files entered the population, every mutation proof became a row wanting a proof of its own. Ascripts/tests/file that enforces a repo invariant with no separate guard behind it is aGUARD, graded normally, and may cite a mutation case in its own file. - Proof —
MUTATION: a clause-level mutation was executed and this named test was witnessed red. The test either performs the disarm itself (a monkeypatch, a deselection, a removed marker) or a recorded incident shows it going red when the clause went away.BEHAVIOUR-ONLY: a real test drives the guard through its real entry point with good and bad input, but no one has demonstrated that removing the clause reddens it.NONE. - Proof ref —
file.py::function, verified to exist by the inventory test.
The distinction between MUTATION and BEHAVIOUR-ONLY is the whole point of #775 and is not a
grading curve. A behavioural test proves the guard reacts to its input; only a mutation proves the
guard is load-bearing. #685 shipped two guards on one condition where deleting either left the
suite green, and every behavioural test passed throughout.
The definition above is the second one. The first read "a named test disarms this guard's clause
and asserts red", and three rows were graded MUTATION against it that do not disarm anything —
they feed the real script an input only that clause rejects (empty stdin, a short page, a full first
page). Cold review caught it, and the objection is right twice over: those tests are the same
species as ones graded BEHAVIOUR-ONLY eight rows away, so the column was being applied as a curve
on the very day it was introduced. They are regraded. The surviving criterion is witnessed, not
plausible — "removing this clause would surely redden that test" is an argument, and this table
exists because arguments of that shape have been wrong here six times.
Scope limit, stated rather than implied
This inventory covers guard files, discovered by globbing .claude/hooks/*.sh, .husky/* and
scripts/tests/test_*.py, plus every scripts/… path referenced by a workflow or a hook. Six
classes are outside that population. They are listed because the first version of this section named
only the first one, and cold review found that the very guards this inventory shipped with were
sitting in the gap:
- Guards inline in workflow YAML — most importantly
pr-checks.yml:ci-image-pin. "Which jobs are guards" needs a judgement call per job the filesystem cannot supply. Two were audited under #774 and one fixed; extending the population is tracked in #786. - C# and TypeScript guards —
ErsatzTV.Mcp.Tests/ToolCatalogTests.csandweb/src/api/pageSizeCallSites.guard.test.tsare both structural guards and neither has a row. - Mentions counted as call sites. The
scripts/…scrape matches any occurrence, including inside a comment or an::error::string.scripts/update-openapi.shis named in apr-checks.ymlerror message, so removing the step that runs it would leave its row intact. - Nested and non-lowercase paths beyond
scripts/tests/— a guard underscripts/scripted-schedules/, or with an uppercase name, is invisible to the scrape. .yamlworkflows and non-.shhooks — the globs are*.ymland*.shonly.- Transitive calls — a script invoked only by another script, rather than by a workflow or hook, is not discovered.
Hook wiring is checked (test_every_hook_file_is_actually_WIRED reads .claude/settings.json
and the husky hooks with full-line comments stripped), so a hook file whose registration is deleted
fails rather than keeping a row that reads as coverage. Its limit, stated because the check reads
stronger than it is: it is a substring test for the basename, not a parse of the invocation.
: # .claude/hooks/decisions-guard.sh disabled still reads as wired, and conversely a hook invoked
through a wrapper or a constructed path reads as unwired. It catches deletion, which is the common
case; it does not catch deliberate disablement. The check does not extend to the scripts/ half at
all.
Inventory
| Guard | Blocks | Kind | Proof | Proof ref |
|---|---|---|---|---|
.claude/hooks/decisions-guard.sh |
a commit | GUARD | NONE | — |
.claude/hooks/design-sync-reminder.sh |
the first Stop after a UI change (one-shot, then allows) | GUARD | NONE | — |
.claude/hooks/posttooluse-worktree-marker.sh |
nothing (writes the marker the worktree guard reads) | GUARD | NONE | — |
.claude/hooks/prepush-clean-worktree-check.sh |
a push with uncommitted changes in the pushed set | GUARD | NONE | — |
.claude/hooks/prepush-donewhen.sh |
a direct push to main with unticked Done-when boxes |
GUARD | NONE | — |
.claude/hooks/prepush-rebase-check.sh |
a push from a branch behind origin/main |
GUARD | BEHAVIOUR-ONLY | test_prepush_rebase_check_tag_exemption.py::test_zero_ref_lines_does_not_exempt |
.claude/hooks/pretooluse-agent-model.sh |
an Agent dispatch naming no model (asks) | GUARD | NONE | — |
.claude/hooks/pretooluse-agent-ram.sh |
an Agent dispatch under 10% free RAM | GUARD | NONE | — |
.claude/hooks/pretooluse-bash-guard.sh |
a Bash call setting ETV_UPDATE_GOLDENS |
GUARD | NONE | — |
.claude/hooks/pretooluse-bom-guard.sh |
a commit/push carrying a BOM in a touched .cs |
GUARD | MUTATION | test_bom_guard_detection.py::test_DISARMING_the_BOM_comparison_stops_detection |
.claude/hooks/pretooluse-merge-consent.sh |
a PR merge without derived consent | GUARD | BEHAVIOUR-ONLY | test_merge_consent_exemption.py::test_protected_path_on_a_LATER_page_is_still_seen |
.claude/hooks/pretooluse-nav-guard.sh |
a browser navigate to a streaming URL | GUARD | NONE | — |
.claude/hooks/pretooluse-worktree-guard.sh |
a commit/merge in a foreign worktree | GUARD | NONE | — |
.husky/commit-msg |
a commit with no Co-Authored-By trailer |
GUARD | NONE | — |
.husky/pre-commit |
a commit failing lint-staged, decisions, root-PNG or format | GUARD | NONE | — |
.husky/pre-push |
a push failing any pre-push hook or the SPA gate | GUARD | NONE | — |
scripts/build_decisions_catalog.py |
the decisions-guard job, on a stale catalog |
GUARD | NONE | — |
scripts/check-kickoff-guard.sh |
the decisions-guard job, on a revived #237 reference |
GUARD | NONE | — |
scripts/check-review-verdict.sh |
the merge-consent hook's verdict classification | GUARD | BEHAVIOUR-ONLY | test_check_review_verdict.py::test_falseopen_token_must_be_a_whole_word |
scripts/ci-detect-already-validated.sh |
nothing directly (feeds the skip gate) | GUARD | NONE | — |
scripts/ci-detect-docs-only.sh |
nothing directly (feeds the skip gate) | GUARD | NONE | — |
scripts/ci-peak-anon.sh |
nothing (samples container memory) | TOOLING | NONE | — |
scripts/ci-prove-ban-detects.sh |
the release path, if the delimiter ban is disarmed | GUARD | NONE | — |
scripts/ci-step-ran.sh |
the two required contexts, on a dropped step | GUARD | MUTATION | test_ci_dropped_step_guard.py::test_dropping_ANY_single_step_FAILS_the_guard |
scripts/decisions_validate.py |
the decisions-guard job, on a lifecycle fault |
GUARD | MUTATION | test_decisions_validate.py::test_main_actually_CALLS_the_wing_scan |
scripts/e2e-functional.sh |
the Functional E2E job, on a failed HTTP contract assertion | GUARD | NONE | — |
scripts/e2e-local.sh |
nothing (boots a local instance) | TOOLING | NONE | — |
scripts/e2e-ui.sh |
nothing (drives the Playwright flows) | TOOLING | NONE | — |
scripts/hook-fire-log.sh |
nothing (records that each hook fired, and reports it) | TOOLING | NONE | — |
scripts/jq-preflight.sh |
the script-tests job, on a jq version change |
GUARD | BEHAVIOUR-ONLY | test_jq_preflight.py::test_below_the_floor_is_LOUD |
scripts/post-review-verdict.sh |
nothing (writes the verdict status) | GUARD | BEHAVIOUR-ONLY | test_post_review_verdict.py::test_never_retargets_the_verdict_at_the_new_head |
scripts/pr-changed-files.sh |
the verdict exemption, on an incomplete enumeration | GUARD | BEHAVIOUR-ONLY | test_pr_changed_files.py::test_a_SHORT_page_does_not_end_the_enumeration |
scripts/prove-fix.sh |
the prove-fix job, on a commit whose Proves: trailer names a test that passes without the fix |
GUARD | MUTATION | test_prove_fix.py::test_MUTATION_disarming_the_UNPROVEN_clause_reddens_the_refusal_test |
scripts/update-openapi.sh |
nothing (regenerates the spec) | TOOLING | NONE | — |
scripts/tests/test_bom_guard_detection.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_build_catalog.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_check_review_verdict.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_ci_dropped_step_guard.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_ci_image_pin_population.py |
the script-tests job, when a container job loses its pin |
GUARD | MUTATION | test_ci_image_pin_population.py::test_a_single_job_losing_its_pin_is_DETECTED |
scripts/tests/test_ci_release_path_scan_job.py |
the script-tests job, on a weakened release-path scan job |
GUARD | NONE | — |
scripts/tests/test_decisions_lib.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_decisions_validate.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_guard_inventory.py |
the script-tests job, on an unclassified guard or a stale proof ref |
GUARD | MUTATION | test_guard_inventory.py::test_the_inventory_covers_exactly_the_guards_that_exist |
scripts/tests/test_hook_fire_log.py |
the script-tests job, on a hook that stops reporting that it fired, or whose reporting changes what the harness sees |
GUARD | MUTATION | test_hook_fire_log.py::test_a_hook_that_LOSES_its_instrumentation_is_DETECTED |
scripts/tests/test_jq_preflight.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_merge_consent_base_change.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_merge_consent_exemption.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_merge_consent_required_check.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_migration_equivalence.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_post_review_verdict.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_pr_changed_files.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_prepush_rebase_check_tag_exemption.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_prove_fix.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_remote_state_inventory.py |
the script-tests job, on an executable that talks to a remote service with no row in docs/remote-state-inventory.md |
GUARD | MUTATION | test_remote_state_inventory.py::test_MUTATION_PROOF_a_dropped_row_and_a_phantom_row_are_both_detected |
What the numbers say
34 guards, 5 tooling scripts, 15 proof files. 8 guards carry a mutation proof; 6 are
behaviour-only; 20 have none. These figures are asserted against the table by
test_the_summary_counts_match_the_table — they were wrong in the first draft (28/4/6/3/19 against
a table holding 27/5/6/3/18), because a hand-maintained summary of a table is a second copy of it,
which is the duplication family this change argues against. Both cold reviewers found the error
independently.
One guard was WITHDRAWN rather than shipped, and it is worth a line here because the inventory
is where a future session will look for it. A test_review_verdict_vocabulary_parity.py asserted
set equality between the H10 verdict vocabulary's write side and read side by extracting both from
shell source with regexes. Six cold-review rounds each found another shell construction that either
escaped it or made it red on a correct tree. It was deleted rather than patched a seventh time: the
testing.guard-derives-population-from-source record says a weak detector is itself the
symptom-keyed mistake, and this was the empirical demonstration. The underlying duplication is real
and is now UNMITIGATED — tracked in #788, which fixes it by removing the second copy.
What that file asserted, and where each part went — enumerated because the first attempt at this
withdrawal silently dropped one and claimed otherwise, which is
process.enumerate-workaround-behaviors-before-deleting failing on its own removal:
| Invariant it asserted | Where it is now |
|---|---|
| Bidirectional write/read vocabulary parity | LOST. The reason for the withdrawal; #788 |
| No READ-side word is both positive and negative | LOST as a universal property; #788 |
| No WRITE-side word is both success and failure | LOST as a universal property; #788 |
| Strict extractor consumes every case arm | Retired with the extractor; #788 |
POS_RE/NEG_RE each assigned exactly once |
Retired with the extractor; #788 |
| Both extractors find non-empty sets (anti-vacuity) | Retired with the extractor; #788 |
| A wrong write source makes the extractors disagree | Retired with the extractor; #788 |
Two guards were ADDED in its place. They are not replacements for rows 2 and 3 and the table above deliberately does not list them as such — an earlier draft did, and cold review was right that relabelling a lost invariant as a narrower surviving one is the exact failure this table exists to prevent, committed by the table:
| Added | What it pins |
|---|---|
test_check_review_verdict.py::test_each_verdict_word_retains_its_established_polarity |
the five established tokens still READ with their established polarity |
test_post_review_verdict.py::test_each_verdict_word_posts_its_established_polarity |
the five established tokens still POST with their established polarity |
Polarity is strictly weaker than disjointness, and the gap is demonstrable. Put MERGEABLE in
BOTH case arms of post-review-verdict.sh: the success arm wins, every polarity assertion stays
green, and the withdrawn disjointness test failed. What the two added guards do catch is the
dangerous direction — a token that a reviewer means as BLOCKED silently reading or posting as
approval, which writes a green review-verdict/h10. A token added to one script and not the other
is untested by either, which is the whole of #788.
A guard was WITHDRAWN from test_hook_fire_log.py after four iterations, recorded here because
this is where a future session will look for it. A test_no_redirection_in_the_sink_PRECEDES_its_stderr_redirect
generalised the exec … 2>/dev/null rule to any command by pattern-matching shell source. Each
iteration fixed a false positive or false negative the previous one introduced — >/dev/null 2>&1
flagged wrongly, then operators preceded by a digit (exec 0<) missed entirely, then multi-command
lines flagged wrongly — and cold review then constructed more of both ([[ "$x" < "$y" ]] 2>/dev/null,
a backslash continuation, a > inside a quoted string). Deleted rather than patched a fifth time,
on the same reasoning as the vocabulary-parity withdrawal above.
**test_hook_fire_log.py asserts TWO clauses
That unproven set carries no number here on purpose. It restated the count, drifted the moment
the BOM guard was regraded, and test_the_summary_counts_match_the_table cannot see it — the parser
checks the formatted summary sentence and nothing else. A second hand-maintained copy of a number is
the duplication family this file argues against, so the copy is removed rather than corrected: the
set is every PreToolUse hook except merge-consent and the BOM guard, and the count is in the
summary above.
They are now observable but still unproven, and the two words carry different weight. Observable:
every hook records its own execution through scripts/hook-fire-log.sh, so "did this hook fire, and
what did it decide" is a measurement — run scripts/hook-fire-log.sh report (#776). Unproven:
nobody has demonstrated any of them is load-bearing, which is what #785 tracks. Observability tells
you a guard ran; only a mutation tells you it would have caught anything. The BOM guard is the case
that shows why the distinction matters — it was firing on every commit the whole time it was
fail-open.
The gaps are not uniform in cost, and the ranking that matters is what a silent failure would let through, not test count:
— now proven, and it was fail-open the whole time. Ranked first here because the defect it guards has recurred three times (#311, #402, #405); that ranking turned out to be right for a worse reason than intended. It detected a BOM withpretooluse-bom-guard.shxxd -p, andxxdships with vim and is absent on the Linux CI runner, so the comparison never matched and every BOM was allowed in silence.odnow, with a clause-level mutation proof. The lesson for the rows below: an unproven guard is not merely untested, it is a guard whose current behaviour nobody has established.pretooluse-worktree-guard.sh+posttooluse-worktree-marker.sh— a two-file mechanism guarding #289, where a regression in either half is invisible and the two halves have never been tested together..husky/pre-push:11'sunset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE— a one-line fix for a real bug (a nestedgit diff --exit-codesilently reporting no diff) that nothing pins. Reordering it after the nested git calls reintroduces the bug silently.scripts/build_decisions_catalog.py— its--checkpath is what CI runs and no test callsmain()at all; the tests exerciserender_catalog()directly.
Filling these is tracked rather than done in one pass, deliberately: a mutation proof written to close a row is the kind of test that passes for the wrong reason.