Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
38 KiB
Guard inventory (ersatztv#774 / #775)
Every executable guard file in this repo, what it blocks, and whether it ships a proof it can go
red. scripts/tests/test_guard_inventory.py derives the population from the git index and the
workflow/hook call sites and asserts set equality against the Guard column, so a new guard
cannot be added without acquiring a row here, and a row cannot name a proof that does not exist.
The index rather than a filesystem walk since ersatztv#806 — the practical consequence is that a new
guard joins the population when it is staged, not when the file appears. Nothing local runs
these checks at all: .husky/pre-commit runs lint-staged, the decisions guard, the root-PNG check
and dotnet format, and grep -rn pytest .husky/ returns nothing. The runner is
pr-checks.yml::script-tests, on: pull_request, so the red arrives in CI — plus
docker-build.yml:723 on the release path, which re-runs two of these files
(test_ci_dropped_step_guard.py and test_ci_release_path_scan_job.py) as a needs: of build.
Read docs/decisions/records/testing/guard-derives-population-from-source.md and
…/guard-ships-with-mutation-proof.md before editing a guard or adding a row.
Columns
- Kind —
GUARD(it can block a commit, a push, a tool call, a merge or a CI job),TOOLING(it does work but asserts nothing; listed so its absence from the guard set is a recorded decision, not an oversight), orPROOF(ascripts/tests/file whose job is to prove another guard).PROOFexists to stop a regress: once test files entered the population, every mutation proof became a row wanting a proof of its own. Ascripts/tests/file that enforces a repo invariant with no separate guard behind it is aGUARD, graded normally, and may cite a mutation case in its own file. - Proof —
MUTATION: a clause-level mutation was executed and this named test was witnessed red. The test either performs the disarm itself (a monkeypatch, a deselection, a removed marker) or a recorded incident shows it going red when the clause went away.BEHAVIOUR-ONLY: a real test drives the guard through its real entry point with good and bad input, but no one has demonstrated that removing the clause reddens it.NONE. - Proof ref —
file.py::function, verified to exist by the inventory test.
The distinction between MUTATION and BEHAVIOUR-ONLY is the whole point of #775 and is not a
grading curve. A behavioural test proves the guard reacts to its input; only a mutation proves the
guard is load-bearing. #685 shipped two guards on one condition where deleting either left the
suite green, and every behavioural test passed throughout.
The definition above is the second one. The first read "a named test disarms this guard's clause
and asserts red", and three rows were graded MUTATION against it that do not disarm anything —
they feed the real script an input only that clause rejects (empty stdin, a short page, a full first
page). Cold review caught it, and the objection is right twice over: those tests are the same
species as ones graded BEHAVIOUR-ONLY eight rows away, so the column was being applied as a curve
on the very day it was introduced. They are regraded. The surviving criterion is witnessed, not
plausible — "removing this clause would surely redden that test" is an argument, and this table
exists because arguments of that shape have been wrong here six times.
Scope limit, stated rather than implied
This inventory covers guard files, discovered by reading .claude/hooks/*.sh, .husky/* and
scripts/tests/test_*.py out of the git index, plus every scripts/… path referenced by a
workflow or a hook. The classes below are outside that population. They are listed because the first
version of this section named only the first one, and cold review found that the very guards this
inventory shipped with were sitting in the gap:
- Guards inline in workflow YAML — most importantly
pr-checks.yml:ci-image-pin. "Which jobs are guards" needs a judgement call per job the filesystem cannot supply. Two were audited under #774 and one fixed; extending the population is tracked in #786. - C# and TypeScript guards —
ErsatzTV.Mcp.Tests/ToolCatalogTests.cs,web/src/api/pageSizeCallSites.guard.test.tsandweb/src/api/completeRequest.guard.test.tsare all structural guards and none has a row. Note what that costs: this list is a HAND-WRITTEN mirror of a population nothing derives, so it goes stale silently and CI stays green — #807 added the third entry, and only review caught that the second had become the only one named. - Mentions counted as call sites. The
scripts/…scrape matches any occurrence, including inside a comment or an::error::string.scripts/update-openapi.shis named in apr-checks.ymlerror message, so removing the step that runs it would leave its row intact. - Nested and non-lowercase paths beyond
scripts/tests/— a guard underscripts/scripted-schedules/, or with an uppercase name, is invisible to the scrape. - Non-
.shhooks — the hook pattern is*.shonly. (.yamlworkflows are no longer in this gap: the caller scan matches*.ymland*.yamlsince #806, because Gitea accepts both.) - Transitive calls — a script invoked only by another script, rather than by a workflow or hook, is not discovered.
- Non-
test_modules underscripts/tests/— the pattern istest_*.py, soconftest.py,mutation_harness_lib.py,mutation_manifest.pyandtracked_files.pyare outside the population and hold no rows. They are not guards (they assert nothing on their own), but the middle two ARE whattest_mutation_harness.pyis made of, so gutting either would take that guard with it. What catches that is the guard's own row: its declared mutation targetsmutation_harness_lib.py, and its proof test refuses to run if the clause it names has moved.tracked_files.pyhas no such backstop — it is load-bearing for every module that imports it (#806), and an edit to it is covered only by those modules' own proofs. Recorded rather than force-fitted: a row for a library would need aKindthe vocabulary does not have. - Nested workflow directories — the workflow scope is direct children of
.gitea/workflows, so a tracked.gitea/workflows/nested/x.ymlis invisible to the caller scan and totest_ci_image_pin_population.py. Left as scope rather than widened: whether Gitea executes nested workflow files was not verified here, and widening on an unverified premise risks a permanent red on a correct tree, which is how a correct guard gets deleted.
Hook wiring is checked (test_every_hook_file_is_actually_WIRED reads .claude/settings.json
and the husky hooks with full-line comments stripped), so a hook file whose registration is deleted
fails rather than keeping a row that reads as coverage. Its limit, stated because the check reads
stronger than it is: it is a substring test for the basename, not a parse of the invocation.
: # .claude/hooks/decisions-guard.sh disabled still reads as wired, and conversely a hook invoked
through a wrapper or a constructed path reads as unwired. It catches deletion, which is the common
case; it does not catch deliberate disablement. The check does not extend to the scripts/ half at
all.
File populations and where they come from (ersatztv#806)
Every guard here whose members are FILES derives them from the git index, never a filesystem
walk. The disk is not an authoritative source: it reports build output and editor droppings and
differs per machine, so a guard derived from it asserts a different population in CI than on the
laptop of the person it is meant to stop. scripts/tests/tracked_files.py is the single derivation
and carries the full rationale; scripts/tests/test_guard_populations_derive_from_git.py proves it,
in both directions: removing EVERY member of each registered derivation from the index one at a time
and requiring it to disappear while still on disk, and watching for a directory LISTING issued while
the derivation runs (reading files stays allowed). Removal alone is blind to a source that
contributes only untracked members — an rglob reaching .husky/_/ adds and never removes — and
any check phrased as "an untracked file must not enter" is itself machine-dependent, because the
untracked file has to exist. Watching for the call needs no arranged state.
That second check is a regression guard against the accidental shapes, not a boundary: what is observed is any call that goes through one of the spies, whenever it happens — the spy records into a list that outlives the patch, so a reference captured during the window and invoked after it still counts. Whether the call goes through a spy is what decides, not when. The instance list — what never reaches a spy at all — lives in the check's own docstring and is deliberately not restated here, because a second copy of it drifted from the first within one commit.
One deliberate exception to "from the index" sits in the same file: the registration check lists
scripts/tests/test_*.py from disk on purpose, because it is a superset check over what pytest
collects — an untracked stray there makes it MORE demanding, never blind, whereas using the index
would let an unstaged new guard escape registration.
Its own limit, stated because a check described as complete stops being re-examined: it finds
derivations by PARSING each test_*.py for an import of the shared helper, so a module that derives
a file population some other way is invisible to it, and no mechanical check can close that (#774
reached the same conclusion about detecting filter-shaped guards by token).
The audit #806 asked for, recorded whichever way it came out, because "we looked and left it" and "we never looked" are indistinguishable a year later:
| Guard | Population | Completeness claim over tracked files? | Outcome |
|---|---|---|---|
test_guard_inventory.py |
.claude/hooks/*.sh, .husky/*, scripts/tests/test_*.py, workflow/hook callers |
yes — set equality against this table | converted to the index; .husky/_/ had been excluded only because _ is a directory, so the obvious "make it recursive" edit would have reintroduced #778's defect here |
test_hook_fire_log.py |
.claude/hooks/*.sh |
yes — every hook must be instrumented | converted; an untracked scratch .sh used to demand instrumentation and redden the suite on that checkout alone |
test_ci_image_pin_population.py |
.gitea/workflows/*.yml + *.yaml |
yes — "docker-build is the ONLY workflow pinning the toolchain image" | converted, and *.yaml added: Gitea accepts both spellings, so a .yaml workflow was structurally invisible while the test read as covering all of them |
test_pr_changed_files.py |
.gitea/workflows/*.y*ml |
yes — "no OTHER workflow writes the review-verdict status" | converted. Not on #806's list and found by cold review on the pushed head: an untracked .yaml dropped in .gitea/workflows/ reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population |
web/src/api/pageSizeCallSites.guard.test.ts |
web/src/**/*.{ts,tsx,mts,cts} via import.meta.glob |
yes — an unregistered discovered site fails | DEFERRED to #819, assessed not skipped. The glob is a documented workaround: @types/node is deliberately out of tsconfig.app.json, and wiring it in was tried and reverted (it leaked Node's setTimeout into the app project and broke three unrelated tests), so there is no node:child_process to reach the index from. Over-enumerates, so it fails loudly rather than going blind |
test_ci_release_path_scan_job.py |
.gitea/workflows/*.y*ml + scripts/** |
no — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: shutil.copytree copied whatever was on disk, including untracked files and scripts/__pycache__, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring |
test_ci_dropped_step_guard.py |
the parsed workflow document | no filesystem population at all | unchanged; its residual is MARKED_JOBS, a SCOPE mirror of the required contexts on main, which #806 does not close |
test_remote_state_inventory.py fixed its own population under #778 and kept a private copy of the
derivation; #806 folded it onto the shared one, so that module is covered by the proof above like
the rest. Across the whole change, every module that derived a file population its own way now goes
through tracked_files.py — one implementation of the rule instead of one per module. The
registered derivations are listed in DERIVATIONS in
scripts/tests/test_guard_populations_derive_from_git.py; this page deliberately keeps no count
of them.
Still on filesystem walks, deliberately out of scope: the decisions corpus
(scripts/decisions_lib.py's active_files(), and the suites over it). Its members are docs/
Markdown with no generated-file pressure and a different lifecycle, and folding it in here would
have been the reflex this milestone argues against — a change with no defect behind it. It is
recorded as unexamined rather than as cleared.
Inventory
| Guard | Blocks | Kind | Proof | Proof ref |
|---|---|---|---|---|
.claude/hooks/decisions-guard.sh |
a commit | GUARD | NONE | — |
.claude/hooks/design-sync-reminder.sh |
the first Stop after a UI change (one-shot, then allows) | GUARD | NONE | — |
.claude/hooks/posttooluse-worktree-marker.sh |
nothing (writes the marker the worktree guard reads) | GUARD | MUTATION | test_worktree_ownership_guard.py::test_MUTATION_a_marker_hook_that_stops_WRITING_makes_the_guard_go_quiet |
.claude/hooks/prepush-clean-worktree-check.sh |
a push with uncommitted changes in the pushed set | GUARD | NONE | — |
.claude/hooks/prepush-donewhen.sh |
a direct push to main with unticked Done-when boxes |
GUARD | NONE | — |
.claude/hooks/prepush-rebase-check.sh |
a push from a branch behind origin/main |
GUARD | BEHAVIOUR-ONLY | test_prepush_rebase_check_tag_exemption.py::test_zero_ref_lines_does_not_exempt |
.claude/hooks/pretooluse-agent-model.sh |
an Agent dispatch naming no model (asks) | GUARD | NONE | — |
.claude/hooks/pretooluse-agent-ram.sh |
an Agent dispatch under 10% free RAM | GUARD | NONE | — |
.claude/hooks/pretooluse-bash-guard.sh |
a Bash call setting ETV_UPDATE_GOLDENS |
GUARD | NONE | — |
.claude/hooks/pretooluse-bom-guard.sh |
a commit/push carrying a BOM in a touched .cs |
GUARD | MUTATION | test_bom_guard_detection.py::test_DISARMING_the_BOM_comparison_stops_detection |
.claude/hooks/pretooluse-merge-consent.sh |
a PR merge without derived consent | GUARD | BEHAVIOUR-ONLY | test_merge_consent_exemption.py::test_protected_path_on_a_LATER_page_is_still_seen |
.claude/hooks/pretooluse-nav-guard.sh |
a browser navigate to a streaming URL | GUARD | NONE | — |
.claude/hooks/pretooluse-worktree-guard.sh |
a commit/merge in a foreign worktree | GUARD | MUTATION | test_worktree_ownership_guard.py::test_MUTATION_disarming_the_guards_MARKER_READ_stops_the_deny |
.husky/commit-msg |
a commit with no Co-Authored-By trailer |
GUARD | NONE | — |
.husky/pre-commit |
a commit failing lint-staged, decisions, root-PNG or format | GUARD | NONE | — |
.husky/pre-push |
a push failing any pre-push hook or the SPA gate | GUARD | MUTATION | test_prepush_unsets_git_env.py::test_MUTATION_DELETING_the_unset_lets_drift_through_silently |
scripts/build_decisions_catalog.py |
the decisions-guard job, on a stale catalog |
GUARD | MUTATION | test_build_catalog_check_path.py::test_MUTATION_disarming_the_stale_comparison_stops_detection |
scripts/check-doc-narrative.py |
nothing, by design (advisory ::warning:: only, exits 0 on every path (the error/degradation arms are defensive and unproven — see the record) — docs.no-session-narrative says a string predicate over prose may not be load-bearing) |
TOOLING | NONE | — |
scripts/check-kickoff-guard.sh |
the decisions-guard job, on a revived #237 reference |
GUARD | NONE | — |
scripts/check-review-verdict.sh |
the merge-consent hook's verdict classification | GUARD | BEHAVIOUR-ONLY | test_check_review_verdict.py::test_falseopen_token_must_be_a_whole_word |
scripts/ci-detect-already-validated.sh |
nothing directly (feeds the skip gate) | GUARD | NONE | — |
scripts/ci-detect-docs-only.sh |
nothing directly (feeds the skip gate) | GUARD | NONE | — |
scripts/ci-peak-anon.sh |
nothing (samples container memory) | TOOLING | NONE | — |
scripts/ci-prove-ban-detects.sh |
the release path, if the delimiter ban is disarmed | GUARD | NONE | — |
scripts/ci-step-ran.sh |
the two required contexts, on a dropped step | GUARD | MUTATION | test_ci_dropped_step_guard.py::test_dropping_ANY_single_step_FAILS_the_guard |
scripts/ci-toolchain-image-resolves.sh |
the toolchain-preflight job, when the pinned CI toolchain image has been deleted from the registry |
GUARD | MUTATION | test_ci_toolchain_image_resolves.py::test_MUTATION_a_deleted_tag_is_reported_as_a_failure |
scripts/decisions_validate.py |
the decisions-guard job, on a lifecycle fault |
GUARD | MUTATION | test_decisions_validate.py::test_main_actually_CALLS_the_wing_scan |
scripts/e2e-functional.sh |
the Functional E2E job, on a failed HTTP contract assertion | GUARD | NONE | — |
scripts/e2e-local.sh |
nothing (boots a local instance) | TOOLING | NONE | — |
scripts/e2e-ui.sh |
nothing (drives the Playwright flows) | TOOLING | NONE | — |
scripts/hook-fire-log.sh |
nothing (records that each hook fired, and reports it) | TOOLING | NONE | — |
scripts/jq-preflight.sh |
the script-tests job, on a jq version change |
GUARD | BEHAVIOUR-ONLY | test_jq_preflight.py::test_below_the_floor_is_LOUD |
scripts/post-review-verdict.sh |
nothing (writes the verdict status) | GUARD | BEHAVIOUR-ONLY | test_post_review_verdict.py::test_never_retargets_the_verdict_at_the_new_head |
scripts/pr-changed-files.sh |
the verdict exemption, on an incomplete enumeration | GUARD | BEHAVIOUR-ONLY | test_pr_changed_files.py::test_a_SHORT_page_does_not_end_the_enumeration |
scripts/prove-fix.sh |
the prove-fix job, on a commit whose Proves: trailer names a test that passes without the fix |
GUARD | MUTATION | test_prove_fix.py::test_MUTATION_disarming_the_UNPROVEN_clause_reddens_the_refusal_test |
scripts/update-openapi.sh |
nothing (regenerates the spec) | TOOLING | NONE | — |
scripts/tests/test_bom_guard_detection.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_build_catalog.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_build_catalog_check_path.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_check_review_verdict.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_check_doc_narrative.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_ci_dropped_step_guard.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_ci_image_pin_population.py |
the script-tests job, when a container job loses its pin |
GUARD | MUTATION | test_ci_image_pin_population.py::test_a_single_job_losing_its_pin_is_DETECTED |
scripts/tests/test_ci_release_path_scan_job.py |
the script-tests job, on a weakened release-path scan job |
GUARD | NONE | — |
scripts/tests/test_ci_toolchain_image_resolves.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_decisions_lib.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_decisions_validate.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_guard_inventory.py |
the script-tests job, on an unclassified guard or a stale proof ref |
GUARD | MUTATION | test_guard_inventory.py::test_the_inventory_covers_exactly_the_guards_that_exist |
scripts/tests/test_guard_populations_derive_from_git.py |
the script-tests job, on a guard whose file population admits a file git does not track |
GUARD | MUTATION | test_guard_populations_derive_from_git.py::test_no_derivation_admits_an_untracked_file |
scripts/tests/test_hook_fire_log.py |
the script-tests job, on a hook that stops reporting that it fired, or whose reporting changes what the harness sees |
GUARD | MUTATION | test_hook_fire_log.py::test_a_hook_that_LOSES_its_instrumentation_is_DETECTED |
scripts/tests/test_jq_preflight.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_merge_consent_base_change.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_merge_consent_exemption.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_merge_consent_required_check.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_migration_equivalence.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_mutation_harness.py |
the script-tests job, when a MUTATION row's declared clause no longer reddens the test the row names |
GUARD | MUTATION | test_mutation_harness.py::test_MUTATION_disarming_the_DIAGNOSTIC_gate_accepts_a_red_for_the_wrong_reason |
scripts/tests/test_post_review_verdict.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_prepush_unsets_git_env.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_pr_changed_files.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_prepush_rebase_check_tag_exemption.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_optional_request_members.py |
the script-tests job, on an OpenAPI request schema that can silently drop a member with no stated disposition |
GUARD | MUTATION | test_optional_request_members.py::test_every_droppable_request_schema_has_a_stated_disposition |
scripts/tests/test_prove_fix.py |
the script-tests job |
PROOF | NONE | — |
scripts/tests/test_remote_state_inventory.py |
the script-tests job, on an executable that talks to a remote service with no row in docs/remote-state-inventory.md |
GUARD | MUTATION | test_remote_state_inventory.py::test_every_in_scope_file_has_a_row_and_every_row_names_a_real_file |
scripts/tests/test_worktree_ownership_guard.py |
the script-tests job |
PROOF | NONE | — |
The MUTATION column is executed, not asserted
Every row graded MUTATION carries a DECLARED clause mutation in scripts/tests/mutation_manifest.py.
scripts/tests/test_mutation_harness.py applies each one to an isolated copy of this repository and
requires that row's own named test to go red; the manifest and the MUTATION rows are compared for
set equality in both directions, so the grade and the mutation cannot drift apart. Adding a row
graded MUTATION without declaring its clause fails the suite. Full contract and its limits:
docs/decisions/records/testing/mutation-claims-are-executed.md.
Two things that column still does not say. It does not say the declared clause is the ONLY thing the
guard hangs on — for three rows (measured 2026-08-22) the redness arrives through the
proof test's own "this clause has moved, RETARGET it" assertion rather than through changed
behaviour, which catches the recorded proof going stale but not much else. And one row is graded DETECTOR rather than CLAUSE in the
manifest: instrumentation_faults accumulates from four independent arms and a stripped hook trips
three at once, so no single-arm disarm reddens its proof. That finer mutation is carried as data and
re-run every suite, and must keep surviving — if it starts reddening, the guard has become
clause-provable and the entry is regraded.
What the numbers say
38 guards, 6 tooling scripts, 20 proof files. 16 guards carry a mutation proof; 6 are
behaviour-only; 16 have none. These figures are asserted against the table by
test_the_summary_counts_match_the_table — they were wrong in the first draft (28/4/6/3/19 against
a table holding 27/5/6/3/18), because a hand-maintained summary of a table is a second copy of it,
which is the duplication family this change argues against. Both cold reviewers found the error
independently.
One guard was WITHDRAWN rather than shipped, and it is worth a line here because the inventory
is where a future session will look for it. A test_review_verdict_vocabulary_parity.py asserted
set equality between the H10 verdict vocabulary's write side and read side by extracting both from
shell source with regexes. Six cold-review rounds each found another shell construction that either
escaped it or made it red on a correct tree. It was deleted rather than patched a seventh time: the
testing.guard-derives-population-from-source record says a weak detector is itself the
symptom-keyed mistake, and this was the empirical demonstration. The underlying duplication is real
and is now UNMITIGATED — tracked in #788, which fixes it by removing the second copy.
What that file asserted, and where each part went — enumerated because the first attempt at this
withdrawal silently dropped one and claimed otherwise, which is
process.enumerate-workaround-behaviors-before-deleting failing on its own removal:
| Invariant it asserted | Where it is now |
|---|---|
| Bidirectional write/read vocabulary parity | LOST. The reason for the withdrawal; #788 |
| No READ-side word is both positive and negative | LOST as a universal property; #788 |
| No WRITE-side word is both success and failure | LOST as a universal property; #788 |
| Strict extractor consumes every case arm | Retired with the extractor; #788 |
POS_RE/NEG_RE each assigned exactly once |
Retired with the extractor; #788 |
| Both extractors find non-empty sets (anti-vacuity) | Retired with the extractor; #788 |
| A wrong write source makes the extractors disagree | Retired with the extractor; #788 |
Two guards were ADDED in its place. They are not replacements for rows 2 and 3 and the table above deliberately does not list them as such — an earlier draft did, and cold review was right that relabelling a lost invariant as a narrower surviving one is the exact failure this table exists to prevent, committed by the table:
| Added | What it pins |
|---|---|
test_check_review_verdict.py::test_each_verdict_word_retains_its_established_polarity |
the five established tokens still READ with their established polarity |
test_post_review_verdict.py::test_each_verdict_word_posts_its_established_polarity |
the five established tokens still POST with their established polarity |
Polarity is strictly weaker than disjointness, and the gap is demonstrable. Put MERGEABLE in
BOTH case arms of post-review-verdict.sh: the success arm wins, every polarity assertion stays
green, and the withdrawn disjointness test failed. What the two added guards do catch is the
dangerous direction — a token that a reviewer means as BLOCKED silently reading or posting as
approval, which writes a green review-verdict/h10. A token added to one script and not the other
is untested by either, which is the whole of #788.
A guard was WITHDRAWN from test_hook_fire_log.py after four iterations, recorded here because
this is where a future session will look for it. A test_no_redirection_in_the_sink_PRECEDES_its_stderr_redirect
generalised the exec … 2>/dev/null rule to any command by pattern-matching shell source. Each
iteration fixed a false positive or false negative the previous one introduced — >/dev/null 2>&1
flagged wrongly, then operators preceded by a digit (exec 0<) missed entirely, then multi-command
lines flagged wrongly — and cold review then constructed more of both ([[ "$x" < "$y" ]] 2>/dev/null,
a backslash continuation, a > inside a quoted string). Deleted rather than patched a fifth time,
on the same reasoning as the vocabulary-parity withdrawal above.
test_hook_fire_log.py proves THREE clauses, and its row claims two of them. Coverage (every
hook reports that it fired) via test_a_hook_that_LOSES_its_instrumentation_is_DETECTED;
transparency (the wrapper changes nothing the harness can see) via
test_instrumentation_changes_NOTHING_the_harness_can_see with its mutation proof
test_DELETING_the_replay_makes_the_differential_go_RED; and placement (etv_hook_fire_begin must
precede the stdin read) via test_begin_placed_AFTER_the_stdin_read_is_DETECTED. The row's Blocks
column covers the first two — "stops reporting that it fired, or whose reporting changes what the
harness sees" — and the Proof ref column holds one ref because the column holds one, not because
the second is unproven. Placement is proved and unclaimed.
Its test_the_suite_does_not_write_to_the_PRODUCTION_log is narrower than its docstring, which
says conftest.py "must isolate every test, not just this file's". What it checks is that the
fixture set ETV_HOOK_FIRE_LOG_DIR for the test currently running and that a hook it drives in
its own sandbox does not touch the real log. It cannot see another suite that snapshots
os.environ at IMPORT time — before the autouse fixture runs — and hands that stale mapping to its
subprocesses. That suite's hooks then write to $HOME/.cache/ersatztv/hook-fire/ while every
assertion stays green, because the fire-log library is fail-open. #785 shipped exactly that defect,
inside the file added to prove those very hooks.
The reproduction is the part worth keeping: restore a module-level {**os.environ} snapshot — or
leave the helper correct and point a single env= argument at one — then run that file and count
records for its synthetic session ids. 58 per run, measured identically on macOS/git 2.55 and
Linux/git 2.47.3. (The accumulated total observed before the fix ran to four figures across many
runs; that is an observation rather than a reproducible measurement, so check the per-run figure.)
The pin is test_worktree_ownership_guard.py::test_driving_a_hook_LANDS_its_records_in_the_ISOLATED_dir,
which asserts the EFFECT — records land in the fixture's dir — rather than the shape of the fix. Its
predecessor asserted the helper's return value, and cold review showed that leaves the call site
unguarded: _env() correct, one env= reverted, all tests green, records still leaking. Pinning to
a hand-written revert rather than to the property is verify-against-the-REAL-predecessor.
It is still narrower than the property: it guards the launch path it drives. A second launcher in the same file that passed a stale environment would leave it green — measured, 18 records — because the hooks it drives would still log correctly. Every hook in that file goes through one helper today, which is what makes it sufficient there. Generalising it is #809, and the reason that is hard is that the obvious version races against a real session's hooks firing during the run.
That unproven set carries no number here on purpose. It restated the count, drifted the moment
the BOM guard was regraded, and test_the_summary_counts_match_the_table cannot see it — the parser
checks the formatted summary sentence and nothing else. A second hand-maintained copy of a number is
the duplication family this file argues against, so the copy is removed rather than corrected: the
set is the eight .claude/hooks/ guards enumerated in the standing list below, and the count is in
the summary above. Naming them by event is what made the earlier wording wrong twice — they do not
share one event, and design-sync-reminder.sh is registered on both PreToolUse and Stop in
.claude/settings.json, so any "every X hook" phrasing double-counts it.
They are now observable but still unproven, and the two words carry different weight. Observable:
every hook records its own execution through scripts/hook-fire-log.sh, so "did this hook fire, and
what did it decide" is a measurement — run scripts/hook-fire-log.sh report (#776). Unproven:
nobody has demonstrated any of them is load-bearing — the first group in the standing list below.
Observability tells you a guard ran; only a mutation tells you it would have caught anything. The
BOM guard is the case that shows why the distinction matters — it was firing on every commit the
whole time it was fail-open.
The gaps are not uniform in cost, and the ranking that matters is what a silent failure would let through, not test count. All four ranked entries now carry clause-level mutation proofs (#785); they are kept here with what each mutation established, because the ranking is the reusable part and because two of them turned out to be worse than the ranking predicted.
— proven, and it was fail-open the whole time. Ranked first because the defect it guards has recurred three times (#311, #402, #405); that ranking turned out to be right for a worse reason than intended. It detected a BOM withpretooluse-bom-guard.shxxd -p, andxxdships with vim and is absent on the Linux CI runner, so the comparison never matched and every BOM was allowed in silence.odnow. The lesson for the rows below: an unproven guard is not merely untested, it is a guard whose current behaviour nobody has established.— proven as a pair. Four clauses were disarmed and witnessed red: the guard's marker read; the guard's ownership comparison (inverted, because disarming it the other way only makes the guard deny more and every deny assertion stays green); thepretooluse-worktree-guard.sh+posttooluse-worktree-marker.shcommit|mergealternation, whosemergehalf every other case in the file left untested while guarding the plumbing-merge path; and — the one that could not exist while the halves were tested apart — the marker hook's write, asserted against the guard's decision. Both hooks are deliberately fail-open, so an absent mechanism and a working one produce the identical "commit allowed".— proven, and the case is the normal one rather than an edge. Git exports.husky/pre-push:11'sunset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILEGIT_DIRtopre-pushwhen the push comes from a worktree and not from the main tree;process.shared-tree-readonlymakes the worktree the mandated way to work here, so every push takes the exposed path. WithGIT_DIRset andGIT_WORK_TREEunset git stops discovering the repo and treats the current directory as the work tree, socd web && npm run check:api'sgit diff --exit-codecompares against index paths that do not exist and reports no diff. Both the deletion and the relocation are proved.— proven, including the wiring. Thescripts/build_decisions_catalog.py--checkcomparison is mutated directly, and a separate subprocess case runs the command derived frompr-checks.ymlagainst a copied corpus. That second case is not redundant: replacingraise SystemExit(main())with a baremain()leaves the script printingdocs/decisions/README.md is staleon stderr while exiting 0, and the workflow step reads nothing but the exit code. Only the subprocess case reddens — the #751/#719 shape.
What a file-level MUTATION grade does and does not claim, because three of these four rows are
multi-purpose files. The grade covers the clause the cited case actually mutates, per
testing.guard-ships-with-mutation-proof; it is not a statement about every line in the file.
.husky/pre-push runs four other things, and its row asserts only that line 11 is load-bearing —
the three hooks it invokes carry their own rows and their own grades. Reading the row as "pre-push
is tested" is the same relabelling error the withdrawn-guard table above exists to prevent.
Clause-level grading is tracked in #790.
The remaining unproven guards, and why each is still NONE rather than merely unattended — the
third ## Done-when box of #785, answered here rather than left implicit. They fall into three
groups, and the split is the point: the first two are backlogs, the third is not.
- The eight remaining hook guards — genuinely unproven, and a real backlog. Four are
PreToolUse(pretooluse-agent-model.sh,pretooluse-agent-ram.sh,pretooluse-bash-guard.sh,pretooluse-nav-guard.sh), two are pre-push (prepush-clean-worktree-check.sh,prepush-donewhen.sh), one is pre-commit (decisions-guard.sh) and one is registered on bothPreToolUseandStop(design-sync-reminder.sh); they are grouped by their status, not by their event, because the events do not partition them. They are now observable (scripts/hook-fire-log.sh report, #776), which is a weaker claim than proven and is stated as such above. .husky/commit-msgand.husky/pre-commit— unproven, and each carries its own clause, so they are a backlog too rather than "covered by what they call".pre-commitdispatches tolint-stagedanddecisions-guard.sh, but the root-level-*.pngrefusal and thedotnet format whitespace --verify-no-changesblock are its own.commit-msgis not a dispatcher at all: it is onegrep -q '^Co-Authored-By:'plus aMERGE_HEADexemption, and nothing tests that the exemption fires only for merges. Neither is covered by what it calls — the trap is to read "it dispatches to guards" as "its own clauses are guarded".ci-detect-already-validated.sh,ci-detect-docs-only.sh,ci-prove-ban-detects.sh,e2e-functional.sh,check-kickoff-guard.sh,test_ci_release_path_scan_job.py— not unattended either, but each needs its own harness rather than a proof written to clear a row.ci-prove-ban-detects.shis the exception noted intesting.guard-ships-with-mutation-proof: it runs its own mutation at CI time, because what it proves is disarmable from inside pytest.
That list is prose and nothing checks it, which is the honest limit — a guard moving out of a group
will not redden anything. It is here so the next session inherits why a row is NONE, which is
the distinction #785 asked for; the machine-checked half remains the table.
Filling the rest is tracked rather than done in one pass, deliberately: a mutation proof written to close a row is the kind of test that passes for the wrong reason.