Files
ersatztv/docs/guard-inventory.md
T
timothy 6a4265d81d
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 40s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m30s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m16s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m52s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Failing after 17s
feat(784): a doc records the end state — generalize the no-session-narrative rule (#811)
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-22 00:23:22 +00:00

27 KiB

Guard inventory (ersatztv#774 / #775)

Every executable guard file in this repo, what it blocks, and whether it ships a proof it can go red. scripts/tests/test_guard_inventory.py derives the population from the filesystem and the workflow/hook call sites and asserts set equality against the Guard column, so a new guard cannot be added without acquiring a row here, and a row cannot name a proof that does not exist.

Read docs/decisions/records/testing/guard-derives-population-from-source.md and …/guard-ships-with-mutation-proof.md before editing a guard or adding a row.

Columns

  • Kind — GUARD (it can block a commit, a push, a tool call, a merge or a CI job), TOOLING (it does work but asserts nothing; listed so its absence from the guard set is a recorded decision, not an oversight), or PROOF (a scripts/tests/ file whose job is to prove another guard). PROOF exists to stop a regress: once test files entered the population, every mutation proof became a row wanting a proof of its own. A scripts/tests/ file that enforces a repo invariant with no separate guard behind it is a GUARD, graded normally, and may cite a mutation case in its own file.
  • Proof — MUTATION: a clause-level mutation was executed and this named test was witnessed red. The test either performs the disarm itself (a monkeypatch, a deselection, a removed marker) or a recorded incident shows it going red when the clause went away. BEHAVIOUR-ONLY: a real test drives the guard through its real entry point with good and bad input, but no one has demonstrated that removing the clause reddens it. NONE.
  • Proof ref — file.py::function, verified to exist by the inventory test.

The distinction between MUTATION and BEHAVIOUR-ONLY is the whole point of #775 and is not a grading curve. A behavioural test proves the guard reacts to its input; only a mutation proves the guard is load-bearing. #685 shipped two guards on one condition where deleting either left the suite green, and every behavioural test passed throughout.

The definition above is the second one. The first read "a named test disarms this guard's clause and asserts red", and three rows were graded MUTATION against it that do not disarm anything — they feed the real script an input only that clause rejects (empty stdin, a short page, a full first page). Cold review caught it, and the objection is right twice over: those tests are the same species as ones graded BEHAVIOUR-ONLY eight rows away, so the column was being applied as a curve on the very day it was introduced. They are regraded. The surviving criterion is witnessed, not plausible — "removing this clause would surely redden that test" is an argument, and this table exists because arguments of that shape have been wrong here six times.

Scope limit, stated rather than implied

This inventory covers guard files, discovered by globbing .claude/hooks/*.sh, .husky/* and scripts/tests/test_*.py, plus every scripts/… path referenced by a workflow or a hook. Six classes are outside that population. They are listed because the first version of this section named only the first one, and cold review found that the very guards this inventory shipped with were sitting in the gap:

  1. Guards inline in workflow YAML — most importantly pr-checks.yml:ci-image-pin. "Which jobs are guards" needs a judgement call per job the filesystem cannot supply. Two were audited under #774 and one fixed; extending the population is tracked in #786.
  2. C# and TypeScript guards — ErsatzTV.Mcp.Tests/ToolCatalogTests.cs and web/src/api/pageSizeCallSites.guard.test.ts are both structural guards and neither has a row.
  3. Mentions counted as call sites. The scripts/… scrape matches any occurrence, including inside a comment or an ::error:: string. scripts/update-openapi.sh is named in a pr-checks.yml error message, so removing the step that runs it would leave its row intact.
  4. Nested and non-lowercase paths beyond scripts/tests/ — a guard under scripts/scripted-schedules/, or with an uppercase name, is invisible to the scrape.
  5. .yaml workflows and non-.sh hooks — the globs are *.yml and *.sh only.
  6. Transitive calls — a script invoked only by another script, rather than by a workflow or hook, is not discovered.

Hook wiring is checked (test_every_hook_file_is_actually_WIRED reads .claude/settings.json and the husky hooks with full-line comments stripped), so a hook file whose registration is deleted fails rather than keeping a row that reads as coverage. Its limit, stated because the check reads stronger than it is: it is a substring test for the basename, not a parse of the invocation. : # .claude/hooks/decisions-guard.sh disabled still reads as wired, and conversely a hook invoked through a wrapper or a constructed path reads as unwired. It catches deletion, which is the common case; it does not catch deliberate disablement. The check does not extend to the scripts/ half at all.

Inventory

Guard Blocks Kind Proof Proof ref
.claude/hooks/decisions-guard.sh a commit GUARD NONE —
.claude/hooks/design-sync-reminder.sh the first Stop after a UI change (one-shot, then allows) GUARD NONE —
.claude/hooks/posttooluse-worktree-marker.sh nothing (writes the marker the worktree guard reads) GUARD MUTATION test_worktree_ownership_guard.py::test_MUTATION_a_marker_hook_that_stops_WRITING_makes_the_guard_go_quiet
.claude/hooks/prepush-clean-worktree-check.sh a push with uncommitted changes in the pushed set GUARD NONE —
.claude/hooks/prepush-donewhen.sh a direct push to main with unticked Done-when boxes GUARD NONE —
.claude/hooks/prepush-rebase-check.sh a push from a branch behind origin/main GUARD BEHAVIOUR-ONLY test_prepush_rebase_check_tag_exemption.py::test_zero_ref_lines_does_not_exempt
.claude/hooks/pretooluse-agent-model.sh an Agent dispatch naming no model (asks) GUARD NONE —
.claude/hooks/pretooluse-agent-ram.sh an Agent dispatch under 10% free RAM GUARD NONE —
.claude/hooks/pretooluse-bash-guard.sh a Bash call setting ETV_UPDATE_GOLDENS GUARD NONE —
.claude/hooks/pretooluse-bom-guard.sh a commit/push carrying a BOM in a touched .cs GUARD MUTATION test_bom_guard_detection.py::test_DISARMING_the_BOM_comparison_stops_detection
.claude/hooks/pretooluse-merge-consent.sh a PR merge without derived consent GUARD BEHAVIOUR-ONLY test_merge_consent_exemption.py::test_protected_path_on_a_LATER_page_is_still_seen
.claude/hooks/pretooluse-nav-guard.sh a browser navigate to a streaming URL GUARD NONE —
.claude/hooks/pretooluse-worktree-guard.sh a commit/merge in a foreign worktree GUARD MUTATION test_worktree_ownership_guard.py::test_MUTATION_disarming_the_guards_MARKER_READ_stops_the_deny
.husky/commit-msg a commit with no Co-Authored-By trailer GUARD NONE —
.husky/pre-commit a commit failing lint-staged, decisions, root-PNG or format GUARD NONE —
.husky/pre-push a push failing any pre-push hook or the SPA gate GUARD MUTATION test_prepush_unsets_git_env.py::test_MUTATION_DELETING_the_unset_lets_drift_through_silently
scripts/build_decisions_catalog.py the decisions-guard job, on a stale catalog GUARD MUTATION test_build_catalog_check_path.py::test_MUTATION_disarming_the_stale_comparison_stops_detection
scripts/check-doc-narrative.py nothing, by design (advisory ::warning:: only, exits 0 on every path (the error/degradation arms are defensive and unproven — see the record) — docs.no-session-narrative says a string predicate over prose may not be load-bearing) TOOLING NONE —
scripts/check-kickoff-guard.sh the decisions-guard job, on a revived #237 reference GUARD NONE —
scripts/check-review-verdict.sh the merge-consent hook's verdict classification GUARD BEHAVIOUR-ONLY test_check_review_verdict.py::test_falseopen_token_must_be_a_whole_word
scripts/ci-detect-already-validated.sh nothing directly (feeds the skip gate) GUARD NONE —
scripts/ci-detect-docs-only.sh nothing directly (feeds the skip gate) GUARD NONE —
scripts/ci-peak-anon.sh nothing (samples container memory) TOOLING NONE —
scripts/ci-prove-ban-detects.sh the release path, if the delimiter ban is disarmed GUARD NONE —
scripts/ci-step-ran.sh the two required contexts, on a dropped step GUARD MUTATION test_ci_dropped_step_guard.py::test_dropping_ANY_single_step_FAILS_the_guard
scripts/decisions_validate.py the decisions-guard job, on a lifecycle fault GUARD MUTATION test_decisions_validate.py::test_main_actually_CALLS_the_wing_scan
scripts/e2e-functional.sh the Functional E2E job, on a failed HTTP contract assertion GUARD NONE —
scripts/e2e-local.sh nothing (boots a local instance) TOOLING NONE —
scripts/e2e-ui.sh nothing (drives the Playwright flows) TOOLING NONE —
scripts/hook-fire-log.sh nothing (records that each hook fired, and reports it) TOOLING NONE —
scripts/jq-preflight.sh the script-tests job, on a jq version change GUARD BEHAVIOUR-ONLY test_jq_preflight.py::test_below_the_floor_is_LOUD
scripts/post-review-verdict.sh nothing (writes the verdict status) GUARD BEHAVIOUR-ONLY test_post_review_verdict.py::test_never_retargets_the_verdict_at_the_new_head
scripts/pr-changed-files.sh the verdict exemption, on an incomplete enumeration GUARD BEHAVIOUR-ONLY test_pr_changed_files.py::test_a_SHORT_page_does_not_end_the_enumeration
scripts/prove-fix.sh the prove-fix job, on a commit whose Proves: trailer names a test that passes without the fix GUARD MUTATION test_prove_fix.py::test_MUTATION_disarming_the_UNPROVEN_clause_reddens_the_refusal_test
scripts/update-openapi.sh nothing (regenerates the spec) TOOLING NONE —
scripts/tests/test_bom_guard_detection.py the script-tests job PROOF NONE —
scripts/tests/test_build_catalog.py the script-tests job PROOF NONE —
scripts/tests/test_build_catalog_check_path.py the script-tests job PROOF NONE —
scripts/tests/test_check_review_verdict.py the script-tests job PROOF NONE —
scripts/tests/test_check_doc_narrative.py the script-tests job PROOF NONE —
scripts/tests/test_ci_dropped_step_guard.py the script-tests job PROOF NONE —
scripts/tests/test_ci_image_pin_population.py the script-tests job, when a container job loses its pin GUARD MUTATION test_ci_image_pin_population.py::test_a_single_job_losing_its_pin_is_DETECTED
scripts/tests/test_ci_release_path_scan_job.py the script-tests job, on a weakened release-path scan job GUARD NONE —
scripts/tests/test_decisions_lib.py the script-tests job PROOF NONE —
scripts/tests/test_decisions_validate.py the script-tests job PROOF NONE —
scripts/tests/test_guard_inventory.py the script-tests job, on an unclassified guard or a stale proof ref GUARD MUTATION test_guard_inventory.py::test_the_inventory_covers_exactly_the_guards_that_exist
scripts/tests/test_hook_fire_log.py the script-tests job, on a hook that stops reporting that it fired, or whose reporting changes what the harness sees GUARD MUTATION test_hook_fire_log.py::test_a_hook_that_LOSES_its_instrumentation_is_DETECTED
scripts/tests/test_jq_preflight.py the script-tests job PROOF NONE —
scripts/tests/test_merge_consent_base_change.py the script-tests job PROOF NONE —
scripts/tests/test_merge_consent_exemption.py the script-tests job PROOF NONE —
scripts/tests/test_merge_consent_required_check.py the script-tests job PROOF NONE —
scripts/tests/test_migration_equivalence.py the script-tests job PROOF NONE —
scripts/tests/test_post_review_verdict.py the script-tests job PROOF NONE —
scripts/tests/test_prepush_unsets_git_env.py the script-tests job PROOF NONE —
scripts/tests/test_pr_changed_files.py the script-tests job PROOF NONE —
scripts/tests/test_prepush_rebase_check_tag_exemption.py the script-tests job PROOF NONE —
scripts/tests/test_prove_fix.py the script-tests job PROOF NONE —
scripts/tests/test_remote_state_inventory.py the script-tests job, on an executable that talks to a remote service with no row in docs/remote-state-inventory.md GUARD MUTATION test_remote_state_inventory.py::test_MUTATION_PROOF_a_dropped_row_and_a_phantom_row_are_both_detected
scripts/tests/test_worktree_ownership_guard.py the script-tests job PROOF NONE —

What the numbers say

34 guards, 6 tooling scripts, 19 proof files. 12 guards carry a mutation proof; 6 are behaviour-only; 16 have none. These figures are asserted against the table by test_the_summary_counts_match_the_table — they were wrong in the first draft (28/4/6/3/19 against a table holding 27/5/6/3/18), because a hand-maintained summary of a table is a second copy of it, which is the duplication family this change argues against. Both cold reviewers found the error independently.

One guard was WITHDRAWN rather than shipped, and it is worth a line here because the inventory is where a future session will look for it. A test_review_verdict_vocabulary_parity.py asserted set equality between the H10 verdict vocabulary's write side and read side by extracting both from shell source with regexes. Six cold-review rounds each found another shell construction that either escaped it or made it red on a correct tree. It was deleted rather than patched a seventh time: the testing.guard-derives-population-from-source record says a weak detector is itself the symptom-keyed mistake, and this was the empirical demonstration. The underlying duplication is real and is now UNMITIGATED — tracked in #788, which fixes it by removing the second copy.

What that file asserted, and where each part went — enumerated because the first attempt at this withdrawal silently dropped one and claimed otherwise, which is process.enumerate-workaround-behaviors-before-deleting failing on its own removal:

Invariant it asserted Where it is now
Bidirectional write/read vocabulary parity LOST. The reason for the withdrawal; #788
No READ-side word is both positive and negative LOST as a universal property; #788
No WRITE-side word is both success and failure LOST as a universal property; #788
Strict extractor consumes every case arm Retired with the extractor; #788
POS_RE/NEG_RE each assigned exactly once Retired with the extractor; #788
Both extractors find non-empty sets (anti-vacuity) Retired with the extractor; #788
A wrong write source makes the extractors disagree Retired with the extractor; #788

Two guards were ADDED in its place. They are not replacements for rows 2 and 3 and the table above deliberately does not list them as such — an earlier draft did, and cold review was right that relabelling a lost invariant as a narrower surviving one is the exact failure this table exists to prevent, committed by the table:

Added What it pins
test_check_review_verdict.py::test_each_verdict_word_retains_its_established_polarity the five established tokens still READ with their established polarity
test_post_review_verdict.py::test_each_verdict_word_posts_its_established_polarity the five established tokens still POST with their established polarity

Polarity is strictly weaker than disjointness, and the gap is demonstrable. Put MERGEABLE in BOTH case arms of post-review-verdict.sh: the success arm wins, every polarity assertion stays green, and the withdrawn disjointness test failed. What the two added guards do catch is the dangerous direction — a token that a reviewer means as BLOCKED silently reading or posting as approval, which writes a green review-verdict/h10. A token added to one script and not the other is untested by either, which is the whole of #788.

A guard was WITHDRAWN from test_hook_fire_log.py after four iterations, recorded here because this is where a future session will look for it. A test_no_redirection_in_the_sink_PRECEDES_its_stderr_redirect generalised the exec … 2>/dev/null rule to any command by pattern-matching shell source. Each iteration fixed a false positive or false negative the previous one introduced — >/dev/null 2>&1 flagged wrongly, then operators preceded by a digit (exec 0<) missed entirely, then multi-command lines flagged wrongly — and cold review then constructed more of both ([[ "$x" < "$y" ]] 2>/dev/null, a backslash continuation, a > inside a quoted string). Deleted rather than patched a fifth time, on the same reasoning as the vocabulary-parity withdrawal above.

test_hook_fire_log.py proves THREE clauses, and its row claims two of them. Coverage (every hook reports that it fired) via test_a_hook_that_LOSES_its_instrumentation_is_DETECTED; transparency (the wrapper changes nothing the harness can see) via test_instrumentation_changes_NOTHING_the_harness_can_see with its mutation proof test_DELETING_the_replay_makes_the_differential_go_RED; and placement (etv_hook_fire_begin must precede the stdin read) via test_begin_placed_AFTER_the_stdin_read_is_DETECTED. The row's Blocks column covers the first two — "stops reporting that it fired, or whose reporting changes what the harness sees" — and the Proof ref column holds one ref because the column holds one, not because the second is unproven. Placement is proved and unclaimed.

Its test_the_suite_does_not_write_to_the_PRODUCTION_log is narrower than its docstring, which says conftest.py "must isolate every test, not just this file's". What it checks is that the fixture set ETV_HOOK_FIRE_LOG_DIR for the test currently running and that a hook it drives in its own sandbox does not touch the real log. It cannot see another suite that snapshots os.environ at IMPORT time — before the autouse fixture runs — and hands that stale mapping to its subprocesses. That suite's hooks then write to $HOME/.cache/ersatztv/hook-fire/ while every assertion stays green, because the fire-log library is fail-open. #785 shipped exactly that defect, inside the file added to prove those very hooks.

The reproduction is the part worth keeping: restore a module-level {**os.environ} snapshot — or leave the helper correct and point a single env= argument at one — then run that file and count records for its synthetic session ids. 58 per run, measured identically on macOS/git 2.55 and Linux/git 2.47.3. (The accumulated total observed before the fix ran to four figures across many runs; that is an observation rather than a reproducible measurement, so check the per-run figure.)

The pin is test_worktree_ownership_guard.py::test_driving_a_hook_LANDS_its_records_in_the_ISOLATED_dir, which asserts the EFFECT — records land in the fixture's dir — rather than the shape of the fix. Its predecessor asserted the helper's return value, and cold review showed that leaves the call site unguarded: _env() correct, one env= reverted, all tests green, records still leaking. Pinning to a hand-written revert rather than to the property is verify-against-the-REAL-predecessor.

It is still narrower than the property: it guards the launch path it drives. A second launcher in the same file that passed a stale environment would leave it green — measured, 18 records — because the hooks it drives would still log correctly. Every hook in that file goes through one helper today, which is what makes it sufficient there. Generalising it is #809, and the reason that is hard is that the obvious version races against a real session's hooks firing during the run.

That unproven set carries no number here on purpose. It restated the count, drifted the moment the BOM guard was regraded, and test_the_summary_counts_match_the_table cannot see it — the parser checks the formatted summary sentence and nothing else. A second hand-maintained copy of a number is the duplication family this file argues against, so the copy is removed rather than corrected: the set is the eight .claude/hooks/ guards enumerated in the standing list below, and the count is in the summary above. Naming them by event is what made the earlier wording wrong twice — they do not share one event, and design-sync-reminder.sh is registered on both PreToolUse and Stop in .claude/settings.json, so any "every X hook" phrasing double-counts it.

They are now observable but still unproven, and the two words carry different weight. Observable: every hook records its own execution through scripts/hook-fire-log.sh, so "did this hook fire, and what did it decide" is a measurement — run scripts/hook-fire-log.sh report (#776). Unproven: nobody has demonstrated any of them is load-bearing — the first group in the standing list below. Observability tells you a guard ran; only a mutation tells you it would have caught anything. The BOM guard is the case that shows why the distinction matters — it was firing on every commit the whole time it was fail-open.

The gaps are not uniform in cost, and the ranking that matters is what a silent failure would let through, not test count. All four ranked entries now carry clause-level mutation proofs (#785); they are kept here with what each mutation established, because the ranking is the reusable part and because two of them turned out to be worse than the ranking predicted.

  1. pretooluse-bom-guard.sh — proven, and it was fail-open the whole time. Ranked first because the defect it guards has recurred three times (#311, #402, #405); that ranking turned out to be right for a worse reason than intended. It detected a BOM with xxd -p, and xxd ships with vim and is absent on the Linux CI runner, so the comparison never matched and every BOM was allowed in silence. od now. The lesson for the rows below: an unproven guard is not merely untested, it is a guard whose current behaviour nobody has established.
  2. pretooluse-worktree-guard.sh + posttooluse-worktree-marker.sh — proven as a pair. Four clauses were disarmed and witnessed red: the guard's marker read; the guard's ownership comparison (inverted, because disarming it the other way only makes the guard deny more and every deny assertion stays green); the commit|merge alternation, whose merge half every other case in the file left untested while guarding the plumbing-merge path; and — the one that could not exist while the halves were tested apart — the marker hook's write, asserted against the guard's decision. Both hooks are deliberately fail-open, so an absent mechanism and a working one produce the identical "commit allowed".
  3. .husky/pre-push:11's unset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE — proven, and the case is the normal one rather than an edge. Git exports GIT_DIR to pre-push when the push comes from a worktree and not from the main tree; process.shared-tree-readonly makes the worktree the mandated way to work here, so every push takes the exposed path. With GIT_DIR set and GIT_WORK_TREE unset git stops discovering the repo and treats the current directory as the work tree, so cd web && npm run check:api's git diff --exit-code compares against index paths that do not exist and reports no diff. Both the deletion and the relocation are proved.
  4. scripts/build_decisions_catalog.py — proven, including the wiring. The --check comparison is mutated directly, and a separate subprocess case runs the command derived from pr-checks.yml against a copied corpus. That second case is not redundant: replacing raise SystemExit(main()) with a bare main() leaves the script printing docs/decisions/README.md is stale on stderr while exiting 0, and the workflow step reads nothing but the exit code. Only the subprocess case reddens — the #751/#719 shape.

What a file-level MUTATION grade does and does not claim, because three of these four rows are multi-purpose files. The grade covers the clause the cited case actually mutates, per testing.guard-ships-with-mutation-proof; it is not a statement about every line in the file. .husky/pre-push runs four other things, and its row asserts only that line 11 is load-bearing — the three hooks it invokes carry their own rows and their own grades. Reading the row as "pre-push is tested" is the same relabelling error the withdrawn-guard table above exists to prevent. Clause-level grading is tracked in #790.

The remaining unproven guards, and why each is still NONE rather than merely unattended — the third ## Done-when box of #785, answered here rather than left implicit. They fall into three groups, and the split is the point: the first two are backlogs, the third is not.

  • The eight remaining hook guards — genuinely unproven, and a real backlog. Four are PreToolUse (pretooluse-agent-model.sh, pretooluse-agent-ram.sh, pretooluse-bash-guard.sh, pretooluse-nav-guard.sh), two are pre-push (prepush-clean-worktree-check.sh, prepush-donewhen.sh), one is pre-commit (decisions-guard.sh) and one is registered on both PreToolUse and Stop (design-sync-reminder.sh); they are grouped by their status, not by their event, because the events do not partition them. They are now observable (scripts/hook-fire-log.sh report, #776), which is a weaker claim than proven and is stated as such above.
  • .husky/commit-msg and .husky/pre-commit — unproven, and each carries its own clause, so they are a backlog too rather than "covered by what they call". pre-commit dispatches to lint-staged and decisions-guard.sh, but the root-level-*.png refusal and the dotnet format whitespace --verify-no-changes block are its own. commit-msg is not a dispatcher at all: it is one grep -q '^Co-Authored-By:' plus a MERGE_HEAD exemption, and nothing tests that the exemption fires only for merges. Neither is covered by what it calls — the trap is to read "it dispatches to guards" as "its own clauses are guarded".
  • ci-detect-already-validated.sh, ci-detect-docs-only.sh, ci-prove-ban-detects.sh, e2e-functional.sh, check-kickoff-guard.sh, test_ci_release_path_scan_job.py — not unattended either, but each needs its own harness rather than a proof written to clear a row. ci-prove-ban-detects.sh is the exception noted in testing.guard-ships-with-mutation-proof: it runs its own mutation at CI time, because what it proves is disarmable from inside pytest.

That list is prose and nothing checks it, which is the honest limit — a guard moving out of a group will not redden anything. It is here so the next session inherits why a row is NONE, which is the distinction #785 asked for; the machine-checked half remains the table.

Filling the rest is tracked rather than done in one pass, deliberately: a mutation proof written to close a row is the kind of test that passes for the wrong reason.