diff --git a/.claude/hooks/pretooluse-merge-consent.sh b/.claude/hooks/pretooluse-merge-consent.sh index 330e6626c..8412c1218 100755 --- a/.claude/hooks/pretooluse-merge-consent.sh +++ b/.claude/hooks/pretooluse-merge-consent.sh @@ -472,25 +472,17 @@ else # directions, and it is rare in practice: as of 2026-08-19 this repo's only rule is the plain # name `main`, which the classifier resolves to `exact` on every run. That is a dated # observation about mutable remote config, not a property to rely on. - bp_verdict=$(printf '%s' "$bp_list" | jq --arg b "$base_ref" -c ' - def esc: gsub("(?[.+?^${}()|\\[\\]\\\\])"; "\\" + .c); - def offs: [match("[*?\\[\\]{}\\\\]"; "g").offset]; - def superset: . as $n | (offs) as $o - | ($n[0:$o[0]] | esc) + ".*" + ($n[($o[-1]+1):] | esc); - def nonascii: explode | any(. > 127); - . as $rules | $b as $base | - ($rules | map(select((.branch_name // .rule_name // "") as $n - | (($n|offs|length) == 0) - and (($n|ascii_downcase) == ($base|ascii_downcase))))) as $exacts | - (($base|nonascii) or ($rules | any((.branch_name // .rule_name // "") as $n - | ($n|offs|length) == 0 and ($n|nonascii)))) as $unfoldable | - if ($rules | any((.branch_name // .rule_name // "") as $n - | (($n|offs|length) > 0) - and ($base | test("^" + ($n|superset) + "$")))) then {verdict:"undecidable"} - elif $unfoldable then {verdict:"undecidable"} - elif ($exacts | length) > 1 then {verdict:"undecidable"} - elif ($exacts | length) == 1 then {verdict:"exact", rule:($exacts | first)} - else {verdict:"none"} end' 2>/dev/null || true) + # The classifier is a FILE now (ersatztv#787), so its absence is a new failure mode: `jq -f` on a + # missing program exits 2 with empty stdout, which reaches the `*)` arm below and asks that "this + # repo's branch-protection rules came back in a shape this hook could not parse" — blaming the + # payload for a missing local file. That is precisely the states-a-cause-that-did-not-happen defect + # the two comments beside that arm were written to fix, so it is checked here rather than inherited. + classifier="$repo_root/scripts/lib/branch-rule-classifier.jq" + if [ ! -r "$classifier" ]; then + rm -f "$bp_file" + decide ask "H6/H10 merge gate: the shared branch-protection rule classifier is missing or unreadable at $classifier, so which rule governs '$base_ref' — and therefore whether 'review-verdict/h10' is required on it — could not be derived (ersatztv#787). Restore the file, or confirm the required checks manually." + fi + bp_verdict=$(printf '%s' "$bp_list" | jq --arg b "$base_ref" -c -f "$classifier" 2>/dev/null || true) case $(printf '%s' "$bp_verdict" | jq -r '.verdict // ""' 2>/dev/null || true) in exact) bp=$(printf '%s' "$bp_verdict" | jq -c '.rule' 2>/dev/null || true); bp_code=200 ;; undecidable) rm -f "$bp_file" @@ -614,6 +606,119 @@ case "$class" in decide ask "H10 merge gate: unrecognized verdict classification '$class' for PR #$pr. Confirm the review covered the latest commit ($short) before merging." ;; esac +# --- (d) Guard-scope freshness (ersatztv#787): the committed mirror of `main`'s required status +# checks must still match the server. ------------------------------------------------------ +# ORDERED LAST, and that is a severity argument rather than a stylistic one. Every check above +# can DENY; this one can only ever downgrade an otherwise-satisfied auto-grant to a prompt. Run +# earlier it would preempt those verdicts and report a stale guard scope at a reader whose merge +# is blocked for a completely different and more serious reason, and it would ask on payloads the +# checks above are about to reject anyway. Placed here it is also PAST the point where the two +# merge paths converge, so it covers both without duplicating anything. +# `scripts/tests/test_ci_dropped_step_guard.py` DERIVES which jobs must carry per-step execution +# markers from `.gitea/required-status-contexts.json`, because its CI job checks out with +# `persist-credentials: false` and cannot ask Gitea. That makes the snapshot the single +# hand-maintained input in the chain: a fourth required context added on the server leaves the +# snapshot — and therefore the guard's scope — silently behind, which is the whole of #787. +# +# THIS RUNS ON BOTH MERGE PATHS, deliberately, and it is placed here rather than beside the +# branch-protection read in the scheduled-auto-merge branch for that reason. +# +# WHAT IT DOES NOT COVER, said here rather than left to be discovered: a PR whose changed files are +# all docs/process — `.gitea/` included — exits at the docs-only passthrough far above, so this arm +# never runs for it. A PR that edits ONLY `.gitea/required-status-contexts.json` is docs-only BY +# CONSTRUCTION, and that is exactly the snapshot-NARROWING direction the decision record names as +# this design's residual. Excluding that path from the allow-list would not buy the protection it +# looks like it would: this arm compares the live server against the snapshot in the LOCAL CHECKOUT, +# not against the version the PR proposes, so it cannot see a narrowing that has not landed yet. +# What does hold is that the passthrough is a passthrough — a human prompt, never an auto-grant — +# which is the `.gitea/` treatment ersatztv#317 asked for. That read is inside +# `else` (mwcs = true) and never executes on an immediate merge, which is the common case; hanging +# the freshness check off it would fire it only when an auto-merge is armed. This file already +# records that exact defect one section up — the base re-read "first landed inside the +# scheduled-auto-merge branch only", and cold review found scheduled+retarget denied while +# immediate+retarget auto-GRANTED. Same shape, so it is not repeated here. +# +# It reads `main` (the branch the snapshot names), NOT `$base_ref`. That is a DIFFERENT question +# from the one the scheduled branch asks — "is review-verdict/h10 required on the base I am merging +# into" — so this is not a second copy of that classifier and the two cannot drift into disagreeing: +# they consume different fields of different rules for different decisions. +# +# ASK, NEVER DENY. Drift does not make THIS merge unsafe: Gitea enforces the live required set +# server-side, so a newly required context with no status blocks the merge on its own. What has gone +# stale is a guard's scope — a different artifact, on a different clock. Denying would state +# something false about the change in front of the reader. Every non-`match` class asks, so a +# comparison that could not be made is surfaced rather than skipped (`unknown` is not `fine`). +# ONE base for both the checker and the snapshot, and it is `$repo_root` — derived from this file's +# own location — rather than `$CLAUDE_PROJECT_DIR`. Two reasons, and the second is the load-bearing +# one. Resolving them from different roots would let the hook classify one checkout's snapshot with +# another checkout's script, mismatched halves of a comparison whose whole job is to detect a +# mismatch. And an ENV VAR is not a sound input to a security decision: a wrong value pointing at a +# tree that happens to contain an executable checker returns `match` about a different checkout +# entirely, which silently authorizes the grant. A missing path only asks, so the failure is quiet +# exactly where it is worst. +ctx_base="$repo_root" +ctx_snapshot="$ctx_base/.gitea/required-status-contexts.json" +ctx_script="$ctx_base/scripts/check-required-contexts.sh" + +# THIS ARM IS ABOUT ONE REPO, and the merge tool is not. Every other check here reads +# `$owner/$repo` from the tool input and is repo-agnostic; this one compares a HARDCODED branch +# against a snapshot committed in THIS checkout. Merging a PR in another repo from a session opened +# here would otherwise weigh that repo's live contexts against this repo's mirror and report a +# confident, flatly false finding about it — measured: server-management returns `[]`, which +# classifies as `nomatch`. So the snapshot names the repo it describes and the arm runs only for it. +# An unreadable snapshot cannot answer "is this my repo?" either, so it asks rather than skipping. +ctx_repo=$(jq -r 'if (.repo | type) == "string" then .repo else "" end' "$ctx_snapshot" 2>/dev/null || true) +if [ -z "$ctx_repo" ]; then + decide ask "H6 merge gate: $ctx_snapshot is missing, unreadable, or names no \`repo\`, so the dropped-step guard's scope could not be checked against branch protection — nor could it be established whether this snapshot even describes $owner/$repo (ersatztv#787). Restore the file, or check the required checks manually." +fi +# CASE-FOLDED, because Gitea resolves owner/repo case-insensitively: verified live, both +# `/repos/timothy/ersatztv` and `/repos/TIMOTHY/ErsatzTV` answer 200. A byte-exact compare would let +# any case variant sail through every other arm and SKIP this one, so drift would go unreported with +# no ask — the gate failing open on a spelling. The hook already treats case folding as +# decision-relevant one section up, where `MAIN` vs `main` makes the governing rule undecidable. +ctx_repo_fold=$(printf '%s' "$ctx_repo" | tr '[:upper:]' '[:lower:]') +target_repo_fold=$(printf '%s' "$owner/$repo" | tr '[:upper:]' '[:lower:]') +if [ "$ctx_repo_fold" = "$target_repo_fold" ]; then +if [ ! -x "$ctx_script" ]; then + decide ask "H6 merge gate: the required-contexts checker is missing or not executable at $ctx_script, so whether the dropped-step guard's scope still matches branch protection on 'main' could not be derived (ersatztv#787). Check it manually, or restore the script." +fi +bpf=$(mktemp) || decide ask "H6 merge gate: could not allocate a temp file to read branch protection for the guard-scope freshness check (ersatztv#787)." +if [ -n "${ETV_GITEA_TOKEN:-}" ]; then + ctx_code=$(curl -s -o "$bpf" -w '%{http_code}' -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true) +else + ctx_code=$(curl -s -o "$bpf" -w '%{http_code}' -u "$ETV_GITEA_BASICAUTH" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true) +fi +if [ "$ctx_code" = "200" ]; then + # stderr is KEPT, not sent to /dev/null. The checker exits 2 with a diagnostic on a usage error — + # an unreadable snapshot, a branch mismatch, a missing classifier — and discarding it made all of + # those arrive at the operator as the catch-all's "returned 'nothing'", which names no cause. That + # is the same states-a-cause-that-did-not-happen shape this arm was careful about elsewhere. + ctx_class=$("$ctx_script" --branch main --snapshot "$ctx_snapshot" < "$bpf" 2>"$bpf.err" || true) + ctx_diag=$(tr '\n' ' ' < "$bpf.err" 2>/dev/null | cut -c1-300 || true) +else + ctx_class=readfail + ctx_diag="" +fi +rm -f "$bpf" "$bpf.err" +case "$ctx_class" in + match) : ;; + drift) + decide ask "H6 merge gate: the required status checks on 'main' no longer match .gitea/required-status-contexts.json (ersatztv#787). scripts/tests/test_ci_dropped_step_guard.py derives its marked-job scope from that snapshot, so until it is reconciled a required context may have NO dropped-step guard — a step the runner drops would conclude success and take that check green having done no work (ersatztv#756). Re-read the live list and update the snapshot in a PR (the guard will then demand markers for any newly required job, or an ACCOUNTED_ELSEWHERE entry naming what covers it). This does not make the merge in front of you unsafe — Gitea enforces the live required set server-side — so approve if you have judged it unrelated." ;; + nomatch) + decide ask "H6 merge gate: no branch-protection rule governs 'main' at all, so the required status checks the dropped-step guard scopes itself to could not be confirmed (ersatztv#787). Branch protection on 'main' is what makes 'review-verdict/h10' load-bearing (ersatztv#743) — check it before merging." ;; + undecidable) + decide ask "H6 merge gate: a glob branch-protection rule could govern 'main', so which rule's required contexts to compare against .gitea/required-status-contexts.json is not derivable without reimplementing Gitea's matcher (ersatztv#787). Confirm the required checks manually." ;; + unreadable) + decide ask "H6 merge gate: branch protection for 'main', or .gitea/required-status-contexts.json itself, came back in a shape the required-contexts checker could not consume, so whether the dropped-step guard's scope is still current is unknown (ersatztv#787). Check the rules and the snapshot manually." ;; + readfail) + decide ask "H6 merge gate: could not read branch protection for the guard-scope freshness check (HTTP '${ctx_code:-none}' — Gitea unreachable, or these credentials lack the repo-admin scope that endpoint needs), so whether .gitea/required-status-contexts.json is still current is unknown (ersatztv#787). Confirm the required checks on 'main' manually." ;; + *) + decide ask "H6 merge gate: the required-contexts checker returned '${ctx_class:-nothing}', which is not a class this hook understands, so the dropped-step guard's scope could not be confirmed against branch protection (ersatztv#787).${ctx_diag:+ It said: ${ctx_diag}}Check scripts/check-required-contexts.sh." ;; +esac +fi # end of the guard-scope freshness arm (opened at `if [ "$ctx_repo_fold" = ... ]` above). The + # body is left unindented to match the rest of this file, which is flat throughout; the marker + # is here because the block is long enough that its extent is otherwise easy to misread. + if [ "$class" = "positive" ]; then # (a) CI + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant. # The reason string must not claim more than was actually checked: on the merge_when_checks_succeed diff --git a/.gitea/required-status-contexts.json b/.gitea/required-status-contexts.json new file mode 100644 index 000000000..8b05607c9 --- /dev/null +++ b/.gitea/required-status-contexts.json @@ -0,0 +1,12 @@ +{ + "repo": "timothy/ersatztv", + "branch": "main", + "read_on": "2026-08-27", + "source": "GET /repos/timothy/ersatztv/branch_protections -> the rule governing `main` -> status_check_contexts", + "why": "ersatztv#787. The committed mirror of the required status checks on `main`. It exists because the guards that make a required context trustworthy run in `pr-checks.yml::script-tests`, which checks out with persist-credentials:false and holds no Gitea credential, so it cannot ask the server. scripts/tests/test_ci_dropped_step_guard.py DERIVES its marked-job scope from `contexts` rather than repeating it as a literal, and scripts/check-required-contexts.sh compares this list against the live one wherever a credential does exist. Editing `contexts` by hand without re-reading the server is the one move that defeats both. The `repo` field exists because the merge-consent hook fires for whatever owner/repo the merge tool was called with: without it, merging a PR in another repo from an ersatztv session compares that repo's live contexts against THIS repo's mirror and reports a confident, flatly false finding about it.", + "contexts": [ + "Build ErsatzTV Image / Build & test (.NET) (pull_request)", + "Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request)", + "review-verdict/h10" + ] +} diff --git a/docs/ci-cd.md b/docs/ci-cd.md index afdbf921d..1f8e20aa6 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -1215,6 +1215,38 @@ and the client-side Husky guards (H6/H11/H13) remain useful friction but were ne they are fail-open and `--no-verify` bypasses them. Tag pushes are unaffected (separate mechanism; `tag_protections` is empty), so the release cut in "Cutting a release" still works unchanged. +**A fourth arm: guard-scope freshness (ersatztv#787).** `scripts/tests/test_ci_dropped_step_guard.py` +decides which CI jobs must carry per-step execution markers by DERIVING that scope from +`.gitea/required-status-contexts.json`, the committed mirror of `main`'s required status checks. It +derives rather than repeating a literal because its own CI job, `pr-checks.yml::script-tests`, checks +out with `persist-credentials: false` and cannot ask Gitea — which leaves the snapshot as the one +hand-maintained input in the chain. So the merge hook reconciles it: it reads +`branch_protections`, hands the payload to `scripts/check-required-contexts.sh`, and **asks** (never +denies) when the live list no longer matches. The comparison is against the EFFECTIVE required set: +a rule with `enable_status_check: false` requires nothing whatever its `status_check_contexts` +still holds, so reading the list without the flag would certify disabled protection as current. + +Two consequences worth knowing before you meet them. **The immediate merge path now reads a +repo-admin-scoped endpoint**: before #787 `branch_protections` was fetched only when arming a +scheduled auto-merge, so a credential without repo-admin scope kept its auto-grant on the common +path and now gets a prompt there instead. That is the fail-closed direction and it is loud, but it +is a real change in what the gate needs. **The arm runs only for the repo the snapshot names** +(its `repo` field): the merge tool is called with whatever owner/repo is being merged, and judging +another repo by this one's mirror would report a confident finding about a repo the mirror does not +describe. + +And what it does not reach: a docs/process-only PR exits at the hook's docs-only passthrough before +this arm, so a PR editing only `.gitea/required-status-contexts.json` — the snapshot-narrowing +direction — never triggers it. That passthrough is a human prompt rather than an auto-grant, and the +arm could not have judged such a PR anyway, since it compares the live server against the snapshot in +the local checkout rather than the one the PR proposes. Ask rather than deny because drift does not make that +merge unsafe — Gitea enforces the live required set server-side — while a stale guard scope means a +required context may have no dropped-step guard at all. The arm runs **last**, so it can only +downgrade an otherwise-satisfied auto-grant and never masks a more serious refusal, and it runs on +**both** merge paths: the pre-existing branch-protection read sits inside the +`merge_when_checks_succeed` branch and never fires on an immediate merge. Rule selection is shared, +not copied — both the hook and the checker load `scripts/lib/branch-rule-classifier.jq`. + **The hole it closes.** `pretooluse-merge-consent.sh` proves its three consent conditions at the moment the merge tool is called. Pass `merge_when_checks_succeed=true` and Gitea performs the merge *later*, against whatever head is green then — while the Done-when and review-verdict checks were diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 057c85e72..503a1f547 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -210,7 +210,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `testing.enumerating-guard-identity-not-position` | A guard that cross-checks a hand-reviewed registry against call sites discovered across the whole repo must key each entry on properties INTRINSIC to the site — file, kind, and the value source text — and never on its absolute line or column. A registry keyed on position is a function of every other file in the repo, so a branch that never touches the guard can invalidate it; and because each PR is green against its own base, that failure is structurally invisible pre-merge and lands on `main` after review and after the merge gate. Dropping the position keeps every mutation the guard exists for — a NEW site, a REMOVED site and a CHANGED value each still fail, since each changes the identity multiset — and costs exactly ONE case, which must be stated rather than implied: a SAME-IDENTITY SUBSTITUTION within one file (delete a registered site, add a different unreviewed one with the same kind and value token, net-zero count) now passes. A REPORTED failure still prints the discovered line:column, because identity and diagnostics need not share a format. Comparison stays a MULTISET count rather than set membership, so two sites in one file sharing an identity must be discovered exactly that many times and a third occurrence still fails. A SCANNER test that asserts real AST positions against FIXED inline fixtures is the opposite case and keeps its line/column identity — it has no churn, because its input does not move. | 2026-07-27 | [link](records/testing/enumerating-guard-identity-not-position.md) | | `testing.fix-ships-a-witnessed-red-test` | A commit claiming to fix something may carry a `Proves: ` trailer; when it does, `scripts/prove-fix.sh` must show that selector GREEN with the fix and RED with the code side reverted, and CI enforces it per-PR. The trailer is opt-in — an unproven commit is allowed — but a claimed proof that does not hold fails the build. | 2026-08-16 | [link](records/testing/fix-ships-a-witnessed-red-test.md) | | `testing.full-replace-asserts-field-list` | Any path that writes a WHOLE entity or a WHOLE child collection — a PUT-replace handler, a hand-built request object, a test comparer standing in for one — derives its field list from the authoritative type and asserts SET EQUALITY against it, rather than enumerating the fields by hand. A hand-written list is correct on the day it is written and structurally unable to report the day it stops being: the field that drifts is the one nobody wrote a line for, so no amount of care in the existing lines can reach it. The failure is silent by construction — a full replace with a field omitted returns HTTP 200 and destroys that field's value (#754 drifted from a 28-property DTO by one and cleared it; the symptom arrived hours later as missing pixels). SECOND CLAUSE, separable from the first: where a replaced child row carries state keyed to its identity — progression, ordering, an enumerator position — the handler RECONCILES BY ID rather than delete-and-reinsert, because reinsertion silently resets state a client never asked to touch (#252: a schedule PUT reset fill-group progression; #500: a dedup fix became permanent data loss because the add filter and the remove filter used different keys, so the two halves must agree on the key). Delete-and-reinsert is acceptable ONLY where no such state exists, and that emptiness is a fact about today's schema that a later feature can silently invalidate — so record it where the handler is, dated, rather than leaving it to be re-derived. The canonical worked example is `ToolCatalogTests.Every_Write_Tool_Should_Declare_Exactly_Its_OpenApi_Request_Body_Fields`, which reads the accepted fields from the generated OpenAPI document and compares both directions. ON THE SPA SIDE the same rule is enforced by the TYPE SYSTEM rather than by a test: a full-replace body is built as `Complete` (`web/src/api/completeRequest.ts`), a mapped type that makes every member of a generated request type required, so a builder that omits one fails `npm run typecheck`. Annotate BOTH the API-wrapper parameter (so later callers inherit it) AND each construction site including every `.map` callback return type, because the excess-property check that catches a PHANTOM field fires only on a fresh literal in a contextually typed position and a generic `.map` callback is not one. Do not infer from "most builders already typecheck" that the gap is closed: a member is omittable exactly when it is absent from the schema `required` array in the ASP.NET-produced OpenAPI document, and two live cases (#807) sat unchecked inside a large majority of checked ones. | 2026-08-21 | [link](records/testing/full-replace-asserts-field-list.md) | -| `testing.guard-derives-population-from-source` | A guard that asserts a COMPLETENESS property enumerates its population from a machine-readable authoritative source — the enum, the generated OpenAPI document, the parsed workflow YAML, the provider list — and asserts SET EQUALITY in BOTH directions against it. It may not narrow that population with a filter, a `Where`, a `grep` or an early `continue` before the assertion, because a filter cannot see the member that is MISSING: the member whose absence is the defect is precisely the one the predicate excludes. A hand-written literal list of members is the same defect in slower motion — a filter frozen at authoring time, correct on the day it was written and unable to report the day it stopped being. Two boundaries bound the rule rather than weaken it. FIRST, filtering to select the SUBJECT of a PER-MEMBER property is legitimate and is not this defect: the excluded members satisfy the property vacuously, so the filtered walk and the whole walk assert the same thing (`ToolCatalogTests.Every_Query_Parameter_Should_Be_A_Declared_Property` filters to tools that declare query parameters, and a tool declaring none has nothing to check). The defect is filtering the population before a COMPLETENESS claim, which is what makes an absent member unrepresentable (#757 filtered on `QueryParameters is {Count: > 0}` and so could not see a tool that should have declared one and did not). SECOND, a population of VALUES always has an external authoritative source and this rule applies directly; a population of SITES IN CODE has no such list, needs find-all-references tooling, and is tracked separately in #777 — do not stretch a set-equality assertion over it. Distinguish the guard SCOPE (which subsystems it covers — a reviewed policy choice, legitimately hand-written) from the guard POPULATION (the members inside that scope — always derived). When the scope itself MIRRORS an authoritative source, the mirror needs its own equality check or a dated staleness marker, or the guard is complete within a scope that has silently gone stale. The canonical worked example in this repo is `ToolCatalogTests.Every_Tool_Should_Declare_Exactly_Its_OpenApi_Query_Parameters`; the canonical residual gap is `MARKED_JOBS` in `scripts/tests/test_ci_dropped_step_guard.py`. WHEN THE POPULATION IS FILES (#806), the authoritative source is the GIT INDEX and never a filesystem walk. A walk is not merely a weaker enumerator, it answers a question about the MACHINE rather than about the repo: it reports build output, generated shims and editor droppings, and it differs between CI and every checkout, so the same guard asserts a different population in each place. Derive with `git ls-files`, take direct children only unless a nested population is stated and wanted, and assert existence rather than filtering on it, because filtering is what makes a missing member unrepresentable. This is an instantiation and not a blanket rewrite: the question per guard remains whether it makes a COMPLETENESS claim over TRACKED files, and a walk that assembles a fixture or selects the SUBJECT of a per-member property stays a walk with its reason written down. | 2026-08-13 | [link](records/testing/guard-derives-population-from-source.md) | +| `testing.guard-derives-population-from-source` | A guard that asserts a COMPLETENESS property enumerates its population from a machine-readable authoritative source — the enum, the generated OpenAPI document, the parsed workflow YAML, the provider list — and asserts SET EQUALITY in BOTH directions against it. It may not narrow that population with a filter, a `Where`, a `grep` or an early `continue` before the assertion, because a filter cannot see the member that is MISSING: the member whose absence is the defect is precisely the one the predicate excludes. A hand-written literal list of members is the same defect in slower motion — a filter frozen at authoring time, correct on the day it was written and unable to report the day it stopped being. Two boundaries bound the rule rather than weaken it. FIRST, filtering to select the SUBJECT of a PER-MEMBER property is legitimate and is not this defect: the excluded members satisfy the property vacuously, so the filtered walk and the whole walk assert the same thing (`ToolCatalogTests.Every_Query_Parameter_Should_Be_A_Declared_Property` filters to tools that declare query parameters, and a tool declaring none has nothing to check). The defect is filtering the population before a COMPLETENESS claim, which is what makes an absent member unrepresentable (#757 filtered on `QueryParameters is {Count: > 0}` and so could not see a tool that should have declared one and did not). SECOND, a population of VALUES always has an external authoritative source and this rule applies directly; a population of SITES IN CODE has no such list, needs find-all-references tooling, and is tracked separately in #777 — do not stretch a set-equality assertion over it. Distinguish the guard SCOPE (which subsystems it covers — a reviewed policy choice, legitimately hand-written) from the guard POPULATION (the members inside that scope — always derived). When the scope itself MIRRORS an authoritative source, the mirror needs its own equality check against that source, or the guard is complete within a scope that has silently gone stale. A DATED STALENESS MARKER does not discharge this and is no longer offered as an alternative (#787): a date records when someone last looked, so it goes stale in exactly the circumstance it exists to report, and nothing reddens when it does. The canonical worked example in this repo is `ToolCatalogTests.Every_Tool_Should_Declare_Exactly_Its_OpenApi_Query_Parameters`. When a scope mirrors an authoritative source across a CREDENTIAL BOUNDARY, split the invariant at that boundary rather than dating a comment: commit the mirror verbatim, derive the scope from it offline, and reconcile the mirror against the live source wherever a credential exists — `MARKED_JOBS` in `scripts/tests/test_ci_dropped_step_guard.py` was this repo's residual gap until #787 closed it that way. WHEN THE POPULATION IS FILES (#806), the authoritative source is the GIT INDEX and never a filesystem walk. A walk is not merely a weaker enumerator, it answers a question about the MACHINE rather than about the repo: it reports build output, generated shims and editor droppings, and it differs between CI and every checkout, so the same guard asserts a different population in each place. Derive with `git ls-files`, take direct children only unless a nested population is stated and wanted, and assert existence rather than filtering on it, because filtering is what makes a missing member unrepresentable. This is an instantiation and not a blanket rewrite: the question per guard remains whether it makes a COMPLETENESS claim over TRACKED files, and a walk that assembles a fixture or selects the SUBJECT of a per-member property stays a walk with its reason written down. | 2026-08-13 | [link](records/testing/guard-derives-population-from-source.md) | | `testing.guard-ships-with-mutation-proof` | A guard is not considered tested because a test involving it passes. It ships with a MUTATION PROOF: remove or disarm THAT GUARD'S CLAUSE ALONE, and a NAMED test must go red. ONE NAMED EXCEPTION, with its limits, because the rule degenerates without it: where the guard IS a test (a checker enforcing a repo invariant, with no separate script behind it), disarming it makes it ABSENT rather than red, so the proof is the contrapositive — INTRODUCE THE DEFECT THE GUARD EXISTS TO CATCH into an isolated copy of the guarded artifact, and the named test must go red. That is a mutation of the guarded SYSTEM rather than of the assertion, and it is admissible ONLY for checker-guards and ONLY when the mutation was executed and witnessed. It is NOT a licence to grade an ordinary script-guard MUTATION for having a bad-input test: feeding a script an input its clause rejects is BEHAVIOUR-ONLY, which is what three rows were regraded for. A file-level grade under this exception covers the clause its cited case actually mutates, not every assertion that later lands in the same file. Three things this excludes, each of which has already shipped here as a green suite over a dead check. FIRST, a behavioural test — one that feeds the guard a good input and a bad input and checks it passes and fails — proves the guard REACTS, never that it is LOAD-BEARING; #685 had two guards on one condition where deleting either left the whole suite green while every behavioural test passed. SECOND, mutating the WHOLE FILE does not count (#510): a whole-file revert cannot show that a test reaches a particular clause, so the mutation must target the clause. THIRD, the guard being WIRED is not the guard RUNNING — #631's suite was invoked by no CI job, #751's step was dropped by the runner and the job reported success in 6s against a normal 14-17s, and #719's new logic was never connected to stdin. Every guard that DERIVES A POPULATION also carries an ANTI-VACUITY assertion, because the characteristic failure of a completeness check is reporting that it proved everything while its population was empty; a guard with no population has nothing for such an assertion to be about, and stating it universally reads as coverage the unproven rows do not have. Mechanical enforcement is possible for the BOOKKEEPING and not for the JUDGEMENT, and the split is the decision: `docs/guard-inventory.md` lists every guard file with its Kind, its Proof class (`MUTATION`/`BEHAVIOUR-ONLY`/`NONE`) and a `file::function` ref, and `scripts/tests/test_guard_inventory.py` derives the guard population from the GIT INDEX and the call sites (#806), asserts SET EQUALITY against the rows, and resolves every claimed ref to a real `def`. So a new guard cannot ship unclassified and a renamed test cannot leave a row silently claiming coverage. Whether a row claiming `MUTATION` is telling the truth is no longer left to review: `testing.mutation-claims-are-executed` (#790) requires each such row to carry a DECLARED clause mutation that is applied to an isolated copy of the repository on every run, with the row's own named test required to go red. | 2026-08-13 | [link](records/testing/guard-ships-with-mutation-proof.md) | | `testing.hook-reports-its-own-execution` | Every script in `.claude/hooks/` sources `scripts/hook-fire-log.sh` and calls `etv_hook_fire_begin