From b1b9c1a9d4fbffc31f9e973314a264970f483179 Mon Sep 17 00:00:00 2001 From: Timothy Date: Thu, 27 Aug 2026 20:32:09 +0200 Subject: [PATCH] fix(787): derive the dropped-step guard's scope, and reconcile its snapshot against the server MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `scripts/tests/test_ci_dropped_step_guard.py` decides which jobs must carry per-step execution markers from `MARKED_JOBS = ("test", "migrations")` — a hand-written mirror of the required status contexts on `main`, annotated `Read live on 2026-08-10` and compared against the server by nothing. A dated comment is a claim about the past. Add a fourth required context and it acquired no dropped-step guard, silently, in the file the decision record names as its own model. The constraint that made this its own issue: `pr-checks.yml::script-tests` checks out with `persist-credentials: false`, so it cannot ask Gitea, and the issue's own candidate fixes were each unsatisfying — a snapshot "just moves the hand-maintenance", a credentialed check that skips without creds is the #756 fail-open. SPLIT THE INVARIANT AT THE CREDENTIAL BOUNDARY and check each half where it can be checked: * `.gitea/required-status-contexts.json` — the mirror, committed verbatim from the API. * The guard DERIVES `MARKED_JOBS` from it offline. Each candidate context string is SYNTHESIZED from the workflow (` / ()`) and matched by equality, never parsed — the delimiters occur inside the names they delimit (`Build & test (.NET)` ends in a parenthesis of its own). A required context mapping to neither a job nor an explicit `ACCOUNTED_ELSEWHERE` disposition is a HARD failure at import, which is what makes a fourth required check impossible to add in silence. Both directions are asserted, so a disposition that stops being required is reported too. * `scripts/check-required-contexts.sh` reconciles the mirror against live `status_check_contexts`, called from `pretooluse-merge-consent.sh`. So the maintenance is now a verbatim copy with an equality check on each side, and updating it to match the server makes CI red until the newly required job is marked. Three decisions worth keeping: * The hook arm ASKS, never denies. Drift does not make that merge unsafe — Gitea enforces the live required set server-side — so a deny would state something false about the change in front of the reader. What is stale is a guard's scope. * It runs LAST, so it can only downgrade an otherwise-satisfied auto-grant and never masks a more serious refusal (asserted, not described). * It runs on BOTH merge paths. The pre-existing branch-protection read sits inside the `merge_when_checks_succeed` branch and never fires on an immediate merge — the common case. This hook already records that exact shape once: the base re-read "first landed inside the scheduled-auto-merge branch only", and cold review found scheduled+retarget denied while immediate+retarget auto-GRANTED. The scope index is built from `docker-build.yml` alone and deliberately NOT from the git index. An earlier draft derived it with `tracked_paths` and broke the RELEASE PATH: `docker-build.yml`'s `scan` job runs this guard inside a poisoned COPY of the tree where `git ls-files` exits 128, which is the trap `test_ci_release_path_scan_job.py` documents in the very paragraph this change edits. A narrow index is safe here because the failure direction is: the completeness claim is over the SNAPSHOT's contexts, so a missing workflow makes a context UNRESOLVABLE and the guard strictly louder, never quieter. That harness now stages the snapshot into its copy, named explicitly rather than by glob. Rule selection was EXTRACTED, not copied. `scripts/lib/branch-rule-classifier.jq` is now loaded by both the hook and the checker: this is a security predicate, and the hook's own history is the argument — the docs-only file enumeration was written twice, four rounds of hardening landed on one copy and never reached the other, and the copy with real authority ended up strictly weaker. An earlier draft of the checker had its own blanket "any glob metacharacter anywhere" test, which called a rule named `a[b` undecidable for `main`; the shared classifier's `superset` test does not. A structural test now pins that both callers reach the one file, replacing a first attempt that banned the token "gobwas" and flagged the comment saying the logic is DELEGATED. Cold cross-family review (Codex) returned BLOCKED with seven findings; all are fixed and each fix's red was witnessed: * THE MUTATION PROOF WAS VACUOUS. The mutant was copied without the classifier it loads beside itself, so it exited 2 with EMPTY stdout — and the assertion `stdout != "drift"` was satisfied by that empty string. The proof never reached the clause, so the `MUTATION` grade was unsupported. It now asserts the mutant's EXACT verdict `(0, "match")` and ships a positive control proving the tmp layout reproduces the real one. * Duplicate synthesized contexts silently overwrote in a dict. Two jobs sharing a `name:` collide on one context string, the later wins, and the other's steps go unguarded — branch protection cannot tell them apart either. Now a multimap with a uniqueness assertion. * `enable_status_check: false` was type-checked and its VALUE ignored, so protection with checks DISABLED certified as `match`. The comparison is now against the EFFECTIVE required set. * The checker and snapshot were resolved through `$CLAUDE_PROJECT_DIR`. An env var is not a sound input to a security decision — a wrong value pointing at a tree containing an executable checker returns `match` about a different checkout. Both now bind to the hook-derived `$repo_root`, and the missing-checker test runs a COPY of the hook, the only way to move that root. * `unreadable` is a DECLARED class that fell to the catch-all, which told the operator the checker had returned a word the hook did not understand. It has its own arm. Its test asserts on the IMMEDIATE path: on the scheduled path the h10 arm answers first, and the first draft passed without reaching the code it names. * `scripts/lib/*.jq` was outside the remote-state population, whose claim is that it has no content filter. Scope widened and the classifier given an `N/A` row. * The release-path Blocker it raised was already fixed in this branch before the review landed. A second cold review round (a different model family, and a re-review of the fix commit) returned BLOCKED on two findings, both fixed: * THE STATED INVARIANT WENT UNENFORCED THE MOMENT ONE SIDE STARTED MOVING. The file says "the delimiter ban is WIDER than the marker set", and while both were literals edited together that held. `MARKED_JOBS` now DERIVES, so on the day a fourth required context arrives the marker half goes red, the author adds markers, CI greens — and that job's `run:` bodies were never checked for the `${{` delimiter that CAUSES the drop the markers detect. `DELIMITER_BAN_JOBS` is now derived from `MARKED_JOBS`, and the property is asserted so a future edit back to a literal cannot re-open it silently. Measured: a planted 4th context grows both sets together. * `docs/guard-inventory.md`'s row claimed the context index came from the git index — true of the reverted draft, false of the shipped code, and in the artifact whose job is to be trusted about exactly that. Nothing asserts that column, so it would not have self-corrected. The cross-workflow half moved to its own guard, `scripts/tests/test_ci_status_context_uniqueness.py`. It has to be separate: a cross-workflow claim needs the whole workflow population, whose authoritative source is the git index, and the dropped-step guard cannot touch the index because the release-path `scan` job runs it in a non-git copy. Split, each half gets the right source. It is MUTATION-graded via the checker-guard shape — the guard IS a test, so disarming its own assertion makes it absent rather than red, and the declared mutation goes into the guarded ARTIFACT instead: a workflow job renamed to collide with the required `Build & test (.NET)` context, which is precisely the evasion. Folded in from the same round: the arm now runs only for the repo the snapshot NAMES (it was comparing any repo's live contexts against this repo's mirror — `server-management` returns `[]`, which classified as `nomatch` and would have reported a flatly false finding about it); the hook guards the shared classifier's readability itself, rather than inheriting a `jq -f` failure that blamed the payload for a missing local file; `unreadable` and the usage exits match their stated contracts; and `docs/ci-cd.md` records that the immediate merge path now needs a repo-admin-scoped credential, which it did not before. A fifth cold round returned MERGEABLE — no Blocker, no High, no Medium. It independently re-ran round 4's exact re-inlining attack and confirmed the suite now reddens on it, disarmed twelve further clauses and found each reddened only its own named test, and verified #857 survived the rebase intact. Its one Low and two nits were fixed rather than deferred, because the first made a claim in this message false: the sweep of the retracted "or a dated staleness marker" wording had covered `docs/` but missed a quoting site in `scripts/tests/test_optional_request_members.py`. The checker's stderr is no longer discarded either — a usage error used to reach the operator as "returned 'nothing'", naming no cause. Two prose overclaims of my own were corrected rather than defended: "neither half is checkable alone; composed, they are" now names the reconciliation as ONE bypassable client-side hook, and "updating it to match the server makes CI RED" now says ADD, because hand-NARROWING the snapshot narrows the derived scope and stays green exactly as the literal it replaced did. A third cold round (the cross-family reviewer had hit a usage limit, so this was the strongest available reasoning on a cold brief) returned BLOCKED. Its findings, all fixed and each red witnessed: * THE CENTRAL CLAUSE OF THIS CHANGE SHIPPED WITH NO PROOF. `assert not unaccounted` is what makes a fourth required check impossible to add in silence, and the decision record sells it as exactly that — but disarming it and planting a fourth context left the derived scope silently behind with 51 tests still green. `_derive_marked_jobs` now takes an injectable context list SOLELY so that clause and the anti-vacuity clause beside it can be proven; production calls it with no argument. * The `enable_status_check:false` fix from round 2 had ZERO coverage: removing that one clause returned `match` for disabled protection and all 71 tests stayed green, because the `rule()` helper's `enabled=` parameter had no call site that ever passed False — this repo's own recorded "a fixture that omits a field tests only the default". * A FIFTH test passing for the wrong reason: the missing-checker test asserted the substring `check-required-contexts.sh`, which the CATCH-ALL arm's reason also contains, so deleting the `[ ! -x ]` clause left it green — two guards masking each other (#685). It now asserts wording unique to the arm AND that no branch-protection fetch happened, which only the early clause prevents. * The repo gate was byte-exact while Gitea resolves owner/repo case-insensitively (verified live: `/repos/TIMOTHY/ErsatzTV` answers 200), so a case variant would skip the freshness arm entirely — the gate failing open on a spelling. * The new `branch_protections` read had no `docs/remote-state-inventory.md` row, and the checker's row delegated its fetch to "the caller's row" that did not exist. Added, graded `UNSAFE-KNOWN` with what it authorizes (nothing: every outcome is `ask` or silence). A fourth cold round returned BLOCKED on two independent grounds, both fixed: * THE SEVENTH TEST PASSING FOR THE WRONG REASON, and the one that mattered most: the "one classifier, two callers" property was asserted with a SUBSTRING over source text, which a COMMENT satisfies. Review proved it by re-inlining a byte-identical copy of the classifier into the hook with a comment naming the shared file — the entire suite stayed green at exactly 1062. The byte-identical inline is the refactor that matters, precisely because it agrees on day one; a weakened copy would have been caught behaviourally. Both callers are now proven by SWAPPING the shared file for a sentinel that can only answer `undecidable` and requiring their behaviour to change. Re-running review's exact attack against the fixed suite now reddens it. * THE HEAD WAS STALE AND THE REBASE WAS RED. `origin/main` had moved to #857, which touches five of the same files; this branch did not apply cleanly, and the guard-inventory counts were computed off the dead base. Rebased onto `45b17e58e`, counts resolved to 42/6/21 and 20/6/16, the generated catalog REGENERATED rather than hand-resolved. The earlier green vouched for a tree that no longer existed. Also fixed from that round: the required-`branch` clause was masked by the mismatch guard one line below it, so its test could not see it (#685 again — it now asserts the arm-unique stderr); three of the four freshness arms had no proof and were carried by the catch-all; the checker's own classifier-readability guard was the untested twin of a fix the hook made loudly; and the unaccounted-context diagnostic offered a remediation that is impossible for a context produced by another workflow. Two coverage overclaims were corrected rather than defended: the arm does NOT run for a docs/process -only PR, so a PR editing only the snapshot — the narrowing direction — never triggers it, and it could not have judged one anyway, since it compares the live server against the snapshot in the local checkout rather than the version the PR proposes. So SEVEN of this change's own tests were caught passing for the wrong reason across four rounds, and the pattern is one class — the mutation proof (a mutant that never ran), the `unreadable` arm (an earlier arm answering), the never-deny assertion, and the repo gate (a stub serving a matching payload either way). Every one asserted the ABSENCE of something. They now assert a specific positive outcome, and each fix's red was witnessed by disarming the clause it names. Self-found in the same round: the snapshot names its own `branch`, and a mismatch against `--branch` is now a usage error rather than a class — comparing one branch's live contexts against another's mirror would otherwise produce a confident verdict about a pair never meant to be compared. Verified: the mutation is witnessed, not asserted — unmutated reports `drift` on an added context, the mutant with the set comparison disarmed reports `match`, a permanent no-op. The checker was run against the LIVE branch_protections payload (`match`) and every other class exercised. Full `scripts/tests` suite green; ruff check + format clean. Sweep by subject: every site claiming this gap is open was updated — `docs/guard-inventory.md`, `test_ci_release_path_scan_job.py`, `test_guard_populations_derive_from_git.py`, `test_ci_image_pin_population.py` (whose remaining `TOOLCHAIN_JOBS` literal is #789), and by clause, the rule's "or a dated staleness marker" alternative, which the new guidance rejects. fixes #787 Decisions-Edit: yes Co-authored-by: Timothy --- .claude/hooks/pretooluse-merge-consent.sh | 143 ++++++- .gitea/required-status-contexts.json | 12 + docs/ci-cd.md | 32 ++ docs/decisions/README.md | 2 +- .../process/check-and-use-pins-a-version.md | 11 +- .../guard-derives-population-from-source.md | 48 ++- docs/guard-inventory.md | 7 +- docs/remote-state-inventory.md | 7 +- scripts/check-required-contexts.sh | 127 ++++++ scripts/lib/branch-rule-classifier.jq | 36 ++ scripts/tests/mutation_manifest.py | 32 ++ scripts/tests/test_check_required_contexts.py | 378 ++++++++++++++++++ scripts/tests/test_ci_dropped_step_guard.py | 213 +++++++++- scripts/tests/test_ci_image_pin_population.py | 4 +- .../tests/test_ci_release_path_scan_job.py | 19 +- .../test_ci_status_context_uniqueness.py | 129 ++++++ .../test_guard_populations_derive_from_git.py | 4 +- .../test_merge_consent_required_check.py | 290 +++++++++++++- .../tests/test_optional_request_members.py | 6 +- scripts/tests/test_remote_state_inventory.py | 7 +- 20 files changed, 1433 insertions(+), 74 deletions(-) create mode 100644 .gitea/required-status-contexts.json create mode 100755 scripts/check-required-contexts.sh create mode 100644 scripts/lib/branch-rule-classifier.jq create mode 100644 scripts/tests/test_check_required_contexts.py create mode 100644 scripts/tests/test_ci_status_context_uniqueness.py diff --git a/.claude/hooks/pretooluse-merge-consent.sh b/.claude/hooks/pretooluse-merge-consent.sh index 330e6626c..8412c1218 100755 --- a/.claude/hooks/pretooluse-merge-consent.sh +++ b/.claude/hooks/pretooluse-merge-consent.sh @@ -472,25 +472,17 @@ else # directions, and it is rare in practice: as of 2026-08-19 this repo's only rule is the plain # name `main`, which the classifier resolves to `exact` on every run. That is a dated # observation about mutable remote config, not a property to rely on. - bp_verdict=$(printf '%s' "$bp_list" | jq --arg b "$base_ref" -c ' - def esc: gsub("(?[.+?^${}()|\\[\\]\\\\])"; "\\" + .c); - def offs: [match("[*?\\[\\]{}\\\\]"; "g").offset]; - def superset: . as $n | (offs) as $o - | ($n[0:$o[0]] | esc) + ".*" + ($n[($o[-1]+1):] | esc); - def nonascii: explode | any(. > 127); - . as $rules | $b as $base | - ($rules | map(select((.branch_name // .rule_name // "") as $n - | (($n|offs|length) == 0) - and (($n|ascii_downcase) == ($base|ascii_downcase))))) as $exacts | - (($base|nonascii) or ($rules | any((.branch_name // .rule_name // "") as $n - | ($n|offs|length) == 0 and ($n|nonascii)))) as $unfoldable | - if ($rules | any((.branch_name // .rule_name // "") as $n - | (($n|offs|length) > 0) - and ($base | test("^" + ($n|superset) + "$")))) then {verdict:"undecidable"} - elif $unfoldable then {verdict:"undecidable"} - elif ($exacts | length) > 1 then {verdict:"undecidable"} - elif ($exacts | length) == 1 then {verdict:"exact", rule:($exacts | first)} - else {verdict:"none"} end' 2>/dev/null || true) + # The classifier is a FILE now (ersatztv#787), so its absence is a new failure mode: `jq -f` on a + # missing program exits 2 with empty stdout, which reaches the `*)` arm below and asks that "this + # repo's branch-protection rules came back in a shape this hook could not parse" — blaming the + # payload for a missing local file. That is precisely the states-a-cause-that-did-not-happen defect + # the two comments beside that arm were written to fix, so it is checked here rather than inherited. + classifier="$repo_root/scripts/lib/branch-rule-classifier.jq" + if [ ! -r "$classifier" ]; then + rm -f "$bp_file" + decide ask "H6/H10 merge gate: the shared branch-protection rule classifier is missing or unreadable at $classifier, so which rule governs '$base_ref' — and therefore whether 'review-verdict/h10' is required on it — could not be derived (ersatztv#787). Restore the file, or confirm the required checks manually." + fi + bp_verdict=$(printf '%s' "$bp_list" | jq --arg b "$base_ref" -c -f "$classifier" 2>/dev/null || true) case $(printf '%s' "$bp_verdict" | jq -r '.verdict // ""' 2>/dev/null || true) in exact) bp=$(printf '%s' "$bp_verdict" | jq -c '.rule' 2>/dev/null || true); bp_code=200 ;; undecidable) rm -f "$bp_file" @@ -614,6 +606,119 @@ case "$class" in decide ask "H10 merge gate: unrecognized verdict classification '$class' for PR #$pr. Confirm the review covered the latest commit ($short) before merging." ;; esac +# --- (d) Guard-scope freshness (ersatztv#787): the committed mirror of `main`'s required status +# checks must still match the server. ------------------------------------------------------ +# ORDERED LAST, and that is a severity argument rather than a stylistic one. Every check above +# can DENY; this one can only ever downgrade an otherwise-satisfied auto-grant to a prompt. Run +# earlier it would preempt those verdicts and report a stale guard scope at a reader whose merge +# is blocked for a completely different and more serious reason, and it would ask on payloads the +# checks above are about to reject anyway. Placed here it is also PAST the point where the two +# merge paths converge, so it covers both without duplicating anything. +# `scripts/tests/test_ci_dropped_step_guard.py` DERIVES which jobs must carry per-step execution +# markers from `.gitea/required-status-contexts.json`, because its CI job checks out with +# `persist-credentials: false` and cannot ask Gitea. That makes the snapshot the single +# hand-maintained input in the chain: a fourth required context added on the server leaves the +# snapshot — and therefore the guard's scope — silently behind, which is the whole of #787. +# +# THIS RUNS ON BOTH MERGE PATHS, deliberately, and it is placed here rather than beside the +# branch-protection read in the scheduled-auto-merge branch for that reason. +# +# WHAT IT DOES NOT COVER, said here rather than left to be discovered: a PR whose changed files are +# all docs/process — `.gitea/` included — exits at the docs-only passthrough far above, so this arm +# never runs for it. A PR that edits ONLY `.gitea/required-status-contexts.json` is docs-only BY +# CONSTRUCTION, and that is exactly the snapshot-NARROWING direction the decision record names as +# this design's residual. Excluding that path from the allow-list would not buy the protection it +# looks like it would: this arm compares the live server against the snapshot in the LOCAL CHECKOUT, +# not against the version the PR proposes, so it cannot see a narrowing that has not landed yet. +# What does hold is that the passthrough is a passthrough — a human prompt, never an auto-grant — +# which is the `.gitea/` treatment ersatztv#317 asked for. That read is inside +# `else` (mwcs = true) and never executes on an immediate merge, which is the common case; hanging +# the freshness check off it would fire it only when an auto-merge is armed. This file already +# records that exact defect one section up — the base re-read "first landed inside the +# scheduled-auto-merge branch only", and cold review found scheduled+retarget denied while +# immediate+retarget auto-GRANTED. Same shape, so it is not repeated here. +# +# It reads `main` (the branch the snapshot names), NOT `$base_ref`. That is a DIFFERENT question +# from the one the scheduled branch asks — "is review-verdict/h10 required on the base I am merging +# into" — so this is not a second copy of that classifier and the two cannot drift into disagreeing: +# they consume different fields of different rules for different decisions. +# +# ASK, NEVER DENY. Drift does not make THIS merge unsafe: Gitea enforces the live required set +# server-side, so a newly required context with no status blocks the merge on its own. What has gone +# stale is a guard's scope — a different artifact, on a different clock. Denying would state +# something false about the change in front of the reader. Every non-`match` class asks, so a +# comparison that could not be made is surfaced rather than skipped (`unknown` is not `fine`). +# ONE base for both the checker and the snapshot, and it is `$repo_root` — derived from this file's +# own location — rather than `$CLAUDE_PROJECT_DIR`. Two reasons, and the second is the load-bearing +# one. Resolving them from different roots would let the hook classify one checkout's snapshot with +# another checkout's script, mismatched halves of a comparison whose whole job is to detect a +# mismatch. And an ENV VAR is not a sound input to a security decision: a wrong value pointing at a +# tree that happens to contain an executable checker returns `match` about a different checkout +# entirely, which silently authorizes the grant. A missing path only asks, so the failure is quiet +# exactly where it is worst. +ctx_base="$repo_root" +ctx_snapshot="$ctx_base/.gitea/required-status-contexts.json" +ctx_script="$ctx_base/scripts/check-required-contexts.sh" + +# THIS ARM IS ABOUT ONE REPO, and the merge tool is not. Every other check here reads +# `$owner/$repo` from the tool input and is repo-agnostic; this one compares a HARDCODED branch +# against a snapshot committed in THIS checkout. Merging a PR in another repo from a session opened +# here would otherwise weigh that repo's live contexts against this repo's mirror and report a +# confident, flatly false finding about it — measured: server-management returns `[]`, which +# classifies as `nomatch`. So the snapshot names the repo it describes and the arm runs only for it. +# An unreadable snapshot cannot answer "is this my repo?" either, so it asks rather than skipping. +ctx_repo=$(jq -r 'if (.repo | type) == "string" then .repo else "" end' "$ctx_snapshot" 2>/dev/null || true) +if [ -z "$ctx_repo" ]; then + decide ask "H6 merge gate: $ctx_snapshot is missing, unreadable, or names no \`repo\`, so the dropped-step guard's scope could not be checked against branch protection — nor could it be established whether this snapshot even describes $owner/$repo (ersatztv#787). Restore the file, or check the required checks manually." +fi +# CASE-FOLDED, because Gitea resolves owner/repo case-insensitively: verified live, both +# `/repos/timothy/ersatztv` and `/repos/TIMOTHY/ErsatzTV` answer 200. A byte-exact compare would let +# any case variant sail through every other arm and SKIP this one, so drift would go unreported with +# no ask — the gate failing open on a spelling. The hook already treats case folding as +# decision-relevant one section up, where `MAIN` vs `main` makes the governing rule undecidable. +ctx_repo_fold=$(printf '%s' "$ctx_repo" | tr '[:upper:]' '[:lower:]') +target_repo_fold=$(printf '%s' "$owner/$repo" | tr '[:upper:]' '[:lower:]') +if [ "$ctx_repo_fold" = "$target_repo_fold" ]; then +if [ ! -x "$ctx_script" ]; then + decide ask "H6 merge gate: the required-contexts checker is missing or not executable at $ctx_script, so whether the dropped-step guard's scope still matches branch protection on 'main' could not be derived (ersatztv#787). Check it manually, or restore the script." +fi +bpf=$(mktemp) || decide ask "H6 merge gate: could not allocate a temp file to read branch protection for the guard-scope freshness check (ersatztv#787)." +if [ -n "${ETV_GITEA_TOKEN:-}" ]; then + ctx_code=$(curl -s -o "$bpf" -w '%{http_code}' -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true) +else + ctx_code=$(curl -s -o "$bpf" -w '%{http_code}' -u "$ETV_GITEA_BASICAUTH" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true) +fi +if [ "$ctx_code" = "200" ]; then + # stderr is KEPT, not sent to /dev/null. The checker exits 2 with a diagnostic on a usage error — + # an unreadable snapshot, a branch mismatch, a missing classifier — and discarding it made all of + # those arrive at the operator as the catch-all's "returned 'nothing'", which names no cause. That + # is the same states-a-cause-that-did-not-happen shape this arm was careful about elsewhere. + ctx_class=$("$ctx_script" --branch main --snapshot "$ctx_snapshot" < "$bpf" 2>"$bpf.err" || true) + ctx_diag=$(tr '\n' ' ' < "$bpf.err" 2>/dev/null | cut -c1-300 || true) +else + ctx_class=readfail + ctx_diag="" +fi +rm -f "$bpf" "$bpf.err" +case "$ctx_class" in + match) : ;; + drift) + decide ask "H6 merge gate: the required status checks on 'main' no longer match .gitea/required-status-contexts.json (ersatztv#787). scripts/tests/test_ci_dropped_step_guard.py derives its marked-job scope from that snapshot, so until it is reconciled a required context may have NO dropped-step guard — a step the runner drops would conclude success and take that check green having done no work (ersatztv#756). Re-read the live list and update the snapshot in a PR (the guard will then demand markers for any newly required job, or an ACCOUNTED_ELSEWHERE entry naming what covers it). This does not make the merge in front of you unsafe — Gitea enforces the live required set server-side — so approve if you have judged it unrelated." ;; + nomatch) + decide ask "H6 merge gate: no branch-protection rule governs 'main' at all, so the required status checks the dropped-step guard scopes itself to could not be confirmed (ersatztv#787). Branch protection on 'main' is what makes 'review-verdict/h10' load-bearing (ersatztv#743) — check it before merging." ;; + undecidable) + decide ask "H6 merge gate: a glob branch-protection rule could govern 'main', so which rule's required contexts to compare against .gitea/required-status-contexts.json is not derivable without reimplementing Gitea's matcher (ersatztv#787). Confirm the required checks manually." ;; + unreadable) + decide ask "H6 merge gate: branch protection for 'main', or .gitea/required-status-contexts.json itself, came back in a shape the required-contexts checker could not consume, so whether the dropped-step guard's scope is still current is unknown (ersatztv#787). Check the rules and the snapshot manually." ;; + readfail) + decide ask "H6 merge gate: could not read branch protection for the guard-scope freshness check (HTTP '${ctx_code:-none}' — Gitea unreachable, or these credentials lack the repo-admin scope that endpoint needs), so whether .gitea/required-status-contexts.json is still current is unknown (ersatztv#787). Confirm the required checks on 'main' manually." ;; + *) + decide ask "H6 merge gate: the required-contexts checker returned '${ctx_class:-nothing}', which is not a class this hook understands, so the dropped-step guard's scope could not be confirmed against branch protection (ersatztv#787).${ctx_diag:+ It said: ${ctx_diag}}Check scripts/check-required-contexts.sh." ;; +esac +fi # end of the guard-scope freshness arm (opened at `if [ "$ctx_repo_fold" = ... ]` above). The + # body is left unindented to match the rest of this file, which is flat throughout; the marker + # is here because the block is long enough that its extent is otherwise easy to misread. + if [ "$class" = "positive" ]; then # (a) CI + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant. # The reason string must not claim more than was actually checked: on the merge_when_checks_succeed diff --git a/.gitea/required-status-contexts.json b/.gitea/required-status-contexts.json new file mode 100644 index 000000000..8b05607c9 --- /dev/null +++ b/.gitea/required-status-contexts.json @@ -0,0 +1,12 @@ +{ + "repo": "timothy/ersatztv", + "branch": "main", + "read_on": "2026-08-27", + "source": "GET /repos/timothy/ersatztv/branch_protections -> the rule governing `main` -> status_check_contexts", + "why": "ersatztv#787. The committed mirror of the required status checks on `main`. It exists because the guards that make a required context trustworthy run in `pr-checks.yml::script-tests`, which checks out with persist-credentials:false and holds no Gitea credential, so it cannot ask the server. scripts/tests/test_ci_dropped_step_guard.py DERIVES its marked-job scope from `contexts` rather than repeating it as a literal, and scripts/check-required-contexts.sh compares this list against the live one wherever a credential does exist. Editing `contexts` by hand without re-reading the server is the one move that defeats both. The `repo` field exists because the merge-consent hook fires for whatever owner/repo the merge tool was called with: without it, merging a PR in another repo from an ersatztv session compares that repo's live contexts against THIS repo's mirror and reports a confident, flatly false finding about it.", + "contexts": [ + "Build ErsatzTV Image / Build & test (.NET) (pull_request)", + "Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request)", + "review-verdict/h10" + ] +} diff --git a/docs/ci-cd.md b/docs/ci-cd.md index afdbf921d..1f8e20aa6 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -1215,6 +1215,38 @@ and the client-side Husky guards (H6/H11/H13) remain useful friction but were ne they are fail-open and `--no-verify` bypasses them. Tag pushes are unaffected (separate mechanism; `tag_protections` is empty), so the release cut in "Cutting a release" still works unchanged. +**A fourth arm: guard-scope freshness (ersatztv#787).** `scripts/tests/test_ci_dropped_step_guard.py` +decides which CI jobs must carry per-step execution markers by DERIVING that scope from +`.gitea/required-status-contexts.json`, the committed mirror of `main`'s required status checks. It +derives rather than repeating a literal because its own CI job, `pr-checks.yml::script-tests`, checks +out with `persist-credentials: false` and cannot ask Gitea — which leaves the snapshot as the one +hand-maintained input in the chain. So the merge hook reconciles it: it reads +`branch_protections`, hands the payload to `scripts/check-required-contexts.sh`, and **asks** (never +denies) when the live list no longer matches. The comparison is against the EFFECTIVE required set: +a rule with `enable_status_check: false` requires nothing whatever its `status_check_contexts` +still holds, so reading the list without the flag would certify disabled protection as current. + +Two consequences worth knowing before you meet them. **The immediate merge path now reads a +repo-admin-scoped endpoint**: before #787 `branch_protections` was fetched only when arming a +scheduled auto-merge, so a credential without repo-admin scope kept its auto-grant on the common +path and now gets a prompt there instead. That is the fail-closed direction and it is loud, but it +is a real change in what the gate needs. **The arm runs only for the repo the snapshot names** +(its `repo` field): the merge tool is called with whatever owner/repo is being merged, and judging +another repo by this one's mirror would report a confident finding about a repo the mirror does not +describe. + +And what it does not reach: a docs/process-only PR exits at the hook's docs-only passthrough before +this arm, so a PR editing only `.gitea/required-status-contexts.json` — the snapshot-narrowing +direction — never triggers it. That passthrough is a human prompt rather than an auto-grant, and the +arm could not have judged such a PR anyway, since it compares the live server against the snapshot in +the local checkout rather than the one the PR proposes. Ask rather than deny because drift does not make that +merge unsafe — Gitea enforces the live required set server-side — while a stale guard scope means a +required context may have no dropped-step guard at all. The arm runs **last**, so it can only +downgrade an otherwise-satisfied auto-grant and never masks a more serious refusal, and it runs on +**both** merge paths: the pre-existing branch-protection read sits inside the +`merge_when_checks_succeed` branch and never fires on an immediate merge. Rule selection is shared, +not copied — both the hook and the checker load `scripts/lib/branch-rule-classifier.jq`. + **The hole it closes.** `pretooluse-merge-consent.sh` proves its three consent conditions at the moment the merge tool is called. Pass `merge_when_checks_succeed=true` and Gitea performs the merge *later*, against whatever head is green then — while the Done-when and review-verdict checks were diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 057c85e72..503a1f547 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -210,7 +210,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `testing.enumerating-guard-identity-not-position` | A guard that cross-checks a hand-reviewed registry against call sites discovered across the whole repo must key each entry on properties INTRINSIC to the site — file, kind, and the value source text — and never on its absolute line or column. A registry keyed on position is a function of every other file in the repo, so a branch that never touches the guard can invalidate it; and because each PR is green against its own base, that failure is structurally invisible pre-merge and lands on `main` after review and after the merge gate. Dropping the position keeps every mutation the guard exists for — a NEW site, a REMOVED site and a CHANGED value each still fail, since each changes the identity multiset — and costs exactly ONE case, which must be stated rather than implied: a SAME-IDENTITY SUBSTITUTION within one file (delete a registered site, add a different unreviewed one with the same kind and value token, net-zero count) now passes. A REPORTED failure still prints the discovered line:column, because identity and diagnostics need not share a format. Comparison stays a MULTISET count rather than set membership, so two sites in one file sharing an identity must be discovered exactly that many times and a third occurrence still fails. A SCANNER test that asserts real AST positions against FIXED inline fixtures is the opposite case and keeps its line/column identity — it has no churn, because its input does not move. | 2026-07-27 | [link](records/testing/enumerating-guard-identity-not-position.md) | | `testing.fix-ships-a-witnessed-red-test` | A commit claiming to fix something may carry a `Proves: ` trailer; when it does, `scripts/prove-fix.sh` must show that selector GREEN with the fix and RED with the code side reverted, and CI enforces it per-PR. The trailer is opt-in — an unproven commit is allowed — but a claimed proof that does not hold fails the build. | 2026-08-16 | [link](records/testing/fix-ships-a-witnessed-red-test.md) | | `testing.full-replace-asserts-field-list` | Any path that writes a WHOLE entity or a WHOLE child collection — a PUT-replace handler, a hand-built request object, a test comparer standing in for one — derives its field list from the authoritative type and asserts SET EQUALITY against it, rather than enumerating the fields by hand. A hand-written list is correct on the day it is written and structurally unable to report the day it stops being: the field that drifts is the one nobody wrote a line for, so no amount of care in the existing lines can reach it. The failure is silent by construction — a full replace with a field omitted returns HTTP 200 and destroys that field's value (#754 drifted from a 28-property DTO by one and cleared it; the symptom arrived hours later as missing pixels). SECOND CLAUSE, separable from the first: where a replaced child row carries state keyed to its identity — progression, ordering, an enumerator position — the handler RECONCILES BY ID rather than delete-and-reinsert, because reinsertion silently resets state a client never asked to touch (#252: a schedule PUT reset fill-group progression; #500: a dedup fix became permanent data loss because the add filter and the remove filter used different keys, so the two halves must agree on the key). Delete-and-reinsert is acceptable ONLY where no such state exists, and that emptiness is a fact about today's schema that a later feature can silently invalidate — so record it where the handler is, dated, rather than leaving it to be re-derived. The canonical worked example is `ToolCatalogTests.Every_Write_Tool_Should_Declare_Exactly_Its_OpenApi_Request_Body_Fields`, which reads the accepted fields from the generated OpenAPI document and compares both directions. ON THE SPA SIDE the same rule is enforced by the TYPE SYSTEM rather than by a test: a full-replace body is built as `Complete` (`web/src/api/completeRequest.ts`), a mapped type that makes every member of a generated request type required, so a builder that omits one fails `npm run typecheck`. Annotate BOTH the API-wrapper parameter (so later callers inherit it) AND each construction site including every `.map` callback return type, because the excess-property check that catches a PHANTOM field fires only on a fresh literal in a contextually typed position and a generic `.map` callback is not one. Do not infer from "most builders already typecheck" that the gap is closed: a member is omittable exactly when it is absent from the schema `required` array in the ASP.NET-produced OpenAPI document, and two live cases (#807) sat unchecked inside a large majority of checked ones. | 2026-08-21 | [link](records/testing/full-replace-asserts-field-list.md) | -| `testing.guard-derives-population-from-source` | A guard that asserts a COMPLETENESS property enumerates its population from a machine-readable authoritative source — the enum, the generated OpenAPI document, the parsed workflow YAML, the provider list — and asserts SET EQUALITY in BOTH directions against it. It may not narrow that population with a filter, a `Where`, a `grep` or an early `continue` before the assertion, because a filter cannot see the member that is MISSING: the member whose absence is the defect is precisely the one the predicate excludes. A hand-written literal list of members is the same defect in slower motion — a filter frozen at authoring time, correct on the day it was written and unable to report the day it stopped being. Two boundaries bound the rule rather than weaken it. FIRST, filtering to select the SUBJECT of a PER-MEMBER property is legitimate and is not this defect: the excluded members satisfy the property vacuously, so the filtered walk and the whole walk assert the same thing (`ToolCatalogTests.Every_Query_Parameter_Should_Be_A_Declared_Property` filters to tools that declare query parameters, and a tool declaring none has nothing to check). The defect is filtering the population before a COMPLETENESS claim, which is what makes an absent member unrepresentable (#757 filtered on `QueryParameters is {Count: > 0}` and so could not see a tool that should have declared one and did not). SECOND, a population of VALUES always has an external authoritative source and this rule applies directly; a population of SITES IN CODE has no such list, needs find-all-references tooling, and is tracked separately in #777 — do not stretch a set-equality assertion over it. Distinguish the guard SCOPE (which subsystems it covers — a reviewed policy choice, legitimately hand-written) from the guard POPULATION (the members inside that scope — always derived). When the scope itself MIRRORS an authoritative source, the mirror needs its own equality check or a dated staleness marker, or the guard is complete within a scope that has silently gone stale. The canonical worked example in this repo is `ToolCatalogTests.Every_Tool_Should_Declare_Exactly_Its_OpenApi_Query_Parameters`; the canonical residual gap is `MARKED_JOBS` in `scripts/tests/test_ci_dropped_step_guard.py`. WHEN THE POPULATION IS FILES (#806), the authoritative source is the GIT INDEX and never a filesystem walk. A walk is not merely a weaker enumerator, it answers a question about the MACHINE rather than about the repo: it reports build output, generated shims and editor droppings, and it differs between CI and every checkout, so the same guard asserts a different population in each place. Derive with `git ls-files`, take direct children only unless a nested population is stated and wanted, and assert existence rather than filtering on it, because filtering is what makes a missing member unrepresentable. This is an instantiation and not a blanket rewrite: the question per guard remains whether it makes a COMPLETENESS claim over TRACKED files, and a walk that assembles a fixture or selects the SUBJECT of a per-member property stays a walk with its reason written down. | 2026-08-13 | [link](records/testing/guard-derives-population-from-source.md) | +| `testing.guard-derives-population-from-source` | A guard that asserts a COMPLETENESS property enumerates its population from a machine-readable authoritative source — the enum, the generated OpenAPI document, the parsed workflow YAML, the provider list — and asserts SET EQUALITY in BOTH directions against it. It may not narrow that population with a filter, a `Where`, a `grep` or an early `continue` before the assertion, because a filter cannot see the member that is MISSING: the member whose absence is the defect is precisely the one the predicate excludes. A hand-written literal list of members is the same defect in slower motion — a filter frozen at authoring time, correct on the day it was written and unable to report the day it stopped being. Two boundaries bound the rule rather than weaken it. FIRST, filtering to select the SUBJECT of a PER-MEMBER property is legitimate and is not this defect: the excluded members satisfy the property vacuously, so the filtered walk and the whole walk assert the same thing (`ToolCatalogTests.Every_Query_Parameter_Should_Be_A_Declared_Property` filters to tools that declare query parameters, and a tool declaring none has nothing to check). The defect is filtering the population before a COMPLETENESS claim, which is what makes an absent member unrepresentable (#757 filtered on `QueryParameters is {Count: > 0}` and so could not see a tool that should have declared one and did not). SECOND, a population of VALUES always has an external authoritative source and this rule applies directly; a population of SITES IN CODE has no such list, needs find-all-references tooling, and is tracked separately in #777 — do not stretch a set-equality assertion over it. Distinguish the guard SCOPE (which subsystems it covers — a reviewed policy choice, legitimately hand-written) from the guard POPULATION (the members inside that scope — always derived). When the scope itself MIRRORS an authoritative source, the mirror needs its own equality check against that source, or the guard is complete within a scope that has silently gone stale. A DATED STALENESS MARKER does not discharge this and is no longer offered as an alternative (#787): a date records when someone last looked, so it goes stale in exactly the circumstance it exists to report, and nothing reddens when it does. The canonical worked example in this repo is `ToolCatalogTests.Every_Tool_Should_Declare_Exactly_Its_OpenApi_Query_Parameters`. When a scope mirrors an authoritative source across a CREDENTIAL BOUNDARY, split the invariant at that boundary rather than dating a comment: commit the mirror verbatim, derive the scope from it offline, and reconcile the mirror against the live source wherever a credential exists — `MARKED_JOBS` in `scripts/tests/test_ci_dropped_step_guard.py` was this repo's residual gap until #787 closed it that way. WHEN THE POPULATION IS FILES (#806), the authoritative source is the GIT INDEX and never a filesystem walk. A walk is not merely a weaker enumerator, it answers a question about the MACHINE rather than about the repo: it reports build output, generated shims and editor droppings, and it differs between CI and every checkout, so the same guard asserts a different population in each place. Derive with `git ls-files`, take direct children only unless a nested population is stated and wanted, and assert existence rather than filtering on it, because filtering is what makes a missing member unrepresentable. This is an instantiation and not a blanket rewrite: the question per guard remains whether it makes a COMPLETENESS claim over TRACKED files, and a walk that assembles a fixture or selects the SUBJECT of a per-member property stays a walk with its reason written down. | 2026-08-13 | [link](records/testing/guard-derives-population-from-source.md) | | `testing.guard-ships-with-mutation-proof` | A guard is not considered tested because a test involving it passes. It ships with a MUTATION PROOF: remove or disarm THAT GUARD'S CLAUSE ALONE, and a NAMED test must go red. ONE NAMED EXCEPTION, with its limits, because the rule degenerates without it: where the guard IS a test (a checker enforcing a repo invariant, with no separate script behind it), disarming it makes it ABSENT rather than red, so the proof is the contrapositive — INTRODUCE THE DEFECT THE GUARD EXISTS TO CATCH into an isolated copy of the guarded artifact, and the named test must go red. That is a mutation of the guarded SYSTEM rather than of the assertion, and it is admissible ONLY for checker-guards and ONLY when the mutation was executed and witnessed. It is NOT a licence to grade an ordinary script-guard MUTATION for having a bad-input test: feeding a script an input its clause rejects is BEHAVIOUR-ONLY, which is what three rows were regraded for. A file-level grade under this exception covers the clause its cited case actually mutates, not every assertion that later lands in the same file. Three things this excludes, each of which has already shipped here as a green suite over a dead check. FIRST, a behavioural test — one that feeds the guard a good input and a bad input and checks it passes and fails — proves the guard REACTS, never that it is LOAD-BEARING; #685 had two guards on one condition where deleting either left the whole suite green while every behavioural test passed. SECOND, mutating the WHOLE FILE does not count (#510): a whole-file revert cannot show that a test reaches a particular clause, so the mutation must target the clause. THIRD, the guard being WIRED is not the guard RUNNING — #631's suite was invoked by no CI job, #751's step was dropped by the runner and the job reported success in 6s against a normal 14-17s, and #719's new logic was never connected to stdin. Every guard that DERIVES A POPULATION also carries an ANTI-VACUITY assertion, because the characteristic failure of a completeness check is reporting that it proved everything while its population was empty; a guard with no population has nothing for such an assertion to be about, and stating it universally reads as coverage the unproven rows do not have. Mechanical enforcement is possible for the BOOKKEEPING and not for the JUDGEMENT, and the split is the decision: `docs/guard-inventory.md` lists every guard file with its Kind, its Proof class (`MUTATION`/`BEHAVIOUR-ONLY`/`NONE`) and a `file::function` ref, and `scripts/tests/test_guard_inventory.py` derives the guard population from the GIT INDEX and the call sites (#806), asserts SET EQUALITY against the rows, and resolves every claimed ref to a real `def`. So a new guard cannot ship unclassified and a renamed test cannot leave a row silently claiming coverage. Whether a row claiming `MUTATION` is telling the truth is no longer left to review: `testing.mutation-claims-are-executed` (#790) requires each such row to carry a DECLARED clause mutation that is applied to an isolated copy of the repository on every run, with the row's own named test required to go red. | 2026-08-13 | [link](records/testing/guard-ships-with-mutation-proof.md) | | `testing.hook-reports-its-own-execution` | Every script in `.claude/hooks/` sources `scripts/hook-fire-log.sh` and calls `etv_hook_fire_begin