From fa06da21f33db49281d7100a55f9f45c1b5d36bf Mon Sep 17 00:00:00 2001 From: Timothy Date: Sat, 5 Sep 2026 11:00:29 +0200 Subject: [PATCH] fix(885): the guard states both invariants, and the second one is executed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two `docs/guard-inventory.md` rows and the module docstring described `test_workflow_persist_credentials.py` as the `actions/checkout` guard only. The deferral to #909 rested on `docs/guard-inventory.md` being held by the session working #881; that issue is closed and its PR is the commit this branch is rebased onto, so the file is free and the edit belongs here under docs-update-is-part-of-done. `MUTATIONS` keys at most one declared clause mutation per guard FILE (`test_the_manifest_covers_exactly_the_MUTATION_rows` asserts `len(MUTATIONS) == len(declared)`), and the grading row's proof-ref column is compared against it, so the route invariant cannot take a second `MUTATIONS` row. It takes a `CLAIMS` entry instead — the population #881 widened this file to carry — bound to the inventory sentence that states it: deleting `build`'s `if: github.event_name != 'pull_request'` from the shipped `docker-build.yml` is applied to a sandbox copy every run and the named proof is required to redden with the collector's own wording. That grows the `CLAIMS` population from three entries to four, which invalidates the cost span `testing.mutation-claims-are-executed` measured over three. Re-taking it here produced 54.3s/149.5s, 81.6s/78.7s and 114.3s/84.2s across three A/B pairs with other builds on the host — two inverted, so the load dominates the signal. The record now says the span is a lower bound and that a re-measurement is owed on a quiet machine, rather than carrying a scaled or invented number. Decisions-Edit: yes Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiV --- .../pr-route-carries-no-stored-credential.md | 18 +++++++------ .../testing/mutation-claims-are-executed.md | 2 +- docs/guard-inventory.md | 4 +-- scripts/tests/mutation_manifest.py | 26 +++++++++++++++++++ .../test_workflow_persist_credentials.py | 7 ++++- 5 files changed, 45 insertions(+), 12 deletions(-) diff --git a/docs/decisions/records/ci/pr-route-carries-no-stored-credential.md b/docs/decisions/records/ci/pr-route-carries-no-stored-credential.md index 092c0bc5e..dfb87f93a 100644 --- a/docs/decisions/records/ci/pr-route-carries-no-stored-credential.md +++ b/docs/decisions/records/ci/pr-route-carries-no-stored-credential.md @@ -92,14 +92,16 @@ expressions a human can see are equivalent, leaves the job IN the population. Pa after repeated defects from that one mechanism; a pin can only ever be too demanding, which is the safe direction, and widening it is then a deliberate edit rather than an accident. -**One inventory row still understates this, tracked as #909.** `docs/guard-inventory.md` describes -`scripts/tests/test_workflow_persist_credentials.py` as the `actions/checkout` guard only, while it -now also holds this invariant with its own derived population and mutation proofs. Nothing goes red -on that: `test_guard_inventory.py` asserts set equality over FILES, and the file was already listed, -so a row whose PROSE decays is invisible to it. Do not read it as a complete description of what its -subject checks. #909's other half — `docs/remote-state-inventory.md`'s row for -`scripts/ci-toolchain-image-resolves.sh`, which listed "an unusable credential" among the shapes that -fail the job — is corrected in this change. +**The guard now holds TWO invariants, and only one grade can name a declared clause.** +`scripts/tests/test_workflow_persist_credentials.py` holds the `actions/checkout` invariant (#835) +and this one, on the same index-derived population. Both of its `docs/guard-inventory.md` rows say +so; nothing MAKES them say so, because `test_guard_inventory.py` asserts set equality over FILES and +the file was already listed, so a row whose PROSE decays stays green. The manifest keys at most one +`MUTATIONS` entry per guard FILE, so the declared clause there stands for the checkout half and the +route half is executed through a `CLAIMS` entry instead, bound to the inventory sentence that states +it (`testing.mutation-claims-are-executed`). That is the same obligation reached through the +population #881 widened, not a weaker one: the clause is declared by hand, applied to a sandbox copy +and its result read on every run, exactly as an inventory row's is. **The residual, stated so nobody cites this record as a boundary.** Head-supplied YAML can still name every secret in the store — the committed workflows' references were never the limit diff --git a/docs/decisions/records/testing/mutation-claims-are-executed.md b/docs/decisions/records/testing/mutation-claims-are-executed.md index b9964afff..4c724ae20 100644 --- a/docs/decisions/records/testing/mutation-claims-are-executed.md +++ b/docs/decisions/records/testing/mutation-claims-are-executed.md @@ -7,7 +7,7 @@ supersedes: none superseded-by: none rule: 'A `MUTATION` row in `docs/guard-inventory.md` is not a statement that someone once witnessed a red. It carries a DECLARED clause mutation in `scripts/tests/mutation_manifest.py`, and `scripts/tests/test_mutation_harness.py` applies that mutation to an isolated copy of the repository on every run and requires the row''s OWN named test to go red. The manifest and the MUTATION rows are compared for SET EQUALITY in both directions, so a row cannot claim the grade without a mutation and a mutation cannot outlive the grade it justifies. EXIT STATUS IS NOT THE VERDICT: each entry also declares the DIAGNOSTIC its red must carry, matched against pytest''s exception output alone, because pytest reports a crashing test exactly as it reports a detecting one and a red for an unrelated reason is evidence about nothing. WHERE THE GUARD IS ITSELF A TEST, `target` may differ from `guard` and the exact-once check applies to the declared TARGET. Two shapes are admissible and the choice is not free. Where the guard''s assertion IS the check — a completeness comparison against a Markdown inventory — the mutation goes into the guarded ARTIFACT, per `testing.guard-ships-with-mutation-proof`''s checker-guard exception, because mutating such a checker''s own POPULATION demonstrates a false POSITIVE while proving nothing about the detection the row claims. Where the guard is a test module wrapping a separately mutable DETECTOR or helper, the clause may be in that detector, since disarming it is a real clause disarm and the module''s own assertion is what notices. THE MUTATION IS DECLARED, NEVER INFERRED: a harness that guessed which clause of a 90-line hook is the guard would manufacture the confident-but-empty coverage this exists to prevent, which is why `testing.guard-ships-with-mutation-proof` rejected a generic runner. Where a proof test already names its clause in source, the manifest reuses THAT string, so a retarget in either place is caught by the other. COARSENESS IS RECORDED, NOT HIDDEN: each entry is graded `CLAUSE` or `DETECTOR`, and a `DETECTOR` entry — one whose detector accumulates faults from independent arms, so disarming any single arm leaves its proof test green — must CARRY the finer mutation that survived, which is re-run every time and required to keep surviving. Guards that are not graded `MUTATION` each carry a STATED reason in that same manifest, keyed on the guard and compared for SET EQUALITY against the inventory''s `GUARD` rows in both directions — so a new guard cannot arrive without someone writing what a proof would need, and a reason cannot outlive the row it is about. Keying the reason on the row''s GRADE instead is tautological (a new guard inherits one and nobody looks at it) and a pinned COUNT moves only on net change; both were tried and are rejected. The sandbox is a real git repository built from `git ls-files` with working-tree content, never a filesystem walk. THE POPULATION IS EVERY DECLARED MUTATION-OUTCOME CLAIM, NOT ONLY THE INVENTORY''S ROWS (ersatztv#881): a sentence asserting that a specific mutation reddens, or does not redden, a named test is the same kind of claim wherever it is written — a code comment, a test docstring, a decision record — and is either a `CLAIMS` entry in `scripts/tests/mutation_manifest.py` that executes every run, or it is not written. A claim entry is keyed on the PROSE rather than on a guard: it carries the tracked `site` and the verbatim `quote`, both checked every run, so a reworded sentence reports as a RETARGET instead of drifting away from the entry that justifies it. THE QUOTE SPANS THE MUTATION AND THE OUTCOME IT ASSERTS: the outcome half is the checkable part, so a quote stopping at the mutation binds the uncontroversial words and leaves the assertion, and the test it names, outside the binding entirely. It declares `RED` or `GREEN`, and GREEN is the half no `MUTATION` row can express — a claim that a mutation is NOT noticed is read by THREE separate clauses, because each alone certifies a run that proves nothing: the run exited 0, something actually PASSED, and the named proof was shown to REACH the mutated clause. THE TWO DIRECTIONS ARE NOT SYMMETRIC, WHICH IS WHY THE THIRD CLAUSE EXISTS ON ONLY ONE OF THEM. A RED claim checks its own relevance for free — a proof that never executes the mutated file stays green and is refused in the same words as a clause that has stopped being load-bearing — while a GREEN claim gets precisely the green it asserts from a proof that reads a different file entirely, so exit status and a passing test TOGETHER still certify a run that never executed the clause. A GREEN entry therefore also declares a SECOND mutation of the SAME clause, required to REDDEN the same proof and carrying its own declared diagnostic, executed through the same red-reading path the inventory rows use; only once that passes is the green evidence about this clause at all. It is worth what the `DETECTOR` survivor was worth: re-run, and required to KEEP surviving. THE REFERENT IS DECLARED, NEVER INFERRED, exactly as the clause is: a checker hunting for sentences that merely SOUND like mutation claims would be a string predicate over prose, which this repo refuses to make load-bearing, so what is enforced is a claim that named its own referent, and the direction that cannot be mechanised — prose that declares nothing — is stated rather than papered over. TWO NEIGHBOURING PROPOSALS ARE SETTLED HERE. A QUOTATION of another file is a claim about that file: adopted, and the `quote` field IS the check, wherever the referent is declared; a general quotation-checker is rejected for the same reason inference is, since a quotation whose referent is unnamed has nothing to resolve. ANCHORING PROSE TO A STATE THE COMMIT ITSELF MOVES (''the last edit to that path'', ''the only X that''): rejected as a detector and kept as a phrasing rule. The rejection is measured, and the scan is PINNED IN THE BODY rather than summarised as a figure here, for the same reason the population scan is — a count quoted without the predicate that produced it is the defect this record is about. What it shows is that the shape is indistinguishable BY PATTERN from ordinary rationale (''X is the only thing that can Y''), which this rule carves out, so a detector would fire mostly on prose it must not flag, and an advisory check that does that stops being read. Anchor to a date or a sha under `docs.no-session-narrative`''s dated-measurement clause; inside this rule''s own population the anchor is the manifest entry, which is re-executed and therefore cannot go stale.' signals: 'mutation harness · declared clause not inferred · MUTATION row is executed · set equality manifest vs inventory · CLAUSE vs DETECTOR granularity · surviving finer mutation is re-run · pytest exit code 1 is the only red that counts · every other exit status is rejected · positive control before any mutation · sandbox from git ls-files · paths: `scripts/tests/test_mutation_harness.py`, `scripts/tests/mutation_manifest.py`, `scripts/tests/mutation_harness_lib.py`, `docs/guard-inventory.md` · prose claim bound to its declared site and verbatim quote · GREEN direction executes a claim that a mutation is NOT noticed · a GREEN entry declares a reach mutation proving its proof touches the clause · quotation of another file is a claim about that file · state anchors rejected as a detector, kept as a phrasing rule · issues: #790, #775, #774, #778, #806, #812, #881' -mechanics: 'The harness builds ONE sandbox per session — tracked files only, from `git ls-files -s`, in an environment with every `GIT_*` variable stripped so an exported `GIT_DIR`/`GIT_COMMON_DIR`/`GIT_CONFIG_*` cannot point its `init`/`add`/`commit` at the real repository — and resets it between mutations with `git reset --hard` TO A BASELINE COMMIT RECORDED OUTSIDE THE REPOSITORY, plus `git clean -qffdx`. Both halves matter: bare `git reset --hard` resets to whatever HEAD is, so a proof test that COMMITS inside the sandbox moves HEAD onto a commit carrying the mutant and every later reset restores it faithfully; and a baseline held as a REF inside the sandbox is one more thing a proof can move. `core.hooksPath`, `commit.gpgsign` and `core.worktree` are pinned on every git invocation for the same reason. The `-ff` removes a nested repository a proof may have left. It runs the full set of named proof tests UNMUTATED first, inside the fixture rather than as a separate test so ordering is a dependency and not a convention. Only pytest exit code 1 counts as red; every other status is rejected. That is what closes the two ways a proof ref goes stale, and they are measured rather than assumed: with an explicit `file.py::function` node id a missing file and a missing function both exit 4, while 5 needs a collection that succeeded and selected nothing. The sandbox skips the `ErsatzTV-macOS` gitlink and recreates the `.claude/skills/jellyfin` symlink as a symlink. `CLAIMS` reuses the same sandbox, the same reset and the same clause application (`_mutate_and_run`, split out so the two directions cannot come to disagree about what a red is worth), and its proof refs join the positive control for the reason the row proofs do, which holds in BOTH directions: over a proof that was ALREADY red, a RED claim is satisfied by redness its own mutation did not cause, and a GREEN claim is REFUSED for a reason that has nothing to do with its mutation — reported as a wrong sentence when what is wrong is the proof. What that control cannot do is see ONE ref that collects nothing — its assertions are over the aggregate of every ref, so the other refs supply the word `passed` — nor ONE ref that collects plenty and touches nothing the claim mutates, which is the larger of the two, since such a proof is green before and after and a GREEN claim would read as verified. Both discriminations are per-claim, in the GREEN direction''s own vacuity and relevance gates. THE RELEVANCE GATE RUNS LAST BECAUSE IT IS THE EXPENSIVE ONE: it is the only one of the three that costs a second full run of the proof, under the reach mutation with a sandbox reset between, so an entry the status or the vacuity gate has already refused never pays for it. What keeps the three gates worth three is that each carries its own disarm proof, rather than the position it sits in. The sandbox is reset between a claim''s two proof runs, for the reason it is reset between mutations. Each claim''s `site` is tested for membership of the git INDEX, not `is_file`, so a claim pointing at an untracked file fails here instead of passing on one machine. Cost, re-measured 2026-09-05 WITH the relevance gate, on an M-series Mac with `-p no:randomly`, whole file, three baseline/branch pairs run back to back: the `CLAIMS` half adds 31.7-43.6%, which is the span of the three pairs and not a rounding of it — 43.6% (46.8s -> 67.2s), 34.1% (50.1s -> 67.2s) and 31.7% (48.9s -> 64.4s), under varying concurrent load. The figure this replaces, 12.7-16.6% measured 2026-09-04, was taken before the GREEN direction had a relevance gate and is retired rather than adjusted: the gate runs a GREEN entry''s proof a SECOND time under the reach mutation, with a sandbox reset between, and its three disarm proofs each drive further inner pytest runs. A measurement is a property of the population that produced it, so growing the population invalidates it — re-measure, never scale. Quote the PERCENTAGE rather than the seconds - the absolute figures moved 30% with machine load in one session, and the earlier ~14s here covered fewer entries. The ~4min `script-tests` suite it was measured against is PRE-#870 and no longer holds: that change made `count_pr_mutations` read to its 20-page cap, and `test_pr_changed_files.py` went from 202s to 474s locally (286 -> 294 tests, same machine, `-p no:randomly`, 528383cf3 vs the #870 branch), so the job is roughly double. Re-measure on a runner before quoting a job figure.' +mechanics: 'The harness builds ONE sandbox per session — tracked files only, from `git ls-files -s`, in an environment with every `GIT_*` variable stripped so an exported `GIT_DIR`/`GIT_COMMON_DIR`/`GIT_CONFIG_*` cannot point its `init`/`add`/`commit` at the real repository — and resets it between mutations with `git reset --hard` TO A BASELINE COMMIT RECORDED OUTSIDE THE REPOSITORY, plus `git clean -qffdx`. Both halves matter: bare `git reset --hard` resets to whatever HEAD is, so a proof test that COMMITS inside the sandbox moves HEAD onto a commit carrying the mutant and every later reset restores it faithfully; and a baseline held as a REF inside the sandbox is one more thing a proof can move. `core.hooksPath`, `commit.gpgsign` and `core.worktree` are pinned on every git invocation for the same reason. The `-ff` removes a nested repository a proof may have left. It runs the full set of named proof tests UNMUTATED first, inside the fixture rather than as a separate test so ordering is a dependency and not a convention. Only pytest exit code 1 counts as red; every other status is rejected. That is what closes the two ways a proof ref goes stale, and they are measured rather than assumed: with an explicit `file.py::function` node id a missing file and a missing function both exit 4, while 5 needs a collection that succeeded and selected nothing. The sandbox skips the `ErsatzTV-macOS` gitlink and recreates the `.claude/skills/jellyfin` symlink as a symlink. `CLAIMS` reuses the same sandbox, the same reset and the same clause application (`_mutate_and_run`, split out so the two directions cannot come to disagree about what a red is worth), and its proof refs join the positive control for the reason the row proofs do, which holds in BOTH directions: over a proof that was ALREADY red, a RED claim is satisfied by redness its own mutation did not cause, and a GREEN claim is REFUSED for a reason that has nothing to do with its mutation — reported as a wrong sentence when what is wrong is the proof. What that control cannot do is see ONE ref that collects nothing — its assertions are over the aggregate of every ref, so the other refs supply the word `passed` — nor ONE ref that collects plenty and touches nothing the claim mutates, which is the larger of the two, since such a proof is green before and after and a GREEN claim would read as verified. Both discriminations are per-claim, in the GREEN direction''s own vacuity and relevance gates. THE RELEVANCE GATE RUNS LAST BECAUSE IT IS THE EXPENSIVE ONE: it is the only one of the three that costs a second full run of the proof, under the reach mutation with a sandbox reset between, so an entry the status or the vacuity gate has already refused never pays for it. What keeps the three gates worth three is that each carries its own disarm proof, rather than the position it sits in. The sandbox is reset between a claim''s two proof runs, for the reason it is reset between mutations. Each claim''s `site` is tested for membership of the git INDEX, not `is_file`, so a claim pointing at an untracked file fails here instead of passing on one machine. Cost, re-measured 2026-09-05 WITH the relevance gate, on an M-series Mac with `-p no:randomly`, whole file, three baseline/branch pairs run back to back: the `CLAIMS` half adds 31.7-43.6%, which is the span of the three pairs and not a rounding of it — 43.6% (46.8s -> 67.2s), 34.1% (50.1s -> 67.2s) and 31.7% (48.9s -> 64.4s), under varying concurrent load. The figure this replaces, 12.7-16.6% measured 2026-09-04, was taken before the GREEN direction had a relevance gate and is retired rather than adjusted: the gate runs a GREEN entry''s proof a SECOND time under the reach mutation, with a sandbox reset between, and its three disarm proofs each drive further inner pytest runs. A measurement is a property of the population that produced it, so growing the population invalidates it — re-measure, never scale. That span belongs to the THREE entries #881 shipped; #885 added a fourth (the PR-route stored-secret invariant, whose guard file already holds the one `MUTATIONS` entry its key allows), so the span is a LOWER BOUND until it is re-taken. Re-taking it was attempted 2026-09-05 on this machine and REFUSED rather than skipped: three A/B pairs — whole file, `-p no:randomly`, arm A deselecting the per-claim execution and arm B the shipped file — came out 54.3s/149.5s, 81.6s/78.7s and 114.3s/84.2s while other builds shared the host, and TWO of the three are inverted, which is a load reading and not a cost reading. Take it on a quiet machine; a number off a contended one would be worse than the dated lower bound it replaced. Quote the PERCENTAGE rather than the seconds - the absolute figures moved 30% with machine load in one session, and the earlier ~14s here covered fewer entries. The ~4min `script-tests` suite it was measured against is PRE-#870 and no longer holds: that change made `count_pr_mutations` read to its 20-page cap, and `test_pr_changed_files.py` went from 202s to 474s locally (286 -> 294 tests, same machine, `-p no:randomly`, 528383cf3 vs the #870 branch), so the job is roughly double. Re-measure on a runner before quoting a job figure.' --- **What #775 left open, in its own words: "nothing checks that a row claiming `MUTATION` is telling diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index 4e1818700..e622de0b9 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -172,7 +172,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_ci_image_pin_population.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — "docker-build is the ONLY workflow pinning the toolchain image" | converted, and `*.yaml` added: Gitea accepts both spellings, so a `.yaml` workflow was structurally invisible while the test read as covering all of them | | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | -| `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | +| `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes, over TWO invariants on one population** — every `actions/checkout` in every workflow must drop the persisted credential (#835), and since #885 no job the `pull_request` trigger reaches, and no workflow scope outside `jobs:`, may name a stored secret (`ci.pr-route-carries-no-stored-credential`) | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires. The second invariant reuses that population and adds two derivations of its own — which jobs the trigger reaches (an `if:` PIN, not a parser) and where a reference to the secret store can be written — and it carries the same no-exemption rule, `secrets.GITEA_TOKEN` excepted as INJECTED rather than stored. THE GRADE BELOW COVERS ONE HALF: the manifest keys at most one declared clause mutation per GUARD file, so that entry stands for the checkout invariant. The route invariant is bound instead by a `CLAIMS` entry on this sentence — deleting `build`'s `if: github.event_name != 'pull_request'` from the shipped `docker-build.yml` reddens `test_no_PULL_REQUEST_route_job_names_a_STORED_secret` — which the harness executes every run for the same reason a `MUTATION` row is executed. A row that understates its guard is invisible to `test_guard_inventory.py`, which compares FILE sets, so this prose is the only thing that carries the second invariant | | `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | @@ -270,7 +270,7 @@ recorded as unexamined rather than as cleared. | `scripts/tests/test_prove_fix.py` | the `script-tests` job | PROOF | NONE | — | | `scripts/tests/test_review_verdict_vocabulary.py` | the `script-tests` job, when the H10 verdict vocabulary stops being shared — a word added to the one declaration that fails to reach the write side or the read side | GUARD | MUTATION | `test_review_verdict_vocabulary.py::test_a_word_added_to_the_shared_source_reaches_BOTH_sides` | | `scripts/tests/test_remote_state_inventory.py` | the `script-tests` job, on an executable that talks to a remote service with no row in `docs/remote-state-inventory.md` | GUARD | MUTATION | `test_remote_state_inventory.py::test_every_in_scope_file_has_a_row_and_every_row_names_a_real_file` | -| `scripts/tests/test_workflow_persist_credentials.py` | the `script-tests` job, when an `actions/checkout` step no longer sets `persist-credentials: false` | GUARD | MUTATION | `test_workflow_persist_credentials.py::test_every_actions_checkout_DROPS_the_persisted_credential` | +| `scripts/tests/test_workflow_persist_credentials.py` | the `script-tests` job, when an `actions/checkout` step no longer sets `persist-credentials: false`, or when a job on the `pull_request` route — or the workflow scope outside `jobs:` — names a stored secret. The declared clause mutation opposite covers the checkout half; the route half is executed every run through the `CLAIMS` entry named in the population table above | GUARD | MUTATION | `test_workflow_persist_credentials.py::test_every_actions_checkout_DROPS_the_persisted_credential` | | `scripts/tests/test_workflow_job_guards.py` | the `script-tests` job, on a workflow JOB that is a guard with no row in the workflow-job table below, or a job with no `CI_JOB_ROLE` at all | GUARD | MUTATION | `test_workflow_job_guards.py::test_the_inventory_covers_exactly_the_guard_JOBS_that_exist` | | `scripts/tests/test_worktree_ownership_guard.py` | the `script-tests` job | PROOF | NONE | — | diff --git a/scripts/tests/mutation_manifest.py b/scripts/tests/mutation_manifest.py index c6e31fa79..cc5478603 100644 --- a/scripts/tests/mutation_manifest.py +++ b/scripts/tests/mutation_manifest.py @@ -679,4 +679,30 @@ CLAIMS: tuple[Claim, ...] = ( "The mutation drops `record_wing_faults()` from the call and keeps `yaml_faults`, which is " "the deletion the sentence describes rather than a caricature of it.", ), + Claim( + site="docs/guard-inventory.md", + quote=( + "deleting `build`'s `if: github.event_name != 'pull_request'` from the shipped " + "`docker-build.yml` reddens `test_no_PULL_REQUEST_route_job_names_a_STORED_secret`" + ), + target=".gitea/workflows/docker-build.yml", + clause=" if: github.event_name != 'pull_request'", + replacement=" # if: clause removed by the mutation harness", + proof="test_workflow_persist_credentials.py::test_no_PULL_REQUEST_route_job_names_a_STORED_secret", + outcome=RED, + expect="job `build` names stored secret(s) on the pull_request route", + why="`MUTATIONS` is keyed one entry per GUARD FILE, and " + "`scripts/tests/test_workflow_persist_credentials.py` holds TWO invariants since #885 — the " + "`actions/checkout` one its declared entry stands for, and the PR-route stored-secret one. " + "The second would otherwise have no executed mutation at all, so the inventory row states it " + "in prose and the prose is bound here, which is the population #881 widened this file to " + "cover. The clause is the ONE thing keeping `build` — the last job naming `REGISTRY_PASSWORD` " + "— off the head-authored route, so the mutation is the defect the invariant exists for and " + "not a caricature: with it deleted the collector must name that job and that secret. The " + "mutation is driven into the SHIPPED workflow rather than a fixture for the same reason the " + "module's own two mutation tests are: a synthetic document would only prove the collector " + "works on YAML this repo does not run. The declared diagnostic is the fault's own wording " + "rather than the assertion's, because a red anywhere else in that test — a parse error, an " + "empty population — prints a different message and is evidence about nothing.", + ), ) diff --git a/scripts/tests/test_workflow_persist_credentials.py b/scripts/tests/test_workflow_persist_credentials.py index 2ff80f83b..53302351c 100644 --- a/scripts/tests/test_workflow_persist_credentials.py +++ b/scripts/tests/test_workflow_persist_credentials.py @@ -1,4 +1,9 @@ -"""Every `actions/checkout` drops the persisted credential (ersatztv#835). +"""TWO credential invariants over one derived workflow population. + +Every `actions/checkout` drops the persisted credential (ersatztv#835) — below — and no job +the `pull_request` trigger reaches, nor the workflow scope outside `jobs:`, names a stored +secret (ersatztv#885). The second is documented at its own banner further down, where the +population it adds to this one is derived; what follows here is the first. WHAT THIS IS PROTECTING. `actions/checkout` writes an `Authorization` header into `.git/config` unless `persist-credentials: false` is set, and this instance's Actions default token permission is