From febaad77d770e114e1a5cefb085468eb755d6b32 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 10:24:00 +0200 Subject: [PATCH 01/16] fix(887): the image build builds the SPA and does not test it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `docker/Dockerfile`'s web-build stage is gitless twice over — the build context is `web/` + `design-system/` so there is no `.git`, and `node:22-bookworm-slim` ships no git binary. Members of the SPA suite need one or the other, so running the suite there required naming the ones that cannot run. That list was a population nothing derived: #883 added a third member without updating the hand-written pair of `--exclude`s, and because `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` the resulting red was unreachable on a PR. It landed on `main` and on the `v*` tag path instead — every image build failed, `:latest` stopped being republished, and a release cut would have failed at the image build. Adding a third `--exclude` re-arms the trap, so the list is removed rather than extended: the stage now lints, typechecks and BUILDS the SPA, and the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout. `build` carries `needs: [test, migrations, scan]`, so no image is published past a red suite. `scripts/tests/test_image_build_delegates_the_spa_suite.py` holds both halves — the negative one alone would be satisfied by deleting the `needs:` edge. Three populations, all derived: tracked Dockerfiles and workflows from the git index, and which npm scripts ARE the suite from `web/package.json` (so `test` is in and the Playwright `test:ui-e2e` is out, with no exemption list). Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step's own `file:` input, which is why `ci-image.yml` is out of scope by derivation rather than by an entry that would outlive its reason. Four mutants witnessed red, each by the intended test: a filtered suite run put back into the Dockerfile, the `needs:` edge deleted, and the gating run narrowed in both the block and the single-line `run:` step forms. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docker/Dockerfile | 60 +- docs/decisions/README.md | 1 + .../ci/image-build-delegates-the-spa-suite.md | 57 ++ docs/guard-inventory.md | 8 +- docs/testing.md | 2 +- scripts/tests/mutation_manifest.py | 18 + .../test_guard_populations_derive_from_git.py | 3 + ...est_image_build_delegates_the_spa_suite.py | 738 ++++++++++++++++++ .../trackedSourceFiles.realgit.test.ts | 9 +- 9 files changed, 858 insertions(+), 38 deletions(-) create mode 100644 docs/decisions/records/ci/image-build-delegates-the-spa-suite.md create mode 100644 scripts/tests/test_image_build_delegates_the_spa_suite.py diff --git a/docker/Dockerfile b/docker/Dockerfile index dd8eac5f3..abe2d1a78 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -9,37 +9,35 @@ WORKDIR /source COPY design-system/. ./design-system/ COPY web/. ./web/ WORKDIR /source/web -# The SPA suite runs here except for two files (ersatztv#819), excluded for OVERLAPPING reasons — -# one needs the git binary, the other needs the binary AND a checkout. Reading them as two separate -# reasons is what broke this stage once already: -# * `web/src/api/pageSizeCallSites.guard.test.ts` needs a git CHECKOUT — and, through it, the -# binary. It derives its file population from `git ls-files` rather than a directory walk, and -# refuses rather than falling back. This stage's context is `web/` + `design-system/` only, so -# there is no `.git`. -# * `web/vite-plugins/trackedSourceFiles.realgit.test.ts` needs the BINARY but no checkout: it -# builds its own temp repository to prove the derivation by executing it. -# `node:22-bookworm-slim` ships no git (`command -v git` -> not found), so it dies with -# `spawnSync git ENOENT`. -# So this is NOT checkout-versus-binary, and adding a `COPY .git` would not let either run here — -# the binary would still be missing. Excluding only the first is not enough either, and a replica -# that merely deletes `.git` cannot show that: verify any change here with the git binary off -# `PATH`, not just with the directory absent. -# Everything else — all but those two files — runs fine gitless and is kept, rather than dropping -# the whole suite for one file as an earlier fix here did. -# The excluded pair is not skipped overall: `docker-build.yml`'s `Build & test (.NET)` job runs the -# whole suite on a real checkout, and `build` (the job that invokes this Dockerfile) carries -# `needs: [test, migrations, scan]`. State that chain precisely, because the `needs:` edge is not -# all of it: `Test SPA` is also gated on `docs_only` and on the #420 revalidate skip, and `build` is -# not gated on `revalidate`. On a push whose tree is byte-identical to an already-green head the -# suite is skipped and the image still builds — carried there by #420's byte-identical-tree -# argument. The `docs_only` arm cannot ship an image at all (`Build and push` is gated on it too), -# and `ci-detect-docs-only.sh` classifies by PATH SHAPE (`docs/` or `*.md`), not by directory, so a -# `web/*.md` would count as docs — there are none today, but do not restate it as "any `web/**`". -RUN npm run lint && npm run typecheck && \ - npm test -- --run \ - --exclude 'src/api/pageSizeCallSites.guard.test.ts' \ - --exclude 'vite-plugins/trackedSourceFiles.realgit.test.ts' && \ - npm run build +# THE VITEST SUITE IS NOT RUN IN THIS STAGE, AND MUST NOT BE RE-ADDED (ersatztv#887). +# This stage is gitless in both senses at once: the build context is `web/` + `design-system/`, so +# there is no `.git`, and `node:22-bookworm-slim` ships no git binary (`command -v git` -> not +# found). Members of the SPA suite need one or the other — `web/src/api/*.guard.test.ts` derive +# their file population from `git ls-files` and refuse to fall back to a directory walk +# (`testing.guard-derives-population-from-source`), and `web/vite-plugins/*.realgit.test.ts` builds +# its own temp repository. So a `COPY .git` alone would not make the suite runnable here; the +# binary would still be missing. +# +# Running the suite here anyway therefore costs a hand-maintained list of the members that cannot +# run — and that list is a population nothing derives. It went stale the first time a guard was +# added without updating it, and because `Build & push image (amd64)` is `if: github.event_name != +# 'pull_request'`, the resulting red is unreachable on a PR and lands on `main` and on the `v*` tag +# path: every image build failed and `:latest` stopped being republished. Excluding one more file +# re-arms that; the list is removed instead. +# +# WHAT STILL VALIDATES THE SUITE. `docker-build.yml`'s `test` job runs it UNFILTERED on a real +# checkout, and `build` — the job that invokes this Dockerfile — carries `needs: [test, migrations, +# scan]`, so an image cannot be published past a red suite. Two skips exist inside `test` and +# neither leaves an image unvalidated: `docs_only` also gates `Build and push`, so that arm ships +# nothing; the #420 revalidate skip fires only on a tree byte-identical to a head that already +# carried a green combined status, i.e. this exact source already passed. Held by +# `scripts/tests/test_image_build_delegates_the_spa_suite.py`, which derives both halves rather +# than restating them. +# +# `lint` and `typecheck` stay. They are gitless-safe with no member that is not, so they carry no +# list and no trap; whether the image build should run them at all is a separate question this +# does not answer. +RUN npm run lint && npm run typecheck && npm run build FROM --platform=linux/amd64 192.168.1.95:3000/timothy/ersatztv-ffmpeg:8.1.2 AS runtime-base COPY --from=dotnet-runtime /usr/share/dotnet /usr/share/dotnet diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 320f9a6bd..3b8bb9f3e 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,6 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md new file mode 100644 index 000000000..ff07195b4 --- /dev/null +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -0,0 +1,57 @@ +--- +key: ci.image-build-delegates-the-spa-suite +title: '2026-08-30 — the image build builds the SPA and does not test it (#887)' +status: active +since: '2026-08-30' +supersedes: none +superseded-by: none +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' +mechanics: 'Guard derives three populations — tracked Dockerfiles, tracked workflows, and the vitest script names read out of `web/package.json` — then asserts no Dockerfile runs the suite, every publishing job whose Dockerfile carries the SPA source transitively `needs:` a job that does, and every such run is unfiltered. Publishing jobs come from the `docker/build-push-action` step; the Dockerfile each builds comes from that step''s `file:` input.' +--- + +The web-build stage is gitless twice over: its build context is `web/` + `design-system/`, so there +is no `.git`, and `node:22-bookworm-slim` ships no git binary. Members of the SPA suite need one or +the other — the `web/src/api/*.guard.test.ts` guards derive their file population from `git ls-files` +and refuse to fall back to a directory walk (`testing.guard-derives-population-from-source`), and +`trackedSourceFiles.realgit.test.ts` builds a real temporary repository. Because the two +prerequisites OVERLAP rather than divide, adding a `COPY .git` fixes neither: the binary is still +missing. + +So running the suite there at all costs a list of the members that cannot run, maintained by hand +beside a suite that grows. That list went stale the first time a guard was added without updating +it, and the failure it produced was invisible where it could have been cheap and fatal where it was +not: green on every PR, red on `main` and on the release tag. + +**Why not the alternatives.** Each was considered and rejected on its own evidence, not by analogy: + +* **Add a third `--exclude`.** Re-arms the same trap for the next guard, which is what the issue was + filed to prevent. +* **Derive the exclusions from source.** Buildable, but it requires a predicate for "needs git" over + spec text — two unrelated shapes today (a virtual-module import, a `git` spawn) and no reason to + think that stays two. A string predicate over source is the wrong instrument for a set whose + members are defined by what they DO at runtime. +* **Give the container git** (`apt-get install git` plus a real `.git`). It works, and it costs a + cache-busting `COPY` of VCS metadata on every commit, so the whole lint/typecheck/test/build layer + re-runs for changes that do not touch `web/`. Synthesising a repository in the image instead — + `git init && git add -A` — is worse than it looks: the resulting index is a filesystem walk with + extra steps, which is precisely the population the guards refuse. + +**What the removal gives up, enumerated rather than waved through** +(`process.enumerate-workaround-behaviors-before-deleting`): + +1. *Blocking a publish on a red suite* — retained, by the `needs:` edge. Two skips exist inside + `test` and neither leaves an image unvalidated: `docs_only` also gates `Build and push`, so that + arm ships nothing, and the #420 revalidate skip fires only on a tree byte-identical to a head + that already carried a green combined status. +2. *Running the suite under `node:22-bookworm-slim` specifically* — **lost**, and this is the real + cost. It is small because the suite is jsdom and pure JS, and because the artifact the image + actually ships is the output of `vite build`, which still runs there: a test that passes in the + CI toolchain image and fails in bookworm-slim would be telling us about the test environment, not + about the image. +3. *`docker build` running the suite for a developer locally* — lost; `npm test -- --run` is the + direct route and is what `docs/testing.md` documents. + +`lint` and `typecheck` stay in the stage. They are gitless-safe with no member that is not, so they +carry no list and no trap. Whether the image build should run them at all is a separate question +this does not answer, and leaving them is not an argument that it should. diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index 133280191..2fefe5284 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -74,7 +74,9 @@ with have sat in these gaps. Treat anything not named here as unassessed, not as and the pair `web/vite-plugins/trackedSourceFiles.test.ts` + its `.realgit.test.ts` sibling, which together prove the population `pageSizeCallSites.guard.test.ts` derives (split because the `realgit` half - needs the git BINARY, which the Docker web-build stage has not got). Note what that costs: this + needs the git BINARY and its thirteen injected-runner siblings do not, so the prerequisite is + confined to one file — it is NOT the Docker exclusion it was originally written for, which + ersatztv#887 removed along with the whole in-image suite run). Note what that costs: this list is a HAND-WRITTEN mirror of a population nothing derives, so it goes stale silently and CI stays green — #807 added the third entry, and nothing mechanical caught that `pageSizeCallSites.guard.test.ts` had become the only one named. @@ -171,6 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | +| `test_image_build_delegates_the_spa_suite.py` | `.gitea/workflows/*.y*ml` + every tracked `Dockerfile*`/`*.Dockerfile`, plus the vitest SCRIPT NAMES read out of `web/package.json` | **yes, in both directions** — no Dockerfile in the population may run the SPA suite, and every image-publishing job whose Dockerfile carries the SPA source must transitively `needs:` a job that runs it unfiltered | ersatztv#887, index-derived from the start. THREE populations rather than one, because a single one leaves a hole the other two close: the Dockerfile set alone cannot see a deleted `needs:` edge, and the job set alone cannot see a suite run added to a gitless stage. The publishing jobs are derived from the `docker/build-push-action` step and the Dockerfile each builds is read from that step's own `file:` input, so `ci-image.yml` — which publishes from `docker/ci/Dockerfile`, a Dockerfile carrying no SPA source — is out of scope by DERIVATION and holds no exemption entry that could outlive its reason. Which npm scripts count as the suite is derived too, from `web/package.json`: `test` is in because `"test": "vitest"` is, and the sibling `test:ui-e2e` is out because it runs Playwright — a hardcoded name would go quiet on a rename, and a `test:`-prefix guess would demand an unfiltered gate on a browser suite. RESIDUALS, stated: a `RUN` that calls a SCRIPT which runs the suite is invisible (the parser reads commands, not what they call — no such script exists today, and the miss is a missed DETECTION, not a false green about the gate, which is derived separately); only `RUN` is read in a Dockerfile, since nothing else executes during a build; and a line whose first non-space character is `#` is inert, which is what lets `docker/Dockerfile` name the rule it is subject to — only a LEADING `#`, so an invocation with a trailing note is still seen | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | @@ -249,6 +252,7 @@ recorded as unexamined rather than as cleared. | `scripts/tests/test_guard_inventory.py` | the `script-tests` job, on an unclassified guard or a stale proof ref | GUARD | MUTATION | `test_guard_inventory.py::test_the_inventory_covers_exactly_the_guards_that_exist` | | `scripts/tests/test_guard_populations_derive_from_git.py` | the `script-tests` job, on a guard whose file population admits a file git does not track | GUARD | MUTATION | `test_guard_populations_derive_from_git.py::test_no_derivation_admits_an_untracked_file` | | `scripts/tests/test_hook_fire_log.py` | the `script-tests` job, on a hook that stops reporting that it fired, or whose reporting changes what the harness sees | GUARD | MUTATION | `test_hook_fire_log.py::test_a_hook_that_LOSES_its_instrumentation_is_DETECTED` | +| `scripts/tests/test_image_build_delegates_the_spa_suite.py` | the `script-tests` job, on a Dockerfile that runs the SPA vitest suite, on an image-publishing job that stops being gated on a job that runs it, or on a gating run that is narrowed by a spec filter | GUARD | MUTATION | `test_image_build_delegates_the_spa_suite.py::test_no_DOCKERFILE_runs_the_spa_suite` | | `scripts/tests/test_jq_preflight.py` | the `script-tests` job | PROOF | NONE | — | | `scripts/tests/test_mcp_smoke.py` | the `script-tests` job, when the MCP smoke checker accepts a server it should refuse — an impostor identity, a reply carrying only an id, a missing expected tool, or an `initialize` pre-answered before the request (the `tools/list` twin is held by two mechanisms jointly and is covered behaviourally, not by the declared clause) | GUARD | MUTATION | `test_mcp_smoke.py::test_MUTATION_a_PRE_ANSWERED_id_is_refused_because_the_request_ids_are_UNGUESSABLE` | | `scripts/tests/test_merge_consent_base_change.py` | the `script-tests` job | PROOF | NONE | — | @@ -375,7 +379,7 @@ clause-provable and the entry is regraded. ## What the numbers say -45 guards, 6 tooling scripts, 23 proof files. **25 guards carry a mutation proof; 5 are +46 guards, 6 tooling scripts, 23 proof files. **26 guards carry a mutation proof; 5 are behaviour-only; 15 have none.** These figures are asserted against the table by `test_the_summary_counts_match_the_table`, because a hand-maintained summary of a table is a second copy of it — the duplication family this change argues against, and one that has already drifted diff --git a/docs/testing.md b/docs/testing.md index 0d93d1685..dbc04b5d5 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -20,7 +20,7 @@ before adding tests, not just `docs/contributing.md` §8 (which now just points | `ErsatzTV.Scanner.Tests` | Library scanning: scan handlers, folder scanners, NFO readers | Handler tests substitute the folder scanners + `ILibraryRepository` and assert the resulting repository writes (e.g. `ScanLocalLibraryHandlerTests` pins which `LastScan` levels a scan records — ersatztv#264). Fakes/`Testably` back the file-system-facing scanners. ~1,485 tests (approximate on purpose — an exact count goes stale on every PR that adds one). Additionally contains `Core/FFmpeg/TranscodingTests` — `[Explicit]` + `[Combinatorial]`, so it never runs in CI or a plain `dotnet test` (it needs real ffmpeg/hardware) and contributes 0 to that count; run it by name when touching the transcoding pipeline. | | `ErsatzTV.Architecture.Tests` | Layering rules via NetArchTest.eNhancedEdition | Core↛Infra/App/EF; FFmpeg↛all; App↛concrete providers. 5 tests. See `docs/contributing.md` §1. | | `ErsatzTV.FFmpeg.Tests` | FFmpeg command construction | Build a pipeline, assert the exact rendered arg string (`PipelineBuilderBaseTests.cs`). | -| `web/` (vitest) | React SPA unit tests | Run alongside typecheck + build (see below). Collects `src/**`, `web/scripts/**` *and* `web/vite-plugins/**`, but deliberately **excludes** `web/e2e/**` (the Playwright specs — vitest's default `**/*.spec.*` glob would otherwise run them under jsdom). Two files have git prerequisites since ersatztv#819. `web/src/api/pageSizeCallSites.guard.test.ts` needs a git **checkout** AND, through it, the **binary** — it derives its file population from `git ls-files` via `web/vite-plugins/trackedSourceFiles.ts` rather than a directory walk, and refuses rather than falling back. `web/vite-plugins/trackedSourceFiles.realgit.test.ts` needs the **binary** but no checkout: it builds its own temp repository to prove that derivation by executing it. So it is not checkout-versus-binary — supplying a `.git` alone would not let either run. Every other file runs fine with neither. `docker/Dockerfile`'s web-build stage runs the suite with exactly those two `--exclude`d, because its context carries no `.git` and `node:22-bookworm-slim` ships no git — and the two exclusions overlap rather than divide — both files need the binary. | +| `web/` (vitest) | React SPA unit tests | Run alongside typecheck + build (see below). Collects `src/**`, `web/scripts/**` *and* `web/vite-plugins/**`, but deliberately **excludes** `web/e2e/**` (the Playwright specs — vitest's default `**/*.spec.*` glob would otherwise run them under jsdom). Some files have git prerequisites since ersatztv#819, and the set is not fixed — ersatztv#883 added one. `web/src/api/pageSizeCallSites.guard.test.ts` and `web/src/api/completeAnnotations.guard.test.ts` need a git **checkout** AND, through it, the **binary**: they derive their file population from `git ls-files` via `web/vite-plugins/trackedSourceFiles.ts` rather than a directory walk, and refuse rather than falling back. `web/vite-plugins/trackedSourceFiles.realgit.test.ts` needs the **binary** but no checkout — it builds its own temp repository to prove that derivation by executing it. So it is not checkout-versus-binary: supplying a `.git` alone would not let any of them run. **The suite therefore runs only where git is present, and `docker/Dockerfile` is not such a place** — its web-build stage builds the SPA and does not test it (ersatztv#887). Enumerating the git-dependent files as Docker `--exclude`s was tried and REVERSED: that list is a population nothing derives, it went stale the first time a guard was added, and the resulting red is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it landed on `main` and on the release tag instead. The image is gated on `docker-build.yml`'s `test` job running the whole suite on a real checkout, held by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | | `web/e2e/` (Playwright) | UI-interactive E2E flows against a **live** instance | Not a unit suite and **not** part of `npm test` — needs a running server, so it runs via `scripts/e2e-ui.sh` (boots its own fresh instance) and in CI as a step of the `functional-e2e` job. Headless Chromium, `serial`, `retries: 0`. Scope rule: assert only what the curl harness structurally cannot. See `docs/e2e-local.md` → "UI-E2E harness". | ## Golden-file nets diff --git a/scripts/tests/mutation_manifest.py b/scripts/tests/mutation_manifest.py index d09fde233..e6ff8f6d5 100644 --- a/scripts/tests/mutation_manifest.py +++ b/scripts/tests/mutation_manifest.py @@ -403,6 +403,24 @@ MUTATIONS: tuple[Mutation, ...] = ( "that it moved). Disarming the comparison inside the script instead would prove the second " "and say nothing about the first, which is the half that was missing.", ), + Mutation( + guard="scripts/tests/test_image_build_delegates_the_spa_suite.py", + target="docker/Dockerfile", + clause="RUN npm run lint && npm run typecheck && npm run build", + replacement="RUN npm run lint && npm run typecheck && npm test -- --run && npm run build", + proof="test_image_build_delegates_the_spa_suite.py::test_no_DOCKERFILE_runs_the_spa_suite", + granularity=CLAUSE, + expect="runs the SPA vitest suite, but its build context has no git", + why="The target is not the guard, per the checker-guard exception: the guard IS a test, so " + "disarming it makes it absent rather than red, and the mutation goes into the guarded " + "ARTIFACT. The clause is the web-build stage's whole command line, and the replacement is " + "the defect itself rather than a caricature of it — ersatztv#883 put a vitest run back into " + "a gitless stage, and every image build failed from that commit until #887. Mutating the " + "detector's own token list instead would prove only that the detector matches its own " + "constants; this proves it reads the real Dockerfile and reports the real shape. Note the " + "mutant is UNFILTERED: a filtered run is what actually broke, but the guard forbids the " + "class, and mutating to the narrower case would leave the broader one unproven.", + ), ) diff --git a/scripts/tests/test_guard_populations_derive_from_git.py b/scripts/tests/test_guard_populations_derive_from_git.py index 12fc69450..ecc437bb5 100644 --- a/scripts/tests/test_guard_populations_derive_from_git.py +++ b/scripts/tests/test_guard_populations_derive_from_git.py @@ -43,6 +43,7 @@ from scripts.tests import test_ci_image_pin_population as image_pin from scripts.tests import test_ci_status_context_uniqueness as ctx_unique from scripts.tests import test_guard_inventory as guard_inventory from scripts.tests import test_hook_fire_log as hook_fire +from scripts.tests import test_image_build_delegates_the_spa_suite as image_build from scripts.tests import test_pr_changed_files as pr_changed from scripts.tests import test_remote_state_inventory as remote_state from scripts.tests import test_workflow_job_guards as job_guards @@ -108,6 +109,8 @@ def _as_relative_strings(members) -> set[str]: DERIVATIONS = ( ("test_guard_inventory.derived_guard_files", guard_inventory.derived_guard_files, 25), ("test_hook_fire_log.hook_scripts", hook_fire.hook_scripts, 10), + ("test_image_build_delegates_the_spa_suite.dockerfiles", image_build.dockerfiles, 4), + ("test_image_build_delegates_the_spa_suite.workflow_files", image_build.workflow_files, 5), ("test_ci_image_pin_population.workflow_files", image_pin.workflow_files, 5), ("test_ci_status_context_uniqueness.workflow_files", ctx_unique.workflow_files, 5), ("test_remote_state_inventory.derived_population", remote_state.derived_population, 40), diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py new file mode 100644 index 000000000..37735732a --- /dev/null +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -0,0 +1,738 @@ +"""The SPA vitest suite never runs inside an image build, and the image is gated on a job that does +run it, unfiltered (ersatztv#887). + +WHAT THIS IS PROTECTING. `docker/Dockerfile`'s `web-build` stage is gitless twice over: its build +context is `web/` + `design-system/`, so there is no `.git`, and `node:22-bookworm-slim` ships no +git binary. Members of the SPA suite need one or the other — the `web/src/api/*.guard.test.ts` +guards derive their file population from `git ls-files` and refuse to fall back to a directory walk +(`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds a +real temporary repository. Running the suite there anyway therefore requires a list of the members +that cannot run, and that list is a population nothing derives: ersatztv#883 added a third member +without updating a hand-written pair of `--exclude`s, and every image build failed from that commit +on. The red was structurally unreachable on a PR — `Build & push image (amd64)` carries +`if: github.event_name != 'pull_request'` — so it landed on `main` and on the `v*` tag path, where a +release cut fails at the image build and `:latest` stops being republished. + +The fix removed the list rather than extending it. This guard is what keeps it removed, and what +holds up the safety argument that licensed removing it, because that argument is a claim about two +OTHER files and would otherwise rot silently in a comment. + +THE INVARIANT, in the two halves it actually has: + + * NEGATIVE — no Dockerfile runs the suite. Not "no `--exclude` appears": an unfiltered suite run + in a gitless stage fails just as hard, and a filtered one is the trap itself. The whole class is + excluded rather than a spelling of it. + * POSITIVE — every job that publishes an image built from a Dockerfile CARRYING THE SPA SOURCE is + transitively gated on a job that runs the suite, and that run is unfiltered. Without this half + the negative half is a hole: deleting the suite run from the Dockerfile and deleting + `needs: [test, ...]` from `build` would both be green. + +POPULATIONS, all derived — never a filesystem walk and never a hand-written list +(`testing.guard-derives-population-from-source`, ersatztv#806): + + * Dockerfiles and workflows: the git index, via `tracked_files`. Both the `*.yml` and `*.yaml` + workflow spellings, for the reason `test_workflow_persist_credentials.py` states — Gitea accepts + either, so a `.yaml` workflow would otherwise be structurally invisible. + * WHICH npm SCRIPTS ARE THE SUITE: read out of `web/package.json`, by finding the scripts whose + body invokes the vitest binary. `npm test` is a suite run because `"test": "vitest"` is; the + sibling `"test:ui-e2e": "playwright test"` is not, and no exemption says so. A guess in either + direction here is a live defect — treating `test:ui-e2e` as the suite would demand a gate on a + Playwright run, and hardcoding the name `test` would go quiet the day the script is renamed. + * Image-publishing jobs: every job with a `docker/build-push-action` step, and the Dockerfile it + builds is read out of that step's `file:` input rather than hardcoded here. That is what keeps + a renamed Dockerfile red instead of silently uncovered, and it is why `ci-image.yml` — which + publishes the CI toolchain image from `docker/ci/Dockerfile`, a Dockerfile that carries no SPA + source — is correctly out of scope WITHOUT an exemption entry. An exemption outlives its reason; + a derivation does not. + +WHY A SHELL-POSITION PARSER RATHER THAN A REGEX. The first version matched `vitest` anywhere in a +command and reported `echo the vitest suite does not run here` as an invocation. A predicate that +cannot tell a command from prose about a command produces false reds in exactly the files that +explain the rule, and those get silenced. Commands are split on shell separators and only the HEAD +of each segment is consulted, so a runner named in an argument or in prose is not a run. + +WHAT IS NOT COVERED, stated rather than implied: + + * A line whose FIRST non-space character is `#` is treated as inert, in both file kinds. That is + right for a Dockerfile comment and for a shell comment at the start of a line inside a `run:` + body, and it is what lets `docker/Dockerfile`'s own explanatory comment name the thing it + forbids. A suite invocation appended after a `#` on a line that also carries code is NOT a + comment and is still seen, because only a leading `#` is stripped. + * In a Dockerfile only `RUN` executes at build time, so only `RUN` is read. A `CMD`/`ENTRYPOINT` + that runs the suite would run in the shipped container, which is a different (and absent) + problem; it is out of scope rather than overlooked. + * A `RUN` that invokes a SCRIPT which in turn runs the suite is invisible to this — the parser + reads commands, not what they call. No such script exists today; the failure mode is a missed + detection, not a false green about the gate, because the positive half is derived independently. + * An image published by something other than `docker/build-push-action` — a raw `docker push` in a + `run:` body — is outside the publishing-job derivation. `test_no_run_BODY_builds_or_pushes_an_image` + asserts that shape does not exist rather than leaving it uncovered. + * A genuinely new value-taking vitest flag is reported as a spec filter, because every non-flag + token that is not the value of a KNOWN value-taking flag is treated as one. That is a false red + a reader fixes by naming the flag here — the safe direction. +""" + +from __future__ import annotations + +import json +import re +import shlex +from pathlib import Path + +import yaml + +from scripts.tests import tracked_files +from scripts.tests.tracked_files import tracked_paths + +REPO_ROOT = Path(__file__).resolve().parents[2] +WORKFLOW_DIR = ".gitea/workflows" +WORKFLOWS = (WORKFLOW_DIR, ("*.yml", "*.yaml")) +PACKAGE_JSON = "web/package.json" + +# The action every image publish in this repo goes through. Matched on the last two path segments +# for the reason `test_workflow_persist_credentials.py` gives: Gitea accepts a full action URL, and +# a host is not required to contain a dot, so no hostname heuristic is safe. +PUBLISH_ACTION = "docker/build-push-action" + +# Shell tokens that end one command and start another. `shlex.split` returns each of these as its +# own token, which is what makes segmenting possible without a shell grammar. +SEPARATORS = frozenset({"&&", "||", ";", "|", "&"}) + +# Prefixes that wrap a command without being one. Skipped so `env FOO=1 npm test` is still a run. +WRAPPERS = frozenset({"env", "sudo", "time", "exec", "command", "nice", "ionice"}) + +# Dockerfile instructions. Only `RUN` executes during a build; the rest are read as not-a-command so +# a `COPY` whose argument happens to be a runner name is not mistaken for one. +DOCKERFILE_INSTRUCTIONS = frozenset( + { + "ADD", + "ARG", + "CMD", + "COPY", + "ENTRYPOINT", + "ENV", + "EXPOSE", + "FROM", + "HEALTHCHECK", + "LABEL", + "MAINTAINER", + "ONBUILD", + "RUN", + "SHELL", + "STOPSIGNAL", + "USER", + "VOLUME", + "WORKDIR", + } +) + +# Flags that narrow WHICH specs run. Every one of these reintroduces the defect: the set of specs +# that ran becomes a thing someone maintains by hand, and nothing compares it to the set that exists. +NARROWING_FLAGS = frozenset( + {"--exclude", "--include", "--dir", "--project", "--shard", "--changed", "--related", "--testNamePattern", "-t"} +) + +# Flags whose NEXT token is a value rather than a spec filter, so `--reporter verbose` is not read as +# a filter. Deliberately short — see the docstring's last residual. +VALUE_TAKING_FLAGS = frozenset( + { + "--reporter", + "--outputFile", + "--config", + "--root", + "--environment", + "--pool", + "--maxWorkers", + "--minWorkers", + "--testTimeout", + "--hookTimeout", + "--retry", + "--coverage.reporter", + "--coverage.provider", + } +) + +_ENV_ASSIGNMENT = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*=") + + +def _canonical_action(uses: str) -> str: + """The last two path segments of a `uses:` value, without its `@version`. + + Same normalisation as `test_workflow_persist_credentials._canonical_action`, and deliberately a + second copy rather than an import: that guard's population is checkout steps, this one's is + publish steps, and sharing the helper would mean a narrowing there silently narrows here too. + """ + ref = uses.strip().strip("'\"").rsplit("@", 1)[0].strip().lower() + ref = re.sub(r"^https?://", "", ref) + segments = [seg for seg in ref.split("/") if seg] + if segments: + segments[-1] = re.sub(r"\.git$", "", segments[-1]) + return "/".join(segments[-2:]) if len(segments) >= 2 else ref + + +def logical_commands(text: str) -> list[tuple[int, str]]: + """`(1-based line number, command)` per command line: comments dropped, `\\` continuations joined. + + Both normalisations are load-bearing. Dropping leading-`#` lines is what lets a Dockerfile + explain the rule it enforces without tripping it. Joining backslash continuations is what makes + a multi-line invocation ONE command with its flags attached — the shape the removed `--exclude` + pair had, and the shape a re-added one would have. Scanning raw lines instead would read + `npm test -- --run` as unfiltered and report it clean. + """ + commands: list[tuple[int, str]] = [] + buffer = "" + start = 0 + for number, raw in enumerate(text.splitlines(), start=1): + if raw.lstrip().startswith("#"): + continue + if not buffer: + start = number + stripped = raw.rstrip() + if stripped.endswith("\\"): + buffer += stripped[:-1] + " " + continue + buffer += stripped + if buffer.strip(): + commands.append((start, buffer.strip())) + buffer = "" + if buffer.strip(): + commands.append((start, buffer.strip())) + return commands + + +def command_segments(command: str) -> list[list[str]]: + """The shell segments of one command, as token lists, or `[]` if nothing executes. + + A leading Dockerfile instruction is consumed: `RUN` yields its command, every other instruction + yields nothing, since only `RUN` runs during a build. + """ + try: + tokens = shlex.split(command) + except ValueError: + # An unbalanced quote is not a command anyone can reason about. Returning the raw split + # keeps it VISIBLE to the detectors rather than silently dropping it. + tokens = command.split() + if not tokens: + return [] + # A YAML step written on one line — `- run: npm test -- --run` — is a command, and roughly half + # the `run:` steps in this repo are written that way. Without this the raw-text cross-check below + # reads `run:` as the program and finds nothing, so a single-line invocation would be invisible + # to every text-based assertion while the block form was caught. + if tokens[0] == "-": + tokens = tokens[1:] + if tokens and tokens[0] == "run:": + tokens = tokens[1:] + if not tokens: + return [] + head = tokens[0] + if head.isupper() and head in DOCKERFILE_INSTRUCTIONS: + if head != "RUN": + return [] + tokens = tokens[1:] + + segments: list[list[str]] = [] + current: list[str] = [] + for token in tokens: + if token in SEPARATORS: + if current: + segments.append(current) + current = [] + continue + current.append(token) + if current: + segments.append(current) + return segments + + +def _strip_prefixes(tokens: list[str]) -> list[str]: + index = 0 + while index < len(tokens) and (_ENV_ASSIGNMENT.match(tokens[index]) or tokens[index] in WRAPPERS): + index += 1 + return tokens[index:] + + +def direct_vitest_args(tokens: list[str]) -> list[str] | None: + """The arguments of a segment that invokes the vitest BINARY, or `None`. + + Covers `vitest …` (npm puts `node_modules/.bin` on PATH for a script body) and + `npx/npm exec/pnpm exec/yarn dlx vitest …`. Kept separate from the npm-script route below so the + `web/package.json` scan can use it without recursing through the script table it is building. + """ + tokens = _strip_prefixes(tokens) + if not tokens: + return None + if tokens[0] == "vitest": + return tokens[1:] + if tokens[0] in {"npx", "pnpm", "npm", "yarn"}: + index = 1 + if tokens[0] != "npx": + if index >= len(tokens) or tokens[index] not in {"exec", "dlx"}: + return None + index += 1 + while index < len(tokens) and tokens[index] in {"--", "-y", "--yes", "--no-install"}: + index += 1 + if index < len(tokens) and tokens[index] == "vitest": + return tokens[index + 1 :] + return None + + +def vitest_scripts() -> set[str]: + """The `web/package.json` script names whose body runs the vitest binary. + + This is the derivation that decides what `npm test` and `npm run X` mean. `"test": "vitest"` is + in; `"test:ui-e2e": "playwright test"` is not, and needs no exemption to be left out. + """ + scripts = json.loads((REPO_ROOT / PACKAGE_JSON).read_text(encoding="utf-8")).get("scripts", {}) + named: set[str] = set() + for name, body in scripts.items(): + if not isinstance(body, str): + continue + if any(direct_vitest_args(segment) is not None for segment in command_segments(body)): + named.add(str(name)) + return named + + +def suite_args(tokens: list[str], scripts: set[str]) -> list[str] | None: + """The arguments of a segment that runs the SPA suite, or `None` if it does not run it.""" + direct = direct_vitest_args(tokens) + if direct is not None: + return direct + tokens = _strip_prefixes(tokens) + if not tokens or tokens[0] not in {"npm", "pnpm", "yarn", "bun"}: + return None + index = 1 + if index < len(tokens) and tokens[index] in {"run", "run-script"}: + index += 1 + if index >= len(tokens) or tokens[index] not in scripts: + return None + return tokens[index + 1 :] + + +def suite_invocations(text: str, scripts: set[str] | None = None) -> list[tuple[int, str]]: + """Every command in `text` that runs the SPA vitest suite.""" + scripts = vitest_scripts() if scripts is None else scripts + found: list[tuple[int, str]] = [] + for number, command in logical_commands(text): + if any(suite_args(segment, scripts) is not None for segment in command_segments(command)): + found.append((number, command)) + return found + + +def narrowing_arguments(command: str, scripts: set[str] | None = None) -> list[str]: + """The arguments in `command` that restrict WHICH specs run, or `[]` when it runs all of them.""" + scripts = vitest_scripts() if scripts is None else scripts + found: list[str] = [] + for segment in command_segments(command): + args = suite_args(segment, scripts) + if args is None: + continue + index = 0 + while index < len(args): + argument = args[index] + index += 1 + if argument == "--": + continue + base = argument.split("=", 1)[0] + if base in NARROWING_FLAGS: + found.append(argument) + if "=" not in argument and index < len(args) and not args[index].startswith("-"): + found.append(args[index]) + index += 1 + continue + if base in VALUE_TAKING_FLAGS and "=" not in argument: + index += 1 # its value is not a filter + continue + if argument.startswith("-"): + continue + found.append(argument) + return found + + +def dockerfiles() -> list[str]: + """Tracked Dockerfiles, by repo-relative path, in every spelling docker recognises. + + `tracked_files._git_ls_files()` through the MODULE, never a `from … import` binding. The shared + proof in `test_guard_populations_derive_from_git.py` narrows the index by patching that module + attribute, and a name bound at import time is not reachable from there — the derivation would go + on returning every Dockerfile after git stopped tracking them, and the proof would say so. A + binding captured before the patch is the same shape that file warns about one block down. + """ + return sorted( + path + for path in tracked_files._git_ls_files() + if path.rpartition("/")[2].startswith("Dockerfile") or path.endswith(".Dockerfile") + ) + + +def workflow_files() -> list[Path]: + return tracked_paths(*WORKFLOWS) + + +def _read(rel: str) -> str: + return (REPO_ROOT / rel).read_text(encoding="utf-8") + + +def carries_spa_source(rel: str) -> bool: + """Does this Dockerfile copy the SPA source tree into the build? + + This is what makes a suite run there possible at all, and it is what separates + `docker/Dockerfile` from `docker/ci/Dockerfile` — which installs npm packages globally but never + copies `web/`. Derived from the `COPY` instructions rather than from a list of Dockerfile paths. + + Two shapes count. `COPY web/… ` names the tree; `COPY . …` carries it without naming it, and the + question is what the stage HAS, not what it says. No Dockerfile here uses the second form today — + it is matched anyway because the gate assertion SKIPS every Dockerfile this answers False for, so + a miss here drops an image out of the check silently rather than reddening. + + A `COPY --from=` is not a context copy: its source is another stage, so `--from=build .` + would not bring the repo in. Flags are dropped before the source argument is read. + """ + for _, command in logical_commands(_read(rel)): + tokens = command.split() + if not tokens or tokens[0].upper() != "COPY": + continue + arguments = [token for token in tokens[1:] if not token.startswith("--")] + if len(arguments) < 2: + continue + if any(token.startswith("--from=") for token in tokens[1:]): + continue + sources = arguments[:-1] + if any(source in {".", "./"} or re.match(r"^\.?/?web/", source) for source in sources): + return True + return False + + +def _jobs(doc: object) -> dict[str, dict]: + jobs = doc.get("jobs") if isinstance(doc, dict) else None + if not isinstance(jobs, dict): + return {} + return {str(key): value for key, value in jobs.items() if isinstance(value, dict)} + + +def _steps(job: dict) -> list[dict]: + steps = job.get("steps") + return [step for step in steps if isinstance(step, dict)] if isinstance(steps, list) else [] + + +def _run_bodies(job: dict) -> list[str]: + return [step["run"] for step in _steps(job) if isinstance(step.get("run"), str)] + + +def _needs(job: dict) -> list[str]: + needs = job.get("needs") + if isinstance(needs, str): + return [needs] + return [str(item) for item in needs] if isinstance(needs, list) else [] + + +def _transitive_needs(job_id: str, jobs: dict[str, dict]) -> set[str]: + seen: set[str] = set() + frontier = list(_needs(jobs.get(job_id, {}))) + while frontier: + current = frontier.pop() + if current in seen: + continue + seen.add(current) + frontier.extend(_needs(jobs.get(current, {}))) + return seen + + +def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]: + """`(workflow rel path, job id, Dockerfile rel path, that workflow's jobs)` per publishing job. + + The Dockerfile comes from the step's own `file:` input, defaulting to docker's own default + (`Dockerfile` at the context root) when the input is absent, so a step that stops naming one is + still resolved rather than dropped from the population. + """ + found: list[tuple[str, str, str, dict[str, dict]]] = [] + for path in workflow_files(): + rel = path.relative_to(REPO_ROOT).as_posix() + jobs = _jobs(yaml.safe_load(path.read_text(encoding="utf-8"))) + for job_id, job in jobs.items(): + for step in _steps(job): + uses = step.get("uses") + if not isinstance(uses, str) or _canonical_action(uses) != PUBLISH_ACTION: + continue + with_block = step.get("with") + named = with_block.get("file") if isinstance(with_block, dict) else None + dockerfile = str(named or "Dockerfile").strip() + found.append((rel, job_id, re.sub(r"^\./", "", dockerfile), jobs)) + return found + + +# ------------------------------------------------------------------------------------------------ +# The invariant +# ------------------------------------------------------------------------------------------------ + + +def test_no_DOCKERFILE_runs_the_spa_suite() -> None: + """The negative half. A gitless stage runs no vitest suite, filtered or not.""" + scripts = vitest_scripts() + faults: list[str] = [] + for rel in dockerfiles(): + for number, command in suite_invocations(_read(rel), scripts): + faults.append( + f"{rel}:{number} runs the SPA vitest suite, but its build context has no git and " + f"`node:22-bookworm-slim` ships no git binary, so the suite's git-dependent members " + f"cannot run there: `{command.strip()}`. Excluding them by name is the defect this " + f"guard exists to prevent — that list is a population nothing derives, it went " + f"stale in ersatztv#883, and the resulting red is unreachable on a PR, so it lands " + f"on `main` and on the release tag. The suite runs in `docker-build.yml`'s `test` " + f"job, which gates the image; delete this invocation rather than filtering it " + f"(ersatztv#887)." + ) + assert not faults, "\n".join(faults) + + +def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> None: + """The positive half, without which deleting `needs:` is silently green.""" + scripts = vitest_scripts() + known = set(dockerfiles()) + faults: list[str] = [] + for rel, job_id, dockerfile, jobs in publishing_jobs(): + if dockerfile not in known: + faults.append( + f"{rel}: job `{job_id}` builds `{dockerfile}`, which is not a tracked Dockerfile. " + f"The population here is derived from the git index, so this job's image is " + f"UNCHECKED — fix the path or widen `dockerfiles()`." + ) + continue + if not carries_spa_source(dockerfile): + continue + gates = _transitive_needs(job_id, jobs) + runners = sorted( + gate for gate in gates if any(suite_invocations(body, scripts) for body in _run_bodies(jobs.get(gate, {}))) + ) + if not runners: + faults.append( + f"{rel}: job `{job_id}` publishes an image built from `{dockerfile}`, which carries " + f"the SPA source, but none of the jobs it transitively `needs:` ({sorted(gates)}) " + f"runs the SPA suite. Since ersatztv#887 the image build deliberately does not run " + f"the suite itself, so this `needs:` edge is the ONLY thing standing between a red " + f"suite and a published image." + ) + assert not faults, "\n".join(faults) + + +def test_every_spa_suite_invocation_is_UNFILTERED() -> None: + """One list, and it is empty. A narrowed run is the hand-maintained population coming back. + + Driven off the YAML walk rather than the file text, so it reads exactly what the gate assertion + above reads. `test_the_yaml_walk_finds_every_suite_invocation_the_TEXT_does` is what stops the + walk from silently narrowing. + """ + scripts = vitest_scripts() + faults: list[str] = [] + for path in workflow_files(): + rel = path.relative_to(REPO_ROOT).as_posix() + jobs = _jobs(yaml.safe_load(path.read_text(encoding="utf-8"))) + for job_id, job in jobs.items(): + for body in _run_bodies(job): + for _, command in suite_invocations(body, scripts): + narrowing = narrowing_arguments(command, scripts) + if narrowing: + faults.append( + f"{rel}: job `{job_id}` runs the SPA suite but restricts which specs run " + f"({narrowing}): `{command.strip()}`. The gating run must be the WHOLE " + f"suite — a filtered one re-creates the hand-maintained population " + f"ersatztv#887 removed." + ) + assert not faults, "\n".join(faults) + + +def test_the_yaml_walk_finds_every_suite_invocation_the_TEXT_does() -> None: + """The walk and a plain text scan must agree per file, or the walk has a blind spot. + + Both assertions above read `jobs..steps[].run`. A step list the walk cannot reach makes a + suite run structurally invisible while the guard still reports a clean pass, and absence of + output is not evidence. The two shapes that would escape BOTH halves — a local composite action + and a `jobs..uses:` reusable workflow — are asserted not to exist AT ALL by + `test_workflow_persist_credentials.py`, repo-wide and for the same reason; they are not + re-asserted here, because two guards on one condition mask each other (ersatztv#685). + """ + scripts = vitest_scripts() + disagreements: list[str] = [] + for path in workflow_files(): + text = path.read_text(encoding="utf-8") + jobs = _jobs(yaml.safe_load(text)) + walked = sum(len(suite_invocations(body, scripts)) for job in jobs.values() for body in _run_bodies(job)) + scanned = len(suite_invocations(text, scripts)) + if walked != scanned: + disagreements.append( + f"{path.name}: the jobs..steps walk found {walked} SPA-suite invocation(s) but " + f"the text scan found {scanned}. The walk is what both assertions above run over, so " + f"the difference is suite runs this guard cannot see — widen `_run_bodies`." + ) + assert not disagreements, "\n".join(disagreements) + + +def test_no_run_BODY_builds_or_pushes_an_image() -> None: + """The publishing-job derivation keys on the action; assert nothing publishes around it. + + Both halves would agree on zero if an image were built by a raw `docker push` in a shell body, + so this reports the shape rather than leaving it uncovered. + """ + stray: list[str] = [] + for path in workflow_files(): + rel = path.relative_to(REPO_ROOT).as_posix() + jobs = _jobs(yaml.safe_load(path.read_text(encoding="utf-8"))) + for job_id, job in jobs.items(): + for body in _run_bodies(job): + for number, command in logical_commands(body): + for segment in command_segments(command): + head = _strip_prefixes(segment) + publishes = head[:3] == ["docker", "buildx", "build"] or ( + len(head) >= 2 and head[0] == "docker" and head[1] in {"push", "build"} + ) + if publishes: + stray.append(f"{rel}: job `{job_id}`, line {number} of a run body: `{command.strip()}`") + assert not stray, ( + f"image build/push outside `{PUBLISH_ACTION}`: {stray}. The publishing-job population above " + f"is derived from that action, so these jobs are NOT checked for the suite gate — extend " + f"`publishing_jobs()` before adding one." + ) + + +# ------------------------------------------------------------------------------------------------ +# Anti-vacuity and negative controls +# ------------------------------------------------------------------------------------------------ + + +def test_the_populations_are_NOT_empty() -> None: + """Every assertion above passes over an empty set, which is how a completeness guard dies.""" + assert dockerfiles(), "`git ls-files` reported no Dockerfile — the derivation is broken, not the repo." + assert workflow_files(), f"`git ls-files` reported no {WORKFLOW_DIR}/*.yml — the derivation is broken." + + scripts = vitest_scripts() + assert scripts, ( + f"no script in {PACKAGE_JSON} was read as running vitest, so every assertion in this file " + "measures an empty set: no command can be recognised as a suite run. `direct_vitest_args` " + "stopped matching the script body, or the scripts moved." + ) + + spa_carrying = [rel for rel in dockerfiles() if carries_spa_source(rel)] + assert spa_carrying, ( + f"none of {dockerfiles()} was read as carrying the SPA source, so " + "`test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite` skipped every " + "job and asserted nothing. `carries_spa_source` stopped matching the `COPY web/` it derives." + ) + + published = publishing_jobs() + assert published, ( + f"no job in {WORKFLOW_DIR} uses `{PUBLISH_ACTION}`, so the gate assertion ran over an empty " + "population. Either the repo stopped publishing images or `_canonical_action` stopped " + "recognising the action." + ) + + runs = [ + (path.name, number) + for path in workflow_files() + for number, _ in suite_invocations(path.read_text(encoding="utf-8"), scripts) + ] + assert runs, ( + "no workflow runs the SPA suite at all. The image gate then rests on a job that runs " + "nothing, and `test_every_spa_suite_invocation_is_UNFILTERED` asserts over an empty set." + ) + + +def test_the_SCRIPT_derivation_separates_vitest_from_playwright() -> None: + """Pinned in both directions: this table is what every other assertion here keys on. + + `test` in and `test:ui-e2e` out is the whole reason there is no exemption list. If the + derivation drifted to include `test:ui-e2e`, the guard would demand an unfiltered gate on a + Playwright run; if it drifted to exclude `test`, nothing in the repo would be recognised as a + suite run and every assertion above would pass vacuously. + """ + scripts = vitest_scripts() + assert "test" in scripts, scripts + assert "test:ui-e2e" not in scripts, scripts + assert "build" not in scripts and "lint" not in scripts, scripts + + +def test_the_detector_recognises_the_SPELLINGS_it_claims_to() -> None: + """Negative control on the predicates, driven directly rather than through the tree. + + The mutation proof drives the real Dockerfile; this covers the spellings the tree does not + contain, so a detector that quietly stopped matching one is caught without waiting for someone + to write it. + """ + scripts = {"test", "test:unit"} + for command in ( + "npm test -- --run", + "npm run test", + "npm run test:unit", + "npx vitest run", + "npm exec vitest run", + "vitest run", + "cd web && npm test -- --run", + "npm run lint && npm test -- --run && npm run build", + "RUN npm test -- --run", + "env CI=1 npm test -- --run", + # The one-line YAML step form, which roughly half this repo's `run:` steps use. + "run: npm test -- --run", + "- run: npm test -- --run", + ): + assert suite_invocations(command, scripts), command + + for command in ( + "dotnet test --configuration Release", + "npm ci", + "npm run build", + "npm run lint", + "npm run typecheck", + "npm run test:ui-e2e", # not in `scripts`, i.e. not a vitest script + "playwright test", + # Prose ABOUT the suite is not a run of it. The first version of this guard reported both of + # these, which is a false red in exactly the files that explain the rule. + "echo the vitest suite does not run here", + "echo skipping npm test in this stage", + # A non-RUN Dockerfile instruction does not execute during a build. + "CMD npm test -- --run", + "COPY vitest.config.ts ./", + ): + assert not suite_invocations(command, scripts), command + + assert narrowing_arguments("npm test -- --run", scripts) == [] + assert narrowing_arguments("npm test -- --run --reporter verbose", scripts) == [] + assert narrowing_arguments("npm test -- --run --exclude 'src/api/x.guard.test.ts'", scripts) == [ + "--exclude", + "src/api/x.guard.test.ts", + ] + assert narrowing_arguments("npm test -- --run --project=spa", scripts) == ["--project=spa"] + assert narrowing_arguments("npm test -- --run src/api", scripts) == ["src/api"] + assert narrowing_arguments("npm test -- --run --changed", scripts) == ["--changed"] + + +def test_a_COMMENT_is_inert_but_only_a_LEADING_hash_is() -> None: + """`docker/Dockerfile` names the rule it enforces; that must not trip the rule. + + The second half is the one that matters: a `#` mid-line does not make what precedes it a + comment, and treating it as one would let a real invocation hide behind a trailing note. + """ + scripts = {"test"} + assert not suite_invocations("# npm test -- --run\n", scripts) + assert not suite_invocations(" # the vitest suite is not run in this stage\n", scripts) + assert suite_invocations("RUN npm test -- --run # left over from before\n", scripts) + + +def test_the_continuation_JOIN_keeps_flags_with_their_command() -> None: + """A multi-line invocation is one command; its flags must not read as separate lines. + + This is the exact shape the removed exclusion pair had, so a guard that scanned raw lines would + see `npm test -- --run` as unfiltered and report it clean. + """ + scripts = {"test"} + multiline = "RUN npm test -- --run \\\n --exclude 'src/api/a.test.ts' \\\n --exclude 'b.test.ts'\n" + found = suite_invocations(multiline, scripts) + assert len(found) == 1, found + assert narrowing_arguments(found[0][1], scripts) == ["--exclude", "src/api/a.test.ts", "--exclude", "b.test.ts"] + + +def test_the_spa_carrying_predicate_separates_the_two_real_DOCKERFILES() -> None: + """Pinned both ways, because the gate assertion is SKIPPED for everything it answers False for. + + A predicate that drifted to False for `docker/Dockerfile` would make the positive half vacuous + while `test_the_populations_are_NOT_empty` still passed on some other file. + """ + assert carries_spa_source("docker/Dockerfile") + assert not carries_spa_source("docker/ci/Dockerfile") diff --git a/web/vite-plugins/trackedSourceFiles.realgit.test.ts b/web/vite-plugins/trackedSourceFiles.realgit.test.ts index f911f11f9..9a81861be 100644 --- a/web/vite-plugins/trackedSourceFiles.realgit.test.ts +++ b/web/vite-plugins/trackedSourceFiles.realgit.test.ts @@ -13,10 +13,11 @@ import { type GitRunner, resolveTrackedSourceFiles } from './trackedSourceFiles' * plugin's refusal. Reading the two exclusions as checkout-versus-binary is what broke the Docker * stage once already; see `docker/Dockerfile`, which retracts that framing. * - * `docker/Dockerfile`'s web-build stage runs on `node:22-bookworm-slim`, which ships no git at all, - * so this file is excluded there alongside the guard. Keeping them apart means - * that exclusion costs two files rather than the thirteen injected-runner tests in - * `trackedSourceFiles.test.ts`, which run fine with no git and no repository. + * Keeping these apart from `trackedSourceFiles.test.ts` confines the git prerequisite to one file: + * its thirteen injected-runner siblings run with no git and no repository. The suite as a whole is + * therefore run only where git is present. `docker/Dockerfile`'s web-build stage is not such a + * place — `node:22-bookworm-slim` ships no git at all — and since ersatztv#887 it does not run the + * suite there rather than naming the files that cannot run, which was a list nothing derived. * * Do NOT make these skip when git is missing. A check that answers "could not tell" by passing is a * permanent no-op, which is the whole failure mode this plugin exists to avoid. -- 2.47.3 From a4df8f7958f601fdd376fa0640b44f2e71d4a5f0 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 11:09:17 +0200 Subject: [PATCH 02/16] =?UTF-8?q?fix(887):=20the=20gate=20must=20be=20REAL?= =?UTF-8?q?,=20not=20merely=20present=20=E2=80=94=20two=20cold=20reviews'?= =?UTF-8?q?=20findings?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both independent reviews (Codex GPT-5.6 cross-family, and a cold Opus agent in an isolated worktree) returned BLOCKED. Both independently confirmed the CI path itself is sound — neither found a route that publishes an image on which the suite never ran — so every finding is about the guard's reach, plus one factual error in the prose. THE STRUCTURAL ONE. The guard asserted a `needs:` edge EXISTS, never that it is load bearing. Since this change deletes the in-image run, that edge is the only remaining layer, so `continue-on-error: true`, `if: false`, a job-level `if:`, `npm test … || true`, a pipe into `tee`, and `set +e` each certified a publish over a red suite with every assertion green. `test_the_gating_suite_run_is_NOT_ADVISORY` closes all six. A filter written into `web/package.json`'s script body was invisible at the call site: `"test": "vitest --exclude x"` with a workflow saying `npm test -- --run` is a filtered gating run reading as clean — the removed defect, one level down. `vitest_scripts()` now derives each script's own narrowing arguments and `suite_args` prepends them. PARSER REACH, every case measured rather than argued. `shlex.split` yields `lint&&npm` as one token, so unspaced `&&` and `;` re-adds were invisible; `shlex` in punctuation_chars mode splits them. Added: `sh -c` payload expansion, `npm --prefix`/`npx -p` flag skipping, `xargs`, heredoc bodies as DATA (a `cat > f <<'EOF' … npm test … EOF` block counted as a real run), `ADD`/JSON-form/no-trailing-slash `COPY` in `carries_spa_source`, and redirections no longer read as spec filters. `--root` and `--config` moved to the narrowing set: both change which specs vitest collects. A FACTUAL ERROR, in five places including the mutation `expect`: "the build context is `web/` + `design-system/`, so there is no `.git`". The context is the repository root (`context: .`) and `.dockerignore` does not exclude `.git`. The true statement is about the STAGE, which copies only those two directories. The conclusion survives — bookworm slim has no git binary either — but a reader who checked would have found `.git` in the context and concluded the note was stale. ONE FINDING WAS MINE, from the mutant battery rather than from either review, and it is the reason the battery exists: `failure_suppressions` tokenised the whole multi-line `run:` body at once. A newline is not a shell separator, so a realistic two-line step — the `ci-step-ran.sh` marker line, then the suite — merged into ONE segment whose head was the marker script, and three suppression mutants passed while my single-line unit test was green. It now works per logical line, and the regression test uses the two-line shape. 17 mutants, 0 missed, each caught by the intended assertion; baseline green. The `docs/guard-inventory.md` residual list is rewritten as MEASURED reach — the previous one was wrong rather than merely short, which cold review rightly called worse than silence. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docker/Dockerfile | 12 +- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 16 +- docs/guard-inventory.md | 4 +- scripts/tests/mutation_manifest.py | 2 +- ...est_image_build_delegates_the_spa_suite.py | 791 ++++++++++++++---- .../trackedSourceFiles.realgit.test.ts | 4 +- 7 files changed, 639 insertions(+), 192 deletions(-) diff --git a/docker/Dockerfile b/docker/Dockerfile index abe2d1a78..729340c3a 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -10,14 +10,14 @@ COPY design-system/. ./design-system/ COPY web/. ./web/ WORKDIR /source/web # THE VITEST SUITE IS NOT RUN IN THIS STAGE, AND MUST NOT BE RE-ADDED (ersatztv#887). -# This stage is gitless in both senses at once: the build context is `web/` + `design-system/`, so -# there is no `.git`, and `node:22-bookworm-slim` ships no git binary (`command -v git` -> not -# found). Members of the SPA suite need one or the other — `web/src/api/*.guard.test.ts` derive +# This stage is gitless in both senses at once. It copies only `web/` and `design-system/`, so it +# holds no `.git` — a property of the STAGE, not of the context, which is the repository root +# (`context: .`) and whose `.dockerignore` does not exclude `.git`: the directory is there to be +# copied and simply is not. And `node:22-bookworm-slim` ships no git binary (`command -v git` -> not +# found), which is why a `COPY .git` would not help either. Members of the SPA suite need one or the +# other — `web/src/api/*.guard.test.ts` derive # their file population from `git ls-files` and refuse to fall back to a directory walk # (`testing.guard-derives-population-from-source`), and `web/vite-plugins/*.realgit.test.ts` builds -# its own temp repository. So a `COPY .git` alone would not make the suite runnable here; the -# binary would still be missing. -# # Running the suite here anyway therefore costs a hand-maintained list of the members that cannot # run — and that list is a population nothing derives. It went stale the first time a guard was # added without updating it, and because `Build & push image (amd64)` is `if: github.event_name != diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 3b8bb9f3e..5311975a0 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present: the gating step must not be advisory (`continue-on-error`, `\|\| true`, a pipe, `set +e`), its job must carry no job-level `if:`, its step `if:` must be one this guard knows, and the run must be unfiltered at the CALL SITE and inside the `web/package.json` script body. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index ff07195b4..d70656cfc 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,14 +5,18 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' -signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' -mechanics: 'Guard derives three populations — tracked Dockerfiles, tracked workflows, and the vitest script names read out of `web/package.json` — then asserts no Dockerfile runs the suite, every publishing job whose Dockerfile carries the SPA source transitively `needs:` a job that does, and every such run is unfiltered. Publishing jobs come from the `docker/build-push-action` step; the Dockerfile each builds comes from that step''s `file:` input.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present: the gating step must not be advisory (`continue-on-error`, `|| true`, a pipe, `set +e`), its job must carry no job-level `if:`, its step `if:` must be one this guard knows, and the run must be unfiltered at the CALL SITE and inside the `web/package.json` script body. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' +mechanics: 'Guard derives three populations — tracked Dockerfiles, tracked workflows, and the vitest script names read out of `web/package.json` — then asserts no Dockerfile runs the suite, every publishing job whose Dockerfile carries the SPA source transitively `needs:` a job that does, every such run is unfiltered at both the call site and in the script body, and the gating step is not advisory. Publishing jobs come from the `docker/build-push-action` step; the Dockerfile each builds comes from that step''s `file:` input.' --- -The web-build stage is gitless twice over: its build context is `web/` + `design-system/`, so there -is no `.git`, and `node:22-bookworm-slim` ships no git binary. Members of the SPA suite need one or -the other — the `web/src/api/*.guard.test.ts` guards derive their file population from `git ls-files` +The web-build stage is gitless twice over. It copies only `web/` and `design-system/`, so it holds +no `.git`. That is a property of the STAGE, not of the build context — the context is the repository +root (`context: .`) and `.dockerignore` does not exclude `.git`, so the directory is available to be +copied and simply is not; stating it the other way invites a reader to check, find `.git` in the +context, and conclude the whole note is stale. And `node:22-bookworm-slim` ships no git binary. Members of the SPA +suite need one or the other — the `web/src/api/*.guard.test.ts` guards derive their file population +from `git ls-files` and refuse to fall back to a directory walk (`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds a real temporary repository. Because the two prerequisites OVERLAP rather than divide, adding a `COPY .git` fixes neither: the binary is still diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index 2fefe5284..1181ab736 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | `.gitea/workflows/*.y*ml` + every tracked `Dockerfile*`/`*.Dockerfile`, plus the vitest SCRIPT NAMES read out of `web/package.json` | **yes, in both directions** — no Dockerfile in the population may run the SPA suite, and every image-publishing job whose Dockerfile carries the SPA source must transitively `needs:` a job that runs it unfiltered | ersatztv#887, index-derived from the start. THREE populations rather than one, because a single one leaves a hole the other two close: the Dockerfile set alone cannot see a deleted `needs:` edge, and the job set alone cannot see a suite run added to a gitless stage. The publishing jobs are derived from the `docker/build-push-action` step and the Dockerfile each builds is read from that step's own `file:` input, so `ci-image.yml` — which publishes from `docker/ci/Dockerfile`, a Dockerfile carrying no SPA source — is out of scope by DERIVATION and holds no exemption entry that could outlive its reason. Which npm scripts count as the suite is derived too, from `web/package.json`: `test` is in because `"test": "vitest"` is, and the sibling `test:ui-e2e` is out because it runs Playwright — a hardcoded name would go quiet on a rename, and a `test:`-prefix guess would demand an unfiltered gate on a browser suite. RESIDUALS, stated: a `RUN` that calls a SCRIPT which runs the suite is invisible (the parser reads commands, not what they call — no such script exists today, and the miss is a missed DETECTION, not a false green about the gate, which is derived separately); only `RUN` is read in a Dockerfile, since nothing else executes during a build; and a line whose first non-space character is `#` is inert, which is what lets `docker/Dockerfile` name the rule it is subject to — only a LEADING `#`, so an invocation with a trailing note is still seen | +| `test_image_build_delegates_the_spa_suite.py` | `.gitea/workflows/*.y*ml` + every tracked `Dockerfile*`/`*.Dockerfile`, plus the vitest SCRIPT NAMES read out of `web/package.json` | **yes, in both directions** — no Dockerfile in the population may run the SPA suite, and every image-publishing job whose Dockerfile carries the SPA source must transitively `needs:` a job that runs it unfiltered | ersatztv#887, index-derived from the start. THREE populations rather than one, because a single one leaves a hole the other two close: the Dockerfile set alone cannot see a deleted `needs:` edge, and the job set alone cannot see a suite run added to a gitless stage. The publishing jobs are derived from the `docker/build-push-action` step and the Dockerfile each builds is read from that step's own `file:` input, so `ci-image.yml` — which publishes from `docker/ci/Dockerfile`, a Dockerfile carrying no SPA source — is out of scope by DERIVATION and holds no exemption entry that could outlive its reason. Which npm scripts count as the suite is derived too, from `web/package.json`: `test` is in because `"test": "vitest"` is, and the sibling `test:ui-e2e` is out because it runs Playwright — a hardcoded name would go quiet on a rename, and a `test:`-prefix guess would demand an unfiltered gate on a browser suite. A THIRD assertion exists because the first two together still certify a publish on which the suite never ran: the gating step must not be advisory. Two independent cold reviews each demonstrated that gap by execution rather than argument, and between them produced twelve mutants the guard passed on arrival — `continue-on-error: true`, `if: false`, a job-level `if:`, an or-true suffix on the suite command, a pipeline into `tee`, `set +e`, `--root src/api`, `--config other.ts`, an unspaced `lint&&npm test`, a `;`-separated re-add, `sh -c "npm test …"`, and `npm --prefix web test`. All twelve now redden; each was re-measured after the fix. THE REACH IS MEASURED, NOT ASSERTED, and the residuals below are what survived that measurement rather than what seemed likely. (1) A `RUN` or step that calls a SCRIPT which runs the suite is invisible — the parser reads commands, not what they call, and `sh -c` is expanded only because its argument IS a command line; no such script exists today, and the miss is a missed DETECTION on the negative assertion, never a false green on the positive one, which is derived from the job graph instead. (2) Only `RUN` is read in a Dockerfile, since nothing else executes during a build. (3) A line whose FIRST non-space character is `#` is inert — that is what lets `docker/Dockerfile` name the rule it is subject to — and only a LEADING `#`, so an invocation with a trailing note is still seen. (4) The step `if:` is checked against a HAND-WRITTEN set of one condition, which is a deliberate review trigger rather than a derivation: a legitimate change reddens and its author states the reason. (5) `test_the_SCRIPT_derivation_separates_vitest_from_playwright` pins the two script NAMES, so renaming the `test` script is a false red — measured, and the derivation itself adapts correctly; the pin is the only thing that catches a wrong-but-non-empty script table, which the anti-vacuity check cannot. (6) A new value-taking vitest flag is reported as a spec filter, a false red fixed by naming it | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | @@ -252,7 +252,7 @@ recorded as unexamined rather than as cleared. | `scripts/tests/test_guard_inventory.py` | the `script-tests` job, on an unclassified guard or a stale proof ref | GUARD | MUTATION | `test_guard_inventory.py::test_the_inventory_covers_exactly_the_guards_that_exist` | | `scripts/tests/test_guard_populations_derive_from_git.py` | the `script-tests` job, on a guard whose file population admits a file git does not track | GUARD | MUTATION | `test_guard_populations_derive_from_git.py::test_no_derivation_admits_an_untracked_file` | | `scripts/tests/test_hook_fire_log.py` | the `script-tests` job, on a hook that stops reporting that it fired, or whose reporting changes what the harness sees | GUARD | MUTATION | `test_hook_fire_log.py::test_a_hook_that_LOSES_its_instrumentation_is_DETECTED` | -| `scripts/tests/test_image_build_delegates_the_spa_suite.py` | the `script-tests` job, on a Dockerfile that runs the SPA vitest suite, on an image-publishing job that stops being gated on a job that runs it, or on a gating run that is narrowed by a spec filter | GUARD | MUTATION | `test_image_build_delegates_the_spa_suite.py::test_no_DOCKERFILE_runs_the_spa_suite` | +| `scripts/tests/test_image_build_delegates_the_spa_suite.py` | the `script-tests` job, on a Dockerfile that runs the SPA vitest suite, on an image-publishing job that stops being gated on a job that runs it, on a gating run narrowed by a spec filter at the call site or in the `web/package.json` script body, or on a gating run made ADVISORY — `continue-on-error`, a trailing or-true, a pipeline, `set +e`, a job-level `if:`, or an unrecognised step `if:` | GUARD | MUTATION | `test_image_build_delegates_the_spa_suite.py::test_no_DOCKERFILE_runs_the_spa_suite` | | `scripts/tests/test_jq_preflight.py` | the `script-tests` job | PROOF | NONE | — | | `scripts/tests/test_mcp_smoke.py` | the `script-tests` job, when the MCP smoke checker accepts a server it should refuse — an impostor identity, a reply carrying only an id, a missing expected tool, or an `initialize` pre-answered before the request (the `tools/list` twin is held by two mechanisms jointly and is covered behaviourally, not by the declared clause) | GUARD | MUTATION | `test_mcp_smoke.py::test_MUTATION_a_PRE_ANSWERED_id_is_refused_because_the_request_ids_are_UNGUESSABLE` | | `scripts/tests/test_merge_consent_base_change.py` | the `script-tests` job | PROOF | NONE | — | diff --git a/scripts/tests/mutation_manifest.py b/scripts/tests/mutation_manifest.py index e6ff8f6d5..642cccfb7 100644 --- a/scripts/tests/mutation_manifest.py +++ b/scripts/tests/mutation_manifest.py @@ -410,7 +410,7 @@ MUTATIONS: tuple[Mutation, ...] = ( replacement="RUN npm run lint && npm run typecheck && npm test -- --run && npm run build", proof="test_image_build_delegates_the_spa_suite.py::test_no_DOCKERFILE_runs_the_spa_suite", granularity=CLAUSE, - expect="runs the SPA vitest suite, but its build context has no git", + expect="runs the SPA vitest suite in a stage that has neither a git CHECKOUT nor the git BINARY", why="The target is not the guard, per the checker-guard exception: the guard IS a test, so " "disarming it makes it absent rather than red, and the mutation goes into the guarded " "ARTIFACT. The clause is the web-build stage's whole command line, and the replacement is " diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index 37735732a..cca679e85 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -1,9 +1,12 @@ """The SPA vitest suite never runs inside an image build, and the image is gated on a job that does -run it, unfiltered (ersatztv#887). +run it, unfiltered and non-advisory (ersatztv#887). -WHAT THIS IS PROTECTING. `docker/Dockerfile`'s `web-build` stage is gitless twice over: its build -context is `web/` + `design-system/`, so there is no `.git`, and `node:22-bookworm-slim` ships no -git binary. Members of the SPA suite need one or the other — the `web/src/api/*.guard.test.ts` +WHAT THIS IS PROTECTING. `docker/Dockerfile`'s `web-build` stage is gitless twice over. It copies +only `web/` and `design-system/`, so it holds no `.git` — note this is a property of the STAGE, not +of the build context, which is the repository root (`context: .`) and whose `.dockerignore` does not +exclude `.git`; the directory is available to be copied and simply is not. And `node:22-bookworm-slim` +ships no git binary, which is why adding a `COPY .git` would still not help. Members of the SPA +suite need one or the other — the `web/src/api/*.guard.test.ts` guards derive their file population from `git ls-files` and refuse to fall back to a directory walk (`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds a real temporary repository. Running the suite there anyway therefore requires a list of the members @@ -17,15 +20,20 @@ The fix removed the list rather than extending it. This guard is what keeps it r holds up the safety argument that licensed removing it, because that argument is a claim about two OTHER files and would otherwise rot silently in a comment. -THE INVARIANT, in the two halves it actually has: +THE INVARIANT, in the three parts it actually has. The first two are the obvious ones; the third +came out of cold review, which pointed out that the first two together still certify a publish on +which the suite never ran: * NEGATIVE — no Dockerfile runs the suite. Not "no `--exclude` appears": an unfiltered suite run in a gitless stage fails just as hard, and a filtered one is the trap itself. The whole class is excluded rather than a spelling of it. * POSITIVE — every job that publishes an image built from a Dockerfile CARRYING THE SPA SOURCE is - transitively gated on a job that runs the suite, and that run is unfiltered. Without this half - the negative half is a hole: deleting the suite run from the Dockerfile and deleting - `needs: [test, ...]` from `build` would both be green. + transitively gated on a job that runs the suite. Without this the negative part is a hole: + deleting the suite run from the Dockerfile and deleting `needs: [test, ...]` would both be green. + * EFFECTIVE — that run is unfiltered AT BOTH LEVELS and is not advisory. A `needs:` edge onto a + job whose suite step carries `continue-on-error: true` blocks nothing, and a filter written into + `web/package.json`'s script body is invisible at the invocation site. Either one reproduces the + exact defect being removed while every other assertion here stays green. POPULATIONS, all derived — never a filesystem walk and never a hand-written list (`testing.guard-derives-population-from-source`, ersatztv#806): @@ -33,11 +41,11 @@ POPULATIONS, all derived — never a filesystem walk and never a hand-written li * Dockerfiles and workflows: the git index, via `tracked_files`. Both the `*.yml` and `*.yaml` workflow spellings, for the reason `test_workflow_persist_credentials.py` states — Gitea accepts either, so a `.yaml` workflow would otherwise be structurally invisible. - * WHICH npm SCRIPTS ARE THE SUITE: read out of `web/package.json`, by finding the scripts whose - body invokes the vitest binary. `npm test` is a suite run because `"test": "vitest"` is; the - sibling `"test:ui-e2e": "playwright test"` is not, and no exemption says so. A guess in either - direction here is a live defect — treating `test:ui-e2e` as the suite would demand a gate on a - Playwright run, and hardcoding the name `test` would go quiet the day the script is renamed. + * WHICH npm SCRIPTS ARE THE SUITE, *and what filters their bodies already carry*: read out of + `web/package.json`. `npm test` is a suite run because `"test": "vitest"` is; the sibling + `"test:ui-e2e": "playwright test"` is not, and no exemption says so. A guess in either direction + is a live defect — treating `test:ui-e2e` as the suite would demand a gate on a Playwright run, + and hardcoding the name `test` would go quiet the day the script is renamed. * Image-publishing jobs: every job with a `docker/build-push-action` step, and the Dockerfile it builds is read out of that step's `file:` input rather than hardcoded here. That is what keeps a renamed Dockerfile red instead of silently uncovered, and it is why `ci-image.yml` — which @@ -45,11 +53,12 @@ POPULATIONS, all derived — never a filesystem walk and never a hand-written li source — is correctly out of scope WITHOUT an exemption entry. An exemption outlives its reason; a derivation does not. -WHY A SHELL-POSITION PARSER RATHER THAN A REGEX. The first version matched `vitest` anywhere in a -command and reported `echo the vitest suite does not run here` as an invocation. A predicate that -cannot tell a command from prose about a command produces false reds in exactly the files that -explain the rule, and those get silenced. Commands are split on shell separators and only the HEAD -of each segment is consulted, so a runner named in an argument or in prose is not a run. +WHY A SHELL-POSITION PARSER RATHER THAN A REGEX. A regex over command text reported +`echo the vitest suite does not run here` as an invocation. A predicate that cannot tell a command +from prose about a command produces false reds in exactly the files that explain the rule, and those +get silenced. Commands are tokenised with `shlex` in `punctuation_chars` mode — which is what splits +the unspaced `lint&&npm test`, where plain `shlex.split` yields the single token `lint&&npm` — then +split on shell separators, and only the HEAD of each segment is consulted. WHAT IS NOT COVERED, stated rather than implied: @@ -61,9 +70,11 @@ WHAT IS NOT COVERED, stated rather than implied: * In a Dockerfile only `RUN` executes at build time, so only `RUN` is read. A `CMD`/`ENTRYPOINT` that runs the suite would run in the shipped container, which is a different (and absent) problem; it is out of scope rather than overlooked. - * A `RUN` that invokes a SCRIPT which in turn runs the suite is invisible to this — the parser - reads commands, not what they call. No such script exists today; the failure mode is a missed - detection, not a false green about the gate, because the positive half is derived independently. + * A `RUN` or a step that invokes a SCRIPT which in turn runs the suite is invisible — the parser + reads commands, not what they call, and `sh -c` is expanded only because its argument IS a + command line. No such script exists today. The failure mode is a missed DETECTION on the + negative part; it cannot produce a false green on the positive part, which is derived from the + job graph rather than from command text. * An image published by something other than `docker/build-push-action` — a raw `docker push` in a `run:` body — is outside the publishing-job derivation. `test_no_run_BODY_builds_or_pushes_an_image` asserts that shape does not exist rather than leaving it uncovered. @@ -77,6 +88,7 @@ from __future__ import annotations import json import re import shlex +from collections.abc import Mapping from pathlib import Path import yaml @@ -94,12 +106,17 @@ PACKAGE_JSON = "web/package.json" # a host is not required to contain a dot, so no hostname heuristic is safe. PUBLISH_ACTION = "docker/build-push-action" -# Shell tokens that end one command and start another. `shlex.split` returns each of these as its -# own token, which is what makes segmenting possible without a shell grammar. -SEPARATORS = frozenset({"&&", "||", ";", "|", "&"}) +# Shell tokens that end one command and start another. +SEPARATORS = frozenset({"&&", "||", ";", "|", "&", "(", ")"}) -# Prefixes that wrap a command without being one. Skipped so `env FOO=1 npm test` is still a run. -WRAPPERS = frozenset({"env", "sudo", "time", "exec", "command", "nice", "ionice"}) +# Prefixes that wrap a command without being one. Skipped so `env FOO=1 npm test` and +# `xargs npm test` are still runs. +WRAPPERS = frozenset({"env", "sudo", "time", "exec", "command", "nice", "ionice", "xargs", "stdbuf"}) + +# Shells whose `-c` argument is itself a command line. `sh -c 'npm test -- --run'` runs the suite, +# and the head of that segment is `sh`, so without expanding the payload the segment reads as +# not-a-run. Measured False before this existed, not reasoned about. +SHELLS = frozenset({"sh", "bash", "zsh", "dash", "ash", "ksh"}) # Dockerfile instructions. Only `RUN` executes during a build; the rest are read as not-a-command so # a `COPY` whose argument happens to be a runner name is not mistaken for one. @@ -126,10 +143,29 @@ DOCKERFILE_INSTRUCTIONS = frozenset( } ) +# Instructions that bring files from the build context into the image. +COPY_INSTRUCTIONS = frozenset({"ADD", "COPY"}) + # Flags that narrow WHICH specs run. Every one of these reintroduces the defect: the set of specs # that ran becomes a thing someone maintains by hand, and nothing compares it to the set that exists. NARROWING_FLAGS = frozenset( - {"--exclude", "--include", "--dir", "--project", "--shard", "--changed", "--related", "--testNamePattern", "-t"} + { + "--exclude", + "--include", + "--dir", + "--project", + "--shard", + "--changed", + "--related", + "--testNamePattern", + "-t", + # `--root` moves the project root and `--config` can point at a config whose `include` + # differs, so both change WHICH specs are collected. Cold review found them classified as + # benign value-taking flags, which made `--root src/api` report as an unfiltered run — a + # narrowed gate reading clean, which is the defect itself. + "--root", + "--config", + } ) # Flags whose NEXT token is a value rather than a spec filter, so `--reporter verbose` is not read as @@ -138,8 +174,6 @@ VALUE_TAKING_FLAGS = frozenset( { "--reporter", "--outputFile", - "--config", - "--root", "--environment", "--pool", "--maxWorkers", @@ -152,8 +186,36 @@ VALUE_TAKING_FLAGS = frozenset( } ) +# npm/npx flags taking a SEPARATE value token. A leading flag has to be consumed before the script +# name can be read: `npm --prefix web test` runs the suite, `npm ci --silent` does not, and the two +# are indistinguishable until the flags are gone. +RUNNER_VALUE_FLAGS = frozenset( + {"--prefix", "-C", "--workspace", "-w", "--loglevel", "--userconfig", "--package", "-p", "--call"} +) + +# The `if:` conditions a suite step may carry. A HAND-WRITTEN MIRROR OF ONE STRING, and deliberately +# so: this step is the only thing standing between a red suite and a published image, so a change to +# when it runs must be a decision someone states, not a diff nobody reads. A third condition here +# reddens rather than silently narrowing the gate. Both current terms are documented — `docs_only` +# (ersatztv#416) cannot ship an image at all, and the revalidate skip (ersatztv#420) fires only on a +# tree byte-identical to an already-green head. +KNOWN_GATE_CONDITIONS = frozenset( + {"steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'"} +) + _ENV_ASSIGNMENT = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*=") +# Redirection operators, which `punctuation_chars` splits out as their own tokens. Neither they nor +# their targets are spec filters: cold review measured `npm test -- --run > test.log` reporting +# `['>', 'test.log']` as a narrowing, which is a false red on an ordinary log capture. +_REDIRECTION = re.compile(r"^\d*(?:>>|>&|&>|<<|>|<)$") + +# `< f <<'EOF' … npm test … EOF` block counts as a suite run: cold +# review showed that satisfies both the anti-vacuity check and the gate assertion while the suite +# never executes. +_HEREDOC = re.compile(r"<<-?\s*(['\"]?)([A-Za-z_][A-Za-z0-9_]*)\1") + def _canonical_action(uses: str) -> str: """The last two path segments of a `uses:` value, without its `@version`. @@ -170,19 +232,43 @@ def _canonical_action(uses: str) -> str: return "/".join(segments[-2:]) if len(segments) >= 2 else ref -def logical_commands(text: str) -> list[tuple[int, str]]: - """`(1-based line number, command)` per command line: comments dropped, `\\` continuations joined. +def tokenize(command: str) -> list[str]: + """Shell tokens, with the separators split out even when unspaced. - Both normalisations are load-bearing. Dropping leading-`#` lines is what lets a Dockerfile - explain the rule it enforces without tripping it. Joining backslash continuations is what makes - a multi-line invocation ONE command with its flags attached — the shape the removed `--exclude` - pair had, and the shape a re-added one would have. Scanning raw lines instead would read - `npm test -- --run` as unfiltered and report it clean. + `shlex.split` yields `lint&&npm` as ONE token for `npm run lint&&npm test`, which hides the + second command entirely. `punctuation_chars` mode is the stdlib answer and splits it correctly. + """ + try: + lexer = shlex.shlex(command, posix=True, punctuation_chars=True) + lexer.whitespace_split = True + return list(lexer) + except ValueError: + # An unbalanced quote is not a command anyone can reason about. The raw split keeps it + # VISIBLE to the detectors rather than silently dropping it. + return command.split() + + +def logical_commands(text: str) -> list[tuple[int, str]]: + """`(1-based line number, command)` per command line, with the three normalisations that matter. + + Leading-`#` lines are dropped, so a Dockerfile can explain the rule it enforces without tripping + it. Backslash continuations are joined, so a multi-line invocation is ONE command with its flags + attached — the shape the removed `--exclude` pair had; scanning raw lines would read + `npm test -- --run` as unfiltered and report it clean. And HEREDOC BODIES ARE SKIPPED, because + the shell reads them as data: a `npm test` line inside one never executes, and counting it makes + the gate assertion pass on a job that ran no suite. """ commands: list[tuple[int, str]] = [] buffer = "" start = 0 + terminator: str | None = None for number, raw in enumerate(text.splitlines(), start=1): + if terminator is not None: + # `<<-` permits leading tabs on the terminator; plain `<<` does not, and accepting a + # stripped match either way only ends the skip EARLIER, never later. + if raw.strip() == terminator: + terminator = None + continue if raw.lstrip().startswith("#"): continue if not buffer: @@ -194,30 +280,25 @@ def logical_commands(text: str) -> list[tuple[int, str]]: buffer += stripped if buffer.strip(): commands.append((start, buffer.strip())) + opener = _HEREDOC.search(buffer) + if opener: + terminator = opener.group(2) buffer = "" if buffer.strip(): commands.append((start, buffer.strip())) return commands -def command_segments(command: str) -> list[list[str]]: +def command_segments(command: str, depth: int = 0) -> list[list[str]]: """The shell segments of one command, as token lists, or `[]` if nothing executes. A leading Dockerfile instruction is consumed: `RUN` yields its command, every other instruction - yields nothing, since only `RUN` runs during a build. + yields nothing. A leading YAML step key is consumed too — `- run: npm test -- --run` is a + command, and roughly half this repo's `run:` steps are written on one line like that. """ - try: - tokens = shlex.split(command) - except ValueError: - # An unbalanced quote is not a command anyone can reason about. Returning the raw split - # keeps it VISIBLE to the detectors rather than silently dropping it. - tokens = command.split() + tokens = tokenize(command) if not tokens: return [] - # A YAML step written on one line — `- run: npm test -- --run` — is a command, and roughly half - # the `run:` steps in this repo are written that way. Without this the raw-text cross-check below - # reads `run:` as the program and finds nothing, so a single-line invocation would be invisible - # to every text-based assertion while the block form was caught. if tokens[0] == "-": tokens = tokens[1:] if tokens and tokens[0] == "run:": @@ -241,22 +322,73 @@ def command_segments(command: str) -> list[list[str]]: current.append(token) if current: segments.append(current) - return segments + return _expand_shell_payloads(segments, depth) + + +def _expand_shell_payloads(segments: list[list[str]], depth: int) -> list[list[str]]: + """Add the segments of any `sh -c ''` payload alongside the segment carrying it. + + The wrapper segment is KEPT as well as expanded: nothing here decides `sh` is uninteresting, only + that its argument is also a command. Bounded at three levels — past anything a Dockerfile or a + `run:` body writes — and hitting the cap costs a missed detection, never a false green about the + gate, which is derived from the job graph instead. + """ + if depth >= 3: + return segments + expanded: list[list[str]] = [] + for segment in segments: + expanded.append(segment) + head = _strip_prefixes(segment) + if len(head) < 3 or head[0] not in SHELLS: + continue + for offset, token in enumerate(head[1:-1], start=1): + # `-c`, and the login/interactive spellings carrying it: `-lc`, `-euxc`, `-ec`. + if token.startswith("-") and not token.startswith("--") and "c" in token.lstrip("-"): + expanded.extend(command_segments(head[offset + 1], depth + 1)) + break + return expanded def _strip_prefixes(tokens: list[str]) -> list[str]: + """Drop leading environment assignments and command wrappers, plus any flags a wrapper takes.""" index = 0 - while index < len(tokens) and (_ENV_ASSIGNMENT.match(tokens[index]) or tokens[index] in WRAPPERS): - index += 1 + while index < len(tokens): + token = tokens[index] + if _ENV_ASSIGNMENT.match(token): + index += 1 + continue + if token in WRAPPERS: + index += 1 + while index < len(tokens) and tokens[index].startswith("-"): + index += 1 + continue + break return tokens[index:] +def _skip_runner_flags(tokens: list[str], index: int) -> int: + """Advance past leading `npm`/`npx` flags to the first positional argument. + + A flag in `RUNNER_VALUE_FLAGS` consumes the token after it unless written `--flag=value`. `--` is + consumed too: `npm exec -- vitest` is the documented spelling. + """ + while index < len(tokens): + word = tokens[index] + if word == "--": + index += 1 + continue + if not word.startswith("-"): + return index + index += 2 if (word.split("=", 1)[0] in RUNNER_VALUE_FLAGS and "=" not in word) else 1 + return index + + def direct_vitest_args(tokens: list[str]) -> list[str] | None: - """The arguments of a segment that invokes the vitest BINARY, or `None`. + """The arguments of a segment invoking the vitest BINARY, or `None`. Covers `vitest …` (npm puts `node_modules/.bin` on PATH for a script body) and - `npx/npm exec/pnpm exec/yarn dlx vitest …`. Kept separate from the npm-script route below so the - `web/package.json` scan can use it without recursing through the script table it is building. + `npx / npm exec / pnpm exec / yarn dlx vitest …`. Kept separate from the npm-script route below + so the `web/package.json` scan can use it without recursing through the table it is building. """ tokens = _strip_prefixes(tokens) if not tokens: @@ -266,49 +398,60 @@ def direct_vitest_args(tokens: list[str]) -> list[str] | None: if tokens[0] in {"npx", "pnpm", "npm", "yarn"}: index = 1 if tokens[0] != "npx": + index = _skip_runner_flags(tokens, index) if index >= len(tokens) or tokens[index] not in {"exec", "dlx"}: return None index += 1 - while index < len(tokens) and tokens[index] in {"--", "-y", "--yes", "--no-install"}: - index += 1 + index = _skip_runner_flags(tokens, index) if index < len(tokens) and tokens[index] == "vitest": return tokens[index + 1 :] return None -def vitest_scripts() -> set[str]: - """The `web/package.json` script names whose body runs the vitest binary. +def vitest_scripts() -> dict[str, list[str]]: + """`web/package.json` script name -> the narrowing arguments its BODY already carries. - This is the derivation that decides what `npm test` and `npm run X` mean. `"test": "vitest"` is - in; `"test:ui-e2e": "playwright test"` is not, and needs no exemption to be left out. + Two questions, one derivation, because they have one answer. Which scripts are the suite decides + what `npm test` means; what their bodies carry decides whether an invocation that looks + unfiltered at the call site actually is. `"test": "vitest --exclude x"` with a workflow saying + `npm test -- --run` is a filtered gating run that reads as clean from the workflow alone — the + exact defect being removed, one level down. Cold review found that hole; it is closed here + rather than noted. """ scripts = json.loads((REPO_ROOT / PACKAGE_JSON).read_text(encoding="utf-8")).get("scripts", {}) - named: set[str] = set() + named: dict[str, list[str]] = {} for name, body in scripts.items(): if not isinstance(body, str): continue - if any(direct_vitest_args(segment) is not None for segment in command_segments(body)): - named.add(str(name)) + for segment in command_segments(body): + args = direct_vitest_args(segment) + if args is not None: + named[str(name)] = _narrowing_in(args) + break return named -def suite_args(tokens: list[str], scripts: set[str]) -> list[str] | None: - """The arguments of a segment that runs the SPA suite, or `None` if it does not run it.""" +def suite_args(tokens: list[str], scripts: Mapping[str, list[str]]) -> list[str] | None: + """The arguments of a segment that runs the SPA suite, or `None` if it does not run it. + + A script's own body arguments come first, so a filter written there is reported by the same + assertion that reports one written at the call site. + """ direct = direct_vitest_args(tokens) if direct is not None: return direct tokens = _strip_prefixes(tokens) if not tokens or tokens[0] not in {"npm", "pnpm", "yarn", "bun"}: return None - index = 1 + index = _skip_runner_flags(tokens, 1) if index < len(tokens) and tokens[index] in {"run", "run-script"}: - index += 1 + index = _skip_runner_flags(tokens, index + 1) if index >= len(tokens) or tokens[index] not in scripts: return None - return tokens[index + 1 :] + return list(scripts[tokens[index]]) + tokens[index + 1 :] -def suite_invocations(text: str, scripts: set[str] | None = None) -> list[tuple[int, str]]: +def suite_invocations(text: str, scripts: Mapping[str, list[str]] | None = None) -> list[tuple[int, str]]: """Every command in `text` that runs the SPA vitest suite.""" scripts = vitest_scripts() if scripts is None else scripts found: list[tuple[int, str]] = [] @@ -318,33 +461,116 @@ def suite_invocations(text: str, scripts: set[str] | None = None) -> list[tuple[ return found -def narrowing_arguments(command: str, scripts: set[str] | None = None) -> list[str]: - """The arguments in `command` that restrict WHICH specs run, or `[]` when it runs all of them.""" +def _strip_redirections(args: list[str]) -> list[str]: + """Drop redirection operators, their targets, and any file descriptor written before them. + + `2>&1` tokenises as `['2', '>&', '1']`, so the descriptor and the target are bare tokens that + would otherwise both be reported as positional spec filters. + """ + kept: list[str] = [] + index = 0 + while index < len(args): + argument = args[index] + if _REDIRECTION.match(argument): + index += 2 # the operator and its target + continue + if ( + argument.isdigit() + and index + 1 < len(args) + and _REDIRECTION.match(args[index + 1]) + and not args[index + 1][:1].isdigit() + ): + index += 3 # descriptor, operator, target + continue + kept.append(argument) + index += 1 + return kept + + +def _narrowing_in(args: list[str]) -> list[str]: + """The subset of `args` that restricts WHICH specs run.""" + args = _strip_redirections(args) + found: list[str] = [] + index = 0 + while index < len(args): + argument = args[index] + index += 1 + if argument == "--": + continue + base = argument.split("=", 1)[0] + if base in NARROWING_FLAGS: + found.append(argument) + if "=" not in argument and index < len(args) and not args[index].startswith("-"): + found.append(args[index]) + index += 1 + continue + if base in VALUE_TAKING_FLAGS and "=" not in argument: + index += 1 # its value is not a filter + continue + if argument.startswith("-"): + continue + found.append(argument) + return found + + +def narrowing_arguments(command: str, scripts: Mapping[str, list[str]] | None = None) -> list[str]: + """Everything restricting which specs `command` runs — from the call site AND the script body.""" scripts = vitest_scripts() if scripts is None else scripts found: list[str] = [] for segment in command_segments(command): args = suite_args(segment, scripts) - if args is None: - continue - index = 0 - while index < len(args): - argument = args[index] - index += 1 - if argument == "--": + if args is not None: + found.extend(_narrowing_in(args)) + return found + + +def failure_suppressions(command: str, scripts: Mapping[str, list[str]], depth: int = 0) -> list[str]: + """The ways `command` stops a red suite from failing its step. + + A `needs:` edge onto a job whose suite command cannot fail blocks nothing, and there are three + spellings of that which every other assertion here reads as an ordinary run — cold review + measured all three green: `npm test … || true` swallows the status outright; `npm test | tee log` + reports `tee`'s status unless the body set `pipefail`; and `set +e` earlier in the body disarms + the step's own `set -e`. Reported by NAME rather than by a boolean so the message can say which. + """ + if depth >= 3: + return [] + found: list[str] = [] + # PER LOGICAL LINE, never over the whole body at once. A newline is not a shell separator, so + # tokenising a real two-line `run:` body — the `ci-step-ran.sh` marker followed by the suite + # command — merged both into ONE segment whose head was the marker script, and every `||`/`|` + # after the suite went unseen. Measured: three mutants passed that way while a single-line + # version of each was caught, which is exactly the shape of a test passing for the wrong reason. + for _, line in logical_commands(command): + tokens = tokenize(line) + if tokens and tokens[0] == "-": + tokens = tokens[1:] + if tokens and tokens[0] == "run:": + tokens = tokens[1:] + if tokens and tokens[0].isupper() and tokens[0] in DOCKERFILE_INSTRUCTIONS: + tokens = tokens[1:] if tokens[0] == "RUN" else [] + + current: list[str] = [] + for word in [*tokens, ";"]: + if word not in SEPARATORS: + current.append(word) continue - base = argument.split("=", 1)[0] - if base in NARROWING_FLAGS: - found.append(argument) - if "=" not in argument and index < len(args) and not args[index].startswith("-"): - found.append(args[index]) - index += 1 - continue - if base in VALUE_TAKING_FLAGS and "=" not in argument: - index += 1 # its value is not a filter - continue - if argument.startswith("-"): - continue - found.append(argument) + if current: + head = _strip_prefixes(current) + if head[:2] == ["set", "+e"]: + found.append("`set +e` disarms the step's own `set -e`") + if suite_args(current, scripts) is not None and word in {"||", "|"}: + found.append( + "a trailing or-true swallows the exit status" + if word == "||" + else "a pipeline reports the LAST command's status" + ) + if len(head) >= 3 and head[0] in SHELLS: + for offset, inner in enumerate(head[1:-1], start=1): + if inner.startswith("-") and not inner.startswith("--") and "c" in inner.lstrip("-"): + found.extend(failure_suppressions(head[offset + 1], scripts, depth + 1)) + break + current = [] return found @@ -354,8 +580,7 @@ def dockerfiles() -> list[str]: `tracked_files._git_ls_files()` through the MODULE, never a `from … import` binding. The shared proof in `test_guard_populations_derive_from_git.py` narrows the index by patching that module attribute, and a name bound at import time is not reachable from there — the derivation would go - on returning every Dockerfile after git stopped tracking them, and the proof would say so. A - binding captured before the patch is the same shape that file warns about one block down. + on returning every Dockerfile after git stopped tracking them, and the proof would say so. """ return sorted( path @@ -372,36 +597,51 @@ def _read(rel: str) -> str: return (REPO_ROOT / rel).read_text(encoding="utf-8") -def carries_spa_source(rel: str) -> bool: - """Does this Dockerfile copy the SPA source tree into the build? +def copies_spa_source(command: str) -> bool: + """Does this ONE instruction bring the SPA source tree into the image? - This is what makes a suite run there possible at all, and it is what separates - `docker/Dockerfile` from `docker/ci/Dockerfile` — which installs npm packages globally but never - copies `web/`. Derived from the `COPY` instructions rather than from a list of Dockerfile paths. - - Two shapes count. `COPY web/… ` names the tree; `COPY . …` carries it without naming it, and the - question is what the stage HAS, not what it says. No Dockerfile here uses the second form today — - it is matched anyway because the gate assertion SKIPS every Dockerfile this answers False for, so - a miss here drops an image out of the check silently rather than reddening. + FOUR spellings count, because the gate assertion SKIPS every Dockerfile answered False — a miss + here drops an image out of the check silently rather than reddening. `ADD` as well as `COPY`; + the JSON exec form `COPY ["web/", "/dest/"]`; a source without a trailing slash + (`COPY web /dest`); and `COPY . …`, which carries the tree without naming it. Cold review found + the first three answering False. A `COPY --from=` is not a context copy: its source is another stage, so `--from=build .` - would not bring the repo in. Flags are dropped before the source argument is read. + would not bring the repo in. Flags are dropped before the source arguments are read. """ - for _, command in logical_commands(_read(rel)): - tokens = command.split() - if not tokens or tokens[0].upper() != "COPY": - continue + tokens = command.split() + if not tokens or tokens[0].upper() not in COPY_INSTRUCTIONS: + return False + if any(token.startswith("--from=") for token in tokens[1:]): + return False + remainder = command[len(tokens[0]) :].strip() + if remainder.startswith("["): + try: + parsed = json.loads(remainder) + except ValueError: + return False + arguments = [str(item) for item in parsed] if isinstance(parsed, list) else [] + else: arguments = [token for token in tokens[1:] if not token.startswith("--")] - if len(arguments) < 2: - continue - if any(token.startswith("--from=") for token in tokens[1:]): - continue - sources = arguments[:-1] - if any(source in {".", "./"} or re.match(r"^\.?/?web/", source) for source in sources): + if len(arguments) < 2: + return False + for source in arguments[:-1]: + normalised = source.removeprefix("./").rstrip("/") + if normalised in {"", ".", "web"} or normalised.startswith("web/"): return True return False +def carries_spa_source(rel: str) -> bool: + """Does this Dockerfile bring the SPA source tree into the build? + + This is what makes a suite run there possible at all, and it separates `docker/Dockerfile` from + `docker/ci/Dockerfile`, which installs npm packages globally but never copies `web/`. Derived + from the copy instructions rather than from a list of Dockerfile paths. + """ + return any(copies_spa_source(command) for _, command in logical_commands(_read(rel))) + + def _jobs(doc: object) -> dict[str, dict]: jobs = doc.get("jobs") if isinstance(doc, dict) else None if not isinstance(jobs, dict): @@ -437,6 +677,21 @@ def _transitive_needs(job_id: str, jobs: dict[str, dict]) -> set[str]: return seen +def suite_steps() -> list[tuple[str, str, dict, dict]]: + """`(workflow rel path, job id, job, step)` for every workflow step that runs the SPA suite.""" + scripts = vitest_scripts() + found: list[tuple[str, str, dict, dict]] = [] + for path in workflow_files(): + rel = path.relative_to(REPO_ROOT).as_posix() + jobs = _jobs(yaml.safe_load(path.read_text(encoding="utf-8"))) + for job_id, job in jobs.items(): + for step in _steps(job): + body = step.get("run") + if isinstance(body, str) and suite_invocations(body, scripts): + found.append((rel, job_id, job, step)) + return found + + def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]: """`(workflow rel path, job id, Dockerfile rel path, that workflow's jobs)` per publishing job. @@ -466,15 +721,15 @@ def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]: def test_no_DOCKERFILE_runs_the_spa_suite() -> None: - """The negative half. A gitless stage runs no vitest suite, filtered or not.""" + """The negative part. A gitless stage runs no vitest suite, filtered or not.""" scripts = vitest_scripts() faults: list[str] = [] for rel in dockerfiles(): for number, command in suite_invocations(_read(rel), scripts): faults.append( - f"{rel}:{number} runs the SPA vitest suite, but its build context has no git and " - f"`node:22-bookworm-slim` ships no git binary, so the suite's git-dependent members " - f"cannot run there: `{command.strip()}`. Excluding them by name is the defect this " + f"{rel}:{number} runs the SPA vitest suite in a stage that has neither a git " + f"CHECKOUT nor the git BINARY, so the suite's git-dependent members cannot run " + f"there: `{command.strip()}`. Excluding them by name is the defect this " f"guard exists to prevent — that list is a population nothing derives, it went " f"stale in ersatztv#883, and the resulting red is unreachable on a PR, so it lands " f"on `main` and on the release tag. The suite runs in `docker-build.yml`'s `test` " @@ -485,7 +740,7 @@ def test_no_DOCKERFILE_runs_the_spa_suite() -> None: def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> None: - """The positive half, without which deleting `needs:` is silently green.""" + """The positive part, without which deleting `needs:` is silently green.""" scripts = vitest_scripts() known = set(dockerfiles()) faults: list[str] = [] @@ -515,55 +770,72 @@ def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> def test_every_spa_suite_invocation_is_UNFILTERED() -> None: - """One list, and it is empty. A narrowed run is the hand-maintained population coming back. + """One list, and it is empty — at the call site AND inside the npm script it runs. Driven off the YAML walk rather than the file text, so it reads exactly what the gate assertion - above reads. `test_the_yaml_walk_finds_every_suite_invocation_the_TEXT_does` is what stops the - walk from silently narrowing. + reads. `test_the_yaml_walk_finds_every_suite_invocation_the_TEXT_does` is what stops the walk + from silently narrowing. """ scripts = vitest_scripts() faults: list[str] = [] - for path in workflow_files(): - rel = path.relative_to(REPO_ROOT).as_posix() - jobs = _jobs(yaml.safe_load(path.read_text(encoding="utf-8"))) - for job_id, job in jobs.items(): - for body in _run_bodies(job): - for _, command in suite_invocations(body, scripts): - narrowing = narrowing_arguments(command, scripts) - if narrowing: - faults.append( - f"{rel}: job `{job_id}` runs the SPA suite but restricts which specs run " - f"({narrowing}): `{command.strip()}`. The gating run must be the WHOLE " - f"suite — a filtered one re-creates the hand-maintained population " - f"ersatztv#887 removed." - ) + for rel, job_id, _, step in suite_steps(): + for _, command in suite_invocations(step["run"], scripts): + narrowing = narrowing_arguments(command, scripts) + if narrowing: + faults.append( + f"{rel}: job `{job_id}` runs the SPA suite but restricts which specs run " + f"({narrowing}): `{command.strip()}`. The restriction may be written at this " + f"call site or inside the `{PACKAGE_JSON}` script it runs — both count, and a " + f"body-level filter is the one that reads as clean from the workflow. The " + f"gating run must be the WHOLE suite: a filtered one re-creates the " + f"hand-maintained population ersatztv#887 removed." + ) assert not faults, "\n".join(faults) -def test_the_yaml_walk_finds_every_suite_invocation_the_TEXT_does() -> None: - """The walk and a plain text scan must agree per file, or the walk has a blind spot. +def test_the_gating_suite_run_is_NOT_ADVISORY() -> None: + """A `needs:` edge onto a job that cannot fail blocks nothing. - Both assertions above read `jobs..steps[].run`. A step list the walk cannot reach makes a - suite run structurally invisible while the guard still reports a clean pass, and absence of - output is not evidence. The two shapes that would escape BOTH halves — a local composite action - and a `jobs..uses:` reusable workflow — are asserted not to exist AT ALL by - `test_workflow_persist_credentials.py`, repo-wide and for the same reason; they are not - re-asserted here, because two guards on one condition mask each other (ersatztv#685). + `continue-on-error: true` on the step or the job turns a red suite into a green job, `build` + publishes, and every other assertion here stays green — the gate certifies a publish on which + the suite failed. The `if:` is pinned for the same reason one step further out: a third + condition would let the suite not run at all. """ scripts = vitest_scripts() - disagreements: list[str] = [] - for path in workflow_files(): - text = path.read_text(encoding="utf-8") - jobs = _jobs(yaml.safe_load(text)) - walked = sum(len(suite_invocations(body, scripts)) for job in jobs.values() for body in _run_bodies(job)) - scanned = len(suite_invocations(text, scripts)) - if walked != scanned: - disagreements.append( - f"{path.name}: the jobs..steps walk found {walked} SPA-suite invocation(s) but " - f"the text scan found {scanned}. The walk is what both assertions above run over, so " - f"the difference is suite runs this guard cannot see — widen `_run_bodies`." + faults: list[str] = [] + for rel, job_id, job, step in suite_steps(): + for scope, holder in (("step", step), ("job", job)): + if holder.get("continue-on-error") in (True, "true"): + faults.append( + f"{rel}: the SPA suite {scope} in job `{job_id}` carries " + f"`continue-on-error: true`, so a red suite reports success and the `needs:` " + f"edge that gates the image blocks nothing." + ) + # The JOB's `if:` as well as the step's: a job that does not run reports `skipped`, and the + # question of what a skipped `needs:` does to `build` is one this repo deliberately does not + # rely on (see docker-build.yml's header on `if:`-skipped required contexts). + if job.get("if") is not None: + faults.append( + f"{rel}: the job `{job_id}` that runs the SPA suite carries a job-level `if:` " + f"({str(job['if']).strip()!r}). None did when ersatztv#887 made this `needs:` edge " + f"the only gate, and a job that does not run cannot gate anything." ) - assert not disagreements, "\n".join(disagreements) + condition = step.get("if") + if condition is not None and str(condition).strip() not in KNOWN_GATE_CONDITIONS: + faults.append( + f"{rel}: the SPA suite step in job `{job_id}` carries an `if:` this guard does not " + f"know: {str(condition).strip()!r}. Since ersatztv#887 that step is the only thing " + f"standing between a red suite and a published image, so a change to WHEN it runs " + f"is a decision to state rather than a diff to skim — add the new condition to " + f"`KNOWN_GATE_CONDITIONS` with its reason." + ) + for suppression in failure_suppressions(step["run"], scripts): + faults.append( + f"{rel}: the SPA suite command in job `{job_id}` cannot fail the step — " + f"{suppression}. That is `continue-on-error` by another spelling: the job reports " + f"success over a red suite and the image publishes." + ) + assert not faults, "\n".join(faults) def test_no_run_BODY_builds_or_pushes_an_image() -> None: @@ -614,7 +886,7 @@ def test_the_populations_are_NOT_empty() -> None: assert spa_carrying, ( f"none of {dockerfiles()} was read as carrying the SPA source, so " "`test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite` skipped every " - "job and asserted nothing. `carries_spa_source` stopped matching the `COPY web/` it derives." + "job and asserted nothing. `copies_spa_source` stopped matching the copy it derives." ) published = publishing_jobs() @@ -624,14 +896,9 @@ def test_the_populations_are_NOT_empty() -> None: "recognising the action." ) - runs = [ - (path.name, number) - for path in workflow_files() - for number, _ in suite_invocations(path.read_text(encoding="utf-8"), scripts) - ] - assert runs, ( - "no workflow runs the SPA suite at all. The image gate then rests on a job that runs " - "nothing, and `test_every_spa_suite_invocation_is_UNFILTERED` asserts over an empty set." + assert suite_steps(), ( + "no workflow STEP runs the SPA suite. The image gate then rests on a job that runs nothing, " + "and both the unfiltered and non-advisory assertions iterate over an empty set." ) @@ -641,12 +908,40 @@ def test_the_SCRIPT_derivation_separates_vitest_from_playwright() -> None: `test` in and `test:ui-e2e` out is the whole reason there is no exemption list. If the derivation drifted to include `test:ui-e2e`, the guard would demand an unfiltered gate on a Playwright run; if it drifted to exclude `test`, nothing in the repo would be recognised as a - suite run and every assertion above would pass vacuously. + suite run and every assertion above would pass vacuously — `test_the_populations_are_NOT_empty` + asserts only that the table is non-empty, so a wrong-but-populated table is caught HERE and + nowhere else. + + The two names are hardcoded, which the module docstring argues against for the DERIVATION and + not for this pin: renaming the script produces a false RED here (measured — the derivation + itself adapts and every gate assertion stays correct), never a silent pass. A rename is then a + two-line edit someone makes deliberately, which is the intended cost. """ scripts = vitest_scripts() assert "test" in scripts, scripts assert "test:ui-e2e" not in scripts, scripts assert "build" not in scripts and "lint" not in scripts, scripts + assert scripts["test"] == [], ( + f"the `test` script body already narrows the suite ({scripts['test']}). That is the " + "call-site-invisible form of the defect ersatztv#887 removed." + ) + + +def test_a_filter_inside_the_npm_SCRIPT_BODY_is_seen() -> None: + """The call site can read as clean while the script it runs is filtered. + + `"test": "vitest --exclude x"` with a workflow saying `npm test -- --run` is a filtered gating + run that no assertion reading only the workflow can see. Driven through the real functions with + a synthetic table, so the mechanism is exercised rather than described. + """ + filtered = {"test": ["--exclude", "src/api/x.guard.test.ts"]} + assert narrowing_arguments("npm test -- --run", filtered) == ["--exclude", "src/api/x.guard.test.ts"] + assert narrowing_arguments("npm test -- --run", {"test": []}) == [] + # The two compose: a body filter plus a call-site filter reports both. + both = narrowing_arguments("npm test -- --run --project=spa", filtered) + assert both == ["--exclude", "src/api/x.guard.test.ts", "--project=spa"], both + # And the derivation really reads a body filter out of package.json rather than assuming none. + assert _narrowing_in(["--exclude", "a.ts"]) == ["--exclude", "a.ts"] def test_the_detector_recognises_the_SPELLINGS_it_claims_to() -> None: @@ -654,9 +949,10 @@ def test_the_detector_recognises_the_SPELLINGS_it_claims_to() -> None: The mutation proof drives the real Dockerfile; this covers the spellings the tree does not contain, so a detector that quietly stopped matching one is caught without waiting for someone - to write it. + to write it. Every entry marked MEASURED was returning the wrong answer before the clause that + handles it was added — cold-review findings, not hypotheticals. """ - scripts = {"test", "test:unit"} + scripts: dict[str, list[str]] = {"test": [], "test:unit": []} for command in ( "npm test -- --run", "npm run test", @@ -671,6 +967,21 @@ def test_the_detector_recognises_the_SPELLINGS_it_claims_to() -> None: # The one-line YAML step form, which roughly half this repo's `run:` steps use. "run: npm test -- --run", "- run: npm test -- --run", + # MEASURED False before `tokenize` used punctuation_chars: plain `shlex.split` yields the + # single token `lint&&npm`, hiding the second command entirely. + "npm run lint&&npm test -- --run", + # MEASURED False before `_skip_runner_flags`. + "npm --prefix web test -- --run", + "npm --prefix=web test -- --run", + "npm -s test -- --run", + "npm --loglevel warn run test", + "npx -p vitest vitest run", + "npm exec -- vitest run", + # MEASURED False before `_expand_shell_payloads` / the `xargs` wrapper. + "sh -c 'npm test -- --run'", + "bash -lc 'npm test -- --run'", + "sh -euxc 'cd web && npm test -- --run'", + "xargs npm test -- --run", ): assert suite_invocations(command, scripts), command @@ -682,13 +993,17 @@ def test_the_detector_recognises_the_SPELLINGS_it_claims_to() -> None: "npm run typecheck", "npm run test:ui-e2e", # not in `scripts`, i.e. not a vitest script "playwright test", - # Prose ABOUT the suite is not a run of it. The first version of this guard reported both of - # these, which is a false red in exactly the files that explain the rule. + # Prose ABOUT the suite is not a run of it. An earlier version of this guard reported both, + # which is a false red in exactly the files that explain the rule. "echo the vitest suite does not run here", "echo skipping npm test in this stage", # A non-RUN Dockerfile instruction does not execute during a build. "CMD npm test -- --run", "COPY vitest.config.ts ./", + # Flag-skipping must not turn a non-suite npm command into one. + "npm --prefix web ci", + "npm --prefix web run build", + "bash scripts/some-script.sh", # a script is not followed; stated as a residual ): assert not suite_invocations(command, scripts), command @@ -701,6 +1016,84 @@ def test_the_detector_recognises_the_SPELLINGS_it_claims_to() -> None: assert narrowing_arguments("npm test -- --run --project=spa", scripts) == ["--project=spa"] assert narrowing_arguments("npm test -- --run src/api", scripts) == ["src/api"] assert narrowing_arguments("npm test -- --run --changed", scripts) == ["--changed"] + # A filter still reads as a filter through a wrapper and through leading runner flags. + assert narrowing_arguments("sh -c 'npm test -- --run --exclude a.ts'", scripts) == ["--exclude", "a.ts"] + assert narrowing_arguments("npm --prefix web test -- --run --exclude a.ts", scripts) == ["--exclude", "a.ts"] + + +def test_a_SUPPRESSED_failure_is_advisory_by_another_spelling() -> None: + """`continue-on-error` is the declarative form; these are the shell forms of the same thing. + + All three were MEASURED green — the gate certified a publish over a suite that could not fail. + Driven directly here so the mechanism is covered without waiting for someone to write it into a + workflow. + """ + scripts: dict[str, list[str]] = {"test": []} + # A REAL step body: the `ci-step-ran.sh` marker line, then the suite. Tokenising the two lines + # together merged them into one segment and missed all three suppressions — pinned here so the + # single-line form can never again stand in for the shape the workflow actually has. + marker = '"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test\n' + assert failure_suppressions(marker + "npm test -- --run || true", scripts) + assert failure_suppressions(marker + "npm test -- --run | tee /tmp/spa.log", scripts) + assert failure_suppressions(marker + "set +e\nnpm test -- --run", scripts) + assert failure_suppressions(marker + "sh -c 'npm test -- --run || true'", scripts) + # And each still reports on its own. + assert failure_suppressions("npm test -- --run || true", scripts) + assert failure_suppressions("npm test -- --run | tee /tmp/spa.log", scripts) + assert failure_suppressions("set +e\nnpm test -- --run", scripts) + # An ordinary run, and a `||` on something that is NOT the suite, are both clean. + assert failure_suppressions(marker + "npm test -- --run", scripts) == [] + assert failure_suppressions("npm run lint && npm test -- --run", scripts) == [] + assert failure_suppressions("npm run lint || true\nnpm test -- --run", scripts) == [] + + +def test_a_REDIRECTION_is_not_a_spec_filter() -> None: + """Capturing a log is not narrowing the suite, and reporting it as one is a false red. + + MEASURED reporting `['>', 'test.log']` before `_strip_redirections` existed. The `2>&1` form + matters separately: it tokenises into three pieces, two of which are bare tokens that the + positional-filter rule would otherwise both report. + """ + scripts: dict[str, list[str]] = {"test": []} + assert narrowing_arguments("npm test -- --run > test.log", scripts) == [] + assert narrowing_arguments("npm test -- --run >> test.log", scripts) == [] + assert narrowing_arguments("npm test -- --run 2>&1", scripts) == [] + assert narrowing_arguments("npm test -- --run > test.log 2>&1", scripts) == [] + # A real filter beside a redirection is still reported. + assert narrowing_arguments("npm test -- --run --exclude a.ts > log", scripts) == ["--exclude", "a.ts"] + + +def test_the_flags_that_move_the_COLLECTED_SET_count_as_filters() -> None: + """`--root` and `--config` change which specs vitest collects, without naming one. + + Both were MEASURED reporting `[]` while classified as benign value-taking flags — a narrowed + gating run reading as clean, which is the defect itself in its least visible spelling. + """ + scripts: dict[str, list[str]] = {"test": []} + assert narrowing_arguments("npm test -- --run --root src/api", scripts) == ["--root", "src/api"] + assert narrowing_arguments("npm test -- --run --config vitest.subset.ts", scripts) == [ + "--config", + "vitest.subset.ts", + ] + # A flag that genuinely only shapes OUTPUT stays benign. + assert narrowing_arguments("npm test -- --run --reporter verbose", scripts) == [] + + +def test_a_HEREDOC_body_is_data_rather_than_a_command() -> None: + """A `npm test` line inside a heredoc never executes, and counting it is a false green. + + Cold review's scenario: write the suite command into a file with `cat > f <<'EOF' … EOF`. The + step is green having run nothing, and a guard counting the heredoc line finds both its + non-vacuous invocation and its suite-running `needs:` gate in a job that ran no tests. + """ + scripts: dict[str, list[str]] = {"test": []} + for opener, terminator in (("<<'EOF'", "EOF"), ("< /tmp/run.sh {opener}\nnpm test -- --run\n{terminator}\n" + assert not suite_invocations(body, scripts), body + # The command AFTER the terminator is live again — the skip must end, not swallow the rest. + resumed = "cat > /tmp/run.sh <<'EOF'\nnpm test -- --run\nEOF\nnpm test -- --run\n" + found = suite_invocations(resumed, scripts) + assert len(found) == 1 and found[0][0] == 4, found def test_a_COMMENT_is_inert_but_only_a_LEADING_hash_is() -> None: @@ -709,7 +1102,7 @@ def test_a_COMMENT_is_inert_but_only_a_LEADING_hash_is() -> None: The second half is the one that matters: a `#` mid-line does not make what precedes it a comment, and treating it as one would let a real invocation hide behind a trailing note. """ - scripts = {"test"} + scripts: dict[str, list[str]] = {"test": []} assert not suite_invocations("# npm test -- --run\n", scripts) assert not suite_invocations(" # the vitest suite is not run in this stage\n", scripts) assert suite_invocations("RUN npm test -- --run # left over from before\n", scripts) @@ -721,18 +1114,66 @@ def test_the_continuation_JOIN_keeps_flags_with_their_command() -> None: This is the exact shape the removed exclusion pair had, so a guard that scanned raw lines would see `npm test -- --run` as unfiltered and report it clean. """ - scripts = {"test"} + scripts: dict[str, list[str]] = {"test": []} multiline = "RUN npm test -- --run \\\n --exclude 'src/api/a.test.ts' \\\n --exclude 'b.test.ts'\n" found = suite_invocations(multiline, scripts) assert len(found) == 1, found assert narrowing_arguments(found[0][1], scripts) == ["--exclude", "src/api/a.test.ts", "--exclude", "b.test.ts"] -def test_the_spa_carrying_predicate_separates_the_two_real_DOCKERFILES() -> None: +def test_the_spa_carrying_predicate_reads_every_COPY_SPELLING() -> None: """Pinned both ways, because the gate assertion is SKIPPED for everything it answers False for. - A predicate that drifted to False for `docker/Dockerfile` would make the positive half vacuous - while `test_the_populations_are_NOT_empty` still passed on some other file. + A predicate that drifted to False for `docker/Dockerfile` would make the positive part vacuous + while `test_the_populations_are_NOT_empty` still passed on some other file. The synthetic + spellings were all MEASURED False before this clause was widened — a new Dockerfile using any of + them would have published without a gate. """ assert carries_spa_source("docker/Dockerfile") assert not carries_spa_source("docker/ci/Dockerfile") + + for spelling in ( + 'COPY ["web/", "/source/web/"]', + "COPY web /source/web", + "ADD web/ /source/web/", + "COPY web/. ./web/", + "COPY . /source/", + "COPY ./ /source/", + "COPY --chown=node:node web/ /source/web/", + ): + assert copies_spa_source(spelling), spelling + + for spelling in ( + "COPY design-system/. ./design-system/", + "COPY --from=web-build /source/ErsatzTV/wwwroot/app/. ./app/", + "COPY webhooks/. ./webhooks/", # `web` must be a whole path segment + "RUN npm ci", + "COPY web/", # a single argument is not a copy + ): + assert not copies_spa_source(spelling), spelling + + +def test_the_yaml_walk_finds_every_suite_invocation_the_TEXT_does() -> None: + """The walk and a plain text scan must agree per file, or the walk has a blind spot. + + Every assertion above reads `jobs..steps[].run`. A step list the walk cannot reach makes a + suite run structurally invisible while the guard still reports a clean pass, and absence of + output is not evidence. The two shapes that would escape BOTH halves — a local composite action + and a `jobs..uses:` reusable workflow — are asserted not to exist AT ALL by + `test_workflow_persist_credentials.py`, repo-wide and for the same reason; they are not + re-asserted here, because two guards on one condition mask each other (ersatztv#685). + """ + scripts = vitest_scripts() + disagreements: list[str] = [] + for path in workflow_files(): + text = path.read_text(encoding="utf-8") + jobs = _jobs(yaml.safe_load(text)) + walked = sum(len(suite_invocations(body, scripts)) for job in jobs.values() for body in _run_bodies(job)) + scanned = len(suite_invocations(text, scripts)) + if walked != scanned: + disagreements.append( + f"{path.name}: the jobs..steps walk found {walked} SPA-suite invocation(s) but " + f"the text scan found {scanned}. The walk is what every assertion above runs over, " + f"so the difference is suite runs this guard cannot see — widen `_run_bodies`." + ) + assert not disagreements, "\n".join(disagreements) diff --git a/web/vite-plugins/trackedSourceFiles.realgit.test.ts b/web/vite-plugins/trackedSourceFiles.realgit.test.ts index 9a81861be..2859b2549 100644 --- a/web/vite-plugins/trackedSourceFiles.realgit.test.ts +++ b/web/vite-plugins/trackedSourceFiles.realgit.test.ts @@ -75,7 +75,9 @@ describe('resolveTrackedSourceFiles against a REAL repository (#819)', () => { // Residual: this asserts the message the MISSING-BINARY path also emits, so in an environment with // no git it passes for the wrong reason. Not silent overall — its two siblings fail ENOENT there, - // loudly — and that is exactly the environment this file is excluded from in `docker/Dockerfile`. + // loudly. Since ersatztv#887 no such environment runs this suite: `docker/Dockerfile` stopped + // running it rather than excluding the files that cannot run, so the residual is now reachable + // only on a developer machine with no git installed. it('really throws outside a repository, rather than returning an empty population', () => { const notARepo = mkdtempSync(join(tmpdir(), 'etv-819-bare-')); repos.push(notARepo); -- 2.47.3 From 090a29db3d82d4a042dd73bf8bf668fb746fa5ea Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 11:13:35 +0200 Subject: [PATCH 03/16] fix(887): two defects the FIX introduced, found by attacking it rather than reading it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both were measured, not reasoned, and both are the shape this repo keeps recording — the fix round introducing an adjacent defect, and a unit test using a simpler input shape than the real file has. `npm test -- --run && echo ok || true` reported NO suppression. `&&`/`||` chain across a whole list, so when the suite fails the `&&` right-hand side is skipped and the `||` right-hand side runs: the list exits 0 and the suite's failure is swallowed even though the `||` is not adjacent to it. The detector looked only at the separator IMMEDIATELY after the suite segment. It is now scoped to the `;`-delimited list, which also catches a backgrounded `npm test &` (status never awaited) and `( npm test ) || true`. A `;` ends the list and resets, so `npm test; other || true` stays clean — that `||` is about the other command. `--exclude 2 > log` reported `['--exclude']`, losing the filter's own value: stripping redirections as a PRE-PASS let the file-descriptor rule claim the `2` before the flag could. Redirections are now consumed inside the walk, after flag values are taken. The mutant battery grew from 17 to 24 and is 0-missed. The `docs/guard-inventory.md` row now states the count and, explicitly, the grading: exactly ONE of the 24 is declared in `mutation_manifest.py` and re-executed every suite; the other 23 were witnessed by hand and are NOT standing. That is the same footing `pageSizeCallSites.guard.test.ts` states for its nine, and saying so is the difference between evidence for the reach and a claim of a per-run proof. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docs/guard-inventory.md | 2 +- ...est_image_build_delegates_the_spa_suite.py | 83 +++++++++++-------- 2 files changed, 49 insertions(+), 36 deletions(-) diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index 1181ab736..f6dc55de5 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | `.gitea/workflows/*.y*ml` + every tracked `Dockerfile*`/`*.Dockerfile`, plus the vitest SCRIPT NAMES read out of `web/package.json` | **yes, in both directions** — no Dockerfile in the population may run the SPA suite, and every image-publishing job whose Dockerfile carries the SPA source must transitively `needs:` a job that runs it unfiltered | ersatztv#887, index-derived from the start. THREE populations rather than one, because a single one leaves a hole the other two close: the Dockerfile set alone cannot see a deleted `needs:` edge, and the job set alone cannot see a suite run added to a gitless stage. The publishing jobs are derived from the `docker/build-push-action` step and the Dockerfile each builds is read from that step's own `file:` input, so `ci-image.yml` — which publishes from `docker/ci/Dockerfile`, a Dockerfile carrying no SPA source — is out of scope by DERIVATION and holds no exemption entry that could outlive its reason. Which npm scripts count as the suite is derived too, from `web/package.json`: `test` is in because `"test": "vitest"` is, and the sibling `test:ui-e2e` is out because it runs Playwright — a hardcoded name would go quiet on a rename, and a `test:`-prefix guess would demand an unfiltered gate on a browser suite. A THIRD assertion exists because the first two together still certify a publish on which the suite never ran: the gating step must not be advisory. Two independent cold reviews each demonstrated that gap by execution rather than argument, and between them produced twelve mutants the guard passed on arrival — `continue-on-error: true`, `if: false`, a job-level `if:`, an or-true suffix on the suite command, a pipeline into `tee`, `set +e`, `--root src/api`, `--config other.ts`, an unspaced `lint&&npm test`, a `;`-separated re-add, `sh -c "npm test …"`, and `npm --prefix web test`. All twelve now redden; each was re-measured after the fix. THE REACH IS MEASURED, NOT ASSERTED, and the residuals below are what survived that measurement rather than what seemed likely. (1) A `RUN` or step that calls a SCRIPT which runs the suite is invisible — the parser reads commands, not what they call, and `sh -c` is expanded only because its argument IS a command line; no such script exists today, and the miss is a missed DETECTION on the negative assertion, never a false green on the positive one, which is derived from the job graph instead. (2) Only `RUN` is read in a Dockerfile, since nothing else executes during a build. (3) A line whose FIRST non-space character is `#` is inert — that is what lets `docker/Dockerfile` name the rule it is subject to — and only a LEADING `#`, so an invocation with a trailing note is still seen. (4) The step `if:` is checked against a HAND-WRITTEN set of one condition, which is a deliberate review trigger rather than a derivation: a legitimate change reddens and its author states the reason. (5) `test_the_SCRIPT_derivation_separates_vitest_from_playwright` pins the two script NAMES, so renaming the `test` script is a false red — measured, and the derivation itself adapts correctly; the pin is the only thing that catches a wrong-but-non-empty script table, which the anti-vacuity check cannot. (6) A new value-taking vitest flag is reported as a spec filter, a false red fixed by naming it | +| `test_image_build_delegates_the_spa_suite.py` | `.gitea/workflows/*.y*ml` + every tracked `Dockerfile*`/`*.Dockerfile`, plus the vitest SCRIPT NAMES read out of `web/package.json` | **yes, in both directions** — no Dockerfile in the population may run the SPA suite, and every image-publishing job whose Dockerfile carries the SPA source must transitively `needs:` a job that runs it unfiltered | ersatztv#887, index-derived from the start. THREE populations rather than one, because a single one leaves a hole the other two close: the Dockerfile set alone cannot see a deleted `needs:` edge, and the job set alone cannot see a suite run added to a gitless stage. The publishing jobs are derived from the `docker/build-push-action` step and the Dockerfile each builds is read from that step's own `file:` input, so `ci-image.yml` — which publishes from `docker/ci/Dockerfile`, a Dockerfile carrying no SPA source — is out of scope by DERIVATION and holds no exemption entry that could outlive its reason. Which npm scripts count as the suite is derived too, from `web/package.json`: `test` is in because `"test": "vitest"` is, and the sibling `test:ui-e2e` is out because it runs Playwright — a hardcoded name would go quiet on a rename, and a `test:`-prefix guess would demand an unfiltered gate on a browser suite. A THIRD assertion exists because the first two together still certify a publish on which the suite never ran: the gating step must not be advisory. Two independent cold reviews each demonstrated that gap by execution rather than argument. A 24-mutant battery now stands behind the reach: **24 applied, 0 missed**, each caught by the assertion intended for it, re-measured after every fix round. Twelve came from the two reviews (`continue-on-error: true`, `if: false`, a job-level `if:`, an or-true suffix, a pipeline into `tee`, `set +e`, `--root`, `--config`, an unspaced `lint&&npm test`, a `;`-separated re-add, `sh -c`, `npm --prefix`); two more were found by the author attacking the fix (`npm test && echo ok || true`, where the or-true is not adjacent to the suite yet still swallows its status, and a backgrounded `&`); the rest cover `xargs`, `npx -p`, `sh -euxc`, the JSON/`ADD`/no-slash copy spellings, and a heredoc standing in for the run — that last one reddens TWO assertions, since the gate loses its runner and anti-vacuity fires. **GRADING, stated rather than implied: exactly ONE of those 24 is DECLARED in `mutation_manifest.py` and re-executed every suite; the other 23 were witnessed by hand during development and are NOT standing** — the same footing `pageSizeCallSites.guard.test.ts` states for its nine. The battery is development evidence for the reach, not a per-run proof of it. THE REACH IS MEASURED, NOT ASSERTED, and the residuals below are what survived that measurement rather than what seemed likely. (1) A `RUN` or step that calls a SCRIPT which runs the suite is invisible — the parser reads commands, not what they call, and `sh -c` is expanded only because its argument IS a command line; no such script exists today, and the miss is a missed DETECTION on the negative assertion, never a false green on the positive one, which is derived from the job graph instead. (2) Only `RUN` is read in a Dockerfile, since nothing else executes during a build. (3) A line whose FIRST non-space character is `#` is inert — that is what lets `docker/Dockerfile` name the rule it is subject to — and only a LEADING `#`, so an invocation with a trailing note is still seen. (4) The step `if:` is checked against a HAND-WRITTEN set of one condition, which is a deliberate review trigger rather than a derivation: a legitimate change reddens and its author states the reason. (5) `test_the_SCRIPT_derivation_separates_vitest_from_playwright` pins the two script NAMES, so renaming the `test` script is a false red — measured, and the derivation itself adapts correctly; the pin is the only thing that catches a wrong-but-non-empty script table, which the anti-vacuity check cannot. (6) A new value-taking vitest flag is reported as a spec filter, a false red fixed by naming it | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index cca679e85..ee5941be9 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -461,35 +461,14 @@ def suite_invocations(text: str, scripts: Mapping[str, list[str]] | None = None) return found -def _strip_redirections(args: list[str]) -> list[str]: - """Drop redirection operators, their targets, and any file descriptor written before them. - - `2>&1` tokenises as `['2', '>&', '1']`, so the descriptor and the target are bare tokens that - would otherwise both be reported as positional spec filters. - """ - kept: list[str] = [] - index = 0 - while index < len(args): - argument = args[index] - if _REDIRECTION.match(argument): - index += 2 # the operator and its target - continue - if ( - argument.isdigit() - and index + 1 < len(args) - and _REDIRECTION.match(args[index + 1]) - and not args[index + 1][:1].isdigit() - ): - index += 3 # descriptor, operator, target - continue - kept.append(argument) - index += 1 - return kept - - def _narrowing_in(args: list[str]) -> list[str]: - """The subset of `args` that restricts WHICH specs run.""" - args = _strip_redirections(args) + """The subset of `args` that restricts WHICH specs run. + + Redirections are consumed HERE rather than in a pre-pass, so a flag's value is claimed before the + file-descriptor rule can see it. As a pre-pass, `--exclude 2 > log` lost the `2`: the digit was + read as the descriptor of the following `>` and dropped with it, and the fault message then named + a filter without its value. Measured. + """ found: list[str] = [] index = 0 while index < len(args): @@ -497,6 +476,17 @@ def _narrowing_in(args: list[str]) -> list[str]: index += 1 if argument == "--": continue + if _REDIRECTION.match(argument): + index += 1 # the operator's target + continue + if ( + argument.isdigit() + and index < len(args) + and _REDIRECTION.match(args[index]) + and not args[index][:1].isdigit() + ): + index += 2 # the operator and its target; `argument` was its descriptor + continue base = argument.split("=", 1)[0] if base in NARROWING_FLAGS: found.append(argument) @@ -551,6 +541,13 @@ def failure_suppressions(command: str, scripts: Mapping[str, list[str]], depth: tokens = tokens[1:] if tokens[0] == "RUN" else [] current: list[str] = [] + # Whether the SUITE has run earlier in this `;`-delimited list. `&&`/`||` chain across a + # whole list, so `npm test && echo ok || true` runs `true` when the suite fails and exits 0 — + # the suite's failure is swallowed even though the `||` does not sit immediately after it. + # Looking only at the NEXT separator missed that; measured. A `;` (or a newline, since each + # line is walked separately) ends the list and resets, because a `||` in the next list is + # about the next command. + suite_in_list = False for word in [*tokens, ";"]: if word not in SEPARATORS: current.append(word) @@ -559,17 +556,21 @@ def failure_suppressions(command: str, scripts: Mapping[str, list[str]], depth: head = _strip_prefixes(current) if head[:2] == ["set", "+e"]: found.append("`set +e` disarms the step's own `set -e`") - if suite_args(current, scripts) is not None and word in {"||", "|"}: - found.append( - "a trailing or-true swallows the exit status" - if word == "||" - else "a pipeline reports the LAST command's status" - ) + if suite_args(current, scripts) is not None: + suite_in_list = True + if word == "|": + found.append("a pipeline reports the LAST command's status") if len(head) >= 3 and head[0] in SHELLS: for offset, inner in enumerate(head[1:-1], start=1): if inner.startswith("-") and not inner.startswith("--") and "c" in inner.lstrip("-"): found.extend(failure_suppressions(head[offset + 1], scripts, depth + 1)) break + if word == "||" and suite_in_list: + found.append("an `||` later in the same list swallows the exit status") + if word == "&" and suite_in_list: + found.append("`&` backgrounds the command, so its status is never awaited") + if word == ";": + suite_in_list = False current = [] return found @@ -1041,10 +1042,19 @@ def test_a_SUPPRESSED_failure_is_advisory_by_another_spelling() -> None: assert failure_suppressions("npm test -- --run || true", scripts) assert failure_suppressions("npm test -- --run | tee /tmp/spa.log", scripts) assert failure_suppressions("set +e\nnpm test -- --run", scripts) + # A `||` LATER in the same list still swallows it: if the suite fails, the `&&` right-hand side + # is skipped and the `||` right-hand side runs, so the list exits 0. Measured missed before the + # detector became list-scoped rather than next-separator-scoped. + assert failure_suppressions("npm test -- --run && echo ok || true", scripts) + assert failure_suppressions("npm test -- --run && npm run build || true", scripts) + assert failure_suppressions("npm test -- --run &", scripts) + assert failure_suppressions("( npm test -- --run ) || true", scripts) # An ordinary run, and a `||` on something that is NOT the suite, are both clean. assert failure_suppressions(marker + "npm test -- --run", scripts) == [] assert failure_suppressions("npm run lint && npm test -- --run", scripts) == [] assert failure_suppressions("npm run lint || true\nnpm test -- --run", scripts) == [] + # A `;` ends the list, so a `||` after it is about the NEXT command, not the suite. + assert failure_suppressions("npm test -- --run; npm run other || true", scripts) == [] def test_a_REDIRECTION_is_not_a_spec_filter() -> None: @@ -1059,8 +1069,11 @@ def test_a_REDIRECTION_is_not_a_spec_filter() -> None: assert narrowing_arguments("npm test -- --run >> test.log", scripts) == [] assert narrowing_arguments("npm test -- --run 2>&1", scripts) == [] assert narrowing_arguments("npm test -- --run > test.log 2>&1", scripts) == [] - # A real filter beside a redirection is still reported. + # A real filter beside a redirection is still reported, INCLUDING a numeric value: stripping + # redirections as a pre-pass ate the `2` in `--exclude 2 > log` by reading it as a descriptor. assert narrowing_arguments("npm test -- --run --exclude a.ts > log", scripts) == ["--exclude", "a.ts"] + assert narrowing_arguments("npm test -- --run --exclude 2 > log", scripts) == ["--exclude", "2"] + assert narrowing_arguments("npm test -- --run 2 --exclude a.ts", scripts) == ["2", "--exclude", "a.ts"] def test_the_flags_that_move_the_COLLECTED_SET_count_as_filters() -> None: -- 2.47.3 From 376aad6774fe041e00ea1eba41e3865e977c80bc Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 12:01:27 +0200 Subject: [PATCH 04/16] =?UTF-8?q?fix(887):=20withdraw=20the=20command=20pa?= =?UTF-8?q?rser=20=E2=80=94=20pin=20the=20command=20TEXT=20instead?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 4, and the third cold review found the same mechanism failing again, so it is removed rather than patched a tenth time. WHAT KEPT BREAKING. Three versions of this guard asked "does this command RUN the suite, and can its failure be swallowed?" of arbitrary shell text. That predicate was wrong NINE times across three review rounds, and twice a clause added to remove a FALSE RED opened a FALSE GREEN on the guard's headline assertion: * heredoc bodies were skipped as data, but BuildKit EXECUTES `RUN < Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docker/Dockerfile | 14 +- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 10 +- docs/guard-inventory.md | 4 +- scripts/tests/mutation_manifest.py | 16 +- ...est_image_build_delegates_the_spa_suite.py | 1308 +++++------------ 6 files changed, 408 insertions(+), 946 deletions(-) diff --git a/docker/Dockerfile b/docker/Dockerfile index 729340c3a..6c9627b2b 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -15,9 +15,12 @@ WORKDIR /source/web # (`context: .`) and whose `.dockerignore` does not exclude `.git`: the directory is there to be # copied and simply is not. And `node:22-bookworm-slim` ships no git binary (`command -v git` -> not # found), which is why a `COPY .git` would not help either. Members of the SPA suite need one or the -# other — `web/src/api/*.guard.test.ts` derive -# their file population from `git ls-files` and refuse to fall back to a directory walk -# (`testing.guard-derives-population-from-source`), and `web/vite-plugins/*.realgit.test.ts` builds +# other: `pageSizeCallSites.guard.test.ts` and `completeAnnotations.guard.test.ts` derive their file +# population from `git ls-files` and refuse to fall back to a directory walk +# (`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds +# its own temp repository. (Not every `*.guard.test.ts` — `completeRequest.guard.test.ts` needs no +# git at all.) +# # Running the suite here anyway therefore costs a hand-maintained list of the members that cannot # run — and that list is a population nothing derives. It went stale the first time a guard was # added without updating it, and because `Build & push image (amd64)` is `if: github.event_name != @@ -31,8 +34,9 @@ WORKDIR /source/web # neither leaves an image unvalidated: `docs_only` also gates `Build and push`, so that arm ships # nothing; the #420 revalidate skip fires only on a tree byte-identical to a head that already # carried a green combined status, i.e. this exact source already passed. Held by -# `scripts/tests/test_image_build_delegates_the_spa_suite.py`, which derives both halves rather -# than restating them. +# `scripts/tests/test_image_build_delegates_the_spa_suite.py`. That guard PINS the command line +# below as text: any edit to it reddens, whatever the edit says. Update the pin in the same commit, +# and do not use it to re-add a suite run. # # `lint` and `typecheck` stay. They are gitless-safe with no member that is not, so they carry no # list and no trap; whether the image build should run them at all is a separate question this diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 5311975a0..e012e38e5 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present: the gating step must not be advisory (`continue-on-error`, `\|\| true`, a pipe, `set +e`), its job must carry no job-level `if:`, its step `if:` must be one this guard knows, and the run must be unfiltered at the CALL SITE and inside the `web/package.json` script body. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds; a pin cannot be wrong in the dangerous direction, only in the loud one. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index d70656cfc..ebad5c2b6 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,9 +5,9 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present: the gating step must not be advisory (`continue-on-error`, `|| true`, a pipe, `set +e`), its job must carry no job-level `if:`, its step `if:` must be one this guard knows, and the run must be unfiltered at the CALL SITE and inside the `web/package.json` script body. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds; a pin cannot be wrong in the dangerous direction, only in the loud one. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' -mechanics: 'Guard derives three populations — tracked Dockerfiles, tracked workflows, and the vitest script names read out of `web/package.json` — then asserts no Dockerfile runs the suite, every publishing job whose Dockerfile carries the SPA source transitively `needs:` a job that does, every such run is unfiltered at both the call site and in the script body, and the gating step is not advisory. Publishing jobs come from the `docker/build-push-action` step; the Dockerfile each builds comes from that step''s `file:` input.' +mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 41 mutants, 0 missed.' --- The web-build stage is gitless twice over. It copies only `web/` and `design-system/`, so it holds @@ -15,8 +15,8 @@ no `.git`. That is a property of the STAGE, not of the build context — the con root (`context: .`) and `.dockerignore` does not exclude `.git`, so the directory is available to be copied and simply is not; stating it the other way invites a reader to check, find `.git` in the context, and conclude the whole note is stale. And `node:22-bookworm-slim` ships no git binary. Members of the SPA -suite need one or the other — the `web/src/api/*.guard.test.ts` guards derive their file population -from `git ls-files` +suite need one or the other: `pageSizeCallSites.guard.test.ts` and +`completeAnnotations.guard.test.ts` derive their file population from `git ls-files` and refuse to fall back to a directory walk (`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds a real temporary repository. Because the two prerequisites OVERLAP rather than divide, adding a `COPY .git` fixes neither: the binary is still @@ -56,6 +56,8 @@ not: green on every PR, red on `main` and on the release tag. 3. *`docker build` running the suite for a developer locally* — lost; `npm test -- --run` is the direct route and is what `docs/testing.md` documents. +**The guard's own mechanism was replaced once, and the reason belongs here rather than only in the commit log, because the next person to widen it needs it.** Three versions asked whether a given command runs the suite and whether its failure can be swallowed. That predicate was wrong nine times across three cold-review rounds — executed heredocs treated as data, `#` truncating a command mid-word (including the live `${#reports[@]}` idiom), compound punctuation welding commands, `npm t` and `./node_modules/.bin/vitest` and `timeout`/`su -c` wrappers unrecognised, `true || npm test` counted as a run it never performs, and `continue-on-error: ${{ … }}` slipping a two-literal check. Twice, a clause added to remove a FALSE RED opened a FALSE GREEN on the guard's headline assertion. The mechanism was withdrawn rather than patched a tenth time: the risky command lines are now compared against a pin, so no spelling has to be recognised in order to be rejected. The cost is real and is the point — a legitimate edit to those lines reddens the guard and its author updates the pin deliberately. + `lint` and `typecheck` stay in the stage. They are gitless-safe with no member that is not, so they carry no list and no trap. Whether the image build should run them at all is a separate question this does not answer, and leaving them is not an argument that it should. diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index f6dc55de5..dcba07771 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | `.gitea/workflows/*.y*ml` + every tracked `Dockerfile*`/`*.Dockerfile`, plus the vitest SCRIPT NAMES read out of `web/package.json` | **yes, in both directions** — no Dockerfile in the population may run the SPA suite, and every image-publishing job whose Dockerfile carries the SPA source must transitively `needs:` a job that runs it unfiltered | ersatztv#887, index-derived from the start. THREE populations rather than one, because a single one leaves a hole the other two close: the Dockerfile set alone cannot see a deleted `needs:` edge, and the job set alone cannot see a suite run added to a gitless stage. The publishing jobs are derived from the `docker/build-push-action` step and the Dockerfile each builds is read from that step's own `file:` input, so `ci-image.yml` — which publishes from `docker/ci/Dockerfile`, a Dockerfile carrying no SPA source — is out of scope by DERIVATION and holds no exemption entry that could outlive its reason. Which npm scripts count as the suite is derived too, from `web/package.json`: `test` is in because `"test": "vitest"` is, and the sibling `test:ui-e2e` is out because it runs Playwright — a hardcoded name would go quiet on a rename, and a `test:`-prefix guess would demand an unfiltered gate on a browser suite. A THIRD assertion exists because the first two together still certify a publish on which the suite never ran: the gating step must not be advisory. Two independent cold reviews each demonstrated that gap by execution rather than argument. A 24-mutant battery now stands behind the reach: **24 applied, 0 missed**, each caught by the assertion intended for it, re-measured after every fix round. Twelve came from the two reviews (`continue-on-error: true`, `if: false`, a job-level `if:`, an or-true suffix, a pipeline into `tee`, `set +e`, `--root`, `--config`, an unspaced `lint&&npm test`, a `;`-separated re-add, `sh -c`, `npm --prefix`); two more were found by the author attacking the fix (`npm test && echo ok || true`, where the or-true is not adjacent to the suite yet still swallows its status, and a backgrounded `&`); the rest cover `xargs`, `npx -p`, `sh -euxc`, the JSON/`ADD`/no-slash copy spellings, and a heredoc standing in for the run — that last one reddens TWO assertions, since the gate loses its runner and anti-vacuity fires. **GRADING, stated rather than implied: exactly ONE of those 24 is DECLARED in `mutation_manifest.py` and re-executed every suite; the other 23 were witnessed by hand during development and are NOT standing** — the same footing `pageSizeCallSites.guard.test.ts` states for its nine. The battery is development evidence for the reach, not a per-run proof of it. THE REACH IS MEASURED, NOT ASSERTED, and the residuals below are what survived that measurement rather than what seemed likely. (1) A `RUN` or step that calls a SCRIPT which runs the suite is invisible — the parser reads commands, not what they call, and `sh -c` is expanded only because its argument IS a command line; no such script exists today, and the miss is a missed DETECTION on the negative assertion, never a false green on the positive one, which is derived from the job graph instead. (2) Only `RUN` is read in a Dockerfile, since nothing else executes during a build. (3) A line whose FIRST non-space character is `#` is inert — that is what lets `docker/Dockerfile` name the rule it is subject to — and only a LEADING `#`, so an invocation with a trailing note is still seen. (4) The step `if:` is checked against a HAND-WRITTEN set of one condition, which is a deliberate review trigger rather than a derivation: a legitimate change reddens and its author states the reason. (5) `test_the_SCRIPT_derivation_separates_vitest_from_playwright` pins the two script NAMES, so renaming the `test` script is a false red — measured, and the derivation itself adapts correctly; the pin is the only thing that catches a wrong-but-non-empty script table, which the anti-vacuity check cannot. (6) A new value-taking vitest flag is reported as a spec filter, a false red fixed by naming it | +| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN <` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | @@ -252,7 +252,7 @@ recorded as unexamined rather than as cleared. | `scripts/tests/test_guard_inventory.py` | the `script-tests` job, on an unclassified guard or a stale proof ref | GUARD | MUTATION | `test_guard_inventory.py::test_the_inventory_covers_exactly_the_guards_that_exist` | | `scripts/tests/test_guard_populations_derive_from_git.py` | the `script-tests` job, on a guard whose file population admits a file git does not track | GUARD | MUTATION | `test_guard_populations_derive_from_git.py::test_no_derivation_admits_an_untracked_file` | | `scripts/tests/test_hook_fire_log.py` | the `script-tests` job, on a hook that stops reporting that it fired, or whose reporting changes what the harness sees | GUARD | MUTATION | `test_hook_fire_log.py::test_a_hook_that_LOSES_its_instrumentation_is_DETECTED` | -| `scripts/tests/test_image_build_delegates_the_spa_suite.py` | the `script-tests` job, on a Dockerfile that runs the SPA vitest suite, on an image-publishing job that stops being gated on a job that runs it, on a gating run narrowed by a spec filter at the call site or in the `web/package.json` script body, or on a gating run made ADVISORY — `continue-on-error`, a trailing or-true, a pipeline, `set +e`, a job-level `if:`, or an unrecognised step `if:` | GUARD | MUTATION | `test_image_build_delegates_the_spa_suite.py::test_no_DOCKERFILE_runs_the_spa_suite` | +| `scripts/tests/test_image_build_delegates_the_spa_suite.py` | the `script-tests` job, on ANY change to the commands an SPA-carrying Dockerfile stage runs, on ANY change to the gating `Test SPA` step (its `run:` body, its `if:`, or a `continue-on-error` in any spelling), on the publish step losing its `docs_only` gate, on an image-publishing job that stops being gated on the job holding that step, or on any other step mentioning the suite | GUARD | MUTATION | `test_image_build_delegates_the_spa_suite.py::test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands` | | `scripts/tests/test_jq_preflight.py` | the `script-tests` job | PROOF | NONE | — | | `scripts/tests/test_mcp_smoke.py` | the `script-tests` job, when the MCP smoke checker accepts a server it should refuse — an impostor identity, a reply carrying only an id, a missing expected tool, or an `initialize` pre-answered before the request (the `tools/list` twin is held by two mechanisms jointly and is covered behaviourally, not by the declared clause) | GUARD | MUTATION | `test_mcp_smoke.py::test_MUTATION_a_PRE_ANSWERED_id_is_refused_because_the_request_ids_are_UNGUESSABLE` | | `scripts/tests/test_merge_consent_base_change.py` | the `script-tests` job | PROOF | NONE | — | diff --git a/scripts/tests/mutation_manifest.py b/scripts/tests/mutation_manifest.py index 642cccfb7..68f305534 100644 --- a/scripts/tests/mutation_manifest.py +++ b/scripts/tests/mutation_manifest.py @@ -408,18 +408,20 @@ MUTATIONS: tuple[Mutation, ...] = ( target="docker/Dockerfile", clause="RUN npm run lint && npm run typecheck && npm run build", replacement="RUN npm run lint && npm run typecheck && npm test -- --run && npm run build", - proof="test_image_build_delegates_the_spa_suite.py::test_no_DOCKERFILE_runs_the_spa_suite", + proof="test_image_build_delegates_the_spa_suite.py::test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands", granularity=CLAUSE, - expect="runs the SPA vitest suite in a stage that has neither a git CHECKOUT nor the git BINARY", + expect="docker/Dockerfile::web-build does not run its pinned commands", why="The target is not the guard, per the checker-guard exception: the guard IS a test, so " "disarming it makes it absent rather than red, and the mutation goes into the guarded " "ARTIFACT. The clause is the web-build stage's whole command line, and the replacement is " "the defect itself rather than a caricature of it — ersatztv#883 put a vitest run back into " - "a gitless stage, and every image build failed from that commit until #887. Mutating the " - "detector's own token list instead would prove only that the detector matches its own " - "constants; this proves it reads the real Dockerfile and reports the real shape. Note the " - "mutant is UNFILTERED: a filtered run is what actually broke, but the guard forbids the " - "class, and mutating to the narrower case would leave the broader one unproven.", + "a gitless stage, and every image build failed from that commit until #887. WHAT THIS PROVES " + "IS NARROWER THAN IT LOOKS, and saying so is the point: the guard no longer decides whether " + "a command RUNS the suite (that predicate was wrong nine times across three review rounds), " + "it compares the stage's commands against a pin. So this mutation proves the pin is " + "compared and reported — not that any particular spelling is recognised, because none needs " + "to be. The mutant is deliberately the UNFILTERED spelling: the filtered one is what broke, " + "but a pin rejects both identically and choosing the narrower case would suggest otherwise.", ), ) diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index ee5941be9..d3258447c 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -1,94 +1,85 @@ -"""The SPA vitest suite never runs inside an image build, and the image is gated on a job that does +"""The SPA vitest suite never runs inside an image build, and the image is gated on a step that does run it, unfiltered and non-advisory (ersatztv#887). WHAT THIS IS PROTECTING. `docker/Dockerfile`'s `web-build` stage is gitless twice over. It copies -only `web/` and `design-system/`, so it holds no `.git` — note this is a property of the STAGE, not -of the build context, which is the repository root (`context: .`) and whose `.dockerignore` does not -exclude `.git`; the directory is available to be copied and simply is not. And `node:22-bookworm-slim` -ships no git binary, which is why adding a `COPY .git` would still not help. Members of the SPA -suite need one or the other — the `web/src/api/*.guard.test.ts` -guards derive their file population from `git ls-files` and refuse to fall back to a directory walk +only `web/` and `design-system/`, so it holds no `.git` — a property of the STAGE, not of the build +context, which is the repository root (`context: .`) and whose `.dockerignore` does not exclude +`.git`: the directory is there to be copied and simply is not. And `node:22-bookworm-slim` ships no +git binary, which is why a `COPY .git` would not help either. Members of the SPA suite need one or +the other — `pageSizeCallSites.guard.test.ts` and `completeAnnotations.guard.test.ts` derive their +file population from `git ls-files` and refuse to fall back to a directory walk (`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds a -real temporary repository. Running the suite there anyway therefore requires a list of the members -that cannot run, and that list is a population nothing derives: ersatztv#883 added a third member -without updating a hand-written pair of `--exclude`s, and every image build failed from that commit -on. The red was structurally unreachable on a PR — `Build & push image (amd64)` carries +real temporary repository. + +Running the suite there anyway therefore requires a list of the members that cannot run, and that +list is a population nothing derives: ersatztv#883 added a third member without updating a +hand-written pair of `--exclude`s, and every image build failed from that commit on. The red was +structurally unreachable on a PR — `Build & push image (amd64)` carries `if: github.event_name != 'pull_request'` — so it landed on `main` and on the `v*` tag path, where a release cut fails at the image build and `:latest` stops being republished. -The fix removed the list rather than extending it. This guard is what keeps it removed, and what -holds up the safety argument that licensed removing it, because that argument is a claim about two -OTHER files and would otherwise rot silently in a comment. +## WHY THIS GUARD PINS TEXT INSTEAD OF PARSING IT -THE INVARIANT, in the three parts it actually has. The first two are the obvious ones; the third -came out of cold review, which pointed out that the first two together still certify a publish on -which the suite never ran: +The first three versions of this file asked "does this command RUN the suite, and can it FAIL?" of +arbitrary shell text. That predicate was wrong nine times across three review rounds, and every one +of the nine was the same mechanism: - * NEGATIVE — no Dockerfile runs the suite. Not "no `--exclude` appears": an unfiltered suite run - in a gitless stage fails just as hard, and a filtered one is the trap itself. The whole class is - excluded rather than a spelling of it. - * POSITIVE — every job that publishes an image built from a Dockerfile CARRYING THE SPA SOURCE is - transitively gated on a job that runs the suite. Without this the negative part is a hole: - deleting the suite run from the Dockerfile and deleting `needs: [test, ...]` would both be green. - * EFFECTIVE — that run is unfiltered AT BOTH LEVELS and is not advisory. A `needs:` edge onto a - job whose suite step carries `continue-on-error: true` blocks nothing, and a filter written into - `web/package.json`'s script body is invisible at the invocation site. Either one reproduces the - exact defect being removed while every other assertion here stays green. + * heredoc bodies — skipped as data, but BuildKit EXECUTES `RUN <::`. Whitespace +# is normalised (see `_normalise`) so a reflow is not a red, but the tokens are exact. +PINNED_STAGE_COMMANDS: dict[str, tuple[str, ...]] = { + "docker/Dockerfile::web-build": ( + "RUN npm ci", + # Lints, typechecks and BUILDS the SPA. It must not TEST it: this stage has neither a git + # checkout nor the git binary, and naming the specs that cannot run there is the + # hand-maintained population ersatztv#887 removed. + "RUN npm run lint && npm run typecheck && npm run build", + ), +} -# npm/npx flags taking a SEPARATE value token. A leading flag has to be consumed before the script -# name can be read: `npm --prefix web test` runs the suite, `npm ci --silent` does not, and the two -# are indistinguishable until the flags are gone. -RUNNER_VALUE_FLAGS = frozenset( - {"--prefix", "-C", "--workspace", "-w", "--loglevel", "--userconfig", "--package", "-p", "--call"} -) +# The step that runs the suite and gates the image, pinned whole. Its body being exact is what makes +# "unfiltered", "actually executes", and "cannot be suppressed" true without parsing any of it. +GATING_WORKFLOW = ".gitea/workflows/docker-build.yml" +GATING_JOB = "test" +GATING_STEP_NAME = "Test SPA" +GATING_STEP_RUN = '"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test\nnpm test -- --run' +# `docs_only` (ersatztv#416) and the #420 revalidate skip. Both are safe for a DIFFERENT reason and +# the difference is why this is pinned rather than pattern-matched: docs-only cannot ship an image at +# all, which is asserted below rather than assumed; the revalidate skip fires only on a tree +# byte-identical to a head that already carried a green combined status. +GATING_STEP_IF = "steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'" -# The `if:` conditions a suite step may carry. A HAND-WRITTEN MIRROR OF ONE STRING, and deliberately -# so: this step is the only thing standing between a red suite and a published image, so a change to -# when it runs must be a decision someone states, not a diff nobody reads. A third condition here -# reddens rather than silently narrowing the gate. Both current terms are documented — `docs_only` -# (ersatztv#416) cannot ship an image at all, and the revalidate skip (ersatztv#420) fires only on a -# tree byte-identical to an already-green head. -KNOWN_GATE_CONDITIONS = frozenset( - {"steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'"} -) +# The premise under the `docs_only` half of `GATING_STEP_IF`: the publish step is gated on it too, so +# the arm that skips the suite ships nothing. Round-1 review found this stated as fact with nothing +# checking it, one level out from the same defect. +PUBLISH_STEP_NAME = "Build and push" +PUBLISH_STEP_IF = "steps.detect.outputs.docs_only != 'true'" -_ENV_ASSIGNMENT = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*=") +# Substrings that mean "a human should look at this step". NOT a semantic predicate — the pinned step +# is allowed to contain them and every other step is not. +SUITE_MENTIONS = ("npm test", "npm run test", "vitest") -# Redirection operators, which `punctuation_chars` splits out as their own tokens. Neither they nor -# their targets are spec filters: cold review measured `npm test -- --run > test.log` reporting -# `['>', 'test.log']` as a narrowing, which is a false red on an ordinary log capture. -_REDIRECTION = re.compile(r"^\d*(?:>>|>&|&>|<<|>|<)$") -# `< f <<'EOF' … npm test … EOF` block counts as a suite run: cold -# review showed that satisfies both the anti-vacuity check and the gate assertion while the suite -# never executes. -_HEREDOC = re.compile(r"<<-?\s*(['\"]?)([A-Za-z_][A-Za-z0-9_]*)\1") +def _normalise(text: str) -> str: + """Collapse runs of whitespace so a reflow is not a red, keeping every token exact.""" + return " ".join(text.split()) def _canonical_action(uses: str) -> str: @@ -232,349 +158,6 @@ def _canonical_action(uses: str) -> str: return "/".join(segments[-2:]) if len(segments) >= 2 else ref -def tokenize(command: str) -> list[str]: - """Shell tokens, with the separators split out even when unspaced. - - `shlex.split` yields `lint&&npm` as ONE token for `npm run lint&&npm test`, which hides the - second command entirely. `punctuation_chars` mode is the stdlib answer and splits it correctly. - """ - try: - lexer = shlex.shlex(command, posix=True, punctuation_chars=True) - lexer.whitespace_split = True - return list(lexer) - except ValueError: - # An unbalanced quote is not a command anyone can reason about. The raw split keeps it - # VISIBLE to the detectors rather than silently dropping it. - return command.split() - - -def logical_commands(text: str) -> list[tuple[int, str]]: - """`(1-based line number, command)` per command line, with the three normalisations that matter. - - Leading-`#` lines are dropped, so a Dockerfile can explain the rule it enforces without tripping - it. Backslash continuations are joined, so a multi-line invocation is ONE command with its flags - attached — the shape the removed `--exclude` pair had; scanning raw lines would read - `npm test -- --run` as unfiltered and report it clean. And HEREDOC BODIES ARE SKIPPED, because - the shell reads them as data: a `npm test` line inside one never executes, and counting it makes - the gate assertion pass on a job that ran no suite. - """ - commands: list[tuple[int, str]] = [] - buffer = "" - start = 0 - terminator: str | None = None - for number, raw in enumerate(text.splitlines(), start=1): - if terminator is not None: - # `<<-` permits leading tabs on the terminator; plain `<<` does not, and accepting a - # stripped match either way only ends the skip EARLIER, never later. - if raw.strip() == terminator: - terminator = None - continue - if raw.lstrip().startswith("#"): - continue - if not buffer: - start = number - stripped = raw.rstrip() - if stripped.endswith("\\"): - buffer += stripped[:-1] + " " - continue - buffer += stripped - if buffer.strip(): - commands.append((start, buffer.strip())) - opener = _HEREDOC.search(buffer) - if opener: - terminator = opener.group(2) - buffer = "" - if buffer.strip(): - commands.append((start, buffer.strip())) - return commands - - -def command_segments(command: str, depth: int = 0) -> list[list[str]]: - """The shell segments of one command, as token lists, or `[]` if nothing executes. - - A leading Dockerfile instruction is consumed: `RUN` yields its command, every other instruction - yields nothing. A leading YAML step key is consumed too — `- run: npm test -- --run` is a - command, and roughly half this repo's `run:` steps are written on one line like that. - """ - tokens = tokenize(command) - if not tokens: - return [] - if tokens[0] == "-": - tokens = tokens[1:] - if tokens and tokens[0] == "run:": - tokens = tokens[1:] - if not tokens: - return [] - head = tokens[0] - if head.isupper() and head in DOCKERFILE_INSTRUCTIONS: - if head != "RUN": - return [] - tokens = tokens[1:] - - segments: list[list[str]] = [] - current: list[str] = [] - for token in tokens: - if token in SEPARATORS: - if current: - segments.append(current) - current = [] - continue - current.append(token) - if current: - segments.append(current) - return _expand_shell_payloads(segments, depth) - - -def _expand_shell_payloads(segments: list[list[str]], depth: int) -> list[list[str]]: - """Add the segments of any `sh -c ''` payload alongside the segment carrying it. - - The wrapper segment is KEPT as well as expanded: nothing here decides `sh` is uninteresting, only - that its argument is also a command. Bounded at three levels — past anything a Dockerfile or a - `run:` body writes — and hitting the cap costs a missed detection, never a false green about the - gate, which is derived from the job graph instead. - """ - if depth >= 3: - return segments - expanded: list[list[str]] = [] - for segment in segments: - expanded.append(segment) - head = _strip_prefixes(segment) - if len(head) < 3 or head[0] not in SHELLS: - continue - for offset, token in enumerate(head[1:-1], start=1): - # `-c`, and the login/interactive spellings carrying it: `-lc`, `-euxc`, `-ec`. - if token.startswith("-") and not token.startswith("--") and "c" in token.lstrip("-"): - expanded.extend(command_segments(head[offset + 1], depth + 1)) - break - return expanded - - -def _strip_prefixes(tokens: list[str]) -> list[str]: - """Drop leading environment assignments and command wrappers, plus any flags a wrapper takes.""" - index = 0 - while index < len(tokens): - token = tokens[index] - if _ENV_ASSIGNMENT.match(token): - index += 1 - continue - if token in WRAPPERS: - index += 1 - while index < len(tokens) and tokens[index].startswith("-"): - index += 1 - continue - break - return tokens[index:] - - -def _skip_runner_flags(tokens: list[str], index: int) -> int: - """Advance past leading `npm`/`npx` flags to the first positional argument. - - A flag in `RUNNER_VALUE_FLAGS` consumes the token after it unless written `--flag=value`. `--` is - consumed too: `npm exec -- vitest` is the documented spelling. - """ - while index < len(tokens): - word = tokens[index] - if word == "--": - index += 1 - continue - if not word.startswith("-"): - return index - index += 2 if (word.split("=", 1)[0] in RUNNER_VALUE_FLAGS and "=" not in word) else 1 - return index - - -def direct_vitest_args(tokens: list[str]) -> list[str] | None: - """The arguments of a segment invoking the vitest BINARY, or `None`. - - Covers `vitest …` (npm puts `node_modules/.bin` on PATH for a script body) and - `npx / npm exec / pnpm exec / yarn dlx vitest …`. Kept separate from the npm-script route below - so the `web/package.json` scan can use it without recursing through the table it is building. - """ - tokens = _strip_prefixes(tokens) - if not tokens: - return None - if tokens[0] == "vitest": - return tokens[1:] - if tokens[0] in {"npx", "pnpm", "npm", "yarn"}: - index = 1 - if tokens[0] != "npx": - index = _skip_runner_flags(tokens, index) - if index >= len(tokens) or tokens[index] not in {"exec", "dlx"}: - return None - index += 1 - index = _skip_runner_flags(tokens, index) - if index < len(tokens) and tokens[index] == "vitest": - return tokens[index + 1 :] - return None - - -def vitest_scripts() -> dict[str, list[str]]: - """`web/package.json` script name -> the narrowing arguments its BODY already carries. - - Two questions, one derivation, because they have one answer. Which scripts are the suite decides - what `npm test` means; what their bodies carry decides whether an invocation that looks - unfiltered at the call site actually is. `"test": "vitest --exclude x"` with a workflow saying - `npm test -- --run` is a filtered gating run that reads as clean from the workflow alone — the - exact defect being removed, one level down. Cold review found that hole; it is closed here - rather than noted. - """ - scripts = json.loads((REPO_ROOT / PACKAGE_JSON).read_text(encoding="utf-8")).get("scripts", {}) - named: dict[str, list[str]] = {} - for name, body in scripts.items(): - if not isinstance(body, str): - continue - for segment in command_segments(body): - args = direct_vitest_args(segment) - if args is not None: - named[str(name)] = _narrowing_in(args) - break - return named - - -def suite_args(tokens: list[str], scripts: Mapping[str, list[str]]) -> list[str] | None: - """The arguments of a segment that runs the SPA suite, or `None` if it does not run it. - - A script's own body arguments come first, so a filter written there is reported by the same - assertion that reports one written at the call site. - """ - direct = direct_vitest_args(tokens) - if direct is not None: - return direct - tokens = _strip_prefixes(tokens) - if not tokens or tokens[0] not in {"npm", "pnpm", "yarn", "bun"}: - return None - index = _skip_runner_flags(tokens, 1) - if index < len(tokens) and tokens[index] in {"run", "run-script"}: - index = _skip_runner_flags(tokens, index + 1) - if index >= len(tokens) or tokens[index] not in scripts: - return None - return list(scripts[tokens[index]]) + tokens[index + 1 :] - - -def suite_invocations(text: str, scripts: Mapping[str, list[str]] | None = None) -> list[tuple[int, str]]: - """Every command in `text` that runs the SPA vitest suite.""" - scripts = vitest_scripts() if scripts is None else scripts - found: list[tuple[int, str]] = [] - for number, command in logical_commands(text): - if any(suite_args(segment, scripts) is not None for segment in command_segments(command)): - found.append((number, command)) - return found - - -def _narrowing_in(args: list[str]) -> list[str]: - """The subset of `args` that restricts WHICH specs run. - - Redirections are consumed HERE rather than in a pre-pass, so a flag's value is claimed before the - file-descriptor rule can see it. As a pre-pass, `--exclude 2 > log` lost the `2`: the digit was - read as the descriptor of the following `>` and dropped with it, and the fault message then named - a filter without its value. Measured. - """ - found: list[str] = [] - index = 0 - while index < len(args): - argument = args[index] - index += 1 - if argument == "--": - continue - if _REDIRECTION.match(argument): - index += 1 # the operator's target - continue - if ( - argument.isdigit() - and index < len(args) - and _REDIRECTION.match(args[index]) - and not args[index][:1].isdigit() - ): - index += 2 # the operator and its target; `argument` was its descriptor - continue - base = argument.split("=", 1)[0] - if base in NARROWING_FLAGS: - found.append(argument) - if "=" not in argument and index < len(args) and not args[index].startswith("-"): - found.append(args[index]) - index += 1 - continue - if base in VALUE_TAKING_FLAGS and "=" not in argument: - index += 1 # its value is not a filter - continue - if argument.startswith("-"): - continue - found.append(argument) - return found - - -def narrowing_arguments(command: str, scripts: Mapping[str, list[str]] | None = None) -> list[str]: - """Everything restricting which specs `command` runs — from the call site AND the script body.""" - scripts = vitest_scripts() if scripts is None else scripts - found: list[str] = [] - for segment in command_segments(command): - args = suite_args(segment, scripts) - if args is not None: - found.extend(_narrowing_in(args)) - return found - - -def failure_suppressions(command: str, scripts: Mapping[str, list[str]], depth: int = 0) -> list[str]: - """The ways `command` stops a red suite from failing its step. - - A `needs:` edge onto a job whose suite command cannot fail blocks nothing, and there are three - spellings of that which every other assertion here reads as an ordinary run — cold review - measured all three green: `npm test … || true` swallows the status outright; `npm test | tee log` - reports `tee`'s status unless the body set `pipefail`; and `set +e` earlier in the body disarms - the step's own `set -e`. Reported by NAME rather than by a boolean so the message can say which. - """ - if depth >= 3: - return [] - found: list[str] = [] - # PER LOGICAL LINE, never over the whole body at once. A newline is not a shell separator, so - # tokenising a real two-line `run:` body — the `ci-step-ran.sh` marker followed by the suite - # command — merged both into ONE segment whose head was the marker script, and every `||`/`|` - # after the suite went unseen. Measured: three mutants passed that way while a single-line - # version of each was caught, which is exactly the shape of a test passing for the wrong reason. - for _, line in logical_commands(command): - tokens = tokenize(line) - if tokens and tokens[0] == "-": - tokens = tokens[1:] - if tokens and tokens[0] == "run:": - tokens = tokens[1:] - if tokens and tokens[0].isupper() and tokens[0] in DOCKERFILE_INSTRUCTIONS: - tokens = tokens[1:] if tokens[0] == "RUN" else [] - - current: list[str] = [] - # Whether the SUITE has run earlier in this `;`-delimited list. `&&`/`||` chain across a - # whole list, so `npm test && echo ok || true` runs `true` when the suite fails and exits 0 — - # the suite's failure is swallowed even though the `||` does not sit immediately after it. - # Looking only at the NEXT separator missed that; measured. A `;` (or a newline, since each - # line is walked separately) ends the list and resets, because a `||` in the next list is - # about the next command. - suite_in_list = False - for word in [*tokens, ";"]: - if word not in SEPARATORS: - current.append(word) - continue - if current: - head = _strip_prefixes(current) - if head[:2] == ["set", "+e"]: - found.append("`set +e` disarms the step's own `set -e`") - if suite_args(current, scripts) is not None: - suite_in_list = True - if word == "|": - found.append("a pipeline reports the LAST command's status") - if len(head) >= 3 and head[0] in SHELLS: - for offset, inner in enumerate(head[1:-1], start=1): - if inner.startswith("-") and not inner.startswith("--") and "c" in inner.lstrip("-"): - found.extend(failure_suppressions(head[offset + 1], scripts, depth + 1)) - break - if word == "||" and suite_in_list: - found.append("an `||` later in the same list swallows the exit status") - if word == "&" and suite_in_list: - found.append("`&` backgrounds the command, so its status is never awaited") - if word == ";": - suite_in_list = False - current = [] - return found - - def dockerfiles() -> list[str]: """Tracked Dockerfiles, by repo-relative path, in every spelling docker recognises. @@ -598,14 +181,54 @@ def _read(rel: str) -> str: return (REPO_ROOT / rel).read_text(encoding="utf-8") +def instructions(text: str) -> list[str]: + """The Dockerfile instructions, comments dropped and `\\` continuations joined. + + This is the whole of the file's syntax handling. It does not interpret what a `RUN` DOES — the + string is compared against a pin — so the shell traps that defeated three earlier versions + (heredocs, `#` inside a command, compound punctuation) cannot reach any decision here. + """ + joined: list[str] = [] + buffer = "" + for raw in text.splitlines(): + if raw.lstrip().startswith("#"): + continue + stripped = raw.rstrip() + if stripped.endswith("\\"): + buffer += stripped[:-1] + " " + continue + buffer += stripped + if buffer.strip(): + joined.append(_normalise(buffer)) + buffer = "" + if buffer.strip(): + joined.append(_normalise(buffer)) + return joined + + +def stages(text: str) -> list[tuple[str, list[str]]]: + """`(stage name, its instructions)` per build stage, in file order. + + A stage is delimited by `FROM`, which is the one Dockerfile construct with no shell content and + therefore nothing to misparse. An unnamed stage is keyed by its index so it is still checkable. + """ + found: list[tuple[str, list[str]]] = [] + for line in instructions(text): + if re.match(r"^FROM\b", line, re.I): + match = re.search(r"\bAS\s+(\S+)\s*$", line, re.I) + found.append((match.group(1) if match else f"", [])) + elif found: + found[-1][1].append(line) + return found + + def copies_spa_source(command: str) -> bool: """Does this ONE instruction bring the SPA source tree into the image? - FOUR spellings count, because the gate assertion SKIPS every Dockerfile answered False — a miss - here drops an image out of the check silently rather than reddening. `ADD` as well as `COPY`; - the JSON exec form `COPY ["web/", "/dest/"]`; a source without a trailing slash - (`COPY web /dest`); and `COPY . …`, which carries the tree without naming it. Cold review found - the first three answering False. + Every spelling counts, because the negative assertion SKIPS every stage answered False — a miss + here drops a stage out of the check silently rather than reddening. `ADD` as well as `COPY`; the + JSON exec form `COPY ["web/", "/dest/"]`, with or without a preceding flag; a source without a + trailing slash (`COPY web /dest`); and `COPY . …`, which carries the tree without naming it. A `COPY --from=` is not a context copy: its source is another stage, so `--from=build .` would not bring the repo in. Flags are dropped before the source arguments are read. @@ -615,15 +238,17 @@ def copies_spa_source(command: str) -> bool: return False if any(token.startswith("--from=") for token in tokens[1:]): return False - remainder = command[len(tokens[0]) :].strip() + remainder = " ".join(token for token in tokens[1:] if not token.startswith("--")).strip() if remainder.startswith("["): try: parsed = json.loads(remainder) except ValueError: - return False + # An exec-form array this cannot read is reported as CARRYING the source: that makes the + # stage pinned rather than skipped, which is the safe direction for an unknown. + return True arguments = [str(item) for item in parsed] if isinstance(parsed, list) else [] else: - arguments = [token for token in tokens[1:] if not token.startswith("--")] + arguments = remainder.split() if len(arguments) < 2: return False for source in arguments[:-1]: @@ -633,14 +258,14 @@ def copies_spa_source(command: str) -> bool: return False -def carries_spa_source(rel: str) -> bool: - """Does this Dockerfile bring the SPA source tree into the build? - - This is what makes a suite run there possible at all, and it separates `docker/Dockerfile` from - `docker/ci/Dockerfile`, which installs npm packages globally but never copies `web/`. Derived - from the copy instructions rather than from a list of Dockerfile paths. - """ - return any(copies_spa_source(command) for _, command in logical_commands(_read(rel))) +def spa_carrying_stages() -> list[tuple[str, str, list[str]]]: + """`(dockerfile, stage name, RUN commands)` for every stage carrying the SPA source.""" + found: list[tuple[str, str, list[str]]] = [] + for rel in dockerfiles(): + for name, lines in stages(_read(rel)): + if any(copies_spa_source(line) for line in lines): + found.append((rel, name, [line for line in lines if re.match(r"^RUN\b", line)])) + return found def _jobs(doc: object) -> dict[str, dict]: @@ -655,10 +280,6 @@ def _steps(job: dict) -> list[dict]: return [step for step in steps if isinstance(step, dict)] if isinstance(steps, list) else [] -def _run_bodies(job: dict) -> list[str]: - return [step["run"] for step in _steps(job) if isinstance(step.get("run"), str)] - - def _needs(job: dict) -> list[str]: needs = job.get("needs") if isinstance(needs, str): @@ -678,19 +299,15 @@ def _transitive_needs(job_id: str, jobs: dict[str, dict]) -> set[str]: return seen -def suite_steps() -> list[tuple[str, str, dict, dict]]: - """`(workflow rel path, job id, job, step)` for every workflow step that runs the SPA suite.""" - scripts = vitest_scripts() - found: list[tuple[str, str, dict, dict]] = [] - for path in workflow_files(): - rel = path.relative_to(REPO_ROOT).as_posix() - jobs = _jobs(yaml.safe_load(path.read_text(encoding="utf-8"))) - for job_id, job in jobs.items(): - for step in _steps(job): - body = step.get("run") - if isinstance(body, str) and suite_invocations(body, scripts): - found.append((rel, job_id, job, step)) - return found +def workflow_jobs(rel: str) -> dict[str, dict]: + return _jobs(yaml.safe_load(_read(rel))) + + +def find_step(rel: str, job_id: str, name: str) -> dict | None: + for step in _steps(workflow_jobs(rel).get(job_id, {})): + if str(step.get("name", "")).strip() == name: + return step + return None def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]: @@ -703,7 +320,7 @@ def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]: found: list[tuple[str, str, str, dict[str, dict]]] = [] for path in workflow_files(): rel = path.relative_to(REPO_ROOT).as_posix() - jobs = _jobs(yaml.safe_load(path.read_text(encoding="utf-8"))) + jobs = workflow_jobs(rel) for job_id, job in jobs.items(): for step in _steps(job): uses = step.get("uses") @@ -717,32 +334,56 @@ def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]: # ------------------------------------------------------------------------------------------------ -# The invariant +# NEGATIVE — the gitless stages run exactly what they are pinned to run # ------------------------------------------------------------------------------------------------ -def test_no_DOCKERFILE_runs_the_spa_suite() -> None: - """The negative part. A gitless stage runs no vitest suite, filtered or not.""" - scripts = vitest_scripts() +def test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands() -> None: + """Set equality on command TEXT, not a predicate over it. + + A suite run put back in ANY spelling — a heredoc, `sh -c`, `npm t`, `timeout npm test`, a + `#`-suffixed line — is simply not equal to the pin. That is the whole point: three earlier + versions asked what a command meant and were wrong nine times; this one asks whether it is the + command that was reviewed. + """ faults: list[str] = [] - for rel in dockerfiles(): - for number, command in suite_invocations(_read(rel), scripts): + seen: set[str] = set() + for rel, stage, commands in spa_carrying_stages(): + key = f"{rel}::{stage}" + seen.add(key) + pinned = PINNED_STAGE_COMMANDS.get(key) + if pinned is None: faults.append( - f"{rel}:{number} runs the SPA vitest suite in a stage that has neither a git " - f"CHECKOUT nor the git BINARY, so the suite's git-dependent members cannot run " - f"there: `{command.strip()}`. Excluding them by name is the defect this " - f"guard exists to prevent — that list is a population nothing derives, it went " - f"stale in ersatztv#883, and the resulting red is unreachable on a PR, so it lands " - f"on `main` and on the release tag. The suite runs in `docker-build.yml`'s `test` " - f"job, which gates the image; delete this invocation rather than filtering it " - f"(ersatztv#887)." + f"{key} carries the SPA source but has no entry in `PINNED_STAGE_COMMANDS`, so " + f"nothing checks what it runs. It runs: {commands}. Add the entry — and if any of " + f"those commands runs the vitest suite, that is the ersatztv#887 defect: the stage " + f"has no git checkout and no git binary, so the suite's git-dependent members " + f"cannot run there, and the red is unreachable on a PR." ) - assert not faults, "\n".join(faults) + continue + if tuple(commands) != pinned: + faults.append( + f"{key} does not run its pinned commands.\n pinned: {list(pinned)}\n actual: {commands}\n" + f" If this change is deliberate, update `PINNED_STAGE_COMMANDS` in the same commit " + f"and say why. If it added a vitest run, do not: this stage cannot run the suite " + f"(ersatztv#887), and excluding the specs that fail there is the hand-maintained " + f"population that broke every image build in ersatztv#883." + ) + for orphan in sorted(set(PINNED_STAGE_COMMANDS) - seen): + faults.append( + f"`PINNED_STAGE_COMMANDS` names {orphan}, which no longer exists or no longer carries " + f"the SPA source. A pin for a stage that is not checked is a sentence nobody is reading." + ) + assert not faults, "\n\n".join(faults) + + +# ------------------------------------------------------------------------------------------------ +# POSITIVE — the image is gated on the job holding the pinned step +# ------------------------------------------------------------------------------------------------ def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> None: - """The positive part, without which deleting `needs:` is silently green.""" - scripts = vitest_scripts() + """Without this, deleting `needs:` is silently green.""" known = set(dockerfiles()) faults: list[str] = [] for rel, job_id, dockerfile, jobs in publishing_jobs(): @@ -753,119 +394,148 @@ def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> f"UNCHECKED — fix the path or widen `dockerfiles()`." ) continue - if not carries_spa_source(dockerfile): + if not any(stage_rel == dockerfile for stage_rel, _, _ in spa_carrying_stages()): continue gates = _transitive_needs(job_id, jobs) - runners = sorted( - gate for gate in gates if any(suite_invocations(body, scripts) for body in _run_bodies(jobs.get(gate, {}))) - ) - if not runners: + if GATING_JOB not in gates: faults.append( f"{rel}: job `{job_id}` publishes an image built from `{dockerfile}`, which carries " - f"the SPA source, but none of the jobs it transitively `needs:` ({sorted(gates)}) " - f"runs the SPA suite. Since ersatztv#887 the image build deliberately does not run " - f"the suite itself, so this `needs:` edge is the ONLY thing standing between a red " - f"suite and a published image." + f"the SPA source, but does not transitively `needs:` `{GATING_JOB}` — the job " + f"holding the pinned suite step. It needs: {sorted(gates)}. Since ersatztv#887 the " + f"image build deliberately does not run the suite itself, so this `needs:` edge is " + f"the ONLY thing standing between a red suite and a published image." ) assert not faults, "\n".join(faults) -def test_every_spa_suite_invocation_is_UNFILTERED() -> None: - """One list, and it is empty — at the call site AND inside the npm script it runs. +# ------------------------------------------------------------------------------------------------ +# EFFECTIVE — the gating step really runs, really fails, and really runs everything +# ------------------------------------------------------------------------------------------------ - Driven off the YAML walk rather than the file text, so it reads exactly what the gate assertion - reads. `test_the_yaml_walk_finds_every_suite_invocation_the_TEXT_does` is what stops the walk - from silently narrowing. + +def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None: + """Its body, its `if:`, and the absence of `continue-on-error` — all pinned. + + Pinning the BODY is what makes "unfiltered" and "not suppressed" true without parsing: a + `--exclude`, a `|| true`, a pipe, a `set +e`, a `true || npm test` that never executes, are each + a different string from the pin. Three earlier versions tried to recognise those individually and + missed `true || npm test` and `continue-on-error: ${{ … }}` among others. """ - scripts = vitest_scripts() + step = find_step(GATING_WORKFLOW, GATING_JOB, GATING_STEP_NAME) + assert step is not None, ( + f"{GATING_WORKFLOW}: job `{GATING_JOB}` has no step named {GATING_STEP_NAME!r}. That step is " + f"the only thing running the SPA suite, and the image is gated on the job holding it." + ) + faults: list[str] = [] - for rel, job_id, _, step in suite_steps(): - for _, command in suite_invocations(step["run"], scripts): - narrowing = narrowing_arguments(command, scripts) - if narrowing: - faults.append( - f"{rel}: job `{job_id}` runs the SPA suite but restricts which specs run " - f"({narrowing}): `{command.strip()}`. The restriction may be written at this " - f"call site or inside the `{PACKAGE_JSON}` script it runs — both count, and a " - f"body-level filter is the one that reads as clean from the workflow. The " - f"gating run must be the WHOLE suite: a filtered one re-creates the " - f"hand-maintained population ersatztv#887 removed." - ) + body = step.get("run") + if not isinstance(body, str) or _normalise(body) != _normalise(GATING_STEP_RUN): + faults.append( + f"the gating step's `run:` is not the pinned command.\n pinned: {GATING_STEP_RUN!r}\n" + f" actual: {body!r}\n Anything but this — a spec filter, a `|| true`, a pipe, a " + f"`set +e`, a suite on the right of a `||` that may never execute — changes whether a " + f"red suite blocks the image. Update the pin deliberately, in the same commit, with a " + f"reason." + ) + + condition = step.get("if") + if condition is None or _normalise(str(condition)) != _normalise(GATING_STEP_IF): + faults.append( + f"the gating step's `if:` is not the pinned condition.\n pinned: {GATING_STEP_IF!r}\n" + f" actual: {condition!r}\n A third term here lets the suite not run at all while the " + f"image still publishes." + ) + + # ANY spelling, not two literals. `continue-on-error: ${{ … }}` is the idiomatic conditional form + # and a check comparing against `True`/`"true"` waves it through — a presence test cannot see + # polarity, and this one is fail-OPEN in exactly the direction that matters. + job = workflow_jobs(GATING_WORKFLOW).get(GATING_JOB, {}) + for scope, holder in (("step", step), ("job", job)): + if "continue-on-error" in holder and holder["continue-on-error"] not in (False, "false"): + faults.append( + f"the gating {scope} carries `continue-on-error: {holder['continue-on-error']!r}`, " + f"so a red suite reports success and the `needs:` edge that gates the image blocks " + f"nothing. Only absent, `false` or `False` is acceptable here." + ) + + if "if" in job: + faults.append( + f"the job `{GATING_JOB}` carries a job-level `if:` ({job['if']!r}). None did when " + f"ersatztv#887 made this `needs:` edge the only gate, and a job that does not run cannot " + f"gate anything — Gitea reports it `skipped`, and what that does to a dependent job is " + f"not something this repo relies on." + ) assert not faults, "\n".join(faults) -def test_the_gating_suite_run_is_NOT_ADVISORY() -> None: - """A `needs:` edge onto a job that cannot fail blocks nothing. +def test_the_DOCS_ONLY_arm_cannot_publish_an_image() -> None: + """The premise under half of `GATING_STEP_IF`, asserted rather than assumed. - `continue-on-error: true` on the step or the job turns a red suite into a green job, `build` - publishes, and every other assertion here stays green — the gate certifies a publish on which - the suite failed. The `if:` is pinned for the same reason one step further out: a third - condition would let the suite not run at all. + The gating step is allowed to skip when `docs_only` is true. That is only safe because the + publish step skips on the same condition, so the arm ships nothing. Round-1 review found this + stated as fact in the prose with nothing checking it — the same shape as the defect one level in. """ - scripts = vitest_scripts() - faults: list[str] = [] - for rel, job_id, job, step in suite_steps(): - for scope, holder in (("step", step), ("job", job)): - if holder.get("continue-on-error") in (True, "true"): - faults.append( - f"{rel}: the SPA suite {scope} in job `{job_id}` carries " - f"`continue-on-error: true`, so a red suite reports success and the `needs:` " - f"edge that gates the image blocks nothing." - ) - # The JOB's `if:` as well as the step's: a job that does not run reports `skipped`, and the - # question of what a skipped `needs:` does to `build` is one this repo deliberately does not - # rely on (see docker-build.yml's header on `if:`-skipped required contexts). - if job.get("if") is not None: - faults.append( - f"{rel}: the job `{job_id}` that runs the SPA suite carries a job-level `if:` " - f"({str(job['if']).strip()!r}). None did when ersatztv#887 made this `needs:` edge " - f"the only gate, and a job that does not run cannot gate anything." - ) - condition = step.get("if") - if condition is not None and str(condition).strip() not in KNOWN_GATE_CONDITIONS: - faults.append( - f"{rel}: the SPA suite step in job `{job_id}` carries an `if:` this guard does not " - f"know: {str(condition).strip()!r}. Since ersatztv#887 that step is the only thing " - f"standing between a red suite and a published image, so a change to WHEN it runs " - f"is a decision to state rather than a diff to skim — add the new condition to " - f"`KNOWN_GATE_CONDITIONS` with its reason." - ) - for suppression in failure_suppressions(step["run"], scripts): - faults.append( - f"{rel}: the SPA suite command in job `{job_id}` cannot fail the step — " - f"{suppression}. That is `continue-on-error` by another spelling: the job reports " - f"success over a red suite and the image publishes." - ) - assert not faults, "\n".join(faults) + step = find_step(GATING_WORKFLOW, "build", PUBLISH_STEP_NAME) + assert step is not None, f"{GATING_WORKFLOW}: job `build` has no step named {PUBLISH_STEP_NAME!r}." + condition = step.get("if") + assert condition is not None and _normalise(str(condition)) == _normalise(PUBLISH_STEP_IF), ( + f"the publish step's `if:` is {condition!r}, not the pinned {PUBLISH_STEP_IF!r}. The gating " + f"step is permitted to skip on `docs_only`, and that is safe ONLY because this step skips " + f"on it too. Without this, a docs-only push to `main` skips the SPA suite and publishes an " + f"image anyway." + ) -def test_no_run_BODY_builds_or_pushes_an_image() -> None: - """The publishing-job derivation keys on the action; assert nothing publishes around it. +def test_only_the_PINNED_step_mentions_the_suite() -> None: + """A substring sweep, deliberately not a semantic predicate. - Both halves would agree on zero if an image were built by a raw `docker push` in a shell body, - so this reports the shape rather than leaving it uncovered. + Every earlier version tried to decide whether a command RUNS the suite and was wrong nine times. + This asks the much weaker question — does any OTHER step's shell body mention it at all — whose + failure mode is a false red asking a human to look, never a false green. """ - stray: list[str] = [] + strays: list[str] = [] for path in workflow_files(): rel = path.relative_to(REPO_ROOT).as_posix() - jobs = _jobs(yaml.safe_load(path.read_text(encoding="utf-8"))) - for job_id, job in jobs.items(): - for body in _run_bodies(job): - for number, command in logical_commands(body): - for segment in command_segments(command): - head = _strip_prefixes(segment) - publishes = head[:3] == ["docker", "buildx", "build"] or ( - len(head) >= 2 and head[0] == "docker" and head[1] in {"push", "build"} - ) - if publishes: - stray.append(f"{rel}: job `{job_id}`, line {number} of a run body: `{command.strip()}`") - assert not stray, ( - f"image build/push outside `{PUBLISH_ACTION}`: {stray}. The publishing-job population above " - f"is derived from that action, so these jobs are NOT checked for the suite gate — extend " - f"`publishing_jobs()` before adding one." + for job_id, job in workflow_jobs(rel).items(): + for step in _steps(job): + body = step.get("run") + if not isinstance(body, str): + continue + if ( + rel == GATING_WORKFLOW + and job_id == GATING_JOB + and str(step.get("name", "")).strip() == GATING_STEP_NAME + ): + continue + hit = [mention for mention in SUITE_MENTIONS if mention in body] + if hit: + strays.append(f"{rel} :: job `{job_id}` step {str(step.get('name', '?'))!r} mentions {hit}") + assert not strays, ( + "these steps mention the SPA suite and are not the pinned gating step:\n " + + "\n ".join(strays) + + "\n\nThis is a SUBSTRING sweep, not a claim that they run it — look, and either move the " + "run into the pinned step or record why this one is not a second, unpinned gate." ) +def test_no_DOCKERFILE_outside_a_pinned_stage_mentions_the_suite() -> None: + """The same weak sweep over Dockerfiles, covering stages that carry no SPA source. + + A stage without `web/` has no suite to run, so `PINNED_STAGE_COMMANDS` does not cover it. This + reports a mention there anyway rather than leaving the class unexamined. + """ + strays: list[str] = [] + for rel in dockerfiles(): + for name, lines in stages(_read(rel)): + if f"{rel}::{name}" in PINNED_STAGE_COMMANDS: + continue + for line in lines: + hit = [mention for mention in SUITE_MENTIONS if mention in line] + if hit: + strays.append(f"{rel} :: stage `{name}`: {line!r} mentions {hit}") + assert not strays, "these unpinned Dockerfile stages mention the SPA suite:\n " + "\n ".join(strays) + + # ------------------------------------------------------------------------------------------------ # Anti-vacuity and negative controls # ------------------------------------------------------------------------------------------------ @@ -876,19 +546,13 @@ def test_the_populations_are_NOT_empty() -> None: assert dockerfiles(), "`git ls-files` reported no Dockerfile — the derivation is broken, not the repo." assert workflow_files(), f"`git ls-files` reported no {WORKFLOW_DIR}/*.yml — the derivation is broken." - scripts = vitest_scripts() - assert scripts, ( - f"no script in {PACKAGE_JSON} was read as running vitest, so every assertion in this file " - "measures an empty set: no command can be recognised as a suite run. `direct_vitest_args` " - "stopped matching the script body, or the scripts moved." - ) - - spa_carrying = [rel for rel in dockerfiles() if carries_spa_source(rel)] - assert spa_carrying, ( - f"none of {dockerfiles()} was read as carrying the SPA source, so " - "`test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite` skipped every " - "job and asserted nothing. `copies_spa_source` stopped matching the copy it derives." + carrying = spa_carrying_stages() + assert carrying, ( + f"no stage in {dockerfiles()} was read as carrying the SPA source, so the negative assertion " + "checked nothing and the positive one skipped every job. `copies_spa_source` or `stages` " + "stopped matching." ) + assert PINNED_STAGE_COMMANDS, "the pin table is empty; the negative assertion asserts nothing." published = publishing_jobs() assert published, ( @@ -896,263 +560,39 @@ def test_the_populations_are_NOT_empty() -> None: "population. Either the repo stopped publishing images or `_canonical_action` stopped " "recognising the action." ) - - assert suite_steps(), ( - "no workflow STEP runs the SPA suite. The image gate then rests on a job that runs nothing, " - "and both the unfiltered and non-advisory assertions iterate over an empty set." + assert any(dockerfile in {rel for rel, _, _ in carrying} for _, _, dockerfile, _ in published), ( + "no publishing job builds an SPA-carrying Dockerfile, so the gate assertion is vacuous." ) -def test_the_SCRIPT_derivation_separates_vitest_from_playwright() -> None: - """Pinned in both directions: this table is what every other assertion here keys on. +def test_the_STAGE_SPLIT_finds_the_real_stages() -> None: + """`stages()` is the only Dockerfile syntax this file interprets; pin what it returns. - `test` in and `test:ui-e2e` out is the whole reason there is no exemption list. If the - derivation drifted to include `test:ui-e2e`, the guard would demand an unfiltered gate on a - Playwright run; if it drifted to exclude `test`, nothing in the repo would be recognised as a - suite run and every assertion above would pass vacuously — `test_the_populations_are_NOT_empty` - asserts only that the table is non-empty, so a wrong-but-populated table is caught HERE and - nowhere else. - - The two names are hardcoded, which the module docstring argues against for the DERIVATION and - not for this pin: renaming the script produces a false RED here (measured — the derivation - itself adapts and every gate assertion stays correct), never a silent pass. A rename is then a - two-line edit someone makes deliberately, which is the intended cost. + A split that stopped recognising `FROM … AS name` would put every instruction in one bucket, the + pin key would never match, and `test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands` + would report an orphan pin — loud, but for the wrong reason. Pinned so the cause is legible. """ - scripts = vitest_scripts() - assert "test" in scripts, scripts - assert "test:ui-e2e" not in scripts, scripts - assert "build" not in scripts and "lint" not in scripts, scripts - assert scripts["test"] == [], ( - f"the `test` script body already narrows the suite ({scripts['test']}). That is the " - "call-site-invisible form of the defect ersatztv#887 removed." - ) + names = [name for name, _ in stages(_read("docker/Dockerfile"))] + assert names[:3] == ["dotnet-runtime", "web-build", "runtime-base"], names + assert [stage for _, stage, _ in spa_carrying_stages()] == ["web-build"], spa_carrying_stages() -def test_a_filter_inside_the_npm_SCRIPT_BODY_is_seen() -> None: - """The call site can read as clean while the script it runs is filtered. +def test_the_SPA_CARRYING_predicate_reads_every_COPY_SPELLING() -> None: + """Pinned both ways: every stage answered False is SKIPPED by the negative assertion. - `"test": "vitest --exclude x"` with a workflow saying `npm test -- --run` is a filtered gating - run that no assertion reading only the workflow can see. Driven through the real functions with - a synthetic table, so the mechanism is exercised rather than described. + The synthetic spellings were MEASURED False by cold review before this was widened; a new stage + using any of them would have gone unpinned and unchecked. """ - filtered = {"test": ["--exclude", "src/api/x.guard.test.ts"]} - assert narrowing_arguments("npm test -- --run", filtered) == ["--exclude", "src/api/x.guard.test.ts"] - assert narrowing_arguments("npm test -- --run", {"test": []}) == [] - # The two compose: a body filter plus a call-site filter reports both. - both = narrowing_arguments("npm test -- --run --project=spa", filtered) - assert both == ["--exclude", "src/api/x.guard.test.ts", "--project=spa"], both - # And the derivation really reads a body filter out of package.json rather than assuming none. - assert _narrowing_in(["--exclude", "a.ts"]) == ["--exclude", "a.ts"] - - -def test_the_detector_recognises_the_SPELLINGS_it_claims_to() -> None: - """Negative control on the predicates, driven directly rather than through the tree. - - The mutation proof drives the real Dockerfile; this covers the spellings the tree does not - contain, so a detector that quietly stopped matching one is caught without waiting for someone - to write it. Every entry marked MEASURED was returning the wrong answer before the clause that - handles it was added — cold-review findings, not hypotheticals. - """ - scripts: dict[str, list[str]] = {"test": [], "test:unit": []} - for command in ( - "npm test -- --run", - "npm run test", - "npm run test:unit", - "npx vitest run", - "npm exec vitest run", - "vitest run", - "cd web && npm test -- --run", - "npm run lint && npm test -- --run && npm run build", - "RUN npm test -- --run", - "env CI=1 npm test -- --run", - # The one-line YAML step form, which roughly half this repo's `run:` steps use. - "run: npm test -- --run", - "- run: npm test -- --run", - # MEASURED False before `tokenize` used punctuation_chars: plain `shlex.split` yields the - # single token `lint&&npm`, hiding the second command entirely. - "npm run lint&&npm test -- --run", - # MEASURED False before `_skip_runner_flags`. - "npm --prefix web test -- --run", - "npm --prefix=web test -- --run", - "npm -s test -- --run", - "npm --loglevel warn run test", - "npx -p vitest vitest run", - "npm exec -- vitest run", - # MEASURED False before `_expand_shell_payloads` / the `xargs` wrapper. - "sh -c 'npm test -- --run'", - "bash -lc 'npm test -- --run'", - "sh -euxc 'cd web && npm test -- --run'", - "xargs npm test -- --run", - ): - assert suite_invocations(command, scripts), command - - for command in ( - "dotnet test --configuration Release", - "npm ci", - "npm run build", - "npm run lint", - "npm run typecheck", - "npm run test:ui-e2e", # not in `scripts`, i.e. not a vitest script - "playwright test", - # Prose ABOUT the suite is not a run of it. An earlier version of this guard reported both, - # which is a false red in exactly the files that explain the rule. - "echo the vitest suite does not run here", - "echo skipping npm test in this stage", - # A non-RUN Dockerfile instruction does not execute during a build. - "CMD npm test -- --run", - "COPY vitest.config.ts ./", - # Flag-skipping must not turn a non-suite npm command into one. - "npm --prefix web ci", - "npm --prefix web run build", - "bash scripts/some-script.sh", # a script is not followed; stated as a residual - ): - assert not suite_invocations(command, scripts), command - - assert narrowing_arguments("npm test -- --run", scripts) == [] - assert narrowing_arguments("npm test -- --run --reporter verbose", scripts) == [] - assert narrowing_arguments("npm test -- --run --exclude 'src/api/x.guard.test.ts'", scripts) == [ - "--exclude", - "src/api/x.guard.test.ts", - ] - assert narrowing_arguments("npm test -- --run --project=spa", scripts) == ["--project=spa"] - assert narrowing_arguments("npm test -- --run src/api", scripts) == ["src/api"] - assert narrowing_arguments("npm test -- --run --changed", scripts) == ["--changed"] - # A filter still reads as a filter through a wrapper and through leading runner flags. - assert narrowing_arguments("sh -c 'npm test -- --run --exclude a.ts'", scripts) == ["--exclude", "a.ts"] - assert narrowing_arguments("npm --prefix web test -- --run --exclude a.ts", scripts) == ["--exclude", "a.ts"] - - -def test_a_SUPPRESSED_failure_is_advisory_by_another_spelling() -> None: - """`continue-on-error` is the declarative form; these are the shell forms of the same thing. - - All three were MEASURED green — the gate certified a publish over a suite that could not fail. - Driven directly here so the mechanism is covered without waiting for someone to write it into a - workflow. - """ - scripts: dict[str, list[str]] = {"test": []} - # A REAL step body: the `ci-step-ran.sh` marker line, then the suite. Tokenising the two lines - # together merged them into one segment and missed all three suppressions — pinned here so the - # single-line form can never again stand in for the shape the workflow actually has. - marker = '"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test\n' - assert failure_suppressions(marker + "npm test -- --run || true", scripts) - assert failure_suppressions(marker + "npm test -- --run | tee /tmp/spa.log", scripts) - assert failure_suppressions(marker + "set +e\nnpm test -- --run", scripts) - assert failure_suppressions(marker + "sh -c 'npm test -- --run || true'", scripts) - # And each still reports on its own. - assert failure_suppressions("npm test -- --run || true", scripts) - assert failure_suppressions("npm test -- --run | tee /tmp/spa.log", scripts) - assert failure_suppressions("set +e\nnpm test -- --run", scripts) - # A `||` LATER in the same list still swallows it: if the suite fails, the `&&` right-hand side - # is skipped and the `||` right-hand side runs, so the list exits 0. Measured missed before the - # detector became list-scoped rather than next-separator-scoped. - assert failure_suppressions("npm test -- --run && echo ok || true", scripts) - assert failure_suppressions("npm test -- --run && npm run build || true", scripts) - assert failure_suppressions("npm test -- --run &", scripts) - assert failure_suppressions("( npm test -- --run ) || true", scripts) - # An ordinary run, and a `||` on something that is NOT the suite, are both clean. - assert failure_suppressions(marker + "npm test -- --run", scripts) == [] - assert failure_suppressions("npm run lint && npm test -- --run", scripts) == [] - assert failure_suppressions("npm run lint || true\nnpm test -- --run", scripts) == [] - # A `;` ends the list, so a `||` after it is about the NEXT command, not the suite. - assert failure_suppressions("npm test -- --run; npm run other || true", scripts) == [] - - -def test_a_REDIRECTION_is_not_a_spec_filter() -> None: - """Capturing a log is not narrowing the suite, and reporting it as one is a false red. - - MEASURED reporting `['>', 'test.log']` before `_strip_redirections` existed. The `2>&1` form - matters separately: it tokenises into three pieces, two of which are bare tokens that the - positional-filter rule would otherwise both report. - """ - scripts: dict[str, list[str]] = {"test": []} - assert narrowing_arguments("npm test -- --run > test.log", scripts) == [] - assert narrowing_arguments("npm test -- --run >> test.log", scripts) == [] - assert narrowing_arguments("npm test -- --run 2>&1", scripts) == [] - assert narrowing_arguments("npm test -- --run > test.log 2>&1", scripts) == [] - # A real filter beside a redirection is still reported, INCLUDING a numeric value: stripping - # redirections as a pre-pass ate the `2` in `--exclude 2 > log` by reading it as a descriptor. - assert narrowing_arguments("npm test -- --run --exclude a.ts > log", scripts) == ["--exclude", "a.ts"] - assert narrowing_arguments("npm test -- --run --exclude 2 > log", scripts) == ["--exclude", "2"] - assert narrowing_arguments("npm test -- --run 2 --exclude a.ts", scripts) == ["2", "--exclude", "a.ts"] - - -def test_the_flags_that_move_the_COLLECTED_SET_count_as_filters() -> None: - """`--root` and `--config` change which specs vitest collects, without naming one. - - Both were MEASURED reporting `[]` while classified as benign value-taking flags — a narrowed - gating run reading as clean, which is the defect itself in its least visible spelling. - """ - scripts: dict[str, list[str]] = {"test": []} - assert narrowing_arguments("npm test -- --run --root src/api", scripts) == ["--root", "src/api"] - assert narrowing_arguments("npm test -- --run --config vitest.subset.ts", scripts) == [ - "--config", - "vitest.subset.ts", - ] - # A flag that genuinely only shapes OUTPUT stays benign. - assert narrowing_arguments("npm test -- --run --reporter verbose", scripts) == [] - - -def test_a_HEREDOC_body_is_data_rather_than_a_command() -> None: - """A `npm test` line inside a heredoc never executes, and counting it is a false green. - - Cold review's scenario: write the suite command into a file with `cat > f <<'EOF' … EOF`. The - step is green having run nothing, and a guard counting the heredoc line finds both its - non-vacuous invocation and its suite-running `needs:` gate in a job that ran no tests. - """ - scripts: dict[str, list[str]] = {"test": []} - for opener, terminator in (("<<'EOF'", "EOF"), ("< /tmp/run.sh {opener}\nnpm test -- --run\n{terminator}\n" - assert not suite_invocations(body, scripts), body - # The command AFTER the terminator is live again — the skip must end, not swallow the rest. - resumed = "cat > /tmp/run.sh <<'EOF'\nnpm test -- --run\nEOF\nnpm test -- --run\n" - found = suite_invocations(resumed, scripts) - assert len(found) == 1 and found[0][0] == 4, found - - -def test_a_COMMENT_is_inert_but_only_a_LEADING_hash_is() -> None: - """`docker/Dockerfile` names the rule it enforces; that must not trip the rule. - - The second half is the one that matters: a `#` mid-line does not make what precedes it a - comment, and treating it as one would let a real invocation hide behind a trailing note. - """ - scripts: dict[str, list[str]] = {"test": []} - assert not suite_invocations("# npm test -- --run\n", scripts) - assert not suite_invocations(" # the vitest suite is not run in this stage\n", scripts) - assert suite_invocations("RUN npm test -- --run # left over from before\n", scripts) - - -def test_the_continuation_JOIN_keeps_flags_with_their_command() -> None: - """A multi-line invocation is one command; its flags must not read as separate lines. - - This is the exact shape the removed exclusion pair had, so a guard that scanned raw lines would - see `npm test -- --run` as unfiltered and report it clean. - """ - scripts: dict[str, list[str]] = {"test": []} - multiline = "RUN npm test -- --run \\\n --exclude 'src/api/a.test.ts' \\\n --exclude 'b.test.ts'\n" - found = suite_invocations(multiline, scripts) - assert len(found) == 1, found - assert narrowing_arguments(found[0][1], scripts) == ["--exclude", "src/api/a.test.ts", "--exclude", "b.test.ts"] - - -def test_the_spa_carrying_predicate_reads_every_COPY_SPELLING() -> None: - """Pinned both ways, because the gate assertion is SKIPPED for everything it answers False for. - - A predicate that drifted to False for `docker/Dockerfile` would make the positive part vacuous - while `test_the_populations_are_NOT_empty` still passed on some other file. The synthetic - spellings were all MEASURED False before this clause was widened — a new Dockerfile using any of - them would have published without a gate. - """ - assert carries_spa_source("docker/Dockerfile") - assert not carries_spa_source("docker/ci/Dockerfile") - for spelling in ( 'COPY ["web/", "/source/web/"]', + 'COPY --chown=node:node ["web/", "/source/web/"]', "COPY web /source/web", "ADD web/ /source/web/", "COPY web/. ./web/", "COPY . /source/", "COPY ./ /source/", "COPY --chown=node:node web/ /source/web/", + "COPY web/package*.json ./web/", ): assert copies_spa_source(spelling), spelling @@ -1166,27 +606,41 @@ def test_the_spa_carrying_predicate_reads_every_COPY_SPELLING() -> None: assert not copies_spa_source(spelling), spelling -def test_the_yaml_walk_finds_every_suite_invocation_the_TEXT_does() -> None: - """The walk and a plain text scan must agree per file, or the walk has a blind spot. +def test_the_INSTRUCTION_JOIN_survives_what_defeated_the_PARSER() -> None: + """The shapes that broke three earlier versions must not break the pin comparison. - Every assertion above reads `jobs..steps[].run`. A step list the walk cannot reach makes a - suite run structurally invisible while the guard still reports a clean pass, and absence of - output is not evidence. The two shapes that would escape BOTH halves — a local composite action - and a `jobs..uses:` reusable workflow — are asserted not to exist AT ALL by - `test_workflow_persist_credentials.py`, repo-wide and for the same reason; they are not - re-asserted here, because two guards on one condition mask each other (ersatztv#685). + None of these needs interpreting — they only need to arrive at the comparison intact, so that a + stage running them is NOT EQUAL to its pin. Each was a measured false green when this file tried + to decide what the command meant. """ - scripts = vitest_scripts() - disagreements: list[str] = [] - for path in workflow_files(): - text = path.read_text(encoding="utf-8") - jobs = _jobs(yaml.safe_load(text)) - walked = sum(len(suite_invocations(body, scripts)) for job in jobs.values() for body in _run_bodies(job)) - scanned = len(suite_invocations(text, scripts)) - if walked != scanned: - disagreements.append( - f"{path.name}: the jobs..steps walk found {walked} SPA-suite invocation(s) but " - f"the text scan found {scanned}. The walk is what every assertion above runs over, " - f"so the difference is suite runs this guard cannot see — widen `_run_bodies`." - ) - assert not disagreements, "\n".join(disagreements) + heredoc = "RUN < None: + """The nine shapes that defeated the predicate, run against the pin instead. + + Not one of them is equal to `RUN npm run lint && npm run typecheck && npm run build`, which is + the entire argument for replacing the mechanism: the pin does not need to recognise a spelling to + reject it. + """ + pinned = PINNED_STAGE_COMMANDS["docker/Dockerfile::web-build"] + for spelling in ( + "RUN < Date: Sun, 30 Aug 2026 12:15:42 +0200 Subject: [PATCH 05/16] =?UTF-8?q?fix(887):=20close=20what=20a=20pin=20cann?= =?UTF-8?q?ot=20see=20=E2=80=94=20meaning=20changed=20without=20changing?= =?UTF-8?q?=20the=20text?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A pin rejects any command it does not equal, so the remaining attack is to change what the pinned text MEANS. Four such vectors, found by attacking the new mechanism rather than reading it; the first was MEASURED escaping and the rest are the same class: * A NEW stage taking the SPA source across with `COPY --from=web-build /source/web` and running the suite there. `copies_spa_source` excluded every `--from=` copy on the grounds that a stage copy is not a context copy — true, but it can still carry the SOURCE TREE from a stage that has it. The receiving stage was therefore unpinned and unchecked, which is exactly the false-NEGATIVE direction this file's own residual warns about. A stage copy now counts when its SOURCE has a whole `web` path segment, which keeps the built-artifact copy this repo actually makes (`/source/ErsatzTV/wwwroot/app/.`) correctly out. * `working-directory:` moved off `web` — `npm test` somewhere else runs a different package, or none, with the pinned command text unchanged. Now pinned. * `defaults.run.shell` changed from `bash` to `sh`. `bash` here is `bash -e`, which is what makes a failing command fail the step; changing it changes whether a red suite blocks the image without touching the step at all. Now pinned. * A `SHELL` instruction in a pinned stage, which redefines what every later `RUN` executes. Refused outright rather than modelled — there is none in this repo, so the honest move is to reject the construct, not to reason about a replacement interpreter. Battery 41 -> 45, 0 missed; the stage-copy mutant reddens three assertions. The residual list gains the two cases that remain in this class and are NOT covered: a stage copy that RENAMES the tree on the way in (no `web` segment in its source), and an `ENV` altering `PATH` so a pinned `RUN` resolves a different `npm`. Naming them is the point — the previous rounds' residual lists read as exhaustive while omitting the largest holes. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docs/guard-inventory.md | 2 +- ...est_image_build_delegates_the_spa_suite.py | 63 ++++++++++++++++++- 2 files changed, 62 insertions(+), 3 deletions(-) diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index dcba07771..c263afbf3 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN <` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index d3258447c..41401f641 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -126,6 +126,12 @@ GATING_STEP_RUN = '"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test # all, which is asserted below rather than assumed; the revalidate skip fires only on a tree # byte-identical to a head that already carried a green combined status. GATING_STEP_IF = "steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'" +# `npm test` means nothing without the directory it runs in: pointed elsewhere it runs a different +# package, or none. Part of the pin because it is part of what the step does. +GATING_STEP_WORKDIR = "web" +# `shell: bash` is `bash -e`, which is what makes a failing command fail the step. A workflow-level +# change here alters that for the gating step without touching the step at all. +GATING_WORKFLOW_SHELL = "bash" # The premise under the `docs_only` half of `GATING_STEP_IF`: the publish step is gated on it too, so # the arm that skips the suite ships nothing. Round-1 review found this stated as fact with nothing @@ -236,9 +242,16 @@ def copies_spa_source(command: str) -> bool: tokens = command.split() if not tokens or tokens[0].upper() not in COPY_INSTRUCTIONS: return False - if any(token.startswith("--from=") for token in tokens[1:]): - return False remainder = " ".join(token for token in tokens[1:] if not token.startswith("--")).strip() + if any(token.startswith("--from=") for token in tokens[1:]): + # A stage copy is not a CONTEXT copy, so `--from=build .` brings in that stage's `/`, not the + # repository. But it can still carry the SPA SOURCE across from a stage that has it, and a + # stage answered False here is UNPINNED and unchecked — measured: `COPY --from=web-build + # /source/web /source/web` was False, so a suite run in the receiving stage was invisible. + # The test is a whole `web` path SEGMENT in the source, which distinguishes the source tree + # from the built artifact this repo actually copies (`/source/ErsatzTV/wwwroot/app/.` has no + # such segment). + return any("web" in source.strip("/").split("/") for source in remainder.split()[:-1]) if remainder.startswith("["): try: parsed = json.loads(remainder) @@ -268,6 +281,23 @@ def spa_carrying_stages() -> list[tuple[str, str, list[str]]]: return found +def stage_shell_overrides() -> list[str]: + """Every `SHELL` instruction inside a pinned stage, which the pin alone cannot see. + + `SHELL ["/bin/sh", "-c"]` redefines what every later `RUN` in the stage EXECUTES while leaving + the `RUN` text — and therefore the pin — untouched. Reported rather than modelled: there is no + `SHELL` in this repo, so the honest move is to refuse the construct here rather than reason about + what a replacement interpreter would do. + """ + overrides: list[str] = [] + for rel in dockerfiles(): + for name, lines in stages(_read(rel)): + if f"{rel}::{name}" not in PINNED_STAGE_COMMANDS: + continue + overrides.extend(f"{rel}::{name}: {line!r}" for line in lines if re.match(r"^SHELL\b", line)) + return overrides + + def _jobs(doc: object) -> dict[str, dict]: jobs = doc.get("jobs") if isinstance(doc, dict) else None if not isinstance(jobs, dict): @@ -438,6 +468,30 @@ def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None: f"reason." ) + workdir = step.get("working-directory") + if workdir is None or str(workdir).strip() != GATING_STEP_WORKDIR: + faults.append( + f"the gating step's `working-directory:` is {workdir!r}, not the pinned " + f"{GATING_STEP_WORKDIR!r}. `npm test` run anywhere else runs a different package, or " + f"none, while the pinned command text is unchanged." + ) + + shell = ((yaml.safe_load(_read(GATING_WORKFLOW)) or {}).get("defaults") or {}).get("run", {}).get("shell") + if shell is None or str(shell).strip() != GATING_WORKFLOW_SHELL: + faults.append( + f"{GATING_WORKFLOW}'s `defaults.run.shell` is {shell!r}, not the pinned " + f"{GATING_WORKFLOW_SHELL!r}. `bash` here means `bash -e`, which is what makes a failing " + f"command fail the step; changing it changes whether a red suite blocks the image, " + f"without touching the step." + ) + + if stage_shell_overrides(): + faults.append( + f"a pinned Dockerfile stage carries a `SHELL` instruction: {stage_shell_overrides()}. " + f"That redefines what every later `RUN` executes while leaving its text — and therefore " + f"the pin — unchanged." + ) + condition = step.get("if") if condition is None or _normalise(str(condition)) != _normalise(GATING_STEP_IF): faults.append( @@ -596,6 +650,11 @@ def test_the_SPA_CARRYING_predicate_reads_every_COPY_SPELLING() -> None: ): assert copies_spa_source(spelling), spelling + # A STAGE copy can still carry the SPA source across. Measured False before this clause: the + # receiving stage was then unpinned, so a suite run in it was invisible. + assert copies_spa_source("COPY --from=web-build /source/web /source/web") + assert copies_spa_source("COPY --from=builder /app/web ./web") + for spelling in ( "COPY design-system/. ./design-system/", "COPY --from=web-build /source/ErsatzTV/wwwroot/app/. ./app/", -- 2.47.3 From 566cabea6ddaeee0d1a52ddc7c3acbc32a72bd1b Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 12:39:29 +0200 Subject: [PATCH 06/16] =?UTF-8?q?fix(887):=20the=20pin's=20real=20limit,?= =?UTF-8?q?=20measured=20=E2=80=94=20and=20three=20absolutes=20it=20does?= =?UTF-8?q?=20not=20support?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A fourth cold review attacked the pin itself. BLOCKER empty, the mechanism upheld, and every finding was prose claiming more than I had measured — plus one one-token gap that was live. THE ABSOLUTES, refuted by execution and now corrected in all three places they appeared (guard docstring, `docs/guard-inventory.md`, the decision record's `rule:`): * "A pin cannot produce a false green." True only in the trivial reading. A pin is immune to a different SPELLING of the command — the entire class that defeated the parser nine times — and is NOT immune to the same text MEANING something else. Two mutants re-armed ersatztv#887 through `web/package.json` alone: `RUN npm run build` executes whatever that file says, so `"build": "vitest run && …"` puts the suite back into the gitless stage with every pin still matching, and `"prepare"` does it via `npm ci`. Now pinned: exactly one script may mention vitest, and its body is fixed. * Residual (1), "a stage that does not carry the SPA source is unpinned — correct, since without `web/` there is no suite there". False. The boundary is what `copies_spa_source` RECOGNISES, which is narrower than "has the suite available". Restated, with the case still outside it named: a stage copy that RENAMES the tree. * The substring sweep's "never a false green". Its reported failures are false reds; what it fails to REPORT is not. `SUITE_MENTIONS` is a hand-written SELECTOR — a third category beside population and pin, and the worst-behaved, because a population going short is caught by an equality and a stale pin reddens loudly, while a selector going short is silent. It was short by exactly one entry: `npm t`, npm's own alias, which this guard already names among the spellings that defeated the parser. A stage running `npm t -- --run` escaped it. Fixed, and the category is now named. ALSO CLOSED: `run: |` -> `run: >` folded the two-line body into one command whose whitespace-normalised text was byte-identical to the pin, so the marker script swallowed the suite as its arguments — the body is now compared LINE BY LINE, since a newline separates two commands. A SECOND step named `Test SPA` inherited the exemption both the pin lookup and the sweep key on; exactly one is now required. And `COPY web*/` — a glob that matches `web/` — was read as not carrying the source, leaving the receiving stage unpinned. Battery 45 -> 51, 0 missed. The remaining meaning-change route, an `ENV` rewriting `PATH` so a pinned `RUN` resolves a different `npm`, is not modelled and is recorded as a residual rather than implied away. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 6 +- docs/guard-inventory.md | 2 +- ...est_image_build_delegates_the_spa_suite.py | 127 +++++++++++++++--- 4 files changed, 113 insertions(+), 24 deletions(-) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index e012e38e5..ffc53638f 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds; a pin cannot be wrong in the dangerous direction, only in the loud one. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`'s scripts — are pinned or refused too, and the one that is not (`ENV` rewriting `PATH`) is recorded as a residual. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index ebad5c2b6..c8465a3a5 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,9 +5,9 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds; a pin cannot be wrong in the dangerous direction, only in the loud one. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`''s scripts — are pinned or refused too, and the one that is not (`ENV` rewriting `PATH`) is recorded as a residual. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' -mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 41 mutants, 0 missed.' +mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 51 mutants, 0 missed.' --- The web-build stage is gitless twice over. It copies only `web/` and `design-system/`, so it holds @@ -58,6 +58,8 @@ not: green on every PR, red on `main` and on the release tag. **The guard's own mechanism was replaced once, and the reason belongs here rather than only in the commit log, because the next person to widen it needs it.** Three versions asked whether a given command runs the suite and whether its failure can be swallowed. That predicate was wrong nine times across three cold-review rounds — executed heredocs treated as data, `#` truncating a command mid-word (including the live `${#reports[@]}` idiom), compound punctuation welding commands, `npm t` and `./node_modules/.bin/vitest` and `timeout`/`su -c` wrappers unrecognised, `true || npm test` counted as a run it never performs, and `continue-on-error: ${{ … }}` slipping a two-literal check. Twice, a clause added to remove a FALSE RED opened a FALSE GREEN on the guard's headline assertion. The mechanism was withdrawn rather than patched a tenth time: the risky command lines are now compared against a pin, so no spelling has to be recognised in order to be rejected. The cost is real and is the point — a legitimate edit to those lines reddens the guard and its author updates the pin deliberately. +A fourth cold review then attacked the pin itself and found the honest limit: the replacement removes the SPELLING problem outright, but the same pinned text can be made to mean something else, and two mutants re-armed this very defect through `web/package.json` — `RUN npm run build` executes whatever that file says — with every pin still matching. Those routes are pinned now. The general lesson is worth more than the fix: replacing a predicate with a pin moves the problem from "can the checker recognise this?" to "is the checked text still the whole story?", and the second question has a short, enumerable answer where the first did not. + `lint` and `typecheck` stay in the stage. They are gitless-safe with no member that is not, so they carry no list and no trap. Whether the image build should run them at all is a separate question this does not answer, and leaving them is not an argument that it should. diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index c263afbf3..24e383288 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index 41401f641..82cf4ee6a 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -40,8 +40,18 @@ Nine defects from one mechanism is `process.enumerate-workaround-behaviors-befor to remove it rather than patch a tenth time, and the mechanism is removed here. **Nothing in this file decides what a command MEANS.** The commands that may run in the risky places are PINNED as text; anything else — a heredoc, a wrapper, a new spelling, a rewrite — is simply not equal to the -pin and reddens. A pin cannot have a false green, only a false red, and a false red is a human -reading a diff they should have read anyway. +pin and reddens. + +WHAT A PIN IS AND IS NOT IMMUNE TO, stated precisely because an earlier version of this paragraph +claimed the absolute "a pin cannot have a false green" and cold review refuted it by execution. A +pin is immune to the class that defeated the parser: a DIFFERENT SPELLING of the command. It is not +immune to the same text MEANING something else, and there are five such routes. Four are closed — +`working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`'s scripts, since +`RUN npm run build` executes whatever that file says — and each is pinned or refused below. The +fifth, an `ENV` rewriting `PATH` so a pinned `RUN` resolves a different `npm`, is not modelled and +is stated as a residual rather than implied away. So: the pin removes the spelling problem entirely +and shrinks the meaning problem to an enumerated list, which is a much better place to be than a +predicate that was wrong nine times — but it is not "cannot be wrong". The cost is stated rather than hidden: a legitimate edit to those command lines reddens this guard and its author must update the pin. That is the same deliberate review trigger `KNOWN_GATE_*` uses, @@ -65,13 +75,24 @@ red). Only the second is safe to write by hand. WHAT IS NOT COVERED, stated rather than implied: - * A Dockerfile stage that does NOT carry the SPA source is unpinned. It could run the suite and - this guard would not care — correctly, since without `web/` there is no suite there to run. The - decision rests on `copies_spa_source`, whose false-NEGATIVE direction is the one that would drop - a stage out of the check; it is pinned in both directions by its own test. - * A step OTHER than the pinned one could run the suite. `test_only_the_PINNED_step_mentions_the_suite` - reports any `run:` body containing `npm test` or `vitest` as a substring — a substring, not a - semantic predicate, so its failure mode is a false red asking a human to look. + * A Dockerfile stage that does not carry the SPA source is unpinned. The boundary is what + `copies_spa_source` recognises — a context copy of `web/` or `.`, a matching GLOB, or a stage + copy whose source names `web` — and NOT the wider "has the suite available to run", which is + what an earlier version of this sentence claimed. A stage copy that RENAMES the tree on the way + in (`COPY --from=web-build /source/web /elsewhere`) has no `web` segment in its source and is + not recognised. The false-NEGATIVE direction is the one that drops a stage out of the check, so + it is pinned in both directions by its own test. + * A step or stage OTHER than a pinned one could run the suite, and the substring sweep is what + reports it. `SUITE_MENTIONS` is a hand-written SELECTOR — a third category the population/pin + split does not name, and the one that behaves worst: a population going short is caught by an + equality, a pin going stale goes loudly red, but a SELECTOR going short is silent. It was short + by exactly one entry (`npm t`, npm's own alias, which this file names among the spellings that + defeated the parser) until cold review measured a stage escaping through it. `PUBLISH_ACTION` is + the same category and is covered by the anti-vacuity assertion instead. + * The gating body is compared LINE BY LINE with each line whitespace-normalised, not byte-exact. + Indentation and reflow are free; line boundaries are not, because a newline separates two + commands — changing `run: |` to `run: >` folds them into one and the whole-text comparison could + not see it. * Gitea's own evaluation of a skipped `needs:` job is not modelled; the guard forbids the job-level `if:` that would create one rather than reasoning about it. """ @@ -80,6 +101,7 @@ from __future__ import annotations import json import re +from fnmatch import fnmatch from pathlib import Path import yaml @@ -139,9 +161,20 @@ GATING_WORKFLOW_SHELL = "bash" PUBLISH_STEP_NAME = "Build and push" PUBLISH_STEP_IF = "steps.detect.outputs.docs_only != 'true'" +# `RUN npm run build` and `RUN npm ci` execute whatever `web/package.json` says they do, and that +# file is neither a Dockerfile nor a workflow. Cold review re-armed ersatztv#887 through it twice +# without touching any pinned line: `"build": "vitest run && tsc -b && vite build"` puts the suite +# back into the gitless stage, and `"prepare"` is run by `npm ci`. So the SCRIPTS THAT MENTION VITEST +# are pinned too — exactly one, and its body. +PACKAGE_JSON = "web/package.json" +PINNED_VITEST_SCRIPTS: dict[str, str] = {"test": "vitest"} + # Substrings that mean "a human should look at this step". NOT a semantic predicate — the pinned step # is allowed to contain them and every other step is not. -SUITE_MENTIONS = ("npm test", "npm run test", "vitest") +SUITE_MENTIONS = ("npm t", "npm run test", "vitest") +# `npm t` rather than `npm test`: it is npm's own documented alias, it SUBSUMES `npm test` as a +# substring, and this file enumerates it among the spellings that defeated the parser. Leaving it out +# meant an unpinned stage running `npm t -- --run` passed the sweep — measured by cold review. def _normalise(text: str) -> str: @@ -149,6 +182,15 @@ def _normalise(text: str) -> str: return " ".join(text.split()) +def _normalise_lines(text: str) -> list[str]: + """Per-line normalisation, so indentation is free but the LINE BOUNDARIES are not. + + A newline separates two commands; collapsing it merges them into one. That is not a cosmetic + difference and the whole-text form cannot see it. + """ + return [_normalise(line) for line in text.strip().splitlines() if line.strip()] + + def _canonical_action(uses: str) -> str: """The last two path segments of a `uses:` value, without its `@version`. @@ -264,11 +306,22 @@ def copies_spa_source(command: str) -> bool: arguments = remainder.split() if len(arguments) < 2: return False - for source in arguments[:-1]: - normalised = source.removeprefix("./").rstrip("/") - if normalised in {"", ".", "web"} or normalised.startswith("web/"): - return True - return False + return any(_source_reaches_web(source) for source in arguments[:-1]) + + +def _source_reaches_web(source: str) -> bool: + """Could this COPY source bring `web/` in? + + Exact (`web`, `web/…`), whole-context (`.`, `./`), and GLOB: `COPY web*/ ./` matches `web/` and + so carries the source, while `webhooks/` does not — measured False for the glob by cold review, + which left the receiving stage unpinned. `fnmatch` against the literal segment `web` is what + separates the two without hand-listing patterns. + """ + normalised = source.removeprefix("./").rstrip("/") + if normalised in {"", ".", "web"} or normalised.startswith("web/"): + return True + head = normalised.split("/")[0] + return any(char in head for char in "*?[") and fnmatch("web", head) def spa_carrying_stages() -> list[tuple[str, str, list[str]]]: @@ -451,15 +504,25 @@ def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None: a different string from the pin. Three earlier versions tried to recognise those individually and missed `true || npm test` and `continue-on-error: ${{ … }}` among others. """ - step = find_step(GATING_WORKFLOW, GATING_JOB, GATING_STEP_NAME) - assert step is not None, ( - f"{GATING_WORKFLOW}: job `{GATING_JOB}` has no step named {GATING_STEP_NAME!r}. That step is " - f"the only thing running the SPA suite, and the image is gated on the job holding it." + named = [ + candidate + for candidate in _steps(workflow_jobs(GATING_WORKFLOW).get(GATING_JOB, {})) + if str(candidate.get("name", "")).strip() == GATING_STEP_NAME + ] + assert len(named) == 1, ( + f"{GATING_WORKFLOW}: job `{GATING_JOB}` has {len(named)} steps named {GATING_STEP_NAME!r}, " + f"not one. That name is what both this assertion and the substring sweep key on, so a " + f"SECOND step wearing it inherits the exemption and is never checked — measured by cold " + f"review with a duplicate carrying `continue-on-error: true` and a spec filter." ) + step = named[0] faults: list[str] = [] body = step.get("run") - if not isinstance(body, str) or _normalise(body) != _normalise(GATING_STEP_RUN): + # PER LINE, not over the whole body: `_normalise` collapses newlines, so changing `run: |` to + # `run: >` folds the two lines into one command whose normalised text is byte-identical to the + # pin. Measured by cold review — the marker script then swallows the suite as its arguments. + if not isinstance(body, str) or _normalise_lines(body) != _normalise_lines(GATING_STEP_RUN): faults.append( f"the gating step's `run:` is not the pinned command.\n pinned: {GATING_STEP_RUN!r}\n" f" actual: {body!r}\n Anything but this — a spec filter, a `|| true`, a pipe, a " @@ -522,6 +585,26 @@ def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None: assert not faults, "\n".join(faults) +def test_only_the_PINNED_npm_SCRIPTS_run_vitest() -> None: + """A pinned `RUN` executes whatever `web/package.json` says; pin that too. + + The Dockerfile stage runs `npm ci` and `npm run build`. Neither line changes when a script body + does, so `"build": "vitest run && …"` or a `"prepare"` that runs vitest re-arms ersatztv#887 with + every pin still matching — measured by cold review, twice. A substring test on `vitest`, not a + predicate: it decides which scripts must be pinned, and going short is caught by the equality + below rather than by the substring. + """ + scripts = json.loads(_read(PACKAGE_JSON)).get("scripts", {}) + mentioning = {name: body for name, body in scripts.items() if isinstance(body, str) and "vitest" in body} + assert mentioning == PINNED_VITEST_SCRIPTS, ( + f"the `{PACKAGE_JSON}` scripts mentioning vitest are {mentioning}, not the pinned " + f"{PINNED_VITEST_SCRIPTS}. Every `RUN npm …` in a pinned Dockerfile stage executes whatever " + f"these say, so a suite run added here reaches the gitless stage with no pinned line " + f"changed — which is ersatztv#887 re-armed through a file nobody would think to check " + f"against this guard. If the change is deliberate, update the pin and say why." + ) + + def test_the_DOCS_ONLY_arm_cannot_publish_an_image() -> None: """The premise under half of `GATING_STEP_IF`, asserted rather than assumed. @@ -647,6 +730,10 @@ def test_the_SPA_CARRYING_predicate_reads_every_COPY_SPELLING() -> None: "COPY ./ /source/", "COPY --chown=node:node web/ /source/web/", "COPY web/package*.json ./web/", + # A GLOB that matches `web/`. Measured False before `_source_reaches_web`, which left a + # stage copying the source this way unpinned and unchecked. + "COPY web*/ ./", + "COPY * /source/", ): assert copies_spa_source(spelling), spelling -- 2.47.3 From 3a88f5dc3d79791da3c408704d47ddecdfec7d36 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 12:50:56 +0200 Subject: [PATCH 07/16] =?UTF-8?q?fix(887):=20six=20meaning-change=20routes?= =?UTF-8?q?,=20not=20five=20=E2=80=94=20counted=20by=20checking=20rather?= =?UTF-8?q?=20than=20asserting?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The previous commit said "five such routes" in three places. Checking rather than restating found six, and the sixth is one this guard must NOT close itself: the gating job runs in a `container:`, whose image decides which `npm` exists at all. That is already pinned by `test_ci_image_pin_population.py`, so it is CITED — two guards on one condition mask each other (ersatztv#685), and the way to find that out is to delete one and look for a red, which nobody does. Also measured rather than assumed: an INDIRECT script chain (`"test": "npm run inner"` with `inner` running vitest) needs no clause of its own. The set-equality against `PINNED_VITEST_SCRIPTS` reddens on it, because `inner` mentions vitest and `test` no longer does — verified across four scenarios, three red and one green. An enumeration is a claim like any other. This one was written from memory of what had been fixed rather than from the code, and it was short by one. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 2 +- docs/guard-inventory.md | 2 +- .../test_image_build_delegates_the_spa_suite.py | 14 +++++++++----- 4 files changed, 12 insertions(+), 8 deletions(-) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index ffc53638f..e3006d27a 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`'s scripts — are pinned or refused too, and the one that is not (`ENV` rewriting `PATH`) is recorded as a residual. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`'s scripts — are pinned or refused too, the gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice, and the one that is not covered (`ENV` rewriting `PATH`) is recorded as a residual. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index c8465a3a5..34fb0ee4b 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,7 +5,7 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`''s scripts — are pinned or refused too, and the one that is not (`ENV` rewriting `PATH`) is recorded as a residual. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`''s scripts — are pinned or refused too, the gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice, and the one that is not covered (`ENV` rewriting `PATH`) is recorded as a residual. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 51 mutants, 0 missed.' --- diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index 24e383288..ad630f244 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | +| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index 82cf4ee6a..25dc0610d 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -45,11 +45,15 @@ pin and reddens. WHAT A PIN IS AND IS NOT IMMUNE TO, stated precisely because an earlier version of this paragraph claimed the absolute "a pin cannot have a false green" and cold review refuted it by execution. A pin is immune to the class that defeated the parser: a DIFFERENT SPELLING of the command. It is not -immune to the same text MEANING something else, and there are five such routes. Four are closed — -`working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`'s scripts, since -`RUN npm run build` executes whatever that file says — and each is pinned or refused below. The -fifth, an `ENV` rewriting `PATH` so a pinned `RUN` resolves a different `npm`, is not modelled and -is stated as a residual rather than implied away. So: the pin removes the spelling problem entirely +immune to the same text MEANING something else, and there are six such routes. Four are closed +here — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`'s scripts, +since `RUN npm run build` executes whatever that file says (an INDIRECT chain, `"test": "npm run +inner"` with `inner` running vitest, is caught by the same equality rather than needing its own +clause). A fifth, the `container:` image the gating job runs in, decides which `npm` exists at all +and is pinned by `test_ci_image_pin_population.py` — cited rather than re-asserted, because two +guards on one condition mask each other (ersatztv#685). The sixth, an `ENV` rewriting `PATH` so a +pinned `RUN` resolves a different `npm`, is not modelled and is stated as a residual rather than +implied away. So: the pin removes the spelling problem entirely and shrinks the meaning problem to an enumerated list, which is a much better place to be than a predicate that was wrong nine times — but it is not "cannot be wrong". -- 2.47.3 From 9b99a23835c0e5287d394265fbc9a5d2aede89c7 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 13:23:36 +0200 Subject: [PATCH 08/16] =?UTF-8?q?fix(887):=20a=20selector=20where=20a=20pi?= =?UTF-8?q?n=20was=20available=20=E2=80=94=20the=20blocker=20round=205=20m?= =?UTF-8?q?easured?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 5 found a BLOCKER, and it is the sharpest kind: I made the exact mistake I had described one screen earlier. `test_only_the_PINNED_npm_SCRIPTS_run_vitest` SELECTED the scripts to pin by asking whether their body contained the literal `vitest` — a selector, the category this file calls the worst-behaved because going short is silent — and then its docstring claimed "going short is caught by the equality below", which is false: a script the substring misses is absent from the compared map, so the equality still holds. Four one-line `web/package.json` edits, none of which spells `vitest`, each put the suite back into the gitless stage with the whole guard green: `"build": "npm run test -- --run && …"`, the same via `npm t`, and the `prebuild` / `preinstall` LIFECYCLE HOOKS, which npm runs for `npm run build` and `npm ci` without anything naming them. That is #883 verbatim, through the route round 4 identified and the previous commit reported closed. The fix is the one the file's own vocabulary prescribes: pin the WHOLE script map. A script that does not exist cannot be a lifecycle hook, and one that changes is not equal. The category disappears rather than being widened by two entries. ALSO CLOSED, all measured: * `web/vite.config.ts`'s `test:` block is now pinned. `npm test -- --run` collects what that file says, so `test.exclude` is where a filter would now naturally be written — it is the only place left after this change removed the Dockerfile's. Three mutants narrowed the gating suite through it with the step's own command unchanged. * A step-level `shell:` and a job-level `defaults:` each override the pinned workflow default. Both forbidden. * `test_no_run_BODY_builds_or_pushes_an_image` is RESTORED — I dropped it in the parser withdrawal, and a job publishing via `run: docker build … && docker push …` was then outside the action-derived population with anti-vacuity none the wiser. * A leading-slash context copy (`COPY /web/. ./web/`) was not recognised. * The sweep gains `yarn test`, `pnpm test`, `bun test`. The residual naming the uncovered COPY shapes was wrong for the SECOND consecutive round — it named `COPY --from=X /source/web /elsewhere`, which is covered (only the destination is renamed). The real gaps are an ANCESTOR source (`/source` brings `/source/web` along) and `/source/.`. Both measured. Route count: five, then six, now seven. It has been wrong at every count, so it is now stated as a running total with that history attached rather than as an enumeration. Battery 51 -> 60, 0 missed. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 8 +- docs/guard-inventory.md | 2 +- ...est_image_build_delegates_the_spa_suite.py | 183 +++++++++++++++--- 4 files changed, 160 insertions(+), 35 deletions(-) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index e3006d27a..25593eb2e 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`'s scripts — are pinned or refused too, the gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice, and the one that is not covered (`ENV` rewriting `PATH`) is recorded as a residual. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), and `web/vite.config.ts`'s `test:` block, which decides what the suite collects. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. Only an `ENV` rewriting `PATH` is unmodelled. SEVEN routes is a running total, not a proof — it was written as five and then six, and both were wrong when checked. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index 34fb0ee4b..bdad3f49d 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,9 +5,9 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`''s scripts — are pinned or refused too, the gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice, and the one that is not covered (`ENV` rewriting `PATH`) is recorded as a residual. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), and `web/vite.config.ts`''s `test:` block, which decides what the suite collects. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. Only an `ENV` rewriting `PATH` is unmodelled. SEVEN routes is a running total, not a proof — it was written as five and then six, and both were wrong when checked. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' -mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 51 mutants, 0 missed.' +mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 60 mutants, 0 missed.' --- The web-build stage is gitless twice over. It copies only `web/` and `design-system/`, so it holds @@ -58,7 +58,9 @@ not: green on every PR, red on `main` and on the release tag. **The guard's own mechanism was replaced once, and the reason belongs here rather than only in the commit log, because the next person to widen it needs it.** Three versions asked whether a given command runs the suite and whether its failure can be swallowed. That predicate was wrong nine times across three cold-review rounds — executed heredocs treated as data, `#` truncating a command mid-word (including the live `${#reports[@]}` idiom), compound punctuation welding commands, `npm t` and `./node_modules/.bin/vitest` and `timeout`/`su -c` wrappers unrecognised, `true || npm test` counted as a run it never performs, and `continue-on-error: ${{ … }}` slipping a two-literal check. Twice, a clause added to remove a FALSE RED opened a FALSE GREEN on the guard's headline assertion. The mechanism was withdrawn rather than patched a tenth time: the risky command lines are now compared against a pin, so no spelling has to be recognised in order to be rejected. The cost is real and is the point — a legitimate edit to those lines reddens the guard and its author updates the pin deliberately. -A fourth cold review then attacked the pin itself and found the honest limit: the replacement removes the SPELLING problem outright, but the same pinned text can be made to mean something else, and two mutants re-armed this very defect through `web/package.json` — `RUN npm run build` executes whatever that file says — with every pin still matching. Those routes are pinned now. The general lesson is worth more than the fix: replacing a predicate with a pin moves the problem from "can the checker recognise this?" to "is the checked text still the whole story?", and the second question has a short, enumerable answer where the first did not. +A fourth cold review then attacked the pin itself and found the honest limit: the replacement removes the SPELLING problem outright, but the same pinned text can be made to mean something else, and two mutants re-armed this very defect through `web/package.json` — `RUN npm run build` executes whatever that file says — with every pin still matching. A fifth round then found the same mistake inside the fix: the package.json check SELECTED on the literal `vitest` where a PIN was available, and four one-line edits that never spell it — `npm run test`, `npm t`, and the `prebuild`/`preinstall` lifecycle hooks — each re-armed the defect with every other pin matching. The whole script map is pinned now, and so is `vite.config.ts`''s `test:` block. + +The general lesson is worth more than any of the fixes: replacing a predicate with a pin moves the problem from "can the checker recognise this?" to "is the checked text still the whole story?". The second question has a short, workable answer where the first did not — but it is still a question, the route list has been wrong at every count so far, and the honest form of that is a running total with its history attached rather than a closure claim. `lint` and `typecheck` stay in the stage. They are gitless-safe with no member that is not, so they carry no list and no trap. Whether the image build should run them at all is a separate question diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index ad630f244..5319b19b7 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | +| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index 25dc0610d..fb821e0c8 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -45,17 +45,21 @@ pin and reddens. WHAT A PIN IS AND IS NOT IMMUNE TO, stated precisely because an earlier version of this paragraph claimed the absolute "a pin cannot have a false green" and cold review refuted it by execution. A pin is immune to the class that defeated the parser: a DIFFERENT SPELLING of the command. It is not -immune to the same text MEANING something else, and there are six such routes. Four are closed -here — `working-directory`, `defaults.run.shell`, a stage `SHELL`, and `web/package.json`'s scripts, -since `RUN npm run build` executes whatever that file says (an INDIRECT chain, `"test": "npm run -inner"` with `inner` running vitest, is caught by the same equality rather than needing its own -clause). A fifth, the `container:` image the gating job runs in, decides which `npm` exists at all -and is pinned by `test_ci_image_pin_population.py` — cited rather than re-asserted, because two -guards on one condition mask each other (ersatztv#685). The sixth, an `ENV` rewriting `PATH` so a -pinned `RUN` resolves a different `npm`, is not modelled and is stated as a residual rather than -implied away. So: the pin removes the spelling problem entirely -and shrinks the meaning problem to an enumerated list, which is a much better place to be than a -predicate that was wrong nine times — but it is not "cannot be wrong". +immune to the same text MEANING something else. SEVEN such routes have been IDENTIFIED, and that +number is a running total, not a proof: it was written as five, then six, and both were wrong when +checked. Six are handled — `working-directory` and the gating step's own `shell:`; the workflow +`defaults.run.shell` and the job-level `defaults` that would override it; a stage `SHELL`, refused +outright; `web/package.json`'s script map, pinned WHOLE after selecting on the literal `vitest` +missed `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS npm runs for +`npm run build` and `npm ci`; `web/vite.config.ts`'s `test:` block, which decides what the suite +COLLECTS and is where a filter would now naturally be written; and the gating job's `container:` +image, pinned by `test_ci_image_pin_population.py` and CITED rather than re-asserted, because two +guards on one condition mask each other (ersatztv#685). The seventh, an `ENV` rewriting `PATH` so a +pinned `RUN` resolves a different `npm`, is not modelled and is stated as a residual. + +So: the pin removes the SPELLING problem outright and reduces the MEANING problem to a list that can +be worked through — a much better place than a predicate that was wrong nine times. It is not +"cannot be wrong", and the list's own history says so. The cost is stated rather than hidden: a legitimate edit to those command lines reddens this guard and its author must update the pin. That is the same deliberate review trigger `KNOWN_GATE_*` uses, @@ -84,8 +88,12 @@ WHAT IS NOT COVERED, stated rather than implied: copy whose source names `web` — and NOT the wider "has the suite available to run", which is what an earlier version of this sentence claimed. A stage copy that RENAMES the tree on the way in (`COPY --from=web-build /source/web /elsewhere`) has no `web` segment in its source and is - not recognised. The false-NEGATIVE direction is the one that drops a stage out of the check, so - it is pinned in both directions by its own test. + not recognised — nor is a source that is an ANCESTOR of it (`COPY --from=web-build /source /src` + brings `/source/web` along without naming it). Both were MEASURED; the previous two attempts at + this sentence each named a shape that was in fact covered while omitting these. The + false-NEGATIVE direction is what drops a stage out of the check, so the predicate is pinned in + both directions by its own test, and the substring sweep below is the backstop for whatever it + still misses. * A step or stage OTHER than a pinned one could run the suite, and the substring sweep is what reports it. `SUITE_MENTIONS` is a hand-written SELECTOR — a third category the population/pin split does not name, and the one that behaves worst: a population going short is caught by an @@ -171,11 +179,42 @@ PUBLISH_STEP_IF = "steps.detect.outputs.docs_only != 'true'" # back into the gitless stage, and `"prepare"` is run by `npm ci`. So the SCRIPTS THAT MENTION VITEST # are pinned too — exactly one, and its body. PACKAGE_JSON = "web/package.json" -PINNED_VITEST_SCRIPTS: dict[str, str] = {"test": "vitest"} +# THE WHOLE MAP, not the scripts that mention vitest. Selecting on the literal `vitest` was a +# SELECTOR where a PIN was available, and cold review re-armed ersatztv#887 through four spellings +# none of which contain it: `"build": "npm run test -- --run && …"`, `"build": "npm t -- …"`, and the +# lifecycle hooks `prebuild` (npm runs it before `npm run build`) and `preinstall` (before `npm ci`). +# Both pinned RUN lines in the gitless stage execute whatever this map says, so the map is pinned. +PINNED_PACKAGE_SCRIPTS: dict[str, str] = { + "dev": "vite --host 0.0.0.0", + "build": "tsc -b && vite build", + "generate:api": "node scripts/generate-openapi-types.mjs", + "check:api": "npm run generate:api && git diff --exit-code -- src/api/generated/v1.d.ts", + "lint": "eslint .", + "test": "vitest", + "test:ui-e2e": "playwright test", + "typecheck": "tsc -b --pretty false", + "prepare": "cd .. && husky", +} + +# `npm test -- --run` collects whatever this config says it does, so `test.exclude`/`include` is the +# remaining place a filter can be written — and, since this change removes the Dockerfile +# `--exclude`s, the FIRST place anyone reaching for one would now look. The `test:` block is pinned +# for the same reason the script map is. +VITE_CONFIG = "web/vite.config.ts" +PINNED_VITE_TEST_BLOCK = """test: { +environment: 'jsdom', +environmentOptions: { +jsdom: { +url: 'http://localhost/app/' +} +}, +exclude: [...configDefaults.exclude, 'e2e/**'], +setupFiles: './src/setupTests.ts' +}""" # Substrings that mean "a human should look at this step". NOT a semantic predicate — the pinned step # is allowed to contain them and every other step is not. -SUITE_MENTIONS = ("npm t", "npm run test", "vitest") +SUITE_MENTIONS = ("npm t", "npm run test", "yarn test", "pnpm test", "bun test", "vitest") # `npm t` rather than `npm test`: it is npm's own documented alias, it SUBSUMES `npm test` as a # substring, and this file enumerates it among the spellings that defeated the parser. Leaving it out # meant an unpinned stage running `npm t -- --run` passed the sweep — measured by cold review. @@ -186,6 +225,31 @@ def _normalise(text: str) -> str: return " ".join(text.split()) +def _vite_test_block() -> list[str]: + """The `test: {` … matching `}` block of `web/vite.config.ts`, normalised per line. + + Brace matching from a single marker — the same class of syntax handling as splitting a Dockerfile + on `FROM`: unambiguous, and it interprets nothing. Comment lines are dropped so the block can be + explained without the explanation being pinned. + """ + text = _read(VITE_CONFIG) + start = text.index("test: {") + depth = 0 + for index in range(start, len(text)): + if text[index] == "{": + depth += 1 + elif text[index] == "}": + depth -= 1 + if depth == 0: + block = text[start : index + 1] + return [ + _normalise(line) + for line in block.splitlines() + if line.strip() and not line.strip().startswith(("//", "*", "/*")) + ] + raise AssertionError(f"{VITE_CONFIG}: the `test: {{` block is not brace-balanced") + + def _normalise_lines(text: str) -> list[str]: """Per-line normalisation, so indentation is free but the LINE BOUNDARIES are not. @@ -321,7 +385,7 @@ def _source_reaches_web(source: str) -> bool: which left the receiving stage unpinned. `fnmatch` against the literal segment `web` is what separates the two without hand-listing patterns. """ - normalised = source.removeprefix("./").rstrip("/") + normalised = source.removeprefix("./").lstrip("/").rstrip("/") if normalised in {"", ".", "web"} or normalised.startswith("web/"): return True head = normalised.split("/")[0] @@ -579,6 +643,18 @@ def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None: f"nothing. Only absent, `false` or `False` is acceptable here." ) + if "shell" in step: + faults.append( + f"the gating step carries its own `shell: {step['shell']!r}`, which overrides the pinned " + f"workflow default. The default is pinned because `bash` means `bash -e`; a step-level " + f"override moves that decision somewhere nothing checks." + ) + if "defaults" in job: + faults.append( + f"the job `{GATING_JOB}` carries `defaults: {job['defaults']!r}`, which overrides the " + f"pinned workflow default for every step in it, including the gating one." + ) + if "if" in job: faults.append( f"the job `{GATING_JOB}` carries a job-level `if:` ({job['if']!r}). None did when " @@ -589,23 +665,42 @@ def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None: assert not faults, "\n".join(faults) -def test_only_the_PINNED_npm_SCRIPTS_run_vitest() -> None: - """A pinned `RUN` executes whatever `web/package.json` says; pin that too. +def test_the_npm_SCRIPT_MAP_is_exactly_what_was_reviewed() -> None: + """The whole map, because a pinned `RUN npm …` executes whatever it says. - The Dockerfile stage runs `npm ci` and `npm run build`. Neither line changes when a script body - does, so `"build": "vitest run && …"` or a `"prepare"` that runs vitest re-arms ersatztv#887 with - every pin still matching — measured by cold review, twice. A substring test on `vitest`, not a - predicate: it decides which scripts must be pinned, and going short is caught by the equality - below rather than by the substring. + An earlier version pinned only the scripts whose body contained `vitest`, which is a SELECTOR — + the category this file elsewhere calls the worst-behaved, because going short is silent. It went + short four ways, each re-arming ersatztv#887 in the gitless stage with every other pin matching: + `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS, which npm runs for + `npm run build` and `npm ci` without anything naming them. Pinning the map removes the category: + a script that does not exist cannot be a hook, and one that changes is not equal. """ scripts = json.loads(_read(PACKAGE_JSON)).get("scripts", {}) - mentioning = {name: body for name, body in scripts.items() if isinstance(body, str) and "vitest" in body} - assert mentioning == PINNED_VITEST_SCRIPTS, ( - f"the `{PACKAGE_JSON}` scripts mentioning vitest are {mentioning}, not the pinned " - f"{PINNED_VITEST_SCRIPTS}. Every `RUN npm …` in a pinned Dockerfile stage executes whatever " - f"these say, so a suite run added here reaches the gitless stage with no pinned line " - f"changed — which is ersatztv#887 re-armed through a file nobody would think to check " - f"against this guard. If the change is deliberate, update the pin and say why." + assert scripts == PINNED_PACKAGE_SCRIPTS, ( + f"`{PACKAGE_JSON}`'s scripts are not the pinned map.\n pinned: {PINNED_PACKAGE_SCRIPTS}\n" + f" actual: {scripts}\n Both pinned `RUN npm …` lines in the gitless `web-build` stage " + f"execute whatever this says, INCLUDING npm's `pre*`/`post*` lifecycle hooks, which nothing " + f"in the Dockerfile names. A suite run added here reaches that stage with no Dockerfile or " + f"workflow line changed — ersatztv#887 through a file nobody would think to check against " + f"this guard. Update the pin deliberately, in the same commit, with a reason." + ) + + +def test_the_VITE_TEST_BLOCK_is_exactly_what_was_reviewed() -> None: + """`npm test -- --run` collects what this config says; "unfiltered" is otherwise unverified. + + `test.exclude` is where the Dockerfile's `--exclude`s would naturally be re-created now that they + are gone, and a filter there leaves every other pin matching while the gating run reports green + over a narrowed suite. Cold review measured three: adding the three git-dependent specs to + `exclude`, narrowing `include` to `src/**`, and `include: ['nope/**']` with `passWithNoTests`. + """ + block = _vite_test_block() + assert block == _normalise_lines(PINNED_VITE_TEST_BLOCK), ( + f"`{VITE_CONFIG}`'s `test:` block is not the pinned one.\n" + f" pinned: {_normalise_lines(PINNED_VITE_TEST_BLOCK)}\n actual: {block}\n" + f" This block decides WHICH specs `npm test -- --run` collects, so a filter here narrows " + f"the gating suite with the step's own command unchanged — the same defect ersatztv#887 " + f"removed from the Dockerfile, relocated. Update the pin deliberately." ) @@ -682,6 +777,34 @@ def test_no_DOCKERFILE_outside_a_pinned_stage_mentions_the_suite() -> None: # ------------------------------------------------------------------------------------------------ +def test_no_run_BODY_builds_or_pushes_an_image() -> None: + """`publishing_jobs()` keys on the ACTION, which is a selector; assert nothing publishes around it. + + A job doing `docker build -f ./docker/Dockerfile … && docker push …` in a shell body is not in + that population, so the positive invariant is silently not applied to it — and anti-vacuity does + not notice, because the real `build` job is still there. Measured MISSED by cold review after + this assertion was dropped in the rewrite; it is restored rather than reasoned away. + """ + stray: list[str] = [] + for path in workflow_files(): + rel = path.relative_to(REPO_ROOT).as_posix() + for job_id, job in workflow_jobs(rel).items(): + for step in _steps(job): + body = step.get("run") + if not isinstance(body, str): + continue + for line in body.splitlines(): + if line.lstrip().startswith("#"): + continue + if re.search(r"\bdocker\s+(?:push|build|buildx\s+build)\b", line): + stray.append(f"{rel} :: job `{job_id}`: {line.strip()!r}") + assert not stray, ( + f"image build/push outside `{PUBLISH_ACTION}`:\n " + "\n ".join(stray) + "\n\n" + "`publishing_jobs()` derives from that action, so these jobs are NOT checked for the suite " + "gate — extend it before adding one." + ) + + def test_the_populations_are_NOT_empty() -> None: """Every assertion above passes over an empty set, which is how a completeness guard dies.""" assert dockerfiles(), "`git ls-files` reported no Dockerfile — the derivation is broken, not the repo." -- 2.47.3 From 848216e277fdfdc380212f1276110bfb467dff85 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 13:37:49 +0200 Subject: [PATCH 09/16] =?UTF-8?q?fix(887):=20a=20second=20config=20file=20?= =?UTF-8?q?outranks=20the=20pinned=20one=20=E2=80=94=20found=20by=20probin?= =?UTF-8?q?g=20vitest=20itself?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pinning `web/vite.config.ts`'s `test:` block is worthless while a file that takes precedence over it can simply be added. Vitest resolves `vitest.config.*` (and `vitest.workspace.*` / `vitest.projects.*`) BEFORE `vite.config.*`. MEASURED, not read: dropping a `web/vitest.config.ts` carrying `include: ['nope/**'], passWithNoTests: true` beside the pinned file made `npx vitest run` report "No test files found, exiting with code 0". The gating step would be green having run NOTHING — worse than the filtered run ersatztv#887 removed, because a filtered suite at least reports on what it ran. The construct is refused rather than modelled: no such file exists, so the guard asserts none appears. Its population is the git INDEX, which is right and worth stating — an untracked config does not exist in a CI checkout either, so the mutant proving this has to STAGE the file. It failed to redden until it did, which is the correct behaviour demonstrating itself. Route count five -> six -> seven -> eight, wrong at every previous count, so it stays a running total with its history attached. Battery 60 -> 61, 0 missed. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 4 +- docs/guard-inventory.md | 2 +- ...est_image_build_delegates_the_spa_suite.py | 49 ++++++++++++++++--- 4 files changed, 46 insertions(+), 11 deletions(-) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 25593eb2e..392971edd 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), and `web/vite.config.ts`'s `test:` block, which decides what the suite collects. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. Only an `ENV` rewriting `PATH` is unmodelled. SEVEN routes is a running total, not a proof — it was written as five and then six, and both were wrong when checked. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`'s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. Only an `ENV` rewriting `PATH` is unmodelled. EIGHT routes is a running total, not a proof — it was written as five, then six, then seven, and each was wrong when checked. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index bdad3f49d..ca4fa38eb 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,9 +5,9 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), and `web/vite.config.ts`''s `test:` block, which decides what the suite collects. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. Only an `ENV` rewriting `PATH` is unmodelled. SEVEN routes is a running total, not a proof — it was written as five and then six, and both were wrong when checked. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`''s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. Only an `ENV` rewriting `PATH` is unmodelled. EIGHT routes is a running total, not a proof — it was written as five, then six, then seven, and each was wrong when checked. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' -mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 60 mutants, 0 missed.' +mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 61 mutants, 0 missed.' --- The web-build stage is gitless twice over. It copies only `web/` and `design-system/`, so it holds diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index 5319b19b7..d0441fadf 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | +| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index fb821e0c8..d9aee7173 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -45,17 +45,22 @@ pin and reddens. WHAT A PIN IS AND IS NOT IMMUNE TO, stated precisely because an earlier version of this paragraph claimed the absolute "a pin cannot have a false green" and cold review refuted it by execution. A pin is immune to the class that defeated the parser: a DIFFERENT SPELLING of the command. It is not -immune to the same text MEANING something else. SEVEN such routes have been IDENTIFIED, and that -number is a running total, not a proof: it was written as five, then six, and both were wrong when -checked. Six are handled — `working-directory` and the gating step's own `shell:`; the workflow +immune to the same text MEANING something else. EIGHT such routes have been IDENTIFIED, and that +number is a running total, not a proof: it was written as five, then six, then seven, and each was +wrong when checked. Six are handled — `working-directory` and the gating step's own `shell:`; the workflow `defaults.run.shell` and the job-level `defaults` that would override it; a stage `SHELL`, refused outright; `web/package.json`'s script map, pinned WHOLE after selecting on the literal `vitest` missed `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS npm runs for `npm run build` and `npm ci`; `web/vite.config.ts`'s `test:` block, which decides what the suite -COLLECTS and is where a filter would now naturally be written; and the gating job's `container:` -image, pinned by `test_ci_image_pin_population.py` and CITED rather than re-asserted, because two -guards on one condition mask each other (ersatztv#685). The seventh, an `ENV` rewriting `PATH` so a -pinned `RUN` resolves a different `npm`, is not modelled and is stated as a residual. +COLLECTS and is where a filter would now naturally be written; the ABSENCE of any +`vitest.config.*`/`vitest.workspace.*`/`vitest.projects.*`, which vitest resolves in PREFERENCE to +that file — pinning one config is worthless while a second can outrank it, and the measured effect +of adding one was a run that collected zero specs and exited 0, which is worse than a filtered one; +and the gating job's `container:` image, pinned by `test_ci_image_pin_population.py` +and CITED rather than re-asserted, because two guards on one condition mask each other (#685). + +The eighth, an `ENV` rewriting `PATH` so a pinned `RUN` resolves a different `npm`, is not modelled +and is stated as a residual. So: the pin removes the SPELLING problem outright and reduces the MEANING problem to a list that can be worked through — a much better place than a predicate that was wrong nine times. It is not @@ -201,6 +206,13 @@ PINNED_PACKAGE_SCRIPTS: dict[str, str] = { # `--exclude`s, the FIRST place anyone reaching for one would now look. The `test:` block is pinned # for the same reason the script map is. VITE_CONFIG = "web/vite.config.ts" +# Vitest resolves its OWN config in preference to `vite.config.*`, so a `web/vitest.config.ts` beside +# the pinned file silently replaces the block below. MEASURED: adding one with +# `include: ['nope/**'], passWithNoTests: true` made `npx vitest run` report +# "No test files found, exiting with code 0" — the gating step green having run NOTHING, and the +# image published. Pinning one file is worthless while a second can outrank it, so the absence of +# every higher-precedence spelling is asserted rather than assumed. +VITEST_CONFIG_PREFIXES = ("vitest.config.", "vitest.workspace.", "vitest.projects.") PINNED_VITE_TEST_BLOCK = """test: { environment: 'jsdom', environmentOptions: { @@ -686,6 +698,29 @@ def test_the_npm_SCRIPT_MAP_is_exactly_what_was_reviewed() -> None: ) +def test_NO_OTHER_vitest_CONFIG_outranks_the_pinned_one() -> None: + """A pinned config is worthless if a second file can take precedence over it. + + Vitest prefers `vitest.config.*` (and `vitest.workspace.*`/`vitest.projects.*`) to + `vite.config.*`. MEASURED: dropping a `web/vitest.config.ts` with `include: ['nope/**']` and + `passWithNoTests: true` beside the pinned file made the suite collect zero specs and exit 0 — the + gating step green having run nothing at all, which is worse than the filtered run ersatztv#887 + removed. The construct is refused rather than modelled: there is no such file today, so the + honest move is to assert none appears. + """ + intruders = sorted( + path + for path in tracked_files._git_ls_files() + if path.startswith("web/") and path.rpartition("/")[2].startswith(VITEST_CONFIG_PREFIXES) + ) + assert not intruders, ( + f"these files outrank the pinned `{VITE_CONFIG}` in vitest's config resolution: {intruders}. " + f"Whatever they say about `include`/`exclude` decides which specs the gating run collects, " + f"and the pin on `{VITE_CONFIG}` cannot see it — a suite that collects nothing still exits 0. " + f"Either fold the settings into `{VITE_CONFIG}` and its pin, or pin this file too." + ) + + def test_the_VITE_TEST_BLOCK_is_exactly_what_was_reviewed() -> None: """`npm test -- --run` collects what this config says; "unfiltered" is otherwise unverified. -- 2.47.3 From 3ec73f3769371bb1a437fcb953a6d883d003dbed Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 14:07:47 +0200 Subject: [PATCH 10/16] fix(887): a pin assumes it is pinning the thing that still DECIDES MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 6 found three more false greens and named the class they share, which is worth more than any of the three fixes: * `web/vitest.config.ts` OUTRANKS the pinned `vite.config.ts` — closed in the previous commit, found by probing vitest rather than reading about it. * A DECOY first `test: {` block. The comparison took `text.index("test: {")`, so a copy of the pin placed above `defineConfig` satisfied it while the real block was narrowed. Exactly one is now required — the same assertion this file already made about the gating step's NAME, for the same reason, not carried across. * A `needs:` edge matched by bare job id. `needs:` resolves within its own workflow, so a SECOND workflow publishing this Dockerfile while needing its own unrelated job called `test` satisfied it. Now bound to `GATING_WORKFLOW`. (The reviewer downgraded this to MEDIUM on measuring that `test_remote_state_inventory.py` forces a human to classify any new workflow — so the hole is "the guard is blind", not "silent". The forced review asks about remote state, not about whether the image is gated, so the one-line fix stands.) * A vite PLUGIN can shell out to the suite from `buildStart()`. The plugin ARRAY is pinned; the plugin BODIES are a stated residual, mitigated because `trackedSourceFilesPlugin` is deliberately lazy — a fact its own comment now marks as LOAD-BEARING for the image build rather than leaving as an optimisation note. THE CLASS: **a pin assumes it is pinning the artifact that still decides.** Every route found so far is authority moving where the pin is not looking — to another FILE, another OCCURRENCE in the same file, another WORKFLOW, or a HOOK the pinned command invokes. That question is now written down for the next person adding a pin, because a list of four instances is not what generalises. Prose, all refuted by execution: the residual naming the uncovered COPY shapes was wrong a THIRD time at the same site (`/source/web /elsewhere` IS recognised — only the destination is renamed — and the file's own test 700 lines below said so); "only an `ENV` is unmodelled" was an absolute and is now a list; "Reach: N mutants, 0 missed" is restated as a DEVELOPMENT BATTERY, since it is not in the repo, nothing re-derives it, and an independent battery found misses against an earlier head; and `PUBLISH_ACTION` was claimed covered by anti-vacuity, which proves the selector is non-empty and cannot prove it complete. Battery 61 -> 64, 0 missed. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 4 +- docs/guard-inventory.md | 2 +- ...est_image_build_delegates_the_spa_suite.py | 67 ++++++++++++++++--- web/vite-plugins/trackedSourceFiles.ts | 4 +- 5 files changed, 66 insertions(+), 13 deletions(-) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 392971edd..379c6915d 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`'s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. Only an `ENV` rewriting `PATH` is unmodelled. EIGHT routes is a running total, not a proof — it was written as five, then six, then seven, and each was wrong when checked. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`'s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. ELEVEN routes is a running total, not a proof — written as five, six, seven and eight, each wrong when checked. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index ca4fa38eb..842af8549 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,9 +5,9 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`''s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. Only an `ENV` rewriting `PATH` is unmodelled. EIGHT routes is a running total, not a proof — it was written as five, then six, then seven, and each was wrong when checked. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`''s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. ELEVEN routes is a running total, not a proof — written as five, six, seven and eight, each wrong when checked. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' -mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 61 mutants, 0 missed.' +mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 64 mutants, 0 missed (development battery).' --- The web-build stage is gitless twice over. It copies only `web/` and `design-system/`, so it holds diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index d0441fadf..f25537176 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | +| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index d9aee7173..bcf667292 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -66,6 +66,14 @@ So: the pin removes the SPELLING problem outright and reduces the MEANING proble be worked through — a much better place than a predicate that was wrong nine times. It is not "cannot be wrong", and the list's own history says so. +THE SHAPE THOSE ROUTES SHARE, which is the thing to check when adding a pin rather than a list to +memorise: **a pin assumes it is pinning the artifact that still DECIDES.** Every instance found so +far is authority moving somewhere the pin is not looking — to another FILE (`vitest.config.*` +outranks `vite.config.ts`), to another OCCURRENCE in the same file (a decoy first `test: {`), to +another WORKFLOW (a `needs:` edge naming a job called `test` that is not this one), or to a HOOK the +pinned command invokes (a vite plugin, an npm lifecycle script). Ask of any new pin: what else could +decide this, and would the pin still match? + The cost is stated rather than hidden: a legitimate edit to those command lines reddens this guard and its author must update the pin. That is the same deliberate review trigger `KNOWN_GATE_*` uses, applied to the two places where a silent change publishes an untested image. @@ -93,9 +101,12 @@ WHAT IS NOT COVERED, stated rather than implied: copy whose source names `web` — and NOT the wider "has the suite available to run", which is what an earlier version of this sentence claimed. A stage copy that RENAMES the tree on the way in (`COPY --from=web-build /source/web /elsewhere`) has no `web` segment in its source and is - not recognised — nor is a source that is an ANCESTOR of it (`COPY --from=web-build /source /src` - brings `/source/web` along without naming it). Both were MEASURED; the previous two attempts at - this sentence each named a shape that was in fact covered while omitting these. The + not recognised. THAT CLAUSE WAS WRONG THREE TIMES and is now stated from a measurement rather + than from memory: `COPY --from=web-build /source/web /elsewhere` IS recognised (only the + DESTINATION is renamed, and only sources are read). What is not: a source that is an ANCESTOR of + the tree (`COPY --from=web-build /source /src` brings `/source/web` along without naming it) or + names it only as `/source/.`. Both measured, and both backstopped by the substring sweep — an + ancestor-copy stage running `npm t`, `npx vitest` or a new npm script is caught. The false-NEGATIVE direction is what drops a stage out of the check, so the predicate is pinned in both directions by its own test, and the substring sweep below is the backstop for whatever it still misses. @@ -213,6 +224,12 @@ VITE_CONFIG = "web/vite.config.ts" # image published. Pinning one file is worthless while a second can outrank it, so the absence of # every higher-precedence spelling is asserted rather than assumed. VITEST_CONFIG_PREFIXES = ("vitest.config.", "vitest.workspace.", "vitest.projects.") +# A vite plugin runs inside the same process as the suite and can shell out from a build hook, so a +# `buildStart()` calling `npm test` re-arms ersatztv#887 from the plugin side with every other pin +# matching (measured). The ARRAY is pinned so no plugin joins silently; the plugin BODIES are not, +# and that is a stated residual — `trackedSourceFiles.ts` is deliberately LAZY for this exact +# reason, which its own comment now marks as load-bearing for the image build. +PINNED_VITE_PLUGINS = "plugins: [react(), trackedSourceFilesPlugin()]," PINNED_VITE_TEST_BLOCK = """test: { environment: 'jsdom', environmentOptions: { @@ -245,6 +262,16 @@ def _vite_test_block() -> list[str]: explained without the explanation being pinned. """ text = _read(VITE_CONFIG) + occurrences = text.count("test: {") + # A decoy `const reviewed = { test: {…} }` copied byte-for-byte from the pin, placed ABOVE + # `defineConfig`, satisfies a comparison that takes the FIRST match while the real block is + # narrowed — measured green. The file already requires exactly ONE step named `Test SPA` for the + # same reason; the reasoning had not been applied here. + assert occurrences == 1, ( + f"{VITE_CONFIG} contains {occurrences} `test: {{` blocks, not one. The pin compares the " + f"FIRST, so a second is either a decoy that satisfies it while another block decides, or a " + f"real second config nobody is checking." + ) start = text.index("test: {") depth = 0 for index in range(start, len(text)): @@ -560,13 +587,20 @@ def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> if not any(stage_rel == dockerfile for stage_rel, _, _ in spa_carrying_stages()): continue gates = _transitive_needs(job_id, jobs) - if GATING_JOB not in gates: + # `needs:` names a job in the SAME workflow, so the edge only reaches the pinned step when + # the publishing job lives in `GATING_WORKFLOW`. Matching a bare job id anywhere was a false + # green: a second workflow publishing this Dockerfile while `needs:`-ing its OWN unrelated + # job called `test` (`run: echo ok`) satisfied it — measured, and it escaped every + # workflow-population guard in the repo, 588 tests green. + if rel != GATING_WORKFLOW or GATING_JOB not in gates: faults.append( f"{rel}: job `{job_id}` publishes an image built from `{dockerfile}`, which carries " - f"the SPA source, but does not transitively `needs:` `{GATING_JOB}` — the job " - f"holding the pinned suite step. It needs: {sorted(gates)}. Since ersatztv#887 the " - f"image build deliberately does not run the suite itself, so this `needs:` edge is " - f"the ONLY thing standing between a red suite and a published image." + f"the SPA source, but does not transitively `needs:` `{GATING_JOB}` IN " + f"`{GATING_WORKFLOW}` — the job holding the pinned suite step. It needs: " + f"{sorted(gates)}. A `needs:` edge resolves within its own workflow, so a job named " + f"`{GATING_JOB}` in a DIFFERENT workflow is a different job and gates nothing. Since " + f"ersatztv#887 the image build does not run the suite itself, so this edge is the " + f"ONLY thing between a red suite and a published image." ) assert not faults, "\n".join(faults) @@ -698,6 +732,23 @@ def test_the_npm_SCRIPT_MAP_is_exactly_what_was_reviewed() -> None: ) +def test_the_VITE_PLUGIN_LIST_is_exactly_what_was_reviewed() -> None: + """A plugin can run the suite from a build hook, so the list of them is pinned. + + Vite plugins execute in the same process as `vite build` and `vitest`, and a `buildStart()` that + shells out to `npm test` puts the suite back into the gitless stage with every command pin + intact — measured, both as an inline plugin and inside an existing one. The list is pinned so a + new plugin cannot arrive unnoticed; what each plugin DOES is a residual, and the mitigation + there is that `trackedSourceFilesPlugin` is deliberately lazy. + """ + text = _read(VITE_CONFIG) + assert text.count(PINNED_VITE_PLUGINS) == 1, ( + f"`{VITE_CONFIG}`'s plugin list is not the pinned one ({PINNED_VITE_PLUGINS!r}). A vite " + f"plugin runs in the same process as the suite and can shell out from a build hook, so a " + f"new one is a decision to state. Update the pin deliberately." + ) + + def test_NO_OTHER_vitest_CONFIG_outranks_the_pinned_one() -> None: """A pinned config is worthless if a second file can take precedence over it. diff --git a/web/vite-plugins/trackedSourceFiles.ts b/web/vite-plugins/trackedSourceFiles.ts index 1624307fe..ede5ddbe0 100644 --- a/web/vite-plugins/trackedSourceFiles.ts +++ b/web/vite-plugins/trackedSourceFiles.ts @@ -150,7 +150,9 @@ export function trackedSourceFilesPlugin(runGit?: GitRunner, fileExists?: FileEx return id === TRACKED_SOURCE_FILES_ID ? RESOLVED_ID : undefined; }, // Deliberately LAZY: git runs only when something imports the virtual module. `vite build` - // never resolves this id and so never shells out to git. + // never resolves this id and so never shells out to git. LOAD-BEARING FOR THE IMAGE BUILD since + // ersatztv#887: `docker/Dockerfile`'s web-build stage runs `npm run build` in a stage with no git + // binary, so making this eager would break every image build for #887's original root cause. load(id) { if (id !== RESOLVED_ID) { return undefined; -- 2.47.3 From 06b80816647c7c8b9ac0e8bb56ef40eb509758bb Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 14:23:27 +0200 Subject: [PATCH 11/16] docs(887): two probed non-routes, recorded so they are not re-derived MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hunting a fifth instance of "the pin assumes it is pinning the thing that still decides" turned up two candidates that look like routes and are not, both measured rather than argued: * `setupFiles` is pinned by NAME while its CONTENT is not, which reads like the package.json hole one level down. It is fail-NOISY: `process.exit(0)` at the top of `src/setupTests.ts` makes vitest report `121 failed (121)`, not a green. * `tsconfig*.json` shapes what `tsc -b` compiles, not what vitest collects. Recorded because a reader who spots either will otherwise spend the same probe to reach the same answer — and because the honest residual beside them is the one that IS open: a dependency's own install script, reached through `npm ci` and `web/package-lock.json`. That is a supply-chain concern wider than this guard, and it is named rather than claimed covered. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- scripts/tests/test_image_build_delegates_the_spa_suite.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index bcf667292..7bbbf988a 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -74,6 +74,14 @@ another WORKFLOW (a `needs:` edge naming a job called `test` that is not this on pinned command invokes (a vite plugin, an npm lifecycle script). Ask of any new pin: what else could decide this, and would the pin still match? +Two candidates for that class were probed and are NOT routes, recorded so the next reader does not +re-derive them. The `setupFiles` entry is pinned by NAME and its CONTENT is not, but a setup file +that terminates the run is FAIL-NOISY: `process.exit(0)` at the top of `src/setupTests.ts` reports +`121 failed (121)`, not a green. And `tsconfig*.json` shapes what `tsc -b` compiles, not what vitest +collects. What remains genuinely unpinned in that direction is a dependency's own install script, +reached through `npm ci` and `web/package-lock.json` — a supply-chain concern wider than this guard, +named rather than claimed to be covered. + The cost is stated rather than hidden: a legitimate edit to those command lines reddens this guard and its author must update the pin. That is the same deliberate review trigger `KNOWN_GATE_*` uses, applied to the two places where a silent change publishes an untested image. -- 2.47.3 From 8141ac7807141c280c8b2c667dcc36bec46bf04c Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 15:34:31 +0200 Subject: [PATCH 12/16] fix(887): the rule applied to one config family and not its sibling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 7. BLOCKER empty; three real holes, two of them introduced by round six's own fixes, which is this repo's recorded `each-fix-round-introduces-an-adjacent-defect` happening inside a single commit. **`web/vite.config.js` outranks the pinned `.ts` for `vite build`.** Read from the pinned tarball rather than remembered: vite 8.1.3's `DEFAULT_CONFIG_FILES` is `["vite.config.js", "vite.config.mjs", "vite.config.ts", …]`. So a `vite.config.js` whose plugin shells out to the suite runs it in the gitless stage — measured with all 1411 tests green. The governing rule was already written down one family over ("pinning one config is worthless while a second can outrank it") and had been applied to `vitest.config.*` and not to this one. The refusal now covers both families. **The decoy fix counted one SPELLING of the marker.** `text.count("test: {")` is defeated by writing the real block `test: {` with two spaces beside a decoy that matches exactly: count is 1, the comparison takes the decoy, and the live block filters out precisely the specs #883 broke on. Now `re.finditer(r"\btest:\s*\{")`. **The plugin pin added in that same commit shipped the identical decoy hole** it was written next door to fix — a raw `text.count(PINNED_VITE_PLUGINS) == 1` with a decoy above `defineConfig`. Both `vite.config.ts` pins now share ONE bracket walk and ONE whitespace-tolerant uniqueness rule, so they cannot drift apart again. PROSE, and this one is a false completion claim in my own previous commit message: I said the `PUBLISH_ACTION`/anti-vacuity sentence and the singular "only an `ENV`" residual were corrected. They were — in the record and the inventory row, and NOT in the guard docstring, which is the artifact a code reader hits first. Both are now fixed there too, the route COUNT is removed from the docstring and the record and kept in ONE place, and the residual that stated its own false version before retracting it now states the boundary once. Also: the `--from=` branch never reached the JSON exec-form parser, so `COPY --from=web-build ["/source/web", "/dest"]` left the receiving stage unpinned; the revalidate arm of the gating `if:` is now described as a DEPENDENCY on `ci-detect-already-validated.sh` (graded `MUTATION: NONE`) rather than as something asserted here, since only the `docs_only` arm is; and the plugin-bodies residual now says there are TWO plugins, `react()`'s being third-party and unmitigated. Battery 64 -> 68, 0 missed. One of those four exists because the battery itself briefly reported NOTHING and exited 0 after a bad splice deleted its `main()` — it now carries an anti-vacuity assert on its own mutant count. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docker/Dockerfile | 6 +- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 14 +- docs/guard-inventory.md | 2 +- ...est_image_build_delegates_the_spa_suite.py | 148 +++++++++++++----- 5 files changed, 121 insertions(+), 51 deletions(-) diff --git a/docker/Dockerfile b/docker/Dockerfile index 6c9627b2b..ad86dcb2a 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -32,8 +32,10 @@ WORKDIR /source/web # checkout, and `build` — the job that invokes this Dockerfile — carries `needs: [test, migrations, # scan]`, so an image cannot be published past a red suite. Two skips exist inside `test` and # neither leaves an image unvalidated: `docs_only` also gates `Build and push`, so that arm ships -# nothing; the #420 revalidate skip fires only on a tree byte-identical to a head that already -# carried a green combined status, i.e. this exact source already passed. Held by +# nothing — and that arm IS asserted, by `test_the_DOCS_ONLY_arm_cannot_publish_an_image`. The #420 +# revalidate skip fires only on a tree byte-identical to a head that already carried a green combined +# status; that arm is NOT asserted here — it rests on `scripts/ci-detect-already-validated.sh`, which +# carries no mutation proof of its own, so it is a dependency rather than a check. Held by # `scripts/tests/test_image_build_delegates_the_spa_suite.py`. That guard PINS the command line # below as text: any edit to it reddens, whatever the edit says. Update the pin in the same commit, # and do not use it to re-add a suite run. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 379c6915d..d7f445e92 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`'s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. ELEVEN routes is a running total, not a proof — written as five, six, seven and eight, each wrong when checked. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`'s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. The count of such routes lives in the `docs/guard-inventory.md` row and nowhere else — it briefly existed in three places with two different values, which is how a number stays wrong. It is a running total, wrong at every count so far. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index 842af8549..6ad6ef5d6 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,9 +5,9 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`''s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. ELEVEN routes is a running total, not a proof — written as five, six, seven and eight, each wrong when checked. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`''s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. The count of such routes lives in the `docs/guard-inventory.md` row and nowhere else — it briefly existed in three places with two different values, which is how a number stays wrong. It is a running total, wrong at every count so far. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' -mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 64 mutants, 0 missed (development battery).' +mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 68 mutants, 0 missed (development battery).' --- The web-build stage is gitless twice over. It copies only `web/` and `design-system/`, so it holds @@ -45,9 +45,13 @@ not: green on every PR, red on `main` and on the release tag. (`process.enumerate-workaround-behaviors-before-deleting`): 1. *Blocking a publish on a red suite* — retained, by the `needs:` edge. Two skips exist inside - `test` and neither leaves an image unvalidated: `docs_only` also gates `Build and push`, so that - arm ships nothing, and the #420 revalidate skip fires only on a tree byte-identical to a head - that already carried a green combined status. + `test`. `docs_only` also gates `Build and push`, so that arm ships nothing — ASSERTED, by + `test_the_DOCS_ONLY_arm_cannot_publish_an_image`, after cold review pointed out it had been + stated as fact with nothing checking it. The #420 revalidate skip fires only on a tree + byte-identical to a head that already carried a green combined status; that arm is a DEPENDENCY + on `scripts/ci-detect-already-validated.sh` rather than something asserted here, and that script + is graded `MUTATION: NONE`. Said plainly because #887 is what made the second arm load-bearing: + before it, the in-image run covered that case. 2. *Running the suite under `node:22-bookworm-slim` specifically* — **lost**, and this is the real cost. It is small because the suite is jsdom and pure JS, and because the artifact the image actually ships is the output of `vite build`, which still runs there: a test that passes in the diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index f25537176..3c444fcc7 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | +| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index 7bbbf988a..e53eff096 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -45,22 +45,33 @@ pin and reddens. WHAT A PIN IS AND IS NOT IMMUNE TO, stated precisely because an earlier version of this paragraph claimed the absolute "a pin cannot have a false green" and cold review refuted it by execution. A pin is immune to the class that defeated the parser: a DIFFERENT SPELLING of the command. It is not -immune to the same text MEANING something else. EIGHT such routes have been IDENTIFIED, and that -number is a running total, not a proof: it was written as five, then six, then seven, and each was -wrong when checked. Six are handled — `working-directory` and the gating step's own `shell:`; the workflow -`defaults.run.shell` and the job-level `defaults` that would override it; a stage `SHELL`, refused -outright; `web/package.json`'s script map, pinned WHOLE after selecting on the literal `vitest` -missed `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS npm runs for -`npm run build` and `npm ci`; `web/vite.config.ts`'s `test:` block, which decides what the suite -COLLECTS and is where a filter would now naturally be written; the ABSENCE of any -`vitest.config.*`/`vitest.workspace.*`/`vitest.projects.*`, which vitest resolves in PREFERENCE to -that file — pinning one config is worthless while a second can outrank it, and the measured effect -of adding one was a run that collected zero specs and exited 0, which is worse than a filtered one; -and the gating job's `container:` image, pinned by `test_ci_image_pin_population.py` -and CITED rather than re-asserted, because two guards on one condition mask each other (#685). +immune to the same text MEANING something else. The routes found so far, and what each is worth, +are enumerated ONCE — in `docs/guard-inventory.md`'s row for this guard, which is where the count +lives. It is not repeated here: it existed in three artifacts with two different values at one +point, and re-syncing three copies is how a number stays wrong (`state-the-invariant-not-the- +measurement`). What this file states is the RULE they are instances of, below. -The eighth, an `ENV` rewriting `PATH` so a pinned `RUN` resolves a different `npm`, is not modelled -and is stated as a residual. +Handled here: `working-directory` and the gating step's own `shell:`; the workflow +`defaults.run.shell` and the job-level `defaults` that would override it; a stage `SHELL`, refused +outright; `web/package.json`'s script map, pinned WHOLE after SELECTING on the literal `vitest` +missed `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS npm runs for +`npm run build` and `npm ci`; `web/vite.config.ts`'s `test:` block and `plugins:` array, both +extracted by the same bracket walk and required to be UNIQUE, because a decoy copy above +`defineConfig` satisfies a comparison that takes the first match; and the ABSENCE of any config file +that outranks the pinned one — `vitest.config.*` for vitest, and `vite.config.js`/`.mjs` for `vite +build`, whose `DEFAULT_CONFIG_FILES` puts them BEFORE `.ts`. Pinning one config is worthless while a +second can outrank it, and both halves of that were measured: a `vitest.config.ts` made the suite +collect zero specs and exit 0, and a `vite.config.js` ran the suite inside the gitless stage with +all 1411 tests green. + +Cited rather than re-asserted: the gating job's `container:` image, pinned by +`test_ci_image_pin_population.py` — two guards on one condition mask each other (#685). + +NOT modelled, as a list rather than an "only": an `ENV` in a pinned stage that changes what a pinned +`RUN` resolves or executes; the BODIES of the pinned plugins (there are two, and `react()`'s is +third-party and unmitigated — only `trackedSourceFilesPlugin`'s laziness is a mitigation); a +dependency's own install script, reached through `npm ci` and `web/package-lock.json`; and a publish +through some action other than `docker/build-push-action`. So: the pin removes the SPELLING problem outright and reduces the MEANING problem to a list that can be worked through — a much better place than a predicate that was wrong nine times. It is not @@ -109,12 +120,12 @@ WHAT IS NOT COVERED, stated rather than implied: copy whose source names `web` — and NOT the wider "has the suite available to run", which is what an earlier version of this sentence claimed. A stage copy that RENAMES the tree on the way in (`COPY --from=web-build /source/web /elsewhere`) has no `web` segment in its source and is - not recognised. THAT CLAUSE WAS WRONG THREE TIMES and is now stated from a measurement rather - than from memory: `COPY --from=web-build /source/web /elsewhere` IS recognised (only the - DESTINATION is renamed, and only sources are read). What is not: a source that is an ANCESTOR of - the tree (`COPY --from=web-build /source /src` brings `/source/web` along without naming it) or - names it only as `/source/.`. Both measured, and both backstopped by the substring sweep — an - ancestor-copy stage running `npm t`, `npx vitest` or a new npm script is caught. The + not recognised. What that boundary EXCLUDES, measured: a source that is an ANCESTOR of the + tree (`COPY --from=web-build /source /src` brings `/source/web` along without naming it), and + one that names it only as `/source/.`. Both are backstopped by the substring sweep — an + ancestor-copy stage running `npm t`, `npx vitest` or a new npm script is caught. (This sentence + was wrong three times before it was written from a measurement; a destination rename such as + `/source/web /elsewhere` IS recognised, because only sources are read.) The false-NEGATIVE direction is what drops a stage out of the check, so the predicate is pinned in both directions by its own test, and the substring sweep below is the backstop for whatever it still misses. @@ -124,7 +135,9 @@ WHAT IS NOT COVERED, stated rather than implied: equality, a pin going stale goes loudly red, but a SELECTOR going short is silent. It was short by exactly one entry (`npm t`, npm's own alias, which this file names among the spellings that defeated the parser) until cold review measured a stage escaping through it. `PUBLISH_ACTION` is - the same category and is covered by the anti-vacuity assertion instead. + the same category and is NOT covered — anti-vacuity proves the selector is non-empty, which the + real `build` job guarantees, and cannot prove it complete: a publish through a different action + is invisible to it. `test_no_run_BODY_builds_or_pushes_an_image` covers only the shell-body form. * The gating body is compared LINE BY LINE with each line whitespace-normalised, not byte-exact. Indentation and reflow are free; line boundaries are not, because a newline separates two commands — changing `run: |` to `run: >` folds them into one and the whole-text comparison could @@ -231,13 +244,20 @@ VITE_CONFIG = "web/vite.config.ts" # "No test files found, exiting with code 0" — the gating step green having run NOTHING, and the # image published. Pinning one file is worthless while a second can outrank it, so the absence of # every higher-precedence spelling is asserted rather than assumed. -VITEST_CONFIG_PREFIXES = ("vitest.config.", "vitest.workspace.", "vitest.projects.") +# Every config filename that can outrank the pinned one, for EITHER tool. Read out of the pinned +# tarballs rather than remembered: vite 8.1.3's `DEFAULT_CONFIG_FILES` is +# `["vite.config.js", "vite.config.mjs", "vite.config.ts", …]` — so a `vite.config.js` beats the +# pinned `.ts` for `vite build`, which is what the gitless stage runs. Vitest 4.1.9 resolves +# `vitest.config.*` before `vite.config.*`. MEASURED: a `web/vite.config.js` whose plugin shells out +# to the suite ran it in the gitless stage with all 1411 tests green. +OUTRANKING_CONFIG_PREFIXES = ("vitest.config.", "vitest.workspace.", "vitest.projects.", "vite.config.") # A vite plugin runs inside the same process as the suite and can shell out from a build hook, so a # `buildStart()` calling `npm test` re-arms ersatztv#887 from the plugin side with every other pin # matching (measured). The ARRAY is pinned so no plugin joins silently; the plugin BODIES are not, # and that is a stated residual — `trackedSourceFiles.ts` is deliberately LAZY for this exact # reason, which its own comment now marks as load-bearing for the image build. -PINNED_VITE_PLUGINS = "plugins: [react(), trackedSourceFilesPlugin()]," +# The ARRAY, as the bracket walk returns it — no `plugins:` prefix, no trailing comma. +PINNED_VITE_PLUGINS = "[react(), trackedSourceFilesPlugin()]" PINNED_VITE_TEST_BLOCK = """test: { environment: 'jsdom', environmentOptions: { @@ -262,6 +282,25 @@ def _normalise(text: str) -> str: return " ".join(text.split()) +def _bracket_span(text: str, start: int, opener: str, closer: str) -> str: + """The balanced `opener`…`closer` span beginning at or after `start`. + + Shared by both `vite.config.ts` pins so they cannot drift: the plugin array shipped a raw + substring comparison while the `test:` block used a walk, and the substring one was defeated by + the very decoy the walk had just been taught to reject. + """ + begin = text.index(opener, start) + depth = 0 + for index in range(begin, len(text)): + if text[index] == opener: + depth += 1 + elif text[index] == closer: + depth -= 1 + if depth == 0: + return text[begin : index + 1] + raise AssertionError(f"{VITE_CONFIG}: unbalanced {opener}{closer} from offset {begin}") + + def _vite_test_block() -> list[str]: """The `test: {` … matching `}` block of `web/vite.config.ts`, normalised per line. @@ -270,17 +309,17 @@ def _vite_test_block() -> list[str]: explained without the explanation being pinned. """ text = _read(VITE_CONFIG) - occurrences = text.count("test: {") - # A decoy `const reviewed = { test: {…} }` copied byte-for-byte from the pin, placed ABOVE - # `defineConfig`, satisfies a comparison that takes the FIRST match while the real block is - # narrowed — measured green. The file already requires exactly ONE step named `Test SPA` for the - # same reason; the reasoning had not been applied here. - assert occurrences == 1, ( - f"{VITE_CONFIG} contains {occurrences} `test: {{` blocks, not one. The pin compares the " - f"FIRST, so a second is either a decoy that satisfies it while another block decides, or a " - f"real second config nobody is checking." + # WHITESPACE-TOLERANT, because counting one SPELLING is not counting the marker: a decoy written + # `test: {` beside a real block written `test: {` made `text.count("test: {") == 1` while the + # comparison took the decoy — measured green with the real block filtering out exactly the specs + # ersatztv#883 broke on. One space defeated the previous fix. + matches = list(re.finditer(r"\btest:\s*\{", text)) + assert len(matches) == 1, ( + f"{VITE_CONFIG} contains {len(matches)} `test:` blocks, not one. The pin compares the FIRST, " + f"so a second is either a decoy that satisfies it while another block decides, or a real " + f"second config nobody is checking." ) - start = text.index("test: {") + start = matches[0].start() depth = 0 for index in range(start, len(text)): if text[index] == "{": @@ -400,6 +439,15 @@ def copies_spa_source(command: str) -> bool: if not tokens or tokens[0].upper() not in COPY_INSTRUCTIONS: return False remainder = " ".join(token for token in tokens[1:] if not token.startswith("--")).strip() + if remainder.startswith("["): + # Parse the exec form BEFORE the `--from=` branch: splitting `["/source/web", "/dest"]` on + # whitespace leaves a trailing comma on the first token, so the segment test never matched + # and the stage went unpinned. Measured False. + try: + parsed = json.loads(remainder) + except ValueError: + return True + remainder = " ".join(str(item) for item in parsed) if isinstance(parsed, list) else "" if any(token.startswith("--from=") for token in tokens[1:]): # A stage copy is not a CONTEXT copy, so `--from=build .` brings in that stage's `/`, not the # repository. But it can still carry the SPA SOURCE across from a stage that has it, and a @@ -750,10 +798,23 @@ def test_the_VITE_PLUGIN_LIST_is_exactly_what_was_reviewed() -> None: there is that `trackedSourceFilesPlugin` is deliberately lazy. """ text = _read(VITE_CONFIG) - assert text.count(PINNED_VITE_PLUGINS) == 1, ( - f"`{VITE_CONFIG}`'s plugin list is not the pinned one ({PINNED_VITE_PLUGINS!r}). A vite " - f"plugin runs in the same process as the suite and can shell out from a build hook, so a " - f"new one is a decision to state. Update the pin deliberately." + # The SAME bracket walk and the SAME uniqueness rule as the `test:` block. The first version of + # this assertion was `text.count(PINNED_VITE_PLUGINS) == 1` — a raw substring — and it shipped + # the decoy hole it was written one function away from fixing: a decoy `const reviewedPlugins = + # { plugins: [react(), trackedSourceFilesPlugin()], }` above `defineConfig` satisfied the count + # while a third plugin sat in the live array. Measured green. That is + # `each-fix-round-introduces-an-adjacent-defect` inside a single commit. + matches = list(re.finditer(r"\bplugins:\s*\[", text)) + assert len(matches) == 1, ( + f"`{VITE_CONFIG}` contains {len(matches)} `plugins:` arrays, not one. The pin compares the " + f"FIRST, so a second is either a decoy or a real list nobody is checking." + ) + array = _bracket_span(text, matches[0].start(), "[", "]") + assert _normalise(array) == _normalise(PINNED_VITE_PLUGINS), ( + f"`{VITE_CONFIG}`'s plugin list is not the pinned one.\n pinned: " + f"{_normalise(PINNED_VITE_PLUGINS.rstrip(','))}\n actual: {_normalise(array)}\n" + f" A vite plugin runs in the same process as the suite and can shell out from a build hook, " + f"so a new one is a decision to state. Update the pin deliberately." ) @@ -770,12 +831,15 @@ def test_NO_OTHER_vitest_CONFIG_outranks_the_pinned_one() -> None: intruders = sorted( path for path in tracked_files._git_ls_files() - if path.startswith("web/") and path.rpartition("/")[2].startswith(VITEST_CONFIG_PREFIXES) + if path.startswith("web/") + and path.rpartition("/")[2].startswith(OUTRANKING_CONFIG_PREFIXES) + and path != VITE_CONFIG ) assert not intruders, ( - f"these files outrank the pinned `{VITE_CONFIG}` in vitest's config resolution: {intruders}. " - f"Whatever they say about `include`/`exclude` decides which specs the gating run collects, " - f"and the pin on `{VITE_CONFIG}` cannot see it — a suite that collects nothing still exits 0. " + f"these files outrank the pinned `{VITE_CONFIG}` in vite's or vitest's config resolution: " + f"{intruders}. Whatever they say decides what the gating run COLLECTS and what `vite build` " + f"loads in the gitless stage, and the pin on `{VITE_CONFIG}` cannot see either — a suite " + f"that collects nothing still exits 0, and a plugin loaded from a `.js` config still runs. " f"Either fold the settings into `{VITE_CONFIG}` and its pin, or pin this file too." ) -- 2.47.3 From 69b30d15ecbf650aeb95108ffea6857178151b02 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 15:35:37 +0200 Subject: [PATCH 13/16] =?UTF-8?q?docs(887):=20the=20shared=20bracket=20wal?= =?UTF-8?q?k=20fails=20RED=20on=20a=20bracket=20in=20a=20string=20?= =?UTF-8?q?=E2=80=94=20measured?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It counts brackets without understanding string literals, so a `]` inside one ends the span early. The direction is what matters and it was measured: the truncated span does not equal the pin, so the outcome is a false RED, never a false green. Stated rather than fixed — parsing TypeScript to do better is exactly the predicate this file withdrew. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- scripts/tests/test_image_build_delegates_the_spa_suite.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index e53eff096..5c8aa1fd7 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -288,6 +288,11 @@ def _bracket_span(text: str, start: int, opener: str, closer: str) -> str: Shared by both `vite.config.ts` pins so they cannot drift: the plugin array shipped a raw substring comparison while the `test:` block used a walk, and the substring one was defeated by the very decoy the walk had just been taught to reject. + + It counts brackets without understanding strings or comments, so a `]` inside a string literal + ends the span early. MEASURED, including the direction that matters: the truncated span does not + equal the pin, so the outcome is a FAIL-RED, never a false green. Stated rather than fixed — + parsing TypeScript to do better is the predicate this file exists to avoid. """ begin = text.index(opener, start) depth = 0 -- 2.47.3 From 58def5faa1f7e4e58358d387a390b268d4003e3d Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 16:25:39 +0200 Subject: [PATCH 14/16] =?UTF-8?q?fix(887):=20withdraw=20partial=20matching?= =?UTF-8?q?=20too=20=E2=80=94=20`web/vite.config.ts`=20is=20pinned=20WHOLE?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 8. Three consecutive rounds had each closed one SPELLING of the same marker match, which is `every-blocker-was-one-mechanism-so-delete-it` at the count where it says withdraw. Round 6 pinned a block; round 7 fixed `test: {` (two spaces); round 8 defeated the repaired matcher four more ways — `test : {`, `"test": {`, and the same two for `plugins:` — plus two that never touched the marker at all: plugins: [react(), trackedSourceFilesPlugin()].concat([evil]) test: { …pinned… }, ...moreTest `defineConfig` is the identity function in BOTH vite and vitest (read from the installed tree), so a spread AFTER the pinned span simply replaces what the pin matched. No respelling of the marker could ever have caught those: the defect was partial matching, not the pattern. So the file is pinned whole. 48 lines, nothing generates it, no marker to respell and nothing after the span. One assertion replaces a bracket walk, a block extractor and two uniqueness assertions — and catches all seven measured routes. Stated cost, which is the same one every other pin here carries: any edit to that file reddens, a comment included. This also retires a claim I made in a commit message AND in the inventory row: that the two pins "share one bracket walk and cannot drift apart again". It was false when written — the block extractor had its own inline copy and never called the shared helper. Verified by spying on the call: the `test:` pin made zero calls to it. Also closed: the second exec-form branch in `copies_spa_source` was DEAD code (measured by poisoning it with a `raise` — every test stayed green), since the round-7 reorder rewrites `remainder` before it; the exec-form `--from` case that reorder was written for had no STANDING test, only the development battery, and now has three; and `OUTRANKING_CONFIG_PREFIXES` matched nested paths, so a `web/e2e/vite.config.ts` — and `web/e2e/` is a real directory — would have reddened with a message asserting it outranks the pinned config, which is false. Direct children of `web/` only. Battery 68 -> 73, 0 missed. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- .../ci/image-build-delegates-the-spa-suite.md | 2 +- docs/guard-inventory.md | 2 +- ...est_image_build_delegates_the_spa_suite.py | 244 ++++++++---------- 3 files changed, 104 insertions(+), 144 deletions(-) diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index 6ad6ef5d6..35fb265ab 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -7,7 +7,7 @@ supersedes: none superseded-by: none rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`''s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. The count of such routes lives in the `docs/guard-inventory.md` row and nowhere else — it briefly existed in three places with two different values, which is how a number stays wrong. It is a running total, wrong at every count so far. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' -mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 68 mutants, 0 missed (development battery).' +mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 73 mutants, 0 missed (development battery).' --- The web-build stage is gitless twice over. It copies only `web/` and `design-system/`, so it holds diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index 3c444fcc7..a3b2e8b11 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | +| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index 5c8aa1fd7..441e2a56d 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -55,9 +55,9 @@ Handled here: `working-directory` and the gating step's own `shell:`; the workfl `defaults.run.shell` and the job-level `defaults` that would override it; a stage `SHELL`, refused outright; `web/package.json`'s script map, pinned WHOLE after SELECTING on the literal `vitest` missed `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS npm runs for -`npm run build` and `npm ci`; `web/vite.config.ts`'s `test:` block and `plugins:` array, both -extracted by the same bracket walk and required to be UNIQUE, because a decoy copy above -`defineConfig` satisfies a comparison that takes the first match; and the ABSENCE of any config file +`npm run build` and `npm ci`; `web/vite.config.ts` PINNED WHOLE — pinning a block of it was +defeated seven measured ways across three rounds, so the partial match was withdrawn rather than +respelled an eighth time; and the ABSENCE of any config file that outranks the pinned one — `vitest.config.*` for vitest, and `vite.config.js`/`.mjs` for `vite build`, whose `DEFAULT_CONFIG_FILES` puts them BEFORE `.ts`. Pinning one config is worthless while a second can outrank it, and both halves of that were measured: a `vitest.config.ts` made the suite @@ -233,17 +233,68 @@ PINNED_PACKAGE_SCRIPTS: dict[str, str] = { "prepare": "cd .. && husky", } -# `npm test -- --run` collects whatever this config says it does, so `test.exclude`/`include` is the -# remaining place a filter can be written — and, since this change removes the Dockerfile -# `--exclude`s, the FIRST place anyone reaching for one would now look. The `test:` block is pinned -# for the same reason the script map is. +# `npm test -- --run` collects whatever this config says, and `vite build` in the gitless stage +# loads it too — so the WHOLE FILE is pinned, not a block within it. Three rounds each closed one +# SPELLING of a marker match (`test: {`, then `test: {`, then `test : {` and `"test": {`), and two +# more routes never touched the marker at all (`].concat([evil])` after the pinned array, +# `...moreTest` after the pinned block — `defineConfig` is the identity function, so a later spread +# REPLACES what the pin matched). That is `every-blocker-was-one-mechanism-so-delete-it` at the +# count where it says withdraw: the mechanism was partial matching, and it is gone. +# +# STATED COST: any edit to this file reddens, including a comment. It is 48 lines, nothing generates +# it, and it is the same move already made for `web/package.json`'s script map — a PIN where a +# partial match was being defended. VITE_CONFIG = "web/vite.config.ts" -# Vitest resolves its OWN config in preference to `vite.config.*`, so a `web/vitest.config.ts` beside -# the pinned file silently replaces the block below. MEASURED: adding one with -# `include: ['nope/**'], passWithNoTests: true` made `npx vitest run` report -# "No test files found, exiting with code 0" — the gating step green having run NOTHING, and the -# image published. Pinning one file is worthless while a second can outrank it, so the absence of -# every higher-precedence spelling is asserted rather than assumed. +PINNED_VITE_CONFIG = """import { fileURLToPath, URL } from 'node:url'; +import react from '@vitejs/plugin-react'; +import { configDefaults, defineConfig } from 'vitest/config'; +import { trackedSourceFilesPlugin } from './vite-plugins/trackedSourceFiles'; + +const aspNetHost = 'http://localhost:8409'; + +export default defineConfig({ + base: '/app/', + plugins: [react(), trackedSourceFilesPlugin()], + resolve: { + alias: { + '@': fileURLToPath(new URL('./src', import.meta.url)), + '@design-system': fileURLToPath(new URL('../design-system', import.meta.url)) + } + }, + build: { + outDir: '../ErsatzTV/wwwroot/app', + emptyOutDir: true + }, + server: { + port: 5173, + strictPort: false, + proxy: { + '/api': aspNetHost, + '/iptv': aspNetHost, + '/artwork': aspNetHost + } + }, + test: { + environment: 'jsdom', + environmentOptions: { + jsdom: { + url: 'http://localhost/app/' + } + }, + // `e2e/` holds the Playwright UI-E2E specs (ersatztv#445), which drive a REAL browser against a + // running instance. Vitest's default `include` glob (`**/*.{test,spec}.*`) would otherwise pick + // them up and run them under jsdom, where `@playwright/test`'s `test()` has no runner and the + // whole file dies. Spread the defaults rather than replacing `exclude` outright — a bare + // `exclude: ['e2e/**']` would drop `**/node_modules/**` and make vitest crawl deps. + // NOTE: excluding (not narrowing `include` to `src/**`) is deliberate — `scripts/` holds a real + // vitest test too (`generate-openapi-types.test.mjs`), which an `src/**`-only include would + // silently stop running. + exclude: [...configDefaults.exclude, 'e2e/**'], + setupFiles: './src/setupTests.ts' + } +}); +""" + # Every config filename that can outrank the pinned one, for EITHER tool. Read out of the pinned # tarballs rather than remembered: vite 8.1.3's `DEFAULT_CONFIG_FILES` is # `["vite.config.js", "vite.config.mjs", "vite.config.ts", …]` — so a `vite.config.js` beats the @@ -251,23 +302,6 @@ VITE_CONFIG = "web/vite.config.ts" # `vitest.config.*` before `vite.config.*`. MEASURED: a `web/vite.config.js` whose plugin shells out # to the suite ran it in the gitless stage with all 1411 tests green. OUTRANKING_CONFIG_PREFIXES = ("vitest.config.", "vitest.workspace.", "vitest.projects.", "vite.config.") -# A vite plugin runs inside the same process as the suite and can shell out from a build hook, so a -# `buildStart()` calling `npm test` re-arms ersatztv#887 from the plugin side with every other pin -# matching (measured). The ARRAY is pinned so no plugin joins silently; the plugin BODIES are not, -# and that is a stated residual — `trackedSourceFiles.ts` is deliberately LAZY for this exact -# reason, which its own comment now marks as load-bearing for the image build. -# The ARRAY, as the bracket walk returns it — no `plugins:` prefix, no trailing comma. -PINNED_VITE_PLUGINS = "[react(), trackedSourceFilesPlugin()]" -PINNED_VITE_TEST_BLOCK = """test: { -environment: 'jsdom', -environmentOptions: { -jsdom: { -url: 'http://localhost/app/' -} -}, -exclude: [...configDefaults.exclude, 'e2e/**'], -setupFiles: './src/setupTests.ts' -}""" # Substrings that mean "a human should look at this step". NOT a semantic predicate — the pinned step # is allowed to contain them and every other step is not. @@ -282,65 +316,6 @@ def _normalise(text: str) -> str: return " ".join(text.split()) -def _bracket_span(text: str, start: int, opener: str, closer: str) -> str: - """The balanced `opener`…`closer` span beginning at or after `start`. - - Shared by both `vite.config.ts` pins so they cannot drift: the plugin array shipped a raw - substring comparison while the `test:` block used a walk, and the substring one was defeated by - the very decoy the walk had just been taught to reject. - - It counts brackets without understanding strings or comments, so a `]` inside a string literal - ends the span early. MEASURED, including the direction that matters: the truncated span does not - equal the pin, so the outcome is a FAIL-RED, never a false green. Stated rather than fixed — - parsing TypeScript to do better is the predicate this file exists to avoid. - """ - begin = text.index(opener, start) - depth = 0 - for index in range(begin, len(text)): - if text[index] == opener: - depth += 1 - elif text[index] == closer: - depth -= 1 - if depth == 0: - return text[begin : index + 1] - raise AssertionError(f"{VITE_CONFIG}: unbalanced {opener}{closer} from offset {begin}") - - -def _vite_test_block() -> list[str]: - """The `test: {` … matching `}` block of `web/vite.config.ts`, normalised per line. - - Brace matching from a single marker — the same class of syntax handling as splitting a Dockerfile - on `FROM`: unambiguous, and it interprets nothing. Comment lines are dropped so the block can be - explained without the explanation being pinned. - """ - text = _read(VITE_CONFIG) - # WHITESPACE-TOLERANT, because counting one SPELLING is not counting the marker: a decoy written - # `test: {` beside a real block written `test: {` made `text.count("test: {") == 1` while the - # comparison took the decoy — measured green with the real block filtering out exactly the specs - # ersatztv#883 broke on. One space defeated the previous fix. - matches = list(re.finditer(r"\btest:\s*\{", text)) - assert len(matches) == 1, ( - f"{VITE_CONFIG} contains {len(matches)} `test:` blocks, not one. The pin compares the FIRST, " - f"so a second is either a decoy that satisfies it while another block decides, or a real " - f"second config nobody is checking." - ) - start = matches[0].start() - depth = 0 - for index in range(start, len(text)): - if text[index] == "{": - depth += 1 - elif text[index] == "}": - depth -= 1 - if depth == 0: - block = text[start : index + 1] - return [ - _normalise(line) - for line in block.splitlines() - if line.strip() and not line.strip().startswith(("//", "*", "/*")) - ] - raise AssertionError(f"{VITE_CONFIG}: the `test: {{` block is not brace-balanced") - - def _normalise_lines(text: str) -> list[str]: """Per-line normalisation, so indentation is free but the LINE BOUNDARIES are not. @@ -462,16 +437,10 @@ def copies_spa_source(command: str) -> bool: # from the built artifact this repo actually copies (`/source/ErsatzTV/wwwroot/app/.` has no # such segment). return any("web" in source.strip("/").split("/") for source in remainder.split()[:-1]) - if remainder.startswith("["): - try: - parsed = json.loads(remainder) - except ValueError: - # An exec-form array this cannot read is reported as CARRYING the source: that makes the - # stage pinned rather than skipped, which is the safe direction for an unknown. - return True - arguments = [str(item) for item in parsed] if isinstance(parsed, list) else [] - else: - arguments = remainder.split() + # `remainder` is already a plain space-joined string by here — the exec form was parsed above, + # before the `--from=` branch. A second `startswith("[")` test used to sit here and was dead + # code: measured by poisoning it with a `raise`, which left every test green. + arguments = remainder.split() if len(arguments) < 2: return False return any(_source_reaches_web(source) for source in arguments[:-1]) @@ -793,33 +762,31 @@ def test_the_npm_SCRIPT_MAP_is_exactly_what_was_reviewed() -> None: ) -def test_the_VITE_PLUGIN_LIST_is_exactly_what_was_reviewed() -> None: - """A plugin can run the suite from a build hook, so the list of them is pinned. +def test_the_VITE_CONFIG_is_exactly_what_was_reviewed() -> None: + """The whole file, because every partial match of it was defeated. - Vite plugins execute in the same process as `vite build` and `vitest`, and a `buildStart()` that - shells out to `npm test` puts the suite back into the gitless stage with every command pin - intact — measured, both as an inline plugin and inside an existing one. The list is pinned so a - new plugin cannot arrive unnoticed; what each plugin DOES is a residual, and the mitigation - there is that `trackedSourceFilesPlugin` is deliberately lazy. + `test.exclude`/`include` decide which specs the gating run collects, and `plugins:` can shell out + to the suite from a build hook — both inside a file the Dockerfile never mentions. Pinning a + BLOCK of it was defeated seven measured ways across three review rounds: a decoy copy above + `defineConfig` with the real member respelled `test: {`, `test : {` or `"test": {`; the same for + `plugins:`; and two that never touched the marker — `[…].concat([evil])` and a trailing + `...moreTest` spread, which replaces the pinned object because `defineConfig` is identity in both + vite and vitest. Comparing the file entire has no marker to respell and nothing after the span. """ - text = _read(VITE_CONFIG) - # The SAME bracket walk and the SAME uniqueness rule as the `test:` block. The first version of - # this assertion was `text.count(PINNED_VITE_PLUGINS) == 1` — a raw substring — and it shipped - # the decoy hole it was written one function away from fixing: a decoy `const reviewedPlugins = - # { plugins: [react(), trackedSourceFilesPlugin()], }` above `defineConfig` satisfied the count - # while a third plugin sat in the live array. Measured green. That is - # `each-fix-round-introduces-an-adjacent-defect` inside a single commit. - matches = list(re.finditer(r"\bplugins:\s*\[", text)) - assert len(matches) == 1, ( - f"`{VITE_CONFIG}` contains {len(matches)} `plugins:` arrays, not one. The pin compares the " - f"FIRST, so a second is either a decoy or a real list nobody is checking." - ) - array = _bracket_span(text, matches[0].start(), "[", "]") - assert _normalise(array) == _normalise(PINNED_VITE_PLUGINS), ( - f"`{VITE_CONFIG}`'s plugin list is not the pinned one.\n pinned: " - f"{_normalise(PINNED_VITE_PLUGINS.rstrip(','))}\n actual: {_normalise(array)}\n" - f" A vite plugin runs in the same process as the suite and can shell out from a build hook, " - f"so a new one is a decision to state. Update the pin deliberately." + actual = _normalise_lines(_read(VITE_CONFIG)) + pinned = _normalise_lines(PINNED_VITE_CONFIG) + assert actual == pinned, ( + f"`{VITE_CONFIG}` is not the pinned file. First difference:\n" + + "\n".join( + f" line {index + 1}:\n pinned: {pinned[index] if index < len(pinned) else ''}\n" + f" actual: {actual[index] if index < len(actual) else ''}" + for index in range(max(len(actual), len(pinned))) + if index >= min(len(actual), len(pinned)) or actual[index] != pinned[index] + ) + + "\n\n This file decides which specs the gating run collects and what `vite build` loads " + "in the gitless stage. Any edit — including a comment — reddens, deliberately: every " + "partial match of it was defeated. Update `PINNED_VITE_CONFIG` in the same commit, with a " + "reason." ) @@ -836,7 +803,11 @@ def test_NO_OTHER_vitest_CONFIG_outranks_the_pinned_one() -> None: intruders = sorted( path for path in tracked_files._git_ls_files() + # DIRECT children of `web/` only: vite resolves its config from the project ROOT, so a + # `web/e2e/vite.config.ts` outranks nothing — and `web/e2e/` is a real directory. Reddening + # on it would be fail-noisy with a message that is simply false. if path.startswith("web/") + and "/" not in path[len("web/") :] and path.rpartition("/")[2].startswith(OUTRANKING_CONFIG_PREFIXES) and path != VITE_CONFIG ) @@ -849,24 +820,6 @@ def test_NO_OTHER_vitest_CONFIG_outranks_the_pinned_one() -> None: ) -def test_the_VITE_TEST_BLOCK_is_exactly_what_was_reviewed() -> None: - """`npm test -- --run` collects what this config says; "unfiltered" is otherwise unverified. - - `test.exclude` is where the Dockerfile's `--exclude`s would naturally be re-created now that they - are gone, and a filter there leaves every other pin matching while the gating run reports green - over a narrowed suite. Cold review measured three: adding the three git-dependent specs to - `exclude`, narrowing `include` to `src/**`, and `include: ['nope/**']` with `passWithNoTests`. - """ - block = _vite_test_block() - assert block == _normalise_lines(PINNED_VITE_TEST_BLOCK), ( - f"`{VITE_CONFIG}`'s `test:` block is not the pinned one.\n" - f" pinned: {_normalise_lines(PINNED_VITE_TEST_BLOCK)}\n actual: {block}\n" - f" This block decides WHICH specs `npm test -- --run` collects, so a filter here narrows " - f"the gating suite with the step's own command unchanged — the same defect ersatztv#887 " - f"removed from the Dockerfile, relocated. Update the pin deliberately." - ) - - def test_the_DOCS_ONLY_arm_cannot_publish_an_image() -> None: """The premise under half of `GATING_STEP_IF`, asserted rather than assumed. @@ -1031,6 +984,13 @@ def test_the_SPA_CARRYING_predicate_reads_every_COPY_SPELLING() -> None: # receiving stage was then unpinned, so a suite run in it was invisible. assert copies_spa_source("COPY --from=web-build /source/web /source/web") assert copies_spa_source("COPY --from=builder /app/web ./web") + # The EXEC form of the same thing. Measured False before the parse order was fixed — splitting + # `["/source/web", "/dest"]` on whitespace left a trailing comma on the first token, so the + # segment test never matched. Pinned here because only the development battery witnessed it, and + # that battery is explicitly not standing. + assert copies_spa_source('COPY --from=web-build ["/source/web", "/dest"]') + assert copies_spa_source('COPY --from=web-build ["/source/web","/dest"]') + assert not copies_spa_source('COPY --from=web-build ["/source/ErsatzTV/wwwroot/app/.", "/dest"]') for spelling in ( "COPY design-system/. ./design-system/", -- 2.47.3 From 1506e5a54556bba090b01881e4173b58ab13d56b Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 16:26:51 +0200 Subject: [PATCH 15/16] =?UTF-8?q?docs(887):=20'pinned=20whole'=20is=20not?= =?UTF-8?q?=20byte=20equality=20=E2=80=94=20say=20what=20it=20cannot=20see?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_normalise_lines` drops blank lines and collapses whitespace WITHIN a line, so a reflow, an indentation change, and a change to the spacing inside a STRING LITERAL are invisible. The first two carry no meaning; the third could, and does not here. Line order and any token change are caught. All four measured. Stated because the phrase 'pinned whole' invites a reader to assume byte equality, and a reader who assumes that will not check the one case where it matters. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- scripts/tests/test_image_build_delegates_the_spa_suite.py | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index 441e2a56d..cd240bafa 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -772,6 +772,13 @@ def test_the_VITE_CONFIG_is_exactly_what_was_reviewed() -> None: `plugins:`; and two that never touched the marker — `[…].concat([evil])` and a trailing `...moreTest` spread, which replaces the pinned object because `defineConfig` is identity in both vite and vitest. Comparing the file entire has no marker to respell and nothing after the span. + + WHAT THE COMPARISON CANNOT SEE, measured: `_normalise_lines` drops blank lines and collapses + whitespace WITHIN a line, so a reflow, an indentation change and a change to the spacing inside + a STRING LITERAL are all invisible. The first two carry no meaning; the third could, and does not + here — no string in this file has consequential internal spacing. Line ORDER and any token change + are caught (both measured). Stated because "pinned whole" invites the reader to assume byte + equality, and it is not. """ actual = _normalise_lines(_read(VITE_CONFIG)) pinned = _normalise_lines(PINNED_VITE_CONFIG) -- 2.47.3 From 9da00204622c55819650f08a3c48b61ebe4fa975 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 30 Aug 2026 17:14:22 +0200 Subject: [PATCH 16/16] =?UTF-8?q?docs(887):=20sweep=20the=20withdrawal=20t?= =?UTF-8?q?hrough=20the=20record=20=E2=80=94=20it=20still=20described=20th?= =?UTF-8?q?e=20withdrawn=20pin?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 9 returned MERGEABLE with BLOCKER and HIGH empty. Every remaining item was a sentence, and every one erred by UNDERSTATING the guard — which is the safe direction and still worth fixing, because the decision record is what CLAUDE.md routes convention lookups to. The record's `rule:` still listed "`web/vite.config.ts`'s `test:` block" among the pinned things — the very mechanism the previous commit withdrew — and named only `vitest.config.*` as the outranking family, omitting `vite.config.js`/`.mjs`, which is the MEASURED attack from round 7 (a `web/vite.config.js` ran the suite in the gitless stage with 1411 tests green). That family went short in round 7 and again in round 8. This is `enumerate-CLAUSES-to-close-a-sweep`: the survivors were phrased in a different category (WHAT is pinned) from the retracted claim (HOW it is extracted), so sweeping for the retracted words missed them. Also: "any edit to this file reddens, including a comment" was an absolute and is refutable — a reindent, added blank lines, tabs, and a form feed all stay green, because `_normalise_lines` collapses whitespace. Restated as what is actually true (a line's TOKEN sequence, a comment's words included) plus the reason the tolerance is currently inert: this file has no template literal and no ASI-sensitive token outside a comment. And a YAML single-quote escape had leaked from the frontmatter into the markdown BODY, where `''` renders literally. No code change; the guard is unchanged and still 73/0. Refs: #887 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF --- docs/decisions/README.md | 2 +- .../ci/image-build-delegates-the-spa-suite.md | 7 +++++-- docs/guard-inventory.md | 2 +- .../test_image_build_delegates_the_spa_suite.py | 12 ++++++++---- 4 files changed, 15 insertions(+), 8 deletions(-) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index d7f445e92..210ccd8de 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -54,7 +54,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job's JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator's own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI's registry secret was the ADMIN account's basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential's route (an ALLOW-LISTED user credential's forgery carries a matching `creator` and is inherited as a verdict; an Actions job's carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger's `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator's own token remains unfixable; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) | | `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) | -| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`'s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. The count of such routes lives in the `docs/guard-inventory.md` row and nowhere else — it briefly existed in three places with two different values, which is how a number stays wrong. It is a running total, wrong at every count so far. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | +| `ci.image-build-delegates-the-spa-suite` | `docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`'s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step's `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step's own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`'s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts` PINNED WHOLE — it decides what the suite collects and what `vite build` loads, and pinning a BLOCK of it was defeated seven measured ways across three rounds, so the partial match was withdrawn rather than respelled an eighth time; and the ABSENCE of any config file that outranks it — `vitest.config.*` for vitest, and `vite.config.js`/`.mjs` for `vite build`, whose `DEFAULT_CONFIG_FILES` lists them BEFORE `.ts`. Pinning one file is worthless while a second can outrank it, and that family went short twice. The gating job's `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. The count of such routes lives in the `docs/guard-inventory.md` row and nowhere else — it briefly existed in three places with two different values, which is how a number stays wrong. It is a running total, wrong at every count so far. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != 'pull_request'` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`. | 2026-08-30 | [link](records/ci/image-build-delegates-the-spa-suite.md) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) | | `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) | diff --git a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md index 35fb265ab..45aee9c77 100644 --- a/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md +++ b/docs/decisions/records/ci/image-build-delegates-the-spa-suite.md @@ -5,7 +5,7 @@ status: active since: '2026-08-30' supersedes: none superseded-by: none -rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts`''s `test:` block, which decides what the suite collects, and the ABSENCE of any `vitest.config.*` — vitest resolves its own config in preference to `vite.config.*`, so pinning one file is worthless while a second can outrank it. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. The count of such routes lives in the `docs/guard-inventory.md` row and nowhere else — it briefly existed in three places with two different values, which is how a number stays wrong. It is a running total, wrong at every count so far. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' +rule: '`docker/Dockerfile` runs no vitest suite. Its web-build stage lints, typechecks and BUILDS the SPA; the suite runs once, unfiltered, in `docker-build.yml`''s `test` job on a real checkout, and `build` carries `needs: [test, migrations, scan]` so no image is published past a red suite. THAT EDGE IS NOW THE ONLY LAYER, so the guard checks it is real and not merely present — and it does so by PINNING TEXT rather than parsing it. The commands each SPA-carrying Dockerfile stage runs, and the gating step''s `run:` body and `if:`, are compared as strings against a declared pin; the step and its job must carry no `continue-on-error` in any spelling, the job no job-level `if:`, and the publish step must keep its own `docs_only` gate. A guard that asks what a command MEANS was wrong nine times in three review rounds. A pin is immune to a different SPELLING of the command, which is that whole class; it is not immune to the same text meaning something else, so the routes to that are pinned or refused too: `working-directory` and the step''s own `shell:`, the workflow `defaults.run.shell` and the job-level `defaults` overriding it, a stage `SHELL`, `web/package.json`''s script map (pinned WHOLE — selecting on the literal `vitest` missed `npm run test`, `npm t` and the `prebuild`/`preinstall` lifecycle hooks), `web/vite.config.ts` PINNED WHOLE — it decides what the suite collects and what `vite build` loads, and pinning a BLOCK of it was defeated seven measured ways across three rounds, so the partial match was withdrawn rather than respelled an eighth time; and the ABSENCE of any config file that outranks it — `vitest.config.*` for vitest, and `vite.config.js`/`.mjs` for `vite build`, whose `DEFAULT_CONFIG_FILES` lists them BEFORE `.ts`. Pinning one file is worthless while a second can outrank it, and that family went short twice. The gating job''s `container:` image is left to `ci.image-pin-population` rather than guarded twice. What is unmodelled is a LIST, not an "only": an `ENV` changing what a pinned `RUN` resolves, the plugin BODIES, and a publish through another action. The count of such routes lives in the `docs/guard-inventory.md` row and nowhere else — it briefly existed in three places with two different values, which is how a number stays wrong. It is a running total, wrong at every count so far. They share one shape, which is the transferable part: A PIN ASSUMES IT IS PINNING THE ARTIFACT THAT STILL DECIDES, and every route found so far is authority moving to another file, another occurrence, another workflow, or a hook the pinned command invokes. Do NOT restore a filtered in-image run by naming the specs that cannot run there: that list is a population nothing derives, and the red it produces when it goes stale is unreachable on a PR — `Build & push image (amd64)` is `if: github.event_name != ''pull_request''` — so it lands on `main` and on the `v*` tag path, where it fails the release cut. Held in both directions by `scripts/tests/test_image_build_delegates_the_spa_suite.py`.' signals: 'image build fails on main only · every image build fails · latest not republished · release cut fails at the image build · gitless build STAGE not context · node:22-bookworm-slim ships no git · virtual:etv-tracked-source-files could not read the git index · hand-maintained exclude list · exclusion list is the underived population · suite members needing the git binary vs a checkout · COPY .git would not help · needs edge is the only gate · unfiltered gating run · paths: `docker/Dockerfile`, `.gitea/workflows/docker-build.yml`, `scripts/tests/test_image_build_delegates_the_spa_suite.py`, `web/vite-plugins/trackedSourceFiles.ts` · issues: #887, #883, #819, #806, #420' mechanics: 'Populations are DERIVED from the git index (tracked Dockerfiles and workflows, and the stages within them carrying the SPA source); the command lines in those places are PINNED as text. A population decides what is CHECKED, so a hand-written one goes silently short; a pin decides what is EXPECTED, so a stale one goes loudly red. Publishing jobs come from the `docker/build-push-action` step and the Dockerfile each builds from that step''s `file:` input, so `ci-image.yml` is out of scope by derivation rather than by an exemption. 73 mutants, 0 missed (development battery).' --- @@ -62,7 +62,10 @@ not: green on every PR, red on `main` and on the release tag. **The guard's own mechanism was replaced once, and the reason belongs here rather than only in the commit log, because the next person to widen it needs it.** Three versions asked whether a given command runs the suite and whether its failure can be swallowed. That predicate was wrong nine times across three cold-review rounds — executed heredocs treated as data, `#` truncating a command mid-word (including the live `${#reports[@]}` idiom), compound punctuation welding commands, `npm t` and `./node_modules/.bin/vitest` and `timeout`/`su -c` wrappers unrecognised, `true || npm test` counted as a run it never performs, and `continue-on-error: ${{ … }}` slipping a two-literal check. Twice, a clause added to remove a FALSE RED opened a FALSE GREEN on the guard's headline assertion. The mechanism was withdrawn rather than patched a tenth time: the risky command lines are now compared against a pin, so no spelling has to be recognised in order to be rejected. The cost is real and is the point — a legitimate edit to those lines reddens the guard and its author updates the pin deliberately. -A fourth cold review then attacked the pin itself and found the honest limit: the replacement removes the SPELLING problem outright, but the same pinned text can be made to mean something else, and two mutants re-armed this very defect through `web/package.json` — `RUN npm run build` executes whatever that file says — with every pin still matching. A fifth round then found the same mistake inside the fix: the package.json check SELECTED on the literal `vitest` where a PIN was available, and four one-line edits that never spell it — `npm run test`, `npm t`, and the `prebuild`/`preinstall` lifecycle hooks — each re-armed the defect with every other pin matching. The whole script map is pinned now, and so is `vite.config.ts`''s `test:` block. +A fourth cold review then attacked the pin itself and found the honest limit: the replacement removes the SPELLING problem outright, but the same pinned text can be made to mean something else, and two mutants re-armed this very defect through `web/package.json` — `RUN npm run build` executes whatever that file says — with every pin still matching. A fifth round then found the same mistake inside the fix: the package.json check SELECTED on the literal `vitest` where a PIN was available, and four one-line edits that never spell it — `npm run test`, `npm t`, and the `prebuild`/`preinstall` lifecycle hooks — each re-armed the defect with every other pin matching. The whole script map is pinned now, and so is `web/vite.config.ts` — WHOLE, after three further +rounds each defeated one spelling of a partial match of it, and two more routes bypassed the marker +entirely because `defineConfig` is the identity function and a later spread replaces what was +matched. The general lesson is worth more than any of the fixes: replacing a predicate with a pin moves the problem from "can the checker recognise this?" to "is the checked text still the whole story?". The second question has a short, workable answer where the first did not — but it is still a question, the route list has been wrong at every count so far, and the honest form of that is a running total with its history attached rather than a closure claim. diff --git a/docs/guard-inventory.md b/docs/guard-inventory.md index a3b2e8b11..d5d033583 100644 --- a/docs/guard-inventory.md +++ b/docs/guard-inventory.md @@ -173,7 +173,7 @@ The audit #806 asked for, recorded whichever way it came out, because "we looked | `test_workflow_job_guards.py` | `.gitea/workflows/*.y*ml` | **yes** — every tracked workflow's jobs must each declare a `CI_JOB_ROLE`, and the guard-declaring ones must match `## Workflow-job guards` both ways | index-derived from the start (ersatztv#786); registered in `test_guard_populations_derive_from_git.py` so the shared proof covers it | | `test_pr_changed_files.py` | `.gitea/workflows/*.y*ml` | **yes** — "no OTHER workflow writes the review-verdict status", and since ersatztv#748 "every tracked workflow declares a `permissions:` block", with NO exemption list (the one `ci-image.yml` briefly needed was deleted when #744 landed, per #835) | converted. Not on #806's list: an untracked `.yaml` dropped in `.gitea/workflows/` reddened two guards while absent from the index — the issue's list of files to assess was a starting point, not the population | | `test_workflow_persist_credentials.py` | `.gitea/workflows/*.yml` + `*.yaml` | **yes** — every `actions/checkout` in every workflow must drop the persisted credential | derived from the index from the start (#835). Shipped with NO exemption list, which is why it waited for #744: the one non-compliant checkout would otherwise have bought a permanent entry that keeps passing after its reason expires | -| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | +| `test_image_build_delegates_the_spa_suite.py` | DERIVED: tracked Dockerfiles and workflows from the git index; the stages within them that carry the SPA source. PINNED (hand-written, and the distinction is the whole design): the commands each such stage runs, and the gating `Test SPA` step's `run:` body and `if:` | **yes on the derived populations, by pin rather than by predicate** — every SPA-carrying stage must have a pin and match it; every image-publishing job whose Dockerfile has such a stage must transitively `needs:` the gating job | ersatztv#887. **THE PARSER WAS WITHDRAWN, and that is the finding worth carrying.** Three versions of this guard asked "does this command RUN the suite, and can it FAIL?" of arbitrary shell text, and that predicate was wrong NINE times across three cold-review rounds, always the same mechanism: heredoc bodies skipped as data when BuildKit EXECUTES `RUN < `run: >` folds two commands into one; (3) Gitea's treatment of a SKIPPED `needs:` job is not modelled — the guard forbids the job-level `if:` that would create one rather than reasoning about it; (4) the pins are hand-written and a legitimate edit reddens this guard, which is the intended review trigger and the stated cost | | `web/src/api/pageSizeCallSites.guard.test.ts` | `import.meta.glob` over `web/src/**/*.{ts,tsx,mts,cts}`, INTERSECTED with the git index | **yes** — an unregistered discovered site fails | converted under #819. The population is the glob INTERSECTED with the git index, and the walk/index divergence is asserted in BOTH directions WITHIN a shared scope: a key the index does not carry is dropped, and a tracked, on-disk, in-scope path that never reached the scanner FAILS rather than shrinking the population silently (that direction catches a dotfile, which the glob cannot match, and a disk/index spelling divergence, which `core.ignorecase` and NFD/NFC normalisation make permanent). The SCOPE itself cancels out of that equality — both sides call one `isInScopeSourcePath` — so narrowing it moves both sets together and no assertion comparing the two sides can see it. That is closed separately, by re-deriving the predicate from its two components over the whole tracked index (`the shared scope predicate is EXACTLY its two documented components`). Two mechanisms carry that. FIRST, the scope predicate is proved by a CLOSED-FORM restatement — it may share no helper, at any depth, with the predicate it checks, since anything shared sits on both sides of the comparison and cancels. FOUR earlier attempts were each measured going green while removing real files: a table of example paths (four of `src/`'s eight directories, missed a 23-file narrowing), a decomposition delegating its filename half to `isScannableSourceFileName` (11 files), one still sharing a `basename` helper (15 files, and blind to a planted call site), and one still sharing the tracked-file array every comparison was derived from (8 files, also blind). SECOND — because that last one showed a restatement cannot police the population it reads — the plugin runs a separate `git ls-files --others` query and the guard requires every in-scope walked path to appear in `tracked ∪ others`; narrowing `tracked` cannot suppress that, since it adds nothing to `others`. SIX residuals, NOT a claim of closure, each with its direction MEASURED by planting a real `pageSize` call site rather than reasoned about. FAIL-NOISY (they redden a checkout; they cannot hide a call site): (2) a scope term matching no tracked path today survives until the day it first matches one; (4) the WIRING in `listSourceFiles`, indistinguishable on a clean checkout where the walk set and the index set agree on every in-scope key (they are not literally equal — the index also carries `.css`, which the glob never yields), so no assertion COMPARING THE TWO POPULATIONS can tell which is passed; (5) `ABSENT_FROM_DISK` emptied. BLIND — each hid a planted `pageSize: 100` with the whole suite green: (1) a COORDINATED edit of the scope predicate and BOTH closed-form restatements, which is three sites and not two, since a second restatement guards `expectedTrackedSources` (measured: the two-site edit reddens, the three-site edit does not) — tolerated because a three-site policy edit is review-visible in a way a one-line slip is not; and (3) a misdescription by the plugin's THREE outputs — not only a mispartition of `tracked`/`others`, which preserves the union any consumer compares, but a FATTENED `absentFromDisk`, which subtracts real files from the hole-detection direction. The fattening is now caught wherever it overlaps a file the walk found (a walked path is on disk by construction, so it can never be legitimately absent); restricted to paths the walk cannot see anyway — a dotfile, a case-divergent name — it stays invisible from the consuming side and is answerable only by testing the derivation directly, and the real-git tests that do so catch an UNCONDITIONAL misdescription, not one keyed on a path pattern. Neither blind residual is tolerated for being harmless. A sixth is specific to WATCH mode and is stale in BOTH directions, so it belongs to neither group: the virtual module has no backing file, so the index is read once per dev-server lifetime while the glob refreshes. A file CREATED mid-session reddens the cross-check misleadingly (noisy); a file already UNTRACKED when the watcher started keeps that classification when staged mid-session, so its call sites are never scanned — MEASURED green across both phases while `npx vitest run` on the same tree reports `UNREGISTERED`, i.e. blind, and a window that did not exist before #819 because the population was then the walk. A watch-mode green is therefore not authoritative for this guard; restarting the watcher clears the noisy case and opens the blind one, so the remedy is `npm test -- --run`. Invalidating the module from `configureServer` was implemented, measured and REJECTED: it fixes the created-mid-session red and additionally blinds the create-then-stage sequence, while the already-untracked-then-staged sequence is blind either way, since `git add` fires no watcher event in either design. The index is read by `web/vite-plugins/trackedSourceFiles.ts` in Vite's own Node context and handed to the app project as the virtual module `virtual:etv-tracked-source-files`. That is what reaches the index without admitting `@types/node` to `tsconfig.app.json` — the obstacle that deferred this, since wiring those types in was tried and reverted (it leaked Node's `setTimeout` into the app project and broke three unrelated tests). The plugin throws rather than falling back to the unfiltered walk when git fails, reports zero files, or cannot learn the Vite root. Residual, stated: a file tracked but DELETED in the working tree is subtracted deliberately, since an unstaged deletion is a normal developer state and a guard red on one gets ignored (#806); it is distinguished from the hole cases above by an on-disk existence check, not conflated with them | | `web/src/api/completeAnnotations.guard.test.ts` | TWO derived populations: the `Complete<…>` annotations across `web/src/**/*.{ts,tsx,mts,cts}` INTERSECTED with the git index, and the droppable SCHEMAS parsed out of the generated `src/api/generated/v1.d.ts` | **partial, and the split is stated** — set equality holds for the SCHEMA population (a new optional member fails until dispositioned) and every tracked in-scope path must be supplied by the glob; there is NO closed-form restatement of the scope predicate, so a coordinated edit to it is caught by five named path pins AND a 0.95 population-ratio floor, not by a completeness proof | ersatztv#820. Turns `test_optional_request_members.py`'s `COVERED` disposition — worded "the builder is annotated `Complete`", a claim about ANOTHER LANGUAGE'S source that nothing checked — into a check, and makes `docs/spa-conventions.md` §4b's prohibitions executable as asserted ABSENCES. It found one live defect: `playouts.ts` declared two request types as hand-written mirrors SHADOWING generated schemas of the same name, so their `Complete<>` was checking a local copy rather than the contract. **NARROWED after four BLOCKED review rounds, and the narrowing is the point.** It originally also derived the write WRAPPERS and required the annotation on the wrapper parameter. Every blocker across those rounds came out of that one mechanism — the obligation attached to the wrong population; reachability mistaken for protection (`Complete` is shallow, so a wrapper annotation never reached a nested schema); body discovery keyed on a parameter NAME, then on parameter-versus-local; and finally an `export function` -> `export const` refactor that removed real protection while both the AST scan and its supposedly independent regex cross-check stayed blind TOGETHER, because both keyed on the same token. Five defects from one mechanism is the `process.enumerate-workaround-behaviors-before-deleting` signal to remove it rather than patch a sixth time, and ~250 lines went with it. WHAT IS NOT COVERED, listed because the earlier version of this row stated residuals in the direction that was actually covered: (1) the obligation is per-SCHEMA, not per-SITE and not per-WRAPPER — deleting the `Complete<>` from an API wrapper stays green as long as some production file still names that schema, so the wrapper half of §4b rests on review; (2) it is a TOKEN-PRESENCE check, so a dead `export type X = Complete` that nothing uses discharges the obligation as well as a live builder does — it catches deletion, which is the failure actually observed in #807, not substitution; (3) the PHANTOM direction needs a fresh literal in a contextually typed position and is not checked at all (sites-in-code, #777); (4) the test-file exclusion covers `*.test.*`, `*.spec.*` and the ONE setup file `vite.config.ts` names, whose path is pinned so a rename reddens — a SECOND `setupFiles` entry is loaded by vitest every run and WOULD discharge the obligation; that needs a three-site coordinated edit and is tolerated on the terms the sibling guard states for its own; (5) a MUST-NOT-ANNOTATE violation inside a `*.guard.test.ts` is not seen, since that file class is excluded to keep `completeRequest.guard.test.ts`'s synthetic `Complete<{…}>` fixtures out of the resolver. The disposition VALUES are cross-checked against the authoritative Python table by `scripts/tests/test_complete_annotation_dispositions.py` — without it, flipping one row from `ANNOTATED` to `CREATE` silently retired the requirement, which cold review demonstrated. Nine mutations were witnessed by hand across development, NOT re-executed per suite, so this guard claims no standing `MUTATION` grade — the same footing as `pageSizeCallSites.guard.test.ts`. (Its Python cross-check `test_complete_annotation_dispositions.py` DOES carry a declared harness-executed mutation and is graded accordingly.) A SIXTH residual, shared with its sibling: every population here derives from `trackedSources.tracked`, so a narrowing inside `web/vite-plugins/trackedSourceFiles.ts` cancels out of every comparison — this guard never reads the plugin's separate `others` query, and relies on `pageSizeCallSites.guard.test.ts` and `trackedSourceFiles.test.ts` policing the plugin. A coordinated scope edit is caught by five named path pins AND a 0.95 population-ratio floor whose denominator is computed by a different expression, not by a completeness proof | | `test_ci_release_path_scan_job.py` | `.gitea/workflows/*.y*ml` + `scripts/**` | **no** — a fixture assembling a tmp harness, asserted about behaviour not membership | takes its file LIST from the index anyway, for hermeticity not completeness: `shutil.copytree` copied whatever was on disk, including untracked files and `scripts/__pycache__`, into a tree whose behaviour the probes then measure. Content still comes from the working tree. The copy is not a git repo, so the two files this step RUNS may not use the helper — see the fixture docstring | diff --git a/scripts/tests/test_image_build_delegates_the_spa_suite.py b/scripts/tests/test_image_build_delegates_the_spa_suite.py index cd240bafa..297f5ac6a 100644 --- a/scripts/tests/test_image_build_delegates_the_spa_suite.py +++ b/scripts/tests/test_image_build_delegates_the_spa_suite.py @@ -241,9 +241,12 @@ PINNED_PACKAGE_SCRIPTS: dict[str, str] = { # REPLACES what the pin matched). That is `every-blocker-was-one-mechanism-so-delete-it` at the # count where it says withdraw: the mechanism was partial matching, and it is gone. # -# STATED COST: any edit to this file reddens, including a comment. It is 48 lines, nothing generates -# it, and it is the same move already made for `web/package.json`'s script map — a PIN where a -# partial match was being defended. +# STATED COST: any edit that changes a LINE'S TOKEN SEQUENCE reddens, a comment's words included. +# Whitespace and blank lines are free — `_normalise_lines` collapses them, which is what lets a +# reflow through. That tolerance is inert for this file's current content (no template literal and +# no ASI-sensitive token: every `return`/`=>`/backtick in it sits inside a `//` comment) and would +# stop being inert if one were introduced. It is 48 lines, nothing generates it, and it is the same +# move already made for `web/package.json`'s script map — a PIN where a partial match was defended. VITE_CONFIG = "web/vite.config.ts" PINNED_VITE_CONFIG = """import { fileURLToPath, URL } from 'node:url'; import react from '@vitejs/plugin-react'; @@ -791,7 +794,8 @@ def test_the_VITE_CONFIG_is_exactly_what_was_reviewed() -> None: if index >= min(len(actual), len(pinned)) or actual[index] != pinned[index] ) + "\n\n This file decides which specs the gating run collects and what `vite build` loads " - "in the gitless stage. Any edit — including a comment — reddens, deliberately: every " + "in the gitless stage. Any edit that changes a line's TOKENS reddens — a comment's words " + "included, whitespace and blank lines not — deliberately, because every " "partial match of it was defeated. Update `PINNED_VITE_CONFIG` in the same commit, with a " "reason." ) -- 2.47.3