Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 35s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 57s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 37s
PR Gates / Docs update reminder (pull_request) Successful in 1m0s
PR Gates / decisions lifecycle (pull_request) Successful in 20s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
review-verdict/h10 Review-verdict: MERGEABLE @ a7d91bf (base: main)
Review verdict / Set review-verdict status (pull_request_target) Successful in 45s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m25s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m17s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 19m27s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m4s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
`docs.no-session-narrative` reaches every durable artifact, but its detector scanned only `docs/**/*.md` and root markdown, and nothing had ever swept the rest. The issue named four sites from one grep and called them a floor. Deriving the population instead — a whitespace-joined sweep over every tracked file outside the detector, for the detector's own phrasings plus the attribution and review-round class #812 found — gave 453 sites in 108 files at `fb5592971`, and a second pass for phrasings the first list missed (hyphenated `round-N`, "an earlier version", "the reviewer proved") added residuals in the same files. Every site was classified with #812's three dispositions (CUT / SEVER / KEEP with its sub-kind) under the who-benefits test; the per-site manifests are on the PR. The rejected designs, tested-and-rejected fixtures, measurements and traps stay; the attribution of who found them and the round in which they were found go. The detector's population grows to `.claude/`, `.gitea/`, `.husky/` and `scripts/` regardless of extension, minus the detector and its own test (whose fixtures ARE the phrasings) and minus `scripts/tests/fixtures/` (test data, including decision-record copies — the same reasoning as the records' own exemption, and what keeps the record's depth measurement true), and `--all` lists tracked REGULAR files only — a symlink's content is its target and a gitlink has none. The #812 argument for leaving `docs/superpowers/**` in the population runs the other way here: `--diff` sees only ADDED lines, and 287 of the 453 sites were under 30 days old — this corpus is where narrative is being added, so the advisory nudge has reach. Density agrees: 56 line-mode hits over the 113 regular files the predicate admits, against 9 over 66 docs files before #812. `web/` and C# stay out on the same measurement (3 of 74 PATTERNS-matching sites, ~4,600 files). The predicate did not grow: PATTERNS matched 74 of 453 sites, and widening the word list to the attribution class is the treadmill the withdrawn parity test ran on. The population oracle is restated over segments with the new arms, the synthetic cross product gains the process heads and non-markdown extensions, a fixture witnesses that a tracked symlink is neither scanned nor counted, a `.py.bak` axis separates a by-name exemption from a `startswith` over the same tuple, and eight mutants (drop the process arm, drop the by-name exemption, exempt by `startswith`, drop or add a prefix, drop the fixtures exemption, list only markdown, drop the symlink filter, test the mode per row instead of per path) each redden it. A pre-existing silent drop in `--diff` goes with it: git tab-terminates a `+++` filename that contains a space, and the kept tab made `is_scanned_path` refuse the file with no notice — fixed, with a positive control and its own mutant. Code is unchanged by construction, measured per file type against `origin/main`: Python modules are AST-equal with docstrings stripped, except `#` lines inside the embedded fixture programs (string literals) of three test modules; workflows differ only in `#` lines inside `run:` block scalars; shell, C#, TypeScript and jq are equal with comment lines stripped. The stated exceptions: the detector and its test, 26 vitest titles that carried review-round or severity labels or a reviewer attribution (call sites whose title changed — every changed title line walked back to its `it(` / `it.each(...)(` anchor, so a `' + '` concatenation counts once), two registry note strings and the mutation manifest's prose fields. scripts/tests: 1565 passed. Web: lint, typecheck, 1319 tests green. Closes #876. Decisions-Edit: yes Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEcBoFw7ctrf3Nb7R7x7wk
1053 lines
57 KiB
Python
1053 lines
57 KiB
Python
"""The SPA vitest suite never runs inside an image build, and the image is gated on a step that does
|
|
run it, unfiltered and non-advisory (ersatztv#887).
|
|
|
|
WHAT THIS IS PROTECTING. `docker/Dockerfile`'s `web-build` stage is gitless twice over. It copies
|
|
only `web/` and `design-system/`, so it holds no `.git` — a property of the STAGE, not of the build
|
|
context, which is the repository root (`context: .`) and whose `.dockerignore` does not exclude
|
|
`.git`: the directory is there to be copied and simply is not. And `node:22-bookworm-slim` ships no
|
|
git binary, which is why a `COPY .git` would not help either. Members of the SPA suite need one or
|
|
the other — `pageSizeCallSites.guard.test.ts` and `completeAnnotations.guard.test.ts` derive their
|
|
file population from `git ls-files` and refuse to fall back to a directory walk
|
|
(`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds a
|
|
real temporary repository.
|
|
|
|
Running the suite there anyway therefore requires a list of the members that cannot run, and that
|
|
list is a population nothing derives: ersatztv#883 added a third member without updating a
|
|
hand-written pair of `--exclude`s, and every image build failed from that commit on. The red was
|
|
structurally unreachable on a PR — `Build & push image (amd64)` carries
|
|
`if: github.event_name != 'pull_request'` — so it landed on `main` and on the `v*` tag path, where a
|
|
release cut fails at the image build and `:latest` stops being republished.
|
|
|
|
## WHY THIS GUARD PINS TEXT INSTEAD OF PARSING IT
|
|
|
|
A predicate asking "does this command RUN the suite, and can it FAIL?" of arbitrary shell text was
|
|
wrong nine times, and every one of the nine was the same mechanism:
|
|
|
|
* heredoc bodies — skipped as data, but BuildKit EXECUTES `RUN <<EOF`, and the opener regex also
|
|
fired inside quotes (`echo "tags<<__EOT__"`), which blinded the scan over the last 303 lines of
|
|
`docker-build.yml`. Wrong in BOTH directions at once.
|
|
* `shlex.shlex` does not clear `commenters` the way `shlex.split` does, so `#` truncated a command
|
|
mid-word — including the live `${#reports[@]}` idiom — and made this file's own stated residual
|
|
("only a LEADING `#` is stripped") false.
|
|
* compound punctuation (`);`, `;;`) welded two commands into one segment.
|
|
* `npm t`, `./node_modules/.bin/vitest`, `pnpm vitest`, `yarn vitest`, `node …/vitest.mjs`,
|
|
`timeout …`, `su -c …`, `if npm test; then` — all invisible.
|
|
* `true || npm test` counted as the gating run while never executing.
|
|
* `continue-on-error: ${{ … }}` passed a check that compared against two literals.
|
|
|
|
Nine defects from one mechanism is `process.enumerate-workaround-behaviors-before-deleting`'s signal
|
|
to remove it rather than patch a tenth time, and the mechanism is removed here. **Nothing in this
|
|
file decides what a command MEANS.** The commands that may run in the risky places are PINNED as
|
|
text; anything else — a heredoc, a wrapper, a new spelling, a rewrite — is simply not equal to the
|
|
pin and reddens.
|
|
|
|
WHAT A PIN IS AND IS NOT IMMUNE TO, stated precisely because the absolute "a pin cannot have a
|
|
false green" is FALSE, refuted by execution. A
|
|
pin is immune to the class that defeated the parser: a DIFFERENT SPELLING of the command. It is not
|
|
immune to the same text MEANING something else. The routes found so far, and what each is worth,
|
|
are enumerated ONCE — in `docs/guard-inventory.md`'s row for this guard, which is where the count
|
|
lives. It is not repeated here: it existed in three artifacts with two different values at one
|
|
point, and re-syncing three copies is how a number stays wrong (`state-the-invariant-not-the-
|
|
measurement`). What this file states is the RULE they are instances of, below.
|
|
|
|
Handled here: `working-directory` and the gating step's own `shell:`; the workflow
|
|
`defaults.run.shell` and the job-level `defaults` that would override it; a stage `SHELL`, refused
|
|
outright; `web/package.json`'s script map, pinned WHOLE after SELECTING on the literal `vitest`
|
|
missed `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS npm runs for
|
|
`npm run build` and `npm ci`; `web/vite.config.ts` PINNED WHOLE — pinning a block of it was
|
|
defeated seven measured ways across three rounds, so the partial match was withdrawn rather than
|
|
respelled an eighth time; and the ABSENCE of any config file
|
|
that outranks the pinned one — `vitest.config.*` for vitest, and `vite.config.js`/`.mjs` for `vite
|
|
build`, whose `DEFAULT_CONFIG_FILES` puts them BEFORE `.ts`. Pinning one config is worthless while a
|
|
second can outrank it, and both halves of that were measured: a `vitest.config.ts` made the suite
|
|
collect zero specs and exit 0, and a `vite.config.js` ran the suite inside the gitless stage with
|
|
all 1411 tests green.
|
|
|
|
Cited rather than re-asserted: the gating job's `container:` image, pinned by
|
|
`test_ci_image_pin_population.py` — two guards on one condition mask each other (#685).
|
|
|
|
NOT modelled, as a list rather than an "only": an `ENV` in a pinned stage that changes what a pinned
|
|
`RUN` resolves or executes; the BODIES of the pinned plugins (there are two, and `react()`'s is
|
|
third-party and unmitigated — only `trackedSourceFilesPlugin`'s laziness is a mitigation); a
|
|
dependency's own install script, reached through `npm ci` and `web/package-lock.json`; and a publish
|
|
through some action other than `docker/build-push-action`.
|
|
|
|
So: the pin removes the SPELLING problem outright and reduces the MEANING problem to a list that can
|
|
be worked through — a much better place than a predicate that was wrong nine times. It is not
|
|
"cannot be wrong", and the list's own history says so.
|
|
|
|
THE SHAPE THOSE ROUTES SHARE, which is the thing to check when adding a pin rather than a list to
|
|
memorise: **a pin assumes it is pinning the artifact that still DECIDES.** Every instance found so
|
|
far is authority moving somewhere the pin is not looking — to another FILE (`vitest.config.*`
|
|
outranks `vite.config.ts`), to another OCCURRENCE in the same file (a decoy first `test: {`), to
|
|
another WORKFLOW (a `needs:` edge naming a job called `test` that is not this one), or to a HOOK the
|
|
pinned command invokes (a vite plugin, an npm lifecycle script). Ask of any new pin: what else could
|
|
decide this, and would the pin still match?
|
|
|
|
Two candidates for that class were probed and are NOT routes, recorded so the next reader does not
|
|
re-derive them. The `setupFiles` entry is pinned by NAME and its CONTENT is not, but a setup file
|
|
that terminates the run is FAIL-NOISY: `process.exit(0)` at the top of `src/setupTests.ts` reports
|
|
`121 failed (121)`, not a green. And `tsconfig*.json` shapes what `tsc -b` compiles, not what vitest
|
|
collects. What remains genuinely unpinned in that direction is a dependency's own install script,
|
|
reached through `npm ci` and `web/package-lock.json` — a supply-chain concern wider than this guard,
|
|
named rather than claimed to be covered.
|
|
|
|
The cost is stated rather than hidden: a legitimate edit to those command lines reddens this guard
|
|
and its author must update the pin. That is the same deliberate review trigger `KNOWN_GATE_*` uses,
|
|
applied to the two places where a silent change publishes an untested image.
|
|
|
|
## THE INVARIANT, in the three parts it has
|
|
|
|
* NEGATIVE — every Dockerfile STAGE that carries the SPA source runs exactly the pinned commands.
|
|
A suite run put back in any spelling is not equal to the pin.
|
|
* POSITIVE — every job publishing an image from an SPA-carrying Dockerfile transitively `needs:`
|
|
the job holding the pinned gating step.
|
|
* EFFECTIVE — that step's `run:` body, its `if:`, and the publishing step's own docs-only `if:`
|
|
are pinned; the step and its job carry no `continue-on-error` in ANY spelling; and the job
|
|
carries no job-level `if:`.
|
|
|
|
POPULATIONS, all derived from the git index via `tracked_files` — never a filesystem walk and never
|
|
a hand-written list (`testing.guard-derives-population-from-source`, ersatztv#806). The PINS are
|
|
hand-written by necessity, and the difference matters: a population decides WHAT IS CHECKED (so a
|
|
hand-written one goes silently short), a pin decides WHAT IS EXPECTED (so a stale one goes loudly
|
|
red). Only the second is safe to write by hand.
|
|
|
|
WHAT IS NOT COVERED, stated rather than implied:
|
|
|
|
* A Dockerfile stage that does not carry the SPA source is unpinned. The boundary is what
|
|
`copies_spa_source` recognises — a context copy of `web/` or `.`, a matching GLOB, or a stage
|
|
copy whose source names `web` — and NOT the wider "has the suite available to run". A stage
|
|
copy that RENAMES the tree on the way
|
|
in (`COPY --from=web-build /source/web /elsewhere`) has no `web` segment in its source and is
|
|
not recognised. What that boundary EXCLUDES, measured: a source that is an ANCESTOR of the
|
|
tree (`COPY --from=web-build /source /src` brings `/source/web` along without naming it), and
|
|
one that names it only as `/source/.`. Both are backstopped by the substring sweep — an
|
|
ancestor-copy stage running `npm t`, `npx vitest` or a new npm script is caught. (MEASURED, not
|
|
reasoned: a destination rename such as `/source/web /elsewhere` IS recognised, because only
|
|
sources are read.) The
|
|
false-NEGATIVE direction is what drops a stage out of the check, so the predicate is pinned in
|
|
both directions by its own test, and the substring sweep below is the backstop for whatever it
|
|
still misses.
|
|
* A step or stage OTHER than a pinned one could run the suite, and the substring sweep is what
|
|
reports it. `SUITE_MENTIONS` is a hand-written SELECTOR — a third category the population/pin
|
|
split does not name, and the one that behaves worst: a population going short is caught by an
|
|
equality, a pin going stale goes loudly red, but a SELECTOR going short is silent. It was short
|
|
by exactly one entry (`npm t`, npm's own alias, which this file names among the spellings that
|
|
defeated the parser) until a stage was MEASURED escaping through it. `PUBLISH_ACTION` is
|
|
the same category and is NOT covered — anti-vacuity proves the selector is non-empty, which the
|
|
real `build` job guarantees, and cannot prove it complete: a publish through a different action
|
|
is invisible to it. `test_no_run_BODY_builds_or_pushes_an_image` covers only the shell-body form.
|
|
* The gating body is compared LINE BY LINE with each line whitespace-normalised, not byte-exact.
|
|
Indentation and reflow are free; line boundaries are not, because a newline separates two
|
|
commands — changing `run: |` to `run: >` folds them into one and the whole-text comparison could
|
|
not see it.
|
|
* Gitea's own evaluation of a skipped `needs:` job is not modelled; the guard forbids the job-level
|
|
`if:` that would create one rather than reasoning about it.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import re
|
|
from fnmatch import fnmatch
|
|
from pathlib import Path
|
|
|
|
import yaml
|
|
|
|
from scripts.tests import tracked_files
|
|
from scripts.tests.tracked_files import tracked_paths
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parents[2]
|
|
WORKFLOW_DIR = ".gitea/workflows"
|
|
WORKFLOWS = (WORKFLOW_DIR, ("*.yml", "*.yaml"))
|
|
|
|
# The action every image publish in this repo goes through. Matched on the last two path segments
|
|
# for the reason `test_workflow_persist_credentials.py` gives: Gitea accepts a full action URL, and
|
|
# a host is not required to contain a dot, so no hostname heuristic is safe.
|
|
PUBLISH_ACTION = "docker/build-push-action"
|
|
|
|
# Instructions that bring files from the build context into the image.
|
|
COPY_INSTRUCTIONS = frozenset({"ADD", "COPY"})
|
|
|
|
# ---------------------------------------------------------------------------------------------
|
|
# THE PINS. Update these deliberately, in the same commit as the change they describe, and say why.
|
|
# ---------------------------------------------------------------------------------------------
|
|
|
|
# Every `RUN` in a Dockerfile stage that carries the SPA source, keyed `<path>::<stage>`. Whitespace
|
|
# is normalised (see `_normalise`) so a reflow is not a red, but the tokens are exact.
|
|
PINNED_STAGE_COMMANDS: dict[str, tuple[str, ...]] = {
|
|
"docker/Dockerfile::web-build": (
|
|
"RUN npm ci",
|
|
# Lints, typechecks and BUILDS the SPA. It must not TEST it: this stage has neither a git
|
|
# checkout nor the git binary, and naming the specs that cannot run there is the
|
|
# hand-maintained population ersatztv#887 removed.
|
|
"RUN npm run lint && npm run typecheck && npm run build",
|
|
),
|
|
}
|
|
|
|
# The step that runs the suite and gates the image, pinned whole. Its body being exact is what makes
|
|
# "unfiltered", "actually executes", and "cannot be suppressed" true without parsing any of it.
|
|
GATING_WORKFLOW = ".gitea/workflows/docker-build.yml"
|
|
GATING_JOB = "test"
|
|
GATING_STEP_NAME = "Test SPA"
|
|
GATING_STEP_RUN = '"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test\nnpm test -- --run'
|
|
# `docs_only` (ersatztv#416) and the #420 revalidate skip. Both are safe for a DIFFERENT reason and
|
|
# the difference is why this is pinned rather than pattern-matched: docs-only cannot ship an image at
|
|
# all, which is asserted below rather than assumed; the revalidate skip fires only on a tree
|
|
# byte-identical to a head that already carried a green combined status.
|
|
GATING_STEP_IF = "steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'"
|
|
# `npm test` means nothing without the directory it runs in: pointed elsewhere it runs a different
|
|
# package, or none. Part of the pin because it is part of what the step does.
|
|
GATING_STEP_WORKDIR = "web"
|
|
# `shell: bash` is `bash -e`, which is what makes a failing command fail the step. A workflow-level
|
|
# change here alters that for the gating step without touching the step at all.
|
|
GATING_WORKFLOW_SHELL = "bash"
|
|
|
|
# The premise under the `docs_only` half of `GATING_STEP_IF`: the publish step is gated on it too, so
|
|
# the arm that skips the suite ships nothing. It is CHECKED rather than stated as fact — an
|
|
# unchecked premise is one level out from the same defect.
|
|
PUBLISH_STEP_NAME = "Build and push"
|
|
PUBLISH_STEP_IF = "steps.detect.outputs.docs_only != 'true'"
|
|
|
|
# `RUN npm run build` and `RUN npm ci` execute whatever `web/package.json` says they do, and that
|
|
# file is neither a Dockerfile nor a workflow. ersatztv#887 was RE-ARMED through it twice
|
|
# without touching any pinned line: `"build": "vitest run && tsc -b && vite build"` puts the suite
|
|
# back into the gitless stage, and `"prepare"` is run by `npm ci`. So the SCRIPTS THAT MENTION VITEST
|
|
# are pinned too — exactly one, and its body.
|
|
PACKAGE_JSON = "web/package.json"
|
|
# THE WHOLE MAP, not the scripts that mention vitest. Selecting on the literal `vitest` was a
|
|
# SELECTOR where a PIN was available, and ersatztv#887 was RE-ARMED through four spellings
|
|
# none of which contain it: `"build": "npm run test -- --run && …"`, `"build": "npm t -- …"`, and the
|
|
# lifecycle hooks `prebuild` (npm runs it before `npm run build`) and `preinstall` (before `npm ci`).
|
|
# Both pinned RUN lines in the gitless stage execute whatever this map says, so the map is pinned.
|
|
PINNED_PACKAGE_SCRIPTS: dict[str, str] = {
|
|
"dev": "vite --host 0.0.0.0",
|
|
"build": "tsc -b && vite build",
|
|
"generate:api": "node scripts/generate-openapi-types.mjs",
|
|
"check:api": "npm run generate:api && git diff --exit-code -- src/api/generated/v1.d.ts",
|
|
"lint": "eslint .",
|
|
"test": "vitest",
|
|
"test:ui-e2e": "playwright test",
|
|
"typecheck": "tsc -b --pretty false",
|
|
"prepare": "cd .. && husky",
|
|
}
|
|
|
|
# `npm test -- --run` collects whatever this config says, and `vite build` in the gitless stage
|
|
# loads it too — so the WHOLE FILE is pinned, not a block within it. Three rounds each closed one
|
|
# SPELLING of a marker match (`test: {`, then `test: {`, then `test : {` and `"test": {`), and two
|
|
# more routes never touched the marker at all (`].concat([evil])` after the pinned array,
|
|
# `...moreTest` after the pinned block — `defineConfig` is the identity function, so a later spread
|
|
# REPLACES what the pin matched). That is `every-blocker-was-one-mechanism-so-delete-it` at the
|
|
# count where it says withdraw: the mechanism was partial matching, and it is gone.
|
|
#
|
|
# STATED COST: any edit that changes a LINE'S TOKEN SEQUENCE reddens, a comment's words included.
|
|
# Whitespace and blank lines are free — `_normalise_lines` collapses them, which is what lets a
|
|
# reflow through. That tolerance is inert for this file's current content (no template literal and
|
|
# no ASI-sensitive token: every `return`/`=>`/backtick in it sits inside a `//` comment) and would
|
|
# stop being inert if one were introduced. It is 48 lines, nothing generates it, and it is the same
|
|
# move already made for `web/package.json`'s script map — a PIN where a partial match was defended.
|
|
VITE_CONFIG = "web/vite.config.ts"
|
|
PINNED_VITE_CONFIG = """import { fileURLToPath, URL } from 'node:url';
|
|
import react from '@vitejs/plugin-react';
|
|
import { configDefaults, defineConfig } from 'vitest/config';
|
|
import { trackedSourceFilesPlugin } from './vite-plugins/trackedSourceFiles';
|
|
|
|
const aspNetHost = 'http://localhost:8409';
|
|
|
|
export default defineConfig({
|
|
base: '/app/',
|
|
plugins: [react(), trackedSourceFilesPlugin()],
|
|
resolve: {
|
|
alias: {
|
|
'@': fileURLToPath(new URL('./src', import.meta.url)),
|
|
'@design-system': fileURLToPath(new URL('../design-system', import.meta.url))
|
|
}
|
|
},
|
|
build: {
|
|
outDir: '../ErsatzTV/wwwroot/app',
|
|
emptyOutDir: true
|
|
},
|
|
server: {
|
|
port: 5173,
|
|
strictPort: false,
|
|
proxy: {
|
|
'/api': aspNetHost,
|
|
'/iptv': aspNetHost,
|
|
'/artwork': aspNetHost
|
|
}
|
|
},
|
|
test: {
|
|
environment: 'jsdom',
|
|
environmentOptions: {
|
|
jsdom: {
|
|
url: 'http://localhost/app/'
|
|
}
|
|
},
|
|
// `e2e/` holds the Playwright UI-E2E specs (ersatztv#445), which drive a REAL browser against a
|
|
// running instance. Vitest's default `include` glob (`**/*.{test,spec}.*`) would otherwise pick
|
|
// them up and run them under jsdom, where `@playwright/test`'s `test()` has no runner and the
|
|
// whole file dies. Spread the defaults rather than replacing `exclude` outright — a bare
|
|
// `exclude: ['e2e/**']` would drop `**/node_modules/**` and make vitest crawl deps.
|
|
// NOTE: excluding (not narrowing `include` to `src/**`) is deliberate — `scripts/` holds a real
|
|
// vitest test too (`generate-openapi-types.test.mjs`), which an `src/**`-only include would
|
|
// silently stop running.
|
|
exclude: [...configDefaults.exclude, 'e2e/**'],
|
|
setupFiles: './src/setupTests.ts'
|
|
}
|
|
});
|
|
"""
|
|
|
|
# Every config filename that can outrank the pinned one, for EITHER tool. Read out of the pinned
|
|
# tarballs rather than remembered: vite 8.1.3's `DEFAULT_CONFIG_FILES` is
|
|
# `["vite.config.js", "vite.config.mjs", "vite.config.ts", …]` — so a `vite.config.js` beats the
|
|
# pinned `.ts` for `vite build`, which is what the gitless stage runs. Vitest 4.1.9 resolves
|
|
# `vitest.config.*` before `vite.config.*`. MEASURED: a `web/vite.config.js` whose plugin shells out
|
|
# to the suite ran it in the gitless stage with all 1411 tests green.
|
|
OUTRANKING_CONFIG_PREFIXES = ("vitest.config.", "vitest.workspace.", "vitest.projects.", "vite.config.")
|
|
|
|
# Substrings that mean "a human should look at this step". NOT a semantic predicate — the pinned step
|
|
# is allowed to contain them and every other step is not.
|
|
SUITE_MENTIONS = ("npm t", "npm run test", "yarn test", "pnpm test", "bun test", "vitest")
|
|
# `npm t` rather than `npm test`: it is npm's own documented alias, it SUBSUMES `npm test` as a
|
|
# substring, and this file enumerates it among the spellings that defeated the parser. Leaving it out
|
|
# meant an unpinned stage running `npm t -- --run` passed the sweep — MEASURED.
|
|
|
|
|
|
def _normalise(text: str) -> str:
|
|
"""Collapse runs of whitespace so a reflow is not a red, keeping every token exact."""
|
|
return " ".join(text.split())
|
|
|
|
|
|
def _normalise_lines(text: str) -> list[str]:
|
|
"""Per-line normalisation, so indentation is free but the LINE BOUNDARIES are not.
|
|
|
|
A newline separates two commands; collapsing it merges them into one. That is not a cosmetic
|
|
difference and the whole-text form cannot see it.
|
|
"""
|
|
return [_normalise(line) for line in text.strip().splitlines() if line.strip()]
|
|
|
|
|
|
def _canonical_action(uses: str) -> str:
|
|
"""The last two path segments of a `uses:` value, without its `@version`.
|
|
|
|
Same normalisation as `test_workflow_persist_credentials._canonical_action`, and deliberately a
|
|
second copy rather than an import: that guard's population is checkout steps, this one's is
|
|
publish steps, and sharing the helper would mean a narrowing there silently narrows here too.
|
|
"""
|
|
ref = uses.strip().strip("'\"").rsplit("@", 1)[0].strip().lower()
|
|
ref = re.sub(r"^https?://", "", ref)
|
|
segments = [seg for seg in ref.split("/") if seg]
|
|
if segments:
|
|
segments[-1] = re.sub(r"\.git$", "", segments[-1])
|
|
return "/".join(segments[-2:]) if len(segments) >= 2 else ref
|
|
|
|
|
|
def dockerfiles() -> list[str]:
|
|
"""Tracked Dockerfiles, by repo-relative path, in every spelling docker recognises.
|
|
|
|
`tracked_files._git_ls_files()` through the MODULE, never a `from … import` binding. The shared
|
|
proof in `test_guard_populations_derive_from_git.py` narrows the index by patching that module
|
|
attribute, and a name bound at import time is not reachable from there — the derivation would go
|
|
on returning every Dockerfile after git stopped tracking them, and the proof would say so.
|
|
"""
|
|
return sorted(
|
|
path
|
|
for path in tracked_files._git_ls_files()
|
|
if path.rpartition("/")[2].startswith("Dockerfile") or path.endswith(".Dockerfile")
|
|
)
|
|
|
|
|
|
def workflow_files() -> list[Path]:
|
|
return tracked_paths(*WORKFLOWS)
|
|
|
|
|
|
def _read(rel: str) -> str:
|
|
return (REPO_ROOT / rel).read_text(encoding="utf-8")
|
|
|
|
|
|
def instructions(text: str) -> list[str]:
|
|
"""The Dockerfile instructions, comments dropped and `\\` continuations joined.
|
|
|
|
This is the whole of the file's syntax handling. It does not interpret what a `RUN` DOES — the
|
|
string is compared against a pin — so the shell traps that defeated the parser
|
|
(heredocs, `#` inside a command, compound punctuation) cannot reach any decision here.
|
|
"""
|
|
joined: list[str] = []
|
|
buffer = ""
|
|
for raw in text.splitlines():
|
|
if raw.lstrip().startswith("#"):
|
|
continue
|
|
stripped = raw.rstrip()
|
|
if stripped.endswith("\\"):
|
|
buffer += stripped[:-1] + " "
|
|
continue
|
|
buffer += stripped
|
|
if buffer.strip():
|
|
joined.append(_normalise(buffer))
|
|
buffer = ""
|
|
if buffer.strip():
|
|
joined.append(_normalise(buffer))
|
|
return joined
|
|
|
|
|
|
def stages(text: str) -> list[tuple[str, list[str]]]:
|
|
"""`(stage name, its instructions)` per build stage, in file order.
|
|
|
|
A stage is delimited by `FROM`, which is the one Dockerfile construct with no shell content and
|
|
therefore nothing to misparse. An unnamed stage is keyed by its index so it is still checkable.
|
|
"""
|
|
found: list[tuple[str, list[str]]] = []
|
|
for line in instructions(text):
|
|
if re.match(r"^FROM\b", line, re.I):
|
|
match = re.search(r"\bAS\s+(\S+)\s*$", line, re.I)
|
|
found.append((match.group(1) if match else f"<unnamed:{len(found)}>", []))
|
|
elif found:
|
|
found[-1][1].append(line)
|
|
return found
|
|
|
|
|
|
def copies_spa_source(command: str) -> bool:
|
|
"""Does this ONE instruction bring the SPA source tree into the image?
|
|
|
|
Every spelling counts, because the negative assertion SKIPS every stage answered False — a miss
|
|
here drops a stage out of the check silently rather than reddening. `ADD` as well as `COPY`; the
|
|
JSON exec form `COPY ["web/", "/dest/"]`, with or without a preceding flag; a source without a
|
|
trailing slash (`COPY web /dest`); and `COPY . …`, which carries the tree without naming it.
|
|
|
|
A `COPY --from=<stage>` is not a context copy: its source is another stage, so `--from=build .`
|
|
would not bring the repo in. Flags are dropped before the source arguments are read.
|
|
"""
|
|
tokens = command.split()
|
|
if not tokens or tokens[0].upper() not in COPY_INSTRUCTIONS:
|
|
return False
|
|
remainder = " ".join(token for token in tokens[1:] if not token.startswith("--")).strip()
|
|
if remainder.startswith("["):
|
|
# Parse the exec form BEFORE the `--from=` branch: splitting `["/source/web", "/dest"]` on
|
|
# whitespace leaves a trailing comma on the first token, so the segment test never matched
|
|
# and the stage went unpinned. Measured False.
|
|
try:
|
|
parsed = json.loads(remainder)
|
|
except ValueError:
|
|
return True
|
|
remainder = " ".join(str(item) for item in parsed) if isinstance(parsed, list) else ""
|
|
if any(token.startswith("--from=") for token in tokens[1:]):
|
|
# A stage copy is not a CONTEXT copy, so `--from=build .` brings in that stage's `/`, not the
|
|
# repository. But it can still carry the SPA SOURCE across from a stage that has it, and a
|
|
# stage answered False here is UNPINNED and unchecked — measured: `COPY --from=web-build
|
|
# /source/web /source/web` was False, so a suite run in the receiving stage was invisible.
|
|
# The test is a whole `web` path SEGMENT in the source, which distinguishes the source tree
|
|
# from the built artifact this repo actually copies (`/source/ErsatzTV/wwwroot/app/.` has no
|
|
# such segment).
|
|
return any("web" in source.strip("/").split("/") for source in remainder.split()[:-1])
|
|
# `remainder` is already a plain space-joined string by here — the exec form was parsed above,
|
|
# before the `--from=` branch. A second `startswith("[")` test here is dead
|
|
# code: measured by poisoning it with a `raise`, which left every test green.
|
|
arguments = remainder.split()
|
|
if len(arguments) < 2:
|
|
return False
|
|
return any(_source_reaches_web(source) for source in arguments[:-1])
|
|
|
|
|
|
def _source_reaches_web(source: str) -> bool:
|
|
"""Could this COPY source bring `web/` in?
|
|
|
|
Exact (`web`, `web/…`), whole-context (`.`, `./`), and GLOB: `COPY web*/ ./` matches `web/` and
|
|
so carries the source, while `webhooks/` does not — MEASURED False for the glob, which left
|
|
the receiving stage unpinned. `fnmatch` against the literal segment `web` is what
|
|
separates the two without hand-listing patterns.
|
|
"""
|
|
normalised = source.removeprefix("./").lstrip("/").rstrip("/")
|
|
if normalised in {"", ".", "web"} or normalised.startswith("web/"):
|
|
return True
|
|
head = normalised.split("/")[0]
|
|
return any(char in head for char in "*?[") and fnmatch("web", head)
|
|
|
|
|
|
def spa_carrying_stages() -> list[tuple[str, str, list[str]]]:
|
|
"""`(dockerfile, stage name, RUN commands)` for every stage carrying the SPA source."""
|
|
found: list[tuple[str, str, list[str]]] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if any(copies_spa_source(line) for line in lines):
|
|
found.append((rel, name, [line for line in lines if re.match(r"^RUN\b", line)]))
|
|
return found
|
|
|
|
|
|
def stage_shell_overrides() -> list[str]:
|
|
"""Every `SHELL` instruction inside a pinned stage, which the pin alone cannot see.
|
|
|
|
`SHELL ["/bin/sh", "-c"]` redefines what every later `RUN` in the stage EXECUTES while leaving
|
|
the `RUN` text — and therefore the pin — untouched. Reported rather than modelled: there is no
|
|
`SHELL` in this repo, so the honest move is to refuse the construct here rather than reason about
|
|
what a replacement interpreter would do.
|
|
"""
|
|
overrides: list[str] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if f"{rel}::{name}" not in PINNED_STAGE_COMMANDS:
|
|
continue
|
|
overrides.extend(f"{rel}::{name}: {line!r}" for line in lines if re.match(r"^SHELL\b", line))
|
|
return overrides
|
|
|
|
|
|
def _jobs(doc: object) -> dict[str, dict]:
|
|
jobs = doc.get("jobs") if isinstance(doc, dict) else None
|
|
if not isinstance(jobs, dict):
|
|
return {}
|
|
return {str(key): value for key, value in jobs.items() if isinstance(value, dict)}
|
|
|
|
|
|
def _steps(job: dict) -> list[dict]:
|
|
steps = job.get("steps")
|
|
return [step for step in steps if isinstance(step, dict)] if isinstance(steps, list) else []
|
|
|
|
|
|
def _needs(job: dict) -> list[str]:
|
|
needs = job.get("needs")
|
|
if isinstance(needs, str):
|
|
return [needs]
|
|
return [str(item) for item in needs] if isinstance(needs, list) else []
|
|
|
|
|
|
def _transitive_needs(job_id: str, jobs: dict[str, dict]) -> set[str]:
|
|
seen: set[str] = set()
|
|
frontier = list(_needs(jobs.get(job_id, {})))
|
|
while frontier:
|
|
current = frontier.pop()
|
|
if current in seen:
|
|
continue
|
|
seen.add(current)
|
|
frontier.extend(_needs(jobs.get(current, {})))
|
|
return seen
|
|
|
|
|
|
def workflow_jobs(rel: str) -> dict[str, dict]:
|
|
return _jobs(yaml.safe_load(_read(rel)))
|
|
|
|
|
|
def find_step(rel: str, job_id: str, name: str) -> dict | None:
|
|
for step in _steps(workflow_jobs(rel).get(job_id, {})):
|
|
if str(step.get("name", "")).strip() == name:
|
|
return step
|
|
return None
|
|
|
|
|
|
def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]:
|
|
"""`(workflow rel path, job id, Dockerfile rel path, that workflow's jobs)` per publishing job.
|
|
|
|
The Dockerfile comes from the step's own `file:` input, defaulting to docker's own default
|
|
(`Dockerfile` at the context root) when the input is absent, so a step that stops naming one is
|
|
still resolved rather than dropped from the population.
|
|
"""
|
|
found: list[tuple[str, str, str, dict[str, dict]]] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
jobs = workflow_jobs(rel)
|
|
for job_id, job in jobs.items():
|
|
for step in _steps(job):
|
|
uses = step.get("uses")
|
|
if not isinstance(uses, str) or _canonical_action(uses) != PUBLISH_ACTION:
|
|
continue
|
|
with_block = step.get("with")
|
|
named = with_block.get("file") if isinstance(with_block, dict) else None
|
|
dockerfile = str(named or "Dockerfile").strip()
|
|
found.append((rel, job_id, re.sub(r"^\./", "", dockerfile), jobs))
|
|
return found
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# NEGATIVE — the gitless stages run exactly what they are pinned to run
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands() -> None:
|
|
"""Set equality on command TEXT, not a predicate over it.
|
|
|
|
A suite run put back in ANY spelling — a heredoc, `sh -c`, `npm t`, `timeout npm test`, a
|
|
`#`-suffixed line — is simply not equal to the pin. That is the whole point: three earlier
|
|
versions asked what a command meant and were wrong nine times; this one asks whether it is the
|
|
command that was reviewed.
|
|
"""
|
|
faults: list[str] = []
|
|
seen: set[str] = set()
|
|
for rel, stage, commands in spa_carrying_stages():
|
|
key = f"{rel}::{stage}"
|
|
seen.add(key)
|
|
pinned = PINNED_STAGE_COMMANDS.get(key)
|
|
if pinned is None:
|
|
faults.append(
|
|
f"{key} carries the SPA source but has no entry in `PINNED_STAGE_COMMANDS`, so "
|
|
f"nothing checks what it runs. It runs: {commands}. Add the entry — and if any of "
|
|
f"those commands runs the vitest suite, that is the ersatztv#887 defect: the stage "
|
|
f"has no git checkout and no git binary, so the suite's git-dependent members "
|
|
f"cannot run there, and the red is unreachable on a PR."
|
|
)
|
|
continue
|
|
if tuple(commands) != pinned:
|
|
faults.append(
|
|
f"{key} does not run its pinned commands.\n pinned: {list(pinned)}\n actual: {commands}\n"
|
|
f" If this change is deliberate, update `PINNED_STAGE_COMMANDS` in the same commit "
|
|
f"and say why. If it added a vitest run, do not: this stage cannot run the suite "
|
|
f"(ersatztv#887), and excluding the specs that fail there is the hand-maintained "
|
|
f"population that broke every image build in ersatztv#883."
|
|
)
|
|
for orphan in sorted(set(PINNED_STAGE_COMMANDS) - seen):
|
|
faults.append(
|
|
f"`PINNED_STAGE_COMMANDS` names {orphan}, which no longer exists or no longer carries "
|
|
f"the SPA source. A pin for a stage that is not checked is a sentence nobody is reading."
|
|
)
|
|
assert not faults, "\n\n".join(faults)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# POSITIVE — the image is gated on the job holding the pinned step
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> None:
|
|
"""Without this, deleting `needs:` is silently green."""
|
|
known = set(dockerfiles())
|
|
faults: list[str] = []
|
|
for rel, job_id, dockerfile, jobs in publishing_jobs():
|
|
if dockerfile not in known:
|
|
faults.append(
|
|
f"{rel}: job `{job_id}` builds `{dockerfile}`, which is not a tracked Dockerfile. "
|
|
f"The population here is derived from the git index, so this job's image is "
|
|
f"UNCHECKED — fix the path or widen `dockerfiles()`."
|
|
)
|
|
continue
|
|
if not any(stage_rel == dockerfile for stage_rel, _, _ in spa_carrying_stages()):
|
|
continue
|
|
gates = _transitive_needs(job_id, jobs)
|
|
# `needs:` names a job in the SAME workflow, so the edge only reaches the pinned step when
|
|
# the publishing job lives in `GATING_WORKFLOW`. Matching a bare job id anywhere was a false
|
|
# green: a second workflow publishing this Dockerfile while `needs:`-ing its OWN unrelated
|
|
# job called `test` (`run: echo ok`) satisfied it — measured, and it escaped every
|
|
# workflow-population guard in the repo, 588 tests green.
|
|
if rel != GATING_WORKFLOW or GATING_JOB not in gates:
|
|
faults.append(
|
|
f"{rel}: job `{job_id}` publishes an image built from `{dockerfile}`, which carries "
|
|
f"the SPA source, but does not transitively `needs:` `{GATING_JOB}` IN "
|
|
f"`{GATING_WORKFLOW}` — the job holding the pinned suite step. It needs: "
|
|
f"{sorted(gates)}. A `needs:` edge resolves within its own workflow, so a job named "
|
|
f"`{GATING_JOB}` in a DIFFERENT workflow is a different job and gates nothing. Since "
|
|
f"ersatztv#887 the image build does not run the suite itself, so this edge is the "
|
|
f"ONLY thing between a red suite and a published image."
|
|
)
|
|
assert not faults, "\n".join(faults)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# EFFECTIVE — the gating step really runs, really fails, and really runs everything
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None:
|
|
"""Its body, its `if:`, and the absence of `continue-on-error` — all pinned.
|
|
|
|
Pinning the BODY is what makes "unfiltered" and "not suppressed" true without parsing: a
|
|
`--exclude`, a `|| true`, a pipe, a `set +e`, a `true || npm test` that never executes, are each
|
|
a different string from the pin. Recognising those individually was tried three times and
|
|
missed `true || npm test` and `continue-on-error: ${{ … }}` among others.
|
|
"""
|
|
named = [
|
|
candidate
|
|
for candidate in _steps(workflow_jobs(GATING_WORKFLOW).get(GATING_JOB, {}))
|
|
if str(candidate.get("name", "")).strip() == GATING_STEP_NAME
|
|
]
|
|
assert len(named) == 1, (
|
|
f"{GATING_WORKFLOW}: job `{GATING_JOB}` has {len(named)} steps named {GATING_STEP_NAME!r}, "
|
|
f"not one. That name is what both this assertion and the substring sweep key on, so a "
|
|
f"SECOND step wearing it inherits the exemption and is never checked — measured by cold "
|
|
f"review with a duplicate carrying `continue-on-error: true` and a spec filter."
|
|
)
|
|
step = named[0]
|
|
|
|
faults: list[str] = []
|
|
body = step.get("run")
|
|
# PER LINE, not over the whole body: `_normalise` collapses newlines, so changing `run: |` to
|
|
# `run: >` folds the two lines into one command whose normalised text is byte-identical to the
|
|
# pin. MEASURED — the marker script then swallows the suite as its arguments.
|
|
if not isinstance(body, str) or _normalise_lines(body) != _normalise_lines(GATING_STEP_RUN):
|
|
faults.append(
|
|
f"the gating step's `run:` is not the pinned command.\n pinned: {GATING_STEP_RUN!r}\n"
|
|
f" actual: {body!r}\n Anything but this — a spec filter, a `|| true`, a pipe, a "
|
|
f"`set +e`, a suite on the right of a `||` that may never execute — changes whether a "
|
|
f"red suite blocks the image. Update the pin deliberately, in the same commit, with a "
|
|
f"reason."
|
|
)
|
|
|
|
workdir = step.get("working-directory")
|
|
if workdir is None or str(workdir).strip() != GATING_STEP_WORKDIR:
|
|
faults.append(
|
|
f"the gating step's `working-directory:` is {workdir!r}, not the pinned "
|
|
f"{GATING_STEP_WORKDIR!r}. `npm test` run anywhere else runs a different package, or "
|
|
f"none, while the pinned command text is unchanged."
|
|
)
|
|
|
|
shell = ((yaml.safe_load(_read(GATING_WORKFLOW)) or {}).get("defaults") or {}).get("run", {}).get("shell")
|
|
if shell is None or str(shell).strip() != GATING_WORKFLOW_SHELL:
|
|
faults.append(
|
|
f"{GATING_WORKFLOW}'s `defaults.run.shell` is {shell!r}, not the pinned "
|
|
f"{GATING_WORKFLOW_SHELL!r}. `bash` here means `bash -e`, which is what makes a failing "
|
|
f"command fail the step; changing it changes whether a red suite blocks the image, "
|
|
f"without touching the step."
|
|
)
|
|
|
|
if stage_shell_overrides():
|
|
faults.append(
|
|
f"a pinned Dockerfile stage carries a `SHELL` instruction: {stage_shell_overrides()}. "
|
|
f"That redefines what every later `RUN` executes while leaving its text — and therefore "
|
|
f"the pin — unchanged."
|
|
)
|
|
|
|
condition = step.get("if")
|
|
if condition is None or _normalise(str(condition)) != _normalise(GATING_STEP_IF):
|
|
faults.append(
|
|
f"the gating step's `if:` is not the pinned condition.\n pinned: {GATING_STEP_IF!r}\n"
|
|
f" actual: {condition!r}\n A third term here lets the suite not run at all while the "
|
|
f"image still publishes."
|
|
)
|
|
|
|
# ANY spelling, not two literals. `continue-on-error: ${{ … }}` is the idiomatic conditional form
|
|
# and a check comparing against `True`/`"true"` waves it through — a presence test cannot see
|
|
# polarity, and this one is fail-OPEN in exactly the direction that matters.
|
|
job = workflow_jobs(GATING_WORKFLOW).get(GATING_JOB, {})
|
|
for scope, holder in (("step", step), ("job", job)):
|
|
if "continue-on-error" in holder and holder["continue-on-error"] not in (False, "false"):
|
|
faults.append(
|
|
f"the gating {scope} carries `continue-on-error: {holder['continue-on-error']!r}`, "
|
|
f"so a red suite reports success and the `needs:` edge that gates the image blocks "
|
|
f"nothing. Only absent, `false` or `False` is acceptable here."
|
|
)
|
|
|
|
if "shell" in step:
|
|
faults.append(
|
|
f"the gating step carries its own `shell: {step['shell']!r}`, which overrides the pinned "
|
|
f"workflow default. The default is pinned because `bash` means `bash -e`; a step-level "
|
|
f"override moves that decision somewhere nothing checks."
|
|
)
|
|
if "defaults" in job:
|
|
faults.append(
|
|
f"the job `{GATING_JOB}` carries `defaults: {job['defaults']!r}`, which overrides the "
|
|
f"pinned workflow default for every step in it, including the gating one."
|
|
)
|
|
|
|
if "if" in job:
|
|
faults.append(
|
|
f"the job `{GATING_JOB}` carries a job-level `if:` ({job['if']!r}). None did when "
|
|
f"ersatztv#887 made this `needs:` edge the only gate, and a job that does not run cannot "
|
|
f"gate anything — Gitea reports it `skipped`, and what that does to a dependent job is "
|
|
f"not something this repo relies on."
|
|
)
|
|
assert not faults, "\n".join(faults)
|
|
|
|
|
|
def test_the_npm_SCRIPT_MAP_is_exactly_what_was_reviewed() -> None:
|
|
"""The whole map, because a pinned `RUN npm …` executes whatever it says.
|
|
|
|
Pinning only the scripts whose body contains `vitest` is a SELECTOR —
|
|
the category this file elsewhere calls the worst-behaved, because going short is silent. It goes
|
|
short four ways, each re-arming ersatztv#887 in the gitless stage with every other pin matching:
|
|
`npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS, which npm runs for
|
|
`npm run build` and `npm ci` without anything naming them. Pinning the map removes the category:
|
|
a script that does not exist cannot be a hook, and one that changes is not equal.
|
|
"""
|
|
scripts = json.loads(_read(PACKAGE_JSON)).get("scripts", {})
|
|
assert scripts == PINNED_PACKAGE_SCRIPTS, (
|
|
f"`{PACKAGE_JSON}`'s scripts are not the pinned map.\n pinned: {PINNED_PACKAGE_SCRIPTS}\n"
|
|
f" actual: {scripts}\n Both pinned `RUN npm …` lines in the gitless `web-build` stage "
|
|
f"execute whatever this says, INCLUDING npm's `pre*`/`post*` lifecycle hooks, which nothing "
|
|
f"in the Dockerfile names. A suite run added here reaches that stage with no Dockerfile or "
|
|
f"workflow line changed — ersatztv#887 through a file nobody would think to check against "
|
|
f"this guard. Update the pin deliberately, in the same commit, with a reason."
|
|
)
|
|
|
|
|
|
def test_the_VITE_CONFIG_is_exactly_what_was_reviewed() -> None:
|
|
"""The whole file, because every partial match of it was defeated.
|
|
|
|
`test.exclude`/`include` decide which specs the gating run collects, and `plugins:` can shell out
|
|
to the suite from a build hook — both inside a file the Dockerfile never mentions. Pinning a
|
|
BLOCK of it was defeated seven measured ways: a decoy copy above
|
|
`defineConfig` with the real member respelled `test: {`, `test : {` or `"test": {`; the same for
|
|
`plugins:`; and two that never touched the marker — `[…].concat([evil])` and a trailing
|
|
`...moreTest` spread, which replaces the pinned object because `defineConfig` is identity in both
|
|
vite and vitest. Comparing the file entire has no marker to respell and nothing after the span.
|
|
|
|
WHAT THE COMPARISON CANNOT SEE, measured: `_normalise_lines` drops blank lines and collapses
|
|
whitespace WITHIN a line, so a reflow, an indentation change and a change to the spacing inside
|
|
a STRING LITERAL are all invisible. The first two carry no meaning; the third could, and does not
|
|
here — no string in this file has consequential internal spacing. Line ORDER and any token change
|
|
are caught (both measured). Stated because "pinned whole" invites the reader to assume byte
|
|
equality, and it is not.
|
|
"""
|
|
actual = _normalise_lines(_read(VITE_CONFIG))
|
|
pinned = _normalise_lines(PINNED_VITE_CONFIG)
|
|
assert actual == pinned, (
|
|
f"`{VITE_CONFIG}` is not the pinned file. First difference:\n"
|
|
+ "\n".join(
|
|
f" line {index + 1}:\n pinned: {pinned[index] if index < len(pinned) else '<absent>'}\n"
|
|
f" actual: {actual[index] if index < len(actual) else '<absent>'}"
|
|
for index in range(max(len(actual), len(pinned)))
|
|
if index >= min(len(actual), len(pinned)) or actual[index] != pinned[index]
|
|
)
|
|
+ "\n\n This file decides which specs the gating run collects and what `vite build` loads "
|
|
"in the gitless stage. Any edit that changes a line's TOKENS reddens — a comment's words "
|
|
"included, whitespace and blank lines not — deliberately, because every "
|
|
"partial match of it was defeated. Update `PINNED_VITE_CONFIG` in the same commit, with a "
|
|
"reason."
|
|
)
|
|
|
|
|
|
def test_NO_OTHER_vitest_CONFIG_outranks_the_pinned_one() -> None:
|
|
"""A pinned config is worthless if a second file can take precedence over it.
|
|
|
|
Vitest prefers `vitest.config.*` (and `vitest.workspace.*`/`vitest.projects.*`) to
|
|
`vite.config.*`. MEASURED: dropping a `web/vitest.config.ts` with `include: ['nope/**']` and
|
|
`passWithNoTests: true` beside the pinned file made the suite collect zero specs and exit 0 — the
|
|
gating step green having run nothing at all, which is worse than the filtered run ersatztv#887
|
|
removed. The construct is refused rather than modelled: there is no such file today, so the
|
|
honest move is to assert none appears.
|
|
"""
|
|
intruders = sorted(
|
|
path
|
|
for path in tracked_files._git_ls_files()
|
|
# DIRECT children of `web/` only: vite resolves its config from the project ROOT, so a
|
|
# `web/e2e/vite.config.ts` outranks nothing — and `web/e2e/` is a real directory. Reddening
|
|
# on it would be fail-noisy with a message that is simply false.
|
|
if path.startswith("web/")
|
|
and "/" not in path[len("web/") :]
|
|
and path.rpartition("/")[2].startswith(OUTRANKING_CONFIG_PREFIXES)
|
|
and path != VITE_CONFIG
|
|
)
|
|
assert not intruders, (
|
|
f"these files outrank the pinned `{VITE_CONFIG}` in vite's or vitest's config resolution: "
|
|
f"{intruders}. Whatever they say decides what the gating run COLLECTS and what `vite build` "
|
|
f"loads in the gitless stage, and the pin on `{VITE_CONFIG}` cannot see either — a suite "
|
|
f"that collects nothing still exits 0, and a plugin loaded from a `.js` config still runs. "
|
|
f"Either fold the settings into `{VITE_CONFIG}` and its pin, or pin this file too."
|
|
)
|
|
|
|
|
|
def test_the_DOCS_ONLY_arm_cannot_publish_an_image() -> None:
|
|
"""The premise under half of `GATING_STEP_IF`, asserted rather than assumed.
|
|
|
|
The gating step is allowed to skip when `docs_only` is true. That is only safe because the
|
|
publish step skips on the same condition, so the arm ships nothing. Stating that in prose with
|
|
nothing checking it is the same shape as the defect one level in.
|
|
"""
|
|
step = find_step(GATING_WORKFLOW, "build", PUBLISH_STEP_NAME)
|
|
assert step is not None, f"{GATING_WORKFLOW}: job `build` has no step named {PUBLISH_STEP_NAME!r}."
|
|
condition = step.get("if")
|
|
assert condition is not None and _normalise(str(condition)) == _normalise(PUBLISH_STEP_IF), (
|
|
f"the publish step's `if:` is {condition!r}, not the pinned {PUBLISH_STEP_IF!r}. The gating "
|
|
f"step is permitted to skip on `docs_only`, and that is safe ONLY because this step skips "
|
|
f"on it too. Without this, a docs-only push to `main` skips the SPA suite and publishes an "
|
|
f"image anyway."
|
|
)
|
|
|
|
|
|
def test_only_the_PINNED_step_mentions_the_suite() -> None:
|
|
"""A substring sweep, deliberately not a semantic predicate.
|
|
|
|
Deciding whether a command RUNS the suite was wrong nine times.
|
|
This asks the much weaker question — does any OTHER step's shell body mention it at all — whose
|
|
failure mode is a false red asking a human to look, never a false green.
|
|
"""
|
|
strays: list[str] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
for job_id, job in workflow_jobs(rel).items():
|
|
for step in _steps(job):
|
|
body = step.get("run")
|
|
if not isinstance(body, str):
|
|
continue
|
|
if (
|
|
rel == GATING_WORKFLOW
|
|
and job_id == GATING_JOB
|
|
and str(step.get("name", "")).strip() == GATING_STEP_NAME
|
|
):
|
|
continue
|
|
hit = [mention for mention in SUITE_MENTIONS if mention in body]
|
|
if hit:
|
|
strays.append(f"{rel} :: job `{job_id}` step {str(step.get('name', '?'))!r} mentions {hit}")
|
|
assert not strays, (
|
|
"these steps mention the SPA suite and are not the pinned gating step:\n "
|
|
+ "\n ".join(strays)
|
|
+ "\n\nThis is a SUBSTRING sweep, not a claim that they run it — look, and either move the "
|
|
"run into the pinned step or record why this one is not a second, unpinned gate."
|
|
)
|
|
|
|
|
|
def test_no_DOCKERFILE_outside_a_pinned_stage_mentions_the_suite() -> None:
|
|
"""The same weak sweep over Dockerfiles, covering stages that carry no SPA source.
|
|
|
|
A stage without `web/` has no suite to run, so `PINNED_STAGE_COMMANDS` does not cover it. This
|
|
reports a mention there anyway rather than leaving the class unexamined.
|
|
"""
|
|
strays: list[str] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if f"{rel}::{name}" in PINNED_STAGE_COMMANDS:
|
|
continue
|
|
for line in lines:
|
|
hit = [mention for mention in SUITE_MENTIONS if mention in line]
|
|
if hit:
|
|
strays.append(f"{rel} :: stage `{name}`: {line!r} mentions {hit}")
|
|
assert not strays, "these unpinned Dockerfile stages mention the SPA suite:\n " + "\n ".join(strays)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# Anti-vacuity and negative controls
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_no_run_BODY_builds_or_pushes_an_image() -> None:
|
|
"""`publishing_jobs()` keys on the ACTION, which is a selector; assert nothing publishes around it.
|
|
|
|
A job doing `docker build -f ./docker/Dockerfile … && docker push …` in a shell body is not in
|
|
that population, so the positive invariant is silently not applied to it — and anti-vacuity does
|
|
not notice, because the real `build` job is still there. MEASURED MISSED without this
|
|
assertion, which is why it is asserted rather than reasoned away.
|
|
"""
|
|
stray: list[str] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
for job_id, job in workflow_jobs(rel).items():
|
|
for step in _steps(job):
|
|
body = step.get("run")
|
|
if not isinstance(body, str):
|
|
continue
|
|
for line in body.splitlines():
|
|
if line.lstrip().startswith("#"):
|
|
continue
|
|
if re.search(r"\bdocker\s+(?:push|build|buildx\s+build)\b", line):
|
|
stray.append(f"{rel} :: job `{job_id}`: {line.strip()!r}")
|
|
assert not stray, (
|
|
f"image build/push outside `{PUBLISH_ACTION}`:\n " + "\n ".join(stray) + "\n\n"
|
|
"`publishing_jobs()` derives from that action, so these jobs are NOT checked for the suite "
|
|
"gate — extend it before adding one."
|
|
)
|
|
|
|
|
|
def test_the_populations_are_NOT_empty() -> None:
|
|
"""Every assertion above passes over an empty set, which is how a completeness guard dies."""
|
|
assert dockerfiles(), "`git ls-files` reported no Dockerfile — the derivation is broken, not the repo."
|
|
assert workflow_files(), f"`git ls-files` reported no {WORKFLOW_DIR}/*.yml — the derivation is broken."
|
|
|
|
carrying = spa_carrying_stages()
|
|
assert carrying, (
|
|
f"no stage in {dockerfiles()} was read as carrying the SPA source, so the negative assertion "
|
|
"checked nothing and the positive one skipped every job. `copies_spa_source` or `stages` "
|
|
"stopped matching."
|
|
)
|
|
assert PINNED_STAGE_COMMANDS, "the pin table is empty; the negative assertion asserts nothing."
|
|
|
|
published = publishing_jobs()
|
|
assert published, (
|
|
f"no job in {WORKFLOW_DIR} uses `{PUBLISH_ACTION}`, so the gate assertion ran over an empty "
|
|
"population. Either the repo stopped publishing images or `_canonical_action` stopped "
|
|
"recognising the action."
|
|
)
|
|
assert any(dockerfile in {rel for rel, _, _ in carrying} for _, _, dockerfile, _ in published), (
|
|
"no publishing job builds an SPA-carrying Dockerfile, so the gate assertion is vacuous."
|
|
)
|
|
|
|
|
|
def test_the_STAGE_SPLIT_finds_the_real_stages() -> None:
|
|
"""`stages()` is the only Dockerfile syntax this file interprets; pin what it returns.
|
|
|
|
A split that stopped recognising `FROM … AS name` would put every instruction in one bucket, the
|
|
pin key would never match, and `test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands`
|
|
would report an orphan pin — loud, but for the wrong reason. Pinned so the cause is legible.
|
|
"""
|
|
names = [name for name, _ in stages(_read("docker/Dockerfile"))]
|
|
assert names[:3] == ["dotnet-runtime", "web-build", "runtime-base"], names
|
|
assert [stage for _, stage, _ in spa_carrying_stages()] == ["web-build"], spa_carrying_stages()
|
|
|
|
|
|
def test_the_SPA_CARRYING_predicate_reads_every_COPY_SPELLING() -> None:
|
|
"""Pinned both ways: every stage answered False is SKIPPED by the negative assertion.
|
|
|
|
The synthetic spellings were MEASURED False before the predicate was widened; a new stage
|
|
using any of them would have gone unpinned and unchecked.
|
|
"""
|
|
for spelling in (
|
|
'COPY ["web/", "/source/web/"]',
|
|
'COPY --chown=node:node ["web/", "/source/web/"]',
|
|
"COPY web /source/web",
|
|
"ADD web/ /source/web/",
|
|
"COPY web/. ./web/",
|
|
"COPY . /source/",
|
|
"COPY ./ /source/",
|
|
"COPY --chown=node:node web/ /source/web/",
|
|
"COPY web/package*.json ./web/",
|
|
# A GLOB that matches `web/`. Measured False before `_source_reaches_web`, which left a
|
|
# stage copying the source this way unpinned and unchecked.
|
|
"COPY web*/ ./",
|
|
"COPY * /source/",
|
|
):
|
|
assert copies_spa_source(spelling), spelling
|
|
|
|
# A STAGE copy can still carry the SPA source across. Measured False before this clause: the
|
|
# receiving stage was then unpinned, so a suite run in it was invisible.
|
|
assert copies_spa_source("COPY --from=web-build /source/web /source/web")
|
|
assert copies_spa_source("COPY --from=builder /app/web ./web")
|
|
# The EXEC form of the same thing. Measured False before the parse order was fixed — splitting
|
|
# `["/source/web", "/dest"]` on whitespace left a trailing comma on the first token, so the
|
|
# segment test never matched. Pinned here because only the development battery witnessed it, and
|
|
# that battery is explicitly not standing.
|
|
assert copies_spa_source('COPY --from=web-build ["/source/web", "/dest"]')
|
|
assert copies_spa_source('COPY --from=web-build ["/source/web","/dest"]')
|
|
assert not copies_spa_source('COPY --from=web-build ["/source/ErsatzTV/wwwroot/app/.", "/dest"]')
|
|
|
|
for spelling in (
|
|
"COPY design-system/. ./design-system/",
|
|
"COPY --from=web-build /source/ErsatzTV/wwwroot/app/. ./app/",
|
|
"COPY webhooks/. ./webhooks/", # `web` must be a whole path segment
|
|
"RUN npm ci",
|
|
"COPY web/", # a single argument is not a copy
|
|
):
|
|
assert not copies_spa_source(spelling), spelling
|
|
|
|
|
|
def test_the_INSTRUCTION_JOIN_survives_what_defeated_the_PARSER() -> None:
|
|
"""The shapes that broke the parser must not break the pin comparison.
|
|
|
|
None of these needs interpreting — they only need to arrive at the comparison intact, so that a
|
|
stage running them is NOT EQUAL to its pin. Each was a measured false green when this file tried
|
|
to decide what the command meant.
|
|
"""
|
|
heredoc = "RUN <<EOF\nnpm test -- --run\nEOF\n"
|
|
assert "RUN <<EOF" in instructions(heredoc)[0], instructions(heredoc)
|
|
assert instructions("RUN echo hi # npm test -- --run\n") == ["RUN echo hi # npm test -- --run"]
|
|
assert instructions("RUN a=(1 2); npm test\n") == ["RUN a=(1 2); npm test"]
|
|
assert instructions("RUN npm run lint && \\\n npm test\n") == ["RUN npm run lint && npm test"]
|
|
# A comment LINE is still dropped, which is what lets the Dockerfile explain the rule.
|
|
assert instructions("# RUN npm test -- --run\nRUN npm ci\n") == ["RUN npm ci"]
|
|
|
|
|
|
def test_the_PIN_rejects_every_spelling_the_PARSER_missed() -> None:
|
|
"""The nine shapes that defeated the predicate, run against the pin instead.
|
|
|
|
Not one of them is equal to `RUN npm run lint && npm run typecheck && npm run build`, which is
|
|
the entire argument for replacing the mechanism: the pin does not need to recognise a spelling to
|
|
reject it.
|
|
"""
|
|
pinned = PINNED_STAGE_COMMANDS["docker/Dockerfile::web-build"]
|
|
for spelling in (
|
|
"RUN <<EOF",
|
|
"RUN echo build#1 && npm test -- --run",
|
|
"RUN (npm run lint); npm test -- --run",
|
|
"RUN npm t -- --run",
|
|
"RUN ./node_modules/.bin/vitest run",
|
|
"RUN pnpm vitest run",
|
|
"RUN timeout 900 npm test -- --run",
|
|
"RUN su node -c 'npm test -- --run'",
|
|
"RUN sh -c 'npm test -- --run'",
|
|
"RUN npm run lint && npm run typecheck && npm test -- --run && npm run build",
|
|
):
|
|
assert spelling not in pinned, spelling
|
|
assert (pinned[0], spelling) != pinned, spelling
|