Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 17s
PR Gates / Docs update reminder (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 16s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 12s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
review-verdict/h10 Review-verdict: MERGEABLE @ 9da0020 (base: main)
Review verdict / Set review-verdict status (pull_request_target) Successful in 14s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m53s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 19m17s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m47s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m2s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
Round 9 returned MERGEABLE with BLOCKER and HIGH empty. Every remaining item was a sentence, and every one erred by UNDERSTATING the guard — which is the safe direction and still worth fixing, because the decision record is what CLAUDE.md routes convention lookups to. The record's `rule:` still listed "`web/vite.config.ts`'s `test:` block" among the pinned things — the very mechanism the previous commit withdrew — and named only `vitest.config.*` as the outranking family, omitting `vite.config.js`/`.mjs`, which is the MEASURED attack from round 7 (a `web/vite.config.js` ran the suite in the gitless stage with 1411 tests green). That family went short in round 7 and again in round 8. This is `enumerate-CLAUSES-to-close-a-sweep`: the survivors were phrased in a different category (WHAT is pinned) from the retracted claim (HOW it is extracted), so sweeping for the retracted words missed them. Also: "any edit to this file reddens, including a comment" was an absolute and is refutable — a reindent, added blank lines, tabs, and a form feed all stay green, because `_normalise_lines` collapses whitespace. Restated as what is actually true (a line's TOKEN sequence, a comment's words included) plus the reason the tolerance is currently inert: this file has no template literal and no ASI-sensitive token outside a comment. And a YAML single-quote escape had leaked from the frontmatter into the markdown BODY, where `''` renders literally. No code change; the guard is unchanged and still 73/0. Refs: #887 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF
1054 lines
57 KiB
Python
1054 lines
57 KiB
Python
"""The SPA vitest suite never runs inside an image build, and the image is gated on a step that does
|
|
run it, unfiltered and non-advisory (ersatztv#887).
|
|
|
|
WHAT THIS IS PROTECTING. `docker/Dockerfile`'s `web-build` stage is gitless twice over. It copies
|
|
only `web/` and `design-system/`, so it holds no `.git` — a property of the STAGE, not of the build
|
|
context, which is the repository root (`context: .`) and whose `.dockerignore` does not exclude
|
|
`.git`: the directory is there to be copied and simply is not. And `node:22-bookworm-slim` ships no
|
|
git binary, which is why a `COPY .git` would not help either. Members of the SPA suite need one or
|
|
the other — `pageSizeCallSites.guard.test.ts` and `completeAnnotations.guard.test.ts` derive their
|
|
file population from `git ls-files` and refuse to fall back to a directory walk
|
|
(`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds a
|
|
real temporary repository.
|
|
|
|
Running the suite there anyway therefore requires a list of the members that cannot run, and that
|
|
list is a population nothing derives: ersatztv#883 added a third member without updating a
|
|
hand-written pair of `--exclude`s, and every image build failed from that commit on. The red was
|
|
structurally unreachable on a PR — `Build & push image (amd64)` carries
|
|
`if: github.event_name != 'pull_request'` — so it landed on `main` and on the `v*` tag path, where a
|
|
release cut fails at the image build and `:latest` stops being republished.
|
|
|
|
## WHY THIS GUARD PINS TEXT INSTEAD OF PARSING IT
|
|
|
|
The first three versions of this file asked "does this command RUN the suite, and can it FAIL?" of
|
|
arbitrary shell text. That predicate was wrong nine times across three review rounds, and every one
|
|
of the nine was the same mechanism:
|
|
|
|
* heredoc bodies — skipped as data, but BuildKit EXECUTES `RUN <<EOF`, and the opener regex also
|
|
fired inside quotes (`echo "tags<<__EOT__"`), which blinded the scan over the last 303 lines of
|
|
`docker-build.yml`. Wrong in BOTH directions at once.
|
|
* `shlex.shlex` does not clear `commenters` the way `shlex.split` does, so `#` truncated a command
|
|
mid-word — including the live `${#reports[@]}` idiom — and made this file's own stated residual
|
|
("only a LEADING `#` is stripped") false.
|
|
* compound punctuation (`);`, `;;`) welded two commands into one segment.
|
|
* `npm t`, `./node_modules/.bin/vitest`, `pnpm vitest`, `yarn vitest`, `node …/vitest.mjs`,
|
|
`timeout …`, `su -c …`, `if npm test; then` — all invisible.
|
|
* `true || npm test` counted as the gating run while never executing.
|
|
* `continue-on-error: ${{ … }}` passed a check that compared against two literals.
|
|
|
|
Nine defects from one mechanism is `process.enumerate-workaround-behaviors-before-deleting`'s signal
|
|
to remove it rather than patch a tenth time, and the mechanism is removed here. **Nothing in this
|
|
file decides what a command MEANS.** The commands that may run in the risky places are PINNED as
|
|
text; anything else — a heredoc, a wrapper, a new spelling, a rewrite — is simply not equal to the
|
|
pin and reddens.
|
|
|
|
WHAT A PIN IS AND IS NOT IMMUNE TO, stated precisely because an earlier version of this paragraph
|
|
claimed the absolute "a pin cannot have a false green" and cold review refuted it by execution. A
|
|
pin is immune to the class that defeated the parser: a DIFFERENT SPELLING of the command. It is not
|
|
immune to the same text MEANING something else. The routes found so far, and what each is worth,
|
|
are enumerated ONCE — in `docs/guard-inventory.md`'s row for this guard, which is where the count
|
|
lives. It is not repeated here: it existed in three artifacts with two different values at one
|
|
point, and re-syncing three copies is how a number stays wrong (`state-the-invariant-not-the-
|
|
measurement`). What this file states is the RULE they are instances of, below.
|
|
|
|
Handled here: `working-directory` and the gating step's own `shell:`; the workflow
|
|
`defaults.run.shell` and the job-level `defaults` that would override it; a stage `SHELL`, refused
|
|
outright; `web/package.json`'s script map, pinned WHOLE after SELECTING on the literal `vitest`
|
|
missed `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS npm runs for
|
|
`npm run build` and `npm ci`; `web/vite.config.ts` PINNED WHOLE — pinning a block of it was
|
|
defeated seven measured ways across three rounds, so the partial match was withdrawn rather than
|
|
respelled an eighth time; and the ABSENCE of any config file
|
|
that outranks the pinned one — `vitest.config.*` for vitest, and `vite.config.js`/`.mjs` for `vite
|
|
build`, whose `DEFAULT_CONFIG_FILES` puts them BEFORE `.ts`. Pinning one config is worthless while a
|
|
second can outrank it, and both halves of that were measured: a `vitest.config.ts` made the suite
|
|
collect zero specs and exit 0, and a `vite.config.js` ran the suite inside the gitless stage with
|
|
all 1411 tests green.
|
|
|
|
Cited rather than re-asserted: the gating job's `container:` image, pinned by
|
|
`test_ci_image_pin_population.py` — two guards on one condition mask each other (#685).
|
|
|
|
NOT modelled, as a list rather than an "only": an `ENV` in a pinned stage that changes what a pinned
|
|
`RUN` resolves or executes; the BODIES of the pinned plugins (there are two, and `react()`'s is
|
|
third-party and unmitigated — only `trackedSourceFilesPlugin`'s laziness is a mitigation); a
|
|
dependency's own install script, reached through `npm ci` and `web/package-lock.json`; and a publish
|
|
through some action other than `docker/build-push-action`.
|
|
|
|
So: the pin removes the SPELLING problem outright and reduces the MEANING problem to a list that can
|
|
be worked through — a much better place than a predicate that was wrong nine times. It is not
|
|
"cannot be wrong", and the list's own history says so.
|
|
|
|
THE SHAPE THOSE ROUTES SHARE, which is the thing to check when adding a pin rather than a list to
|
|
memorise: **a pin assumes it is pinning the artifact that still DECIDES.** Every instance found so
|
|
far is authority moving somewhere the pin is not looking — to another FILE (`vitest.config.*`
|
|
outranks `vite.config.ts`), to another OCCURRENCE in the same file (a decoy first `test: {`), to
|
|
another WORKFLOW (a `needs:` edge naming a job called `test` that is not this one), or to a HOOK the
|
|
pinned command invokes (a vite plugin, an npm lifecycle script). Ask of any new pin: what else could
|
|
decide this, and would the pin still match?
|
|
|
|
Two candidates for that class were probed and are NOT routes, recorded so the next reader does not
|
|
re-derive them. The `setupFiles` entry is pinned by NAME and its CONTENT is not, but a setup file
|
|
that terminates the run is FAIL-NOISY: `process.exit(0)` at the top of `src/setupTests.ts` reports
|
|
`121 failed (121)`, not a green. And `tsconfig*.json` shapes what `tsc -b` compiles, not what vitest
|
|
collects. What remains genuinely unpinned in that direction is a dependency's own install script,
|
|
reached through `npm ci` and `web/package-lock.json` — a supply-chain concern wider than this guard,
|
|
named rather than claimed to be covered.
|
|
|
|
The cost is stated rather than hidden: a legitimate edit to those command lines reddens this guard
|
|
and its author must update the pin. That is the same deliberate review trigger `KNOWN_GATE_*` uses,
|
|
applied to the two places where a silent change publishes an untested image.
|
|
|
|
## THE INVARIANT, in the three parts it has
|
|
|
|
* NEGATIVE — every Dockerfile STAGE that carries the SPA source runs exactly the pinned commands.
|
|
A suite run put back in any spelling is not equal to the pin.
|
|
* POSITIVE — every job publishing an image from an SPA-carrying Dockerfile transitively `needs:`
|
|
the job holding the pinned gating step.
|
|
* EFFECTIVE — that step's `run:` body, its `if:`, and the publishing step's own docs-only `if:`
|
|
are pinned; the step and its job carry no `continue-on-error` in ANY spelling; and the job
|
|
carries no job-level `if:`.
|
|
|
|
POPULATIONS, all derived from the git index via `tracked_files` — never a filesystem walk and never
|
|
a hand-written list (`testing.guard-derives-population-from-source`, ersatztv#806). The PINS are
|
|
hand-written by necessity, and the difference matters: a population decides WHAT IS CHECKED (so a
|
|
hand-written one goes silently short), a pin decides WHAT IS EXPECTED (so a stale one goes loudly
|
|
red). Only the second is safe to write by hand.
|
|
|
|
WHAT IS NOT COVERED, stated rather than implied:
|
|
|
|
* A Dockerfile stage that does not carry the SPA source is unpinned. The boundary is what
|
|
`copies_spa_source` recognises — a context copy of `web/` or `.`, a matching GLOB, or a stage
|
|
copy whose source names `web` — and NOT the wider "has the suite available to run", which is
|
|
what an earlier version of this sentence claimed. A stage copy that RENAMES the tree on the way
|
|
in (`COPY --from=web-build /source/web /elsewhere`) has no `web` segment in its source and is
|
|
not recognised. What that boundary EXCLUDES, measured: a source that is an ANCESTOR of the
|
|
tree (`COPY --from=web-build /source /src` brings `/source/web` along without naming it), and
|
|
one that names it only as `/source/.`. Both are backstopped by the substring sweep — an
|
|
ancestor-copy stage running `npm t`, `npx vitest` or a new npm script is caught. (This sentence
|
|
was wrong three times before it was written from a measurement; a destination rename such as
|
|
`/source/web /elsewhere` IS recognised, because only sources are read.) The
|
|
false-NEGATIVE direction is what drops a stage out of the check, so the predicate is pinned in
|
|
both directions by its own test, and the substring sweep below is the backstop for whatever it
|
|
still misses.
|
|
* A step or stage OTHER than a pinned one could run the suite, and the substring sweep is what
|
|
reports it. `SUITE_MENTIONS` is a hand-written SELECTOR — a third category the population/pin
|
|
split does not name, and the one that behaves worst: a population going short is caught by an
|
|
equality, a pin going stale goes loudly red, but a SELECTOR going short is silent. It was short
|
|
by exactly one entry (`npm t`, npm's own alias, which this file names among the spellings that
|
|
defeated the parser) until cold review measured a stage escaping through it. `PUBLISH_ACTION` is
|
|
the same category and is NOT covered — anti-vacuity proves the selector is non-empty, which the
|
|
real `build` job guarantees, and cannot prove it complete: a publish through a different action
|
|
is invisible to it. `test_no_run_BODY_builds_or_pushes_an_image` covers only the shell-body form.
|
|
* The gating body is compared LINE BY LINE with each line whitespace-normalised, not byte-exact.
|
|
Indentation and reflow are free; line boundaries are not, because a newline separates two
|
|
commands — changing `run: |` to `run: >` folds them into one and the whole-text comparison could
|
|
not see it.
|
|
* Gitea's own evaluation of a skipped `needs:` job is not modelled; the guard forbids the job-level
|
|
`if:` that would create one rather than reasoning about it.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import re
|
|
from fnmatch import fnmatch
|
|
from pathlib import Path
|
|
|
|
import yaml
|
|
|
|
from scripts.tests import tracked_files
|
|
from scripts.tests.tracked_files import tracked_paths
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parents[2]
|
|
WORKFLOW_DIR = ".gitea/workflows"
|
|
WORKFLOWS = (WORKFLOW_DIR, ("*.yml", "*.yaml"))
|
|
|
|
# The action every image publish in this repo goes through. Matched on the last two path segments
|
|
# for the reason `test_workflow_persist_credentials.py` gives: Gitea accepts a full action URL, and
|
|
# a host is not required to contain a dot, so no hostname heuristic is safe.
|
|
PUBLISH_ACTION = "docker/build-push-action"
|
|
|
|
# Instructions that bring files from the build context into the image.
|
|
COPY_INSTRUCTIONS = frozenset({"ADD", "COPY"})
|
|
|
|
# ---------------------------------------------------------------------------------------------
|
|
# THE PINS. Update these deliberately, in the same commit as the change they describe, and say why.
|
|
# ---------------------------------------------------------------------------------------------
|
|
|
|
# Every `RUN` in a Dockerfile stage that carries the SPA source, keyed `<path>::<stage>`. Whitespace
|
|
# is normalised (see `_normalise`) so a reflow is not a red, but the tokens are exact.
|
|
PINNED_STAGE_COMMANDS: dict[str, tuple[str, ...]] = {
|
|
"docker/Dockerfile::web-build": (
|
|
"RUN npm ci",
|
|
# Lints, typechecks and BUILDS the SPA. It must not TEST it: this stage has neither a git
|
|
# checkout nor the git binary, and naming the specs that cannot run there is the
|
|
# hand-maintained population ersatztv#887 removed.
|
|
"RUN npm run lint && npm run typecheck && npm run build",
|
|
),
|
|
}
|
|
|
|
# The step that runs the suite and gates the image, pinned whole. Its body being exact is what makes
|
|
# "unfiltered", "actually executes", and "cannot be suppressed" true without parsing any of it.
|
|
GATING_WORKFLOW = ".gitea/workflows/docker-build.yml"
|
|
GATING_JOB = "test"
|
|
GATING_STEP_NAME = "Test SPA"
|
|
GATING_STEP_RUN = '"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test\nnpm test -- --run'
|
|
# `docs_only` (ersatztv#416) and the #420 revalidate skip. Both are safe for a DIFFERENT reason and
|
|
# the difference is why this is pinned rather than pattern-matched: docs-only cannot ship an image at
|
|
# all, which is asserted below rather than assumed; the revalidate skip fires only on a tree
|
|
# byte-identical to a head that already carried a green combined status.
|
|
GATING_STEP_IF = "steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'"
|
|
# `npm test` means nothing without the directory it runs in: pointed elsewhere it runs a different
|
|
# package, or none. Part of the pin because it is part of what the step does.
|
|
GATING_STEP_WORKDIR = "web"
|
|
# `shell: bash` is `bash -e`, which is what makes a failing command fail the step. A workflow-level
|
|
# change here alters that for the gating step without touching the step at all.
|
|
GATING_WORKFLOW_SHELL = "bash"
|
|
|
|
# The premise under the `docs_only` half of `GATING_STEP_IF`: the publish step is gated on it too, so
|
|
# the arm that skips the suite ships nothing. Round-1 review found this stated as fact with nothing
|
|
# checking it, one level out from the same defect.
|
|
PUBLISH_STEP_NAME = "Build and push"
|
|
PUBLISH_STEP_IF = "steps.detect.outputs.docs_only != 'true'"
|
|
|
|
# `RUN npm run build` and `RUN npm ci` execute whatever `web/package.json` says they do, and that
|
|
# file is neither a Dockerfile nor a workflow. Cold review re-armed ersatztv#887 through it twice
|
|
# without touching any pinned line: `"build": "vitest run && tsc -b && vite build"` puts the suite
|
|
# back into the gitless stage, and `"prepare"` is run by `npm ci`. So the SCRIPTS THAT MENTION VITEST
|
|
# are pinned too — exactly one, and its body.
|
|
PACKAGE_JSON = "web/package.json"
|
|
# THE WHOLE MAP, not the scripts that mention vitest. Selecting on the literal `vitest` was a
|
|
# SELECTOR where a PIN was available, and cold review re-armed ersatztv#887 through four spellings
|
|
# none of which contain it: `"build": "npm run test -- --run && …"`, `"build": "npm t -- …"`, and the
|
|
# lifecycle hooks `prebuild` (npm runs it before `npm run build`) and `preinstall` (before `npm ci`).
|
|
# Both pinned RUN lines in the gitless stage execute whatever this map says, so the map is pinned.
|
|
PINNED_PACKAGE_SCRIPTS: dict[str, str] = {
|
|
"dev": "vite --host 0.0.0.0",
|
|
"build": "tsc -b && vite build",
|
|
"generate:api": "node scripts/generate-openapi-types.mjs",
|
|
"check:api": "npm run generate:api && git diff --exit-code -- src/api/generated/v1.d.ts",
|
|
"lint": "eslint .",
|
|
"test": "vitest",
|
|
"test:ui-e2e": "playwright test",
|
|
"typecheck": "tsc -b --pretty false",
|
|
"prepare": "cd .. && husky",
|
|
}
|
|
|
|
# `npm test -- --run` collects whatever this config says, and `vite build` in the gitless stage
|
|
# loads it too — so the WHOLE FILE is pinned, not a block within it. Three rounds each closed one
|
|
# SPELLING of a marker match (`test: {`, then `test: {`, then `test : {` and `"test": {`), and two
|
|
# more routes never touched the marker at all (`].concat([evil])` after the pinned array,
|
|
# `...moreTest` after the pinned block — `defineConfig` is the identity function, so a later spread
|
|
# REPLACES what the pin matched). That is `every-blocker-was-one-mechanism-so-delete-it` at the
|
|
# count where it says withdraw: the mechanism was partial matching, and it is gone.
|
|
#
|
|
# STATED COST: any edit that changes a LINE'S TOKEN SEQUENCE reddens, a comment's words included.
|
|
# Whitespace and blank lines are free — `_normalise_lines` collapses them, which is what lets a
|
|
# reflow through. That tolerance is inert for this file's current content (no template literal and
|
|
# no ASI-sensitive token: every `return`/`=>`/backtick in it sits inside a `//` comment) and would
|
|
# stop being inert if one were introduced. It is 48 lines, nothing generates it, and it is the same
|
|
# move already made for `web/package.json`'s script map — a PIN where a partial match was defended.
|
|
VITE_CONFIG = "web/vite.config.ts"
|
|
PINNED_VITE_CONFIG = """import { fileURLToPath, URL } from 'node:url';
|
|
import react from '@vitejs/plugin-react';
|
|
import { configDefaults, defineConfig } from 'vitest/config';
|
|
import { trackedSourceFilesPlugin } from './vite-plugins/trackedSourceFiles';
|
|
|
|
const aspNetHost = 'http://localhost:8409';
|
|
|
|
export default defineConfig({
|
|
base: '/app/',
|
|
plugins: [react(), trackedSourceFilesPlugin()],
|
|
resolve: {
|
|
alias: {
|
|
'@': fileURLToPath(new URL('./src', import.meta.url)),
|
|
'@design-system': fileURLToPath(new URL('../design-system', import.meta.url))
|
|
}
|
|
},
|
|
build: {
|
|
outDir: '../ErsatzTV/wwwroot/app',
|
|
emptyOutDir: true
|
|
},
|
|
server: {
|
|
port: 5173,
|
|
strictPort: false,
|
|
proxy: {
|
|
'/api': aspNetHost,
|
|
'/iptv': aspNetHost,
|
|
'/artwork': aspNetHost
|
|
}
|
|
},
|
|
test: {
|
|
environment: 'jsdom',
|
|
environmentOptions: {
|
|
jsdom: {
|
|
url: 'http://localhost/app/'
|
|
}
|
|
},
|
|
// `e2e/` holds the Playwright UI-E2E specs (ersatztv#445), which drive a REAL browser against a
|
|
// running instance. Vitest's default `include` glob (`**/*.{test,spec}.*`) would otherwise pick
|
|
// them up and run them under jsdom, where `@playwright/test`'s `test()` has no runner and the
|
|
// whole file dies. Spread the defaults rather than replacing `exclude` outright — a bare
|
|
// `exclude: ['e2e/**']` would drop `**/node_modules/**` and make vitest crawl deps.
|
|
// NOTE: excluding (not narrowing `include` to `src/**`) is deliberate — `scripts/` holds a real
|
|
// vitest test too (`generate-openapi-types.test.mjs`), which an `src/**`-only include would
|
|
// silently stop running.
|
|
exclude: [...configDefaults.exclude, 'e2e/**'],
|
|
setupFiles: './src/setupTests.ts'
|
|
}
|
|
});
|
|
"""
|
|
|
|
# Every config filename that can outrank the pinned one, for EITHER tool. Read out of the pinned
|
|
# tarballs rather than remembered: vite 8.1.3's `DEFAULT_CONFIG_FILES` is
|
|
# `["vite.config.js", "vite.config.mjs", "vite.config.ts", …]` — so a `vite.config.js` beats the
|
|
# pinned `.ts` for `vite build`, which is what the gitless stage runs. Vitest 4.1.9 resolves
|
|
# `vitest.config.*` before `vite.config.*`. MEASURED: a `web/vite.config.js` whose plugin shells out
|
|
# to the suite ran it in the gitless stage with all 1411 tests green.
|
|
OUTRANKING_CONFIG_PREFIXES = ("vitest.config.", "vitest.workspace.", "vitest.projects.", "vite.config.")
|
|
|
|
# Substrings that mean "a human should look at this step". NOT a semantic predicate — the pinned step
|
|
# is allowed to contain them and every other step is not.
|
|
SUITE_MENTIONS = ("npm t", "npm run test", "yarn test", "pnpm test", "bun test", "vitest")
|
|
# `npm t` rather than `npm test`: it is npm's own documented alias, it SUBSUMES `npm test` as a
|
|
# substring, and this file enumerates it among the spellings that defeated the parser. Leaving it out
|
|
# meant an unpinned stage running `npm t -- --run` passed the sweep — measured by cold review.
|
|
|
|
|
|
def _normalise(text: str) -> str:
|
|
"""Collapse runs of whitespace so a reflow is not a red, keeping every token exact."""
|
|
return " ".join(text.split())
|
|
|
|
|
|
def _normalise_lines(text: str) -> list[str]:
|
|
"""Per-line normalisation, so indentation is free but the LINE BOUNDARIES are not.
|
|
|
|
A newline separates two commands; collapsing it merges them into one. That is not a cosmetic
|
|
difference and the whole-text form cannot see it.
|
|
"""
|
|
return [_normalise(line) for line in text.strip().splitlines() if line.strip()]
|
|
|
|
|
|
def _canonical_action(uses: str) -> str:
|
|
"""The last two path segments of a `uses:` value, without its `@version`.
|
|
|
|
Same normalisation as `test_workflow_persist_credentials._canonical_action`, and deliberately a
|
|
second copy rather than an import: that guard's population is checkout steps, this one's is
|
|
publish steps, and sharing the helper would mean a narrowing there silently narrows here too.
|
|
"""
|
|
ref = uses.strip().strip("'\"").rsplit("@", 1)[0].strip().lower()
|
|
ref = re.sub(r"^https?://", "", ref)
|
|
segments = [seg for seg in ref.split("/") if seg]
|
|
if segments:
|
|
segments[-1] = re.sub(r"\.git$", "", segments[-1])
|
|
return "/".join(segments[-2:]) if len(segments) >= 2 else ref
|
|
|
|
|
|
def dockerfiles() -> list[str]:
|
|
"""Tracked Dockerfiles, by repo-relative path, in every spelling docker recognises.
|
|
|
|
`tracked_files._git_ls_files()` through the MODULE, never a `from … import` binding. The shared
|
|
proof in `test_guard_populations_derive_from_git.py` narrows the index by patching that module
|
|
attribute, and a name bound at import time is not reachable from there — the derivation would go
|
|
on returning every Dockerfile after git stopped tracking them, and the proof would say so.
|
|
"""
|
|
return sorted(
|
|
path
|
|
for path in tracked_files._git_ls_files()
|
|
if path.rpartition("/")[2].startswith("Dockerfile") or path.endswith(".Dockerfile")
|
|
)
|
|
|
|
|
|
def workflow_files() -> list[Path]:
|
|
return tracked_paths(*WORKFLOWS)
|
|
|
|
|
|
def _read(rel: str) -> str:
|
|
return (REPO_ROOT / rel).read_text(encoding="utf-8")
|
|
|
|
|
|
def instructions(text: str) -> list[str]:
|
|
"""The Dockerfile instructions, comments dropped and `\\` continuations joined.
|
|
|
|
This is the whole of the file's syntax handling. It does not interpret what a `RUN` DOES — the
|
|
string is compared against a pin — so the shell traps that defeated three earlier versions
|
|
(heredocs, `#` inside a command, compound punctuation) cannot reach any decision here.
|
|
"""
|
|
joined: list[str] = []
|
|
buffer = ""
|
|
for raw in text.splitlines():
|
|
if raw.lstrip().startswith("#"):
|
|
continue
|
|
stripped = raw.rstrip()
|
|
if stripped.endswith("\\"):
|
|
buffer += stripped[:-1] + " "
|
|
continue
|
|
buffer += stripped
|
|
if buffer.strip():
|
|
joined.append(_normalise(buffer))
|
|
buffer = ""
|
|
if buffer.strip():
|
|
joined.append(_normalise(buffer))
|
|
return joined
|
|
|
|
|
|
def stages(text: str) -> list[tuple[str, list[str]]]:
|
|
"""`(stage name, its instructions)` per build stage, in file order.
|
|
|
|
A stage is delimited by `FROM`, which is the one Dockerfile construct with no shell content and
|
|
therefore nothing to misparse. An unnamed stage is keyed by its index so it is still checkable.
|
|
"""
|
|
found: list[tuple[str, list[str]]] = []
|
|
for line in instructions(text):
|
|
if re.match(r"^FROM\b", line, re.I):
|
|
match = re.search(r"\bAS\s+(\S+)\s*$", line, re.I)
|
|
found.append((match.group(1) if match else f"<unnamed:{len(found)}>", []))
|
|
elif found:
|
|
found[-1][1].append(line)
|
|
return found
|
|
|
|
|
|
def copies_spa_source(command: str) -> bool:
|
|
"""Does this ONE instruction bring the SPA source tree into the image?
|
|
|
|
Every spelling counts, because the negative assertion SKIPS every stage answered False — a miss
|
|
here drops a stage out of the check silently rather than reddening. `ADD` as well as `COPY`; the
|
|
JSON exec form `COPY ["web/", "/dest/"]`, with or without a preceding flag; a source without a
|
|
trailing slash (`COPY web /dest`); and `COPY . …`, which carries the tree without naming it.
|
|
|
|
A `COPY --from=<stage>` is not a context copy: its source is another stage, so `--from=build .`
|
|
would not bring the repo in. Flags are dropped before the source arguments are read.
|
|
"""
|
|
tokens = command.split()
|
|
if not tokens or tokens[0].upper() not in COPY_INSTRUCTIONS:
|
|
return False
|
|
remainder = " ".join(token for token in tokens[1:] if not token.startswith("--")).strip()
|
|
if remainder.startswith("["):
|
|
# Parse the exec form BEFORE the `--from=` branch: splitting `["/source/web", "/dest"]` on
|
|
# whitespace leaves a trailing comma on the first token, so the segment test never matched
|
|
# and the stage went unpinned. Measured False.
|
|
try:
|
|
parsed = json.loads(remainder)
|
|
except ValueError:
|
|
return True
|
|
remainder = " ".join(str(item) for item in parsed) if isinstance(parsed, list) else ""
|
|
if any(token.startswith("--from=") for token in tokens[1:]):
|
|
# A stage copy is not a CONTEXT copy, so `--from=build .` brings in that stage's `/`, not the
|
|
# repository. But it can still carry the SPA SOURCE across from a stage that has it, and a
|
|
# stage answered False here is UNPINNED and unchecked — measured: `COPY --from=web-build
|
|
# /source/web /source/web` was False, so a suite run in the receiving stage was invisible.
|
|
# The test is a whole `web` path SEGMENT in the source, which distinguishes the source tree
|
|
# from the built artifact this repo actually copies (`/source/ErsatzTV/wwwroot/app/.` has no
|
|
# such segment).
|
|
return any("web" in source.strip("/").split("/") for source in remainder.split()[:-1])
|
|
# `remainder` is already a plain space-joined string by here — the exec form was parsed above,
|
|
# before the `--from=` branch. A second `startswith("[")` test used to sit here and was dead
|
|
# code: measured by poisoning it with a `raise`, which left every test green.
|
|
arguments = remainder.split()
|
|
if len(arguments) < 2:
|
|
return False
|
|
return any(_source_reaches_web(source) for source in arguments[:-1])
|
|
|
|
|
|
def _source_reaches_web(source: str) -> bool:
|
|
"""Could this COPY source bring `web/` in?
|
|
|
|
Exact (`web`, `web/…`), whole-context (`.`, `./`), and GLOB: `COPY web*/ ./` matches `web/` and
|
|
so carries the source, while `webhooks/` does not — measured False for the glob by cold review,
|
|
which left the receiving stage unpinned. `fnmatch` against the literal segment `web` is what
|
|
separates the two without hand-listing patterns.
|
|
"""
|
|
normalised = source.removeprefix("./").lstrip("/").rstrip("/")
|
|
if normalised in {"", ".", "web"} or normalised.startswith("web/"):
|
|
return True
|
|
head = normalised.split("/")[0]
|
|
return any(char in head for char in "*?[") and fnmatch("web", head)
|
|
|
|
|
|
def spa_carrying_stages() -> list[tuple[str, str, list[str]]]:
|
|
"""`(dockerfile, stage name, RUN commands)` for every stage carrying the SPA source."""
|
|
found: list[tuple[str, str, list[str]]] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if any(copies_spa_source(line) for line in lines):
|
|
found.append((rel, name, [line for line in lines if re.match(r"^RUN\b", line)]))
|
|
return found
|
|
|
|
|
|
def stage_shell_overrides() -> list[str]:
|
|
"""Every `SHELL` instruction inside a pinned stage, which the pin alone cannot see.
|
|
|
|
`SHELL ["/bin/sh", "-c"]` redefines what every later `RUN` in the stage EXECUTES while leaving
|
|
the `RUN` text — and therefore the pin — untouched. Reported rather than modelled: there is no
|
|
`SHELL` in this repo, so the honest move is to refuse the construct here rather than reason about
|
|
what a replacement interpreter would do.
|
|
"""
|
|
overrides: list[str] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if f"{rel}::{name}" not in PINNED_STAGE_COMMANDS:
|
|
continue
|
|
overrides.extend(f"{rel}::{name}: {line!r}" for line in lines if re.match(r"^SHELL\b", line))
|
|
return overrides
|
|
|
|
|
|
def _jobs(doc: object) -> dict[str, dict]:
|
|
jobs = doc.get("jobs") if isinstance(doc, dict) else None
|
|
if not isinstance(jobs, dict):
|
|
return {}
|
|
return {str(key): value for key, value in jobs.items() if isinstance(value, dict)}
|
|
|
|
|
|
def _steps(job: dict) -> list[dict]:
|
|
steps = job.get("steps")
|
|
return [step for step in steps if isinstance(step, dict)] if isinstance(steps, list) else []
|
|
|
|
|
|
def _needs(job: dict) -> list[str]:
|
|
needs = job.get("needs")
|
|
if isinstance(needs, str):
|
|
return [needs]
|
|
return [str(item) for item in needs] if isinstance(needs, list) else []
|
|
|
|
|
|
def _transitive_needs(job_id: str, jobs: dict[str, dict]) -> set[str]:
|
|
seen: set[str] = set()
|
|
frontier = list(_needs(jobs.get(job_id, {})))
|
|
while frontier:
|
|
current = frontier.pop()
|
|
if current in seen:
|
|
continue
|
|
seen.add(current)
|
|
frontier.extend(_needs(jobs.get(current, {})))
|
|
return seen
|
|
|
|
|
|
def workflow_jobs(rel: str) -> dict[str, dict]:
|
|
return _jobs(yaml.safe_load(_read(rel)))
|
|
|
|
|
|
def find_step(rel: str, job_id: str, name: str) -> dict | None:
|
|
for step in _steps(workflow_jobs(rel).get(job_id, {})):
|
|
if str(step.get("name", "")).strip() == name:
|
|
return step
|
|
return None
|
|
|
|
|
|
def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]:
|
|
"""`(workflow rel path, job id, Dockerfile rel path, that workflow's jobs)` per publishing job.
|
|
|
|
The Dockerfile comes from the step's own `file:` input, defaulting to docker's own default
|
|
(`Dockerfile` at the context root) when the input is absent, so a step that stops naming one is
|
|
still resolved rather than dropped from the population.
|
|
"""
|
|
found: list[tuple[str, str, str, dict[str, dict]]] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
jobs = workflow_jobs(rel)
|
|
for job_id, job in jobs.items():
|
|
for step in _steps(job):
|
|
uses = step.get("uses")
|
|
if not isinstance(uses, str) or _canonical_action(uses) != PUBLISH_ACTION:
|
|
continue
|
|
with_block = step.get("with")
|
|
named = with_block.get("file") if isinstance(with_block, dict) else None
|
|
dockerfile = str(named or "Dockerfile").strip()
|
|
found.append((rel, job_id, re.sub(r"^\./", "", dockerfile), jobs))
|
|
return found
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# NEGATIVE — the gitless stages run exactly what they are pinned to run
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands() -> None:
|
|
"""Set equality on command TEXT, not a predicate over it.
|
|
|
|
A suite run put back in ANY spelling — a heredoc, `sh -c`, `npm t`, `timeout npm test`, a
|
|
`#`-suffixed line — is simply not equal to the pin. That is the whole point: three earlier
|
|
versions asked what a command meant and were wrong nine times; this one asks whether it is the
|
|
command that was reviewed.
|
|
"""
|
|
faults: list[str] = []
|
|
seen: set[str] = set()
|
|
for rel, stage, commands in spa_carrying_stages():
|
|
key = f"{rel}::{stage}"
|
|
seen.add(key)
|
|
pinned = PINNED_STAGE_COMMANDS.get(key)
|
|
if pinned is None:
|
|
faults.append(
|
|
f"{key} carries the SPA source but has no entry in `PINNED_STAGE_COMMANDS`, so "
|
|
f"nothing checks what it runs. It runs: {commands}. Add the entry — and if any of "
|
|
f"those commands runs the vitest suite, that is the ersatztv#887 defect: the stage "
|
|
f"has no git checkout and no git binary, so the suite's git-dependent members "
|
|
f"cannot run there, and the red is unreachable on a PR."
|
|
)
|
|
continue
|
|
if tuple(commands) != pinned:
|
|
faults.append(
|
|
f"{key} does not run its pinned commands.\n pinned: {list(pinned)}\n actual: {commands}\n"
|
|
f" If this change is deliberate, update `PINNED_STAGE_COMMANDS` in the same commit "
|
|
f"and say why. If it added a vitest run, do not: this stage cannot run the suite "
|
|
f"(ersatztv#887), and excluding the specs that fail there is the hand-maintained "
|
|
f"population that broke every image build in ersatztv#883."
|
|
)
|
|
for orphan in sorted(set(PINNED_STAGE_COMMANDS) - seen):
|
|
faults.append(
|
|
f"`PINNED_STAGE_COMMANDS` names {orphan}, which no longer exists or no longer carries "
|
|
f"the SPA source. A pin for a stage that is not checked is a sentence nobody is reading."
|
|
)
|
|
assert not faults, "\n\n".join(faults)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# POSITIVE — the image is gated on the job holding the pinned step
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> None:
|
|
"""Without this, deleting `needs:` is silently green."""
|
|
known = set(dockerfiles())
|
|
faults: list[str] = []
|
|
for rel, job_id, dockerfile, jobs in publishing_jobs():
|
|
if dockerfile not in known:
|
|
faults.append(
|
|
f"{rel}: job `{job_id}` builds `{dockerfile}`, which is not a tracked Dockerfile. "
|
|
f"The population here is derived from the git index, so this job's image is "
|
|
f"UNCHECKED — fix the path or widen `dockerfiles()`."
|
|
)
|
|
continue
|
|
if not any(stage_rel == dockerfile for stage_rel, _, _ in spa_carrying_stages()):
|
|
continue
|
|
gates = _transitive_needs(job_id, jobs)
|
|
# `needs:` names a job in the SAME workflow, so the edge only reaches the pinned step when
|
|
# the publishing job lives in `GATING_WORKFLOW`. Matching a bare job id anywhere was a false
|
|
# green: a second workflow publishing this Dockerfile while `needs:`-ing its OWN unrelated
|
|
# job called `test` (`run: echo ok`) satisfied it — measured, and it escaped every
|
|
# workflow-population guard in the repo, 588 tests green.
|
|
if rel != GATING_WORKFLOW or GATING_JOB not in gates:
|
|
faults.append(
|
|
f"{rel}: job `{job_id}` publishes an image built from `{dockerfile}`, which carries "
|
|
f"the SPA source, but does not transitively `needs:` `{GATING_JOB}` IN "
|
|
f"`{GATING_WORKFLOW}` — the job holding the pinned suite step. It needs: "
|
|
f"{sorted(gates)}. A `needs:` edge resolves within its own workflow, so a job named "
|
|
f"`{GATING_JOB}` in a DIFFERENT workflow is a different job and gates nothing. Since "
|
|
f"ersatztv#887 the image build does not run the suite itself, so this edge is the "
|
|
f"ONLY thing between a red suite and a published image."
|
|
)
|
|
assert not faults, "\n".join(faults)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# EFFECTIVE — the gating step really runs, really fails, and really runs everything
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None:
|
|
"""Its body, its `if:`, and the absence of `continue-on-error` — all pinned.
|
|
|
|
Pinning the BODY is what makes "unfiltered" and "not suppressed" true without parsing: a
|
|
`--exclude`, a `|| true`, a pipe, a `set +e`, a `true || npm test` that never executes, are each
|
|
a different string from the pin. Three earlier versions tried to recognise those individually and
|
|
missed `true || npm test` and `continue-on-error: ${{ … }}` among others.
|
|
"""
|
|
named = [
|
|
candidate
|
|
for candidate in _steps(workflow_jobs(GATING_WORKFLOW).get(GATING_JOB, {}))
|
|
if str(candidate.get("name", "")).strip() == GATING_STEP_NAME
|
|
]
|
|
assert len(named) == 1, (
|
|
f"{GATING_WORKFLOW}: job `{GATING_JOB}` has {len(named)} steps named {GATING_STEP_NAME!r}, "
|
|
f"not one. That name is what both this assertion and the substring sweep key on, so a "
|
|
f"SECOND step wearing it inherits the exemption and is never checked — measured by cold "
|
|
f"review with a duplicate carrying `continue-on-error: true` and a spec filter."
|
|
)
|
|
step = named[0]
|
|
|
|
faults: list[str] = []
|
|
body = step.get("run")
|
|
# PER LINE, not over the whole body: `_normalise` collapses newlines, so changing `run: |` to
|
|
# `run: >` folds the two lines into one command whose normalised text is byte-identical to the
|
|
# pin. Measured by cold review — the marker script then swallows the suite as its arguments.
|
|
if not isinstance(body, str) or _normalise_lines(body) != _normalise_lines(GATING_STEP_RUN):
|
|
faults.append(
|
|
f"the gating step's `run:` is not the pinned command.\n pinned: {GATING_STEP_RUN!r}\n"
|
|
f" actual: {body!r}\n Anything but this — a spec filter, a `|| true`, a pipe, a "
|
|
f"`set +e`, a suite on the right of a `||` that may never execute — changes whether a "
|
|
f"red suite blocks the image. Update the pin deliberately, in the same commit, with a "
|
|
f"reason."
|
|
)
|
|
|
|
workdir = step.get("working-directory")
|
|
if workdir is None or str(workdir).strip() != GATING_STEP_WORKDIR:
|
|
faults.append(
|
|
f"the gating step's `working-directory:` is {workdir!r}, not the pinned "
|
|
f"{GATING_STEP_WORKDIR!r}. `npm test` run anywhere else runs a different package, or "
|
|
f"none, while the pinned command text is unchanged."
|
|
)
|
|
|
|
shell = ((yaml.safe_load(_read(GATING_WORKFLOW)) or {}).get("defaults") or {}).get("run", {}).get("shell")
|
|
if shell is None or str(shell).strip() != GATING_WORKFLOW_SHELL:
|
|
faults.append(
|
|
f"{GATING_WORKFLOW}'s `defaults.run.shell` is {shell!r}, not the pinned "
|
|
f"{GATING_WORKFLOW_SHELL!r}. `bash` here means `bash -e`, which is what makes a failing "
|
|
f"command fail the step; changing it changes whether a red suite blocks the image, "
|
|
f"without touching the step."
|
|
)
|
|
|
|
if stage_shell_overrides():
|
|
faults.append(
|
|
f"a pinned Dockerfile stage carries a `SHELL` instruction: {stage_shell_overrides()}. "
|
|
f"That redefines what every later `RUN` executes while leaving its text — and therefore "
|
|
f"the pin — unchanged."
|
|
)
|
|
|
|
condition = step.get("if")
|
|
if condition is None or _normalise(str(condition)) != _normalise(GATING_STEP_IF):
|
|
faults.append(
|
|
f"the gating step's `if:` is not the pinned condition.\n pinned: {GATING_STEP_IF!r}\n"
|
|
f" actual: {condition!r}\n A third term here lets the suite not run at all while the "
|
|
f"image still publishes."
|
|
)
|
|
|
|
# ANY spelling, not two literals. `continue-on-error: ${{ … }}` is the idiomatic conditional form
|
|
# and a check comparing against `True`/`"true"` waves it through — a presence test cannot see
|
|
# polarity, and this one is fail-OPEN in exactly the direction that matters.
|
|
job = workflow_jobs(GATING_WORKFLOW).get(GATING_JOB, {})
|
|
for scope, holder in (("step", step), ("job", job)):
|
|
if "continue-on-error" in holder and holder["continue-on-error"] not in (False, "false"):
|
|
faults.append(
|
|
f"the gating {scope} carries `continue-on-error: {holder['continue-on-error']!r}`, "
|
|
f"so a red suite reports success and the `needs:` edge that gates the image blocks "
|
|
f"nothing. Only absent, `false` or `False` is acceptable here."
|
|
)
|
|
|
|
if "shell" in step:
|
|
faults.append(
|
|
f"the gating step carries its own `shell: {step['shell']!r}`, which overrides the pinned "
|
|
f"workflow default. The default is pinned because `bash` means `bash -e`; a step-level "
|
|
f"override moves that decision somewhere nothing checks."
|
|
)
|
|
if "defaults" in job:
|
|
faults.append(
|
|
f"the job `{GATING_JOB}` carries `defaults: {job['defaults']!r}`, which overrides the "
|
|
f"pinned workflow default for every step in it, including the gating one."
|
|
)
|
|
|
|
if "if" in job:
|
|
faults.append(
|
|
f"the job `{GATING_JOB}` carries a job-level `if:` ({job['if']!r}). None did when "
|
|
f"ersatztv#887 made this `needs:` edge the only gate, and a job that does not run cannot "
|
|
f"gate anything — Gitea reports it `skipped`, and what that does to a dependent job is "
|
|
f"not something this repo relies on."
|
|
)
|
|
assert not faults, "\n".join(faults)
|
|
|
|
|
|
def test_the_npm_SCRIPT_MAP_is_exactly_what_was_reviewed() -> None:
|
|
"""The whole map, because a pinned `RUN npm …` executes whatever it says.
|
|
|
|
An earlier version pinned only the scripts whose body contained `vitest`, which is a SELECTOR —
|
|
the category this file elsewhere calls the worst-behaved, because going short is silent. It went
|
|
short four ways, each re-arming ersatztv#887 in the gitless stage with every other pin matching:
|
|
`npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS, which npm runs for
|
|
`npm run build` and `npm ci` without anything naming them. Pinning the map removes the category:
|
|
a script that does not exist cannot be a hook, and one that changes is not equal.
|
|
"""
|
|
scripts = json.loads(_read(PACKAGE_JSON)).get("scripts", {})
|
|
assert scripts == PINNED_PACKAGE_SCRIPTS, (
|
|
f"`{PACKAGE_JSON}`'s scripts are not the pinned map.\n pinned: {PINNED_PACKAGE_SCRIPTS}\n"
|
|
f" actual: {scripts}\n Both pinned `RUN npm …` lines in the gitless `web-build` stage "
|
|
f"execute whatever this says, INCLUDING npm's `pre*`/`post*` lifecycle hooks, which nothing "
|
|
f"in the Dockerfile names. A suite run added here reaches that stage with no Dockerfile or "
|
|
f"workflow line changed — ersatztv#887 through a file nobody would think to check against "
|
|
f"this guard. Update the pin deliberately, in the same commit, with a reason."
|
|
)
|
|
|
|
|
|
def test_the_VITE_CONFIG_is_exactly_what_was_reviewed() -> None:
|
|
"""The whole file, because every partial match of it was defeated.
|
|
|
|
`test.exclude`/`include` decide which specs the gating run collects, and `plugins:` can shell out
|
|
to the suite from a build hook — both inside a file the Dockerfile never mentions. Pinning a
|
|
BLOCK of it was defeated seven measured ways across three review rounds: a decoy copy above
|
|
`defineConfig` with the real member respelled `test: {`, `test : {` or `"test": {`; the same for
|
|
`plugins:`; and two that never touched the marker — `[…].concat([evil])` and a trailing
|
|
`...moreTest` spread, which replaces the pinned object because `defineConfig` is identity in both
|
|
vite and vitest. Comparing the file entire has no marker to respell and nothing after the span.
|
|
|
|
WHAT THE COMPARISON CANNOT SEE, measured: `_normalise_lines` drops blank lines and collapses
|
|
whitespace WITHIN a line, so a reflow, an indentation change and a change to the spacing inside
|
|
a STRING LITERAL are all invisible. The first two carry no meaning; the third could, and does not
|
|
here — no string in this file has consequential internal spacing. Line ORDER and any token change
|
|
are caught (both measured). Stated because "pinned whole" invites the reader to assume byte
|
|
equality, and it is not.
|
|
"""
|
|
actual = _normalise_lines(_read(VITE_CONFIG))
|
|
pinned = _normalise_lines(PINNED_VITE_CONFIG)
|
|
assert actual == pinned, (
|
|
f"`{VITE_CONFIG}` is not the pinned file. First difference:\n"
|
|
+ "\n".join(
|
|
f" line {index + 1}:\n pinned: {pinned[index] if index < len(pinned) else '<absent>'}\n"
|
|
f" actual: {actual[index] if index < len(actual) else '<absent>'}"
|
|
for index in range(max(len(actual), len(pinned)))
|
|
if index >= min(len(actual), len(pinned)) or actual[index] != pinned[index]
|
|
)
|
|
+ "\n\n This file decides which specs the gating run collects and what `vite build` loads "
|
|
"in the gitless stage. Any edit that changes a line's TOKENS reddens — a comment's words "
|
|
"included, whitespace and blank lines not — deliberately, because every "
|
|
"partial match of it was defeated. Update `PINNED_VITE_CONFIG` in the same commit, with a "
|
|
"reason."
|
|
)
|
|
|
|
|
|
def test_NO_OTHER_vitest_CONFIG_outranks_the_pinned_one() -> None:
|
|
"""A pinned config is worthless if a second file can take precedence over it.
|
|
|
|
Vitest prefers `vitest.config.*` (and `vitest.workspace.*`/`vitest.projects.*`) to
|
|
`vite.config.*`. MEASURED: dropping a `web/vitest.config.ts` with `include: ['nope/**']` and
|
|
`passWithNoTests: true` beside the pinned file made the suite collect zero specs and exit 0 — the
|
|
gating step green having run nothing at all, which is worse than the filtered run ersatztv#887
|
|
removed. The construct is refused rather than modelled: there is no such file today, so the
|
|
honest move is to assert none appears.
|
|
"""
|
|
intruders = sorted(
|
|
path
|
|
for path in tracked_files._git_ls_files()
|
|
# DIRECT children of `web/` only: vite resolves its config from the project ROOT, so a
|
|
# `web/e2e/vite.config.ts` outranks nothing — and `web/e2e/` is a real directory. Reddening
|
|
# on it would be fail-noisy with a message that is simply false.
|
|
if path.startswith("web/")
|
|
and "/" not in path[len("web/") :]
|
|
and path.rpartition("/")[2].startswith(OUTRANKING_CONFIG_PREFIXES)
|
|
and path != VITE_CONFIG
|
|
)
|
|
assert not intruders, (
|
|
f"these files outrank the pinned `{VITE_CONFIG}` in vite's or vitest's config resolution: "
|
|
f"{intruders}. Whatever they say decides what the gating run COLLECTS and what `vite build` "
|
|
f"loads in the gitless stage, and the pin on `{VITE_CONFIG}` cannot see either — a suite "
|
|
f"that collects nothing still exits 0, and a plugin loaded from a `.js` config still runs. "
|
|
f"Either fold the settings into `{VITE_CONFIG}` and its pin, or pin this file too."
|
|
)
|
|
|
|
|
|
def test_the_DOCS_ONLY_arm_cannot_publish_an_image() -> None:
|
|
"""The premise under half of `GATING_STEP_IF`, asserted rather than assumed.
|
|
|
|
The gating step is allowed to skip when `docs_only` is true. That is only safe because the
|
|
publish step skips on the same condition, so the arm ships nothing. Round-1 review found this
|
|
stated as fact in the prose with nothing checking it — the same shape as the defect one level in.
|
|
"""
|
|
step = find_step(GATING_WORKFLOW, "build", PUBLISH_STEP_NAME)
|
|
assert step is not None, f"{GATING_WORKFLOW}: job `build` has no step named {PUBLISH_STEP_NAME!r}."
|
|
condition = step.get("if")
|
|
assert condition is not None and _normalise(str(condition)) == _normalise(PUBLISH_STEP_IF), (
|
|
f"the publish step's `if:` is {condition!r}, not the pinned {PUBLISH_STEP_IF!r}. The gating "
|
|
f"step is permitted to skip on `docs_only`, and that is safe ONLY because this step skips "
|
|
f"on it too. Without this, a docs-only push to `main` skips the SPA suite and publishes an "
|
|
f"image anyway."
|
|
)
|
|
|
|
|
|
def test_only_the_PINNED_step_mentions_the_suite() -> None:
|
|
"""A substring sweep, deliberately not a semantic predicate.
|
|
|
|
Every earlier version tried to decide whether a command RUNS the suite and was wrong nine times.
|
|
This asks the much weaker question — does any OTHER step's shell body mention it at all — whose
|
|
failure mode is a false red asking a human to look, never a false green.
|
|
"""
|
|
strays: list[str] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
for job_id, job in workflow_jobs(rel).items():
|
|
for step in _steps(job):
|
|
body = step.get("run")
|
|
if not isinstance(body, str):
|
|
continue
|
|
if (
|
|
rel == GATING_WORKFLOW
|
|
and job_id == GATING_JOB
|
|
and str(step.get("name", "")).strip() == GATING_STEP_NAME
|
|
):
|
|
continue
|
|
hit = [mention for mention in SUITE_MENTIONS if mention in body]
|
|
if hit:
|
|
strays.append(f"{rel} :: job `{job_id}` step {str(step.get('name', '?'))!r} mentions {hit}")
|
|
assert not strays, (
|
|
"these steps mention the SPA suite and are not the pinned gating step:\n "
|
|
+ "\n ".join(strays)
|
|
+ "\n\nThis is a SUBSTRING sweep, not a claim that they run it — look, and either move the "
|
|
"run into the pinned step or record why this one is not a second, unpinned gate."
|
|
)
|
|
|
|
|
|
def test_no_DOCKERFILE_outside_a_pinned_stage_mentions_the_suite() -> None:
|
|
"""The same weak sweep over Dockerfiles, covering stages that carry no SPA source.
|
|
|
|
A stage without `web/` has no suite to run, so `PINNED_STAGE_COMMANDS` does not cover it. This
|
|
reports a mention there anyway rather than leaving the class unexamined.
|
|
"""
|
|
strays: list[str] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if f"{rel}::{name}" in PINNED_STAGE_COMMANDS:
|
|
continue
|
|
for line in lines:
|
|
hit = [mention for mention in SUITE_MENTIONS if mention in line]
|
|
if hit:
|
|
strays.append(f"{rel} :: stage `{name}`: {line!r} mentions {hit}")
|
|
assert not strays, "these unpinned Dockerfile stages mention the SPA suite:\n " + "\n ".join(strays)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# Anti-vacuity and negative controls
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_no_run_BODY_builds_or_pushes_an_image() -> None:
|
|
"""`publishing_jobs()` keys on the ACTION, which is a selector; assert nothing publishes around it.
|
|
|
|
A job doing `docker build -f ./docker/Dockerfile … && docker push …` in a shell body is not in
|
|
that population, so the positive invariant is silently not applied to it — and anti-vacuity does
|
|
not notice, because the real `build` job is still there. Measured MISSED by cold review after
|
|
this assertion was dropped in the rewrite; it is restored rather than reasoned away.
|
|
"""
|
|
stray: list[str] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
for job_id, job in workflow_jobs(rel).items():
|
|
for step in _steps(job):
|
|
body = step.get("run")
|
|
if not isinstance(body, str):
|
|
continue
|
|
for line in body.splitlines():
|
|
if line.lstrip().startswith("#"):
|
|
continue
|
|
if re.search(r"\bdocker\s+(?:push|build|buildx\s+build)\b", line):
|
|
stray.append(f"{rel} :: job `{job_id}`: {line.strip()!r}")
|
|
assert not stray, (
|
|
f"image build/push outside `{PUBLISH_ACTION}`:\n " + "\n ".join(stray) + "\n\n"
|
|
"`publishing_jobs()` derives from that action, so these jobs are NOT checked for the suite "
|
|
"gate — extend it before adding one."
|
|
)
|
|
|
|
|
|
def test_the_populations_are_NOT_empty() -> None:
|
|
"""Every assertion above passes over an empty set, which is how a completeness guard dies."""
|
|
assert dockerfiles(), "`git ls-files` reported no Dockerfile — the derivation is broken, not the repo."
|
|
assert workflow_files(), f"`git ls-files` reported no {WORKFLOW_DIR}/*.yml — the derivation is broken."
|
|
|
|
carrying = spa_carrying_stages()
|
|
assert carrying, (
|
|
f"no stage in {dockerfiles()} was read as carrying the SPA source, so the negative assertion "
|
|
"checked nothing and the positive one skipped every job. `copies_spa_source` or `stages` "
|
|
"stopped matching."
|
|
)
|
|
assert PINNED_STAGE_COMMANDS, "the pin table is empty; the negative assertion asserts nothing."
|
|
|
|
published = publishing_jobs()
|
|
assert published, (
|
|
f"no job in {WORKFLOW_DIR} uses `{PUBLISH_ACTION}`, so the gate assertion ran over an empty "
|
|
"population. Either the repo stopped publishing images or `_canonical_action` stopped "
|
|
"recognising the action."
|
|
)
|
|
assert any(dockerfile in {rel for rel, _, _ in carrying} for _, _, dockerfile, _ in published), (
|
|
"no publishing job builds an SPA-carrying Dockerfile, so the gate assertion is vacuous."
|
|
)
|
|
|
|
|
|
def test_the_STAGE_SPLIT_finds_the_real_stages() -> None:
|
|
"""`stages()` is the only Dockerfile syntax this file interprets; pin what it returns.
|
|
|
|
A split that stopped recognising `FROM … AS name` would put every instruction in one bucket, the
|
|
pin key would never match, and `test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands`
|
|
would report an orphan pin — loud, but for the wrong reason. Pinned so the cause is legible.
|
|
"""
|
|
names = [name for name, _ in stages(_read("docker/Dockerfile"))]
|
|
assert names[:3] == ["dotnet-runtime", "web-build", "runtime-base"], names
|
|
assert [stage for _, stage, _ in spa_carrying_stages()] == ["web-build"], spa_carrying_stages()
|
|
|
|
|
|
def test_the_SPA_CARRYING_predicate_reads_every_COPY_SPELLING() -> None:
|
|
"""Pinned both ways: every stage answered False is SKIPPED by the negative assertion.
|
|
|
|
The synthetic spellings were MEASURED False by cold review before this was widened; a new stage
|
|
using any of them would have gone unpinned and unchecked.
|
|
"""
|
|
for spelling in (
|
|
'COPY ["web/", "/source/web/"]',
|
|
'COPY --chown=node:node ["web/", "/source/web/"]',
|
|
"COPY web /source/web",
|
|
"ADD web/ /source/web/",
|
|
"COPY web/. ./web/",
|
|
"COPY . /source/",
|
|
"COPY ./ /source/",
|
|
"COPY --chown=node:node web/ /source/web/",
|
|
"COPY web/package*.json ./web/",
|
|
# A GLOB that matches `web/`. Measured False before `_source_reaches_web`, which left a
|
|
# stage copying the source this way unpinned and unchecked.
|
|
"COPY web*/ ./",
|
|
"COPY * /source/",
|
|
):
|
|
assert copies_spa_source(spelling), spelling
|
|
|
|
# A STAGE copy can still carry the SPA source across. Measured False before this clause: the
|
|
# receiving stage was then unpinned, so a suite run in it was invisible.
|
|
assert copies_spa_source("COPY --from=web-build /source/web /source/web")
|
|
assert copies_spa_source("COPY --from=builder /app/web ./web")
|
|
# The EXEC form of the same thing. Measured False before the parse order was fixed — splitting
|
|
# `["/source/web", "/dest"]` on whitespace left a trailing comma on the first token, so the
|
|
# segment test never matched. Pinned here because only the development battery witnessed it, and
|
|
# that battery is explicitly not standing.
|
|
assert copies_spa_source('COPY --from=web-build ["/source/web", "/dest"]')
|
|
assert copies_spa_source('COPY --from=web-build ["/source/web","/dest"]')
|
|
assert not copies_spa_source('COPY --from=web-build ["/source/ErsatzTV/wwwroot/app/.", "/dest"]')
|
|
|
|
for spelling in (
|
|
"COPY design-system/. ./design-system/",
|
|
"COPY --from=web-build /source/ErsatzTV/wwwroot/app/. ./app/",
|
|
"COPY webhooks/. ./webhooks/", # `web` must be a whole path segment
|
|
"RUN npm ci",
|
|
"COPY web/", # a single argument is not a copy
|
|
):
|
|
assert not copies_spa_source(spelling), spelling
|
|
|
|
|
|
def test_the_INSTRUCTION_JOIN_survives_what_defeated_the_PARSER() -> None:
|
|
"""The shapes that broke three earlier versions must not break the pin comparison.
|
|
|
|
None of these needs interpreting — they only need to arrive at the comparison intact, so that a
|
|
stage running them is NOT EQUAL to its pin. Each was a measured false green when this file tried
|
|
to decide what the command meant.
|
|
"""
|
|
heredoc = "RUN <<EOF\nnpm test -- --run\nEOF\n"
|
|
assert "RUN <<EOF" in instructions(heredoc)[0], instructions(heredoc)
|
|
assert instructions("RUN echo hi # npm test -- --run\n") == ["RUN echo hi # npm test -- --run"]
|
|
assert instructions("RUN a=(1 2); npm test\n") == ["RUN a=(1 2); npm test"]
|
|
assert instructions("RUN npm run lint && \\\n npm test\n") == ["RUN npm run lint && npm test"]
|
|
# A comment LINE is still dropped, which is what lets the Dockerfile explain the rule.
|
|
assert instructions("# RUN npm test -- --run\nRUN npm ci\n") == ["RUN npm ci"]
|
|
|
|
|
|
def test_the_PIN_rejects_every_spelling_the_PARSER_missed() -> None:
|
|
"""The nine shapes that defeated the predicate, run against the pin instead.
|
|
|
|
Not one of them is equal to `RUN npm run lint && npm run typecheck && npm run build`, which is
|
|
the entire argument for replacing the mechanism: the pin does not need to recognise a spelling to
|
|
reject it.
|
|
"""
|
|
pinned = PINNED_STAGE_COMMANDS["docker/Dockerfile::web-build"]
|
|
for spelling in (
|
|
"RUN <<EOF",
|
|
"RUN echo build#1 && npm test -- --run",
|
|
"RUN (npm run lint); npm test -- --run",
|
|
"RUN npm t -- --run",
|
|
"RUN ./node_modules/.bin/vitest run",
|
|
"RUN pnpm vitest run",
|
|
"RUN timeout 900 npm test -- --run",
|
|
"RUN su node -c 'npm test -- --run'",
|
|
"RUN sh -c 'npm test -- --run'",
|
|
"RUN npm run lint && npm run typecheck && npm test -- --run && npm run build",
|
|
):
|
|
assert spelling not in pinned, spelling
|
|
assert (pinned[0], spelling) != pinned, spelling
|