timothyandClaude Opus 5
58def5faa1
Round 8. Three consecutive rounds had each closed one SPELLING of the same marker match,
which is `every-blocker-was-one-mechanism-so-delete-it` at the count where it says
withdraw. Round 6 pinned a block; round 7 fixed `test: {` (two spaces); round 8 defeated
the repaired matcher four more ways — `test : {`, `"test": {`, and the same two for
`plugins:` — plus two that never touched the marker at all:
plugins: [react(), trackedSourceFilesPlugin()].concat([evil])
test: { …pinned… }, ...moreTest
`defineConfig` is the identity function in BOTH vite and vitest (read from the installed
tree), so a spread AFTER the pinned span simply replaces what the pin matched. No
respelling of the marker could ever have caught those: the defect was partial matching,
not the pattern.
So the file is pinned whole. 48 lines, nothing generates it, no marker to respell and
nothing after the span. One assertion replaces a bracket walk, a block extractor and two
uniqueness assertions — and catches all seven measured routes. Stated cost, which is the
same one every other pin here carries: any edit to that file reddens, a comment included.
This also retires a claim I made in a commit message AND in the inventory row: that the
two pins "share one bracket walk and cannot drift apart again". It was false when
written — the block extractor had its own inline copy and never called the shared helper.
Verified by spying on the call: the `test:` pin made zero calls to it.
Also closed: the second exec-form branch in `copies_spa_source` was DEAD code (measured by
poisoning it with a `raise` — every test stayed green), since the round-7 reorder rewrites
`remainder` before it; the exec-form `--from` case that reorder was written for had no
STANDING test, only the development battery, and now has three; and
`OUTRANKING_CONFIG_PREFIXES` matched nested paths, so a `web/e2e/vite.config.ts` — and
`web/e2e/` is a real directory — would have reddened with a message asserting it outranks
the pinned config, which is false. Direct children of `web/` only.
Battery 68 -> 73, 0 missed.
Refs: #887
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019T79beF1Ufid3dXju4yqkF
1043 lines
56 KiB
Python
1043 lines
56 KiB
Python
"""The SPA vitest suite never runs inside an image build, and the image is gated on a step that does
|
|
run it, unfiltered and non-advisory (ersatztv#887).
|
|
|
|
WHAT THIS IS PROTECTING. `docker/Dockerfile`'s `web-build` stage is gitless twice over. It copies
|
|
only `web/` and `design-system/`, so it holds no `.git` — a property of the STAGE, not of the build
|
|
context, which is the repository root (`context: .`) and whose `.dockerignore` does not exclude
|
|
`.git`: the directory is there to be copied and simply is not. And `node:22-bookworm-slim` ships no
|
|
git binary, which is why a `COPY .git` would not help either. Members of the SPA suite need one or
|
|
the other — `pageSizeCallSites.guard.test.ts` and `completeAnnotations.guard.test.ts` derive their
|
|
file population from `git ls-files` and refuse to fall back to a directory walk
|
|
(`testing.guard-derives-population-from-source`), and `trackedSourceFiles.realgit.test.ts` builds a
|
|
real temporary repository.
|
|
|
|
Running the suite there anyway therefore requires a list of the members that cannot run, and that
|
|
list is a population nothing derives: ersatztv#883 added a third member without updating a
|
|
hand-written pair of `--exclude`s, and every image build failed from that commit on. The red was
|
|
structurally unreachable on a PR — `Build & push image (amd64)` carries
|
|
`if: github.event_name != 'pull_request'` — so it landed on `main` and on the `v*` tag path, where a
|
|
release cut fails at the image build and `:latest` stops being republished.
|
|
|
|
## WHY THIS GUARD PINS TEXT INSTEAD OF PARSING IT
|
|
|
|
The first three versions of this file asked "does this command RUN the suite, and can it FAIL?" of
|
|
arbitrary shell text. That predicate was wrong nine times across three review rounds, and every one
|
|
of the nine was the same mechanism:
|
|
|
|
* heredoc bodies — skipped as data, but BuildKit EXECUTES `RUN <<EOF`, and the opener regex also
|
|
fired inside quotes (`echo "tags<<__EOT__"`), which blinded the scan over the last 303 lines of
|
|
`docker-build.yml`. Wrong in BOTH directions at once.
|
|
* `shlex.shlex` does not clear `commenters` the way `shlex.split` does, so `#` truncated a command
|
|
mid-word — including the live `${#reports[@]}` idiom — and made this file's own stated residual
|
|
("only a LEADING `#` is stripped") false.
|
|
* compound punctuation (`);`, `;;`) welded two commands into one segment.
|
|
* `npm t`, `./node_modules/.bin/vitest`, `pnpm vitest`, `yarn vitest`, `node …/vitest.mjs`,
|
|
`timeout …`, `su -c …`, `if npm test; then` — all invisible.
|
|
* `true || npm test` counted as the gating run while never executing.
|
|
* `continue-on-error: ${{ … }}` passed a check that compared against two literals.
|
|
|
|
Nine defects from one mechanism is `process.enumerate-workaround-behaviors-before-deleting`'s signal
|
|
to remove it rather than patch a tenth time, and the mechanism is removed here. **Nothing in this
|
|
file decides what a command MEANS.** The commands that may run in the risky places are PINNED as
|
|
text; anything else — a heredoc, a wrapper, a new spelling, a rewrite — is simply not equal to the
|
|
pin and reddens.
|
|
|
|
WHAT A PIN IS AND IS NOT IMMUNE TO, stated precisely because an earlier version of this paragraph
|
|
claimed the absolute "a pin cannot have a false green" and cold review refuted it by execution. A
|
|
pin is immune to the class that defeated the parser: a DIFFERENT SPELLING of the command. It is not
|
|
immune to the same text MEANING something else. The routes found so far, and what each is worth,
|
|
are enumerated ONCE — in `docs/guard-inventory.md`'s row for this guard, which is where the count
|
|
lives. It is not repeated here: it existed in three artifacts with two different values at one
|
|
point, and re-syncing three copies is how a number stays wrong (`state-the-invariant-not-the-
|
|
measurement`). What this file states is the RULE they are instances of, below.
|
|
|
|
Handled here: `working-directory` and the gating step's own `shell:`; the workflow
|
|
`defaults.run.shell` and the job-level `defaults` that would override it; a stage `SHELL`, refused
|
|
outright; `web/package.json`'s script map, pinned WHOLE after SELECTING on the literal `vitest`
|
|
missed `npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS npm runs for
|
|
`npm run build` and `npm ci`; `web/vite.config.ts` PINNED WHOLE — pinning a block of it was
|
|
defeated seven measured ways across three rounds, so the partial match was withdrawn rather than
|
|
respelled an eighth time; and the ABSENCE of any config file
|
|
that outranks the pinned one — `vitest.config.*` for vitest, and `vite.config.js`/`.mjs` for `vite
|
|
build`, whose `DEFAULT_CONFIG_FILES` puts them BEFORE `.ts`. Pinning one config is worthless while a
|
|
second can outrank it, and both halves of that were measured: a `vitest.config.ts` made the suite
|
|
collect zero specs and exit 0, and a `vite.config.js` ran the suite inside the gitless stage with
|
|
all 1411 tests green.
|
|
|
|
Cited rather than re-asserted: the gating job's `container:` image, pinned by
|
|
`test_ci_image_pin_population.py` — two guards on one condition mask each other (#685).
|
|
|
|
NOT modelled, as a list rather than an "only": an `ENV` in a pinned stage that changes what a pinned
|
|
`RUN` resolves or executes; the BODIES of the pinned plugins (there are two, and `react()`'s is
|
|
third-party and unmitigated — only `trackedSourceFilesPlugin`'s laziness is a mitigation); a
|
|
dependency's own install script, reached through `npm ci` and `web/package-lock.json`; and a publish
|
|
through some action other than `docker/build-push-action`.
|
|
|
|
So: the pin removes the SPELLING problem outright and reduces the MEANING problem to a list that can
|
|
be worked through — a much better place than a predicate that was wrong nine times. It is not
|
|
"cannot be wrong", and the list's own history says so.
|
|
|
|
THE SHAPE THOSE ROUTES SHARE, which is the thing to check when adding a pin rather than a list to
|
|
memorise: **a pin assumes it is pinning the artifact that still DECIDES.** Every instance found so
|
|
far is authority moving somewhere the pin is not looking — to another FILE (`vitest.config.*`
|
|
outranks `vite.config.ts`), to another OCCURRENCE in the same file (a decoy first `test: {`), to
|
|
another WORKFLOW (a `needs:` edge naming a job called `test` that is not this one), or to a HOOK the
|
|
pinned command invokes (a vite plugin, an npm lifecycle script). Ask of any new pin: what else could
|
|
decide this, and would the pin still match?
|
|
|
|
Two candidates for that class were probed and are NOT routes, recorded so the next reader does not
|
|
re-derive them. The `setupFiles` entry is pinned by NAME and its CONTENT is not, but a setup file
|
|
that terminates the run is FAIL-NOISY: `process.exit(0)` at the top of `src/setupTests.ts` reports
|
|
`121 failed (121)`, not a green. And `tsconfig*.json` shapes what `tsc -b` compiles, not what vitest
|
|
collects. What remains genuinely unpinned in that direction is a dependency's own install script,
|
|
reached through `npm ci` and `web/package-lock.json` — a supply-chain concern wider than this guard,
|
|
named rather than claimed to be covered.
|
|
|
|
The cost is stated rather than hidden: a legitimate edit to those command lines reddens this guard
|
|
and its author must update the pin. That is the same deliberate review trigger `KNOWN_GATE_*` uses,
|
|
applied to the two places where a silent change publishes an untested image.
|
|
|
|
## THE INVARIANT, in the three parts it has
|
|
|
|
* NEGATIVE — every Dockerfile STAGE that carries the SPA source runs exactly the pinned commands.
|
|
A suite run put back in any spelling is not equal to the pin.
|
|
* POSITIVE — every job publishing an image from an SPA-carrying Dockerfile transitively `needs:`
|
|
the job holding the pinned gating step.
|
|
* EFFECTIVE — that step's `run:` body, its `if:`, and the publishing step's own docs-only `if:`
|
|
are pinned; the step and its job carry no `continue-on-error` in ANY spelling; and the job
|
|
carries no job-level `if:`.
|
|
|
|
POPULATIONS, all derived from the git index via `tracked_files` — never a filesystem walk and never
|
|
a hand-written list (`testing.guard-derives-population-from-source`, ersatztv#806). The PINS are
|
|
hand-written by necessity, and the difference matters: a population decides WHAT IS CHECKED (so a
|
|
hand-written one goes silently short), a pin decides WHAT IS EXPECTED (so a stale one goes loudly
|
|
red). Only the second is safe to write by hand.
|
|
|
|
WHAT IS NOT COVERED, stated rather than implied:
|
|
|
|
* A Dockerfile stage that does not carry the SPA source is unpinned. The boundary is what
|
|
`copies_spa_source` recognises — a context copy of `web/` or `.`, a matching GLOB, or a stage
|
|
copy whose source names `web` — and NOT the wider "has the suite available to run", which is
|
|
what an earlier version of this sentence claimed. A stage copy that RENAMES the tree on the way
|
|
in (`COPY --from=web-build /source/web /elsewhere`) has no `web` segment in its source and is
|
|
not recognised. What that boundary EXCLUDES, measured: a source that is an ANCESTOR of the
|
|
tree (`COPY --from=web-build /source /src` brings `/source/web` along without naming it), and
|
|
one that names it only as `/source/.`. Both are backstopped by the substring sweep — an
|
|
ancestor-copy stage running `npm t`, `npx vitest` or a new npm script is caught. (This sentence
|
|
was wrong three times before it was written from a measurement; a destination rename such as
|
|
`/source/web /elsewhere` IS recognised, because only sources are read.) The
|
|
false-NEGATIVE direction is what drops a stage out of the check, so the predicate is pinned in
|
|
both directions by its own test, and the substring sweep below is the backstop for whatever it
|
|
still misses.
|
|
* A step or stage OTHER than a pinned one could run the suite, and the substring sweep is what
|
|
reports it. `SUITE_MENTIONS` is a hand-written SELECTOR — a third category the population/pin
|
|
split does not name, and the one that behaves worst: a population going short is caught by an
|
|
equality, a pin going stale goes loudly red, but a SELECTOR going short is silent. It was short
|
|
by exactly one entry (`npm t`, npm's own alias, which this file names among the spellings that
|
|
defeated the parser) until cold review measured a stage escaping through it. `PUBLISH_ACTION` is
|
|
the same category and is NOT covered — anti-vacuity proves the selector is non-empty, which the
|
|
real `build` job guarantees, and cannot prove it complete: a publish through a different action
|
|
is invisible to it. `test_no_run_BODY_builds_or_pushes_an_image` covers only the shell-body form.
|
|
* The gating body is compared LINE BY LINE with each line whitespace-normalised, not byte-exact.
|
|
Indentation and reflow are free; line boundaries are not, because a newline separates two
|
|
commands — changing `run: |` to `run: >` folds them into one and the whole-text comparison could
|
|
not see it.
|
|
* Gitea's own evaluation of a skipped `needs:` job is not modelled; the guard forbids the job-level
|
|
`if:` that would create one rather than reasoning about it.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import re
|
|
from fnmatch import fnmatch
|
|
from pathlib import Path
|
|
|
|
import yaml
|
|
|
|
from scripts.tests import tracked_files
|
|
from scripts.tests.tracked_files import tracked_paths
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parents[2]
|
|
WORKFLOW_DIR = ".gitea/workflows"
|
|
WORKFLOWS = (WORKFLOW_DIR, ("*.yml", "*.yaml"))
|
|
|
|
# The action every image publish in this repo goes through. Matched on the last two path segments
|
|
# for the reason `test_workflow_persist_credentials.py` gives: Gitea accepts a full action URL, and
|
|
# a host is not required to contain a dot, so no hostname heuristic is safe.
|
|
PUBLISH_ACTION = "docker/build-push-action"
|
|
|
|
# Instructions that bring files from the build context into the image.
|
|
COPY_INSTRUCTIONS = frozenset({"ADD", "COPY"})
|
|
|
|
# ---------------------------------------------------------------------------------------------
|
|
# THE PINS. Update these deliberately, in the same commit as the change they describe, and say why.
|
|
# ---------------------------------------------------------------------------------------------
|
|
|
|
# Every `RUN` in a Dockerfile stage that carries the SPA source, keyed `<path>::<stage>`. Whitespace
|
|
# is normalised (see `_normalise`) so a reflow is not a red, but the tokens are exact.
|
|
PINNED_STAGE_COMMANDS: dict[str, tuple[str, ...]] = {
|
|
"docker/Dockerfile::web-build": (
|
|
"RUN npm ci",
|
|
# Lints, typechecks and BUILDS the SPA. It must not TEST it: this stage has neither a git
|
|
# checkout nor the git binary, and naming the specs that cannot run there is the
|
|
# hand-maintained population ersatztv#887 removed.
|
|
"RUN npm run lint && npm run typecheck && npm run build",
|
|
),
|
|
}
|
|
|
|
# The step that runs the suite and gates the image, pinned whole. Its body being exact is what makes
|
|
# "unfiltered", "actually executes", and "cannot be suppressed" true without parsing any of it.
|
|
GATING_WORKFLOW = ".gitea/workflows/docker-build.yml"
|
|
GATING_JOB = "test"
|
|
GATING_STEP_NAME = "Test SPA"
|
|
GATING_STEP_RUN = '"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test\nnpm test -- --run'
|
|
# `docs_only` (ersatztv#416) and the #420 revalidate skip. Both are safe for a DIFFERENT reason and
|
|
# the difference is why this is pinned rather than pattern-matched: docs-only cannot ship an image at
|
|
# all, which is asserted below rather than assumed; the revalidate skip fires only on a tree
|
|
# byte-identical to a head that already carried a green combined status.
|
|
GATING_STEP_IF = "steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'"
|
|
# `npm test` means nothing without the directory it runs in: pointed elsewhere it runs a different
|
|
# package, or none. Part of the pin because it is part of what the step does.
|
|
GATING_STEP_WORKDIR = "web"
|
|
# `shell: bash` is `bash -e`, which is what makes a failing command fail the step. A workflow-level
|
|
# change here alters that for the gating step without touching the step at all.
|
|
GATING_WORKFLOW_SHELL = "bash"
|
|
|
|
# The premise under the `docs_only` half of `GATING_STEP_IF`: the publish step is gated on it too, so
|
|
# the arm that skips the suite ships nothing. Round-1 review found this stated as fact with nothing
|
|
# checking it, one level out from the same defect.
|
|
PUBLISH_STEP_NAME = "Build and push"
|
|
PUBLISH_STEP_IF = "steps.detect.outputs.docs_only != 'true'"
|
|
|
|
# `RUN npm run build` and `RUN npm ci` execute whatever `web/package.json` says they do, and that
|
|
# file is neither a Dockerfile nor a workflow. Cold review re-armed ersatztv#887 through it twice
|
|
# without touching any pinned line: `"build": "vitest run && tsc -b && vite build"` puts the suite
|
|
# back into the gitless stage, and `"prepare"` is run by `npm ci`. So the SCRIPTS THAT MENTION VITEST
|
|
# are pinned too — exactly one, and its body.
|
|
PACKAGE_JSON = "web/package.json"
|
|
# THE WHOLE MAP, not the scripts that mention vitest. Selecting on the literal `vitest` was a
|
|
# SELECTOR where a PIN was available, and cold review re-armed ersatztv#887 through four spellings
|
|
# none of which contain it: `"build": "npm run test -- --run && …"`, `"build": "npm t -- …"`, and the
|
|
# lifecycle hooks `prebuild` (npm runs it before `npm run build`) and `preinstall` (before `npm ci`).
|
|
# Both pinned RUN lines in the gitless stage execute whatever this map says, so the map is pinned.
|
|
PINNED_PACKAGE_SCRIPTS: dict[str, str] = {
|
|
"dev": "vite --host 0.0.0.0",
|
|
"build": "tsc -b && vite build",
|
|
"generate:api": "node scripts/generate-openapi-types.mjs",
|
|
"check:api": "npm run generate:api && git diff --exit-code -- src/api/generated/v1.d.ts",
|
|
"lint": "eslint .",
|
|
"test": "vitest",
|
|
"test:ui-e2e": "playwright test",
|
|
"typecheck": "tsc -b --pretty false",
|
|
"prepare": "cd .. && husky",
|
|
}
|
|
|
|
# `npm test -- --run` collects whatever this config says, and `vite build` in the gitless stage
|
|
# loads it too — so the WHOLE FILE is pinned, not a block within it. Three rounds each closed one
|
|
# SPELLING of a marker match (`test: {`, then `test: {`, then `test : {` and `"test": {`), and two
|
|
# more routes never touched the marker at all (`].concat([evil])` after the pinned array,
|
|
# `...moreTest` after the pinned block — `defineConfig` is the identity function, so a later spread
|
|
# REPLACES what the pin matched). That is `every-blocker-was-one-mechanism-so-delete-it` at the
|
|
# count where it says withdraw: the mechanism was partial matching, and it is gone.
|
|
#
|
|
# STATED COST: any edit to this file reddens, including a comment. It is 48 lines, nothing generates
|
|
# it, and it is the same move already made for `web/package.json`'s script map — a PIN where a
|
|
# partial match was being defended.
|
|
VITE_CONFIG = "web/vite.config.ts"
|
|
PINNED_VITE_CONFIG = """import { fileURLToPath, URL } from 'node:url';
|
|
import react from '@vitejs/plugin-react';
|
|
import { configDefaults, defineConfig } from 'vitest/config';
|
|
import { trackedSourceFilesPlugin } from './vite-plugins/trackedSourceFiles';
|
|
|
|
const aspNetHost = 'http://localhost:8409';
|
|
|
|
export default defineConfig({
|
|
base: '/app/',
|
|
plugins: [react(), trackedSourceFilesPlugin()],
|
|
resolve: {
|
|
alias: {
|
|
'@': fileURLToPath(new URL('./src', import.meta.url)),
|
|
'@design-system': fileURLToPath(new URL('../design-system', import.meta.url))
|
|
}
|
|
},
|
|
build: {
|
|
outDir: '../ErsatzTV/wwwroot/app',
|
|
emptyOutDir: true
|
|
},
|
|
server: {
|
|
port: 5173,
|
|
strictPort: false,
|
|
proxy: {
|
|
'/api': aspNetHost,
|
|
'/iptv': aspNetHost,
|
|
'/artwork': aspNetHost
|
|
}
|
|
},
|
|
test: {
|
|
environment: 'jsdom',
|
|
environmentOptions: {
|
|
jsdom: {
|
|
url: 'http://localhost/app/'
|
|
}
|
|
},
|
|
// `e2e/` holds the Playwright UI-E2E specs (ersatztv#445), which drive a REAL browser against a
|
|
// running instance. Vitest's default `include` glob (`**/*.{test,spec}.*`) would otherwise pick
|
|
// them up and run them under jsdom, where `@playwright/test`'s `test()` has no runner and the
|
|
// whole file dies. Spread the defaults rather than replacing `exclude` outright — a bare
|
|
// `exclude: ['e2e/**']` would drop `**/node_modules/**` and make vitest crawl deps.
|
|
// NOTE: excluding (not narrowing `include` to `src/**`) is deliberate — `scripts/` holds a real
|
|
// vitest test too (`generate-openapi-types.test.mjs`), which an `src/**`-only include would
|
|
// silently stop running.
|
|
exclude: [...configDefaults.exclude, 'e2e/**'],
|
|
setupFiles: './src/setupTests.ts'
|
|
}
|
|
});
|
|
"""
|
|
|
|
# Every config filename that can outrank the pinned one, for EITHER tool. Read out of the pinned
|
|
# tarballs rather than remembered: vite 8.1.3's `DEFAULT_CONFIG_FILES` is
|
|
# `["vite.config.js", "vite.config.mjs", "vite.config.ts", …]` — so a `vite.config.js` beats the
|
|
# pinned `.ts` for `vite build`, which is what the gitless stage runs. Vitest 4.1.9 resolves
|
|
# `vitest.config.*` before `vite.config.*`. MEASURED: a `web/vite.config.js` whose plugin shells out
|
|
# to the suite ran it in the gitless stage with all 1411 tests green.
|
|
OUTRANKING_CONFIG_PREFIXES = ("vitest.config.", "vitest.workspace.", "vitest.projects.", "vite.config.")
|
|
|
|
# Substrings that mean "a human should look at this step". NOT a semantic predicate — the pinned step
|
|
# is allowed to contain them and every other step is not.
|
|
SUITE_MENTIONS = ("npm t", "npm run test", "yarn test", "pnpm test", "bun test", "vitest")
|
|
# `npm t` rather than `npm test`: it is npm's own documented alias, it SUBSUMES `npm test` as a
|
|
# substring, and this file enumerates it among the spellings that defeated the parser. Leaving it out
|
|
# meant an unpinned stage running `npm t -- --run` passed the sweep — measured by cold review.
|
|
|
|
|
|
def _normalise(text: str) -> str:
|
|
"""Collapse runs of whitespace so a reflow is not a red, keeping every token exact."""
|
|
return " ".join(text.split())
|
|
|
|
|
|
def _normalise_lines(text: str) -> list[str]:
|
|
"""Per-line normalisation, so indentation is free but the LINE BOUNDARIES are not.
|
|
|
|
A newline separates two commands; collapsing it merges them into one. That is not a cosmetic
|
|
difference and the whole-text form cannot see it.
|
|
"""
|
|
return [_normalise(line) for line in text.strip().splitlines() if line.strip()]
|
|
|
|
|
|
def _canonical_action(uses: str) -> str:
|
|
"""The last two path segments of a `uses:` value, without its `@version`.
|
|
|
|
Same normalisation as `test_workflow_persist_credentials._canonical_action`, and deliberately a
|
|
second copy rather than an import: that guard's population is checkout steps, this one's is
|
|
publish steps, and sharing the helper would mean a narrowing there silently narrows here too.
|
|
"""
|
|
ref = uses.strip().strip("'\"").rsplit("@", 1)[0].strip().lower()
|
|
ref = re.sub(r"^https?://", "", ref)
|
|
segments = [seg for seg in ref.split("/") if seg]
|
|
if segments:
|
|
segments[-1] = re.sub(r"\.git$", "", segments[-1])
|
|
return "/".join(segments[-2:]) if len(segments) >= 2 else ref
|
|
|
|
|
|
def dockerfiles() -> list[str]:
|
|
"""Tracked Dockerfiles, by repo-relative path, in every spelling docker recognises.
|
|
|
|
`tracked_files._git_ls_files()` through the MODULE, never a `from … import` binding. The shared
|
|
proof in `test_guard_populations_derive_from_git.py` narrows the index by patching that module
|
|
attribute, and a name bound at import time is not reachable from there — the derivation would go
|
|
on returning every Dockerfile after git stopped tracking them, and the proof would say so.
|
|
"""
|
|
return sorted(
|
|
path
|
|
for path in tracked_files._git_ls_files()
|
|
if path.rpartition("/")[2].startswith("Dockerfile") or path.endswith(".Dockerfile")
|
|
)
|
|
|
|
|
|
def workflow_files() -> list[Path]:
|
|
return tracked_paths(*WORKFLOWS)
|
|
|
|
|
|
def _read(rel: str) -> str:
|
|
return (REPO_ROOT / rel).read_text(encoding="utf-8")
|
|
|
|
|
|
def instructions(text: str) -> list[str]:
|
|
"""The Dockerfile instructions, comments dropped and `\\` continuations joined.
|
|
|
|
This is the whole of the file's syntax handling. It does not interpret what a `RUN` DOES — the
|
|
string is compared against a pin — so the shell traps that defeated three earlier versions
|
|
(heredocs, `#` inside a command, compound punctuation) cannot reach any decision here.
|
|
"""
|
|
joined: list[str] = []
|
|
buffer = ""
|
|
for raw in text.splitlines():
|
|
if raw.lstrip().startswith("#"):
|
|
continue
|
|
stripped = raw.rstrip()
|
|
if stripped.endswith("\\"):
|
|
buffer += stripped[:-1] + " "
|
|
continue
|
|
buffer += stripped
|
|
if buffer.strip():
|
|
joined.append(_normalise(buffer))
|
|
buffer = ""
|
|
if buffer.strip():
|
|
joined.append(_normalise(buffer))
|
|
return joined
|
|
|
|
|
|
def stages(text: str) -> list[tuple[str, list[str]]]:
|
|
"""`(stage name, its instructions)` per build stage, in file order.
|
|
|
|
A stage is delimited by `FROM`, which is the one Dockerfile construct with no shell content and
|
|
therefore nothing to misparse. An unnamed stage is keyed by its index so it is still checkable.
|
|
"""
|
|
found: list[tuple[str, list[str]]] = []
|
|
for line in instructions(text):
|
|
if re.match(r"^FROM\b", line, re.I):
|
|
match = re.search(r"\bAS\s+(\S+)\s*$", line, re.I)
|
|
found.append((match.group(1) if match else f"<unnamed:{len(found)}>", []))
|
|
elif found:
|
|
found[-1][1].append(line)
|
|
return found
|
|
|
|
|
|
def copies_spa_source(command: str) -> bool:
|
|
"""Does this ONE instruction bring the SPA source tree into the image?
|
|
|
|
Every spelling counts, because the negative assertion SKIPS every stage answered False — a miss
|
|
here drops a stage out of the check silently rather than reddening. `ADD` as well as `COPY`; the
|
|
JSON exec form `COPY ["web/", "/dest/"]`, with or without a preceding flag; a source without a
|
|
trailing slash (`COPY web /dest`); and `COPY . …`, which carries the tree without naming it.
|
|
|
|
A `COPY --from=<stage>` is not a context copy: its source is another stage, so `--from=build .`
|
|
would not bring the repo in. Flags are dropped before the source arguments are read.
|
|
"""
|
|
tokens = command.split()
|
|
if not tokens or tokens[0].upper() not in COPY_INSTRUCTIONS:
|
|
return False
|
|
remainder = " ".join(token for token in tokens[1:] if not token.startswith("--")).strip()
|
|
if remainder.startswith("["):
|
|
# Parse the exec form BEFORE the `--from=` branch: splitting `["/source/web", "/dest"]` on
|
|
# whitespace leaves a trailing comma on the first token, so the segment test never matched
|
|
# and the stage went unpinned. Measured False.
|
|
try:
|
|
parsed = json.loads(remainder)
|
|
except ValueError:
|
|
return True
|
|
remainder = " ".join(str(item) for item in parsed) if isinstance(parsed, list) else ""
|
|
if any(token.startswith("--from=") for token in tokens[1:]):
|
|
# A stage copy is not a CONTEXT copy, so `--from=build .` brings in that stage's `/`, not the
|
|
# repository. But it can still carry the SPA SOURCE across from a stage that has it, and a
|
|
# stage answered False here is UNPINNED and unchecked — measured: `COPY --from=web-build
|
|
# /source/web /source/web` was False, so a suite run in the receiving stage was invisible.
|
|
# The test is a whole `web` path SEGMENT in the source, which distinguishes the source tree
|
|
# from the built artifact this repo actually copies (`/source/ErsatzTV/wwwroot/app/.` has no
|
|
# such segment).
|
|
return any("web" in source.strip("/").split("/") for source in remainder.split()[:-1])
|
|
# `remainder` is already a plain space-joined string by here — the exec form was parsed above,
|
|
# before the `--from=` branch. A second `startswith("[")` test used to sit here and was dead
|
|
# code: measured by poisoning it with a `raise`, which left every test green.
|
|
arguments = remainder.split()
|
|
if len(arguments) < 2:
|
|
return False
|
|
return any(_source_reaches_web(source) for source in arguments[:-1])
|
|
|
|
|
|
def _source_reaches_web(source: str) -> bool:
|
|
"""Could this COPY source bring `web/` in?
|
|
|
|
Exact (`web`, `web/…`), whole-context (`.`, `./`), and GLOB: `COPY web*/ ./` matches `web/` and
|
|
so carries the source, while `webhooks/` does not — measured False for the glob by cold review,
|
|
which left the receiving stage unpinned. `fnmatch` against the literal segment `web` is what
|
|
separates the two without hand-listing patterns.
|
|
"""
|
|
normalised = source.removeprefix("./").lstrip("/").rstrip("/")
|
|
if normalised in {"", ".", "web"} or normalised.startswith("web/"):
|
|
return True
|
|
head = normalised.split("/")[0]
|
|
return any(char in head for char in "*?[") and fnmatch("web", head)
|
|
|
|
|
|
def spa_carrying_stages() -> list[tuple[str, str, list[str]]]:
|
|
"""`(dockerfile, stage name, RUN commands)` for every stage carrying the SPA source."""
|
|
found: list[tuple[str, str, list[str]]] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if any(copies_spa_source(line) for line in lines):
|
|
found.append((rel, name, [line for line in lines if re.match(r"^RUN\b", line)]))
|
|
return found
|
|
|
|
|
|
def stage_shell_overrides() -> list[str]:
|
|
"""Every `SHELL` instruction inside a pinned stage, which the pin alone cannot see.
|
|
|
|
`SHELL ["/bin/sh", "-c"]` redefines what every later `RUN` in the stage EXECUTES while leaving
|
|
the `RUN` text — and therefore the pin — untouched. Reported rather than modelled: there is no
|
|
`SHELL` in this repo, so the honest move is to refuse the construct here rather than reason about
|
|
what a replacement interpreter would do.
|
|
"""
|
|
overrides: list[str] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if f"{rel}::{name}" not in PINNED_STAGE_COMMANDS:
|
|
continue
|
|
overrides.extend(f"{rel}::{name}: {line!r}" for line in lines if re.match(r"^SHELL\b", line))
|
|
return overrides
|
|
|
|
|
|
def _jobs(doc: object) -> dict[str, dict]:
|
|
jobs = doc.get("jobs") if isinstance(doc, dict) else None
|
|
if not isinstance(jobs, dict):
|
|
return {}
|
|
return {str(key): value for key, value in jobs.items() if isinstance(value, dict)}
|
|
|
|
|
|
def _steps(job: dict) -> list[dict]:
|
|
steps = job.get("steps")
|
|
return [step for step in steps if isinstance(step, dict)] if isinstance(steps, list) else []
|
|
|
|
|
|
def _needs(job: dict) -> list[str]:
|
|
needs = job.get("needs")
|
|
if isinstance(needs, str):
|
|
return [needs]
|
|
return [str(item) for item in needs] if isinstance(needs, list) else []
|
|
|
|
|
|
def _transitive_needs(job_id: str, jobs: dict[str, dict]) -> set[str]:
|
|
seen: set[str] = set()
|
|
frontier = list(_needs(jobs.get(job_id, {})))
|
|
while frontier:
|
|
current = frontier.pop()
|
|
if current in seen:
|
|
continue
|
|
seen.add(current)
|
|
frontier.extend(_needs(jobs.get(current, {})))
|
|
return seen
|
|
|
|
|
|
def workflow_jobs(rel: str) -> dict[str, dict]:
|
|
return _jobs(yaml.safe_load(_read(rel)))
|
|
|
|
|
|
def find_step(rel: str, job_id: str, name: str) -> dict | None:
|
|
for step in _steps(workflow_jobs(rel).get(job_id, {})):
|
|
if str(step.get("name", "")).strip() == name:
|
|
return step
|
|
return None
|
|
|
|
|
|
def publishing_jobs() -> list[tuple[str, str, str, dict[str, dict]]]:
|
|
"""`(workflow rel path, job id, Dockerfile rel path, that workflow's jobs)` per publishing job.
|
|
|
|
The Dockerfile comes from the step's own `file:` input, defaulting to docker's own default
|
|
(`Dockerfile` at the context root) when the input is absent, so a step that stops naming one is
|
|
still resolved rather than dropped from the population.
|
|
"""
|
|
found: list[tuple[str, str, str, dict[str, dict]]] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
jobs = workflow_jobs(rel)
|
|
for job_id, job in jobs.items():
|
|
for step in _steps(job):
|
|
uses = step.get("uses")
|
|
if not isinstance(uses, str) or _canonical_action(uses) != PUBLISH_ACTION:
|
|
continue
|
|
with_block = step.get("with")
|
|
named = with_block.get("file") if isinstance(with_block, dict) else None
|
|
dockerfile = str(named or "Dockerfile").strip()
|
|
found.append((rel, job_id, re.sub(r"^\./", "", dockerfile), jobs))
|
|
return found
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# NEGATIVE — the gitless stages run exactly what they are pinned to run
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands() -> None:
|
|
"""Set equality on command TEXT, not a predicate over it.
|
|
|
|
A suite run put back in ANY spelling — a heredoc, `sh -c`, `npm t`, `timeout npm test`, a
|
|
`#`-suffixed line — is simply not equal to the pin. That is the whole point: three earlier
|
|
versions asked what a command meant and were wrong nine times; this one asks whether it is the
|
|
command that was reviewed.
|
|
"""
|
|
faults: list[str] = []
|
|
seen: set[str] = set()
|
|
for rel, stage, commands in spa_carrying_stages():
|
|
key = f"{rel}::{stage}"
|
|
seen.add(key)
|
|
pinned = PINNED_STAGE_COMMANDS.get(key)
|
|
if pinned is None:
|
|
faults.append(
|
|
f"{key} carries the SPA source but has no entry in `PINNED_STAGE_COMMANDS`, so "
|
|
f"nothing checks what it runs. It runs: {commands}. Add the entry — and if any of "
|
|
f"those commands runs the vitest suite, that is the ersatztv#887 defect: the stage "
|
|
f"has no git checkout and no git binary, so the suite's git-dependent members "
|
|
f"cannot run there, and the red is unreachable on a PR."
|
|
)
|
|
continue
|
|
if tuple(commands) != pinned:
|
|
faults.append(
|
|
f"{key} does not run its pinned commands.\n pinned: {list(pinned)}\n actual: {commands}\n"
|
|
f" If this change is deliberate, update `PINNED_STAGE_COMMANDS` in the same commit "
|
|
f"and say why. If it added a vitest run, do not: this stage cannot run the suite "
|
|
f"(ersatztv#887), and excluding the specs that fail there is the hand-maintained "
|
|
f"population that broke every image build in ersatztv#883."
|
|
)
|
|
for orphan in sorted(set(PINNED_STAGE_COMMANDS) - seen):
|
|
faults.append(
|
|
f"`PINNED_STAGE_COMMANDS` names {orphan}, which no longer exists or no longer carries "
|
|
f"the SPA source. A pin for a stage that is not checked is a sentence nobody is reading."
|
|
)
|
|
assert not faults, "\n\n".join(faults)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# POSITIVE — the image is gated on the job holding the pinned step
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_every_IMAGE_PUBLISHING_job_is_gated_on_the_job_that_runs_the_suite() -> None:
|
|
"""Without this, deleting `needs:` is silently green."""
|
|
known = set(dockerfiles())
|
|
faults: list[str] = []
|
|
for rel, job_id, dockerfile, jobs in publishing_jobs():
|
|
if dockerfile not in known:
|
|
faults.append(
|
|
f"{rel}: job `{job_id}` builds `{dockerfile}`, which is not a tracked Dockerfile. "
|
|
f"The population here is derived from the git index, so this job's image is "
|
|
f"UNCHECKED — fix the path or widen `dockerfiles()`."
|
|
)
|
|
continue
|
|
if not any(stage_rel == dockerfile for stage_rel, _, _ in spa_carrying_stages()):
|
|
continue
|
|
gates = _transitive_needs(job_id, jobs)
|
|
# `needs:` names a job in the SAME workflow, so the edge only reaches the pinned step when
|
|
# the publishing job lives in `GATING_WORKFLOW`. Matching a bare job id anywhere was a false
|
|
# green: a second workflow publishing this Dockerfile while `needs:`-ing its OWN unrelated
|
|
# job called `test` (`run: echo ok`) satisfied it — measured, and it escaped every
|
|
# workflow-population guard in the repo, 588 tests green.
|
|
if rel != GATING_WORKFLOW or GATING_JOB not in gates:
|
|
faults.append(
|
|
f"{rel}: job `{job_id}` publishes an image built from `{dockerfile}`, which carries "
|
|
f"the SPA source, but does not transitively `needs:` `{GATING_JOB}` IN "
|
|
f"`{GATING_WORKFLOW}` — the job holding the pinned suite step. It needs: "
|
|
f"{sorted(gates)}. A `needs:` edge resolves within its own workflow, so a job named "
|
|
f"`{GATING_JOB}` in a DIFFERENT workflow is a different job and gates nothing. Since "
|
|
f"ersatztv#887 the image build does not run the suite itself, so this edge is the "
|
|
f"ONLY thing between a red suite and a published image."
|
|
)
|
|
assert not faults, "\n".join(faults)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# EFFECTIVE — the gating step really runs, really fails, and really runs everything
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_the_GATING_STEP_is_exactly_what_was_reviewed() -> None:
|
|
"""Its body, its `if:`, and the absence of `continue-on-error` — all pinned.
|
|
|
|
Pinning the BODY is what makes "unfiltered" and "not suppressed" true without parsing: a
|
|
`--exclude`, a `|| true`, a pipe, a `set +e`, a `true || npm test` that never executes, are each
|
|
a different string from the pin. Three earlier versions tried to recognise those individually and
|
|
missed `true || npm test` and `continue-on-error: ${{ … }}` among others.
|
|
"""
|
|
named = [
|
|
candidate
|
|
for candidate in _steps(workflow_jobs(GATING_WORKFLOW).get(GATING_JOB, {}))
|
|
if str(candidate.get("name", "")).strip() == GATING_STEP_NAME
|
|
]
|
|
assert len(named) == 1, (
|
|
f"{GATING_WORKFLOW}: job `{GATING_JOB}` has {len(named)} steps named {GATING_STEP_NAME!r}, "
|
|
f"not one. That name is what both this assertion and the substring sweep key on, so a "
|
|
f"SECOND step wearing it inherits the exemption and is never checked — measured by cold "
|
|
f"review with a duplicate carrying `continue-on-error: true` and a spec filter."
|
|
)
|
|
step = named[0]
|
|
|
|
faults: list[str] = []
|
|
body = step.get("run")
|
|
# PER LINE, not over the whole body: `_normalise` collapses newlines, so changing `run: |` to
|
|
# `run: >` folds the two lines into one command whose normalised text is byte-identical to the
|
|
# pin. Measured by cold review — the marker script then swallows the suite as its arguments.
|
|
if not isinstance(body, str) or _normalise_lines(body) != _normalise_lines(GATING_STEP_RUN):
|
|
faults.append(
|
|
f"the gating step's `run:` is not the pinned command.\n pinned: {GATING_STEP_RUN!r}\n"
|
|
f" actual: {body!r}\n Anything but this — a spec filter, a `|| true`, a pipe, a "
|
|
f"`set +e`, a suite on the right of a `||` that may never execute — changes whether a "
|
|
f"red suite blocks the image. Update the pin deliberately, in the same commit, with a "
|
|
f"reason."
|
|
)
|
|
|
|
workdir = step.get("working-directory")
|
|
if workdir is None or str(workdir).strip() != GATING_STEP_WORKDIR:
|
|
faults.append(
|
|
f"the gating step's `working-directory:` is {workdir!r}, not the pinned "
|
|
f"{GATING_STEP_WORKDIR!r}. `npm test` run anywhere else runs a different package, or "
|
|
f"none, while the pinned command text is unchanged."
|
|
)
|
|
|
|
shell = ((yaml.safe_load(_read(GATING_WORKFLOW)) or {}).get("defaults") or {}).get("run", {}).get("shell")
|
|
if shell is None or str(shell).strip() != GATING_WORKFLOW_SHELL:
|
|
faults.append(
|
|
f"{GATING_WORKFLOW}'s `defaults.run.shell` is {shell!r}, not the pinned "
|
|
f"{GATING_WORKFLOW_SHELL!r}. `bash` here means `bash -e`, which is what makes a failing "
|
|
f"command fail the step; changing it changes whether a red suite blocks the image, "
|
|
f"without touching the step."
|
|
)
|
|
|
|
if stage_shell_overrides():
|
|
faults.append(
|
|
f"a pinned Dockerfile stage carries a `SHELL` instruction: {stage_shell_overrides()}. "
|
|
f"That redefines what every later `RUN` executes while leaving its text — and therefore "
|
|
f"the pin — unchanged."
|
|
)
|
|
|
|
condition = step.get("if")
|
|
if condition is None or _normalise(str(condition)) != _normalise(GATING_STEP_IF):
|
|
faults.append(
|
|
f"the gating step's `if:` is not the pinned condition.\n pinned: {GATING_STEP_IF!r}\n"
|
|
f" actual: {condition!r}\n A third term here lets the suite not run at all while the "
|
|
f"image still publishes."
|
|
)
|
|
|
|
# ANY spelling, not two literals. `continue-on-error: ${{ … }}` is the idiomatic conditional form
|
|
# and a check comparing against `True`/`"true"` waves it through — a presence test cannot see
|
|
# polarity, and this one is fail-OPEN in exactly the direction that matters.
|
|
job = workflow_jobs(GATING_WORKFLOW).get(GATING_JOB, {})
|
|
for scope, holder in (("step", step), ("job", job)):
|
|
if "continue-on-error" in holder and holder["continue-on-error"] not in (False, "false"):
|
|
faults.append(
|
|
f"the gating {scope} carries `continue-on-error: {holder['continue-on-error']!r}`, "
|
|
f"so a red suite reports success and the `needs:` edge that gates the image blocks "
|
|
f"nothing. Only absent, `false` or `False` is acceptable here."
|
|
)
|
|
|
|
if "shell" in step:
|
|
faults.append(
|
|
f"the gating step carries its own `shell: {step['shell']!r}`, which overrides the pinned "
|
|
f"workflow default. The default is pinned because `bash` means `bash -e`; a step-level "
|
|
f"override moves that decision somewhere nothing checks."
|
|
)
|
|
if "defaults" in job:
|
|
faults.append(
|
|
f"the job `{GATING_JOB}` carries `defaults: {job['defaults']!r}`, which overrides the "
|
|
f"pinned workflow default for every step in it, including the gating one."
|
|
)
|
|
|
|
if "if" in job:
|
|
faults.append(
|
|
f"the job `{GATING_JOB}` carries a job-level `if:` ({job['if']!r}). None did when "
|
|
f"ersatztv#887 made this `needs:` edge the only gate, and a job that does not run cannot "
|
|
f"gate anything — Gitea reports it `skipped`, and what that does to a dependent job is "
|
|
f"not something this repo relies on."
|
|
)
|
|
assert not faults, "\n".join(faults)
|
|
|
|
|
|
def test_the_npm_SCRIPT_MAP_is_exactly_what_was_reviewed() -> None:
|
|
"""The whole map, because a pinned `RUN npm …` executes whatever it says.
|
|
|
|
An earlier version pinned only the scripts whose body contained `vitest`, which is a SELECTOR —
|
|
the category this file elsewhere calls the worst-behaved, because going short is silent. It went
|
|
short four ways, each re-arming ersatztv#887 in the gitless stage with every other pin matching:
|
|
`npm run test`, `npm t`, and the `prebuild`/`preinstall` LIFECYCLE HOOKS, which npm runs for
|
|
`npm run build` and `npm ci` without anything naming them. Pinning the map removes the category:
|
|
a script that does not exist cannot be a hook, and one that changes is not equal.
|
|
"""
|
|
scripts = json.loads(_read(PACKAGE_JSON)).get("scripts", {})
|
|
assert scripts == PINNED_PACKAGE_SCRIPTS, (
|
|
f"`{PACKAGE_JSON}`'s scripts are not the pinned map.\n pinned: {PINNED_PACKAGE_SCRIPTS}\n"
|
|
f" actual: {scripts}\n Both pinned `RUN npm …` lines in the gitless `web-build` stage "
|
|
f"execute whatever this says, INCLUDING npm's `pre*`/`post*` lifecycle hooks, which nothing "
|
|
f"in the Dockerfile names. A suite run added here reaches that stage with no Dockerfile or "
|
|
f"workflow line changed — ersatztv#887 through a file nobody would think to check against "
|
|
f"this guard. Update the pin deliberately, in the same commit, with a reason."
|
|
)
|
|
|
|
|
|
def test_the_VITE_CONFIG_is_exactly_what_was_reviewed() -> None:
|
|
"""The whole file, because every partial match of it was defeated.
|
|
|
|
`test.exclude`/`include` decide which specs the gating run collects, and `plugins:` can shell out
|
|
to the suite from a build hook — both inside a file the Dockerfile never mentions. Pinning a
|
|
BLOCK of it was defeated seven measured ways across three review rounds: a decoy copy above
|
|
`defineConfig` with the real member respelled `test: {`, `test : {` or `"test": {`; the same for
|
|
`plugins:`; and two that never touched the marker — `[…].concat([evil])` and a trailing
|
|
`...moreTest` spread, which replaces the pinned object because `defineConfig` is identity in both
|
|
vite and vitest. Comparing the file entire has no marker to respell and nothing after the span.
|
|
"""
|
|
actual = _normalise_lines(_read(VITE_CONFIG))
|
|
pinned = _normalise_lines(PINNED_VITE_CONFIG)
|
|
assert actual == pinned, (
|
|
f"`{VITE_CONFIG}` is not the pinned file. First difference:\n"
|
|
+ "\n".join(
|
|
f" line {index + 1}:\n pinned: {pinned[index] if index < len(pinned) else '<absent>'}\n"
|
|
f" actual: {actual[index] if index < len(actual) else '<absent>'}"
|
|
for index in range(max(len(actual), len(pinned)))
|
|
if index >= min(len(actual), len(pinned)) or actual[index] != pinned[index]
|
|
)
|
|
+ "\n\n This file decides which specs the gating run collects and what `vite build` loads "
|
|
"in the gitless stage. Any edit — including a comment — reddens, deliberately: every "
|
|
"partial match of it was defeated. Update `PINNED_VITE_CONFIG` in the same commit, with a "
|
|
"reason."
|
|
)
|
|
|
|
|
|
def test_NO_OTHER_vitest_CONFIG_outranks_the_pinned_one() -> None:
|
|
"""A pinned config is worthless if a second file can take precedence over it.
|
|
|
|
Vitest prefers `vitest.config.*` (and `vitest.workspace.*`/`vitest.projects.*`) to
|
|
`vite.config.*`. MEASURED: dropping a `web/vitest.config.ts` with `include: ['nope/**']` and
|
|
`passWithNoTests: true` beside the pinned file made the suite collect zero specs and exit 0 — the
|
|
gating step green having run nothing at all, which is worse than the filtered run ersatztv#887
|
|
removed. The construct is refused rather than modelled: there is no such file today, so the
|
|
honest move is to assert none appears.
|
|
"""
|
|
intruders = sorted(
|
|
path
|
|
for path in tracked_files._git_ls_files()
|
|
# DIRECT children of `web/` only: vite resolves its config from the project ROOT, so a
|
|
# `web/e2e/vite.config.ts` outranks nothing — and `web/e2e/` is a real directory. Reddening
|
|
# on it would be fail-noisy with a message that is simply false.
|
|
if path.startswith("web/")
|
|
and "/" not in path[len("web/") :]
|
|
and path.rpartition("/")[2].startswith(OUTRANKING_CONFIG_PREFIXES)
|
|
and path != VITE_CONFIG
|
|
)
|
|
assert not intruders, (
|
|
f"these files outrank the pinned `{VITE_CONFIG}` in vite's or vitest's config resolution: "
|
|
f"{intruders}. Whatever they say decides what the gating run COLLECTS and what `vite build` "
|
|
f"loads in the gitless stage, and the pin on `{VITE_CONFIG}` cannot see either — a suite "
|
|
f"that collects nothing still exits 0, and a plugin loaded from a `.js` config still runs. "
|
|
f"Either fold the settings into `{VITE_CONFIG}` and its pin, or pin this file too."
|
|
)
|
|
|
|
|
|
def test_the_DOCS_ONLY_arm_cannot_publish_an_image() -> None:
|
|
"""The premise under half of `GATING_STEP_IF`, asserted rather than assumed.
|
|
|
|
The gating step is allowed to skip when `docs_only` is true. That is only safe because the
|
|
publish step skips on the same condition, so the arm ships nothing. Round-1 review found this
|
|
stated as fact in the prose with nothing checking it — the same shape as the defect one level in.
|
|
"""
|
|
step = find_step(GATING_WORKFLOW, "build", PUBLISH_STEP_NAME)
|
|
assert step is not None, f"{GATING_WORKFLOW}: job `build` has no step named {PUBLISH_STEP_NAME!r}."
|
|
condition = step.get("if")
|
|
assert condition is not None and _normalise(str(condition)) == _normalise(PUBLISH_STEP_IF), (
|
|
f"the publish step's `if:` is {condition!r}, not the pinned {PUBLISH_STEP_IF!r}. The gating "
|
|
f"step is permitted to skip on `docs_only`, and that is safe ONLY because this step skips "
|
|
f"on it too. Without this, a docs-only push to `main` skips the SPA suite and publishes an "
|
|
f"image anyway."
|
|
)
|
|
|
|
|
|
def test_only_the_PINNED_step_mentions_the_suite() -> None:
|
|
"""A substring sweep, deliberately not a semantic predicate.
|
|
|
|
Every earlier version tried to decide whether a command RUNS the suite and was wrong nine times.
|
|
This asks the much weaker question — does any OTHER step's shell body mention it at all — whose
|
|
failure mode is a false red asking a human to look, never a false green.
|
|
"""
|
|
strays: list[str] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
for job_id, job in workflow_jobs(rel).items():
|
|
for step in _steps(job):
|
|
body = step.get("run")
|
|
if not isinstance(body, str):
|
|
continue
|
|
if (
|
|
rel == GATING_WORKFLOW
|
|
and job_id == GATING_JOB
|
|
and str(step.get("name", "")).strip() == GATING_STEP_NAME
|
|
):
|
|
continue
|
|
hit = [mention for mention in SUITE_MENTIONS if mention in body]
|
|
if hit:
|
|
strays.append(f"{rel} :: job `{job_id}` step {str(step.get('name', '?'))!r} mentions {hit}")
|
|
assert not strays, (
|
|
"these steps mention the SPA suite and are not the pinned gating step:\n "
|
|
+ "\n ".join(strays)
|
|
+ "\n\nThis is a SUBSTRING sweep, not a claim that they run it — look, and either move the "
|
|
"run into the pinned step or record why this one is not a second, unpinned gate."
|
|
)
|
|
|
|
|
|
def test_no_DOCKERFILE_outside_a_pinned_stage_mentions_the_suite() -> None:
|
|
"""The same weak sweep over Dockerfiles, covering stages that carry no SPA source.
|
|
|
|
A stage without `web/` has no suite to run, so `PINNED_STAGE_COMMANDS` does not cover it. This
|
|
reports a mention there anyway rather than leaving the class unexamined.
|
|
"""
|
|
strays: list[str] = []
|
|
for rel in dockerfiles():
|
|
for name, lines in stages(_read(rel)):
|
|
if f"{rel}::{name}" in PINNED_STAGE_COMMANDS:
|
|
continue
|
|
for line in lines:
|
|
hit = [mention for mention in SUITE_MENTIONS if mention in line]
|
|
if hit:
|
|
strays.append(f"{rel} :: stage `{name}`: {line!r} mentions {hit}")
|
|
assert not strays, "these unpinned Dockerfile stages mention the SPA suite:\n " + "\n ".join(strays)
|
|
|
|
|
|
# ------------------------------------------------------------------------------------------------
|
|
# Anti-vacuity and negative controls
|
|
# ------------------------------------------------------------------------------------------------
|
|
|
|
|
|
def test_no_run_BODY_builds_or_pushes_an_image() -> None:
|
|
"""`publishing_jobs()` keys on the ACTION, which is a selector; assert nothing publishes around it.
|
|
|
|
A job doing `docker build -f ./docker/Dockerfile … && docker push …` in a shell body is not in
|
|
that population, so the positive invariant is silently not applied to it — and anti-vacuity does
|
|
not notice, because the real `build` job is still there. Measured MISSED by cold review after
|
|
this assertion was dropped in the rewrite; it is restored rather than reasoned away.
|
|
"""
|
|
stray: list[str] = []
|
|
for path in workflow_files():
|
|
rel = path.relative_to(REPO_ROOT).as_posix()
|
|
for job_id, job in workflow_jobs(rel).items():
|
|
for step in _steps(job):
|
|
body = step.get("run")
|
|
if not isinstance(body, str):
|
|
continue
|
|
for line in body.splitlines():
|
|
if line.lstrip().startswith("#"):
|
|
continue
|
|
if re.search(r"\bdocker\s+(?:push|build|buildx\s+build)\b", line):
|
|
stray.append(f"{rel} :: job `{job_id}`: {line.strip()!r}")
|
|
assert not stray, (
|
|
f"image build/push outside `{PUBLISH_ACTION}`:\n " + "\n ".join(stray) + "\n\n"
|
|
"`publishing_jobs()` derives from that action, so these jobs are NOT checked for the suite "
|
|
"gate — extend it before adding one."
|
|
)
|
|
|
|
|
|
def test_the_populations_are_NOT_empty() -> None:
|
|
"""Every assertion above passes over an empty set, which is how a completeness guard dies."""
|
|
assert dockerfiles(), "`git ls-files` reported no Dockerfile — the derivation is broken, not the repo."
|
|
assert workflow_files(), f"`git ls-files` reported no {WORKFLOW_DIR}/*.yml — the derivation is broken."
|
|
|
|
carrying = spa_carrying_stages()
|
|
assert carrying, (
|
|
f"no stage in {dockerfiles()} was read as carrying the SPA source, so the negative assertion "
|
|
"checked nothing and the positive one skipped every job. `copies_spa_source` or `stages` "
|
|
"stopped matching."
|
|
)
|
|
assert PINNED_STAGE_COMMANDS, "the pin table is empty; the negative assertion asserts nothing."
|
|
|
|
published = publishing_jobs()
|
|
assert published, (
|
|
f"no job in {WORKFLOW_DIR} uses `{PUBLISH_ACTION}`, so the gate assertion ran over an empty "
|
|
"population. Either the repo stopped publishing images or `_canonical_action` stopped "
|
|
"recognising the action."
|
|
)
|
|
assert any(dockerfile in {rel for rel, _, _ in carrying} for _, _, dockerfile, _ in published), (
|
|
"no publishing job builds an SPA-carrying Dockerfile, so the gate assertion is vacuous."
|
|
)
|
|
|
|
|
|
def test_the_STAGE_SPLIT_finds_the_real_stages() -> None:
|
|
"""`stages()` is the only Dockerfile syntax this file interprets; pin what it returns.
|
|
|
|
A split that stopped recognising `FROM … AS name` would put every instruction in one bucket, the
|
|
pin key would never match, and `test_every_SPA_CARRYING_STAGE_runs_exactly_its_pinned_commands`
|
|
would report an orphan pin — loud, but for the wrong reason. Pinned so the cause is legible.
|
|
"""
|
|
names = [name for name, _ in stages(_read("docker/Dockerfile"))]
|
|
assert names[:3] == ["dotnet-runtime", "web-build", "runtime-base"], names
|
|
assert [stage for _, stage, _ in spa_carrying_stages()] == ["web-build"], spa_carrying_stages()
|
|
|
|
|
|
def test_the_SPA_CARRYING_predicate_reads_every_COPY_SPELLING() -> None:
|
|
"""Pinned both ways: every stage answered False is SKIPPED by the negative assertion.
|
|
|
|
The synthetic spellings were MEASURED False by cold review before this was widened; a new stage
|
|
using any of them would have gone unpinned and unchecked.
|
|
"""
|
|
for spelling in (
|
|
'COPY ["web/", "/source/web/"]',
|
|
'COPY --chown=node:node ["web/", "/source/web/"]',
|
|
"COPY web /source/web",
|
|
"ADD web/ /source/web/",
|
|
"COPY web/. ./web/",
|
|
"COPY . /source/",
|
|
"COPY ./ /source/",
|
|
"COPY --chown=node:node web/ /source/web/",
|
|
"COPY web/package*.json ./web/",
|
|
# A GLOB that matches `web/`. Measured False before `_source_reaches_web`, which left a
|
|
# stage copying the source this way unpinned and unchecked.
|
|
"COPY web*/ ./",
|
|
"COPY * /source/",
|
|
):
|
|
assert copies_spa_source(spelling), spelling
|
|
|
|
# A STAGE copy can still carry the SPA source across. Measured False before this clause: the
|
|
# receiving stage was then unpinned, so a suite run in it was invisible.
|
|
assert copies_spa_source("COPY --from=web-build /source/web /source/web")
|
|
assert copies_spa_source("COPY --from=builder /app/web ./web")
|
|
# The EXEC form of the same thing. Measured False before the parse order was fixed — splitting
|
|
# `["/source/web", "/dest"]` on whitespace left a trailing comma on the first token, so the
|
|
# segment test never matched. Pinned here because only the development battery witnessed it, and
|
|
# that battery is explicitly not standing.
|
|
assert copies_spa_source('COPY --from=web-build ["/source/web", "/dest"]')
|
|
assert copies_spa_source('COPY --from=web-build ["/source/web","/dest"]')
|
|
assert not copies_spa_source('COPY --from=web-build ["/source/ErsatzTV/wwwroot/app/.", "/dest"]')
|
|
|
|
for spelling in (
|
|
"COPY design-system/. ./design-system/",
|
|
"COPY --from=web-build /source/ErsatzTV/wwwroot/app/. ./app/",
|
|
"COPY webhooks/. ./webhooks/", # `web` must be a whole path segment
|
|
"RUN npm ci",
|
|
"COPY web/", # a single argument is not a copy
|
|
):
|
|
assert not copies_spa_source(spelling), spelling
|
|
|
|
|
|
def test_the_INSTRUCTION_JOIN_survives_what_defeated_the_PARSER() -> None:
|
|
"""The shapes that broke three earlier versions must not break the pin comparison.
|
|
|
|
None of these needs interpreting — they only need to arrive at the comparison intact, so that a
|
|
stage running them is NOT EQUAL to its pin. Each was a measured false green when this file tried
|
|
to decide what the command meant.
|
|
"""
|
|
heredoc = "RUN <<EOF\nnpm test -- --run\nEOF\n"
|
|
assert "RUN <<EOF" in instructions(heredoc)[0], instructions(heredoc)
|
|
assert instructions("RUN echo hi # npm test -- --run\n") == ["RUN echo hi # npm test -- --run"]
|
|
assert instructions("RUN a=(1 2); npm test\n") == ["RUN a=(1 2); npm test"]
|
|
assert instructions("RUN npm run lint && \\\n npm test\n") == ["RUN npm run lint && npm test"]
|
|
# A comment LINE is still dropped, which is what lets the Dockerfile explain the rule.
|
|
assert instructions("# RUN npm test -- --run\nRUN npm ci\n") == ["RUN npm ci"]
|
|
|
|
|
|
def test_the_PIN_rejects_every_spelling_the_PARSER_missed() -> None:
|
|
"""The nine shapes that defeated the predicate, run against the pin instead.
|
|
|
|
Not one of them is equal to `RUN npm run lint && npm run typecheck && npm run build`, which is
|
|
the entire argument for replacing the mechanism: the pin does not need to recognise a spelling to
|
|
reject it.
|
|
"""
|
|
pinned = PINNED_STAGE_COMMANDS["docker/Dockerfile::web-build"]
|
|
for spelling in (
|
|
"RUN <<EOF",
|
|
"RUN echo build#1 && npm test -- --run",
|
|
"RUN (npm run lint); npm test -- --run",
|
|
"RUN npm t -- --run",
|
|
"RUN ./node_modules/.bin/vitest run",
|
|
"RUN pnpm vitest run",
|
|
"RUN timeout 900 npm test -- --run",
|
|
"RUN su node -c 'npm test -- --run'",
|
|
"RUN sh -c 'npm test -- --run'",
|
|
"RUN npm run lint && npm run typecheck && npm test -- --run && npm run build",
|
|
):
|
|
assert spelling not in pinned, spelling
|
|
assert (pinned[0], spelling) != pinned, spelling
|