fix(806): guard populations over FILES derive from the git index, not a filesystem walk
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 10s
PR Gates / Docs update reminder (pull_request) Successful in 13s
PR Gates / decisions lifecycle (pull_request) Successful in 18s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 11s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 25s
review-verdict/h10 Awaiting review verdict for b0f42f1
Review verdict / Set review-verdict status (pull_request_target) Successful in 10s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 5m38s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m35s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m11s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s

`testing.guard-derives-population-from-source` (#774) says a completeness guard
derives its population from an authoritative source, and its worked examples are an
enum and the generated OpenAPI document. It was silent on the commonest population
in our own guards — files in a directory — and every one of them answered with a
filesystem walk. A walk is not an authoritative source: it reports build output,
generated shims and editor droppings, and it differs per machine.

#778 measured the cost by getting the same population wrong three times in one PR
while implementing the milestone that exists to prevent it. The worst shape was
`Path.rglob` enumerating `.husky/_/` — 17 shims `npm ci` writes, gitignored and
untracked — which made that guard RED on every developer checkout and GREEN in CI,
whose `script-tests` job never runs `npm ci`. Only that one guard was fixed then.
This audits the rest.

CONVERTED (a completeness claim over tracked files):
  * `test_guard_inventory.py` — its `.husky/` walk excluded `.husky/_/` only because
    `_` is a directory, so the obvious "make it recursive" edit would have
    reintroduced #778's defect in the repo's own model guard. Both halves are gated
    on the index: the callers, and the `scripts/…` paths they name — the second was
    left on `Path.exists()` in the first cut and found by cold review.
  * `test_hook_fire_log.py` — an untracked scratch `.sh` in `.claude/hooks/` was
    demanded to carry instrumentation.
  * `test_ci_image_pin_population.py` — and `*.yaml` added: Gitea accepts both
    spellings, so a `.yaml` workflow adopting the toolchain image was structurally
    invisible while the test read as covering every workflow.
  * `test_remote_state_inventory.py` — folded onto the shared derivation, so the
    rule has one implementation rather than two.

ASSESSED AND RECORDED, not silently skipped:
  * `test_ci_release_path_scan_job.py::_repo_copy` — not a completeness claim, but
    it takes its file LIST from the index anyway for hermeticity, since `copytree`
    copied untracked files and `__pycache__` into a tree whose behaviour the probes
    measure. Content still comes from the working tree, and the copy is NOT a git
    repo, so neither file that step runs may use the helper — written down because
    `MARKED_JOBS` is left open as a residual gap, which invites editing exactly
    that file.
  * `test_ci_dropped_step_guard.py` — no filesystem population at all; its members
    come from the parsed workflow.
  * The decisions corpus keeps its walks and is recorded as unexamined rather than
    cleared. This is not "replace every glob".

`scripts/tests/tracked_files.py` is the single derivation.
`test_guard_populations_derive_from_git.py` proves it in two directions, which are
complements rather than a second opinion — measured both ways:
  * REMOVAL, exhaustively: every member of every registered derivation is dropped
    from the index in turn and must disappear while still on disk. One victim was
    not enough — `derived_guard_files` unions four contributors, so a mutant putting
    only one back on a walk passed. This catches a hardcoded `.exists()` admit and
    memoisation, which the other direction cannot.
  * The CALL LOG: a derivation may READ a file but must not LIST a directory. This
    catches an append-only source that yields nothing on this machine — #778's
    shape — which removal cannot see, because it has nothing to remove. Its limits
    are stated in one place and are true in both directions.

Both the population and the proofs are registered against a hand-written scope
mirror that carries its own equality check, and the import matcher's 16 forms are
pinned as a table so the next edit cannot silently re-open one.

The mutation this guard declares in `mutation_manifest.py` (#790) is the real
defect cold review found in its own first round.

Docs: `testing.guard-derives-population-from-source` gains the file-population case
and why the disk is not authoritative; `docs/guard-inventory.md` carries the
per-guard audit table, including the two no-change verdicts and the decisions
corpus recorded as unexamined.

Seven rounds of independent cold review (Codex GPT-5.6 and Opus, alternating) —
the per-round findings and their measurements are in the PR.

fixes #806

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-22 15:48:46 +02:00
co-authored by Claude Opus 5
parent 3a53c50c45
commit b0f42f14a0
13 changed files with 1112 additions and 74 deletions
+20 -11
View File
@@ -34,9 +34,10 @@ from __future__ import annotations
import fnmatch
import re
import subprocess
from pathlib import Path
from scripts.tests import tracked_files
REPO_ROOT = Path(__file__).resolve().parents[2]
INVENTORY = REPO_ROOT / "docs" / "remote-state-inventory.md"
@@ -86,8 +87,9 @@ SCOPE = (
# The lesson each time was the same one this repo already wrote down — derive the population from an
# AUTHORITATIVE source — and the filesystem is not one. It reports build output, editor droppings and
# anything else that happens to be on disk, and it varies per machine. The repo's index is
# authoritative, versioned, identical for every checkout and for CI, and it excludes untracked
# generated files by construction rather than by an exclusion list that must be maintained.
# authoritative: it holds the same set of files every checkout receives from a clone, and it
# excludes untracked generated files by construction rather than by an exclusion list that must be
# maintained.
#
# `scripts/tests/` is still excluded explicitly, because those files ARE tracked. That exclusion is a
# scope decision, reviewable in one line: they run only under pytest and authorize nothing. Their
@@ -104,15 +106,22 @@ _ROW = re.compile(r"^\|\s*`([^`]+?)`[^|]*\|\s*`([^`]+)`\s*\|", re.M)
def _tracked_files() -> list[str]:
"""Every file git tracks, as repo-relative posix paths.
Fails LOUDLY rather than returning nothing: an empty population would make every completeness
assertion below pass vacuously, which is the exact failure this guard exists to prevent.
A thin wrapper over the SHARED derivation rather than a second copy of it (ersatztv#806): one
definition of the rule, not two, which is detector C — dedup by construction — applied to the
file that first stated the rule. The wrapper survives because this module patches
`_tracked_files` by name in its own proofs, and because its scope is RECURSIVE over `scripts/`
where `tracked_children` is deliberately flat.
It therefore carries BOTH its own proofs and a row in that file's `DERIVATIONS`, which is not
the duplication that masks: they cut at different seams — the in-file pair patches
`_tracked_files` (this wrapper), the shared pair patches `_git_ls_files` (the subprocess) — and
each was witnessed red independently, so neither can hide the other's total failure.
Fails LOUDLY rather than returning nothing, now including git's own stderr: an empty population
would make every completeness assertion below pass vacuously, which is the exact failure this
guard exists to prevent.
"""
proc = subprocess.run(
["git", "-C", str(REPO_ROOT), "ls-files", "-z"],
capture_output=True,
check=True,
)
return [p for p in proc.stdout.decode().split("\0") if p]
return tracked_files._git_ls_files()
def derived_population() -> set[str]: