Files
ersatztv/scripts
timothy e855a79999 fix(786,789): make the new proofs exercise the shipped detectors, and rename advisory -> report-only
Round three, reviewed twice in parallel (Codex gpt-5.6 and a cold Claude brief). Both converged
independently on the same two MEDIUM findings, which are the branch's own defect class reproduced
by its most recent commits: a test asserting a guarantee it does not provide.

1. THE PROOFS DID NOT EXERCISE THE CODE THEY NAMED. `test_MUTATION_a_DUPLICATE_row_is_reported`
   rebuilt the comparison with its own `len(set(...))`, and `test_MUTATION_a_SHADOWING_env_key_is_
   reported` built a synthetic dict and intersected it by hand. Both stayed GREEN with the live
   check disarmed — a behavioural test wearing a MUTATION name, which is exactly what
   `testing.guard-ships-with-mutation-proof` excludes.

   `duplicate_row_faults(rows)` and `env_shadow_faults(docs)` are now named detectors; the live
   test and its proof both call them. Verified by reproducing the reviewers' own demonstrations
   against the fix:
     duplicate detector neutered -> REDDENED "the DETECTOR did not report a second row for ..."
     shadow    detector neutered -> REDDENED "the DETECTOR did not report a shadowing job-level key"

2. `test_a_commented_out_marker_does_NOT_count` CLAIMED TO GUARD A TEXT-SCAN REGRESSION AND DID
   NOT. It re-implemented the read inline and never called `declared_roles`. One reviewer built a
   comment-blind grep reader reproducing all 17 current values byte-identically and ran every
   non-parametrised test in the file: nothing went red.

   `declared_roles(paths=None)` now takes an injectable population, so the test drives the REAL
   reader over a fixture, asserts the fault function reports it, and pins that a live marker in the
   same shape is still read (so the assertion cannot be satisfied by a reader returning None for
   everything). Verified: installing the grep reader now reddens it.

RENAMED `advisory` -> `report-only`. docs/ci-cd.md already calls `functional-e2e` an advisory job
in a DIFFERENT sense — not a required context, though it can certainly fail — while this branch
declares that job `guard`. A reader would infer the wrong marker for exactly the job whose marker
is `guard`. The value is now unambiguous, its Kind is `REPORT-ONLY`, and the manifest's declared
`expect` string moved in lockstep with the message it matches.

The rename also caught its own adjacent defect: `REPORT-ONLY` contains a hyphen, which the row
regex's `[A-Z]+` Kind group did not match, so the docs-reminder row stopped parsing. That surfaced
as a loud red naming the job — the fail-closed property documented one commit earlier, working.
Widened to `[A-Z-]+`, matching the `BEHAVIOUR-ONLY` convention in the sibling table.

Nine further accuracy findings, each measured before fixing:
  * "would therefore flag every job" — false. git/jq/python3/curl flag 4 of 8. And the REASON was
    wrong too: `small` is git-only and provisions Python via actions/setup-python, while `build`
    runs on stock ubuntu-latest, so "the bare runner has them" conflated two lanes. They are
    excluded because they do not DISCRIMINATE.
  * "four lines from the container: block" / "in one diff hunk" — false. Measured 6, 58, 6, 7, 7;
    `migrations` has the whole `services:` block in between. The property the guard rests on is
    INDEPENDENCE, not proximity, and the docs now say so.
  * "the merge semantics never have to be relied upon" — too strong. The check proves no NAME
    COLLISION; it does not prove the runner merges. That evidence is in-repo and live and is now
    cited: `build`'s Smoke step declares its own step-level `env:` and consumes workflow-level
    `${IMAGE}` on the same line, on every push to main.
  * "every malformed row fails closed" — over-broad. A malformed DUPLICATE is invisible: its job is
    already satisfied by the well-formed row. Stated.
  * The `scan` Blocks cell said "a marked job's run: body"; the ban's scope is DELIMITER_BAN_JOBS
    (the required contexts PLUS `build`), which is wider, and `scan` carries no `if:` so it runs on
    PRs too.
  * The functional-e2e row credited `e2e-local.sh` (TOOLING, only boots the instance) for curl
    assertions that live in `e2e-functional.sh` (GUARD).
  * `report-only` was defined nowhere; now defined as a verdict the job cannot enforce, with the
    limit stated (nothing detects a `guard` job whose checks are all continue-on-error).
  * "inline rows carry Proof: NONE" was stated universally while two rows cite a pytest; hedged,
    and each such row names which PART it covers.
  * "Exactly three jobs are `none`" was an unchecked prose enumeration in three sites, in a branch
    that deleted one for that reason. Removed; the markers are the source.
  * Plus: the ci-cd tool list was short by `dotnet-ef` and `playwright`; docs/README said only a
    `guard` job needs a row; "all three checks fail together" read as the opposite of "go blind";
    and #774's unrealised consequence is now subjunctive rather than past tense.

Suite: 1178 passed, 2 skipped, 1 known flake (test_hook_fire_log's production-log guard, which
compares mtimes under the SHARED ~/.cache/ersatztv/hook-fire/ — four session logs were written
during the window; 78 passed in isolation. ersatztv#809/#822, evidence posted there).

Refs #786
Refs #789
2026-08-28 22:14:44 +02:00
..
2024-01-15 10:29:07 -06:00
2024-01-16 13:28:46 -06:00