fix(649): point the ENFORCED review-verdict gate at the shared PR-file enumeration #666

Merged
timothy merged 4 commits from fix/649-enforced-verdict-guard into main 2026-07-27 00:04:07 +02:00
4 Commits
Author SHA1 Message Date
timothy e960d5b918 test(649): make the POST-wiring assertion unable to opt out or accept the wrong host
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
PR Gates / Script tests (pytest) (pull_request) Successful in 36s
Review verdict / Set review-verdict status (pull_request) Successful in 40s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
review-verdict/h10 Review-verdict: MERGEABLE @ e960d5b
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m28s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 23m3s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 24m2s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Re-review found the verifier could disable itself two ways, both of which look like
coverage:

- it was guarded by `if url_file.exists()`, so deleting the recorder in the stub turned
  the whole assertion into a no-op and every test stayed green;
- it compared only the URL SUFFIX, so a POST to the right path on the wrong HOST or the
  wrong REPO passed — which is exactly the class the assertion was added to catch.

It now requires the URL to have been recorded whenever a status was posted, and compares
the full URL against the env the job was given. Mutation-verified three ways: wrong host,
wrong repo, and deleting the recorder each redden the suite.

Refs #649
2026-07-26 23:38:25 +02:00
timothy 3885fd6aea docs(649): narrow the enumeration's stated guarantees to what it actually proves
Two limitations the cold review surfaced are now written where the guarantees are
described, rather than living only in a review transcript. Both are pre-existing and
tracked separately (#663, #664); neither is fixed here.

- Head-sha binding detects ONE-WAY movement. An A->B->A force-push round trip restores
  the expected sha, so the binding holds while the pages came from two states. The record
  previously read as though the race were closed.
- A commit status is repo-GLOBAL, so a success earned on one PR is inherited by any other
  PR with the same head. Same property that makes the per-sha binding work, read from the
  other end.

Refs #649

Decisions-Edit: yes
2026-07-26 23:29:44 +02:00
timothy 322dd43d10 fix(649): close the test-isolation gaps cold review found, and make the job's Gitea config authoritative
Four findings acted on; two more are real but pre-existing and are being filed rather
than fixed here (see below).

**The job's Gitea config was not authoritative.** `pr-changed-files.sh` resolves
`ETV_GITEA_URL` BEFORE `GITEA_BASE_URL` (and `ETV_GITEA_TOKEN` before `GITEA_TOKEN`),
because its other caller is a developer Mac using the ETV_* convention. Setting only the
GITEA_* names meant a runner exporting a stale ETV_GITEA_URL would enumerate a DIFFERENT
Gitea instance and this job would post a verdict here from a diff read there. Both names
are now set to the same value, so precedence cannot matter.

**Three guards passed their tests for the wrong reason.** Each was confirmed by deleting
the clause and watching the suite stay green — the reviewer asserted it, mutation proved
it:

- The explicit empty-response clause was uncovered on jq 1.8, because jq 1.8 rejects
  empty input by itself. jq 1.6 does not, and THE RUNNER SHIPS 1.6 — so the one
  environment where the clause is load-bearing had no coverage. That is the #643/#647
  failure class reproduced inside the suite meant to prevent it. Now covered by importing
  the existing jq-1.6 shim (imported, not copied — a second quirk emulator is the same
  drift problem one level down), with a verify-the-verifier test and a positive control.
- `type == "array"` needed a body whose VALUES are valid rows. Two earlier attempts
  failed for a third reason: `jq`'s `all(.[]; …)` iterates an object's values, so
  `{"message":"…"}` and a single flat row are both rejected by `.filename` erroring on a
  string. Only `{"0": {…valid row…}}` reaches the fail-open, where a non-array body
  enumerates as a complete docs-only list.
- `.filename | ok` is now isolated by a row carrying a valid `.status` and no filename,
  removing the closed-allow-list as a second reason to reject.

**Two assertions proved less than their names claimed.** `"jq-preflight.sh" in code` also
matched the `[ -x … ]` presence guard, so deleting the invocation left it green; it now
requires an invoking line. `_run_classify` accepted every POST, so a status aimed at the
wrong endpoint or sha would not have been noticed; it now asserts the POST lands on
`/statuses/<full head sha>`.

**One test name overclaimed** and is narrowed rather than left implying coverage it does
not have: the head-movement test proves "final head != expected sha", not movement
*during* enumeration.

Deferred, both pre-existing and neither introduced here — filed as follow-ups:
- A commit status is repo-global, so a `review-verdict/h10=success` obtained for head H
  on one PR is inherited by any other PR with the same head, including one opened against
  a different base. Same class as #632, reached by a third route.
- The A->B->A force-push race: paging is several round-trips and the head is re-read once
  at the end, so a restore to the original sha passes the binding while the pages came
  from two states. Inherent to enumerating a mutable list over an API with no
  commit-pinned files endpoint.

Refs #649
2026-07-26 23:23:52 +02:00
timothy 9114a7e8af fix(649): point the ENFORCED review-verdict gate at the shared PR-file enumeration
#658 landed the shared implementation, `scripts/pr-changed-files.sh`, and rewired the
ADVISORY hook onto it. The ENFORCED copy — the one that writes the branch-protection-
required `review-verdict/h10` status — was left byte-identical to main, so its
fail-closed behaviour on a malformed or empty response stayed INCIDENTAL: an empty `n`
erroring `[ "$n" -lt 50 ]` to false. That is #649's second Done-when box, and the whole
point of the issue was that the gate with real authority was weaker than the gate with
none.

`review-verdict.yml` now:

- checks out the PR's BASE ref (`base.sha`, `persist-credentials: false`), never the
  head, so a PR cannot supply the code that judges it;
- runs `scripts/jq-preflight.sh` in FLOOR-ONLY mode — `--expect` here would deadlock
  every merge on `main` the day the runner's jq changes;
- calls `scripts/pr-changed-files.sh` and reads its EXIT STATUS, never its stdout on a
  failure path. The env trap flagged in review is handled: the script reads
  GITEA_BASE_URL and takes owner/repo as two separate arguments, so passing BASE_URL and
  a combined `owner/repo` would have silently fallen back to the hardcoded LAN default.

The ~40 lines of inline enumeration are deleted, so the two copies can no longer drift.
A base ref predating #658 has no such script; that posts `pending` with the reason
rather than dying with no status at all.

The drift guard is re-tightened from "the hook uses the shared script" to "BOTH callers
do", and the workflow's own preconditions are pinned by parsing the YAML rather than
substring-matching it — `head.sha` for `base.sha` is a nine-character diff.

Verified by mutation, six mutants, each killed by its intended test: ignore the exit
status; check out the head; drop `persist-credentials`; add `--expect`; re-inline a
`pulls/N/files?` fetch; delete the PROTECTED clause.

That last one initially MISSED, and the miss was the useful finding. The test used a
docs-only-plus-protected file list and passed with the clause deleted, because
PROTECTED (`.claude/ .gitea/ .husky/ scripts/ docker/ci/`) and DOCS_ONLY (`docs/`, root
`*.md`) are disjoint — on the docs-only path that clause can never fire, and DOCS_ONLY
was doing all the work. PROTECTED is load-bearing only on the BOT path, so the test now
covers a Renovate PR editing the shared script, with a positive control proving the bot
exemption fires at all.

The caller contract is tested by EXECUTING the workflow's `run:` block against a stubbed
enumeration that fails while emitting a perfectly docs-only list — the one combination
the "every failure path also happens to print nothing" redundancy cannot absorb, and the
exact mutation that survived the whole suite last round.

Docs: both "Landing note" blocks removed, and the record's base-ref paragraph converted
from a future-tense requirement to present-tense fact with its staging rationale kept as
history.

Refs #649

Decisions-Edit: yes
2026-07-26 23:05:14 +02:00