fix(751): review round — the twin null-shape, a governance self-exemption, and four over-claims
Two independent cold reviews (a cross-family GPT-5.6 pass and an isolated Opus pass).
Neither found a path to a green `review-verdict/h10` on an unreviewed head. Both found
real defects BESIDE the fix, which is the failure mode this file keeps producing.
THE TWIN, and the reason not to trust "I fixed the two I could see". `GET
/commits/{sha}/status` returns `statuses: null` — not `[]` — for a head with no
statuses yet: `{"state":"pending","total_count":0,"statuses":null}`, measured on PR
#739's head. `read_existing_verdict` gated on `.statuses | type == "array"` and took
its `exit 1` path, posting NOTHING. Fail-closed, but the user-visible outcome is the
one this issue is about: an exempt PR with no status and, since #743, no bypass. Its
double printed `{"statuses": []}` at all three no-verdict sites, so that branch was
unreachable in the suite — the same unfaithful-double story as the timeline, one
function over. `null` is accepted only when `total_count` is 0, so a body that merely
lost its array is still refused and an existing verdict is still protected. Swept
`scripts/pr-changed-files.sh` too: `pulls/{n}/files` returns `[]`, unaffected. The
generalisable rule is that a nil Go slice serialises to `null`, so every list-shaped
field on this API is suspect and only a per-endpoint measurement settles it.
A GOVERNANCE SELF-EXEMPTION, reachable again precisely because this change works.
`DOCS_ONLY` matched `CLAUDE.md` and `AGENTS.md` — the documents that DEFINE the
completion protocol, the merge-consent convention and the H10 rule. Driving the real
classify body with a lone `CLAUDE.md` change produced `review-verdict/h10=success`.
Protecting `.claude/` while the file specifying what it enforces stayed exemptible is
the same self-exemption the header rules out, one directory over. Both added to
PROTECTED; `README.md` deliberately not (ordinary prose, no enforcement).
FOUR OVER-CLAIMS, corrected rather than defended:
* The repo-wide expression test does NOT catch "any payload that cannot evaluate".
It checks the HEAD TOKEN of each dotted path. `${{ github.ref == }}` and
`${{ …head.sha + }}` pass; so does a renamed output, since tokens after the first
are skipped by design. Claim corrected in the docstring, `docs/ci-cd.md` and the
record. The test is kept permissive on purpose: a red here blocks every merge.
* The strict test's anti-vacuity half banned expressions ANYWHERE outside
`with:`/`env:`, so the standard `if: ${{ always() }}` spelling and even a delimiter
in an inert top-level comment went red — a guard more dangerous than its target.
Replaced with the honest property: the YAML walk saw every `run:` body it declares.
* The `if:` assertion demanded the bare `always()` exactly; now normalised, since the
wrapped form is identical to the runner.
* `exit 1` was matched anywhere in the guard body, so an unreachable
`if false; then exit 1; fi` satisfied it while the real branch said `exit 0`. Now
required INSIDE the missing-marker branch — and the new behavioural test settles it
properly by EXECUTING the guard body both ways.
* The record asserted a repo-wide obligation to guard consequential steps. It is not
repo-wide: `docker-build.yml`'s `test`/`migrations` are also required contexts and a
dropped step there is fail-OPEN (green having done no work), strictly worse than
here. Scoped to this file and tracked as #756 rather than asserted as done.
Also: comments in both files still said it was unestablished whether a later step runs
after a drop — runs 1863/1866 established it, so they now record the measurement; a
cited test name that never existed; `kind` leaked to global scope; a mangled comment
wrap; and an already-false "one event on page 1".
Hardening of my own: `null` now counts as exhaustion only from page 2 ON. Every real
PR's first page carries events (4, 2, 9, 5, 10 across #752/#753/#749/#739/#717), so a
terminator on page 1 means no page was ever read, and certifying "no retarget" from a
response we cannot explain is the one thing the fence exists to refuse. Narrows rather
than closes it: a wrong `null` on page 3 still reads as exhaustion.
Verification: 137 in this file / 452 total green; ELEVEN mutations each red —
reintroducing the defect, deleting the guard, deleting the marker write, removing
`if: always()`, `exit 1`→`exit 0`, a delimiter in the guard body, the fence gate (20
red), the TWIN gate (70 red), dropping the governance paths, accepting a null first
page, and diverging the marker path between the two steps.
Refs: #751
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+37
-4
@@ -1041,10 +1041,15 @@ Three things now hold the line, and they are deliberately different in kind:
|
||||
that absence would be silent as well.
|
||||
- **Two static guards**, in `scripts/tests/test_pr_changed_files.py`: no delimiter in *any* `run:`
|
||||
body of this file (absolute — a dropped step here is a dead merge gate, and its bodies are ~700
|
||||
lines of prose), and repo-wide, every expression payload must name a context or function the runner
|
||||
can resolve (permissive, because the other workflows interpolate into `run:` legitimately — 7
|
||||
occurrences today). The second catches the class: a payload that cannot evaluate, wherever it sits,
|
||||
including in an `if:`.
|
||||
lines of prose), and repo-wide, every expression payload's **head token** must name a context or
|
||||
function the runner can resolve (permissive, because the other workflows interpolate into `run:`
|
||||
legitimately — 7 occurrences today). Be precise about the second one's reach: it catches the
|
||||
historical defect (`pr number`) and a nonexistent context, but **not** a syntactically invalid
|
||||
payload whose tokens are all known (`${{ github.ref == }}` passes), nor a renamed output
|
||||
(`steps.metadata.outputs.shortsha` passes — every token after the first is preceded by `.` and is
|
||||
skipped), nor an unclosed opener. Catching those needs an expression parser. An earlier draft of
|
||||
this section claimed it caught "a payload that cannot evaluate, wherever it sits"; that was false,
|
||||
and the corrected claim is the one to rely on.
|
||||
|
||||
Worth knowing why nothing caught this for three days: every *other* workflow-shape test in that file
|
||||
reads `_code_lines()`, which strips comments. That is correct for what it was for, but it encodes the
|
||||
@@ -1073,6 +1078,34 @@ Two things kept it invisible, and both are worth generalising:
|
||||
The type is now read as a value (`case` over `jq -r 'type'`) rather than through `jq -e`, whose
|
||||
exit-status semantics already bit this workflow once at jq 1.6, and both `null` and `[]` terminate the
|
||||
walk. The regression test is parameterised over both shapes because both are live on this server.
|
||||
`null` is accepted as exhaustion only from **page 2 on** — every real PR's first page carries events
|
||||
(measured: 4, 2, 9, 5, 10 across #752/#753/#749/#739/#717), so a `null` first page is anomalous rather
|
||||
than empty, and the walk should not certify "no retarget happened" from a response it cannot explain.
|
||||
|
||||
**The same nil-slice shape bites `/commits/{sha}/status`** — a third instance, found by cold review of
|
||||
the fix for the second. A head with no statuses yet returns
|
||||
`{"state":"pending","total_count":0,"statuses":null}` (measured on PR #739's head). `read_existing_verdict`
|
||||
gated on `.statuses | type == "array"`, so it hit its `exit 1` and posted nothing at all — fail-closed,
|
||||
same user-visible outcome. `null` is now accepted there only when `total_count` is 0, so a body that
|
||||
merely lost its array is still refused and an existing verdict is still protected from a transient
|
||||
error. `scripts/pr-changed-files.sh` was swept and is unaffected (`pulls/{n}/files` returns `[]`).
|
||||
**The generalisable rule: a nil Go slice serialises to `null`, so every list-shaped field on this API
|
||||
is suspect and only a per-endpoint measurement settles it.**
|
||||
|
||||
**`CLAUDE.md` and `AGENTS.md` are now PROTECTED paths.** `DOCS_ONLY` matched them, so the documents
|
||||
that *define* the completion protocol, the merge-consent convention and the H10 rule were themselves
|
||||
docs-only-exemptible while `.claude/` was protected — the same self-exemption the gate rules out, one
|
||||
directory over. Driving the real classify body with a lone `CLAUDE.md` change produced
|
||||
`review-verdict/h10=success`. It is fixed here rather than deferred because restoring the exemptions is
|
||||
what makes it reachable: no exemption `success` was writable at all while the classify step was
|
||||
dropped. `README.md` is deliberately not listed — ordinary prose, no enforcement. For the same reason,
|
||||
#706's known residual returns with the working fence: while `rt_ok` was never `yes`, route 1 was closed
|
||||
by accident.
|
||||
|
||||
**The guard is scoped to this file, and that is a gap rather than a decision** — `docker-build.yml`'s
|
||||
`test` and `migrations` jobs are also required contexts, and there a dropped step is **fail-OPEN**: the
|
||||
required check goes green having done no work, which is strictly worse than an absent status (compare
|
||||
#684). Tracked as ersatztv#756.
|
||||
|
||||
It lives in its **own workflow file** on purpose: `pr-checks.yml` sets `cancel-in-progress: true`,
|
||||
and a cancelled run there would leave an exempt PR with no status and no further push to
|
||||
|
||||
Reference in New Issue
Block a user