Round 5: raw HTML is the third code-block form. `<pre>`, `<code>` and HTML comments all render their contents literally, so a verdict inside one is an example, not an approval: <pre> / <code> / <!-- ... --> containing a verdict -> positive Now stripped, tracked as a marker count rather than parsed — the direction of error is to strip MORE, which can only ever withhold approval. Mutation-verified: removing the stripper fails all five cases. AND THE HARDENING STOPS HERE, deliberately. The record now says so, because otherwise the next session re-derives it: this is a best-effort heuristic, not a markdown parser. It covers the three code-block forms markdown has (fenced; indented, via the column-0 rule; raw HTML) and is not proof against every way to render text as non-prose. Stopping is safe because the comment is NOT the load-bearing gate. Since #622 the authoritative signal is the `review-verdict/h10` commit status, written only by post-review-verdict.sh from explicit arguments — a comment cannot forge it. This classifier is condition (c) of the PreToolUse hook: defense in depth on an agent's merge call. A residual false-open means the hook does not object; it does not mean a merge happens. Five rounds found five code-block forms, four of them introduced while fixing the previous round. The generalisable rule, now in the record: when a heuristic keeps failing at the edges, check whether it is actually the thing enforcing the invariant before spending another round on it. Also measured, against the real corpus: a "verdict must be the first line" rule would have killed every code-block form at once, but 14 of 18 verdict markers ever posted in this repo are NOT on the first line — so it was rejected as a retroactive break, not deferred. 178 tests. refs #629 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Decisions-Edit: yes
11 KiB
key, title, status, since, supersedes, superseded-by, rule, signals, mechanics
| key | title | status | since | supersedes | superseded-by | rule | signals | mechanics |
|---|---|---|---|---|---|---|---|---|
| release.review-verdict-gate | 2026-07-12 — Review-verdict merge-gate: latest commit must be reviewed (#303 H10) | active | 2026-07-12 | none | none | A PR may not merge until a `Review-verdict: <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> @ <head-sha>` comment references the PR's current head sha (short-sha prefix match against the verdict's OWN `@ <sha>` field, marker at COLUMN 0 (no indent, so indented code blocks cannot self-approve), whole-word verdict token, fenced code blocks stripped with markdown fence-length semantics, negative wins over positive on the same head); folds into the H6 merge-consent hook as condition (c). The grammar lives in ONE tested place, `scripts/check-review-verdict.sh` — #629 found three false-opens that survived because it was implemented inline and untested while this record described stricter behaviour than the code had. | review-verdict, head-sha match, stale-review prevention, verdict false-open, MERGEABLE-LATER, fenced code block verdict, sha from a URL, unknown verdict token · paths: `scripts/check-review-verdict.sh`, `.claude/hooks/pretooluse-merge-consent.sh`, `.claude/settings.json` · issues: #303 (H10), #242, #629 | `scripts/check-review-verdict.sh` (the grammar, + `scripts/tests/test_check_review_verdict.py`); `pretooluse-merge-consent.sh` (maps a class onto allow/deny/ask); `scripts/post-review-verdict.sh`; CLAUDE.md → Task Completion Protocol (H10 convention) |
A PR may not merge until a Review-verdict: comment on it references the PR's CURRENT head sha —
so the latest commit is proven-reviewed, not a stale earlier diff. This mechanizes the ersatztv#242
lesson ("re-review the fix commit, not just the initial PR diff": a review of an earlier revision does
not license merging a head that carries un-reviewed follow-up commits). It folds into the existing H6
pretooluse-merge-consent.sh as condition (c), reusing its PR fetch, docs-only exemption, and
Gitea-auth-from-env (no second hook → no detection drift, per the #303 methodology review).
Convention: after reviewing a PR (or its latest fix commit), post a PR comment (issue-style, not a
Gitea formal-review body — the gate reads issues/{pr}/comments) whose line starts with the marker:
Review-verdict: <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> @ <head-sha> (short ≥7-char or full sha).
The gate counts a line as a verdict only when the marker is at line-start (after optional indent) —
a comment that merely quotes the template mid-sentence (an instruction "please post: Review-verdict:
MERGEABLE @ …", or the gate's own suggestion text echoed back) does not self-approve the merge
(adversarial re-review false-open, folded pre-merge). It then classifies each verdict line by the sha
in its @ <sha> field, matched to the head by git short-sha prefix semantics (head begins with
the token, token ≥7 chars) — NOT a loose substring test, so an older sha that merely contains the head
prefix does not count.
#629 — three of the protections described here were asserted but not implemented. The grammar lived inline in the hook with no tests, and each of these graded as a positive verdict until #629 (every one reproduced, then fixed, then mutation-verified):
- the verdict token was prefix-matched, so
MERGEABLE-LATER,APPROVED-PENDING-QAandLGTMishall read as positive. A token is now matched as a whole word, and one in neither vocabulary is classifiedunknown— never positive, and never guessed into a block either; - a verdict inside a fenced code block counted, because the line-start anchor is satisfied inside
a fence. So documentation showing the convention was itself a verdict. Fenced blocks are now
stripped, with fence state reset per comment body (blockquotes never needed handling — a
>prefix already fails the anchor); - "the head prefix appearing in an unrelated URL on the line does not count" — this paragraph's
own earlier claim — was false. The implementation took the first
@<hex>anywhere on the line, soReview-verdict: MERGEABLE [x](https://e/@0123456)was graded against the link. The sha is now read from the verdict's own@ <sha>field, the one immediately following the token, which also makes multi-@lines unambiguous.
A cross-family review of that first fix found three more, all reproduced before fixing — worth recording because each is the same fix done half-way:
- fences were stripped for
```only, but markdown also accepts~~~; - the sha field matched
{7,40}with no right boundary, so an over-long or malformed token was silently truncated into a passing one:@ <40-hex-head>fand@ <40-hex-head>ZZZboth graded as a verdict for head. The hex run is now matched whole, must end at a non-alphanumeric boundary, and its length is validated separately — an out-of-range token is rejected, never trimmed to fit; - fence stripping toggled on any line of three-or-more markers. Markdown closes a fence only
with N-or-more of the same marker it was opened with, so a ```` block legitimately contains a
- comment bodies were joined with a literal
\x01BODY-BOUNDARY\x01line so fence state could reset per comment. An in-band delimiter is forgeable by whoever writes the data — and here that is anyone who can comment on the PR: a body containing that line reset the fence mid-comment and exposed a verdict still inside an unclosed fence. Bodies are now carried out-of-band (one JSON-encoded string per line), which removes the class rather than escaping the sentinel.
A fourth round found the last one, and it is why the grammar was narrowed rather than patched
again: markdown has a second code-block form — indented blocks (4 spaces or a tab) — which the fence
stripper does not cover, so a pasted indented example still self-approved. Rather than add a second
stripper (the same instance-not-class move that produced the previous three rounds), the marker must
now sit at COLUMN 0. That removes every indentation-based ambiguity at once. The cost is that a
verdict indented under a list item is ignored and classifies absent — which asks a human, the safe
direction for a gate. Fence detection keeps its leading-whitespace tolerance, because stripping more
is always safe.
Four review rounds, four sets of real findings, and rounds 2–4 each found a bug introduced while fixing the round before — every one the same fix applied half-way (one fence marker but not the other; a bounded left side but not the right; state reset per comment but via a delimiter the writer controls). For attacker-writable text, budget several rounds and prefer eliminating a class over enumerating its members.
A fifth round found raw HTML — <pre>, <code>, HTML comments — the third code-block form. It is
stripped too, and hardening stopped there, deliberately.
Scope limit, stated so nobody re-derives it: the comment scanner is a best-effort heuristic, not a markdown parser. It handles the three code-block forms markdown actually has (fenced, indented via the column-0 rule, raw HTML) and errs toward stripping more, because every error in that direction can only withhold approval. It is not proof against every conceivable way to render text as non-prose, and chasing that was demonstrably not converging: five review rounds, five code-block forms, four of them introduced while fixing the previous one.
Stopping is safe because the comment is not the load-bearing gate. Since #622 the authoritative
signal is the review-verdict/h10 commit status, written only by scripts/post-review-verdict.sh
from explicit arguments — a comment cannot forge it, whatever it contains. The comment classifier
is condition (c) of the PreToolUse hook, i.e. defense in depth on an agent's merge call. A residual
false-open there means the hook does not object; it does not mean a merge happens.
The rule that generalises: when a heuristic keeps failing at the edges, check whether it is actually the thing enforcing the invariant before spending another round on it.
The lesson is about where a grammar lives, not about any one regex: this record described the intended
behaviour accurately, the code did something looser, and nothing compared them. The grammar now lives
in scripts/check-review-verdict.sh — one implementation, called by the hook and by
post-review-verdict.sh's cross-check, covered by scripts/tests/test_check_review_verdict.py.
Two of those fixes also broke previously-green tests, which is the useful part: an over-long token
became no-sha where a fixture expected stale (the fixture was 45 hex chars, so it had been
testing the length guard while claiming to test the prefix rule), and jq -e exits 4 on no
output, so an empty comment list started reading as a malformed payload — turning "no comments yet"
into an input error on a gate whose callers fail closed.
The classifications:
- a MERGEABLE/APPROVED/LGTM verdict whose
@ <sha>is the current head → allow; - a negative verdict (BLOCKED/NOT-MERGEABLE) on the head → deny, and it wins over a positive
one on the same head (a later BLOCKED retracts an earlier MERGEABLE; to retract, re-review head and
post BLOCKED @ head). Staleness is symmetric on purpose: a negative for an older commit is stale
exactly like a positive for an older commit, and does NOT override a fresh head-positive — otherwise a
pre-fix
BLOCKED @ oldshawould block forever even after the fix changes the sha and earns a freshMERGEABLE @ head(the normal flow). So a genuine block must reference head, per the convention; - verdict comment(s) exist but reference only older commits → deny — the stale-review case #242 targets;
- a verdict line whose token is in neither vocabulary (
MERGEABLE-LATER,SHIP-IT, …) → ask (#629). Deliberately not read as approval, and deliberately not read as a block either — an unrecognized token means the reviewer's intent is unknown, so it goes to a human; - a
Review-verdict:marker with no@ <sha>in its own field → ask (a lazy/quoted marker; not mislabelled as stale); - no
Review-verdict:comment at all → ask (graceful adoption, mirrors H6's "no Done-when → ask": surface, don't hard-block a PR that hasn't adopted the convention yet); - comments unfetchable / head sha unresolvable → ask.
Scope: the Claude PreToolUse gate on the Gitea merge tool only. A direct git push origin main has no
PR comments to check, so the .husky/pre-push backstop is not extended for H10 (the merge tool is the
real merge path; docs-only PRs remain exempt via H6's file-set exemption). Rationale, as with the whole
Wave-1/2/3 hook set: make the process rule a derivation/hook, not prose to remember (#303).