From 628ae46f7f6d0ec29fcd1d33526d0a730bba2d88 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sat, 25 Jul 2026 21:24:09 +0200 Subject: [PATCH 1/6] fix(622): bind H10 merge consent to the reviewed sha via a required commit status MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `pretooluse-merge-consent.sh` proves all three consent conditions at the moment the merge tool is called. With `merge_when_checks_succeed=true`, condition (a) is delegated to Gitea, which then merges whatever head is green at ITS merge time — while (b) Done-when and (c) the head-referencing verdict were proven against the head at SCHEDULING time. Every commit pushed in between merges unreviewed. The gate is not bypassed; it is satisfied against a snapshot that stops being true. Demonstrated as a controlled A/B rather than inferred, with a CI check left pending so Gitea waits as it really does: without a required verdict context: unreviewed commit B MERGED with it: same sequence REFUSED after reviewing B and posting it: merges — blocked, not deadlocked NOTE the anecdote in #622 is wrong and is corrected in the docs rather than repeated: PR #619 does carry `Review-verdict: MERGEABLE @ 02c82b35`, posted six seconds before the merge, explicitly re-reviewing the follow-up commits. #622 was filed off a lagging API read. The hole is real regardless, and structural — nothing FORCED that re-review inside the window Gitea would have merged in. This turns a property that held by diligence into one that holds by construction. The fix is the sha, not a smarter check. A Gitea commit status belongs to exactly one commit, so a status written for a parent cannot be inherited by a child that did not exist yet. `review-verdict/h10` becomes a REQUIRED status check on main: push a new head and the context is simply absent, which Gitea reads as not-passing (verified against 1.25.4: missing AND pending both block, and auto-merge re-checks the current head). It also covers merge paths the hook never sees — Gitea UI, raw API, another agent's session. - scripts/post-review-verdict.sh writes the comment and the status together so they cannot drift, and re-reads head after commenting: if a commit landed mid-flight it writes NO status and exits non-zero rather than retargeting the verdict at a commit nobody read. - .gitea/workflows/review-verdict.yml auto-passes the two exempt classes that would otherwise deadlock — Renovate-authored (platformAutomerge) and docs-only — and marks everything else pending. Exemptions are void when the PR touches .claude/, .gitea/, .husky/, scripts/ or docker/ci/. - The hook refuses to SCHEDULE an auto-merge unless that status is green on head, and no longer claims "CI green" on the mwcs path, where it never read CI. Two silent false negatives in the exemption path, both found by verifying rather than reasoning, both fixed at BOTH call sites (workflow and the hook's pre-existing docs-only carve-out): - The files endpoint caps at 50 rows and IGNORES a larger `limit` — PR #619 has 194 changed files and `?limit=100` returns 50. A single page saw ZERO protected paths there where the full enumeration finds ten. Both now page to exhaustion and withhold the exemption if they cannot complete. - A rename is ONE row whose `filename` is the destination, the source only in `previous_filename`. Verified live: `.gitea/workflows/renovate.yml` -> `docs/innocuous-note.md` presented as docs-only with no protected path visible. Both now read BOTH sides. Limits are documented, not papered over: base changes leave the head sha (and so the verdict) unchanged, and a PR editing the workflow is judged by its own edited copy — so PROTECTED is a guardrail against accident, not a tamper-proof control. fixes #622 --- .claude/hooks/pretooluse-merge-consent.sh | 64 ++++- .gitea/workflows/review-verdict.yml | 163 +++++++++++ CLAUDE.md | 5 +- docs/ci-cd.md | 63 ++++- docs/decisions/README.md | 1 + .../records/release/verdict-status-check.md | 115 ++++++++ docs/handoffs/chicorytv-issue-queue.md | 5 + scripts/post-review-verdict.sh | 128 +++++++++ scripts/tests/test_post_review_verdict.py | 254 ++++++++++++++++++ 9 files changed, 789 insertions(+), 9 deletions(-) create mode 100644 .gitea/workflows/review-verdict.yml create mode 100644 docs/decisions/records/release/verdict-status-check.md create mode 100755 scripts/post-review-verdict.sh create mode 100644 scripts/tests/test_post_review_verdict.py diff --git a/.claude/hooks/pretooluse-merge-consent.sh b/.claude/hooks/pretooluse-merge-consent.sh index c5b1e5592..cb48b0d3f 100755 --- a/.claude/hooks/pretooluse-merge-consent.sh +++ b/.claude/hooks/pretooluse-merge-consent.sh @@ -7,6 +7,14 @@ # (c) a review-verdict comment on the PR references the CURRENT head sha (H10) — proving the # LATEST commit was reviewed, not a stale earlier diff (the ersatztv#242 failure mode: # "re-review the fix commit, not just the initial PR diff"). +# +# EVERY ONE OF THOSE IS A SNAPSHOT, taken when the merge tool is called. That is sound for an +# immediate merge and UNSOUND for a scheduled one: with merge_when_checks_succeed, Gitea merges +# later, against whatever head is green then (ersatztv#622). So the sha-bound half of H10 is +# enforced by the SERVER, not here — `review-verdict/h10` is a required status check on `main`, +# written per-sha by scripts/post-review-verdict.sh, and a new commit cannot inherit it. This hook +# additionally refuses to SCHEDULE an auto-merge unless that status is already green on head, so the +# two mechanisms agree at the only moment they can both observe the same commit. # The "## Done-when" issue-body checklist is the convention (docs/decisions.md, CLAUDE.md Task # Completion Protocol). One box is "adversarial review passed"; the others are per-issue. # The H10 review-verdict convention: after reviewing a PR (or its latest fix commit), post a PR @@ -77,8 +85,26 @@ sha=$(printf '%s' "$prjson" | jq -r '.head.sha // ""' 2>/dev/null || true) body=$(printf '%s' "$prjson" | jq -r '.body // ""' 2>/dev/null || true) # --- Docs-only exemption: if every changed file is docs/process, skip the gate. --- -files=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=100" | jq -r '.[].filename // empty' 2>/dev/null || true) -if [ -n "$files" ] && ! printf '%s\n' "$files" | grep -qvE '^(docs/|\.claude/|\.husky/|\.gitea/|.*\.md$)'; then +# The file list must be enumerated EXHAUSTIVELY or the exemption is unsafe. Gitea caps this +# endpoint at 50 rows per page and silently ignores a larger `limit` (verified: PR #619 has 194 +# changed files and `?limit=100` returns exactly 50), so the previous single-page read could see 50 +# docs files, miss the code in positions 51+, and exempt a PR that is not remotely docs-only. +# Page until a short page proves the end; anything else leaves `files_complete=no`, which withholds +# the exemption and falls through to the full gate (ersatztv#622). +files=""; files_complete=no; page=1 +while [ "$page" -le 40 ]; do + raw=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=50&page=$page") + # BOTH sides of a rename: Gitea reports a `git mv` as ONE row whose `filename` is the DESTINATION, + # with the source in `previous_filename`. Reading only `filename` would let a PR move code into + # docs/ and claim the docs-only exemption. Page size is measured in ROWS, not paths. + n=$(printf '%s' "$raw" | jq -r '.[].filename // empty' 2>/dev/null | grep -c . || true) + chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)' 2>/dev/null || true) + [ -n "$chunk" ] && files=$(printf '%s\n%s' "$files" "$chunk") + if [ "${n:-0}" -lt 50 ]; then files_complete=yes; break; fi + page=$((page + 1)) +done +files=$(printf '%s\n' "$files" | grep -v '^$' || true) +if [ "$files_complete" = yes ] && [ -n "$files" ] && ! printf '%s\n' "$files" | grep -qvE '^(docs/|\.claude/|\.husky/|\.gitea/|.*\.md$)'; then # Docs/process-only PR: the Done-when + review-verdict gate doesn't apply — but this exemption is a # file-TYPE bypass, NOT the a+b+c "provably reviewed & ready" proof, so it does NOT auto-grant. It # passes through to normal permissioning (one prompt). This deliberately keeps a human in the loop for @@ -119,6 +145,32 @@ if [ "$mwcs" != "true" ]; then "") decide ask "H6 merge gate: could not read CI status for PR #$pr ($sha). Verify CI is green before merging." ;; *) decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success'. Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging." ;; esac +else + # --- SCHEDULED auto-merge: everything this hook proves is a SNAPSHOT (ersatztv#622). ---------- + # With merge_when_checks_succeed, Gitea performs the merge later, against whatever head is green + # at THAT moment — but (b) and (c) below are evaluated against the head that exists right now. + # Any commit pushed in between would merge with no verdict covering it. Demonstrated as a + # controlled A/B (#622): with a slow CI check pending so Gitea waits, an unreviewed commit pushed + # after scheduling MERGED without the required verdict context and was REFUSED with it. + # + # The durable fix is server-side and lives outside this hook: `review-verdict/h10` is a REQUIRED + # status check on `main`, and a commit status belongs to exactly ONE sha, so a later commit cannot + # inherit it and Gitea's own gate refuses to merge until that head is re-reviewed. + # + # What we add HERE is the matching precondition at SCHEDULING time: refuse to arm an auto-merge + # unless the sha-bound status already exists on this head. Checking the comment alone (condition + # (c) below) is not enough for this path — the comment is what a human reads, the status is what + # the server enforces, and only the latter survives a new push. Deny rather than ask: the remedy + # is a single documented command, so there is nothing here for a human to adjudicate. + [ -n "$sha" ] || decide ask "H6 merge gate: could not resolve PR #$pr head sha to check the review-verdict status. Verify the review covered the latest commit before scheduling an auto-merge." + vstate=$(gq "repos/$owner/$repo/statuses/$sha?limit=100" \ + | jq -r '[.[] | select(.context == "review-verdict/h10")] | first | .status // ""' 2>/dev/null || true) + case "$vstate" in + success) : ;; + "") decide deny "H6/H10 merge gate: BLOCKED — PR #$pr has no 'review-verdict/h10' commit status on head ${sha:0:7}, so scheduling an auto-merge would freeze consent at a head Gitea may not be the one to merge (ersatztv#622). Review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;; + pending) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is still pending on PR #$pr head ${sha:0:7} (no verdict posted for this commit yet). Review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;; + *) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is '$vstate' on PR #$pr head ${sha:0:7}. Resolve the findings, then run: scripts/post-review-verdict.sh $pr MERGEABLE" ;; + esac fi # --- (c) Review-verdict freshness (ersatztv#303 H10): a review-verdict comment must reference the @@ -170,7 +222,13 @@ if [ "$head_neg" = 1 ]; then decide deny "H10 merge gate: BLOCKED — a review verdict for the current head ($short) is negative (BLOCKED/NOT-MERGEABLE). Resolve the findings and post a fresh 'Review-verdict: MERGEABLE @ $short' before merging PR #$pr." fi if [ "$head_pos" = 1 ]; then - # (a) CI green + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant. + # (a) CI + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant. + # The reason string must not claim more than was actually checked: on the merge_when_checks_succeed + # path this hook never read the CI status at all (it is delegated to Gitea), so saying "CI green" + # there was a plain falsehood in the one message a human reads to decide whether to trust the gate. + if [ "$mwcs" = "true" ]; then + decide grant "H6/H10 merge gate: satisfied — all Done-when boxes ticked, and both a positive Review-verdict comment and the 'review-verdict/h10' status cover the current head ($short). CI is gated by Gitea (merge_when_checks_succeed), and because the verdict status is bound to this sha, a commit pushed before Gitea merges will clear it and block the merge (ersatztv#622). Auto-granted." + fi decide grant "H6/H10 merge gate: satisfied — CI green, all Done-when boxes ticked, and a positive Review-verdict references the current head ($short). Auto-granted (no separate confirmation needed)." fi if [ "$stale" = 1 ]; then diff --git a/.gitea/workflows/review-verdict.yml b/.gitea/workflows/review-verdict.yml new file mode 100644 index 000000000..e3080f8e1 --- /dev/null +++ b/.gitea/workflows/review-verdict.yml @@ -0,0 +1,163 @@ +name: Review verdict + +# ersatztv#622 — server-side H10 enforcement. +# +# THE INVARIANT. `review-verdict/h10` is a REQUIRED status check on `main`, and a Gitea commit +# status belongs to exactly ONE sha. So a commit that did not exist when a verdict was written can +# never inherit that verdict: push a new head and the required context is simply absent, which +# Gitea's merge-requirement check treats as not-passing. `merge_when_checks_succeed` therefore +# refuses to fire until someone re-reviews THAT head. This is what closes the ersatztv#622 hole, +# where the PreToolUse hook proved conditions (b) and (c) against the head at SCHEDULING time and +# Gitea then merged whatever head happened to be green minutes later. +# +# WHAT THIS WORKFLOW DOES — and, importantly, does NOT do. It does NOT decide whether code was +# reviewed; only a human/agent review does that, via `scripts/post-review-verdict.sh`, which writes +# the `review-verdict/h10` status directly. This workflow only handles the two EXEMPT classes that +# would otherwise deadlock, and marks everything else `pending` so the PR shows an explicit, +# actionable blocking reason instead of a silently-missing check: +# +# 1. Bot-authored PRs (Renovate). Renovate uses `platformAutomerge: true` — i.e. Gitea's OWN +# auto-merge — to land patch bumps unattended. A required verdict context with no exemption +# would stall every dependency PR forever waiting on a human verdict. +# 2. Docs-only PRs, mirroring the merge-consent hook's existing docs-only carve-out. +# +# BOTH exemptions are void when the PR touches a PROTECTED path (see PROTECTED below): the gate, +# the CI definition, the git hooks, the scripts they call, and the CI toolchain image. A PR that +# weakens the merge gate must never be able to exempt itself from the merge gate — that is the one +# self-referential failure worth spending an explicit rule on. Note this also (deliberately) means +# Renovate's `docker/ci/Dockerfile` base bumps need a real verdict; those already require the +# manual publish-then-pin two-step (docs/ci-cd.md -> "CI toolchain image"), so unattended merge was +# never correct for them anyway. +# +# WHY ITS OWN FILE, not a job in pr-checks.yml: that workflow sets `cancel-in-progress: true`, so a +# superseding push cancels its runs. A cancelled run here would leave an EXEMPT PR with no success +# status and no further pushes to re-trigger it — Renovate would stall silently. This workflow +# therefore takes no cancelling concurrency group. +# +# This job's OWN status context ("Review verdict / Set review-verdict status (pull_request)") is +# NOT the required check and is not what gates merges — `review-verdict/h10`, the status it POSTS, +# is. Keeping them distinct is deliberate: a workflow cannot be allowed to satisfy the gate merely +# by running successfully. + +on: + pull_request: + types: [opened, reopened, synchronize, ready_for_review] + +defaults: + run: + shell: bash + +jobs: + set-verdict-status: + name: Set review-verdict status + runs-on: small # a few API calls; keep it off the build runners + steps: + - name: Classify the PR and post the review-verdict status + env: + GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} + BASE_URL: ${{ github.server_url }}/api/v1 + REPO: ${{ github.repository }} + PR: ${{ github.event.pull_request.number }} + SHA: ${{ github.event.pull_request.head.sha }} + AUTHOR: ${{ github.event.pull_request.user.login }} + PR_URL: ${{ github.event.pull_request.html_url }} + run: | + set -euo pipefail + + CONTEXT="review-verdict/h10" + # Accounts whose PRs may merge without a human verdict. Renovate only — keep this list + # minimal and explicit; every entry is an account that can land code unreviewed. + BOTS="renovate" + # Paths where NEITHER exemption applies, because a change here can alter the gate itself, + # what CI runs, or what the hooks enforce. + PROTECTED='^(\.claude/|\.gitea/|\.husky/|scripts/|docker/ci/)' + # Docs-only: prose and decision records. Deliberately narrower than the hook's pattern, + # which also lets .claude/.gitea/.husky through — that carve-out is safe there only + # because it falls through to a HUMAN PROMPT, whereas here it would post a green status + # with nobody in the loop. + DOCS_ONLY='^(docs/|[^/]*\.md$)' + + if [ -z "${GITEA_TOKEN:-}" ]; then + echo "::error::No GITEA_TOKEN available, so the ${CONTEXT} status cannot be written. An exempt (bot/docs-only) PR will stall until this is fixed; a normal PR is unaffected — post its verdict with scripts/post-review-verdict.sh." + exit 1 + fi + + gh() { curl -sf -H "Authorization: token $GITEA_TOKEN" "$@"; } + + # --- Already decided for THIS sha? Never overwrite a real verdict. ------------------- + # A human/agent verdict for this exact head may already exist (the reviewer ran the + # script before this workflow finished, or a rerun). Re-posting `pending` over it would + # un-approve a reviewed head and stall the PR. + existing=$(gh "$BASE_URL/repos/$REPO/statuses/$SHA?limit=100" \ + | jq -r --arg c "$CONTEXT" '[.[] | select(.context == $c)] | first | .status // ""' 2>/dev/null || true) + if [ "$existing" = "success" ] || [ "$existing" = "failure" ]; then + echo "${CONTEXT} is already '${existing}' on ${SHA:0:7} — leaving the existing verdict alone." + exit 0 + fi + + # --- Changed files: PAGE to exhaustion, and fail CLOSED if we cannot. ---------------- + # Gitea caps this endpoint at 50 rows per page and SILENTLY IGNORES a larger `limit` + # (verified: PR #619 has 194 changed files and `?limit=100` returns exactly 50). A + # single-page read is therefore a silent false negative: a protected path sitting at + # position 51+ would simply not be seen, and a bot-authored PR that edits the gate could + # exempt itself from the gate. Page until a short page proves the end. + PAGE_SIZE=50 + MAX_PAGES=40 # 2000 files; beyond this we refuse rather than guess + files="" + page=1 + complete=no + while [ "$page" -le "$MAX_PAGES" ]; do + raw=$(gh "$BASE_URL/repos/$REPO/pulls/$PR/files?limit=${PAGE_SIZE}&page=${page}" || true) + # BOTH sides of a rename. Gitea reports a `git mv` as ONE row with status=renamed, + # `filename` = the DESTINATION and `previous_filename` = the source. Reading only + # `filename` therefore lets a PR move a protected file INTO docs/ and be classified + # docs-only: verified live — `.gitea/workflows/renovate.yml` -> `docs/innocuous-note.md` + # showed no protected path at all. Counting rows (not paths) keeps the page-size test + # honest, since one renamed row contributes two paths. + n=$(printf '%s' "$raw" | jq -r '.[].filename // empty' 2>/dev/null | grep -c . || true) + chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)' 2>/dev/null || true) + [ -n "$chunk" ] && files=$(printf '%s\n%s' "$files" "$chunk") + if [ "${n:-0}" -lt "$PAGE_SIZE" ]; then complete=yes; break; fi + page=$((page + 1)) + done + files=$(printf '%s\n' "$files" | grep -v '^$' || true) + count=$(printf '%s\n' "$files" | grep -c . || true) + echo "Changed files (${count}, complete=${complete}, pages=${page}):" + printf '%s\n' "$files" | sed 's/^/ /' + + exempt=no + reason="" + if [ "$complete" != yes ]; then + reason="could not enumerate the changed files exhaustively (stopped at ${count}) — no exemption" + elif [ "${count:-0}" -eq 0 ]; then + reason="no changed files could be read from the API — no exemption" + elif printf '%s\n' "$files" | grep -qE "$PROTECTED"; then + reason="touches a protected path (gate/CI/hooks/scripts/ci-image) — exemptions do not apply" + elif printf '%s\n' "$BOTS" | tr ' ' '\n' | grep -qxF "$AUTHOR"; then + exempt=yes + reason="authored by the '$AUTHOR' bot account and touches no protected path" + elif ! printf '%s\n' "$files" | grep -qvE "$DOCS_ONLY"; then + exempt=yes + reason="docs-only change (no code, no protected path)" + else + reason="awaiting an H10 review verdict for head ${SHA:0:7}" + fi + + if [ "$exempt" = yes ]; then + state=success + desc="Exempt: $reason" + else + state=pending + desc="Awaiting review verdict for ${SHA:0:7}" + fi + echo "Decision: state=${state} — ${reason}" + + payload=$(jq -n --arg s "$state" --arg c "$CONTEXT" --arg d "$desc" --arg u "$PR_URL" \ + '{state:$s, context:$c, description:$d, target_url:$u}') + gh -X POST -H 'Content-Type: application/json' -d "$payload" \ + "$BASE_URL/repos/$REPO/statuses/$SHA" >/dev/null + echo "Posted ${CONTEXT}=${state} on ${SHA:0:7}." + + if [ "$state" = "pending" ]; then + echo "::notice::This PR needs an H10 review verdict for head ${SHA:0:7} before it can merge. After reviewing, run: scripts/post-review-verdict.sh ${PR} MERGEABLE" + fi diff --git a/CLAUDE.md b/CLAUDE.md index f691c13f5..e718b9c27 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -52,7 +52,7 @@ docker build -f docker/Dockerfile -t ersatztv:dev . - Test with **NUnit** + Shouldly + NSubstitute (the existing `*.Tests` projects); xUnit is **not** used here - **Dependencies use Central Package Management**: versions live in the repo-root `Directory.Packages.props`; csproj reference packages by name only. Add/upgrade by editing the central `` — never put `Version=` back on a `` (trips `NU1008`). See `docs/ci-cd.md` → Dependency management. - **DB migrations target BOTH providers**: a `TvContext` model change needs a migration in `ErsatzTV.Infrastructure.Sqlite` **and** `ErsatzTV.Infrastructure.MySql` — run `scripts/add-migration.sh ` (does both). CI's `migrations` job enforces model-drift + apply-to-fresh-DB per provider. See `docs/ci-cd.md` → Migration integrity. -- **Renovate** is live (`.gitea/workflows/renovate.yml`, weekly + `workflow_dispatch`): opens dependency-update + OSV vuln-fix PRs and a Dependency Dashboard issue; patch bumps to test/dev-only packages auto-merge once `Build & test` passes, the rest are manual. Cross-repo rollout: server-management#484. See `docs/ci-cd.md` → Dependency management. +- **Renovate** is live (`.gitea/workflows/renovate.yml`, weekly + `workflow_dispatch`): opens dependency-update + OSV vuln-fix PRs and a Dependency Dashboard issue; patch bumps to test/dev-only packages auto-merge once `Build & test` passes (their `review-verdict/h10` required check is auto-passed as a bot PR — unless they touch `.claude/`/`.gitea/`/`.husky/`/`scripts/`/`docker/ci/`, which need a real verdict), the rest are manual. Cross-repo rollout: server-management#484. See `docs/ci-cd.md` → Dependency management. - **Versioning**: release tags are `vYY..` (year · sequential release-within-year · patch) — inherited from upstream, **not** year.month. `v26.3.1` = our infra rebuild of upstream 26.3.0 (no app changes); `v26.4.0` is reserved for the first release with app changes. Never `[skip ci]` a commit you'll tag (it suppresses the release build). Full policy: `docs/ci-cd.md` → Versioning & releases. - Backlog tracked via [Gitea Issues](http://192.168.1.95:3000/timothy/ersatztv/issues) @@ -61,7 +61,8 @@ docker build -f docker/Dockerfile -t ersatztv:dev . Every task that closes a Gitea issue MUST complete ALL of these before it is considered done. Use `/done ` to run through this automatically. **Merge-consent is derived from state, not asserted (`## Done-when` convention — ersatztv#303 H6 + H10).** Any issue whose PR will merge to `main` should carry a `## Done-when` section in its **issue body** — a checklist of completion criteria (always include an "adversarial review passed" box; add per-issue criteria like tests-green, docs-updated, live-E2E). Two hooks derive merge-consent from it so a premature merge is blocked *by construction*, not by memory: -- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), post a PR comment with a line `Review-verdict: @ ` — this proves the *latest* commit was reviewed, not a stale earlier diff (ersatztv#242). +- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), run **`scripts/post-review-verdict.sh [note]`** — it posts both the `Review-verdict: … @ ` comment and the sha-bound `review-verdict/h10` commit status, proving the *latest* commit was reviewed rather than a stale earlier diff (ersatztv#242). Do not hand-write the comment: the **status** is the required check branch protection enforces, and a comment alone leaves it absent. +- **The gate is enforced server-side, per sha (ersatztv#622).** `review-verdict/h10` is a required status check on `main`. Because a commit status belongs to one sha, a commit pushed *after* an auto-merge is scheduled clears it and blocks the merge — closing the hole where `merge_when_checks_succeed` froze consent at scheduling time and Gitea later merged an unreviewed head. Renovate-authored and docs-only PRs are auto-passed by `.gitea/workflows/review-verdict.yml`, **except** when they touch `.claude/`, `.gitea/`, `.husky/`, `scripts/` or `docker/ci/`. See `docs/ci-cd.md` → Review-verdict gate. - `.husky/pre-push` → `prepush-donewhen.sh` — a fail-open backstop that blocks a direct `git push origin main` whose commits `fix #N` an issue with unticked boxes. Both need Gitea read creds in the env to enforce (**`ETV_GITEA_BASICAUTH=user:pass`** or `ETV_GITEA_TOKEN`; `ETV_GITEA_URL` overrides the base). Without them the merge hook asks and the push backstop is a no-op — the gate degrades to today's manual confirmation, never a silent pass. Docs-only PRs/pushes are exempt. diff --git a/docs/ci-cd.md b/docs/ci-cd.md index e1718a6e4..97ee42b21 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -620,10 +620,65 @@ grep of `docker-build.yml` still validates the five pin-bearing jobs (`test`/`migrations`/`functional-e2e`/`api-docs`/`format`) that remain there. `api-docs` and `format` stay in `docker-build.yml` because they carry the shared-image `container:` + pin and run on the healthy `ubuntu-latest` lane (where they skipped correctly). None of the three moved jobs is -a **required** check — branch protection requires only `Build & test (.NET)` and `EF migration -integrity` — so relocating them (their status-context prefix changes from `Build ErsatzTV Image / -…` to `PR Gates / …`) does not affect merges. The file declares `defaults: run: shell: bash` -because `ci-image-pin` uses `mapfile`/`set -o pipefail`. +a **required** check — branch protection requires `Build & test (.NET)`, `EF migration integrity` +and `review-verdict/h10` (next section) — so relocating them (their status-context prefix changes +from `Build ErsatzTV Image / …` to `PR Gates / …`) does not affect merges. The file declares +`defaults: run: shell: bash` because `ci-image-pin` uses `mapfile`/`set -o pipefail`. + +## Review-verdict gate (`review-verdict/h10`, required — `.gitea/workflows/review-verdict.yml`) + +**A required status check named `review-verdict/h10`, written per-sha, is what actually stops an +unreviewed commit from merging** (ersatztv#622). It is not produced by a job's success/failure; it +is a commit status that `scripts/post-review-verdict.sh` POSTs onto one specific sha. + +**The hole it closes.** `pretooluse-merge-consent.sh` proves its three consent conditions at the +moment the merge tool is called. Pass `merge_when_checks_succeed=true` and Gitea performs the merge +*later*, against whatever head is green then — while the Done-when and review-verdict checks were +proven against the head at **scheduling** time. Every commit pushed in between merges unreviewed. + +This was demonstrated as a controlled A/B rather than inferred (`ci/fake` stands in for a slow CI +check so Gitea waits, as it really does): review head A → post its verdict → schedule auto-merge → +push an unreviewed commit B → CI greens on B. **Without** the required verdict context, B merged. +**With** it, the same sequence was refused, and merged only once B itself was reviewed. + +Note the motivating anecdote in ersatztv#622 — "PR #619 merged 263 insertions with no verdict" — +is **wrong**: #619 does carry `Review-verdict: MERGEABLE @ 02c82b35`, posted six seconds before the +merge, explicitly re-reviewing the follow-up commits. It was filed from an API read that lagged. +The gap is real anyway, and structural: nothing *forced* that re-review inside the 45-minute window +where Gitea would have merged whatever went green. This turns diligence into construction. + +**Why a commit status fixes it and a smarter hook cannot.** A status belongs to exactly one sha, so +a new commit *cannot inherit it*: the required context is simply absent on the new head, Gitea's +merge-requirement check reads that as not-passing, and the scheduled auto-merge refuses to fire. +The invariant self-invalidates — nothing has to notice the push. It also covers merge paths the +hook never sees (Gitea UI, raw API, another agent's session). + +**Posting a verdict.** After reviewing a PR's *current* head: + +```bash +ETV_GITEA_BASICAUTH=user:pass scripts/post-review-verdict.sh MERGEABLE [note...] +``` + +That posts both the `Review-verdict: … @ ` comment (the human-readable artifact, and the +hook's condition (c)) and the `review-verdict/h10` status on the same sha. `BLOCKED` / +`NOT-MERGEABLE` post a `failure` status instead. The script re-reads the head after commenting: if +a commit landed mid-flight it writes **no** status and exits non-zero rather than retargeting your +verdict at a commit you never read. + +**Exemptions** are handled by `review-verdict.yml` on every `pull_request` event, which posts the +status as `success` for **Renovate-authored** PRs (it uses `platformAutomerge: true`, so a required +verdict with no exemption would stall every dependency bump) and for **docs-only** PRs, and as +`pending` for everything else so the block has a visible reason. Both exemptions are **void when the +PR touches `.claude/`, `.gitea/`, `.husky/`, `scripts/` or `docker/ci/`** — a PR that can weaken the +gate must not be able to exempt itself from the gate. That includes Renovate's `docker/ci` base +bumps, which already need the manual publish-then-pin two-step anyway. + +It lives in its **own workflow file** on purpose: `pr-checks.yml` sets `cancel-in-progress: true`, +and a cancelled run there would leave an exempt PR with no status and no further push to +re-trigger it. Its own job context (`Review verdict / Set review-verdict status`) is **not** the +required check — a workflow must not satisfy the gate merely by running successfully. + +Full rationale: `docs/decisions/records/release/verdict-status-check.md`. ## CI toolchain image (`docker/ci/Dockerfile`, `.gitea/workflows/ci-image.yml`) diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 67e890272..de0c0c119 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -111,6 +111,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `release.prepush-clean-worktree-guard` | A fail-open pre-push hook blocks a push when any file in the branch's diff vs `origin/main` also has uncommitted working-tree or index changes, since a stale-index commit (e.g. `git reset --soft` + `git add` over an edited-but-unstaged fix) can silently push, CI-test, and get reviewed a different tree than the one on disk. Scope is precise to pushed-diff files; escape hatch `ETV_ALLOW_DIRTY_PUSH=1`. | 2026-07-17 | [link](records/release/prepush-clean-worktree-guard.md) | | `release.promotion-floating-prod` | Prod tracks the floating `:prod` image reference; a tag build's immutable `:` image is scanned first, then promotion happens via a separate manual `DeployStack`, with daily auto-update only as a fallback — tag with enough runway before 03:00 to avoid an unscanned promotion. | 2026-07-13 | [link](records/release/promotion-floating-prod.md) | | `release.review-verdict-gate` | A PR may not merge until a `Review-verdict: @ ` comment references the PR's current head sha (short-sha prefix match, line-start marker only, negative wins over positive on the same head); folds into the H6 merge-consent hook as condition (c). | 2026-07-12 | [link](records/release/review-verdict-gate.md) | +| `release.verdict-status-check` | The H10 review verdict is written as a `review-verdict/h10` Gitea **commit status** on the exact reviewed sha by `scripts/post-review-verdict.sh`, and that context is a REQUIRED status check on `main`. Because a status belongs to one sha, a later commit cannot inherit it, so Gitea's own `merge_when_checks_succeed` refuses to merge a head no one reviewed. The PreToolUse hook additionally refuses to SCHEDULE an auto-merge unless that status is already green on head. A `pull_request` workflow auto-passes the two exempt classes (Renovate-authored, docs-only) unless the PR touches a protected path (`.claude/`, `.gitea/`, `.husky/`, `scripts/`, `docker/ci/`). This extends — does not supersede — `release.review-verdict-gate` (#303 H10), whose comment convention remains the human-readable artifact and the hook's condition (c). | 2026-07-25 | [link](records/release/verdict-status-check.md) | | `rulebuilder.relative-date-macros` | The visual rule builder's `inLast`/`notInLast` date operators compile to/parse from the pre-existing `CustomMultiFieldQueryParser` macros `released_inthelast`/`released_notinthelast` and `added_inthelast`/`added_notinthelast`, value form `" day\|week\|month\|year"`; there is no backend change. | 2026-07-23 | [link](records/rulebuilder/relative-date-macros.md) | | `scan.collections-scan-status` | `GET /api/v1/media-sources/collections-scan-status` reports a family-global (not per-source), boolean-only active-scan set read from `IEntityLocker`; the SPA reconciles authoritatively against it (with a grace-tick helper) instead of a fixed client-side timeout. | 2026-07-12 | [link](records/scan/collections-scan-status.md) | | `scan.getoraddfolder-db-lookup` | `ILibraryRepository.GetOrAddFolder` resolves the existing folder via a DB query on `(LibraryPathId, Path)`, not the caller's `LibraryPath.LibraryFolders` in-memory navigation, since that navigation is only eager-loaded on the local scan path and is null on remote (Jellyfin) callers. | 2026-07-20 | [link](records/scan/getoraddfolder-db-lookup.md) | diff --git a/docs/decisions/records/release/verdict-status-check.md b/docs/decisions/records/release/verdict-status-check.md new file mode 100644 index 000000000..b941bc717 --- /dev/null +++ b/docs/decisions/records/release/verdict-status-check.md @@ -0,0 +1,115 @@ +--- +key: release.verdict-status-check +title: '2026-07-25 — H10 is enforced server-side as a per-sha required status check, not only by the PreToolUse hook (#622)' +status: active +since: '2026-07-25' +supersedes: none +superseded-by: none +rule: 'The H10 review verdict is written as a `review-verdict/h10` Gitea **commit status** on the exact reviewed sha by `scripts/post-review-verdict.sh`, and that context is a REQUIRED status check on `main`. Because a status belongs to one sha, a later commit cannot inherit it, so Gitea''s own `merge_when_checks_succeed` refuses to merge a head no one reviewed. The PreToolUse hook additionally refuses to SCHEDULE an auto-merge unless that status is already green on head. A `pull_request` workflow auto-passes the two exempt classes (Renovate-authored, docs-only) unless the PR touches a protected path (`.claude/`, `.gitea/`, `.husky/`, `scripts/`, `docker/ci/`). This extends — does not supersede — `release.review-verdict-gate` (#303 H10), whose comment convention remains the human-readable artifact and the hook''s condition (c).' +signals: 'merge_when_checks_succeed freezes consent, auto-merge merges an unreviewed head, verdict bound to sha, review-verdict/h10 required check, post-review-verdict.sh, Renovate platformAutomerge exemption · paths: `scripts/post-review-verdict.sh`, `.gitea/workflows/review-verdict.yml`, `.claude/hooks/pretooluse-merge-consent.sh` · issues: #622, #303 (H6/H10), #242, #619' +mechanics: '`scripts/tests/test_post_review_verdict.py` (incl. a TOCTOU head-moved case and cross-checks against the hook''s own condition-(c) regexes); branch protection `status_check_contexts` on `main`' +--- + +`pretooluse-merge-consent.sh` evaluates all three consent conditions **at the moment the merge tool is +called**. That is sound for an immediate merge and unsound for a scheduled one. When the call passes +`merge_when_checks_succeed=true`, condition (a) is deliberately delegated to Gitea — but Gitea then +merges whatever head is green at *its* merge time, while (b) Done-when and (c) the head-referencing +verdict were proven against the head that existed at **scheduling** time. Any commit pushed in between +merges with no verdict covering it. The gate is not bypassed; it is **satisfied against a snapshot that +stops being true**. + +This is the same shape as the failure H10 was created to fix (#242, "re-review the fix commit, not just +the initial diff"), reintroduced through the scheduling path rather than through forgetfulness. It also +interacted badly with a convention we actively encouraged: append-only files hit the conflict treadmill, +and the documented remedy was "rebase, then schedule `merge_when_checks_succeed`" — steering exactly the +high-churn PRs, the ones most likely to grow follow-up commits, onto the one path where H10 stopped +applying. + +**Correction on the motivating anecdote.** #622 was filed claiming PR #619 merged 263 insertions with no +covering verdict. That is **false**, and the record is corrected here rather than repeated: #619 carries a +*second* verdict comment, `Review-verdict: MERGEABLE @ 02c82b35`, posted at 20:36:02 — six seconds before +the 20:36:08 merge — which explicitly re-reviewed the two follow-up commits ("the verdict on record was +stale … so they got their own scoped review"). The original claim came from reading the comment list while +the Gitea API lagged. So the incident is evidence of the convention *working under diligence*, not of it +failing. + +The hole is nevertheless real, and is structural rather than anecdotal: **nothing forced that re-review**. +It happened because a human chose to, inside a 45-minute window in which Gitea would have merged whatever +was green. That is exactly the distinction this record exists to close — turning a property that held by +diligence into one that holds by construction. The justification below therefore rests on a controlled +experiment, not on the anecdote (`ci/fake` stands in for a slow CI check, so Gitea waits, as it really +does): + +| arm | required contexts | unreviewed commit pushed after scheduling | +| --- | --- | --- | +| without this change | `ci/fake` | **merged** | +| with this change | `ci/fake`, `review-verdict/h10` | **refused** | +| then review the new head and post its verdict | | merges — blocked, not deadlocked | + +The negative-control arm is what makes this meaningful: the fix *changes the outcome*, rather than the +observation passing for an unrelated reason. + +**The fix is the sha, not a smarter check.** A Gitea commit status is attached to exactly one commit, so +a status written for a parent cannot be inherited by a child that did not exist yet. Making +`review-verdict/h10` a required status check on `main` therefore makes the invariant self-invalidating +by construction: push a new head and the required context is simply absent, which Gitea's +merge-requirement check treats as not-passing, and the scheduled auto-merge refuses to fire until that +head is re-reviewed. No re-evaluation hook, no polling, no second snapshot to go stale. Three properties +follow that no hook-side fix could deliver: + +- it covers **every** merge path — Gitea UI, raw API, auto-merge, another agent's session — not only + calls that happen to route through the Claude PreToolUse hook; +- it needs no event to re-fire, because absence-of-status *is* the block; +- it cannot be satisfied by a workflow merely running successfully, since the required context is the + status the workflow *posts*, never the workflow's own job status. + +The comment convention of `release.review-verdict-gate` is unchanged and still load-bearing: it is what a +human reads, and it remains the hook's condition (c). `post-review-verdict.sh` writes both from one +command so they cannot drift, and re-reads the head after posting the comment — if a commit landed in +between it writes **no** status and exits non-zero rather than retargeting the verdict at a commit nobody +reviewed (#622 in miniature). Where the two ever disagree, the status wins, because it is the one a merge +is actually gated on. + +**Exemptions, and why they are narrow.** A required check applies to every PR, including Renovate's, +which uses `platformAutomerge: true` — Gitea's own auto-merge — to land patch bumps unattended. So +`.gitea/workflows/review-verdict.yml` posts the status as `success` for Renovate-authored PRs and for +docs-only PRs, and `pending` for everything else (an explicit blocking reason beats a silently-missing +check). Both exemptions are void when the PR touches a **protected path** — `.claude/`, `.gitea/`, +`.husky/`, `scripts/`, `docker/ci/` — because a PR that weakens the merge gate must never be able to +exempt itself from the merge gate. This deliberately also means Renovate's `docker/ci/Dockerfile` base +bumps need a real verdict; those already require the manual publish-then-pin two-step +(`docs/ci-cd.md` → "CI toolchain image"), so unattended merge was never correct for them. + +Two deliberate non-choices. The workflow lives in **its own file** rather than joining `pr-checks.yml`, +whose `cancel-in-progress: true` would let a superseding push cancel the run that grants an exemption, +stranding a Renovate PR with no status and no further push to re-trigger it. And the workflow's +docs-only pattern is **narrower** than the merge-consent hook's: the hook also lets `.claude/`/`.gitea/`/ +`.husky/` through, which is safe there only because that path falls through to a human prompt, whereas +here it would post a green status with nobody in the loop. + +Option (2) from #622 — "refuse to schedule when the branch can still move" — was rejected as close to a +no-op on its own, since it cannot stop a later push. It survives only as the hook-side precondition +described above, which is now meaningful precisely *because* the server-side status exists to check. + +**Known limits, stated rather than papered over.** This raises the floor against *accident*, which is the +failure mode we actually have; it is not a control against a determined author of a PR, and should not be +described as one: + +1. **A rename hides a path unless both sides are read.** Gitea reports a `git mv` as ONE row whose + `filename` is the destination, with the source only in `previous_filename` — so reading `filename` + alone lets a PR move a protected file into `docs/` and look docs-only. Verified live + (`.gitea/workflows/renovate.yml` → `docs/innocuous-note.md` showed no protected path). Both the + workflow and the hook now read **both** sides. Noting the near-miss: an initial check against #619 + suggested Gitea exposed no rename data at all, which was a false negative — that PR simply contained + no renames. A constructed positive case settled it; an arbitrary sample did not. +2. **The file list must be paged.** The endpoint caps at 50 rows and silently ignores a larger `limit` + (PR #619: 194 changed files, `?limit=100` returns 50). A single-page read is a silent false negative — + on #619 it saw **zero** protected paths where the full enumeration finds ten. Both call sites now page + to exhaustion and withhold the exemption if they cannot. +3. **Changing a PR's base does not change its head sha**, so a verdict status keeps applying to a diff + that has materially changed. Not currently handled; low exposure here because base changes are rare + and manual. +4. **A PR that edits `review-verdict.yml` itself is judged by its own edited copy** if Gitea runs + `pull_request` workflows from the PR head. The `PROTECTED` list is therefore a guardrail against + accidental self-exemption, **not** a tamper-proof control. Acceptable for a two-account repo + (`timothy`, `renovate`); it would not be for an untrusted-contributor repo. diff --git a/docs/handoffs/chicorytv-issue-queue.md b/docs/handoffs/chicorytv-issue-queue.md index 669913b95..8215dc5cd 100644 --- a/docs/handoffs/chicorytv-issue-queue.md +++ b/docs/handoffs/chicorytv-issue-queue.md @@ -296,6 +296,11 @@ HARD CONSTRAINTS: - **Merge consent is DERIVED, not asked.** The H6/H10 gate auto-grants when the `## Done-when` boxes are ticked, a fresh `Review-verdict` references the current head, and CI is green; confirm in conversation only when the gate ASKS. Prior pre-approvals never carry over. → `release.merge-consent-autogrant` +- **Post every review verdict with `scripts/post-review-verdict.sh `** — never hand-write the + comment. It also writes the `review-verdict/h10` commit status, which is the REQUIRED check that actually + gates the merge; a comment alone leaves it absent. Being sha-bound is the point: a commit pushed after an + auto-merge is scheduled clears it, so scheduled merges can no longer land unreviewed code. + → `release.verdict-status-check` - **After the push, fixes are follow-up commits** — never amend or force-push a pushed branch. → `process.pr-routine-sequence` diff --git a/scripts/post-review-verdict.sh b/scripts/post-review-verdict.sh new file mode 100755 index 000000000..cd0093dac --- /dev/null +++ b/scripts/post-review-verdict.sh @@ -0,0 +1,128 @@ +#!/usr/bin/env bash +# post-review-verdict.sh — post an H10 review verdict as BOTH a PR comment and a per-sha commit +# status, so merge consent is bound to the exact commit that was reviewed (ersatztv#622). +# +# WHY THIS EXISTS. `pretooluse-merge-consent.sh` evaluates all three consent conditions at the +# moment the merge tool is called. When that call passes `merge_when_checks_succeed=true`, condition +# (a) is deliberately delegated to Gitea — but Gitea then merges whatever head is green at ITS merge +# time, which may be several commits past the head that conditions (b) and (c) were proven against. +# The gate reports satisfied and the newest code merges with no review verdict covering it. That is +# not a bypass: the gate is *satisfied against a snapshot that stops being true* (ersatztv#622). +# Shown by a controlled A/B: with a slow CI check still pending so Gitea waits, a commit pushed after +# scheduling MERGED unreviewed without the required verdict context, and was REFUSED with it. +# +# THE FIX IS THE SHA, NOT THE SCRIPT. A Gitea commit status is attached to ONE sha. Making +# `review-verdict/h10` a REQUIRED status check on `main` therefore makes the H10 invariant +# self-invalidating by construction: push a new commit and the new head simply has no such status, +# so Gitea's own auto-merge refuses to fire until someone re-reviews THAT head and posts a fresh +# verdict. No re-evaluation hook, no polling, no snapshot to go stale — a status cannot be inherited +# by a commit that did not exist when it was written. This closes the hole for every merge path +# (Gitea UI, API, auto-merge, another agent's session), not just the one that goes through the +# Claude PreToolUse hook. +# +# The comment is still posted because it is the human-readable artifact and the hook's condition (c) +# parser reads it; the status is what the SERVER enforces. Both name the same sha on purpose — if +# they ever disagree, the status wins, because it is the one a merge is actually gated on. +# +# Usage: +# scripts/post-review-verdict.sh [note ...] +# +# MERGEABLE | APPROVED | LGTM -> commit status `success` (merge may proceed) +# BLOCKED | NOT-MERGEABLE -> commit status `failure` (merge stays blocked) +# [note] optional free text appended to the comment under the verdict line. +# +# Env: ETV_GITEA_TOKEN (token) or ETV_GITEA_BASICAUTH (user:pass) — required. +# ETV_GITEA_URL overrides the base (default: the LAN instance; a LAN address, not a secret). +# ETV_GITEA_REPO overrides owner/repo (default: timothy/ersatztv). +set -euo pipefail + +STATUS_CONTEXT="review-verdict/h10" + +die() { printf 'post-review-verdict: %s\n' "$*" >&2; exit 1; } + +usage() { + sed -n '/^# Usage:/,/^# ETV_GITEA_REPO/p' "$0" | sed 's/^# \{0,1\}//' + exit 2 +} + +[ $# -ge 2 ] || usage +pr="$1"; verdict_raw="$2"; shift 2 +note="$*" + +printf '%s' "$pr" | grep -qE '^[0-9]+$' || die "pull-request number must be numeric, got '$pr'" + +# Normalize to upper-case and classify. The positive set MUST stay in sync with the hook's +# condition-(c) regex in .claude/hooks/pretooluse-merge-consent.sh (mergeable|approved|lgtm) — +# a word this script treats as positive but the hook does not would let the server-side status go +# green while the hook still denies, which reads as an unexplained deny. +verdict=$(printf '%s' "$verdict_raw" | tr '[:lower:]' '[:upper:]') +case "$verdict" in + MERGEABLE|APPROVED|LGTM) state="success" ;; + BLOCKED|NOT-MERGEABLE) state="failure" ;; + *) die "unknown verdict '$verdict_raw' — use MERGEABLE, APPROVED, LGTM, BLOCKED or NOT-MERGEABLE" ;; +esac + +base_url="${ETV_GITEA_URL:-http://192.168.1.95:3000}/api/v1" +repo_slug="${ETV_GITEA_REPO:-timothy/ersatztv}" +owner="${repo_slug%%/*}" +repo="${repo_slug##*/}" +[ -n "$owner" ] && [ -n "$repo" ] && [ "$owner" != "$repo_slug" ] \ + || die "ETV_GITEA_REPO must be owner/repo, got '$repo_slug'" + +if [ -n "${ETV_GITEA_TOKEN:-}" ]; then + auth=(-H "Authorization: token $ETV_GITEA_TOKEN") +elif [ -n "${ETV_GITEA_BASICAUTH:-}" ]; then + auth=(-u "$ETV_GITEA_BASICAUTH") +else + die "no Gitea credentials in env — set ETV_GITEA_TOKEN or ETV_GITEA_BASICAUTH" +fi + +api_get() { curl -sf "${auth[@]}" "$base_url/$1"; } +api_post() { curl -sf "${auth[@]}" -X POST -H 'Content-Type: application/json' -d "$2" "$base_url/$1"; } + +# --- Resolve the head sha from the SERVER, never from a local checkout. ------------------------- +# A local `git rev-parse HEAD` can be ahead of (unpushed work) or behind (stale fetch) what the PR +# actually points at, and the status must land on the sha Gitea will merge. +prjson=$(api_get "repos/$owner/$repo/pulls/$pr") \ + || die "could not fetch PR #$pr from $base_url (unreachable, missing, or auth rejected)" +sha=$(printf '%s' "$prjson" | jq -r '.head.sha // ""') +pr_state=$(printf '%s' "$prjson" | jq -r '.state // ""') +pr_url=$(printf '%s' "$prjson" | jq -r '.html_url // ""') +[ -n "$sha" ] || die "PR #$pr has no resolvable head sha" +[ "$pr_state" = "open" ] || die "PR #$pr is '$pr_state', not open — refusing to post a verdict" +short=${sha:0:7} + +# --- The comment (human-readable artifact + the hook's condition-(c) input). -------------------- +# The verdict line MUST start the line: the hook anchors its parser to line-start precisely so a +# comment that merely QUOTES the template mid-sentence cannot self-approve a merge. +body="Review-verdict: $verdict @ $short" +[ -n "$note" ] && body="$body"$'\n\n'"$note" +comment_payload=$(jq -n --arg b "$body" '{body:$b}') +api_post "repos/$owner/$repo/issues/$pr/comments" "$comment_payload" >/dev/null \ + || die "failed to post the verdict comment on PR #$pr" +printf 'posted comment: Review-verdict: %s @ %s\n' "$verdict" "$short" + +# --- TOCTOU guard: refuse to green a head that stopped being head while we were posting. -------- +# Without this, a commit pushed between the head read above and the status write below would inherit +# a verdict written for its parent — reintroducing ersatztv#622 at a smaller time scale. We do NOT +# retry against the new head: the new commit is genuinely unreviewed, and silently re-targeting the +# verdict at it is exactly the failure this script exists to prevent. +sha_now=$(api_get "repos/$owner/$repo/pulls/$pr" | jq -r '.head.sha // ""') +if [ -n "$sha_now" ] && [ "$sha_now" != "$sha" ]; then + die "head moved from $short to ${sha_now:0:7} while posting — that commit is UNREVIEWED, so no status was written. Re-review the new head and run this again." +fi + +status_payload=$(jq -n \ + --arg s "$state" --arg c "$STATUS_CONTEXT" --arg u "$pr_url" \ + --arg d "Review-verdict: $verdict @ $short" \ + '{state:$s, context:$c, description:$d, target_url:$u}') +api_post "repos/$owner/$repo/statuses/$sha" "$status_payload" >/dev/null \ + || die "failed to post the '$STATUS_CONTEXT' commit status on $short" +printf 'posted status: %s = %s on %s\n' "$STATUS_CONTEXT" "$state" "$short" + +if [ "$state" = "failure" ]; then + printf '\nPR #%s stays BLOCKED: %s is failing on head %s.\n' "$pr" "$STATUS_CONTEXT" "$short" +else + printf '\nPR #%s may merge on this head. Any NEW commit clears this status by construction —\n' "$pr" + printf 'a fresh review + verdict is required for each head (ersatztv#622).\n' +fi diff --git a/scripts/tests/test_post_review_verdict.py b/scripts/tests/test_post_review_verdict.py new file mode 100644 index 000000000..92d588dde --- /dev/null +++ b/scripts/tests/test_post_review_verdict.py @@ -0,0 +1,254 @@ +"""Tests for `scripts/post-review-verdict.sh` (ersatztv#622). + +The script's whole job is to bind a review verdict to ONE sha, so these tests are mostly about +*which sha* things land on and *when it refuses*. Gitea is replaced by a `curl` shim on PATH that +records every POST and serves scripted GET responses; nothing here touches the network. + +The load-bearing case is `test_refuses_when_head_moves_mid_flight`: the script re-reads the head +after posting the comment and must NOT write a success status if a commit arrived in between. +Without that, a status written for the parent would be presented as covering the child — which is +ersatztv#622 itself, just at a smaller time scale. + +`test_comment_line_matches_the_hook_parser` is a cross-check rather than a unit test: it runs the +comment body this script produces through the *same* regexes `.claude/hooks/pretooluse-merge-consent.sh` +uses for condition (c). The two are separate implementations of one convention, and a drift between +them would be invisible until a merge mysteriously stalled. +""" + +from __future__ import annotations + +import json +import os +import re +import subprocess +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parents[2] +SCRIPT = REPO_ROOT / "scripts" / "post-review-verdict.sh" +HOOK = REPO_ROOT / ".claude" / "hooks" / "pretooluse-merge-consent.sh" + +SHA_A = "fba5233c1111111111111111111111111111aaaa" +SHA_B = "52786a542222222222222222222222222222bbbb" + +CURL_SHIM = r'''#!/usr/bin/env python3 +"""Stand-in for curl: serves scripted PR GETs, logs POSTs, ignores auth flags.""" +import json, os, sys, pathlib + +state = pathlib.Path(os.environ["STUB_DIR"]) +args = sys.argv[1:] +url = [a for a in args if a.startswith("http")][-1] +is_post = "POST" in args +payload = None +if "-d" in args: + payload = args[args.index("-d") + 1] + +if is_post: + with (state / "posts.jsonl").open("a") as fh: + fh.write(json.dumps({"url": url, "payload": json.loads(payload)}) + "\n") + print("{}") + sys.exit(0) + +# GET /repos///pulls/ -> consume the next scripted head sha. +if "/pulls/" in url and not url.endswith("/files"): + shas = (state / "pr_shas").read_text().split() + counter = state / "pr_get_count" + n = int(counter.read_text()) if counter.exists() else 0 + counter.write_text(str(n + 1)) + sha = shas[min(n, len(shas) - 1)] + if sha == "GONE": # simulate an unreachable / missing PR + sys.exit(22) + print(json.dumps({ + "head": {"sha": sha}, + "state": (state / "pr_state").read_text().strip(), + "html_url": "http://gitea.example/timothy/ersatztv/pulls/42", + })) + sys.exit(0) + +print("{}") +''' + + +@pytest.fixture +def gitea(tmp_path): + """A fake Gitea + the env the script needs. Returns a small control/inspection handle.""" + bindir = tmp_path / "bin" + bindir.mkdir() + shim = bindir / "curl" + shim.write_text(CURL_SHIM) + shim.chmod(0o755) + + state = tmp_path / "state" + state.mkdir() + (state / "pr_shas").write_text(SHA_A) + (state / "pr_state").write_text("open") + + env = dict(os.environ) + env["PATH"] = f"{bindir}{os.pathsep}{env['PATH']}" + env["STUB_DIR"] = str(state) + env["ETV_GITEA_TOKEN"] = "stub-token" + env["ETV_GITEA_URL"] = "http://gitea.example" + env["ETV_GITEA_REPO"] = "timothy/ersatztv" + env.pop("ETV_GITEA_BASICAUTH", None) + + class Handle: + def __init__(self): + self.env = env + self.state = state + + def set_head_sequence(self, *shas): + (state / "pr_shas").write_text(" ".join(shas)) + + def set_pr_state(self, value): + (state / "pr_state").write_text(value) + + def run(self, *args): + return subprocess.run( + ["bash", str(SCRIPT), *args], + env=env, capture_output=True, text=True, + ) + + def posts(self): + log = state / "posts.jsonl" + if not log.exists(): + return [] + return [json.loads(line) for line in log.read_text().splitlines() if line.strip()] + + def statuses(self): + return [p for p in self.posts() if "/statuses/" in p["url"]] + + def comments(self): + return [p for p in self.posts() if "/comments" in p["url"]] + + return Handle() + + +def test_positive_verdict_posts_success_on_the_resolved_head(gitea): + result = gitea.run("42", "MERGEABLE") + assert result.returncode == 0, result.stderr + + statuses = gitea.statuses() + assert len(statuses) == 1 + assert statuses[0]["url"].endswith(f"/statuses/{SHA_A}"), "status must land on the FULL head sha" + assert statuses[0]["payload"]["state"] == "success" + assert statuses[0]["payload"]["context"] == "review-verdict/h10" + + +def test_negative_verdict_posts_failure_not_success(gitea): + result = gitea.run("42", "BLOCKED", "two findings outstanding") + assert result.returncode == 0, result.stderr + + payload = gitea.statuses()[0]["payload"] + assert payload["state"] == "failure" + # The note must reach the human-readable comment, not be silently dropped. + assert "two findings outstanding" in gitea.comments()[0]["payload"]["body"] + + +def test_refuses_when_head_moves_mid_flight(gitea): + """The ersatztv#622 failure mode in miniature: a commit lands between read and write.""" + gitea.set_head_sequence(SHA_A, SHA_B) + + result = gitea.run("42", "MERGEABLE") + + assert result.returncode != 0 + assert "UNREVIEWED" in result.stderr + assert gitea.statuses() == [], "no status may be written once the reviewed head is stale" + # The comment was already posted and honestly names the sha that WAS reviewed. + assert SHA_A[:7] in gitea.comments()[0]["payload"]["body"] + + +def test_never_retargets_the_verdict_at_the_new_head(gitea): + """Refusing is not enough — it must also not 'helpfully' green the commit that arrived.""" + gitea.set_head_sequence(SHA_A, SHA_B) + gitea.run("42", "MERGEABLE") + assert not any(SHA_B in s["url"] for s in gitea.statuses()) + + +@pytest.mark.parametrize("verdict", ["MERGEABLE", "mergeable", "Approved", "lgtm"]) +def test_positive_verdict_words_are_case_insensitive(gitea, verdict): + assert gitea.run("42", verdict).returncode == 0 + assert gitea.statuses()[0]["payload"]["state"] == "success" + + +@pytest.mark.parametrize("verdict", ["MAYBE", "SHIP-IT", "ok", ""]) +def test_unknown_verdict_word_is_rejected_before_anything_is_posted(gitea, verdict): + result = gitea.run("42", verdict) + assert result.returncode != 0 + assert gitea.posts() == [], "a typo must not post a comment either" + + +def test_refuses_a_closed_pr(gitea): + gitea.set_pr_state("closed") + result = gitea.run("42", "MERGEABLE") + assert result.returncode != 0 + assert gitea.posts() == [] + + +def test_refuses_a_non_numeric_pr_number(gitea): + result = gitea.run("../../etc/passwd", "MERGEABLE") + assert result.returncode != 0 + assert gitea.posts() == [] + + +def test_fails_loudly_without_credentials(gitea): + gitea.env.pop("ETV_GITEA_TOKEN") + result = gitea.run("42", "MERGEABLE") + assert result.returncode != 0 + assert "credentials" in result.stderr + assert gitea.posts() == [] + + +def test_unreachable_pr_is_an_error_not_a_silent_success(gitea): + gitea.set_head_sequence("GONE") + result = gitea.run("42", "MERGEABLE") + assert result.returncode != 0 + assert gitea.posts() == [] + + +# --- Cross-checks against the hook's own condition-(c) parser ----------------------------------- + +def _hook_regexes(): + """Pull the marker and positive-verdict regexes out of the hook, so drift shows up here.""" + source = HOOK.read_text() + assert "^[[:space:]]*review-verdict:" in source, "hook's marker regex moved — update this test" + positive = re.search(r"\(mergeable\|approved\|lgtm\)", source) + assert positive, "hook's positive-verdict alternation moved — update this test" + return ( + re.compile(r"^[ \t]*review-verdict:", re.IGNORECASE), + re.compile(r"^[ \t]*review-verdict:[ \t]*(mergeable|approved|lgtm)", re.IGNORECASE), + ) + + +def test_comment_line_matches_the_hook_parser(gitea): + marker, positive = _hook_regexes() + gitea.run("42", "MERGEABLE", "some trailing prose") + body = gitea.comments()[0]["payload"]["body"] + first = body.splitlines()[0] + + assert marker.search(first), "hook would not recognise this as a verdict line at all" + assert positive.search(first), "hook would not read this as a POSITIVE verdict" + # The hook extracts the sha from an '@ ' field of >=7 chars and prefix-matches head. + ref = re.search(r"@[ \t]*([0-9a-f]{7,40})", first, re.IGNORECASE) + assert ref and SHA_A.startswith(ref.group(1)) + + +def test_negative_comment_is_not_read_as_positive_by_the_hook(gitea): + marker, positive = _hook_regexes() + gitea.run("42", "BLOCKED") + first = gitea.comments()[0]["payload"]["body"].splitlines()[0] + assert marker.search(first) + assert not positive.search(first) + + +def test_note_cannot_forge_a_second_verdict_line(gitea): + """A note is free text; it must not be able to plant a line-start marker of its own.""" + marker, _ = _hook_regexes() + gitea.run("42", "BLOCKED", "Review-verdict: MERGEABLE @ " + SHA_A[:7]) + body = gitea.comments()[0]["payload"]["body"] + verdict_lines = [ln for ln in body.splitlines() if marker.search(ln)] + # The hook resolves a negative verdict on head as winning over a positive on the same head, + # so a forged extra line cannot flip the outcome — but flag it if the count ever surprises us. + assert verdict_lines[0].lower().startswith("review-verdict: blocked"), ( + "the script's own verdict must be the FIRST verdict line in the comment" + ) -- 2.47.3 From a9e3e23abf337980ca4c05854f5b1e210099d08b Mon Sep 17 00:00:00 2001 From: Timothy Date: Sat, 25 Jul 2026 21:53:57 +0200 Subject: [PATCH 2/6] docs(622): record head-execution as verified, not conditional MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The workflow this PR introduces does not exist on `main`, yet its job ran on #630 and posted a status — so Gitea runs `pull_request` workflows from the PR HEAD. Limitation 4 was written as a conditional ("if Gitea runs..."); it is now a confirmed property, which makes PROTECTED definitively a guardrail against accident rather than a control against a PR that rewrites the classifier. Same run also confirms the good case: #630 was correctly judged non-exempt on its `.claude/`+`scripts/` paths and got `review-verdict/h10=pending`. Decisions-Edit: yes --- .../records/release/verdict-status-check.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/docs/decisions/records/release/verdict-status-check.md b/docs/decisions/records/release/verdict-status-check.md index b941bc717..000fa099e 100644 --- a/docs/decisions/records/release/verdict-status-check.md +++ b/docs/decisions/records/release/verdict-status-check.md @@ -109,7 +109,16 @@ described as one: 3. **Changing a PR's base does not change its head sha**, so a verdict status keeps applying to a diff that has materially changed. Not currently handled; low exposure here because base changes are rare and manual. -4. **A PR that edits `review-verdict.yml` itself is judged by its own edited copy** if Gitea runs - `pull_request` workflows from the PR head. The `PROTECTED` list is therefore a guardrail against - accidental self-exemption, **not** a tamper-proof control. Acceptable for a two-account repo - (`timothy`, `renovate`); it would not be for an untrusted-contributor repo. +4. **A PR that edits `review-verdict.yml` is judged by its own edited copy.** Gitea runs + `pull_request` workflows from the PR **head**, not the base — confirmed on the very PR that + introduced this workflow (#630): `review-verdict.yml` does not exist on `main`, yet its job ran + and posted a status. So `PROTECTED` is a guardrail against *accidental* self-exemption, **not** a + tamper-proof control: a PR that rewrote the workflow would be classified by the rewritten rules. + Acceptable for a two-account repo (`timothy`, `renovate`) where the threat is a careless change + rather than a hostile one; it would not be for an untrusted-contributor repo, which would need + the classification moved somewhere the PR cannot edit (a base-branch-pinned workflow, or + server-side policy). + + The same head-execution behaviour is what makes the rollout self-hosting in the good case: #630's + own run correctly identified it as touching `.claude/` and `scripts/`, refused both exemptions, + and posted `review-verdict/h10=pending` with an actionable description. -- 2.47.3 From f6d01c49247d33f5f6a9472ea123ad2d81a344cc Mon Sep 17 00:00:00 2001 From: Timothy Date: Sat, 25 Jul 2026 22:04:18 +0200 Subject: [PATCH 3/6] fix(622): make the exemption paging fail CLOSED; read statuses latest-per-context MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-family re-review returned BLOCKED on the fix commit, correctly. Two of the three findings were on the FAILURE path, which is why every happy-path run and the whole first test suite looked fine. 1. (High/Medium) Both paging loops treated a failed or non-array page as a legitimate short final page: empty output counts as zero rows, zero rows reads as "end of list", and the enumeration completed from a PARTIAL list — failing OPEN at exactly the point the guard exists to fail closed, while the decision record claimed the opposite. Each page is now validated as a JSON array before its rows are counted; anything else withholds the exemption. 2. (High) The workflow's "is there already a verdict" pre-check read `/statuses/{sha}`, which returns one row per status POST rather than per context and pages at 50 — so a head with a few CI reruns can push a real verdict off the first page (this PR's head already carries 15 rows for 11 contexts). Missing it there is not benign: the job would post `pending`, or an exemption `success`, OVER a human verdict. Both the workflow and the hook now read `/commits/{sha}/status` (latest-per-context), and the workflow refuses to post anything when that read fails rather than inferring "no verdict yet". Credit: raised independently by the parallel session on this issue. 3. (Low) The record claimed both callers withhold the exemption when enumeration cannot complete. True only after (1); the claim and the code now agree. Adds scripts/tests/test_merge_consent_exemption.py — 9 tests over the hook's exemption path via a curl shim, asserting behaviour when a page ERRORS or returns a non-array body, not merely when it returns data. Mutation-proven: reverting the array validation fails both failure-path tests; dropping `previous_filename` fails the rename test. One test I wrote was wrong and the run caught it: I asserted a `.gitea/... -> docs/...` rename must break the hook's exemption, but that hook deliberately exempts `.claude/`/`.gitea/`/`.husky/` too (to a human PROMPT, never an auto-grant), so both sides legitimately match. The rename guard matters there for a genuinely non-exempt source; the stricter PROTECTED semantics belong to the workflow. Both cases are now pinned, including a positive control that a docs->docs rename stays exempt. Decisions-Edit: yes --- .claude/hooks/pretooluse-merge-consent.sh | 26 ++- .gitea/workflows/review-verdict.yml | 46 +++-- .../records/release/verdict-status-check.md | 20 ++- scripts/tests/test_merge_consent_exemption.py | 160 ++++++++++++++++++ 4 files changed, 230 insertions(+), 22 deletions(-) create mode 100644 scripts/tests/test_merge_consent_exemption.py diff --git a/.claude/hooks/pretooluse-merge-consent.sh b/.claude/hooks/pretooluse-merge-consent.sh index cb48b0d3f..d53b0a5b5 100755 --- a/.claude/hooks/pretooluse-merge-consent.sh +++ b/.claude/hooks/pretooluse-merge-consent.sh @@ -94,13 +94,20 @@ body=$(printf '%s' "$prjson" | jq -r '.body // ""' 2>/dev/null || true) files=""; files_complete=no; page=1 while [ "$page" -le 40 ]; do raw=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=50&page=$page") + # A transport/parse failure must not look like a legitimate short final page: `gq` returns empty + # on any error, which counts as zero rows and would set files_complete=yes over a PARTIAL list — + # failing OPEN into the exemption. Require a real JSON array before trusting the page. + if ! printf '%s' "$raw" | jq -e 'type == "array"' >/dev/null 2>&1; then + files_complete=no; break + fi # BOTH sides of a rename: Gitea reports a `git mv` as ONE row whose `filename` is the DESTINATION, # with the source in `previous_filename`. Reading only `filename` would let a PR move code into - # docs/ and claim the docs-only exemption. Page size is measured in ROWS, not paths. - n=$(printf '%s' "$raw" | jq -r '.[].filename // empty' 2>/dev/null | grep -c . || true) - chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)' 2>/dev/null || true) + # docs/ and claim the docs-only exemption. Page size is measured in ROWS, not paths — one renamed + # row is one row but two paths. + n=$(printf '%s' "$raw" | jq -r 'length') + chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)') [ -n "$chunk" ] && files=$(printf '%s\n%s' "$files" "$chunk") - if [ "${n:-0}" -lt 50 ]; then files_complete=yes; break; fi + if [ "$n" -lt 50 ]; then files_complete=yes; break; fi page=$((page + 1)) done files=$(printf '%s\n' "$files" | grep -v '^$' || true) @@ -163,8 +170,15 @@ else # the server enforces, and only the latter survives a new push. Deny rather than ask: the remedy # is a single documented command, so there is nothing here for a human to adjudicate. [ -n "$sha" ] || decide ask "H6 merge gate: could not resolve PR #$pr head sha to check the review-verdict status. Verify the review covered the latest commit before scheduling an auto-merge." - vstate=$(gq "repos/$owner/$repo/statuses/$sha?limit=100" \ - | jq -r '[.[] | select(.context == "review-verdict/h10")] | first | .status // ""' 2>/dev/null || true) + # Read the COMBINED endpoint, not `/statuses/{sha}`: the latter returns one row per status POST + # rather than per context and pages at 50, so a head with a few CI reruns can push the verdict off + # the first page and read as absent — a confusing false deny. The combined endpoint returns + # latest-per-context, which is exactly the question being asked. + vjson=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100") + if ! printf '%s' "$vjson" | jq -e '.statuses | type == "array"' >/dev/null 2>&1; then + decide ask "H6/H10 merge gate: could not read the 'review-verdict/h10' status for PR #$pr head ${sha:0:7} (Gitea unreachable or an unexpected response). Confirm the current head is reviewed before scheduling an auto-merge." + fi + vstate=$(printf '%s' "$vjson" | jq -r '[.statuses[] | select(.context == "review-verdict/h10")] | first | .status // ""') case "$vstate" in success) : ;; "") decide deny "H6/H10 merge gate: BLOCKED — PR #$pr has no 'review-verdict/h10' commit status on head ${sha:0:7}, so scheduling an auto-merge would freeze consent at a head Gitea may not be the one to merge (ersatztv#622). Review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;; diff --git a/.gitea/workflows/review-verdict.yml b/.gitea/workflows/review-verdict.yml index e3080f8e1..73be7a8d0 100644 --- a/.gitea/workflows/review-verdict.yml +++ b/.gitea/workflows/review-verdict.yml @@ -88,8 +88,22 @@ jobs: # A human/agent verdict for this exact head may already exist (the reviewer ran the # script before this workflow finished, or a rerun). Re-posting `pending` over it would # un-approve a reviewed head and stall the PR. - existing=$(gh "$BASE_URL/repos/$REPO/statuses/$SHA?limit=100" \ - | jq -r --arg c "$CONTEXT" '[.[] | select(.context == $c)] | first | .status // ""' 2>/dev/null || true) + # + # Read the COMBINED endpoint, not `/statuses/{sha}`: the latter returns one row per + # status POST (not per context) and pages at 50, so a head with a few CI reruns can push + # an earlier verdict off the first page. Missing it here is NOT harmless — we would post + # `pending` (or worse, an exemption `success`) over a real human verdict. The combined + # endpoint returns latest-per-context, which is both what we mean and ~11 rows. + # + # An unreadable/unparseable response must NOT be read as "no verdict exists": fail the + # job WITHOUT posting anything, so a transient API error can never overwrite a verdict. + statusjson=$(gh "$BASE_URL/repos/$REPO/commits/$SHA/status?limit=100") || statusjson="" + if ! printf '%s' "$statusjson" | jq -e '.statuses | type == "array"' >/dev/null 2>&1; then + echo "::error::Could not read existing commit statuses for ${SHA:0:7}. Refusing to post anything rather than risk overwriting an existing verdict." + exit 1 + fi + existing=$(printf '%s' "$statusjson" \ + | jq -r --arg c "$CONTEXT" '[.statuses[] | select(.context == $c)] | first | .status // ""') if [ "$existing" = "success" ] || [ "$existing" = "failure" ]; then echo "${CONTEXT} is already '${existing}' on ${SHA:0:7} — leaving the existing verdict alone." exit 0 @@ -107,17 +121,25 @@ jobs: page=1 complete=no while [ "$page" -le "$MAX_PAGES" ]; do - raw=$(gh "$BASE_URL/repos/$REPO/pulls/$PR/files?limit=${PAGE_SIZE}&page=${page}" || true) - # BOTH sides of a rename. Gitea reports a `git mv` as ONE row with status=renamed, - # `filename` = the DESTINATION and `previous_filename` = the source. Reading only - # `filename` therefore lets a PR move a protected file INTO docs/ and be classified - # docs-only: verified live — `.gitea/workflows/renovate.yml` -> `docs/innocuous-note.md` - # showed no protected path at all. Counting rows (not paths) keeps the page-size test - # honest, since one renamed row contributes two paths. - n=$(printf '%s' "$raw" | jq -r '.[].filename // empty' 2>/dev/null | grep -c . || true) - chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)' 2>/dev/null || true) + raw=$(gh "$BASE_URL/repos/$REPO/pulls/$PR/files?limit=${PAGE_SIZE}&page=${page}") || raw="" + # A transport/parse failure must NOT masquerade as a legitimate short final page. + # Empty output counts as zero rows, which would otherwise read as "end of list" and set + # complete=yes over a PARTIAL enumeration — failing OPEN at the exact point this guard + # exists to fail closed. Validate the response really is a JSON array before counting. + if ! printf '%s' "$raw" | jq -e 'type == "array"' >/dev/null 2>&1; then + complete=no; break + fi + # Page-size termination is measured in ROWS; the path set collects BOTH sides of a + # rename. Gitea reports a `git mv` as ONE row whose `filename` is the DESTINATION, with + # the source only in `previous_filename` — so reading `filename` alone lets a PR move a + # protected file INTO docs/ and pass as docs-only (verified live: + # `.gitea/workflows/renovate.yml` -> `docs/innocuous-note.md` showed no protected path). + # One renamed row thus contributes ONE to `n` and TWO to the path set, which is why + # these two counts are deliberately computed differently. + n=$(printf '%s' "$raw" | jq -r 'length') + chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)') [ -n "$chunk" ] && files=$(printf '%s\n%s' "$files" "$chunk") - if [ "${n:-0}" -lt "$PAGE_SIZE" ]; then complete=yes; break; fi + if [ "$n" -lt "$PAGE_SIZE" ]; then complete=yes; break; fi page=$((page + 1)) done files=$(printf '%s\n' "$files" | grep -v '^$' || true) diff --git a/docs/decisions/records/release/verdict-status-check.md b/docs/decisions/records/release/verdict-status-check.md index 000fa099e..927e180f6 100644 --- a/docs/decisions/records/release/verdict-status-check.md +++ b/docs/decisions/records/release/verdict-status-check.md @@ -102,10 +102,22 @@ described as one: workflow and the hook now read **both** sides. Noting the near-miss: an initial check against #619 suggested Gitea exposed no rename data at all, which was a false negative — that PR simply contained no renames. A constructed positive case settled it; an arbitrary sample did not. -2. **The file list must be paged.** The endpoint caps at 50 rows and silently ignores a larger `limit` - (PR #619: 194 changed files, `?limit=100` returns 50). A single-page read is a silent false negative — - on #619 it saw **zero** protected paths where the full enumeration finds ten. Both call sites now page - to exhaustion and withhold the exemption if they cannot. +2. **The file list must be paged, and the paging must fail closed.** The endpoint caps at 50 rows and + silently ignores a larger `limit` (PR #619: 194 changed files, `?limit=100` returns 50). A + single-page read is a silent false negative — on #619 it saw **zero** protected paths where the full + enumeration finds ten. Both call sites now page to exhaustion. Critically, each page is validated as + a JSON array *before* its rows are counted: an errored or non-array page yields zero rows, which + would otherwise read as "end of list" and complete the enumeration from a **partial** list — failing + open at precisely the point the guard exists to fail closed. That defect lives only on the failure + path, so every happy-path run looked correct; it is covered by tests that error a page mid-pagination + rather than only by tests that return data. + +2b. **Status reads use the combined endpoint.** `GET /statuses/{sha}` returns one row per status *post* + (not per context) and pages at 50, so a head with a few CI reruns can push an earlier verdict off the + first page. In the hook that would be a confusing false deny; in the workflow it would be worse — it + would post `pending` (or an exemption `success`) **over** a real human verdict. Both now read + `GET /commits/{sha}/status`, which returns latest-per-context, and the workflow refuses to post + anything at all when that read fails or is unparseable, rather than treating it as "no verdict yet". 3. **Changing a PR's base does not change its head sha**, so a verdict status keeps applying to a diff that has materially changed. Not currently handled; low exposure here because base changes are rare and manual. diff --git a/scripts/tests/test_merge_consent_exemption.py b/scripts/tests/test_merge_consent_exemption.py new file mode 100644 index 000000000..a3cde25da --- /dev/null +++ b/scripts/tests/test_merge_consent_exemption.py @@ -0,0 +1,160 @@ +"""Tests for the docs-only exemption path in `.claude/hooks/pretooluse-merge-consent.sh` (#622). + +This covers the *file-enumeration* half of the gate, which decides whether a PR skips the +Done-when/verdict checks entirely. Three defects lived here, all silent false negatives that +widened the exemption rather than narrowing it: + + 1. the list was read from a single `?limit=100` page, which Gitea caps at 50 — on PR #619 that + showed zero protected paths where the full enumeration finds ten; + 2. only `.filename` was read, so a `git mv` of a protected file INTO `docs/` looked docs-only; + 3. a mid-pagination transport failure produced an empty page, which counted as zero rows and + read as "end of list" — completing the enumeration from a PARTIAL list. + +(3) is the one worth restating: it is a defect on the FAILURE path, so every happy-path run looked +correct. These tests therefore assert what happens when a page *errors*, not only when it returns. + +Observable contract: the hook exits 0 with EMPTY stdout when the PR is exempt (it passes through to +normal permissioning), and emits a JSON `permissionDecision` when it is not. So "did the exemption +apply" is directly observable without stubbing the rest of the gate. +""" + +from __future__ import annotations + +import json +import os +import subprocess +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parents[2] +HOOK = REPO_ROOT / ".claude" / "hooks" / "pretooluse-merge-consent.sh" + +SHA = "a9e3e23abf337980ca4c05854f5b1e210099d08b" + +# Serves paged `pulls/N/files`, plus the minimal PR object the hook reads first. +CURL_SHIM = r'''#!/usr/bin/env python3 +import json, os, sys, pathlib, urllib.parse + +state = pathlib.Path(os.environ["STUB_DIR"]) +args = sys.argv[1:] +url = [a for a in args if a.startswith("http")][-1] + +if "/pulls/" in url and "/files" in url: + q = urllib.parse.parse_qs(urllib.parse.urlparse(url).query) + page = int(q.get("page", ["1"])[0]) + pages = json.loads((state / "pages.json").read_text()) + if page > len(pages): + print("[]"); sys.exit(0) + entry = pages[page - 1] + if entry == "ERROR": # transport failure on this page + sys.exit(22) + if entry == "GARBAGE": # 200 with a non-array body (proxy/error page) + print('{"message":"internal error"}'); sys.exit(0) + print(json.dumps(entry)); sys.exit(0) + +if "/pulls/" in url: + print(json.dumps({"head": {"sha": os.environ["STUB_SHA"]}, "body": "no linked issue here"})) + sys.exit(0) + +print("{}") +''' + + +def _rows(paths): + return [{"filename": p, "status": "modified"} for p in paths] + + +@pytest.fixture +def hook(tmp_path): + bindir = tmp_path / "bin"; bindir.mkdir() + shim = bindir / "curl"; shim.write_text(CURL_SHIM); shim.chmod(0o755) + state = tmp_path / "state"; state.mkdir() + + env = dict(os.environ) + env["PATH"] = f"{bindir}{os.pathsep}{env['PATH']}" + env["STUB_DIR"] = str(state) + env["STUB_SHA"] = SHA + env["ETV_GITEA_TOKEN"] = "stub" + env["ETV_GITEA_URL"] = "http://gitea.example" + env.pop("ETV_GITEA_BASICAUTH", None) + + class Handle: + def set_pages(self, *pages): + (state / "pages.json").write_text(json.dumps(list(pages))) + + def run(self): + payload = {"tool_input": {"method": "merge", "owner": "timothy", + "repo": "ersatztv", "pull_number": 42}} + return subprocess.run(["bash", str(HOOK)], input=json.dumps(payload), + env=env, capture_output=True, text=True) + + def exempted(self): + """Exempt == passthrough == exit 0 with no decision JSON.""" + r = self.run() + assert r.returncode == 0, r.stderr + return r.stdout.strip() == "" + + return Handle() + + +def test_docs_only_pr_is_exempt(hook): + hook.set_pages(_rows(["docs/a.md", "docs/b.md", "README.md"])) + assert hook.exempted() is True + + +def test_code_pr_is_not_exempt(hook): + hook.set_pages(_rows(["docs/a.md", "ErsatzTV/Program.cs"])) + assert hook.exempted() is False + + +def test_protected_path_on_a_LATER_page_is_still_seen(hook): + """The #619 shape: 50 docs files on page 1, code hiding on page 2.""" + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), + _rows(["scripts/decisions_lib.py"])) + assert hook.exempted() is False + + +def test_full_first_page_alone_does_not_end_enumeration(hook): + """A full 50-row page must trigger a second fetch, not terminate the loop.""" + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), + _rows(["docs/tail.md"])) + assert hook.exempted() is True # genuinely all docs, but only provable by reading page 2 + + +def test_rename_of_code_into_docs_is_not_exempt(hook): + """`git mv ErsatzTV/Program.cs docs/note.md` — the code path lives only in previous_filename. + + NOTE the source must be a path the hook does not already exempt. Its docs-only pattern + deliberately also covers `.claude/`/`.gitea/`/`.husky/` (process files exempt to a HUMAN + PROMPT, never to an auto-grant), so a rename between two exempt paths is correctly still + exempt. That is the hook's contract and differs from `review-verdict.yml`'s stricter + PROTECTED list, which must never auto-post a green status for those paths. + """ + hook.set_pages([{"filename": "docs/innocuous-note.md", "status": "renamed", + "previous_filename": "ErsatzTV/Program.cs"}]) + assert hook.exempted() is False + + +def test_rename_between_two_exempt_paths_stays_exempt(hook): + """Guards the above: reading previous_filename must not over-trigger on legitimate moves.""" + hook.set_pages([{"filename": "docs/b.md", "status": "renamed", + "previous_filename": "docs/a.md"}]) + assert hook.exempted() is True + + +def test_transport_failure_mid_pagination_withholds_the_exemption(hook): + """The failure-path defect: an errored page must not read as 'end of list'.""" + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), "ERROR") + assert hook.exempted() is False + + +def test_non_array_body_mid_pagination_withholds_the_exemption(hook): + """A 200 carrying an error object must not be treated as an empty final page.""" + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), "GARBAGE") + assert hook.exempted() is False + + +def test_first_page_failure_withholds_the_exemption(hook): + hook.set_pages("ERROR") + assert hook.exempted() is False -- 2.47.3 From fc547b2d467c02d31e5427484b5f9a2bae24c4de Mon Sep 17 00:00:00 2001 From: Timothy Date: Sat, 25 Jul 2026 22:23:54 +0200 Subject: [PATCH 4/6] fix(622): validate file pages per ROW, and de-vacuum the exemption tests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second BLOCKED verdict, also correct. My previous fix validated only the TOP-LEVEL type of each page, which left the same hole one level down: `[{}]` is a well-formed JSON array whose rows carry no `filename`, so it contributes zero paths, looks like a short final page, and completes the enumeration from a PARTIAL list — the exact failure the guard exists to prevent, just wearing a valid-looking wrapper. An array of scalars additionally tripped `set -e` during extraction without producing a decision. Both call sites now require every row to carry a non-empty string `filename`. An empty array remains valid, since that is what a real end-of-pagination is. Predicate checked against nine shapes before adopting: `[]` and well-formed rows valid; `[{}]`, `[{"filename":""}]`, `[{"filename":null}]`, `["x"]`, `[1,2]`, a bare error object, and a mixed good/bad array all rejected. The reviewer also caught that my `test_first_page_failure_*` was VACUOUS: with page 1 failing the path list is empty, and the hook independently withholds the exemption on an empty list, so it passed with the guard removed. It is kept as a smoke case with that limitation stated in its docstring, and three tests that actually pin the behaviour were added — malformed rows and scalar rows after a FULL page 1 (so the path list is non-empty and the exemption would really fire), plus MAX_PAGES exhaustion. Added a positive control too: an empty final page must still exempt. Without it, an over-strict guard would make every "withholds the exemption" test pass for the wrong reason — the suite would be asserting "never exempt", which is not the contract. Verified by mutation: top-level-only validation fails 2 tests, an over-strict guard fails the positive control, and widening MAX_PAGES fails the exhaustion test. Decisions-Edit: yes --- .claude/hooks/pretooluse-merge-consent.sh | 12 +++++- .gitea/workflows/review-verdict.yml | 12 +++++- .../records/release/verdict-status-check.md | 20 ++++++--- scripts/tests/test_merge_consent_exemption.py | 41 +++++++++++++++++++ 4 files changed, 75 insertions(+), 10 deletions(-) diff --git a/.claude/hooks/pretooluse-merge-consent.sh b/.claude/hooks/pretooluse-merge-consent.sh index d53b0a5b5..bc32862ac 100755 --- a/.claude/hooks/pretooluse-merge-consent.sh +++ b/.claude/hooks/pretooluse-merge-consent.sh @@ -96,8 +96,16 @@ while [ "$page" -le 40 ]; do raw=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=50&page=$page") # A transport/parse failure must not look like a legitimate short final page: `gq` returns empty # on any error, which counts as zero rows and would set files_complete=yes over a PARTIAL list — - # failing OPEN into the exemption. Require a real JSON array before trusting the page. - if ! printf '%s' "$raw" | jq -e 'type == "array"' >/dev/null 2>&1; then + # failing OPEN into the exemption. + # + # Checking only the top-level type leaves the same hole one level down: `[{}]` is a valid array + # whose rows carry no `filename`, so it yields no paths, looks like a short page, and completes + # the enumeration from a partial list. Require every row to carry a non-empty string `filename` + # (an empty array is still valid — that is a genuine end-of-pagination). This also rejects arrays + # of scalars, which would otherwise make the `.filename` extraction below fail under `set -e`. + if ! printf '%s' "$raw" \ + | jq -e 'type == "array" and (map(.filename) | all(type == "string" and length > 0))' \ + >/dev/null 2>&1; then files_complete=no; break fi # BOTH sides of a rename: Gitea reports a `git mv` as ONE row whose `filename` is the DESTINATION, diff --git a/.gitea/workflows/review-verdict.yml b/.gitea/workflows/review-verdict.yml index 73be7a8d0..91d8b4203 100644 --- a/.gitea/workflows/review-verdict.yml +++ b/.gitea/workflows/review-verdict.yml @@ -125,8 +125,16 @@ jobs: # A transport/parse failure must NOT masquerade as a legitimate short final page. # Empty output counts as zero rows, which would otherwise read as "end of list" and set # complete=yes over a PARTIAL enumeration — failing OPEN at the exact point this guard - # exists to fail closed. Validate the response really is a JSON array before counting. - if ! printf '%s' "$raw" | jq -e 'type == "array"' >/dev/null 2>&1; then + # exists to fail closed. + # + # Validating only the TOP-LEVEL type is not enough: a page like `[{}]` is a well-formed + # array whose rows carry no `filename`, so it contributes no paths, counts as a short + # page, and completes the enumeration from a partial list — the same failure one level + # down. Require every row to carry a non-empty string `filename`; an empty array stays + # valid, since that is what a genuine end-of-pagination looks like. + if ! printf '%s' "$raw" \ + | jq -e 'type == "array" and (map(.filename) | all(type == "string" and length > 0))' \ + >/dev/null 2>&1; then complete=no; break fi # Page-size termination is measured in ROWS; the path set collects BOTH sides of a diff --git a/docs/decisions/records/release/verdict-status-check.md b/docs/decisions/records/release/verdict-status-check.md index 927e180f6..3c6f4404f 100644 --- a/docs/decisions/records/release/verdict-status-check.md +++ b/docs/decisions/records/release/verdict-status-check.md @@ -105,12 +105,20 @@ described as one: 2. **The file list must be paged, and the paging must fail closed.** The endpoint caps at 50 rows and silently ignores a larger `limit` (PR #619: 194 changed files, `?limit=100` returns 50). A single-page read is a silent false negative — on #619 it saw **zero** protected paths where the full - enumeration finds ten. Both call sites now page to exhaustion. Critically, each page is validated as - a JSON array *before* its rows are counted: an errored or non-array page yields zero rows, which - would otherwise read as "end of list" and complete the enumeration from a **partial** list — failing - open at precisely the point the guard exists to fail closed. That defect lives only on the failure - path, so every happy-path run looked correct; it is covered by tests that error a page mid-pagination - rather than only by tests that return data. + enumeration finds ten. Both call sites now page to exhaustion. Critically, each page is validated + *before* its rows are counted, and validated **per row**, not just at the top level: every row must + carry a non-empty string `filename`. An errored page, a non-array body, an array of scalars, and an + array of `filename`-less objects all yield zero usable paths, and any of them would otherwise read as + "end of list" and complete the enumeration from a **partial** list — failing open at precisely the + point the guard exists to fail closed. An empty array stays valid, because that is what a genuine + end-of-pagination looks like. + + This whole class lives only on the *failure* path, so every happy-path run looked correct and the + first test suite was green throughout. It took two rounds to close: the first fix validated only the + top-level type, which left `[{}]` as a working bypass. The tests therefore error and malform pages + mid-pagination rather than only returning data, and include a positive control (an empty final page + must still exempt) so that an over-strict guard — which would make every "withholds the exemption" + test pass for the wrong reason — is also caught. 2b. **Status reads use the combined endpoint.** `GET /statuses/{sha}` returns one row per status *post* (not per context) and pages at 50, so a head with a few CI reruns can push an earlier verdict off the diff --git a/scripts/tests/test_merge_consent_exemption.py b/scripts/tests/test_merge_consent_exemption.py index a3cde25da..c337305dd 100644 --- a/scripts/tests/test_merge_consent_exemption.py +++ b/scripts/tests/test_merge_consent_exemption.py @@ -51,6 +51,7 @@ if "/pulls/" in url and "/files" in url: sys.exit(22) if entry == "GARBAGE": # 200 with a non-array body (proxy/error page) print('{"message":"internal error"}'); sys.exit(0) + # anything else is served verbatim as the page body (rows, [], malformed rows, scalars) print(json.dumps(entry)); sys.exit(0) if "/pulls/" in url: @@ -156,5 +157,45 @@ def test_non_array_body_mid_pagination_withholds_the_exemption(hook): def test_first_page_failure_withholds_the_exemption(hook): + """NOTE this one is deliberately weak on its own — see the test below. + + With page 1 failing, the path list is empty, and the hook independently withholds the + exemption on an empty list. So this passes even with the validation guard removed. It is + kept as a smoke case, but it does NOT pin the guard; `test_malformed_rows_*` does. + """ hook.set_pages("ERROR") assert hook.exempted() is False + + +def test_malformed_rows_mid_pagination_withhold_the_exemption(hook): + """Isolates the row-schema hole: `[{}]` is a valid ARRAY whose rows carry no filename. + + Validating only the top-level type lets this through — it contributes no paths, so it looks + like a short final page and completes the enumeration from a PARTIAL list. The 50 docs rows on + page 1 are what make this non-vacuous: the path list is non-empty, so the exemption would + genuinely fire if `complete` were wrongly set. + """ + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), [{}]) + assert hook.exempted() is False + + +def test_scalar_rows_mid_pagination_withhold_the_exemption(hook): + """An array of scalars must be rejected, not crash the extraction under `set -e`.""" + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), ["unexpected"]) + assert hook.exempted() is False + + +def test_empty_final_page_is_a_legitimate_end_of_pagination(hook): + """Positive control for the guard above: `[]` must still count as a clean end, not a failure. + + Without this, a stricter guard could withhold every exemption and the tests above would still + pass — the suite would be asserting 'never exempt', which is not the contract. + """ + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), []) + assert hook.exempted() is True + + +def test_exceeding_max_pages_withholds_the_exemption(hook): + """41 full pages: enumeration cannot be proven exhaustive, so no exemption.""" + hook.set_pages(*[_rows([f"docs/p{p}f{i}.md" for i in range(50)]) for p in range(41)]) + assert hook.exempted() is False -- 2.47.3 From 07e1e8cfbcbefce715d0039913028cbea43119a4 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sat, 25 Jul 2026 22:43:30 +0200 Subject: [PATCH 5/6] fix(622): validate rename rows too; name the real blocker when only the verdict is pending MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Third BLOCKED verdict, third legitimate finding. The per-row guard validated the DESTINATION only, so `{"filename":"docs/x.md","status":"renamed"}` with no `previous_filename` passed while its source silently vanished — precisely the path-hiding that collecting both sides exists to prevent. A rename row must now carry a non-empty `previous_filename`. Required for rename rows ONLY. Demanding it globally would reject every ordinary modified/added row and make the gate refuse all exemptions — which every "withholds the exemption" test would happily pass through, so that direction gets its own positive control. Predicate checked against ten shapes before adopting; mutation-verified in both directions (dropping the clause fails 2 tests, applying it globally fails 3 including the controls). Also fixes a wart this PR introduced. `review-verdict/h10` is itself folded into the COMBINED status, so a PR awaiting its verdict reports combined 'pending' and the hook's condition (a) reported it as a CI problem — sending a reader to build logs when the missing thing is the review, and exiting before the H10 branch that would have said so. The message now names the outstanding contexts, and says plainly when the verdict is the only one left. Scope boundary, stated in the record: real Gitea populates `previous_filename` on renames and returns well-formed pages. Everything past this point defends against shapes with no evidence of existing, so the guard's claim stays "any page we cannot fully classify withholds the exemption" rather than growing to cover unobserved responses. Decisions-Edit: yes --- .claude/hooks/pretooluse-merge-consent.sh | 15 +++++++-- .gitea/workflows/review-verdict.yml | 2 +- .../records/release/verdict-status-check.md | 21 +++++++++++-- scripts/tests/test_merge_consent_exemption.py | 31 +++++++++++++++++++ 4 files changed, 63 insertions(+), 6 deletions(-) diff --git a/.claude/hooks/pretooluse-merge-consent.sh b/.claude/hooks/pretooluse-merge-consent.sh index bc32862ac..4ceaa0cdf 100755 --- a/.claude/hooks/pretooluse-merge-consent.sh +++ b/.claude/hooks/pretooluse-merge-consent.sh @@ -104,7 +104,7 @@ while [ "$page" -le 40 ]; do # (an empty array is still valid — that is a genuine end-of-pagination). This also rejects arrays # of scalars, which would otherwise make the `.filename` extraction below fail under `set -e`. if ! printf '%s' "$raw" \ - | jq -e 'type == "array" and (map(.filename) | all(type == "string" and length > 0))' \ + | jq -e 'type == "array" and all(.[]; (.filename | type == "string" and length > 0) and (if .status == "renamed" then (.previous_filename | type == "string" and length > 0) else true end))' \ >/dev/null 2>&1; then files_complete=no; break fi @@ -158,7 +158,18 @@ if [ "$mwcs" != "true" ]; then case "$state" in success) : ;; "") decide ask "H6 merge gate: could not read CI status for PR #$pr ($sha). Verify CI is green before merging." ;; - *) decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success'. Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging." ;; + *) + # `review-verdict/h10` is itself one of the contexts folded into the COMBINED state, so a PR + # awaiting its verdict reports combined 'pending' and would otherwise be reported as a CI + # problem — sending the reader to look at build logs when the actual missing thing is the + # review. Name the real blocker when the verdict is the only thing outstanding. + vpending=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100" \ + | jq -r '[.statuses[]? | select(.status == "pending" or .status == "failure")] | map(.context) | join(", ")' 2>/dev/null || true) + if [ "$vpending" = "review-verdict/h10" ]; then + decide deny "H6/H10 merge gate: BLOCKED — every CI check on PR #$pr is green; the only outstanding context is 'review-verdict/h10' on head ${sha:0:7}, i.e. this head has no review verdict yet. Review it and run: scripts/post-review-verdict.sh $pr MERGEABLE" + fi + decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success' (outstanding: ${vpending:-unknown}). Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging." + ;; esac else # --- SCHEDULED auto-merge: everything this hook proves is a SNAPSHOT (ersatztv#622). ---------- diff --git a/.gitea/workflows/review-verdict.yml b/.gitea/workflows/review-verdict.yml index 91d8b4203..69e66a80f 100644 --- a/.gitea/workflows/review-verdict.yml +++ b/.gitea/workflows/review-verdict.yml @@ -133,7 +133,7 @@ jobs: # down. Require every row to carry a non-empty string `filename`; an empty array stays # valid, since that is what a genuine end-of-pagination looks like. if ! printf '%s' "$raw" \ - | jq -e 'type == "array" and (map(.filename) | all(type == "string" and length > 0))' \ + | jq -e 'type == "array" and all(.[]; (.filename | type == "string" and length > 0) and (if .status == "renamed" then (.previous_filename | type == "string" and length > 0) else true end))' \ >/dev/null 2>&1; then complete=no; break fi diff --git a/docs/decisions/records/release/verdict-status-check.md b/docs/decisions/records/release/verdict-status-check.md index 3c6f4404f..614d6706b 100644 --- a/docs/decisions/records/release/verdict-status-check.md +++ b/docs/decisions/records/release/verdict-status-check.md @@ -113,12 +113,27 @@ described as one: point the guard exists to fail closed. An empty array stays valid, because that is what a genuine end-of-pagination looks like. + Rename rows carry one extra requirement: a row whose `status` is `renamed` must also carry a + non-empty `previous_filename`, or the row validates while its SOURCE silently vanishes — which is + exactly the path-hiding that collecting both sides exists to prevent. It is required *only* for + rename rows; demanding it globally would reject every ordinary modified/added row and make the gate + refuse all exemptions, a failure the "withholds the exemption" tests could not distinguish from + working, so there is a positive control for that too. + This whole class lives only on the *failure* path, so every happy-path run looked correct and the first test suite was green throughout. It took two rounds to close: the first fix validated only the top-level type, which left `[{}]` as a working bypass. The tests therefore error and malform pages - mid-pagination rather than only returning data, and include a positive control (an empty final page - must still exempt) so that an over-strict guard — which would make every "withholds the exemption" - test pass for the wrong reason — is also caught. + mid-pagination rather than only returning data, and include positive controls (an empty final page + must still exempt; an ordinary row without `previous_filename` must still be valid) so that an + over-strict guard — which would make every "withholds the exemption" test pass for the wrong + reason — is also caught. + + **Where this hardening stops.** Three review rounds closed: single-page truncation, renames, + failure-path pages, top-level-only validation, and malformed rename rows. Real Gitea populates + `previous_filename` on renames and returns well-formed pages; everything past this point defends + against response shapes with no evidence of existing. The guard's contract is now "any page we + cannot fully classify withholds the exemption", and it is worth keeping that claim narrow and true + rather than extending it to shapes nobody has observed. 2b. **Status reads use the combined endpoint.** `GET /statuses/{sha}` returns one row per status *post* (not per context) and pages at 50, so a head with a few CI reruns can push an earlier verdict off the diff --git a/scripts/tests/test_merge_consent_exemption.py b/scripts/tests/test_merge_consent_exemption.py index c337305dd..ac6bbd9c4 100644 --- a/scripts/tests/test_merge_consent_exemption.py +++ b/scripts/tests/test_merge_consent_exemption.py @@ -195,6 +195,37 @@ def test_empty_final_page_is_a_legitimate_end_of_pagination(hook): assert hook.exempted() is True +def test_malformed_rename_row_withholds_the_exemption(hook): + """A row marked `renamed` with NO `previous_filename` hides its source path. + + The destination alone is `docs/...`, so validating only `filename` would accept the row and + silently drop whatever it was renamed FROM — defeating the reason both sides are collected. + Real Gitea always populates it (verified by constructing a rename), so this is the + malformed-2xx class the guard claims to fail closed on; the claim should match the behaviour. + """ + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), + [{"filename": "docs/moved.md", "status": "renamed"}]) + assert hook.exempted() is False + + +def test_rename_row_with_empty_previous_filename_withholds_the_exemption(hook): + hook.set_pages(_rows([f"docs/f{i}.md" for i in range(50)]), + [{"filename": "docs/moved.md", "status": "renamed", "previous_filename": ""}]) + assert hook.exempted() is False + + +def test_ordinary_row_without_previous_filename_is_still_valid(hook): + """Positive control: `previous_filename` must be required ONLY for rename rows. + + Requiring it globally would reject every normal modified/added row and make the gate refuse + all exemptions — which the 'withholds' tests above could not distinguish from working. + """ + hook.set_pages([{"filename": "docs/a.md", "status": "modified"}, + {"filename": "docs/b.md", "status": "added"}, + {"filename": "docs/c.md"}]) + assert hook.exempted() is True + + def test_exceeding_max_pages_withholds_the_exemption(hook): """41 full pages: enumeration cannot be proven exhaustive, so no exemption.""" hook.set_pages(*[_rows([f"docs/p{p}f{i}.md" for i in range(50)]) for p in range(41)]) -- 2.47.3 From 9f5317bfa55fef0077674fba11b1790acd6d9bb0 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sat, 25 Jul 2026 23:16:30 +0200 Subject: [PATCH 6/6] fix(622): classify every non-success CI state, and word a negative verdict correctly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fourth review round, two Low findings, both in the condition-(a) message change I added last round. Both are message-only — every path still denies — but both would have printed something flatly false at the moment someone is trying to understand why a merge is blocked. 1. The filter selected only `pending` and `failure`, but Gitea also has `error` (and `warning`). With `review-verdict/h10=pending` and a build in `error`, the errored build fell out of the set, leaving the verdict as the lone entry — and the hook then claimed "every CI check is green" while a build was erroring. Non-green is now anything that is not `success`. `skipped` is deliberately still counted as GREEN: the image-push job skips on every PR (#593 — a skipped context is not red), so treating it as non-green would have permanently suppressed the H10-specific message in the one situation it exists for. 2. If the sole non-green context was `review-verdict/h10=failure` — i.e. someone reviewed this head and REJECTED it — the message said no verdict existed yet and told the reader to post MERGEABLE. It now branches on the verdict's own state: pending means nobody has reviewed this head, failure/error means it was reviewed and rejected, so resolve the findings. Also folds the two combined-status fetches into one. Verified against eight payloads: verdict-only-pending, verdict-pending-plus-error, verdict-failure-alone, skipped-plus-verdict-pending, two-pending, a real build failure, a warning state, and an unreadable body — each producing the intended message and no other. Both review channels independently confirmed this round that the rename validation is correct at both call sites, that the tests meaningfully pin it, and that no ALLOW/GRANT path became more permissive. --- .claude/hooks/pretooluse-merge-consent.sh | 34 +++++++++++++++++------ 1 file changed, 25 insertions(+), 9 deletions(-) diff --git a/.claude/hooks/pretooluse-merge-consent.sh b/.claude/hooks/pretooluse-merge-consent.sh index 4ceaa0cdf..cba4cd552 100755 --- a/.claude/hooks/pretooluse-merge-consent.sh +++ b/.claude/hooks/pretooluse-merge-consent.sh @@ -154,21 +154,37 @@ done # --- (a) CI combined status must be green (unless deferring to Gitea's own check-gate). --- if [ "$mwcs" != "true" ]; then [ -n "$sha" ] || decide ask "H6 merge gate: could not resolve PR #$pr head sha to check CI. Verify CI is green before merging." - state=$(gq "repos/$owner/$repo/commits/$sha/status" | jq -r '.state // ""' 2>/dev/null || true) + cistatus=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100") + state=$(printf '%s' "$cistatus" | jq -r '.state // ""' 2>/dev/null || true) case "$state" in success) : ;; "") decide ask "H6 merge gate: could not read CI status for PR #$pr ($sha). Verify CI is green before merging." ;; *) # `review-verdict/h10` is itself one of the contexts folded into the COMBINED state, so a PR # awaiting its verdict reports combined 'pending' and would otherwise be reported as a CI - # problem — sending the reader to look at build logs when the actual missing thing is the - # review. Name the real blocker when the verdict is the only thing outstanding. - vpending=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100" \ - | jq -r '[.statuses[]? | select(.status == "pending" or .status == "failure")] | map(.context) | join(", ")' 2>/dev/null || true) - if [ "$vpending" = "review-verdict/h10" ]; then - decide deny "H6/H10 merge gate: BLOCKED — every CI check on PR #$pr is green; the only outstanding context is 'review-verdict/h10' on head ${sha:0:7}, i.e. this head has no review verdict yet. Review it and run: scripts/post-review-verdict.sh $pr MERGEABLE" - fi - decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success' (outstanding: ${vpending:-unknown}). Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging." + # problem — sending the reader to build logs when the missing thing is the review. Name the + # real blocker when the verdict is the only thing outstanding. + # + # "Not green" is anything that is not `success`, NOT just pending/failure: Gitea also has + # `error` (and `warning`), and omitting those would let an errored build hide behind the + # verdict and produce the flatly false claim "every CI check is green". `skipped` IS treated + # as green — the image-push job skips on every PR (ersatztv#593: a skipped context is not red). + nongreen=$(printf '%s' "$cistatus" \ + | jq -r '[.statuses[]? | select(.status != "success" and .status != "skipped")] + | map("\(.context)=\(.status)") | join(", ")' 2>/dev/null || true) + # The verdict's OWN state decides the wording: absent/pending means nobody has reviewed this + # head, while failure/error means someone reviewed it and said no. Telling a reviewer to "post + # a verdict" when they already posted a BLOCKED one would be actively misleading. + vonly=$(printf '%s' "$cistatus" \ + | jq -r '[.statuses[]? | select(.status != "success" and .status != "skipped")] + | if (length == 1 and .[0].context == "review-verdict/h10") then .[0].status else "" end' 2>/dev/null || true) + case "$vonly" in + pending) + decide deny "H6/H10 merge gate: BLOCKED — every CI check on PR #$pr is green; the only outstanding context is 'review-verdict/h10' on head ${sha:0:7}, i.e. this head has no review verdict yet. Review it and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;; + failure|error) + decide deny "H6/H10 merge gate: BLOCKED — every CI check on PR #$pr is green, but 'review-verdict/h10' is '$vonly' on head ${sha:0:7}: this head was reviewed and REJECTED. Resolve the findings, then run: scripts/post-review-verdict.sh $pr MERGEABLE" ;; + esac + decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success' (not green: ${nongreen:-unknown}). Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging." ;; esac else -- 2.47.3