timothy
5e7623b8d5
Two real defects, and three docs claims that were simply wrong. jq-preflight.sh parsed the version by stripping around the first `-` and `.`, which assumed the format is exactly `jq-X.Y`. A build printing `jq version 1.6` left major empty; the sanity check concatenated major+minor into "6", which is non-empty and all-digits, so it PASSED. The floor comparison then ran `[ "" -lt 1 ]`, which errors — and `set -e` exempts a failing command in an `if` condition, so the conditional read false and the script exited 0 having asserted nothing, after printing a plausible "parsed" line. The silently-untested-axis failure this script exists to eliminate, reproduced inside the script itself. Now parsed by explicit regex, failing closed with a diagnosis when there is no <digits>.<digits> match. Also: `--expect` with no value exited 1 with empty output on both streams. The hook's exit-status check was pinned by nothing: mutating `if files=$(...)` into `files=$(...) || true; files_complete=yes` left the ENTIRE suite green. It survived only by redundancy — the script writes stdout once, right before exit 0, so failures also happen to yield empty stdout and `[ -n "$files" ]` catches it. Safe by accident, which is the exact criticism this branch levels at the old code. Four tests now pin it, with a stub that FAILS while emitting a docs-only list (the one case redundancy cannot absorb) plus a positive control proving the harness can see the difference. Verified: the mutation now turns exactly those tests red. Docs corrections. The record claimed the --expect pin was safe because script-tests is "advisory, not a required check" — false. The merge-consent hook reads the COMBINED status (ci.advisory-red-blocks-the-merge-gate, #598), so firing the tripwire blocks every non-docs-only merge until someone re-pins. Kept anyway, for a stated reason, but no longer described as free. The record also asserted in the present tense that review-verdict.yml checks out the base ref; it has no checkout step at all, so that is now a future-tense requirement on the follow-up. And the documented .status allow-list named GitHub's `removed`, which the code rejects. The drift-guard regex anchored on `?limit=`, so a re-inlined copy written `files?page=1&limit=50` would have walked past it. Decisions-Edit: yes
197 lines
11 KiB
Bash
Executable File
197 lines
11 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Exhaustively enumerate a PR's changed file paths, or fail closed.
|
|
#
|
|
# ersatztv#649. This is the ONE implementation of the security-critical half of the merge gate.
|
|
# It exists because the same logic was written twice — once in `.claude/hooks/pretooluse-merge-consent.sh`
|
|
# (advisory: a failure produces a human prompt) and once in `.gitea/workflows/review-verdict.yml`
|
|
# (ENFORCED: it writes the branch-protection-required `review-verdict/h10` status). The advisory copy
|
|
# accumulated four rounds of hardening (ersatztv#643) that the enforced copy never received, leaving the
|
|
# copy with real authority strictly weaker than the copy without. Two copies of a security predicate
|
|
# drift; one cannot.
|
|
#
|
|
# SCOPE — mechanism, not policy. This script answers exactly one question: "what is the complete set of
|
|
# paths this PR touches, at one head, or can we not tell?" It deliberately does NOT classify the PR.
|
|
# The two callers' allow-lists differ ON PURPOSE and must stay separate:
|
|
# * the hook's docs-only pattern also lets .claude/ .gitea/ .husky/ through, which is safe there only
|
|
# because it falls through to a HUMAN PROMPT;
|
|
# * the workflow's is narrower, because there a match posts a green status with nobody in the loop.
|
|
# Sharing the enumeration fixes the drift; sharing the classification would erase an intended difference.
|
|
#
|
|
# CONTRACT
|
|
# Usage: pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha>
|
|
# stdout: newline-delimited paths, BOTH sides of every rename, no blank lines. May be empty.
|
|
# exit 0 the enumeration is COMPLETE and bound to <expected-head-sha>. stdout is authoritative.
|
|
# exit 1 the enumeration could NOT be completed or verified. stdout is meaningless — the caller
|
|
# MUST fail closed (withhold any exemption). A diagnostic goes to stderr.
|
|
# exit 2 usage error.
|
|
# Callers must treat any non-zero exit as "no exemption". Never read stdout without checking the status.
|
|
#
|
|
# AUTH/TRANSPORT is caller-supplied via env, because the two callers authenticate differently:
|
|
# ETV_GITEA_TOKEN | GITEA_TOKEN -> `Authorization: token`
|
|
# ETV_GITEA_BASICAUTH -> curl -u user:pass
|
|
# ETV_GITEA_URL | GITEA_BASE_URL -> API base; defaults to the homelab Gitea. A value ending in
|
|
# /api/v1 is used as-is, otherwise /api/v1 is appended.
|
|
#
|
|
# jq COMPATIBILITY (ersatztv#648). This runs on the CI runner, which ships **jq 1.6**, while it is
|
|
# authored on Macs shipping 1.8.x. It is therefore written to the 1.6-compatible subset:
|
|
# * never rely on `jq -e`'s exit status over EMPTY input — 1.6 exits 0 where >=1.7 exits 4, which is
|
|
# precisely the fail-open that ersatztv#647 found live in the enforced gate. Emptiness is always
|
|
# checked explicitly in shell FIRST.
|
|
# * never use `contains()` for substring tests — on 1.6 `contains("<NUL>")` is true for every string.
|
|
# * never distinguish a parse error from "no output" by exit code — 1.6 returns 4 for both.
|
|
# See docs/ci-cd.md -> "The jq contract".
|
|
|
|
set -euo pipefail
|
|
|
|
if [ "$#" -ne 4 ]; then
|
|
echo "usage: pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha>" >&2
|
|
exit 2
|
|
fi
|
|
|
|
owner=$1
|
|
repo=$2
|
|
pr=$3
|
|
expected_sha=$4
|
|
|
|
if [ -z "$owner" ] || [ -z "$repo" ] || [ -z "$pr" ] || [ -z "$expected_sha" ]; then
|
|
echo "pr-changed-files: empty owner/repo/pr/sha argument" >&2
|
|
exit 2
|
|
fi
|
|
|
|
base_url="${ETV_GITEA_URL:-${GITEA_BASE_URL:-http://192.168.1.95:3000}}"
|
|
case "$base_url" in
|
|
*/api/v1) : ;;
|
|
*/) base_url="${base_url}api/v1" ;;
|
|
*) base_url="${base_url}/api/v1" ;;
|
|
esac
|
|
|
|
# Empty output on ANY failure, so every caller path treats a transport error the same way. The
|
|
# emptiness is then rejected explicitly below — never inferred from a jq exit code.
|
|
gq() {
|
|
local path="$1"
|
|
if [ -n "${ETV_GITEA_TOKEN:-}" ]; then
|
|
curl -sf -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/$path" 2>/dev/null || true
|
|
elif [ -n "${GITEA_TOKEN:-}" ]; then
|
|
curl -sf -H "Authorization: token $GITEA_TOKEN" "$base_url/$path" 2>/dev/null || true
|
|
elif [ -n "${ETV_GITEA_BASICAUTH:-}" ]; then
|
|
curl -sf -u "$ETV_GITEA_BASICAUTH" "$base_url/$path" 2>/dev/null || true
|
|
else
|
|
printf ''
|
|
fi
|
|
}
|
|
|
|
if [ -z "${ETV_GITEA_TOKEN:-}" ] && [ -z "${GITEA_TOKEN:-}" ] && [ -z "${ETV_GITEA_BASICAUTH:-}" ]; then
|
|
echo "pr-changed-files: no Gitea credentials in env — cannot enumerate, failing closed" >&2
|
|
exit 1
|
|
fi
|
|
|
|
PAGE_SIZE=50
|
|
MAX_PAGES=40 # 2000 files; beyond this we refuse rather than guess
|
|
|
|
files=""
|
|
page=1
|
|
complete=no
|
|
|
|
while [ "$page" -le "$MAX_PAGES" ]; do
|
|
raw=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=${PAGE_SIZE}&page=${page}")
|
|
|
|
# An EMPTY body is rejected in SHELL, before jq sees it. `jq -e` over empty input exits 4 on
|
|
# jq >= 1.7 but 0 on jq 1.6, and the runner ships 1.6 — leaving this to jq's exit status is the
|
|
# exact fail-open ersatztv#647 found in the enforced copy. A transport failure must never
|
|
# masquerade as a legitimate short final page.
|
|
if [ -z "${raw//[[:space:]]/}" ]; then
|
|
echo "pr-changed-files: empty/unreadable response for page ${page}" >&2
|
|
complete=no; break
|
|
fi
|
|
|
|
# VALIDATE EVERY FIELD THE EXTRACTION BELOW CONSUMES, on EVERY row.
|
|
#
|
|
# * Top-level type alone is not enough: `[{}]` is a well-formed array whose rows carry no
|
|
# `filename`, so it contributes no paths, looks like a short page, and would complete the
|
|
# enumeration from a PARTIAL list — the same failure one level down. It also rejects arrays of
|
|
# scalars, which would otherwise make the extraction fail under `set -e`.
|
|
# * CR/LF in a path is rejected outright. `chunk` flattens paths into newline-delimited text, so a
|
|
# filename containing a newline splits into TWO lines matched against the allow-list separately:
|
|
# "safe.md\ndocs/Program.cs" yields `safe.md` and `docs/Program.cs`, both of which pass, while the
|
|
# real single path ends in `.cs`. Git permits newlines in filenames, so this is reachable and was
|
|
# reproduced against the hook.
|
|
# * `previous_filename` is validated on EVERY row, not only `renamed` ones, because `chunk` emits it
|
|
# for every row regardless of `.status`. Validating it only where it is semantically "supposed to"
|
|
# appear left a hole one predicate wide: a `status: "modified"` row carrying a newline in
|
|
# `previous_filename` was reproducibly exempted. The validation domain must match the CONSUMPTION
|
|
# domain.
|
|
# * `..` is rejected because the callers' allow-lists anchor `^docs/`, so `docs/../ErsatzTV/Program.cs`
|
|
# matches one. Git will not produce such a path; this guard's job is to fail closed on unexpected
|
|
# 2xx shapes rather than assume a well-behaved peer.
|
|
# * `.status` is checked against a CLOSED set. Be precise about what this does and does not do:
|
|
# the extraction below emits `(.previous_filename // empty)` UNCONDITIONALLY, so a present
|
|
# `previous_filename` is never dropped on account of `.status`. What the closed set actually buys
|
|
# is rejecting rows whose vocabulary we do not recognise — where a source path may be absent, or
|
|
# carried in some other field we are not reading. Without it, `"Renamed"` with a capital R, or an
|
|
# absent status, silently takes the `else true` branch of the clause below and skips the
|
|
# "renamed rows MUST carry previous_filename" requirement entirely. (An earlier version of this
|
|
# comment claimed the source path would be "dropped", which is not the mechanism; a maintainer
|
|
# who tested that claim would find it false and might conclude the check is redundant.)
|
|
# `modified` is accepted alongside `changed` deliberately: live Gitea 1.25.4 emits `changed`, but a
|
|
# closed allow-list built from the wrong vocabulary is a worse failure than the hole it closes — it
|
|
# would gate every genuine docs-only PR on any version that spells it differently. The property is
|
|
# "reject values we do not recognise", not "enumerate one version exactly".
|
|
if ! printf '%s' "$raw" \
|
|
| jq -e 'def ok: type == "string" and length > 0
|
|
and (test("[\\r\\n]") | not)
|
|
and (split("/") | index("..") | not);
|
|
type == "array" and all(.[];
|
|
(.filename | ok)
|
|
and (.previous_filename == null or (.previous_filename | ok))
|
|
and ((.status // "") as $s | ($s | type) == "string"
|
|
and (["added","deleted","changed","modified","renamed","copied"] | index($s)) != null)
|
|
and (if .status == "renamed"
|
|
then (.previous_filename | type == "string" and length > 0)
|
|
else true end))' \
|
|
>/dev/null 2>&1; then
|
|
echo "pr-changed-files: page ${page} failed row validation" >&2
|
|
complete=no; break
|
|
fi
|
|
|
|
# BOTH sides of a rename: Gitea reports a `git mv` as ONE row whose `filename` is the DESTINATION,
|
|
# with the source only in `previous_filename`. Reading `filename` alone lets a PR move a protected
|
|
# file INTO docs/ and pass as docs-only (verified live: `.gitea/workflows/renovate.yml` ->
|
|
# `docs/innocuous-note.md` showed no protected path). One renamed row is therefore ONE row but TWO
|
|
# paths, which is why the two counts below are computed differently.
|
|
n=$(printf '%s' "$raw" | jq -r 'length')
|
|
chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)')
|
|
[ -n "$chunk" ] && files=$(printf '%s\n%s' "$files" "$chunk")
|
|
|
|
# Terminate ONLY on an explicitly validated EMPTY page — never on a merely SHORT one.
|
|
# "Fewer than 50 rows means last page" assumes the server's page size is the 50 we asked for, but
|
|
# Gitea caps `limit` at the server-wide MAX_RESPONSE_ITEMS (default 50, configurable) and is free to
|
|
# return fewer. A 30-row page followed by a page of code would complete the enumeration over a
|
|
# PARTIAL list — the same fail-open, reached without any transport error. Costs one extra request;
|
|
# the MAX_PAGES cap still fails closed.
|
|
if [ "$n" -eq 0 ]; then complete=yes; break; fi
|
|
page=$((page + 1))
|
|
done
|
|
|
|
if [ "$complete" != yes ]; then
|
|
echo "pr-changed-files: enumeration incomplete (stopped at page ${page}) — failing closed" >&2
|
|
exit 1
|
|
fi
|
|
|
|
# Bind the enumeration to ONE head. Paging is several round-trips; a force-push between them means
|
|
# page 1 came from head A and page 2 from head B, so the assembled list belongs to no single commit —
|
|
# B's code page can be skipped entirely while B's docs page reads as a clean short tail. Re-read the
|
|
# head and refuse if it moved.
|
|
prjson=$(gq "repos/$owner/$repo/pulls/$pr")
|
|
if [ -z "${prjson//[[:space:]]/}" ]; then
|
|
echo "pr-changed-files: could not re-read PR head to bind the enumeration — failing closed" >&2
|
|
exit 1
|
|
fi
|
|
sha_after=$(printf '%s' "$prjson" | jq -r '.head.sha // ""' 2>/dev/null || true)
|
|
if [ -z "$sha_after" ] || [ "$sha_after" != "$expected_sha" ]; then
|
|
echo "pr-changed-files: head moved during enumeration (${expected_sha:0:7} -> ${sha_after:0:7}) — failing closed" >&2
|
|
exit 1
|
|
fi
|
|
|
|
printf '%s\n' "$files" | grep -v '^$' || true
|
|
exit 0
|