ersatztv/docs/decisions/records/ci/grep-q-pipefail-inversion.md

key, title, status, since, supersedes, superseded-by, rule, signals, mechanics

key	title	status	since	supersedes	superseded-by	rule	signals	mechanics
ci.grep-q-pipefail-inversion	2026-07-29 — never feed `grep -q` from a pipe under `set -o pipefail`: SIGPIPE turns a MATCH into a failed pipeline and inverts the guard (#698)	active	2026-07-29	none	none	In any script running under `set -o pipefail`, a security or classification predicate of the form `producer | grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" | grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`.	grep -q pipefail, exit 141, SIGPIPE in a shell guard, large PR classified docs-only, protected path guard skipped, printf pipe grep -q, classification inverts on big input, pipe buffer 64K shell predicate · paths: `.gitea/workflows/review-verdict.yml`, `.claude/hooks/pretooluse-merge-consent.sh`, `scripts/tests/test_pr_changed_files.py` · issues: #698, #649	`count_matching` / `count_not_matching` helpers in `review-verdict.yml`, DEFINED BEFORE FIRST USE, results precomputed into `n_protected`/`n_not_manifest`/`n_not_docs` at top level and validated numeric; regression tests `test_a_LARGE_pr_*` build 1900+ paths (~171KB) to cross the pipe buffer, `test_the_classify_step_runs_without_SHELL_ERRORS` catches a dead guard, `test_a_protected_path_is_rejected_BY_THE_PROTECTED_BRANCH` asserts the branch rather than the outcome

Found by cross-family adversarial review of the #698 PR, then reproduced directly. It is not a defect of that change — the construct shipped with the ORIGINAL docs-only check and was live on main.

The mechanism. grep -q stops reading at its first match. If the writer still has data queued, it gets SIGPIPE and exits 141. pipefail makes the pipeline's status that of the failing member, so a successful MATCH is reported as a FAILED pipeline. Every guard written as if ! producer | grep -q… or if producer | grep -q… therefore flips, and only for large inputs — which is why every existing test missed it: they all used a handful of short paths, far below the buffer.

Both directions were reproduced, at 1901 paths / ~171KB, inside the enumerator's 2000-file cap:

• A.cs followed by 1900 docs paths → docs_only=yes, pipeline status 141. A C# PR classified docs-only and auto-granted review-verdict/h10=success.
• .gitea/workflows/review-verdict.yml followed by 1900 docs paths → the PROTECTED guard MISSES, so a PR editing the gate's own definition skips the protected-path check and falls through to the exemptions.

The second is the serious one: it reaches a green required status on a PR that rewrites the gate, with no retarget, no bot account and no forged status — nothing but a large PR. It is a more direct hole than any of the three routes #698 was filed for.

The here-string was the FIRST fix, and it was wrong. grep -q… <<< "$data" does remove the SIGPIPE, and it shipped for one round. But bash materialises a large here-string through temporary storage, so it fails when temp space is full or unwritable — and because these predicates sit inside if/!, that failure flips them exactly as SIGPIPE did. It did not reproduce on macOS bash 3.2 and did on Linux bash 5.x, which is the environment CI runs; the disagreement is itself the argument for a construct that cannot fail either way. Counting with grep -c uses an ordinary pipe and drains stdin, so neither failure mode exists.

Two follow-on traps, both found only by re-review. First, the helpers were defined AFTER the classification chain that called them, so count_matching was command not found on every run and the PROTECTED branch never fired — while three "protected path" tests stayed green, because a protected path is also not a manifest and not docs-only, so the job reached pending down another route. Second, exit 1 inside those helpers only left the command-substitution SUBSHELL, and since the substitution sat in a conditional, set -e never fired either. Hence the rule: define before use, evaluate once at top level, validate the result is numeric, and fail closed when it is not.

Two testing lessons. When several branches produce the SAME outcome, asserting the outcome cannot tell you which branch ran — assert the discriminator (here the Decision: reason line). And a cheap stderr sweep for command not found / integer expression expected / unbound variable catches a whole family of silently-skipped guards, because each of those makes an if condition merely false while the job exits 0 and posts a plausible status.

The input-size lesson. The whole class was invisible because every test used small inputs. A guard whose behaviour depends on a BUFFER THRESHOLD needs a test that crosses the threshold; otherwise the suite is measuring the wrong regime entirely and full coverage of the small regime proves nothing. The regression tests pair each large-input negative with a large-input POSITIVE control, so "large lists now fail closed" (a merge deadlock) cannot masquerade as a fix.