fix(845): "14 of 36" was a grep LINE count, and alignment ADDS a named failure rather than replacing them

Seventh cold review. Three findings, all in prose I wrote correcting earlier prose.

HIGH — "14 of 36 did" is not a failure count. `grep -c` over pytest output counts LINES, and
`--tb=line` prints each failure twice, so 14 was 7 failures doubled; only 5 distinct tests are
involved, and with truncation disabled the figure moves the other way (31 of 36). Measured:

  raw grep -c                    14
    of which '^E ' rows           7
    of which '^/' tb=line rows    7
  DISTINCT failing tests          5

A number that changes with pytest's display options is not a fact about the code, so no figure
is given at all now — which is what the same comment already said it was doing about counts.
The trap itself is recorded so one is not added back.

MEDIUM — "ONE named guard failure INSTEAD OF half the suite reddening" asserts a substitution
that does not happen, and contradicted the sentence directly above it. Measured on the aligned
tree: the guard failure appears AND the ~36 unrelated reds still appear, in the same job.
Alignment adds a diagnosis; it replaces nothing. Now "ALONGSIDE".

MEDIUM — the `set -u` sweep was incomplete, and the previous commit message's "gone repo-wide"
was false. Four files still asserted the refuted mechanism. The sharpest was
`scripts/check-review-verdict.sh`, whose comment said "see the sentinel comment in the library"
— and following that pointer landed on the retraction saying the opposite. Corrected there, in
`docs/ci-cd.md` and in `test_review_verdict_vocabulary.py`, all three already touched by this
branch. Every remaining mention in code, tests and docs is now a quotation INSIDE a retraction;
the only standing assertion left is the decision record, which is its own change and stays with
ersatztv#886 (now narrowed to exactly that).

CORRECTION to the previous commit message: it said the retracted wording was "gone repo-wide".
That was true only of the two phrases the round-6 reviewer checked, not of the mechanism. It is
true now for code, tests and docs.

Also: dropped a "for the same reason" back-reference that resolved to the wrong reason (drift,
where the actual reason is falsity), and removed the session narrative from the figure notes —
"a first attempt said 14" is autobiography; the durable form is the trap and its consequence.

Verified again on this tree: full suite 1278 passed / 2 skipped, catalog in sync,
decisions-validate OK, doc-narrative clean.

fixes #845
refs #886

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
This commit is contained in:
2026-08-30 03:38:47 +02:00
co-authored by Claude Opus 5
parent 06873a0113
commit 3bbe4f0635
7 changed files with 71 additions and 42 deletions
+11 -5
View File
@@ -1515,11 +1515,17 @@ nothing. Words are interpolated into a regex, so a `.*` in the positive list wou
explicit `BLOCKED` as `positive`; validation restricts every word to `[a-z][a-z-]*`.
**What enforces that is a data dependency, not the `if` around the validator** — worth knowing
before editing either script, because the obvious reading is wrong and the first version of #788
shipped the hole. Under `set -u`, an unbound-variable error inside a function aborts the function
but not the script, and in an `if ! validate` form neither branch runs, so the `exit 2` is skipped.
The reader has no `set -e`, so unvalidated words reached the regex and an explicit `BLOCKED @ head`
classified `positive`, exit 0. Validation now sets a sentinel on its last line and the derived views
before editing either script, because the first version of #788 shipped the hole. The fail-open is
measured: with the negative list written as a scalar and `.*` in the positive list, unvalidated words
reached the regex and an explicit `BLOCKED @ head` classified `positive`, exit 0.
The MECHANISM first recorded for it was wrong, corrected in ersatztv#845 and stated here so it is not
re-derived: it held that under `set -u` an unbound-variable error inside a function aborts the
function but not the script, so an `if ! validate` form takes neither branch and the `exit 2` is
skipped. Measured on bash 3.2.57 and 5.3.15, under `set -u` alone as well as `set -eu` — the
distinction that matters, since the reader deliberately runs without `set -e` — the shell EXITS
outright, so nothing continues past the gate. That reading holds only inside a command substitution.
What the sentinel actually covers is the caller that never runs validation in the first place. Validation now sets a sentinel on its last line and the derived views
refuse without it. Likewise a top-level `exit` in a sourced library terminates the sourcing script
where it stands, so both consumers probe-source the library in a subshell and require an
end-of-file marker before trusting it.
@@ -252,15 +252,20 @@ same literal with the same anchors OVER THE SAME POPULATION — the whole file
reddens CI on the PR that makes it. The population is part of the rule, not an implementation
detail: that guard used to scope itself to the classify step's `run:` body, and a second anchored
assignment in any other `run:` block then left it green while making the writer refuse every verdict.
Measured before it was aligned: the guard green, and about half the writer suite red. Precisely what
was missing is worth stating, since the obvious phrasing is wrong: those failures DO carry the
library's refusal, which names the count and the variable — 14 of 36 did. What was missing is a
failure POINTING AT THE WORKFLOW, because the reds surface under unrelated test names and the cause
has to be inferred from a stack of them. Such an edit reaching `main` WOULD be a repo-wide merge outage — no verdict could be
Measured before it was aligned: the guard green, and about half the writer suite red. What was
missing is worth stating precisely, because the obvious phrasing is wrong twice over. Those failures
DO carry the library's refusal, which names the count and the variable. And alignment ADDS a named
guard failure rather than replacing them. What was missing is a failure POINTING AT THE WORKFLOW,
since the reds surface under unrelated test names and the cause had to be inferred from a stack of
them. No figure is given for how many name it, and the trap is worth recording so one is not added
back: a `grep -c` over pytest output counts LINES rather than failures and `--tb=line` prints each
failure twice, so such a count overstates by 2x, while truncation moves it the other way (7 under the
default display, 31 under `--tb=long -vv` on the same run). A number that changes with display
options is not a fact about the code. Such an edit reaching `main` WOULD be a repo-wide merge outage — no verdict could be
posted and `review-verdict/h10` is required — but the guard is not the only thing standing in the
way, and saying so would overstate it: the writer suite reddens too, in the ADVISORY `script-tests`
job (the required contexts, measured 2026-08-30, are `Build & test`, `EF migration integrity` and
`review-verdict/h10`). What alignment buys is ONE named guard failure pointing at the workflow, instead of half the suite reddening under unrelated
`review-verdict/h10`). What alignment buys is ONE named guard failure pointing at the workflow, ALONGSIDE the half-suite of unrelated
failures. The route by which the outage is actually reachable is the snapshot boundary below — a
verdict posted from a checkout predating the edit — which is why the two paragraphs belong together.
+13 -5
View File
@@ -84,11 +84,19 @@ esac
# verdict word is; the one thing it must never do is fall through to `absent`, which the hook reads
# as "convention not adopted" and downgrades into an `ask`.
#
# THE GATE IS THE SENTINEL, NOT THIS `if`. Stated because the obvious reading is wrong: under
# `set -u` an unbound-variable error inside a function aborts the function and NOT the script, and
# in an `if ! validate` form neither branch is then taken, so the `exit 2` is skipped and execution
# continues. That is a measured fail-open, not a hypothetical — see the sentinel comment in the
# library. `etv_verdict_alternation` refusing without the sentinel is what actually holds.
# THE GATE IS THE SENTINEL, NOT THIS `if`. The fail-open this guards against is measured and real:
# with the negative list written as a scalar and `.*` in the positive list, an explicit
# `BLOCKED @ <head>` classified `positive`, exit 0, with only a stderr line the calling hook
# discards.
#
# THE MECHANISM ORIGINALLY GIVEN HERE WAS WRONG and is corrected rather than repeated (ersatztv#845).
# It said an unbound-variable error inside a function aborts the function and NOT the script, so an
# `if ! validate` form takes neither branch and execution continues past the gate. Measured on bash
# 3.2.57 and 5.3.15, and with `set -u` ALONE as well as `set -eu` — which matters here, since this
# reader deliberately runs without `set -e` — the shell EXITS outright: no branch runs and nothing
# after the gate runs. That reading holds only inside a command substitution, which is not this
# shape. `etv_verdict_alternation` refusing without the sentinel is what actually holds, and what it
# covers is the caller that never runs validation at all.
_here=$(CDPATH='' cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) || {
printf 'check-review-verdict: could not resolve this script own directory to locate the shared verdict vocabulary\n' >&2; exit 2; }
VOCABULARY_LIB="$_here/lib/review-verdict-vocabulary.sh"
+12 -8
View File
@@ -40,15 +40,19 @@
# That guard used to scope itself to the classify step's `run:` body, which is NOT the same tree this
# file walks: a second anchored assignment in any other `run:` block left it green while making this
# refuse every verdict. Measured before the fix: the guard green, and about half the writer suite red.
# BE PRECISE ABOUT WHAT WAS MISSING, because the obvious phrasing is wrong: those failures DO carry
# this library's refusal, which names the count and the variable — 14 of 36 did. What was missing is
# a failure that POINTS AT THE WORKFLOW: the reds surface under unrelated test names
# (`test_positive_verdict_posts_success_on_the_resolved_head`, …), so the cause has to be inferred
# from a stack of them. Aligning the populations buys ONE named guard failure instead. Counts are
# avoided elsewhere in this comment because they drift as tests are added.
# BE PRECISE ABOUT WHAT WAS MISSING, because the obvious phrasing is wrong twice over. Those failures
# DO carry this library's refusal, which names the count and the variable. And alignment does not
# REPLACE them — it ADDS a named guard failure alongside them. What was missing is a failure that
# POINTS AT THE WORKFLOW: the reds surface under unrelated test names
# (`test_positive_verdict_posts_success_on_the_resolved_head`, …), so the cause had to be inferred
# from a stack of them.
#
# Stated as "about half" rather than "the whole suite" for the same reason — 36 of 75 at the time of
# writing, so "whole" would be false.
# NO FIGURE IS GIVEN FOR HOW MANY NAME IT, deliberately, and the trap is worth stating so nobody
# adds one back. A `grep -c` over pytest output counts LINES, not failures, and `--tb=line` prints
# each failure twice — so that count overstates by 2x. Truncation moves it again: the same run shows
# 7 under the default display and 31 under `--tb=long -vv`. A figure that changes with display
# options is not a fact about the code. "About half" survives because it is coarse enough to be
# robust.
#
# BE EXACT ABOUT THE CONSEQUENCE, because the obvious phrasing overstates it. Such an edit landing on
# `main` WOULD be a repo-wide merge outage — no verdict could be posted, and `review-verdict/h10` is
+3 -4
View File
@@ -77,10 +77,9 @@ ETV_VERDICT_NEGATIVE_WORDS=(blocked not-mergeable)
# validation in the first place: the words are unobtainable unless it ran all the way to its final
# line, so a derived view cannot be reached by a path that simply did not call the validator.
#
# The same refuted mechanism still appears in `release.verdict-vocabulary-shared` and in
# `test_review_verdict_vocabulary.py`; correcting a decision record is its own change, tracked as
# ersatztv#886. The fail-open that record documents is real and is not in question — only the
# mechanism it attributes.
# The same refuted mechanism still stands in `release.verdict-vocabulary-shared`; correcting a
# decision record is its own change, tracked as ersatztv#886. The fail-open that record documents is
# real and is not in question — only the mechanism it attributes.
ETV_VERDICT_VOCABULARY_OK=0
# --- Validation. -------------------------------------------------------------------------------
+7 -5
View File
@@ -2388,14 +2388,16 @@ def _h10_reviewers_literal():
# classify step therefore does not walk the same tree as its subject: a second anchored
# assignment in ANY OTHER `run:` block is invisible here while making the writer refuse every
# verdict. Measured on this file: injecting one into the last step left this test green and
# reddened about half the writer suite. Those failures DO carry the library's refusal, which
# names the count and the variable (14 of 36 did) — what they do NOT do is point at the workflow,
# since they surface under unrelated test names and the cause must be inferred from a stack of
# them. "About half" rather than "the whole suite" because 36 of 75 is not all of it. Such an edit
# reddened about half the writer suite (36 of 75). Those failures DO carry the library's
# refusal, which names the count and the variable — what they do NOT do is point at the
# workflow, since they surface under unrelated test names. No figure is given for how many name
# it: `grep -c` over pytest output counts LINES rather than failures, and `--tb=line` prints each
# failure twice, so such a count overstates by 2x; truncation moves it the other way. Such an edit
# reaching `main` would be a repo-wide merge outage — no verdict could be posted and
# `review-verdict/h10` is required — but the honest claim for THIS guard is narrower: the writer
# suite already reddens, in the ADVISORY `script-tests` job, so what aligning the populations
# buys is ONE named guard failure pointing at the workflow, rather than half the suite reddening.
# buys is ONE named guard failure pointing at the workflow, ALONGSIDE those unrelated reds —
# it adds a diagnosis, it does not replace them.
# The classify step remains what the GATE runs, and
# `_classify_step()` is still the right scope for every other assertion about that body.
body = WORKFLOW.read_text()
@@ -318,18 +318,23 @@ def test_the_write_side_maps_the_shipped_words_onto_the_same_states(sandbox, tok
assert sandbox.posted_status_state() == state
# --- A control-flow gate can be SKIPPED; the sentinel is what actually holds. --------------------
# --- The sentinel, not the `if`, is what holds. ---------------------------------------------------
#
# Under `set -u`, an unbound-variable error inside a FUNCTION aborts the function but not the
# script, and in an `if ! validate` form NEITHER branch is then taken — so the `exit 2` never runs
# and execution continues past the gate. `check-review-verdict.sh` runs without `set -e`, so the
# words reached the regex unvalidated: with `.*` in the positive list and the negative list written
# as a scalar, an explicit `BLOCKED @ <head>` classified `positive`, exit 0. The calling hook
# discards stderr, so the only signal was invisible.
# The fail-open is measured and real: with `.*` in the positive list and the negative list written as
# a scalar, unvalidated words reached the regex and an explicit `BLOCKED @ <head>` classified
# `positive`, exit 0. The calling hook discards stderr, so the only signal was invisible.
#
# THE MECHANISM ORIGINALLY RECORDED FOR IT WAS WRONG, corrected in ersatztv#845 and restated here
# rather than repeated: it held that under `set -u` an unbound-variable error inside a FUNCTION
# aborts the function but not the script, so an `if ! validate` form takes NEITHER branch and the
# `exit 2` never runs. Measured on bash 3.2.57 and 5.3.15, under `set -u` ALONE as well as `set -eu`
# — the case that matters, since `check-review-verdict.sh` deliberately runs without `set -e` — the
# shell EXITS outright, so nothing continues past the gate. That reading holds only inside a command
# substitution.
#
# The fix is a DATA dependency rather than a control-flow one: validation sets
# `ETV_VERDICT_VOCABULARY_OK=1` on its last line, and the derived views refuse without it. These
# tests pin the malformation classes that produce the abort.
# `ETV_VERDICT_VOCABULARY_OK=1` on its last line, and the derived views refuse without it. What that
# covers is the caller that never runs validation at all. These tests pin the malformation classes.
def test_a_SCALAR_word_list_cannot_reach_the_regex_unvalidated(sandbox):