fix(845): "14 of 36" was a grep LINE count, and alignment ADDS a named failure rather than replacing them
Seventh cold review. Three findings, all in prose I wrote correcting earlier prose.
HIGH — "14 of 36 did" is not a failure count. `grep -c` over pytest output counts LINES, and
`--tb=line` prints each failure twice, so 14 was 7 failures doubled; only 5 distinct tests are
involved, and with truncation disabled the figure moves the other way (31 of 36). Measured:
raw grep -c 14
of which '^E ' rows 7
of which '^/' tb=line rows 7
DISTINCT failing tests 5
A number that changes with pytest's display options is not a fact about the code, so no figure
is given at all now — which is what the same comment already said it was doing about counts.
The trap itself is recorded so one is not added back.
MEDIUM — "ONE named guard failure INSTEAD OF half the suite reddening" asserts a substitution
that does not happen, and contradicted the sentence directly above it. Measured on the aligned
tree: the guard failure appears AND the ~36 unrelated reds still appear, in the same job.
Alignment adds a diagnosis; it replaces nothing. Now "ALONGSIDE".
MEDIUM — the `set -u` sweep was incomplete, and the previous commit message's "gone repo-wide"
was false. Four files still asserted the refuted mechanism. The sharpest was
`scripts/check-review-verdict.sh`, whose comment said "see the sentinel comment in the library"
— and following that pointer landed on the retraction saying the opposite. Corrected there, in
`docs/ci-cd.md` and in `test_review_verdict_vocabulary.py`, all three already touched by this
branch. Every remaining mention in code, tests and docs is now a quotation INSIDE a retraction;
the only standing assertion left is the decision record, which is its own change and stays with
ersatztv#886 (now narrowed to exactly that).
CORRECTION to the previous commit message: it said the retracted wording was "gone repo-wide".
That was true only of the two phrases the round-6 reviewer checked, not of the mechanism. It is
true now for code, tests and docs.
Also: dropped a "for the same reason" back-reference that resolved to the wrong reason (drift,
where the actual reason is falsity), and removed the session narrative from the figure notes —
"a first attempt said 14" is autobiography; the durable form is the trap and its consequence.
Verified again on this tree: full suite 1278 passed / 2 skipped, catalog in sync,
decisions-validate OK, doc-narrative clean.
fixes #845
refs #886
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
This commit is contained in:
+11
-5
@@ -1515,11 +1515,17 @@ nothing. Words are interpolated into a regex, so a `.*` in the positive list wou
|
||||
explicit `BLOCKED` as `positive`; validation restricts every word to `[a-z][a-z-]*`.
|
||||
|
||||
**What enforces that is a data dependency, not the `if` around the validator** — worth knowing
|
||||
before editing either script, because the obvious reading is wrong and the first version of #788
|
||||
shipped the hole. Under `set -u`, an unbound-variable error inside a function aborts the function
|
||||
but not the script, and in an `if ! validate` form neither branch runs, so the `exit 2` is skipped.
|
||||
The reader has no `set -e`, so unvalidated words reached the regex and an explicit `BLOCKED @ head`
|
||||
classified `positive`, exit 0. Validation now sets a sentinel on its last line and the derived views
|
||||
before editing either script, because the first version of #788 shipped the hole. The fail-open is
|
||||
measured: with the negative list written as a scalar and `.*` in the positive list, unvalidated words
|
||||
reached the regex and an explicit `BLOCKED @ head` classified `positive`, exit 0.
|
||||
|
||||
The MECHANISM first recorded for it was wrong, corrected in ersatztv#845 and stated here so it is not
|
||||
re-derived: it held that under `set -u` an unbound-variable error inside a function aborts the
|
||||
function but not the script, so an `if ! validate` form takes neither branch and the `exit 2` is
|
||||
skipped. Measured on bash 3.2.57 and 5.3.15, under `set -u` alone as well as `set -eu` — the
|
||||
distinction that matters, since the reader deliberately runs without `set -e` — the shell EXITS
|
||||
outright, so nothing continues past the gate. That reading holds only inside a command substitution.
|
||||
What the sentinel actually covers is the caller that never runs validation in the first place. Validation now sets a sentinel on its last line and the derived views
|
||||
refuse without it. Likewise a top-level `exit` in a sourced library terminates the sourcing script
|
||||
where it stands, so both consumers probe-source the library in a subshell and require an
|
||||
end-of-file marker before trusting it.
|
||||
|
||||
@@ -252,15 +252,20 @@ same literal with the same anchors OVER THE SAME POPULATION — the whole file
|
||||
reddens CI on the PR that makes it. The population is part of the rule, not an implementation
|
||||
detail: that guard used to scope itself to the classify step's `run:` body, and a second anchored
|
||||
assignment in any other `run:` block then left it green while making the writer refuse every verdict.
|
||||
Measured before it was aligned: the guard green, and about half the writer suite red. Precisely what
|
||||
was missing is worth stating, since the obvious phrasing is wrong: those failures DO carry the
|
||||
library's refusal, which names the count and the variable — 14 of 36 did. What was missing is a
|
||||
failure POINTING AT THE WORKFLOW, because the reds surface under unrelated test names and the cause
|
||||
has to be inferred from a stack of them. Such an edit reaching `main` WOULD be a repo-wide merge outage — no verdict could be
|
||||
Measured before it was aligned: the guard green, and about half the writer suite red. What was
|
||||
missing is worth stating precisely, because the obvious phrasing is wrong twice over. Those failures
|
||||
DO carry the library's refusal, which names the count and the variable. And alignment ADDS a named
|
||||
guard failure rather than replacing them. What was missing is a failure POINTING AT THE WORKFLOW,
|
||||
since the reds surface under unrelated test names and the cause had to be inferred from a stack of
|
||||
them. No figure is given for how many name it, and the trap is worth recording so one is not added
|
||||
back: a `grep -c` over pytest output counts LINES rather than failures and `--tb=line` prints each
|
||||
failure twice, so such a count overstates by 2x, while truncation moves it the other way (7 under the
|
||||
default display, 31 under `--tb=long -vv` on the same run). A number that changes with display
|
||||
options is not a fact about the code. Such an edit reaching `main` WOULD be a repo-wide merge outage — no verdict could be
|
||||
posted and `review-verdict/h10` is required — but the guard is not the only thing standing in the
|
||||
way, and saying so would overstate it: the writer suite reddens too, in the ADVISORY `script-tests`
|
||||
job (the required contexts, measured 2026-08-30, are `Build & test`, `EF migration integrity` and
|
||||
`review-verdict/h10`). What alignment buys is ONE named guard failure pointing at the workflow, instead of half the suite reddening under unrelated
|
||||
`review-verdict/h10`). What alignment buys is ONE named guard failure pointing at the workflow, ALONGSIDE the half-suite of unrelated
|
||||
failures. The route by which the outage is actually reachable is the snapshot boundary below — a
|
||||
verdict posted from a checkout predating the edit — which is why the two paragraphs belong together.
|
||||
|
||||
|
||||
@@ -84,11 +84,19 @@ esac
|
||||
# verdict word is; the one thing it must never do is fall through to `absent`, which the hook reads
|
||||
# as "convention not adopted" and downgrades into an `ask`.
|
||||
#
|
||||
# THE GATE IS THE SENTINEL, NOT THIS `if`. Stated because the obvious reading is wrong: under
|
||||
# `set -u` an unbound-variable error inside a function aborts the function and NOT the script, and
|
||||
# in an `if ! validate` form neither branch is then taken, so the `exit 2` is skipped and execution
|
||||
# continues. That is a measured fail-open, not a hypothetical — see the sentinel comment in the
|
||||
# library. `etv_verdict_alternation` refusing without the sentinel is what actually holds.
|
||||
# THE GATE IS THE SENTINEL, NOT THIS `if`. The fail-open this guards against is measured and real:
|
||||
# with the negative list written as a scalar and `.*` in the positive list, an explicit
|
||||
# `BLOCKED @ <head>` classified `positive`, exit 0, with only a stderr line the calling hook
|
||||
# discards.
|
||||
#
|
||||
# THE MECHANISM ORIGINALLY GIVEN HERE WAS WRONG and is corrected rather than repeated (ersatztv#845).
|
||||
# It said an unbound-variable error inside a function aborts the function and NOT the script, so an
|
||||
# `if ! validate` form takes neither branch and execution continues past the gate. Measured on bash
|
||||
# 3.2.57 and 5.3.15, and with `set -u` ALONE as well as `set -eu` — which matters here, since this
|
||||
# reader deliberately runs without `set -e` — the shell EXITS outright: no branch runs and nothing
|
||||
# after the gate runs. That reading holds only inside a command substitution, which is not this
|
||||
# shape. `etv_verdict_alternation` refusing without the sentinel is what actually holds, and what it
|
||||
# covers is the caller that never runs validation at all.
|
||||
_here=$(CDPATH='' cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) || {
|
||||
printf 'check-review-verdict: could not resolve this script own directory to locate the shared verdict vocabulary\n' >&2; exit 2; }
|
||||
VOCABULARY_LIB="$_here/lib/review-verdict-vocabulary.sh"
|
||||
|
||||
@@ -40,15 +40,19 @@
|
||||
# That guard used to scope itself to the classify step's `run:` body, which is NOT the same tree this
|
||||
# file walks: a second anchored assignment in any other `run:` block left it green while making this
|
||||
# refuse every verdict. Measured before the fix: the guard green, and about half the writer suite red.
|
||||
# BE PRECISE ABOUT WHAT WAS MISSING, because the obvious phrasing is wrong: those failures DO carry
|
||||
# this library's refusal, which names the count and the variable — 14 of 36 did. What was missing is
|
||||
# a failure that POINTS AT THE WORKFLOW: the reds surface under unrelated test names
|
||||
# (`test_positive_verdict_posts_success_on_the_resolved_head`, …), so the cause has to be inferred
|
||||
# from a stack of them. Aligning the populations buys ONE named guard failure instead. Counts are
|
||||
# avoided elsewhere in this comment because they drift as tests are added.
|
||||
# BE PRECISE ABOUT WHAT WAS MISSING, because the obvious phrasing is wrong twice over. Those failures
|
||||
# DO carry this library's refusal, which names the count and the variable. And alignment does not
|
||||
# REPLACE them — it ADDS a named guard failure alongside them. What was missing is a failure that
|
||||
# POINTS AT THE WORKFLOW: the reds surface under unrelated test names
|
||||
# (`test_positive_verdict_posts_success_on_the_resolved_head`, …), so the cause had to be inferred
|
||||
# from a stack of them.
|
||||
#
|
||||
# Stated as "about half" rather than "the whole suite" for the same reason — 36 of 75 at the time of
|
||||
# writing, so "whole" would be false.
|
||||
# NO FIGURE IS GIVEN FOR HOW MANY NAME IT, deliberately, and the trap is worth stating so nobody
|
||||
# adds one back. A `grep -c` over pytest output counts LINES, not failures, and `--tb=line` prints
|
||||
# each failure twice — so that count overstates by 2x. Truncation moves it again: the same run shows
|
||||
# 7 under the default display and 31 under `--tb=long -vv`. A figure that changes with display
|
||||
# options is not a fact about the code. "About half" survives because it is coarse enough to be
|
||||
# robust.
|
||||
#
|
||||
# BE EXACT ABOUT THE CONSEQUENCE, because the obvious phrasing overstates it. Such an edit landing on
|
||||
# `main` WOULD be a repo-wide merge outage — no verdict could be posted, and `review-verdict/h10` is
|
||||
|
||||
@@ -77,10 +77,9 @@ ETV_VERDICT_NEGATIVE_WORDS=(blocked not-mergeable)
|
||||
# validation in the first place: the words are unobtainable unless it ran all the way to its final
|
||||
# line, so a derived view cannot be reached by a path that simply did not call the validator.
|
||||
#
|
||||
# The same refuted mechanism still appears in `release.verdict-vocabulary-shared` and in
|
||||
# `test_review_verdict_vocabulary.py`; correcting a decision record is its own change, tracked as
|
||||
# ersatztv#886. The fail-open that record documents is real and is not in question — only the
|
||||
# mechanism it attributes.
|
||||
# The same refuted mechanism still stands in `release.verdict-vocabulary-shared`; correcting a
|
||||
# decision record is its own change, tracked as ersatztv#886. The fail-open that record documents is
|
||||
# real and is not in question — only the mechanism it attributes.
|
||||
ETV_VERDICT_VOCABULARY_OK=0
|
||||
|
||||
# --- Validation. -------------------------------------------------------------------------------
|
||||
|
||||
@@ -2388,14 +2388,16 @@ def _h10_reviewers_literal():
|
||||
# classify step therefore does not walk the same tree as its subject: a second anchored
|
||||
# assignment in ANY OTHER `run:` block is invisible here while making the writer refuse every
|
||||
# verdict. Measured on this file: injecting one into the last step left this test green and
|
||||
# reddened about half the writer suite. Those failures DO carry the library's refusal, which
|
||||
# names the count and the variable (14 of 36 did) — what they do NOT do is point at the workflow,
|
||||
# since they surface under unrelated test names and the cause must be inferred from a stack of
|
||||
# them. "About half" rather than "the whole suite" because 36 of 75 is not all of it. Such an edit
|
||||
# reddened about half the writer suite (36 of 75). Those failures DO carry the library's
|
||||
# refusal, which names the count and the variable — what they do NOT do is point at the
|
||||
# workflow, since they surface under unrelated test names. No figure is given for how many name
|
||||
# it: `grep -c` over pytest output counts LINES rather than failures, and `--tb=line` prints each
|
||||
# failure twice, so such a count overstates by 2x; truncation moves it the other way. Such an edit
|
||||
# reaching `main` would be a repo-wide merge outage — no verdict could be posted and
|
||||
# `review-verdict/h10` is required — but the honest claim for THIS guard is narrower: the writer
|
||||
# suite already reddens, in the ADVISORY `script-tests` job, so what aligning the populations
|
||||
# buys is ONE named guard failure pointing at the workflow, rather than half the suite reddening.
|
||||
# buys is ONE named guard failure pointing at the workflow, ALONGSIDE those unrelated reds —
|
||||
# it adds a diagnosis, it does not replace them.
|
||||
# The classify step remains what the GATE runs, and
|
||||
# `_classify_step()` is still the right scope for every other assertion about that body.
|
||||
body = WORKFLOW.read_text()
|
||||
|
||||
@@ -318,18 +318,23 @@ def test_the_write_side_maps_the_shipped_words_onto_the_same_states(sandbox, tok
|
||||
assert sandbox.posted_status_state() == state
|
||||
|
||||
|
||||
# --- A control-flow gate can be SKIPPED; the sentinel is what actually holds. --------------------
|
||||
# --- The sentinel, not the `if`, is what holds. ---------------------------------------------------
|
||||
#
|
||||
# Under `set -u`, an unbound-variable error inside a FUNCTION aborts the function but not the
|
||||
# script, and in an `if ! validate` form NEITHER branch is then taken — so the `exit 2` never runs
|
||||
# and execution continues past the gate. `check-review-verdict.sh` runs without `set -e`, so the
|
||||
# words reached the regex unvalidated: with `.*` in the positive list and the negative list written
|
||||
# as a scalar, an explicit `BLOCKED @ <head>` classified `positive`, exit 0. The calling hook
|
||||
# discards stderr, so the only signal was invisible.
|
||||
# The fail-open is measured and real: with `.*` in the positive list and the negative list written as
|
||||
# a scalar, unvalidated words reached the regex and an explicit `BLOCKED @ <head>` classified
|
||||
# `positive`, exit 0. The calling hook discards stderr, so the only signal was invisible.
|
||||
#
|
||||
# THE MECHANISM ORIGINALLY RECORDED FOR IT WAS WRONG, corrected in ersatztv#845 and restated here
|
||||
# rather than repeated: it held that under `set -u` an unbound-variable error inside a FUNCTION
|
||||
# aborts the function but not the script, so an `if ! validate` form takes NEITHER branch and the
|
||||
# `exit 2` never runs. Measured on bash 3.2.57 and 5.3.15, under `set -u` ALONE as well as `set -eu`
|
||||
# — the case that matters, since `check-review-verdict.sh` deliberately runs without `set -e` — the
|
||||
# shell EXITS outright, so nothing continues past the gate. That reading holds only inside a command
|
||||
# substitution.
|
||||
#
|
||||
# The fix is a DATA dependency rather than a control-flow one: validation sets
|
||||
# `ETV_VERDICT_VOCABULARY_OK=1` on its last line, and the derived views refuse without it. These
|
||||
# tests pin the malformation classes that produce the abort.
|
||||
# `ETV_VERDICT_VOCABULARY_OK=1` on its last line, and the derived views refuse without it. What that
|
||||
# covers is the caller that never runs validation at all. These tests pin the malformation classes.
|
||||
|
||||
|
||||
def test_a_SCALAR_word_list_cannot_reach_the_regex_unvalidated(sandbox):
|
||||
|
||||
Reference in New Issue
Block a user