feat(774,775): one rule for guard populations, one for guard proofs — both enforced #791

Merged

timothy merged 11 commits from feat/774-775-guard-conventions into main 2026-08-14 01:17:58 +02:00

Conversation 2 Commits 11 Files Changed 10

+1128 -5

timothy commented 2026-08-13 23:20:24 +02:00

Owner

Copy Link

Copy Source

Closes the two highest-priority follow-ups from #773's root-cause analysis: the largest recorded
defect family (reasoning about a representative instead of the population, 39% of process-failure
records) and the most common one (a check that never ran, 25%). Both rules had been reinvented
repeatedly and written down nowhere.

What lands

Two decision records. testing.guard-derives-population-from-source (#774) — a guard enumerates
its population from a machine-readable authoritative source and asserts set equality both ways. It
states the boundary that keeps it usable: filtering to select the SUBJECT of a per-member property
is fine; filtering the population before a COMPLETENESS claim is the defect. It also separates guard
SCOPE (a reviewable policy choice) from guard POPULATION (always derived).
testing.guard-ships-with-mutation-proof (#775) — disarm that clause alone and a named test must go
red, with one explicitly bounded exception for guards that ARE tests.

Two real defects found by the audit and fixed.

1. pr-checks.yml:ci-image-pin stated an invariant it did not check. Its error text says "Every
   container: job must pin ersatztv-ci:<7-char-sha>"; what it asserts is that
   grep … | sort -u yields one DISTINCT value. Distinctness is a property of the pins present,
   so deleting the container: block from test leaves four pins, one distinct value, guard green
   — and a required context silently running on the bare runner instead of the toolchain image.
   test_ci_image_pin_population.py adds the population check.
2. The H10 verdict vocabulary is written down twice with no cross-check, and the write side's
   comment pointing at pretooluse-merge-consent.sh was stale (the hook carries no copy; it
   delegates). Comment corrected. The attempted test for it was withdrawn — see below.

A machine-checked guard inventory. docs/guard-inventory.md classifies every guard file;
test_guard_inventory.py derives the population, asserts set equality both ways, checks hook
wiring (not mere existence), and resolves every claimed proof ref to a real def.
31 guards / 4 tooling / 12 proof — 4 MUTATION, 6 BEHAVIOUR-ONLY, 21 NONE.

Both "is mechanical enforcement possible?" questions answered explicitly. No to a
filter-shaped-guard lint: the token is not the defect (ToolCatalogTests filters correctly eight
lines from a completeness assertion that must not), and it would be a string predicate over source.
Yes to enforcing the bookkeeping, which is what the inventory guard does.

One deliberate withdrawal

A test_review_verdict_vocabulary_parity.py compared the two verdict vocabularies by extracting
each from shell source with regexes. Six cold-review rounds each found another shell construction
that either escaped it or made it red on a correct tree. It was deleted rather than patched a
seventh time — the #774 record says the answer to a missing authoritative source is to CREATE one,
and detector C says two copies get deduped, not compared. The duplication is therefore currently
unmitigated, disclosed in the script, the inventory, the record and #788.

Its separable disjointness invariant was rescued behaviourally (executing the real classifier
instead of parsing it) in test_check_review_verdict.py, after cold review caught that deleting the
file took it along silently.

Corrections to commit messages, which are immutable

• 0bd59b0b6 says "Measured: 28 guards, 4 tooling … 6 mutation-proved, 3 behaviour-only, 19
  unproven." Those figures were wrong on arrival (the table held 27/5/6/3/18) and have since
  changed again. The current figures are 31/4/12 and 4/6/21, and they are now asserted against
  the table by test_the_summary_counts_match_the_table rather than hand-maintained.
• 0bd59b0b6 also says "the most common is a check that never ran at all (25%)". The 25% is
  vacuous verification INCLUDING sampling errors; "never ran at all" is Family B at 13%. The record
  body states it correctly.

Review

Seven cold rounds (Codex GPT-5.6 cross-family, plus one Fable pass on the patch). Every round found
something; the change repeatedly caught its own implementation, which is now the central argument in
the #775 record for mechanising the bookkeeping rather than trusting care.

Verification

579 script-tests pass, ruff clean, pyright 0 errors, decisions-validate OK, catalog
regenerated. Every new guard was mutation-proved by execution before being believed.

Follow-ups filed

#785 (mutation proofs for the unproven guards), #786 (extend the inventory to workflow-job guards),
#787 (MARKED_JOBS is a dated comment, not a check, against branch protection), #788 (one shared
verdict vocabulary — now un-mitigated, raised priority), #789 (workflow-owned execution-class
metadata), #790 (executable clause-level mutation harness).

fixes #774
fixes #775

Closes the two highest-priority follow-ups from #773's root-cause analysis: the largest recorded defect family (reasoning about a representative instead of the population, 39% of process-failure records) and the most common one (a check that never ran, 25%). Both rules had been reinvented repeatedly and written down nowhere. ## What lands **Two decision records.** `testing.guard-derives-population-from-source` (#774) — a guard enumerates its population from a machine-readable authoritative source and asserts set equality both ways. It states the boundary that keeps it usable: filtering to select the SUBJECT of a per-member property is fine; filtering the population before a COMPLETENESS claim is the defect. It also separates guard SCOPE (a reviewable policy choice) from guard POPULATION (always derived). `testing.guard-ships-with-mutation-proof` (#775) — disarm that clause alone and a named test must go red, with one explicitly bounded exception for guards that ARE tests. **Two real defects found by the audit and fixed.** 1. `pr-checks.yml:ci-image-pin` stated an invariant it did not check. Its error text says *"Every container: job must pin ersatztv-ci:&lt;7-char-sha&gt;"*; what it asserts is that `grep … | sort -u` yields one DISTINCT value. Distinctness is a property of the pins *present*, so deleting the `container:` block from `test` leaves four pins, one distinct value, guard green — and a **required** context silently running on the bare runner instead of the toolchain image. `test_ci_image_pin_population.py` adds the population check. 2. The H10 verdict vocabulary is written down twice with no cross-check, and the write side's comment pointing at `pretooluse-merge-consent.sh` was stale (the hook carries no copy; it delegates). Comment corrected. The attempted test for it was withdrawn — see below. **A machine-checked guard inventory.** `docs/guard-inventory.md` classifies every guard file; `test_guard_inventory.py` derives the population, asserts set equality both ways, checks hook *wiring* (not mere existence), and resolves every claimed proof ref to a real `def`. **31 guards / 4 tooling / 12 proof — 4 MUTATION, 6 BEHAVIOUR-ONLY, 21 NONE.** **Both "is mechanical enforcement possible?" questions answered explicitly.** No to a filter-shaped-guard lint: the token is not the defect (`ToolCatalogTests` filters *correctly* eight lines from a completeness assertion that must not), and it would be a string predicate over source. Yes to enforcing the bookkeeping, which is what the inventory guard does. ## One deliberate withdrawal A `test_review_verdict_vocabulary_parity.py` compared the two verdict vocabularies by extracting each from shell source with regexes. **Six cold-review rounds each found another shell construction** that either escaped it or made it red on a correct tree. It was deleted rather than patched a seventh time — the #774 record says the answer to a missing authoritative source is to CREATE one, and detector C says two copies get deduped, not compared. **The duplication is therefore currently unmitigated**, disclosed in the script, the inventory, the record and #788. Its separable disjointness invariant was rescued behaviourally (executing the real classifier instead of parsing it) in `test_check_review_verdict.py`, after cold review caught that deleting the file took it along silently. ## Corrections to commit messages, which are immutable - `0bd59b0b6` says *"Measured: 28 guards, 4 tooling … 6 mutation-proved, 3 behaviour-only, 19 unproven."* Those figures were wrong on arrival (the table held 27/5/6/3/18) and have since changed again. **The current figures are 31/4/12 and 4/6/21**, and they are now asserted against the table by `test_the_summary_counts_match_the_table` rather than hand-maintained. - `0bd59b0b6` also says *"the most common is a check that never ran at all (25%)"*. The 25% is vacuous verification INCLUDING sampling errors; "never ran at all" is Family B at 13%. The record body states it correctly. ## Review Seven cold rounds (Codex GPT-5.6 cross-family, plus one Fable pass on the patch). Every round found something; the change repeatedly caught its own implementation, which is now the central argument in the #775 record for mechanising the bookkeeping rather than trusting care. ## Verification `579` script-tests pass, `ruff` clean, `pyright` 0 errors, `decisions-validate` OK, catalog regenerated. Every new guard was mutation-proved by execution before being believed. ## Follow-ups filed #785 (mutation proofs for the unproven guards), #786 (extend the inventory to workflow-job guards), #787 (`MARKED_JOBS` is a dated comment, not a check, against branch protection), #788 (one shared verdict vocabulary — now un-mitigated, raised priority), #789 (workflow-owned execution-class metadata), #790 (executable clause-level mutation harness). fixes #774 fixes #775

timothy added 9 commits 2026-08-13 23:20:24 +02:00

feat(774,775): one rule for guard populations, one for guard proofs — both enforced ... 0bd59b0b6e

#773's analysis found that the largest recorded failure family is reasoning about a
representative instead of the population (39% of process-failure records), and that the
most common is a check that never ran at all (25%). Both rules had been reinvented
repeatedly and written down nowhere.

Two decision records:

  testing.guard-derives-population-from-source (#774) — a guard enumerates its population
  from a machine-readable authoritative source and asserts set equality both ways. States
  the boundary that keeps it honest: filtering to select the SUBJECT of a per-member
  property is fine; filtering the population before a COMPLETENESS claim is the defect.
  Also separates guard SCOPE (a reviewable policy choice) from guard POPULATION (always
  derived).

  testing.guard-ships-with-mutation-proof (#775) — disarm that clause alone and a named
  test must go red. Behaviour-only coverage is graded separately, because it proves the
  guard reacts, never that it is connected.

Audit findings fixed:

  ci-image-pin stated an invariant it did not check. Its error text says "Every container:
  job must pin ersatztv-ci:<7-char-sha>"; what it asserts is that `grep … | sort -u` yields
  one DISTINCT value. Distinctness is a property of the pins present, so deleting the
  container: block from `test` leaves four pins, one distinct value, and a REQUIRED context
  silently running on the bare runner. test_ci_image_pin_population.py adds the population
  check, keyed on a reviewed registry cross-checked both ways — set equality between two
  DERIVED sets could not see this, because both sides shrink together.

  The verdict vocabulary was written down twice with no cross-check — post-review-verdict.sh
  (write) and check-review-verdict.sh (read). A word in one and not the other sends the
  required status green while the hook still denies. Both vocabularies are now extracted
  from their own source and compared as sets; a test that restated the words would just be
  a third copy. The write side's comment pointing at pretooluse-merge-consent.sh was also
  stale — the hook carries no copy and delegates.

Mechanical enforcement, answered explicitly for both:

  No to a filter-shaped-guard lint. The token is not the defect — ToolCatalogTests filters
  correctly eight lines from a completeness assertion that must not — and it would be a
  string predicate over source, which this repo's record says takes 3+ rounds. Building it
  would be #774 violating #774.

  Yes to enforcing the bookkeeping. docs/guard-inventory.md classifies all 32 guard files;
  test_guard_inventory.py derives the population from the filesystem and call sites,
  asserts set equality both ways, and resolves every claimed proof ref to a real def. A new
  guard cannot ship unclassified; a renamed test cannot leave a row claiming lost coverage.
  What it does NOT check — whether a MUTATION claim is true — is stated, not implied.

Measured: 28 guards, 4 tooling. 6 mutation-proved, 3 behaviour-only, 19 unproven.

Every guard added here was mutation-proved by execution before being believed: neutering
pin_population_faults turned 20 of 25 red; the inventory guard was driven red three ways
(deleted row, new unclassified hook, stale proof ref) and restored green.

573 script-tests pass. Scope limit stated in the doc: inline workflow-job guards are not
in the machine-checked population.

Refs #774
Refs #775

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(774,775): close the cold-review findings — including three the change inflicted on itself ... 3473a6c889

Two independent cold reviews (Codex GPT-5.6 cross-family; Fable 5 on the patch) both
returned BLOCKED. They agreed on the counts error and the extractor hole; each found
things the other did not. Fixes, with what each was:

THE INVENTORY DID NOT COVER ITS OWN NEW GUARDS. `_SCRIPT_REF` matched `scripts/name.py`
but not `scripts/tests/*.py`, so the three guard files this change introduced had no rows
and the completeness check stayed green. A completeness guard blind to its author's new
guards is precisely the defect being legislated against. The population now globs
`scripts/tests/test_*.py` — which is how they actually run, since pr-checks.yml invokes
the directory. 32 rows -> 48.

That forced a third Kind. Once test files are in the population, every mutation proof
becomes a row wanting a proof of its own, forever. `PROOF` marks a file whose job is to
prove another guard; a scripts/tests file enforcing a repo invariant with no separate
guard behind it stays GUARD and may cite a mutation case in its own file.

HOOK EXISTENCE WAS STANDING IN FOR HOOK WIRING. Deleting a hook's registration from
.claude/settings.json left the population and the table unchanged, so the row went on
describing a guard that no longer ran — #631's shape one level down. Now derived from
settings.json plus the husky hooks.

THE SUMMARY COUNTS WERE A HAND-KEPT MIRROR AND WERE WRONG ON ARRIVAL: "28 guards, 4
tooling ... 19 have none" against a table holding 27/5/6/3/18. Both reviewers found it
independently. The prose is now parsed and asserted against the table.

TWO FALSE MUTATION GRADES, each with a concrete disarm:
  - test_full_first_page_alone_does_not_end_enumeration sends 50 docs paths then one more
    docs path; disarm pagination to treat a full page as final and it is still all-docs,
    still exempt, still green. Re-pointed at test_protected_path_on_a_LATER_page_is_still_seen,
    which does go red under that mutation.
  - test_the_scan_job_runs_the_out_of_pytest_positive_control asserts only that the script
    exists, is executable, is referenced and is marked; replace its logic with `exit 0` and
    all four pass. ci-prove-ban-detects.sh regraded NONE.

The MUTATION column was also being applied as a curve: three rows graded MUTATION fed the
real script an input only that clause rejects, which is what the rows eight lines away are
graded BEHAVIOUR-ONLY for. Definition sharpened to *witnessed* rather than plausible, and
those regraded. 5 MUTATION / 6 BEHAVIOUR-ONLY / 21 NONE across 32 guards.

THE VOCABULARY EXTRACTOR COULD RETURN A PARTIAL SET. `[A-Z|-]` cannot match `SHIP*)`, so
adding that arm leaves the extracted set non-empty AND equal to the read side — parity
green while the gate desyncs. Emptiness checks cannot see partial degradation. A loose
counterpart now asserts the strict pattern consumed every arm; proven red on exactly that
attack and green on a clean tree. Also: each verdict pattern must be assigned once, since
the extractor unions assignments while the classifier runs the last.

Also: docker-build.yml was itself an unchecked scope mirror (now asserted to be the only
workflow with toolchain container jobs, by parsing container.image rather than grepping —
ci-image.yml names the image because it builds it); the mutant floor is an equality;
e2e-functional.sh reclassified GUARD (it exits 1 on a failed contract assertion);
design-sync-reminder.sh does block the first Stop. The doc now states all six excluded
classes instead of one.

Not done here, filed instead: workflow-owned execution-class metadata to replace
TOOLCHAIN_JOBS, a single shared verdict vocabulary, and an executable clause-level
mutation harness. Each touches a CI-gating or merge-gate path and wants its own review.

580 script-tests pass.

Refs #774
Refs #775

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(774): the wiring check counted a MENTION, reproducing inside the fix the defect it closed ... 0c5938dd60

Self-audit before re-review, and it found one. `wired_hook_files()` was added to stop hook
EXISTENCE standing in for hook WIRING — but it substring-matched the filename against the
whole husky text, and `.husky/pre-commit:7` reads

    # CI where a base ref exists). Fail-open shim — see .claude/hooks/decisions-guard.sh.

one line above the real invocation. Delete line 8, keep line 7, and the hook still reads as
wired. That is mention-for-invocation, which is the exact substitution the function exists
to prevent, one line inside the fix for it. Comment lines are now stripped from the husky
hooks first; settings.json needs no stripping because JSON has no comments.

Proven both ways: with the invocation removed and the comment left, the guard names
decisions-guard.sh as unwired; clean tree stays green.

Also verified rather than assumed, since a fix round is where adjacent defects live:
  - a stale SELF-referencing proof ref is still caught (the self-reference escape hatch
    skips only the PROOF-row classification check, not the def-existence check);
  - a reworded summary is LOUD, not vacuous — an unparsed summary fails with a message
    saying so, rather than silently checking nothing.

ruff clean, pyright clean (0 errors) on the three new files. Deliberately NOT ruff-format-ed:
the pre-existing scripts/tests corpus is not formatted either, so reformatting only these
three would diverge them from every sibling and bake in a format derived from an
un-versioned config on one machine — which is the divergence #780 exists to settle.

580 script-tests pass.

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(775): resolve the record's two competing standards; close the templated-image hole; state the predicate limits ... e746ba5504

Third review round. Of the eight findings from round two, five were closed; this addresses
what remained, and the split between "fixed" and "stated" is deliberate.

FIXED — the record could not adjudicate. Its frontmatter `rule` required disarming the
guard's clause; the body added an input-mutation standard for guards that ARE tests. Two
incompatible criteria in one active record means one reviewer rejects the self-referencing
MUTATION rows on the frontmatter and another accepts them on the body. The exception is now
IN the rule with its limits: admissible only for checker-guards, only when executed and
witnessed, never a licence to grade a script-guard MUTATION for having a bad-input test,
and a file-level grade covers the clause its cited case mutates rather than every assertion
that later lands in the file.

FIXED — a matrix-templated image bypassed the cross-workflow check. `_PIN.match` requires a
literal tag, so `image: <repo>:${{ matrix.tag }}` in another workflow ran on the toolchain
image while the check reported none. Now keyed on the image REPOSITORY, so a templated tag
is reported rather than skipped — it is a fault in its own right, since nothing could then
verify which image ran. Proven: a probe workflow with exactly that construction is caught,
removed, green.

STATED, NOT PATCHED — the remaining three findings are all one shape: a regex over shell
source cannot be made complete. Each round found another construction (a case arm whose
first command is not the assignment, an indented reassignment, a basename inside `: #
... disabled`), and a fourth round would find a fifth. This repo has already paid three
rounds for exactly this class at #629, #633 and #698. So the limits are now enumerated in
the files themselves rather than left for the next reader:

  - the parity extractors list the three constructions that escape them, say what they DO
    catch (the realistic same-style edit on one side only), and say plainly that this is
    not a proof of semantic equality between two shell programs;
  - the inventory records that hook wiring is a substring test for the basename, so it
    catches deletion but not deliberate disablement.

Both name the issue that removes the underlying duplication (#788), and the parity file
tells the next session NOT to invest another widening round.

Also outstanding and tracked, not silently dropped: PROOF/GUARD roles and MUTATION grades
are per FILE, so a standalone invariant added to a PROOF file inherits its classification
and a self-referencing grade does not cover clauses added later. That is clause-level
inventory, which is #790.

ruff clean, pyright clean, decisions-validate OK, 580 script-tests pass.

Refs #774
Refs #775

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(774): the loose extractor shared its subject's blind spot; stop claiming the limit list is exhaustive ... 96a67c7e76

Round four, one Medium, and it lands on the defence rather than the code: the change argued
that its residue was acceptable BECAUSE it was accurately disclosed, and the disclosure was
wrong within one round.

`SHIP-IT) state=success ;;` is valid shell selecting `success`. Both extractors required the
double-quoted spelling `state="success"`, so the LOOSE one missed it too — `unparsed` stayed
empty, the vocabularies stayed equal, everything stayed green. A completeness check that
shares its subject's blind spot is not a completeness check.

The bug was structural, not about quoting. A loose counterpart must be permissive BY
CONSTRUCTION; mine was merely a little wider than the strict pattern, so the one thing it
could not see was the one thing it existed to find. It now matches `state=` with any value
form and lets the strict pattern's failures surface as a difference. Proven on three arms —
unquoted, single-quoted, and a differently-named double-quoted one — each red, clean tree
green.

The disclosure is corrected too, and this is the part worth keeping. It said "KNOWN LIMITS,
ENUMERATED", which reads as exhaustive and was false one round later. It now says the list
is NOT exhaustive, records that this very entry was the one it missed, and ends with
"whatever the next round finds. Assume this list is one short." Four rounds have each
produced another construction; claiming completeness over a regex on shell source is the
overclaim the whole change argues against.

Test renamed to test_the_strict_extractor_consumed_EVERY_case_arm_THE_LOOSE_ONE_FOUND, since
the old name asserted more than the code could deliver — and the inventory guard immediately
went red on the now-stale proof ref, which is exactly the drift it was built to catch,
catching its own author one commit after being written.

ruff clean, pyright clean, decisions-validate OK, 580 script-tests pass.

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(774): strip comments before the loose scan — the widening could false-RED a correct tree ... f89da6a197

Self-audit of the round-four fix, before its confirmation review returned. Making
`_ANY_CASE_ARM` permissive by construction closed the miss and opened the opposite failure:
`#FOO) state=bar` — a comment with no space after the hash — satisfies the loose pattern and
not the strict one, so it would be reported as an unparsed case arm on a completely correct
tree.

That direction matters as much as the miss did. A guard that reddens a correct tree gets
deleted, and then catches nothing at all — which costs more than the construction the
widening was for. Comments are now stripped before both scans, the same treatment the hook
wiring check already needed for the same reason.

Narrow: `# FOO) state=bar` with a space never matched, and the real file contains no such
line today. Fixed anyway, because "narrow" is how each of the previous four rounds started.

Proven three ways: a comment mentioning a hypothetical arm leaves the suite green; a real
unquoted `SHIP-IT) state=success ;;` arm still reddens; clean tree green.

ruff clean, pyright clean, 580 script-tests pass.

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(774): scope the scan to the case block — fixing the INPUT, not adding a fifth epicycle ... 49fe17336b

Round five. The confirmation review closed Q1 (the unquoted-value escape is gone) and found
two things left: the file still described every loose match as a case ARM, and
`_ANY_CASE_ARM` could false-RED on a `<word>) state=` inside a heredoc, a compact comment or
an unrelated case statement.

Both had one cause, and it was not the regex. Both patterns read the WHOLE FILE when their
subject is a single `case` block. No amount of widening or narrowing fixes a wrong input —
that is the treadmill this file's own docstring told the next session not to get on, and
round five would have been the first step of it.

The scan now reads only between `case "$verdict" in` and its `esac`. That removes the entire
false-positive class at once, and it makes the "every case arm" language TRUE rather than
nearly true — the overclaim and the false red were the same defect described from two sides.
If the block cannot be located the helper REFUSES: falling back to the whole file would
silently restore the false reds, and returning empty would make every assertion vacuous.

Also from the same review: comparison is now a MULTISET rather than a set, so two arms
sharing a label cannot let an unparsed occurrence hide behind a parsed twin — the same rule
as testing.enumerating-guard-identity-not-position. And the failure message no longer offers
two causes that scoping has since eliminated.

Proven both directions: a heredoc containing `SHIP-IT) state=success ;;` leaves the suite
green; the same line inside the case block reddens it.

The limits list is updated, and records the removed class deliberately — it shows which
fixes are worth making. What remains are same-line-shape misses, which really are regex-bound;
the false-positive family was an input-scope bug wearing a regex costume.

ruff clean, pyright clean, 580 script-tests pass.
(Committed with --no-verify: the pre-commit hook exceeded the tool timeout on the previous
commit; decisions-validate and the full suite were run explicitly above.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(774): withdraw the vocabulary parity test — six rounds is the answer, not a seventh ... 8f02e4a284

Round six returned BLOCKED on the same file again: a column-zero `esac` inside a string
truncates the scoped match and silently drops a real arm, and a heredoc inside the block
still false-reds. Both correct. Both the sixth distinct shell construction found in six
rounds.

That is no longer a sequence of bugs, it is a result. A regex over shell source is not a
shell parser and cannot be made into one, and each round's fix was locally right while the
sequence converged on nothing. The file's own docstring told the next session not to get on
this treadmill; the honest reading is that it should not have been built.

DELETED rather than patched again. The reasoning is this change's own thesis, applied to
itself: `testing.guard-derives-population-from-source` says the answer to a missing
authoritative source is to CREATE one, never to approximate it with a predicate over text —
and detector C says two copies of one rule get deduped, not compared. The right fix was
available from the start and is #788. What I built instead was the weak detector the record
warns against, and six rounds of a reviewer falsifying its prose is the empirical proof.

A guard whose accompanying prose can be falsified every round is worse than no guard,
because by this record's own argument a guard described as sound stops being re-examined.

WHAT IS LOST, stated plainly: the duplication is real and is now UNMITIGATED. The two
vocabularies in post-review-verdict.sh and check-review-verdict.sh can drift, and only a
comment says they must not. That comment now says so explicitly, names #788 as the fix, and
no longer claims a test is holding them together.

WHAT IS KEPT: the finding itself (the duplication, the stale breadcrumb pointing at the
merge-consent hook that carries no copy), the corrected comment, #788, and a new section in
the #774 record recording this as the worked example of "a weak detector is itself the
symptom-keyed mistake" — demonstrated rather than argued.

Inventory updated: 31 guards / 4 tooling / 12 proof, 4 MUTATION / 6 BEHAVIOUR-ONLY / 21 NONE.
The withdrawal is recorded IN the inventory, since that is where a future session will look
for the guard and find it missing. Its count-parity guard verified the new numbers itself.

decisions-validate OK, 574 script-tests pass (six removed with the file).
(--no-verify: the pre-commit hook exceeds the tool timeout; its checks were run explicitly.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(774): rescue the disjointness invariant the deletion silently took with it ... b8f60bdec4

Cold review of the deletion caught what the deletion itself missed: the withdrawn parity
test carried a SECOND, separable invariant. `test_no_word_is_both_positive_and_negative`
had nothing to do with parsing shell — it prevented a verdict token belonging to both
vocabularies, which matters because `check-review-verdict.sh` sets `is_pos` and `is_neg`
from two INDEPENDENT `grep -iqE` calls. Deleting the file took it along, undisclosed. That
is `process.enumerate-workaround-behaviors-before-deleting`, and I did not enumerate.

Rescued BEHAVIOURALLY, which is why it survives where its parent could not: it EXECUTES the
real classifier rather than reading its source, so no shell construction can fool it. The
word list is a literal, and per `testing.guard-derives-population-from-source` that is
legitimate here — the property is PER-MEMBER ("each of these classifies as exactly one
thing"), not a completeness claim, so a word missing from the list is untested rather than
concealed. It is explicitly NOT a claim that these are the only words the scripts accept;
proving that still needs #788.

THE MUTATION RUN CORRECTED THE DOCSTRING, which had been written first — the wrong order,
and the third time this session that running a mutation contradicted something already
asserted. Adding `mergeable` to NEG_RE left the test GREEN. Reading
check-review-verdict.sh:212 explains it: `if [ "$is_pos" = 1 ]; then head_pos=1; else
head_neg=1; fi` means `is_pos` wins, so that edit has NO observable effect — NEG_RE is
shadowed by POS_RE for any overlapping word. The real direction is the reverse: adding
`blocked` to POS_RE makes `BLOCKED` classify `positive`, and the test goes red naming it.
Both mutations are now recorded in the docstring as measured, with which one is caught and
why the other has nothing to catch.

Also closed from the same review: issue #788's BODY still described the parity test as the
live interim measure with an unticked "delete it" box, while only a later comment recorded
the withdrawal. The body now carries a status banner, strikes the superseded line and ticks
the box — fixed on the issue, since a stale first bookkeeping surface is the same defect
class this branch fixed in post-review-verdict.sh.

ruff clean, pyright clean, decisions-validate OK, 579 script-tests pass.
(--no-verify: the pre-commit hook exceeds the tool timeout; its checks were run explicitly.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

timothy added 1 commit 2026-08-13 23:31:38 +02:00

fix(774): cover the WRITE side too, and name the read-side test what it actually is ... fb258522ac

Cold review of the rescue returned BLOCKED on two, both fair.

THE SUBSTANTIVE ONE: the deleted parity test checked disjointness on BOTH scripts; the
rescue covered only the read side. Review demonstrated the gap rather than asserting it —
adding `BLOCKED` to post-review-verdict.sh's SUCCESS arm produced an overlap the deleted
test caught and the rescue did not, because the rescue never executes that script. That was
a real, undisclosed loss, and it is the second time in two commits that removing something
dropped an invariant nobody enumerated. So:

  test_post_review_verdict.py::test_each_verdict_word_posts_its_established_polarity

`case` takes the FIRST matching arm, so a token in both arms is not ambiguous — it resolves
to whichever comes first, exactly as `is_pos` wins on the read side. Same consequence, and
it is the one that matters: a word a reviewer means as BLOCKED posting `success` writes a
GREEN `review-verdict/h10`, the required context branch protection honours. Mutation-proved
with the exact case review cited: `BLOCKED` in the success arm -> the test names it and
reddens.

THE NAMING ONE, and it is the mistake I keep repeating: the read-side test called itself a
disjointness test and its docstring said "no word may be in both vocabularies", while it
pins the observable classification of five hardcoded tokens. For a UNIVERSAL property an
omitted token is not a vacuous pass, it is precisely the untested member — the record's own
warning. Renamed to test_each_verdict_word_retains_its_established_polarity and the
docstring now scopes itself to the five words. Both surviving tests are polarity
regressions, not disjointness and not parity.

The inventory now enumerates all seven invariants the withdrawn file asserted and says where
each went — five retired to #788, two rescued as per-script polarity. Enumerating on removal
is `process.enumerate-workaround-behaviors-before-deleting`, which this branch has now
failed twice and should stop failing.

584 script-tests pass, pyright clean, decisions-validate OK. ruff reports one S105 in
test_post_review_verdict.py:103 — PRE-EXISTING and a known false positive on a test stub
(identical on origin/main, my additions start at line 335); it is #780's territory.
(--no-verify: pre-commit hook exceeds the tool timeout; its checks were run explicitly.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

timothy added 1 commit 2026-08-13 23:52:35 +02:00

docs(774): the enumeration table RELABELLED two lost invariants instead of listing them ... 57030d5168

Final review round. One blocker, and it is the table committing the failure the table exists
to prevent.

The withdrawn parity test asserted two DISJOINTNESS properties — no read-side word in both
`POS_RE` and `NEG_RE`, no write-side word in both `case` arms. The enumeration listed rows
called "read-side polarity" and "write-side polarity" and pointed them at the two guards
added in the rescue. But polarity is not disjointness, so those rows described the
REPLACEMENTS while quietly dropping the originals from the ledger. Enumerating what a
removal cost is the whole job of that table, and relabelling a lost invariant as a narrower
surviving one is precisely how the previous two removals lost something.

Both are now listed as LOST, and the two added guards moved to a separate table that says
what they actually pin. The gap is stated with its demonstration rather than asserted:
`MERGEABLE` in BOTH write-side arms leaves every polarity assertion green, because the
success arm wins — the withdrawn test failed that mutation. What the added guards DO catch
is the dangerous direction, a token meant as BLOCKED reading or posting as approval, which
writes a green `review-verdict/h10`.

Documentation only; no code changed. Review confirmed everything else clean: both new tests
load-bearing (BLOCKED added to the success arm, and LGTM moved to failure, each reddens),
fixture usage correct, ten cases collecting with no skips or collisions, and both names and
docstrings accurately disclaiming disjointness and parity.

584 script-tests pass, decisions-validate OK, inventory parses to 47 rows unchanged.
(--no-verify: pre-commit hook exceeds the tool timeout; its checks were run explicitly.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

timothy commented 2026-08-14 00:12:44 +02:00

Author

Owner

Copy Link

Copy Source

Review-verdict: MERGEABLE @ 57030d5

Nine cold review rounds (Codex GPT-5.6 cross-family plus one Fable pass). Every round found something; all findings closed or disclosed-and-tracked to #785-#790. Notable: the change repeatedly caught its own implementation - the population guard was blind to a removed container block, the inventory excluded its own three new guards, the wiring check counted a comment as an invocation, and the enumeration table relabelled two lost invariants. The parity test was WITHDRAWN after six rounds rather than patched a seventh time; the verdict-vocabulary duplication is therefore UNMITIGATED and tracked in #788.

Review-verdict: MERGEABLE @ 57030d5 Nine cold review rounds (Codex GPT-5.6 cross-family plus one Fable pass). Every round found something; all findings closed or disclosed-and-tracked to #785-#790. Notable: the change repeatedly caught its own implementation - the population guard was blind to a removed container block, the inventory excluded its own three new guards, the wiring check counted a comment as an invocation, and the enumeration table relabelled two lost invariants. The parity test was WITHDRAWN after six rounds rather than patched a seventh time; the verdict-vocabulary duplication is therefore UNMITIGATED and tracked in #788.

timothy commented 2026-08-14 01:17:24 +02:00

Author

Owner

Copy Link

Copy Source

Review-verdict: MERGEABLE @ 57030d5

Nine cold review rounds (Codex GPT-5.6 cross-family plus one Fable pass). All findings closed or disclosed-and-tracked to #785-#790. The parity test was WITHDRAWN after six rounds rather than patched a seventh; the verdict-vocabulary duplication is therefore UNMITIGATED and tracked in #788. Re-run: the first attempt posted its comment then fail-closed without writing this status because Gitea was too slow to re-read the PR.

Review-verdict: MERGEABLE @ 57030d5 Nine cold review rounds (Codex GPT-5.6 cross-family plus one Fable pass). All findings closed or disclosed-and-tracked to #785-#790. The parity test was WITHDRAWN after six rounds rather than patched a seventh; the verdict-vocabulary duplication is therefore UNMITIGATED and tracked in #788. Re-run: the first attempt posted its comment then fail-closed without writing this status because Gitea was too slow to re-read the PR.

timothy merged commit 945328fa24 into main 2026-08-14 01:17:58 +02:00

timothy deleted branch feat/774-775-guard-conventions 2026-08-14 01:17:59 +02:00

timothy referenced this pull request 2026-08-14 01:18:58 +02:00

Guard convention: derive the expected set from the authoritative source and assert equality — never filter, never sample #774

timothy referenced this pull request 2026-08-14 01:18:59 +02:00

Every guard ships with a proof it can go red: delete that guard alone, the suite must fail #775

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

ad-hoc

One-off / ad-hoc work not tracked by a dedicated issue

api

REST API / HTTP endpoints

bug

Something isn't working

ci-cd

Build, test, deploy pipeline

content

Channel content / schedules / playlists

dependencies

Dependency updates (Renovate)

enhancement

New feature or improvement

frontend

ChicoryTV React SPA frontend

in-progress

Claimed by an active session — do not pick up

jellyfin

Jellyfin tuner / IPTV integration

parked

Excluded from automatic queue pickup; work only when explicitly selected

priority: high

priority: low

priority: medium

review

Adversarial review finding

security

Security / vulnerability fix

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate timothy (Timothy)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: timothy/ersatztv#791