docs(796): remove the last hand-maintained numbers, on a MERGEABLE verdict

Round 5 returned MERGEABLE and found nothing false. These are its durability findings — statements that are true today and that an unrelated future edit would quietly falsify. Taking them because each is one line and each removes a number that nothing checks; the class has cost four rounds already. - "yet FOUR of them carry MUTATION rows — <closed list>" was the most exposed claim in the change: the same sentence calls the remaining hooks "unproven debt", so it invites the very upgrade that falsifies its own count, in three places (the `rule:`, the body, the mirrored catalog row), with nothing checking it. The argument needs at-least-one, not exactly-four. Now "several … among them". - "The only place that RUNS this recipe is `check-local-lsp.sh:107`" was true, but the record exists to publish that recipe for reuse — the first script to adopt it falsifies the claim, and that author has no reason to open this record. The line number rots on any edit above it. Both dropped. - "scope limit item 6"/"item 7" are positional references into another file's numbered list, bound by nothing: appending is safe, inserting silently repoints them. Now named as well as numbered. - "so there is one list rather than a number" — the inventory row's "what it blocks" cell is a second enumeration, mandated by the row format. "a list". - The defect table sits ABOVE the paragraph that said "the five defects below". - `scripts/mcp_smoke.py`'s ACCEPTED LIMIT note said the caps "caused three defects in an earlier round", which contradicts the record that is now the canonical account and counts them as one of five. Pre-existing, but this PR is what makes it a contradiction, so it is fixed here rather than left for a reader to reconcile. Docstring only — the declared mutation clause is untouched and still unique in the file. Verification: 1104 passed, 2 skipped; ruff format+check clean over all 47 tracked Python files; decisions-validate OK; catalog up to date. Refs: #796 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 21:42:35 +02:00
parent 23624af50b
commit 1080e0bba4
4 changed files with 8 additions and 8 deletions
@@ -218,7 +218,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
 | `testing.playwright-mcp-download-and-recovery` | In Playwright-MCP E2E, fetch file-download endpoints with curl — never a browser tab or `window.open` — and if browser tools stall repeatedly, `pkill -f ms-playwright-mcp` and drive a fresh session. | 2026-07-21 | [link](records/testing/playwright-mcp-download-and-recovery.md) |
 | `testing.scripted-playout-golden-deferred` | The `PlayoutBuildGoldenTests` in-memory golden net covers Sequential (YAML) as of #381. Scripted's *end-to-end pipeline* is excluded — `ScriptedPlayoutBuilder` runs a user-authored external program that drives the engine over HTTP loopback, which the in-memory harness can't pin — so that full-pipeline (integration) harness is deferred to #563. But the scheduling *behavior* those scripts drive lives entirely in the in-process `SchedulingEngine` (the `ScriptedScheduleController` is a 1:1 pass-through to it), which IS directly unit/golden-testable; the earlier "Scripted is un-golden-able by construction" framing overstated the constraint by conflating transport with engine. #395 extracts that shared switch to `ContentEnumeratorBuilder` and adds a direct regression net (`ContentEnumeratorBuilderTests`) over it. | 2026-07-22 | [link](records/testing/scripted-playout-golden-deferred.md) |
 | `testing.troubleshoot-path-cannot-test-branding` | Verify logo/watermark/bug changes through a real channel playout — a green troubleshoot run proves nothing about branding. | 2026-07-21 | [link](records/testing/troubleshoot-path-cannot-test-branding.md) |
-| `testing.verification-code-needs-its-own-proof` | VERIFICATION CODE IS CODE UNDER TEST. The obligation in `testing.guard-ships-with-mutation-proof` attaches to whatever emits a PASS/FAIL somebody acts on — a smoke test, a timeout wrapper, an operator-run checker, the harness that applies the mutations — and NOT only to files sitting in the guard population or named like guards; every one of the five defects measured below was in code nobody had counted as a guard. THE DISCRIMINATOR IS WHETHER A PROOF TEST CAN DRIVE THE CONSTRUCT HERMETICALLY, NOT WHETHER CI RUNS IT, and the difference is load-bearing rather than pedantic: `.claude/hooks/*.sh` and `.husky/pre-push` are operator-run and CI never fires them AS HOOKS, yet FOUR of them carry `MUTATION` rows — `posttooluse-worktree-marker.sh`, `pretooluse-bom-guard.sh`, `pretooluse-worktree-guard.sh` and `.husky/pre-push` — because a pytest proof drives them in a sandbox. A does-CI-run-it test would sort those four already-proven guards into the exempt branch. (Most of the remaining hooks are graded `NONE`; that is unproven debt, not evidence for the other discriminator.) "IT NEEDS A REAL SERVER" IS AN EXEMPTION CLAIM AND IT IS USUALLY FALSE: `scripts/mcp_smoke.py` takes its config path and server name as POSITIONAL ARGUMENTS, so a test hands it a synthetic config in a tmpdir pointing at a stub responder and drives it with no gitignored `.mcp.json` and no language server, which is how it is proven in `scripts/tests/test_mcp_smoke.py`. That coverage is a SUBSET and no count of it is kept here: the file's own docstring enumerates the cases it drives and names what it does not, so there is one list rather than a number to keep in step with it. A checker is never excused because its CALLER cannot run in CI. WHERE THE PROOF LIVES: the checker itself needs NO `docs/guard-inventory.md` row, because that population is derived from workflow and hook call sites and a transitively-reached script is rejected as a phantom; the proof goes in an ordinary `scripts/tests/test_*.py`, which joins the population automatically and carries the row, while the manifest entry names THAT test as `guard` and the checker as `target`. That shape already exists for `mutation_harness_lib.py` and needs no change to any population. ONLY WHAT NO PROOF TEST CAN DRIVE HERMETICALLY falls back to a WITNESSED NEGATIVE CONTROL — the failure path executed once, the false-green form reproduced, both recorded in the PR, and its decay stated because nothing re-runs it. `scripts/check-local-lsp.sh` is the genuine case: it asserts developer-machine installs (a global `csharp-ls`, a `host/fxr` under `DOTNET_ROOT`, a root `node_modules`) that no runner has. STOP-AND-SUBTRACT, adopted here as this record's own threshold: when a review round's finding was CREATED by the previous round's fix and that happens TWICE CONSECUTIVELY, delete the layer generating them rather than guarding it. READING THE CONSTRUCT IS NOT EXECUTING IT: ShellCheck 0.11.0 flagged NEITHER of the two shell defects below, and each is indistinguishable from correct by reading. | 2026-08-28 | [link](records/testing/verification-code-needs-its-own-proof.md) |
+| `testing.verification-code-needs-its-own-proof` | VERIFICATION CODE IS CODE UNDER TEST. The obligation in `testing.guard-ships-with-mutation-proof` attaches to whatever emits a PASS/FAIL somebody acts on — a smoke test, a timeout wrapper, an operator-run checker, the harness that applies the mutations — and NOT only to files sitting in the guard population or named like guards; every one of the five defects measured below was in code nobody had counted as a guard. THE DISCRIMINATOR IS WHETHER A PROOF TEST CAN DRIVE THE CONSTRUCT HERMETICALLY, NOT WHETHER CI RUNS IT, and the difference is load-bearing rather than pedantic: `.claude/hooks/*.sh` and `.husky/pre-push` are operator-run and CI never fires them AS HOOKS, yet several of them carry `MUTATION` rows — among them `pretooluse-bom-guard.sh`, `pretooluse-worktree-guard.sh` and `.husky/pre-push` — because a pytest proof drives them in a sandbox. A does-CI-run-it test would sort those already-proven guards into the exempt branch. (Most of the remaining hooks are graded `NONE`; that is unproven debt, not evidence for the other discriminator.) "IT NEEDS A REAL SERVER" IS AN EXEMPTION CLAIM AND IT IS USUALLY FALSE: `scripts/mcp_smoke.py` takes its config path and server name as POSITIONAL ARGUMENTS, so a test hands it a synthetic config in a tmpdir pointing at a stub responder and drives it with no gitignored `.mcp.json` and no language server, which is how it is proven in `scripts/tests/test_mcp_smoke.py`. That coverage is a SUBSET and no count of it is kept here: the file's own docstring enumerates the cases it drives and names what it does not, so there is a list rather than a number to keep in step with it. A checker is never excused because its CALLER cannot run in CI. WHERE THE PROOF LIVES: the checker itself needs NO `docs/guard-inventory.md` row, because that population is derived from workflow and hook call sites and a transitively-reached script is rejected as a phantom; the proof goes in an ordinary `scripts/tests/test_*.py`, which joins the population automatically and carries the row, while the manifest entry names THAT test as `guard` and the checker as `target`. That shape already exists for `mutation_harness_lib.py` and needs no change to any population. ONLY WHAT NO PROOF TEST CAN DRIVE HERMETICALLY falls back to a WITNESSED NEGATIVE CONTROL — the failure path executed once, the false-green form reproduced, both recorded in the PR, and its decay stated because nothing re-runs it. `scripts/check-local-lsp.sh` is the genuine case: it asserts developer-machine installs (a global `csharp-ls`, a `host/fxr` under `DOTNET_ROOT`, a root `node_modules`) that no runner has. STOP-AND-SUBTRACT, adopted here as this record's own threshold: when a review round's finding was CREATED by the previous round's fix and that happens TWICE CONSECUTIVELY, delete the layer generating them rather than guarding it. READING THE CONSTRUCT IS NOT EXECUTING IT: ShellCheck 0.11.0 flagged NEITHER of the two shell defects below, and each is indistinguishable from correct by reading. | 2026-08-28 | [link](records/testing/verification-code-needs-its-own-proof.md) |

 ## Review due

@@ -5,9 +5,9 @@ status: active
 since: '2026-08-28'
 supersedes: none
 superseded-by: none
-rule: 'VERIFICATION CODE IS CODE UNDER TEST. The obligation in `testing.guard-ships-with-mutation-proof` attaches to whatever emits a PASS/FAIL somebody acts on — a smoke test, a timeout wrapper, an operator-run checker, the harness that applies the mutations — and NOT only to files sitting in the guard population or named like guards; every one of the five defects measured below was in code nobody had counted as a guard. THE DISCRIMINATOR IS WHETHER A PROOF TEST CAN DRIVE THE CONSTRUCT HERMETICALLY, NOT WHETHER CI RUNS IT, and the difference is load-bearing rather than pedantic: `.claude/hooks/*.sh` and `.husky/pre-push` are operator-run and CI never fires them AS HOOKS, yet FOUR of them carry `MUTATION` rows — `posttooluse-worktree-marker.sh`, `pretooluse-bom-guard.sh`, `pretooluse-worktree-guard.sh` and `.husky/pre-push` — because a pytest proof drives them in a sandbox. A does-CI-run-it test would sort those four already-proven guards into the exempt branch. (Most of the remaining hooks are graded `NONE`; that is unproven debt, not evidence for the other discriminator.) "IT NEEDS A REAL SERVER" IS AN EXEMPTION CLAIM AND IT IS USUALLY FALSE: `scripts/mcp_smoke.py` takes its config path and server name as POSITIONAL ARGUMENTS, so a test hands it a synthetic config in a tmpdir pointing at a stub responder and drives it with no gitignored `.mcp.json` and no language server, which is how it is proven in `scripts/tests/test_mcp_smoke.py`. That coverage is a SUBSET and no count of it is kept here: the file''s own docstring enumerates the cases it drives and names what it does not, so there is one list rather than a number to keep in step with it. A checker is never excused because its CALLER cannot run in CI. WHERE THE PROOF LIVES: the checker itself needs NO `docs/guard-inventory.md` row, because that population is derived from workflow and hook call sites and a transitively-reached script is rejected as a phantom; the proof goes in an ordinary `scripts/tests/test_*.py`, which joins the population automatically and carries the row, while the manifest entry names THAT test as `guard` and the checker as `target`. That shape already exists for `mutation_harness_lib.py` and needs no change to any population. ONLY WHAT NO PROOF TEST CAN DRIVE HERMETICALLY falls back to a WITNESSED NEGATIVE CONTROL — the failure path executed once, the false-green form reproduced, both recorded in the PR, and its decay stated because nothing re-runs it. `scripts/check-local-lsp.sh` is the genuine case: it asserts developer-machine installs (a global `csharp-ls`, a `host/fxr` under `DOTNET_ROOT`, a root `node_modules`) that no runner has. STOP-AND-SUBTRACT, adopted here as this record''s own threshold: when a review round''s finding was CREATED by the previous round''s fix and that happens TWICE CONSECUTIVELY, delete the layer generating them rather than guarding it. READING THE CONSTRUCT IS NOT EXECUTING IT: ShellCheck 0.11.0 flagged NEITHER of the two shell defects below, and each is indistinguishable from correct by reading.'
+rule: 'VERIFICATION CODE IS CODE UNDER TEST. The obligation in `testing.guard-ships-with-mutation-proof` attaches to whatever emits a PASS/FAIL somebody acts on — a smoke test, a timeout wrapper, an operator-run checker, the harness that applies the mutations — and NOT only to files sitting in the guard population or named like guards; every one of the five defects measured below was in code nobody had counted as a guard. THE DISCRIMINATOR IS WHETHER A PROOF TEST CAN DRIVE THE CONSTRUCT HERMETICALLY, NOT WHETHER CI RUNS IT, and the difference is load-bearing rather than pedantic: `.claude/hooks/*.sh` and `.husky/pre-push` are operator-run and CI never fires them AS HOOKS, yet several of them carry `MUTATION` rows — among them `pretooluse-bom-guard.sh`, `pretooluse-worktree-guard.sh` and `.husky/pre-push` — because a pytest proof drives them in a sandbox. A does-CI-run-it test would sort those already-proven guards into the exempt branch. (Most of the remaining hooks are graded `NONE`; that is unproven debt, not evidence for the other discriminator.) "IT NEEDS A REAL SERVER" IS AN EXEMPTION CLAIM AND IT IS USUALLY FALSE: `scripts/mcp_smoke.py` takes its config path and server name as POSITIONAL ARGUMENTS, so a test hands it a synthetic config in a tmpdir pointing at a stub responder and drives it with no gitignored `.mcp.json` and no language server, which is how it is proven in `scripts/tests/test_mcp_smoke.py`. That coverage is a SUBSET and no count of it is kept here: the file''s own docstring enumerates the cases it drives and names what it does not, so there is a list rather than a number to keep in step with it. A checker is never excused because its CALLER cannot run in CI. WHERE THE PROOF LIVES: the checker itself needs NO `docs/guard-inventory.md` row, because that population is derived from workflow and hook call sites and a transitively-reached script is rejected as a phantom; the proof goes in an ordinary `scripts/tests/test_*.py`, which joins the population automatically and carries the row, while the manifest entry names THAT test as `guard` and the checker as `target`. That shape already exists for `mutation_harness_lib.py` and needs no change to any population. ONLY WHAT NO PROOF TEST CAN DRIVE HERMETICALLY falls back to a WITNESSED NEGATIVE CONTROL — the failure path executed once, the false-green form reproduced, both recorded in the PR, and its decay stated because nothing re-runs it. `scripts/check-local-lsp.sh` is the genuine case: it asserts developer-machine installs (a global `csharp-ls`, a `host/fxr` under `DOTNET_ROOT`, a root `node_modules`) that no runner has. STOP-AND-SUBTRACT, adopted here as this record''s own threshold: when a review round''s finding was CREATED by the previous round''s fix and that happens TWICE CONSECUTIVELY, delete the layer generating them rather than guarding it. READING THE CONSTRUCT IS NOT EXECUTING IT: ShellCheck 0.11.0 flagged NEITHER of the two shell defects below, and each is indistinguishable from correct by reading.'
 signals: 'the verification harness is itself unproven code · a false green in the checker rather than the checked · `[ -x "$cmd" ]` is true for a DIRECTORY · perl alarm/exec exits 0 without `or die` · macOS has no `timeout` and no `gtimeout` · smoke test accepts an impostor server · a server that pre-answers a guessable id · each fix created the next round''s defect · stop and subtract rather than add a layer · hermetically drivable is the test, not does-CI-run-it · a checker is not exempt because its CALLER cannot run in CI · the checker needs no inventory row — name the proof test as `guard` and the checker as `target` · a transitively-reached script is rejected as a PHANTOM · witnessed negative control for what cannot be driven · paths: `scripts/mcp_smoke.py`, `scripts/tests/test_mcp_smoke.py`, `scripts/check-local-lsp.sh`, `scripts/tests/mutation_manifest.py`, `docs/guard-inventory.md` · issues: #796, #793, #790, #785, #777, #776'
-mechanics: 'THE macOS TIMEOUT RECIPE, recorded here because the WRONG form was the recommended one and a session that followed it verbatim produced a false green (#793): `perl -e ''alarm shift; exec @ARGV or die "exec failed: $!\n"'' <secs> <cmd...>`. The `or die` is load-bearing — without it perl exits 0 when `exec` fails, so a missing or misspelt binary reports SUCCESS having run nothing (measured 2026-08-28: `perl -e ''alarm shift; exec @ARGV'' 5 /nope` exits 0; the `or die` form exits 2 and says why). macOS has neither `timeout` (GNU coreutils) nor `gtimeout` on this fleet, and an absent `gtimeout` exits 127 with empty output, which reads as a clean pass to any caller grepping stdout for a failure string. Two traps travel with it: `[ -x "$cmd" ]` does not establish that a command is runnable, because a directory satisfies `-x` (`[ -x /bin ]` is true); and whatever wrapper is used, its EXIT CODE must be checked rather than its stdout. The only place that RUNS this recipe is `scripts/check-local-lsp.sh:107` (the other in-repo occurrences of the idiom are in this record); the `-x` trap appears there only in a comment recording what it deliberately does not do.'
+mechanics: 'THE macOS TIMEOUT RECIPE, recorded here because the WRONG form was the recommended one and a session that followed it verbatim produced a false green (#793): `perl -e ''alarm shift; exec @ARGV or die "exec failed: $!\n"'' <secs> <cmd...>`. The `or die` is load-bearing — without it perl exits 0 when `exec` fails, so a missing or misspelt binary reports SUCCESS having run nothing (measured 2026-08-28: `perl -e ''alarm shift; exec @ARGV'' 5 /nope` exits 0; the `or die` form exits 2 and says why). macOS has neither `timeout` (GNU coreutils) nor `gtimeout` on this fleet, and an absent `gtimeout` exits 127 with empty output, which reads as a clean pass to any caller grepping stdout for a failure string. Two traps travel with it: `[ -x "$cmd" ]` does not establish that a command is runnable, because a directory satisfies `-x` (`[ -x /bin ]` is true); and whatever wrapper is used, its EXIT CODE must be checked rather than its stdout. The place that RUNS this recipe is `scripts/check-local-lsp.sh`; the other in-repo occurrences of the idiom are in this record. The `-x` trap appears there only in a comment recording what it deliberately does not do.'
 ---

 **The measurement.** #793 added one small operator-run check — a smoke test that starts the
@@ -27,11 +27,11 @@ reproduced with a control before being fixed:
 proof` is scoped by a population — the guard files `docs/guard-inventory.md` derives and grades.
 Defects 1 and 3 are a `[ -x ]` test and a `perl` wrapper inside `scripts/check-local-lsp.sh`, which
 no workflow or hook invokes; 2, 4 and 5 live in `scripts/mcp_smoke.py`, reached only through that
-script and so excluded by scope limit item 6. Neither file is in the population. That scoping is what keeps the inventory from becoming a chore that gets
+script and so excluded by the transitive-calls scope limit (item 6). Neither file is in the population. That scoping is what keeps the inventory from becoming a chore that gets
 rubber-stamped; what was missing is the claim that the obligation follows the *verdict*, not the *file*.

 **The discriminator, stated precisely because the obvious one is wrong.** It is not "does CI run
-this": CI never fires `.claude/hooks/*.sh` or `.husky/pre-push` as hooks either, yet four of them
+this": CI never fires `.claude/hooks/*.sh` or `.husky/pre-push` as hooks either, yet several of them
 are graded `MUTATION`, because a pytest proof drives them in a sandbox. The question is whether a
 proof test can drive the construct *hermetically*, and "it needs a live server" usually fails that
 test rather than passing it. `mcp_smoke.py` takes its config path and server name as positional
@@ -41,7 +41,7 @@ file's docstring rather than counted here. `check-local-lsp.sh` genuinely
 cannot be driven that way: it asserts a global `csharp-ls` on PATH, a `host/fxr` under
 `DOTNET_ROOT`, and a root `node_modules`, which is developer-machine state no runner has. That, and
 not "it is operator-run", is what puts it in the exempt branch. Note what that concedes: two of
-the five defects below (the `-x` test and the `perl` wrapper) were in `check-local-lsp.sh` itself,
+the five defects above (the `-x` test and the `perl` wrapper) were in `check-local-lsp.sh` itself,
 so the exempt branch is not defect-free — it is the part where a recorded control is the best
 available proof. Of the three in `mcp_smoke.py`, defects 2 and 5 are pinned by tests; defect 4's
 territory is deliberately left uncovered, because the message caps that would bound it WERE that
@@ -22,7 +22,7 @@ deliberately share a stage code, so read the message, not the number.

 ACCEPTED LIMIT: a server that writes a gigabyte with no newline can still exhaust
 memory before the timeout fires. Guarding that needs the very frame-capping reader
-whose caps caused three defects in an earlier round, and the input here is our OWN
+whose caps were themselves a defect in an earlier round, and the input here is our OWN
 configured server on a developer machine — not an adversary. Stated rather than
 silently unhandled.

@@ -10,7 +10,7 @@ its-own-proof` is the rule that says a checker owes an executed proof; this is t

 `mcp_smoke.py` itself gets no `docs/guard-inventory.md` row: the inventory's population is derived
 from workflow and hook call sites and this script is reached only transitively, through
-`scripts/check-local-lsp.sh` (scope limit item 6), so a row for it is rejected as a phantom. THIS
+`scripts/check-local-lsp.sh` (the transitive-calls scope limit, item 6), so a row for it is rejected as a phantom. THIS
 file joins the population as an ordinary `scripts/tests/test_*.py` and carries the row, and the
 manifest entry names it as `guard` with `scripts/mcp_smoke.py` as `target` — the same shape
 `test_mutation_harness.py` uses for `mutation_harness_lib.py`.