From c3ccfa95e06e258bfb61201e16ccc8313bf56198 Mon Sep 17 00:00:00 2001 From: Timothy Date: Thu, 13 Aug 2026 20:01:25 +0200 Subject: [PATCH] docs(773): root-cause the recurring defect shapes across the full closed-issue corpus MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Analysis over all 349 closed issues (95 carry a `## Closing record`), classified by five independent raters against a written taxonomy, with an inter-rater control (12/15 blind agreement) and a backward-generalization check over the 254 pre-convention issues. Findings that change the picture the issue started from: - The ranking reverses. Vacuous verification is the most common shape (17/69), not twin-missed (14/69), over the full corpus vs the issue's 50-issue sample. - The issue's central hypothesis holds and extends: twin-missed, vacuous-by- sampling and symptom-keyed guards are one error (reasoning about a representative instead of the population) — ~32/69 (46%), one detector. - The shapes predate the closing-record convention (#1, #215, #232, #403), so they are not an artifact of recent guard-building work. - A class the taxonomy missed entirely: check-and-use races over mutable state (#536, #622, #632, #706, #707) — five instances, proposed independently by three raters and re-derived by a fourth, blind. - Overclaim drops to 4% as a primary cause; it is a modifier, not a class. Part 2 (tooling audit), all measured rather than assumed: csharp-lsp cannot initialize and typescript-lsp cannot resolve typescript, the `LSP` tool has 0 calls across 811 transcripts, ruff/pyright are enforced nowhere despite the global instruction, and PreToolUse/PostToolUse hook execution leaves no durable trace — so we cannot currently tell whether our own guards fire. Names the classes where no mechanical detector is plausible rather than inventing weak ones. Spawns #774-#781; this issue is analysis, not implementation. Refs #773 Co-Authored-By: Claude Opus 5 (1M context) --- docs/README.md | 7 + docs/defect-shapes-773.md | 351 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 358 insertions(+) create mode 100644 docs/defect-shapes-773.md diff --git a/docs/README.md b/docs/README.md index 873ac2e90..6411227fc 100644 --- a/docs/README.md +++ b/docs/README.md @@ -24,6 +24,7 @@ doc below, or that changes which sections a task signal points to.** | Concurrency / optimistic-locking work | `docs/api-conventions.md` §7a/b/c + `docs/decisions/optimistic-concurrency.md` | | Auth / security-surface work | `docs/decisions/api-auth-security.md` | | CI / release pipeline work | `docs/ci-cd.md` + `docs/decisions/release-ci-governance.md` | +| Proposing a new guard / CI check / regression test convention | `docs/defect-shapes-773.md` §4 (detector menu + the classes where no detector is plausible) | | Live local run / Playwright-MCP verification | `docs/e2e-local.md` + `scripts/e2e-local.sh` | | Adding/changing a UI-E2E browser flow | `docs/e2e-local.md` → "UI-E2E harness" + `scripts/e2e-ui.sh` | | What does a test suite cover | `docs/testing.md` | @@ -98,6 +99,12 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K that file's standing kickoff for the two concurrent tracks (orientation ‖ selection). ersatztv#237 is a closed, archival historical tracker (superseded by `startup.parallel-orientation` in `docs/decisions.md`) — not a live pointer. +- **`docs/defect-shapes-773.md`** — root-cause analysis of the recurring defect shapes across the + whole closed-issue corpus (#773): the measured class ranking, the four families they consolidate + into, the cheapest mechanical detector per class, the classes where **no** detector is plausible, + and an audit of which configured hooks/MCP servers/LSPs are actually invoked. Read it before + proposing a new guard or CI check — §4 is the detector menu, and it argues against enumerating + cases one incident at a time. - **`docs/tracker-retrofit-triage-237.md`** — audit trail for the #524 triage of ersatztv#237's 111 comments (method, per-comment classification, totals). Evidence for the `docs.tracker-comment-retrofit` decision; read it only when triaging another over-cap tracker. diff --git a/docs/defect-shapes-773.md b/docs/defect-shapes-773.md new file mode 100644 index 000000000..e02ed52d2 --- /dev/null +++ b/docs/defect-shapes-773.md @@ -0,0 +1,351 @@ +# Recurring defect shapes across the closed-issue corpus (ersatztv#773) + +Root-cause analysis over every closed issue in the tracker, the detectors that would have caught +each class, and an audit of the tooling already configured in this repo. This is an **analysis +artifact**, not a rulebook: the rules it argues for land as decision records and follow-up issues, +which are linked per class below. + +Measured 2026-08-13 against the tracker as of issue #773 and `origin/main` at `f9f8f65ce`. + +--- + +## 1. Method, and what these numbers can and cannot support + +**Corpus.** All 349 closed issues (`state=closed&type=issues`, 7 pages, 349 distinct numbers, +#1–#757 — pull requests excluded). Of those, **95 carry a `## Closing record`** comment. Each +record was classified by one of five independent agents against a written taxonomy, with +`NEW:` available so the taxonomy could not be self-fulfilling. A sixth agent re-rated a +stratified 15-issue sample blind, as an inter-rater control. A seventh sampled 25 of the 254 +pre-convention closed issues from their raw bodies and comments, to test whether the shapes exist +outside the era that writes records. + +**Three limits, stated up front because they bound every number below.** + +1. **Record coverage is not uniform.** The `## Closing record` convention starts at #520 + (2026-07-21). Coverage is 31/32 for #600–699 and 4/69 for #1–99. So frequencies over the 95 + records are frequencies over *recent* practice. +2. **Corpus composition shifts underneath the measurement.** The share of records describing no + process failure at all falls from 12/19 (#60–#438) to 0/19 (#616–#671). Early issues are + feature work; recent issues are largely CI/process hardening. Guard-shaped defects therefore + dominate the recent records *partly by construction* — we have lately been building guards, so + our defects are in guards. +3. **A closing record is the closing author's self-report.** Raters were told to flag records whose + own claims outrun what they describe, and several did (#60, #651, #691, #720). But the corpus + systematically cannot contain a defect nobody noticed. + +**What limit 2 does *not* undermine.** The obvious worry — that these classes are an artifact of +recent guard-building — was tested and **refuted**. The backward sample found high-confidence +instances of the same shapes well before the convention existed: twin-missed at #215 (REST playout +mutations skip the build-lock gating the Blazor path had) and #403 (an unhandled enum silent at 5 of +6 dispatch sites); wrapper drift at #198 and #287 (OpenAPI spec diverged from runtime JSON); +masking guards at #232 and #234 ("one bug papering over another"); overclaim at #1 ("the earlier +'confirmed' was based on one curl test and several unverified assertions"). The convention changed +the **density and legibility** of the evidence, not the existence of the shapes. Several older +issues (#94, #157, #172, #265, #377) genuinely cannot be classified from what they record, and are +reported as `insufficient-evidence` rather than forced into a bucket. + +--- + +## 2. What the corpus actually says + +95 records; **69 describe a process failure**, 26 describe none. Percentages are of the 69. + +| Rank | Class | n | % | Shape | +|---|---|---|---|---| +| 1 | **Vacuous verification** | 17 | 25% | A check was green having done no work | +| 2 | **Twin-missed** | 14 | 20% | The fix hit one instance and missed its structural twin | +| 3 | **Symptom-keyed guard** | 9 | 13% | The guard keyed on the symptom seen, not the defect's mechanism | +| 4 | *(new classes — see §3)* | 8 | 12% | Proposed by raters as fitting nothing above | +| 5 | **Environment divergence** | 7 | 10% | Behaviour differs by interpreter/provider/hardware, untested there | +| 6 | **Wrapper drift** | 5 | 7% | A hand-maintained mirror drifted from the contract it mirrors | +| 7 | **Overclaim / stale claim** | 3 | 4% | An assertion stronger than its evidence, or since gone false | +| 7 | **String-predicate churn** | 3 | 4% | A regex/grep predicate needing repeated rounds | +| 9 | **Identity-not-capability** | 2 | 3% | Authorization scoped by *who*, not by *what is being changed* | +| 10 | **Masking guards** | 1 | 1% | Two guards on one condition; one hid the other's total failure | + +**This reorders the issue's own ranking.** #773 put twin-missed first (9 instances) and vacuous +verification second (8), from a 50-issue sample. Over the full record corpus they swap: **vacuous +verification is the most common single shape.** Overclaim also drops sharply as a *primary* cause +(4%) — raters overwhelmingly assigned it as a *secondary*. It is better understood as a modifier +that rides on another failure than as a class with its own detector. + +**Inter-rater control (measured, not asserted).** A sixth rater re-classified a stratified 15-issue +sample blind. **Raw agreement 12/15 (80%)** across an 11-category scheme. The three disagreements +are informative rather than noise: + +- **#177** (twin-missed → no-failure) and **#521** (no-failure → twin-missed) disagree at the + "did a process failure occur" boundary and point in **opposite directions**, so that boundary is + noisy but not systematically biased. The 26 clean-record count should be read as ±3. +- **#649** was rated twin-missed by one and wrapper-drift by the other — which is precisely the blur + that Family C in §3 exists to resolve. Both raters named the other class as secondary. + +The top-three separation is far larger than the disagreement. Treat ±1 rank as noise. + +--- + +## 3. Consolidation: four families, one of which the issue's taxonomy missed + +### Family A — reasoning about a representative instead of the population + +**#773's central hypothesis, confirmed, and it extends further than proposed.** The issue asked +whether twin-missed and vacuous verification are the same underlying error. They are — and +symptom-keyed guards belong with them. All three are the same mistake at different targets: + +- **twin-missed** — the *fix* was applied to a sample of the population, +- **vacuous-by-sampling** — the *verification* sampled an empty or unrepresentative subset, +- **symptom-keyed guard** — the *guard* matched only the member that was noticed. + +**Two numbers, because the honest one is smaller than the headline.** Merging the three classes as +rated gives **40 of 69 records (58%)**. But vacuous verification splits: only part of it is a +sampling error, and the rest belongs to Family B below. Assigning each of the 17 vacuous records to +one half or the other — **my split, not the raters'** — puts ~9 in the sampling half and ~8 in the +never-ran half. So Family A is **~32 of 69 (46%)**, and 58% is the upper bound that holds only if +you accept the issue's original hypothesis wholesale. Either way it is the largest family by a wide +margin. + +The merge is not merely rhetorical; it predicts a single detector, and the corpus already contains +that detector, independently reinvented several times: + +> "the by-id handler covered 4 of 10 media types… **Sweep by FIELD, not by the call site the issue +> names**" — #671 +> "#644's guard keyed on the *symptom* (an inflated pageSize) rather than the *defect*… an at-cap +> request was structurally invisible to it" — #650 +> "filtered on tools that *already* declared `pageNum`, so a tool wrapping a paged endpoint with no +> paging args escaped it entirely" — #616 +> "the pre-existing test filters `Where(t => t.QueryParameters is {Count: > 0})`, so a tool that +> lost its query parameters escaped it entirely" — #757 + +Four sessions, four reinventions of one rule → **detector A** in §4. + +### Family B — the check never ran at all + +The other ~8 of the 17 vacuous-verification records are not sampling errors: the check was *dead*. `scripts/tests/` +invoked by no CI job (#631); a validator whose call site could be deleted with the suite still green +(#621); a `${{ }}` in a shell **comment** silently dropping a step while the job reported success in +6s (#751); `if ! cmd; then status=$?` reading bash's logical negation, so a failing spec run exited 0 +(#445); new pre-push logic never wired to receive stdin, "dead code that every unit test still passes +over" (#719). A sampling detector does nothing here. This needs **detector B**: proof the guard can +go red. + +### Family C — two copies of one thing + +Wrapper drift plus a chunk of twin-missed: `.claude/` and its byte-identical `.codex/` mirror where +only one was in the `PROTECTED` list (#711); an advisory local hook and the server-enforced workflow +where four rounds of hardening landed on the copy with the lower stakes (#649); a hand-maintained MCP +schema drifting from the generated OpenAPI DTO by one field, silently clearing it (#754, again #757); +a hand-rolled frontmatter parser accepting YAML that PyYAML rejects (#674); two byte-identical Lucene +escapers in different files (#440). The honest detector here is not a parity check — it is +**deletion**: one implementation, called from both places. + +### Family D — check-and-use race over mutable state *(the taxonomy missed this entirely)* + +Three raters independently proposed overlapping new classes that consolidate into one — and the +blind re-rater, having seen none of their output, proposed the same class again for #536 under +its own name (`toctou-partial-atomicity` vs `half-atomic-toctou`). That is convergent validity, not +a prompt echo: + +| # | The race | +|---|---| +| 536 | Check-then-act split by an `await`; the write side was atomic, the read side a stale `Volatile.Read` | +| 622 | Merge consent bound to a head sha, then Gitea's async auto-merge evaluated against a later head | +| 632 | Head sha bound, but retargeting the PR base changes the diff while moving neither sha nor status | +| 706 | Status writes are read-then-write with no compare-and-set; an older run can finish last and win | +| 707 | Paged file enumeration diffs each page against the base's *live* tip, so a mid-paging advance drops rows | + +**Five instances — larger than wrapper drift, identity-not-capability, string-predicate churn and +masking guards individually**, and #773's taxonomy had no bucket for it. The unifying property: +a check and the action it authorizes are separated in time over state that can change in between, +with nothing pinning a version. #622's own record puts it exactly: *"The gate was never bypassed — +it was satisfied against a snapshot that stops being true."* + +### Two further cross-era classes, from the pre-convention sample + +- **Fail-open by default** — a surface that defaults to permissive when config is absent or a step + vanishes: API writes open when `Api:WriteKey` is unset (#280); a dropped step is fail-CLOSED in + `review-verdict.yml` but fail-OPEN in `docker-build.yml` (#751, #756, #768); a jq-1.6-inert + classifier (#647). Distinct from identity-not-capability: nothing is mis-scoped, the *default* + is wrong. +- **Destructive replace** — a full-replace path that silently destroys state a reconcile-by-id + would have kept: a schedule PUT cascade-deleting fill-group progression on every save, "even a + no-op" (#252); a full-replace wrapper missing one field and clearing it (#754); a dedup fix that + turned a duplicate row into permanent data loss because the remove filter used a different key + (#500). + +--- + +## 4. Detectors + +Ranked by instances covered per unit of build cost. "Rule status" distinguishes a **docs problem** +(no rule exists) from a **hooks problem** (the rule exists and nothing enforces it) — they have +different fixes, and conflating them is why several of these recurred. + +| # | Detector | Covers | Rule status | +|---|---|---|---| +| **A** | **A guard derives its expected set from the authoritative source and asserts set equality — never `filter`, never a sample.** The population comes from the enum / the OpenAPI doc / the workflow YAML / the provider list, and the assertion is equality against it. A filter over the population cannot see the member that is missing. | Family A: ~32 records (46%); 40 (58%) if the whole vacuous class merges in | **rule-missing** as a general rule; instantiated ad hoc in #616, #644→#650, #671, #757 | +| **B** | **Every guard ships with a proof it can go red: delete or disarm *that guard alone* and the suite must fail.** Not "a test exists" — a mutation. | Family B + masking guards: ~9 records | **rule-present-unenforced** — stated in #685's record and in project memory, enforced nowhere | +| **C** | **Dedup by construction.** When two copies of one rule exist, delete one and have both callers invoke it. Prefer this over any parity check; a parity check is itself a Family-A guard that can be keyed wrong. | Family C: ~9 records | **rule-missing** | +| **D** | **Anything read-then-written against live Gitea/remote state pins a version or uses compare-and-set** — bind the base sha as well as the head, page against a pinned tip, make the status write conditional. | Family D: 5 records | **rule-missing** (partially addressed for the merge path by the per-sha required check from #622) | +| **E** | **Run the check under the interpreter/provider it will actually run under**, and preflight-log the version. | Environment divergence: 7 records | **rule-present** and working — `scripts/jq-preflight.sh` + `ci.jq-version-contract` closed the jq axis after #643/#647/#648. **Unclosed axes**: SQLite-vs-MySQL query semantics (#668), GPU generation (#505), CI-VM speed (#512) | +| **F** | **Test the DENY path with the production config value.** #756's lesson generalised: a fixture that omits a field tests only the default, so a fail-open in the production value stays invisible. Parametrise the whole matrix, including "config absent". | Fail-open-by-default: ~5 records | **rule-missing** | +| **G** | **Full-replace endpoints assert their complete field list in a test**, and prefer reconcile-by-id over delete-and-reinsert where child state exists. | Destructive replace: ~3 records | **rule-missing** | + +**Detector A is the highest-value single change in this analysis** — one convention, ~46% of the +recorded process failures, and it is already proven four times in this repo under four different +names. + +### Classes where no mechanical detector is plausible — stated rather than papered over + +The issue explicitly asked for this, and inventing a weak detector here would itself be the +symptom-keyed-guard mistake. + +- **Overclaim / stale claim.** No check can tell that a sentence is stronger than the evidence + behind it. Partial mitigations exist and should not be oversold: `stale-after` frontmatter (#603) + dates a claim, and #578's retracted-term grep catches a *specific known* retraction propagating + into generated artifacts. Neither detects a fresh overclaim. This stays a review responsibility. +- **Per-task review blindness** (#60). A review scoped to one task's diff structurally cannot see a + defect that only exists once several tasks compose — five Important findings at #60 were invisible + to six per-task reviews and surfaced only in a whole-branch pass. The fix is a *process* step + (a whole-branch review before close), not a check. +- **Omitted brief constraint** (#586). A delegated brief silent on a hazard gets a plausible-but-wrong + default — "an omitted rule isn't an unenforced rule, it's a rule replaced by whatever default the + agent reaches for." A brief lint is conceivable but would be a keyword matcher, i.e. exactly the + string-predicate class. A hazards checklist in the brief template is the honest ceiling. +- **Guard-parity by verb** (#458), and re-deriving an inherited exemption against a new failure mode + (#484). Both need judgment about whether a prior rationale still applies. + +### A meta-finding about this repo's own knowledge base + +`docs/decisions/records/` holds **189 active records**, and the two most relevant to this analysis +(`testing.enumerating-guard-identity-not-position`, `ci.required-job-step-execution-markers`) are +extraordinarily detailed — each a full account of one incident. The corpus grows **one record per +instance**. That is Family A operating on our own process: we are enumerating cases rather than +removing the mechanism. Detector A, C and B are class-level rules precisely because the per-instance +record has already been tried 189 times. + +--- + +## 5. Part 2 — audit of the tooling already configured + +Judged against the measured classes above, per the issue's sequencing. Everything here was run, not +assumed. + +### 5.1 The LSPs: two of three are broken, and the one that matters most is the most broken + +#773 asked whether routine LSP use would catch Family A (find *all* references rather than the one +in front of you). It would help — **and it is not available.** Measured in this session: + +| LSP | State | Evidence | +|---|---|---| +| `csharp-lsp` | **Broken — cannot initialize** | `findReferences` on `ChannelPlaylist.ToM3U()` → `System.InvalidOperationException: .NET SDK cannot be resolved, because libhostfxr.dylib cannot be found inside /opt/homebrew/Cellar/dotnet/10.0.302/bin/host/fxr`. That directory **does not exist** — Homebrew's dotnet layout is not what MSBuildLocator expects. `dotnet --version` works (10.0.302), so builds are fine; only the language server is dead. | +| `typescript-lsp` | **Broken at the repo root** | `Could not find a valid TypeScript installation… ensure that the "typescript" dependency is installed in the workspace`. `typescript` lives in `web/node_modules`, not at the workspace root. Configuration problem, not a missing dependency. | +| `pyright-lsp` | **Works** | `documentSymbol` on `scripts/decisions_lib.py` returned the full symbol tree; `findReferences` on `active_files` correctly returned 4 references across 3 files, including cross-file hits in `decisions_validate.py` and `migrate_decisions_split.py`. | + +**Two conclusions, and the second is the sharper one.** + +1. The LSP that would help most is the one that is dead. C# is where Family A bites hardest — #671 + (4 of 10 media types), #510 (three switch arms, not one), #403 (5 of 6 dispatch sites) are all + "find every site" problems in C#, and `csharp-lsp` cannot answer a single query. +2. **A dispatched subagent could not reach the LSP tool at all.** The agent tasked with testing the + three LSPs reported `ToolSearch` returning "No matching deferred tools found" for every query, + while the same tool resolved immediately in this main session. So the standing note + *"workflow agents must use csharp-lsp"* is doubly rotten: the server is broken **and** the agents + it addresses cannot invoke it even when it works. This is a live instance of overclaim/stale + claim (§2 rank 7) sitting in our own guidance. + +### 5.2 Python: the global instruction and this repo disagree, and the repo is silent + +`~/.claude/CLAUDE.md` instructs `ruff check`, `ruff format --check`, and `pyright` after modifying +Python. Measured against reality: + +| Claim | Measured | +|---|---| +| ruff/pyright run in CI or hooks | **No.** `grep -rn "ruff\|pyright" .gitea/workflows/ .husky/` returns **nothing**. | +| What *does* run on Python | `pr-checks.yml` → `decisions-guard` (`decisions_validate.py`, `build_decisions_catalog.py --check`) and `script-tests` (`pytest scripts/tests`). Neither Husky hook touches Python. | +| `ruff check scripts/` | **47 errors** — but 28 are `E702` (semicolons) and **7 `S105` "hardcoded password" are false positives** on test stubs (`env["ETV_GITEA_TOKEN"] = "stub"`). The real cleanup is small. | +| `ruff format --check scripts/` | **10 of 20 files would be reformatted.** | +| `pyright scripts/` | **2 errors**, both `reportMissingImports` for `etv_client` in `scripts/scripted-schedules/entrypoint.py` — a package resolvable only in that script's deploy environment. Effectively clean. | +| Repo-level ruff config | **None.** No `ruff.toml`/`pyproject.toml`. Ruff silently falls back to whichever `~/.config/ruff/ruff.toml` the operator's machine happens to have. | + +**The last row is the finding that matters**, and it is Family E (environment divergence) in our own +toolchain: Python lint behaviour here is a function of an un-versioned file on one laptop. A second +machine lints differently, or not at all. One of the two must move — either the repo adopts a +committed ruff config and enforces it, or the global instruction stops claiming this repo enforces +something it does not. + +### 5.3 Configured vs actually invoked + +Measured over the session transcript corpus (811 files under +`~/.claude/projects/-Users-timothy-ersatztv/`, this session excluded). + +| Tool | Configured | Actually invoked | Verdict | +|---|---|---|---| +| `gitea` MCP | project `.mcp.json` **and** user scope (duplicate, identical values) | Heavy, same-day | Keep; de-duplicate the config | +| `ssh-mcp` | project `.mcp.json` | 16 transcripts, same-day | Keep | +| `mempalace` MCP | user scope | 10 transcripts, **last hit 2026-07-25** (19 days) | Keep, but see below | +| **`LSP` (all three servers)** | 3 plugins enabled + `csharp-lsp` in `enabledMcpjsonServers` | **0 calls in 811 transcripts** | Broken *and* unused | +| `context7` | enabled, `CONTEXT7_API_KEY` set | 0 calls | Dead | +| `playwright` | enabled | 145 calls, last 2026-07-25 | Keep | +| `superpowers` | enabled | 29 transcripts, last 2026-08-04 | Keep | +| `feature-dev`, `ralph-loop`, `security-guidance` | enabled | 0 skill invocations | Dead as configured | +| `codex` plugin | enabled | 0 *skill* invocations — but the documented workflow is `codex exec` via Bash, which this search cannot see | Not dead; measured the wrong surface | +| `nuget`, `docker-mcp` | in `.mcp.json`, explicitly **disabled** | 0 (expected) | Correctly off | + +The zero for `LSP` is verified rather than assumed: the same query shape returns 23,661 `Bash` calls +and 5,102 `Read` calls over the same corpus, so the search was demonstrably capable of finding hits. +*(The first attempt at this count returned 0 for `Bash` as well — a `rg -c` output-format slip that +summed the file path instead of the count. A zero without a positive control is exactly the vacuous +verification this document is about.)* + +**Hooks are the good news.** All 13 scripts in `.claude/hooks/` are wired from either +`.claude/settings.json` or `.husky/*`, and no settings entry points at a missing path — there are +**no dead hook scripts**, contrary to the issue's suspicion. Husky hooks have genuine fired-output +evidence in transcripts (`husky - dotnet format found whitespace/BOM issues`, `husky - commit message +missing Co-Authored-By trailer`, `husky - refusing to commit root-level screenshot(s)`). + +### 5.4 We cannot currently tell whether our own hooks fire + +The most consequential Part 2 finding, and it was volunteered by the inventory as a limit on its own +evidence. In this harness version only **`Stop`** hooks emit a structured record +(`stop_hook_summary`/`hookInfos`). `PreToolUse` and `PostToolUse` hooks — which is *every* guard that +matters here: merge consent, worktree, BOM, agent model/RAM — leave **no durable execution trace**. +The counts for those rows are filename mentions in settings dumps and prior audits, not proof of +execution. Only `design-sync-reminder.sh`'s Stop path and the three husky message strings are +confirmed fires. + +So the guards this repo relies on most are exactly the ones whose execution we cannot observe. That +is Family B (the check never ran) applied to the hook layer itself, and this project has already +paid for it twice at the CI layer — #751's guard that reported success in 6s having run nothing, and +#756's fail-open twin. The standing lesson from #756 applies unchanged: **make the system report it +rather than infer it.** A one-line append to a session-scoped log from each hook would convert every +row in that table from inference to measurement, and it is cheap. + +**On MemPalace**, the 19-day gap deserves care rather than a verdict: `CLAUDE.md` makes it the +documented discovery path for decisions, so either sessions are skipping the documented path, or +they are correctly reading `docs/decisions/README.md` directly (which the same contract permits, and +which is authoritative). The measurement cannot distinguish those, and I am not going to guess. + +--- + +## 6. Follow-ups + +This issue is analysis; it spawns implementation rather than doing it. + +| Issue | Detector / finding | Covers | Priority | +|---|---|---|---| +| #774 | **A** — a guard derives its population from the authoritative source and asserts set equality | Family A, ~32 records (46%) | high | +| #775 | **B** — every guard ships a mutation proof: delete that guard alone, see red | Family B + masking guards, ~9 records | high | +| #776 | Make hooks report that they fired — PreToolUse/PostToolUse execution is unobservable | the whole hook layer | high | +| #777 | `csharp-lsp` and `typescript-lsp` are broken; the "workflow agents must use csharp-lsp" note is stale | partial mitigation for Family A | medium | +| #778 | **D** — pin a version or use compare-and-set for read-then-write against live remote state | Family D, 5 records | medium | +| #779 | **F + G** — test the deny path with the production config value; assert full-replace field lists | fail-open + destructive-replace, ~8 records | medium | +| #780 | Reconcile Python tooling — commit a ruff config and enforce it, or stop claiming we do | environment divergence in our own toolchain | medium | +| #781 | Retire enabled-but-never-invoked plugins/MCP servers; de-duplicate the `gitea` server | config that reads as coverage | low | + +**Detector C (dedup by construction) has no issue of its own on purpose.** It is not a thing to build; +it is the shape the fixes in #774 and #778 should take when they find two copies of one rule. Filing +it separately would invite a parity check, which is the weaker construction the corpus already +rejected twice (#649, #711). + +Not filed, deliberately: overclaim/stale-claim, per-task review blindness, and omitted brief +constraints. §4 argues no plausible mechanical detector exists for these, and filing an issue for +each would produce exactly the weak enumerating guard this analysis recommends against.