From b760de8b77a277d4fc7dcd6d22b8a26d770d40a1 Mon Sep 17 00:00:00 2001 From: Timothy Date: Fri, 28 Aug 2026 14:55:27 +0200 Subject: [PATCH 1/5] docs(781,799): re-measure the tooling audit from a derived population, and adopt serena MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §5.3's verdicts rested on a single surface (tool/skill invocation counts), which manufactured three false zeros: `codex` is used via `codex exec` inside Bash, `security-guidance` exposes no tool at all and runs as a hook, and `feature-dev` is used through its agents rather than its skill. Re-measured across all six surfaces a thing can actually be used through, with the population derived from the config files instead of listed by hand. The retirement half of #781 is answered *no* on evidence rather than left open: the zeros split five ways (broken, wrong surface, reachable-but-uncalled, aged out of a rolling corpus, genuinely unused) and only the last is grounds for removal. Eight plugins are genuinely unused and are deliberately kept — operator decision, recorded in the table. #799's premise is refuted rather than fixed: serena's tools appear in the tool-availability listing of six transcripts on 2026-08-14, the day the issue claimed they never reach a session, and in 191 transcripts since. It was never unreachable, only never called — what was missing is an `activate_project`. Adopted and documented as the third code-intelligence surface, with `.serena/` gitignored because activation writes it into the project root. Also removed the disabled `nuget` entry from the local `.mcp.json` (its command names `/usr/local/share/dotnet/dotnet`, verified absent) and the dangling `nuget` reference it left in `disabledMcpjsonServers`. Both files are gitignored, so that part is not visible in this diff. fixes #781 fixes #799 Co-Authored-By: Claude Opus 5 (1M context) --- .gitignore | 5 ++ docs/README.md | 12 ++--- docs/defect-shapes-773.md | 108 ++++++++++++++++++++++---------------- docs/local-lsp-tooling.md | 55 ++++++++++++++++++- 4 files changed, 128 insertions(+), 52 deletions(-) diff --git a/.gitignore b/.gitignore index 467575a81..0c9ad4ad3 100644 --- a/.gitignore +++ b/.gitignore @@ -95,3 +95,8 @@ web/playwright-report/ # plaintext Gitea credential and absolute /Users paths, so it is neither portable nor safe to # commit. See ersatztv#711 for the related merge-gate gap. .codex/ + +# serena's per-project state, written by `activate_project` (ersatztv#799). Holds project.yml, +# project.local.yml and a language-server cache; the path is machine-specific and the cache is +# large. Activation is per DIRECTORY, so each worktree writes its own copy. +.serena/ diff --git a/docs/README.md b/docs/README.md index 664bbe5ca..af59a63f1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -34,7 +34,7 @@ doc below, or that changes which sections a task signal points to.** | Writing or editing any doc, or answering a review finding in prose | `docs/decisions/records/docs/no-session-narrative.md` — the doc records the END STATE; the path to it goes in the commit message. Apply the who-benefits test, and read the carve-out before you cut (dated measurements, stated snapshot boundaries and tested-and-rejected results stay) | | Adding / changing / deleting a guard file | `docs/guard-inventory.md` — every guard's row is machine-checked by `scripts/tests/test_guard_inventory.py`, so a new guard must acquire a row before the suite goes green, and a row graded `MUTATION` must also acquire a declared clause in `scripts/tests/mutation_manifest.py` | | Writing code that reads live Gitea/remote state and then acts on it | `docs/decisions/records/process/check-and-use-pins-a-version.md`, then `docs/remote-state-inventory.md` — a new executable under `scripts/` (**excluding `scripts/tests/`**), `.claude/hooks/`, `.husky/` or `.gitea/workflows/` must acquire a row there before `scripts/tests/test_remote_state_inventory.py` goes green | -| Finding every site that references a symbol (multi-site fix/sweep) | `docs/local-lsp-tooling.md` — which surface answers, and why a delegated agent must be pointed at the `csharp-lsp` MCP tools rather than the `LSP` tool | +| Finding every site that references a symbol (multi-site fix/sweep) | `docs/local-lsp-tooling.md` — which of the three surfaces answers, and why a delegated agent must be pointed at an MCP server (`csharp-lsp`, or `serena` after an `activate_project`) rather than the `LSP` tool | | Live local run / Playwright-MCP verification | `docs/e2e-local.md` + `scripts/e2e-local.sh` | | Adding/changing a UI-E2E browser flow | `docs/e2e-local.md` → "UI-E2E harness" + `scripts/e2e-ui.sh` | | What does a test suite cover | `docs/testing.md` | @@ -77,11 +77,11 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K - **`docs/spa-conventions.md`** — playbook for adding a screen to the ChicoryTV React SPA. - **`docs/e2e-local.md`** (+ `scripts/e2e-local.sh`) — how to run a live local instance for manual or Playwright-MCP verification. -- **`docs/local-lsp-tooling.md`** — the code-intelligence surfaces (the `LSP` tool's three servers and - the `csharp-lsp` MCP server): how each is configured, which one a **subagent** can actually reach, - the traps (a cold server answers the first query with a confidently partial result), and - `scripts/check-local-lsp.sh` to verify the preconditions. Read before briefing an agent to find - every site referencing a symbol. +- **`docs/local-lsp-tooling.md`** — the code-intelligence surfaces (the `LSP` tool's three servers, + the `csharp-lsp` MCP server, and `serena`): how each is configured, which ones a **subagent** can + actually reach, the traps (a cold server answers the first query with a confidently partial result; + serena needs an `activate_project` per directory), and `scripts/check-local-lsp.sh` to verify the + preconditions. Read before briefing an agent to find every site referencing a symbol. - **`docs/testing.md`** — testing map: what each `*.Tests` project / `web` suite covers, golden-file nets, the timezone-independence rule, how to run subsets, the per-PR verification gate. diff --git a/docs/defect-shapes-773.md b/docs/defect-shapes-773.md index fa0637f02..24d64688b 100644 --- a/docs/defect-shapes-773.md +++ b/docs/defect-shapes-773.md @@ -485,56 +485,75 @@ list and the measured exclude matrix: `ci.python-lint-ruff-config-committed`. ### 5.3 Configured vs actually invoked -Measured over the session transcript corpus (811 files under -`~/.claude/projects/-Users-timothy-ersatztv/`, this session excluded). +**Measured 2026-08-28 (#781/#799).** Population derived from config, not listed by hand: +`~/.claude/settings.json` `enabledPlugins`, `~/.claude.json` `mcpServers` (plus its per-project +`projects[].mcpServers`), the project `.mcp.json`, and `.claude/settings.local.json`. Corpus: **486 +transcripts** under `~/.claude/projects/` via `rglob` (a top-level glob sees 209 and manufactures +false zeros), this session excluded. Positive control on the same query shape: 22,927 `Bash`, 1,420 +`Read`, 1,116 `Edit`. -| Tool | Configured | Actually invoked | Verdict | +**A count is only meaningful against the surface a thing is actually used through**, so each row +below names its surface. Six surfaces exist and an audit that checks one of them will invent zeros: +MCP tool calls (`mcp____*`), `Skill` invocations, `Agent` `subagent_type`, user-typed slash +commands (``), hooks (fire in `stop_hook_summary`, never as a tool call), and CLIs +invoked inside `Bash` (`codex exec`). + +| Entry | Scope / surface | Measured | Verdict | |---|---|---|---| -| `gitea` MCP | project `.mcp.json` **and** user scope (duplicate, identical values) | Heavy, same-day | Keep; de-duplicate the config | -| `ssh-mcp` | project `.mcp.json` | 16 transcripts, same-day | Keep | -| `mempalace` MCP | user scope | 10 transcripts, **last hit 2026-07-25** (19 days) | Keep, but see below | -| **`LSP` (all three servers)** | 3 plugins enabled + `csharp-lsp` in `enabledMcpjsonServers` | **0 calls in 811 transcripts** | Broken *and* unused — both C# and TS servers repaired 2026-08-14 (#777); the zero is the baseline a future measurement is compared against | -| `context7` | enabled, `CONTEXT7_API_KEY` set | 0 calls | Dead | -| `playwright` | enabled | 145 calls, last 2026-07-25 | Keep | -| `superpowers` | enabled | 29 transcripts, last 2026-08-04 | Keep | -| `feature-dev`, `ralph-loop`, `security-guidance` | enabled | 0 skill invocations | Dead as configured | -| `codex` plugin | enabled | 0 *skill* invocations — but the documented workflow is `codex exec` via Bash, which this search cannot see | Not dead; measured the wrong surface | -| `nuget`, `docker-mcp` | in `.mcp.json`, explicitly **disabled** | 0 (expected) | Correctly off | +| `gitea` MCP | user `~/.claude.json` | **975 calls**, 105 transcripts, last 2026-08-28 | Keep — heaviest tool in the corpus | +| `codex` plugin | `codex exec` inside `Bash` | **256 calls**, 25 transcripts, last 2026-08-27 | Keep — second-heaviest; invisible to a skill-name search | +| `playwright` | plugin MCP | 64 calls, last 2026-08-27 | Keep | +| `security-guidance` | **hooks** (`hooks/security_reminder_hook.py`) | Fires on 20 distinct days, last 2026-08-28 | Keep — **not** dead; it exposes no tool at all | +| `ssh-mcp` | project `.mcp.json` | 16 calls, 9 transcripts, last 2026-08-22 | Keep | +| `superpowers` | `Skill` | 13 invocations (`systematic-debugging` 6, `brainstorming` 5, …), last 2026-08-26 | Keep | +| `mempalace` | user `~/.claude.json` | 11 calls, 7 transcripts, last 2026-08-27 | Keep — in active use | +| `feature-dev` | **`Agent` subagent_type** | `feature-dev:code-reviewer` ×2, last 2026-08-21 | Keep — **not** dead; used via agents, not skills | +| `serena` | plugin MCP | 0 invocations, but reachable throughout and **demonstrated working 2026-08-28** | Keep — see #799 and `docs/local-lsp-tooling.md` | +| `LSP` tool (`csharp-lsp`, `pyright-lsp`, `typescript-lsp` plugins) | `LSP` | 13 calls, last 2026-08-14 — all §5.1/#777 probes | Repaired but unadopted; no post-repair use | +| `csharp-lsp` MCP server | project `.mcp.json` | **0** since it became startable on 2026-08-14 | Reachable, unadopted | +| `context7`, `code-review`, `code-simplifier`, `ralph-loop`, `commit-commands`, `claude-md-management`, `skill-creator`, `frontend-design` | MCP / `Skill` / `Agent` / slash command | **0 on every one of the six surfaces**; installed since March | Genuinely unused. **Deliberately kept** — operator decision 2026-08-28 | +| `nuget` | project `.mcp.json`, explicitly disabled | 0 (expected); `command` names `/usr/local/share/dotnet/dotnet`, **verified absent** | Entry removed 2026-08-28 | +| `docker-mcp` | project `.mcp.json`, explicitly disabled | 0 (expected) | Correctly off | +| `homeassistant` | `~/.claude.json` `projects[/Users/timothy/server-management]` | 129 calls | Not in this repo's population — scoped to another project | -The zero for `LSP` is verified rather than assumed: the same query shape returns 23,661 `Bash` calls -and 5,102 `Read` calls over the same corpus, so the search was demonstrably capable of finding hits. -**Re-derive it with that positive control**, not on its own — `rg -c` over multiple files prints -`path:count`, so the obvious way to total it sums the paths and returns zero for everything. +`claude-in-chrome` also appears in the corpus (154 calls) and is **not** configured anywhere in these +files: it is harness-provided. An audit deriving its population from config alone will not see it. -**Re-measured 2026-08-14 (#781), and the table above needs two corrections plus seven additions.** -The population here was assembled **by hand**; deriving it from the config files instead -(`~/.claude/settings.json` `enabledPlugins`, `~/.claude.json` `mcpServers`, the project `.mcp.json`, -`.claude/settings.local.json`) surfaces **seven enabled plugins the table omits** — `serena`, -`code-review`, `code-simplifier`, `commit-commands`, `claude-md-management`, `skill-creator`, -`frontend-design`. An audit whose completeness rests on remembering every entry is exactly the -failure `testing.guard-derives-population-from-source` exists to prevent, applied to an audit. +**Why "retire what is enabled and never invoked" was not carried out.** #781 assumed a zero means +useless. Across two re-measurements the zeros here split **five** ways, and only the last is grounds +for removal on its own: -The corpus is also larger than counted: **964** transcripts via `rglob`, not 811 — a top-level-only -glob sees 209 here and manufactures false zeros. Positive control on the same query: 24,762 `Bash`, -5,275 `Read`, 4,037 `Edit`. +1. **Broken** — `csharp-ls` and the `csharp-lsp` MCP server named a dotnet root this machine does not + have (#777). Unreachable, not unwanted. +2. **Measured on the wrong surface** — `codex` (256 `Bash` calls), `security-guidance` (hooks only), + `feature-dev` (agents, not skills). Three separate instances of one mistake; each looked like a + confident zero. +3. **Reachable but never called** — `serena`. See below; this is the one that most resembles + "unreachable" and is not. +4. **Evidence aged out** — the corpus is a rolling window. It held 964 transcripts on 2026-08-14 and + **486** on 2026-08-28, reaching back only to 2026-07-29. A zero can now mean the calls expired, + and an older measurement's counts are not reproducible from a later corpus. +5. **Genuinely unused** — the eight plugins above, zero across all six surfaces. -| Row above | Re-measured | -|---|---| -| `mempalace` "last hit 2026-07-25 (19 days)" | **31 calls, last seen 2026-08-14.** In active use; the gap was a snapshot artifact, not disuse. | -| `codex` "0 skill invocations" | **113 `codex exec` calls across 17 sessions** — the third-heaviest tool in the corpus. The caveat was right; this is the number. | -| `LSP` "0 calls" | Not zero. The handful of calls are from sessions *testing whether the LSP works* — including §5.1's own probes. | -| `gitea` "duplicate, identical values" | Duplicated but **not identical** — the user-scope entry carries `"type": "stdio"`. **Resolved:** the *project* copy was removed, not the user one, because `server-management` and `homelab-docs` have no `.mcp.json` and depend on user scope. | -| *(absent)* `serena` | Enabled; starts and initializes standalone (Serena 1.28.1) but its tools never appear in a session and `ToolSearch` cannot resolve them. Cause unproven → **#799**. | -| *(absent)* six further plugins | Three (`claude-md-management`, `skill-creator`, `frontend-design`) were enabled **on 2026-08-14**, so their zeros mean "not yet tried". The rest are reachable and simply unused. | +Establishing *which* of the five applies is the precondition for acting, and doing so cost two +sessions here. What was actually wrong with this configuration was **duplication** and **breakage**, +never excess. -**The issue's framing does not survive the measurement, and that is the finding.** "Retire what is -enabled and never invoked" treats a zero as evidence of uselessness. These zeros split **four** ways: -**broken** (`csharp-lsp` MCP and two LSP servers — #777), **unreachable** (`serena`), **just enabled** -(three plugins), and **measured on the wrong surface** (`codex`). Establishing *why* a counter is zero -is a precondition for acting on it — and on this evidence almost nothing should be removed. What was -actually wrong was **duplication** and **breakage**, not excess. The one entry still reading as -coverage it does not provide is `nuget`, which names the same dead `/usr/local/share/dotnet` path -#777 fixed elsewhere; it is inert only because it is explicitly disabled. +**`serena` was never unreachable (#799).** #799 recorded "its tools never appear in a session and +`ToolSearch` cannot resolve them". That is refuted by the transcripts: on **2026-08-14 itself**, the +day it was filed, `mcp__plugin_serena_serena__*` appears in the tool-availability listing of **6 +transcripts**, and in 191 transcripts across 11 days since — always as an availability listing, +never merely as prose discussing the issue. The tools were offered every session and never called. +What was genuinely missing was an `activate_project` call, without which the first query returns a +"no active project" error naming the projects it does know. Both #799's ToolSearch phrasings resolve +today. The lesson is #781's own, turned on itself: **a probe that reports "absent" is a measurement, +and it needs a positive control exactly like a count does.** + +`.mcp.json` is **gitignored**, so none of its content is reviewable from this repo and the +de-duplication recorded below happened on one machine only. Two scopes defined `gitea` (the user +entry additionally carrying `"type": "stdio"`); the **project** copy was removed rather than the user +one, because `server-management` and `homelab-docs` have no `.mcp.json` and depend on user scope. +Verified afterwards by starting the surviving server: 54 tools. **Hooks are the good news.** All 13 scripts in `.claude/hooks/` are wired from either `.claude/settings.json` or `.husky/*`, and no settings entry points at a missing path — there are @@ -675,7 +694,8 @@ This issue is analysis; it spawns implementation rather than doing it. | #778 | **D** — pin a version or use compare-and-set for read-then-write against live remote state | Family D, 5 records | medium | | #779 | **F + G** — test the deny path with the production config value; assert full-replace field lists | fail-open + destructive-replace, ~8 records | medium | | #780 | Reconcile Python tooling — commit a ruff config and enforce it, or stop claiming we do | environment divergence in our own toolchain | medium | -| #781 | Retire enabled-but-never-invoked plugins/MCP servers; de-duplicate the `gitea` server | config that reads as coverage | low | +| #781 | Retire enabled-but-never-invoked plugins/MCP servers; de-duplicate the `gitea` server | config that reads as coverage | low — **DONE**, §5.3 is re-measured from a derived population; the retirement half was answered *no* on evidence | +| #799 | `serena` is enabled but never invoked — establish why, then use it or disable it | one row of §5.3 | low — **DONE**, it was reachable all along and is now adopted (`docs/local-lsp-tooling.md`) | **Detector C (dedup by construction) has no issue of its own on purpose.** It is not a thing to build; it is the shape the fixes in #774 and #778 should take when they find two copies of one rule — with diff --git a/docs/local-lsp-tooling.md b/docs/local-lsp-tooling.md index 25ea59a49..56f7c3c17 100644 --- a/docs/local-lsp-tooling.md +++ b/docs/local-lsp-tooling.md @@ -1,4 +1,4 @@ -# Local code-intelligence tooling (LSP + the csharp-lsp MCP server) +# Local code-intelligence tooling (LSP, the csharp-lsp MCP server, serena) What is available for "find every site that references this symbol", how each surface is configured, and how to verify it rather than assume it. Run `scripts/check-local-lsp.sh` to check all of it at @@ -11,7 +11,7 @@ not cover the residue where the population is *sites in code* — #403 (5 of 6 d (a by-id handler covering 4 of 10 media types). Find-all-references is the tool for that residue, and it is overwhelmingly a C# problem here. -## The two surfaces, and which one a subagent can reach +## The three surfaces, and which one a subagent can reach This distinction is the whole reason both halves are documented together. @@ -19,6 +19,7 @@ This distinction is the whole reason both halves are documented together. | --- | --- | --- | | The **`LSP` tool** (Claude Code plugins) | `csharp-ls`, `typescript-language-server`, `pyright-langserver` | **No dispatched subagent has been observed to reach it.** `ToolSearch` returns "No matching deferred tools found" for `select:LSP` in a subagent while the same query resolves in the main session — Claude Code 2.1.232, agent types `general-purpose` and `Explore`, 2026-08-14, plus the independent observation in `docs/defect-shapes-773.md` §5.1. Three observations on one harness version: treat it as measured behaviour to design around, not an architectural guarantee. | | The **`csharp-lsp` MCP server** (`.mcp.json`) | wraps `csharp-ls`; serves `csharp_references`, `csharp_diagnostics`, `csharp_hover`, `csharp_definition`, `csharp_symbols`, `csharp_completions`, `csharp_set_workspace`, … (16 tools) | **Main session and subagents.** Subagents demonstrably call MCP tools here: 326 MCP calls from inside subagent turns across the transcript corpus, spanning four servers (`gitea` 288, `playwright` 26, `ssh-mcp` 6, `mempalace` 6). Not yet demonstrated for `csharp-lsp` specifically — that server only became startable on 2026-08-14 — so this is the general MCP boundary, evidenced, rather than a per-server measurement. | +| The **`serena` MCP server** (plugin `serena@claude-plugins-official`) | its own bundled Roslyn C# server, plus ~30 other language servers it selects per project | **Main session and subagents** (MCP, same boundary as above). Cross-language, and the only surface here that is not C#-only. Requires an explicit `activate_project` per directory — see below. | So an instruction telling *delegated agents* to use C# code intelligence must point them at the **MCP tools**, never at the `LSP` tool. Briefing harder does not help: an agent that cannot resolve @@ -153,6 +154,56 @@ The alternative is to install the .NET 8 SDK and build upstream unmodified. Eith not fine is leaving it undocumented, because `scripts/check-local-lsp.sh` now *starts* the server, so a wrong pin or a missing patch surfaces as a failed smoke test rather than a false pass. +### The `serena` MCP server + +`serena` is an enabled plugin (`serena@claude-plugins-official`) declaring one MCP server: + +```json +"serena": { + "command": "uvx", + "args": ["--from", "git+https://github.com/oraios/serena", "serena", "start-mcp-server"] +} +``` + +Nothing about it needs configuring in this repo. It answers the same find-all-references question as +`csharp-lsp`, across languages rather than C# only, and being MCP it is reachable from dispatched +subagents — which is the gap the `LSP` tool leaves. + +**It requires an explicit project activation, and that is what made it look broken.** A fresh server +has no active project; the first real call fails with a message naming the projects it does know: + +``` +No active project. Ask the user to provide the project path or to select a project +from this list of known projects: ['adversarial-reviewer', 'server-management', ...] +``` + +That error is the tell, and it is the system reporting the cause rather than the caller inferring it +from an empty result. Activate by absolute path, once per directory: + +``` +mcp__plugin_serena_serena__activate_project project=/Users/timothy/ersatztv +``` + +Demonstrated in this repo on 2026-08-28 — `find_symbol` on `ChannelPlaylist` +(`ErsatzTV.Core/Iptv/ChannelPlaylist.cs`) returns the class and its constructor, and +`find_referencing_symbols` on `ErsatzTV.Core.Iptv/ChannelPlaylist` returns references across **7 +files** spanning `ErsatzTV.Core.Tests`, `ErsatzTV/Controllers`, `ErsatzTV/Formatters` and +`ErsatzTV.Application` — the cross-project spread that makes it useful for the sites-in-code +residue. + +Traps specific to it: + +- **Activation is per directory, not per repository.** A git worktree is a different path, so it + registers as a *separate* serena project named after the worktree directory. Activate each one you + work in; do not assume the canonical checkout's activation carries over. +- **It writes `.serena/` into the project root** — `project.yml`, `project.local.yml` and a language + server cache. Gitignored here since #799; activating in a checkout that predates that entry leaves + an untracked directory. +- **It starts its own Roslyn server**, `Microsoft.CodeAnalysis.LanguageServer.dll` under + `~/.serena/language_servers/`, independent of `csharp-ls`. That is a *second* C# language server + with its own multi-minute warm-up, so the cold-query trap below applies to it too — and running it + alongside a `dotnet build` or a `csharp-lsp` solution load competes for the same cores. + ## Traps - **A query issued before the project graph is loaded is answered anyway, and answered wrongly.** -- 2.47.3 From 3f99aecddcb4d276261e882c3e6f2a13d97b0ad2 Mon Sep 17 00:00:00 2001 From: Timothy Date: Fri, 28 Aug 2026 14:59:09 +0200 Subject: [PATCH 2/5] docs(781): tighten two claims that outran their evidence The surface enumeration asserted "six surfaces exist", which is an absolute claim about the world rather than about this corpus. MCP resources/prompts are a seventh; they measure 0 across every server, so naming them costs a sentence and removes a hole a reader could fall into. The security-guidance row said "fires on 20 distinct days" from a loose grep that would also match a session merely discussing the hook. Recounted against `stop_hook_summary` records naming the hook itself: 1,080 executions across 79 transcripts, same 20 days. Co-Authored-By: Claude Opus 5 (1M context) --- docs/defect-shapes-773.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/defect-shapes-773.md b/docs/defect-shapes-773.md index 24d64688b..43f9a059f 100644 --- a/docs/defect-shapes-773.md +++ b/docs/defect-shapes-773.md @@ -493,17 +493,20 @@ false zeros), this session excluded. Positive control on the same query shape: 2 `Read`, 1,116 `Edit`. **A count is only meaningful against the surface a thing is actually used through**, so each row -below names its surface. Six surfaces exist and an audit that checks one of them will invent zeros: -MCP tool calls (`mcp____*`), `Skill` invocations, `Agent` `subagent_type`, user-typed slash -commands (``), hooks (fire in `stop_hook_summary`, never as a tool call), and CLIs -invoked inside `Bash` (`codex exec`). +below names its surface. Six carry real use in this corpus, and an audit that checks only one of +them will invent zeros: MCP tool calls (`mcp____*`), `Skill` invocations, `Agent` +`subagent_type`, user-typed slash commands (``), hooks (which fire in +`stop_hook_summary` and never appear as a tool call), and CLIs invoked inside `Bash` (`codex exec`). +A seventh — MCP **resources/prompts** (`ListMcpResourcesTool`, `ReadMcpResourceTool`) — measures **0 +across every server**, so it is a surface no entry is used through rather than one that would +rescue a zero below. | Entry | Scope / surface | Measured | Verdict | |---|---|---|---| | `gitea` MCP | user `~/.claude.json` | **975 calls**, 105 transcripts, last 2026-08-28 | Keep — heaviest tool in the corpus | | `codex` plugin | `codex exec` inside `Bash` | **256 calls**, 25 transcripts, last 2026-08-27 | Keep — second-heaviest; invisible to a skill-name search | | `playwright` | plugin MCP | 64 calls, last 2026-08-27 | Keep | -| `security-guidance` | **hooks** (`hooks/security_reminder_hook.py`) | Fires on 20 distinct days, last 2026-08-28 | Keep — **not** dead; it exposes no tool at all | +| `security-guidance` | **hooks** (`hooks/security_reminder_hook.py`) | **1,080 execution records** (`stop_hook_summary` naming the hook) across 79 transcripts / 20 distinct days, last 2026-08-28 | Keep — **not** dead; it exposes no tool at all | | `ssh-mcp` | project `.mcp.json` | 16 calls, 9 transcripts, last 2026-08-22 | Keep | | `superpowers` | `Skill` | 13 invocations (`systematic-debugging` 6, `brainstorming` 5, …), last 2026-08-26 | Keep | | `mempalace` | user `~/.claude.json` | 11 calls, 7 transcripts, last 2026-08-27 | Keep — in active use | -- 2.47.3 From dabbd4c3e86bf3531fade5b2de3361b49761efac Mon Sep 17 00:00:00 2001 From: Timothy Date: Fri, 28 Aug 2026 15:30:42 +0200 Subject: [PATCH 3/5] fix(781,799): correct the serena causal story and the enablement-blind verdicts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two independent cold reviews (one cross-family) blocked the previous revision, both on the same defect the section exists to name: a zero read as "unused" without checking whether the thing was switched on. They were right. serena was not "reachable all along". It was `false` in ~/.claude/settings.json until 2026-08-14T12:31Z, when a concurrent session enabled it; its tools appear in no transcript before 12:42:54Z that day. The #799 session had started at 12:01Z and never ran /reload-plugins, so its probe at 13:41Z correctly reported the tools absent — while the settings file already said `true`. Both reviewers had half of this; the reload boundary is what reconciles them, since the session that performed the toggle DID acquire serena mid-run after reloading three times. So #799's observation was accurate and its cause is now established, rather than refuted as the previous revision claimed. The error underneath it was dating transcript evidence by file mtime instead of record timestamp, which places a session's records at the moment it last wrote and made a first-appearance look like a pre-existing state. Both rules are now recorded in the section. Six of the eight plugins called "genuinely unused" were disabled for 16 of the 30 corpus days (settings.json.bak-doctor), and ralph-loop is not unused at all — its Stop hook has 1,083 executions, the same cadence as security-guidance, since both fire on every stop. It is the sole enabled plugin owning hooks/stop-hook.sh, so the attribution is unambiguous. That is a fourth instance of the wrong-surface mistake, found in the act of documenting the first three. Also: "256 codex exec calls" counted Bash inputs *containing* the string, including pgrep and echo; at command position it is 117. The surface list no longer claims a fixed count, omits neither `LSP` nor `attributionSkill`, and says what was not separately identified. In local-lsp-tooling.md the subagent claim is marked expected-but-undemonstrated, "the only surface not C#-only" is dropped as false, ~30 language servers becomes the config's actual 73, and the multi-minute warm-up assertion is replaced by the one cold run's real timings. Co-Authored-By: Claude Opus 5 (1M context) --- .gitignore | 12 +++- docs/README.md | 2 +- docs/defect-shapes-773.md | 113 +++++++++++++++++++++++--------------- docs/local-lsp-tooling.md | 27 ++++++--- 4 files changed, 100 insertions(+), 54 deletions(-) diff --git a/.gitignore b/.gitignore index 0c9ad4ad3..7d8c7ae20 100644 --- a/.gitignore +++ b/.gitignore @@ -96,7 +96,13 @@ web/playwright-report/ # commit. See ersatztv#711 for the related merge-gate gap. .codex/ -# serena's per-project state, written by `activate_project` (ersatztv#799). Holds project.yml, -# project.local.yml and a language-server cache; the path is machine-specific and the cache is -# large. Activation is per DIRECTORY, so each worktree writes its own copy. +# serena's per-project state, written by `activate_project` (ersatztv#799): project.yml, +# project.local.yml, a language-server cache, and memories/. +# +# This deliberately rejects serena's own versioning model. Its nested .serena/.gitignore excludes +# only `cache` and `project.local.yml`, and project.local.yml says project.yml "is intended to be +# versioned" — but activation here is per DIRECTORY, and every worktree generates a project.yml +# whose project_name is that worktree's folder (e.g. `781-tooling`). A committed copy would name +# the wrong project in every checkout but the one that produced it. memories/ is ignored with it: +# it is serena's own written notes, and this repo's durable knowledge lives in docs/ instead. .serena/ diff --git a/docs/README.md b/docs/README.md index af59a63f1..154afc50e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -34,7 +34,7 @@ doc below, or that changes which sections a task signal points to.** | Writing or editing any doc, or answering a review finding in prose | `docs/decisions/records/docs/no-session-narrative.md` — the doc records the END STATE; the path to it goes in the commit message. Apply the who-benefits test, and read the carve-out before you cut (dated measurements, stated snapshot boundaries and tested-and-rejected results stay) | | Adding / changing / deleting a guard file | `docs/guard-inventory.md` — every guard's row is machine-checked by `scripts/tests/test_guard_inventory.py`, so a new guard must acquire a row before the suite goes green, and a row graded `MUTATION` must also acquire a declared clause in `scripts/tests/mutation_manifest.py` | | Writing code that reads live Gitea/remote state and then acts on it | `docs/decisions/records/process/check-and-use-pins-a-version.md`, then `docs/remote-state-inventory.md` — a new executable under `scripts/` (**excluding `scripts/tests/`**), `.claude/hooks/`, `.husky/` or `.gitea/workflows/` must acquire a row there before `scripts/tests/test_remote_state_inventory.py` goes green | -| Finding every site that references a symbol (multi-site fix/sweep) | `docs/local-lsp-tooling.md` — which of the three surfaces answers, and why a delegated agent must be pointed at an MCP server (`csharp-lsp`, or `serena` after an `activate_project`) rather than the `LSP` tool | +| Finding every site that references a symbol (multi-site fix/sweep) | `docs/local-lsp-tooling.md` — which of the three surfaces answers, and why a delegated agent must be pointed at an MCP server (`csharp-lsp`, or `serena` after an `activate_project`) rather than the `LSP` tool, which no subagent has been observed to reach | | Live local run / Playwright-MCP verification | `docs/e2e-local.md` + `scripts/e2e-local.sh` | | Adding/changing a UI-E2E browser flow | `docs/e2e-local.md` → "UI-E2E harness" + `scripts/e2e-ui.sh` | | What does a test suite cover | `docs/testing.md` | diff --git a/docs/defect-shapes-773.md b/docs/defect-shapes-773.md index 43f9a059f..1b68492c9 100644 --- a/docs/defect-shapes-773.md +++ b/docs/defect-shapes-773.md @@ -493,28 +493,32 @@ false zeros), this session excluded. Positive control on the same query shape: 2 `Read`, 1,116 `Edit`. **A count is only meaningful against the surface a thing is actually used through**, so each row -below names its surface. Six carry real use in this corpus, and an audit that checks only one of -them will invent zeros: MCP tool calls (`mcp____*`), `Skill` invocations, `Agent` -`subagent_type`, user-typed slash commands (``), hooks (which fire in -`stop_hook_summary` and never appear as a tool call), and CLIs invoked inside `Bash` (`codex exec`). -A seventh — MCP **resources/prompts** (`ListMcpResourcesTool`, `ReadMcpResourceTool`) — measures **0 -across every server**, so it is a surface no entry is used through rather than one that would -rescue a zero below. +below names its surface. Do not reason from a surface *count* — enumerate the event shapes and say +which you measured, because the whole failure here is an audit that checked one and reported zeros +for the rest. Measured: MCP tool calls (`mcp____*`), `Skill` invocations, `Agent` +`subagent_type`, user-typed slash commands (``), `LSP` tool calls, hooks (which appear +as `hookInfos` inside a `stop_hook_summary` record and *never* as a tool call), and CLIs invoked +inside `Bash`. Measured at zero everywhere: MCP **resource** tools (`ListMcpResourcesTool`, +`ReadMcpResourceTool`). **Not** separately identified: MCP prompts, and the `attributionSkill` +records that also appear in transcripts — so a zero below is a zero on the shapes listed, not on +every shape that exists. | Entry | Scope / surface | Measured | Verdict | |---|---|---|---| | `gitea` MCP | user `~/.claude.json` | **975 calls**, 105 transcripts, last 2026-08-28 | Keep — heaviest tool in the corpus | -| `codex` plugin | `codex exec` inside `Bash` | **256 calls**, 25 transcripts, last 2026-08-27 | Keep — second-heaviest; invisible to a skill-name search | +| `codex` plugin | `codex exec` inside `Bash`, plus `SessionStart`/`SessionEnd`/`Stop` hooks | **117 `Bash` inputs invoke it at command position**, 21 transcripts, last 2026-08-28 (258 merely *contain* the string — polling and inspection commands mention it, so a substring count overstates it) | Keep — heavily used, and invisible to a skill-name search | | `playwright` | plugin MCP | 64 calls, last 2026-08-27 | Keep | -| `security-guidance` | **hooks** (`hooks/security_reminder_hook.py`) | **1,080 execution records** (`stop_hook_summary` naming the hook) across 79 transcripts / 20 distinct days, last 2026-08-28 | Keep — **not** dead; it exposes no tool at all | +| `security-guidance` | **hooks** (`SessionStart`, `UserPromptSubmit`, `PostToolUse`, `Stop`) | **1,084 execution records** in `hookInfos` naming `security_reminder_hook`, across 79 transcripts / 23 UTC days, last 2026-08-28 | Keep — **not** dead; it exposes no tool at all | | `ssh-mcp` | project `.mcp.json` | 16 calls, 9 transcripts, last 2026-08-22 | Keep | | `superpowers` | `Skill` | 13 invocations (`systematic-debugging` 6, `brainstorming` 5, …), last 2026-08-26 | Keep | | `mempalace` | user `~/.claude.json` | 11 calls, 7 transcripts, last 2026-08-27 | Keep — in active use | | `feature-dev` | **`Agent` subagent_type** | `feature-dev:code-reviewer` ×2, last 2026-08-21 | Keep — **not** dead; used via agents, not skills | -| `serena` | plugin MCP | 0 invocations, but reachable throughout and **demonstrated working 2026-08-28** | Keep — see #799 and `docs/local-lsp-tooling.md` | +| `serena` | plugin MCP | **Disabled until 2026-08-14T12:31Z**; 0 invocations in the 14 days since it was enabled; **demonstrated working 2026-08-28** | Keep — see #799 and `docs/local-lsp-tooling.md` | | `LSP` tool (`csharp-lsp`, `pyright-lsp`, `typescript-lsp` plugins) | `LSP` | 13 calls, last 2026-08-14 — all §5.1/#777 probes | Repaired but unadopted; no post-repair use | | `csharp-lsp` MCP server | project `.mcp.json` | **0** since it became startable on 2026-08-14 | Reachable, unadopted | -| `context7`, `code-review`, `code-simplifier`, `ralph-loop`, `commit-commands`, `claude-md-management`, `skill-creator`, `frontend-design` | MCP / `Skill` / `Agent` / slash command | **0 on every one of the six surfaces**; installed since March | Genuinely unused. **Deliberately kept** — operator decision 2026-08-28 | +| `ralph-loop` | **hooks** (`Stop`) + slash command | Its `Stop` hook ran **1,083 times** across 78 transcripts / 23 days — the same cadence as `security-guidance`, because both fire on every stop. The `/ralph-loop` command itself: **0** | Keep — the hook executes and exits when no loop state exists; the *feature* is unused, the plugin is not inert | +| `context7` | plugin MCP | **0**, and it was enabled across the whole retained window | No observed use in the window. **Deliberately kept** — operator decision 2026-08-28 | +| `code-review`, `code-simplifier`, `commit-commands`, `claude-md-management`, `skill-creator`, `frontend-design` | `Skill` / `Agent` / slash command | **0** — but all six were **disabled** until 2026-08-14T12:41Z, so the zero covers only the 14 days since | Not yet exercised rather than rejected. **Deliberately kept** — operator decision 2026-08-28 | | `nuget` | project `.mcp.json`, explicitly disabled | 0 (expected); `command` names `/usr/local/share/dotnet/dotnet`, **verified absent** | Entry removed 2026-08-28 | | `docker-mcp` | project `.mcp.json`, explicitly disabled | 0 (expected) | Correctly off | | `homeassistant` | `~/.claude.json` `projects[/Users/timothy/server-management]` | 129 calls | Not in this repo's population — scoped to another project | @@ -523,34 +527,57 @@ rescue a zero below. files: it is harness-provided. An audit deriving its population from config alone will not see it. **Why "retire what is enabled and never invoked" was not carried out.** #781 assumed a zero means -useless. Across two re-measurements the zeros here split **five** ways, and only the last is grounds -for removal on its own: +useless. Across three re-measurements the zeros here split **six** ways, and none of them is grounds +for removal until you know which one you are looking at: 1. **Broken** — `csharp-ls` and the `csharp-lsp` MCP server named a dotnet root this machine does not have (#777). Unreachable, not unwanted. -2. **Measured on the wrong surface** — `codex` (256 `Bash` calls), `security-guidance` (hooks only), - `feature-dev` (agents, not skills). Three separate instances of one mistake; each looked like a - confident zero. -3. **Reachable but never called** — `serena`. See below; this is the one that most resembles - "unreachable" and is not. -4. **Evidence aged out** — the corpus is a rolling window. It held 964 transcripts on 2026-08-14 and - **486** on 2026-08-28, reaching back only to 2026-07-29. A zero can now mean the calls expired, - and an older measurement's counts are not reproducible from a later corpus. -5. **Genuinely unused** — the eight plugins above, zero across all six surfaces. +2. **Measured on the wrong surface** — `codex` (used through `Bash`), `security-guidance` and + `ralph-loop` (hooks, which never appear as a tool call), `feature-dev` (agents, not skills). Four + separate instances of one mistake, and each looked like a confident zero. +3. **Not enabled** — `serena` was `false` until 2026-08-14T12:31Z, and six of the plugins above until + 12:41Z the same day. *Installed* is not *enabled*: `installedAt` says March for all of them, and + only `enabledPlugins` says whether a thing could have been used at all. +4. **Enabled but never called** — `serena` since 2026-08-14, and the `csharp-lsp` MCP server since it + became startable. This is the only shape that resembles the issue's premise. +5. **Evidence aged out** — the corpus is a rolling window: 964 transcripts on 2026-08-14, **486** on + 2026-08-28, reaching back only to 2026-07-29. A zero can mean the calls expired, and an older + measurement's counts are not reproducible from a later corpus. +6. **No observed use in the window** — `context7`. Note the wording: the corpus starts 2026-07-29 and + these plugins date to March, so the evidence cannot distinguish "never used" from "not used + lately", and it cannot support "excess" as a historical claim. -Establishing *which* of the five applies is the precondition for acting, and doing so cost two -sessions here. What was actually wrong with this configuration was **duplication** and **breakage**, -never excess. +Establishing *which* of the six applies is the precondition for acting, and doing so cost three +sessions here. What was demonstrably wrong with this configuration was **duplication** and +**breakage**; excess was never shown. -**`serena` was never unreachable (#799).** #799 recorded "its tools never appear in a session and -`ToolSearch` cannot resolve them". That is refuted by the transcripts: on **2026-08-14 itself**, the -day it was filed, `mcp__plugin_serena_serena__*` appears in the tool-availability listing of **6 -transcripts**, and in 191 transcripts across 11 days since — always as an availability listing, -never merely as prose discussing the issue. The tools were offered every session and never called. -What was genuinely missing was an `activate_project` call, without which the first query returns a -"no active project" error naming the projects it does know. Both #799's ToolSearch phrasings resolve -today. The lesson is #781's own, turned on itself: **a probe that reports "absent" is a measurement, -and it needs a positive control exactly like a count does.** +**`serena` was disabled, and #799's observation was correct (#799).** The plugin was `false` in +`~/.claude/settings.json` until **2026-08-14T12:31Z**, when a *concurrent* session enabled it — +`~/.claude/settings.json.bak-doctor`, written moments earlier, still records `serena: false`. Its +tools appear in **no transcript before 2026-08-14T12:42:54Z**, then in 6 transcripts that day and +186 across the 10 days since, always in a harness-emitted availability listing rather than prose +(counts as of the 2026-08-28 snapshot; the corpus is live and grows). The +#799 session had started at 12:01Z, so its tool set never contained them; its `ToolSearch` probe at +13:41Z correctly reported them absent, and that probe *did* carry a positive control — two other +`ToolSearch` queries resolved in the same session minutes before. + +**A running session does not pick up a plugin a sibling session enables; it acquires it only on an +explicit `/reload-plugins` or a restart.** The session that performed the toggle ran +`/reload-plugins` three times and gained serena mid-run at 12:54Z despite having started at 12:05Z, +*before* the toggle — so this is a reload boundary, not a startup one. The #799 session never +reloaded. Its author read `serena: true` from the settings file and `serena: absent` from their own +tool set, and both were true simultaneously. + +Two things follow for any future audit. **Read enablement as of the evidence, not as of now** — an +audit that compares *current* config against *historical* usage will call a plugin unused for a +window in which it was switched off. And **date transcript evidence by record timestamp, never by +file mtime**: mtime is when a session last wrote, which can be hours or days after the event being +proved, and here it is what made a first-appearance look like a pre-existing state. + +The `activate_project` requirement is a separate, real trap, reachable only once the tools are +present: without it the first query returns a "no active project" error naming the projects it does +know. Both of #799's phrasings resolve today, and the demonstrated query is in +`docs/local-lsp-tooling.md`. `.mcp.json` is **gitignored**, so none of its content is reviewable from this repo and the de-duplication recorded below happened on one machine only. Two scopes defined `gitea` (the user @@ -690,15 +717,15 @@ This issue is analysis; it spawns implementation rather than doing it. | Issue | Detector / finding | Covers | Priority | |---|---|---|---| -| #774 | **A** — a guard derives its population from the authoritative source and asserts set equality | Family A, 27 records (39%) | high | -| #775 | **B** — every guard ships a mutation proof: delete that guard alone, see red | Family B, 9 records (13%) | high | -| #776 | Make hooks report that they fired — PreToolUse/PostToolUse execution is unobservable | the whole hook layer | high — **DONE**, §5.4 is measured | -| #777 | `csharp-lsp` and `typescript-lsp` are broken; the "workflow agents must use csharp-lsp" note is stale | partial mitigation for Family A | medium | -| #778 | **D** — pin a version or use compare-and-set for read-then-write against live remote state | Family D, 5 records | medium | -| #779 | **F + G** — test the deny path with the production config value; assert full-replace field lists | fail-open + destructive-replace, ~8 records | medium | -| #780 | Reconcile Python tooling — commit a ruff config and enforce it, or stop claiming we do | environment divergence in our own toolchain | medium | -| #781 | Retire enabled-but-never-invoked plugins/MCP servers; de-duplicate the `gitea` server | config that reads as coverage | low — **DONE**, §5.3 is re-measured from a derived population; the retirement half was answered *no* on evidence | -| #799 | `serena` is enabled but never invoked — establish why, then use it or disable it | one row of §5.3 | low — **DONE**, it was reachable all along and is now adopted (`docs/local-lsp-tooling.md`) | +| #774 | **A** — a guard derives its population from the authoritative source and asserts set equality | Family A, 27 records (39%) | high — **CLOSED** | +| #775 | **B** — every guard ships a mutation proof: delete that guard alone, see red | Family B, 9 records (13%) | high — **CLOSED** | +| #776 | Make hooks report that they fired — PreToolUse/PostToolUse execution is unobservable | the whole hook layer | high — **CLOSED**, §5.4 is measured | +| #777 | `csharp-lsp` and `typescript-lsp` are broken; the "workflow agents must use csharp-lsp" note is stale | partial mitigation for Family A | medium — **CLOSED**, §5.1 | +| #778 | **D** — pin a version or use compare-and-set for read-then-write against live remote state | Family D, 5 records | medium — **CLOSED** | +| #779 | **F + G** — test the deny path with the production config value; assert full-replace field lists | fail-open + destructive-replace, ~8 records | medium — **CLOSED** | +| #780 | Reconcile Python tooling — commit a ruff config and enforce it, or stop claiming we do | environment divergence in our own toolchain | medium — **CLOSED**, §5.2 | +| #781 | Retire enabled-but-never-invoked plugins/MCP servers; de-duplicate the `gitea` server | config that reads as coverage | low — **CLOSED**, §5.3 is re-measured from a derived population; the retirement half was answered *no* on evidence | +| #799 | `serena` is enabled but never invoked — establish why, then use it or disable it | one row of §5.3 | low — **CLOSED**, it had been *disabled* until 2026-08-14 and is now adopted (`docs/local-lsp-tooling.md`) | **Detector C (dedup by construction) has no issue of its own on purpose.** It is not a thing to build; it is the shape the fixes in #774 and #778 should take when they find two copies of one rule — with diff --git a/docs/local-lsp-tooling.md b/docs/local-lsp-tooling.md index 56f7c3c17..0cb8e8f0e 100644 --- a/docs/local-lsp-tooling.md +++ b/docs/local-lsp-tooling.md @@ -19,7 +19,7 @@ This distinction is the whole reason both halves are documented together. | --- | --- | --- | | The **`LSP` tool** (Claude Code plugins) | `csharp-ls`, `typescript-language-server`, `pyright-langserver` | **No dispatched subagent has been observed to reach it.** `ToolSearch` returns "No matching deferred tools found" for `select:LSP` in a subagent while the same query resolves in the main session — Claude Code 2.1.232, agent types `general-purpose` and `Explore`, 2026-08-14, plus the independent observation in `docs/defect-shapes-773.md` §5.1. Three observations on one harness version: treat it as measured behaviour to design around, not an architectural guarantee. | | The **`csharp-lsp` MCP server** (`.mcp.json`) | wraps `csharp-ls`; serves `csharp_references`, `csharp_diagnostics`, `csharp_hover`, `csharp_definition`, `csharp_symbols`, `csharp_completions`, `csharp_set_workspace`, … (16 tools) | **Main session and subagents.** Subagents demonstrably call MCP tools here: 326 MCP calls from inside subagent turns across the transcript corpus, spanning four servers (`gitea` 288, `playwright` 26, `ssh-mcp` 6, `mempalace` 6). Not yet demonstrated for `csharp-lsp` specifically — that server only became startable on 2026-08-14 — so this is the general MCP boundary, evidenced, rather than a per-server measurement. | -| The **`serena` MCP server** (plugin `serena@claude-plugins-official`) | its own bundled Roslyn C# server, plus ~30 other language servers it selects per project | **Main session and subagents** (MCP, same boundary as above). Cross-language, and the only surface here that is not C#-only. Requires an explicit `activate_project` per directory — see below. | +| The **`serena` MCP server** (plugin `serena@claude-plugins-official`) | its own bundled Roslyn C# server; its generated config lists 73 selectable language-server ids, one chosen per project | **Main session** demonstrated. Subagent use is *expected* (it is MCP, so the boundary above applies) but **not yet demonstrated** — every serena call on record is from a main session. Requires an explicit `activate_project` before the first query — see below. | So an instruction telling *delegated agents* to use C# code intelligence must point them at the **MCP tools**, never at the `LSP` tool. Briefing harder does not help: an agent that cannot resolve @@ -184,7 +184,9 @@ from an empty result. Activate by absolute path, once per directory: mcp__plugin_serena_serena__activate_project project=/Users/timothy/ersatztv ``` -Demonstrated in this repo on 2026-08-28 — `find_symbol` on `ChannelPlaylist` +Demonstrated on 2026-08-28 against a worktree of this repo (`/Users/timothy/ersatztv-wt/781-tooling` +— the path above is the canonical checkout, which is the one to activate for ordinary work) — +`find_symbol` on `ChannelPlaylist` (`ErsatzTV.Core/Iptv/ChannelPlaylist.cs`) returns the class and its constructor, and `find_referencing_symbols` on `ErsatzTV.Core.Iptv/ChannelPlaylist` returns references across **7 files** spanning `ErsatzTV.Core.Tests`, `ErsatzTV/Controllers`, `ErsatzTV/Formatters` and @@ -193,16 +195,27 @@ residue. Traps specific to it: -- **Activation is per directory, not per repository.** A git worktree is a different path, so it - registers as a *separate* serena project named after the worktree directory. Activate each one you - work in; do not assume the canonical checkout's activation carries over. +- **Activation is per directory, not per repository, and it is a registration — not a session + setting.** A git worktree is a different path, so it registers as a *separate* serena project + named after the worktree directory. Activate each directory you work in the first time; a running + server also has one *active* project at a time, so switching directories means activating again + even if both are already registered. - **It writes `.serena/` into the project root** — `project.yml`, `project.local.yml` and a language server cache. Gitignored here since #799; activating in a checkout that predates that entry leaves an untracked directory. - **It starts its own Roslyn server**, `Microsoft.CodeAnalysis.LanguageServer.dll` under `~/.serena/language_servers/`, independent of `csharp-ls`. That is a *second* C# language server - with its own multi-minute warm-up, so the cold-query trap below applies to it too — and running it - alongside a `dotnet build` or a `csharp-lsp` solution load competes for the same cores. + competing for the same cores as a `dotnet build` or a `csharp-lsp` solution load. Its warm-up cost + has **not** been characterised: the one cold run on record (2026-08-28) activated in 1.7 s and + answered `find_symbol` in 6.8 s and `find_referencing_symbols` in 13.1 s, which is far short of + `csharp-ls`'s multi-minute solution load. One observation is not a warm-up profile — treat the + cold-query trap below as applying until someone measures it. +- **A truncated answer degrades silently into a summary.** `find_referencing_symbols` with + `max_answer_chars: 4000` on `ChannelPlaylist` returned *"The answer is too long (5758 + characters)"* and then a "References without surrounding lines" list instead. That summary is + still complete on files and symbols — but it is a different shape from what the tool normally + returns, and reading it as the full result is the mistake to avoid. Raise `max_answer_chars` + rather than accepting the degraded form. ## Traps -- 2.47.3 From d040c031fb20a28706b6f04784595ad832978a36 Mon Sep 17 00:00:00 2001 From: Timothy Date: Fri, 28 Aug 2026 15:55:21 +0200 Subject: [PATCH 4/5] fix(781,799): bound the reload claim, and measure what was asserted MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Second review round, both reviewers again. The reload account was right in substance but overreached: two sessions cannot establish the harness's contract, and "started at 12:01Z, SO its tool set never contained them" implied a startup cause that the evidence contradicts — the enabling session also started before the toggle. It now states what separates the two (one reloaded, one did not) and gives the operating rule rather than a mechanism. The /reload-plugins count was itself a substring count, in the paragraph that establishes "count invocations, not substrings". Replaced with the cause-effect pair: reload at 12:54:32.694Z, `Reloaded: 17 plugins`, serena in the next delta 19s later at 12:54:51.402Z. The codex figure took three parsers to three answers (117, 112/126, 109), which is the point at which a hand-written command-position grammar stops earning its keep. Stated as 109-126 with the grammar named, and the range left unresolved on purpose. The truncation trap claimed the degraded form was "still complete on files and symbols" without ever re-running at a higher limit. Re-run at 60000: same 7 files, same 17 symbols, only the content_around_reference snippets lost. Now measured instead of assumed. Also: activation is no longer described as what made serena look broken (it is a second prerequisite, met only after the tools resolve); the subagent claim in the serena intro no longer contradicts the caveat in the surfaces table; both hook rows carry the same triple; and the identical-cadence claim is tied to both plugins registering a Stop hook in hooks.json rather than asserted as necessity. Co-Authored-By: Claude Opus 5 (1M context) --- docs/defect-shapes-773.md | 23 ++++++++++++++--------- docs/local-lsp-tooling.md | 25 ++++++++++++++----------- 2 files changed, 28 insertions(+), 20 deletions(-) diff --git a/docs/defect-shapes-773.md b/docs/defect-shapes-773.md index 1b68492c9..4ea87d81e 100644 --- a/docs/defect-shapes-773.md +++ b/docs/defect-shapes-773.md @@ -506,9 +506,9 @@ every shape that exists. | Entry | Scope / surface | Measured | Verdict | |---|---|---|---| | `gitea` MCP | user `~/.claude.json` | **975 calls**, 105 transcripts, last 2026-08-28 | Keep — heaviest tool in the corpus | -| `codex` plugin | `codex exec` inside `Bash`, plus `SessionStart`/`SessionEnd`/`Stop` hooks | **117 `Bash` inputs invoke it at command position**, 21 transcripts, last 2026-08-28 (258 merely *contain* the string — polling and inspection commands mention it, so a substring count overstates it) | Keep — heavily used, and invisible to a skill-name search | +| `codex` plugin | `codex exec` inside `Bash`, plus `SessionStart`/`SessionEnd`/`Stop` hooks | **109–126 `Bash` inputs invoke it at command position** across ~21 transcripts, last 2026-08-28 — 109 under a quote- and heredoc-aware parse, rising toward 126 if wrappers such as `nohup` are admitted. The range is stated rather than resolved: three successive parsers gave three answers, which is the point at which a hand-written command-position grammar stops being worth another round. **258** inputs merely *contain* the string — `pgrep -f "codex exec"` polling loops and heredoc bodies mention it — so a substring count overstates it roughly twofold | Keep — heavily used, and invisible to a skill-name search | | `playwright` | plugin MCP | 64 calls, last 2026-08-27 | Keep | -| `security-guidance` | **hooks** (`SessionStart`, `UserPromptSubmit`, `PostToolUse`, `Stop`) | **1,084 execution records** in `hookInfos` naming `security_reminder_hook`, across 79 transcripts / 23 UTC days, last 2026-08-28 | Keep — **not** dead; it exposes no tool at all | +| `security-guidance` | **hooks** (`SessionStart`, `UserPromptSubmit`, `PostToolUse`, `Stop`) | **1,086 execution records** in `hookInfos` naming `security_reminder_hook`, across 78 transcripts / 23 UTC days, last 2026-08-28 | Keep — **not** dead; it exposes no tool at all | | `ssh-mcp` | project `.mcp.json` | 16 calls, 9 transcripts, last 2026-08-22 | Keep | | `superpowers` | `Skill` | 13 invocations (`systematic-debugging` 6, `brainstorming` 5, …), last 2026-08-26 | Keep | | `mempalace` | user `~/.claude.json` | 11 calls, 7 transcripts, last 2026-08-27 | Keep — in active use | @@ -516,7 +516,7 @@ every shape that exists. | `serena` | plugin MCP | **Disabled until 2026-08-14T12:31Z**; 0 invocations in the 14 days since it was enabled; **demonstrated working 2026-08-28** | Keep — see #799 and `docs/local-lsp-tooling.md` | | `LSP` tool (`csharp-lsp`, `pyright-lsp`, `typescript-lsp` plugins) | `LSP` | 13 calls, last 2026-08-14 — all §5.1/#777 probes | Repaired but unadopted; no post-repair use | | `csharp-lsp` MCP server | project `.mcp.json` | **0** since it became startable on 2026-08-14 | Reachable, unadopted | -| `ralph-loop` | **hooks** (`Stop`) + slash command | Its `Stop` hook ran **1,083 times** across 78 transcripts / 23 days — the same cadence as `security-guidance`, because both fire on every stop. The `/ralph-loop` command itself: **0** | Keep — the hook executes and exits when no loop state exists; the *feature* is unused, the plugin is not inert | +| `ralph-loop` | **hooks** (`Stop`) + slash command | Its `Stop` hook ran **1,086 times** across 78 transcripts / 23 days — the same triple as `security-guidance` at this snapshot, consistent with both registering a `Stop` hook in their `hooks.json`. The `/ralph-loop` command itself: **0** | Keep — the hook executes and exits when no loop state exists; the *feature* is unused, the plugin is not inert | | `context7` | plugin MCP | **0**, and it was enabled across the whole retained window | No observed use in the window. **Deliberately kept** — operator decision 2026-08-28 | | `code-review`, `code-simplifier`, `commit-commands`, `claude-md-management`, `skill-creator`, `frontend-design` | `Skill` / `Agent` / slash command | **0** — but all six were **disabled** until 2026-08-14T12:41Z, so the zero covers only the 14 days since | Not yet exercised rather than rejected. **Deliberately kept** — operator decision 2026-08-28 | | `nuget` | project `.mcp.json`, explicitly disabled | 0 (expected); `command` names `/usr/local/share/dotnet/dotnet`, **verified absent** | Entry removed 2026-08-28 | @@ -561,13 +561,18 @@ tools appear in **no transcript before 2026-08-14T12:42:54Z**, then in 6 transcr 13:41Z correctly reported them absent, and that probe *did* carry a positive control — two other `ToolSearch` queries resolved in the same session minutes before. -**A running session does not pick up a plugin a sibling session enables; it acquires it only on an -explicit `/reload-plugins` or a restart.** The session that performed the toggle ran -`/reload-plugins` three times and gained serena mid-run at 12:54Z despite having started at 12:05Z, -*before* the toggle — so this is a reload boundary, not a startup one. The #799 session never +**Enabling a plugin did not reach a session that was already running; a reload did.** Both sessions +started *before* the 12:31Z toggle, so start time alone separates nothing — what separates them is +the reload. The enabling session ran `/reload-plugins` at **12:54:32.694Z**, printed +`Reloaded: 17 plugins`, and its next tool delta added serena **19 seconds later at 12:54:51.402Z**. +The #799 session ran for 3h54m, emitted exactly one tool delta (12:01:27Z, no serena), and never reloaded. Its author read `serena: true` from the settings file and `serena: absent` from their own tool set, and both were true simultaneously. +Two sessions are not a proof of the harness's contract, so treat this as the operating rule rather +than a mechanism: **do not assume a plugin a sibling session enabled is available to yours — reload +or restart, then confirm the tool resolves.** + Two things follow for any future audit. **Read enablement as of the evidence, not as of now** — an audit that compares *current* config against *historical* usage will call a plugin unused for a window in which it was switched off. And **date transcript evidence by record timestamp, never by @@ -724,8 +729,8 @@ This issue is analysis; it spawns implementation rather than doing it. | #778 | **D** — pin a version or use compare-and-set for read-then-write against live remote state | Family D, 5 records | medium — **CLOSED** | | #779 | **F + G** — test the deny path with the production config value; assert full-replace field lists | fail-open + destructive-replace, ~8 records | medium — **CLOSED** | | #780 | Reconcile Python tooling — commit a ruff config and enforce it, or stop claiming we do | environment divergence in our own toolchain | medium — **CLOSED**, §5.2 | -| #781 | Retire enabled-but-never-invoked plugins/MCP servers; de-duplicate the `gitea` server | config that reads as coverage | low — **CLOSED**, §5.3 is re-measured from a derived population; the retirement half was answered *no* on evidence | -| #799 | `serena` is enabled but never invoked — establish why, then use it or disable it | one row of §5.3 | low — **CLOSED**, it had been *disabled* until 2026-08-14 and is now adopted (`docs/local-lsp-tooling.md`) | +| #781 | Retire enabled-but-never-invoked plugins/MCP servers; de-duplicate the `gitea` server | config that reads as coverage | low — **DONE**, §5.3 is re-measured from a derived population; the retirement half was answered *no* on evidence | +| #799 | `serena` is enabled but never invoked — establish why, then use it or disable it | one row of §5.3 | low — **DONE**, it had been *disabled* until 2026-08-14 and is now adopted (`docs/local-lsp-tooling.md`) | **Detector C (dedup by construction) has no issue of its own on purpose.** It is not a thing to build; it is the shape the fixes in #774 and #778 should take when they find two copies of one rule — with diff --git a/docs/local-lsp-tooling.md b/docs/local-lsp-tooling.md index 0cb8e8f0e..57557fc46 100644 --- a/docs/local-lsp-tooling.md +++ b/docs/local-lsp-tooling.md @@ -19,7 +19,7 @@ This distinction is the whole reason both halves are documented together. | --- | --- | --- | | The **`LSP` tool** (Claude Code plugins) | `csharp-ls`, `typescript-language-server`, `pyright-langserver` | **No dispatched subagent has been observed to reach it.** `ToolSearch` returns "No matching deferred tools found" for `select:LSP` in a subagent while the same query resolves in the main session — Claude Code 2.1.232, agent types `general-purpose` and `Explore`, 2026-08-14, plus the independent observation in `docs/defect-shapes-773.md` §5.1. Three observations on one harness version: treat it as measured behaviour to design around, not an architectural guarantee. | | The **`csharp-lsp` MCP server** (`.mcp.json`) | wraps `csharp-ls`; serves `csharp_references`, `csharp_diagnostics`, `csharp_hover`, `csharp_definition`, `csharp_symbols`, `csharp_completions`, `csharp_set_workspace`, … (16 tools) | **Main session and subagents.** Subagents demonstrably call MCP tools here: 326 MCP calls from inside subagent turns across the transcript corpus, spanning four servers (`gitea` 288, `playwright` 26, `ssh-mcp` 6, `mempalace` 6). Not yet demonstrated for `csharp-lsp` specifically — that server only became startable on 2026-08-14 — so this is the general MCP boundary, evidenced, rather than a per-server measurement. | -| The **`serena` MCP server** (plugin `serena@claude-plugins-official`) | its own bundled Roslyn C# server; its generated config lists 73 selectable language-server ids, one chosen per project | **Main session** demonstrated. Subagent use is *expected* (it is MCP, so the boundary above applies) but **not yet demonstrated** — every serena call on record is from a main session. Requires an explicit `activate_project` before the first query — see below. | +| The **`serena` MCP server** (plugin `serena@claude-plugins-official`) | its own bundled Roslyn C# server; its generated config lists 73 selectable language-server ids, one or more chosen per project (first match wins per file) | **Main session** demonstrated. Subagent use is *expected* (it is MCP, so the boundary above applies) but **not yet demonstrated** — every serena call on record is from a main session. Requires an explicit `activate_project` before the first query — see below. | So an instruction telling *delegated agents* to use C# code intelligence must point them at the **MCP tools**, never at the `LSP` tool. Briefing harder does not help: an agent that cannot resolve @@ -166,11 +166,13 @@ a wrong pin or a missing patch surfaces as a failed smoke test rather than a fal ``` Nothing about it needs configuring in this repo. It answers the same find-all-references question as -`csharp-lsp`, across languages rather than C# only, and being MCP it is reachable from dispatched -subagents — which is the gap the `LSP` tool leaves. +`csharp-lsp`, and being MCP it is *expected* to be reachable from dispatched subagents — the gap the +`LSP` tool leaves — though that has not yet been demonstrated for serena specifically. -**It requires an explicit project activation, and that is what made it look broken.** A fresh server -has no active project; the first real call fails with a message naming the projects it does know: +**Once it is available, it still requires an explicit project activation.** This was *not* the cause +of #799 — that session never had serena's tools at all (see `docs/defect-shapes-773.md` §5.3) — it is +a second prerequisite you meet only after the tools resolve. A fresh server has no active project, +and the first real call fails with a message naming the projects it does know: ``` No active project. Ask the user to provide the project path or to select a project @@ -210,12 +212,13 @@ Traps specific to it: answered `find_symbol` in 6.8 s and `find_referencing_symbols` in 13.1 s, which is far short of `csharp-ls`'s multi-minute solution load. One observation is not a warm-up profile — treat the cold-query trap below as applying until someone measures it. -- **A truncated answer degrades silently into a summary.** `find_referencing_symbols` with - `max_answer_chars: 4000` on `ChannelPlaylist` returned *"The answer is too long (5758 - characters)"* and then a "References without surrounding lines" list instead. That summary is - still complete on files and symbols — but it is a different shape from what the tool normally - returns, and reading it as the full result is the mistake to avoid. Raise `max_answer_chars` - rather than accepting the degraded form. +- **A truncated answer degrades into a summary, and says so only in its first line.** + `find_referencing_symbols` with `max_answer_chars: 4000` on `ChannelPlaylist` returned *"The answer + is too long (5758 characters)"* followed by a "References without surrounding lines" list. Re-run + at `max_answer_chars: 60000`, the full answer names the **same 7 files and the same 17 symbols** — + so on this query the degraded form lost only the `content_around_reference` snippets, not + coverage. One comparison is not a guarantee for every query: raise the limit and diff the two + before treating a truncated result as exhaustive. ## Traps -- 2.47.3 From 9b4844c1a8a9246d050dbc33c61a03eaedb79132 Mon Sep 17 00:00:00 2001 From: Timothy Date: Fri, 28 Aug 2026 16:09:37 +0200 Subject: [PATCH 5/5] fix(781,799): remove the last startup-cause clause, and two count nits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round 3. The startup-cause clause the previous commit set out to delete survived one paragraph above its own refutation: :560 still read "started at 12:01Z, so its tool set never contained them" while :564 said start time separates nothing. Sweeping to where the reviewer pointed rather than to every site making the claim is the trap this repo calls "sweep by subject, not verb". The clause now carries the variable that actually differs — "and never reloaded" — so the two paragraphs agree. Dropping the /reload-plugins count was the right call and is now proven so: the 12:05:35Z record I had counted as a second invocation is a /doctor command whose arguments quote the words "run /reload-plugins". The reliable instrument is the harness's own receipt — `Reloaded: N plugins`, which occurs exactly three times corpus-wide — not the command string. Two nits: the codex range is paired with ~21-26 transcripts rather than the strict parse's 21 alone, and the truncation comparison says 17 reference entries across 16 distinct symbols, since GetChannelPlaylistHandler/Handle is referenced twice. Also "the system reporting the cause" now reads "this prerequisite", since the paragraph above it establishes that activation was not #799's cause. Co-Authored-By: Claude Opus 5 (1M context) --- docs/defect-shapes-773.md | 6 +++--- docs/local-lsp-tooling.md | 10 +++++----- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/defect-shapes-773.md b/docs/defect-shapes-773.md index 4ea87d81e..11801fe8d 100644 --- a/docs/defect-shapes-773.md +++ b/docs/defect-shapes-773.md @@ -506,7 +506,7 @@ every shape that exists. | Entry | Scope / surface | Measured | Verdict | |---|---|---|---| | `gitea` MCP | user `~/.claude.json` | **975 calls**, 105 transcripts, last 2026-08-28 | Keep — heaviest tool in the corpus | -| `codex` plugin | `codex exec` inside `Bash`, plus `SessionStart`/`SessionEnd`/`Stop` hooks | **109–126 `Bash` inputs invoke it at command position** across ~21 transcripts, last 2026-08-28 — 109 under a quote- and heredoc-aware parse, rising toward 126 if wrappers such as `nohup` are admitted. The range is stated rather than resolved: three successive parsers gave three answers, which is the point at which a hand-written command-position grammar stops being worth another round. **258** inputs merely *contain* the string — `pgrep -f "codex exec"` polling loops and heredoc bodies mention it — so a substring count overstates it roughly twofold | Keep — heavily used, and invisible to a skill-name search | +| `codex` plugin | `codex exec` inside `Bash`, plus `SessionStart`/`SessionEnd`/`Stop` hooks | **109–126 `Bash` inputs invoke it at command position** across ~21–26 transcripts, last 2026-08-28 — 109 under a quote- and heredoc-aware parse, rising toward 126 if wrappers such as `nohup` are admitted. The range is stated rather than resolved: three successive parsers gave three answers, which is the point at which a hand-written command-position grammar stops being worth another round. **258** inputs merely *contain* the string — `pgrep -f "codex exec"` polling loops and heredoc bodies mention it — so a substring count overstates it roughly twofold | Keep — heavily used, and invisible to a skill-name search | | `playwright` | plugin MCP | 64 calls, last 2026-08-27 | Keep | | `security-guidance` | **hooks** (`SessionStart`, `UserPromptSubmit`, `PostToolUse`, `Stop`) | **1,086 execution records** in `hookInfos` naming `security_reminder_hook`, across 78 transcripts / 23 UTC days, last 2026-08-28 | Keep — **not** dead; it exposes no tool at all | | `ssh-mcp` | project `.mcp.json` | 16 calls, 9 transcripts, last 2026-08-22 | Keep | @@ -556,8 +556,8 @@ sessions here. What was demonstrably wrong with this configuration was **duplica `~/.claude/settings.json.bak-doctor`, written moments earlier, still records `serena: false`. Its tools appear in **no transcript before 2026-08-14T12:42:54Z**, then in 6 transcripts that day and 186 across the 10 days since, always in a harness-emitted availability listing rather than prose -(counts as of the 2026-08-28 snapshot; the corpus is live and grows). The -#799 session had started at 12:01Z, so its tool set never contained them; its `ToolSearch` probe at +(counts as of the 2026-08-28 snapshot; the corpus is live and grows). The #799 session had started +at 12:01Z **and never reloaded**, so its tool set never contained them; its `ToolSearch` probe at 13:41Z correctly reported them absent, and that probe *did* carry a positive control — two other `ToolSearch` queries resolved in the same session minutes before. diff --git a/docs/local-lsp-tooling.md b/docs/local-lsp-tooling.md index 57557fc46..b53336609 100644 --- a/docs/local-lsp-tooling.md +++ b/docs/local-lsp-tooling.md @@ -179,8 +179,8 @@ No active project. Ask the user to provide the project path or to select a proje from this list of known projects: ['adversarial-reviewer', 'server-management', ...] ``` -That error is the tell, and it is the system reporting the cause rather than the caller inferring it -from an empty result. Activate by absolute path, once per directory: +That error is the tell, and it is the system reporting this prerequisite rather than the caller +inferring it from an empty result. Activate by absolute path, once per directory: ``` mcp__plugin_serena_serena__activate_project project=/Users/timothy/ersatztv @@ -215,9 +215,9 @@ Traps specific to it: - **A truncated answer degrades into a summary, and says so only in its first line.** `find_referencing_symbols` with `max_answer_chars: 4000` on `ChannelPlaylist` returned *"The answer is too long (5758 characters)"* followed by a "References without surrounding lines" list. Re-run - at `max_answer_chars: 60000`, the full answer names the **same 7 files and the same 17 symbols** — - so on this query the degraded form lost only the `content_around_reference` snippets, not - coverage. One comparison is not a guarantee for every query: raise the limit and diff the two + at `max_answer_chars: 60000`, the full answer names the **same 7 files and the same 17 reference + entries across 16 distinct symbols** (`GetChannelPlaylistHandler/Handle` is referenced twice) — so + on this query the degraded form lost only the `content_around_reference` snippets, not coverage. One comparison is not a guarantee for every query: raise the limit and diff the two before treating a truncated result as exhaustive. ## Traps -- 2.47.3