docs(781,799): re-measure the tooling audit from a derived population, and adopt serena #862

Merged

timothy merged 5 commits from fix/781-799-tooling-hygiene into main 2026-08-28 16:37:35 +02:00

Conversation 1 Commits 5 Files Changed 4

+192 -59

timothy commented 2026-08-28 16:14:15 +02:00

Owner

Copy Link

Copy Source

Re-measures the docs/defect-shapes-773.md §5.3 tooling audit from a population derived from
config, and adopts serena as a third code-intelligence surface.

fixes #781
fixes #799

What was wrong

§5.3 judged every entry on a single surface — tool/skill invocation counts — which manufactured
four false zeros. codex is driven through codex exec inside Bash; security-guidance and
ralph-loop expose no tool at all and run as hooks (1,086 executions each); feature-dev is
used through its agents, not its skill. Each looked like a confident zero.

Worse, the audit compared current enablement against historical usage. Six of the eight plugins
it called "genuinely unused" were disabled for 16 of the 30 corpus days
(~/.claude/settings.json.bak-doctor), and serena was disabled entirely until 2026-08-14T12:31Z.

#799 — the premise was right, the diagnosis was not

#799 recorded that serena's tools never reached its session. That observation is correct, and
the cause is now established rather than inferred from a zero:

Time (UTC, 2026-08-14)	Event
12:01:16Z	#799's session starts. Its one tool delta carries 357 tools, no serena
12:31Z	A concurrent session flips serena to true
12:54:32Z	That session runs /reload-plugins → Reloaded: 17 plugins
12:54:51Z	serena appears in its next delta, 19 s later
13:41Z	#799's session probes ToolSearch → correctly finds nothing. It never reloaded

Both sessions started before the toggle, so start time separates nothing — the reload does. The
author read serena: true from the settings file and serena: absent from their own tool set, and
both were true at once.

serena is now demonstrated working here: find_referencing_symbols on
ErsatzTV.Core.Iptv/ChannelPlaylist returns references across 7 files in 4 projects. It is
documented in docs/local-lsp-tooling.md as the third surface, with its traps — per-directory
activation, its own Roslyn server, and silent truncation into a summary.

The retirement half is answered no, on evidence

The zeros split six ways (broken · wrong surface · not enabled · enabled-but-never-called ·
aged out of a rolling corpus · no observed use). Only the last is grounds for removal, and the
corpus starts 2026-07-29 while the plugins date to March — so it cannot support "excess" as a
historical claim at all. Operator decision 2026-08-28: keep them, since that config is shared
by every project. #781's first ## Done-when box was amended to match, with the reason recorded on
the issue.

Removed on evidence: the disabled nuget entry, whose command names
/usr/local/share/dotnet/dotnet (verified absent), and the dangling reference it left in
disabledMcpjsonServers. Both files are gitignored, so that change is not visible in this diff.

Review

Four rounds, two independent cold reviewers (one cross-family, Codex). Rounds 1–3 were BLOCKED
and every finding was applied — including two that were the same defect this section exists to
name, found in the act of documenting it: I dated transcript evidence by file mtime instead of
record timestamp, and I counted codex exec and /reload-plugins by substring rather than
invocation. The codex figure is deliberately shipped as a range (109–126) rather than a fourth
parser's answer.

Verification

• python3 -m pytest scripts/tests -q → 1083 passed, 2 skipped (re-run after every round)
• scripts/check-doc-narrative.py --diff origin/main → 3 files scanned, 0 warnings
• Every number re-derived independently by both reviewers; gitea 975, playwright 64, ssh-mcp
  16, mempalace 11, LSP 13 and the corpus size of 486 all reproduced exactly.

Re-measures the `docs/defect-shapes-773.md` §5.3 tooling audit from a population derived from config, and adopts `serena` as a third code-intelligence surface. fixes #781 fixes #799 ## What was wrong §5.3 judged every entry on a single surface — tool/skill invocation counts — which manufactured four false zeros. `codex` is driven through `codex exec` inside `Bash`; `security-guidance` and `ralph-loop` expose no tool at all and run as **hooks** (1,086 executions each); `feature-dev` is used through its **agents**, not its skill. Each looked like a confident zero. Worse, the audit compared *current* enablement against *historical* usage. Six of the eight plugins it called "genuinely unused" were **disabled** for 16 of the 30 corpus days (`~/.claude/settings.json.bak-doctor`), and `serena` was disabled entirely until 2026-08-14T12:31Z. ## #799 — the premise was right, the diagnosis was not #799 recorded that serena's tools never reached its session. That observation is **correct**, and the cause is now established rather than inferred from a zero: | Time (UTC, 2026-08-14) | Event | |---|---| | 12:01:16Z | #799's session starts. Its one tool delta carries 357 tools, no serena | | 12:31Z | A **concurrent** session flips `serena` to `true` | | 12:54:32Z | That session runs `/reload-plugins` → `Reloaded: 17 plugins` | | 12:54:51Z | serena appears in its next delta, 19 s later | | 13:41Z | #799's session probes `ToolSearch` → correctly finds nothing. It never reloaded | Both sessions started *before* the toggle, so start time separates nothing — the reload does. The author read `serena: true` from the settings file and `serena: absent` from their own tool set, and both were true at once. serena is now demonstrated working here: `find_referencing_symbols` on `ErsatzTV.Core.Iptv/ChannelPlaylist` returns references across 7 files in 4 projects. It is documented in `docs/local-lsp-tooling.md` as the third surface, with its traps — per-directory activation, its own Roslyn server, and silent truncation into a summary. ## The retirement half is answered *no*, on evidence The zeros split **six** ways (broken · wrong surface · not enabled · enabled-but-never-called · aged out of a rolling corpus · no observed use). Only the last is grounds for removal, and the corpus starts 2026-07-29 while the plugins date to March — so it cannot support "excess" as a historical claim at all. **Operator decision 2026-08-28: keep them**, since that config is shared by every project. #781's first `## Done-when` box was amended to match, with the reason recorded on the issue. Removed on evidence: the disabled `nuget` entry, whose `command` names `/usr/local/share/dotnet/dotnet` (verified absent), and the dangling reference it left in `disabledMcpjsonServers`. Both files are gitignored, so that change is not visible in this diff. ## Review Four rounds, two independent cold reviewers (one cross-family, Codex). Rounds 1–3 were **BLOCKED** and every finding was applied — including two that were the *same defect this section exists to name*, found in the act of documenting it: I dated transcript evidence by file mtime instead of record timestamp, and I counted `codex exec` and `/reload-plugins` by substring rather than invocation. The `codex` figure is deliberately shipped as a range (109–126) rather than a fourth parser's answer. ## Verification - `python3 -m pytest scripts/tests -q` → **1083 passed, 2 skipped** (re-run after every round) - `scripts/check-doc-narrative.py --diff origin/main` → 3 files scanned, 0 warnings - Every number re-derived independently by both reviewers; `gitea` 975, `playwright` 64, `ssh-mcp` 16, `mempalace` 11, `LSP` 13 and the corpus size of 486 all reproduced exactly.

timothy added 5 commits 2026-08-28 16:14:15 +02:00

docs(781,799): re-measure the tooling audit from a derived population, and adopt serena ... b760de8b77

§5.3's verdicts rested on a single surface (tool/skill invocation counts), which
manufactured three false zeros: `codex` is used via `codex exec` inside Bash,
`security-guidance` exposes no tool at all and runs as a hook, and `feature-dev`
is used through its agents rather than its skill. Re-measured across all six
surfaces a thing can actually be used through, with the population derived from
the config files instead of listed by hand.

The retirement half of #781 is answered *no* on evidence rather than left open:
the zeros split five ways (broken, wrong surface, reachable-but-uncalled, aged
out of a rolling corpus, genuinely unused) and only the last is grounds for
removal. Eight plugins are genuinely unused and are deliberately kept — operator
decision, recorded in the table.

#799's premise is refuted rather than fixed: serena's tools appear in the
tool-availability listing of six transcripts on 2026-08-14, the day the issue
claimed they never reach a session, and in 191 transcripts since. It was never
unreachable, only never called — what was missing is an `activate_project`.
Adopted and documented as the third code-intelligence surface, with `.serena/`
gitignored because activation writes it into the project root.

Also removed the disabled `nuget` entry from the local `.mcp.json` (its command
names `/usr/local/share/dotnet/dotnet`, verified absent) and the dangling
`nuget` reference it left in `disabledMcpjsonServers`. Both files are
gitignored, so that part is not visible in this diff.

fixes #781
fixes #799

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

docs(781): tighten two claims that outran their evidence ... 3f99aecddc

The surface enumeration asserted "six surfaces exist", which is an absolute
claim about the world rather than about this corpus. MCP resources/prompts are
a seventh; they measure 0 across every server, so naming them costs a sentence
and removes a hole a reader could fall into.

The security-guidance row said "fires on 20 distinct days" from a loose grep
that would also match a session merely discussing the hook. Recounted against
`stop_hook_summary` records naming the hook itself: 1,080 executions across 79
transcripts, same 20 days.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(781,799): correct the serena causal story and the enablement-blind verdicts ... dabbd4c3e8

Two independent cold reviews (one cross-family) blocked the previous revision,
both on the same defect the section exists to name: a zero read as "unused"
without checking whether the thing was switched on. They were right.

serena was not "reachable all along". It was `false` in ~/.claude/settings.json
until 2026-08-14T12:31Z, when a concurrent session enabled it; its tools appear
in no transcript before 12:42:54Z that day. The #799 session had started at
12:01Z and never ran /reload-plugins, so its probe at 13:41Z correctly reported
the tools absent — while the settings file already said `true`. Both reviewers
had half of this; the reload boundary is what reconciles them, since the session
that performed the toggle DID acquire serena mid-run after reloading three
times. So #799's observation was accurate and its cause is now established,
rather than refuted as the previous revision claimed.

The error underneath it was dating transcript evidence by file mtime instead of
record timestamp, which places a session's records at the moment it last wrote
and made a first-appearance look like a pre-existing state. Both rules are now
recorded in the section.

Six of the eight plugins called "genuinely unused" were disabled for 16 of the
30 corpus days (settings.json.bak-doctor), and ralph-loop is not unused at all —
its Stop hook has 1,083 executions, the same cadence as security-guidance, since
both fire on every stop. It is the sole enabled plugin owning hooks/stop-hook.sh,
so the attribution is unambiguous. That is a fourth instance of the wrong-surface
mistake, found in the act of documenting the first three.

Also: "256 codex exec calls" counted Bash inputs *containing* the string,
including pgrep and echo; at command position it is 117. The surface list no
longer claims a fixed count, omits neither `LSP` nor `attributionSkill`, and says
what was not separately identified. In local-lsp-tooling.md the subagent claim is
marked expected-but-undemonstrated, "the only surface not C#-only" is dropped as
false, ~30 language servers becomes the config's actual 73, and the multi-minute
warm-up assertion is replaced by the one cold run's real timings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(781,799): bound the reload claim, and measure what was asserted ... d040c031fb

Second review round, both reviewers again. The reload account was right in
substance but overreached: two sessions cannot establish the harness's contract,
and "started at 12:01Z, SO its tool set never contained them" implied a startup
cause that the evidence contradicts — the enabling session also started before
the toggle. It now states what separates the two (one reloaded, one did not) and
gives the operating rule rather than a mechanism.

The /reload-plugins count was itself a substring count, in the paragraph that
establishes "count invocations, not substrings". Replaced with the cause-effect
pair: reload at 12:54:32.694Z, `Reloaded: 17 plugins`, serena in the next delta
19s later at 12:54:51.402Z.

The codex figure took three parsers to three answers (117, 112/126, 109), which
is the point at which a hand-written command-position grammar stops earning its
keep. Stated as 109-126 with the grammar named, and the range left unresolved on
purpose.

The truncation trap claimed the degraded form was "still complete on files and
symbols" without ever re-running at a higher limit. Re-run at 60000: same 7
files, same 17 symbols, only the content_around_reference snippets lost. Now
measured instead of assumed.

Also: activation is no longer described as what made serena look broken (it is a
second prerequisite, met only after the tools resolve); the subagent claim in the
serena intro no longer contradicts the caveat in the surfaces table; both hook
rows carry the same triple; and the identical-cadence claim is tied to both
plugins registering a Stop hook in hooks.json rather than asserted as necessity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

fix(781,799): remove the last startup-cause clause, and two count nits ... 9b4844c1a8

Round 3. The startup-cause clause the previous commit set out to delete
survived one paragraph above its own refutation: :560 still read "started at
12:01Z, so its tool set never contained them" while :564 said start time
separates nothing. Sweeping to where the reviewer pointed rather than to every
site making the claim is the trap this repo calls "sweep by subject, not verb".
The clause now carries the variable that actually differs — "and never
reloaded" — so the two paragraphs agree.

Dropping the /reload-plugins count was the right call and is now proven so: the
12:05:35Z record I had counted as a second invocation is a /doctor command whose
arguments quote the words "run /reload-plugins". The reliable instrument is the
harness's own receipt — `<local-command-stdout>Reloaded: N plugins`, which
occurs exactly three times corpus-wide — not the command string.

Two nits: the codex range is paired with ~21-26 transcripts rather than the
strict parse's 21 alone, and the truncation comparison says 17 reference entries
across 16 distinct symbols, since GetChannelPlaylistHandler/Handle is referenced
twice. Also "the system reporting the cause" now reads "this prerequisite",
since the paragraph above it establishes that activation was not #799's cause.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

timothy commented 2026-08-28 16:14:51 +02:00

Author

Owner

Copy Link

Copy Source

Review-verdict: MERGEABLE @ 9b4844c

Four review rounds, two independent cold reviewers (one cross-family). Rounds 1-3 BLOCKED; all findings applied. Clean at 9b4844c1a.

Review-verdict: MERGEABLE @ 9b4844c Four review rounds, two independent cold reviewers (one cross-family). Rounds 1-3 BLOCKED; all findings applied. Clean at 9b4844c1a.

timothy merged commit b16ec15d6c into main 2026-08-28 16:37:35 +02:00

timothy deleted branch fix/781-799-tooling-hygiene 2026-08-28 16:37:37 +02:00

timothy referenced this pull request 2026-08-28 16:38:01 +02:00

Plugin/MCP config hygiene: retire what is enabled and never invoked, de-duplicate the gitea server #781

timothy referenced this pull request 2026-08-28 16:38:02 +02:00

serena is enabled and starts fine, but its tools never reach a session — establish why, then use or disable it #799

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

ad-hoc

One-off / ad-hoc work not tracked by a dedicated issue

api

REST API / HTTP endpoints

bug

Something isn't working

ci-cd

Build, test, deploy pipeline

content

Channel content / schedules / playlists

dependencies

Dependency updates (Renovate)

enhancement

New feature or improvement

frontend

ChicoryTV React SPA frontend

in-progress

Claimed by an active session — do not pick up

jellyfin

Jellyfin tuner / IPTV integration

parked

Excluded from automatic queue pickup; work only when explicitly selected

priority: high

priority: low

priority: medium

review

Adversarial review finding

security

Security / vulnerability fix

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate timothy (Timothy)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: timothy/ersatztv#862