docs(542): record the workflow lore, then prune the kickoff doc to instructions #544

Merged
timothy merged 1 commits from chore/542-prune-handoff-lore into main 2026-07-21 20:43:06 +02:00
5 changed files with 655 additions and 373 deletions
+1
View File
@@ -62,6 +62,7 @@ in-file entries.
- [`decisions/api-auth-security.md`](decisions/api-auth-security.md) — API/SPA auth & security posture (#197 bundles + #279 headers, #206, #283, #292, #295, #301, #319, #330).
- [`decisions/release-ci-governance.md`](decisions/release-ci-governance.md) — release / CI / merge governance (#303 H3/H4/H5/H6/H9/H10, #311, #314, #315, #335).
- [`decisions/spa-modularization.md`](decisions/spa-modularization.md) — App.tsx screen/shell extraction epic #243 (#244, #245, #247).
- [`decisions/workflow-process.md`](decisions/workflow-process.md) — session workflow, CI-run triage, review routing, worktree/parallel-session hygiene and tooling gotchas (#542). Extracted from the kickoff handoff doc, which was their only copy.
**In this file:**
+32
View File
@@ -29,15 +29,25 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `api.versioning-v1` | The entire `/api` surface is versioned to `/api/v1` uniformly (no unversioned corner); legacy unversioned callers are rewritten in-pipeline (not redirected) with Deprecation/Link/Sunset headers, and post-freeze `/api/v1` is additive-only — a breaking change requires `/api/v2`. | 2026-07-13 | [link](../decisions.md#2026-07-13--api-versioning-the-whole-api-surface-is-mounted-at-apiv1-additive-only-after-freeze-286) |
| `blazor.rollback-tag` | The commit immediately preceding the Blazor-removal merge is tagged `blazor-final` (not a `v*` tag, so it doesn't trigger a prod release build) as the documented rollback/restore path. | 2026-07-11 | [link](../decisions.md#2026-07-11--pre-removal-blazor-rollback-tag-blazor-final-205) |
| `blazor.ui-removed` | The legacy Blazor Server UI (`Pages/`, `Shared/`, `ViewModels/`, `Validators/`, MudBlazor + 8 other packages, Blazor Startup wiring) is fully deleted now that the SPA has parity; the legacy `MapWhen` branch is kept only for controllers/docs/OpenAPI/`LegacyUiRedirects`, and the catch-all fallback 302s any unmatched non-api/artwork/docs/openapi path to `/app`. | 2026-07-11 | [link](../decisions.md#2026-07-11--blazor-server-ui-removed-91-phase-b) |
| `ci.batch-pushes-no-cancel-route` | Hold review fixes, doc corrections and format fixes locally and push **once** — a superseded run cannot be cancelled from the agent side and holds a runner slot until it finishes. | 2026-07-21 | [link](workflow-process.md#2026-07-21--batch-your-pushes-there-is-no-agent-side-cancel-route-on-gitea-1254-542) |
| `ci.build-once-rejected` | CI build-once (a shared compile artifact across jobs) was implemented, measured, and rejected for a 40-85% wall-clock regression; keep the #420 cross-run tree-identity skip instead. | 2026-07-18 | [link](../decisions.md#2026-07-18--ci-build-once-was-measured-and-rejected-keep-the-420-tree-skip) |
| `ci.cancelled-is-not-a-verdict` | Treat a `cancelled` conclusion as "no verdict" — never as pass or fail — and report FAILED and CANCELLED counts separately in any CI monitor. | 2026-07-21 | [link](workflow-process.md#2026-07-21--cancelled-is-not-failure-a-cancelled-run-is-no-verdict-542) |
| `ci.decisions-lifecycle-flake` | When `decisions lifecycle` is the **only** red job, do not investigate and do not create a new run to clear it — no rebase, no `--amend`, no no-op push; the operator reruns that single job from the Gitea UI. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-lone-decisions-lifecycle-red-is-a-known-infra-flake-do-nothing-542) |
| `ci.docs-only-detect-shallow-safe` | The docs-only detect script must diff against `FETCH_HEAD` (always resolves after `git fetch`, even shallow) using a two-dot tree diff — not `origin/<base>` with three-dot — because a `fetch-depth: 1` shallow clone has no remote-tracking ref and no merge-base, which silently fails the original detect into `docs_only=false` (full matrix, no functional error). A CI-behavior change must be verified by measuring the effect (job durations), not just a green check. | 2026-07-17 | [link](../decisions.md#2026-07-17--docs-only-detect-must-be-shallow-checkout-safe-fetch_head--two-dot-not-originmain--three-dot-416-follow-up) |
| `ci.docs-only-skip-steps` | A docs-only change must still run every required job (`test`, `migrations`) so their commit-status contexts always report; each heavy job runs `scripts/ci-detect-docs-only.sh` first and gates its real STEPS on `if: steps.detect.outputs.docs_only != 'true'`, never `if:`-skips the whole job (an `if:`-skipped job reports `skipped`, not `success`, which branch protection may never unblock on). Detection biases toward running more on any doubt. | 2026-07-17 | [link](../decisions.md#2026-07-17--docs-only-ci-skip-gates-steps-in-always-running-required-jobs-never-if-skips-them-416) |
| `ci.format-gate-folder-mode` | The blocking `format` CI job (and matching pre-commit hook) runs `dotnet format whitespace . --folder --include <files>` instead of loading the full MSBuild/Roslyn solution, cutting the gate from ~480s to ~0.5s with unchanged whitespace/charset coverage. | 2026-07-19 | [link](../decisions.md#2026-07-19--the-format-gate-runs-dotnet-format-whitespace----folder-not-the-full-solution-format-469) |
| `ci.functional-e2e-harness` | The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs curl-only, deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. | 2026-07-16 | [link](../decisions.md#2026-07-16--functional-e2e-ci-harness-advisory-curl-contract-job-over-an-app-booted-from-source-299) |
| `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=<name>` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](workflow-process.md#2026-07-21--giteas-milestones-issue-filter-silently-no-ops-on-names-containing--or--542) |
| `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](workflow-process.md#2026-07-21--an-infra-shaped-red-under-host-load-is-not-a-code-failure-542) |
| `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-killed-ci-job-reports-conclusion-failure-read-the-log-tail-before-diagnosing-the-diff-542) |
| `ci.monitor-armed-at-pr-open` | Arm a CI monitor on the PR head sha the moment the PR opens, polling the commit-status endpoint — not at the end of the work. | 2026-07-21 | [link](workflow-process.md#2026-07-21--arm-the-ci-monitor-at-pr-open-via-the-commit-status-endpoint-542) |
| `ci.no-host-health-gating` | Push when your work is validated — never SSH to bumblebee to sample load/RAM first, and never hand-schedule around other sessions' runs. | 2026-07-21 | [link](workflow-process.md#2026-07-21--do-not-gate-or-throttle-pushes-on-host-health-542) |
| `ci.peak-anon-measurement` | The `test` job's headline memory figure is a sampled high-water mark of cgroup `anon`, produced by `scripts/ci-peak-anon.sh`; `memory.peak` and the end-of-job `anon`/`file` split are kept only as a cache-inflated reference. | 2026-07-19 | [link](../decisions.md#2026-07-19--ci-test-job-reports-a-sampled-true-peak-anon-not-cache-inflated-memorypeak-412) |
| `ci.root-screenshot-guard` | The Husky `pre-commit` hook refuses a staged root-level `*.png` (belt-and-suspenders with the `.gitignore` rule); nested `*.png` real assets are unaffected. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--root-screenshot-guard-pre-commit-refuses-root-level-png-303-h3) |
| `ci.runner-placement` | No persistent Roslyn compiler server survives a CI build (`UseSharedCompilation=false` etc., runner env + Dockerfile `ENV`); every `services:` container gets its own explicit `--memory`/`--memory-swap`/`--cpus` cap (it does not inherit the job container's). | 2026-07-17 | [link](../decisions.md#2026-07-17--no-persistent-compiler-servers-in-ci-every-services-container-gets-an-explicit-cap-390s-small-lane-move-reversed-406) |
| `ci.small-lane-git-only` | `runs-on: small` is defined by what a job does (git-only), not its usual runtime; the two `docker build` jobs (docker-build.yml, ci-image.yml) move to `ubuntu-latest` because their worst-case memory, not median runtime, was pinning the small lane's per-slot cap. | 2026-07-20 | [link](../decisions.md#2026-07-20--runs-on-small-means-git-only-the-two-docker-build-jobs-move-to-ubuntu-latest-server-management639) |
| `ci.verify-locally-ci-confirms` | Treat the local build/verify/review pass as the decision point and CI as confirmation — don't idle waiting on a run you have no reason to doubt. | 2026-07-21 | [link](workflow-process.md#2026-07-21--build-and-verify-locally-then-trust-it-ci-confirms-542) |
| `ci.web-test-per-test-timeouts` | Give heavy-render web tests an explicit per-test vitest timeout (e.g. 15s); never raise the global default to fix one slow test. | 2026-07-21 | [link](workflow-process.md#2026-07-21--heavy-render-web-tests-need-explicit-per-test-vitest-timeouts-on-the-ci-vm-542) |
| `concurrency.diff-scalar-fanout` | The frozen Block optimistic-concurrency recipe (api-conventions §7a) fans out to Collection/Playout×2/MultiCollection/RerunCollection, keeping a guard-returned `PreconditionFailedError` out of any handler's generic `catch(Exception)`→422 mapping, and preserving each aggregate's existing `SaveChangesAsync() > 0` gate semantics under the new unconditional `Version++`. | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--253-pr3-diff--scalar-concurrency-fan-out-collection--playout2--multicollection--reruncollection) |
| `concurrency.etag-rotation-completion` | Every handler that mutates a versioned root's editor-visible config state must bump `Version` (rotating the ETag) with no per-aggregate carve-outs, short-circuiting on a genuine no-op before the bump so idempotent re-submits don't fire spurious rebuild fan-out; `SaveChangesForcingVersion` rebases the retry (stored + pending delta), never adopts the stored token verbatim. | 2026-07-12 | [link](optimistic-concurrency.md#2026-07-12--cross-editor-etag-rotation-completed-for-collectionplayout-config-siblings-269) |
| `concurrency.force-write-non-ifmatch` | Any handler that leaves a versioned root `Modified` or `Deleted` but takes no `If-Match` (deletes, item add/remove bumpers, scalar-config writers) must save through `ConcurrencyExtensions.SaveChangesForcingVersion` — force-write past a concurrent `Version` bump rather than throw an unhandled `DbUpdateConcurrencyException` (500). | 2026-07-12 | [link](optimistic-concurrency.md#2026-07-12-269--non-if-match-root-writers-force-write-past-a-concurrent-version-bump) |
@@ -63,6 +73,24 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `media.remote-stream-probe` | `ValidatePlayoutItemPath` probes the Plex/Jellyfin/Emby remote-stream URL via `IRemoteStreamProber` before returning it; only a redirected 404 fails closed (`PlayoutItemNotAvailableFromMediaServer`), everything else fails open, and there is no toggle. | 2026-07-19 | [link](../decisions.md#2026-07-19--media-server-remote-stream-urls-are-probed-before-use-a-redirected-404-fails-closed-everything-else-fails-open-no-toggle-473) |
| `media.remote-stream-probe-externaljson` | External-JSON playout channels' `StreamRemotely` now probes the remote-stream URL through the same `IRemoteStreamProber` seam as the generated-playout path, closing the #473 scope gap for a channel kind with no DB `PlayoutItem` rows. | 2026-07-20 | [link](../decisions.md#2026-07-20--external-json-playout-channels-now-probe-the-remote-stream-url-too-closing-the-473-scope-gap-480) |
| `media.source-mgmt-write-api` | Media-source management (local/Plex/Jellyfin/Emby) is a REST write API + SPA under `/app/libraries/*`, wrapping existing MediatR commands 1:1 with no new commands or DB migration; connection GETs never leak a stored `apiKey`, and each PUT-replace family's identity contract is documented per-family (not assumed uniform). | 2026-07-11 | [link](../decisions.md#2026-07-11--media-source-management-rest-write-api--spa-202) |
| `process.bom-format-detection-recipe` | Before any push touching `.cs`, detect BOMs with the `xxd` byte check and verify the format gate with `dotnet format --include` run under `bash -c`, never bare zsh. | 2026-07-21 | [link](workflow-process.md#2026-07-21--bomformat-pre-push-detection-use-the-xxd-loop-and-run-dotnet-format---include-under-bash--c-542) |
| `process.branch-off-feature-branch` | To fix work on an unmerged feature branch, branch off that branch and land by fast-forward push — and after creating a worktree, drive the first Edit/Read from ITS absolute paths and `git status` it before building. | 2026-07-21 | [link](workflow-process.md#2026-07-21--fix-onto-an-unmerged-feature-branch-by-branching-off-it-and-ff-pushing-542) |
| `process.build-concurrency-limits` | Run at most 3–4 concurrent dotnet/npm builds on this Mac, gate launches on FREE RAM rather than CPU load, and never set `ETV_UPDATE_GOLDENS` / `ETV_UPDATE_PLAYOUT_GOLDENS`. | 2026-07-21 | [link](workflow-process.md#2026-07-21--bound-parallel-builds-by-free-ram-never-regenerate-goldens-542) |
| `process.codex-cheap-worker-launch` | For bounded tool-bearing selector/recon work, launch a Codex worker with `codex exec -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only`; `spawn_agent` buys parallelism but no cost savings. | 2026-07-21 | [link](workflow-process.md#2026-07-21--launch-codex-cheap-workers-via-codex-exec-not-spawn_agent-542) |
| `process.consistency-fix-new-code-scrutiny` | Review a "make X consistent with Y" change as new code, not as a mechanical copy — and for any timer or effect involved, ask explicitly "when does this fire?", including on mount. | 2026-07-21 | [link](workflow-process.md#2026-07-21--make-x-consistent-with-y-review-findings-are-new-code-and-get-new-code-scrutiny-542) |
| `process.enumerate-workaround-behaviors-before-deleting` | When an issue says "delete X", enumerate every behavior X provided before removing it — a workaround often serves a second purpose that outlives the first. | 2026-07-21 | [link](workflow-process.md#2026-07-21--before-deleting-a-workaround-enumerate-every-behavior-it-provided-542) |
| `process.foreign-worktree-plumbing-merge` | Never commit or merge inside a worktree another session created; land the merge with git plumbing against the branch ref instead. | 2026-07-21 | [link](workflow-process.md#2026-07-21--never-commit-or-merge-inside-a-worktree-you-did-not-create-542) |
| `process.harden-with-runtime-posture-not-clamp` | When a security fix constrains a capability the roadmap will later want, make the safe state the DEFAULT OF A SWITCH rather than a wall — and read the feature's own issue for its end-state first. | 2026-07-21 | [link](workflow-process.md#2026-07-21--harden-a-soon-to-grow-feature-with-a-runtime-posture-not-a-hardcoded-clamp-542) |
| `process.independent-review-rubric` | Run an independent review pass — preferably a different model family, otherwise a cold-context review-only agent — on any diff touching locks/concurrency, auth/security, API write-path handlers, or DB migrations, or larger than ~150 changed C# lines; skip only for a pure-SPA/docs leaf with no server-state effect, and state the skip and its reason in the PR or close comment. | 2026-07-21 | [link](workflow-process.md#2026-07-21--independent-cross-model-review-is-mandatory-on-risky-diffs-a-skip-is-a-stated-auditable-exemption-542) |
| `process.issue-qualification-audit` | Run `scripts/issue-qualification-audit.sh` at session end and label everything it flags, including issues you filed that session. | 2026-07-21 | [link](workflow-process.md#2026-07-21--every-open-issue-carries-a-priority-label--run-the-h12-audit-at-session-end-542) |
| `process.local-gate-before-push` | Run the local build/test gate and a cold-context, scoped "review only" adversarial review over the diff, fold the fixes, and only then push or open the PR. | 2026-07-21 | [link](workflow-process.md#2026-07-21--local-buildtest-gate--cold-context-review-run-before-the-push-not-after-542) |
| `process.lock-ownership-enumerate-producers` | Before trusting any "single owner / no double release / no cross-release" claim, grep the whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-lockchannel-no-cross-release-verdict-must-enumerate-every-producer-via-grep-542) |
| `process.one-worktree-one-committing-agent` | Never run two committing agents concurrently on one worktree — give each parallel slice its own worktree branched off the feature branch and merge back. | 2026-07-21 | [link](workflow-process.md#2026-07-21--one-worktree-one-committing-agent-542) |
| `process.parallel-session-claim` | Apply the `in-progress` label before starting an issue, and still read its dependency notes before touching shared surfaces — a claim prevents duplicate pickup, not overlapping code changes. | 2026-07-21 | [link](workflow-process.md#2026-07-21--claim-with-in-progress-before-working-claiming-is-not-collision-safety-542) |
| `process.pr-routine-sequence` | Worktree off origin/main → implement → regenerate API artifacts → full local tests + cold review + live-E2E ALL before the push → push, open PR, arm the CI monitor at open → fixes after the push are follow-up commits, never amend/force-push. | 2026-07-21 | [link](workflow-process.md#2026-07-21--the-pr-routine-is-a-fixed-sequence-validate-locally-then-push-then-only-follow-up-commits-542) |
| `process.review-disagreement-frontier-judge` | When independent reviews disagree on a gate PR, escalate to the frontier judge, and put the proposed fix approach in front of it — not just the disputed finding. | 2026-07-21 | [link](workflow-process.md#2026-07-21--review-disagreement-on-a-gate-pr-escalates-to-the-frontier-judge--and-the-proposed-fix-escalates-with-it-542) |
| `process.shared-tree-readonly` | Never commit in `/Users/timothy/ersatztv` and never read its `git log`/`git status`/HEAD to infer anything about `main` — work in a worktree off `origin/main`, which is the only source of truth. | 2026-07-21 | [link](workflow-process.md#2026-07-21--the-shared-tree-at-userstimothyersatztv-is-read-only-and-tells-you-nothing-about-main-542) |
| `process.subagent-drop-resume` | Treat a subagent connection drop as laptop sleep or transient network and re-resume via SendMessage — the work survives. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-dropped-subagent-connection-is-transient-resume-dont-restart-542) |
| `release.api-contract-ci-gate` | A PR touching `ErsatzTV/Controllers/Api/**` or `ErsatzTV.Core/Api/**` must ship regenerated OpenAPI artifacts (`v1.json`, `v1.d.ts`, `endpoint-index.md`) in the same diff, enforced by a blocking `api-docs` CI job that regenerates-and-diffs against a fresh build. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--blocking-ci-gate-for-api-contract-artifacts-303-h4h5) |
| `release.done-when-merge-consent` | A PR may merge only when its linked issue's `## Done-when` checklist is fully ticked and the PR's CI is green, enforced by a PreToolUse hook on the Gitea merge tool (deny/allow/ask) plus a pre-push backstop for direct pushes to main. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--merge-consent-derived-from-state-via-a--done-when-issue-checklist-303-h6) |
| `release.format-as-you-touch-rebase` | A blocking `format` CI job runs `dotnet format --verify-no-changes` scoped only to the PR's changed `.cs` files (never the legacy BOM backlog), and a PR branch must be kept current by rebasing on `origin/main` (never merging main in), enforced by `.husky/pre-push` → `prepush-rebase-check.sh`. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--formatting-as-you-touch-enforced-rebase-not-merge-for-pr-branches-311-h11--format-ci) |
@@ -122,3 +150,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `spa.topbar-primary-action` | The TopBar's primary-action "+" button renders only when the active route declares a non-empty `primaryAction`, is wired (via a shared `usePrimaryAction` hook) only on single-unambiguous-create-flow list screens, and is dropped everywhere else rather than left as a dead/no-op button. | 2026-07-12 | [link](../decisions.md#2026-07-12--topbar-primary-action-button-wire-creates-drop-the-rest-238) |
| `spa.yaml-validator-textarea` | The YAML playout validator takes pasted YAML via a `<textarea>`, not a server-side file path, since the SPA has no filesystem access. | 2026-07-09 | [link](../decisions.md#2026-07-09--yaml-playout-validator-paste-textarea-instead-of-a-server-file-path) |
| `startup.parallel-orientation` | A fresh session runs two concurrent tracks at startup — Orientation (`AGENTS.md`/`CLAUDE.md` → `docs/README.md` task-signal map → the active decisions catalog `docs/decisions/README.md`) and, only when no issue is named, Selection (`scripts/select-queue.sh N`, deterministic live-Gitea ranking). A named issue skips Selection entirely. ersatztv#237, the closed pickup tracker this replaces, is reduced to a single archival breadcrumb and MUST NOT be read for live state. | 2026-07-21 | [link](../decisions.md#2026-07-21--parallel-orientation--selection-is-the-startup-protocol-237-retired-520) |
| `testing.e2e-local-fresh-config-dir` | Always point `scripts/e2e-local.sh` at a fresh config dir — a reused one never emits the probe's ready line and the script kills a healthy server. | 2026-07-21 | [link](workflow-process.md#2026-07-21--run-scriptse2e-localsh-against-a-fresh-config-dir-a-reused-one-hangs-the-readiness-probe-542) |
| `testing.live-e2e-prepush-timing` | Run live-E2E via `scripts/e2e-local.sh` before pushing a write-path or UI change, and exercise download endpoints with curl, never a browser tab. | 2026-07-21 | [link](workflow-process.md#2026-07-21--live-e2e-runs-before-the-push-and-downloads-are-curled-not-browsed-542) |
| `testing.playwright-mcp-download-and-recovery` | In Playwright-MCP E2E, fetch file-download endpoints with curl — never a browser tab or `window.open` — and if browser tools stall repeatedly, `pkill -f ms-playwright-mcp` and drive a fresh session. | 2026-07-21 | [link](workflow-process.md#2026-07-21--playwright-mcp-curl-download-endpoints-never-open-a-tab-or-windowopen-542) |
| `testing.troubleshoot-path-cannot-test-branding` | Verify logo/watermark/bug changes through a real channel playout — a green troubleshoot run proves nothing about branding. | 2026-07-21 | [link](workflow-process.md#2026-07-21--channel-branding-is-not-testable-through-the-troubleshooting-playback-api-542) |
+526
View File
@@ -0,0 +1,526 @@
# Session-workflow, CI-triage & review-process decisions (#542)
How we *work* — orchestration, CI-run triage, review routing, worktree and parallel-session hygiene,
and the tooling gotchas that have cost real sessions. Distinct from the engineering "why the code
does X" records: nothing here decides product behaviour.
These records were extracted from `docs/handoffs/chicorytv-issue-queue.md` in #542. That doc had
become the **only** copy of most of this knowledge — an inventory found ~30 of ~38 lore bullets had
no decision record at all — so it was neither retrievable via the catalog nor safe from a careless
prune. The rules are now records; the handoff doc keeps the instruction and points here for the why.
Related: `release-ci-governance.md` (merge/release gates and hooks), `startup.parallel-orientation`
(session-start tracks), `session.shared-checkout-refresh` (session-end shared-tree refresh).
---
## 2026-07-21 — A killed CI job reports `conclusion: failure`; read the log tail before diagnosing the diff (#542)
`key: ci.killed-job-triage` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure.
**Signals:** killed job · runner restart · conclusion failure · log truncated mid-step · `❌ Failure - Main` marker · `Unable to pull refs/heads/v4` · act cache noise · UTC vs UTC+2 · semantically-null diff · paths: n/a · issues: #542
**Mechanics:** Gitea Actions job logs + runner container uptime on the runner host.
When the operator restarts the runners (a retune, a reboot), in-flight jobs die and Gitea marks them
**`failure`**, not `cancelled`. The tell is that the log **stops mid-step with no error and no
`❌ Failure - Main …` marker** — a real failure always leaves that marker. On run 1006, `EF migration
integrity` and `Functional E2E` both "failed" on a **BOM-removal-only** diff (6 files, one line each,
zero content change) that could not possibly break them; the logs simply truncated mid-`dotnet build`
at 12:08 UTC and both runners showed `Up About an hour` — the retune had killed them.
**Log timestamps are UTC; the host is UTC+2.** Convert before correlating, or the restart looks two
hours off and a correct theory gets wrongly discarded.
Corollary: a diff that *cannot* cause a failure is evidence the failure isn't yours — when a job that
passed on the previous head fails on a semantically-null delta, suspect the environment and go read
the log rather than re-litigating the diff. Beware warnings that look fatal: `Unable to pull
refs/heads/v4: …` is act refreshing its `/root/.cache/act` action cache and is followed by `Cloned …`
— it is noise, not a cause. Grep for the failure marker, not for the word "error".
## 2026-07-21 — `cancelled` is not `failure`; a cancelled run is no verdict (#542)
`key: ci.cancelled-is-not-a-verdict` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Treat a `cancelled` conclusion as "no verdict" — never as pass or fail — and report FAILED and CANCELLED counts separately in any CI monitor.
**Signals:** conclusion cancelled · run-level vs job-level conclusion · pre-cancel genuine failure · CI monitor state != pending · phantom failure · paths: n/a · issues: #542
**Mechanics:** Gitea Actions run/job API; monitor logic, e.g. `fail=[j for j in jobs if j['conclusion']=='failure']; canc=[j for j in jobs if j['conclusion']=='cancelled']`.
The operator cancels runs by hand — they are the only party who can (see
`ci.batch-pushes-no-cancel-route`) — and a run-level `conclusion: cancelled` means the run produced
**no verdict** about your diff. Two traps follow. First, a run whose *overall* state is `failure` may
hold a **genuine job failure that happened before the cancel**: check job-level `conclusion` plus
timestamps rather than attributing the red to the cancel. Second, a cancelled run tells you nothing,
so never claim green on one.
A monitor that only asks "is state != pending" will report a cancelled run as a failure and send the
next session debugging a phantom. Split the two counts explicitly.
## 2026-07-21 — An infra-shaped red under host load is not a code failure (#542)
`key: ci.infra-shaped-red-under-load` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure.
**Signals:** load average 243 · `Setup .NET` 11ms failure · `remove /root/.cache/act/…/.gitignore: no such file or directory` · buildx `DeadlineExceeded: failed to compute cache key` · `Build & push image` · `EF migration integrity` · `FactoryServer-L` game server · phantom CI bug · paths: n/a · issues: #542
**Mechanics:** `uptime` on the runner host (bumblebee); Gitea Actions job step logs.
On 2026-07-17, with bumblebee at load average **243**, two unrelated jobs died in ways that look like
code bugs but weren't: `EF migration integrity` failed in ~11ms inside **`Setup .NET`** with
`remove /root/.cache/act/…/.gitignore: no such file or directory` (act's shared cache), and main's
`Build & push image` died on buildx `DeadlineExceeded: failed to compute cache key` after ~210s of
retries. Both re-ran green at normal load, and the migration job had no model change to test in the
first place.
The diagnostic tell is the **location** of the failure: a setup or cache step, before your code
compiles. One sample under load is not evidence of a systemic problem — this nearly got filed as #390
lane-rebalance fallout, which the evidence did not support, and would have sent the next session
chasing a phantom. Note also that a game server (`FactoryServer-L`) shares that host with the
runners, so high load is not always CI's own doing.
## 2026-07-21 — Batch your pushes: there is no agent-side cancel route on Gitea 1.25.4 (#542)
`key: ci.batch-pushes-no-cancel-route` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Hold review fixes, doc corrections and format fixes locally and push **once** — a superseded run cannot be cancelled from the agent side and holds a runner slot until it finishes.
**Signals:** cancel run 404 · Gitea 1.25.4 · `POST /api/v1/.../actions/runs/{id}/cancel` · MCP `actions_run_write` `cancel_run` · session+CSRF · `dispatch_workflow` · orphaned run · 4-slot runner · paths: n/a · issues: #542
**Mechanics:** Gitea Actions REST + MCP `actions_run_write`; operator-only cancel in the browser UI.
Cancellation is impossible from the agent side on this Gitea (**1.25.4**):
`POST /api/v1/.../actions/runs/{id}/cancel` returns **404**, MCP `actions_run_write`'s `cancel_run`
returns **404**, and the web-UI route needs a session + CSRF that does not script (login 303s with no
session cookie). **Only the operator can cancel, in the browser** — so if you must supersede a live
run, say so explicitly instead of leaving it burning. (`dispatch_workflow` is a different route and
still works for re-triggering a **main** run.)
This corrects the older "superseded runs drain on their own" framing: they do finish, but they hold
one of the 4 runner slots while doing it, which is a real cost with several parallel sessions.
## 2026-07-21 — Do not gate or throttle pushes on host health (#542)
`key: ci.no-host-health-gating` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Push when your work is validated — never SSH to bumblebee to sample load/RAM first, and never hand-schedule around other sessions' runs.
**Signals:** trust the build queue · runner retune · don't sample load before pushing · queueing is the queue's job · paths: n/a · issues: #542
**Mechanics:** Gitea build queue (two runners, 4 slots).
The runners were retuned for stability (operator, 2026-07-17); queueing is the queue's job, not the
agent's. Sampling host load before a push is both unreliable and unnecessary, and hand-scheduling
around other sessions produces coordination that no one can verify.
Batch your pushes (see `ci.batch-pushes-no-cancel-route`) because orphaned runs cannot be cancelled —
**not** because the host needs protecting. The two rules have the same action and different reasons;
conflating them leads to load-watching behaviour that this record forbids.
## 2026-07-21 — A lone `decisions lifecycle` red is a known infra flake: do nothing (#542)
`key: ci.decisions-lifecycle-flake` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** When `decisions lifecycle` is the **only** red job, do not investigate and do not create a new run to clear it — no rebase, no `--amend`, no no-op push; the operator reruns that single job from the Gitea UI.
**Signals:** decisions lifecycle red · append-only gate · known flake · operator selective rerun · no-op push · convincing local explanation · paths: `docs/decisions.md` · issues: #542, #473, PR #479
**Mechanics:** Gitea Actions `decisions lifecycle` job; operator-driven single-job rerun in the web UI.
Operator-stated, 2026-07-19. Report it as a known flake and carry on; only if *other* jobs are red too
does the run deserve diagnosis. Same family as the killed-job rule — a spurious single-job red is
cleared by the operator's selective rerun, never by pushing, and pushes cannot be cancelled anyway.
**The trap is that a convincing local explanation is always available.** On #473/PR #479 the job went
red just after `main` landed its own `decisions.md` entry, so "mine is no longer at EOF, I must
rebase" looked airtight. The rebase happened — and the job went red **again** on a head whose diff was
a verified pure EOF append with zero deleted lines. A rebase that provably satisfied the gate's stated
rule did not turn it green, which is the proof that the red was never about the diff. Check this rule
*before* theorising about a cause.
## 2026-07-21 — Arm the CI monitor at PR-open, via the commit-status endpoint (#542)
`key: ci.monitor-armed-at-pr-open` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Arm a CI monitor on the PR head sha the moment the PR opens, polling the commit-status endpoint — not at the end of the work.
**Signals:** arm monitor at PR open · commit status endpoint · head sha · red run sat unnoticed · MySQL-apply flake · paths: n/a · issues: #542, #216
**Mechanics:** `GET /api/v1/repos/timothy/ersatztv/commits/{sha}/status`.
CI runs concurrently with the review and E2E work that follows a push, so the cost of a late-armed
monitor is entirely wasted wall-clock. In the #216 session three PR runs sat red for roughly an hour
on a CI-only flake while review and E2E ran to completion — the reds were only discovered afterwards.
Related context for interpreting an early red: the old MySQL host-port 3306 collision is fixed on main
(`ef8915f1`), so a lone MySQL-apply red now indicates the known infra flake and warrants a rerun
rather than diagnosis.
## 2026-07-21 — Heavy-render web tests need explicit per-test vitest timeouts on the CI VM (#542)
`key: ci.web-test-per-test-timeouts` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Give heavy-render web tests an explicit per-test vitest timeout (e.g. 15s); never raise the global default to fix one slow test.
**Signals:** vitest timeout · 5s default · 100+ item grid · CI VM slower than local · run 686 · paths: `web/` tests · issues: #542
**Mechanics:** per-test timeout argument in the vitest test declaration.
The CI VM is materially slower than a dev Mac for render-heavy work. A web test rendering a 100+ item
grid runs in about 1s locally but hit the **5s** vitest default on the CI VM (run 686) and went red
for no code reason.
Bump the timeout on the specific test. Raising the global default hides genuine hangs across the whole
suite in exchange for fixing one known-slow case.
## 2026-07-21 — Build and verify locally, then trust it; CI confirms (#542)
`key: ci.verify-locally-ci-confirms` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Treat the local build/verify/review pass as the decision point and CI as confirmation — don't idle waiting on a run you have no reason to doubt.
**Signals:** ~9 min warm CI · docs-only PR seconds · CI VM 127 · commit status watch · local gate first · paths: n/a · issues: #542, #416
**Mechanics:** `GET /api/v1/repos/timothy/ersatztv/commits/{sha}/status`; CI on VM 127.
A warm full CI run takes roughly **9 minutes**; a docs-only PR completes in seconds since ersatztv#416's
docs-only skip. Neither duration justifies blocking on the run when the same gates already passed
locally.
Watch the run by commit status rather than by polling the UI, and continue working. CI *confirms* the
local verdict; it is not the first line of defence.
## 2026-07-21 — Independent cross-model review is mandatory on risky diffs; a skip is a stated, auditable exemption (#542)
`key: process.independent-review-rubric` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Run an independent review pass — preferably a different model family, otherwise a cold-context review-only agent — on any diff touching locks/concurrency, auth/security, API write-path handlers, or DB migrations, or larger than ~150 changed C# lines; skip only for a pure-SPA/docs leaf with no server-state effect, and state the skip and its reason in the PR or close comment.
**Signals:** cross-model review rotation · correlated blindspots · skip rubric · self-exemption · review-only agent · quota exhaustion · reviewer does not implement · paths: n/a · issues: #542, #235, #250
**Mechanics:** rotation — Codex reviews Claude-authored changes, Claude reviews Codex-authored changes; use whatever native integration is installed (client helpers like `codex:rescue` are optional accelerators, not protocol dependencies). On quota/tool exhaustion, fall back to a cold-context review-only agent in the current client and retry cross-model next window. Independent reviewers do review and stuck-diagnosis work only — never implementation on the same change they review.
The justification is correlated blindspots: independent model families miss *different* classes of
defect, so a single reviewer on a risky diff is the gap, not a saving. #235 (a reviewer clearing a lock
as sound because it never grepped for every producer) and #250 (a vacuous negative control that
silently ran a stale dll) are both cases the second, differently-blind pass caught.
The skip clause is deliberately narrow and deliberately loud. A self-exemption is an auditable claim,
not a vibe — silence is indistinguishable from forgetting, so the exemption must be written down where
the merge record can be re-read later.
## 2026-07-21 — Local build/test gate + cold-context review run BEFORE the push, not after (#542)
`key: process.local-gate-before-push` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Run the local build/test gate and a cold-context, scoped "review only" adversarial review over the diff, fold the fixes, and only then push or open the PR.
**Signals:** local gate before push · cold review before CI · CI is not the first line of defence · uncancellable run · queue pressure · paths: n/a · issues: #542
**Mechanics:** local build/test gate + a scoped review-only agent over `origin/main...HEAD`; CI (armed on the PR head sha at PR-open) *confirms* the result.
Gitea runners should only ever see already-validated work. CI here is a scarce shared slow resource and
a run cannot be cancelled on this Gitea version, so spending a run to surface a lint error or a
review-catchable bug is pure waste plus queue pressure on every other in-flight session.
CI confirms; it is not the first line of defence. Treating it as the first line inverts the cost: the
cheap fast private check is skipped in favour of the expensive slow shared one.
## 2026-07-21 — Review disagreement on a gate PR escalates to the frontier judge — and the proposed FIX escalates with it (#542)
`key: process.review-disagreement-frontier-judge` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** When independent reviews disagree on a gate PR, escalate to the frontier judge, and put the proposed fix approach in front of it — not just the disputed finding.
**Signals:** review disagreement · frontier judge · escalate the fix not just the verdict · gate PR reconciliation · paths: n/a · issues: #542, #235
**Mechanics:** n/a — escalation is a routing decision by the orchestrator.
The reconciliation is not only "who is right about the finding". On #235 the frontier model ratified
the disputed High *and* caught that the orchestrator's own proposed fix (a per-source lock-or-skip over
a fixed loop order) would deterministically starve the 2nd and later media source — prescribing instead
the pattern that shipped.
That is the whole argument: a judge given only the verdict question would have returned "the High is
real" and let a starving fix ship. Send the fix approach up with the finding.
## 2026-07-21 — "Make X consistent with Y" review findings are new code and get new-code scrutiny (#542)
`key: process.consistency-fix-new-code-scrutiny` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Review a "make X consistent with Y" change as new code, not as a mechanical copy — and for any timer or effect involved, ask explicitly "when does this fire?", including on mount.
**Signals:** consistency fix · propagating the pattern propagates the bug · mount-debounce · timer/effect fire timing · paths: n/a · issues: #542, #216
**Mechanics:** n/a — a review-attention rule.
Propagating a pattern propagates its bugs. A #216 review fix copied MediaBrowseScreen's mount-debounce
selection-wipe race into SearchScreen instead of catching it — the finding said "be consistent", and
consistency was achieved, including with the defect.
The mount question is the cheap general form of the check: most of this class hides in an effect or
timer that fires once more than the author pictured.
## 2026-07-21 — A lock/channel "no cross-release" verdict must enumerate every producer via grep (#542)
`key: process.lock-ownership-enumerate-producers` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Before trusting any "single owner / no double release / no cross-release" claim, grep the whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer.
**Signals:** cross-release · lock ownership · channel message producers · scheduler enqueues too · grep the host project · paths: n/a · issues: #542, #235, #267, #250
**Mechanics:** grep for every sender of the message type across the host project (e.g. all `Synchronize{X}Collections(` call sites), not just the controllers present in the PR diff. General form of "re-derive from a grep rather than enumerating from a list": see `sched.weighted-shuffle`.
On #235/#267 a cold fork cleared a new `scan-collections` per-provider lock as sound — "only the
acquirer enqueues, so only its message triggers the scanner unlock" — having checked the three new
controllers in the diff and stopped there. Codex grepped: `SchedulerService` also enqueues those
messages periodically and unlocked, and `ScannerService`'s `finally` released the lock whenever held,
so a scheduled scan cross-releases an API scan's lock (the #250 class).
Corollary: the independent pass earns its keep even when the first reviewer returns MERGEABLE. The
first verdict was not careless — it was complete over the evidence it looked at, which is exactly the
failure mode a second, differently-scoped reviewer catches.
## 2026-07-21 — Before deleting a "workaround", enumerate every behavior it provided (#542)
`key: process.enumerate-workaround-behaviors-before-deleting` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** When an issue says "delete X", enumerate every behavior X provided before removing it — a workaround often serves a second purpose that outlives the first.
**Signals:** remove the workaround · double duty · second purpose · bounded version re-scoped · PENDING_GRACE_TICKS · paths: n/a · issues: #542, #232
**Mechanics:** n/a — a scoping rule applied when reading a removal issue.
#232 said "remove the SPA `PENDING_GRACE_TICKS` grace-tick workaround now the API is honest". The grace
ticks did double duty: compensating the lying-200 (genuinely obsolete) *and* absorbing the inherent
queue→observed-active lag and fast-completion race (still needed). Removing them wholesale wedged the
scan button disabled-until-reload for short scans; both the fork and Codex caught it.
The landing move is usually not "delete" or "keep" but a bounded version re-scoped to the purpose that
still holds.
## 2026-07-21 — The shared tree at /Users/timothy/ersatztv is read-only and tells you nothing about `main` (#542)
`key: process.shared-tree-readonly` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Never commit in `/Users/timothy/ersatztv` and never read its `git log`/`git status`/HEAD to infer anything about `main` — work in a worktree off `origin/main`, which is the only source of truth.
**Signals:** shared checkout · "main checkout" · worktree discipline · stale HEAD · origin/main is truth · node_modules copy · paths: `/Users/timothy/ersatztv`, `.claude/worktrees/`, `.worktrees/` · issues: #542
**Mechanics:** `git worktree add -b <branch> <path> origin/main`; `git fetch origin main` and reference `origin/main` explicitly — the local `main` ref lags and nobody is obliged to update it. The one sanctioned use of the shared tree is read-only: copying `web/node_modules` out of it (keep that fresh — a stale copy broke typecheck in a #198 worktree).
There is no "main checkout". `/Users/timothy/ersatztv` is a shared mutable working tree whose HEAD is
whatever the last session left there; its name lies. The rule holds even for a one-line docs fix — make
a worktree.
This is a design flaw, not a discipline failure: do not "fix" it by telling sessions to check harder,
because a check does not stay true. It broke for two different sessions on the same day, 2026-07-17,
both doing the obvious thing. Session A assumed it was on `main` — correct when it pulled at 12:46,
stale by 14:17 — and committed; the commit landed on the CI session's `ci/604-lane-rebalance`, 24
seconds after that session's own commit at 14:17:28. Two writers, one tree; nothing tangled, by luck.
Session B ran `git log` there, read that same branch's HEAD, and concluded `main` was "4 behind origin"
— a phantom. A session-start `git status` *appears to confirm* the false assumption, which makes it
worse than a plainly wrong claim.
Cross-reference `session.shared-checkout-refresh`, which covers the complementary failure mode: any
FILE (including a pasted kickoff prompt) read out of that tree may be stale, and the session-end
refresh script that mitigates it. This record covers the git-state failure mode only.
## 2026-07-21 — One worktree, one committing agent (#542)
`key: process.one-worktree-one-committing-agent` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Never run two committing agents concurrently on one worktree — give each parallel slice its own worktree branched off the feature branch and merge back.
**Signals:** parallel agents · concurrent commits · worktree per slice · read-only review agent · paths: n/a · issues: #542
**Mechanics:** `git worktree add` a fresh tree per slice off the feature branch; merge the slices back into the feature branch when they land.
Read-only review agents sharing a tree are fine — the constraint is on *writers*. Sequence slices only
where one genuinely depends on another's output; "backend-first narrow, SPA-wide after" worked well for
#216.
## 2026-07-21 — Never commit or merge inside a worktree you did not create (#542)
`key: process.foreign-worktree-plumbing-merge` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Never commit or merge inside a worktree another session created; land the merge with git plumbing against the branch ref instead.
**Signals:** sibling session worktree · branch already checked out · plumbing merge · read-tree · commit-tree · GIT_INDEX_FILE · paths: n/a · issues: #542
**Mechanics:** `git read-tree -m <base> <ours> <theirs>` into a temporary `GIT_INDEX_FILE` → `git write-tree` → `git commit-tree -p <ours> -p <theirs>` → push the resulting commit to the branch ref. The owning session then `git pull`s it.
With several sessions on one machine, any given branch may already be checked out in a sibling
session's worktree. Touching that checkout writes into another agent's working state; the plumbing
route mutates only the ref, so the merge lands without disturbing the owner's tree.
## 2026-07-21 — Fix onto an unmerged feature branch by branching off it and FF-pushing (#542)
`key: process.branch-off-feature-branch` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** To fix work on an unmerged feature branch, branch off that branch and land by fast-forward push — and after creating a worktree, drive the first Edit/Read from ITS absolute paths and `git status` it before building.
**Signals:** unmerged feature branch · FF-push · path leak · sibling worktree recon · wrong tree edits · "0 new tests" · paths: n/a · issues: #542
**Mechanics:** `git worktree add -b <fix-branch> <path> <feature-branch>`; land with a fast-forward push to the feature branch. Never touch a sibling worktree that has the branch checked out (see `process.foreign-worktree-plumbing-merge`).
The path-leak trap, hit on 2026-07-12 (#289 → PR#76): reconnaissance by `Read`ing a *sibling* worktree
leaked that tree's absolute paths into subsequent `Edit`s, so the edits landed there rather than in the
fresh worktree. `dotnet` then built the unchanged tree, and the tell was a build reporting "0 new
tests". A `git status` in the new worktree before the first build catches this in one command.
## 2026-07-21 — Bound parallel builds by FREE RAM, never regenerate goldens (#542)
`key: process.build-concurrency-limits` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Run at most 3–4 concurrent dotnet/npm builds on this Mac, gate launches on FREE RAM rather than CPU load, and never set `ETV_UPDATE_GOLDENS` / `ETV_UPDATE_PLAYOUT_GOLDENS`.
**Signals:** build concurrency · memory_pressure · RAM starvation · fan-out cap · golden regeneration · paths: n/a · issues: #542
**Mechanics:** `memory_pressure -Q` — under 20% free, do not launch more build agents; under 10%, pause or stagger. Host is an M4, 10 cores, 16 GB.
Parallelize by default when slices are genuinely disjoint: use up to the client's concurrency limit,
normally 2–3 delegated agents plus the orchestrator. Fan out bounded recon early, but do not invent
artificial slices or exceed a runtime-imposed agent cap. CPU load spikes during builds are benign and
are not a stop signal; the resource that actually runs out is memory — the historic crash was RAM
starvation from an 8–9-way fan-out, which is why 5+ concurrent builds is never acceptable.
Separately: never set the golden-regeneration env vars. Regenerating goldens to make a diff disappear
hides exactly the change the golden net exists to surface.
## 2026-07-21 — A dropped subagent connection is transient; resume, don't restart (#542)
`key: process.subagent-drop-resume` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Treat a subagent connection drop as laptop sleep or transient network and re-resume via SendMessage — the work survives.
**Signals:** subagent disconnect · connection drop · laptop sleep · SendMessage resume · paths: n/a · issues: #542
**Mechanics:** n/a — client-side resume.
Restarting the agent from scratch discards context the agent still holds; the drop is at the transport,
not the agent.
## 2026-07-21 — Every open issue carries a `priority:` label — run the H12 audit at session end (#542)
`key: process.issue-qualification-audit` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Run `scripts/issue-qualification-audit.sh` at session end and label everything it flags, including issues you filed that session.
**Signals:** issue qualification · priority label · queue tiering · unqualified issues · H12 · session-end check · paths: `scripts/issue-qualification-audit.sh` · issues: #542, #312
**Mechanics:** `scripts/issue-qualification-audit.sh` (H12, ersatztv#312) lists OPEN issues missing a `priority:` label. It is fail-open without Gitea creds and advisory — exit 1 when any issue is unqualified. Sibling to H11 (both #311/#312).
Queue tiering keys off `priority:` and gate labels, so an issue without one is simply invisible to the
selector — it is filed but will never be picked. "Fully qualified" means the issue has a
`priority: {high,medium,low}` label, which signals triage ran; gate-vs-backlog is then derivable from
the `review` label and milestone. A milestone is **not** required — backlog is unmilestoned.
## 2026-07-21 — Claim with `in-progress` before working; claiming is not collision safety (#542)
`key: process.parallel-session-claim` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Apply the `in-progress` label before starting an issue, and still read its dependency notes before touching shared surfaces — a claim prevents duplicate pickup, not overlapping code changes.
**Signals:** parallel sessions · in-progress label · claim race · dependency notes · shared surfaces · lore pruning · paths: `docs/handoffs/chicorytv-issue-queue.md` · issues: #542
**Mechanics:** `in-progress` label on the Gitea issue. The tiny read→claim race window is accepted; the later claimant backs off. Runner topology: two runners (ci-runner VM 127 + bumblebee-runner), 4 slots total.
The critical nuance is what a claim does *not* buy. Two sessions on non-overlapping issues can still
collide in the same files. Before touching a shared surface, read the issue's dependency notes ("#234
after #231", "coordinate with #215").
When editing the standing lore/handoff doc, prune covered and stale bullets rather than appending — it
is not append-only, and git keeps the history. `git pull --rebase` before committing it, since it is
the single most contended file across parallel sessions.
## 2026-07-21 — BOM/format pre-push detection: use the `xxd` loop, and run `dotnet format --include` under `bash -c` (#542)
`key: process.bom-format-detection-recipe` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Before any push touching `.cs`, detect BOMs with the `xxd` byte check and verify the format gate with `dotnet format --include` run under `bash -c`, never bare zsh.
**Signals:** UTF-8 BOM · `efbbbf` · `dotnet format --verify-no-changes` · `--include` · `mapfile` · zsh vs bash · #311 format gate · detector verification · paths: `ErsatzTV.sln`, `.editorconfig` · issues: #542, #311, #70, PR #402, PR #405
**Mechanics:** Local pre-push shell; CI's format job recipe (`ci.format-gate-folder-mode`). The BOM *policy* is `release.format-as-you-touch-rebase`; this record is the mechanics only.
Detection loop, verbatim:
```bash
for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; done
```
**Verify your detector.** An `od -An -c | grep '357 273 277'` check reported all-clean while 19 files
were actually dirty. A detector that can only say "ok" is worse than none — use the `xxd` form or read
the bytes directly, and distrust a clean result you didn't prove can go dirty.
**`dotnet format --include` DOES work here** — an earlier note claiming it silently no-ops was WRONG.
The apparent no-op was **the shell**: CI's recipe uses `mapfile`, which is bash-only, and the default
shell here is zsh → empty array → zero files → exit 0. Working form:
```bash
bash -c 'mapfile -t files < <(git diff --name-only --diff-filter=ACM origin/main...HEAD -- "*.cs")
dotnet format ErsatzTV.sln --no-restore --verify-no-changes --include "${files[@]}"' # exit 0 = gate passes
```
## 2026-07-21 — Run `scripts/e2e-local.sh` against a FRESH config dir; a reused one hangs the readiness probe (#542)
`key: testing.e2e-local-fresh-config-dir` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Always point `scripts/e2e-local.sh` at a fresh config dir — a reused one never emits the probe's ready line and the script kills a healthy server.
**Signals:** `READY_LINE` · `Done migrating search index` · `Search index is already version 50` · readiness probe timeout · e2e-local hang · paths: `scripts/e2e-local.sh`, `docs/e2e-local.md` · issues: #542, #533, #502, #524
**Mechanics:** The script greps one fixed `READY_LINE` — there is no "wait for either line" option today.
The probe waits for `Done migrating search index`, which only a **fresh** config emits. A reused config
logs `Search index is already version 50` instead, so the line never arrives, the probe times out, and
the script kills a perfectly healthy server — a failure that reads as a broken build.
Surfaced in the #502 session (two failed launches before the cause was obvious) and recovered from the
old tracker's comment log by the #524 triage. `docs/e2e-local.md` documents the ready line and
recommends a fresh dir, but not this failure mode. Widening the probe to
`Done migrating search index|Search index is already version` is tracked as **#533**.
## 2026-07-21 — Channel branding is NOT testable through the troubleshooting-playback API (#542)
`key: testing.troubleshoot-path-cannot-test-branding` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Verify logo/watermark/bug changes through a real channel playout — a green troubleshoot run proves nothing about branding.
**Signals:** `PrepareTroubleshootingPlaybackHandler` · synthetic channel · `Artwork = []` · `ChannelId > 0` branch · watermark · channel logo · bug overlay · paths: `ErsatzTV.Application` troubleshooting handlers · issues: #542
**Mechanics:** n/a — a property of the handler, not a toggle.
`PrepareTroubleshootingPlaybackHandler` builds a **synthetic** channel with `Artwork = []`, and its
`ChannelId > 0` branch ignores the media item. Both together mean no branding state ever reaches the
pipeline through that path.
So the troubleshoot endpoint is a false witness for this class: it exercises the transcode pipeline but
not the overlay inputs. Route branding verification through a real channel playout instead.
## 2026-07-21 — Playwright-MCP: curl download endpoints, never open a tab or `window.open` (#542)
`key: testing.playwright-mcp-download-and-recovery` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** In Playwright-MCP E2E, fetch file-download endpoints with curl — never a browser tab or `window.open` — and if browser tools stall repeatedly, `pkill -f ms-playwright-mcp` and drive a fresh session.
**Signals:** Playwright MCP · file download endpoint · window.open · browser tab stall · `ms-playwright-mcp` · paths: n/a · issues: #542
**Mechanics:** `pkill -f ms-playwright-mcp` then re-drive from a fresh MCP session.
Download endpoints hand the browser a file rather than a page; the tab has nothing to snapshot and the
tool waits on navigation that never completes. curl gives you the bytes and the status code directly.
Repeated stalls are a wedged MCP browser process, not a slow app — kill it and restart rather than
retrying the same call.
## 2026-07-21 — Launch Codex cheap workers via `codex exec`, not `spawn_agent` (#542)
`key: process.codex-cheap-worker-launch` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** For bounded tool-bearing selector/recon work, launch a Codex worker with `codex exec -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only`; `spawn_agent` buys parallelism but no cost savings.
**Signals:** codex exec · `--ephemeral` · `spawn_agent` · `gpt-5.4-mini` · `gpt-5.1-codex-mini` HTTP 400 · `model_reasoning_effort` · `~/.codex/models_cache.json` · paths: `~/.codex/models_cache.json` · issues: #542
**Mechanics:** Local `codex` CLI on this ChatGPT-authenticated host. Tested 2026-07-14.
The native `spawn_agent` interface currently exposes no model or effort selector, so it parallelises at
full cost. Launch a separate worker instead:
```
codex exec --ephemeral --color never -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only -C <repo> "<hard-capped scope contract>"
```
GPT-5.4-Mini is the current supported small/cost-efficient profile on this host; the older
`gpt-5.1-codex-mini` guess fails with **HTTP 400**. Keep `low` whenever any shell/MCP/web tool is
exposed; reserve `minimal` for explicitly tool-free synthesis over already-supplied bounded evidence.
On model rollover, inspect the visible entries in `~/.codex/models_cache.json` rather than guessing
names, then update this record once verified.
## 2026-07-21 — Live-E2E runs BEFORE the push, and downloads are curled, not browsed (#542)
`key: testing.live-e2e-prepush-timing` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Run live-E2E via `scripts/e2e-local.sh` before pushing a write-path or UI change, and exercise download endpoints with curl, never a browser tab.
**Signals:** live E2E timing · pre-push validation · `scripts/e2e-local.sh` · download endpoint · browser tab · paths: `scripts/e2e-local.sh`, `docs/e2e-local.md` · issues: #542
**Mechanics:** Seeding recipe: `docs/e2e-local.md` → "Seeding a local TV library for E2E" (tiny ffmpeg testsrc MKVs + `LibraryPath` SQL rows + scan). The *requirement* itself is `release.live-e2e-required`; this record adds only the timing and the download rule.
Live-E2E after the push wastes a CI run to learn something the local harness would have said first —
and on this Gitea a run cannot be cancelled. Fold E2E findings into the branch while nothing is pushed.
Download endpoints must be curled for the same reason as the Playwright rule: a tab has no page to
settle on and the browser tool hangs.
## 2026-07-21 — The PR routine is a fixed sequence: validate locally, then push, then only follow-up commits (#542)
`key: process.pr-routine-sequence` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Worktree off origin/main → implement → regenerate API artifacts → full local tests + cold review + live-E2E ALL before the push → push, open PR, arm the CI monitor at open → fixes after the push are follow-up commits, never amend/force-push.
**Signals:** PR routine · worktree off origin/main · `./scripts/update-openapi.sh` · `npm run generate:api` · cold-context review · CI monitor at PR open · no force-push on a pushed branch · derived-consent merge gate · paths: `scripts/update-openapi.sh`, `scripts/e2e-local.sh` · issues: #542
**Mechanics:** CI watch by commit status: `/api/v1/repos/timothy/ersatztv/commits/{sha}/status`. Merge via the derived-consent gate (`pretooluse-merge-consent.sh`).
The sequence: create a worktree off `origin/main`; implement; for **API changes build the app project
FIRST**, then `./scripts/update-openapi.sh`, then `npm run generate:api` (order matters — the generator
reads the built app's spec). Then run the full local test pass, a cold-context adversarial review
scoped to "review only", and live-E2E for UI/write-path changes — **all before the push**. Fold
pre-push fixes into the branch; amend/reword is fine while nothing is pushed.
Then push, open the PR, and arm the CI monitor **at open**, not at the end. Once the branch is pushed,
every further fix — a post-push cross-model pass, or a CI-surfaced issue — lands as a **follow-up
commit**; never amend or force-push a pushed branch. On green, merge through the derived-consent gate
and write the structured close comments the applicable `AGENTS.md` / `CLAUDE.md` protocol requires.
## 2026-07-21 — Harden a soon-to-grow feature with a runtime posture, not a hardcoded clamp (#542)
`key: process.harden-with-runtime-posture-not-clamp` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** When a security fix constrains a capability the roadmap will later want, make the safe state the DEFAULT OF A SWITCH rather than a wall — and read the feature's own issue for its end-state first.
**Signals:** runtime posture · feature toggle · `ERSATZTV_ALLOW_WRITES` · read-only default · hardcoded clamp · roadmap end-state · paths: n/a · issues: #542, #289, #58
**Mechanics:** `ERSATZTV_ALLOW_WRITES` env var, default `false`. See `mcp.server-foundation`.
The MCP hardening in #289 could have shipped a GET-only clamp. But the MCP's final design (#58) is read
**and** write, so that clamp would have been ripped out within a release — churn plus a window where
the removal, not the original fix, is the risky change.
A runtime read-only posture (`ERSATZTV_ALLOW_WRITES`, default false) gives the same safety today and
grows into the intended design without a revert. The generalisable step is the cheap one: check the
feature's issue for its end-state *before* choosing between a switch and a wall.
## 2026-07-21 — Gitea's `?milestones=` issue filter silently no-ops on names containing `:` or `+` (#542)
`key: ci.gitea-milestone-filter-noop` · `status: active` · `since: 2026-07-21` · `supersedes: none` · `superseded-by: none`
**Rule:** Never filter issues with the server-side `?milestones=<name>` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`.
**Signals:** `GET /issues?milestones=` · milestone name filter · returns the whole open-issue list · `:` or `+` in a milestone title · mis-tiered issues · count looks like "all open issues" · paths: `scripts/select-queue.sh` · issues: #542, #77, #72
**Mechanics:** `?state=open&type=issues&limit=50`, then filter in the client, e.g. `[i for i in issues if (i['milestone'] or {}).get('title') == NAME]`. `scripts/select-queue.sh` already does this; the trap is for anyone writing a *new* query.
The filter does not error on a name it cannot round-trip — it returns the **entire open-issue list** as
though every issue matched. Milestone titles here routinely contain the offending characters (e.g.
`Scheduling: refactor + distribution`), so the failure is the normal case, not an edge case.
Anything trusting that response mis-tiers issues: during #77 selection it made unmilestoned #72 look
like a member of the milestone and hid #77's true sibling set. **The tell is a result count suspiciously
equal to the total number of open issues** — if a filtered query returns everything, the filter silently
failed; re-derive membership locally rather than reasoning about the result.
The dependencies API is unaffected: `POST /issues/{n}/dependencies` with `{"owner","repo","index"}` sets
blocked-by correctly, though the bare `{index}` form returns 201 without reliably attaching — verify
with the GET. (That last sentence is carried forward from an earlier revision of the handoff doc,
commit `f93458c7`, where it was dropped by a later prune rather than disproved.)
+90 -372
View File
@@ -15,7 +15,7 @@ history in `docs/decisions/archive/startup.md`). Do not read it for current queu
The ChicoryTV rewrite + go-live arc is **COMPLETE** and closed out. The project is now in
**maintenance / backlog mode**. This is a *terminal* fact (the arc will not re-open), so it belongs
in this standing file — unlike any "current frontier issue", which must never be written here (read
it live; see the queue-drift lesson in the archived section below).
it live — status embedded in prose always drifts).
So a fresh session stops re-discovering this every time:
- The **arc / gate / open-`review` tiers are normally EMPTY now.** An empty arc frontier is the
@@ -115,8 +115,8 @@ never orchestrator-tier work:
`GET /issues/{n}/dependencies` on every candidate (dropping any with an OPEN blocker), tiers by
LOCAL `.milestone.state`/`review`/`priority:` filters (never the server `?milestones=` name filter,
which no-ops on `:`/`+`), and orders by (tier, priority, issue#). **Its DEPS/tiering/ordering are
deterministic — trust them, do not re-derive them by hand** (see the archived section below for why
that used to be necessary and isn't anymore). It also raises two JUDGMENT flags it deliberately does
deterministic — trust them, do not re-derive them by hand** (the hand-derivation failure modes the
script encodes are recorded at `ci.gitea-milestone-filter-noop`). It also raises two JUDGMENT flags it deliberately does
not decide — `CLAIM?` (a claim can precede the label) and `UMBRELLA?` (an epic whose children are the
real pickups) — resolve ONLY those by reading the flagged issue's comments/body. The script does not
rank an arc tier (there is no active arc right now — maintenance/backlog mode, see above); if a new
@@ -255,382 +255,100 @@ is both the human-readable summary and the per-issue unit MemPalace mines for re
```
HARD CONSTRAINTS:
- Work in worktrees off `origin/main` — ALWAYS, including for a one-line docs commit. `/Users/timothy/ersatztv`
is a **shared mutable tree, not a `main` mirror**: never commit in it, and never read its HEAD/`git log`
to learn anything about `main` (you'll get another session's branch). Its only sanctioned use is
**read-only**: copying `web/node_modules` out of it. `origin/main` is the source of truth — `git fetch
origin main` and reference it explicitly. See the lore bullet for the two sessions this bit on 2026-07-17.
- PARALLELIZE BY DEFAULT when there are genuinely disjoint slices: use up to the client's concurrency
limit, normally 2–3 delegated agents plus the orchestrator. Fan out bounded recon early, but do not
invent artificial slices or exceed a runtime-imposed agent cap. At most 3–4 concurrent
dotnet/npm builds are fine on this Mac (M4, 10 cores, 16 GB); the go/no-go signal is FREE
RAM, not CPU load (`memory_pressure -Q`: <20% free → don't launch more build agents; <10% →
pause/stagger). CPU load spikes during builds are benign. Never 5+ builds (the historic
crash was RAM starvation from an 8–9-way fan-out). NEVER set ETV_UPDATE_GOLDENS.
- Never two committing agents on ONE worktree — give each parallel slice its own worktree
branched off the feature branch and merge back. Sequence only where a slice genuinely depends
on another's output (backend-first narrow, SPA-wide after worked well for #216).
- **BATCH YOUR PUSHES — you cannot cancel a CI run** (operator-requested, 2026-07-17). Hold review
fixes, doc corrections and format fixes locally, verify, and push **once**; each push spawns a run
that orphans the previous one on a 4-slot runner. Cancellation is impossible from the agent side on
this Gitea (**1.25.4**): `POST /api/v1/.../actions/runs/{id}/cancel` → **404**, MCP
`actions_run_write` `cancel_run` → **404**, and the web-UI route needs a session+CSRF that does not
script (login 303s, no session cookie). **Only the operator can cancel, in the browser** — so if you
must supersede a live run, SAY SO explicitly instead of leaving it burning. (`dispatch_workflow` is
a different route and still works for re-triggering a **main** run.) Corrects the older "superseded
runs drain on their own" note below: they do finish, but they hold a slot while doing it.
- **A lone `decisions lifecycle` red is a KNOWN INFRA FLAKE — do NOTHING** (operator-stated,
2026-07-19). When it is the **only** red: do not investigate it, and do **not** create a new run or
commit to clear it — no rebase, no `--amend`, no no-op push. **The operator reruns that single job
from the Gitea UI.** Report it as a known flake and carry on; only if *other* jobs are red too does
the run deserve diagnosis. Same family as the killed-job rule below (a spurious single-job red is
cleared by the operator's selective rerun, never by pushing — and pushes can't be cancelled, see the
batching constraint above). **The trap is that a convincing local explanation is always available.**
On #473/PR #479 the job went red just after `main` landed its own `decisions.md` entry, so "mine is
no longer at EOF, I must rebase" looked airtight — I rebased, and it went red **again** on a head
whose diff was a verified pure EOF append with zero deleted lines. A rebase that provably satisfied
the gate's stated rule did not turn it green, which is the proof the red was never about the diff.
**Check this rule before theorising about the cause.**
- **Do NOT gate or throttle your push on host health — trust the Gitea build queue.** The runners were
retuned for stability (operator, 2026-07-17); queueing is the queue's job, not yours. Don't SSH to
bumblebee to sample load/RAM before pushing, and don't hand-schedule around other sessions' runs. Batch
your pushes (above) because orphaned runs can't be cancelled — not because the host needs protecting.
- **An infra-shaped red under host pressure is not a code failure — check the host before diagnosing the
diff** (2026-07-17). While bumblebee was saturated, two unrelated jobs died in ways that look like code
bugs: `EF migration integrity` failed in ~11ms inside **`Setup .NET`** with
`remove /root/.cache/act/…/.gitignore: no such file or directory` (act's shared cache), and main's
`Build & push image` died on buildx `DeadlineExceeded: failed to compute cache key` after ~210s of
retries. Both re-ran green once the host recovered; the migration job had no model change to test in the
first place. Tell: the failure is in a *setup/cache* step, before your code compiles. Don't file a CI bug
off one sample under pressure — you'll send the next session chasing a phantom (this nearly got filed as
#390 lane-rebalance fallout, which the evidence did not support).
- **Before any push touching `.cs`, check the touched set for a UTF-8 BOM.** The #311 gate is
fix-as-you-touch: a legacy file's inherited BOM becomes *your* problem the moment you touch it, and
`charset=utf-8` means no BOM.
`for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; done`
Strip any hit (1-line diff, zero content change). Bit **two sessions on 2026-07-17** — PR #405 (6
files) and #70/PR #402 (19 files, where Python `io.open(..., encoding='utf-8-sig')` wrote BOMs back
while "preserving existing style"). **Verify your detector**: an `od -An -c | grep '357 273 277'`
check reported all-clean while 19 files were dirty — a detector that can only say "ok" is worse than
none. Use the `xxd` form or read bytes directly.
**`dotnet format --include` DOES work here** (corrected 2026-07-17, #70/PR #402 — an earlier note
claiming it silently no-ops was WRONG). The apparent no-op was **the shell**: CI's recipe uses
`mapfile`, which is **bash-only**, and the default shell here is **zsh** → empty array → zero files
→ exit 0. Run it under `bash -c`:
```bash
bash -c 'mapfile -t files < <(git diff --name-only --diff-filter=ACM origin/main...HEAD -- "*.cs")
dotnet format ErsatzTV.sln --no-restore --verify-no-changes --include "${files[@]}"' # exit 0 = gate passes
```
- Merge consent is DERIVED, not asked: the H6/H10 gate (`pretooluse-merge-consent.sh`) auto-grants a
merge when the linked issue's `## Done-when` boxes are all ticked, a fresh positive `Review-verdict`
references the current head, and CI is green — no conversational confirmation on that satisfied path
(ersatztv#314). Confirm in-conversation only when the gate ASKS (state not derivable: no creds, no
linked issue, no Done-when/verdict, Gitea down). Prior pre-approvals still do NOT carry over.
- **Local gate + cold review run BEFORE the push to Gitea CI, not after** (Timothy, 2026-07-17):
run the local build/test gate AND a cold-context adversarial review agent (SCOPED "review only")
over the diff, fold any fixes, and only THEN push / open the PR. Gitea runners should only ever
see already-validated work — CI is a scarce shared slow resource and a run cannot be cancelled on
this Gitea, so spending a run to surface a lint error or a review-caught bug is pure waste + queue
pressure. CI (armed at PR-open, below) *confirms*; it is not the first line of defence.
- Arm a CI monitor on the PR head sha AT PR-OPEN (commit-status endpoint), not at the end.
- Live-E2E via scripts/e2e-local.sh (also before the push for write-path / UI changes); NEVER
exercise download endpoints via browser tabs (curl them).
- **Work in a worktree off `origin/main` — always**, including a one-line docs commit. Never commit in
`/Users/timothy/ersatztv` and never read its git state to learn about `main`. → `process.shared-tree-readonly`
- **One committing agent per worktree.** Give each parallel slice its own; read-only review agents may
share. → `process.one-worktree-one-committing-agent`
- **Never commit or merge inside a worktree you did not create** — land on a sibling-owned branch with a
plumbing merge. → `process.foreign-worktree-plumbing-merge`
- **Parallelize by default** on genuinely disjoint slices (normally 2–3 delegated agents). Cap builds at
3–4 and gate on FREE RAM, not CPU. **Never set `ETV_UPDATE_GOLDENS`.** → `process.build-concurrency-limits`
- **Local gate + cold-context review BEFORE the push**, never after. → `process.local-gate-before-push`
- **Independent review is mandatory** for locks/concurrency, auth/security, API write-path handlers, DB
migrations, or >~150 changed C# lines; a skip must be stated with its reason. → `process.independent-review-rubric`
- **Batch your pushes — you cannot cancel a CI run.** Only the operator can cancel, in the browser.
→ `ci.batch-pushes-no-cancel-route`
- **Do not gate your push on host health** — trust the build queue. → `ci.no-host-health-gating`
- **Arm the CI monitor at PR-open**, on the head sha. → `ci.monitor-armed-at-pr-open`
- **Before any push touching `.cs`, BOM-check the touched set** and run the format gate under `bash -c`.
→ `process.bom-format-detection-recipe`
- **Keep a PR branch current by REBASING on `origin/main`** — never merge main in; regenerate (never
hand-resolve) generated artifacts after a conflicted rebase. → `release.format-as-you-touch-rebase`
- **Live-E2E via `scripts/e2e-local.sh` before the push** for write-path/UI changes; curl download
endpoints, never browser tabs; use a fresh config dir.
→ `testing.live-e2e-prepush-timing`, `testing.e2e-local-fresh-config-dir`
- **Merge consent is DERIVED, not asked.** The H6/H10 gate auto-grants when the `## Done-when` boxes are
ticked, a fresh `Review-verdict` references the current head, and CI is green; confirm in conversation
only when the gate ASKS. Prior pre-approvals never carry over. → `release.merge-consent-autogrant`
- **After the push, fixes are follow-up commits** — never amend or force-push a pushed branch.
→ `process.pr-routine-sequence`
---
# Lessons / workflow lore (STANDING workflow/orchestration rules only)
> **Scope**: this section is *how we work* (orchestration, CI, review, cross-session hygiene) —
> not engineering decisions. **Every "why the code does X" decision lives in `docs/decisions.md`**
> (+ `api-conventions.md` §7a/b/c for the concurrency/side-effect mechanics, `spa-conventions.md`,
> `e2e-local.md`). Do NOT re-record a code/API/SPA decision here — it will duplicate and drift.
> Pruned 2026-07-12: issue-specific engineering bullets now covered in those docs, and dead
> Blazor-parity process notes, were removed (git history has them). Pruned 2026-07-21: pre-script
> selector lore and the #237-specific queue-drift narrative moved to the archived section below —
> the mechanism they describe no longer exists.
> **Scope**: *how we work* — orchestration, CI triage, review routing, cross-session hygiene. Each rule
> below is one or two lines plus its decision `key:`; the evidence, the incident that produced it and the
> full rationale live in that record (`docs/decisions/workflow-process.md` for most of them). **Keep it
> that way** — this file is pasted into every session's kickoff, so a paragraph of narrative here is a tax
> paid by every future session, while a record is retrieved only when it is needed (#542).
>
> Editing: prune covered and stale bullets rather than appending — this is not append-only and git keeps
> the history. If you add a rule, write the record first and cite it here.
- READ docs/README.md → the convention docs FIRST; point recon/implementer agents at specific
doc sections. Only recon the task-specific delta.
- **The mechanical selection checks live in `scripts/select-queue.sh` — run it instead of
re-deriving them by hand.** It does the deterministic work in code (dependency exclusion via
`GET /dependencies`, LOCAL `.milestone.state`/`review`/`priority:` tiering, (tier, priority, issue#)
ordering, `in-progress`/`parked`/PR exclusion) and only *flags* `CLAIM?`/`UMBRELLA?` for judgment.
**Trust its DEPS/tiering/ordering; do the focused recheck only on the flagged rows.** If you extend
the tiers/filters, update the script first, then these bullets. (See the archived section below for
the pre-script failure modes that motivated writing it, if you need the history.)
- **Codex cheap-worker launch (tested 2026-07-14)**: the native `spawn_agent` interface currently
has no model/effort selector, so it provides parallelism but not cost savings. For bounded,
tool-bearing selector/recon work, launch a separate worker with
`codex exec --ephemeral --color never -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only
-C <repo> "<hard-capped scope contract>"`. GPT-5.4-Mini is the current supported small/cost-efficient
profile on this ChatGPT-authenticated host; the older `gpt-5.1-codex-mini` guess fails with HTTP 400.
Keep `low` whenever any shell/MCP/web tool is exposed; reserve `minimal` for explicitly tool-free
synthesis over already-supplied bounded evidence. If the model rolls over, inspect the visible entries
in `~/.codex/models_cache.json` instead of guessing names, then update this dated note once verified.
- Keep the MAIN checkout's `web/node_modules` fresh (`npm install` after pulling a PR that
adds a dep) — worktrees copy it, and a stale copy broke typecheck in a #198 worktree.
- **Playwright-MCP E2E: never open tabs/window.open for file-download endpoints** — curl them.
If browser tools stall repeatedly, `pkill -f ms-playwright-mcp` and drive a fresh session.
- **THERE IS NO "main checkout". `/Users/timothy/ersatztv` is a shared mutable working tree whose HEAD
is whatever the last session left there — it is NOT a `main` mirror, and its name lies.** Treat it as
**read-only, and never as a source of truth**:
- **Never commit in it** — not even a one-line docs fix. Need a commit on `main`? Make a worktree off
`origin/main` (`git worktree add -b <branch> <path> origin/main`) and commit there.
- **Never read `git log` / `git status` / `HEAD` there and infer anything about `main`** — you will be
reading some other session's branch.
- **`origin/main` is the only source of truth.** `git fetch origin main` and reference `origin/main`
explicitly. Do not trust the local `main` ref either (it lags; nobody is obliged to update it).
- Its **one sanctioned use is read-only**: copying `web/node_modules` into your worktree (keep it
fresh — a stale copy broke typecheck in a #198 worktree).
**Orientation**
- Read `docs/README.md` → the convention docs first; point agents at named sections, recon only the
task-specific delta. → `docs.convention-docs-session-start`
- Run `scripts/select-queue.sh` for queue selection; trust its deps/tiering/ordering and resolve only its
`CLAIM?`/`UMBRELLA?` flags. → `startup.parallel-orientation`
- Claim with `in-progress` before working — but a claim prevents duplicate *pickup*, not overlapping code
changes. → `process.parallel-session-claim`
**This is a design flaw, not a discipline failure — do not "fix" it by telling sessions to check
harder.** A check does not stay true. Worktree discipline is healthy (10 feature worktrees on
2026-07-17: Claude under `.claude/worktrees/`, Codex under `.worktrees/`, some in `/private/tmp/`);
the shared tree is the single place the model breaks, and it broke for **two different sessions on the
same day, both doing the obvious thing**:
- Session A assumed it was on `main` (true at 12:46 when it pulled — **correct when made**, stale by
14:17) and committed → the commit landed on the #604/CI session's `ci/604-lane-rebalance`, **24
seconds after that session's own commit at 14:17:28**. Two writers, one tree. Nothing tangled, by luck.
- Session B ran `git log` there, read `ci/604-lane-rebalance`'s HEAD, and concluded `main` was "4
behind origin" — a phantom.
**Building and reviewing**
- The PR routine is a fixed sequence; for API changes build the app project FIRST, then
`./scripts/update-openapi.sh`, then `npm run generate:api`. → `process.pr-routine-sequence`
- Re-review the FIX COMMIT, not just the initial diff; loop until a clean verdict or an explicit
acceptable-defer, and file a tracked issue for anything deferred. → `release.review-verdict-gate`
- When independent reviews disagree, escalate to the frontier judge — and escalate the proposed FIX with
the finding. → `process.review-disagreement-frontier-judge`
- "Make X consistent with Y" findings are new code; for any timer/effect ask when it fires, including on
mount. → `process.consistency-fix-new-code-scrutiny`
- Before trusting a "single owner / no cross-release" claim, grep the whole host project for every
producer — the background scheduler is the usual miss. → `process.lock-ownership-enumerate-producers`
- Before deleting a "workaround", enumerate every behavior it provided.
→ `process.enumerate-workaround-behaviors-before-deleting`
- Concurrency tests need a stress loop and a negative control that inverts the real condition — never
`if (true)`. → `ffmpeg.work-ahead-slot-atomic`
- Enumerate lazy LanguageExt returns in tests; live-E2E is the only net for that class.
→ `release.live-e2e-required`
- Harden a soon-to-grow feature with a runtime posture, not a hardcoded clamp.
→ `process.harden-with-runtime-posture-not-clamp`
A session-start `git status` *appears to confirm* the false assumption, which makes it worse than a
plainly wrong claim. (Supersedes the old "the user's main checkout stays ON main and pulled each
session" note.) Same family as the sibling bullet below: never commit inside a checkout or worktree
you didn't create.
**Reading a red CI run** (check these before diagnosing your diff)
- A **killed** job reports `conclusion: failure` — read the log tail for the `❌ Failure - Main` marker;
log timestamps are UTC, the host is UTC+2. → `ci.killed-job-triage`
- **`cancelled` is not a verdict** — report FAILED and CANCELLED separately. → `ci.cancelled-is-not-a-verdict`
- A failure inside a **setup/cache step**, before your code compiles, is environmental.
→ `ci.infra-shaped-red-under-load`
- A lone **`decisions lifecycle`** red is a known flake: do **nothing**, the operator reruns it. A
convincing local explanation is always available — check this rule before theorising.
→ `ci.decisions-lifecycle-flake`
- Heavy-render web tests need explicit per-test vitest timeouts. → `ci.web-test-per-test-timeouts`
- Verify locally and trust it; CI confirms (~9 min warm, seconds for docs-only).
→ `ci.verify-locally-ci-confirms`
**Also treat any FILE read out of that tree as stale** — including a kickoff prompt pasted from it.
The guards above only fire when you run `git` there; a stale checkout serves stale docs silently.
Session end runs `scripts/refresh-shared-checkout.sh` to keep it current (see
`session.shared-checkout-refresh` in decisions.md).
- Subagent connection drops = laptop sleep/transient; re-resume via SendMessage (work survives).
- Build/verify (and review) locally, then trust it; CI (VM 127) confirms (~9 min warm; seconds for
a docs-only PR since ersatztv#416). Watch by commit
status: `/api/v1/repos/timothy/ersatztv/commits/{sha}/status`.
- PR routine (works): worktree off origin/main → implement (strongest tier for judgment-heavy /
balanced tier for mechanical work when model routing is available; doc pointers + exact facts;
use the available C# language tooling) → API changes: build app project
FIRST then `./scripts/update-openapi.sh` then `npm run generate:api` → **full local test pass +
cold-context adversarial review (SCOPED "review only") + live-E2E for UI/write-path changes
(`scripts/e2e-local.sh`), ALL BEFORE the push** — fold pre-push review fixes into the branch
(amend/reword is fine while nothing is pushed) → push, PR (arm the CI monitor at open) → CI
*confirms* → any further review fixes (a post-push cross-model pass, or CI-surfaced issues) as
follow-up COMMITS (never amend/force-push a pushed branch) → CI green → merge (derived consent
gate) → structured close comments per the applicable `AGENTS.md` / `CLAUDE.md` protocol.
- Never run TWO committing agents on one worktree concurrently. Read-only review agents are fine.
- **A KILLED job reports `conclusion: failure` — the conclusion field is not enough. Read the log tail.**
(2026-07-17.) When the operator restarts the runners (a retune, a reboot), in-flight jobs die and Gitea
marks them **`failure`**, not `cancelled`. Tell: the log **stops mid-step with no error and no
`❌ Failure - Main …` marker** — a real failure always leaves a marker. Run 1006's `EF migration
integrity` + `Functional E2E` "failed" on a **BOM-removal-only** diff (6 files, 1 line each, zero
content change) that could not possibly break them; the logs simply truncated mid-`dotnet build` at
12:08 **UTC**, and both runners showed `Up About an hour` — the retune had killed them. **Log
timestamps are UTC; the host is UTC+2** — convert before correlating, or the restart looks 2h off and
the theory gets wrongly discarded.
Corollary: **a diff that cannot cause a failure is evidence the failure isn't yours** — when a job that
passed on the previous head fails on a semantically-null delta, suspect the environment and go read the
log, don't re-litigate the diff. Also beware log **warnings that look fatal**: `Unable to pull
refs/heads/v4: …` is act refreshing its `/root/.cache/act` action cache and is followed by `Cloned …`
— it is NOISE, not a cause. Grep for the failure marker, not for the word "error".
- **`cancelled` is NOT `failure` — never read a cancel as a CI verdict** (2026-07-17, operator-surfaced).
The operator cancels runs by hand (they're the only one who can — see the batching constraint above), and a
run-level `conclusion: cancelled` means **no verdict**, not a pass and not a fail. Two traps: (1) a
run whose overall state is `failure` may hold a *genuine* job failure that happened **before** the
cancel — check job-level `conclusion` + timestamps, don't attribute it to the cancel; (2) a
`cancelled` run tells you nothing about your diff, so don't claim green on it. Report FAILED and
CANCELLED counts **separately** in any CI monitor:
`fail=[j for j in jobs if j['conclusion']=='failure']; canc=[j for j in jobs if j['conclusion']=='cancelled']`
— a monitor that only asks "is state != pending" will call a cancelled run a failure and send you
debugging a phantom.
- **An infra-shaped red under host load is not a code failure — check `uptime` on bumblebee BEFORE
diagnosing the diff** (2026-07-17). At load average **243**, two unrelated jobs died in ways that
look like code bugs but aren't: `EF migration integrity` failed in ~11ms inside **`Setup .NET`**
with `remove /root/.cache/act/…/.gitignore: no such file or directory` (act's shared cache), and
main's `Build & push image` died on buildx `DeadlineExceeded: failed to compute cache key` after
~210s of retries. Both re-ran green at normal load; the migration job had no model change to test in
the first place. Tell: the failure is in a *setup/cache* step, before your code compiles. Don't file
a CI bug off one sample under load — you'll send the next session chasing a phantom (this nearly got
filed as #390 lane-rebalance fallout, which the evidence did not support). A game server
(`FactoryServer-L`) shares that host with the runners, so load isn't always CI's doing.
- **Arm a CI monitor the moment the PR opens** (commit-status endpoint), not at the end — in the
#216 session three PR runs sat red ~an hour on a CI-only flake while review/E2E ran. (The old
MySQL host-port 3306 collision is fixed on main, `ef8915f1`; a lone MySQL-apply red now = the
known infra flake, rerun — see MEMORY / `ci-migration-job-mysql-flake`.)
- **"Make X consistent with Y" review findings deserve new-code scrutiny** — a #216 review fix
propagated MediaBrowseScreen's mount-debounce selection-wipe race to SearchScreen instead of
catching it. For any timer/effect, ask "when does this fire?" (incl. mount).
- Live-E2E seeding recipe now lives in `docs/e2e-local.md` → "Seeding a local TV library for E2E"
(tiny ffmpeg testsrc MKVs + `LibraryPath` SQL rows + scan). Live-E2E is a **stated requirement**
for write-path handler changes — see that doc's "When live-E2E is required" + the decisions.md
entry; it's the only net for the lazy-enumeration / reload-through-read-path 500 class (#229).
- **`scripts/e2e-local.sh` hangs its readiness probe on a REUSED config dir** (surfaced in the #502
session; recovered from #237's comment log by the #524 triage). The probe waits for
`Done migrating search index` (`e2e-local.sh` `READY_LINE`), which a **fresh** config emits — a
reused one logs `Search index is already version 50` instead and the line never comes, so the
script times out and kills a perfectly healthy server. `docs/e2e-local.md` documents the ready line
and recommends a fresh dir, but not this failure mode. **Use a fresh config dir** — the script
greps one fixed `READY_LINE`, so there is no "wait for either line" option today (widening it to
`Done migrating search index|Search index is already version` is tracked as #533). Cost that
session two failed launches before the cause was obvious.
- **The troubleshooting-playback API cannot exercise channel branding** (same origin).
`PrepareTroubleshootingPlaybackHandler` builds a synthetic channel with `Artwork = []`, and its
`ChannelId > 0` branch ignores the media item — so logo/watermark/bug behavior is **not** testable
through the troubleshoot path. Verify branding changes through a real channel playout instead;
a green troubleshoot run proves nothing about the bug overlay.
- **Parallel sessions**: claim before working (`in-progress` label — the tiny read→claim race window
is accepted; later claimant backs off). Claiming prevents duplicate pickup, NOT overlapping code
changes — check the issue's dependency notes ("#234 after #231", "coordinate with #215") before
touching shared surfaces. Editing THIS lore: prune covered/stale bullets too (not append-only — git
keeps history), `git pull --rebase` before commit. Two runners (ci-runner VM 127 + bumblebee-runner,
4 slots); **no cancel route exists on this Gitea version — verified 1.25.4: REST and MCP
`cancel_run` both 404** (see the batching constraint above). Superseded runs do finish on their own,
but they hold a runner slot while doing it, so with 4 parallel sessions they are a real cost — batch
pushes rather than relying on them to drain.
- **Two sessions touching one machine**: a branch may be checked out in ANOTHER session's
worktree — never commit/merge inside a worktree you didn't create. To land a merge on such a
branch without touching their checkout: plumbing merge (`git read-tree -m base ours theirs`
into a temp GIT_INDEX_FILE → `write-tree` → `commit-tree -p ours -p theirs` → push the
commit to the branch ref); the owning session then `git pull`s.
- **Cross-model independent review rotation** (2026-07-11): use the other client/model family when
it is available — Codex reviews Claude-authored changes; Claude reviews Codex-authored changes.
Use whatever native integration is actually installed; client-specific helpers such as
`codex:rescue` are optional accelerators, not protocol dependencies. On quota/tool exhaustion,
use a cold-context review-only agent in the current client and retry cross-model review next window.
Independent reviewers do review and stuck-diagnosis work, not implementation on the same change.
**Skip rubric (self-exemption is an auditable claim, not a vibe)**: an *independent* review pass
(preferably cross-model; otherwise a cold agent) is **mandatory** for any diff touching
locks/concurrency, auth/security, API write-path handlers, or DB migrations, or larger than
~150 changed C# lines. It is **skippable** only for a pure-SPA/docs leaf with no server-state
effect — and when you skip it, **say so and why** in the PR or close comment (a stated exemption,
not silence). This is the correlated-blindspot net: independent model families miss *different* classes
(see the #235 "enumerate every producer" and #250 lessons), so a single reviewer on a risky diff
is the gap.
- **Force enumeration of lazy LanguageExt returns in tests** (2026-07-11, PR #229): a write-path 500
survived a green fixed-point test because the handler returned a lazy `Map` the test never enumerated.
Enumerate handler return values in tests; live E2E remains the only net for this class. (Write-path
reload-through-read-path mechanics: api-conventions §7.)
- **Keep a PR branch current by REBASING on origin/main — do NOT merge main in** (ersatztv#311,
enforced by the H11 pre-push hook: a push from a branch behind origin/main is blocked → `git fetch
origin main && git rebase origin/main`). A *merge* commit pulls in every file main changed, incl.
files you never touched (e.g. the ~2500 legacy-BOM `.cs`), which then trip the format hook/CI on code
that isn't yours (the #309 session). Rebasing keeps your diff to exactly what you changed. After a
rebase that hits conflicts in **generated artifacts** (v1.json, v1.d.ts, endpoint-index), REGENERATE
them (`./scripts/update-openapi.sh` + `npm run generate:api`) — never hand-resolve; git text-merges
them plausibly-but-wrong and `npm run check:api` is the guard. Escape hatch for a deliberate
non-rebased push: `ETV_SKIP_REBASE_CHECK=1 git push`.
- **H12 issue-qualification audit** (ersatztv#312, `scripts/issue-qualification-audit.sh`): a
session-end check that lists OPEN issues missing a `priority:` label — the queue tiering keys off
`priority:`/gate labels, so an unlabeled issue is invisible to it. "Fully qualified" = has a
`priority: {high,medium,low}` label (that signals triage ran; gate-vs-backlog is then derivable
from the `review` label / milestone, and a milestone is NOT required — backlog is unmilestoned).
Run it at session end and label anything it flags (esp. issues you filed this session). Fail-open
without creds; advisory (exit 1 when any are unqualified). Sibling to H11 (both #311/#312).
- **CI VM test timeouts**: heavy-render web tests (100+ item grids) need explicit vitest
timeouts (e.g. 15s) — the CI VM hit the 5s default on a test that runs in ~1s locally
(run 686). Bump per-test, don't raise the global default.
- **Concurrency tests need a negative control + a stress loop, not a single round** (2026-07-11,
#231): a one-shot `Barrier(N)` + `Task.WhenAll` "exactly one winner" test did NOT catch a
deliberately non-atomic check-then-set flag on this Mac — the load→store window is too narrow to
collide when threads release once. It green-lit broken code. Fix: hammer the race over many rounds
(N worker threads × ~20k rounds, two Barriers per round: acquire → validate one-winner → reset),
and ALWAYS prove non-vacuous by temporarily breaking the primitive and watching the test fail.
Cheap (sub-second) and deterministic-enough. Same discipline as #12's negative control.
- **Before deleting a "workaround", check it isn't serving a SECOND purpose** (2026-07-11, #232):
an issue said "remove the SPA `PENDING_GRACE_TICKS` grace-tick workaround now the API is honest".
The grace ticks did double duty — compensating the lying-200 (now genuinely obsolete) AND
absorbing the inherent queue→observed-active lag / fast-completion race (still needed). Removing
wholesale wedged the scan button disabled-until-reload for short scans; both fork + Codex caught
it. Keep a bounded version, re-scoped honestly. When an issue says "delete X", enumerate every
behavior X provided before deleting.
- **Re-review the FIX COMMIT, not just the initial PR diff, before closing** (2026-07-11, #242,
user-surfaced): the pre-fix review is only half the loop — the fix commit is new code no
independent reviewer has seen. Here the first fix (gate Edit on `saving`) only PARTIALLY closed
the in-flight-save clobber class; a Codex re-review of the fix commit caught that **Delete** was
the one schedule-switch path (`applySwitch`) still missing the guard. Fold a scoped re-review
(cross-model if quota, else a cold review agent) over the fix commit into the close protocol; loop until a clean
MERGEABLE verdict or an explicit acceptable-defer (file a tracked issue for the deferral, as #248
here). Cheap relative to shipping a half-fix that reopens the issue.
- **Negative-controlling a lock/flag fix: invert the condition, don't stub `if (true)`** (2026-07-11,
#250): to prove a lock-release test is non-vacuous I temporarily broke the fix. First attempt made
the guard `if (true)` — but that left the ownership flag "assigned but never read" → **CS0219 under
warnings-as-errors → the Application build silently FAILED → `dotnet test --no-build` ran the STALE
(fixed) dll → false PASS** that looked like the control had failed to catch the bug. The tell: build
output must show `Build succeeded` / `0 Error(s)`, not just a truncated tail. Break the primitive by
**inverting** the real condition (`if (!lockAcquired)`) so the flag stays read and the code compiles;
always grep the build for `error CS` before trusting a `--no-build` test result. (Codex-found High
this session: an outer `catch` that unlocked unconditionally cross-released another session's lock on
a pre-acquisition exception — the fix was a Handle-scoped ownership flag gating the catch.)
- **A lock/channel-ownership "no cross-release" verdict must enumerate EVERY producer of that message
type, not just the enqueuers visible in the PR diff** (2026-07-11, #235/#267). The cold fork cleared the
new `scan-collections` per-provider lock as sound — "only the acquirer enqueues, so only its message
triggers the scanner unlock" — because it checked the three new controllers but never grepped for ALL
`Synchronize{X}Collections(` senders. Codex did: `SchedulerService` also enqueues them *periodically and
unlocked*, and `ScannerService`'s `finally` released the lock whenever held → a scheduled scan
cross-releases an API scan's lock (#250 class). Rule: before trusting any "single owner / no double
release / no cross-release" claim, `grep` the whole host project for every writer of that channel message
(or acquirer of that lock) — the background scheduler/worker is the usual missing producer. This is why the
independent Codex pass earns its keep alongside the fork even when the fork returns MERGEABLE.
- **Independent-review disagreement on a gate PR → frontier judge; judge the fix, not just the verdict**
(2026-07-11, #235): the reconciliation isn't only "who's right" — Fable historically ratified Codex's High AND caught
that the orchestrator's proposed *fix* (per-source lock-or-skip) would deterministically starve the 2nd+
media source (fixed loop order), prescribing the shipped library-scan §C7 pattern instead (lock once per
provider, batch-enqueue, release on the `Unlock: isLast` message). Escalate the FIX approach too, not just
the finding.
- **Harden a soon-to-grow feature with a runtime posture/toggle, not a hardcoded clamp** (2026-07-12, #289):
the MCP's final design (#58) is read+write, so a GET-clamp would be ripped out — shipped a runtime read-only
posture (`ERSATZTV_ALLOW_WRITES`, default false) instead. When a security fix constrains a capability the
roadmap will later want, make the safe state the *default of a switch*, not a wall; check the feature's issue
for its end-state first.
- **Fixing onto an unmerged feature branch: branch off it, land by FF-push, never touch a sibling worktree that
has it checked out** (2026-07-12, #289 → PR#76). Gotcha: recon by `Read`ing a *sibling* worktree leaked its
paths into my later `Edit`s → they landed there, not the fresh worktree (`dotnet` built the unchanged tree →
"0 new tests" tipped it off). After creating a worktree, do the first Edit/Read against ITS absolute paths and
`git status` it before building.
- **`HttpCompletionOption.ResponseHeadersRead` moves the body read OUTSIDE `HttpClient.Timeout`** (2026-07-12,
#289): with a `None` read token a slow-drip upstream hangs forever. Fix = per-request linked
`CancellationTokenSource.CancelAfter(timeout)` into BOTH `SendAsync` and the stream reads +
`HttpClient.Timeout = InfiniteTimeSpan`; catch transport/timeout exceptions and turn them into a response.
(Not yet in decisions.md — #289 landed on PR#76, not main.)
**Tooling gotchas**
- Playwright-MCP: curl download endpoints, never open a tab; `pkill -f ms-playwright-mcp` on repeated
stalls. → `testing.playwright-mcp-download-and-recovery`
- Channel branding is NOT testable through the troubleshooting-playback API.
→ `testing.troubleshoot-path-cannot-test-branding`
- Codex cheap workers launch via `codex exec`, not `spawn_agent`. → `process.codex-cheap-worker-launch`
- A dropped subagent connection is transient — resume via SendMessage, don't restart.
→ `process.subagent-drop-resume`
- Fixing onto an unmerged feature branch: branch off it, FF-push, and beware the sibling-worktree path
leak. → `process.branch-off-feature-branch`
---
## Archived — pre-script selector history (do not follow)
> This section is history, not instruction. It explains why `scripts/select-queue.sh` exists and
> why the closed #237 tracker used to be the queue-state store; none of it is a live protocol. The
> current protocol is "Two concurrent tracks at session start" above.
- **(Pre-2026-07-19) Before the selector script existed**, a cheap model was dispatched each session
to rank the backlog by reading tracker prose directly, and the orchestrator was told to RE-DERIVE
its mechanical claims because the model kept getting them wrong: it reported blocked issues as
`deps:clear` (never called `/dependencies`), mis-tiered issues when the server-side `?milestones=`
filter silently no-ops on `:`/`+` names, ranked by issue number and ignored priority-within-tier,
and missed live `in-progress` claims. `scripts/select-queue.sh` was written to do the mechanical
parts deterministically in code instead — the evidence bullets below are kept for context, not
because you need to re-derive anything by hand anymore.
- **Gitea milestone-name issue filter silently NO-OPS on names with special chars (2026-07-17,
#77 selection)**: `GET /issues?milestones=<name>` returns the WHOLE open-issue list — not a
filtered set — when the milestone title contains `:` or `+` (e.g. `Scheduling: refactor +
distribution`), because the raw name doesn't round-trip through Gitea's filter. A selector that
trusts that response will mis-tier issues (it made #72 look like a milestone-10 member when it's
unmilestoned, and hid #77's true sibling set). Robust recipe (now encoded in the script): fetch
all open issues once and filter LOCALLY on each issue's `.milestone.title`.
- **Confirmed again + widened (2026-07-17, #72/#164 selection): the packet does not reliably see
`blocked-by`, priority-within-tier, `in-progress` claim state, or umbrella-vs-child** — that run's
dependency data was correct, yet it still ranked #383 first while missing that #384 was
`in-progress` (claimed 40 minutes earlier) and that #383 is that bundle's umbrella. The script's
`CLAIM?`/`UMBRELLA?` flags now surface exactly this instead of relying on a full manual re-check.
- **The cheap selector's dependency + priority data was NOT trustworthy — the fix was to make the
script compute them, not to re-derive the whole contested tier by hand every session** (2026-07-17,
#73 selection): the old packet reported #385/#386 as `deps:clear` when both were blocked, and
missed #383 entirely (the only `priority: medium` in tier 1). The script now runs
`GET /issues/{n}/dependencies` on every surviving candidate and tiers by priority-within-tier
deterministically, closing this class.
- **Queue-drift root cause (2026-07-12, user-surfaced, historical)**: pickups repeatedly re-picked
already-done work because DONE/OPEN status was read from **prose** (the old tracker's arc body
inline markers + session comments' "Recommended next"), which lags real issue state — worst under
parallel sessions narrating each other's merges. STANDING FIX (still the rule, just no longer tied
to any specific tracker): **live Gitea state is the ONLY source of truth for status**; a comment's
"Recommended next" is a candidate that MUST be re-verified OPEN before pickup, never propagated
when stale. Do not name today's frontier issue in this standing file — that's how the drift
started the first time.
**Session end**
- Run the H12 audit and label everything it flags. → `process.issue-qualification-audit`
- Run `scripts/refresh-shared-checkout.sh` so the next session's checkout is current.
→ `session.shared-checkout-refresh`
+6 -1
View File
@@ -291,7 +291,12 @@ def main(argv=None) -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--base")
ap.add_argument("--head")
ap.add_argument("--budget", type=int, default=4800) # aggregate; re-baselined post-#521 migration
# Aggregate active-corpus budget. Re-baselined 2026-07-21 (#542): 4800 -> 5600. The corpus grew
# by ~530 lines because 31 workflow/CI/review rules were MOVED into it from
# docs/handoffs/chicorytv-issue-queue.md, which was their only copy. That is the corpus doing its
# job, not drift — the knowledge existed already, it just wasn't retrievable. Do not raise this
# again to accommodate genuinely new records; that is what the warning is for.
ap.add_argument("--budget", type=int, default=5600)
args = ap.parse_args(argv)
records = dl.all_active_records()