Files
ersatztv/docs/handoffs/chicorytv-issue-queue.md
T
timothyandClaude Opus 4.8 1b207db87c
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 13s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 16s
Build ErsatzTV Image / decisions lifecycle (pull_request) Successful in 37s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m22s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m20s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m19s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m22s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
fix(520,521): clear stale migration-map + narrow convention-docs-session-start rule to the task-map protocol [decisions-edit]
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 12:04:35 +02:00

50 KiB
Raw Blame History

ChicoryTV issue-queue handoff (client-neutral kickoff + workflow lore)

Static kickoff prompt + workflow lore, pruned (not appended) as protocols change — git history has the rest. Queue state is live Gitea state, resolved fresh every session by scripts/select-queue.sh; never written here as a snapshot, never read from stale prose or a closed tracker.

ersatztv#237 was the pinned pickup tracker through 2026-07-21; it is CLOSED

and now purely historical (superseded by startup.parallel-orientation in docs/decisions.md; full history in docs/decisions/archive/startup.md). Do not read it for current queue state.


Current phase — maintenance / backlog mode (terminal transition, 2026-07-13)

The ChicoryTV rewrite + go-live arc is COMPLETE and closed out. The project is now in maintenance / backlog mode. This is a terminal fact (the arc will not re-open), so it belongs in this standing file — unlike any "current frontier issue", which must never be written here (read it live; see the queue-drift lesson in the archived section below).

So a fresh session stops re-discovering this every time:

  • The arc / gate / open-review tiers are normally EMPTY now. An empty arc frontier is the expected steady state — not an anomaly to investigate, not a reason to stop, and not a reason to ask the user what to work on. Fall straight through to the priority:-label cascade and claim the top-ranked eligible backlog issue.
  • Selection is priority-label-driven: OPEN-milestoned → priority: highpriority: mediumpriority: low; within a tier, lowest issue number. Exclude in-progress, pull requests, and any issue with an open blocked-by dependency — Gitea tracks these (GET /issues/{n}/dependencies) and auto-clears the block when the blocker closes, so a blocked issue becomes eligible automatically once its blocker lands. Do not pick an issue ahead of its open blocker (e.g. #70 waits on #380).
  • parked → closed, not parked (convention 2026-07-17): parked means "no concrete plans," and the decision was to close such issues rather than let them linger (reopen instantly if revived). So parked should normally not appear; if it does, treat it as excluded and surface it to the user.
  • Bundles (single-session sweeps): related issues are grouped two ways — (a) an active-arc milestone (e.g. "Scheduling: refactor + distribution", "Auto-Tune DetailPanel", "Design system (Claude Design ↔ prod)"), or (b) a cross-reference comment on backlog clusters (e.g. "🔗 Session bundle — security hardening: #293, #376"). When you claim an issue, check its milestone AND its cross-references / backlinks for co-workable siblings, and sweep the whole disjoint cluster in one session. Active-arc milestones auto-sort above the flat priority backlog; leave individually-important backlog bugs unmilestoned so their priority: label still orders them.
  • A new gate re-appears only when a future adversarial-review Blocker/High finding is slotted as a priority pickup, or a new OPEN milestone re-establishes an arc. Until then, backlog features (ordered by milestone/priority: label) are the normal, expected work.

Orchestrator launch profile (operator-facing; do not paste)

As of 2026-07-14:

  • Codex: use GPT-5.6 Sol, ultra for the orchestrator. ultra is the Codex multi-agent orchestration setting; if it is unavailable, use GPT-5.6 Sol, max effort (xhigh only when you deliberately want to trade some capability for quota/latency).
  • Claude Code: use Opus 4.8, xhigh effort (or ultracode, which is the Claude Code orchestration preset built on xhigh). Reserve Fable 5, high effort for the frontier escalations listed below; use xhigh there only when the decision remains genuinely difficult.
  • Queue preflight: when the user has not named an issue, use the client's cheapest suitable fast/small model at low effort. Queue selection is tool-bearing, so do not use minimal: Codex validates the full enabled tool profile, and enabled web_search can reject minimal even when the selector intends to call only Gitea MCP. Route it as a subagent when possible; otherwise run a separate low-cost selector session and pass its compact selection packet to the orchestrator. Never spend the orchestrator tier on queue ranking.
  • If those names are unavailable, choose the client's strongest long-horizon coding/agentic model at its high extended-reasoning setting. Do not weaken the orchestrator to a fast/mini model; use smaller models for bounded delegated slices, with a separate low-cost queue preflight when the client cannot select a cheaper model per agent.

The kickoff below deliberately describes capability tiers instead of model names so it survives model rollovers. The operator-facing recommendations above may be updated without changing the queue protocol.


STANDING KICKOFF PROMPT (paste into a fresh Codex or Claude Code session unchanged)

You are the ORCHESTRATOR in the main Codex or Claude Code session. First identify your client, active model, effort/reasoning level, available delegation mechanisms, and concurrency limit. Orchestration means: decompose, delegate independent slices, integrate their results, verify the whole, and keep canonical issue state accurate. Use the client's native agent/subagent tools; never assume a named tool, command, plugin, model-routing feature, or fork mechanism exists.

Route by capability when the client supports per-agent model selection: fast/small for bounded recon, balanced for mechanical changes, and the strongest coding/agentic tier for judgment-heavy work. Otherwise use the active model for every slice except the mandatory queue preflight below. Independent review MUST start from a cold, review-only brief; prefer a different model family/client when one is available, otherwise use a fresh agent with no implementation role.

Low-cost routing applies throughout the session, not only to queue selection. Before any batch of bounded searches, inventories, log triage, URL/status sampling, or report drafting, dispatch the cheapest suitable fast/small model. If any tool is enabled or required, start at low; this includes enabled web_search even when the prompt names only Gitea MCP. Reserve minimal for explicitly tool-free synthesis over already-supplied bounded evidence. Never retry a known tool-bearing profile at minimal. Strictly cap its allowed files, queries, log window, result/evidence count, and output; instruct it to stop and return partial evidence at a cap instead of broadening scope. The orchestrator receives a compact evidence packet and performs only the minimum focused live recheck. Same-tier agents add parallelism, not cost savings, while direct tool-only mechanical checks may stay inline. If a tool-bearing launch is rejected at minimal, retry once with the same model and unchanged scope at low. If a cheap worker lacks a required tool, give it already-collected bounded evidence for explicitly tool-free synthesis or report the limitation instead of silently expanding orchestrator reconnaissance. Global Codex hook enforcement is tracked in timothy/server-management#592; the wider Claude-hook port is tracked in timothy/server-management#593.

Two concurrent tracks at session start

A fresh session runs two independent tracks — neither blocks the other, and both are cheap/mechanical, never orchestrator-tier work:

  • Track A — Orientation (always, regardless of whether an issue is named): read AGENTS.md and CLAUDE.md when present, then docs/README.md's task-signal map, and the active decisions catalog docs/decisions/README.md. This is what replaces re-deriving conventions from source or from a pinned tracker's prose — see "Knowledge retrieval" below for the full catalog-first / breadcrumb-rule contract.
  • Track B — Selection (only when the user has NOT named an issue): run the deterministic selector, ETV_GITEA_BASICAUTH=user:pass scripts/select-queue.sh [N]. It does the mechanical parts a cheap model used to get wrong IN CODE: excludes in-progress/parked/PRs, resolves GET /issues/{n}/dependencies on every candidate (dropping any with an OPEN blocker), tiers by LOCAL .milestone.state/review/priority: filters (never the server ?milestones= name filter, which no-ops on :/+), and orders by (tier, priority, issue#). Its DEPS/tiering/ordering are deterministic — trust them, do not re-derive them by hand (see the archived section below for why that used to be necessary and isn't anymore). It also raises two JUDGMENT flags it deliberately does not decide — CLAIM? (a claim can precede the label) and UMBRELLA? (an epic whose children are the real pickups) — resolve ONLY those by reading the flagged issue's comments/body. The script does not rank an arc tier (there is no active arc right now — maintenance/backlog mode, see above); if a new arc/milestone is ever established, add it as a script tier rather than reverting to prose-derived ranking. If the script is unavailable (no creds / Gitea down), it fails open (prints a notice, exits 0) — in that case fall back to a single low-cost model session querying live tiers directly (OPEN-milestone → reviewpriority: high/medium/low), never to inline sorting by the orchestrator. If the user names an issue, skip Track B entirely and go straight to focused retrieval (below) plus a live claimability check on that issue; this explicit user choice is the sole path by which a parked issue may be worked.

Dispatch exactly one selector on the cheapest suitable model at low effort when the client can route a cheaper subagent; otherwise run it in a separate low-cost session before starting or resuming the orchestrator and pass in its packet. Do not fall back to inline sorting or an equally expensive selector. If no cheaper route or session is available, pause and request the selector packet rather than consuming orchestrator tokens on queue ranking.

FIRST read AGENTS.md and CLAUDE.md when present, then docs/README.md and the sections its task-signal map points to for your task, plus the Lessons below. Apply both client instruction files; where they differ, follow the stricter safety/completion requirement unless a higher-priority instruction resolves the conflict.

FRONTIER ESCALATION — these moments go to the strongest available reasoning model. In Claude Code, prefer Fable; in Codex, prefer the strongest frontier model at extended-high reasoning. If you are already running that profile, handle them inline. Otherwise delegate with a SELF-CONTAINED brief (scope, exact question, evidence, file paths, and doc pointers), or ask the user to switch models when the client cannot route the escalation. Do not block merely because one brand-specific model is absent:

  • Architecture/API design decisions (e.g. #202's media-source API shape) and anything [PLAN-MODE]-tagged.
  • Parity verdicts for blazor-route-parity.md (per-capability diff sign-off) and #91b removal go/no-go.
  • Reconciling contradictory independent-review findings and the final pre-merge judgment on gate PRs.
  • Stuck after two genuinely different approaches, or evidence contradicts the issue/docs. Everything else (claiming, worktrees, dispatching implementers, CI monitoring, protocol bookkeeping, routine merges of green reviewed PRs with user consent) stays at your level.

Knowledge retrieval (MemPalace + catalog + Gitea — the #642 seam)

MemPalace = candidate discovery only, retrieved at a bounded k; every passage verified against its cited Markdown/Gitea source before use. Default wing = ErsatzTV-Decisions (active docs/decisions.md + docs/decisions/*.md + catalog). History wings ErsatzTV-Decisions-Archive (docs/decisions/archive/**) and Gitea-ErsatzTV (issues/comments incl. closing records) are touched only when the question is explicitly "what did the rule used to be." "What is the current rule for X" never touches the history wings.

The four load-bearing orientation bullets — carry these VERBATIM wherever this contract is referenced (docs/README.md too):

  1. Current conventions/decisions → catalog-first. Start at docs/decisions/README.md; discover via the ErsatzTV-Decisions wing (active) / ErsatzTV-Decisions-Archive (superseded/retired). Resolve by topic/key, never by chasing a file path.
  2. Issue history → evidence, not authority. The Gitea-ErsatzTV wing is historical narrative that may be stale; it never overrides current Markdown.
  3. The breadcrumb rule (the crux behavior change). A file path named inside a historical issue comment (e.g. "grep docs/decisions.md 2026-07-17", "see …") is a breadcrumb, not a live pointer. Find the current rule via the catalog / active wing by concept; do not treat the named path as current. (Why it's safe: still-current → in the active wing, breadcrumb resolves; superseded → the active wing returns the successor and a literal follow lands on a record that announces its own status: superseded; retired → the active wing returns nothing, which is itself the signal. The validator-enforced move-to-archive/ is what prevents the catastrophic "superseded rule read as current" case.)
  4. Fallback when MemPalace is stale/down: docs/decisions/README.md catalog, then rg '^`key: <dotted.key>`' docs/decisions/. MemPalace is never authority nor sole fallback.

Graceful degradation (calibration, not a correctness cliff): even an under-oriented agent that literally greps docs/decisions.md post-#521 gets valid-but-incomplete results (misses topic files

  • archived records), never wrong ones — that file holds only active records. So this is a completeness/latency risk during rollout, not a correctness cliff.

Altitude / precedence: docs/decisions.md (+ topic files) is normative/current ("the rule is X") — the ErsatzTV-Decisions wing, authority for "what is the current rule." Gitea issues are evidentiary/historical ("we worked X on date D; spec, discussion, outcome") — the Gitea-ErsatzTV wing, authority for "how was this handled / what was the context." A decision record points to its issue; the issue carries the provenance the record compresses. Same fact, two altitudes — canonical Markdown wins for "current."

Issues in MemPalace are for DISCOVERY, not queue state. Encounter a problem → search MemPalace → find relevant docs AND relevant open issue(s): this avoids filing duplicates and surfaces bundles (related issues to pick up together). But a MemPalace issue hit is a candidate: open/closed/claimed/blocked/priority flip constantly and MemPalace lags (seconds1h, up to a week if the webhook is down), so a hit may be a since-closed issue shown open, or miss a just-filed one. Always re-confirm live state in Gitea (scripts/select-queue.sh) before acting — never treat a MemPalace issue hit as current queue state.

What to mine per issue: open issue → the BODY (dedup + bundle-clustering; no closing record yet). Closed issue → body + the ## Closing record (how it was resolved). Ephemeral comments (claim / progress / Review-verdict:) → skip or de-weight both ways — process exhaust, not the knowledge store. The ## Closing record is the one structured per-issue summary meant for retrieval + export.

Staleness bounds (so agents know when to distrust a hit): webhook re-mine in seconds; hourly reconcile; weekly full sweep catches file moves/deletes. Worst case before a supersession takes effect in retrieval: ~1h, or ~1 week if the webhook is down and only the move happened. When in doubt, fall back to exact search (bullet 4 above).

Never derive live queue state from MemPalace, #237, or historical comments.

Then work the queue:

  1. Run the two tracks above (orientation always; selection unless an issue is named).
  2. If Track B ran: apply its ranked shortlist. The orchestrator makes one focused live recheck of the winner (still OPEN, unclaimed, not blocked) before claiming; if it changed, check the next fallback. An empty backlog is not a stopping condition — if the selector returns any eligible candidate, claim its top-ranked winner; do not ask the user to choose merely because candidates belong to different workstreams. Never invent a fix-size, recency, or perceived-relevance tiebreaker.
  3. Claim it: add the in-progress label + a "claiming" comment on the issue(s); reviewer-repo audits are claimed by comment only. Treat that claim as live until a later comment explicitly releases or abandons it, and exclude audits with a posted deliverable even while the issue remains open for implementer replies.
  4. Read the issue bodies (they carry the task context/evidence) and work the item under the HARD CONSTRAINTS below.
  5. Finish the session: run the H12 qualification audit (ETV_GITEA_BASICAUTH=user:pass scripts/issue-qualification-audit.sh) and add a priority: label to anything it lists (every issue you filed this session included); post a ## Closing record (template below) on each issue you closed or substantially progressed; remove your in-progress labels; and complete the per-issue Task Completion Protocol from the applicable AGENTS.md / CLAUDE.md instructions (including the done workflow when required).

Closing record (session-end / issue-close artifact)

Post this as the structured closing comment on any issue you close (or substantially progress) — it is both the human-readable summary and the per-issue unit MemPalace mines for retrieval:

## Closing record
**Outcome:** <what shipped / what didn't; PR link>
**Root cause:** <for bug fixes/incidents — why the problem existed, or "unknown, see follow-up #N">
**Decisions/conventions changed:** <keys added/superseded in docs/decisions.md, or "none">
**Reusable knowledge:** <a fact/gotcha worth surfacing to a future session or MemPalace search>
**Verification:** <tests run, live-E2E, CI status>
**Deferred:** <anything explicitly punted, with a follow-up issue link, or "none">
**Docs updated:** <which docs/*.md files changed in this PR, or "none required and why">

HARD CONSTRAINTS:

  • Work in worktrees off origin/main — ALWAYS, including for a one-line docs commit. /Users/timothy/ersatztv is a shared mutable tree, not a main mirror: never commit in it, and never read its HEAD/git log to learn anything about main (you'll get another session's branch). Its only sanctioned use is read-only: copying web/node_modules out of it. origin/main is the source of truth — git fetch origin main and reference it explicitly. See the lore bullet for the two sessions this bit on 2026-07-17.
  • PARALLELIZE BY DEFAULT when there are genuinely disjoint slices: use up to the client's concurrency limit, normally 23 delegated agents plus the orchestrator. Fan out bounded recon early, but do not invent artificial slices or exceed a runtime-imposed agent cap. At most 34 concurrent dotnet/npm builds are fine on this Mac (M4, 10 cores, 16 GB); the go/no-go signal is FREE RAM, not CPU load (memory_pressure -Q: <20% free → don't launch more build agents; <10% → pause/stagger). CPU load spikes during builds are benign. Never 5+ builds (the historic crash was RAM starvation from an 89-way fan-out). NEVER set ETV_UPDATE_GOLDENS.
  • Never two committing agents on ONE worktree — give each parallel slice its own worktree branched off the feature branch and merge back. Sequence only where a slice genuinely depends on another's output (backend-first narrow, SPA-wide after worked well for #216).
  • BATCH YOUR PUSHES — you cannot cancel a CI run (operator-requested, 2026-07-17). Hold review fixes, doc corrections and format fixes locally, verify, and push once; each push spawns a run that orphans the previous one on a 4-slot runner. Cancellation is impossible from the agent side on this Gitea (1.25.4): POST /api/v1/.../actions/runs/{id}/cancel404, MCP actions_run_write cancel_run404, and the web-UI route needs a session+CSRF that does not script (login 303s, no session cookie). Only the operator can cancel, in the browser — so if you must supersede a live run, SAY SO explicitly instead of leaving it burning. (dispatch_workflow is a different route and still works for re-triggering a main run.) Corrects the older "superseded runs drain on their own" note below: they do finish, but they hold a slot while doing it.
  • A lone decisions lifecycle red is a KNOWN INFRA FLAKE — do NOTHING (operator-stated, 2026-07-19). When it is the only red: do not investigate it, and do not create a new run or commit to clear it — no rebase, no --amend, no no-op push. The operator reruns that single job from the Gitea UI. Report it as a known flake and carry on; only if other jobs are red too does the run deserve diagnosis. Same family as the killed-job rule below (a spurious single-job red is cleared by the operator's selective rerun, never by pushing — and pushes can't be cancelled, see the batching constraint above). The trap is that a convincing local explanation is always available. On #473/PR #479 the job went red just after main landed its own decisions.md entry, so "mine is no longer at EOF, I must rebase" looked airtight — I rebased, and it went red again on a head whose diff was a verified pure EOF append with zero deleted lines. A rebase that provably satisfied the gate's stated rule did not turn it green, which is the proof the red was never about the diff. Check this rule before theorising about the cause.
  • Do NOT gate or throttle your push on host health — trust the Gitea build queue. The runners were retuned for stability (operator, 2026-07-17); queueing is the queue's job, not yours. Don't SSH to bumblebee to sample load/RAM before pushing, and don't hand-schedule around other sessions' runs. Batch your pushes (above) because orphaned runs can't be cancelled — not because the host needs protecting.
  • An infra-shaped red under host pressure is not a code failure — check the host before diagnosing the diff (2026-07-17). While bumblebee was saturated, two unrelated jobs died in ways that look like code bugs: EF migration integrity failed in ~11ms inside Setup .NET with remove /root/.cache/act/…/.gitignore: no such file or directory (act's shared cache), and main's Build & push image died on buildx DeadlineExceeded: failed to compute cache key after ~210s of retries. Both re-ran green once the host recovered; the migration job had no model change to test in the first place. Tell: the failure is in a setup/cache step, before your code compiles. Don't file a CI bug off one sample under pressure — you'll send the next session chasing a phantom (this nearly got filed as #390 lane-rebalance fallout, which the evidence did not support).
  • Before any push touching .cs, check the touched set for a UTF-8 BOM. The #311 gate is fix-as-you-touch: a legacy file's inherited BOM becomes your problem the moment you touch it, and charset=utf-8 means no BOM. for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; done Strip any hit (1-line diff, zero content change). Bit two sessions on 2026-07-17 — PR #405 (6 files) and #70/PR #402 (19 files, where Python io.open(..., encoding='utf-8-sig') wrote BOMs back while "preserving existing style"). Verify your detector: an od -An -c | grep '357 273 277' check reported all-clean while 19 files were dirty — a detector that can only say "ok" is worse than none. Use the xxd form or read bytes directly. dotnet format --include DOES work here (corrected 2026-07-17, #70/PR #402 — an earlier note claiming it silently no-ops was WRONG). The apparent no-op was the shell: CI's recipe uses mapfile, which is bash-only, and the default shell here is zsh → empty array → zero files → exit 0. Run it under bash -c:
    bash -c 'mapfile -t files < <(git diff --name-only --diff-filter=ACM origin/main...HEAD -- "*.cs")
    dotnet format ErsatzTV.sln --no-restore --verify-no-changes --include "${files[@]}"'   # exit 0 = gate passes
    
  • Merge consent is DERIVED, not asked: the H6/H10 gate (pretooluse-merge-consent.sh) auto-grants a merge when the linked issue's ## Done-when boxes are all ticked, a fresh positive Review-verdict references the current head, and CI is green — no conversational confirmation on that satisfied path (ersatztv#314). Confirm in-conversation only when the gate ASKS (state not derivable: no creds, no linked issue, no Done-when/verdict, Gitea down). Prior pre-approvals still do NOT carry over.
  • Local gate + cold review run BEFORE the push to Gitea CI, not after (Timothy, 2026-07-17): run the local build/test gate AND a cold-context adversarial review agent (SCOPED "review only") over the diff, fold any fixes, and only THEN push / open the PR. Gitea runners should only ever see already-validated work — CI is a scarce shared slow resource and a run cannot be cancelled on this Gitea, so spending a run to surface a lint error or a review-caught bug is pure waste + queue pressure. CI (armed at PR-open, below) confirms; it is not the first line of defence.
  • Arm a CI monitor on the PR head sha AT PR-OPEN (commit-status endpoint), not at the end.
  • Live-E2E via scripts/e2e-local.sh (also before the push for write-path / UI changes); NEVER exercise download endpoints via browser tabs (curl them).

Lessons / workflow lore (STANDING workflow/orchestration rules only)

Scope: this section is how we work (orchestration, CI, review, cross-session hygiene) — not engineering decisions. Every "why the code does X" decision lives in docs/decisions.md (+ api-conventions.md §7a/b/c for the concurrency/side-effect mechanics, spa-conventions.md, e2e-local.md). Do NOT re-record a code/API/SPA decision here — it will duplicate and drift. Pruned 2026-07-12: issue-specific engineering bullets now covered in those docs, and dead Blazor-parity process notes, were removed (git history has them). Pruned 2026-07-21: pre-script selector lore and the #237-specific queue-drift narrative moved to the archived section below — the mechanism they describe no longer exists.

  • READ docs/README.md → the convention docs FIRST; point recon/implementer agents at specific doc sections. Only recon the task-specific delta.

  • The mechanical selection checks live in scripts/select-queue.sh — run it instead of re-deriving them by hand. It does the deterministic work in code (dependency exclusion via GET /dependencies, LOCAL .milestone.state/review/priority: tiering, (tier, priority, issue#) ordering, in-progress/parked/PR exclusion) and only flags CLAIM?/UMBRELLA? for judgment. Trust its DEPS/tiering/ordering; do the focused recheck only on the flagged rows. If you extend the tiers/filters, update the script first, then these bullets. (See the archived section below for the pre-script failure modes that motivated writing it, if you need the history.)

  • Codex cheap-worker launch (tested 2026-07-14): the native spawn_agent interface currently has no model/effort selector, so it provides parallelism but not cost savings. For bounded, tool-bearing selector/recon work, launch a separate worker with codex exec --ephemeral --color never -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only -C <repo> "<hard-capped scope contract>". GPT-5.4-Mini is the current supported small/cost-efficient profile on this ChatGPT-authenticated host; the older gpt-5.1-codex-mini guess fails with HTTP 400. Keep low whenever any shell/MCP/web tool is exposed; reserve minimal for explicitly tool-free synthesis over already-supplied bounded evidence. If the model rolls over, inspect the visible entries in ~/.codex/models_cache.json instead of guessing names, then update this dated note once verified.

  • Keep the MAIN checkout's web/node_modules fresh (npm install after pulling a PR that adds a dep) — worktrees copy it, and a stale copy broke typecheck in a #198 worktree.

  • Playwright-MCP E2E: never open tabs/window.open for file-download endpoints — curl them. If browser tools stall repeatedly, pkill -f ms-playwright-mcp and drive a fresh session.

  • THERE IS NO "main checkout". /Users/timothy/ersatztv is a shared mutable working tree whose HEAD is whatever the last session left there — it is NOT a main mirror, and its name lies. Treat it as read-only, and never as a source of truth:

    • Never commit in it — not even a one-line docs fix. Need a commit on main? Make a worktree off origin/main (git worktree add -b <branch> <path> origin/main) and commit there.
    • Never read git log / git status / HEAD there and infer anything about main — you will be reading some other session's branch.
    • origin/main is the only source of truth. git fetch origin main and reference origin/main explicitly. Do not trust the local main ref either (it lags; nobody is obliged to update it).
    • Its one sanctioned use is read-only: copying web/node_modules into your worktree (keep it fresh — a stale copy broke typecheck in a #198 worktree).

    This is a design flaw, not a discipline failure — do not "fix" it by telling sessions to check harder. A check does not stay true. Worktree discipline is healthy (10 feature worktrees on 2026-07-17: Claude under .claude/worktrees/, Codex under .worktrees/, some in /private/tmp/); the shared tree is the single place the model breaks, and it broke for two different sessions on the same day, both doing the obvious thing:

    • Session A assumed it was on main (true at 12:46 when it pulled — correct when made, stale by 14:17) and committed → the commit landed on the #604/CI session's ci/604-lane-rebalance, 24 seconds after that session's own commit at 14:17:28. Two writers, one tree. Nothing tangled, by luck.
    • Session B ran git log there, read ci/604-lane-rebalance's HEAD, and concluded main was "4 behind origin" — a phantom.

    A session-start git status appears to confirm the false assumption, which makes it worse than a plainly wrong claim. (Supersedes the old "the user's main checkout stays ON main and pulled each session" note.) Same family as the sibling bullet below: never commit inside a checkout or worktree you didn't create.

  • Subagent connection drops = laptop sleep/transient; re-resume via SendMessage (work survives).

  • Build/verify (and review) locally, then trust it; CI (VM 127) confirms (~9 min warm; seconds for a docs-only PR since ersatztv#416). Watch by commit status: /api/v1/repos/timothy/ersatztv/commits/{sha}/status.

  • PR routine (works): worktree off origin/main → implement (strongest tier for judgment-heavy / balanced tier for mechanical work when model routing is available; doc pointers + exact facts; use the available C# language tooling) → API changes: build app project FIRST then ./scripts/update-openapi.sh then npm run generate:apifull local test pass + cold-context adversarial review (SCOPED "review only") + live-E2E for UI/write-path changes (scripts/e2e-local.sh), ALL BEFORE the push — fold pre-push review fixes into the branch (amend/reword is fine while nothing is pushed) → push, PR (arm the CI monitor at open) → CI confirms → any further review fixes (a post-push cross-model pass, or CI-surfaced issues) as follow-up COMMITS (never amend/force-push a pushed branch) → CI green → merge (derived consent gate) → structured close comments per the applicable AGENTS.md / CLAUDE.md protocol.

  • Never run TWO committing agents on one worktree concurrently. Read-only review agents are fine.

  • A KILLED job reports conclusion: failure — the conclusion field is not enough. Read the log tail. (2026-07-17.) When the operator restarts the runners (a retune, a reboot), in-flight jobs die and Gitea marks them failure, not cancelled. Tell: the log stops mid-step with no error and no ❌ Failure - Main … marker — a real failure always leaves a marker. Run 1006's EF migration integrity + Functional E2E "failed" on a BOM-removal-only diff (6 files, 1 line each, zero content change) that could not possibly break them; the logs simply truncated mid-dotnet build at 12:08 UTC, and both runners showed Up About an hour — the retune had killed them. Log timestamps are UTC; the host is UTC+2 — convert before correlating, or the restart looks 2h off and the theory gets wrongly discarded. Corollary: a diff that cannot cause a failure is evidence the failure isn't yours — when a job that passed on the previous head fails on a semantically-null delta, suspect the environment and go read the log, don't re-litigate the diff. Also beware log warnings that look fatal: Unable to pull refs/heads/v4: … is act refreshing its /root/.cache/act action cache and is followed by Cloned … — it is NOISE, not a cause. Grep for the failure marker, not for the word "error".

  • cancelled is NOT failure — never read a cancel as a CI verdict (2026-07-17, operator-surfaced). The operator cancels runs by hand (they're the only one who can — see the batching constraint above), and a run-level conclusion: cancelled means no verdict, not a pass and not a fail. Two traps: (1) a run whose overall state is failure may hold a genuine job failure that happened before the cancel — check job-level conclusion + timestamps, don't attribute it to the cancel; (2) a cancelled run tells you nothing about your diff, so don't claim green on it. Report FAILED and CANCELLED counts separately in any CI monitor: fail=[j for j in jobs if j['conclusion']=='failure']; canc=[j for j in jobs if j['conclusion']=='cancelled'] — a monitor that only asks "is state != pending" will call a cancelled run a failure and send you debugging a phantom.

  • An infra-shaped red under host load is not a code failure — check uptime on bumblebee BEFORE diagnosing the diff (2026-07-17). At load average 243, two unrelated jobs died in ways that look like code bugs but aren't: EF migration integrity failed in ~11ms inside Setup .NET with remove /root/.cache/act/…/.gitignore: no such file or directory (act's shared cache), and main's Build & push image died on buildx DeadlineExceeded: failed to compute cache key after ~210s of retries. Both re-ran green at normal load; the migration job had no model change to test in the first place. Tell: the failure is in a setup/cache step, before your code compiles. Don't file a CI bug off one sample under load — you'll send the next session chasing a phantom (this nearly got filed as #390 lane-rebalance fallout, which the evidence did not support). A game server (FactoryServer-L) shares that host with the runners, so load isn't always CI's doing.

  • Arm a CI monitor the moment the PR opens (commit-status endpoint), not at the end — in the #216 session three PR runs sat red ~an hour on a CI-only flake while review/E2E ran. (The old MySQL host-port 3306 collision is fixed on main, ef8915f1; a lone MySQL-apply red now = the known infra flake, rerun — see MEMORY / ci-migration-job-mysql-flake.)

  • "Make X consistent with Y" review findings deserve new-code scrutiny — a #216 review fix propagated MediaBrowseScreen's mount-debounce selection-wipe race to SearchScreen instead of catching it. For any timer/effect, ask "when does this fire?" (incl. mount).

  • Live-E2E seeding recipe now lives in docs/e2e-local.md → "Seeding a local TV library for E2E" (tiny ffmpeg testsrc MKVs + LibraryPath SQL rows + scan). Live-E2E is a stated requirement for write-path handler changes — see that doc's "When live-E2E is required" + the decisions.md entry; it's the only net for the lazy-enumeration / reload-through-read-path 500 class (#229).

  • Parallel sessions: claim before working (in-progress label — the tiny read→claim race window is accepted; later claimant backs off). Claiming prevents duplicate pickup, NOT overlapping code changes — check the issue's dependency notes ("#234 after #231", "coordinate with #215") before touching shared surfaces. Editing THIS lore: prune covered/stale bullets too (not append-only — git keeps history), git pull --rebase before commit. Two runners (ci-runner VM 127 + bumblebee-runner, 4 slots); no cancel route exists on this Gitea version — verified 1.25.4: REST and MCP cancel_run both 404 (see the batching constraint above). Superseded runs do finish on their own, but they hold a runner slot while doing it, so with 4 parallel sessions they are a real cost — batch pushes rather than relying on them to drain.

  • Two sessions touching one machine: a branch may be checked out in ANOTHER session's worktree — never commit/merge inside a worktree you didn't create. To land a merge on such a branch without touching their checkout: plumbing merge (git read-tree -m base ours theirs into a temp GIT_INDEX_FILE → write-treecommit-tree -p ours -p theirs → push the commit to the branch ref); the owning session then git pulls.

  • Cross-model independent review rotation (2026-07-11): use the other client/model family when it is available — Codex reviews Claude-authored changes; Claude reviews Codex-authored changes. Use whatever native integration is actually installed; client-specific helpers such as codex:rescue are optional accelerators, not protocol dependencies. On quota/tool exhaustion, use a cold-context review-only agent in the current client and retry cross-model review next window. Independent reviewers do review and stuck-diagnosis work, not implementation on the same change. Skip rubric (self-exemption is an auditable claim, not a vibe): an independent review pass (preferably cross-model; otherwise a cold agent) is mandatory for any diff touching locks/concurrency, auth/security, API write-path handlers, or DB migrations, or larger than ~150 changed C# lines. It is skippable only for a pure-SPA/docs leaf with no server-state effect — and when you skip it, say so and why in the PR or close comment (a stated exemption, not silence). This is the correlated-blindspot net: independent model families miss different classes (see the #235 "enumerate every producer" and #250 lessons), so a single reviewer on a risky diff is the gap.

  • Force enumeration of lazy LanguageExt returns in tests (2026-07-11, PR #229): a write-path 500 survived a green fixed-point test because the handler returned a lazy Map the test never enumerated. Enumerate handler return values in tests; live E2E remains the only net for this class. (Write-path reload-through-read-path mechanics: api-conventions §7.)

  • Keep a PR branch current by REBASING on origin/main — do NOT merge main in (ersatztv#311, enforced by the H11 pre-push hook: a push from a branch behind origin/main is blocked → git fetch origin main && git rebase origin/main). A merge commit pulls in every file main changed, incl. files you never touched (e.g. the ~2500 legacy-BOM .cs), which then trip the format hook/CI on code that isn't yours (the #309 session). Rebasing keeps your diff to exactly what you changed. After a rebase that hits conflicts in generated artifacts (v1.json, v1.d.ts, endpoint-index), REGENERATE them (./scripts/update-openapi.sh + npm run generate:api) — never hand-resolve; git text-merges them plausibly-but-wrong and npm run check:api is the guard. Escape hatch for a deliberate non-rebased push: ETV_SKIP_REBASE_CHECK=1 git push.

  • H12 issue-qualification audit (ersatztv#312, scripts/issue-qualification-audit.sh): a session-end check that lists OPEN issues missing a priority: label — the queue tiering keys off priority:/gate labels, so an unlabeled issue is invisible to it. "Fully qualified" = has a priority: {high,medium,low} label (that signals triage ran; gate-vs-backlog is then derivable from the review label / milestone, and a milestone is NOT required — backlog is unmilestoned). Run it at session end and label anything it flags (esp. issues you filed this session). Fail-open without creds; advisory (exit 1 when any are unqualified). Sibling to H11 (both #311/#312).

  • CI VM test timeouts: heavy-render web tests (100+ item grids) need explicit vitest timeouts (e.g. 15s) — the CI VM hit the 5s default on a test that runs in ~1s locally (run 686). Bump per-test, don't raise the global default.

  • Concurrency tests need a negative control + a stress loop, not a single round (2026-07-11, #231): a one-shot Barrier(N) + Task.WhenAll "exactly one winner" test did NOT catch a deliberately non-atomic check-then-set flag on this Mac — the load→store window is too narrow to collide when threads release once. It green-lit broken code. Fix: hammer the race over many rounds (N worker threads × ~20k rounds, two Barriers per round: acquire → validate one-winner → reset), and ALWAYS prove non-vacuous by temporarily breaking the primitive and watching the test fail. Cheap (sub-second) and deterministic-enough. Same discipline as #12's negative control.

  • Before deleting a "workaround", check it isn't serving a SECOND purpose (2026-07-11, #232): an issue said "remove the SPA PENDING_GRACE_TICKS grace-tick workaround now the API is honest". The grace ticks did double duty — compensating the lying-200 (now genuinely obsolete) AND absorbing the inherent queue→observed-active lag / fast-completion race (still needed). Removing wholesale wedged the scan button disabled-until-reload for short scans; both fork + Codex caught it. Keep a bounded version, re-scoped honestly. When an issue says "delete X", enumerate every behavior X provided before deleting.

  • Re-review the FIX COMMIT, not just the initial PR diff, before closing (2026-07-11, #242, user-surfaced): the pre-fix review is only half the loop — the fix commit is new code no independent reviewer has seen. Here the first fix (gate Edit on saving) only PARTIALLY closed the in-flight-save clobber class; a Codex re-review of the fix commit caught that Delete was the one schedule-switch path (applySwitch) still missing the guard. Fold a scoped re-review (cross-model if quota, else a cold review agent) over the fix commit into the close protocol; loop until a clean MERGEABLE verdict or an explicit acceptable-defer (file a tracked issue for the deferral, as #248 here). Cheap relative to shipping a half-fix that reopens the issue.

  • Negative-controlling a lock/flag fix: invert the condition, don't stub if (true) (2026-07-11, #250): to prove a lock-release test is non-vacuous I temporarily broke the fix. First attempt made the guard if (true) — but that left the ownership flag "assigned but never read" → CS0219 under warnings-as-errors → the Application build silently FAILED → dotnet test --no-build ran the STALE (fixed) dll → false PASS that looked like the control had failed to catch the bug. The tell: build output must show Build succeeded / 0 Error(s), not just a truncated tail. Break the primitive by inverting the real condition (if (!lockAcquired)) so the flag stays read and the code compiles; always grep the build for error CS before trusting a --no-build test result. (Codex-found High this session: an outer catch that unlocked unconditionally cross-released another session's lock on a pre-acquisition exception — the fix was a Handle-scoped ownership flag gating the catch.)

  • A lock/channel-ownership "no cross-release" verdict must enumerate EVERY producer of that message type, not just the enqueuers visible in the PR diff (2026-07-11, #235/#267). The cold fork cleared the new scan-collections per-provider lock as sound — "only the acquirer enqueues, so only its message triggers the scanner unlock" — because it checked the three new controllers but never grepped for ALL Synchronize{X}Collections( senders. Codex did: SchedulerService also enqueues them periodically and unlocked, and ScannerService's finally released the lock whenever held → a scheduled scan cross-releases an API scan's lock (#250 class). Rule: before trusting any "single owner / no double release / no cross-release" claim, grep the whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer. This is why the independent Codex pass earns its keep alongside the fork even when the fork returns MERGEABLE.

  • Independent-review disagreement on a gate PR → frontier judge; judge the fix, not just the verdict (2026-07-11, #235): the reconciliation isn't only "who's right" — Fable historically ratified Codex's High AND caught that the orchestrator's proposed fix (per-source lock-or-skip) would deterministically starve the 2nd+ media source (fixed loop order), prescribing the shipped library-scan §C7 pattern instead (lock once per provider, batch-enqueue, release on the Unlock: isLast message). Escalate the FIX approach too, not just the finding.

  • Harden a soon-to-grow feature with a runtime posture/toggle, not a hardcoded clamp (2026-07-12, #289): the MCP's final design (#58) is read+write, so a GET-clamp would be ripped out — shipped a runtime read-only posture (ERSATZTV_ALLOW_WRITES, default false) instead. When a security fix constrains a capability the roadmap will later want, make the safe state the default of a switch, not a wall; check the feature's issue for its end-state first.

  • Fixing onto an unmerged feature branch: branch off it, land by FF-push, never touch a sibling worktree that has it checked out (2026-07-12, #289 → PR#76). Gotcha: recon by Reading a sibling worktree leaked its paths into my later Edits → they landed there, not the fresh worktree (dotnet built the unchanged tree → "0 new tests" tipped it off). After creating a worktree, do the first Edit/Read against ITS absolute paths and git status it before building.

  • HttpCompletionOption.ResponseHeadersRead moves the body read OUTSIDE HttpClient.Timeout (2026-07-12, #289): with a None read token a slow-drip upstream hangs forever. Fix = per-request linked CancellationTokenSource.CancelAfter(timeout) into BOTH SendAsync and the stream reads + HttpClient.Timeout = InfiniteTimeSpan; catch transport/timeout exceptions and turn them into a response. (Not yet in decisions.md — #289 landed on PR#76, not main.)


Archived — pre-script selector history (do not follow)

This section is history, not instruction. It explains why scripts/select-queue.sh exists and why the closed #237 tracker used to be the queue-state store; none of it is a live protocol. The current protocol is "Two concurrent tracks at session start" above.

  • (Pre-2026-07-19) Before the selector script existed, a cheap model was dispatched each session to rank the backlog by reading tracker prose directly, and the orchestrator was told to RE-DERIVE its mechanical claims because the model kept getting them wrong: it reported blocked issues as deps:clear (never called /dependencies), mis-tiered issues when the server-side ?milestones= filter silently no-ops on :/+ names, ranked by issue number and ignored priority-within-tier, and missed live in-progress claims. scripts/select-queue.sh was written to do the mechanical parts deterministically in code instead — the evidence bullets below are kept for context, not because you need to re-derive anything by hand anymore.
  • Gitea milestone-name issue filter silently NO-OPS on names with special chars (2026-07-17, #77 selection): GET /issues?milestones=<name> returns the WHOLE open-issue list — not a filtered set — when the milestone title contains : or + (e.g. Scheduling: refactor + distribution), because the raw name doesn't round-trip through Gitea's filter. A selector that trusts that response will mis-tier issues (it made #72 look like a milestone-10 member when it's unmilestoned, and hid #77's true sibling set). Robust recipe (now encoded in the script): fetch all open issues once and filter LOCALLY on each issue's .milestone.title.
  • Confirmed again + widened (2026-07-17, #72/#164 selection): the packet does not reliably see blocked-by, priority-within-tier, in-progress claim state, or umbrella-vs-child — that run's dependency data was correct, yet it still ranked #383 first while missing that #384 was in-progress (claimed 40 minutes earlier) and that #383 is that bundle's umbrella. The script's CLAIM?/UMBRELLA? flags now surface exactly this instead of relying on a full manual re-check.
  • The cheap selector's dependency + priority data was NOT trustworthy — the fix was to make the script compute them, not to re-derive the whole contested tier by hand every session (2026-07-17, #73 selection): the old packet reported #385/#386 as deps:clear when both were blocked, and missed #383 entirely (the only priority: medium in tier 1). The script now runs GET /issues/{n}/dependencies on every surviving candidate and tiers by priority-within-tier deterministically, closing this class.
  • Queue-drift root cause (2026-07-12, user-surfaced, historical): pickups repeatedly re-picked already-done work because DONE/OPEN status was read from prose (the old tracker's arc body inline markers + session comments' "Recommended next"), which lags real issue state — worst under parallel sessions narrating each other's merges. STANDING FIX (still the rule, just no longer tied to any specific tracker): live Gitea state is the ONLY source of truth for status; a comment's "Recommended next" is a candidate that MUST be re-verified OPEN before pickup, never propagated when stale. Do not name today's frontier issue in this standing file — that's how the drift started the first time.