Files
ersatztv/docs/handoffs/chicorytv-issue-queue.md
T

52 KiB
Raw Blame History

ChicoryTV issue-queue handoff (client-neutral kickoff + workflow lore)

PROTOCOL CHANGED 2026-07-11 (decisions.md entry of same date). Queue state lives in the pinned Gitea tracker ersatztv#237, not in this file. Do NOT write session state, queue order, or next-session prompts here — sessions ending under the old protocol should append their session comment to #237 instead. This file holds only stable operating guidance: the standing kickoff prompt and the workflow lore. Historical per-session state: git log of this file (last state-bearing revision: 8b77d5e7).


Current phase — maintenance / backlog mode (terminal transition, 2026-07-13)

The ChicoryTV rewrite + go-live arc is COMPLETE: the pinned arc tracker ersatztv#237 is CLOSED and all six numbered arc items shipped. The project is now in maintenance / backlog mode. This is a terminal fact (the arc will not re-open), so it belongs in this standing file — unlike any "current frontier issue", which must never be written here (read it live; see the queue-drift lesson at the end of this file).

So a fresh session stops re-discovering this every time:

  • The arc / gate / open-review tiers are normally EMPTY now. An empty arc frontier is the expected steady state — not an anomaly to investigate, not a reason to stop, and not a reason to ask the user what to work on. Fall straight through to the priority:-label cascade and claim the top-ranked eligible backlog issue.
  • Selection is priority-label-driven: OPEN-milestoned → priority: highpriority: mediumpriority: low; within a tier, lowest issue number. Exclude in-progress, pull requests, and any issue with an open blocked-by dependency — Gitea tracks these (GET /issues/{n}/dependencies) and auto-clears the block when the blocker closes, so a blocked issue becomes eligible automatically once its blocker lands. Do not pick an issue ahead of its open blocker (e.g. #70 waits on #380).
  • parked → closed, not parked (convention 2026-07-17): parked means "no concrete plans," and the decision was to close such issues rather than let them linger (reopen instantly if revived). So parked should normally not appear; if it does, treat it as excluded and surface it to the user.
  • Bundles (single-session sweeps): related issues are grouped two ways — (a) an active-arc milestone (e.g. "Scheduling: refactor + distribution", "Auto-Tune DetailPanel", "Design system (Claude Design ↔ prod)"), or (b) a cross-reference comment on backlog clusters (e.g. "🔗 Session bundle — security hardening: #293, #376"). When you claim an issue, check its milestone AND its cross-references / backlinks for co-workable siblings, and sweep the whole disjoint cluster in one session. Active-arc milestones auto-sort above the flat priority backlog; leave individually-important backlog bugs unmilestoned so their priority: label still orders them.
  • A new gate re-appears only when a future adversarial-review Blocker/High finding is slotted as a priority pickup, or a new OPEN milestone re-establishes an arc. Until then, backlog features (ordered by milestone/priority: label) are the normal, expected work.

Orchestrator launch profile (operator-facing; do not paste)

As of 2026-07-14:

  • Codex: use GPT-5.6 Sol, ultra for the orchestrator. ultra is the Codex multi-agent orchestration setting; if it is unavailable, use GPT-5.6 Sol, max effort (xhigh only when you deliberately want to trade some capability for quota/latency).
  • Claude Code: use Opus 4.8, xhigh effort (or ultracode, which is the Claude Code orchestration preset built on xhigh). Reserve Fable 5, high effort for the frontier escalations listed below; use xhigh there only when the decision remains genuinely difficult.
  • Queue preflight: when the user has not named an issue, use the client's cheapest suitable fast/small model at low effort. Queue selection is tool-bearing, so do not use minimal: Codex validates the full enabled tool profile, and enabled web_search can reject minimal even when the selector intends to call only Gitea MCP. Route it as a subagent when possible; otherwise run a separate low-cost selector session and pass its compact selection packet to the orchestrator. Never spend the orchestrator tier on queue ranking.
  • If those names are unavailable, choose the client's strongest long-horizon coding/agentic model at its high extended-reasoning setting. Do not weaken the orchestrator to a fast/mini model; use smaller models for bounded delegated slices, with a separate low-cost queue preflight when the client cannot select a cheaper model per agent.

The kickoff below deliberately describes capability tiers instead of model names so it survives model rollovers. The operator-facing recommendations above may be updated without changing the queue protocol.


STANDING KICKOFF PROMPT (paste into a fresh Codex or Claude Code session unchanged)

You are the ORCHESTRATOR in the main Codex or Claude Code session. First identify your client, active model, effort/reasoning level, available delegation mechanisms, and concurrency limit. Orchestration means: decompose, delegate independent slices, integrate their results, verify the whole, and keep canonical issue state accurate. Use the client's native agent/subagent tools; never assume a named tool, command, plugin, model-routing feature, or fork mechanism exists.

Route by capability when the client supports per-agent model selection: fast/small for bounded recon, balanced for mechanical changes, and the strongest coding/agentic tier for judgment-heavy work. Otherwise use the active model for every slice except the mandatory queue preflight below. Independent review MUST start from a cold, review-only brief; prefer a different model family/client when one is available, otherwise use a fresh agent with no implementation role.

Low-cost routing applies throughout the session, not only to queue selection. Before any batch of bounded searches, inventories, log triage, URL/status sampling, or report drafting, dispatch the cheapest suitable fast/small model. If any tool is enabled or required, start at low; this includes enabled web_search even when the prompt names only Gitea MCP. Reserve minimal for explicitly tool-free synthesis over already-supplied bounded evidence. Never retry a known tool-bearing profile at minimal. Strictly cap its allowed files, queries, log window, result/evidence count, and output; instruct it to stop and return partial evidence at a cap instead of broadening scope. The orchestrator receives a compact evidence packet and performs only the minimum focused live recheck. Same-tier agents add parallelism, not cost savings, while direct tool-only mechanical checks may stay inline. If a tool-bearing launch is rejected at minimal, retry once with the same model and unchanged scope at low. If a cheap worker lacks a required tool, give it already-collected bounded evidence for explicitly tool-free synthesis or report the limitation instead of silently expanding orchestrator reconnaissance. Global Codex hook enforcement is tracked in timothy/server-management#592; the wider Claude-hook port is tracked in timothy/server-management#593.

When the user has not named an issue, queue selection is mechanical fast/small work, never orchestrator work. FIRST run the deterministic selector — ETV_GITEA_BASICAUTH=user:pass scripts/select-queue.sh [N] — which does the parts a cheap model kept getting wrong IN CODE: it excludes in-progress/parked/PRs, resolves GET /issues/{n}/dependencies on every candidate (dropping any with an OPEN blocker), tiers by LOCAL .milestone.state/review/priority: filters (never the server ?milestones= name filter, which no-ops on :/+), and orders by (tier, priority, issue#). Its DEPS/tiering/ordering are deterministic — trust them, do not re-derive them by hand. It also raises two JUDGMENT flags it deliberately does not decide — CLAIM? (a claim can precede the label) and UMBRELLA? (an epic whose children are the real pickups); the orchestrator (or a cheap model) resolves ONLY those by reading the flagged issue's comments/body. The script does not rank the arc tier (arc order is #237 prose, currently CLOSED) — read #237 if an arc re-opens. If the script is unavailable (no creds / Gitea down), fall back to the cheap-model path below. Dispatch exactly one selector on the cheapest suitable model at low effort. Give it #237's body and last ~6 comments, then have it query live candidates as a cascade in this exact tier order: arc → OPEN issues assigned to OPEN milestones → open review → unmilestoned/unreviewed priority: highpriority: mediumpriority: low. Every candidate row represents an issue (or eligible reviewer audit): milestone records provide tier metadata and are NEVER pickup candidates. An OPEN milestone with zero eligible OPEN issues contributes zero candidates. Deduplicate candidates, apply the live eligibility exclusions at each tier (including automatic exclusion of parked issues and any issue with an open blocked-by dependency), and stop as soon as five eligible issues have been accumulated (or all tiers are exhausted). It returns only that ranked top-five shortlist, with one-line rationales and live-state evidence; a small result packet is the desired behavior, but an empty or undersized higher tier must fall through to the next tier. The orchestrator receives only that compact packet, then performs a focused live recheck of the winner before claiming. If the cheap worker lacks a repository-scoped issue-list tool, it must immediately return the literal result TOOL_LIMITATION; it must not rank issue IDs mentioned in tracker prose, turn milestone records into candidates, or infer that any tier is empty. The orchestrator then performs only the mechanical tier queries (repository + state + milestone/label), passes at most the bounded raw ISSUE rows back to the cheap worker, and leaves all filtering/ranking to that worker. Never substitute an owner-wide/global search or treat that tool limitation as an empty tier. Do not load implementation docs or issue bodies into the selector. If the client cannot route a cheaper subagent, run the selector in a separate low-cost session before starting or resuming the orchestrator and pass in its packet. Do not fall back to inline sorting or an equally expensive selector. If no cheaper route or session is available, pause and request the selector packet rather than consuming orchestrator tokens on queue ranking. If the user names an issue, skip selection and only verify that issue's live claimability; this explicit user choice is the sole path by which a parked issue may be worked.

FIRST read AGENTS.md and CLAUDE.md when present, then docs/README.md, the convention docs it indexes, and the Lessons below. Apply both client instruction files; where they differ, follow the stricter safety/completion requirement unless a higher-priority instruction resolves the conflict.

FRONTIER ESCALATION — these moments go to the strongest available reasoning model. In Claude Code, prefer Fable; in Codex, prefer the strongest frontier model at extended-high reasoning. If you are already running that profile, handle them inline. Otherwise delegate with a SELF-CONTAINED brief (scope, exact question, evidence, file paths, and doc pointers), or ask the user to switch models when the client cannot route the escalation. Do not block merely because one brand-specific model is absent:

  • Architecture/API design decisions (e.g. #202's media-source API shape) and anything [PLAN-MODE]-tagged.
  • Parity verdicts for blazor-route-parity.md (per-capability diff sign-off) and #91b removal go/no-go.
  • Reconciling contradictory independent-review findings and the final pre-merge judgment on gate PRs.
  • Stuck after two genuinely different approaches, or evidence contradicts the issue/docs. Everything else (claiming, worktrees, dispatching implementers, CI monitoring, protocol bookkeeping, routine merges of green reviewed PRs with user consent) stays at your level.

Then work the queue:

  1. Unless the user named an issue, dispatch or obtain the mandatory low-cost selector packet. The selector, not the orchestrator, reads the pinned tracker ersatztv#237 — body = goal + ordered arc + session protocol — and its last ~6 session comments (newest-first; the full comment payload is large, so stop at ~6). SOURCE OF TRUTH = live Gitea issue state, NEVER the prose. The arc body carries ORDER + goal only; a session comment's "Recommended next" is a forward guess written before the next session acted. Both go stale the instant an item closes (especially under parallel sessions narrating each other's work as "the gate that unblocks X"). So derive the candidate set from LIVE state, not from any inline marker or a prior comment's "next": the current gate = the lowest-numbered OPEN arc item in #237's arc list; its open children are the gate cluster (query them by the review label). Cross-check every arc / "recommended" item's real open/closed state (issue and milestone) before trusting it — do NOT hardcode which issue is the frontier; read it. Candidate discovery is a bounded cascade, not a full inventory: query arc → OPEN issues assigned to OPEN milestones → open review → unmilestoned/unreviewed priority: highpriority: mediumpriority: low, carrying eligible unique results forward until the shortlist contains five issues or all tiers are exhausted. Milestone records are tier metadata, NEVER pickup candidates; an OPEN milestone with zero eligible OPEN issues contributes zero candidates. This priority-label fall-through is mandatory: an empty arc/milestone/review pool is never evidence that the queue is empty. Each priority tier queries all OPEN timothy/ersatztv issues carrying that label, not only IDs mentioned in #237 or recent comments, and excludes pull requests. Treat an umbrella/epic as a container rather than a pickup when #237 names its eligible children. In the review tier, ALSO list open ersatztv-labeled issues in timothy/adversarial-reviewer — unclaimed audits there are pickup candidates too (read-only, parallel-safe; see the tracker's "Pending adversarial reviews" section). Reviewer audits are claimed by comment: a claim remains active until a later comment explicitly releases or abandons it, and a posted audit/review deliverable is completed work even when its issue stays open for implementer replies. If the Gitea MCP is down, hit the REST API directly (use credentials supplied by the environment or your global client instructions; never print them): curl -u <user>:<pass> http://192.168.1.95:3000/api/v1/repos/timothy/ersatztv/issues/237. The authoritative pickup + ranking protocol lives in #237's "Session protocol" section — this is a summary; if the two ever disagree, #237 wins.
  2. The selector returns the highest-ranked eligible candidate plus up to four fallbacks. Apply this strict tier order: (1) open arc items in #237 arc order; (2) OPEN issues assigned to an OPEN milestone; (3) OPEN review-labeled issues and eligible reviewer-repo audits; (4) remaining unmilestoned/unreviewed priority: high issues; (5) priority: medium; (6) priority: low. Within milestone and review tiers, order by priority: high > medium > low; for otherwise equal candidates preserve #237's explicit eligible order, then use lowest issue number. At every tier exclude pull requests, CLOSED issues, in-progress claims, parked issues, issues blocked by a CLOSED milestone, any issue with an open blocked-by dependency (GET /issues/{n}/dependencies; Gitea auto-clears the block when the blocker closes — do not pick an issue ahead of its open blocker), containers whose children are the pickups, and reviewer audits that are already claimed or have a posted deliverable. Note the 2026-07-17 convention: parked (no concrete plans) issues are closed, not left parked, so parked should rarely appear; if it does, treat it as excluded and surface it to the user. Query the next tier only while fewer than five eligible unique candidates have been collected; do not enumerate the remainder after the shortlist is full. The orchestrator then makes one focused Gitea read to confirm the proposed winner is still OPEN, unclaimed, and not blocked by a closed milestone; if it changed, check the next supplied fallback. An empty arc frontier is not a stopping condition. If the selector returns any eligible candidate, claim its top-ranked winner; do not ask the user to choose merely because candidates belong to different workstreams. Do not reread the full tracker or comments for selection. If the prose says "recommended next / now unblocked" but the issue (or its milestone) is already CLOSED, it is done — skip it and fix the stale line in your session comment. Prose lags live state; live state wins; tier + priority: labels decide order, not the prose. Never invent a fix-size, recency, or perceived-relevance tiebreaker. (This mirrors #237's Session-protocol ranking — #237 is canonical.)
  3. Claim it: add the in-progress label + a "claiming" comment on the issue(s); reviewer-repo audits are claimed by comment only. Treat that claim as live until a later comment explicitly releases or abandons it, and exclude audits with a posted deliverable even while the issue remains open for implementer replies.
  4. Read the issue bodies (they carry the task context/evidence) and work the item under the HARD CONSTRAINTS below.
  5. Finish by following the session-end protocol in #237: run the H12 qualification audit (ETV_GITEA_BASICAUTH=user:pass scripts/issue-qualification-audit.sh) and add a priority: label to anything it lists (every issue you filed this session included); then ONE session comment on the tracker (template in the tracker body, incl. triage verdicts for any new issues), remove your in-progress labels, and complete the per-issue Task Completion Protocol from the applicable AGENTS.md / CLAUDE.md instructions (including the done workflow when required).

HARD CONSTRAINTS:

  • Work in worktrees off origin/main — ALWAYS, including for a one-line docs commit. /Users/timothy/ersatztv is a shared mutable tree, not a main mirror: never commit in it, and never read its HEAD/git log to learn anything about main (you'll get another session's branch). Its only sanctioned use is read-only: copying web/node_modules out of it. origin/main is the source of truth — git fetch origin main and reference it explicitly. See the lore bullet for the two sessions this bit on 2026-07-17.
  • PARALLELIZE BY DEFAULT when there are genuinely disjoint slices: use up to the client's concurrency limit, normally 23 delegated agents plus the orchestrator. Fan out bounded recon early, but do not invent artificial slices or exceed a runtime-imposed agent cap. At most 34 concurrent dotnet/npm builds are fine on this Mac (M4, 10 cores, 16 GB); the go/no-go signal is FREE RAM, not CPU load (memory_pressure -Q: <20% free → don't launch more build agents; <10% → pause/stagger). CPU load spikes during builds are benign. Never 5+ builds (the historic crash was RAM starvation from an 89-way fan-out). NEVER set ETV_UPDATE_GOLDENS.
  • Never two committing agents on ONE worktree — give each parallel slice its own worktree branched off the feature branch and merge back. Sequence only where a slice genuinely depends on another's output (backend-first narrow, SPA-wide after worked well for #216).
  • BATCH YOUR PUSHES — you cannot cancel a CI run (operator-requested, 2026-07-17). Hold review fixes, doc corrections and format fixes locally, verify, and push once; each push spawns a run that orphans the previous one on a 4-slot runner. Cancellation is impossible from the agent side on this Gitea (1.25.4): POST /api/v1/.../actions/runs/{id}/cancel404, MCP actions_run_write cancel_run404, and the web-UI route needs a session+CSRF that does not script (login 303s, no session cookie). Only the operator can cancel, in the browser — so if you must supersede a live run, SAY SO explicitly instead of leaving it burning. (dispatch_workflow is a different route and still works for re-triggering a main run.) Corrects the older "superseded runs drain on their own" note below: they do finish, but they hold a slot while doing it.
  • A lone decisions lifecycle red is a KNOWN INFRA FLAKE — do NOTHING (operator-stated, 2026-07-19). When it is the only red: do not investigate it, and do not create a new run or commit to clear it — no rebase, no --amend, no no-op push. The operator reruns that single job from the Gitea UI. Report it as a known flake and carry on; only if other jobs are red too does the run deserve diagnosis. Same family as the killed-job rule below (a spurious single-job red is cleared by the operator's selective rerun, never by pushing — and pushes can't be cancelled, see the batching constraint above). The trap is that a convincing local explanation is always available. On #473/PR #479 the job went red just after main landed its own decisions.md entry, so "mine is no longer at EOF, I must rebase" looked airtight — I rebased, and it went red again on a head whose diff was a verified pure EOF append with zero deleted lines. A rebase that provably satisfied the gate's stated rule did not turn it green, which is the proof the red was never about the diff. Check this rule before theorising about the cause.
  • Do NOT gate or throttle your push on host health — trust the Gitea build queue. The runners were retuned for stability (operator, 2026-07-17); queueing is the queue's job, not yours. Don't SSH to bumblebee to sample load/RAM before pushing, and don't hand-schedule around other sessions' runs. Batch your pushes (above) because orphaned runs can't be cancelled — not because the host needs protecting.
  • An infra-shaped red under host pressure is not a code failure — check the host before diagnosing the diff (2026-07-17). While bumblebee was saturated, two unrelated jobs died in ways that look like code bugs: EF migration integrity failed in ~11ms inside Setup .NET with remove /root/.cache/act/…/.gitignore: no such file or directory (act's shared cache), and main's Build & push image died on buildx DeadlineExceeded: failed to compute cache key after ~210s of retries. Both re-ran green once the host recovered; the migration job had no model change to test in the first place. Tell: the failure is in a setup/cache step, before your code compiles. Don't file a CI bug off one sample under pressure — you'll send the next session chasing a phantom (this nearly got filed as #390 lane-rebalance fallout, which the evidence did not support).
  • Before any push touching .cs, check the touched set for a UTF-8 BOM. The #311 gate is fix-as-you-touch: a legacy file's inherited BOM becomes your problem the moment you touch it, and charset=utf-8 means no BOM. for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; done Strip any hit (1-line diff, zero content change). Bit two sessions on 2026-07-17 — PR #405 (6 files) and #70/PR #402 (19 files, where Python io.open(..., encoding='utf-8-sig') wrote BOMs back while "preserving existing style"). Verify your detector: an od -An -c | grep '357 273 277' check reported all-clean while 19 files were dirty — a detector that can only say "ok" is worse than none. Use the xxd form or read bytes directly. dotnet format --include DOES work here (corrected 2026-07-17, #70/PR #402 — an earlier note claiming it silently no-ops was WRONG). The apparent no-op was the shell: CI's recipe uses mapfile, which is bash-only, and the default shell here is zsh → empty array → zero files → exit 0. Run it under bash -c:
    bash -c 'mapfile -t files < <(git diff --name-only --diff-filter=ACM origin/main...HEAD -- "*.cs")
    dotnet format ErsatzTV.sln --no-restore --verify-no-changes --include "${files[@]}"'   # exit 0 = gate passes
    
  • Merge consent is DERIVED, not asked: the H6/H10 gate (pretooluse-merge-consent.sh) auto-grants a merge when the linked issue's ## Done-when boxes are all ticked, a fresh positive Review-verdict references the current head, and CI is green — no conversational confirmation on that satisfied path (ersatztv#314). Confirm in-conversation only when the gate ASKS (state not derivable: no creds, no linked issue, no Done-when/verdict, Gitea down). Prior pre-approvals still do NOT carry over.
  • Local gate + cold review run BEFORE the push to Gitea CI, not after (Timothy, 2026-07-17): run the local build/test gate AND a cold-context adversarial review agent (SCOPED "review only") over the diff, fold any fixes, and only THEN push / open the PR. Gitea runners should only ever see already-validated work — CI is a scarce shared slow resource and a run cannot be cancelled on this Gitea, so spending a run to surface a lint error or a review-caught bug is pure waste + queue pressure. CI (armed at PR-open, below) confirms; it is not the first line of defence.
  • Arm a CI monitor on the PR head sha AT PR-OPEN (commit-status endpoint), not at the end.
  • Live-E2E via scripts/e2e-local.sh (also before the push for write-path / UI changes); NEVER exercise download endpoints via browser tabs (curl them).

Lessons / workflow lore (STANDING workflow/orchestration rules only)

Scope: this section is how we work (orchestration, CI, review, cross-session hygiene) — not engineering decisions. Every "why the code does X" decision lives in docs/decisions.md (+ api-conventions.md §7a/b/c for the concurrency/side-effect mechanics, spa-conventions.md, e2e-local.md). Do NOT re-record a code/API/SPA decision here — it will duplicate and drift. Pruned 2026-07-12: issue-specific engineering bullets now covered in those docs, and dead Blazor-parity process notes, were removed (git history has them).

  • READ docs/README.md → the convention docs FIRST; point recon/implementer agents at specific doc sections. Only recon the task-specific delta.

  • The mechanical selection checks now live in scripts/select-queue.sh — run it instead of re-deriving them by hand (2026-07-19). The three bullets below are the evidence for why the script exists: a cheap model repeatedly reported blocked issues as deps:clear, mis-tiered on the no-op ?milestones= filter, and ranked by issue number instead of priority-within-tier — so the lore grew to "re-derive the whole contested tier every session," which is exactly the tax the script pays down. The script does the deterministic work in code (dependency exclusion via GET /dependencies, LOCAL .milestone.state/review/priority: tiering, (tier, priority, issue#) ordering, in-progress/parked/PR exclusion) and only flags CLAIM?/UMBRELLA? for judgment. Trust its DEPS/tiering/ordering; do the focused recheck only on the flagged rows. Keep the evidence bullets below (they explain the failure modes the script encodes), but the prescription is now "run the script," not "hand-re-derive." If you extend the tiers/filters, update the script first, then these bullets.

  • Gitea milestone-name issue filter silently NO-OPS on names with special chars (2026-07-17, #77 selection): GET /issues?milestones=<name> returns the WHOLE open-issue list — not a filtered set — when the milestone title contains : or + (e.g. Scheduling: refactor + distribution), because the raw name doesn't round-trip through Gitea's filter. A selector that trusts that response will mis-tier issues (it made #72 look like a milestone-10 member when it's unmilestoned, and hid #77's true sibling set). Robust recipe: fetch all open issues once and filter LOCALLY on each issue's .milestone.title (?state=open&type=issues&limit=50, then [i for i in issues if (i['milestone'] or {}).get('title')==NAME]) — do not rely on the server-side milestones= name filter. If a count looks suspiciously like "all open issues," that filter silently failed; re-derive membership locally. (Dependencies API is fine: POST /issues/{n}/dependencies with {"owner","repo","index"} sets blocked-by; the bare {index} form 201s but may not attach — verify with the GET.)

  • Confirmed again + widened (2026-07-17, #72/#164 selection): treat EVERY mechanical claim in the selector packet as unverified — not just deps and priority. That run's dependency data was correct, yet it still ranked #383 first while missing that #384 was in-progress (claimed 40 minutes earlier) and that #383 is that bundle's umbrella (its body: "Follow-up arc to #69" with #384#386 as deferred sub-work). Claiming its #1 pick would have collided head-on with a live parallel session. The packet does not reliably see: blocked-by, priority-within-tier, in-progress claim state, or umbrella-vs-child. Before claiming, check the candidate's last comments for a live claim (a claim stays live until explicitly released) and read its body for umbrella framing. An empty-ish tier is normal in maintenance mode — three of the four open milestone issues that day were legitimately blocked or claimed, which is the expected steady state, not a reason to relax the checks.

  • The cheap selector's dependency + priority data is NOT trustworthy — re-derive the whole contested tier, not just the winner (2026-07-17, #73 selection): the selector packet reported #385/#386 as deps:clear when both were blocked (#385 by open #70; #386 by open #385+#384) — it never called GET /issues/{n}/dependencies — and missed #383 entirely (the only priority: medium in tier 1), ranking by issue number and ignoring the priority-within-tier rule. Either error alone produces a bad pickup: working a blocked issue, or skipping the top-priority one. The kickoff's "orchestrator makes one focused recheck of the winner" is insufficient whenever a tier has >1 candidate — a wrong winner is invisible to a winner-only recheck. Recipe: fetch all open issues once, filter locally on .milestone.title/labels (the ?milestones= filter no-ops on :/+ names — see the bullet above; both current open milestones have them), then run GET /issues/{n}/dependencies yourself on every candidate you'd plausibly claim. That's ~3 cheap calls and it is the only thing standing between the packet and a wasted session. Keep delegating selection (it's still not orchestrator work) — just verify its two mechanical outputs (deps, priority order) rather than its ranking prose.

  • Codex cheap-worker launch (tested 2026-07-14): the native spawn_agent interface currently has no model/effort selector, so it provides parallelism but not cost savings. For bounded, tool-bearing selector/recon work, launch a separate worker with codex exec --ephemeral --color never -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only -C <repo> "<hard-capped scope contract>". GPT-5.4-Mini is the current supported small/cost-efficient profile on this ChatGPT-authenticated host; the older gpt-5.1-codex-mini guess fails with HTTP 400. Keep low whenever any shell/MCP/web tool is exposed; reserve minimal for explicitly tool-free synthesis over already-supplied bounded evidence. If the model rolls over, inspect the visible entries in ~/.codex/models_cache.json instead of guessing names, then update this dated note once verified.

  • Keep the MAIN checkout's web/node_modules fresh (npm install after pulling a PR that adds a dep) — worktrees copy it, and a stale copy broke typecheck in a #198 worktree.

  • Playwright-MCP E2E: never open tabs/window.open for file-download endpoints — curl them. If browser tools stall repeatedly, pkill -f ms-playwright-mcp and drive a fresh session.

  • THERE IS NO "main checkout". /Users/timothy/ersatztv is a shared mutable working tree whose HEAD is whatever the last session left there — it is NOT a main mirror, and its name lies. Treat it as read-only, and never as a source of truth:

    • Never commit in it — not even a one-line docs fix. Need a commit on main? Make a worktree off origin/main (git worktree add -b <branch> <path> origin/main) and commit there.
    • Never read git log / git status / HEAD there and infer anything about main — you will be reading some other session's branch.
    • origin/main is the only source of truth. git fetch origin main and reference origin/main explicitly. Do not trust the local main ref either (it lags; nobody is obliged to update it).
    • Its one sanctioned use is read-only: copying web/node_modules into your worktree (keep it fresh — a stale copy broke typecheck in a #198 worktree).

    This is a design flaw, not a discipline failure — do not "fix" it by telling sessions to check harder. A check does not stay true. Worktree discipline is healthy (10 feature worktrees on 2026-07-17: Claude under .claude/worktrees/, Codex under .worktrees/, some in /private/tmp/); the shared tree is the single place the model breaks, and it broke for two different sessions on the same day, both doing the obvious thing:

    • Session A assumed it was on main (true at 12:46 when it pulled — correct when made, stale by 14:17) and committed → the commit landed on the #604/CI session's ci/604-lane-rebalance, 24 seconds after that session's own commit at 14:17:28. Two writers, one tree. Nothing tangled, by luck.
    • Session B ran git log there, read ci/604-lane-rebalance's HEAD, and concluded main was "4 behind origin" — a phantom.

    A session-start git status appears to confirm the false assumption, which makes it worse than a plainly wrong claim. (Supersedes the old "the user's main checkout stays ON main and pulled each session" note.) Same family as the sibling bullet below: never commit inside a checkout or worktree you didn't create.

  • Subagent connection drops = laptop sleep/transient; re-resume via SendMessage (work survives).

  • Build/verify (and review) locally, then trust it; CI (VM 127) confirms (~9 min warm; seconds for a docs-only PR since ersatztv#416). Watch by commit status: /api/v1/repos/timothy/ersatztv/commits/{sha}/status.

  • PR routine (works): worktree off origin/main → implement (strongest tier for judgment-heavy / balanced tier for mechanical work when model routing is available; doc pointers + exact facts; use the available C# language tooling) → API changes: build app project FIRST then ./scripts/update-openapi.sh then npm run generate:apifull local test pass + cold-context adversarial review (SCOPED "review only") + live-E2E for UI/write-path changes (scripts/e2e-local.sh), ALL BEFORE the push — fold pre-push review fixes into the branch (amend/reword is fine while nothing is pushed) → push, PR (arm the CI monitor at open) → CI confirms → any further review fixes (a post-push cross-model pass, or CI-surfaced issues) as follow-up COMMITS (never amend/force-push a pushed branch) → CI green → merge (derived consent gate) → structured close comments per the applicable AGENTS.md / CLAUDE.md protocol.

  • Never run TWO committing agents on one worktree concurrently. Read-only review agents are fine.

  • A KILLED job reports conclusion: failure — the conclusion field is not enough. Read the log tail. (2026-07-17.) When the operator restarts the runners (a retune, a reboot), in-flight jobs die and Gitea marks them failure, not cancelled. Tell: the log stops mid-step with no error and no ❌ Failure - Main … marker — a real failure always leaves a marker. Run 1006's EF migration integrity + Functional E2E "failed" on a BOM-removal-only diff (6 files, 1 line each, zero content change) that could not possibly break them; the logs simply truncated mid-dotnet build at 12:08 UTC, and both runners showed Up About an hour — the retune had killed them. Log timestamps are UTC; the host is UTC+2 — convert before correlating, or the restart looks 2h off and the theory gets wrongly discarded. Corollary: a diff that cannot cause a failure is evidence the failure isn't yours — when a job that passed on the previous head fails on a semantically-null delta, suspect the environment and go read the log, don't re-litigate the diff. Also beware log warnings that look fatal: Unable to pull refs/heads/v4: … is act refreshing its /root/.cache/act action cache and is followed by Cloned … — it is NOISE, not a cause. Grep for the failure marker, not for the word "error".

  • cancelled is NOT failure — never read a cancel as a CI verdict (2026-07-17, operator-surfaced). The operator cancels runs by hand (they're the only one who can — see the batching constraint), and a run-level conclusion: cancelled means no verdict, not a pass and not a fail. Two traps: (1) a run whose overall state is failure may hold a genuine job failure that happened before the cancel — check job-level conclusion + timestamps, don't attribute it to the cancel; (2) a cancelled run tells you nothing about your diff, so don't claim green on it. Report FAILED and CANCELLED counts separately in any CI monitor: fail=[j for j in jobs if j['conclusion']=='failure']; canc=[j for j in jobs if j['conclusion']=='cancelled'] — a monitor that only asks "is state != pending" will call a cancelled run a failure and send you debugging a phantom.

  • An infra-shaped red under host load is not a code failure — check uptime on bumblebee BEFORE diagnosing the diff (2026-07-17). At load average 243, two unrelated jobs died in ways that look like code bugs but aren't: EF migration integrity failed in ~11ms inside Setup .NET with remove /root/.cache/act/…/.gitignore: no such file or directory (act's shared cache), and main's Build & push image died on buildx DeadlineExceeded: failed to compute cache key after ~210s of retries. Both re-ran green at normal load; the migration job had no model change to test in the first place. Tell: the failure is in a setup/cache step, before your code compiles. Don't file a CI bug off one sample under load — you'll send the next session chasing a phantom (this nearly got filed as #390 lane-rebalance fallout, which the evidence did not support). A game server (FactoryServer-L) shares that host with the runners, so load isn't always CI's doing.

  • Arm a CI monitor the moment the PR opens (commit-status endpoint), not at the end — in the #216 session three PR runs sat red ~an hour on a CI-only flake while review/E2E ran. (The old MySQL host-port 3306 collision is fixed on main, ef8915f1; a lone MySQL-apply red now = the known infra flake, rerun — see MEMORY / ci-migration-job-mysql-flake.)

  • "Make X consistent with Y" review findings deserve new-code scrutiny — a #216 review fix propagated MediaBrowseScreen's mount-debounce selection-wipe race to SearchScreen instead of catching it. For any timer/effect, ask "when does this fire?" (incl. mount).

  • Live-E2E seeding recipe now lives in docs/e2e-local.md → "Seeding a local TV library for E2E" (tiny ffmpeg testsrc MKVs + LibraryPath SQL rows + scan). Live-E2E is a stated requirement for write-path handler changes — see that doc's "When live-E2E is required" + the decisions.md entry; it's the only net for the lazy-enumeration / reload-through-read-path 500 class (#229).

  • Parallel sessions (2026-07-11 protocol): claim before working (in-progress label — the tiny read→claim race window is accepted; later claimant backs off). Claiming prevents duplicate pickup, NOT overlapping code changes — check the tracker's dependency notes ("#234 after #231", "coordinate with #215") before touching shared surfaces. Editing THIS lore: prune covered/stale bullets too (not append-only — git keeps history), git pull --rebase before commit. Two runners (ci-runner VM 127 + bumblebee-runner, 4 slots); no cancel route exists on this Gitea version — verified 1.25.4: REST and MCP cancel_run both 404 (see the batching constraint above). Superseded runs do finish on their own, but they hold a runner slot while doing it, so with 4 parallel sessions they are a real cost — batch pushes rather than relying on them to drain.

  • Two sessions touching one machine: a branch may be checked out in ANOTHER session's worktree — never commit/merge inside a worktree you didn't create. To land a merge on such a branch without touching their checkout: plumbing merge (git read-tree -m base ours theirs into a temp GIT_INDEX_FILE → write-treecommit-tree -p ours -p theirs → push the commit to the branch ref); the owning session then git pulls.

  • Cross-model independent review rotation (2026-07-11): use the other client/model family when it is available — Codex reviews Claude-authored changes; Claude reviews Codex-authored changes. Use whatever native integration is actually installed; client-specific helpers such as codex:rescue are optional accelerators, not protocol dependencies. On quota/tool exhaustion, use a cold-context review-only agent in the current client and retry cross-model review next window. Independent reviewers do review and stuck-diagnosis work, not implementation on the same change. Skip rubric (self-exemption is an auditable claim, not a vibe): an independent review pass (preferably cross-model; otherwise a cold agent) is mandatory for any diff touching locks/concurrency, auth/security, API write-path handlers, or DB migrations, or larger than ~150 changed C# lines. It is skippable only for a pure-SPA/docs leaf with no server-state effect — and when you skip it, say so and why in the PR or close comment (a stated exemption, not silence). This is the correlated-blindspot net: independent model families miss different classes (see the #235 "enumerate every producer" and #250 lessons), so a single reviewer on a risky diff is the gap.

  • Force enumeration of lazy LanguageExt returns in tests (2026-07-11, PR #229): a write-path 500 survived a green fixed-point test because the handler returned a lazy Map the test never enumerated. Enumerate handler return values in tests; live E2E remains the only net for this class. (Write-path reload-through-read-path mechanics: api-conventions §7.)

  • Keep a PR branch current by REBASING on origin/main — do NOT merge main in (ersatztv#311, enforced by the H11 pre-push hook: a push from a branch behind origin/main is blocked → git fetch origin main && git rebase origin/main). A merge commit pulls in every file main changed, incl. files you never touched (e.g. the ~2500 legacy-BOM .cs), which then trip the format hook/CI on code that isn't yours (the #309 session). Rebasing keeps your diff to exactly what you changed. After a rebase that hits conflicts in generated artifacts (v1.json, v1.d.ts, endpoint-index), REGENERATE them (./scripts/update-openapi.sh + npm run generate:api) — never hand-resolve; git text-merges them plausibly-but-wrong and npm run check:api is the guard. Escape hatch for a deliberate non-rebased push: ETV_SKIP_REBASE_CHECK=1 git push.

  • H12 issue-qualification audit (ersatztv#312, scripts/issue-qualification-audit.sh): a session-end check that lists OPEN issues missing a priority: label — the #237 ranking keys off priority:/gate labels, so an unlabeled issue is invisible to it. "Fully qualified" = has a priority: {high,medium,low} label (that signals triage ran; gate-vs-backlog is then derivable from the review label / milestone, and a milestone is NOT required — backlog is unmilestoned). Run it at session end and label anything it flags (esp. issues you filed this session). Fail-open without creds; advisory (exit 1 when any are unqualified). Sibling to H11 (both #311/#312).

  • CI VM test timeouts: heavy-render web tests (100+ item grids) need explicit vitest timeouts (e.g. 15s) — the CI VM hit the 5s default on a test that runs in ~1s locally (run 686). Bump per-test, don't raise the global default.

  • Concurrency tests need a negative control + a stress loop, not a single round (2026-07-11, #231): a one-shot Barrier(N) + Task.WhenAll "exactly one winner" test did NOT catch a deliberately non-atomic check-then-set flag on this Mac — the load→store window is too narrow to collide when threads release once. It green-lit broken code. Fix: hammer the race over many rounds (N worker threads × ~20k rounds, two Barriers per round: acquire → validate one-winner → reset), and ALWAYS prove non-vacuous by temporarily breaking the primitive and watching the test fail. Cheap (sub-second) and deterministic-enough. Same discipline as #12's negative control.

  • Before deleting a "workaround", check it isn't serving a SECOND purpose (2026-07-11, #232): an issue said "remove the SPA PENDING_GRACE_TICKS grace-tick workaround now the API is honest". The grace ticks did double duty — compensating the lying-200 (now genuinely obsolete) AND absorbing the inherent queue→observed-active lag / fast-completion race (still needed). Removing wholesale wedged the scan button disabled-until-reload for short scans; both fork + Codex caught it. Keep a bounded version, re-scoped honestly. When an issue says "delete X", enumerate every behavior X provided before deleting.

  • Re-review the FIX COMMIT, not just the initial PR diff, before closing (2026-07-11, #242, user-surfaced): the pre-fix review is only half the loop — the fix commit is new code no independent reviewer has seen. Here the first fix (gate Edit on saving) only PARTIALLY closed the in-flight-save clobber class; a Codex re-review of the fix commit caught that Delete was the one schedule-switch path (applySwitch) still missing the guard. Fold a scoped re-review (cross-model if quota, else a cold review agent) over the fix commit into the close protocol; loop until a clean MERGEABLE verdict or an explicit acceptable-defer (file a tracked issue for the deferral, as #248 here). Cheap relative to shipping a half-fix that reopens the issue.

  • Negative-controlling a lock/flag fix: invert the condition, don't stub if (true) (2026-07-11, #250): to prove a lock-release test is non-vacuous I temporarily broke the fix. First attempt made the guard if (true) — but that left the ownership flag "assigned but never read" → CS0219 under warnings-as-errors → the Application build silently FAILED → dotnet test --no-build ran the STALE (fixed) dll → false PASS that looked like the control had failed to catch the bug. The tell: build output must show Build succeeded / 0 Error(s), not just a truncated tail. Break the primitive by inverting the real condition (if (!lockAcquired)) so the flag stays read and the code compiles; always grep the build for error CS before trusting a --no-build test result. (Codex-found High this session: an outer catch that unlocked unconditionally cross-released another session's lock on a pre-acquisition exception — the fix was a Handle-scoped ownership flag gating the catch.)

  • A lock/channel-ownership "no cross-release" verdict must enumerate EVERY producer of that message type, not just the enqueuers visible in the PR diff (2026-07-11, #235/#267). The cold fork cleared the new scan-collections per-provider lock as sound — "only the acquirer enqueues, so only its message triggers the scanner unlock" — because it checked the three new controllers but never grepped for ALL Synchronize{X}Collections( senders. Codex did: SchedulerService also enqueues them periodically and unlocked, and ScannerService's finally released the lock whenever held → a scheduled scan cross-releases an API scan's lock (#250 class). Rule: before trusting any "single owner / no double release / no cross-release" claim, grep the whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer. This is why the independent Codex pass earns its keep alongside the fork even when the fork returns MERGEABLE.

  • Independent-review disagreement on a gate PR → frontier judge; judge the fix, not just the verdict (2026-07-11, #235): the reconciliation isn't only "who's right" — Fable historically ratified Codex's High AND caught that the orchestrator's proposed fix (per-source lock-or-skip) would deterministically starve the 2nd+ media source (fixed loop order), prescribing the shipped library-scan §C7 pattern instead (lock once per provider, batch-enqueue, release on the Unlock: isLast message). Escalate the FIX approach too, not just the finding.

  • Harden a soon-to-grow feature with a runtime posture/toggle, not a hardcoded clamp (2026-07-12, #289): the MCP's final design (#58) is read+write, so a GET-clamp would be ripped out — shipped a runtime read-only posture (ERSATZTV_ALLOW_WRITES, default false) instead. When a security fix constrains a capability the roadmap will later want, make the safe state the default of a switch, not a wall; check the feature's issue for its end-state first.

  • Fixing onto an unmerged feature branch: branch off it, land by FF-push, never touch a sibling worktree that has it checked out (2026-07-12, #289 → PR#76). Gotcha: recon by Reading a sibling worktree leaked its paths into my later Edits → they landed there, not the fresh worktree (dotnet built the unchanged tree → "0 new tests" tipped it off). After creating a worktree, do the first Edit/Read against ITS absolute paths and git status it before building.

  • HttpCompletionOption.ResponseHeadersRead moves the body read OUTSIDE HttpClient.Timeout (2026-07-12, #289): with a None read token a slow-drip upstream hangs forever. Fix = per-request linked CancellationTokenSource.CancelAfter(timeout) into BOTH SendAsync and the stream reads + HttpClient.Timeout = InfiniteTimeSpan; catch transport/timeout exceptions and turn them into a response. (Not yet in decisions.md — #289 landed on PR#76, not main.)

  • Queue-drift root cause + standing rule (2026-07-12, user-surfaced): pickups repeatedly re-picked already-done work — #91b was framed "recommended next / now unblocked" for two sessions after it had merged (2026-07-11), and the #251/#252 priority-pickups sat listed "open" after closing — requiring a reactive body correction (tracker comment 16:00). ROOT CAUSE: DONE/OPEN status was read from prose (the arc body's inline "DONE" markers + each session comment's "Recommended next"), which is append-only and hand-edited, so it lags real issue state — worst under parallel sessions that narrate each other's merges as "the gate that unblocks X" (#271 got cast as the last gate for an already-merged #91b). It is a structural bug, not a stale-writer bug: any status embedded in prose will drift. STANDING FIX (don't just re-patch the body next time): live Gitea state is the ONLY source of truth for status. The arc body carries order+goal; a comment's "Recommended next" is a candidate that MUST be re-verified OPEN (issue and milestone) at pickup. When prose disagrees with live state, live state wins — correct the prose in your session comment, never propagate it. STRUCTURAL CURE (2026-07-12 review, Fable): #237's arc no longer carries inline DONE markers — closed items move to a "Done (history)" section, so status lives ONLY in live Gitea state and can't drift; and the gate/frontier is defined structurally (lowest-numbered open arc item), never hardcoded to an issue number. Beware: the first pass at this fix re-planted the very bug by hardcoding "#197 cluster" / "#91b milestone CLOSED" into the kickoff — if you name today's frontier issue in this standing file, you are writing the next drift. Gate cluster = open review-labeled issues serving the current open arc item.