Third review pass: MERGEABLE WITH NITS. Taking the one finding it asked for before merge, plus a doc nit. The cancellation filter added last commit had no token check, and it spans the whole Transcode body -- including every mediator send (ffprobe via CliWrap, media-server API calls, subtitle extraction, song-video generation). TaskCanceledException is also what HttpClient throws on its OWN timeout, so a real timeout in any of those was being downgraded from an ERROR with a stack trace to a routine "Terminating HLS session" Information line. Behaviour was unchanged (both arms return false) but the fault signal was lost, and this repo has been bitten before by "empty log != the event didn't happen". Now filters on cancellationToken.IsCancellationRequested, so only genuine caller cancellation is treated as a graceful teardown. Doc nit: the <exception> block said cancellation "is thrown"; it is only thrown when the token trips while the probe is in flight -- cancelling after it completes returns normally. Now says "may propagate". Declined the reviewer's optional suggestion to drain until a 0-return instead of reading exactly one byte: reading exactly one byte is what makes the guard safe BY CONSTRUCTION, since a server or proxy that answers 206 with a wider range than requested still cannot be drained unboundedly. 206-only was confirmed correct rather than extended to short 200s, since deciding "short" from Content-Length would reopen the unbounded path for a chunked or Content-Length-less response. Also records the operator's standing rule in the handoff lore: a lone `decisions.md append-only` red is a known infra flake -- do not investigate, rebase, amend or push to clear it; the operator reruns that job from the UI. I violated this earlier in this PR with a tidy-but-wrong "my entry is no longer at EOF" theory, and the rebase did not fix it -- the job went red again on a verified pure-append diff, which is the proof the red was never about the diff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
52 KiB
ChicoryTV issue-queue handoff (client-neutral kickoff + workflow lore)
PROTOCOL CHANGED 2026-07-11 (decisions.md entry of same date). Queue state lives in the pinned Gitea tracker ersatztv#237, not in this file. Do NOT write session state, queue order, or next-session prompts here — sessions ending under the old protocol should append their session comment to #237 instead. This file holds only stable operating guidance: the standing kickoff prompt and the workflow lore. Historical per-session state:
git logof this file (last state-bearing revision:8b77d5e7).
Current phase — maintenance / backlog mode (terminal transition, 2026-07-13)
The ChicoryTV rewrite + go-live arc is COMPLETE: the pinned arc tracker ersatztv#237 is CLOSED and all six numbered arc items shipped. The project is now in maintenance / backlog mode. This is a terminal fact (the arc will not re-open), so it belongs in this standing file — unlike any "current frontier issue", which must never be written here (read it live; see the queue-drift lesson at the end of this file).
So a fresh session stops re-discovering this every time:
- The arc / gate / open-
reviewtiers are normally EMPTY now. An empty arc frontier is the expected steady state — not an anomaly to investigate, not a reason to stop, and not a reason to ask the user what to work on. Fall straight through to thepriority:-label cascade and claim the top-ranked eligible backlog issue. - Selection is priority-label-driven: OPEN-milestoned →
priority: high→priority: medium→priority: low; within a tier, lowest issue number. Excludein-progress, pull requests, and any issue with an openblocked-bydependency — Gitea tracks these (GET /issues/{n}/dependencies) and auto-clears the block when the blocker closes, so a blocked issue becomes eligible automatically once its blocker lands. Do not pick an issue ahead of its open blocker (e.g. #70 waits on #380). parked→ closed, not parked (convention 2026-07-17):parkedmeans "no concrete plans," and the decision was to close such issues rather than let them linger (reopen instantly if revived). Soparkedshould normally not appear; if it does, treat it as excluded and surface it to the user.- Bundles (single-session sweeps): related issues are grouped two ways — (a) an active-arc
milestone (e.g. "Scheduling: refactor + distribution", "Auto-Tune DetailPanel", "Design system
(Claude Design ↔ prod)"), or (b) a cross-reference comment on backlog clusters (e.g. "🔗 Session
bundle — security hardening: #293, #376"). When you claim an issue, check its milestone AND its
cross-references / backlinks for co-workable siblings, and sweep the whole disjoint cluster in one
session. Active-arc milestones auto-sort above the flat priority backlog; leave individually-important
backlog bugs unmilestoned so their
priority:label still orders them. - A new gate re-appears only when a future adversarial-review Blocker/High finding is slotted
as a priority pickup, or a new OPEN milestone re-establishes an arc. Until then, backlog
features (ordered by milestone/
priority:label) are the normal, expected work.
Orchestrator launch profile (operator-facing; do not paste)
As of 2026-07-14:
- Codex: use GPT-5.6 Sol,
ultrafor the orchestrator.ultrais the Codex multi-agent orchestration setting; if it is unavailable, use GPT-5.6 Sol,maxeffort (xhighonly when you deliberately want to trade some capability for quota/latency). - Claude Code: use Opus 4.8,
xhigheffort (or ultracode, which is the Claude Code orchestration preset built onxhigh). Reserve Fable 5,higheffort for the frontier escalations listed below; usexhighthere only when the decision remains genuinely difficult. - Queue preflight: when the user has not named an issue, use the client's cheapest suitable
fast/small model at low effort. Queue selection is tool-bearing, so do not use
minimal: Codex validates the full enabled tool profile, and enabledweb_searchcan rejectminimaleven when the selector intends to call only Gitea MCP. Route it as a subagent when possible; otherwise run a separate low-cost selector session and pass its compact selection packet to the orchestrator. Never spend the orchestrator tier on queue ranking. - If those names are unavailable, choose the client's strongest long-horizon coding/agentic model at its high extended-reasoning setting. Do not weaken the orchestrator to a fast/mini model; use smaller models for bounded delegated slices, with a separate low-cost queue preflight when the client cannot select a cheaper model per agent.
The kickoff below deliberately describes capability tiers instead of model names so it survives model rollovers. The operator-facing recommendations above may be updated without changing the queue protocol.
STANDING KICKOFF PROMPT (paste into a fresh Codex or Claude Code session unchanged)
You are the ORCHESTRATOR in the main Codex or Claude Code session. First identify your client, active model, effort/reasoning level, available delegation mechanisms, and concurrency limit. Orchestration means: decompose, delegate independent slices, integrate their results, verify the whole, and keep canonical issue state accurate. Use the client's native agent/subagent tools; never assume a named tool, command, plugin, model-routing feature, or fork mechanism exists.
Route by capability when the client supports per-agent model selection: fast/small for bounded recon, balanced for mechanical changes, and the strongest coding/agentic tier for judgment-heavy work. Otherwise use the active model for every slice except the mandatory queue preflight below. Independent review MUST start from a cold, review-only brief; prefer a different model family/client when one is available, otherwise use a fresh agent with no implementation role.
Low-cost routing applies throughout the session, not only to queue selection. Before any batch of
bounded searches, inventories, log triage, URL/status sampling, or report drafting, dispatch the cheapest
suitable fast/small model. If any tool is enabled or required, start at low; this includes enabled
web_search even when the prompt names only Gitea MCP. Reserve minimal for explicitly tool-free synthesis
over already-supplied bounded evidence. Never retry a known tool-bearing profile at minimal. Strictly
cap its allowed files, queries, log window, result/evidence count, and output; instruct it to stop and return
partial evidence at a cap instead of broadening scope. The orchestrator receives a compact evidence packet
and performs only the minimum focused live recheck. Same-tier agents add parallelism, not cost savings,
while direct tool-only mechanical checks may stay inline. If a tool-bearing launch is rejected at minimal,
retry once with the same model and unchanged scope at low. If a cheap worker lacks a required tool, give it
already-collected bounded evidence for explicitly tool-free synthesis or report the limitation instead of
silently expanding orchestrator reconnaissance. Global Codex hook enforcement is tracked in
timothy/server-management#592; the wider Claude-hook port is tracked in timothy/server-management#593.
When the user has not named an issue, queue selection is mechanical fast/small work, never orchestrator
work. FIRST run the deterministic selector — ETV_GITEA_BASICAUTH=user:pass scripts/select-queue.sh [N]
— which does the parts a cheap model kept getting wrong IN CODE: it excludes in-progress/parked/PRs,
resolves GET /issues/{n}/dependencies on every candidate (dropping any with an OPEN blocker), tiers by
LOCAL .milestone.state/review/priority: filters (never the server ?milestones= name filter, which
no-ops on :/+), and orders by (tier, priority, issue#). Its DEPS/tiering/ordering are deterministic —
trust them, do not re-derive them by hand. It also raises two JUDGMENT flags it deliberately does not
decide — CLAIM? (a claim can precede the label) and UMBRELLA? (an epic whose children are the real
pickups); the orchestrator (or a cheap model) resolves ONLY those by reading the flagged issue's
comments/body. The script does not rank the arc tier (arc order is #237 prose, currently CLOSED) — read
#237 if an arc re-opens. If the script is unavailable (no creds / Gitea down), fall back to the cheap-model
path below. Dispatch exactly one selector on the cheapest suitable model at low effort. Give it #237's
body and last ~6 comments, then have it query live candidates as a cascade in this exact tier order:
arc → OPEN issues assigned to OPEN milestones → open review → unmilestoned/unreviewed
priority: high → priority: medium → priority: low. Every candidate row represents an issue
(or eligible reviewer audit): milestone records provide tier metadata and are NEVER pickup candidates.
An OPEN milestone with zero eligible OPEN issues contributes zero candidates. Deduplicate candidates,
apply the live eligibility exclusions at each tier (including automatic exclusion of parked issues and any issue with an open blocked-by dependency),
and stop as soon as five eligible issues have been accumulated (or all tiers are exhausted). It returns
only that ranked top-five shortlist, with one-line rationales and live-state evidence; a small result
packet is the desired behavior, but an empty or undersized higher tier must fall through to the next tier.
The orchestrator receives only that compact packet, then performs a focused live recheck of the winner
before claiming. If the cheap worker lacks a repository-scoped issue-list tool, it must immediately return
the literal result TOOL_LIMITATION; it must not rank issue IDs mentioned in tracker prose, turn milestone
records into candidates, or infer that any tier is empty. The orchestrator then performs only the mechanical
tier queries (repository + state + milestone/label), passes at most the bounded raw ISSUE rows back to the
cheap worker, and leaves all filtering/ranking to that worker. Never substitute an owner-wide/global search
or treat that tool limitation as an empty tier.
Do not load implementation docs or issue bodies into the selector. If the client cannot route a
cheaper subagent, run the selector in a separate low-cost session before starting or resuming the orchestrator
and pass in its packet. Do not fall back to inline sorting or an equally expensive selector. If no cheaper
route or session is available, pause and request the selector packet rather than consuming orchestrator tokens
on queue ranking. If the user names an issue, skip selection and only verify that issue's live claimability;
this explicit user choice is the sole path by which a parked issue may be worked.
FIRST read AGENTS.md and CLAUDE.md when present, then docs/README.md, the convention docs it
indexes, and the Lessons below. Apply both client instruction files; where they differ, follow the
stricter safety/completion requirement unless a higher-priority instruction resolves the conflict.
FRONTIER ESCALATION — these moments go to the strongest available reasoning model. In Claude Code, prefer Fable; in Codex, prefer the strongest frontier model at extended-high reasoning. If you are already running that profile, handle them inline. Otherwise delegate with a SELF-CONTAINED brief (scope, exact question, evidence, file paths, and doc pointers), or ask the user to switch models when the client cannot route the escalation. Do not block merely because one brand-specific model is absent:
- Architecture/API design decisions (e.g. #202's media-source API shape) and anything
[PLAN-MODE]-tagged. - Parity verdicts for blazor-route-parity.md (per-capability diff sign-off) and #91b removal go/no-go.
- Reconciling contradictory independent-review findings and the final pre-merge judgment on gate PRs.
- Stuck after two genuinely different approaches, or evidence contradicts the issue/docs. Everything else (claiming, worktrees, dispatching implementers, CI monitoring, protocol bookkeeping, routine merges of green reviewed PRs with user consent) stays at your level.
Then work the queue:
- Unless the user named an issue, dispatch or obtain the mandatory low-cost selector packet. The
selector, not the orchestrator, reads the pinned tracker ersatztv#237 — body = goal + ordered
arc + session protocol — and its last ~6 session comments (newest-first; the full comment
payload is large, so stop at ~6).
SOURCE OF TRUTH = live Gitea issue state, NEVER the prose. The arc body carries ORDER + goal
only; a session comment's "Recommended next" is a forward guess written before the next session
acted. Both go stale the instant an item closes (especially under parallel sessions narrating each
other's work as "the gate that unblocks X"). So derive the candidate set from LIVE state, not from
any inline marker or a prior comment's "next": the current gate = the lowest-numbered OPEN arc
item in #237's arc list; its open children are the gate cluster (query them by the
reviewlabel). Cross-check every arc / "recommended" item's real open/closed state (issue and milestone) before trusting it — do NOT hardcode which issue is the frontier; read it. Candidate discovery is a bounded cascade, not a full inventory: query arc → OPEN issues assigned to OPEN milestones → openreview→ unmilestoned/unreviewedpriority: high→priority: medium→priority: low, carrying eligible unique results forward until the shortlist contains five issues or all tiers are exhausted. Milestone records are tier metadata, NEVER pickup candidates; an OPEN milestone with zero eligible OPEN issues contributes zero candidates. This priority-label fall-through is mandatory: an empty arc/milestone/review pool is never evidence that the queue is empty. Each priority tier queries all OPENtimothy/ersatztvissues carrying that label, not only IDs mentioned in #237 or recent comments, and excludes pull requests. Treat an umbrella/epic as a container rather than a pickup when #237 names its eligible children. In the review tier, ALSO list openersatztv-labeled issues in timothy/adversarial-reviewer — unclaimed audits there are pickup candidates too (read-only, parallel-safe; see the tracker's "Pending adversarial reviews" section). Reviewer audits are claimed by comment: a claim remains active until a later comment explicitly releases or abandons it, and a posted audit/review deliverable is completed work even when its issue stays open for implementer replies. If the Gitea MCP is down, hit the REST API directly (use credentials supplied by the environment or your global client instructions; never print them):curl -u <user>:<pass> http://192.168.1.95:3000/api/v1/repos/timothy/ersatztv/issues/237. The authoritative pickup + ranking protocol lives in #237's "Session protocol" section — this is a summary; if the two ever disagree, #237 wins. - The selector returns the highest-ranked eligible candidate plus up to four fallbacks. Apply this
strict tier order: (1) open arc items in #237 arc order; (2) OPEN issues assigned to an OPEN
milestone;
(3) OPEN
review-labeled issues and eligible reviewer-repo audits; (4) remaining unmilestoned/unreviewedpriority: highissues; (5)priority: medium; (6)priority: low. Within milestone and review tiers, order bypriority: high>medium>low; for otherwise equal candidates preserve #237's explicit eligible order, then use lowest issue number. At every tier exclude pull requests, CLOSED issues,in-progressclaims,parkedissues, issues blocked by a CLOSED milestone, any issue with an openblocked-bydependency (GET /issues/{n}/dependencies; Gitea auto-clears the block when the blocker closes — do not pick an issue ahead of its open blocker), containers whose children are the pickups, and reviewer audits that are already claimed or have a posted deliverable. Note the 2026-07-17 convention:parked(no concrete plans) issues are closed, not left parked, soparkedshould rarely appear; if it does, treat it as excluded and surface it to the user. Query the next tier only while fewer than five eligible unique candidates have been collected; do not enumerate the remainder after the shortlist is full. The orchestrator then makes one focused Gitea read to confirm the proposed winner is still OPEN, unclaimed, and not blocked by a closed milestone; if it changed, check the next supplied fallback. An empty arc frontier is not a stopping condition. If the selector returns any eligible candidate, claim its top-ranked winner; do not ask the user to choose merely because candidates belong to different workstreams. Do not reread the full tracker or comments for selection. If the prose says "recommended next / now unblocked" but the issue (or its milestone) is already CLOSED, it is done — skip it and fix the stale line in your session comment. Prose lags live state; live state wins; tier +priority:labels decide order, not the prose. Never invent a fix-size, recency, or perceived-relevance tiebreaker. (This mirrors #237's Session-protocol ranking — #237 is canonical.) - Claim it: add the
in-progresslabel + a "claiming" comment on the issue(s); reviewer-repo audits are claimed by comment only. Treat that claim as live until a later comment explicitly releases or abandons it, and exclude audits with a posted deliverable even while the issue remains open for implementer replies. - Read the issue bodies (they carry the task context/evidence) and work the item under the HARD CONSTRAINTS below.
- Finish by following the session-end protocol in #237: run the H12 qualification audit
(
ETV_GITEA_BASICAUTH=user:pass scripts/issue-qualification-audit.sh) and add apriority:label to anything it lists (every issue you filed this session included); then ONE session comment on the tracker (template in the tracker body, incl. triage verdicts for any new issues), remove yourin-progresslabels, and complete the per-issue Task Completion Protocol from the applicableAGENTS.md/CLAUDE.mdinstructions (including thedoneworkflow when required).
HARD CONSTRAINTS:
- Work in worktrees off
origin/main— ALWAYS, including for a one-line docs commit./Users/timothy/ersatztvis a shared mutable tree, not amainmirror: never commit in it, and never read its HEAD/git logto learn anything aboutmain(you'll get another session's branch). Its only sanctioned use is read-only: copyingweb/node_modulesout of it.origin/mainis the source of truth —git fetch origin mainand reference it explicitly. See the lore bullet for the two sessions this bit on 2026-07-17. - PARALLELIZE BY DEFAULT when there are genuinely disjoint slices: use up to the client's concurrency
limit, normally 2–3 delegated agents plus the orchestrator. Fan out bounded recon early, but do not
invent artificial slices or exceed a runtime-imposed agent cap. At most 3–4 concurrent
dotnet/npm builds are fine on this Mac (M4, 10 cores, 16 GB); the go/no-go signal is FREE
RAM, not CPU load (
memory_pressure -Q: <20% free → don't launch more build agents; <10% → pause/stagger). CPU load spikes during builds are benign. Never 5+ builds (the historic crash was RAM starvation from an 8–9-way fan-out). NEVER set ETV_UPDATE_GOLDENS. - Never two committing agents on ONE worktree — give each parallel slice its own worktree branched off the feature branch and merge back. Sequence only where a slice genuinely depends on another's output (backend-first narrow, SPA-wide after worked well for #216).
- BATCH YOUR PUSHES — you cannot cancel a CI run (operator-requested, 2026-07-17). Hold review
fixes, doc corrections and format fixes locally, verify, and push once; each push spawns a run
that orphans the previous one on a 4-slot runner. Cancellation is impossible from the agent side on
this Gitea (1.25.4):
POST /api/v1/.../actions/runs/{id}/cancel→ 404, MCPactions_run_writecancel_run→ 404, and the web-UI route needs a session+CSRF that does not script (login 303s, no session cookie). Only the operator can cancel, in the browser — so if you must supersede a live run, SAY SO explicitly instead of leaving it burning. (dispatch_workflowis a different route and still works for re-triggering a main run.) Corrects the older "superseded runs drain on their own" note below: they do finish, but they hold a slot while doing it. - A lone
decisions.md append-onlyred is a KNOWN INFRA FLAKE — do NOTHING (operator-stated, 2026-07-19). When it is the only red: do not investigate it, and do not create a new run or commit to clear it — no rebase, no--amend, no no-op push. The operator reruns that single job from the Gitea UI. Report it as a known flake and carry on; only if other jobs are red too does the run deserve diagnosis. Same family as the killed-job rule below (a spurious single-job red is cleared by the operator's selective rerun, never by pushing — and pushes can't be cancelled, see the batching constraint above). The trap is that a convincing local explanation is always available. On #473/PR #479 the job went red just aftermainlanded its owndecisions.mdentry, so "mine is no longer at EOF, I must rebase" looked airtight — I rebased, and it went red again on a head whose diff was a verified pure EOF append with zero deleted lines. A rebase that provably satisfied the gate's stated rule did not turn it green, which is the proof the red was never about the diff. Check this rule before theorising about the cause. - Do NOT gate or throttle your push on host health — trust the Gitea build queue. The runners were retuned for stability (operator, 2026-07-17); queueing is the queue's job, not yours. Don't SSH to bumblebee to sample load/RAM before pushing, and don't hand-schedule around other sessions' runs. Batch your pushes (above) because orphaned runs can't be cancelled — not because the host needs protecting.
- An infra-shaped red under host pressure is not a code failure — check the host before diagnosing the
diff (2026-07-17). While bumblebee was saturated, two unrelated jobs died in ways that look like code
bugs:
EF migration integrityfailed in ~11ms insideSetup .NETwithremove /root/.cache/act/…/.gitignore: no such file or directory(act's shared cache), and main'sBuild & push imagedied on buildxDeadlineExceeded: failed to compute cache keyafter ~210s of retries. Both re-ran green once the host recovered; the migration job had no model change to test in the first place. Tell: the failure is in a setup/cache step, before your code compiles. Don't file a CI bug off one sample under pressure — you'll send the next session chasing a phantom (this nearly got filed as #390 lane-rebalance fallout, which the evidence did not support). - Before any push touching
.cs, check the touched set for a UTF-8 BOM. The #311 gate is fix-as-you-touch: a legacy file's inherited BOM becomes your problem the moment you touch it, andcharset=utf-8means no BOM.for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q "^efbbbf" && echo "BOM: $f"; doneStrip any hit (1-line diff, zero content change). Bit two sessions on 2026-07-17 — PR #405 (6 files) and #70/PR #402 (19 files, where Pythonio.open(..., encoding='utf-8-sig')wrote BOMs back while "preserving existing style"). Verify your detector: anod -An -c | grep '357 273 277'check reported all-clean while 19 files were dirty — a detector that can only say "ok" is worse than none. Use thexxdform or read bytes directly.dotnet format --includeDOES work here (corrected 2026-07-17, #70/PR #402 — an earlier note claiming it silently no-ops was WRONG). The apparent no-op was the shell: CI's recipe usesmapfile, which is bash-only, and the default shell here is zsh → empty array → zero files → exit 0. Run it underbash -c:bash -c 'mapfile -t files < <(git diff --name-only --diff-filter=ACM origin/main...HEAD -- "*.cs") dotnet format ErsatzTV.sln --no-restore --verify-no-changes --include "${files[@]}"' # exit 0 = gate passes - Merge consent is DERIVED, not asked: the H6/H10 gate (
pretooluse-merge-consent.sh) auto-grants a merge when the linked issue's## Done-whenboxes are all ticked, a fresh positiveReview-verdictreferences the current head, and CI is green — no conversational confirmation on that satisfied path (ersatztv#314). Confirm in-conversation only when the gate ASKS (state not derivable: no creds, no linked issue, no Done-when/verdict, Gitea down). Prior pre-approvals still do NOT carry over. - Local gate + cold review run BEFORE the push to Gitea CI, not after (Timothy, 2026-07-17): run the local build/test gate AND a cold-context adversarial review agent (SCOPED "review only") over the diff, fold any fixes, and only THEN push / open the PR. Gitea runners should only ever see already-validated work — CI is a scarce shared slow resource and a run cannot be cancelled on this Gitea, so spending a run to surface a lint error or a review-caught bug is pure waste + queue pressure. CI (armed at PR-open, below) confirms; it is not the first line of defence.
- Arm a CI monitor on the PR head sha AT PR-OPEN (commit-status endpoint), not at the end.
- Live-E2E via scripts/e2e-local.sh (also before the push for write-path / UI changes); NEVER exercise download endpoints via browser tabs (curl them).
Lessons / workflow lore (STANDING workflow/orchestration rules only)
Scope: this section is how we work (orchestration, CI, review, cross-session hygiene) — not engineering decisions. Every "why the code does X" decision lives in
docs/decisions.md(+api-conventions.md§7a/b/c for the concurrency/side-effect mechanics,spa-conventions.md,e2e-local.md). Do NOT re-record a code/API/SPA decision here — it will duplicate and drift. Pruned 2026-07-12: issue-specific engineering bullets now covered in those docs, and dead Blazor-parity process notes, were removed (git history has them).
-
READ docs/README.md → the convention docs FIRST; point recon/implementer agents at specific doc sections. Only recon the task-specific delta.
-
The mechanical selection checks now live in
scripts/select-queue.sh— run it instead of re-deriving them by hand (2026-07-19). The three bullets below are the evidence for why the script exists: a cheap model repeatedly reported blocked issues asdeps:clear, mis-tiered on the no-op?milestones=filter, and ranked by issue number instead of priority-within-tier — so the lore grew to "re-derive the whole contested tier every session," which is exactly the tax the script pays down. The script does the deterministic work in code (dependency exclusion viaGET /dependencies, LOCAL.milestone.state/review/priority:tiering, (tier, priority, issue#) ordering,in-progress/parked/PR exclusion) and only flagsCLAIM?/UMBRELLA?for judgment. Trust its DEPS/tiering/ordering; do the focused recheck only on the flagged rows. Keep the evidence bullets below (they explain the failure modes the script encodes), but the prescription is now "run the script," not "hand-re-derive." If you extend the tiers/filters, update the script first, then these bullets. -
Gitea milestone-name issue filter silently NO-OPS on names with special chars (2026-07-17, #77 selection):
GET /issues?milestones=<name>returns the WHOLE open-issue list — not a filtered set — when the milestone title contains:or+(e.g.Scheduling: refactor + distribution), because the raw name doesn't round-trip through Gitea's filter. A selector that trusts that response will mis-tier issues (it made #72 look like a milestone-10 member when it's unmilestoned, and hid #77's true sibling set). Robust recipe: fetch all open issues once and filter LOCALLY on each issue's.milestone.title(?state=open&type=issues&limit=50, then[i for i in issues if (i['milestone'] or {}).get('title')==NAME]) — do not rely on the server-sidemilestones=name filter. If a count looks suspiciously like "all open issues," that filter silently failed; re-derive membership locally. (Dependencies API is fine:POST /issues/{n}/dependencieswith{"owner","repo","index"}sets blocked-by; the bare{index}form 201s but may not attach — verify with the GET.) -
Confirmed again + widened (2026-07-17, #72/#164 selection): treat EVERY mechanical claim in the selector packet as unverified — not just deps and priority. That run's dependency data was correct, yet it still ranked #383 first while missing that #384 was
in-progress(claimed 40 minutes earlier) and that #383 is that bundle's umbrella (its body: "Follow-up arc to #69" with #384–#386 as deferred sub-work). Claiming its #1 pick would have collided head-on with a live parallel session. The packet does not reliably see:blocked-by, priority-within-tier,in-progressclaim state, or umbrella-vs-child. Before claiming, check the candidate's last comments for a live claim (a claim stays live until explicitly released) and read its body for umbrella framing. An empty-ish tier is normal in maintenance mode — three of the four open milestone issues that day were legitimately blocked or claimed, which is the expected steady state, not a reason to relax the checks. -
The cheap selector's dependency + priority data is NOT trustworthy — re-derive the whole contested tier, not just the winner (2026-07-17, #73 selection): the selector packet reported #385/#386 as
deps:clearwhen both were blocked (#385 by open #70; #386 by open #385+#384) — it never calledGET /issues/{n}/dependencies— and missed #383 entirely (the onlypriority: mediumin tier 1), ranking by issue number and ignoring the priority-within-tier rule. Either error alone produces a bad pickup: working a blocked issue, or skipping the top-priority one. The kickoff's "orchestrator makes one focused recheck of the winner" is insufficient whenever a tier has >1 candidate — a wrong winner is invisible to a winner-only recheck. Recipe: fetch all open issues once, filter locally on.milestone.title/labels (the?milestones=filter no-ops on:/+names — see the bullet above; both current open milestones have them), then runGET /issues/{n}/dependenciesyourself on every candidate you'd plausibly claim. That's ~3 cheap calls and it is the only thing standing between the packet and a wasted session. Keep delegating selection (it's still not orchestrator work) — just verify its two mechanical outputs (deps, priority order) rather than its ranking prose. -
Codex cheap-worker launch (tested 2026-07-14): the native
spawn_agentinterface currently has no model/effort selector, so it provides parallelism but not cost savings. For bounded, tool-bearing selector/recon work, launch a separate worker withcodex exec --ephemeral --color never -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only -C <repo> "<hard-capped scope contract>". GPT-5.4-Mini is the current supported small/cost-efficient profile on this ChatGPT-authenticated host; the oldergpt-5.1-codex-miniguess fails with HTTP 400. Keeplowwhenever any shell/MCP/web tool is exposed; reserveminimalfor explicitly tool-free synthesis over already-supplied bounded evidence. If the model rolls over, inspect the visible entries in~/.codex/models_cache.jsoninstead of guessing names, then update this dated note once verified. -
Keep the MAIN checkout's
web/node_modulesfresh (npm installafter pulling a PR that adds a dep) — worktrees copy it, and a stale copy broke typecheck in a #198 worktree. -
Playwright-MCP E2E: never open tabs/window.open for file-download endpoints — curl them. If browser tools stall repeatedly,
pkill -f ms-playwright-mcpand drive a fresh session. -
THERE IS NO "main checkout".
/Users/timothy/ersatztvis a shared mutable working tree whose HEAD is whatever the last session left there — it is NOT amainmirror, and its name lies. Treat it as read-only, and never as a source of truth:- Never commit in it — not even a one-line docs fix. Need a commit on
main? Make a worktree offorigin/main(git worktree add -b <branch> <path> origin/main) and commit there. - Never read
git log/git status/HEADthere and infer anything aboutmain— you will be reading some other session's branch. origin/mainis the only source of truth.git fetch origin mainand referenceorigin/mainexplicitly. Do not trust the localmainref either (it lags; nobody is obliged to update it).- Its one sanctioned use is read-only: copying
web/node_modulesinto your worktree (keep it fresh — a stale copy broke typecheck in a #198 worktree).
This is a design flaw, not a discipline failure — do not "fix" it by telling sessions to check harder. A check does not stay true. Worktree discipline is healthy (10 feature worktrees on 2026-07-17: Claude under
.claude/worktrees/, Codex under.worktrees/, some in/private/tmp/); the shared tree is the single place the model breaks, and it broke for two different sessions on the same day, both doing the obvious thing:- Session A assumed it was on
main(true at 12:46 when it pulled — correct when made, stale by 14:17) and committed → the commit landed on the #604/CI session'sci/604-lane-rebalance, 24 seconds after that session's own commit at 14:17:28. Two writers, one tree. Nothing tangled, by luck. - Session B ran
git logthere, readci/604-lane-rebalance's HEAD, and concludedmainwas "4 behind origin" — a phantom.
A session-start
git statusappears to confirm the false assumption, which makes it worse than a plainly wrong claim. (Supersedes the old "the user's main checkout stays ON main and pulled each session" note.) Same family as the sibling bullet below: never commit inside a checkout or worktree you didn't create. - Never commit in it — not even a one-line docs fix. Need a commit on
-
Subagent connection drops = laptop sleep/transient; re-resume via SendMessage (work survives).
-
Build/verify (and review) locally, then trust it; CI (VM 127) confirms (~9 min warm; seconds for a docs-only PR since ersatztv#416). Watch by commit status:
/api/v1/repos/timothy/ersatztv/commits/{sha}/status. -
PR routine (works): worktree off origin/main → implement (strongest tier for judgment-heavy / balanced tier for mechanical work when model routing is available; doc pointers + exact facts; use the available C# language tooling) → API changes: build app project FIRST then
./scripts/update-openapi.shthennpm run generate:api→ full local test pass + cold-context adversarial review (SCOPED "review only") + live-E2E for UI/write-path changes (scripts/e2e-local.sh), ALL BEFORE the push — fold pre-push review fixes into the branch (amend/reword is fine while nothing is pushed) → push, PR (arm the CI monitor at open) → CI confirms → any further review fixes (a post-push cross-model pass, or CI-surfaced issues) as follow-up COMMITS (never amend/force-push a pushed branch) → CI green → merge (derived consent gate) → structured close comments per the applicableAGENTS.md/CLAUDE.mdprotocol. -
Never run TWO committing agents on one worktree concurrently. Read-only review agents are fine.
-
A KILLED job reports
conclusion: failure— the conclusion field is not enough. Read the log tail. (2026-07-17.) When the operator restarts the runners (a retune, a reboot), in-flight jobs die and Gitea marks themfailure, notcancelled. Tell: the log stops mid-step with no error and no❌ Failure - Main …marker — a real failure always leaves a marker. Run 1006'sEF migration integrity+Functional E2E"failed" on a BOM-removal-only diff (6 files, 1 line each, zero content change) that could not possibly break them; the logs simply truncated mid-dotnet buildat 12:08 UTC, and both runners showedUp About an hour— the retune had killed them. Log timestamps are UTC; the host is UTC+2 — convert before correlating, or the restart looks 2h off and the theory gets wrongly discarded. Corollary: a diff that cannot cause a failure is evidence the failure isn't yours — when a job that passed on the previous head fails on a semantically-null delta, suspect the environment and go read the log, don't re-litigate the diff. Also beware log warnings that look fatal:Unable to pull refs/heads/v4: …is act refreshing its/root/.cache/actaction cache and is followed byCloned …— it is NOISE, not a cause. Grep for the failure marker, not for the word "error". -
cancelledis NOTfailure— never read a cancel as a CI verdict (2026-07-17, operator-surfaced). The operator cancels runs by hand (they're the only one who can — see the batching constraint), and a run-levelconclusion: cancelledmeans no verdict, not a pass and not a fail. Two traps: (1) a run whose overall state isfailuremay hold a genuine job failure that happened before the cancel — check job-levelconclusion+ timestamps, don't attribute it to the cancel; (2) acancelledrun tells you nothing about your diff, so don't claim green on it. Report FAILED and CANCELLED counts separately in any CI monitor:fail=[j for j in jobs if j['conclusion']=='failure']; canc=[j for j in jobs if j['conclusion']=='cancelled']— a monitor that only asks "is state != pending" will call a cancelled run a failure and send you debugging a phantom. -
An infra-shaped red under host load is not a code failure — check
uptimeon bumblebee BEFORE diagnosing the diff (2026-07-17). At load average 243, two unrelated jobs died in ways that look like code bugs but aren't:EF migration integrityfailed in ~11ms insideSetup .NETwithremove /root/.cache/act/…/.gitignore: no such file or directory(act's shared cache), and main'sBuild & push imagedied on buildxDeadlineExceeded: failed to compute cache keyafter ~210s of retries. Both re-ran green at normal load; the migration job had no model change to test in the first place. Tell: the failure is in a setup/cache step, before your code compiles. Don't file a CI bug off one sample under load — you'll send the next session chasing a phantom (this nearly got filed as #390 lane-rebalance fallout, which the evidence did not support). A game server (FactoryServer-L) shares that host with the runners, so load isn't always CI's doing. -
Arm a CI monitor the moment the PR opens (commit-status endpoint), not at the end — in the #216 session three PR runs sat red ~an hour on a CI-only flake while review/E2E ran. (The old MySQL host-port 3306 collision is fixed on main,
ef8915f1; a lone MySQL-apply red now = the known infra flake, rerun — see MEMORY /ci-migration-job-mysql-flake.) -
"Make X consistent with Y" review findings deserve new-code scrutiny — a #216 review fix propagated MediaBrowseScreen's mount-debounce selection-wipe race to SearchScreen instead of catching it. For any timer/effect, ask "when does this fire?" (incl. mount).
-
Live-E2E seeding recipe now lives in
docs/e2e-local.md→ "Seeding a local TV library for E2E" (tiny ffmpeg testsrc MKVs +LibraryPathSQL rows + scan). Live-E2E is a stated requirement for write-path handler changes — see that doc's "When live-E2E is required" + the decisions.md entry; it's the only net for the lazy-enumeration / reload-through-read-path 500 class (#229). -
Parallel sessions (2026-07-11 protocol): claim before working (
in-progresslabel — the tiny read→claim race window is accepted; later claimant backs off). Claiming prevents duplicate pickup, NOT overlapping code changes — check the tracker's dependency notes ("#234 after #231", "coordinate with #215") before touching shared surfaces. Editing THIS lore: prune covered/stale bullets too (not append-only — git keeps history),git pull --rebasebefore commit. Two runners (ci-runner VM 127 + bumblebee-runner, 4 slots); no cancel route exists on this Gitea version — verified 1.25.4: REST and MCPcancel_runboth 404 (see the batching constraint above). Superseded runs do finish on their own, but they hold a runner slot while doing it, so with 4 parallel sessions they are a real cost — batch pushes rather than relying on them to drain. -
Two sessions touching one machine: a branch may be checked out in ANOTHER session's worktree — never commit/merge inside a worktree you didn't create. To land a merge on such a branch without touching their checkout: plumbing merge (
git read-tree -m base ours theirsinto a temp GIT_INDEX_FILE →write-tree→commit-tree -p ours -p theirs→ push the commit to the branch ref); the owning session thengit pulls. -
Cross-model independent review rotation (2026-07-11): use the other client/model family when it is available — Codex reviews Claude-authored changes; Claude reviews Codex-authored changes. Use whatever native integration is actually installed; client-specific helpers such as
codex:rescueare optional accelerators, not protocol dependencies. On quota/tool exhaustion, use a cold-context review-only agent in the current client and retry cross-model review next window. Independent reviewers do review and stuck-diagnosis work, not implementation on the same change. Skip rubric (self-exemption is an auditable claim, not a vibe): an independent review pass (preferably cross-model; otherwise a cold agent) is mandatory for any diff touching locks/concurrency, auth/security, API write-path handlers, or DB migrations, or larger than ~150 changed C# lines. It is skippable only for a pure-SPA/docs leaf with no server-state effect — and when you skip it, say so and why in the PR or close comment (a stated exemption, not silence). This is the correlated-blindspot net: independent model families miss different classes (see the #235 "enumerate every producer" and #250 lessons), so a single reviewer on a risky diff is the gap. -
Force enumeration of lazy LanguageExt returns in tests (2026-07-11, PR #229): a write-path 500 survived a green fixed-point test because the handler returned a lazy
Mapthe test never enumerated. Enumerate handler return values in tests; live E2E remains the only net for this class. (Write-path reload-through-read-path mechanics: api-conventions §7.) -
Keep a PR branch current by REBASING on origin/main — do NOT merge main in (ersatztv#311, enforced by the H11 pre-push hook: a push from a branch behind origin/main is blocked →
git fetch origin main && git rebase origin/main). A merge commit pulls in every file main changed, incl. files you never touched (e.g. the ~2500 legacy-BOM.cs), which then trip the format hook/CI on code that isn't yours (the #309 session). Rebasing keeps your diff to exactly what you changed. After a rebase that hits conflicts in generated artifacts (v1.json, v1.d.ts, endpoint-index), REGENERATE them (./scripts/update-openapi.sh+npm run generate:api) — never hand-resolve; git text-merges them plausibly-but-wrong andnpm run check:apiis the guard. Escape hatch for a deliberate non-rebased push:ETV_SKIP_REBASE_CHECK=1 git push. -
H12 issue-qualification audit (ersatztv#312,
scripts/issue-qualification-audit.sh): a session-end check that lists OPEN issues missing apriority:label — the #237 ranking keys offpriority:/gate labels, so an unlabeled issue is invisible to it. "Fully qualified" = has apriority: {high,medium,low}label (that signals triage ran; gate-vs-backlog is then derivable from thereviewlabel / milestone, and a milestone is NOT required — backlog is unmilestoned). Run it at session end and label anything it flags (esp. issues you filed this session). Fail-open without creds; advisory (exit 1 when any are unqualified). Sibling to H11 (both #311/#312). -
CI VM test timeouts: heavy-render web tests (100+ item grids) need explicit vitest timeouts (e.g. 15s) — the CI VM hit the 5s default on a test that runs in ~1s locally (run 686). Bump per-test, don't raise the global default.
-
Concurrency tests need a negative control + a stress loop, not a single round (2026-07-11, #231): a one-shot
Barrier(N)+Task.WhenAll"exactly one winner" test did NOT catch a deliberately non-atomic check-then-set flag on this Mac — the load→store window is too narrow to collide when threads release once. It green-lit broken code. Fix: hammer the race over many rounds (N worker threads × ~20k rounds, two Barriers per round: acquire → validate one-winner → reset), and ALWAYS prove non-vacuous by temporarily breaking the primitive and watching the test fail. Cheap (sub-second) and deterministic-enough. Same discipline as #12's negative control. -
Before deleting a "workaround", check it isn't serving a SECOND purpose (2026-07-11, #232): an issue said "remove the SPA
PENDING_GRACE_TICKSgrace-tick workaround now the API is honest". The grace ticks did double duty — compensating the lying-200 (now genuinely obsolete) AND absorbing the inherent queue→observed-active lag / fast-completion race (still needed). Removing wholesale wedged the scan button disabled-until-reload for short scans; both fork + Codex caught it. Keep a bounded version, re-scoped honestly. When an issue says "delete X", enumerate every behavior X provided before deleting. -
Re-review the FIX COMMIT, not just the initial PR diff, before closing (2026-07-11, #242, user-surfaced): the pre-fix review is only half the loop — the fix commit is new code no independent reviewer has seen. Here the first fix (gate Edit on
saving) only PARTIALLY closed the in-flight-save clobber class; a Codex re-review of the fix commit caught that Delete was the one schedule-switch path (applySwitch) still missing the guard. Fold a scoped re-review (cross-model if quota, else a cold review agent) over the fix commit into the close protocol; loop until a clean MERGEABLE verdict or an explicit acceptable-defer (file a tracked issue for the deferral, as #248 here). Cheap relative to shipping a half-fix that reopens the issue. -
Negative-controlling a lock/flag fix: invert the condition, don't stub
if (true)(2026-07-11, #250): to prove a lock-release test is non-vacuous I temporarily broke the fix. First attempt made the guardif (true)— but that left the ownership flag "assigned but never read" → CS0219 under warnings-as-errors → the Application build silently FAILED →dotnet test --no-buildran the STALE (fixed) dll → false PASS that looked like the control had failed to catch the bug. The tell: build output must showBuild succeeded/0 Error(s), not just a truncated tail. Break the primitive by inverting the real condition (if (!lockAcquired)) so the flag stays read and the code compiles; always grep the build forerror CSbefore trusting a--no-buildtest result. (Codex-found High this session: an outercatchthat unlocked unconditionally cross-released another session's lock on a pre-acquisition exception — the fix was a Handle-scoped ownership flag gating the catch.) -
A lock/channel-ownership "no cross-release" verdict must enumerate EVERY producer of that message type, not just the enqueuers visible in the PR diff (2026-07-11, #235/#267). The cold fork cleared the new
scan-collectionsper-provider lock as sound — "only the acquirer enqueues, so only its message triggers the scanner unlock" — because it checked the three new controllers but never grepped for ALLSynchronize{X}Collections(senders. Codex did:SchedulerServicealso enqueues them periodically and unlocked, andScannerService'sfinallyreleased the lock whenever held → a scheduled scan cross-releases an API scan's lock (#250 class). Rule: before trusting any "single owner / no double release / no cross-release" claim,grepthe whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer. This is why the independent Codex pass earns its keep alongside the fork even when the fork returns MERGEABLE. -
Independent-review disagreement on a gate PR → frontier judge; judge the fix, not just the verdict (2026-07-11, #235): the reconciliation isn't only "who's right" — Fable historically ratified Codex's High AND caught that the orchestrator's proposed fix (per-source lock-or-skip) would deterministically starve the 2nd+ media source (fixed loop order), prescribing the shipped library-scan §C7 pattern instead (lock once per provider, batch-enqueue, release on the
Unlock: isLastmessage). Escalate the FIX approach too, not just the finding. -
Harden a soon-to-grow feature with a runtime posture/toggle, not a hardcoded clamp (2026-07-12, #289): the MCP's final design (#58) is read+write, so a GET-clamp would be ripped out — shipped a runtime read-only posture (
ERSATZTV_ALLOW_WRITES, default false) instead. When a security fix constrains a capability the roadmap will later want, make the safe state the default of a switch, not a wall; check the feature's issue for its end-state first. -
Fixing onto an unmerged feature branch: branch off it, land by FF-push, never touch a sibling worktree that has it checked out (2026-07-12, #289 → PR#76). Gotcha: recon by
Reading a sibling worktree leaked its paths into my laterEdits → they landed there, not the fresh worktree (dotnetbuilt the unchanged tree → "0 new tests" tipped it off). After creating a worktree, do the first Edit/Read against ITS absolute paths andgit statusit before building. -
HttpCompletionOption.ResponseHeadersReadmoves the body read OUTSIDEHttpClient.Timeout(2026-07-12, #289): with aNoneread token a slow-drip upstream hangs forever. Fix = per-request linkedCancellationTokenSource.CancelAfter(timeout)into BOTHSendAsyncand the stream reads +HttpClient.Timeout = InfiniteTimeSpan; catch transport/timeout exceptions and turn them into a response. (Not yet in decisions.md — #289 landed on PR#76, not main.) -
Queue-drift root cause + standing rule (2026-07-12, user-surfaced): pickups repeatedly re-picked already-done work — #91b was framed "recommended next / now unblocked" for two sessions after it had merged (2026-07-11), and the #251/#252 priority-pickups sat listed "open" after closing — requiring a reactive body correction (tracker comment 16:00). ROOT CAUSE: DONE/OPEN status was read from prose (the arc body's inline "DONE" markers + each session comment's "Recommended next"), which is append-only and hand-edited, so it lags real issue state — worst under parallel sessions that narrate each other's merges as "the gate that unblocks X" (#271 got cast as the last gate for an already-merged #91b). It is a structural bug, not a stale-writer bug: any status embedded in prose will drift. STANDING FIX (don't just re-patch the body next time): live Gitea state is the ONLY source of truth for status. The arc body carries order+goal; a comment's "Recommended next" is a candidate that MUST be re-verified OPEN (issue and milestone) at pickup. When prose disagrees with live state, live state wins — correct the prose in your session comment, never propagate it. STRUCTURAL CURE (2026-07-12 review, Fable): #237's arc no longer carries inline
DONEmarkers — closed items move to a "Done (history)" section, so status lives ONLY in live Gitea state and can't drift; and the gate/frontier is defined structurally (lowest-numbered open arc item), never hardcoded to an issue number. Beware: the first pass at this fix re-planted the very bug by hardcoding "#197 cluster" / "#91b milestone CLOSED" into the kickoff — if you name today's frontier issue in this standing file, you are writing the next drift. Gate cluster = openreview-labeled issues serving the current open arc item.