Kickoff: make per-agent model routing a hard constraint + a PreToolUse gate #583
Closed
opened 2026-07-25 11:01:34 +02:00 by timothy
·
1 comment
No Branch/Tag Specified
main
renovate/dotnet-monorepo
renovate/meziantou.analyzer-3.x
renovate/lucene.net
renovate/sqlitepclraw.bundle_e_sqlite3-3.x
renovate/cliwrap-3.x
913-scripted-playout-401
release/v26.15.0-notes
issue-806-guard-populations
scratch/767b-poisoned
scratch/767b-control
release/v26.14.0-notes
release/v26.14.0
docs/510-skill-logo-bug-policy
fix/510-watermark-resolution-policy
fix/629-verdict-classifier-falseopens
fix/609-decisions-edit-token-scope
issue-135-clear-to-none
release/v26.12.0-notes
fix/409b-lastscan-api-parity
fix/401-updatechannel-mirror-422
fix/327-playlist-rename-validation
fix/410-scancancel-log-level
fix/409-447-librariesscreen-neverscanned
fix/338-zap-exit-code
fix/367-plex-budget-message
fix/310-debom-legacy-cs
ci/604-lane-rebalance
feat/388-design-mirror
feat/247-test-ownership
feat/247-primary-action
feat/357-player-owned-playback
feat/357-jellyfin-plugin-poc
fix/289-mcp-hardening
issue58-mcp
feat/244-channels-extract
ci/auto-bump-prod-compose
feat/multi-rerun-collections-api
feat/collections-api
feat/quick-wins
feat/185-docs-part2
feat/140-collections-screen
feat/146-channel-edit
feat/147-classic-ui-link
issue22-renovate-dashboard
feat/91-cutover
feat/63-composite-create
feat/65-library-browse
feat/85-epg
feat/86-schedule-editor
feat/109-dashboard-data
feat/99-session-tracking
fix/dockerfile-node-tag
feat/59-spa-foundation
docs/59-ui-redesign-brief
feat/102-json-guide
feat/111-schedule-durations
feat/104-artwork-upload
feat/103-media-sources-api
feat/playouts-read-api
feat/108-health-api
feat/105-picker-list-endpoints
issue-97-channel-state-api
issue42-jellyfin-musicvideos
issue46-rest-api-error-contract
dependabot/nuget/ErsatzTV.FFmpeg.Tests/multi-d307a2e06f
qsv-improvements
hdr-vulkan-cuda-test
v26.16.0
v26.15.0
v26.14.0
v26.13.0
v26.12.0
v26.11.0
v26.10.0
v26.9.0
v26.8.0
v26.7.0
blazor-final
v26.6.0
v26.5.0
v26.4.0
v26.3.1
v26.3.0
v26.2.0
v26.1.1
v26.1.0
v25.9.0
v25.8.0
v25.7.1
v25.7.0
v25.6.0
v25.5.0
v25.4.0
v25.3.1
v25.3.0
v25.2.0
v25.1.0
v0.8.8-beta
v0.8.7-beta
v0.8.6-beta
v0.8.5-beta
v0.8.4-beta
v0.8.3-beta
v0.8.2-beta
v0.8.1-beta
v0.8.0-beta
v0.7.9-beta
v0.7.8-beta
v0.7.7-beta
v0.7.6-beta
v0.7.5-beta
v0.7.4-beta
v0.7.3-beta
v0.7.2-beta
v0.7.1-beta
v0.7.0-beta
v0.6.9-beta
v0.6.8-beta
v0.6.7-beta
v0.6.6-beta
v0.6.5-beta
v0.6.4-beta
v0.6.3-beta
v0.6.2-beta
v0.6.1-beta
v0.6.0-beta
v0.5.8-beta
v0.5.7-beta
v0.5.6-beta
v0.5.5-beta
v0.5.4-beta
v0.5.3-beta
v0.5.2-beta
v0.5.1-beta
v0.5.0-beta
v0.4.5-alpha
v0.4.4-alpha
v0.4.3-alpha
v0.4.2-alpha
v0.4.1-alpha
v0.4.0-alpha
v0.3.8-alpha
v0.3.7-alpha
develop
v0.3.6-alpha
v0.3.5-alpha
v0.3.4-alpha
v0.3.3-alpha
v0.3.2-alpha
v0.3.1-alpha
v0.3.0-alpha
v0.2.5-alpha
v0.2.4-alpha
v0.2.3-alpha
v0.2.2-alpha
v0.2.1-alpha
v0.2.0-alpha
v0.1.5-alpha
v0.1.4-alpha
v0.1.3-alpha
v0.1.2-alpha
v0.1.1-alpha
v0.1.0-alpha
v0.0.62-alpha
v0.0.61-alpha
v0.0.60-alpha
v0.0.59-alpha
v0.0.58-alpha
v0.0.57-alpha
v0.0.56-alpha
v0.0.55-alpha
v0.0.54-alpha
v0.0.53-alpha
v0.0.52-alpha
v0.0.51-alpha
v0.0.50-alpha
v0.0.49-prealpha
v0.0.48-prealpha
v0.0.47-prealpha
v0.0.46-prealpha
v0.0.45-prealpha
v0.0.44-prealpha
v0.0.43-prealpha
v0.0.42-prealpha
v0.0.41-prealpha
v0.0.40-prealpha
v0.0.39-prealpha
v0.0.38-prealpha
v0.0.37-prealpha
v0.0.36-prealpha
v0.0.35-prealpha
v0.0.34-prealpha
v0.0.33-prealpha
v0.0.32-prealpha
v0.0.31-prealpha
v0.0.30-prealpha
v0.0.29-prealpha
v0.0.28-prealpha
v0.0.27-prealpha
v0.0.26-prealpha
v0.0.25-prealpha
v0.0.24-prealpha
v0.0.23-prealpha
v0.0.22-prealpha
v0.0.21-prealpha
v0.0.20-prealpha
v0.0.19-prealpha
v0.0.18-prealpha
v0.0.17-prealpha
v0.0.16-prealpha
v0.0.15-prealpha
v0.0.14-prealpha
v0.0.13-prealpha
v0.0.12-prealpha
v0.0.11-prealpha
v0.0.10-prealpha
v0.0.9-prealpha
v0.0.8-prealpha
v0.0.7-prealpha
v0.0.6-prealpha
v0.0.5-prealpha
v0.0.4-prealpha
v0.0.3-prealpha
v0.0.2-prealpha
v0.0.1-prealpha
Labels
Clear labels
ad-hoc
api
bug
ci-cd
content
dependencies
enhancement
frontend
in-progress
jellyfin
parked
priority: high
priority: low
priority: medium
review
security
One-off / ad-hoc work not tracked by a dedicated issue
REST API / HTTP endpoints
Something isn't working
Build, test, deploy pipeline
Channel content / schedules / playlists
Dependency updates (Renovate)
New feature or improvement
ChicoryTV React SPA frontend
Claimed by an active session — do not pick up
Jellyfin tuner / IPTV integration
Excluded from automatic queue pickup; work only when explicitly selected
Adversarial review finding
Security / vulnerability fix
Milestone
No items
No Milestone
Projects
Clear projects
No projects
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: timothy/ersatztv#583
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
The kickoff prompt (
docs/handoffs/chicorytv-issue-queue.md) tells the orchestrator to route by capability — fast/small for bounded recon, balanced for mechanical changes, strongest tier for judgment-heavy work. On 2026-07-25 a session dispatched two implementer agents (#436, #440) withmodelomitted on bothAgentcalls, so both silently inherited the Opus orchestrator tier. #440 was a mechanical SPA slice against an already-shipped backend contract and was a plausible Sonnet candidate.Root cause
Not forgetfulness — a structural asymmetry:
HARD CONSTRAINTS(worktree-off-main, one-committing-agent-per-worktree, parallelize, local-gate-before-push, batch-pushes) was followed in that same session. The one rule in prose was the one that got defaulted.modelproduces no artifact — nothing in the session report revealed it; the operator had to ask.Change
HARD CONSTRAINTSbullet requiring the tier to be stated in the dispatch message (converts an invisible omission into visible output), and tighten the prose paragraph to point at the key instead of restating it — the kickoff is pasted into every session, so duplication is a per-session tax (#542)..claude/hooks/pretooluse-agent-model.sh: PreToolUse onAgent,askwhen a dispatch names nomodel.process.per-agent-model-routing.Design revised under review. The first cut gated only implementer-looking dispatches via a prompt-text heuristic. Independent review confirmed it both over- and under-fired — notably "author the change and open a PR" passed silently, missing the exact case it existed to catch — and that its "read-only types cannot commit" rationale was false (
Explore/Plan/claude-code-guideall carry Bash). Root cause: the hook contradicted its own rule ("every dispatched agent"). Inverted to ask whenevermodelis absent, exempting onlyfork(whosemodeloverride the tool ignores). Prompt prose is not a reliable authority signal.Done-when
HARD CONSTRAINTSbullet added + prose paragraph tightened (no duplicated table)Agent|TaskPreToolUse matcher, and its decision matrix verified against real payloads (including a replay of the 2026-07-25 dispatch that missed)process.per-agent-model-routingadded + catalog regeneratedClosing record
Outcome: Shipped via PR #585 (merged, head
7e5d20be). Per-agent model routing is now a keyedHARD CONSTRAINTSbullet requiring the tier to be stated in the dispatch itself, backed by.claude/hooks/pretooluse-agent-model.sh— a PreToolUse gate onAgentthatasks whenevermodelis absent (exempting onlyfork, whosemodeloverride the tool ignores). The prose paragraph was tightened to point at the key rather than restate the table.Root cause: Not forgetfulness — a structural asymmetry. Every bullet in
HARD CONSTRAINTSwas followed in the same session that missed this (worktree offorigin/main, one committing agent per worktree, parallelize, local gate before push, batch pushes). Routing was the one rule living only in a prose paragraph. Three aggravating factors: the low-cost-routing paragraph is written entirely about queue selection and recon and never names implementers, so the largest-cost dispatch fell in a scope gap; omittingmodelis the silent path, producing no artifact (nothing in the session report revealed it — the operator had to ask); and the rule sits ~line 84 while dispatch happens much later, after orientation/claiming/bundle-scan.Decisions/conventions changed: Added
process.per-agent-model-routing(2026-07-25) indocs/decisions/workflow-process.md. No supersession.Reusable knowledge:
git commit,worktree,fixes #). Review confirmed it both over-fired (a read-only recon brief mentioning "worktree") and under-fired —"author the change and open a PR","land this on the branch"and"make the changes and commit them"all passed silently, missing exactly the case it existed to catch. A gate with an unreliable catch rate is worse than none, because it gets trusted.Explore,Planandclaude-code-guideall carry Bash — the "read-only agents cannot commit" premise behind the original exemption list was simply false.Verification: Decision matrix exercised against 11 crafted payloads including a verbatim replay of the 2026-07-25 dispatch that missed, all three review findings, the invariants (model named → never fires;
forkexempt; non-Agentpasses) and robustness (malformed JSON, empty payload, missingtool_input, embedded backticks/quotes/newlines, 2MB prompt,$(...)injection probe on thelabelinterpolation). Never crashes, never blocks, never emits malformed JSON (jq -n --argthroughout). Two independent review rounds (initial + re-review of the fix commit);decisions-validate: OK; CI green on7e5d20be.Deferred: One accepted non-blocking nit — the broad design fires on every model-less dispatch, so an operator could habituate to reflex-approving without reading the reason, eroding the gate without formally disabling it (harder to detect than a kill-switch). The record does not document a fallback posture. Accepted deliberately; if noise materializes in practice, a fallback (e.g. a cost threshold below which dispatches free-pass) gets its own issue. Watch post-merge.
Docs updated:
docs/handoffs/chicorytv-issue-queue.md(HARD CONSTRAINTS bullet + tightened prose),docs/decisions/workflow-process.md(new record),docs/decisions/README.md(regenerated),.claude/settings.json+.claude/hooks/pretooluse-agent-model.sh(new hook).