chore(583): make per-agent model routing a hard constraint + PreToolUse gate #585
Executable
+76
@@ -0,0 +1,76 @@
|
||||
#!/usr/bin/env bash
|
||||
# PreToolUse / Agent — ask when an agent is dispatched without an explicit `model`.
|
||||
#
|
||||
# The kickoff prompt (docs/handoffs/chicorytv-issue-queue.md) says to route by capability: cheap/fast
|
||||
# for bounded recon, mid tier for a mechanical slice against a documented contract, orchestrator tier
|
||||
# for judgment-heavy work. That rule lived only in prose, and on 2026-07-25 an orchestrator dispatched
|
||||
# two implementers with `model` omitted — both silently inherited the Opus orchestrator tier. Nothing
|
||||
# in the session report revealed it; the operator had to ask.
|
||||
#
|
||||
# WHY a hook: omitting `model` is the SILENT path. Every other constraint in that kickoff has a hook,
|
||||
# a CI job or a script behind it, and those were all followed in the same session — the one rule with
|
||||
# no forcing function was the one that got defaulted. A check that runs beats a rule you must remember
|
||||
# (the same reasoning as pretooluse-bom-guard.sh).
|
||||
#
|
||||
# SCOPE — gate EVERY dispatch that names no model, not just implementer-looking ones. The first cut
|
||||
# tried to be clever: it fired only when the prompt text matched implementer signals (`git commit`,
|
||||
# `worktree`, `fixes #`…). Review of that version (#583) confirmed the heuristic both over- and
|
||||
# under-fired — a read-only recon brief mentioning "worktree" nagged, while "author the change and
|
||||
# open a PR", "land this on the branch" and "make the changes and commit them" all sailed through
|
||||
# silently, i.e. it missed the exact case it existed to catch. Prompt prose is not a reliable signal
|
||||
# for authority, and a gate with an unreliable catch rate is worse than an honest one.
|
||||
#
|
||||
# Two further reasons the broad form is correct here:
|
||||
# - The HARD CONSTRAINT itself says "every dispatched agent". A narrower hook contradicted the rule
|
||||
# it was built to enforce.
|
||||
# - Routing matters MOST for the cheap cases. The old exemption list ("read-only, so routing barely
|
||||
# matters") had it backwards: bounded recon is precisely what should be explicitly routed DOWN to
|
||||
# a fast tier, and that review also showed the premise was false — Explore, Plan and
|
||||
# claude-code-guide all carry Bash, so none of them provably "cannot commit".
|
||||
#
|
||||
# The prompt costs nothing to avoid: name a tier and this never fires. That is the habit being built.
|
||||
#
|
||||
# Exempt: `fork` only — a fork ALWAYS inherits the parent model and the tool IGNORES a `model`
|
||||
# override, so asking would demand something unachievable.
|
||||
#
|
||||
# "ask", never "deny": routing is a judgment call with no derivable right answer, unlike the
|
||||
# merge-consent gate (H6/H10) which derives a verifiable state. This gate exists to make an invisible
|
||||
# default visible, not to impose a tier.
|
||||
#
|
||||
# Fail-open by design: any parse trouble -> allow (exit 0, no output).
|
||||
set -uo pipefail
|
||||
|
||||
input=$(cat)
|
||||
|
||||
tool=$(printf '%s' "$input" | jq -r '.tool_name // ""' 2>/dev/null || true)
|
||||
[ "$tool" = "Agent" ] || exit 0
|
||||
|
||||
# An explicit choice was made — nothing to surface. This is the path to prefer.
|
||||
model=$(printf '%s' "$input" | jq -r '.tool_input.model // ""' 2>/dev/null || true)
|
||||
[ -z "$model" ] || exit 0
|
||||
|
||||
subagent=$(printf '%s' "$input" | jq -r '.tool_input.subagent_type // ""' 2>/dev/null || true)
|
||||
|
||||
# A fork's model is fixed to the parent's by the tool; a prompt here could not be acted on.
|
||||
[ "$subagent" = "fork" ] && exit 0
|
||||
|
||||
label="${subagent:-general-purpose}"
|
||||
reason="Dispatching an agent (subagent_type: ${label}) with no explicit \`model\`.
|
||||
|
||||
It will silently inherit this session's model — which may be right, but it is a default, not a choice.
|
||||
Name the tier (and say so in the dispatch message), per the kickoff routing rule
|
||||
\`process.per-agent-model-routing\`:
|
||||
|
||||
- bounded recon / inventory / log triage -> cheapest fast tier (haiku)
|
||||
- mechanical slice against a documented contract -> mid tier (sonnet)
|
||||
- judgment-heavy: design, compiler/parser, security,
|
||||
migrations, review arbitration -> orchestrator tier (opus)
|
||||
|
||||
Independent review should also prefer a DIFFERENT model family than the implementer — a cold
|
||||
same-family review is worth less than a cross-family one.
|
||||
|
||||
Pass \`model\` on the Agent call and this never fires. Approve as-is only if inheriting the
|
||||
orchestrator tier is the deliberate call."
|
||||
|
||||
jq -n --arg r "$reason" '{hookSpecificOutput:{hookEventName:"PreToolUse",permissionDecision:"ask",permissionDecisionReason:$r}}'
|
||||
exit 0
|
||||
@@ -39,6 +39,11 @@
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR/.claude/hooks/pretooluse-agent-ram.sh\"",
|
||||
"timeout": 10
|
||||
},
|
||||
{
|
||||
"type": "command",
|
||||
"command": "\"$CLAUDE_PROJECT_DIR/.claude/hooks/pretooluse-agent-model.sh\"",
|
||||
"timeout": 10
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -93,6 +93,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
| `process.lock-ownership-enumerate-producers` | Before trusting any "single owner / no double release / no cross-release" claim, grep the whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-lockchannel-no-cross-release-verdict-must-enumerate-every-producer-via-grep-542) |
|
||||
| `process.one-worktree-one-committing-agent` | Never run two committing agents concurrently on one worktree — give each parallel slice its own worktree branched off the feature branch and merge back. | 2026-07-21 | [link](workflow-process.md#2026-07-21--one-worktree-one-committing-agent-542) |
|
||||
| `process.parallel-session-claim` | Apply the `in-progress` label before starting an issue, and still read its dependency notes before touching shared surfaces — a claim prevents duplicate pickup, not overlapping code changes. | 2026-07-21 | [link](workflow-process.md#2026-07-21--claim-with-in-progress-before-working-claiming-is-not-collision-safety-542) |
|
||||
| `process.per-agent-model-routing` | State the model tier (and effort, where the client exposes it) in the dispatch itself for every delegated agent — bounded recon → cheapest fast tier at `low`; mechanical slice against a documented contract → mid tier; judgment-heavy work → orchestrator tier; independent review → a different model family than the implementer. | 2026-07-25 | [link](workflow-process.md#2026-07-25--name-the-model-tier-for-every-dispatched-agent-a-pretooluse-gate-makes-the-silent-default-visible-583) |
|
||||
| `process.pr-routine-sequence` | Worktree off origin/main → implement → regenerate API artifacts → full local tests + cold review + live-E2E ALL before the push → push, open PR, arm the CI monitor at open → fixes after the push are follow-up commits, never amend/force-push. | 2026-07-21 | [link](workflow-process.md#2026-07-21--the-pr-routine-is-a-fixed-sequence-validate-locally-then-push-then-only-follow-up-commits-542) |
|
||||
| `process.review-disagreement-frontier-judge` | When independent reviews disagree on a gate PR, escalate to the frontier judge, and put the proposed fix approach in front of it — not just the disputed finding. | 2026-07-21 | [link](workflow-process.md#2026-07-21--review-disagreement-on-a-gate-pr-escalates-to-the-frontier-judge--and-the-proposed-fix-escalates-with-it-542) |
|
||||
| `process.shared-tree-readonly` | Never commit in `/Users/timothy/ersatztv` and never read its `git log`/`git status`/HEAD to infer anything about `main` — work in a worktree off `origin/main`, which is the only source of truth. | 2026-07-21 | [link](workflow-process.md#2026-07-21--the-shared-tree-at-userstimothyersatztv-is-read-only-and-tells-you-nothing-about-main-542) |
|
||||
|
||||
@@ -524,3 +524,64 @@ The dependencies API is unaffected: `POST /issues/{n}/dependencies` with `{"owne
|
||||
blocked-by correctly, though the bare `{index}` form returns 201 without reliably attaching — verify
|
||||
with the GET. (That last sentence is carried forward from an earlier revision of the handoff doc,
|
||||
commit `f93458c7`, where it was dropped by a later prune rather than disproved.)
|
||||
|
||||
## 2026-07-25 — Name the model tier for every dispatched agent; a PreToolUse gate makes the silent default visible (#583)
|
||||
|
||||
`key: process.per-agent-model-routing` · `status: active` · `since: 2026-07-25` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** State the model tier (and effort, where the client exposes it) in the dispatch itself for every delegated agent — bounded recon → cheapest fast tier at `low`; mechanical slice against a documented contract → mid tier; judgment-heavy work → orchestrator tier; independent review → a different model family than the implementer.
|
||||
**Signals:** subagent model routing · `model` omitted · silent tier inheritance · orchestrator tier for a mechanical slice · capability routing · prose rule vs HARD CONSTRAINT · dispatch-time checkpoint · paths: `.claude/hooks/pretooluse-agent-model.sh`, `docs/handoffs/chicorytv-issue-queue.md` · issues: #583, #436, #440
|
||||
**Mechanics:** `Agent` tool `model` parameter; PreToolUse hook on the existing `Agent|Task` matcher in `.claude/settings.json`.
|
||||
|
||||
On 2026-07-25 a session dispatched two implementers (#436, #440) with `model` omitted on both calls;
|
||||
both silently inherited the Opus orchestrator tier. #440 was a mechanical SPA slice against an
|
||||
already-shipped backend contract — a plausible mid-tier candidate.
|
||||
|
||||
The interesting part is *why*, because it wasn't forgetfulness. **Every rule in `HARD CONSTRAINTS` was
|
||||
followed in that same session** — worktree off `origin/main`, one committing agent per worktree,
|
||||
parallelize on disjoint slices, local gate before push. Routing was the one instruction living only in
|
||||
a prose paragraph, and it was the one that got defaulted. Treat that as the general lesson: in a
|
||||
kickoff doc that is pasted into every session, **the bulleted imperative list is what actually
|
||||
functions as the checklist**, and prose above it is read as background. A rule you want followed
|
||||
belongs in the list, keyed, or it is advisory in practice.
|
||||
|
||||
Three aggravating factors, all worth checking when writing any future rule here:
|
||||
|
||||
- **Scope gap.** The low-cost-routing paragraph is written entirely about queue selection and recon
|
||||
("bounded searches, inventories, log triage, report drafting"). It never named *implementers*, and
|
||||
gave no default for the bounded-but-not-trivial case — so the largest-cost dispatch fell in a gap.
|
||||
- **The wrong default is the silent one.** Omitting `model` produces no artifact. Nothing in the
|
||||
session report revealed the tier; the operator had to ask. Contrast the BOM trap
|
||||
(`process.bom-format-detection-recipe`), where a hook fires because a memory describing the trap
|
||||
demonstrably failed to prevent it twice in one day.
|
||||
- **Distance from the decision point.** The rule sits ~line 84 of the kickoff; dispatch happens after
|
||||
orientation, claiming, the bundle scan and doc reading.
|
||||
|
||||
Hence the two-part fix: a keyed HARD CONSTRAINT that requires the tier to be **stated out loud in the
|
||||
dispatch** (a self-correcting mechanism — it turns an invisible omission into visible output), plus
|
||||
`.claude/hooks/pretooluse-agent-model.sh`, which `ask`s whenever an agent is dispatched with no
|
||||
explicit `model`. It exempts only `fork`, whose `model` override the tool ignores by design, so a
|
||||
prompt there could not be acted on. It is `ask`, never `deny`: routing is a judgment call with no
|
||||
derivable right answer, unlike the H6/H10 merge gate (`release.merge-consent-autogrant`), which
|
||||
derives a verifiable state and can therefore grant or refuse outright.
|
||||
|
||||
**The first cut was narrower, and review killed it** — worth recording, because the reasoning was
|
||||
seductive. It fired only when the prompt *text* matched implementer signals (`git commit`, `worktree`,
|
||||
`fixes #`), on the theory that a gate firing on every fan-out trains one-shot dismissal. Independent
|
||||
review confirmed the heuristic both over- and under-fired: a read-only recon brief merely *mentioning*
|
||||
"worktree" nagged, while "author the change and open a PR", "land this on the branch" and "make the
|
||||
changes and commit them" all passed silently — it missed precisely the case it existed to catch.
|
||||
**Prompt prose is not a reliable signal for authority**, and a gate with an unreliable catch rate is
|
||||
worse than none, because it gets trusted.
|
||||
|
||||
Two further reasons the broad form is right, both of which the narrow version had backwards:
|
||||
|
||||
- **It now matches the rule it enforces.** The HARD CONSTRAINT says "every dispatched agent"; a hook
|
||||
gating only implementer-looking dispatches contradicted its own rule.
|
||||
- **Routing matters MOST for the cheap cases.** The old exemption list justified itself as
|
||||
"read-only, so routing barely matters" — but bounded recon is exactly what should be explicitly
|
||||
routed *down* to a fast tier. The premise was also false: `Explore`, `Plan` and `claude-code-guide`
|
||||
all carry Bash, so none of them provably "cannot commit".
|
||||
|
||||
The noise objection is answered by the escape hatch rather than by scoping: **naming a tier costs one
|
||||
parameter and the hook never fires again**. The prompt is self-eliminating for anyone following the
|
||||
rule, which is the habit being built.
|
||||
|
||||
@@ -81,11 +81,12 @@ Orchestration means: decompose, delegate independent slices, integrate their res
|
||||
whole, and keep canonical issue state accurate. Use the client's native agent/subagent tools; never
|
||||
assume a named tool, command, plugin, model-routing feature, or fork mechanism exists.
|
||||
|
||||
Route by capability when the client supports per-agent model selection: fast/small for bounded recon,
|
||||
balanced for mechanical changes, and the strongest coding/agentic tier for judgment-heavy work.
|
||||
Otherwise use the active model for every slice except the mandatory queue preflight below. Independent
|
||||
review MUST start from a cold, review-only brief; prefer a different model family/client when one is
|
||||
available, otherwise use a fresh agent with no implementation role.
|
||||
Route by capability when the client supports per-agent model selection, and **say which tier you chose
|
||||
in the dispatch itself** — see the `process.per-agent-model-routing` HARD CONSTRAINT below for the
|
||||
table. Where the client cannot route per agent, use the active model for every slice except the
|
||||
mandatory queue preflight below. Independent review MUST start from a cold, review-only brief; prefer
|
||||
a different model family/client when one is available, otherwise use a fresh agent with no
|
||||
implementation role.
|
||||
|
||||
**Low-cost routing applies throughout the session, not only to queue selection.** Before any batch of
|
||||
bounded searches, inventories, log triage, URL/status sampling, or report drafting, dispatch the cheapest
|
||||
@@ -273,6 +274,11 @@ HARD CONSTRAINTS:
|
||||
plumbing merge. → `process.foreign-worktree-plumbing-merge`
|
||||
- **Parallelize by default** on genuinely disjoint slices (normally 2–3 delegated agents). Cap builds at
|
||||
3–4 and gate on FREE RAM, not CPU. **Never set `ETV_UPDATE_GOLDENS`.** → `process.build-concurrency-limits`
|
||||
- **Name the model + effort for every dispatched agent, in the dispatch itself** — bounded recon → cheapest
|
||||
fast tier at `low`; mechanical slice against a documented contract → mid tier; judgment-heavy (design,
|
||||
compiler/parser, security, migrations, review arbitration) → orchestrator tier; independent review →
|
||||
a different model family than the implementer. Omitting it silently inherits the orchestrator tier, so
|
||||
state the choice out loud. → `process.per-agent-model-routing`
|
||||
- **Local gate + cold-context review BEFORE the push**, never after. → `process.local-gate-before-push`
|
||||
- **Independent review is mandatory** for locks/concurrency, auth/security, API write-path handlers, DB
|
||||
migrations, or >~150 changed C# lines; a skip must be stated with its reason. → `process.independent-review-rubric`
|
||||
|
||||
Reference in New Issue
Block a user