From 708e6021977ced7dd390a61e88ac15592dc9d11b Mon Sep 17 00:00:00 2001 From: Timothy Date: Sun, 19 Jul 2026 01:09:57 +0200 Subject: [PATCH] =?UTF-8?q?feat(queue):=20deterministic=20scripts/select-q?= =?UTF-8?q?ueue.sh=20=E2=80=94=20stop=20re-deriving=20the=20selector=20by?= =?UTF-8?q?=20hand?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The kickoff dispatches a cheap model to rank the backlog, and the lore then made the orchestrator re-derive its mechanical claims (deps, milestone tiering, priority order, in-progress state) because a small model kept getting them wrong. This pays that tax down: the mechanical checks are now CODE — GET /dependencies exclusion, LOCAL .milestone.state/review/priority tiering (never the no-op ?milestones= filter), (tier,priority,issue#) ordering, in-progress/parked/PR exclusion — leaving only two JUDGMENT flags (CLAIM?, UMBRELLA?) for a human/model to resolve. - scripts/select-queue.sh: fail-open on no-creds/unreachable; ~1.3s; shellcheck clean; UMBRELLA? computed in the jq pass (no per-candidate body fetch). - handoff kickoff: run the script FIRST; trust its deps/tiering/ordering, recheck only flags. - handoff lore: the three 'cheap selector unreliable → re-derive by hand' bullets kept as the EVIDENCE for why the script exists; the prescription is redirected to 'run the script'. Operator-requested this session: 'rather than have the lore make us redo the selector's work, improve the selector.' Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/handoffs/chicorytv-issue-queue.md | 24 +++- scripts/select-queue.sh | 165 +++++++++++++++++++++++++ 2 files changed, 188 insertions(+), 1 deletion(-) create mode 100755 scripts/select-queue.sh diff --git a/docs/handoffs/chicorytv-issue-queue.md b/docs/handoffs/chicorytv-issue-queue.md index 679df3b33..0355ff869 100644 --- a/docs/handoffs/chicorytv-issue-queue.md +++ b/docs/handoffs/chicorytv-issue-queue.md @@ -101,7 +101,17 @@ silently expanding orchestrator reconnaissance. Global Codex hook enforcement is `timothy/server-management#592`; the wider Claude-hook port is tracked in `timothy/server-management#593`. **When the user has not named an issue, queue selection is mechanical fast/small work, never orchestrator -work.** Dispatch exactly one selector on the cheapest suitable model at `low` effort. Give it #237's +work.** FIRST run the deterministic selector — `ETV_GITEA_BASICAUTH=user:pass scripts/select-queue.sh [N]` +— which does the parts a cheap model kept getting wrong IN CODE: it excludes `in-progress`/`parked`/PRs, +resolves `GET /issues/{n}/dependencies` on every candidate (dropping any with an OPEN blocker), tiers by +LOCAL `.milestone.state`/`review`/`priority:` filters (never the server `?milestones=` name filter, which +no-ops on `:`/`+`), and orders by (tier, priority, issue#). **Its DEPS/tiering/ordering are deterministic — +trust them, do not re-derive them by hand.** It also raises two JUDGMENT flags it deliberately does not +decide — `CLAIM?` (a claim can precede the label) and `UMBRELLA?` (an epic whose children are the real +pickups); the orchestrator (or a cheap model) resolves ONLY those by reading the flagged issue's +comments/body. The script does not rank the arc tier (arc order is #237 prose, currently CLOSED) — read +#237 if an arc re-opens. If the script is unavailable (no creds / Gitea down), fall back to the cheap-model +path below. Dispatch exactly one selector on the cheapest suitable model at `low` effort. Give it #237's body and last ~6 comments, then have it query live candidates as a cascade in this exact tier order: **arc → OPEN issues assigned to OPEN milestones → open `review` → unmilestoned/unreviewed `priority: high` → `priority: medium` → `priority: low`**. Every candidate row represents an issue @@ -297,6 +307,18 @@ HARD CONSTRAINTS: - READ docs/README.md → the convention docs FIRST; point recon/implementer agents at specific doc sections. Only recon the task-specific delta. +- **The mechanical selection checks now live in `scripts/select-queue.sh` — run it instead of + re-deriving them by hand (2026-07-19).** The three bullets below are the *evidence* for why the + script exists: a cheap model repeatedly reported blocked issues as `deps:clear`, mis-tiered on the + no-op `?milestones=` filter, and ranked by issue number instead of priority-within-tier — so the + lore grew to "re-derive the whole contested tier every session," which is exactly the tax the + script pays down. The script does the deterministic work in code (dependency exclusion via + `GET /dependencies`, LOCAL `.milestone.state`/`review`/`priority:` tiering, (tier, priority, issue#) + ordering, `in-progress`/`parked`/PR exclusion) and only *flags* `CLAIM?`/`UMBRELLA?` for judgment. + **Trust its DEPS/tiering/ordering; do the focused recheck only on the flagged rows.** Keep the + evidence bullets below (they explain the failure modes the script encodes), but the *prescription* + is now "run the script," not "hand-re-derive." If you extend the tiers/filters, update the script + first, then these bullets. - **Gitea milestone-name issue filter silently NO-OPS on names with special chars (2026-07-17, #77 selection)**: `GET /issues?milestones=` returns the WHOLE open-issue list — not a filtered set — when the milestone title contains `:` or `+` (e.g. `Scheduling: refactor + diff --git a/scripts/select-queue.sh b/scripts/select-queue.sh new file mode 100755 index 000000000..078b26c97 --- /dev/null +++ b/scripts/select-queue.sh @@ -0,0 +1,165 @@ +#!/usr/bin/env bash +# scripts/select-queue.sh — deterministic queue selector for the #237 pickup protocol. +# +# WHY THIS EXISTS. The kickoff dispatches a cheap model to rank the backlog, and the lore then makes +# the orchestrator RE-DERIVE the selector's mechanical claims because a small model kept getting them +# wrong: it reported blocked issues as `deps:clear` (never called /dependencies), mis-tiered issues +# when the server-side `?milestones=` filter silently no-ops on `:`/`+` names, ranked by issue number +# and ignored priority-within-tier, and missed live `in-progress` claims. Re-deriving all of that by +# hand every session is the tax. The fix is to stop asking a model to do arithmetic: this script does +# the MECHANICAL parts deterministically so there is nothing to re-verify, and only FLAGS the two +# checks that are genuinely judgment (a claim buried in a comment, umbrella-vs-child framing) for a +# human/model to resolve on the shortlist. +# +# WHAT IS DETERMINISTIC HERE (trust it, don't re-derive): +# - dependency exclusion: GET /issues/{n}/dependencies on every surviving candidate; any OPEN +# blocker => excluded (Gitea auto-clears the block when the blocker closes). +# - tiering: OPEN-milestone membership by LOCAL `.milestone.state=="open"` filter (never the +# server `?milestones=` name filter, which no-ops on `:`/`+`); `review` label; `priority:` label. +# - ordering: within a tier, priority high>medium>low, then lowest issue number. +# - label exclusions: `in-progress` and `parked` are dropped; pull requests are excluded +# (type=issues + a .pull_request guard). +# +# WHAT STAYS JUDGMENT (the script FLAGS, it does not decide): +# - CLAIM? — a "claiming" note can precede the `in-progress` label; the script flags any candidate +# whose most-recent comments look like a live claim so the orchestrator reads them before claiming. +# - UMBRELLA? — an epic/umbrella whose children are the real pickups reads like a normal issue; the +# script flags bodies that open with umbrella framing so the orchestrator treats it as a container. +# - ARC tier — arc order lives in #237's prose (currently CLOSED / arc complete), not in any label, +# so this script deliberately does NOT rank an arc tier. If an arc ever re-opens, read #237. +# +# Usage: +# ETV_GITEA_BASICAUTH=user:pass scripts/select-queue.sh [N] +# N size of the ranked shortlist (default 5) +# Creds from env: ETV_GITEA_TOKEN or ETV_GITEA_BASICAUTH (user:pass). ETV_GITEA_URL overrides the +# base (default: the LAN instance); ETV_GITEA_REPO overrides owner/repo (default timothy/ersatztv). +# +# Output: a ranked table (most-eligible first) with tier, priority, milestone, deps evidence, and the +# CLAIM?/UMBRELLA? flags. Fail-OPEN with no creds / Gitea unreachable (prints a notice, exits 0). +# Exit 0 when a shortlist is produced or cleanly skipped; exit 2 only on a usage error. +set -uo pipefail + +N="${1:-5}" +case "$N" in ''|*[!0-9]*) echo "usage: select-queue.sh [N] (N = shortlist size, integer)" >&2; exit 2;; esac + +base_url="${ETV_GITEA_URL:-http://192.168.1.95:3000}/api/v1" +repo="${ETV_GITEA_REPO:-timothy/ersatztv}" + +command -v jq >/dev/null || { echo "select-queue: jq not found — install jq." >&2; exit 2; } + +gq() { + local path="$1" + if [ -n "${ETV_GITEA_TOKEN:-}" ]; then + curl -sf -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/$path" 2>/dev/null + elif [ -n "${ETV_GITEA_BASICAUTH:-}" ]; then + curl -sf -u "$ETV_GITEA_BASICAUTH" "$base_url/$path" 2>/dev/null + else + return 1 + fi +} + +if [ -z "${ETV_GITEA_TOKEN:-}" ] && [ -z "${ETV_GITEA_BASICAUTH:-}" ]; then + echo "select-queue: no Gitea creds in env (ETV_GITEA_TOKEN or ETV_GITEA_BASICAUTH) — skipping (no-op)." + exit 0 +fi + +# Reachability probe up front (a mid-pagination `exit` inside the $(...) below would only exit the +# subshell, not this script — so fail-open explicitly here where it can actually stop us). +gq "repos/$repo/issues?state=open&type=issues&limit=1&page=1" >/dev/null || { + echo "select-queue: Gitea unreachable or creds rejected — skipping (no-op)."; exit 0; } + +# 1) Page through ALL open issues once (type=issues excludes PRs). Filter and tier LOCALLY — never +# trust the server-side milestone-name filter. The list payload already carries .body, so the +# UMBRELLA? flag is computed here (no per-candidate body fetch). Emit one TSV row per +# eligible-by-label candidate, pre-sorted by (tier, priority-rank, number) so the deps walk below +# only has to go top-down. +rows=$( + page=1 + while :; do + batch=$(gq "repos/$repo/issues?state=open&type=issues&limit=50&page=$page") || break + count=$(printf '%s' "$batch" | jq 'length' 2>/dev/null || echo 0) + [ "${count:-0}" -eq 0 ] && break + printf '%s' "$batch" | jq -r ' + .[] + # drop PRs and label-excluded issues up front + | select(.pull_request == null) + | (.labels | map(.name)) as $L + | select(($L | index("in-progress")) | not) + | select(($L | index("parked")) | not) + # priority rank: high=0, medium=1, low=2, none=3 (none still selectable in a milestone/review tier) + | (if ($L | index("priority: high")) then 0 + elif ($L | index("priority: medium")) then 1 + elif ($L | index("priority: low")) then 2 + else 3 end) as $prank + | (($L | index("review")) != null) as $isReview + | ((.milestone != null) and (.milestone.state == "open")) as $inOpenMs + # tier: 2=open-milestone, 3=review, 4/5/6=priority high/medium/low (unmilestoned/unreviewed). + # (Tier 1 = arc; not label-derivable, handled by the orchestrator via #237 prose.) + | (if $inOpenMs then 2 + elif $isReview then 3 + elif $prank==0 then 4 + elif $prank==1 then 5 + elif $prank==2 then 6 + else empty end) as $tier # no priority label AND not milestone/review => not a candidate + # UMBRELLA? — body opens with epic/umbrella framing (computed from the list payload) + | (((.body // "")[0:400]) | test("(?i)follow-up arc to|umbrella|this epic|tracked (here|by) as sub|child issues?:|sub-work|sub-issues?:")) as $umbrella + | [ $tier, $prank, .number, + (.milestone.title // "-"), + (.comments // 0), + (if $umbrella then "1" else "0" end), + (.title | gsub("[\t\n]";" ")) ] + | @tsv + ' 2>/dev/null + page=$((page+1)) + done | sort -t$'\t' -k1,1n -k2,2n -k3,3n +) + +if [ -z "$rows" ]; then + echo "select-queue: no label-eligible candidates found (arc/milestone/review/priority all empty)." + echo " An empty backlog is unusual — confirm OPEN issues carry priority: labels (see the H12 audit)." + exit 0 +fi + +# 2) Walk the pre-sorted candidates top-down; for each, resolve dependencies (deterministic exclusion) +# and compute the two judgment FLAGS. Stop once N eligible candidates are collected. +tier_name() { case "$1" in 2) echo "milestone";; 3) echo "review";; 4) echo "prio-high";; 5) echo "prio-medium";; 6) echo "prio-low";; *) echo "t$1";; esac; } +prio_name() { case "$1" in 0) echo "high";; 1) echo "medium";; 2) echo "low";; *) echo "-";; esac; } + +printf '%-6s %-11s %-7s %-22s %-16s %-8s %s\n' "ISSUE" "TIER" "PRIO" "MILESTONE" "DEPS" "FLAGS" "TITLE" +printf '%s\n' "----------------------------------------------------------------------------------------------------" + +collected=0 +while IFS=$'\t' read -r tier prank number ms ncomments umbrella title; do + [ "$collected" -ge "$N" ] && break + + # --- deterministic: dependency exclusion --- + deps=$(gq "repos/$repo/issues/$number/dependencies") + open_blockers=$(printf '%s' "$deps" | jq -r '[.[] | select(.state=="open") | (.number|tostring)] | join(",")' 2>/dev/null) + if [ -n "$open_blockers" ]; then + printf ' (skip #%s: blocked-by open #%s)\n' "$number" "$open_blockers" + continue + fi + closed_blockers=$(printf '%s' "$deps" | jq -r '[.[] | select(.state!="open") | (.number|tostring)] | join(",")' 2>/dev/null) + deps_cell="clear"; [ -n "$closed_blockers" ] && deps_cell="clear(was #$closed_blockers)" + + # --- judgment FLAGS (surface, do not decide) --- + flags="" + # UMBRELLA? — precomputed from the body in the jq pass above + [ "$umbrella" = "1" ] && flags="${flags}UMBRELLA? " + # CLAIM? — a claim can precede the in-progress label; scan the last comments for claim language + if [ "${ncomments:-0}" -gt 0 ]; then + lastc=$(gq "repos/$repo/issues/$number/comments" | jq -r '.[-2:][]?.body // ""' 2>/dev/null) + if printf '%s' "$lastc" | grep -qiE 'claim(ing|ed)?\b|picking (this|it) up|taking this'; then + flags="${flags}CLAIM? " + fi + fi + [ -z "$flags" ] && flags="-" + + printf '#%-5s %-11s %-7s %-22.22s %-16s %-8s %.50s\n' \ + "$number" "$(tier_name "$tier")" "$(prio_name "$prank")" "$ms" "$deps_cell" "$flags" "$title" + collected=$((collected+1)) +done <<< "$rows" + +echo +echo "Ranked by (tier, priority, issue#). DEPS/tiering/ordering are deterministic — trust them." +echo "Resolve CLAIM?/UMBRELLA? flags before claiming (read the issue's comments/body). Arc tier: read #237."