ersatztv/docs/superpowers/specs/2026-07-21-decision-lifecycle-and-kickoff-design.md

Decision-lifecycle + retrieval-efficient startup — design (ersatztv #520 + #521)

Date: 2026-07-21 Issues: #521 (decision knowledge lifecycle), #520 (retire #237 from startup; parallel orientation + selection). Both originate from a bounded knowledge-retrieval audit in timothy/adversarial-reviewer.

Goal

Two coupled problems from the same audit:

1. #521 — docs/decisions.md (~2,468 lines; ~3,469 incl. topic files) is optimized for append safety, not current-state retrieval. Status lives in prose, supersession is inferred at release time, and fresh agents are told to read the corpus broadly before the task is known.
2. #520 — Startup still tells agents (and the selector fallback) to read the closed arc tracker #237 and its stale forward-looking comments, and preloads the whole doc corpus before the task is known. scripts/select-queue.sh already does live mechanical selection correctly.

The fix is one arc: give decisions a stable identity + explicit lifecycle + a compact generated active view (#521), then point startup at that compact view and run orientation ‖ selection in parallel (#520). Keep Git-backed Markdown authoritative; MemPalace stays candidate-discovery only.

Non-goals

• Making MemPalace the source of truth; storing live queue state in decision docs.
• Reopening #237 or creating a new manual session-log issue / Markdown ledger.
• Implementing the MemPalace/Gitea exporter (server-management#642).
• Changing select-queue.sh's dependency/tier/priority algorithm (only its comments/output text), unless a test exposes a bug.

Delegation model (agreed)

• Judgment (orchestrator / escalated): schema, validator, catalog generator, kickoff + docs rewrite, and reconciling the prose-only reversals (#390→#406; #411 measurement obsolescence).
• Mechanical (balanced/cheap subagents, fanned out by topic batch): reformatting existing entries into the keyed schema. Each subagent gets exact schema rules + the batch's entries + a worktree/branch of its own; results merge back. No two committing agents share a worktree.

Guard rework (the crux)

The current append-only guard blocks any modified/deleted line in decisions.md unless the commit carries [decisions-edit] (.claude/hooks/decisions-guard.sh, wired to Husky commit-msg and the CI decisions-guard job). That line-level mechanic is incompatible with a lifecycle model where migration reformats every entry, supersession edits a predecessor's superseded-by: line, and retiring moves a record to archive.

Decision: replace the line-level mechanic with a lifecycle validator that enforces the same spirit — rationale is never silently rewritten or deleted; every history touch is deliberate and reviewable. The [decisions-edit] token is kept narrow, not retired (revised after Fable review): routine lifecycle metadata writes (add record, set superseded-by, relocate to archive, regen catalog) are token-free and proven well-formed by the validator; a change to a record's rationale prose still requires [decisions-edit], enforced by the validator's body-diff. Retiring the token entirely would let an agent silently rewrite rationale (or archive a falsified body) and pass — the exact case the old guard existed for.

Spirit preserved by these validator invariants (see Validator below): ≤1 active record per key; no record block may vanish (if it leaves the active set it must reappear under docs/decisions/archive/); a surviving record's rationale prose can't change, and an archived copy must body-match its pre-move active version, without [decisions-edit] in the commit range; supersession links are reciprocal and resolvable; status only moves active → superseded/retired. The no-vanish/body-diff checks run in CI over merge-base(base,head)…head across decisions.md and every topic file.

Delivery: two PRs, one arc (revised after Fable review). PR1 = machinery (parser, validator, catalog generator, guard swap, header rewrite, exemplar + append-only→lifecycle supersession); PR2 = full corpus migration + #520 startup rewrite + kickoff guard + retrieval-eval. Matches #521's "bounded topic batches, avoid a single conflict-heavy rewrite" and keeps each diff's conflict surface small on the treadmill-prone decisions.md.

Information model (#521)

Each decision record carries a compact, visible, deterministically-parseable metadata block immediately under its ## heading (front-loaded so MemPalace indexes key/status/rule first):

## 2026-07-17 — No persistent compiler servers in CI … (#406)
`key: ci.runner-placement` · `status: active` · `since: 2026-07-17` · `supersedes: none` · `superseded-by: none`
**Rule:** Every CI `services:` container gets an explicit CPU/mem cap; no shared persistent compiler daemon.
**Signals:** ci, runners, services, memory-cap · paths: .gitea/workflows/docker-build.yml · issues: #390 #406
**Mechanics:** docs/ci-cd.md → CI lanes
<existing rationale prose, incl. rejected alternatives — unchanged>

Fields (issue-required):

• key — stable subject identity, dotted lowercase (^[a-z0-9]+(\.[a-z0-9-]+)+$), e.g. ci.runner-placement. A superseding record reuses the predecessor's key.
• status — active | superseded | retired | legacy-unmigrated (the last is a transitional marker; see Migration). "Relocated"/"consolidated" are maintenance actions, not statuses.
• since — effective date (YYYY-MM-DD) or version.
• supersedes / superseded-by — predecessor/successor key+date, or none.
• **Rule:** — one-line current rule, before the long rationale.
• **Signals:** — concepts, paths:, issues: for task-driven retrieval.
• **Mechanics:** — link(s) to the current convention/source/test anchors.
• Rationale prose (incl. rejected alternatives) follows, unchanged from today where migrated.

Rejected alternatives stay inside their accepted record as rationale; they are never active records.

Document architecture (#521)

• docs/decisions.md — stays the chronological record store for in-file (non-topic) decisions, now in schema form. Header rewritten: append-only-log framing → lifecycle framing.
• docs/decisions/*.md — existing topic files, entries migrated to schema form.
• docs/decisions/README.md — NEW generated active catalog / task router. A compact table (key · one-line rule · status · record link · signals) built by scripts/build-decisions-catalog.py from all active records across decisions.md + topic files. Regenerated like endpoint-index.md; drift fails CI. This is the compact "active view" startup points at.
• docs/decisions/archive/ — NEW. superseded/retired records relocated here, out of the default startup path, with forward/back links preserved. Not scanned by the catalog generator.

Validator (#521)

scripts/decisions-validate.py, run locally (a Husky hook replacing the append-only commit-msg check) and in the CI decisions-guard job (renamed → lifecycle guard). Checks:

1. Required metadata present + well-formed on every structured record.
2. Valid status enum; valid key format.
3. ≤1 active record per key.
4. Reciprocal, resolvable supersedes/superseded-by links (successor exists; predecessor points back).
5. No-vanish (diff-aware): any record block removed from the active files (decisions.md + topic files) between base and HEAD must be present under docs/decisions/archive/. Fail-open on missing refs, like the current guard.
6. Active-catalog completeness + freshness (regenerate to a temp file, diff — like check:api).
7. Archive/active placement consistency (no active record in archive; no superseded/retired in the active set once migration of its key is complete).
8. Aggregate active-corpus budget — sum of lines across decisions.md + topic files + catalog (excluding archive). Warn over a threshold (start at the current 1800 applied to the aggregate; tune during migration). Report the legacy-unmigrated remainder count; it must be visible and trend to zero.
9. Best-effort broken-link check on Mechanics:/archive links where practical.

Fail-open on tooling trouble, matching the current guard's philosophy.

Startup / kickoff rewrite (#520)

• docs/handoffs/chicorytv-issue-queue.md — retire #237 from normal startup. Two concurrent tracks when no issue is named: orientation (AGENTS.md, CLAUDE.md, compact docs/README.md map + the active catalog) ‖ selection (scripts/select-queue.sh 5). After both: resolve only CLAIM?/UMBRELLA? flags + the winner, recheck live state, claim, then build a focused task-specific knowledge packet. Named-issue path skips selection → focused retrieval directly. Archive the contradictory pre-script selector lore (the "re-derive the whole contested tier by hand" bullets) into a clearly-labeled historical section. #237 appears only as labeled archival history.
• docs/README.md — mandatory 1–10 reading order → compact task-signal → minimal sections authority/task map, plus a pointer to docs/decisions/README.md as the decision active view.
• CLAUDE.md — docs-first guidance requires the map + relevant sections, not the whole corpus. Task-completion protocol gains the structured ## Closing record template.
• scripts/select-queue.sh — comments/output: active tiers lead with open milestones/review/priorities; drop "read #237" phrasing.
• Structured closing record — documented template (Outcome / Root cause / Decisions-conventions changed / Reusable knowledge / Verification / Deferred-follow-ups / Docs updated) as the future per-issue history record. No second manual ledger.

Regression check (#520 "cannot regress" box)

scripts/check-kickoff-guard.sh (shares the lifecycle CI job): fails if active kickoff/README/ CLAUDE text reintroduces "#237 is the queue / source of truth / read #237 for current state" patterns. Allow-list the explicitly-archival mentions.

Migration plan (#521 — full, via subagents)

1. Land schema rules + validator + catalog generator + archive skeleton first (validator tolerant of legacy-unmigrated).
2. Inventory legacy entries by stable subject (not just chronology); assign keys.
3. Reconcile known prose-only reversals/retirements first (#390→#406 supersession; #411 obsolescence) — orchestrator/escalated, not a cheap subagent.
4. Fan out mechanical reformatting in bounded topic batches (each subagent: exact schema + its entries + its own branch). Merge back.
5. Move superseded/retired records to docs/decisions/archive/ with links.
6. Regenerate the catalog; flip aggregate budget check on; drive legacy-unmigrated to zero.
7. Preserve an auditable mapping from every legacy heading to its active record or archive location (a migration map committed alongside).

Safety valve: if mechanical migration balloons, land complete machinery + a partial migration with a visible remainder count + a tracked follow-up issue (issue done-when explicitly allows a bounded remainder with no ambiguous active rules).

Retrieval evaluation (#521)

A bounded question bank (committed, e.g. docs/decisions/retrieval-eval.md or a test fixture) covering: paraphrased task→decision discovery; exact code/path lookup; active-vs-superseded; retired features; rationale/rejected-alternatives; and at least one "convention already implemented" question that must prevent reimplementation. Success = correct active-record selection + citation, not merely a semantically-related passage. Executed as the cold-agent step below (deterministic grading where the answer is a specific key/citation).

Verification

• Deterministic: validator + regression check + catalog freshness all green locally before push.
• Behavioral (cold-agent sim, #520 done-when): a fresh subagent handed only the new kickoff/README produces the startup flow — orientation ‖ selector, resolve flags, recheck+claim winner, focused retrieval — and never reads #237. Second cold agent runs a sample of the retrieval question bank against the new active view.
• Independent adversarial review of the whole diff (mandatory: >150 lines, touches CI) before push; cross-model if available, else a cold review-only agent.
• Docs updated in the same PR (README index, decisions header, ci-cd release ritual, CLAUDE.md).

CI / release-ritual changes

• Rename/extend the decisions-guard job → run decisions-validate.py + check-kickoff-guard.sh + catalog freshness.
• Release ritual (docs/ci-cd.md → Versioning & releases): change "discover superseded entries" → "validate lifecycle metadata + reciprocal links; archive already-classified history; refresh the active catalog; enforce the aggregate budget; report unresolved legacy records."
• Remove [decisions-edit] from the Husky commit-msg hook and the CI job; update docs/memory that reference the token.

Risks

• Conflict surface: decisions.md conflicts on nearly every CI cycle here; this arc rewrites it wholesale. Mitigation: single PR, rebase-then-merge_when_checks_succeed, land fast.
• Regenerated-artifact merges: on rebase conflicts in the generated catalog, regenerate (never hand-merge), same discipline as v1.json.
• Memory/lore references to [decisions-edit] and "#237 is the queue" are spread across MEMORY.md and the handoff lore; sweep by subject.