From 2af3cd024e13147b5162979dfde18668057e06b4 Mon Sep 17 00:00:00 2001 From: Timothy Date: Tue, 21 Jul 2026 01:20:04 +0200 Subject: [PATCH] =?UTF-8?q?docs(520,521):=20design=20spec=20=E2=80=94=20de?= =?UTF-8?q?cision=20lifecycle=20+=20retrieval-efficient=20startup?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Design for the decision-knowledge lifecycle schema/catalog/validator (#521) and the #237-retirement / parallel-startup kickoff rewrite (#520), as one arc. Refs #520 #521 Co-Authored-By: Claude Opus 4.8 (1M context) --- ...1-decision-lifecycle-and-kickoff-design.md | 205 ++++++++++++++++++ 1 file changed, 205 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-21-decision-lifecycle-and-kickoff-design.md diff --git a/docs/superpowers/specs/2026-07-21-decision-lifecycle-and-kickoff-design.md b/docs/superpowers/specs/2026-07-21-decision-lifecycle-and-kickoff-design.md new file mode 100644 index 000000000..b360ffe79 --- /dev/null +++ b/docs/superpowers/specs/2026-07-21-decision-lifecycle-and-kickoff-design.md @@ -0,0 +1,205 @@ +# Decision-lifecycle + retrieval-efficient startup — design (ersatztv #520 + #521) + +Date: 2026-07-21 +Issues: [#521](http://192.168.1.95:3000/timothy/ersatztv/issues/521) (decision knowledge lifecycle), +[#520](http://192.168.1.95:3000/timothy/ersatztv/issues/520) (retire #237 from startup; parallel orientation + selection). +Both originate from a bounded knowledge-retrieval audit in `timothy/adversarial-reviewer`. + +## Goal + +Two coupled problems from the same audit: + +1. **#521** — `docs/decisions.md` (~2,468 lines; ~3,469 incl. topic files) is optimized for *append + safety*, not *current-state retrieval*. Status lives in prose, supersession is inferred at + release time, and fresh agents are told to read the corpus broadly before the task is known. +2. **#520** — Startup still tells agents (and the selector fallback) to read the closed arc tracker + #237 and its stale forward-looking comments, and preloads the whole doc corpus before the task is + known. `scripts/select-queue.sh` already does live mechanical selection correctly. + +The fix is one arc: **give decisions a stable identity + explicit lifecycle + a compact generated +active view (#521), then point startup at that compact view and run orientation ‖ selection in +parallel (#520).** Keep Git-backed Markdown authoritative; MemPalace stays candidate-discovery only. + +## Non-goals + +- Making MemPalace the source of truth; storing live queue state in decision docs. +- Reopening #237 or creating a new manual session-log issue / Markdown ledger. +- Implementing the MemPalace/Gitea exporter (`server-management#642`). +- Changing `select-queue.sh`'s dependency/tier/priority algorithm (only its comments/output text), + unless a test exposes a bug. + +## Delegation model (agreed) + +- **Judgment (orchestrator / escalated):** schema, validator, catalog generator, kickoff + docs + rewrite, and reconciling the prose-only reversals (#390→#406; #411 measurement obsolescence). +- **Mechanical (balanced/cheap subagents, fanned out by topic batch):** reformatting existing + entries into the keyed schema. Each subagent gets exact schema rules + the batch's entries + a + worktree/branch of its own; results merge back. No two committing agents share a worktree. + +## Guard rework (the crux) + +The current append-only guard blocks **any** modified/deleted line in `decisions.md` unless the +commit carries `[decisions-edit]` (`.claude/hooks/decisions-guard.sh`, wired to Husky `commit-msg` +and the CI `decisions-guard` job). That line-level mechanic is **incompatible** with a lifecycle +model where migration reformats every entry, supersession *edits* a predecessor's `superseded-by:` +line, and retiring *moves* a record to archive. + +**Decision:** replace the line-level mechanic with a **lifecycle validator** that enforces the same +*spirit* — rationale is never silently rewritten or deleted; every history touch is deliberate and +reviewable. The `[decisions-edit]` token is **retired** (removed from the Husky hook, the CI job, +and docs/memory references); routine lifecycle writes are expected and are proven well-formed by the +validator instead of gated by a token. + +Spirit preserved by these validator invariants (see Validator below): ≤1 active record per key; no +record block may vanish (if it leaves the active set it must reappear under `docs/decisions/archive/`); +supersession links are reciprocal and resolvable; status only moves active → superseded/retired. + +## Information model (#521) + +Each decision record carries a compact, **visible**, deterministically-parseable metadata block +immediately under its `##` heading (front-loaded so MemPalace indexes key/status/rule first): + +``` +## 2026-07-17 — No persistent compiler servers in CI … (#406) +`key: ci.runner-placement` · `status: active` · `since: 2026-07-17` · `supersedes: none` · `superseded-by: none` +**Rule:** Every CI `services:` container gets an explicit CPU/mem cap; no shared persistent compiler daemon. +**Signals:** ci, runners, services, memory-cap · paths: .gitea/workflows/docker-build.yml · issues: #390 #406 +**Mechanics:** docs/ci-cd.md → CI lanes + +``` + +Fields (issue-required): + +- `key` — stable subject identity, dotted lowercase (`^[a-z0-9]+(\.[a-z0-9-]+)+$`), e.g. + `ci.runner-placement`. A superseding record reuses the predecessor's key. +- `status` — `active` | `superseded` | `retired` | `legacy-unmigrated` (the last is a transitional + marker; see Migration). "Relocated"/"consolidated" are maintenance actions, not statuses. +- `since` — effective date (YYYY-MM-DD) or version. +- `supersedes` / `superseded-by` — predecessor/successor key+date, or `none`. +- `**Rule:**` — one-line current rule, before the long rationale. +- `**Signals:**` — concepts, `paths:`, `issues:` for task-driven retrieval. +- `**Mechanics:**` — link(s) to the current convention/source/test anchors. +- Rationale prose (incl. rejected alternatives) follows, unchanged from today where migrated. + +Rejected alternatives stay *inside* their accepted record as rationale; they are never active +records. + +## Document architecture (#521) + +- `docs/decisions.md` — stays the chronological record store for in-file (non-topic) decisions, now + in schema form. Header rewritten: append-only-log framing → lifecycle framing. +- `docs/decisions/*.md` — existing topic files, entries migrated to schema form. +- **`docs/decisions/README.md` — NEW generated active catalog / task router.** A compact table + (key · one-line rule · status · record link · signals) built by + `scripts/build-decisions-catalog.py` from all `active` records across `decisions.md` + topic files. + Regenerated like `endpoint-index.md`; drift fails CI. This is the compact "active view" startup + points at. +- **`docs/decisions/archive/` — NEW.** `superseded`/`retired` records relocated here, out of the + default startup path, with forward/back links preserved. Not scanned by the catalog generator. + +## Validator (#521) + +`scripts/decisions-validate.py`, run locally (a Husky hook replacing the append-only commit-msg +check) and in the CI `decisions-guard` job (renamed → lifecycle guard). Checks: + +1. Required metadata present + well-formed on every structured record. +2. Valid `status` enum; valid `key` format. +3. **≤1 `active` record per key.** +4. Reciprocal, resolvable `supersedes`/`superseded-by` links (successor exists; predecessor points + back). +5. **No-vanish (diff-aware):** any record block removed from the active files (`decisions.md` + + topic files) between base and HEAD must be present under `docs/decisions/archive/`. Fail-open on + missing refs, like the current guard. +6. Active-catalog completeness + freshness (regenerate to a temp file, diff — like `check:api`). +7. Archive/active placement consistency (no `active` record in archive; no `superseded`/`retired` + in the active set once migration of its key is complete). +8. **Aggregate active-corpus budget** — sum of lines across `decisions.md` + topic files + catalog + (excluding archive). Warn over a threshold (start at the current 1800 applied to the aggregate; + tune during migration). Report the `legacy-unmigrated` remainder count; it must be visible and + trend to zero. +9. Best-effort broken-link check on `Mechanics:`/archive links where practical. + +Fail-open on tooling trouble, matching the current guard's philosophy. + +## Startup / kickoff rewrite (#520) + +- **`docs/handoffs/chicorytv-issue-queue.md`** — retire #237 from normal startup. Two concurrent + tracks when no issue is named: **orientation** (`AGENTS.md`, `CLAUDE.md`, compact `docs/README.md` + map + the active catalog) ‖ **selection** (`scripts/select-queue.sh 5`). After both: resolve only + `CLAIM?`/`UMBRELLA?` flags + the winner, recheck live state, claim, then build a focused + task-specific knowledge packet. Named-issue path skips selection → focused retrieval directly. + Archive the contradictory *pre-script* selector lore (the "re-derive the whole contested tier by + hand" bullets) into a clearly-labeled historical section. #237 appears only as labeled archival + history. +- **`docs/README.md`** — mandatory 1–10 reading order → compact **task-signal → minimal sections** + authority/task map, plus a pointer to `docs/decisions/README.md` as the decision active view. +- **`CLAUDE.md`** — docs-first guidance requires the map + relevant sections, not the whole corpus. + Task-completion protocol gains the structured `## Closing record` template. +- **`scripts/select-queue.sh`** — comments/output: active tiers lead with open + milestones/review/priorities; drop "read #237" phrasing. +- **Structured closing record** — documented template (Outcome / Root cause / Decisions-conventions + changed / Reusable knowledge / Verification / Deferred-follow-ups / Docs updated) as the future + per-issue history record. No second manual ledger. + +### Regression check (#520 "cannot regress" box) + +`scripts/check-kickoff-guard.sh` (shares the lifecycle CI job): fails if active kickoff/README/ +CLAUDE text reintroduces "#237 is the queue / source of truth / read #237 for current state" +patterns. Allow-list the explicitly-archival mentions. + +## Migration plan (#521 — full, via subagents) + +1. Land schema rules + validator + catalog generator + archive skeleton first (validator tolerant of + `legacy-unmigrated`). +2. Inventory legacy entries by stable subject (not just chronology); assign keys. +3. Reconcile known prose-only reversals/retirements first (#390→#406 supersession; #411 + obsolescence) — orchestrator/escalated, not a cheap subagent. +4. Fan out mechanical reformatting in bounded topic batches (each subagent: exact schema + its + entries + its own branch). Merge back. +5. Move superseded/retired records to `docs/decisions/archive/` with links. +6. Regenerate the catalog; flip aggregate budget check on; drive `legacy-unmigrated` to zero. +7. Preserve an auditable mapping from every legacy heading to its active record or archive location + (a migration map committed alongside). + +Safety valve: if mechanical migration balloons, land complete machinery + a partial migration with a +visible remainder count + a tracked follow-up issue (issue done-when explicitly allows a bounded +remainder with no ambiguous active rules). + +## Retrieval evaluation (#521) + +A bounded question bank (committed, e.g. `docs/decisions/retrieval-eval.md` or a test fixture) +covering: paraphrased task→decision discovery; exact code/path lookup; active-vs-superseded; retired +features; rationale/rejected-alternatives; and at least one "convention already implemented" +question that must prevent reimplementation. Success = correct **active-record selection + citation**, +not merely a semantically-related passage. Executed as the cold-agent step below (deterministic +grading where the answer is a specific key/citation). + +## Verification + +- **Deterministic:** validator + regression check + catalog freshness all green locally before push. +- **Behavioral (cold-agent sim, #520 done-when):** a fresh subagent handed only the new + kickoff/README produces the startup flow — orientation ‖ selector, resolve flags, recheck+claim + winner, focused retrieval — and **never reads #237**. Second cold agent runs a sample of the + retrieval question bank against the new active view. +- **Independent adversarial review** of the whole diff (mandatory: >150 lines, touches CI) before + push; cross-model if available, else a cold review-only agent. +- Docs updated in the same PR (README index, decisions header, ci-cd release ritual, CLAUDE.md). + +## CI / release-ritual changes + +- Rename/extend the `decisions-guard` job → run `decisions-validate.py` + `check-kickoff-guard.sh` + + catalog freshness. +- Release ritual (`docs/ci-cd.md` → Versioning & releases): change "discover superseded entries" → + "validate lifecycle metadata + reciprocal links; archive already-classified history; refresh the + active catalog; enforce the aggregate budget; report unresolved legacy records." +- Remove `[decisions-edit]` from the Husky `commit-msg` hook and the CI job; update docs/memory that + reference the token. + +## Risks + +- **Conflict surface:** `decisions.md` conflicts on nearly every CI cycle here; this arc rewrites it + wholesale. Mitigation: single PR, rebase-then-`merge_when_checks_succeed`, land fast. +- **Regenerated-artifact merges:** on rebase conflicts in the generated catalog, regenerate (never + hand-merge), same discipline as `v1.json`. +- **Memory/lore references** to `[decisions-edit]` and "#237 is the queue" are spread across + MEMORY.md and the handoff lore; sweep by subject.