# Decision-lifecycle + retrieval-efficient startup — design (ersatztv #520 + #521)

Date: 2026-07-21
Issues: [#521](http://192.168.1.95:3000/timothy/ersatztv/issues/521) (decision knowledge lifecycle),
[#520](http://192.168.1.95:3000/timothy/ersatztv/issues/520) (retire #237 from startup; parallel orientation + selection).
Both originate from a bounded knowledge-retrieval audit in `timothy/adversarial-reviewer`.

## Goal

Two coupled problems from the same audit:

1. **#521** — `docs/decisions.md` (~2,468 lines; ~3,469 incl. topic files) is optimized for *append
   safety*, not *current-state retrieval*. Status lives in prose, supersession is inferred at
   release time, and fresh agents are told to read the corpus broadly before the task is known.
2. **#520** — Startup still tells agents (and the selector fallback) to read the closed arc tracker
   #237 and its stale forward-looking comments, and preloads the whole doc corpus before the task is
   known. `scripts/select-queue.sh` already does live mechanical selection correctly.

The fix is one arc: **give decisions a stable identity + explicit lifecycle + a compact generated
active view (#521), then point startup at that compact view and run orientation ‖ selection in
parallel (#520).** Keep Git-backed Markdown authoritative; MemPalace stays candidate-discovery only.

## Non-goals

- Making MemPalace the source of truth; storing live queue state in decision docs.
- Reopening #237 or creating a new manual session-log issue / Markdown ledger.
- Implementing the MemPalace/Gitea exporter (`server-management#642`).
- Changing `select-queue.sh`'s dependency/tier/priority algorithm (only its comments/output text),
  unless a test exposes a bug.

## Delegation model (agreed)

- **Judgment (orchestrator / escalated):** schema, validator, catalog generator, kickoff + docs
  rewrite, and reconciling the prose-only reversals (#390→#406; #411 measurement obsolescence).
- **Mechanical (balanced/cheap subagents, fanned out by topic batch):** reformatting existing
  entries into the keyed schema. Each subagent gets exact schema rules + the batch's entries + a
  worktree/branch of its own; results merge back. No two committing agents share a worktree.

## Guard rework (the crux)

The current append-only guard blocks **any** modified/deleted line in `decisions.md` unless the
commit carries `[decisions-edit]` (`.claude/hooks/decisions-guard.sh`, wired to Husky `commit-msg`
and the CI `decisions-guard` job). That line-level mechanic is **incompatible** with a lifecycle
model where migration reformats every entry, supersession *edits* a predecessor's `superseded-by:`
line, and retiring *moves* a record to archive.

**Decision:** replace the line-level mechanic with a **lifecycle validator** that enforces the same
*spirit* — rationale is never silently rewritten or deleted; every history touch is deliberate and
reviewable. The `[decisions-edit]` token is **kept narrow, not retired** (revised after Fable review):
routine lifecycle *metadata* writes (add record, set `superseded-by`, relocate to archive, regen
catalog) are token-free and proven well-formed by the validator; a change to a record's *rationale
prose* still requires `[decisions-edit]`, enforced by the validator's **body-diff**. Retiring the
token entirely would let an agent silently rewrite rationale (or archive a falsified body) and pass —
the exact case the old guard existed for.

Spirit preserved by these validator invariants (see Validator below): ≤1 active record per key; no
record block may vanish (if it leaves the active set it must reappear under `docs/decisions/archive/`);
**a surviving record's rationale prose can't change, and an archived copy must body-match its pre-move
active version, without `[decisions-edit]` in the commit range**; supersession links are reciprocal
and resolvable; status only moves active → superseded/retired. The no-vanish/body-diff checks run in
CI over `merge-base(base,head)…head` across `decisions.md` **and** every topic file.

**Delivery: two PRs, one arc** (revised after Fable review). PR1 = machinery (parser, validator,
catalog generator, guard swap, header rewrite, exemplar + append-only→lifecycle supersession); PR2 =
full corpus migration + #520 startup rewrite + kickoff guard + retrieval-eval. Matches #521's
"bounded topic batches, avoid a single conflict-heavy rewrite" and keeps each diff's conflict surface
small on the treadmill-prone `decisions.md`.

## Information model (#521)

Each decision record carries a compact, **visible**, deterministically-parseable metadata block
immediately under its `##` heading (front-loaded so MemPalace indexes key/status/rule first):

```
## 2026-07-17 — No persistent compiler servers in CI … (#406)
`key: ci.runner-placement` · `status: active` · `since: 2026-07-17` · `supersedes: none` · `superseded-by: none`
**Rule:** Every CI `services:` container gets an explicit CPU/mem cap; no shared persistent compiler daemon.
**Signals:** ci, runners, services, memory-cap · paths: .gitea/workflows/docker-build.yml · issues: #390 #406
**Mechanics:** docs/ci-cd.md → CI lanes
<existing rationale prose, incl. rejected alternatives — unchanged>
```

Fields (issue-required):

- `key` — stable subject identity, dotted lowercase (`^[a-z0-9]+(\.[a-z0-9-]+)+$`), e.g.
  `ci.runner-placement`. A superseding record reuses the predecessor's key.
- `status` — `active` | `superseded` | `retired` | `legacy-unmigrated` (the last is a transitional
  marker; see Migration). "Relocated"/"consolidated" are maintenance actions, not statuses.
- `since` — effective date (YYYY-MM-DD) or version.
- `supersedes` / `superseded-by` — predecessor/successor key+date, or `none`.
- `**Rule:**` — one-line current rule, before the long rationale.
- `**Signals:**` — concepts, `paths:`, `issues:` for task-driven retrieval.
- `**Mechanics:**` — link(s) to the current convention/source/test anchors.
- Rationale prose (incl. rejected alternatives) follows, unchanged from today where migrated.

Rejected alternatives stay *inside* their accepted record as rationale; they are never active
records.

## Document architecture (#521)

- `docs/decisions.md` — stays the chronological record store for in-file (non-topic) decisions, now
  in schema form. Header rewritten: append-only-log framing → lifecycle framing.
- `docs/decisions/*.md` — existing topic files, entries migrated to schema form.
- **`docs/decisions/README.md` — NEW generated active catalog / task router.** A compact table
  (key · one-line rule · status · record link · signals) built by
  `scripts/build-decisions-catalog.py` from all `active` records across `decisions.md` + topic files.
  Regenerated like `endpoint-index.md`; drift fails CI. This is the compact "active view" startup
  points at.
- **`docs/decisions/archive/` — NEW.** `superseded`/`retired` records relocated here, out of the
  default startup path, with forward/back links preserved. Not scanned by the catalog generator.

## Validator (#521)

`scripts/decisions-validate.py`, run locally (a Husky hook replacing the append-only commit-msg
check) and in the CI `decisions-guard` job (renamed → lifecycle guard). Checks:

1. Required metadata present + well-formed on every structured record.
2. Valid `status` enum; valid `key` format.
3. **≤1 `active` record per key.**
4. Reciprocal, resolvable `supersedes`/`superseded-by` links (successor exists; predecessor points
   back).
5. **No-vanish (diff-aware):** any record block removed from the active files (`decisions.md` +
   topic files) between base and HEAD must be present under `docs/decisions/archive/`. Fail-open on
   missing refs, like the current guard.
6. Active-catalog completeness + freshness (regenerate to a temp file, diff — like `check:api`).
7. Archive/active placement consistency (no `active` record in archive; no `superseded`/`retired`
   in the active set once migration of its key is complete).
8. **Aggregate active-corpus budget** — sum of lines across `decisions.md` + topic files + catalog
   (excluding archive). Warn over a threshold (start at the current 1800 applied to the aggregate;
   tune during migration). Report the `legacy-unmigrated` remainder count; it must be visible and
   trend to zero.
9. Best-effort broken-link check on `Mechanics:`/archive links where practical.

Fail-open on tooling trouble, matching the current guard's philosophy.

## Startup / kickoff rewrite (#520)

- **`docs/handoffs/chicorytv-issue-queue.md`** — retire #237 from normal startup. Two concurrent
  tracks when no issue is named: **orientation** (`AGENTS.md`, `CLAUDE.md`, compact `docs/README.md`
  map + the active catalog) ‖ **selection** (`scripts/select-queue.sh 5`). After both: resolve only
  `CLAIM?`/`UMBRELLA?` flags + the winner, recheck live state, claim, then build a focused
  task-specific knowledge packet. Named-issue path skips selection → focused retrieval directly.
  Archive the contradictory *pre-script* selector lore (the "re-derive the whole contested tier by
  hand" bullets) into a clearly-labeled historical section. #237 appears only as labeled archival
  history.
- **`docs/README.md`** — mandatory 1–10 reading order → compact **task-signal → minimal sections**
  authority/task map, plus a pointer to `docs/decisions/README.md` as the decision active view.
- **`CLAUDE.md`** — docs-first guidance requires the map + relevant sections, not the whole corpus.
  Task-completion protocol gains the structured `## Closing record` template.
- **`scripts/select-queue.sh`** — comments/output: active tiers lead with open
  milestones/review/priorities; drop "read #237" phrasing.
- **Structured closing record** — documented template (Outcome / Root cause / Decisions-conventions
  changed / Reusable knowledge / Verification / Deferred-follow-ups / Docs updated) as the future
  per-issue history record. No second manual ledger.

### Regression check (#520 "cannot regress" box)

`scripts/check-kickoff-guard.sh` (shares the lifecycle CI job): fails if active kickoff/README/
CLAUDE text reintroduces "#237 is the queue / source of truth / read #237 for current state"
patterns. Allow-list the explicitly-archival mentions.

## Migration plan (#521 — full, via subagents)

1. Land schema rules + validator + catalog generator + archive skeleton first (validator tolerant of
   `legacy-unmigrated`).
2. Inventory legacy entries by stable subject (not just chronology); assign keys.
3. Reconcile known prose-only reversals/retirements first (#390→#406 supersession; #411
   obsolescence) — orchestrator/escalated, not a cheap subagent.
4. Fan out mechanical reformatting in bounded topic batches (each subagent: exact schema + its
   entries + its own branch). Merge back.
5. Move superseded/retired records to `docs/decisions/archive/` with links.
6. Regenerate the catalog; flip aggregate budget check on; drive `legacy-unmigrated` to zero.
7. Preserve an auditable mapping from every legacy heading to its active record or archive location
   (a migration map committed alongside).

Safety valve: if mechanical migration balloons, land complete machinery + a partial migration with a
visible remainder count + a tracked follow-up issue (issue done-when explicitly allows a bounded
remainder with no ambiguous active rules).

## Retrieval evaluation (#521)

A bounded question bank (committed, e.g. `docs/decisions/retrieval-eval.md` or a test fixture)
covering: paraphrased task→decision discovery; exact code/path lookup; active-vs-superseded; retired
features; rationale/rejected-alternatives; and at least one "convention already implemented"
question that must prevent reimplementation. Success = correct **active-record selection + citation**,
not merely a semantically-related passage. Executed as the cold-agent step below (deterministic
grading where the answer is a specific key/citation).

## Verification

- **Deterministic:** validator + regression check + catalog freshness all green locally before push.
- **Behavioral (cold-agent sim, #520 done-when):** a fresh subagent handed only the new
  kickoff/README produces the startup flow — orientation ‖ selector, resolve flags, recheck+claim
  winner, focused retrieval — and **never reads #237**. Second cold agent runs a sample of the
  retrieval question bank against the new active view.
- **Independent adversarial review** of the whole diff (mandatory: >150 lines, touches CI) before
  push; cross-model if available, else a cold review-only agent.
- Docs updated in the same PR (README index, decisions header, ci-cd release ritual, CLAUDE.md).

## CI / release-ritual changes

- Rename/extend the `decisions-guard` job → run `decisions-validate.py` + `check-kickoff-guard.sh` +
  catalog freshness.
- Release ritual (`docs/ci-cd.md` → Versioning & releases): change "discover superseded entries" →
  "validate lifecycle metadata + reciprocal links; archive already-classified history; refresh the
  active catalog; enforce the aggregate budget; report unresolved legacy records."
- Remove `[decisions-edit]` from the Husky `commit-msg` hook and the CI job; update docs/memory that
  reference the token.

## Risks

- **Conflict surface:** `decisions.md` conflicts on nearly every CI cycle here; this arc rewrites it
  wholesale. Mitigation: single PR, rebase-then-`merge_when_checks_succeed`, land fast.
- **Regenerated-artifact merges:** on rebase conflicts in the generated catalog, regenerate (never
  hand-merge), same discipline as `v1.json`.
- **Memory/lore references** to `[decisions-edit]` and "#237 is the queue" are spread across
  MEMORY.md and the handoff lore; sweep by subject.
