feat(610): split the decision corpus into one YAML-frontmatter file per record
168 records -> docs/decisions/records/<area>/<topic>.md (163 active, 23 dirs) and docs/decisions/archive/<area>/<topic>.md (5 archived). The filename IS the key, so one-active-record-per-key becomes a filesystem property rather than a validator check, and supersession becomes a `git mv`. WHY: the monolith was a concurrency problem before an aesthetic one. A 3,900-line append target made parallel sessions collide -- PR #605 and PR #614 both hit append-vs-append conflicts during routine rebases, and hand-resolving those inside the corpus is exactly the operation the rationale-rewrite guard exists to police. HOW IT IS VERIFIED: a ~170-file diff cannot be meaningfully read, so correctness does not rest on reading it. The parser was taught BOTH formats first, so the body-diff guard parses the old form at the merge-base and the new form at head -- the migration validates itself, no bypass. The proof is a field-level equivalence harness: 168 records before and after, zero lost, zero gained, zero field mismatches, zero rationale bodies differing. Reviewers should scrutinise the harness; it is the actual evidence. What measuring caught that reading would not have: - ~500 lines sit OUTSIDE any record -- decisions.md's lifecycle schema and each topic file's preamble, mostly the only copy. Source files are kept and stripped, never deleted. They also cannot be filed per-area: topic files hold several areas and 4 of 23 areas span several files. - Archive discovery was a non-recursive glob; after the split it found ZERO archived records, surfacing as four bogus "supersedes points to unknown key" errors rather than an obvious failure. - ~32 live docs point into the corpus BY DATE, which the split dangles. Each stripped file now ends with a generated "Records formerly in this file" index, which also rescues the identical breadcrumbs in old issue comments. - decisions.md's "In this file:" list was 97 same-file anchor bullets that the split makes WRONG, not merely stale. Dropped; the generated index replaces them with links that resolve. The equivalence harness now runs against a checked-in FIXTURE, not the live corpus. The earlier version migrated the real tree, which made it a one-shot: the moment the migration landed there was nothing left to move and the tests failed for reasons unrelated to the code. A fixture keeps them testing the SCRIPT rather than the repo's current state. Keys preserved verbatim, warts included: `sched` (12) and `scheduling` (1) remain two directories for one concept. Renaming a key is not a move -- it changes identity, breaks the equivalence proof, and invalidates MemPalace's per-key drawers. Taxonomy normalisation is separate work. refs #610
This commit is contained in:
@@ -13,15 +13,19 @@ Two properties this script exists to guarantee, both checked by
|
||||
1. **Nothing is rewritten.** Every record's rationale body is copied byte-for-byte, and every
|
||||
metadata field round-trips unchanged. The migration is a MOVE, so a field-level before/after
|
||||
comparison is a complete correctness proof.
|
||||
2. **No prose is lost.** 506 lines of the corpus are NOT inside any record — file preambles that
|
||||
explain why a topic file exists, cross-reference notes, and decisions.md's lifecycle schema
|
||||
header. Those files are kept and stripped to exactly that prose, never deleted. (They cannot be
|
||||
filed per-area: topic files hold several areas, and 4 of 23 areas span several files.)
|
||||
2. **No prose is lost.** A few hundred lines of the corpus are NOT inside any record — file
|
||||
preambles that explain why a topic file exists, cross-reference notes, and decisions.md's
|
||||
lifecycle schema header. Those files are kept and stripped to exactly that prose, never deleted.
|
||||
(They cannot be filed per-area: topic files hold several areas, and 4 of 23 areas span several
|
||||
files.) The one thing deliberately dropped is same-file anchor bullets, which the split makes
|
||||
*wrong* rather than merely stale — the generated index replaces them with links that resolve.
|
||||
Run with `--dry-run` for the current counts rather than trusting a number written here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
@@ -121,8 +125,19 @@ def _moved_index(records: list[dl.Record]) -> str:
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# A bullet whose link is a bare same-file anchor, e.g. `- [2026-07-10 — Title](#2026-07-10--title)`.
|
||||
# decisions.md's "In this file:" index is ~97 of these. Once the records move out, every one of them
|
||||
# points at a heading that no longer exists in that file — and the generated "Records formerly in
|
||||
# this file" section replaces them with links that actually resolve. Anything else keeps its bullet.
|
||||
_SAME_FILE_ANCHOR_BULLET = re.compile(r"^\s*[-*]\s*\[[^\]]*\]\(#[^)]*\)\s*$")
|
||||
|
||||
|
||||
def _preamble(path: Path, records: list[dl.Record]) -> str:
|
||||
"""Everything in `path` that is not inside a record — kept, never deleted."""
|
||||
"""Everything in `path` that is not inside a record — kept, never deleted.
|
||||
|
||||
Two exceptions are dropped rather than kept, because the split makes them *wrong* rather than
|
||||
merely stale: same-file anchor bullets (see above).
|
||||
"""
|
||||
text = path.read_text(encoding="utf-8")
|
||||
lines = text.splitlines()
|
||||
drop: set[int] = set()
|
||||
@@ -130,10 +145,16 @@ def _preamble(path: Path, records: list[dl.Record]) -> str:
|
||||
start = r.lineno - 1 # heading line, 1-indexed in Record
|
||||
for n in range(start, start + 1 + len(r.body.splitlines())):
|
||||
drop.add(n)
|
||||
kept = [ln for n, ln in enumerate(lines) if n not in drop]
|
||||
while kept and not kept[-1].strip():
|
||||
kept.pop()
|
||||
return "\n".join(kept)
|
||||
kept = [ln for n, ln in enumerate(lines) if n not in drop and not _SAME_FILE_ANCHOR_BULLET.match(ln)]
|
||||
# collapse the blank runs left behind where a bullet list used to be
|
||||
out: list[str] = []
|
||||
for ln in kept:
|
||||
if not ln.strip() and out and not out[-1].strip():
|
||||
continue
|
||||
out.append(ln)
|
||||
while out and not out[-1].strip():
|
||||
out.pop()
|
||||
return "\n".join(out)
|
||||
|
||||
|
||||
def plan() -> tuple[list[tuple[dl.Record, Path]], dict[Path, str], dict[Path, int]]:
|
||||
|
||||
Reference in New Issue
Block a user