This is why `decisions lifecycle` went red, and it was NOT the known flake. I
came close to dismissing it as one for the second time this session, because an
earlier red on another branch genuinely was.
The dual-format parser imported PyYAML to read frontmatter. `decisions-guard`
does `setup-python` and installs NOTHING, so once the corpus was migrated every
record became unparseable there: ModuleNotFoundError, job fails. The same would
hit the Husky pre-commit hook and every contributor's machine.
Installing PyYAML in CI is the wrong fix: READING happens everywhere -- CI, the
hook, every dev -- while WRITING happens once, in a migration a human runs
deliberately. So the read path is now dependency-free and only
`migrate_decisions_split` (the writer) still imports yaml.
A hand-rolled parser is only safe if it provably matches the library that WROTE
the files, so `test_frontmatter_reader_matches_pyyaml_on_every_real_record`
compares the two field-by-field across all 169 real records (importorskip, so it
is skipped rather than failing where PyYAML is absent) with a >100-file guard
against near-vacuity. It is narrow by construction: the frontmatter is
machine-generated with default_flow_style=False and width=10**9, so every value
is a single-line scalar, and the reader bails to None on anything nested.
Verified by running all four affected entry points against a shim that makes
`import yaml` raise: validate --base/--head, catalog --check, the kickoff guard,
and the plain validate the pre-commit hook calls. All exit 0.
refs #610
Live docs point into the corpus BY DATE -- "see `decisions.md` 2026-07-10" --
about 32 such references across 12 files, plus the same form in historical issue
comments. The split would dangle every one of them.
Each stripped file now ends with a generated "Records formerly in this file"
index: date, title, and a link to the record's new path. A reader following a
date pointer lands on the file it names and resolves from there. That is far
cheaper and less error-prone than rewriting 32 references by hand, and it also
covers the issue-comment breadcrumbs, which cannot be rewritten at all.
Caught while verifying it: the generated `## Records formerly in this file`
heading is itself an H2, so the record parser counted one legacy-unmigrated
record per stripped file -- the notice went 0 -> 6. Same treatment as the
existing `## Index` section: skip it by name. SKIP_HEADINGS moved to
decisions_lib as the single source of truth, since three modules now need it.
Found by reading the validator's notice output on a trial migration, not by
inspection -- the corpus still validated OK, so nothing else would have flagged it.
First step of the one-file-per-record split. `decisions_lib.parse_text` now
dispatches on a leading `---`: the legacy "many ## records per file, metadata on
a backtick line" form and the new "one record per file, YAML frontmatter" form
both produce the same Record.
This is the load-bearing move for the migration. Because the parser understands
both, the body-diff guard can parse the OLD form at the merge-base and the NEW
form at head -- both keyed on `key` -- so the commit that relocates 166 records
validates itself with no one-time bypass and no escape hatch.
Details:
- Frontmatter uses the SAME field vocabulary as the backtick line
(`superseded-by`, `stale-after`), so on-disk names don't shift mid-migration.
- YAML `None` (a key with no value) is preserved as "" rather than collapsing to
None, keeping the empty-vs-absent distinction `stale-after` depends on (#603).
- `active_files()` also walks `docs/decisions/records/**`, so both layouts can
coexist while the migration lands.
- Malformed frontmatter (unterminated fence, non-dict, YAML error) yields no
record rather than raising; the validator then reports it as a missing record.
1. `date.fromisoformat` is not a YYYY-MM-DD validator. On Python >= 3.11 it also
accepts ISO basic format ("20270101") and week dates ("2027-W01-1"), so a
malformed-looking value passed the blocking check — and which forms parse
depends on the interpreter, meaning the same corpus could validate differently
on a dev machine and on the runner (pr-checks.yml pins only python-version
'3.x'). Knock-on: the catalog's Review-due section sorts on the raw STRING, so
an accepted "20270101" sorted AFTER "2027-01-15" ('-' < '0'), contradicting the
section's own "sorted soonest-first" text. Gate on ^\d{4}-\d{2}-\d{2}$ first,
which fixes both — a fixed-width zero-padded form makes string sort == date sort.
2. A present-but-empty `stale-after:` was collapsed to None by `or None` in the
parser and then skipped by a truthiness guard in the validator, so it passed as
"absent" — a field that silently never fires, which is the exact failure mode
the blocking check exists to prevent. Keep "" distinct from None and test with
`is not None`.
3. `test_catalog_is_date_independent` was partly vacuous: with no date in either
render, both sides were trivially equal after the .replace(). It did still catch
an injected clock-derived marker, but it passed with the feature deleted. Assert
the dates are present.
4. The malformed-date check ran only over the active set, exempting archive
records. Staleness is moot there, but a typo is still a typo — check both wings.
Adds regression tests for each, plus a Review-due row for a topic-file record
(pinning the `../decisions.md` vs bare-filename link forms).
Evaluated the Open Knowledge Format (GoogleCloudPlatform/knowledge-catalog okf
v0.2, scaccogatto/okf-skills) as a replacement for our decision-record system and
rejected it: its conformance rules are deliberately permissive exactly where ours
are strict (broken links, unknown types and missing fields must all be tolerated;
`deprecated` points at no successor), and its stable identity is the file path,
which the breadcrumb rule tells agents not to trust.
Adopted two of its optional families instead, additively:
- `stale-after: YYYY-MM-DD` on the metadata line — marks a record asserting an
outside-world fact as due for re-confirmation. Absolute date, no TTL.
- `**Sources:**` in the metadata block — the evidence a record rests on, as
distinct from `Signals:` (recall keywords).
Neither is required; absence is never an error. A malformed `stale-after` is
blocking (it would silently never fire), but a past-due record is only a
non-blocking `::notice::` — going stale is the passage of time, not a defect in
whatever commit is under test. The catalog's new "Review due" section renders the
date only and never a clock-derived verdict, so it cannot drift `--check` red on a
calendar boundary with no commit touching the corpus.
No backfill: no existing record adopts either field here.
fixes#603
- Exclude docs/decisions/retrieval-eval.md from active decision parsing
(_NON_DECISION_FILES); its `## N.` eval-question headings were being
miscounted as 7 legacy-unmigrated records.
- Add decisions_lib.metadata_line_count() + a decisions_validate guard
that fails a record with more than one `key:` metadata line, so a
stacked-metadata-block migration bug (which the parser silently
tolerated by reading only the first block) can't recur unnoticed.
TDD: test_duplicate_metadata_block_fails / test_single_metadata_block_passes.
- De-duplicate the 6 docs/decisions.md records left with two stacked
metadata blocks (scan.getoraddfolder-db-lookup #488,
scan.musicvideo-reconciliation #494, scan.jellyfin-mixed-content-library
#489, iptv.logo-drives-bug-preset #67, ffmpeg.qsv-decode-encode-split
#498, ci.small-lane-git-only server-management#639), merging the union
of Signals/paths/issues/Mechanics from both blocks and keeping the
richer Rule wording; rationale prose untouched.
- Fill in the deferred Q6b row in docs/decisions/retrieval-eval.md now
that startup.parallel-orientation is active in docs/decisions.md,
scoring it as a real active-vs-superseded question against the
archived docs.queue-state-gitea-tracker.
- Regenerate docs/decisions/README.md via build_decisions_catalog.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds scripts/decisions_lib.py, the shared parser for ErsatzTV decision
records (docs/decisions.md + docs/decisions/*.md). Parses H2 sections
into Record dataclasses, distinguishing migrated records (visible
metadata block: key/status/since/supersedes/superseded-by + Rule/
Signals/Mechanics) from legacy-unmigrated ones with no metadata line.
scripts/ is now an importable package (scripts/__init__.py,
scripts/tests/__init__.py) so later tools can `import scripts.decisions_lib`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
timothy
