feat(603): adopt OKF's optional stale-after and Sources: decision-record metadata #605

Merged
timothy merged 4 commits from worktree-603-decisions-stale-after-sources into main 2026-07-25 16:16:00 +02:00
Owner

fixes #603

Evaluated OKF as a replacement for our decision-record system, rejected it, and adopted two of its optional metadata families additively. Rationale is in the new record (docs.decision-optional-provenance); the short version is that OKF's conformance rules are permissive exactly where ours are strict — broken cross-links, unknown types and missing fields must all be tolerated, deprecated points at no successor, and its stable identity is the file path, which the breadcrumb rule tells agents not to trust.

Three commits: the feature, the fixes from an adversarial review, and the backfill.

What changed

decisions_lib.py parse stale-after off the metadata line, **Sources:** off the metadata block
decisions_validate.py malformed stale-after → blocking error; past-due active record → non-blocking ::notice:: via stale_records()
build_decisions_catalog.py new Review due section, sorted soonest-first, rendering the date only
docs/decisions.md, docs/decisions/workflow-process.md header schema, the new active record, and the backfill onto three ci.* records
docs/README.md schema line kept accurate

Neither field is in REQUIRED_META. No format migration, no change to key/status/since/supersedes/superseded-by/Signals:.

The two design constraints that drove it

Both exist to avoid a self-inflicted CI outage:

  1. Past-due is never blocking. A record going stale is the passage of time, not a defect in whatever commit happens to be under test. A blocking check would turn an unrelated PR red on a calendar boundary.
  2. The generated catalog is clock-free. It's checked with --check in CI, so anything time-dependent in it would silently drift red with no commit touching the corpus. Evaluating today >= stale-after lives only on the validator's notice path; the catalog renders the date.

A malformed date is blocking, because it would otherwise silently never fire.

Adversarial review found four defects — all fixed in f72bcc9a

Two of them broke the very invariant the feature claims to enforce. Worth reading:

  1. date.fromisoformat is not a YYYY-MM-DD validator. On Python ≥3.11 it also accepts ISO basic format (20270101) and week dates (2027-W01-1), so malformed-looking values passed the blocking check — and which forms parse depends on the interpreter, meaning the same corpus could validate differently on a dev machine than on the runner (pr-checks.yml pins only python-version: '3.x'). Knock-on: the Review-due section sorts on the raw string, so an accepted 20270101 sorted after 2027-01-15 ('-' < '0'), contradicting the section's own "soonest-first" text. Fixed with a ^\d{4}-\d{2}-\d{2}$ gate, which closes both — a fixed-width zero-padded form is what makes string sort ≡ date sort.
  2. An empty stale-after: passed as "absent". or None in the parser plus a truthiness guard in the validator collapsed "" into None, so a blank or truncated value produced no error and no notice, ever — precisely the silently-never-fires failure the blocking check exists to prevent.
  3. test_catalog_is_date_independent was partly vacuous — with no date in either render, both sides were trivially equal after the .replace(). It did still catch an injected clock-derived marker, but it passed with the feature deleted.
  4. The malformed-date check exempted archive records. Staleness is moot there, but a typo is still a typo.

Each has a regression test. Fixes verified by independently reproducing the reviewer's exploit cases against the patched code, not by re-running the author's own tests.

Backfill (738c0e25)

The mechanism shipped unused, so three records that encode measured host behavior now adopt both fields, staggered so they don't all come due at once:

Record stale-after Why it drifts
ci.runner-placement 2027-01-15 Encodes bumblebee's sizing (25 GiB / 12 cores) and container caps. Premise has already partly moved — transcoders left for jazz 2026-07-20, runners retuned since.
ci.infra-shaped-red-under-load 2027-02-15 A triage heuristic calibrated against load 243 on a since-retuned host.
ci.peak-anon-measurement 2027-03-15 Weakest of the three, but does assert memory.peak is cache-inflated — a cgroup fact.

Confirmed empirically that amending three existing records needs no [decisions-edit] token: both fields land inside the contiguous metadata block that _rationale() strips before the body-diff, consistent with Rule/Signals/Mechanics. --base main --head HEAD passes with zero tokens in the branch.

Verification

51 tests passing; ruff, ruff format and pyright clean on the touched files; decisions-validate: OK; catalog up to date. Beyond unit tests, a live probe against the real corpus confirmed past-due → exit 0 + notice, malformed → exit 1, and past-vs-future catalogs byte-identical modulo the date string.

Recording one process note: the first run of that probe reported two failures, both artifacts of the probe's own arrangement — I validated before regenerating the catalog (so the real error was catalog-drift, not staleness), and my "is today baked in?" check matched 2026-07-25, which is legitimately in the file as a record's since: value. A sloppier read would have sent me debugging code that was already correct.

Known / deliberate

  • Corpus budget: 5695 → ~5750 against a 5600 limit. Already over before this PR; non-blocking warning, deliberately not raised — that's what the warning is for. Consolidation belongs to its own issue.
  • OKF's verified: {by, at} trust tiers were considered and deferred, not rejected.

🤖 Generated with Claude Code

fixes #603 Evaluated OKF as a replacement for our decision-record system, **rejected it**, and adopted two of its optional metadata families additively. Rationale is in the new record (`docs.decision-optional-provenance`); the short version is that OKF's conformance rules are permissive exactly where ours are strict — broken cross-links, unknown types and missing fields must all be tolerated, `deprecated` points at no successor, and its stable identity is the file *path*, which the breadcrumb rule tells agents not to trust. Three commits: the feature, the fixes from an adversarial review, and the backfill. ## What changed | | | |---|---| | `decisions_lib.py` | parse `stale-after` off the metadata line, `**Sources:**` off the metadata block | | `decisions_validate.py` | malformed `stale-after` → **blocking** error; past-due active record → **non-blocking** `::notice::` via `stale_records()` | | `build_decisions_catalog.py` | new **Review due** section, sorted soonest-first, rendering the date only | | `docs/decisions.md`, `docs/decisions/workflow-process.md` | header schema, the new active record, and the backfill onto three `ci.*` records | | `docs/README.md` | schema line kept accurate | Neither field is in `REQUIRED_META`. No format migration, no change to `key`/`status`/`since`/`supersedes`/`superseded-by`/`Signals:`. ## The two design constraints that drove it Both exist to avoid a self-inflicted CI outage: 1. **Past-due is never blocking.** A record going stale is the passage of time, not a defect in whatever commit happens to be under test. A blocking check would turn an unrelated PR red on a calendar boundary. 2. **The generated catalog is clock-free.** It's checked with `--check` in CI, so anything time-dependent in it would silently drift red with *no commit touching the corpus*. Evaluating `today >= stale-after` lives only on the validator's notice path; the catalog renders the date. A malformed date *is* blocking, because it would otherwise silently never fire. ## Adversarial review found four defects — all fixed in `f72bcc9a` Two of them broke the very invariant the feature claims to enforce. Worth reading: 1. **`date.fromisoformat` is not a YYYY-MM-DD validator.** On Python ≥3.11 it also accepts ISO *basic* format (`20270101`) and week dates (`2027-W01-1`), so malformed-looking values passed the blocking check — and *which* forms parse depends on the interpreter, meaning the same corpus could validate differently on a dev machine than on the runner (`pr-checks.yml` pins only `python-version: '3.x'`). Knock-on: the Review-due section sorts on the raw **string**, so an accepted `20270101` sorted *after* `2027-01-15` (`'-' < '0'`), contradicting the section's own "soonest-first" text. Fixed with a `^\d{4}-\d{2}-\d{2}$` gate, which closes both — a fixed-width zero-padded form is what makes string sort ≡ date sort. 2. **An empty `stale-after:` passed as "absent".** `or None` in the parser plus a truthiness guard in the validator collapsed `""` into `None`, so a blank or truncated value produced no error and no notice, ever — precisely the silently-never-fires failure the blocking check exists to prevent. 3. **`test_catalog_is_date_independent` was partly vacuous** — with no date in either render, both sides were trivially equal after the `.replace()`. It did still catch an injected clock-derived marker, but it passed with the feature deleted. 4. **The malformed-date check exempted archive records.** Staleness is moot there, but a typo is still a typo. Each has a regression test. Fixes verified by independently reproducing the reviewer's exploit cases against the patched code, not by re-running the author's own tests. ## Backfill (`738c0e25`) The mechanism shipped unused, so three records that encode *measured host behavior* now adopt both fields, staggered so they don't all come due at once: | Record | `stale-after` | Why it drifts | |---|---|---| | `ci.runner-placement` | 2027-01-15 | Encodes bumblebee's sizing (25 GiB / 12 cores) and container caps. Premise has already partly moved — transcoders left for jazz 2026-07-20, runners retuned since. | | `ci.infra-shaped-red-under-load` | 2027-02-15 | A triage heuristic calibrated against load 243 on a since-retuned host. | | `ci.peak-anon-measurement` | 2027-03-15 | Weakest of the three, but does assert `memory.peak` is cache-inflated — a cgroup fact. | Confirmed empirically that amending three *existing* records needs **no** `[decisions-edit]` token: both fields land inside the contiguous metadata block that `_rationale()` strips before the body-diff, consistent with `Rule`/`Signals`/`Mechanics`. `--base main --head HEAD` passes with zero tokens in the branch. ## Verification 51 tests passing; ruff, ruff format and pyright clean on the touched files; `decisions-validate: OK`; `catalog up to date`. Beyond unit tests, a live probe against the real corpus confirmed past-due → exit 0 + notice, malformed → exit 1, and past-vs-future catalogs byte-identical modulo the date string. Recording one process note: the *first* run of that probe reported two failures, both artifacts of the probe's own arrangement — I validated before regenerating the catalog (so the real error was catalog-drift, not staleness), and my "is today baked in?" check matched `2026-07-25`, which is legitimately in the file as a record's `since:` value. A sloppier read would have sent me debugging code that was already correct. ## Known / deliberate - **Corpus budget**: 5695 → ~5750 against a 5600 limit. Already over before this PR; non-blocking warning, deliberately *not* raised — that's what the warning is for. Consolidation belongs to its own issue. - OKF's `verified: {by, at}` trust tiers were considered and **deferred**, not rejected. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
timothy force-pushed worktree-603-decisions-stale-after-sources from 738c0e2564 to dd4f2c96d9 2026-07-25 15:23:21 +02:00 Compare
timothy force-pushed worktree-603-decisions-stale-after-sources from dd4f2c96d9 to f04412a95b 2026-07-25 15:31:05 +02:00 Compare
Author
Owner

Adversarial review — 3 passes

Cold same-model review (Codex CLI is not installed in this environment, so no cross-family pass; stating the exemption explicitly).

Pass 1 (@ 1d3d2185) found four defects, two of which broke the invariant the feature claims to enforce:

  1. date.fromisoformat is not a YYYY-MM-DD validator — accepts 20270101 and 2027-W01-1 on py≥3.11, making the blocking check interpreter-dependent, and (since the catalog sorts on the raw string) sorting 20270101 after 2027-01-15.
  2. An empty stale-after: collapsed to "absent" — the silently-never-fires failure the check exists to prevent.
  3. test_catalog_is_date_independent passed with the feature deleted.
  4. Archive records exempt from the malformed-date check.

All fixed in 75139bbf. Pass 2 verified them airtight — brute-forced all 1,114,112 codepoints against the regex (unicode-digit forms pass the regex, then fail closed through except ValueError), and proved the de-vacuumed test by mutation.

Pass 2 (@ 738c0e25) also caught that the record's own prose claimed "no backfill was done here" while the backfill commit falsified it — fixed free, before it became surviving rationale prose.

Pass 3 (@ dd4f2c96) BLOCKED on a self-inflicted gate failure worth recording: my commit message explaining the edit token contained the bracketed literal, and decisions_validate matches it as a bare substring over the whole commit range. That armed it and suppressed all three rationale-rewrite comparisons — a green --base/--head run that was vacuous on the body-diff dimension, in the one PR that hand-resolved a conflict inside the corpus that guard protects. Amended in f04412a9; hardening filed as #609 with a required negative test.

Final pass (@ f04412a9) confirmed the guard is now live by independent method rather than trusting my positive control: 328 _rationale invocations — exactly 160 active pairs ×2 + 4 archive pairs ×2 + 0, so no comparison was skipped — plus an in-memory poisoning of ci.runner-placement to prove sensitivity, then clean on the real tree. Also confirmed not fail-open (base parsed 160 records, head 161).

Rebased onto origin/main (abeef63f); the append-vs-append conflict in docs/decisions.md was resolved keeping both records, verified as +1 H2 / +1 parsed record / zero records gone, with main's ci.ui-e2e-harness byte-identical to its origin copy.

51 tests, validator OK, catalog --check OK. Remaining budget warning (5846 vs 5600) is pre-existing — base abeef63f already measured 5782.

Review-verdict: MERGEABLE @ f04412a95b

## Adversarial review — 3 passes Cold same-model review (Codex CLI is not installed in this environment, so no cross-family pass; stating the exemption explicitly). **Pass 1** (@ 1d3d2185) found four defects, two of which broke the invariant the feature claims to enforce: 1. `date.fromisoformat` is not a YYYY-MM-DD validator — accepts `20270101` and `2027-W01-1` on py≥3.11, making the blocking check interpreter-dependent, and (since the catalog sorts on the raw string) sorting `20270101` *after* `2027-01-15`. 2. An empty `stale-after:` collapsed to "absent" — the silently-never-fires failure the check exists to prevent. 3. `test_catalog_is_date_independent` passed with the feature deleted. 4. Archive records exempt from the malformed-date check. All fixed in `75139bbf`. Pass 2 verified them airtight — brute-forced all 1,114,112 codepoints against the regex (unicode-digit forms pass the regex, then fail *closed* through `except ValueError`), and proved the de-vacuumed test by mutation. **Pass 2** (@ 738c0e25) also caught that the record's own prose claimed "no backfill was done here" while the backfill commit falsified it — fixed free, before it became surviving rationale prose. **Pass 3** (@ dd4f2c96) BLOCKED on a self-inflicted gate failure worth recording: my commit message *explaining* the edit token contained the bracketed literal, and `decisions_validate` matches it as a **bare substring over the whole commit range**. That armed it and suppressed all three rationale-rewrite comparisons — a green `--base/--head` run that was **vacuous on the body-diff dimension**, in the one PR that hand-resolved a conflict inside the corpus that guard protects. Amended in `f04412a9`; hardening filed as #609 with a required negative test. **Final pass** (@ f04412a9) confirmed the guard is now live by independent method rather than trusting my positive control: 328 `_rationale` invocations — exactly 160 active pairs ×2 + 4 archive pairs ×2 + 0, so no comparison was skipped — plus an in-memory poisoning of `ci.runner-placement` to prove sensitivity, then clean on the real tree. Also confirmed not fail-open (base parsed 160 records, head 161). Rebased onto `origin/main` (`abeef63f`); the append-vs-append conflict in `docs/decisions.md` was resolved keeping both records, verified as +1 H2 / +1 parsed record / zero records gone, with main's `ci.ui-e2e-harness` byte-identical to its origin copy. 51 tests, validator OK, catalog `--check` OK. Remaining budget warning (5846 vs 5600) is pre-existing — base `abeef63f` already measured 5782. Review-verdict: MERGEABLE @ f04412a95b599d2f1ed1dd5c4ece15d5a2aead3a
timothy scheduled this pull request to auto merge when all checks succeed 2026-07-25 15:33:49 +02:00
timothy merged commit c3beee1629 into main 2026-07-25 16:16:00 +02:00
timothy deleted branch worktree-603-decisions-stale-after-sources 2026-07-25 16:16:00 +02:00
Sign in to join this conversation.