diff --git a/docs/README.md b/docs/README.md index 1a3d6b8ef..cb140bf04 100644 --- a/docs/README.md +++ b/docs/README.md @@ -73,7 +73,8 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K the Blazor Server UI is removed and every legacy route now 302-redirects to its SPA equivalent (or falls through to the catch-all → `/app`). Read it for the full legacy→SPA route inventory. - **`docs/decisions.md`** + **`docs/decisions/*.md`** — active decision records (lifecycle schema: - key/status/since/supersedes/superseded-by). **Generated active view**: `docs/decisions/README.md` + key/status/since/supersedes/superseded-by, plus optional `stale-after`/`Sources:` — + ersatztv#603). **Generated active view**: `docs/decisions/README.md` (catalog / task router) — start there. Superseded/retired records live in `docs/decisions/archive/` and are read only for history, never for "what is the current rule." - **`docs/ci-cd.md`** — build/test/release pipeline, versioning, dependency management. diff --git a/docs/decisions.md b/docs/decisions.md index 51ecd95e2..6970a7565 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -29,6 +29,16 @@ one of: (An H2 with no metadata block is `legacy-unmigrated` — not yet ported to this schema. The validator tolerates these and reports a count; they trend to zero over time, not required to hit zero at once.) +**Two optional fields (ersatztv#603).** `` `stale-after: YYYY-MM-DD` `` on the metadata line marks a +record that asserts a fact about the *outside world* as due for re-confirmation once that date +passes — an absolute date, no TTL. A **`Sources:`** line in the metadata block cites the evidence the +record rests on (a CI run, a measurement, a log line), as distinct from `Signals:`, which is recall +keywords. Both are optional and never required; a malformed `stale-after` is a blocking error (it +would silently never fire), but a *past-due* record is only a non-blocking notice — going stale is +the passage of time, not a defect in whatever commit happens to be under test. Dated records are +listed in the catalog's **Review due** section, which renders the date only and never a +clock-derived verdict. + **Supersession is same-PR, not a later consolidation pass**: add the new active record, then relocate the predecessor's file (or section) into `docs/decisions/archive/` with the metadata above rewritten to `status: superseded` (or `retired`) and the reciprocal `supersedes`/`superseded-by` links filled @@ -3727,3 +3737,39 @@ ownership instead of proving it. (b) Re-raise signals, so a cancelled run cannot `if ! cmd; then status=$?` — under `!` bash sets `$?` to the logical negation, so it reads 0 and a FAILING run exits 0, silently passing CI. (d) `exec` the app inside the backgrounded subshell, or `$!` is the SUBSHELL on bash 3.2 (stock macOS) and every PID-based kill targets the wrong process. + +## 2026-07-25 — OKF evaluated and rejected as a replacement; two of its optional fields adopted (#603) +`key: docs.decision-optional-provenance` · `status: active` · `since: 2026-07-25` · `supersedes: none` · `superseded-by: none` +**Rule:** Decision records gain two OPTIONAL fields — `stale-after: YYYY-MM-DD` on the metadata line and a `**Sources:**` line in the metadata block; the Open Knowledge Format (OKF) itself is NOT adopted as the record format. +**Signals:** OKF, Open Knowledge Format, knowledge-catalog, okf-skills, stale-after, staleness, Sources, provenance, re-confirmation, review due · paths: scripts/decisions_lib.py scripts/decisions_validate.py scripts/build_decisions_catalog.py · issues: #603 +**Mechanics:** docs/decisions.md → header schema; docs/decisions/README.md → Review due +**Sources:** https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/okf (SPEC.md v0.2) · https://github.com/scaccogatto/okf-skills + +**OKF is not a replacement for this system, and the reason is structural, not stylistic.** Its +conformance rules are deliberately permissive exactly where ours are deliberately strict: a consumer +MUST NOT reject a bundle for broken cross-links, unknown `type` values, or missing optional fields, +and its `status: deprecated` is a bare flag with nothing pointing at the successor. The single +catastrophic failure this corpus exists to prevent is *a superseded rule read as current*, and what +prevents it is the validator — one active record per key, reciprocal `supersedes`/`superseded-by`, +no record vanishing from the active set without an archive copy, catalog-drift detection. OKF has no +equivalent of any of those. It also has no `key`: its stable identity is the file *path*, which is +precisely what the breadcrumb rule tells agents not to trust. Adopting it would additionally mean a +second ingestion format alongside MemPalace for the same corpus. + +**What it does have that we lacked is lifecycle and provenance metadata.** Records that encode +measured facts about hosts, runners or upstream behavior drift silently — nothing marked them due for +re-confirmation. `stale-after` fixes that, and `Sources:` gives the *evidence* a record rests on a +home other than buried prose. Both are additive: no format migration, nothing changes for the +existing corpus, and no backfill was done here — adopt them opportunistically on records that +actually assert an external fact. + +**The staleness check is non-blocking, and the generated catalog is clock-free — both on purpose.** +A calendar date turning an unrelated PR red, or silently drifting `docs/decisions/README.md` into a +`--check` failure with no commit touching the corpus, is a self-inflicted CI outage. So evaluating +`today >= stale-after` lives only on the validator's `::notice::` path, and the catalog's Review-due +section renders the date itself, never a stale/fresh verdict. + +**OKF's `verified: {by, at}` trust tiers were considered and deferred**, not rejected. Given how +much of this corpus is agent-authored, distinguishing agent-inferred from human-confirmed-after- +measuring is real signal; it just needs its own actor convention and wasn't worth coupling to this +change. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index e5a693aab..8ad3f5bd9 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -62,6 +62,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `concurrency.schedule-item-child-identity` | `PUT /api/schedules/{id}/items` reconciles by an optional round-tripped child `Id` (null/absent/0 ⇒ new item), never by array position, so fill-group/shuffle state follows the logical item across reorders; an unknown or duplicate id is rejected 422 (checked after the §7a `CheckVersion`, so 412 precedes 422). | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--stable-child-identity-for-schedule-item-replace-259-split-from-252253) | | `docs.convention-docs-session-start` | Docs-first, not source-first: conventions (api-conventions, spa-conventions, e2e-local, blazor-route-parity, domain-model, decisions, README) are read from docs, not reverse-engineered from code, via `docs/README.md`'s task-signal map — only the sections it points to for the task at hand, not the whole set. Each doc is updated in the same PR that changes what it documents, replacing deferred/follow-up doc updates. | 2026-07-07 | [link](../decisions.md#2026-07-07--convention-docs-read-at-session-start-updated-in-pr) | | `docs.decision-lifecycle` | every decision `##` record (active or archived) carries a 5-field metadata block (`key`, `status`, `since`, `supersedes`, `superseded-by`) checked by `scripts/decisions_validate.py`; a record is never deleted or line-edited to reverse a call — it is moved to `docs/decisions/archive/` with `status: superseded`/`retired` and a reciprocal `superseded-by`/`supersedes` key pair to its replacement. | 2026-07-21 | [link](../decisions.md#2026-07-21--decision-records-carry-a-lifecycle-schema-validated-by-a-script-append-only-by-diff-is-retired-521) | +| `docs.decision-optional-provenance` | Decision records gain two OPTIONAL fields — `stale-after: YYYY-MM-DD` on the metadata line and a `**Sources:**` line in the metadata block; the Open Knowledge Format (OKF) itself is NOT adopted as the record format. | 2026-07-25 | [link](../decisions.md#2026-07-25--okf-evaluated-and-rejected-as-a-replacement-two-of-its-optional-fields-adopted-603) | | `docs.tracker-comment-retrofit` | When the knowledge exporter flags an over-cap tracker issue and excludes it from ingestion, triage its comments instead of assuming a retrofit is owed — and for each decision-shaped item check the **worked issue first**, because a tracker session comment is by construction a précis of the fuller closing record posted on the issue it narrates. Applied to #237 (111 comments) this yielded **zero** records, so server-management#642's "a fact found only in a #237 comment" retrieval row has no valid subject and its interim target (an already-migrated record) is permanent. | 2026-07-21 | [link](../decisions.md#2026-07-21--check-the-worked-issue-before-the-decision-corpus-a-closed-trackers-comments-need-no-retrofit-524) | | `ffmpeg.external-logo-graphics-engine` | External-URL channel logos pass through to the graphics engine like any other watermark source; `WatermarkSelector` must never gate them on `File.Exists` (always false for a URL) and never route them through the ffmpeg-native overlay shortcut. | 2026-07-20 | [link](../decisions.md#2026-07-20--external-url-channel-logos-pass-through-to-the-graphics-engine-never-fileexists-gated-never-ffmpeg-native-502) | | `ffmpeg.hls-cold-start-burst` | HLS cold-start latency is fixed with a bounded `-readrate_initial_burst` (gated on FFmpeg ≥6.1 capability detection), not by raising `work_ahead_limit`, which would remove the concurrency guarantee it exists for. | 2026-07-20 | [link](../decisions.md#2026-07-20--hls-cold-start-is-fixed-with--readrate_initial_burst-not-by-raising-the-work-ahead-limit-350) | diff --git a/scripts/build_decisions_catalog.py b/scripts/build_decisions_catalog.py index a8f50be58..b83ca0056 100644 --- a/scripts/build_decisions_catalog.py +++ b/scripts/build_decisions_catalog.py @@ -62,6 +62,26 @@ def render_catalog(records) -> str: rule = (r.rule or "").replace("|", "\\|") lines.append(f"| `{r.key}` | {rule} | {r.since or ''} | [link]({_rel(r)}) |") lines.append("") + + # Review-due section (ersatztv#603). Renders the DATE only — never "today", never a computed + # stale/fresh verdict. This file is checked with --check in CI, so anything time-dependent here + # would drift it red on a calendar boundary with no commit touching the corpus. Evaluating + # today >= stale-after is the validator's job, on its non-blocking notice path. + due = sorted([r for r in active if r.stale_after], key=lambda r: (r.stale_after or "", r.key or "")) + if due: + lines += [ + "## Review due", + "", + "Active records that assert facts about the outside world and carry a `stale-after` date.", + "Once that date passes, re-confirm the fact and either extend the date or supersede the", + "record. Sorted soonest-first.", + "", + "| Stale after | Key | Record |", + "| --- | --- | --- |", + ] + for r in due: + lines.append(f"| {r.stale_after} | `{r.key}` | [link]({_rel(r)}) |") + lines.append("") return "\n".join(lines) diff --git a/scripts/decisions_lib.py b/scripts/decisions_lib.py index 8fbff2c28..4d64d6b86 100644 --- a/scripts/decisions_lib.py +++ b/scripts/decisions_lib.py @@ -12,6 +12,10 @@ block as its first non-blank content: An H2 with no metadata line is treated as status `legacy-unmigrated` (migration target). + +Two OPTIONAL fields (ersatztv#603, adopted from OKF v0.2's lifecycle/provenance families) may also +appear — `stale-after: YYYY-MM-DD` on the metadata line, and a `**Sources:**` line in the metadata +block. Absence is never an error; see decisions_validate for how they are checked. """ from __future__ import annotations @@ -44,6 +48,10 @@ class Record: rule: str | None = None signals: str | None = None mechanics: str | None = None + # Optional (ersatztv#603). `stale_after` is an absolute date, no TTL — the record is due for + # re-confirmation once today >= it. `sources` cites the evidence the record rests on. + stale_after: str | None = None + sources: str | None = None body: str = "" @@ -76,6 +84,7 @@ def parse_text(text: str, source: Path) -> list[Record]: rec.since = meta.get("since") or None rec.supersedes = meta.get("supersedes") or None rec.superseded_by = meta.get("superseded-by") or None + rec.stale_after = meta.get("stale-after") or None # The metadata block is contiguous: scan only until the first blank line, so a # bolded **Rule:** appearing later inside rationale prose can't overwrite the real one. for bl2 in body_lines[k + 1 :]: @@ -87,6 +96,8 @@ def parse_text(text: str, source: Path) -> list[Record]: rec.signals = bl2[len("**Signals:**") :].strip() elif bl2.startswith("**Mechanics:**"): rec.mechanics = bl2[len("**Mechanics:**") :].strip() + elif bl2.startswith("**Sources:**"): + rec.sources = bl2[len("**Sources:**") :].strip() break rec.body = "\n".join(body_lines).strip() records.append(rec) diff --git a/scripts/decisions_validate.py b/scripts/decisions_validate.py index adc053aca..17f95c351 100644 --- a/scripts/decisions_validate.py +++ b/scripts/decisions_validate.py @@ -13,6 +13,7 @@ from __future__ import annotations import argparse import subprocess import sys +from datetime import date from pathlib import Path import scripts.decisions_lib as dl # noqa: E402 (run with PYTHONPATH=. or as module) @@ -26,6 +27,33 @@ REQUIRED_META = ("key", "status", "since", "supersedes", "superseded_by", "signa EDIT_TOKEN = "[decisions-edit]" # noqa: S105 (a commit-message marker, not a credential) +def _parse_stale_after(value: str | None) -> date | None: + """`stale-after` as a date, or None if absent/malformed. Callers distinguish the two.""" + if not value: + return None + try: + return date.fromisoformat(value.strip()) + except ValueError: + return None + + +def stale_records(records, today: date) -> list[tuple[str, str]]: + """(heading, stale-after) for ACTIVE records that are past due — today >= stale-after. + + OKF v0.2 semantics: an absolute date, no TTL. Deliberately NOT part of `validate()` — a record + going stale is the passage of time, not a defect in the commit under test, so it must never fail + a PR that didn't touch it. main() reports these as a non-blocking notice. + """ + out = [] + for r in records: + if r.status != "active": + continue + d = _parse_stale_after(r.stale_after) + if d is not None and today >= d: + out.append((r.heading, r.stale_after)) + return sorted(out) + + def _key_of(ref: str | None) -> str | None: if not ref or ref == "none": return None @@ -59,6 +87,10 @@ def validate( for f in REQUIRED_META: if getattr(r, f) in (None, ""): errs.append(f"{r.heading}: missing required metadata {f}") + # `stale-after` is OPTIONAL, but a malformed one is a defect in the commit that wrote it — + # it would silently never fire. Absent is fine; unparseable is not. + if r.stale_after and _parse_stale_after(r.stale_after) is None: + errs.append(f"{r.heading}: stale-after {r.stale_after!r} is not a YYYY-MM-DD date") if r.status == "active": active_by_key[r.key] = active_by_key.get(r.key, 0) + 1 if r.status in ("superseded", "retired"): @@ -328,6 +360,15 @@ def main(argv=None) -> int: file=sys.stderr, ) + stale = stale_records(records, date.today()) + if stale: + listed = "; ".join(f"{h} (stale-after {d})" for h, d in stale) + print( + f"::notice::decisions-validate: {len(stale)} active record(s) past their stale-after date " + f"— re-confirm or extend: {listed}", + file=sys.stderr, + ) + unmigrated = [r for r in records if r.status == "legacy-unmigrated" and r.heading not in SKIP_HEADINGS] if unmigrated: print(f"::notice::{len(unmigrated)} legacy-unmigrated decision record(s) remain (must trend to 0).") diff --git a/scripts/tests/fixtures/sample_decisions.md b/scripts/tests/fixtures/sample_decisions.md index ab3bbafb1..987e4cfa5 100644 --- a/scripts/tests/fixtures/sample_decisions.md +++ b/scripts/tests/fixtures/sample_decisions.md @@ -11,5 +11,14 @@ Rationale prose here. +## 2026-07-19 — Optional stale-after / Sources example (#603) +`key: ci.peak-anon-measurement` · `status: active` · `since: 2026-07-19` · `supersedes: none` · `superseded-by: none` · `stale-after: 2027-01-15` +**Rule:** The test job reports a sampled true peak anon. +**Signals:** ci, memory · paths: scripts/ci-peak-anon.sh · issues: #412 +**Mechanics:** docs/ci-cd.md → CI lanes +**Sources:** gitea run 4471 (measured) · scripts/ci-peak-anon.sh + +Rationale prose here. + ## 2026-07-11 — Legacy unmigrated example (#231) Some prose with no metadata line at all. diff --git a/scripts/tests/test_build_catalog.py b/scripts/tests/test_build_catalog.py index 820463390..7da42136a 100644 --- a/scripts/tests/test_build_catalog.py +++ b/scripts/tests/test_build_catalog.py @@ -1,4 +1,5 @@ from pathlib import Path +from typing import Any import scripts.build_decisions_catalog as bc import scripts.decisions_lib as dl @@ -49,3 +50,43 @@ def test_anchor_matches_gitea_double_hyphen_slug(): b = bc._anchor("2026-07-16 — Optional advertised IPTV base URL (`iptv.base_url`)…") assert "iptvbase_url" in b + + +def _r(key: str, **kw: Any) -> dl.Record: + base: dict[str, Any] = dict( + heading=f"H {key}", + source=Path("decisions.md"), + lineno=1, + key=key, + status="active", + rule="r", + ) + base.update(kw) + return dl.Record(**base) + + +def test_review_due_section_lists_dated_records_soonest_first(): + out = bc.render_catalog( + [ + _r("b.later", stale_after="2027-03-01"), + _r("a.sooner", stale_after="2026-09-01"), + _r("c.undated"), + ] + ) + assert "## Review due" in out + assert out.index("2026-09-01") < out.index("2027-03-01") + assert "c.undated" not in out.split("## Review due")[1] + + +def test_review_due_section_omitted_when_no_record_is_dated(): + assert "## Review due" not in bc.render_catalog([_r("a.b")]) + + +def test_catalog_is_date_independent(): + """The generated file must never bake in 'today' — CI checks it with --check, so a + time-dependent render would drift red on a calendar boundary with no commit touching it.""" + past = bc.render_catalog([_r("a.b", stale_after="2020-01-01")]) + future = bc.render_catalog([_r("a.b", stale_after="2099-01-01")]) + # A long-past and a far-future date must produce byte-identical output apart from the date + # itself: no stale/fresh verdict, no marker, nothing derived from the clock. + assert past.replace("2020-01-01", "D") == future.replace("2099-01-01", "D") diff --git a/scripts/tests/test_decisions_lib.py b/scripts/tests/test_decisions_lib.py index 35ad6db08..df255ed93 100644 --- a/scripts/tests/test_decisions_lib.py +++ b/scripts/tests/test_decisions_lib.py @@ -32,3 +32,18 @@ def test_legacy_record_is_unmigrated(): def test_index_section_parses_as_heading(): recs = dl.parse_file(FIX) assert any(r.heading == "Index" for r in recs) + + +def test_parses_optional_stale_after_and_sources(): + recs = dl.parse_file(FIX) + r = next(r for r in recs if r.key == "ci.peak-anon-measurement") + assert r.stale_after == "2027-01-15" + assert r.sources is not None + assert "gitea run 4471" in r.sources + + +def test_optional_fields_default_to_none_when_absent(): + recs = dl.parse_file(FIX) + r = next(r for r in recs if r.key == "ci.runner-placement") + assert r.stale_after is None + assert r.sources is None diff --git a/scripts/tests/test_decisions_validate.py b/scripts/tests/test_decisions_validate.py index b9a9b7998..353406936 100644 --- a/scripts/tests/test_decisions_validate.py +++ b/scripts/tests/test_decisions_validate.py @@ -411,3 +411,48 @@ def test_diff_engine_demoted_migrated_to_legacy_unmigrated(tmp_path, monkeypatch _removed, _rewritten, demoted = dv._diff_findings("HEAD~1", "HEAD") assert "2026-01-01 — Some decision (#1)" in demoted + + +# ---- optional stale-after / Sources metadata (ersatztv#603) ---- + + +def test_stale_after_absent_is_not_an_error(): + assert _v([_rec(key="a.b")]) == [] + + +def test_wellformed_stale_after_is_not_an_error(): + assert _v([_rec(key="a.b", stale_after="2027-01-01")]) == [] + + +def test_malformed_stale_after_fails(): + for bad in ("2027-13-01", "01-01-2027", "soon", "2027-02-30"): + errs = _v([_rec(key="a.b", stale_after=bad)]) + assert any("not a YYYY-MM-DD date" in e for e in errs), bad + + +def test_past_due_record_does_not_fail_validation(): + """A record going stale is the passage of time, not a defect in the commit under test.""" + errs = _v([_rec(key="a.b", stale_after="2020-01-01")]) + assert errs == [] + + +def test_stale_records_reports_only_past_due_active(): + today = dv.date(2026, 7, 25) + recs = [ + _rec(key="past.due", heading="Past", stale_after="2026-07-24"), + _rec(key="due.today", heading="Today", stale_after="2026-07-25"), # today >= date → stale + _rec(key="not.yet", heading="Future", stale_after="2026-07-26"), + _rec(key="no.date", heading="Undated"), + _rec(key="arch.ived", heading="Archived", status="superseded", stale_after="2020-01-01"), + ] + assert dv.stale_records(recs, today) == [("Past", "2026-07-24"), ("Today", "2026-07-25")] + + +def test_stale_records_ignores_malformed_date(): + """A malformed date is caught as a blocking format error; it must not also crash the notice path.""" + assert dv.stale_records([_rec(key="a.b", stale_after="whenever")], dv.date(2026, 7, 25)) == [] + + +def test_sources_is_optional_and_never_required(): + assert _v([_rec(key="a.b")]) == [] + assert _v([_rec(key="a.b", sources="ci run 1234 · docs/ci-cd.md")]) == []