Commit Graph
27 Commits
Author SHA1 Message Date
timothyandClaude Opus 5 8d0df55398 fix(629): review fixes — tilde fences, an unbounded sha field, and a forgeable comment boundary
Cross-family review of 38a96f47 returned BLOCKED with three findings. All reproduced first:

  ~~~ fence           -> positive   fences were stripped for ``` only; markdown also takes ~~~
  @ <40hex>f / ZZZ    -> positive   the sha matched {7,40} with NO right boundary, so an
                                    over-long or malformed token was TRUNCATED into a passing one
  \x01BODY-BOUNDARY\x01 -> positive an in-band separator joined comment bodies, so a body
                                    containing that line forged a boundary, reset fence state
                                    mid-comment, and exposed a verdict inside an unclosed fence

Fixes: both fence markers honoured; the hex run matched whole, required to end at a
non-alphanumeric boundary, with its length validated separately so an out-of-range token is
rejected rather than trimmed to fit; and bodies carried OUT-OF-BAND (one JSON-encoded string
per line), which removes the forgery class instead of escaping the sentinel.

The third is the one worth remembering: an in-band delimiter is forgeable by whoever writes the
data, and here that is anyone who can comment on the PR.

Two of these fixes broke previously-green tests, both of which were right to break:
- an over-long token now classifies `no-sha`, not `stale`. The fixture asserting `stale` was 45
  hex chars, so it had been exercising the length guard while claiming to test the prefix rule.
  Rebuilt as a well-formed 40-char sha that contains the head prefix without starting with it.
- `jq -e` exits 4 when a filter produces NO output, which is the legitimate empty-comment-list
  case. Treating it as an error turned "no comments yet" into an input error — and callers fail
  closed on those, so a new PR would have read as unclassifiable. Exit 4 is now accepted.

44 classifier tests, 155 total. `~~~` and the boundary fixes are each mutation-verified; the
out-of-band fix has no equivalent mutation (it is structural, not a regex) so its evidence is the
direct reproduction against the previous commit.

refs #629

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Decisions-Edit: yes
2026-07-26 00:17:17 +02:00
timothyandClaude Opus 5 38a96f47cc fix(629): close three false-opens in the H10 verdict grammar, and give it tests
The H10 classification lived inline in `pretooluse-merge-consent.sh` with no tests. Three
protections the `release.review-verdict-gate` record described were never actually
implemented, and each graded an unreviewed head as approved. All three reproduced first:

  1 MERGEABLE-LATER   -> positive     the token was prefix-matched, so any word STARTING
                                      with mergeable/approved/lgtm passed
  2 fenced code block -> positive     the line-start anchor is satisfied inside ```, so
                                      documentation showing the convention was a verdict
  3 URL-borne sha     -> positive     the sha came from the first `@<hex>` ANYWHERE on the
                                      line, so a markdown link could supply it

Fixes: whole-word token matching, with a token in neither vocabulary classified `unknown`
(never positive, and not guessed into a block either — it goes to a human); fenced blocks
stripped with fence state reset per comment body; the sha read from the verdict's OWN
`@ <sha>` field, which also makes multi-`@` lines unambiguous.

The grammar moves to `scripts/check-review-verdict.sh` so it can be tested at all — 38 tests,
and each fix mutation-verified: restoring the old regex/extraction makes exactly the
corresponding test fail, control green.

#629's fourth reported item is NOT a defect and is not claimed as a fix. A later `@ <head>`
on a BLOCKED line was reported as "masking a negative"; under the documented grammar that
line is a verdict for the sha in its own field, so `stale` is correct — and was correct
before this change too. Kept as a characterization test.

`test_post_review_verdict.py`'s cross-check re-implemented the hook's regexes in Python and
asserted the shell still contained them. That mirror is removed: it is the same duplication
that let these three survive, and a Python copy would keep passing while the shell drifted.
It now runs the real classifier.

The decision record is corrected — it asserted the URL protection this commit actually adds.

Note: the active corpus is 5637 lines against a 5600 budget, so the validator emits its
consolidation warning (non-blocking). That is #620's subject, not regressed here.

fixes #629

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Decisions-Edit: yes
2026-07-25 23:46:23 +02:00
timothy 07e1e8cfbc fix(622): validate rename rows too; name the real blocker when only the verdict is pending
review-verdict/h10 Awaiting review verdict for 07e1e8c
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
Review verdict / Set review-verdict status (pull_request) Successful in 10s
PR Gates / decisions lifecycle (pull_request) Successful in 21s
PR Gates / Docs update reminder (pull_request) Successful in 19s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m19s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m28s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m5s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 15m31s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 15m44s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Third BLOCKED verdict, third legitimate finding. The per-row guard validated the
DESTINATION only, so `{"filename":"docs/x.md","status":"renamed"}` with no
`previous_filename` passed while its source silently vanished — precisely the
path-hiding that collecting both sides exists to prevent. A rename row must now
carry a non-empty `previous_filename`.

Required for rename rows ONLY. Demanding it globally would reject every ordinary
modified/added row and make the gate refuse all exemptions — which every
"withholds the exemption" test would happily pass through, so that direction gets
its own positive control. Predicate checked against ten shapes before adopting;
mutation-verified in both directions (dropping the clause fails 2 tests, applying
it globally fails 3 including the controls).

Also fixes a wart this PR introduced. `review-verdict/h10` is itself folded into
the COMBINED status, so a PR awaiting its verdict reports combined 'pending' and
the hook's condition (a) reported it as a CI problem — sending a reader to build
logs when the missing thing is the review, and exiting before the H10 branch that
would have said so. The message now names the outstanding contexts, and says
plainly when the verdict is the only one left.

Scope boundary, stated in the record: real Gitea populates `previous_filename` on
renames and returns well-formed pages. Everything past this point defends against
shapes with no evidence of existing, so the guard's claim stays "any page we
cannot fully classify withholds the exemption" rather than growing to cover
unobserved responses.

Decisions-Edit: yes
2026-07-25 22:43:30 +02:00
timothy fc547b2d46 fix(622): validate file pages per ROW, and de-vacuum the exemption tests
review-verdict/h10 Awaiting review verdict for fc547b2
Review verdict / Set review-verdict status (pull_request) Successful in 11s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 23s
PR Gates / decisions lifecycle (pull_request) Successful in 30s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 18s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 15s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m8s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 15m20s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 15m56s
Second BLOCKED verdict, also correct. My previous fix validated only the
TOP-LEVEL type of each page, which left the same hole one level down: `[{}]` is
a well-formed JSON array whose rows carry no `filename`, so it contributes zero
paths, looks like a short final page, and completes the enumeration from a
PARTIAL list — the exact failure the guard exists to prevent, just wearing a
valid-looking wrapper. An array of scalars additionally tripped `set -e` during
extraction without producing a decision.

Both call sites now require every row to carry a non-empty string `filename`.
An empty array remains valid, since that is what a real end-of-pagination is.
Predicate checked against nine shapes before adopting: `[]` and well-formed rows
valid; `[{}]`, `[{"filename":""}]`, `[{"filename":null}]`, `["x"]`, `[1,2]`, a
bare error object, and a mixed good/bad array all rejected.

The reviewer also caught that my `test_first_page_failure_*` was VACUOUS: with
page 1 failing the path list is empty, and the hook independently withholds the
exemption on an empty list, so it passed with the guard removed. It is kept as a
smoke case with that limitation stated in its docstring, and three tests that
actually pin the behaviour were added — malformed rows and scalar rows after a
FULL page 1 (so the path list is non-empty and the exemption would really fire),
plus MAX_PAGES exhaustion.

Added a positive control too: an empty final page must still exempt. Without it,
an over-strict guard would make every "withholds the exemption" test pass for the
wrong reason — the suite would be asserting "never exempt", which is not the
contract. Verified by mutation: top-level-only validation fails 2 tests,
an over-strict guard fails the positive control, and widening MAX_PAGES fails the
exhaustion test.

Decisions-Edit: yes
2026-07-25 22:23:54 +02:00
timothy f6d01c4924 fix(622): make the exemption paging fail CLOSED; read statuses latest-per-context
review-verdict/h10 Awaiting review verdict for f6d01c4
Review verdict / Set review-verdict status (pull_request) Successful in 3s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / Docs update reminder (pull_request) Successful in 15s
PR Gates / decisions lifecycle (pull_request) Successful in 22s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m59s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 10s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m9s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m51s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Cross-family re-review returned BLOCKED on the fix commit, correctly. Two of the
three findings were on the FAILURE path, which is why every happy-path run and
the whole first test suite looked fine.

1. (High/Medium) Both paging loops treated a failed or non-array page as a
   legitimate short final page: empty output counts as zero rows, zero rows reads
   as "end of list", and the enumeration completed from a PARTIAL list — failing
   OPEN at exactly the point the guard exists to fail closed, while the decision
   record claimed the opposite. Each page is now validated as a JSON array before
   its rows are counted; anything else withholds the exemption.

2. (High) The workflow's "is there already a verdict" pre-check read
   `/statuses/{sha}`, which returns one row per status POST rather than per
   context and pages at 50 — so a head with a few CI reruns can push a real
   verdict off the first page (this PR's head already carries 15 rows for 11
   contexts). Missing it there is not benign: the job would post `pending`, or an
   exemption `success`, OVER a human verdict. Both the workflow and the hook now
   read `/commits/{sha}/status` (latest-per-context), and the workflow refuses to
   post anything when that read fails rather than inferring "no verdict yet".
   Credit: raised independently by the parallel session on this issue.

3. (Low) The record claimed both callers withhold the exemption when enumeration
   cannot complete. True only after (1); the claim and the code now agree.

Adds scripts/tests/test_merge_consent_exemption.py — 9 tests over the hook's
exemption path via a curl shim, asserting behaviour when a page ERRORS or returns
a non-array body, not merely when it returns data. Mutation-proven: reverting the
array validation fails both failure-path tests; dropping `previous_filename`
fails the rename test.

One test I wrote was wrong and the run caught it: I asserted a
`.gitea/... -> docs/...` rename must break the hook's exemption, but that hook
deliberately exempts `.claude/`/`.gitea/`/`.husky/` too (to a human PROMPT, never
an auto-grant), so both sides legitimately match. The rename guard matters there
for a genuinely non-exempt source; the stricter PROTECTED semantics belong to the
workflow. Both cases are now pinned, including a positive control that a
docs->docs rename stays exempt.

Decisions-Edit: yes
2026-07-25 22:04:18 +02:00
timothy 628ae46f7f fix(622): bind H10 merge consent to the reviewed sha via a required commit status
review-verdict/h10 Awaiting review verdict for 628ae46
Review verdict / Set review-verdict status (pull_request) Successful in 9s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 15s
PR Gates / decisions lifecycle (pull_request) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Failing after 1m28s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 16m59s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 19m5s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
`pretooluse-merge-consent.sh` proves all three consent conditions at the moment
the merge tool is called. With `merge_when_checks_succeed=true`, condition (a) is
delegated to Gitea, which then merges whatever head is green at ITS merge time —
while (b) Done-when and (c) the head-referencing verdict were proven against the
head at SCHEDULING time. Every commit pushed in between merges unreviewed. The
gate is not bypassed; it is satisfied against a snapshot that stops being true.

Demonstrated as a controlled A/B rather than inferred, with a CI check left
pending so Gitea waits as it really does:

  without a required verdict context: unreviewed commit B MERGED
  with it:                            same sequence REFUSED
  after reviewing B and posting it:   merges — blocked, not deadlocked

NOTE the anecdote in #622 is wrong and is corrected in the docs rather than
repeated: PR #619 does carry `Review-verdict: MERGEABLE @ 02c82b35`, posted six
seconds before the merge, explicitly re-reviewing the follow-up commits. #622 was
filed off a lagging API read. The hole is real regardless, and structural —
nothing FORCED that re-review inside the window Gitea would have merged in. This
turns a property that held by diligence into one that holds by construction.

The fix is the sha, not a smarter check. A Gitea commit status belongs to exactly
one commit, so a status written for a parent cannot be inherited by a child that
did not exist yet. `review-verdict/h10` becomes a REQUIRED status check on main:
push a new head and the context is simply absent, which Gitea reads as
not-passing (verified against 1.25.4: missing AND pending both block, and
auto-merge re-checks the current head). It also covers merge paths the hook never
sees — Gitea UI, raw API, another agent's session.

- scripts/post-review-verdict.sh writes the comment and the status together so
  they cannot drift, and re-reads head after commenting: if a commit landed
  mid-flight it writes NO status and exits non-zero rather than retargeting the
  verdict at a commit nobody read.
- .gitea/workflows/review-verdict.yml auto-passes the two exempt classes that
  would otherwise deadlock — Renovate-authored (platformAutomerge) and docs-only
  — and marks everything else pending. Exemptions are void when the PR touches
  .claude/, .gitea/, .husky/, scripts/ or docker/ci/.
- The hook refuses to SCHEDULE an auto-merge unless that status is green on head,
  and no longer claims "CI green" on the mwcs path, where it never read CI.

Two silent false negatives in the exemption path, both found by verifying rather
than reasoning, both fixed at BOTH call sites (workflow and the hook's
pre-existing docs-only carve-out):

- The files endpoint caps at 50 rows and IGNORES a larger `limit` — PR #619 has
  194 changed files and `?limit=100` returns 50. A single page saw ZERO protected
  paths there where the full enumeration finds ten. Both now page to exhaustion
  and withhold the exemption if they cannot complete.
- A rename is ONE row whose `filename` is the destination, the source only in
  `previous_filename`. Verified live: `.gitea/workflows/renovate.yml` ->
  `docs/innocuous-note.md` presented as docs-only with no protected path visible.
  Both now read BOTH sides.

Limits are documented, not papered over: base changes leave the head sha (and so
the verdict) unchanged, and a PR editing the workflow is judged by its own edited
copy — so PROTECTED is a guardrail against accident, not a tamper-proof control.

fixes #622
2026-07-25 21:51:02 +02:00
timothy 02c82b35ea fix(610): make the frontmatter READ path dependency-free — CI has no PyYAML
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
PR Gates / Docs update reminder (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m15s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m6s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
This is why `decisions lifecycle` went red, and it was NOT the known flake. I
came close to dismissing it as one for the second time this session, because an
earlier red on another branch genuinely was.

The dual-format parser imported PyYAML to read frontmatter. `decisions-guard`
does `setup-python` and installs NOTHING, so once the corpus was migrated every
record became unparseable there: ModuleNotFoundError, job fails. The same would
hit the Husky pre-commit hook and every contributor's machine.

Installing PyYAML in CI is the wrong fix: READING happens everywhere -- CI, the
hook, every dev -- while WRITING happens once, in a migration a human runs
deliberately. So the read path is now dependency-free and only
`migrate_decisions_split` (the writer) still imports yaml.

A hand-rolled parser is only safe if it provably matches the library that WROTE
the files, so `test_frontmatter_reader_matches_pyyaml_on_every_real_record`
compares the two field-by-field across all 169 real records (importorskip, so it
is skipped rather than failing where PyYAML is absent) with a >100-file guard
against near-vacuity. It is narrow by construction: the frontmatter is
machine-generated with default_flow_style=False and width=10**9, so every value
is a single-line scalar, and the reader bails to None on anything nested.

Verified by running all four affected entry points against a shim that makes
`import yaml` raise: validate --base/--head, catalog --check, the kickoff guard,
and the plain validate the pre-commit hook calls. All exit 0.

refs #610
2026-07-25 19:57:22 +02:00
timothy 52786a545c feat(610): re-key the body-diff guard to key; enforce path<->key; pin the filesystem invariant
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Failing after 14s
PR Gates / Docs update reminder (pull_request) Successful in 19s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m24s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m19s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m41s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Three Done-when items that were genuinely unfinished. I audited the checklist
before ticking it rather than after, and these were not done.

1. BODY-DIFF GUARD RE-KEYED FROM HEADING TO `key`. Heading-keying made a pure
   RENAME look like a removal plus an unrelated addition, so retitling a record
   failed CI as an "unlogged removal" -- a trap that has bitten this repo before.
   Records are now MATCHED by key and REPORTED by heading (a key alone is not
   enough for a human to find the record). Demotion, which is inherently about a
   record losing its key, is detected on the heading axis instead, and a demoted
   record is no longer double-reported as removed. Key-matching is also what lets
   the comparison work straight through the legacy->frontmatter migration, since
   `key` survives both the format change and the move between files.

   Pinned by a test driving the REAL git-backed diff engine: retitle a record,
   same key, byte-identical prose -> removed/rewritten/demoted all empty.
   Mutation-verified: restoring heading-keying makes it fail with exactly
   "a retitle was reported as a removal".

2. PATH <-> KEY ENFORCED BY THE VALIDATOR. The filename is derived from the key,
   which is what makes one-active-per-key a filesystem property rather than a
   check -- but only if the two cannot drift. Now an error when they disagree,
   with legacy multi-record files explicitly exempt (they have no key-derived
   path to match).

3. ONE-ACTIVE-PER-KEY PINNED AS A FILESYSTEM PROPERTY. Two records sharing a key
   derive the same path, so the filesystem refuses the second; the migration
   aborts on a destination collision rather than silently overwriting. Both are
   now tested, the latter end-to-end through `main()`.

Also fixed a vacuous assertion ruff caught in my own new test -- `assert X or
True` is always true.

Note on verification: my first positive control for the guard reported it NOT
firing. That was the probe, not the code -- "load 340" appears in both the
`sources:` frontmatter field and the body, and replace(...,1) hit the metadata
copy, which is correctly token-free. Re-run against genuine body prose, the guard
fires with exit 1.

refs #610
2026-07-25 19:53:08 +02:00
timothy fba5233caf feat(610): split the decision corpus into one YAML-frontmatter file per record
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Failing after 23s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m17s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m5s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 16m5s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 17m6s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
168 records -> docs/decisions/records/<area>/<topic>.md (163 active, 23 dirs) and
docs/decisions/archive/<area>/<topic>.md (5 archived). The filename IS the key,
so one-active-record-per-key becomes a filesystem property rather than a
validator check, and supersession becomes a `git mv`.

WHY: the monolith was a concurrency problem before an aesthetic one. A
3,900-line append target made parallel sessions collide -- PR #605 and PR #614
both hit append-vs-append conflicts during routine rebases, and hand-resolving
those inside the corpus is exactly the operation the rationale-rewrite guard
exists to police.

HOW IT IS VERIFIED: a ~170-file diff cannot be meaningfully read, so correctness
does not rest on reading it. The parser was taught BOTH formats first, so the
body-diff guard parses the old form at the merge-base and the new form at head --
the migration validates itself, no bypass. The proof is a field-level equivalence
harness: 168 records before and after, zero lost, zero gained, zero field
mismatches, zero rationale bodies differing. Reviewers should scrutinise the
harness; it is the actual evidence.

What measuring caught that reading would not have:

- ~500 lines sit OUTSIDE any record -- decisions.md's lifecycle schema and each
  topic file's preamble, mostly the only copy. Source files are kept and
  stripped, never deleted. They also cannot be filed per-area: topic files hold
  several areas and 4 of 23 areas span several files.
- Archive discovery was a non-recursive glob; after the split it found ZERO
  archived records, surfacing as four bogus "supersedes points to unknown key"
  errors rather than an obvious failure.
- ~32 live docs point into the corpus BY DATE, which the split dangles. Each
  stripped file now ends with a generated "Records formerly in this file" index,
  which also rescues the identical breadcrumbs in old issue comments.
- decisions.md's "In this file:" list was 97 same-file anchor bullets that the
  split makes WRONG, not merely stale. Dropped; the generated index replaces
  them with links that resolve.

The equivalence harness now runs against a checked-in FIXTURE, not the live
corpus. The earlier version migrated the real tree, which made it a one-shot:
the moment the migration landed there was nothing left to move and the tests
failed for reasons unrelated to the code. A fixture keeps them testing the
SCRIPT rather than the repo's current state.

Keys preserved verbatim, warts included: `sched` (12) and `scheduling` (1) remain
two directories for one concept. Renaming a key is not a move -- it changes
identity, breaks the equivalence proof, and invalidates MemPalace's per-key
drawers. Taxonomy normalisation is separate work.

refs #610
2026-07-25 19:45:09 +02:00
timothy 8578dc1ca7 fix(610): de-brittle the count assertion; stop conflating index lines with preserved prose
Both surfaced when main gained two records mid-flight.

- test_record_count_is_the_expected_166 hardcoded the total, so it failed the
  moment a record landed on main -- a merge turning an unrelated test red. The
  real invariant is before == after; the count only needs to prove the harness
  isn't parsing a stub corpus, so it is now equality plus a floor.

- The migration's "lines preserved" figure silently absorbed the generated
  where-did-it-go index once that was threaded into the preamble string, jumping
  507 -> 759 with no new prose preserved. It now reports the two separately:
  514 lines of original prose, plus 245 generated index lines. A number that
  quietly changes meaning is worse than no number.
2026-07-25 19:08:58 +02:00
timothy 64b65fd2db feat(610): generated where-did-it-go index on each stripped file
Live docs point into the corpus BY DATE -- "see `decisions.md` 2026-07-10" --
about 32 such references across 12 files, plus the same form in historical issue
comments. The split would dangle every one of them.

Each stripped file now ends with a generated "Records formerly in this file"
index: date, title, and a link to the record's new path. A reader following a
date pointer lands on the file it names and resolves from there. That is far
cheaper and less error-prone than rewriting 32 references by hand, and it also
covers the issue-comment breadcrumbs, which cannot be rewritten at all.

Caught while verifying it: the generated `## Records formerly in this file`
heading is itself an H2, so the record parser counted one legacy-unmigrated
record per stripped file -- the notice went 0 -> 6. Same treatment as the
existing `## Index` section: skip it by name. SKIP_HEADINGS moved to
decisions_lib as the single source of truth, since three modules now need it.

Found by reading the validator's notice output on a trial migration, not by
inspection -- the corpus still validated OK, so nothing else would have flagged it.
2026-07-25 19:08:58 +02:00
timothy d52268f2ed feat(610): migration script + field-level equivalence harness
The migration is a MOVE, so correctness is provable rather than reviewable:
parse the corpus before, migrate, parse after, assert the Record sets are
identical field-for-field with byte-identical rationale.

scripts/migrate_decisions_split.py
  166 records -> docs/decisions/records/<area>/<topic>.md (active) or
  docs/decisions/archive/<area>/<topic>.md (superseded/retired), 26 directories.
  Path is DERIVED from the key, so identity stays the key. Refuses to run if any
  record lacks a key, and aborts on a destination collision.

  Source files are KEPT, stripped to their narrative -- not deleted. 506 lines of
  the corpus sit outside any record: decisions.md's lifecycle-schema header (the
  status vocabulary, supersession rules, the edit-token contract) and each topic
  file's preamble explaining why those records exist. For most of it that is the
  only copy. It also cannot be filed per-area -- topic files hold several areas
  and 4 of 23 areas span several files -- so the files themselves stay.

scripts/tests/test_migration_equivalence.py
  Runs the real migration against a COPY of the real corpus in tmp_path, never
  the working tree. Asserts: all 166 records survive with the same keys, every
  metadata field round-trips, titles round-trip from headings, every rationale
  body is byte-identical, path matches key, archived records land in the archive
  wing, the legacy files keep their narrative, and no parseable record is left
  behind in them.

Proven non-vacuous: corrupting one migrated record's prose is caught by the
byte-identical check, and deleting one is caught by the survival check.

One test-authoring note: an early assertion string-matched "## " to prove no
records were left in decisions.md. That is wrong -- the schema header quotes an
illustrative "## 2026-07-17 ..." example in prose. Whether records remain is a
PARSING question, so the parser-based leftover test is the real invariant.
2026-07-25 19:08:58 +02:00
timothyandClaude Opus 5 cc9481f541 fix(609): review fixes — exclude merge commits, unfold folded values, correct the fail-posture doc
Cross-family review (High + Medium + Low), all three reproduced before fixing:

High -- on a pull_request event actions/checkout lands on a synthetic merge commit whose
body the forge composes from the PR description, so a description ending in an example
marker armed a guard no author armed. Excluded merge commits from the range; merging main
into a PR branch is separately forbidden, so no author-written commit is skipped.

Medium -- a folded value (`no` + continuation ` yes`) was split into independent lines and
the continuation armed on its own, inverting the value the author wrote. Read with `unfold`
so the value is judged whole.

Low -- the module docstring promised fail-open while marker resolution deliberately fails
closed. The posture is right; the docstring was wrong. Documented as the one exception.

Both new negative controls verified red against the unfixed matcher. Decision record
amended to state the residual honestly rather than overclaim: a quoted example that is the
FINAL paragraph of an ordinary commit is a trailer by git's own grammar and does arm. What
the change buys is that discussing the marker can no longer disable anything.

fixes #609

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 18:29:46 +02:00
timothyandClaude Opus 5 c597c49f02 fix(609): arm the decisions body-diff with a git trailer, not a bare substring
The rationale-edit exemption was a substring test over the whole commit range, so any
message containing the literal marker armed it -- including prose explaining why no
marker was needed, which is how it fired live in PR #605: a green --base/--head run
that was vacuous on the body-diff dimension, in the one PR that hand-resolved a merge
conflict inside the corpus the guard exists to police.

Now read as an affirmative `Decisions-Edit:` git trailer. Git parses trailers only in
the final paragraph, so a quoted example message cannot arm it -- which matters because
this commit and its decision record both quote one. A non-affirmative value (`no`) does
not arm it either; the retired substring arms nothing and gets a ::warning:: nudge.

Tests: negative controls (prose mention, quoted mid-body trailer, `no` value, retired
substring) plus positive controls (trailer, uppercase, alongside Co-Authored-By). All
four negative controls verified red against the old matcher.

fixes #609

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 18:21:12 +02:00
timothy 75139bbf31 fix(603): close four defects found in adversarial review of the stale-after fields
1. `date.fromisoformat` is not a YYYY-MM-DD validator. On Python >= 3.11 it also
   accepts ISO basic format ("20270101") and week dates ("2027-W01-1"), so a
   malformed-looking value passed the blocking check — and which forms parse
   depends on the interpreter, meaning the same corpus could validate differently
   on a dev machine and on the runner (pr-checks.yml pins only python-version
   '3.x'). Knock-on: the catalog's Review-due section sorts on the raw STRING, so
   an accepted "20270101" sorted AFTER "2027-01-15" ('-' < '0'), contradicting the
   section's own "sorted soonest-first" text. Gate on ^\d{4}-\d{2}-\d{2}$ first,
   which fixes both — a fixed-width zero-padded form makes string sort == date sort.

2. A present-but-empty `stale-after:` was collapsed to None by `or None` in the
   parser and then skipped by a truthiness guard in the validator, so it passed as
   "absent" — a field that silently never fires, which is the exact failure mode
   the blocking check exists to prevent. Keep "" distinct from None and test with
   `is not None`.

3. `test_catalog_is_date_independent` was partly vacuous: with no date in either
   render, both sides were trivially equal after the .replace(). It did still catch
   an injected clock-derived marker, but it passed with the feature deleted. Assert
   the dates are present.

4. The malformed-date check ran only over the active set, exempting archive
   records. Staleness is moot there, but a typo is still a typo — check both wings.

Adds regression tests for each, plus a Review-due row for a topic-file record
(pinning the `../decisions.md` vs bare-filename link forms).
2026-07-25 15:22:50 +02:00
timothy 0ad02db651 feat(603): adopt OKF's optional stale-after and Sources decision-record metadata
Evaluated the Open Knowledge Format (GoogleCloudPlatform/knowledge-catalog okf
v0.2, scaccogatto/okf-skills) as a replacement for our decision-record system and
rejected it: its conformance rules are deliberately permissive exactly where ours
are strict (broken links, unknown types and missing fields must all be tolerated;
`deprecated` points at no successor), and its stable identity is the file path,
which the breadcrumb rule tells agents not to trust.

Adopted two of its optional families instead, additively:

- `stale-after: YYYY-MM-DD` on the metadata line — marks a record asserting an
  outside-world fact as due for re-confirmation. Absolute date, no TTL.
- `**Sources:**` in the metadata block — the evidence a record rests on, as
  distinct from `Signals:` (recall keywords).

Neither is required; absence is never an error. A malformed `stale-after` is
blocking (it would silently never fire), but a past-due record is only a
non-blocking `::notice::` — going stale is the passage of time, not a defect in
whatever commit is under test. The catalog's new "Review due" section renders the
date only and never a clock-derived verdict, so it cannot drift `--check` red on a
calendar boundary with no commit touching the corpus.

No backfill: no existing record adopts either field here.

fixes #603
2026-07-25 15:22:50 +02:00
timothyandClaude Opus 4.8 0ef053a209 ci(545): require a **Signals:** line on decision records
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / Docs update reminder (pull_request) Successful in 22s
PR Gates / decisions lifecycle (pull_request) Successful in 37s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m23s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m21s
The decisions-guard validator enforced lifecycle metadata (key/status/since/
supersedes/superseded-by + reciprocal links) but not the **Signals:** line —
which is exactly what MemPalace's keyword recall matches on. A record without
it ingests with weak recall metadata and produces confident false-negatives
for the "MemPalace to find, file to confirm" retrieval workflow.

Add "signals" to REQUIRED_META so a migrated record with a missing or empty
**Signals:** line fails the same way a missing key does. All 114 active + 2
archive records already carry a Signals line, so this is non-breaking on the
current corpus. Archive records are intentionally out of scope (recall targets
the active corpus).

- scripts/decisions_validate.py: signals in REQUIRED_META (+ rationale comment)
- scripts/tests/test_decisions_validate.py: _rec() default + missing/empty/present cases
- docs/decisions.md: header + Enforcement note the requirement and why

fixes #545

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 20:19:10 +02:00
timothyandClaude Opus 4.8 fb6720ea27 fix(521): de-dup 6 overlapped records; guard duplicate metadata blocks; exclude retrieval-eval; complete eval bank [decisions-edit]
- Exclude docs/decisions/retrieval-eval.md from active decision parsing
  (_NON_DECISION_FILES); its `## N.` eval-question headings were being
  miscounted as 7 legacy-unmigrated records.
- Add decisions_lib.metadata_line_count() + a decisions_validate guard
  that fails a record with more than one `key:` metadata line, so a
  stacked-metadata-block migration bug (which the parser silently
  tolerated by reading only the first block) can't recur unnoticed.
  TDD: test_duplicate_metadata_block_fails / test_single_metadata_block_passes.
- De-duplicate the 6 docs/decisions.md records left with two stacked
  metadata blocks (scan.getoraddfolder-db-lookup #488,
  scan.musicvideo-reconciliation #494, scan.jellyfin-mixed-content-library
  #489, iptv.logo-drives-bug-preset #67, ffmpeg.qsv-decode-encode-split
  #498, ci.small-lane-git-only server-management#639), merging the union
  of Signals/paths/issues/Mechanics from both blocks and keeping the
  richer Rule wording; rationale prose untouched.
- Fill in the deferred Q6b row in docs/decisions/retrieval-eval.md now
  that startup.parallel-orientation is active in docs/decisions.md,
  scoring it as a real active-vs-superseded question against the
  archived docs.queue-state-gitea-tracker.
- Regenerate docs/decisions/README.md via build_decisions_catalog.py.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:54:52 +02:00
timothyandClaude Opus 4.8 ab2d80b47a fix(521): reciprocity covers archive<->archive pairs; skip topic-file Contents heading
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:14:00 +02:00
timothyandClaude Opus 4.8 f93458c76c fix(521): whole-branch review — correct Gitea anchors, guard archive+demotion, reciprocal links, stale append-only refs, budget warning [decisions-edit]
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:00:15 +02:00
timothyandClaude Opus 4.8 458ab2111f fix(521): catalog _anchor collapses punctuation runs; single trailing newline
_anchor() mapped each space/hyphen to its own '-' without collapsing runs,
so the standard heading separator " — " (space, em-dash, space) produced a
double hyphen in every generated anchor. Since nearly every real decision
record heading uses that separator, this made the catalog emit a dead link
for essentially every row. Fix: after building the char list, collapse
consecutive '-' into one and strip leading/trailing '-' via re.sub, matching
how Goldmark/GitHub/Gitea sluggers behave.

Also fixed main() writing an extra trailing newline (want already ends in
"\n", then "+ \n" appended a second one) so docs/decisions/README.md now
ends with exactly one trailing newline; --check still compares via .strip().

Added test_anchor_collapses_em_dash_and_keeps_underscore to pin the anchor
behavior against the reported iptv.base_url case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 67619b2bf6 feat(521): active-catalog generator
Adds scripts/build_decisions_catalog.py, which renders docs/decisions/README.md
as a compact table of only 'active' decision records (sorted by key), and its
test scripts/tests/test_build_catalog.py. Supports --check for CI drift
detection. No decision records are migrated yet, so the generated catalog is
currently empty (banner + header only) — expected at this stage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 bdcc59ff80 test(521): add non-vacuous append-smuggle regression test for _rationale bound
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 7b8ac751d7 fix(521): bound _rationale strip window; restore REQUIRED_META; fail-open _run; archive-placement invariant
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 9266437d68 feat(521): decision lifecycle validator
Adds scripts/decisions_validate.py: lifecycle invariant checks (unique
active key, key format, reciprocal supersession, removed-without-archive,
rationale-rewrite-without-token, catalog staleness, corpus budget) plus
git-diff helpers for merge-base-based CI checks. Deviates from the task
brief in one spot: REQUIRED_META narrowed to (key, status) — the brief's
(key, status, since, supersedes, superseded_by) makes its own
test_clean_corpus_passes fail, since since/supersedes/superseded_by
default to None on bare Record() instances built without going through
the markdown parser.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 2cd78aa0bf style(521): lint/type cleanup on decisions_lib parser + test
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 ba39ca65ae feat(521): decision-record parser (decisions_lib)
Adds scripts/decisions_lib.py, the shared parser for ErsatzTV decision
records (docs/decisions.md + docs/decisions/*.md). Parses H2 sections
into Record dataclasses, distinguishing migrated records (visible
metadata block: key/status/since/supersedes/superseded-by + Rule/
Signals/Mechanics) from legacy-unmigrated ones with no metadata line.

scripts/ is now an importable package (scripts/__init__.py,
scripts/tests/__init__.py) so later tools can `import scripts.decisions_lib`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00