49 Commits

Include renames

Author	SHA1	Message	Date
timothy	f0f8708a6e	fix(632): fail closed when the head/base re-read itself fails ... Self-review of the previous commit. Folding the head and base re-reads into one `prjson_now=$(api_get ... || true)` swallowed a guard that used to be implicit: the old `sha_now=$(api_get ... | jq ...)` aborted under `set -e` + `pipefail` when the GET failed, before any status was written. With `|| true`, both `sha_now` and `base_now` come back empty, both `[ -n ... ]` guards no-op, and the status is written having confirmed nothing about either the head or the base — a fail-open regression introduced by the refactor itself. Confirmed the old behaviour empirically rather than by reading it: a failed piped command substitution under `set -euo pipefail` exits with curl's status. The refusal is now explicit, and pinned by a test — nothing asserted it before, which is exactly why the refactor could drop it silently. Mutation-verified: restoring `|| true` reddens that test alone. Refs #632	2026-07-26 23:16:35 +02:00
timothy	00e623c066	fix(632): bind a review verdict to its BASE branch, not only to its head sha ... #622 made `review-verdict/h10` a per-sha required status, so a new commit cannot inherit an old verdict — the required context is simply absent on the new head. Retargeting a PR's base reaches the same end from the opposite direction: the head sha and the status both hold still while the merge-base, and therefore the effective diff the verdict was formed against, changes underneath them. #622's record claimed the invariant holds "by construction"; this was the documented exception, and an unrecorded exception is how a guarantee degrades into a habit. `post-review-verdict.sh` now records the base branch in the status description as a trailing `(base: <ref>)`, and refuses to write a status at all if the base moved between reading the PR and posting — the same TOCTOU window the head check already covers, which the head check cannot see because retargeting does not move the head. `pretooluse-merge-consent.sh` reads the field back and denies when it no longer matches the PR's live `base.ref`. Two choices are load-bearing, and each is pinned by a test rather than left to a comment: - The comparator is `base.ref`, NOT `base.sha`. `base.sha` tracks the base branch's tip, which moves whenever anything merges to `main` — comparing it would invalidate every open verdict on every unrelated merge, converting a rare-event guard into a permanent merge deadlock. A base that merely advances is out of scope by design: rebasing onto it moves the head sha, which the per-sha binding already covers. - The field goes in the status DESCRIPTION, not the verdict comment. The comment body is parsed by `scripts/check-review-verdict.sh`, whose grammar had three false-opens in its history (#629); nothing parses the description, so this adds a field without reopening that surface. Scope is stated honestly rather than overclaimed: this is DETECTION on the hook path only. A commit status carries no base of its own, so the server-side required check cannot see a retarget, and a merge driven through the Gitea UI or API is unaffected. That is the accepted exposure — base changes are rare, manual, and this is a two-account repo — but it now fails loud in the one place that evaluates consent, instead of living only in a doc. Verdicts posted before this change carry no `(base: …)` and get NO opinion rather than a deny; denying would block every in-flight PR the day it lands, and the window closes on its own since verdicts are per-head and short-lived. Verified by mutation, six mutants, each killed by its intended test: remove the hook's deny; compare base.sha instead of base.ref; drop graceful adoption; stop recording the base; drop the TOCTOU guard; accept a PR with no resolvable base. The positive controls matter more than usual here — the test PR is deliberately non-docs (a docs-only PR short-circuits the whole gate and would never reach the base check) and the rest of the gate is unstubbed, so "the hook denied" alone proves nothing. Refs #632 Decisions-Edit: yes	2026-07-26 23:13:50 +02:00
timothy	b255b7ffdc	test(648): close the mutation gaps round 5 found — two tests passed for the wrong reason ... Round 5 could not break the predicate itself: 28,930 real runs of the script across 14,465 crafted --version strings on bash 3.2.57 and 5.3.15 produced zero fail-opens, and `{1,9}` is honoured on bash 3.2, so round 4's bound is not void on the authoring Macs. What it did find is that two of round 4's changes were unpinned, and the tests that looked like they covered them did not. Reverting BOTH the first-line slice and `[[:blank:]]`→`[[:space:]]` together left the whole suite green. The four filler cases are all killed by the SEPARATOR restriction alone, so they attributed the fix to the wrong layer. Added three cases carrying the literal word `version` (`jq\nversion\n9.9` and friends), which satisfy the separator rule and can only be stopped by confining the parse to line one with a newline-free blank class. The CR-strip test was worse: vacuous through two independent mechanisms. `str.splitlines()` also splits on `\r`, so a per-line view dropped the stray CR; and `subprocess.run(text=True)` translates `\r` to `\n` outright, so even a raw-string check on stdout was unfalsifiable. The mutant demonstrably emits `... = jq-1.6<CR> (parsed 1.6; ...)` at the byte level while the test reported green. Added `run_bytes()` and a bytes comparison. Both gaps are now mutation-verified: reverting either change reddens exactly its own test. Also records the operational edge this parser acquires in the follow-up: it is strictly fail-closed by design, so once the floor mode gates the required check, a jq wrapper that prints a banner line would deadlock merges. The fix there is to widen the accepted forms, never to relax fail-closed. Decisions-Edit: yes	2026-07-26 22:21:07 +02:00
timothy	807ebbd38e	fix(648): round 4 — the round-1 fail-open was still reachable, via an over-long number ... Round 4 found the round-1 MECHANISM alive in round 3's regex. The pattern guaranteed the operands were digits but not that they fit `test`'s integer range, so a 23-digit major made `[ "$major" -lt "$min_major" ]` error with "integer expression expected" — and `set -e` exempts a failing command in an `if` condition, so the conditional read false and the floor was never asserted. Exit 0. That is precisely what the empty string did in round 1: same shape, third occurrence, same predicate. Bounding the runs with {1,9} was not sufficient on its own. The pattern is unanchored at the end, so `jq-1.99999999999999999999999` simply matched the first 9 digits of the minor and compared THAT — a mis-parse that passes the floor rather than an error that skips it. The trailing `([^0-9]|$)` is what actually closes it. Second hole: `[[:space:]]` matches NEWLINES, so round 3's "anchor" still scanned the whole output. `jq\n2.34: cannot load shared library` matched `jq`, crossed the newline as separator, and parsed 2.34. Now the first line only, with `[[:blank:]]`. Third: the separator class `[-[:blank:]]{1,4}` could be walked across filler — `jq -- 2.34 (real jq-1.6)` parsed as 2.34, `jq<TAB><TAB>9.9` as 9.9. It is now one of the two forms real jq emits: `jq-1.6` or `jq version 1.6` (a blank separator REQUIRES the literal word `version`). Verified across a 20-case matrix: every legitimate form still parses to the right numbers (jq-1.6, jq version 1.6, jq-1.7.1, jq-1.6-dirty, jq-1.10 numerically, jq-1.6 (Debian 1.6-2.1), jq-v1.6, JQ-1.6, jq-1.6.0, CRLF), and every constructed attack fails closed. Four mutations, each reddening exactly its own tests. The real jq 1.8.2 on this machine still reports cleanly. Also: the log line now interpolates the first line, so a multi-line --version cannot split the single grep-able line the no-arg mode exists to emit. None of these are reachable from a real jq build. They are recorded and fixed because the guard's own stated invariant — never assert a floor against something it did not parse — was still violable three rounds in, and the follow-up PR moves this exact code into the branch-protection-required check.	2026-07-26 22:21:07 +02:00
timothy	4e094637c6	fix(648): the version parser was fail-OPEN on a jq that cannot start ... Round 3, and it found that round 2's fix was a REGRESSION on the case that matters most. `raw=$(jq --version 2>&1 || true)` did two wrong things at once: folded stderr into the parse input and discarded the exit status. Combined with a pattern that matched the first <digits>.<digits> ANYWHERE, a jq broken by a glibc mismatch — which exits 127 and writes "version `GLIBC_2.34' not found" to stderr — parsed as version 2.34 and PASSED the floor. The strip-based parse this replaced failed CLOSED there. So the fix for a fail-open bug introduced a worse fail-open bug, in the one script whose entire purpose is to refuse to certify a version it did not parse. Same mechanism, second symptom: an unanchored match let a prefix outrank the real version. `2026.07.26 jq-1.6` parsed as 2026.07; a leading warning line carrying any number won too. Now: jq's exit status is captured explicitly (`$?` inside `if ! cmd` is the NEGATED status, so that needed care too), stderr is kept out of the parse, and the pattern is anchored to the leading `jq` token. Every legitimate form still parses — `jq-1.6`, `jq version 1.6`, `jq-1.7.1`, `jq-1.6-dirty`, `jq-1.6 (Debian 1.6-2.1)`, `jq-1.10` (numeric compare, so the two-digit minor is not read lexically). The tests could not have caught any of this: the shim always exited 0 and never wrote to stderr, so every case it could express was clean. It now takes stderr and an exit code, and the four new cases turn red under the exact mutation. Also: the drift guard now strips comment lines before matching. A future comment citing `pulls/$pr/files?limit=100` as an example of what not to do would otherwise have reddened script-tests — which, per this branch's own correction, blocks merges. And the record no longer over-corrects: the combined-status read is guarded by `if [ "$mwcs" != "true" ]`, so a red script-tests blocks the hook-mediated merge path, not literally every merge. Decisions-Edit: yes	2026-07-26 22:21:07 +02:00
timothy	5e7623b8d5	fix(648,649): security-review round 2 — close the version-parse hole and the untested caller contract ... Two real defects, and three docs claims that were simply wrong. jq-preflight.sh parsed the version by stripping around the first `-` and `.`, which assumed the format is exactly `jq-X.Y`. A build printing `jq version 1.6` left major empty; the sanity check concatenated major+minor into "6", which is non-empty and all-digits, so it PASSED. The floor comparison then ran `[ "" -lt 1 ]`, which errors — and `set -e` exempts a failing command in an `if` condition, so the conditional read false and the script exited 0 having asserted nothing, after printing a plausible "parsed" line. The silently-untested-axis failure this script exists to eliminate, reproduced inside the script itself. Now parsed by explicit regex, failing closed with a diagnosis when there is no <digits>.<digits> match. Also: `--expect` with no value exited 1 with empty output on both streams. The hook's exit-status check was pinned by nothing: mutating `if files=$(...)` into `files=$(...) || true; files_complete=yes` left the ENTIRE suite green. It survived only by redundancy — the script writes stdout once, right before exit 0, so failures also happen to yield empty stdout and `[ -n "$files" ]` catches it. Safe by accident, which is the exact criticism this branch levels at the old code. Four tests now pin it, with a stub that FAILS while emitting a docs-only list (the one case redundancy cannot absorb) plus a positive control proving the harness can see the difference. Verified: the mutation now turns exactly those tests red. Docs corrections. The record claimed the --expect pin was safe because script-tests is "advisory, not a required check" — false. The merge-consent hook reads the COMBINED status (ci.advisory-red-blocks-the-merge-gate, #598), so firing the tripwire blocks every non-docs-only merge until someone re-pins. Kept anyway, for a stated reason, but no longer described as free. The record also asserted in the present tense that review-verdict.yml checks out the base ref; it has no checkout step at all, so that is now a future-tense requirement on the follow-up. And the documented .status allow-list named GitHub's `removed`, which the code rejects. The drift-guard regex anchored on `?limit=`, so a re-inlined copy written `files?page=1&limit=50` would have walked past it. Decisions-Edit: yes	2026-07-26 22:21:07 +02:00
timothy	2c10f057b8	fix(648,649): stage the enforced-gate wiring behind the scripts it calls ... Splits the review-verdict.yml rewiring out of this PR. That workflow checks out the PR's BASE ref — deliberately, so a PR cannot rewrite the gate that judges it — and the base is main, which does not yet contain scripts/pr-changed-files.sh or scripts/jq-preflight.sh. Wiring it here would make the job exit 127 on its own PR and block the merge gate through the combined status, which reads red jobs as blocking. So this PR lands the scripts, their tests, the hook rewiring and the script-tests jq pin; the follow-up points review-verdict.yml at them once they exist on main. The two tests that asserted on review-verdict.yml are scoped accordingly, each carrying the reason. test_review_verdict_never_pins_a_jq_version is asserted NOW rather than in the follow-up, so the no-pin constraint on the required check is already enforced when the wiring lands. Decisions-Edit: yes	2026-07-26 22:21:07 +02:00
timothy	2fd798cccf	fix(648,649): one shared PR-file enumeration + an explicit jq version contract ... #649 — the enforced review-verdict.yml guard had drifted strictly WEAKER than the advisory merge-consent hook: four rounds of #643 hardening landed on the copy whose failures produce a human prompt, and never reached the copy that writes the branch-protection-required review-verdict/h10 status. Its fail-closed behaviour on a garbage response was also incidental (an empty `n` erroring a bash conditional to false), not designed. Extract scripts/pr-changed-files.sh as the single implementation both call. Shared MECHANISM, not policy: the two docs-only allow-lists differ deliberately and stay separate. review-verdict.yml now checks out the BASE ref, never the PR head, so a PR cannot rewrite the gate that judges it. #648 — baking jq into docker/ci/Dockerfile provably cannot cover the gate that broke: review-verdict.yml is runs-on:small with no toolchain pin, so it gets the host's jq 1.6 (checked, not assumed). Add scripts/jq-preflight.sh: floor+observable everywhere, and a --expect tripwire on script-tests only — pinning the required merge check would deadlock every merge on a jq bump. Verified by mutation: six guards individually broken, each turning exactly its own test red, then restored byte-identical. fixes #648 fixes #649	2026-07-26 22:21:07 +02:00
timothyandClaude Opus 5	fefd11dffe	fix(620): signal corpus size per RECORD; the aggregate becomes an unthresholded trend ... Squashed from 7 commits (4 review rounds) to keep the rebase onto #621 tractable; the full round-by-round history is on PR #642. corpus was 5658/5600 — over budget and warning again — 3h35m after #619 put it at 5228, with nobody consolidating anything. So this does NOT re-baseline. An aggregate over a monotonically growing corpus can only ratchet; that is the "permanently red, therefore no signal" state #542 re-baselined away from, and growth is not even a smooth rate to plan against (the corpus FELL from 5089 to 5042 across four days, then gained 427 in one evening as two large records landed). Replaces it with a per-record prose ceiling (default 60), non-blocking, naming each record over it — not monotonic, so it can go red AND green, and it points at a file. The aggregate is still printed as an unthresholded trend notice, with record prose and non-record scaffolding reported separately because they are not the same unit. The GENERATED catalog is no longer counted at all: it gains one row per record and no consolidation can shrink it, which made the metric partly a record COUNT in a line-count costume. The calibration test took FOUR versions, and the failures are the durable lesson: v1 true by construction (`max(under) <= 60 < min(over)` over lists built by that test) v2 a gap WIDTH — a ceiling of 200 also sits in a wide gap, so it passed v3 fraction band + "clear air" vs the nearest record above — hostage to an unrelated record: one ordinary 62-line addition reddened it with the ceiling correctly placed, and the only remedy was to RAISE the ceiling. That is this very treadmill, as a hard failure in what #631 makes a blocking job. v4 `p90 <= ceiling <= p95` — the property stated directly and scale-free. Two rules recorded: a guard test must depend only on the thing it guards, and a threshold over a growing population must be expressed in that population's own terms. Candidates: all over-ceiling records assessed, each actioned or declined with a reason. The largest (scan.libraryfolder-unique-identity, 230 lines) is a legitimate DECLINE — a dozen-odd distinct traps whose only copy that is. Nothing pruned, so no archive or supersession was required. An automated redundancy metric is explicitly rejected. Also: `--budget` is accepted but announces its retirement rather than no-opping silently; the dead `budget_ok` parameter is gone; and five "untresholded" typos are fixed, one of which was propagating into the generated catalog row and MemPalace's per-key drawer. Refs #620 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-26 13:57:01 +02:00
timothyandClaude Opus 5	bb1809fbf0	fix(647): the jq-1.6 fail-open was ALSO in review-verdict.yml — the enforced gate ... Round-4 review caught the miss that matters most: #643 fixed the `jq -e`-on-empty fail-open in .claude/hooks/pretooluse-merge-consent.sh, but the SAME construct sits in .gitea/workflows/review-verdict.yml — and that is the copy that runs on the CI runner, where jq is 1.6, and that feeds the branch-protection-required review-verdict/h10 status. Reproduced: `printf "" | jq -e '.statuses | type == "array"'` exits 4 on jq 1.8.2 (guard fires, correct) and 0 on jq 1.6 (guard passes). So on a transient API error `statusjson` is empty, the guard lets it through, `existing` reads "", and the job posts `pending` — or for a bot/docs-only PR an exemption `success` — over a possibly-existing human verdict. That is precisely what the comment three lines above it says must never happen. The hook version was harmless in practice because it runs on a dev Mac with jq 1.8. This one is live. Fixed identically, with a comment naming why the sibling fix missed it, and the same hardening applied to the changed-files read in the same workflow. Also from round 4: - LOW, reproduced — an ARRAY-valued `.status` dodged the closed allow-list. `index` is polymorphic: with an array argument it does SUBSEQUENCE matching, so `[...,"renamed",...] | index(["renamed"])` is truthy while `.status == "renamed"` is false — the row passed the allow-list AND skipped the `previous_filename REQUIRED` clause. The same `git mv code -> docs/` dodge the closed set exists to block, one type away. Now requires `.status` to be a string first; mutation-verified. - The record now carries all THREE jq rules rather than the one, and notes that the durable fix is pinning/preflighting the runner's jq version rather than patching constructs one at a time (tracked on #647). 200 tests pass under BOTH jq 1.8.2 and jq 1.6. Refs #647, #643, #631 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-26 13:24:20 +02:00
timothyandClaude Opus 5	f4473926d4	fix(643): close the status-string dodge, pin the allow-list anchors, fix two doc claims ... Round-3 review (MERGEABLE, all findings Low/Nit) — applied anyway, because each is the over-claim or unpinned-anchor family this PR keeps hitting. LOW — the `renamed => previous_filename REQUIRED` clause matched `.status` by exact lowercase string, so any other value took the `else true` branch: a row with `"Renamed"`, or with no status at all, validated fine and silently dropped its SOURCE path, letting `git mv ErsatzTV/Program.cs -> docs/a.md` read as docs-only. `.status` is now checked against a closed set and an unknown value fails closed. Two things that fix taught me, both caught by my own positive control rather than by review: 1. The first predicate was WRONG in a way that gated everything: inside `[...] | index(.status)`, jq's `.` is the ARRAY, so `.status` was null and every row failed. `$s` is now bound from the row before the context switches. A security check that rejects everything looks identical to a working one from the failing side — only test_gitea_real_status_values_are_accepted caught it. 2. The set includes BOTH `changed` and `modified`. Live Gitea 1.25.4 emits `changed`, but a closed allow-list built from the wrong vocabulary is worse than the hole it closes: it would gate every genuine docs-only PR. The property wanted is "reject what we don't recognise", not "enumerate one version exactly". LOW — three allow-list anchors had no test at all: dropping `^` from the `docs/` alternative (`ErsatzTV/docs/Program.cs` would exempt), dropping `$` from `.md` (`x.md.cs` would exempt), and dropping the non-empty-list guard. Since the round-3 `..` finding WAS an anchor subversion, they are now pinned; all three mutation-verified. NIT — docs/ci-cd.md called this job "a checkout plus a pure-stdlib pytest run", which the same file contradicts 450 lines later and which this PR's own record names as the bug that turned the job red on its first CI run. Also replaced a frozen "111 tests" with an explicitly indicative figure — the suite is ~190 after rebasing onto main, and a frozen count has rotted four times this session. 198 passed under both jq 1.8.2 and jq 1.6. Refs #643, #631 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-26 13:24:20 +02:00
timothy	54c875414c	fix(643): validate every field the extraction consumes; fix a vacuous test ... Re-review of the previous fix commit found it incomplete and its test vacuous. Both reproduced before fixing. MEDIUM — the validation domain did not match the CONSUMPTION domain. `chunk` emits `(.previous_filename // empty)` for EVERY row regardless of status, but the guard validated that field only when `.status == "renamed"`. So a row marked `modified` — or Gitea's distinct `copied` — carrying a newline in previous_filename was still exempted. Verified EXEMPT for both statuses against the previous commit. Now validated whenever present, with the `renamed` => REQUIRED clause kept on top. MEDIUM — test_newline_in_previous_filename_is_also_rejected asserted the right outcome without exercising the mechanism: its payload's second segment was `ErsatzTV/Program.cs`, which the allow-list rejects on its own, so the test passed with the newline guard entirely removed. That is why the hole above went unnoticed — the same filter-hides-the-defect trap the guard itself is about. Payload changed to a segment that PASSES the allow-list, so the test now discriminates, plus parametrized regressions for modified/copied/added. LOW — `..` components rejected. The allow-list anchors `^docs/`, so `docs/../ErsatzTV/Program.cs` matched it (reproduced). Git will not produce such a path, but this guard exists to fail closed on unexpected 2xx shapes. Added a positive control (a legitimate docs->docs rename still exempts) so the tightened row schema cannot be satisfied by never exempting anything. Severity calibrated in the record: the docs-only exemption ends in `decide allow ""`, a passthrough to the normal permission prompt, NOT an auto-grant. Every bypass here downgrades a mechanical deny/ask to a human prompt; none can silently self-merge. Real, worth fixing, but not what an earlier framing of #643 implied. All mutation-verified; 123 passed under BOTH jq 1.8.2 and jq 1.6. Refs #643, #631	2026-07-26 13:24:20 +02:00
timothy	c046add10a	fix(643): close two more fail-opens in the docs-only enumeration, found by cold review ... An independent cross-family review of the jq-1.6 fix found two further ways the docs-only exemption can fire over an incomplete file list — both reachable with NO transport error, so neither had anything to do with the original bug. 1. HIGH — a path containing a newline. `chunk` flattens paths into newline-delimited text before the allow-list grep, so a filename of "safe.md\ndocs/Program.cs" splits into two lines that BOTH match the allow-list, while the real single path ends in .cs. Git permits newlines in filenames and the reviewer reproduced the bypass against this hook. Now rejected outright at the row-schema guard, on both `filename` and `previous_filename`: no docs path contains a control character, so failing closed costs nothing. 2. HIGH — a short page read as the last page. `n < 50` assumed the server's page size is the 50 we requested, but Gitea caps `limit` at the server-wide MAX_RESPONSE_ITEMS (default 50, configurable) and may return fewer. A 30-row docs page followed by a page of code completed the enumeration over a PARTIAL list. Only a validated EMPTY page may now terminate it; the page<=40 cap still fails closed, and the cost is one extra request. 3. MEDIUM — the enumeration was not bound to one head. Paging is several round-trips, so a force-push between them assembles a list belonging to no single commit: page 1 from head A plus a short docs tail from head B, with B's code page never read. The head sha is re-read after enumeration and the exemption refused if it moved. All three mutation-verified: reverting each fix reddens exactly its own test and nothing else. A positive control (short page then empty page) pins that the stricter terminator still exempts a genuinely docs-only PR, so "never terminate early" cannot be satisfied by never exempting anything. 118 passed under BOTH jq 1.8.2 and jq 1.6. The record now states the generalisable lesson: every defect here was an exhaustiveness failure in an enumeration whose completeness is load-bearing. When a security decision depends on having seen ALL of something, the termination condition must be positive and explicit, never inferred from a proxy. Refs #643, #631	2026-07-26 13:24:20 +02:00
timothy	5f068a2488	fix(631): close a fail-open in the merge-consent gate that only CI could see ... Running scripts/tests in CI for the first time turned up a real defect in the gate itself, not just in the harness. `jq -e` over EMPTY input exits 4 on jq >= 1.7 but 0 on jq 1.6 — verified against both binaries, not inferred. The docs-only pagination guard in pretooluse-merge-consent.sh leaned on that exit status to reject a transport failure. On jq 1.6, which the CI runner ships: page 2 errors -> gq returns empty -> jq guard wrongly PASSES -> n is empty so [ "$n" -lt 50 ] errors into false -> the loop walks PAST the failed page -> page 3 legitimately returns [] -> files_complete=yes over a PARTIAL list -> the docs-only exemption fires over unread pages that may be pure code. That is the very defect the guard's own comment describes, reintroduced one layer down by a jq version difference. Fixed by rejecting an empty body explicitly rather than inferring it from jq's exit status. The same hardening is applied to the review-verdict status read, which fell through to `vstate=""` -> deny (fail-CLOSED, so never a hole) but would have surfaced the wrong message. Why it survived: the existing transport-failure test asserts the right thing but can only observe the bug where jq is 1.6, so it passes on a developer Mac with the bug fully present — and the suite had never run anywhere else. The new test removes that dependency by shimming ONLY jq 1.6's empty-input exit status, so it pins the property on any host. Mutation-verified: revert the fix and it goes red on jq 1.8.2. The shim is deliberately narrow (no `-n`): a broader first version swallowed the `jq -n` calls `decide` uses to build its JSON, so the hook emitted nothing and every decision read as a passthrough — the verifier manufacturing the exemption it was meant to disprove. test_jq16_shim_actually_reproduces_the_quirk now pins the shim itself, `-n` case included. Suite: 113 passed under jq 1.8.2 AND under jq 1.6. Refs #631	2026-07-26 13:24:20 +02:00
timothy	3d720a6bc1	fix(621): exempt by exact path, not basename — my "no basename exemption" claim was false ... Re-review found the basename exemption was not actually removed: the code still dropped any wing-root README.md, and the docstring defended that with "no such file exists today; the catalog README lives in TOPIC_DIR". That is false — docs/decisions/archive/ README.md exists and parses to []. So the rule text in the record, the catalog row and ci-cd.md all asserted something the shipped code contradicted. It was also a live hole, not just wrong prose: dl.active_files() globs RECORDS_DIR.rglob with NO filter, so a future docs/decisions/records/README.md would be a corpus source while exempt from the guard — verbatim the hole this PR cites as the reason to stop using _NON_DECISION_FILES, reintroduced one directory over. Now exempt by exact RELATIVE PATH ({archive/README.md}), with a test pinning that the same basename in the ACTIVE wing is still faulted. Mutation-verified. DISAGREEING with one review finding, deliberately. It asked for the empty-wing anti-vacuity check to be symmetric across both wings. The semantics differ: zero ACTIVE records means the scan measured nothing and any clean result is vacuous, while zero ARCHIVED records is a normal state — nothing superseded yet, true of any young repo and every fresh clone before the first supersession. Making it symmetric fails a correct corpus (it reddened 7 tests). The asymmetry stays, with the reason in the code and a test pinning it so nobody "fixes" it back. 127 passed. Refs #621	2026-07-26 12:03:43 +02:00
timothy	98b3e8715b	fix(621): close three exemption holes and pin the guard's own wiring, per cold review ... An independent cold-context review found the guard could be disabled or bypassed in four ways, one of them silently. HIGH — the guard's only wiring was untested. Deleting `wing_faults=record_wing_faults()` from main() left the ENTIRE suite green (120 passed) while a real block-scalar record vanished again with `decisions-validate: OK`. Every existing test either called the collector directly or handed validate() a hand-built list, so nothing pinned that main() ever invokes it — the #609 "prints OK while doing nothing" defect, one level up, inside the fix for that same defect. Independently reproduced before fixing. Now pinned by a monkeypatched sentinel test. MEDIUM — a record could escape by BASENAME. `_NON_DECISION_FILES` is a set of TOPIC-dir names ({README, migration-map, retrieval-eval}) and was applied to the wings too, so a genuine record at records/docs/retrieval-eval.md was skipped. Not hypothetical: the path<->key rule forces key `docs.retrieval-eval` to that exact filename, and dl.active_files() applies that filter only to the TOPIC_DIR glob — so the file would be a corpus source while exempt from the guard. Now nothing is exempt by basename. MEDIUM — the archive exemption was by LOCATION but justified by IDENTITY. Anything directly in archive/ was skipped, so a new unparseable archive/foo.md would vanish in the one directory left unwatched. Now exempt only if it really is a #610 stripped index: one keyless record with a known generated heading. LOW — parse-to-WRONG. `rule: >-` with an UNINDENTED continuation containing a colon parses to one VALID keyed record whose rule is literally ">-", plus a junk key, silently truncating the real value. PyYAML rejects that input, so the hand reader is more permissive than the writer. Now faulted via an unknown-frontmatter-key check. LOW — an absent or empty wing reported clean. Now loud: refusing to certify a corpus from a scan of nothing. Doc corrections, all measured rather than asserted: "all 167 records use single quoted lines" was false (117 of 168 rule: values are unquoted plain scalars — the invariant is single-LINE); the archive shape in the record, ci-cd.md and the rule: field said `archive/<area>/` where the code deliberately uses archive/** ; the mutation count is 10, not five; and the "converts ANY reader limitation" claim is softened to the class it actually covers. 125 passed. Refs #621	2026-07-26 12:03:43 +02:00
timothy	b42df5f15f	fix(621): close a depth blind spot in the record-wing scan ... Adversarial self-review: the archive side used `glob("*/*.md")` while records used rglob. Both exempt the top-level stripped legacy files correctly, but the one-level glob would ALSO skip anything nested deeper, letting a record at archive/<area>/<sub>/x.md escape the check entirely. A path silently escaping the scan is precisely the failure mode this guard exists to close, so the exemption is now expressed as the actual condition — "directly in archive/" — rather than a glob shape that happens to match today's layout. Test added and mutation-verified: reverting to the one-level glob turns it red. 120 passed. Refs #621	2026-07-26 12:03:43 +02:00
timothy	0f565b1f7e	fix(621): make an unparseable decision record loud instead of silently invisible ... A file under the record wings that the dependency-free frontmatter reader cannot parse returned [] and simply vanished from the corpus: decisions_validate.py printed OK, build_decisions_catalog.py --check said "up to date", the record was absent, and nothing anywhere errored. Reproduced end-to-end with a YAML block scalar (`rule: >-`), which is the natural thing to reach for on this corpus's very long rule: values and which parsed fine under PyYAML before #610's dependency-free reader. An EXISTING record disappearing was already loud (the no-vanish diff check). The hole was a NEWLY ADDED record, which that check structurally cannot see — no base state to diff against — so the author's own PR looks clean. Hence a per-PATH check, not a per-construct or diff-driven one: asserting "this path yields exactly one keyed record" turns any present or future reader limitation from silent to loud in one move. Adds record_wing_files/record_wing_faults to decisions_validate.py, surfaced through validate() as ERRORS (a file in the record wings that is not a record is a mistake by definition) and reported first, since a file that failed to parse leaves every downstream check silently evaluating an incomplete corpus. The five top-level stripped legacy archive files are exempt — they are generated "Records formerly in this file" indexes, keyless by construction, and are what keeps older date-based pointers resolvable. _read_frontmatter is deliberately NOT extended to accept block scalars; rationale in the new docs.record-wing-parse-guard record. 8 tests, mutation-verified: with the check neutered 5 go red, restored all 119 pass. A live-corpus positive control asserts the wings are non-empty so a clean result can never be vacuous. Refs #621	2026-07-26 12:03:43 +02:00
timothyandClaude Opus 5	50bcd7b0c7	fix(629): strip raw HTML blocks, and state where the hardening stops ... Round 5: raw HTML is the third code-block form. `<pre>`, `<code>` and HTML comments all render their contents literally, so a verdict inside one is an example, not an approval: <pre> / <code> / <!-- ... --> containing a verdict -> positive Now stripped, tracked as a marker count rather than parsed — the direction of error is to strip MORE, which can only ever withhold approval. Mutation-verified: removing the stripper fails all five cases. AND THE HARDENING STOPS HERE, deliberately. The record now says so, because otherwise the next session re-derives it: this is a best-effort heuristic, not a markdown parser. It covers the three code-block forms markdown has (fenced; indented, via the column-0 rule; raw HTML) and is not proof against every way to render text as non-prose. Stopping is safe because the comment is NOT the load-bearing gate. Since #622 the authoritative signal is the `review-verdict/h10` commit status, written only by post-review-verdict.sh from explicit arguments — a comment cannot forge it. This classifier is condition (c) of the PreToolUse hook: defense in depth on an agent's merge call. A residual false-open means the hook does not object; it does not mean a merge happens. Five rounds found five code-block forms, four of them introduced while fixing the previous round. The generalisable rule, now in the record: when a heuristic keeps failing at the edges, check whether it is actually the thing enforcing the invariant before spending another round on it. Also measured, against the real corpus: a "verdict must be the first line" rule would have killed every code-block form at once, but 14 of 18 verdict markers ever posted in this repo are NOT on the first line — so it was rejected as a retroactive break, not deferred. 178 tests. refs #629 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Decisions-Edit: yes	2026-07-26 01:14:59 +02:00
timothyandClaude Opus 5	28a9d0dbfd	fix(629): round-4 — require the marker at COLUMN 0, narrowing the grammar instead of patching again ... Round 4 found the last false-open: markdown has a SECOND code-block form the fence stripper does not cover — indented blocks (4 spaces or a tab). A pasted indented example still self-approved: Example: Review-verdict: MERGEABLE @ <head> -> positive Adding an indented-code stripper would be the same move that produced rounds 2, 3 and 4: fixing the instance, not the class. So the grammar is narrowed instead — the marker must sit at COLUMN 0. That kills every indentation-based ambiguity at once (4-space, tab, list-nested, arbitrary indent). Cost, accepted deliberately: a verdict indented under a list item is now ignored and classifies `absent`, which asks a human. For a gate, erring toward ignoring is the safe direction. Fence detection KEEPS its leading-whitespace tolerance, because stripping more is always safe. `test_leading_indent_is_tolerated` asserted the old behaviour and is replaced by `test_falseopen_an_indented_verdict_is_not_a_verdict`, parameterised over four indent shapes and mutation-verified: restoring `^[[:space:]]*` fails all four, control green. 172 tests. Round 4 verified clean by execution: the rc plumbing fails closed for a forced failure in the inner jq, awk, grep AND the pipeline producer (rc 3/4/5/93 -> exit 2); fence-length semantics, mismatched markers, CRLF fences, 10-marker fences, blockquote fences; emoji, CRLF, a 120k line, 200 comments; NUL rejection with no JSON-encoding bypass. refs #629 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Decisions-Edit: yes	2026-07-26 00:52:11 +02:00
timothyandClaude Opus 5	299e7b27af	fix(629): round-3 fixes — fence LENGTH semantics, and stop masking reader failures ... Third review round, third set of real findings. Both reproduced before fixing. 1. High: markdown closes a fence only with N-or-more of the SAME marker it was opened with, so a ```` block legitimately CONTAINS a ``` line as content. Toggling on any 3+ marker left the fence at that inner line and graded the verdict below it as a real approval: ````text / ``` / Review-verdict: MERGEABLE @ <head> / ```` -> positive Now tracks the opening marker's character and length; a shorter or different marker while a fence is open is content, so it neither closes the fence nor escapes it. 2. Medium: `awk ... | grep ... || true` flattened "no match" (grep rc 1, normal) together with a real tool failure (rc >= 2). A failing reader produced no verdict lines at all — `absent` — silently discarding a real BLOCKED verdict. awk and grep are now checked separately, and only "no match" is tolerated. Also fixed while writing (2): `[ rc = 0 ] && printf` as the loop body's LAST command would leave the subshell exiting 1 whenever the newest comment carried no verdict, which the rc check would then report as a failure to read comment bodies — an ordinary PR reading as broken. Uses an `if`. 169 tests. Both findings mutation-verified: restoring the naive fence toggle fails all four longer-fence cases, restoring `|| true` fails the failing-grep case, control green. Verified the ordinary shapes still work: plain ``` and ~~~ fences and lang-tagged fences still stripped, an unclosed fence still swallows, a real verdict beside a fenced example still counts, and a fenced positive alongside a real BLOCKED still classifies negative. refs #629 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Decisions-Edit: yes	2026-07-26 00:47:40 +02:00
timothyandClaude Opus 5	3bfd925baf	fix(629): re-review fixes — the READ path must fail closed too ... Re-review of 8d0df553 returned BLOCKED. Both findings are the same class as the bugs this issue is about, one level down: the reader, not the grammar. 1. A here-document makes bash materialise a temp file. When that fails (read-only or restricted TMPDIR) the loop reads NOTHING, and the classifier returned `absent` — silently discarding a real BLOCKED verdict. Reproduced with TMPDIR=/nonexistent: `absent` before, `negative` now. Both loops are off here-docs (a pipe for the reader, process substitution for the classifier so its flags stay in the current shell). "The environment could not supply a temp file" is not evidence that a PR was approved. 2. A malformed payload degraded to `absent` instead of an input error: an object-valued `.body` reached the scanner and simply matched nothing. Shape is now asserted in jq — must be an array, bodies must be strings — so it exits 2 and callers fail closed. Also closed a pre-existing false-open the review found while probing (present before #629, so not a regression, but cheap to fix here): bash strips NULs in command substitution, so a body of `Review<NUL>-verdict: MERGEABLE @ <head>` arrived at the matcher as a valid verdict line — text that is not a verdict becoming one in transit. Bodies containing NUL are now rejected. 161 tests. Verified: 80k body still classifies; empty array, null body still `absent`; malformed JSON, non-array, array-of-non-objects, object body, NUL body all exit 2; a hostile TMPDIR still returns `negative`. refs #629 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-26 00:47:40 +02:00
timothyandClaude Opus 5	f151b93245	fix(629): review fixes — tilde fences, an unbounded sha field, and a forgeable comment boundary ... Cross-family review of 38a96f47 returned BLOCKED with three findings. All reproduced first: ~~~ fence -> positive fences were stripped for ``` only; markdown also takes ~~~ @ <40hex>f / ZZZ -> positive the sha matched {7,40} with NO right boundary, so an over-long or malformed token was TRUNCATED into a passing one \x01BODY-BOUNDARY\x01 -> positive an in-band separator joined comment bodies, so a body containing that line forged a boundary, reset fence state mid-comment, and exposed a verdict inside an unclosed fence Fixes: both fence markers honoured; the hex run matched whole, required to end at a non-alphanumeric boundary, with its length validated separately so an out-of-range token is rejected rather than trimmed to fit; and bodies carried OUT-OF-BAND (one JSON-encoded string per line), which removes the forgery class instead of escaping the sentinel. The third is the one worth remembering: an in-band delimiter is forgeable by whoever writes the data, and here that is anyone who can comment on the PR. Two of these fixes broke previously-green tests, both of which were right to break: - an over-long token now classifies `no-sha`, not `stale`. The fixture asserting `stale` was 45 hex chars, so it had been exercising the length guard while claiming to test the prefix rule. Rebuilt as a well-formed 40-char sha that contains the head prefix without starting with it. - `jq -e` exits 4 when a filter produces NO output, which is the legitimate empty-comment-list case. Treating it as an error turned "no comments yet" into an input error — and callers fail closed on those, so a new PR would have read as unclassifiable. Exit 4 is now accepted. 44 classifier tests, 155 total. `~~~` and the boundary fixes are each mutation-verified; the out-of-band fix has no equivalent mutation (it is structural, not a regex) so its evidence is the direct reproduction against the previous commit. refs #629 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Decisions-Edit: yes	2026-07-26 00:47:40 +02:00
timothyandClaude Opus 5	0f83b54334	fix(629): close three false-opens in the H10 verdict grammar, and give it tests ... The H10 classification lived inline in `pretooluse-merge-consent.sh` with no tests. Three protections the `release.review-verdict-gate` record described were never actually implemented, and each graded an unreviewed head as approved. All three reproduced first: 1 MERGEABLE-LATER -> positive the token was prefix-matched, so any word STARTING with mergeable/approved/lgtm passed 2 fenced code block -> positive the line-start anchor is satisfied inside ```, so documentation showing the convention was a verdict 3 URL-borne sha -> positive the sha came from the first `@<hex>` ANYWHERE on the line, so a markdown link could supply it Fixes: whole-word token matching, with a token in neither vocabulary classified `unknown` (never positive, and not guessed into a block either — it goes to a human); fenced blocks stripped with fence state reset per comment body; the sha read from the verdict's OWN `@ <sha>` field, which also makes multi-`@` lines unambiguous. The grammar moves to `scripts/check-review-verdict.sh` so it can be tested at all — 38 tests, and each fix mutation-verified: restoring the old regex/extraction makes exactly the corresponding test fail, control green. #629's fourth reported item is NOT a defect and is not claimed as a fix. A later `@ <head>` on a BLOCKED line was reported as "masking a negative"; under the documented grammar that line is a verdict for the sha in its own field, so `stale` is correct — and was correct before this change too. Kept as a characterization test. `test_post_review_verdict.py`'s cross-check re-implemented the hook's regexes in Python and asserted the shell still contained them. That mirror is removed: it is the same duplication that let these three survive, and a Python copy would keep passing while the shell drifted. It now runs the real classifier. The decision record is corrected — it asserted the URL protection this commit actually adds. Note: the active corpus is 5637 lines against a 5600 budget, so the validator emits its consolidation warning (non-blocking). That is #620's subject, not regressed here. fixes #629 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Decisions-Edit: yes	2026-07-26 00:47:40 +02:00
timothy	07e1e8cfbc	fix(622): validate rename rows too; name the real blocker when only the verdict is pending ... Third BLOCKED verdict, third legitimate finding. The per-row guard validated the DESTINATION only, so `{"filename":"docs/x.md","status":"renamed"}` with no `previous_filename` passed while its source silently vanished — precisely the path-hiding that collecting both sides exists to prevent. A rename row must now carry a non-empty `previous_filename`. Required for rename rows ONLY. Demanding it globally would reject every ordinary modified/added row and make the gate refuse all exemptions — which every "withholds the exemption" test would happily pass through, so that direction gets its own positive control. Predicate checked against ten shapes before adopting; mutation-verified in both directions (dropping the clause fails 2 tests, applying it globally fails 3 including the controls). Also fixes a wart this PR introduced. `review-verdict/h10` is itself folded into the COMBINED status, so a PR awaiting its verdict reports combined 'pending' and the hook's condition (a) reported it as a CI problem — sending a reader to build logs when the missing thing is the review, and exiting before the H10 branch that would have said so. The message now names the outstanding contexts, and says plainly when the verdict is the only one left. Scope boundary, stated in the record: real Gitea populates `previous_filename` on renames and returns well-formed pages. Everything past this point defends against shapes with no evidence of existing, so the guard's claim stays "any page we cannot fully classify withholds the exemption" rather than growing to cover unobserved responses. Decisions-Edit: yes	2026-07-25 22:43:30 +02:00
timothy	fc547b2d46	fix(622): validate file pages per ROW, and de-vacuum the exemption tests ... Second BLOCKED verdict, also correct. My previous fix validated only the TOP-LEVEL type of each page, which left the same hole one level down: `[{}]` is a well-formed JSON array whose rows carry no `filename`, so it contributes zero paths, looks like a short final page, and completes the enumeration from a PARTIAL list — the exact failure the guard exists to prevent, just wearing a valid-looking wrapper. An array of scalars additionally tripped `set -e` during extraction without producing a decision. Both call sites now require every row to carry a non-empty string `filename`. An empty array remains valid, since that is what a real end-of-pagination is. Predicate checked against nine shapes before adopting: `[]` and well-formed rows valid; `[{}]`, `[{"filename":""}]`, `[{"filename":null}]`, `["x"]`, `[1,2]`, a bare error object, and a mixed good/bad array all rejected. The reviewer also caught that my `test_first_page_failure_*` was VACUOUS: with page 1 failing the path list is empty, and the hook independently withholds the exemption on an empty list, so it passed with the guard removed. It is kept as a smoke case with that limitation stated in its docstring, and three tests that actually pin the behaviour were added — malformed rows and scalar rows after a FULL page 1 (so the path list is non-empty and the exemption would really fire), plus MAX_PAGES exhaustion. Added a positive control too: an empty final page must still exempt. Without it, an over-strict guard would make every "withholds the exemption" test pass for the wrong reason — the suite would be asserting "never exempt", which is not the contract. Verified by mutation: top-level-only validation fails 2 tests, an over-strict guard fails the positive control, and widening MAX_PAGES fails the exhaustion test. Decisions-Edit: yes	2026-07-25 22:23:54 +02:00
timothy	f6d01c4924	fix(622): make the exemption paging fail CLOSED; read statuses latest-per-context ... Cross-family re-review returned BLOCKED on the fix commit, correctly. Two of the three findings were on the FAILURE path, which is why every happy-path run and the whole first test suite looked fine. 1. (High/Medium) Both paging loops treated a failed or non-array page as a legitimate short final page: empty output counts as zero rows, zero rows reads as "end of list", and the enumeration completed from a PARTIAL list — failing OPEN at exactly the point the guard exists to fail closed, while the decision record claimed the opposite. Each page is now validated as a JSON array before its rows are counted; anything else withholds the exemption. 2. (High) The workflow's "is there already a verdict" pre-check read `/statuses/{sha}`, which returns one row per status POST rather than per context and pages at 50 — so a head with a few CI reruns can push a real verdict off the first page (this PR's head already carries 15 rows for 11 contexts). Missing it there is not benign: the job would post `pending`, or an exemption `success`, OVER a human verdict. Both the workflow and the hook now read `/commits/{sha}/status` (latest-per-context), and the workflow refuses to post anything when that read fails rather than inferring "no verdict yet". Credit: raised independently by the parallel session on this issue. 3. (Low) The record claimed both callers withhold the exemption when enumeration cannot complete. True only after (1); the claim and the code now agree. Adds scripts/tests/test_merge_consent_exemption.py — 9 tests over the hook's exemption path via a curl shim, asserting behaviour when a page ERRORS or returns a non-array body, not merely when it returns data. Mutation-proven: reverting the array validation fails both failure-path tests; dropping `previous_filename` fails the rename test. One test I wrote was wrong and the run caught it: I asserted a `.gitea/... -> docs/...` rename must break the hook's exemption, but that hook deliberately exempts `.claude/`/`.gitea/`/`.husky/` too (to a human PROMPT, never an auto-grant), so both sides legitimately match. The rename guard matters there for a genuinely non-exempt source; the stricter PROTECTED semantics belong to the workflow. Both cases are now pinned, including a positive control that a docs->docs rename stays exempt. Decisions-Edit: yes	2026-07-25 22:04:18 +02:00
timothy	628ae46f7f	fix(622): bind H10 merge consent to the reviewed sha via a required commit status ... `pretooluse-merge-consent.sh` proves all three consent conditions at the moment the merge tool is called. With `merge_when_checks_succeed=true`, condition (a) is delegated to Gitea, which then merges whatever head is green at ITS merge time — while (b) Done-when and (c) the head-referencing verdict were proven against the head at SCHEDULING time. Every commit pushed in between merges unreviewed. The gate is not bypassed; it is satisfied against a snapshot that stops being true. Demonstrated as a controlled A/B rather than inferred, with a CI check left pending so Gitea waits as it really does: without a required verdict context: unreviewed commit B MERGED with it: same sequence REFUSED after reviewing B and posting it: merges — blocked, not deadlocked NOTE the anecdote in #622 is wrong and is corrected in the docs rather than repeated: PR #619 does carry `Review-verdict: MERGEABLE @ 02c82b35`, posted six seconds before the merge, explicitly re-reviewing the follow-up commits. #622 was filed off a lagging API read. The hole is real regardless, and structural — nothing FORCED that re-review inside the window Gitea would have merged in. This turns a property that held by diligence into one that holds by construction. The fix is the sha, not a smarter check. A Gitea commit status belongs to exactly one commit, so a status written for a parent cannot be inherited by a child that did not exist yet. `review-verdict/h10` becomes a REQUIRED status check on main: push a new head and the context is simply absent, which Gitea reads as not-passing (verified against 1.25.4: missing AND pending both block, and auto-merge re-checks the current head). It also covers merge paths the hook never sees — Gitea UI, raw API, another agent's session. - scripts/post-review-verdict.sh writes the comment and the status together so they cannot drift, and re-reads head after commenting: if a commit landed mid-flight it writes NO status and exits non-zero rather than retargeting the verdict at a commit nobody read. - .gitea/workflows/review-verdict.yml auto-passes the two exempt classes that would otherwise deadlock — Renovate-authored (platformAutomerge) and docs-only — and marks everything else pending. Exemptions are void when the PR touches .claude/, .gitea/, .husky/, scripts/ or docker/ci/. - The hook refuses to SCHEDULE an auto-merge unless that status is green on head, and no longer claims "CI green" on the mwcs path, where it never read CI. Two silent false negatives in the exemption path, both found by verifying rather than reasoning, both fixed at BOTH call sites (workflow and the hook's pre-existing docs-only carve-out): - The files endpoint caps at 50 rows and IGNORES a larger `limit` — PR #619 has 194 changed files and `?limit=100` returns 50. A single page saw ZERO protected paths there where the full enumeration finds ten. Both now page to exhaustion and withhold the exemption if they cannot complete. - A rename is ONE row whose `filename` is the destination, the source only in `previous_filename`. Verified live: `.gitea/workflows/renovate.yml` -> `docs/innocuous-note.md` presented as docs-only with no protected path visible. Both now read BOTH sides. Limits are documented, not papered over: base changes leave the head sha (and so the verdict) unchanged, and a PR editing the workflow is judged by its own edited copy — so PROTECTED is a guardrail against accident, not a tamper-proof control. fixes #622	2026-07-25 21:51:02 +02:00
timothy	02c82b35ea	fix(610): make the frontmatter READ path dependency-free — CI has no PyYAML ... This is why `decisions lifecycle` went red, and it was NOT the known flake. I came close to dismissing it as one for the second time this session, because an earlier red on another branch genuinely was. The dual-format parser imported PyYAML to read frontmatter. `decisions-guard` does `setup-python` and installs NOTHING, so once the corpus was migrated every record became unparseable there: ModuleNotFoundError, job fails. The same would hit the Husky pre-commit hook and every contributor's machine. Installing PyYAML in CI is the wrong fix: READING happens everywhere -- CI, the hook, every dev -- while WRITING happens once, in a migration a human runs deliberately. So the read path is now dependency-free and only `migrate_decisions_split` (the writer) still imports yaml. A hand-rolled parser is only safe if it provably matches the library that WROTE the files, so `test_frontmatter_reader_matches_pyyaml_on_every_real_record` compares the two field-by-field across all 169 real records (importorskip, so it is skipped rather than failing where PyYAML is absent) with a >100-file guard against near-vacuity. It is narrow by construction: the frontmatter is machine-generated with default_flow_style=False and width=10**9, so every value is a single-line scalar, and the reader bails to None on anything nested. Verified by running all four affected entry points against a shim that makes `import yaml` raise: validate --base/--head, catalog --check, the kickoff guard, and the plain validate the pre-commit hook calls. All exit 0. refs #610	2026-07-25 19:57:22 +02:00
timothy	52786a545c	feat(610): re-key the body-diff guard to key; enforce path<->key; pin the filesystem invariant ... Three Done-when items that were genuinely unfinished. I audited the checklist before ticking it rather than after, and these were not done. 1. BODY-DIFF GUARD RE-KEYED FROM HEADING TO `key`. Heading-keying made a pure RENAME look like a removal plus an unrelated addition, so retitling a record failed CI as an "unlogged removal" -- a trap that has bitten this repo before. Records are now MATCHED by key and REPORTED by heading (a key alone is not enough for a human to find the record). Demotion, which is inherently about a record losing its key, is detected on the heading axis instead, and a demoted record is no longer double-reported as removed. Key-matching is also what lets the comparison work straight through the legacy->frontmatter migration, since `key` survives both the format change and the move between files. Pinned by a test driving the REAL git-backed diff engine: retitle a record, same key, byte-identical prose -> removed/rewritten/demoted all empty. Mutation-verified: restoring heading-keying makes it fail with exactly "a retitle was reported as a removal". 2. PATH <-> KEY ENFORCED BY THE VALIDATOR. The filename is derived from the key, which is what makes one-active-per-key a filesystem property rather than a check -- but only if the two cannot drift. Now an error when they disagree, with legacy multi-record files explicitly exempt (they have no key-derived path to match). 3. ONE-ACTIVE-PER-KEY PINNED AS A FILESYSTEM PROPERTY. Two records sharing a key derive the same path, so the filesystem refuses the second; the migration aborts on a destination collision rather than silently overwriting. Both are now tested, the latter end-to-end through `main()`. Also fixed a vacuous assertion ruff caught in my own new test -- `assert X or True` is always true. Note on verification: my first positive control for the guard reported it NOT firing. That was the probe, not the code -- "load 340" appears in both the `sources:` frontmatter field and the body, and replace(...,1) hit the metadata copy, which is correctly token-free. Re-run against genuine body prose, the guard fires with exit 1. refs #610	2026-07-25 19:53:08 +02:00
timothy	fba5233caf	feat(610): split the decision corpus into one YAML-frontmatter file per record ... 168 records -> docs/decisions/records/<area>/<topic>.md (163 active, 23 dirs) and docs/decisions/archive/<area>/<topic>.md (5 archived). The filename IS the key, so one-active-record-per-key becomes a filesystem property rather than a validator check, and supersession becomes a `git mv`. WHY: the monolith was a concurrency problem before an aesthetic one. A 3,900-line append target made parallel sessions collide -- PR #605 and PR #614 both hit append-vs-append conflicts during routine rebases, and hand-resolving those inside the corpus is exactly the operation the rationale-rewrite guard exists to police. HOW IT IS VERIFIED: a ~170-file diff cannot be meaningfully read, so correctness does not rest on reading it. The parser was taught BOTH formats first, so the body-diff guard parses the old form at the merge-base and the new form at head -- the migration validates itself, no bypass. The proof is a field-level equivalence harness: 168 records before and after, zero lost, zero gained, zero field mismatches, zero rationale bodies differing. Reviewers should scrutinise the harness; it is the actual evidence. What measuring caught that reading would not have: - ~500 lines sit OUTSIDE any record -- decisions.md's lifecycle schema and each topic file's preamble, mostly the only copy. Source files are kept and stripped, never deleted. They also cannot be filed per-area: topic files hold several areas and 4 of 23 areas span several files. - Archive discovery was a non-recursive glob; after the split it found ZERO archived records, surfacing as four bogus "supersedes points to unknown key" errors rather than an obvious failure. - ~32 live docs point into the corpus BY DATE, which the split dangles. Each stripped file now ends with a generated "Records formerly in this file" index, which also rescues the identical breadcrumbs in old issue comments. - decisions.md's "In this file:" list was 97 same-file anchor bullets that the split makes WRONG, not merely stale. Dropped; the generated index replaces them with links that resolve. The equivalence harness now runs against a checked-in FIXTURE, not the live corpus. The earlier version migrated the real tree, which made it a one-shot: the moment the migration landed there was nothing left to move and the tests failed for reasons unrelated to the code. A fixture keeps them testing the SCRIPT rather than the repo's current state. Keys preserved verbatim, warts included: `sched` (12) and `scheduling` (1) remain two directories for one concept. Renaming a key is not a move -- it changes identity, breaks the equivalence proof, and invalidates MemPalace's per-key drawers. Taxonomy normalisation is separate work. refs #610	2026-07-25 19:45:09 +02:00
timothy	8578dc1ca7	fix(610): de-brittle the count assertion; stop conflating index lines with preserved prose ... Both surfaced when main gained two records mid-flight. - test_record_count_is_the_expected_166 hardcoded the total, so it failed the moment a record landed on main -- a merge turning an unrelated test red. The real invariant is before == after; the count only needs to prove the harness isn't parsing a stub corpus, so it is now equality plus a floor. - The migration's "lines preserved" figure silently absorbed the generated where-did-it-go index once that was threaded into the preamble string, jumping 507 -> 759 with no new prose preserved. It now reports the two separately: 514 lines of original prose, plus 245 generated index lines. A number that quietly changes meaning is worse than no number.	2026-07-25 19:08:58 +02:00
timothy	64b65fd2db	feat(610): generated where-did-it-go index on each stripped file ... Live docs point into the corpus BY DATE -- "see `decisions.md` 2026-07-10" -- about 32 such references across 12 files, plus the same form in historical issue comments. The split would dangle every one of them. Each stripped file now ends with a generated "Records formerly in this file" index: date, title, and a link to the record's new path. A reader following a date pointer lands on the file it names and resolves from there. That is far cheaper and less error-prone than rewriting 32 references by hand, and it also covers the issue-comment breadcrumbs, which cannot be rewritten at all. Caught while verifying it: the generated `## Records formerly in this file` heading is itself an H2, so the record parser counted one legacy-unmigrated record per stripped file -- the notice went 0 -> 6. Same treatment as the existing `## Index` section: skip it by name. SKIP_HEADINGS moved to decisions_lib as the single source of truth, since three modules now need it. Found by reading the validator's notice output on a trial migration, not by inspection -- the corpus still validated OK, so nothing else would have flagged it.	2026-07-25 19:08:58 +02:00
timothy	d52268f2ed	feat(610): migration script + field-level equivalence harness ... The migration is a MOVE, so correctness is provable rather than reviewable: parse the corpus before, migrate, parse after, assert the Record sets are identical field-for-field with byte-identical rationale. scripts/migrate_decisions_split.py 166 records -> docs/decisions/records/<area>/<topic>.md (active) or docs/decisions/archive/<area>/<topic>.md (superseded/retired), 26 directories. Path is DERIVED from the key, so identity stays the key. Refuses to run if any record lacks a key, and aborts on a destination collision. Source files are KEPT, stripped to their narrative -- not deleted. 506 lines of the corpus sit outside any record: decisions.md's lifecycle-schema header (the status vocabulary, supersession rules, the edit-token contract) and each topic file's preamble explaining why those records exist. For most of it that is the only copy. It also cannot be filed per-area -- topic files hold several areas and 4 of 23 areas span several files -- so the files themselves stay. scripts/tests/test_migration_equivalence.py Runs the real migration against a COPY of the real corpus in tmp_path, never the working tree. Asserts: all 166 records survive with the same keys, every metadata field round-trips, titles round-trip from headings, every rationale body is byte-identical, path matches key, archived records land in the archive wing, the legacy files keep their narrative, and no parseable record is left behind in them. Proven non-vacuous: corrupting one migrated record's prose is caught by the byte-identical check, and deleting one is caught by the survival check. One test-authoring note: an early assertion string-matched "## " to prove no records were left in decisions.md. That is wrong -- the schema header quotes an illustrative "## 2026-07-17 ..." example in prose. Whether records remain is a PARSING question, so the parser-based leftover test is the real invariant.	2026-07-25 19:08:58 +02:00
timothyandClaude Opus 5	cc9481f541	fix(609): review fixes — exclude merge commits, unfold folded values, correct the fail-posture doc ... Cross-family review (High + Medium + Low), all three reproduced before fixing: High -- on a pull_request event actions/checkout lands on a synthetic merge commit whose body the forge composes from the PR description, so a description ending in an example marker armed a guard no author armed. Excluded merge commits from the range; merging main into a PR branch is separately forbidden, so no author-written commit is skipped. Medium -- a folded value (`no` + continuation ` yes`) was split into independent lines and the continuation armed on its own, inverting the value the author wrote. Read with `unfold` so the value is judged whole. Low -- the module docstring promised fail-open while marker resolution deliberately fails closed. The posture is right; the docstring was wrong. Documented as the one exception. Both new negative controls verified red against the unfixed matcher. Decision record amended to state the residual honestly rather than overclaim: a quoted example that is the FINAL paragraph of an ordinary commit is a trailer by git's own grammar and does arm. What the change buys is that discussing the marker can no longer disable anything. fixes #609 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 18:29:46 +02:00
timothyandClaude Opus 5	c597c49f02	fix(609): arm the decisions body-diff with a git trailer, not a bare substring ... The rationale-edit exemption was a substring test over the whole commit range, so any message containing the literal marker armed it -- including prose explaining why no marker was needed, which is how it fired live in PR #605: a green --base/--head run that was vacuous on the body-diff dimension, in the one PR that hand-resolved a merge conflict inside the corpus the guard exists to police. Now read as an affirmative `Decisions-Edit:` git trailer. Git parses trailers only in the final paragraph, so a quoted example message cannot arm it -- which matters because this commit and its decision record both quote one. A non-affirmative value (`no`) does not arm it either; the retired substring arms nothing and gets a ::warning:: nudge. Tests: negative controls (prose mention, quoted mid-body trailer, `no` value, retired substring) plus positive controls (trailer, uppercase, alongside Co-Authored-By). All four negative controls verified red against the old matcher. fixes #609 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-25 18:21:12 +02:00
timothy	75139bbf31	fix(603): close four defects found in adversarial review of the stale-after fields ... 1. `date.fromisoformat` is not a YYYY-MM-DD validator. On Python >= 3.11 it also accepts ISO basic format ("20270101") and week dates ("2027-W01-1"), so a malformed-looking value passed the blocking check — and which forms parse depends on the interpreter, meaning the same corpus could validate differently on a dev machine and on the runner (pr-checks.yml pins only python-version '3.x'). Knock-on: the catalog's Review-due section sorts on the raw STRING, so an accepted "20270101" sorted AFTER "2027-01-15" ('-' < '0'), contradicting the section's own "sorted soonest-first" text. Gate on ^\d{4}-\d{2}-\d{2}$ first, which fixes both — a fixed-width zero-padded form makes string sort == date sort. 2. A present-but-empty `stale-after:` was collapsed to None by `or None` in the parser and then skipped by a truthiness guard in the validator, so it passed as "absent" — a field that silently never fires, which is the exact failure mode the blocking check exists to prevent. Keep "" distinct from None and test with `is not None`. 3. `test_catalog_is_date_independent` was partly vacuous: with no date in either render, both sides were trivially equal after the .replace(). It did still catch an injected clock-derived marker, but it passed with the feature deleted. Assert the dates are present. 4. The malformed-date check ran only over the active set, exempting archive records. Staleness is moot there, but a typo is still a typo — check both wings. Adds regression tests for each, plus a Review-due row for a topic-file record (pinning the `../decisions.md` vs bare-filename link forms).	2026-07-25 15:22:50 +02:00
timothy	0ad02db651	feat(603): adopt OKF's optional stale-after and Sources decision-record metadata ... Evaluated the Open Knowledge Format (GoogleCloudPlatform/knowledge-catalog okf v0.2, scaccogatto/okf-skills) as a replacement for our decision-record system and rejected it: its conformance rules are deliberately permissive exactly where ours are strict (broken links, unknown types and missing fields must all be tolerated; `deprecated` points at no successor), and its stable identity is the file path, which the breadcrumb rule tells agents not to trust. Adopted two of its optional families instead, additively: - `stale-after: YYYY-MM-DD` on the metadata line — marks a record asserting an outside-world fact as due for re-confirmation. Absolute date, no TTL. - `**Sources:**` in the metadata block — the evidence a record rests on, as distinct from `Signals:` (recall keywords). Neither is required; absence is never an error. A malformed `stale-after` is blocking (it would silently never fire), but a past-due record is only a non-blocking `::notice::` — going stale is the passage of time, not a defect in whatever commit is under test. The catalog's new "Review due" section renders the date only and never a clock-derived verdict, so it cannot drift `--check` red on a calendar boundary with no commit touching the corpus. No backfill: no existing record adopts either field here. fixes #603	2026-07-25 15:22:50 +02:00
timothyandClaude Opus 4.8	0ef053a209	ci(545): require a **Signals:** line on decision records ... The decisions-guard validator enforced lifecycle metadata (key/status/since/ supersedes/superseded-by + reciprocal links) but not the **Signals:** line — which is exactly what MemPalace's keyword recall matches on. A record without it ingests with weak recall metadata and produces confident false-negatives for the "MemPalace to find, file to confirm" retrieval workflow. Add "signals" to REQUIRED_META so a migrated record with a missing or empty **Signals:** line fails the same way a missing key does. All 114 active + 2 archive records already carry a Signals line, so this is non-breaking on the current corpus. Archive records are intentionally out of scope (recall targets the active corpus). - scripts/decisions_validate.py: signals in REQUIRED_META (+ rationale comment) - scripts/tests/test_decisions_validate.py: _rec() default + missing/empty/present cases - docs/decisions.md: header + Enforcement note the requirement and why fixes #545 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 20:19:10 +02:00
timothyandClaude Opus 4.8	fb6720ea27	fix(521): de-dup 6 overlapped records; guard duplicate metadata blocks; exclude retrieval-eval; complete eval bank [decisions-edit] ... - Exclude docs/decisions/retrieval-eval.md from active decision parsing (_NON_DECISION_FILES); its `## N.` eval-question headings were being miscounted as 7 legacy-unmigrated records. - Add decisions_lib.metadata_line_count() + a decisions_validate guard that fails a record with more than one `key:` metadata line, so a stacked-metadata-block migration bug (which the parser silently tolerated by reading only the first block) can't recur unnoticed. TDD: test_duplicate_metadata_block_fails / test_single_metadata_block_passes. - De-duplicate the 6 docs/decisions.md records left with two stacked metadata blocks (scan.getoraddfolder-db-lookup #488, scan.musicvideo-reconciliation #494, scan.jellyfin-mixed-content-library #489, iptv.logo-drives-bug-preset #67, ffmpeg.qsv-decode-encode-split #498, ci.small-lane-git-only server-management#639), merging the union of Signals/paths/issues/Mechanics from both blocks and keeping the richer Rule wording; rationale prose untouched. - Fill in the deferred Q6b row in docs/decisions/retrieval-eval.md now that startup.parallel-orientation is active in docs/decisions.md, scoring it as a real active-vs-superseded question against the archived docs.queue-state-gitea-tracker. - Regenerate docs/decisions/README.md via build_decisions_catalog.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 11:54:52 +02:00
timothyandClaude Opus 4.8	ab2d80b47a	fix(521): reciprocity covers archive<->archive pairs; skip topic-file Contents heading ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 11:14:00 +02:00
timothyandClaude Opus 4.8	f93458c76c	fix(521): whole-branch review — correct Gitea anchors, guard archive+demotion, reciprocal links, stale append-only refs, budget warning [decisions-edit] ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 03:00:15 +02:00
timothyandClaude Opus 4.8	458ab2111f	fix(521): catalog _anchor collapses punctuation runs; single trailing newline ... _anchor() mapped each space/hyphen to its own '-' without collapsing runs, so the standard heading separator " — " (space, em-dash, space) produced a double hyphen in every generated anchor. Since nearly every real decision record heading uses that separator, this made the catalog emit a dead link for essentially every row. Fix: after building the char list, collapse consecutive '-' into one and strip leading/trailing '-' via re.sub, matching how Goldmark/GitHub/Gitea sluggers behave. Also fixed main() writing an extra trailing newline (want already ends in "\n", then "+ \n" appended a second one) so docs/decisions/README.md now ends with exactly one trailing newline; --check still compares via .strip(). Added test_anchor_collapses_em_dash_and_keeps_underscore to pin the anchor behavior against the reported iptv.base_url case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8	67619b2bf6	feat(521): active-catalog generator ... Adds scripts/build_decisions_catalog.py, which renders docs/decisions/README.md as a compact table of only 'active' decision records (sorted by key), and its test scripts/tests/test_build_catalog.py. Supports --check for CI drift detection. No decision records are migrated yet, so the generated catalog is currently empty (banner + header only) — expected at this stage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8	bdcc59ff80	test(521): add non-vacuous append-smuggle regression test for _rationale bound ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8	7b8ac751d7	fix(521): bound _rationale strip window; restore REQUIRED_META; fail-open _run; archive-placement invariant ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8	9266437d68	feat(521): decision lifecycle validator ... Adds scripts/decisions_validate.py: lifecycle invariant checks (unique active key, key format, reciprocal supersession, removed-without-archive, rationale-rewrite-without-token, catalog staleness, corpus budget) plus git-diff helpers for merge-base-based CI checks. Deviates from the task brief in one spot: REQUIRED_META narrowed to (key, status) — the brief's (key, status, since, supersedes, superseded_by) makes its own test_clean_corpus_passes fail, since since/supersedes/superseded_by default to None on bare Record() instances built without going through the markdown parser. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8	2cd78aa0bf	style(521): lint/type cleanup on decisions_lib parser + test ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8	ba39ca65ae	feat(521): decision-record parser (decisions_lib) ... Adds scripts/decisions_lib.py, the shared parser for ErsatzTV decision records (docs/decisions.md + docs/decisions/*.md). Parses H2 sections into Record dataclasses, distinguishing migrated records (visible metadata block: key/status/since/supersedes/superseded-by + Rule/ Signals/Mechanics) from legacy-unmigrated ones with no metadata line. scripts/ is now an importable package (scripts/__init__.py, scripts/tests/__init__.py) so later tools can `import scripts.decisions_lib`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 02:58:58 +02:00