10 Commits

Include renames

Author	SHA1	Message	Date
3 people	5d955000f3	fix(845): the verdict writer checks that the gate will honour what it just posted (#889) ... `review-verdict.yml` inherits an existing `review-verdict/h10=success` only from a status whose `.creator.login` is on its `H10_REVIEWERS` allow-list (#742). `post-review-verdict.sh` wrote those verdicts with whatever account owned the credential in the environment and never asked whose it was. Two coupled values, nothing asserting the coupling, and the failure was the silent kind: the status is written, the tool reports success, and the next `pull_request_target` event re-derives it and posts over it. The PR stalls with no visible cause. The writer now READS ITS OWN STATUS BACK, identifies that write by state and description, and refuses — before the verdict comment, so the surviving half-state is the documented `ask` one — unless the recorded creator is allow-listed. Measured after the write rather than probed before it: that tests what Gitea recorded as the author, which is the value the gate reads, and needs no scope beyond the repo access the POST already required. Membership is required for a `success` ONLY, mirroring the gate's own asymmetry: a `failure` is inherited from any attributable account, so requiring it there would refuse a verdict the gate honours and leave an off-list reviewer no supported way to record a rejection. The allow-list is DERIVED from the gate's own literal by the new `scripts/lib/h10-reviewers.sh` — one declaration, not two plus a parity test. It is a parse rather than a shared declaration both sides source because the gate runs against a checkout of the PR's BASE sha: a PR whose base predates such a file would not have it, and a missing `source` under `set -euo pipefail` kills the job, which posts no `review-verdict/h10` at all and blocks every merge including its own repair (#743). `scripts/post-review-verdict.sh` moves BEHAVIOUR-ONLY -> MUTATION in the guard inventory, which the manifest's own note called "the most valuable upgrade on this list". The declared clause lives in the GATE: rewriting `H10_REVIEWERS` while the posting account stays fixed reddens the accept path only if the writer reads the list live AND the comparison gates the outcome. Two defects were caught by probing the live instance rather than re-reading the code. Reading `.state` instead of `.status` per row would have refused EVERY verdict — a repo-wide deadlock, shipped green, because the test shim replayed the POST payload as the read-back body and so agreed with the parser by construction. Then a `(.status // .state)` fallback added as defensiveness recreated #845 exactly: the writer would accept a shape the gate cannot read and report success. Nine independent cold review rounds, all worktree-isolated, one cross-family (GPT-5.6 via Codex). Round 8 caught the most important one: a `set -u` "correction" made mid-branch had inverted a TRUE statement in live merge-gate code, because the probe used a plain `$UNSET` while the validator uses `${#arr[@]}` — different shapes, different behaviour. Withdrawn wholesale; both libraries are byte-identical to `main` again. Verification: full `scripts/tests` suite green (1278 passed, 2 skipped); the declared mutation executes every run and reddens its named proof with the manifest's `expect` string; every clause disarmed individually and confirmed to redden its own named test; live probes against Gitea 1.27.1 for the row shape, the description round-trip, the paging order and the required-check list. Docs: `ci.exemption-provenance` records the coupling as asserted rather than as a tracked residual, plus `docs/ci-cd.md`, `CLAUDE.md`, `docs/guard-inventory.md`, `docs/remote-state-inventory.md`, `ci.script-tests-job` and the `script-tests` population comment in `pr-checks.yml`. Deferred: the refused-verdict residual (a non-inheritable status left standing with no comment) is the `ask` half-state `release.verdict-writes-status-before-comment` designates as safe; a second corrective write is the sticky-sentinel mechanism #849 is separately designing. fixes #845 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>	2026-08-30 02:43:08 +00:00
timothyandtimothy	d4c72697f2	feat(780): commit a ruff config and enforce it in CI (#813) ... Python lint here was a property of the operator's laptop: the global instructions say to run ruff, no workflow ran it, and with no committed config ruff fell back to whichever ~/.config/ruff/ruff.toml the machine happened to have. - ruff.toml at the root, pinned ruff==0.12.11 in the script-tests job. - Both lint steps pass an EXPLICIT population from `git ls-files` with `--no-force-exclude`, never `ruff check .` — an `exclude` empties a discovery-based run into a GREEN one (top level empties both commands, [lint] empties check, [format] empties format --check), and `ruff check .` over zero files exits 0 with only a stderr warning. Guarded by an empty-population arm. - Tree clean: 74 findings at 706674272, 57 fixed in code, 17 per-site noqa with reasons inline. S105 deliberately per-site, not a directory blanket. RUF100 selected so a suppression that suppresses nothing is itself a finding. - pyright stays ungated; reasoning in the record. Both steps witnessed red on the runner against the shipped bodies: run 2173 job 9176 (ruff check) and run 2170 job 9163 (ruff format --check). Docs: new record ci.python-lint-ruff-config-committed, ci.script-tests-job cross-ref, docs/ci-cd.md (also correcting a stale ~190-tests/~10s figure to the measured 773 tests / ~4.5 min), docs/defect-shapes-773.md §5.2 resolved. fixes #780 Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>	2026-08-22 00:33:18 +00:00
timothy	63fa81fbb5	docs(648,649): the jq contract + the shared PR-file enumeration record ... Adds docs/ci-cd.md "The jq contract" (1.6 floor, the three divergent constructs, and the deliberate pin-vs-floor asymmetry with its merge-deadlock reason), plus two decision records: ci.jq-version-contract and ci.shared-pr-file-enumeration. ci.script-tests-job stops restating the three jq rules and points at the new record. Also corrects the script-tests preflight description: it is now two steps (git presence, then jq VERSION via scripts/jq-preflight.sh --expect 1.6), not one. A literal NUL byte had crept into the ci-cd.md paragraph describing jq 1.6's NUL truncation — which git treats as a binary file. Replaced with the literal text. Decisions-Edit: yes	2026-07-26 22:21:07 +02:00
timothyandClaude Opus 5	bb1809fbf0	fix(647): the jq-1.6 fail-open was ALSO in review-verdict.yml — the enforced gate ... Round-4 review caught the miss that matters most: #643 fixed the `jq -e`-on-empty fail-open in .claude/hooks/pretooluse-merge-consent.sh, but the SAME construct sits in .gitea/workflows/review-verdict.yml — and that is the copy that runs on the CI runner, where jq is 1.6, and that feeds the branch-protection-required review-verdict/h10 status. Reproduced: `printf "" | jq -e '.statuses | type == "array"'` exits 4 on jq 1.8.2 (guard fires, correct) and 0 on jq 1.6 (guard passes). So on a transient API error `statusjson` is empty, the guard lets it through, `existing` reads "", and the job posts `pending` — or for a bot/docs-only PR an exemption `success` — over a possibly-existing human verdict. That is precisely what the comment three lines above it says must never happen. The hook version was harmless in practice because it runs on a dev Mac with jq 1.8. This one is live. Fixed identically, with a comment naming why the sibling fix missed it, and the same hardening applied to the changed-files read in the same workflow. Also from round 4: - LOW, reproduced — an ARRAY-valued `.status` dodged the closed allow-list. `index` is polymorphic: with an array argument it does SUBSEQUENCE matching, so `[...,"renamed",...] | index(["renamed"])` is truthy while `.status == "renamed"` is false — the row passed the allow-list AND skipped the `previous_filename REQUIRED` clause. The same `git mv code -> docs/` dodge the closed set exists to block, one type away. Now requires `.status` to be a string first; mutation-verified. - The record now carries all THREE jq rules rather than the one, and notes that the durable fix is pinning/preflighting the runner's jq version rather than patching constructs one at a time (tracked on #647). 200 tests pass under BOTH jq 1.8.2 and jq 1.6. Refs #647, #643, #631 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-26 13:24:20 +02:00
timothyandClaude Opus 5	f4473926d4	fix(643): close the status-string dodge, pin the allow-list anchors, fix two doc claims ... Round-3 review (MERGEABLE, all findings Low/Nit) — applied anyway, because each is the over-claim or unpinned-anchor family this PR keeps hitting. LOW — the `renamed => previous_filename REQUIRED` clause matched `.status` by exact lowercase string, so any other value took the `else true` branch: a row with `"Renamed"`, or with no status at all, validated fine and silently dropped its SOURCE path, letting `git mv ErsatzTV/Program.cs -> docs/a.md` read as docs-only. `.status` is now checked against a closed set and an unknown value fails closed. Two things that fix taught me, both caught by my own positive control rather than by review: 1. The first predicate was WRONG in a way that gated everything: inside `[...] | index(.status)`, jq's `.` is the ARRAY, so `.status` was null and every row failed. `$s` is now bound from the row before the context switches. A security check that rejects everything looks identical to a working one from the failing side — only test_gitea_real_status_values_are_accepted caught it. 2. The set includes BOTH `changed` and `modified`. Live Gitea 1.25.4 emits `changed`, but a closed allow-list built from the wrong vocabulary is worse than the hole it closes: it would gate every genuine docs-only PR. The property wanted is "reject what we don't recognise", not "enumerate one version exactly". LOW — three allow-list anchors had no test at all: dropping `^` from the `docs/` alternative (`ErsatzTV/docs/Program.cs` would exempt), dropping `$` from `.md` (`x.md.cs` would exempt), and dropping the non-empty-list guard. Since the round-3 `..` finding WAS an anchor subversion, they are now pinned; all three mutation-verified. NIT — docs/ci-cd.md called this job "a checkout plus a pure-stdlib pytest run", which the same file contradicts 450 lines later and which this PR's own record names as the bug that turned the job red on its first CI run. Also replaced a frozen "111 tests" with an explicitly indicative figure — the suite is ~190 after rebasing onto main, and a frozen count has rotted four times this session. 198 passed under both jq 1.8.2 and jq 1.6. Refs #643, #631 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-26 13:24:20 +02:00
timothy	54c875414c	fix(643): validate every field the extraction consumes; fix a vacuous test ... Re-review of the previous fix commit found it incomplete and its test vacuous. Both reproduced before fixing. MEDIUM — the validation domain did not match the CONSUMPTION domain. `chunk` emits `(.previous_filename // empty)` for EVERY row regardless of status, but the guard validated that field only when `.status == "renamed"`. So a row marked `modified` — or Gitea's distinct `copied` — carrying a newline in previous_filename was still exempted. Verified EXEMPT for both statuses against the previous commit. Now validated whenever present, with the `renamed` => REQUIRED clause kept on top. MEDIUM — test_newline_in_previous_filename_is_also_rejected asserted the right outcome without exercising the mechanism: its payload's second segment was `ErsatzTV/Program.cs`, which the allow-list rejects on its own, so the test passed with the newline guard entirely removed. That is why the hole above went unnoticed — the same filter-hides-the-defect trap the guard itself is about. Payload changed to a segment that PASSES the allow-list, so the test now discriminates, plus parametrized regressions for modified/copied/added. LOW — `..` components rejected. The allow-list anchors `^docs/`, so `docs/../ErsatzTV/Program.cs` matched it (reproduced). Git will not produce such a path, but this guard exists to fail closed on unexpected 2xx shapes. Added a positive control (a legitimate docs->docs rename still exempts) so the tightened row schema cannot be satisfied by never exempting anything. Severity calibrated in the record: the docs-only exemption ends in `decide allow ""`, a passthrough to the normal permission prompt, NOT an auto-grant. Every bypass here downgrades a mechanical deny/ask to a human prompt; none can silently self-merge. Real, worth fixing, but not what an earlier framing of #643 implied. All mutation-verified; 123 passed under BOTH jq 1.8.2 and jq 1.6. Refs #643, #631	2026-07-26 13:24:20 +02:00
timothy	c046add10a	fix(643): close two more fail-opens in the docs-only enumeration, found by cold review ... An independent cross-family review of the jq-1.6 fix found two further ways the docs-only exemption can fire over an incomplete file list — both reachable with NO transport error, so neither had anything to do with the original bug. 1. HIGH — a path containing a newline. `chunk` flattens paths into newline-delimited text before the allow-list grep, so a filename of "safe.md\ndocs/Program.cs" splits into two lines that BOTH match the allow-list, while the real single path ends in .cs. Git permits newlines in filenames and the reviewer reproduced the bypass against this hook. Now rejected outright at the row-schema guard, on both `filename` and `previous_filename`: no docs path contains a control character, so failing closed costs nothing. 2. HIGH — a short page read as the last page. `n < 50` assumed the server's page size is the 50 we requested, but Gitea caps `limit` at the server-wide MAX_RESPONSE_ITEMS (default 50, configurable) and may return fewer. A 30-row docs page followed by a page of code completed the enumeration over a PARTIAL list. Only a validated EMPTY page may now terminate it; the page<=40 cap still fails closed, and the cost is one extra request. 3. MEDIUM — the enumeration was not bound to one head. Paging is several round-trips, so a force-push between them assembles a list belonging to no single commit: page 1 from head A plus a short docs tail from head B, with B's code page never read. The head sha is re-read after enumeration and the exemption refused if it moved. All three mutation-verified: reverting each fix reddens exactly its own test and nothing else. A positive control (short page then empty page) pins that the stricter terminator still exempts a genuinely docs-only PR, so "never terminate early" cannot be satisfied by never exempting anything. 118 passed under BOTH jq 1.8.2 and jq 1.6. The record now states the generalisable lesson: every defect here was an exhaustiveness failure in an enumeration whose completeness is load-bearing. When a security decision depends on having seen ALL of something, the termination condition must be positive and explicit, never inferred from a proxy. Refs #643, #631	2026-07-26 13:24:20 +02:00
timothy	5f068a2488	fix(631): close a fail-open in the merge-consent gate that only CI could see ... Running scripts/tests in CI for the first time turned up a real defect in the gate itself, not just in the harness. `jq -e` over EMPTY input exits 4 on jq >= 1.7 but 0 on jq 1.6 — verified against both binaries, not inferred. The docs-only pagination guard in pretooluse-merge-consent.sh leaned on that exit status to reject a transport failure. On jq 1.6, which the CI runner ships: page 2 errors -> gq returns empty -> jq guard wrongly PASSES -> n is empty so [ "$n" -lt 50 ] errors into false -> the loop walks PAST the failed page -> page 3 legitimately returns [] -> files_complete=yes over a PARTIAL list -> the docs-only exemption fires over unread pages that may be pure code. That is the very defect the guard's own comment describes, reintroduced one layer down by a jq version difference. Fixed by rejecting an empty body explicitly rather than inferring it from jq's exit status. The same hardening is applied to the review-verdict status read, which fell through to `vstate=""` -> deny (fail-CLOSED, so never a hole) but would have surfaced the wrong message. Why it survived: the existing transport-failure test asserts the right thing but can only observe the bug where jq is 1.6, so it passes on a developer Mac with the bug fully present — and the suite had never run anywhere else. The new test removes that dependency by shimming ONLY jq 1.6's empty-input exit status, so it pins the property on any host. Mutation-verified: revert the fix and it goes red on jq 1.8.2. The shim is deliberately narrow (no `-n`): a broader first version swallowed the `jq -n` calls `decide` uses to build its JSON, so the hook emitted nothing and every decision read as a passthrough — the verifier manufacturing the exemption it was meant to disprove. test_jq16_shim_actually_reproduces_the_quirk now pins the shim itself, `-n` case included. Suite: 113 passed under jq 1.8.2 AND under jq 1.6. Refs #631	2026-07-26 13:24:20 +02:00
timothy	73577f484f	ci(631): install pyyaml too — the suite is not pure-stdlib, and CI proved it ... First CI run went red on `ModuleNotFoundError: No module named 'yaml'` at collection: test_migration_equivalence.py imports scripts/migrate_decisions_split.py, which uses PyYAML by design. It passed locally only because this machine already had PyYAML. The original claim came from reading decisions_lib/decisions_validate/build_catalog and the test files themselves — which is precisely the set that does NOT import yaml. Replaced with an AST import scan over all of scripts/, which gives the complete third-party set: pytest and yaml. (etv_client is imported only by scripts/scripted-schedules/entrypoint.py, a container entrypoint pytest never collects.) This does not contradict the dependency-free decisions READ path: decisions_lib._read_frontmatter is hand-written exactly so validation runs where nothing is installed. The one-shot WRITE path is allowed PyYAML, and its test needs it. Verified the way it should have been the first time: a clean venv with pytest alone reproduces CI's collection error; adding pyyaml gives 111 passed. Comments, ci-cd.md and the decision record all corrected to drop the "pure stdlib" claim. Refs #631	2026-07-26 13:24:19 +02:00
timothy	69d8d3ccfe	ci(631): run scripts/tests in CI as its own job, not inside the flake-covered decisions-guard ... Nothing executed scripts/tests/. No workflow and no Husky hook invoked pytest: decisions-guard runs decisions_validate.py and build_decisions_catalog.py directly (the code, never its tests) and the `test` job is dotnet test only. The 111 tests guarding the decision corpus, the #610 migration-equivalence harness, the merge-consent exemption logic and the #622 review-verdict poster were enforced only by whoever remembered to run pytest locally. Adds a `script-tests` job to pr-checks.yml rather than the step inside decisions-guard the issue proposed. decisions-guard is covered by ci.decisions-lifecycle-flake, a standing instruction that a lone `decisions lifecycle` red is a known flake and must not be investigated — so a genuine pytest regression there would surface as exactly the red every session is told to wave through. A gate inherits the credibility of the job it lives in. Runs unconditionally: the suite executes the real post-review-verdict.sh and pretooluse-merge-consent.sh, so a `scripts/**` path filter would miss a .claude/hooks/** edit. ~10s, pure stdlib + pytest. Negative control verified locally (broken test -> exit 1); CI-red measurement follows on a scratch PR per the Done-when. Refs #631	2026-07-26 13:24:19 +02:00