DO NOT MERGE — probe(631): negative control, proves script-tests can go red #638

2026-07-26T00:40:32+02:00

timothy commented

2026-07-26 00:40:32 +02:00

Throwaway. Do not merge. Will be closed and the branch deleted once measured.

This is the negative control for #631 / PR #637. Its Done-when requires the new enforcement be
"verified by measurement, not a green tick — deliberately break one test on a scratch branch and
confirm CI goes red".

Contents = PR #637's branch + one deliberately failing test appended to
scripts/tests/test_decisions_lib.py.

Expected result: the PR Gates / Script tests (pytest) job FAILS. Every other job passes.
If script-tests came back green here, the job would be decorative — exactly the defect #631 exists
to close — and #637 would not be mergeable.

Kept off #637's own branch so that PR's history contains no knowingly-broken commit.

**Throwaway. Do not merge. Will be closed and the branch deleted once measured.** This is the negative control for #631 / PR #637. Its Done-when requires the new enforcement be *"verified by measurement, not a green tick — deliberately break one test on a scratch branch and confirm CI goes red"*. Contents = PR #637's branch + one deliberately failing test appended to `scripts/tests/test_decisions_lib.py`. **Expected result:** the `PR Gates / Script tests (pytest)` job **FAILS**. Every other job passes. If `script-tests` came back green here, the job would be decorative — exactly the defect #631 exists to close — and #637 would not be mergeable. Kept off #637's own branch so that PR's history contains no knowingly-broken commit.

timothy added 2 commits 2026-07-26 00:40:32 +02:00

ci(631): run scripts/tests in CI as its own job, not inside the flake-covered decisions-guard fedec84997

Nothing executed scripts/tests/. No workflow and no Husky hook invoked pytest:
decisions-guard runs decisions_validate.py and build_decisions_catalog.py directly
(the code, never its tests) and the `test` job is dotnet test only. The 111 tests
guarding the decision corpus, the #610 migration-equivalence harness, the
merge-consent exemption logic and the #622 review-verdict poster were enforced only
by whoever remembered to run pytest locally.

Adds a `script-tests` job to pr-checks.yml rather than the step inside decisions-guard
the issue proposed. decisions-guard is covered by ci.decisions-lifecycle-flake, a
standing instruction that a lone `decisions lifecycle` red is a known flake and must
not be investigated — so a genuine pytest regression there would surface as exactly
the red every session is told to wave through. A gate inherits the credibility of the
job it lives in.

Runs unconditionally: the suite executes the real post-review-verdict.sh and
pretooluse-merge-consent.sh, so a `scripts/**` path filter would miss a
.claude/hooks/** edit. ~10s, pure stdlib + pytest.

Negative control verified locally (broken test -> exit 1); CI-red measurement follows
on a scratch PR per the Done-when.

Refs #631

ci(631): preflight jq/git in script-tests, and reflow the header comment

PR Gates / CI image pin matches docker/ci (pull_request) Successful in 10s

Details

PR Gates / Docs update reminder (pull_request) Successful in 12s

Details

review-verdict/h10 Awaiting review verdict for 8d3fbe2

Details

PR Gates / Script tests (pytest) (pull_request) Failing after 15s

Details

Review verdict / Set review-verdict status (pull_request) Successful in 4s

Details

PR Gates / decisions lifecycle (pull_request) Successful in 23s

Details

Build ErsatzTV Image / Build & test (.NET) (pull_request) Failing after 4m13s

Details

Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m42s

Details

Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 13s

Details

Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s

Details

Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 17m23s

Details

Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped

Details

8d3fbe215b

test_post_review_verdict.py and test_merge_consent_exemption.py exec the real
post-review-verdict.sh and pretooluse-merge-consent.sh, which shell out to jq ~26
times. Those tests shim `curl` on PATH but NOT `jq`, and no existing small-lane job
proves jq is present on that image — so a missing jq would land as ~20 opaque
assertion failures rather than a diagnosis.

Checks rather than installs: #390 deliberately removed run-time apt-get from CI, so
the correct fix for a genuine miss is baking the tool into the runner image, and the
error message says so.

Refs #631

timothy force-pushed probe/631-negative-control from b56dd0953b to dbdaaf2148

2026-07-26 00:44:34 +02:00

Compare

timothy force-pushed probe/631-negative-control from dbdaaf2148 to 52a8f16224

2026-07-26 01:00:41 +02:00

Compare

timothy referenced this pull request

2026-07-26 01:11:42 +02:00

`scripts/tests/` is never executed by CI — the Python test suite is local-only #631

timothy closed this pull request

2026-07-26 01:11:46 +02:00

timothy referenced this pull request