Files
ersatztv/scripts/tests/test_check_required_contexts.py
T
timothyandtimothy cf5f42edf9
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 10m49s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m41s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 7m18s
Build ErsatzTV Image / Build & push image (amd64) (push) Failing after 1m26s
fix(858,859): a rule the classifier cannot read is not a rule that matches nothing (#897)
#859 was filed as a wrong STATED CAUSE. It was masking a live false-open in the merge gate.

Gitea reports a GLOB branch-protection rule with an EMPTY `branch_name` — the canonical
name lives only in `rule_name`. Measured 2026-08-30 on a scratch repo against 1.27.1.
jq's `//` fires on null and false but NOT on `""`, so `(.branch_name // .rule_name // "")`
resolved every glob rule to the empty string — a name with no metacharacters — and the
glob test, the entire basis of the classifier's undecidable-first ordering, never saw it.

Measured on the predecessor: glob `m*` (not requiring review-verdict/h10) beside plain
`main` (requiring it) resolved to `exact` on `main` and AUTO-GRANTED a scheduled merge,
while Gitea — ordering by Priority then plain-name-ness — may be applying `m*`. That is
#622's hole, reached through the ordering written to close it. Mirror case: a glob alone
resolved to `none` and DENIED about a rule that provably governs the base.

A name is now a non-empty string. Each field resolves to a NAME, a SKIP (absent/null/
empty — fall through), or POISON (present, wrong type — poisons whichever field carries
it). A rule with no usable name is a distinct `unreadable` verdict with its own operator
cause, instead of feeding `none`, whose whole authority is "the full rule list was read
and none matches". The short-circuit is STRUCTURAL: jq binds `as` eagerly, so the flat
form still evaluated `offs`/`nonascii` on the bad name and died before reaching the arm
meant to prevent that.

Also #859: `branch_protections` is fetched ONCE per run, not twice. The round trip is the
smaller half — it is mutable config, so two reads can disagree and the two arms then
decide about different repo states with neither able to notice.

#858: `verdict_script` resolves from `$repo_root`, not `$CLAUDE_PROJECT_DIR`. And the
finding that mattered more — `ETV_HOOK_FIRE_LIB` is `. `-SOURCED, so it is CODE running
before stdin is read and before `decide` exists. A first draft exempted it as "telemetry,
not a predicate"; cold review refuted that by execution: a decoy hook-fire-log.sh in an
env-var-named tree printing an allow and exiting 0 GRANTS THE MERGE, bypassing every
check. Classify a path by how it is CONSUMED, never by what it is called. This hook's copy
is self-located; the other twelve are #891 (high/security), which records the reachable
case — husky launches the prepush hooks by RELATIVE path, so the two roots diverge there.

check-required-contexts.sh gains an array-type gate (a JSON object previously printed
`nomatch`, a positive claim about server config from a body it cannot consume).

Verification: 1377 passed / 2 skipped; 11 declared mutants, 11 detected, disjoint
reddened sets; classifier executed across jq 1.8.2 and 1.6 with identical results; both
env-var tests ship a negative control, because the passing outcome is also what an inert
decoy produces.

Four cold review rounds plus a bounded prose check. Every round found defects the
previous round's fixes introduced — a type conflation that re-opened the auto-grant, a
comment asserting the opposite of the line its own commit changed, and a corrected
sentence whose identical twin survived in the same diff.

Docs: new record `process.hook-resolves-inputs-from-repo-root`; both inline sites cite it
rather than arguing it twice. docs/remote-state-inventory.md's row for the second read
updated. Follow-ups filed: #891 (the other 12 hooks), #895 ("all N tests green" claims).

fixes #858
fixes #859

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-30 11:33:50 +00:00

556 lines
28 KiB
Python

"""Proof for `scripts/check-required-contexts.sh` (ersatztv#787).
The script reconciles the LIVE required status checks on `main` against the committed snapshot
`.gitea/required-status-contexts.json`, from which `test_ci_dropped_step_guard.py` derives its
marked-job scope. It is a pure classifier over a payload its caller fetched, which is the whole
reason it is a script at all rather than a few lines inside `pretooluse-merge-consent.sh`: the
merge hook's own comments record that while its verdict classification lived inline it had no tests
and three false-opens survived in it (#629).
Every test here execs the REAL script. Nothing reimplements its logic — a second copy of the
classifier in the test would agree with the first by construction and prove nothing.
"""
from __future__ import annotations
import json
import shutil
import subprocess
import tempfile
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[2]
SCRIPT = REPO_ROOT / "scripts" / "check-required-contexts.sh"
SNAPSHOT = REPO_ROOT / ".gitea" / "required-status-contexts.json"
# THE CONTRACT, pinned. `pretooluse-merge-consent.sh` maps each of these onto a hook decision and
# treats anything else as "could not tell" -> ask. Pinning the set here means a sixth class cannot be
# added to the script without this test demanding the hook learn about it: an unmapped class would
# otherwise reach the hook's catch-all and read as a malfunction rather than the finding it is.
CLASSES = {"match", "drift", "nomatch", "undecidable", "unreadable"}
SNAP_CONTEXTS = json.loads(SNAPSHOT.read_text())["contexts"]
def run(payload: str, snapshot: Path | None = None, branch: str = "main") -> tuple[int, str, str]:
proc = subprocess.run(
[str(SCRIPT), "--branch", branch, "--snapshot", str(snapshot or SNAPSHOT)],
input=payload,
capture_output=True,
text=True,
check=False,
)
return proc.returncode, proc.stdout.strip(), proc.stderr
def rule(contexts, name="main", enabled=True):
return {"branch_name": name, "enable_status_check": enabled, "status_check_contexts": contexts}
def test_the_script_is_executable():
assert SCRIPT.is_file() and SCRIPT.stat().st_mode & 0o111, f"{SCRIPT} must exist and be executable"
def test_the_committed_snapshot_is_well_formed_and_not_empty():
"""ANTI-VACUITY. An empty snapshot would make `match` reachable against an empty live list while
`test_ci_dropped_step_guard.py` derived an empty scope and collected zero parametrized cases."""
data = json.loads(SNAPSHOT.read_text())
assert isinstance(data.get("contexts"), list) and data["contexts"]
assert all(isinstance(c, str) and c.strip() for c in data["contexts"])
assert data.get("branch") == "main"
assert data.get("read_on"), "the snapshot must date itself — it is a claim about a moment"
def test_live_equal_to_the_snapshot_is_match():
assert run(json.dumps([rule(list(SNAP_CONTEXTS))]))[1] == "match"
def test_the_comparison_is_by_SET_not_by_order():
"""Gitea does not promise an order, so an order-sensitive compare would report drift forever —
and a guard that cries wolf on every merge is one somebody switches off."""
assert run(json.dumps([rule(list(reversed(SNAP_CONTEXTS)))]))[1] == "match"
def test_an_ADDED_required_context_is_drift():
"""THE FINDING #787 EXISTS FOR: a fourth required check whose job carries no execution marker."""
code, out, _ = run(json.dumps([rule([*SNAP_CONTEXTS, "Build ErsatzTV Image / Fourth (pull_request)"])]))
assert (code, out) == (0, "drift")
def test_a_REMOVED_required_context_is_drift():
assert run(json.dumps([rule(list(SNAP_CONTEXTS[:-1]))]))[1] == "drift"
def test_null_contexts_against_a_non_empty_snapshot_is_drift():
"""`null` is a legitimate payload meaning "none required", not a read failure — so it is the
FINDING that every required context was removed, never `unreadable`."""
payload = json.dumps([{"branch_name": "main", "enable_status_check": True, "status_check_contexts": None}])
assert run(payload)[1] == "drift"
def test_a_rule_with_status_checks_DISABLED_is_drift_against_a_nonempty_snapshot():
"""The EFFECTIVE required set, not the raw list.
`enable_status_check: false` means Gitea requires nothing on the branch whatever
`status_check_contexts` still holds. Type-checking the flag and ignoring its VALUE certifies
disabled protection as a current mirror — measured: with that clause removed the same payload
returns `match`, and the ENTIRE suite across this file and the merge-consent file stayed green,
because `rule()`'s `enabled=` parameter had no call site that ever passed False. (Stated as an
invariant: this sentence carried a test COUNT until ersatztv#859, and a count is falsified by
the next person to add a test — which is exactly what that PR did to it.)
"""
assert run(json.dumps([rule(list(SNAP_CONTEXTS), enabled=False)]))[1] == "drift"
def test_a_rule_with_status_checks_DISABLED_matches_an_EMPTY_snapshot(tmp_path):
"""The other side of the same clause: disabled protection IS an accurate mirror of a snapshot
that requires nothing, so it must not be reported as drift."""
empty = tmp_path / "snap.json"
empty.write_text(json.dumps({"repo": "timothy/ersatztv", "branch": "main", "contexts": []}))
assert run(json.dumps([rule(list(SNAP_CONTEXTS), enabled=False)]), snapshot=empty)[1] == "match"
def test_no_rule_governing_the_branch_is_nomatch():
assert run(json.dumps([rule([], name="develop")]))[1] == "nomatch"
def test_a_glob_rule_makes_it_undecidable_even_when_an_exact_rule_matches():
"""ORDER MATTERS, and this is the test that pins it. Gitea picks the governing rule by Priority
with gobwas/glob semantics, so a glob rule can outrank an exactly-named one. Preferring the exact
rule would compare against a rule Gitea may not be applying and could report `match` on a branch
whose real required set has drifted — a false-open in the direction that costs something."""
payload = json.dumps([rule([], name="m*"), rule(list(SNAP_CONTEXTS))])
assert run(payload)[1] == "undecidable"
def test_two_rules_folding_to_the_same_name_are_undecidable():
assert run(json.dumps([rule(list(SNAP_CONTEXTS)), rule(list(SNAP_CONTEXTS), name="MAIN")]))[1] == "undecidable"
@pytest.mark.parametrize(
("label", "payload"),
[
("not json", "not json at all"),
("top level object", '{"branch_name": "main"}'),
(
"enable_status_check as a string",
json.dumps([{"branch_name": "main", "enable_status_check": "true", "status_check_contexts": []}]),
),
("a non-string context member", json.dumps([rule(["ok", 123])])),
("contexts as an object", json.dumps([rule({"a": 1})])),
],
)
def test_shapes_this_decision_cannot_consume_are_unreadable(label, payload):
"""Validated to the depth it is CONSUMED at, not just the top-level type. An object-typed check
that never inspects its members is the one-level-down swallow the merge hook fixed twice."""
assert run(payload)[1] == "unreadable", label
def test_every_class_the_script_emits_is_one_the_contract_declares():
"""Every class the contract DECLARES is reachable, and these five payloads produce no other.
Deliberately not claimed: that a sixth class cannot ship. This drives five canned inputs, so a
class emitted only on a sixth input shape would leave `seen` unchanged and pass — the hook's
catch-all arm is what keeps that fail-closed, not this test. What it does catch is a declared
class going unreachable (dead contract) or one of these five silently changing.
"""
seen = {
run(json.dumps([rule(list(SNAP_CONTEXTS))]))[1],
run(json.dumps([rule([*SNAP_CONTEXTS, "x (pull_request)"])]))[1],
run(json.dumps([rule([], name="develop")]))[1],
run(json.dumps([rule([], name="m*")]))[1],
run("not json")[1],
}
assert seen == CLASSES, f"classes reached {sorted(seen)} != contract {sorted(CLASSES)}"
def test_a_usage_error_exits_2_and_prints_NO_class():
"""A broken invocation must not be mistakable for a finding. If it printed a class AND exited
non-zero, a caller reading only stdout would act on a comparison that never happened."""
proc = subprocess.run([str(SCRIPT), "--bogus"], input="[]", capture_output=True, text=True, check=False)
assert proc.returncode == 2
assert proc.stdout.strip() == ""
assert not (CLASSES & set(proc.stdout.split()))
def test_an_unreadable_snapshot_is_a_usage_error_not_a_class():
proc = subprocess.run(
[str(SCRIPT), "--snapshot", str(REPO_ROOT / "does" / "not" / "exist.json")],
input="[]",
capture_output=True,
text=True,
check=False,
)
assert proc.returncode == 2 and proc.stdout.strip() == ""
def test_a_snapshot_naming_NO_branch_is_a_usage_error(tmp_path):
"""The snapshot must name the branch it mirrors. Treating an absent `branch` as "compare against
whatever was asked for" reaches the same defect as a mismatched one — one branch's live contexts
weighed against another's mirror — by omission rather than by disagreement."""
bad = tmp_path / "snap.json"
bad.write_text(json.dumps({"contexts": list(SNAP_CONTEXTS)}))
code, out, err = run(json.dumps([rule(list(SNAP_CONTEXTS))]), snapshot=bad)
assert (code, out) == (2, ""), "an unbranded snapshot must be a usage error, never a class"
# THE ARM-UNIQUE WORDING. Exit 2 alone cannot see this clause: with `.branch` absent
# `snap_branch` is empty, and `--branch` can never be empty, so the MISMATCH guard on the next
# line fires anyway with the same exit code and the same empty stdout. Two guards masking each
# other (#685) — measured: deleting the required-branch clause left this file at 27 passed.
assert "names no branch" in err, (
f"the required-branch clause did not fire; the mismatch guard answered instead: {err.strip()[:160]}"
)
def test_a_snapshot_for_a_DIFFERENT_branch_is_a_usage_error():
code, out, _ = run(json.dumps([rule(list(SNAP_CONTEXTS))]), branch="develop")
assert (code, out) == (2, ""), "a branch mismatch must be a usage error, never a class"
def test_a_malformed_snapshot_is_unreadable_not_a_confident_drift(tmp_path):
"""The snapshot is validated to the same depth as the live payload. A `contexts` holding a
non-string would otherwise compare unequal against a well-formed live list and be reported as
`drift` — a confident finding derived from data that was never understood."""
bad = tmp_path / "snap.json"
bad.write_text(json.dumps({"contexts": ["ok", 7]}))
assert run(json.dumps([rule(list(SNAP_CONTEXTS))]), snapshot=bad)[1] == "unreadable"
# The clause this guard hangs on: the set comparison that turns a differing live list into `drift`.
COMPARE_CLAUSE = "($rule.status_check_contexts | sort | unique) == $snap"
def test_a_glob_that_could_NOT_govern_the_branch_does_not_block_the_comparison():
"""Rule selection is delegated to `scripts/lib/branch-rule-classifier.jq`, the SAME program
`pretooluse-merge-consent.sh` loads. A rule named `a[b` carries a metacharacter but cannot govern
`main`, so it must not make the comparison undecidable — a blanket "any metacharacter anywhere"
test would ask forever in a repo that has one, which is how a correct guard gets switched off."""
payload = json.dumps([rule([], name="a[b"), rule(list(SNAP_CONTEXTS))])
assert run(payload)[1] == "match"
CLASSIFIER = REPO_ROOT / "scripts" / "lib" / "branch-rule-classifier.jq"
MERGE_HOOK = REPO_ROOT / ".claude" / "hooks" / "pretooluse-merge-consent.sh"
SENTINEL_CLASSIFIER = (
"# sentinel: a classifier that answers only one way, so a caller that loads THIS file changes\n"
'{verdict: "undecidable"}\n'
)
def test_ONE_classifier_TWO_callers_BEHAVIOURALLY():
"""The structural property #787 relies on, PROVEN by swapping the file — not by reading source.
A string-presence check was the seventh test in this change to pass for the wrong reason: cold
review re-inlined a BYTE-IDENTICAL copy of the classifier into the merge hook, left a comment
naming `scripts/lib/branch-rule-classifier.jq` above it, and the whole suite stayed green at
exactly 1062 passed. A comment satisfies a substring assertion. And the byte-identical inline is
precisely the drift-creating refactor that matters, because it AGREES on day one — a weakened
copy would have been caught behaviourally.
So the proof replaces the shared file with a sentinel that can only answer `undecidable`, and
requires the caller's behaviour to change. Nothing but actually loading that path can do that.
"""
assert CLASSIFIER.is_file(), f"the shared classifier is missing at {CLASSIFIER}"
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
victim = root / "check-required-contexts.sh"
victim.write_text(SCRIPT.read_text(encoding="utf-8"))
victim.chmod(0o755)
(root / "lib").mkdir()
(root / "lib" / "branch-rule-classifier.jq").write_text(SENTINEL_CLASSIFIER)
payload = json.dumps([rule(list(SNAP_CONTEXTS))])
proc = subprocess.run(
[str(victim), "--branch", "main", "--snapshot", str(SNAPSHOT)],
input=payload,
capture_output=True,
text=True,
check=False,
)
assert (proc.returncode, proc.stdout.strip()) == (0, "undecidable"), (
"swapping scripts/lib/branch-rule-classifier.jq did not change this script's verdict "
f"(exit={proc.returncode} stdout={proc.stdout.strip()!r}) — it is not loading that file, so "
"the one-copy property is a claim rather than a fact"
)
def _mutant_tree(tmp_path, source_text):
"""The script in a RUNNABLE layout: it resolves `lib/branch-rule-classifier.jq` relative to its
own directory, so a bare copy exits 2 before reaching any of its logic.
This is not incidental plumbing. The first version of the mutation proof below copied only the
script, so the mutant died on `classifier not readable` with EMPTY stdout — and an assertion that
stdout merely differed from `drift` was satisfied by that empty string. The proof passed for the
wrong reason and the `MUTATION` grade it justified was unsupported.
"""
victim = tmp_path / "mutant.sh"
victim.write_text(source_text)
victim.chmod(0o755)
shutil.copytree(REPO_ROOT / "scripts" / "lib", tmp_path / "lib")
return victim
def test_the_mutation_harness_layout_ITSELF_reproduces_the_real_verdict(tmp_path):
"""POSITIVE CONTROL for the harness below, without which it can only prove a broken copy.
The UNMUTATED script, in the same tmp layout, must reach the same verdict it reaches in the
repo. If this fails, the mutation test's red says nothing about the clause.
"""
victim = _mutant_tree(tmp_path, SCRIPT.read_text(encoding="utf-8"))
drifted = json.dumps([rule([*SNAP_CONTEXTS, "Build ErsatzTV Image / Fourth (pull_request)"])])
proc = subprocess.run(
[str(victim), "--branch", "main", "--snapshot", str(SNAPSHOT)],
input=drifted,
capture_output=True,
text=True,
check=False,
)
assert (proc.returncode, proc.stdout.strip()) == (0, "drift"), (
f"the unmutated script did not report drift in the harness layout "
f"(exit={proc.returncode} stdout={proc.stdout.strip()!r} stderr={proc.stderr.strip()[:160]!r}) — "
"the mutation proof below would then be measuring a broken copy, not a disarmed clause"
)
def test_a_MISSING_shared_classifier_is_a_usage_error_not_an_unreadable_CLASS():
"""The twin of a fix the hook made loudly, on the second caller.
Without this guard `jq -f` on a missing program fails, `verdict` is empty, and the script prints
the CLASS `unreadable` with exit 0 — reporting "the payload came back in a shape I could not
consume" about a missing LOCAL FILE. That is the states-a-cause-that-did-not-happen defect the
hook added its own `[ -r "$classifier" ]` check to avoid; measured, removing this one left the
file at 27 passed.
"""
with tempfile.TemporaryDirectory() as tmp:
victim = Path(tmp) / "check-required-contexts.sh"
victim.write_text(SCRIPT.read_text(encoding="utf-8"))
victim.chmod(0o755)
# deliberately NO lib/ beside it
proc = subprocess.run(
[str(victim), "--branch", "main", "--snapshot", str(SNAPSHOT)],
input=json.dumps([rule(list(SNAP_CONTEXTS))]),
capture_output=True,
text=True,
check=False,
)
assert proc.returncode == 2, f"a missing classifier must be a usage error, got exit {proc.returncode}"
assert proc.stdout.strip() == "", (
f"a missing local file was reported as the class {proc.stdout.strip()!r} — a cause that did not happen"
)
assert "classifier not readable" in proc.stderr
def test_MUTATION_disarming_the_set_comparison_stops_every_drift_report(tmp_path):
"""`testing.guard-ships-with-mutation-proof` (#775) — the DECLARED clause, re-run every suite.
The set comparison is the guard's entire finding. Disarm that clause alone — make it constantly
true — and a live list that genuinely differs from the snapshot must stop reporting `drift`. If
it still did, the finding comes from somewhere other than the clause this guard hangs on and
every passing test above proves nothing about it.
"""
text = SCRIPT.read_text(encoding="utf-8")
assert COMPARE_CLAUSE in text, (
"the set-comparison clause has moved or been reworded; RETARGET this mutation at its new "
"location rather than loosening the string match — a mutation that silently stops mutating "
"is the exact failure this file exists to catch"
)
mutated = text.replace(COMPARE_CLAUSE, "true", 1)
assert mutated != text and COMPARE_CLAUSE not in mutated, "the replacement did not change the source"
victim = _mutant_tree(tmp_path, mutated)
drifted = json.dumps([rule([*SNAP_CONTEXTS, "Build ErsatzTV Image / Fourth (pull_request)"])])
proc = subprocess.run(
[str(victim), "--branch", "main", "--snapshot", str(SNAPSHOT)],
input=drifted,
capture_output=True,
text=True,
check=False,
)
# EXACT, not `!= "drift"`. The mutant must RUN and reach the disarmed clause, so the only
# acceptable evidence is the specific wrong answer it now gives. Any exit code but 0, or an
# empty stdout, means it died on the way and proves nothing.
assert (proc.returncode, proc.stdout.strip()) == (0, "match"), (
f"the mutant did not reach the disarmed clause and report `match` "
f"(exit={proc.returncode} stdout={proc.stdout.strip()!r} stderr={proc.stderr.strip()[:160]!r}). "
"With the set comparison constantly true it must certify a drifted list as current; "
"anything else means the proof measured a broken copy rather than a disarmed guard"
)
# --- ersatztv#859: a rule with no usable NAME is `unreadable`, not `nomatch` ---------------------
@pytest.mark.parametrize(
"bad_rule",
[
{"enable_status_check": True, "status_check_contexts": SNAP_CONTEXTS},
{"branch_name": None, "enable_status_check": True, "status_check_contexts": SNAP_CONTEXTS},
{"branch_name": False, "enable_status_check": True, "status_check_contexts": SNAP_CONTEXTS},
{"branch_name": 7, "enable_status_check": True, "status_check_contexts": SNAP_CONTEXTS},
{"branch_name": "", "rule_name": "", "enable_status_check": True, "status_check_contexts": SNAP_CONTEXTS},
],
ids=["absent", "null", "false", "numeric", "both-empty"],
)
def test_a_rule_with_an_UNUSABLE_NAME_is_unreadable_not_nomatch(bad_rule):
"""`(.branch_name // .rule_name // "")` collapsed all of these to the empty string — a VALID
name that matches nothing — so the shared classifier answered `none` and this script reported
`nomatch`: "no rule governs the branch at all (protection removed)".
That is a finding about a list the program could not read. `nomatch` is this script's only class
that asserts something POSITIVE about the server's configuration, so it is the one that must not
be reachable from an unread rule. On this caller the wrong answer was merely safe — the hook asks
on `nomatch` — while the sibling caller DENIED on it; the classifier is shared, so both are fixed
by the same change and both are pinned by their own suite.
`numeric` is here for a second reason: it used to make jq THROW, so this script reached
`unreadable` through its catch-all rather than by classifying. Same output, no longer resting on
a crash.
"""
code, out, err = run(json.dumps([bad_rule]))
assert (code, out) == (0, "unreadable"), (
f"a rule with no usable name classified as {out!r} (exit {code}, stderr {err.strip()[:160]!r}) "
"— `nomatch` asserts protection was REMOVED, which is a claim about a list that was never read"
)
def test_an_EMPTY_branch_name_falls_through_to_rule_name():
"""jq's `//` fires on null and false but NOT on `""`, so an empty `branch_name` SHADOWED a
perfectly good `rule_name` and the fully-protected branch reported `nomatch`.
REACHABLE, and measured: Gitea 1.27.1 reports a GLOB rule as `{"branch_name":"","rule_name":
"release/*"}` — verified 2026-08-30 by creating one rule of each kind on a scratch repo and
reading the list back. An earlier probe against this repo saw only its single plain rule and
wrongly concluded both fields are always populated; the positive case had to be constructed.
So every glob rule reached this classifier as a name with no metacharacters, and `nomatch` — a
positive claim that protection was REMOVED — was returned for branches a glob provably governs.
"""
payload = json.dumps(
[{"branch_name": "", "rule_name": "main", "enable_status_check": True, "status_check_contexts": SNAP_CONTEXTS}]
)
code, out, err = run(payload)
assert (code, out) == (0, "match"), (
f"an empty `branch_name` still hid the real rule name: {out!r} (exit {code}, stderr {err.strip()[:160]!r})"
)
def test_a_readable_rule_beside_an_unreadable_one_still_poisons_the_batch():
"""The whole LIST is the input, so one unreadable member is enough. A rule the program cannot
read might be the rule Gitea applies — Priority ordering is the server's, not this script's — so
classifying the readable one and ignoring the other would answer confidently about a list that
was only partly understood. Conservative direction: the caller asks instead of comparing."""
payload = json.dumps([rule(SNAP_CONTEXTS), {"branch_name": 7, "enable_status_check": True}])
code, out, _ = run(payload)
assert (code, out) == (0, "unreadable"), (
f"a readable rule beside an unreadable one produced {out!r} — the batch was classified on partial information"
)
CLASSIFIER = REPO_ROOT / "scripts" / "lib" / "branch-rule-classifier.jq"
def classify(payload: str, branch: str = "main") -> tuple[int, str]:
"""The classifier ALONE, with no script wrapping it. `check-required-contexts.sh` maps both a
declared `unreadable` verdict and a jq CRASH onto the same output word, so a test that only reads
that word cannot tell the two apart — cold review demonstrated exactly that by mutating the
classifier's structural guard to `if false`, making jq exit 5, and watching the wrapper still
print `unreadable`. Asserting the raw verdict is what pins the mechanism."""
proc = subprocess.run(
["jq", "--arg", "b", branch, "-c", "-f", str(CLASSIFIER)],
input=payload,
capture_output=True,
text=True,
check=False,
)
return proc.returncode, proc.stdout.strip()
@pytest.mark.parametrize(
"bad_rule",
[
{"enable_status_check": True},
{"branch_name": None},
{"branch_name": False},
{"branch_name": 7},
{"branch_name": "", "rule_name": ""},
],
ids=["absent", "null", "false", "numeric", "both-empty"],
)
def test_the_CLASSIFIER_ITSELF_returns_the_declared_unreadable_verdict(bad_rule):
"""Exit 0 AND the verdict word, not merely the wrapper's output. A crash exits 5 with empty
stdout; a classification exits 0 with `{"verdict":"unreadable"}`. Only the second is the
contract the two callers were taught to consume."""
code, out = classify(json.dumps([bad_rule]))
assert (code, out) == (0, '{"verdict":"unreadable"}'), (
f"the classifier did not CLASSIFY an unusable name — it produced exit={code} out={out!r}. "
"An exit 5 with empty stdout is a crash the callers absorb, not the declared verdict."
)
def test_the_classifier_still_reaches_its_OTHER_verdicts():
"""Anti-vacuity. Every assertion above is satisfied by a classifier that answers `unreadable` to
everything, which would be a catastrophic permanent `ask`. These pin that the other three arms
are still reachable, so the new one is an addition rather than a swallow."""
assert classify(json.dumps([rule(SNAP_CONTEXTS)]))[1].startswith('{"verdict":"exact"'), "exact is unreachable"
assert classify(json.dumps([rule(SNAP_CONTEXTS, name="m*")])) == (0, '{"verdict":"undecidable"}'), (
"undecidable is unreachable"
)
assert classify(json.dumps([rule(SNAP_CONTEXTS, name="develop")])) == (0, '{"verdict":"none"}'), (
"none is unreachable"
)
@pytest.mark.parametrize(
"payload",
["{}", '{"a":{"branch_name":"main","enable_status_check":true,"status_check_contexts":[]}}', '"main"', "42"],
ids=["empty-object", "object-of-rules", "string", "number"],
)
def test_a_NON_ARRAY_payload_is_unreadable_not_a_finding(payload):
"""`nomatch` is this script's only class that asserts something POSITIVE about the server —
"no rule governs the branch at all (protection removed)". It must not be reachable from a body
that is not the shape this script consumes.
Measured before the gate: `{}` reached the classifier, matched no rule, and printed `nomatch`.
The merge hook has had this gate since #778; this script did not, so the same payload produced a
confident finding here and a correct refusal there. Pre-existing rather than introduced by
ersatztv#859, and fixed with it because the whole issue is arms asserting findings they never
established.
"""
code, out, _ = run(payload)
assert (code, out) == (0, "unreadable"), (
f"a non-array payload classified as {out!r} — anything but `unreadable` is a claim about "
"branch protection derived from a body that was never understood"
)
def test_an_EMPTY_array_is_still_a_real_finding():
"""NEGATIVE CONTROL for the gate above. `[]` IS the shape this script consumes and it genuinely
establishes absence — the list was read and holds no rule. A gate that swallowed it too would
turn the one payload that legitimately proves "protection removed" into a shrug, and every test
above would still pass."""
code, out, _ = run("[]")
assert (code, out) == (0, "nomatch"), (
f"the array gate swallowed a legitimately empty rule list: {out!r} — `[]` is the only "
"payload that establishes absence, and it must stay a finding"
)
def test_a_MALFORMED_rule_name_poisons_even_beside_a_GOOD_branch_name():
"""Poison is field-agnostic, which the first draft of the fix got wrong in one direction.
It short-circuited on `branch_name`, so `{"branch_name":42,"rule_name":"main"}` poisoned while
`{"branch_name":"main","rule_name":42}` answered `exact` off the good field and never looked at
the malformed one — an invariant the comment claimed and the code did not hold. Gitea 1.27.1 does
not send that second shape, so this pins a property rather than a bug: "a malformed field poisons
the list" has to be true of BOTH fields or it is not the rule that is written down.
"""
for bad in (42, [], {}, True):
payload = json.dumps([{"branch_name": "main", "rule_name": bad, "enable_status_check": True}])
code, out = classify(payload)
assert (code, out) == (0, '{"verdict":"unreadable"}'), (
f"a malformed `rule_name` ({bad!r}) beside a usable `branch_name` produced {out!r} "
"— poison must not depend on which field carries it"
)