test(649): make the POST-wiring assertion unable to opt out or accept the wrong host
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
PR Gates / Script tests (pytest) (pull_request) Successful in 36s
Review verdict / Set review-verdict status (pull_request) Successful in 40s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
review-verdict/h10 Review-verdict: MERGEABLE @ e960d5b
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m28s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 23m3s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 24m2s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped

Re-review found the verifier could disable itself two ways, both of which look like
coverage:

- it was guarded by `if url_file.exists()`, so deleting the recorder in the stub turned
  the whole assertion into a no-op and every test stayed green;
- it compared only the URL SUFFIX, so a POST to the right path on the wrong HOST or the
  wrong REPO passed — which is exactly the class the assertion was added to catch.

It now requires the URL to have been recorded whenever a status was posted, and compares
the full URL against the env the job was given. Mutation-verified three ways: wrong host,
wrong repo, and deleting the recorder each redden the suite.

Refs #649
This commit is contained in:
2026-07-26 23:38:25 +02:00
parent 3885fd6aea
commit e960d5b918
+16 -7
View File
@@ -607,15 +607,24 @@ def _run_classify(tmp_path, enum_stub: str | None, author: str = "timothy"):
script.write_text(_classify_step()["run"])
r = subprocess.run(["bash", str(script)], cwd=tmp_path, env=env,
capture_output=True, text=True)
# Assert the WIRING, not only the classification. The stub accepts every POST, so a status aimed
# at the wrong endpoint, sha, host or repo would otherwise leave these tests green while the real
# required check was never written.
#
# Two ways this check could disable itself, both found by cold review of an earlier draft:
# * it was guarded by `if url_file.exists()`, so deleting the recorder in the stub turned it
# into a no-op and every test stayed green — a verifier that silently opts out;
# * it compared only the URL SUFFIX, so a POST to the right path on the WRONG HOST OR REPO
# passed. Compare the whole URL against the env this job was given.
posted = tmp_path / "posted.json"
url_file = tmp_path / "posted_url.txt"
if url_file.exists():
# Assert the WIRING, not only the classification. Cold review's point: the stub accepts every
# POST, so a status aimed at the wrong endpoint or the wrong sha would leave these tests green
# while the real required check was never written.
url = url_file.read_text().strip()
assert url.endswith(f"/statuses/{SHA}"), (
f"the status was POSTed to {url!r}, not to /statuses/<full head sha>")
if posted.exists():
assert url_file.exists(), (
"a status was POSTed but its URL was not recorded — the wiring assertion below would "
"have silently skipped")
expected = f"{env['BASE_URL']}/repos/{env['REPO']}/statuses/{SHA}"
assert url_file.read_text().strip() == expected, (
f"the status was POSTed to {url_file.read_text().strip()!r}, expected {expected!r}")
return (json.loads(posted.read_text()) if posted.exists() else None), r