test(649): make the POST-wiring assertion unable to opt out or accept the wrong host
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
PR Gates / Script tests (pytest) (pull_request) Successful in 36s
Review verdict / Set review-verdict status (pull_request) Successful in 40s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
review-verdict/h10 Review-verdict: MERGEABLE @ e960d5b
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m28s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 23m3s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 24m2s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
PR Gates / Script tests (pytest) (pull_request) Successful in 36s
Review verdict / Set review-verdict status (pull_request) Successful in 40s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
review-verdict/h10 Review-verdict: MERGEABLE @ e960d5b
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m28s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 23m3s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 24m2s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Re-review found the verifier could disable itself two ways, both of which look like coverage: - it was guarded by `if url_file.exists()`, so deleting the recorder in the stub turned the whole assertion into a no-op and every test stayed green; - it compared only the URL SUFFIX, so a POST to the right path on the wrong HOST or the wrong REPO passed — which is exactly the class the assertion was added to catch. It now requires the URL to have been recorded whenever a status was posted, and compares the full URL against the env the job was given. Mutation-verified three ways: wrong host, wrong repo, and deleting the recorder each redden the suite. Refs #649
This commit is contained in:
@@ -607,15 +607,24 @@ def _run_classify(tmp_path, enum_stub: str | None, author: str = "timothy"):
|
||||
script.write_text(_classify_step()["run"])
|
||||
r = subprocess.run(["bash", str(script)], cwd=tmp_path, env=env,
|
||||
capture_output=True, text=True)
|
||||
# Assert the WIRING, not only the classification. The stub accepts every POST, so a status aimed
|
||||
# at the wrong endpoint, sha, host or repo would otherwise leave these tests green while the real
|
||||
# required check was never written.
|
||||
#
|
||||
# Two ways this check could disable itself, both found by cold review of an earlier draft:
|
||||
# * it was guarded by `if url_file.exists()`, so deleting the recorder in the stub turned it
|
||||
# into a no-op and every test stayed green — a verifier that silently opts out;
|
||||
# * it compared only the URL SUFFIX, so a POST to the right path on the WRONG HOST OR REPO
|
||||
# passed. Compare the whole URL against the env this job was given.
|
||||
posted = tmp_path / "posted.json"
|
||||
url_file = tmp_path / "posted_url.txt"
|
||||
if url_file.exists():
|
||||
# Assert the WIRING, not only the classification. Cold review's point: the stub accepts every
|
||||
# POST, so a status aimed at the wrong endpoint or the wrong sha would leave these tests green
|
||||
# while the real required check was never written.
|
||||
url = url_file.read_text().strip()
|
||||
assert url.endswith(f"/statuses/{SHA}"), (
|
||||
f"the status was POSTed to {url!r}, not to /statuses/<full head sha>")
|
||||
if posted.exists():
|
||||
assert url_file.exists(), (
|
||||
"a status was POSTed but its URL was not recorded — the wiring assertion below would "
|
||||
"have silently skipped")
|
||||
expected = f"{env['BASE_URL']}/repos/{env['REPO']}/statuses/{SHA}"
|
||||
assert url_file.read_text().strip() == expected, (
|
||||
f"the status was POSTed to {url_file.read_text().strip()!r}, expected {expected!r}")
|
||||
return (json.loads(posted.read_text()) if posted.exists() else None), r
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user