From bbd919bfe396bf9c10d06abd89f817fb83f8c425 Mon Sep 17 00:00:00 2001 From: Timothy Date: Sat, 5 Sep 2026 00:28:37 +0200 Subject: [PATCH] fix(885): a refusal carrying NO challenge never ran the token leg, so it stops claiming one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `probe` enters `acquire_token` on a `401` only. A registry answering `403` on the first read — or a `401` with no `Www-Authenticate` — therefore leaves `token_leg_done=0` and `token=""`, the guard at the `401|403` arm is false, and the run fell through to the message that says the read was refused "even after a Bearer token was obtained". Probed 2026-09-05 with a curl shim answering `403` and dumping only `HTTP/1.1 403 Forbidden`: that message is printed, EXIT=1, and no token was ever requested. The fail direction was safe; the diagnosis was not. It sends an operator to package visibility on evidence that does not exist (`dont-narrate-mechanisms-you-didnt-measure`) — in a script whose whole design is that its refusal messages are worded apart on purpose. The arm now branches on what actually ran, `token` first so the never-asked case cannot borrow either other mechanism: * `token` non-empty -> refused after a GOOD bearer (an answer about the PACKAGE) * token leg attempted -> challenged but produced no token (about the TOKEN ENDPOINT) * neither -> refused with no challenge at all (about ACCESS) The pre-existing `403` test could not reach this: `CURL_SHIM` answered `401` + a challenge to every unauthenticated read regardless of the configured code, so the `403` parameter was only ever observable AFTER the token leg. The shim grew a challenge-less behaviour (`CHALLENGE=none`, `REFUSAL=403|401`) rather than the assertion being written against the old one, and both codes are driven because they take different paths — the challenge-less `401` still enters and abandons the token leg. Witnessed red on the predecessor script (2 failed) and green on the fix. `docs/ci-cd.md`'s "Cutting a release" runbook — the section an operator reads at cut time — gains the `v*` tag protection, the account it whitelists, the fact that its positive half is unverified, and the `DELETE .../tag_protections/1` unblock. The tag-protection note already in this file sits inside the `main`-direct-push discussion, which is not where a release cut is driven from, and `release.tag-protection-v-star` names its own failure mode as a cut that will not push. `ci.pr-route-carries-no-stored-credential` records that `docs/remote-state-inventory.md`'s row for the preflight still lists "an unusable credential" among the shapes that fail the job, which this issue deleted. That file is held by a concurrent change, so the one-clause edit is tracked as #909 rather than made here. Refs #885, #909 Decisions-Edit: yes Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_015QqCpYFsKgnAnx6jVwrKiV --- docs/ci-cd.md | 12 ++++ scripts/ci-toolchain-image-resolves.sh | 32 +++++++---- .../tests/test_ci_toolchain_image_resolves.py | 57 ++++++++++++++++++- 3 files changed, 90 insertions(+), 11 deletions(-) diff --git a/docs/ci-cd.md b/docs/ci-cd.md index 6f96d8af0..8510bde96 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -101,6 +101,18 @@ commit in the range (see the `decisions.md` header) — routine lifecycle metada 1. Confirm `main` CI is green; run the full local gate plus `dotnet list package --vulnerable --include-transitive`; then push a `vYY.N.P` tag on that exact `main` commit. + + > **`v*` tags are protected — push as `timothy`.** Since ersatztv#885 the repo carries one + > tag-protection rule (`name_pattern: v*`, `whitelist_usernames: ["timothy"]`, + > `release.tag-protection-v-star`), because a `v*` push builds and publishes `:prod`. A push from + > any other account is refused. **The positive half of that rule is UNVERIFIED**: it was applied + > 2026-09-04 and only its negative half was measured (a non-`v*` tag still pushes), because + > pushing a `v*` tag outside a real cut would publish `:prod`. This cut is its first real + > exercise. If the tag push is refused, unblock with + > `DELETE /api/v1/repos/timothy/ersatztv/tag_protections/1`, push, then re-`POST` the rule + > (prefer a `PATCH` adding the account if the cut has moved to a different operator, so the + > protection is never left off) — and record the outcome on `release.tag-protection-v-star`, + > which is waiting for it. 2. Wait for tag CI to build `:prod` + the immutable `:` + `:` images. Run `scripts/security-scan.sh` on jazz against **the immutable `:` image**, not a moving tag, and triage every ZAP/semgrep finding. diff --git a/scripts/ci-toolchain-image-resolves.sh b/scripts/ci-toolchain-image-resolves.sh index d4ab3e80f..95abe988a 100755 --- a/scripts/ci-toolchain-image-resolves.sh +++ b/scripts/ci-toolchain-image-resolves.sh @@ -191,22 +191,34 @@ for pin in $pins; do # `fail` rather than `rc=1`: unlike a 404, this says nothing about the pin, and it will say # the same thing about every remaining one. Abandoning the loop keeps the log to one cause. # - # Reached only after the token leg has been ATTEMPTED, and the two ways of getting here send - # an operator to different places, so they get different messages — the same reason `404` and - # `could NOT VERIFY` are worded apart. A challenge that yielded no usable token is an - # infrastructure answer about the TOKEN ENDPOINT; a refusal that survived a good token is an - # answer about this PACKAGE, and since ersatztv#885 it is not a preflight-only problem: every - # `container:` job pulls the same image with no credential, so they fail at image pull too, - # including both required contexts. + # THREE ways to get here, and they send an operator to three different places, so they are + # worded apart — the same reason `404` and `could NOT VERIFY` are. Each message states only + # what actually ran, because a message naming a step that did not happen is evidence for a + # diagnosis nobody performed: + # + # * a refusal that survived a GOOD token is an answer about this PACKAGE, and since + # ersatztv#885 it is not a preflight-only problem: every `container:` job pulls the same + # image with no credential, so they fail at image pull too, including both required + # contexts; + # * a challenge that yielded no usable token is an infrastructure answer about the TOKEN + # ENDPOINT; + # * a refusal carrying NO Bearer challenge at all never reached the token leg — `probe` + # enters it on a 401 only, so a first-read 403 (or a 401 with no `Www-Authenticate`, which + # the token leg then abandons) leaves `token` empty having asked for nothing. This is an + # answer about ACCESS to the registry, and the branch order below is `token` first + # precisely so this case cannot borrow either of the other two mechanisms. # # A failed token leg is NOT retried, deliberately: `token_leg_done` is set before the attempt # so a registry genuinely refusing anonymous reads is asked once rather than once per pin. The # cost is that a transient token-endpoint outage fails the job on its first try; the message # below says so rather than blaming the package. - if [ "$token_leg_done" -eq 1 ] && [ -z "$token" ]; then - fail "could NOT OBTAIN an anonymous pull token for $registry/$image_repo:$pin — the registry challenged (HTTP $code) but the token leg produced none: either the challenge named no realm, or the token endpoint did not answer with a token. The pin was NOT checked. Look at the registry's token endpoint, not at the pin." + if [ -n "$token" ]; then + fail "the registry refused an ANONYMOUS read (HTTP $code) of $registry/$image_repo:$pin even after a Bearer token was obtained, so the pin could not be checked. Every container: job pulls this image without a credential too, so they will fail at image pull. Check that timothy/ersatztv and its ersatztv-ci package are still PUBLIC — do not read this as a pass." fi - fail "the registry refused an ANONYMOUS read (HTTP $code) of $registry/$image_repo:$pin even after a Bearer token was obtained, so the pin could not be checked. Every container: job pulls this image without a credential too, so they will fail at image pull. Check that timothy/ersatztv and its ersatztv-ci package are still PUBLIC — do not read this as a pass." + if [ "$token_leg_done" -eq 1 ]; then + fail "could NOT OBTAIN an anonymous pull token for $registry/$image_repo:$pin — the registry answered HTTP $code and the token leg produced none: either there was no Www-Authenticate challenge, or it named no realm, or the token endpoint did not answer with a token. The pin was NOT checked. Look at the registry's token endpoint, not at the pin." + fi + fail "the registry refused an ANONYMOUS read (HTTP $code) of $registry/$image_repo:$pin WITHOUT issuing a Bearer challenge, so no token was ever requested and the pin could not be checked. A challenge is what tells a client where a token can be had; an outright refusal is an answer about ACCESS to the registry, not about the pin. Every container: job pulls this image without a credential too, so they will fail at image pull. Check that timothy/ersatztv and its ersatztv-ci package are still PUBLIC, and that nothing (a proxy, an ACL) is answering for the registry — do not read this as a pass." ;; *) # NOT gone, and NOT a pass either. Deliberately worded apart from the 404 message: this sends diff --git a/scripts/tests/test_ci_toolchain_image_resolves.py b/scripts/tests/test_ci_toolchain_image_resolves.py index 99334b0a6..9a117eaa6 100644 --- a/scripts/tests/test_ci_toolchain_image_resolves.py +++ b/scripts/tests/test_ci_toolchain_image_resolves.py @@ -85,6 +85,17 @@ code = codes.get(tag, codes.get("*", "200")) if code == "TRANSPORT": sys.exit(7) +if authorization != "Bearer anon-token" and codes.get("CHALLENGE") == "none": + # A registry that REFUSES an unauthenticated read outright instead of challenging: no + # Www-Authenticate header at all, so a client has nowhere to ask for a token. `REFUSAL` picks + # the code, because a 403 and a challenge-less 401 take DIFFERENT paths through the script -- + # `probe` enters the token leg on 401 only. + refusal = codes.get("REFUSAL", "403") + if dump: + pathlib.Path(dump).write_text("HTTP/1.1 %s Refused\r\n" % refusal) + print("{}\n%s" % refusal, end="") + sys.exit(0) + if authorization != "Bearer anon-token": # 401 for a live tag and a deleted one alike -- the challenge is the ONLY thing that tells a # client where a token can be had. @@ -271,8 +282,52 @@ def test_a_401_or_403_AFTER_the_token_leg_REFUSES_rather_than_passing(preflight, result = preflight.run() assert result.returncode != 0 assert "refused an ANONYMOUS read" in result.stderr + assert "even after a Bearer token was obtained" in result.stderr, ( + "this arm is the one where a token really was obtained, so it is the only one allowed to " + f"say so. stderr={result.stderr}" + ) assert "PUBLIC" in result.stderr, "the message must name the cause an operator can act on" - assert preflight.token_calls(), "the token leg must have been attempted before refusing" + assert preflight.authenticated_manifest_calls(), ( + "a message claiming the read survived a Bearer token must be reached with a read that " + "CARRIED one — `token_calls` only shows the token was asked for" + ) + + +@pytest.mark.parametrize("refusal", ["403", "401"]) +def test_a_refusal_with_NO_CHALLENGE_never_claims_a_token_was_obtained(preflight, refusal): + """The message must not name a mechanism the run did not perform. + + `probe` enters the token leg on a `401` only, so a registry answering `403` on the first read — + or a `401` carrying no `Www-Authenticate` — leaves the script with no token having asked for + nothing. Measured 2026-09-05 on the predecessor at 59003d5a3, the `403` shape reported "even + after a Bearer token was obtained", which sends an operator to package visibility on evidence + that does not exist (`dont-narrate-mechanisms-you-didnt-measure`). The two codes are BOTH driven + because they take different paths: the challenge-less `401` still enters and abandons the token + leg, the `403` never enters it. + + The shim answering 401 + a challenge to every unauthenticated read is why the pre-existing + `403` case could not reach this — it could only ever be observed AFTER the token leg — so the + shim grew a challenge-less behaviour rather than the assertion being written against the old one. + """ + preflight.set_codes({"*": "200", "CHALLENGE": "none", "REFUSAL": refusal}) + result = preflight.run() + assert result.returncode != 0, "an unreadable registry is not a pass" + assert "even after a Bearer token was obtained" not in result.stderr, ( + f"no token was obtained on this path. stderr={result.stderr}" + ) + assert preflight.authenticated_manifest_calls() == [], "no read can have carried a token here" + if refusal == "403": + assert preflight.token_calls() == [], "a 403 first read must not even ask for a token" + assert "WITHOUT issuing a Bearer challenge" in result.stderr, result.stderr + assert "PUBLIC" in result.stderr, "the message must still name a cause an operator can act on" + else: + # A challenge-less 401 DOES enter the token leg (and abandons it for want of a realm), so it + # is the token-endpoint diagnosis rather than the never-asked one. + assert preflight.token_calls() == [], "there was no realm to request a token from" + assert "could NOT OBTAIN an anonymous pull token" in result.stderr, result.stderr + assert "no Www-Authenticate challenge" in result.stderr, ( + f"the token-leg message must admit the challenge was missing. stderr={result.stderr}" + ) def test_the_TOKEN_LEG_actually_runs_and_the_bearer_REACHES_the_registry(preflight):