diff --git a/docs/ci-cd.md b/docs/ci-cd.md index 6f96d8af0..8510bde96 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -101,6 +101,18 @@ commit in the range (see the `decisions.md` header) — routine lifecycle metada 1. Confirm `main` CI is green; run the full local gate plus `dotnet list package --vulnerable --include-transitive`; then push a `vYY.N.P` tag on that exact `main` commit. + + > **`v*` tags are protected — push as `timothy`.** Since ersatztv#885 the repo carries one + > tag-protection rule (`name_pattern: v*`, `whitelist_usernames: ["timothy"]`, + > `release.tag-protection-v-star`), because a `v*` push builds and publishes `:prod`. A push from + > any other account is refused. **The positive half of that rule is UNVERIFIED**: it was applied + > 2026-09-04 and only its negative half was measured (a non-`v*` tag still pushes), because + > pushing a `v*` tag outside a real cut would publish `:prod`. This cut is its first real + > exercise. If the tag push is refused, unblock with + > `DELETE /api/v1/repos/timothy/ersatztv/tag_protections/1`, push, then re-`POST` the rule + > (prefer a `PATCH` adding the account if the cut has moved to a different operator, so the + > protection is never left off) — and record the outcome on `release.tag-protection-v-star`, + > which is waiting for it. 2. Wait for tag CI to build `:prod` + the immutable `:` + `:` images. Run `scripts/security-scan.sh` on jazz against **the immutable `:` image**, not a moving tag, and triage every ZAP/semgrep finding. diff --git a/scripts/ci-toolchain-image-resolves.sh b/scripts/ci-toolchain-image-resolves.sh index d4ab3e80f..95abe988a 100755 --- a/scripts/ci-toolchain-image-resolves.sh +++ b/scripts/ci-toolchain-image-resolves.sh @@ -191,22 +191,34 @@ for pin in $pins; do # `fail` rather than `rc=1`: unlike a 404, this says nothing about the pin, and it will say # the same thing about every remaining one. Abandoning the loop keeps the log to one cause. # - # Reached only after the token leg has been ATTEMPTED, and the two ways of getting here send - # an operator to different places, so they get different messages — the same reason `404` and - # `could NOT VERIFY` are worded apart. A challenge that yielded no usable token is an - # infrastructure answer about the TOKEN ENDPOINT; a refusal that survived a good token is an - # answer about this PACKAGE, and since ersatztv#885 it is not a preflight-only problem: every - # `container:` job pulls the same image with no credential, so they fail at image pull too, - # including both required contexts. + # THREE ways to get here, and they send an operator to three different places, so they are + # worded apart — the same reason `404` and `could NOT VERIFY` are. Each message states only + # what actually ran, because a message naming a step that did not happen is evidence for a + # diagnosis nobody performed: + # + # * a refusal that survived a GOOD token is an answer about this PACKAGE, and since + # ersatztv#885 it is not a preflight-only problem: every `container:` job pulls the same + # image with no credential, so they fail at image pull too, including both required + # contexts; + # * a challenge that yielded no usable token is an infrastructure answer about the TOKEN + # ENDPOINT; + # * a refusal carrying NO Bearer challenge at all never reached the token leg — `probe` + # enters it on a 401 only, so a first-read 403 (or a 401 with no `Www-Authenticate`, which + # the token leg then abandons) leaves `token` empty having asked for nothing. This is an + # answer about ACCESS to the registry, and the branch order below is `token` first + # precisely so this case cannot borrow either of the other two mechanisms. # # A failed token leg is NOT retried, deliberately: `token_leg_done` is set before the attempt # so a registry genuinely refusing anonymous reads is asked once rather than once per pin. The # cost is that a transient token-endpoint outage fails the job on its first try; the message # below says so rather than blaming the package. - if [ "$token_leg_done" -eq 1 ] && [ -z "$token" ]; then - fail "could NOT OBTAIN an anonymous pull token for $registry/$image_repo:$pin — the registry challenged (HTTP $code) but the token leg produced none: either the challenge named no realm, or the token endpoint did not answer with a token. The pin was NOT checked. Look at the registry's token endpoint, not at the pin." + if [ -n "$token" ]; then + fail "the registry refused an ANONYMOUS read (HTTP $code) of $registry/$image_repo:$pin even after a Bearer token was obtained, so the pin could not be checked. Every container: job pulls this image without a credential too, so they will fail at image pull. Check that timothy/ersatztv and its ersatztv-ci package are still PUBLIC — do not read this as a pass." fi - fail "the registry refused an ANONYMOUS read (HTTP $code) of $registry/$image_repo:$pin even after a Bearer token was obtained, so the pin could not be checked. Every container: job pulls this image without a credential too, so they will fail at image pull. Check that timothy/ersatztv and its ersatztv-ci package are still PUBLIC — do not read this as a pass." + if [ "$token_leg_done" -eq 1 ]; then + fail "could NOT OBTAIN an anonymous pull token for $registry/$image_repo:$pin — the registry answered HTTP $code and the token leg produced none: either there was no Www-Authenticate challenge, or it named no realm, or the token endpoint did not answer with a token. The pin was NOT checked. Look at the registry's token endpoint, not at the pin." + fi + fail "the registry refused an ANONYMOUS read (HTTP $code) of $registry/$image_repo:$pin WITHOUT issuing a Bearer challenge, so no token was ever requested and the pin could not be checked. A challenge is what tells a client where a token can be had; an outright refusal is an answer about ACCESS to the registry, not about the pin. Every container: job pulls this image without a credential too, so they will fail at image pull. Check that timothy/ersatztv and its ersatztv-ci package are still PUBLIC, and that nothing (a proxy, an ACL) is answering for the registry — do not read this as a pass." ;; *) # NOT gone, and NOT a pass either. Deliberately worded apart from the 404 message: this sends diff --git a/scripts/tests/test_ci_toolchain_image_resolves.py b/scripts/tests/test_ci_toolchain_image_resolves.py index 99334b0a6..9a117eaa6 100644 --- a/scripts/tests/test_ci_toolchain_image_resolves.py +++ b/scripts/tests/test_ci_toolchain_image_resolves.py @@ -85,6 +85,17 @@ code = codes.get(tag, codes.get("*", "200")) if code == "TRANSPORT": sys.exit(7) +if authorization != "Bearer anon-token" and codes.get("CHALLENGE") == "none": + # A registry that REFUSES an unauthenticated read outright instead of challenging: no + # Www-Authenticate header at all, so a client has nowhere to ask for a token. `REFUSAL` picks + # the code, because a 403 and a challenge-less 401 take DIFFERENT paths through the script -- + # `probe` enters the token leg on 401 only. + refusal = codes.get("REFUSAL", "403") + if dump: + pathlib.Path(dump).write_text("HTTP/1.1 %s Refused\r\n" % refusal) + print("{}\n%s" % refusal, end="") + sys.exit(0) + if authorization != "Bearer anon-token": # 401 for a live tag and a deleted one alike -- the challenge is the ONLY thing that tells a # client where a token can be had. @@ -271,8 +282,52 @@ def test_a_401_or_403_AFTER_the_token_leg_REFUSES_rather_than_passing(preflight, result = preflight.run() assert result.returncode != 0 assert "refused an ANONYMOUS read" in result.stderr + assert "even after a Bearer token was obtained" in result.stderr, ( + "this arm is the one where a token really was obtained, so it is the only one allowed to " + f"say so. stderr={result.stderr}" + ) assert "PUBLIC" in result.stderr, "the message must name the cause an operator can act on" - assert preflight.token_calls(), "the token leg must have been attempted before refusing" + assert preflight.authenticated_manifest_calls(), ( + "a message claiming the read survived a Bearer token must be reached with a read that " + "CARRIED one — `token_calls` only shows the token was asked for" + ) + + +@pytest.mark.parametrize("refusal", ["403", "401"]) +def test_a_refusal_with_NO_CHALLENGE_never_claims_a_token_was_obtained(preflight, refusal): + """The message must not name a mechanism the run did not perform. + + `probe` enters the token leg on a `401` only, so a registry answering `403` on the first read — + or a `401` carrying no `Www-Authenticate` — leaves the script with no token having asked for + nothing. Measured 2026-09-05 on the predecessor at 59003d5a3, the `403` shape reported "even + after a Bearer token was obtained", which sends an operator to package visibility on evidence + that does not exist (`dont-narrate-mechanisms-you-didnt-measure`). The two codes are BOTH driven + because they take different paths: the challenge-less `401` still enters and abandons the token + leg, the `403` never enters it. + + The shim answering 401 + a challenge to every unauthenticated read is why the pre-existing + `403` case could not reach this — it could only ever be observed AFTER the token leg — so the + shim grew a challenge-less behaviour rather than the assertion being written against the old one. + """ + preflight.set_codes({"*": "200", "CHALLENGE": "none", "REFUSAL": refusal}) + result = preflight.run() + assert result.returncode != 0, "an unreadable registry is not a pass" + assert "even after a Bearer token was obtained" not in result.stderr, ( + f"no token was obtained on this path. stderr={result.stderr}" + ) + assert preflight.authenticated_manifest_calls() == [], "no read can have carried a token here" + if refusal == "403": + assert preflight.token_calls() == [], "a 403 first read must not even ask for a token" + assert "WITHOUT issuing a Bearer challenge" in result.stderr, result.stderr + assert "PUBLIC" in result.stderr, "the message must still name a cause an operator can act on" + else: + # A challenge-less 401 DOES enter the token leg (and abandons it for want of a realm), so it + # is the token-endpoint diagnosis rather than the never-asked one. + assert preflight.token_calls() == [], "there was no realm to request a token from" + assert "could NOT OBTAIN an anonymous pull token" in result.stderr, result.stderr + assert "no Www-Authenticate challenge" in result.stderr, ( + f"the token-leg message must admit the challenge was missing. stderr={result.stderr}" + ) def test_the_TOKEN_LEG_actually_runs_and_the_bearer_REACHES_the_registry(preflight):