timothy/ersatztv
1
0
Fork 0
Code Issues 58 Pull Requests 8 Actions Packages Projects Releases 9 Wiki Activity

docs(931): state the scan host's requirement instead of naming hosts, and every script exit

Browse Source 
Round-3 review: the Security scanning section named bumblebee (has semgrep) and
jazz (needs it) with no date or source, the Exit-code contract still said the
script exits 0 for clean/WARN and 1 for FAIL, the exit-2 sentence said only the
SAST half, and the --selftest sentence claimed more main-path coverage than it
pinned.

- Security scanning drops host names: the requirement is a docker host with
  registry access, a checkout of the commit under test (semgrep scans the
  checkout the script lives in), and semgrep on the non-interactive PATH, or
  the opt-out plus a separate SAST run.
- The Exit-code contract lists 0/1/2/other; 2 is "a scanner half did not run"
  (semgrep missing or erroring, or docker missing). The --selftest sentence
  names exactly the cases it pins, now including the main-path ones.
- The Hosts table and release step 2 sent the scan to jazz. That dates from
  d6aec6de7 (2026-07-20), which moved it there with the transcoders, but the
  scan boots its own throwaway target from the registry and needs nothing
  jazz-local. Step 2 now states the requirement, with the one measured host
  dated and sourced (bumblebee, 2026-10-27, server-management#1072). jazz had
  no semgrep when measured 2026-10-27, so following the old step would fail
  the gate with exit 2.
- Edited prose rewrapped to the section's ~105 columns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: f9ce6bc6-5b36-491a-8d72-89c1f70dd3e1
This commit is contained in:
timothy
2026-10-30 00:03:24 +01:00
co-authored by Claude Opus 5.5
parent 2e4de81309
commit 0e64328ca2
1 changed files with 35 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ci-cd.md
+35 -24
View File
@@ -9,7 +9,7 @@ server-management#481.
| Host | IP | Role |
|---|---|---|
| **jazz** | 192.168.1.29 | **Docker host for the media transcoders** — prod `ersatztv` (8409), `ersatztv-test` (8410), Jellyfin. Release scans (`security-scan.sh`) and the prod-copy `migration-smoke.sh` run here. |
| **jazz** | 192.168.1.29 | **Docker host for the media transcoders** — prod `ersatztv` (8409), `ersatztv-test` (8410), Jellyfin. The prod-copy `migration-smoke.sh` runs here. The release scan (`security-scan.sh`) boots its own throwaway target, so it needs a docker host that meets its requirements (Security scanning → Where/when), not this one. |
| **bumblebee** | 192.168.1.99 | CI runners (`bumblebee-runner`, `small-runner`), plus every other Docker stack. All the memory/lane measurements below were taken here. |
| **ci-runner** | VM 127 (pve4) | The other `ubuntu-latest` CI runner. |
@@ -116,8 +116,12 @@ commit in the range (see the `decisions.md` header) — routine lifecycle metada
> protection is never left off) — and record the outcome on `release.tag-protection-v-star`,
> which is waiting for it.
2. Wait for tag CI to build `:prod` + the immutable `:<version>` + `:<sha>` images. Run
`scripts/security-scan.sh` on jazz against **the immutable `:<version>` image**, not a
moving tag, and triage every ZAP/semgrep finding.
`scripts/security-scan.sh` against **the immutable `:<version>` image**, not a moving tag, from a
checkout of the tagged commit on a docker host whose non-interactive `PATH` has semgrep: check with
`ssh <host> command -v semgrep` first (bumblebee passed that check on 2026-10-27,
server-management#1072). A host without semgrep fails the gate with exit 2; the only alternative is
`ETV_SCAN_SKIP_SEMGREP=1` plus a separate semgrep run against the tagged commit, recorded as such.
Triage every ZAP/semgrep finding. Requirements: Security scanning → Where/when.
3. Only after the candidate passes, manually `DeployStack jazz-media` and observe its pre-deploy
output. Prod's compose deliberately follows floating `:prod` (Timothy's 2026-07-11 decision), so
no CI push or pin bump is needed.
@@ -2660,19 +2664,21 @@ running product from outside our C#/review stack.
`/api/v1` operation, injecting `X-Api-Key` on every request via a ZAP replacer rule so it reaches the
`[RequiresAuthentication]` + `RequireKeyForReads` surface (not just the `/app` shell an unauthenticated
spider sees). Then a **semgrep** SAST cross-check (`p/security-audit` + `p/secrets` + `p/csharp`). The
container is torn down on exit. **The SAST half is required**: a host without
semgrep on `PATH` fails the script up front (exit 2, naming the tool) before the image is pulled, and the
only skip is the explicit opt-out `ETV_SCAN_SKIP_SEMGREP=1` (DAST only; a gate run that sets it has not
run the SAST half, so run semgrep separately and say so in the release record). A semgrep that is installed but exits non-zero (bad config,
registry unreachable) also fails the script, since findings alone exit 0. `--selftest` pins the
missing, opted-out, present and failing cases through the script's own exit status (ersatztv#931). A non-interactive ssh shell needs semgrep on
its `PATH` (`~/.zshenv`, not `.zshrc`).
- **Where/when.** Runs on a **docker host** with registry access (bumblebee, which has semgrep on its non-interactive `PATH`; a host without semgrep fails the gate, so jazz needs it installed first), like
`migration-smoke.sh`: `scripts/security-scan.sh [IMAGE] [PORT]` (defaults `…:latest` / `8411`). It is a
**manual release-gate**, deliberately **not** a per-PR CI job — it needs docker + a booted image, takes
several minutes, and is noisy (expect to tune, not take raw). The continuous layer is the per-PR
white-box gates + the weekly `dependency-scan`; this is the per-release black-box pass. Re-run it each
release and before any change to the exposure posture.
container is torn down on exit. **The SAST half is required** (ersatztv#931): with no semgrep on `PATH`
the script exits 2 naming the tool, before the image is pulled. The only skip is the explicit opt-out
`ETV_SCAN_SKIP_SEMGREP=1` (any other value, `0` included, does not skip). A run that sets it is DAST
only and has not passed the SAST half: run semgrep separately and say so in the release record. A
semgrep that is installed but exits non-zero (bad config, registry unreachable) also fails the script,
since findings alone exit 0.
- **Where/when.** Runs on a **docker host** with registry access, from a checkout of the commit under test
(semgrep scans the checkout the script lives in), whose **non-interactive** `PATH` has semgrep: the
`PATH` that `ssh <host> command -v semgrep` sees, which for zsh comes from `~/.zshenv`, not `.zshrc`. A
host without it fails the gate by construction; the alternative is the opt-out plus a separate SAST
run. Like `migration-smoke.sh`: `scripts/security-scan.sh [IMAGE] [PORT]` (defaults `…:latest` /
`8411`). It is a **manual release-gate**, deliberately **not** a per-PR CI job — it needs docker + a
booted image, takes several minutes, and is noisy (expect to tune, not take raw). The continuous layer
is the per-PR white-box gates + the weekly `dependency-scan`; this is the per-release black-box pass.
Re-run it each release and before any change to the exposure posture.
- **Exit-code contract (ersatztv#338).** `zap-api-scan.py`'s raw exit code is NOT a simple pass/fail — it
conflates a clean run with a warnings-only run unless you know its wrapper contract: **0** clean (no
FAIL or WARN alerts), **2** WARN-only (triage required, but **not** release-blocking), **1** FAIL (at
@@ -2680,14 +2686,19 @@ running product from outside our C#/review stack.
hung post-scan cleanup (the report written before the hang is still usable — triage it), any other
code means the scanner/tool itself errored (not a scan result at all). `scripts/security-scan.sh`
encodes this in `classify_zap_exit()` and prints an unambiguous `==> ZAP result: <PASS|WARN|FAIL|
TIMEOUT|TOOL ERROR> ...` line; the script's own exit status reflects that classification (0 for
clean/WARN, 1 for FAIL/timeout/tool-error) rather than ZAP's raw code, so a warnings-only run no longer
reads as a failed scan. Found when the v26.8.0 release scan (#335) returned raw exit 2 for a report with
`FAIL-NEW: 0` and two known/expected warning classes — the shell result looked like a failure though the
release gate had actually passed. Run `scripts/security-scan.sh --selftest` for a docker-free regression
check of the classification logic.
**The script's own exit 2 is not ZAP's WARN**: it means the SAST half did not run (semgrep missing
before the scan, or present and erroring after it) and is release-blocking.
TIMEOUT|TOOL ERROR> ...` line. The script's **own** exit is not ZAP's raw code, so a warnings-only run
no longer reads as a failed scan. Found when the v26.8.0 release scan (#335) returned raw exit 2 for a
report with `FAIL-NEW: 0` and two known/expected warning classes — the shell result looked like a
failure though the release gate had actually passed. Every non-zero script exit is release-blocking:
**0** ZAP clean or WARN-only and the SAST half ran clean (or was opted out); **1** ZAP FAIL, timeout or
tool error, or the throwaway target never came up or its key was unreadable; **2** — *not* ZAP's WARN —
a scanner half did not run: semgrep missing (before the scan) or erroring (after it), or docker
missing; if ZAP already classified non-zero, that status stands and the semgrep error prints beside
it; **any other** code is a setup command (e.g. `docker pull`) failing under `set -e`. Run
`scripts/security-scan.sh --selftest` for a docker-free regression check: it pins every ZAP
classification, and runs the script itself with shimmed tools (ZAP reporting clean) to pin the
semgrep gate — missing → exit 2 before docker is checked, erroring → 2, succeeding → 0 with semgrep
run, opt-out → 0 with semgrep not run (`=0` still runs it). It exercises no real ZAP or semgrep.
- **Triage.** Triage each WARN/FAIL finding false-positive vs real. Real, in-scope, go-live-blocking
findings get fixed (e.g. the security headers from the #319 baseline; the Microsoft.OpenApi pin above);
LAN-expected noise (Private-IP disclosure) is revisited only for genuine remote exposure. nuclei