feat(ci): #314 — authenticated black-box security scan + Microsoft.OpenApi HIGH-vuln pin #331

Merged
timothy merged 3 commits from fix/314-security-scan into main 2026-07-13 08:42:43 +02:00
4 changed files with 149 additions and 2 deletions
+4
View File
@@ -35,6 +35,10 @@
<PackageVersion Include="Microsoft.Extensions.Identity.Core" Version="10.0.2" />
<PackageVersion Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="10.0.2" />
<PackageVersion Include="Microsoft.AspNetCore.OpenApi" Version="10.0.2" />
<!-- Direct-pin over the 2.0.0 transitive (from Microsoft.AspNetCore.OpenApi + Scalar.AspNetCore):
2.0.0 is GHSA-v5pm-xwqc-g5wc (High — stack overflow parsing a circular $ref). Fixed in 2.7.5.
Referenced directly in ErsatzTV.csproj so the override actually resolves (CPM). See ersatztv#314/#8. -->
<PackageVersion Include="Microsoft.OpenApi" Version="2.7.5" />
<PackageVersion Include="Microsoft.AspNetCore.SpaServices.Extensions" Version="10.0.2" />
<PackageVersion Include="Microsoft.EntityFrameworkCore" Version="[9.0.12,10)" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.Design" Version="[9.0.12,10)" />
+4
View File
@@ -37,6 +37,10 @@
<PackageReference Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" />
<PackageReference Include="Microsoft.AspNetCore.OpenApi" />
<!-- Direct ref forces the patched Microsoft.OpenApi 2.7.5 over the vulnerable 2.0.0 that
Microsoft.AspNetCore.OpenApi + Scalar.AspNetCore pull transitively (GHSA-v5pm-xwqc-g5wc).
Version lives in Directory.Packages.props (CPM). -->
<PackageReference Include="Microsoft.OpenApi" />
<PackageReference Include="Microsoft.AspNetCore.SpaServices.Extensions" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design">
<PrivateAssets>all</PrivateAssets>
+34 -2
View File
@@ -167,8 +167,11 @@ opt-out (`ManagePackageVersionsCentrally=false`). (ersatztv#14)
`Directory.Build.props` demotes low/moderate/high advisories (NU1901-1903) to warnings
and promotes NU1904 (critical) to an error in **every** project via `WarningsAsErrors`.
The advisories that prompted this were resolved in ersatztv#8 (NCalcSync→6.x; SQLitePCLRaw
bundle 3.x). The NU1901-1903 demotion is **kept by design**: criticals (NU1904) still hard-
block, while low/moderate/high advisories surface as warnings + via the weekly scan and
bundle 3.x) and ersatztv#314 (**Microsoft.OpenApi 2.0.0→2.7.5**, GHSA-v5pm-xwqc-g5wc High —
direct-pinned in `ErsatzTV.csproj` over the 2.0.0 that `Microsoft.AspNetCore.OpenApi` +
`Scalar.AspNetCore` pull transitively; the SQLitePCLRaw override pattern; regenerates the
OpenAPI doc byte-identically). The NU1901-1903 demotion is **kept by design**: criticals (NU1904)
still hard-block, while low/moderate/high advisories surface as warnings + via the weekly scan and
Renovate security PRs, rather than breaking unrelated PRs the moment a new transitive
advisory drops.
@@ -206,6 +209,35 @@ Core `[9.0.x,10)`) are respected — no v10 jump. PR volume is throttled (`prCon
to drain faster. `workflow_dispatch` defaults to a safe **dry run**. Cross-repo rollout
tracked in server-management#484. (server-management#484)
## Security scanning — black-box DAST + SAST (`scripts/security-scan.sh`, ersatztv#314)
Every other security check we run is **in-ecosystem / white-box** — SonarAnalyzer, NetArchTest, the
adversarial fork + Codex review passes, the api-docs/format/decisions CI gates, `dotnet list package
--vulnerable` — so they share our blind spots. `scripts/security-scan.sh` is the **out-of-ecosystem,
black-box** complement and a **#197 exit criterion** (HARD GATE before remote exposure): it drives the
running product from outside our C#/review stack.
- **What it does.** Boots a **throwaway** container from the image under test (fresh empty config volume;
never the deployed prod/test container — the authenticated *active* scan sends attack payloads to write
endpoints), reads the generated machine key, and runs an **authenticated OWASP ZAP** API scan
(`zap-api-scan.py`) that imports the static `/openapi/v1.json` so it exercises **every** declared
`/api/v1` operation, injecting `X-Api-Key` on every request via a ZAP replacer rule so it reaches the
`[RequiresAuthentication]` + `RequireKeyForReads` surface (not just the `/app` shell an unauthenticated
spider sees). Then a **semgrep** SAST cross-check (`p/security-audit` + `p/secrets` + `p/csharp`). The
container is torn down on exit.
- **Where/when.** Runs on the **docker host** (bumblebee — the Mac has no docker), like
`migration-smoke.sh`: `scripts/security-scan.sh [IMAGE] [PORT]` (defaults `…:latest` / `8411`). It is a
**manual release-gate**, deliberately **not** a per-PR CI job — it needs docker + a booted image, takes
several minutes, and is noisy (expect to tune, not take raw). The continuous layer is the per-PR
white-box gates + the weekly `dependency-scan`; this is the per-release black-box pass. Re-run it each
release and before any change to the exposure posture.
- **Triage.** ZAP exits non-zero on any FAIL-level alert; the tooling is noisy, so triage each finding
false-positive vs real. Real, in-scope, go-live-blocking findings get fixed (e.g. the security headers
from the #319 baseline; the Microsoft.OpenApi pin above); LAN-expected noise (Private-IP disclosure) is
revisited only for genuine remote exposure. nuclei (template-based CVE fingerprinting) is an optional
third pass — deferred while its template fetch is blocked in the runner env (pre-seed a template volume
to add it); ZAP covers the DAST baseline and semgrep the SAST, so it is not on the critical path.
## Static analysis & formatting
**Analyzer packs** — `Directory.Build.targets` references **Roslynator**, **SonarAnalyzer.CSharp**,
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env bash
# scripts/security-scan.sh — out-of-ecosystem, black-box security scan of a release image (ersatztv#314).
#
# WHY: every OTHER security check we run is in-ecosystem / white-box (SonarAnalyzer, NetArchTest, the
# adversarial fork + Codex review passes, the api-docs/format/decisions CI gates, `dotnet list package
# --vulnerable`). They reason about our source the way we do, so they share our blind spots. This drives
# the RUNNING product from outside our C#/review stack — OWASP ZAP as an authenticated DAST against every
# `/api/v1` endpoint, plus semgrep as a SAST cross-check — to catch the classes structural review misses:
# missing/incorrect security headers, an endpoint that forgot its auth attribute, verb tampering,
# injection reflections, known-CVE fingerprints. It is a #197 exit criterion — a HARD GATE before remote
# exposure — and should be re-run each release (see docs/ci-cd.md → Security scanning).
#
# It runs on the docker host (bumblebee) against a THROWAWAY container booted from the image under test —
# NEVER the deployed prod (or test) container: the authenticated active scan sends attack payloads to
# write endpoints (creating/deleting rows, triggering playout builds), so it must target a disposable
# instance with a fresh empty config volume. The container is torn down on exit.
#
# Usage:
# scripts/security-scan.sh [IMAGE] [PORT]
# IMAGE container image to scan (default: 192.168.1.95:3000/timothy/ersatztv:latest)
# PORT host port for the throwaway app (default: 8411)
# Env:
# ETV_SCAN_OUT host dir for reports (default: /tmp/etv-scan-out)
# ETV_SCAN_SKIP_SEMGREP=1 skip the SAST pass (DAST only)
#
# Reports land in $ETV_SCAN_OUT: zap-api-report.{html,md,json}. Exit code is ZAP's:
# 0 = no FAIL-level alerts, non-zero = at least one FAIL (triage it — this tooling is noisy; expect to
# tune the ignore list in .zap/ rather than take the raw report at face value).
set -euo pipefail
IMAGE="${1:-192.168.1.95:3000/timothy/ersatztv:latest}"
PORT="${2:-8411}"
OUT="${ETV_SCAN_OUT:-/tmp/etv-scan-out}"
TARGET_NAME="etv-secscan-target"
ZAP_IMAGE="ghcr.io/zaproxy/zaproxy:stable"
command -v docker >/dev/null || { echo "error: docker not found — run this on the docker host (bumblebee)" >&2; exit 2; }
CFG="$(mktemp -d /tmp/etv-secscan-cfg-XXXXXX)"
cleanup() {
# Reap the ZAP scanner container too — in the trap (not just inline after the scan) so an interrupt
# (SIGINT/SIGTERM) or a `timeout` kill mid-scan doesn't leave it running.
docker ps --filter ancestor="$ZAP_IMAGE" -q | xargs -r docker kill >/dev/null 2>&1 || true
docker rm -f "$TARGET_NAME" >/dev/null 2>&1 || true
rm -rf "$CFG" 2>/dev/null || true
}
trap cleanup EXIT
echo "==> pulling $IMAGE"
docker pull "$IMAGE" >/dev/null
echo "==> booting throwaway scan target ($TARGET_NAME) on :$PORT with a fresh config volume"
docker rm -f "$TARGET_NAME" >/dev/null 2>&1 || true
docker run -d --name "$TARGET_NAME" -p "$PORT:8409" -v "$CFG":/config "$IMAGE" >/dev/null
echo -n "==> waiting for the app to serve /app/ "
ready=0
for _ in $(seq 1 60); do
if [ "$(curl -s -o /dev/null -w '%{http_code}' "http://localhost:$PORT/app/" 2>/dev/null)" = "200" ]; then
ready=1; echo "ready"; break
fi
echo -n "."; sleep 3
done
[ "$ready" = 1 ] || { echo "FAILED: app never became ready"; docker logs "$TARGET_NAME" 2>&1 | tail -30; exit 1; }
# The machine key is generated at /config/api.key (0600, root-owned on the host mount) — read it from
# inside the container. ZAP injects it as X-Api-Key on every request via a replacer rule so the scan
# reaches the authenticated (`RequiresAuthentication` + RequireKeyForReads) surface, not just the /app shell.
KEY="$(docker exec "$TARGET_NAME" cat /config/api.key 2>/dev/null)"
[ -n "$KEY" ] || { echo "FAILED: could not read /config/api.key"; exit 1; }
echo "==> got machine key (${#KEY} chars); it will be injected as X-Api-Key on every request"
mkdir -p "$OUT"; chmod 777 "$OUT"
# zap-api-scan.py imports the OpenAPI spec (served as a static file at /openapi/v1.json) so it exercises
# every declared /api/v1 operation — not just what a spider finds — then runs passive + active scanners.
# The reports are written when the scan finishes but BEFORE the wrapper exits; zap-api-scan has been seen
# to hang in post-scan cleanup (draining the passive queue / holding a connection to a streaming endpoint),
# so the whole run is wrapped in `timeout` — a hung wrapper is killed and the already-written report is
# still used. ETV_SCAN_TIMEOUT (default 45m) bounds it.
echo "==> running authenticated OWASP ZAP API scan (this can take several minutes)"
set +e
timeout "${ETV_SCAN_TIMEOUT:-45m}" docker run --rm --network host -v "$OUT":/zap/wrk:rw "$ZAP_IMAGE" \
zap-api-scan.py -t "http://localhost:$PORT/openapi/v1.json" -f openapi \
-r zap-api-report.html -w zap-api-report.md -J zap-api-report.json \
-z "replacer.full_list(0).description=apikey;replacer.full_list(0).enabled=true;replacer.full_list(0).matchtype=REQ_HEADER;replacer.full_list(0).matchstr=X-Api-Key;replacer.full_list(0).regex=false;replacer.full_list(0).replacement=$KEY"
zap_rc=$?
[ "$zap_rc" = 124 ] && echo "NOTE: ZAP wrapper hit the ${ETV_SCAN_TIMEOUT:-45m} timeout (usually a post-scan cleanup hang) — the report written before the hang is still valid; triage it."
set -e
# (The ZAP container + target + temp dir are reaped by cleanup() on the EXIT trap — every path, incl.
# a `timeout` kill or Ctrl-C mid-scan.)
echo "==> ZAP report: $OUT/zap-api-report.html (+ .md/.json). Exit code: $zap_rc"
if [ "${ETV_SCAN_SKIP_SEMGREP:-0}" = "1" ]; then
echo "==> semgrep SAST skipped (ETV_SCAN_SKIP_SEMGREP=1)"
elif ! command -v semgrep >/dev/null; then
echo "WARN: semgrep not installed — SAST cross-check SKIPPED (install semgrep for full coverage)" >&2
else
echo "==> semgrep SAST cross-check (repo root)"
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
semgrep scan --config p/security-audit --config p/secrets --config p/csharp \
--exclude web/node_modules --exclude design-system --metrics off --quiet \
"$REPO_ROOT" || echo "(semgrep reported findings — triage in the report above)"
fi
exit "$zap_rc"