diff --git a/Directory.Packages.props b/Directory.Packages.props index 59e56e359..436d5cfa7 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -35,6 +35,10 @@ + + diff --git a/ErsatzTV/ErsatzTV.csproj b/ErsatzTV/ErsatzTV.csproj index a3a4d2c18..c94ffb3de 100644 --- a/ErsatzTV/ErsatzTV.csproj +++ b/ErsatzTV/ErsatzTV.csproj @@ -37,6 +37,10 @@ + + all diff --git a/docs/ci-cd.md b/docs/ci-cd.md index 2fb86c248..3aff5bee5 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -167,8 +167,11 @@ opt-out (`ManagePackageVersionsCentrally=false`). (ersatztv#14) `Directory.Build.props` demotes low/moderate/high advisories (NU1901-1903) to warnings and promotes NU1904 (critical) to an error in **every** project via `WarningsAsErrors`. The advisories that prompted this were resolved in ersatztv#8 (NCalcSync→6.x; SQLitePCLRaw -bundle 3.x). The NU1901-1903 demotion is **kept by design**: criticals (NU1904) still hard- -block, while low/moderate/high advisories surface as warnings + via the weekly scan and +bundle 3.x) and ersatztv#314 (**Microsoft.OpenApi 2.0.0→2.7.5**, GHSA-v5pm-xwqc-g5wc High — +direct-pinned in `ErsatzTV.csproj` over the 2.0.0 that `Microsoft.AspNetCore.OpenApi` + +`Scalar.AspNetCore` pull transitively; the SQLitePCLRaw override pattern; regenerates the +OpenAPI doc byte-identically). The NU1901-1903 demotion is **kept by design**: criticals (NU1904) +still hard-block, while low/moderate/high advisories surface as warnings + via the weekly scan and Renovate security PRs, rather than breaking unrelated PRs the moment a new transitive advisory drops. @@ -206,6 +209,35 @@ Core `[9.0.x,10)`) are respected — no v10 jump. PR volume is throttled (`prCon to drain faster. `workflow_dispatch` defaults to a safe **dry run**. Cross-repo rollout tracked in server-management#484. (server-management#484) +## Security scanning — black-box DAST + SAST (`scripts/security-scan.sh`, ersatztv#314) + +Every other security check we run is **in-ecosystem / white-box** — SonarAnalyzer, NetArchTest, the +adversarial fork + Codex review passes, the api-docs/format/decisions CI gates, `dotnet list package +--vulnerable` — so they share our blind spots. `scripts/security-scan.sh` is the **out-of-ecosystem, +black-box** complement and a **#197 exit criterion** (HARD GATE before remote exposure): it drives the +running product from outside our C#/review stack. + +- **What it does.** Boots a **throwaway** container from the image under test (fresh empty config volume; + never the deployed prod/test container — the authenticated *active* scan sends attack payloads to write + endpoints), reads the generated machine key, and runs an **authenticated OWASP ZAP** API scan + (`zap-api-scan.py`) that imports the static `/openapi/v1.json` so it exercises **every** declared + `/api/v1` operation, injecting `X-Api-Key` on every request via a ZAP replacer rule so it reaches the + `[RequiresAuthentication]` + `RequireKeyForReads` surface (not just the `/app` shell an unauthenticated + spider sees). Then a **semgrep** SAST cross-check (`p/security-audit` + `p/secrets` + `p/csharp`). The + container is torn down on exit. +- **Where/when.** Runs on the **docker host** (bumblebee — the Mac has no docker), like + `migration-smoke.sh`: `scripts/security-scan.sh [IMAGE] [PORT]` (defaults `…:latest` / `8411`). It is a + **manual release-gate**, deliberately **not** a per-PR CI job — it needs docker + a booted image, takes + several minutes, and is noisy (expect to tune, not take raw). The continuous layer is the per-PR + white-box gates + the weekly `dependency-scan`; this is the per-release black-box pass. Re-run it each + release and before any change to the exposure posture. +- **Triage.** ZAP exits non-zero on any FAIL-level alert; the tooling is noisy, so triage each finding + false-positive vs real. Real, in-scope, go-live-blocking findings get fixed (e.g. the security headers + from the #319 baseline; the Microsoft.OpenApi pin above); LAN-expected noise (Private-IP disclosure) is + revisited only for genuine remote exposure. nuclei (template-based CVE fingerprinting) is an optional + third pass — deferred while its template fetch is blocked in the runner env (pre-seed a template volume + to add it); ZAP covers the DAST baseline and semgrep the SAST, so it is not on the critical path. + ## Static analysis & formatting **Analyzer packs** — `Directory.Build.targets` references **Roslynator**, **SonarAnalyzer.CSharp**, diff --git a/scripts/security-scan.sh b/scripts/security-scan.sh new file mode 100755 index 000000000..013534d38 --- /dev/null +++ b/scripts/security-scan.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# scripts/security-scan.sh — out-of-ecosystem, black-box security scan of a release image (ersatztv#314). +# +# WHY: every OTHER security check we run is in-ecosystem / white-box (SonarAnalyzer, NetArchTest, the +# adversarial fork + Codex review passes, the api-docs/format/decisions CI gates, `dotnet list package +# --vulnerable`). They reason about our source the way we do, so they share our blind spots. This drives +# the RUNNING product from outside our C#/review stack — OWASP ZAP as an authenticated DAST against every +# `/api/v1` endpoint, plus semgrep as a SAST cross-check — to catch the classes structural review misses: +# missing/incorrect security headers, an endpoint that forgot its auth attribute, verb tampering, +# injection reflections, known-CVE fingerprints. It is a #197 exit criterion — a HARD GATE before remote +# exposure — and should be re-run each release (see docs/ci-cd.md → Security scanning). +# +# It runs on the docker host (bumblebee) against a THROWAWAY container booted from the image under test — +# NEVER the deployed prod (or test) container: the authenticated active scan sends attack payloads to +# write endpoints (creating/deleting rows, triggering playout builds), so it must target a disposable +# instance with a fresh empty config volume. The container is torn down on exit. +# +# Usage: +# scripts/security-scan.sh [IMAGE] [PORT] +# IMAGE container image to scan (default: 192.168.1.95:3000/timothy/ersatztv:latest) +# PORT host port for the throwaway app (default: 8411) +# Env: +# ETV_SCAN_OUT host dir for reports (default: /tmp/etv-scan-out) +# ETV_SCAN_SKIP_SEMGREP=1 skip the SAST pass (DAST only) +# +# Reports land in $ETV_SCAN_OUT: zap-api-report.{html,md,json}. Exit code is ZAP's: +# 0 = no FAIL-level alerts, non-zero = at least one FAIL (triage it — this tooling is noisy; expect to +# tune the ignore list in .zap/ rather than take the raw report at face value). + +set -euo pipefail + +IMAGE="${1:-192.168.1.95:3000/timothy/ersatztv:latest}" +PORT="${2:-8411}" +OUT="${ETV_SCAN_OUT:-/tmp/etv-scan-out}" +TARGET_NAME="etv-secscan-target" +ZAP_IMAGE="ghcr.io/zaproxy/zaproxy:stable" + +command -v docker >/dev/null || { echo "error: docker not found — run this on the docker host (bumblebee)" >&2; exit 2; } + +CFG="$(mktemp -d /tmp/etv-secscan-cfg-XXXXXX)" +cleanup() { + # Reap the ZAP scanner container too — in the trap (not just inline after the scan) so an interrupt + # (SIGINT/SIGTERM) or a `timeout` kill mid-scan doesn't leave it running. + docker ps --filter ancestor="$ZAP_IMAGE" -q | xargs -r docker kill >/dev/null 2>&1 || true + docker rm -f "$TARGET_NAME" >/dev/null 2>&1 || true + rm -rf "$CFG" 2>/dev/null || true +} +trap cleanup EXIT + +echo "==> pulling $IMAGE" +docker pull "$IMAGE" >/dev/null + +echo "==> booting throwaway scan target ($TARGET_NAME) on :$PORT with a fresh config volume" +docker rm -f "$TARGET_NAME" >/dev/null 2>&1 || true +docker run -d --name "$TARGET_NAME" -p "$PORT:8409" -v "$CFG":/config "$IMAGE" >/dev/null + +echo -n "==> waiting for the app to serve /app/ " +ready=0 +for _ in $(seq 1 60); do + if [ "$(curl -s -o /dev/null -w '%{http_code}' "http://localhost:$PORT/app/" 2>/dev/null)" = "200" ]; then + ready=1; echo "ready"; break + fi + echo -n "."; sleep 3 +done +[ "$ready" = 1 ] || { echo "FAILED: app never became ready"; docker logs "$TARGET_NAME" 2>&1 | tail -30; exit 1; } + +# The machine key is generated at /config/api.key (0600, root-owned on the host mount) — read it from +# inside the container. ZAP injects it as X-Api-Key on every request via a replacer rule so the scan +# reaches the authenticated (`RequiresAuthentication` + RequireKeyForReads) surface, not just the /app shell. +KEY="$(docker exec "$TARGET_NAME" cat /config/api.key 2>/dev/null)" +[ -n "$KEY" ] || { echo "FAILED: could not read /config/api.key"; exit 1; } +echo "==> got machine key (${#KEY} chars); it will be injected as X-Api-Key on every request" + +mkdir -p "$OUT"; chmod 777 "$OUT" + +# zap-api-scan.py imports the OpenAPI spec (served as a static file at /openapi/v1.json) so it exercises +# every declared /api/v1 operation — not just what a spider finds — then runs passive + active scanners. +# The reports are written when the scan finishes but BEFORE the wrapper exits; zap-api-scan has been seen +# to hang in post-scan cleanup (draining the passive queue / holding a connection to a streaming endpoint), +# so the whole run is wrapped in `timeout` — a hung wrapper is killed and the already-written report is +# still used. ETV_SCAN_TIMEOUT (default 45m) bounds it. +echo "==> running authenticated OWASP ZAP API scan (this can take several minutes)" +set +e +timeout "${ETV_SCAN_TIMEOUT:-45m}" docker run --rm --network host -v "$OUT":/zap/wrk:rw "$ZAP_IMAGE" \ + zap-api-scan.py -t "http://localhost:$PORT/openapi/v1.json" -f openapi \ + -r zap-api-report.html -w zap-api-report.md -J zap-api-report.json \ + -z "replacer.full_list(0).description=apikey;replacer.full_list(0).enabled=true;replacer.full_list(0).matchtype=REQ_HEADER;replacer.full_list(0).matchstr=X-Api-Key;replacer.full_list(0).regex=false;replacer.full_list(0).replacement=$KEY" +zap_rc=$? +[ "$zap_rc" = 124 ] && echo "NOTE: ZAP wrapper hit the ${ETV_SCAN_TIMEOUT:-45m} timeout (usually a post-scan cleanup hang) — the report written before the hang is still valid; triage it." +set -e +# (The ZAP container + target + temp dir are reaped by cleanup() on the EXIT trap — every path, incl. +# a `timeout` kill or Ctrl-C mid-scan.) +echo "==> ZAP report: $OUT/zap-api-report.html (+ .md/.json). Exit code: $zap_rc" + +if [ "${ETV_SCAN_SKIP_SEMGREP:-0}" = "1" ]; then + echo "==> semgrep SAST skipped (ETV_SCAN_SKIP_SEMGREP=1)" +elif ! command -v semgrep >/dev/null; then + echo "WARN: semgrep not installed — SAST cross-check SKIPPED (install semgrep for full coverage)" >&2 +else + echo "==> semgrep SAST cross-check (repo root)" + REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" + semgrep scan --config p/security-audit --config p/secrets --config p/csharp \ + --exclude web/node_modules --exclude design-system --metrics off --quiet \ + "$REPO_ROOT" || echo "(semgrep reported findings — triage in the report above)" +fi + +exit "$zap_rc"