fix(931): pin the main-path semgrep block in --selftest and state every exit the script produces
Round-3 review found two clause mutants of the main path surviving the selftest: the sg_rc check forced true (M9) and the main-path opt-out widened to any non-empty value (M10). The present case was pinned only through require_semgrep, and no full shimmed run used a semgrep that succeeds. --selftest now runs the script end to end with shimmed docker/curl/timeout (ZAP clean) and a semgrep shim that records each call: succeeding (unset and =0) -> exit 0 with semgrep run; failing -> exit 2; explicit opt-out with semgrep present -> exit 0 with semgrep not run. Witnessed red: M9 fails two cases, M10 fails the skip=0 case, and forcing the main-path skip off (`if false`) fails the opt-out case; M1-M8 stay red. The header's exit contract claimed exit 2 always meant the SAST half did not run, but a missing docker also exits 2. It now lists 0/1/2/other, with 2 as "a scanner half did not run", and the host line states the requirement instead of naming bumblebee. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: f9ce6bc6-5b36-491a-8d72-89c1f70dd3e1
This commit is contained in:
Claude Opus 5.5
+37
-14
@@ -10,7 +10,8 @@
|
||||
# injection reflections, known-CVE fingerprints. It is a #197 exit criterion — a HARD GATE before remote
|
||||
# exposure — and should be re-run each release (see docs/ci-cd.md → Security scanning).
|
||||
#
|
||||
# It runs on the docker host (bumblebee) against a THROWAWAY container booted from the image under test —
|
||||
# It runs on a docker host with registry access and semgrep on its non-interactive PATH (docs/ci-cd.md →
|
||||
# Security scanning → Where/when), against a THROWAWAY container booted from the image under test —
|
||||
# NEVER the deployed prod (or test) container: the authenticated active scan sends attack payloads to
|
||||
# write endpoints (creating/deleting rows, triggering playout builds), so it must target a disposable
|
||||
# instance with a fresh empty config volume. The container is torn down on exit.
|
||||
@@ -31,10 +32,14 @@
|
||||
# 1 FAIL — at least one FAIL-level alert — release-blocking
|
||||
# 124 our own `timeout` wrapper killed a hung wrapper (see the NOTE below) — report is still usable
|
||||
# other the scanner/tool itself errored (bad args, crash) — not a scan result at all
|
||||
# classify_zap_exit() (below) is the single place this contract is encoded; the script's own final exit
|
||||
# code reflects the CLASSIFICATION (0 for clean/WARN, 1 for FAIL/timeout/tool-error), not ZAP's raw code.
|
||||
# The one script-level exit 2 is NOT ZAP's WARN: it means the SAST half did not run (semgrep missing, or
|
||||
# present and erroring) and IS release-blocking.
|
||||
# classify_zap_exit() (below) is the single place this contract is encoded. The SCRIPT's own exit is not
|
||||
# ZAP's raw code; every non-zero exit is release-blocking:
|
||||
# 0 ZAP clean or WARN-only, AND the SAST half ran clean (or was opted out: ETV_SCAN_SKIP_SEMGREP=1)
|
||||
# 1 ZAP FAIL / timeout / tool error, or the throwaway target never came up or its key was unreadable
|
||||
# 2 NOT ZAP's WARN: a scanner half did not run — semgrep missing (checked before the scan) or erroring
|
||||
# (after it), or docker missing. If ZAP already classified non-zero, that status stands and the
|
||||
# semgrep error is printed alongside it.
|
||||
# other a setup command failed under `set -e` (e.g. `docker pull`), exiting with that command's status
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
@@ -121,22 +126,40 @@ if [ "${1:-}" = "--selftest" ]; then
|
||||
else
|
||||
echo "SELFTEST FAIL: script with opt-out -> rc=$rc out='$out' (want to pass the semgrep gate)" >&2; selftest_failed=1
|
||||
fi
|
||||
# Full run with shimmed docker/curl/timeout and a semgrep that FAILS (exit 2): the script must exit non-zero.
|
||||
# Full runs through the MAIN-path SAST block, with shimmed docker/curl/timeout (ZAP reports clean, so the
|
||||
# script's exit is decided by the semgrep half alone). The semgrep shim records each call in
|
||||
# $ETV_SCAN_OUT/semgrep.called and exits $SG_SHIM_RC, so a case can assert both the exit and whether
|
||||
# SAST actually ran. These pin the main-path opt-out test and the sg_rc check, not just the helper.
|
||||
sg_shim="$(mktemp -d)"; sg_out="$(mktemp -d)"
|
||||
printf '#!/bin/sh\n[ "$1" = exec ] && echo shimkey\nexit 0\n' > "$sg_shim/docker"
|
||||
printf '#!/bin/sh\necho 200\n' > "$sg_shim/curl"
|
||||
printf '#!/bin/sh\nshift\nexec "$@"\n' > "$sg_shim/timeout"
|
||||
printf '#!/bin/sh\necho "fake semgrep: registry unreachable" >&2\nexit 2\n' > "$sg_shim/semgrep"
|
||||
printf '#!/bin/sh\necho called >> "$ETV_SCAN_OUT/semgrep.called"\n[ "${SG_SHIM_RC:-0}" = 0 ] || echo "fake semgrep: registry unreachable" >&2\nexit "${SG_SHIM_RC:-0}"\n' > "$sg_shim/semgrep"
|
||||
chmod +x "$sg_shim"/*
|
||||
out="$(env -u ETV_SCAN_SKIP_SEMGREP PATH="$sg_shim:$PATH" ETV_SCAN_OUT="$sg_out" "$BASH" "${BASH_SOURCE[0]}" 2>&1)" && rc=0 || rc=$?
|
||||
if [ "$rc" != 0 ] && [[ "$out" == *"semgrep failed"* ]]; then
|
||||
echo "selftest ok: script, semgrep failing -> exit $rc naming semgrep"
|
||||
else
|
||||
echo "SELFTEST FAIL: script with failing semgrep -> rc=$rc (want non-zero naming semgrep)" >&2; selftest_failed=1
|
||||
fi
|
||||
# full_check LABEL SG_SHIM_RC SKIP WANT_RC WANT_CALLED WANT_SUBSTR — SKIP is "unset" or the value to export.
|
||||
full_check() {
|
||||
local label="$1" sg_shim_rc="$2" skip="$3" want_rc="$4" want_called="$5" want_substr="$6" out rc called
|
||||
rm -f "$sg_out/semgrep.called"
|
||||
if [ "$skip" = unset ]; then
|
||||
out="$(env -u ETV_SCAN_SKIP_SEMGREP SG_SHIM_RC="$sg_shim_rc" PATH="$sg_shim:$PATH" ETV_SCAN_OUT="$sg_out" "$BASH" "${BASH_SOURCE[0]}" 2>&1)" && rc=0 || rc=$?
|
||||
else
|
||||
out="$(ETV_SCAN_SKIP_SEMGREP="$skip" SG_SHIM_RC="$sg_shim_rc" PATH="$sg_shim:$PATH" ETV_SCAN_OUT="$sg_out" "$BASH" "${BASH_SOURCE[0]}" 2>&1)" && rc=0 || rc=$?
|
||||
fi
|
||||
if [ -e "$sg_out/semgrep.called" ]; then called=1; else called=0; fi
|
||||
if [ "$rc" = "$want_rc" ] && [ "$called" = "$want_called" ] && [[ "$out" == *"$want_substr"* ]]; then
|
||||
echo "selftest ok: script, $label -> exit $rc, semgrep called=$called"
|
||||
else
|
||||
echo "SELFTEST FAIL: script, $label -> rc=$rc called=$called (want rc=$want_rc called=$want_called, output containing '$want_substr')" >&2
|
||||
selftest_failed=1
|
||||
fi
|
||||
}
|
||||
full_check "semgrep succeeds" 0 unset 0 1 "==> semgrep SAST cross-check"
|
||||
full_check "semgrep succeeds, skip=0" 0 0 0 1 "==> semgrep SAST cross-check"
|
||||
full_check "semgrep fails" 2 unset 2 1 "error: semgrep failed (exit 2)"
|
||||
full_check "explicit opt-out, semgrep present" 0 1 0 0 "semgrep SAST skipped (ETV_SCAN_SKIP_SEMGREP=1)"
|
||||
rm -rf "$sg_dir" "$sg_empty" "$sg_shim" "$sg_out"
|
||||
if [ "$selftest_failed" = 0 ]; then
|
||||
echo "selftest: all ZAP exit classifications OK"; exit 0
|
||||
echo "selftest: all ZAP exit classifications and semgrep-gate cases OK"; exit 0
|
||||
else
|
||||
echo "selftest: FAILURES ABOVE" >&2; exit 1
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user