Files
ersatztv/scripts/e2e-functional.sh
T
timothyandClaude Opus 4.8 0d7803079c
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 9s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 10s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 5m54s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 17s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 18s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 4m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 6m42s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 4m22s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 5m38s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 6m2s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m41s
ci: add advisory functional-E2E curl harness (fixes #299)
Codify the manual live-E2E curl flows sessions have been re-running by hand
into a CI regression net: a new `functional-e2e` job boots the app from source
(via scripts/e2e-local.sh, parameterized for Release) and drives scripts/
e2e-functional.sh against it.

First-cut contracts (all curl-only, deterministic, no seeded media/ffmpeg/browser):
- legacy->SPA redirect sweep + the /api,/artwork never-redirect exemption
- auth/CSRF/security-stamp flow (setup-claim, read-gate, CSRF, login, logout+revoke)
- library-scan status contract (404/202/scan-status)
- optimistic-concurrency If-Match/412 round-trip

Advisory by design (separate job, not a `build` dependency, not a required
check) so a functional-E2E flake can't block the unit-test gate; promote once
proven, mirroring the migrations-job rollout. SQLite default -> no DB service.

Deferred to #299 follow-ups (need scanner+seeded media or a browser to be
deterministic): the racy 409 re-trigger, playout-build lock 409, Playwright UI
flows.

Assertions verified 30/30 green against a real Release-built instance; caught
/artwork/* returning 400 (not the 404 a static read suggested).

Docs updated same PR: docs/ci-cd.md (new job), docs/e2e-local.md (harness),
docs/decisions.md (append-only).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 13:49:33 +02:00

218 lines
12 KiB
Bash
Executable File

#!/usr/bin/env bash
# scripts/e2e-functional.sh — drive the manual live-E2E functional flows against a RUNNING
# ErsatzTV instance and assert their HTTP contracts. This is the codified, automatable half of the
# ad-hoc curl scenarios sessions have been running by hand (ersatztv#299); it runs both locally and
# in CI (the `functional-e2e` job in .gitea/workflows/docker-build.yml).
#
# It does NOT boot the app — pair it with scripts/e2e-local.sh (which builds/launches and prints the
# CONFIG_DIR), or point it at any already-running instance:
#
# scripts/e2e-functional.sh [BASE_URL] [CONFIG_DIR]
#
# BASE_URL default http://localhost:8409 (the app's UI+API port; ETV_UI_PORT)
# CONFIG_DIR the instance's config folder — REQUIRED, so we can read its machine api key
# ($CONFIG_DIR/api.key). The /api surface is fail-closed (Api:RequireKeyForReads
# defaults true), so every /api call sends X-Api-Key.
#
# Scope (all curl-only, deterministic, no seeded media / ffmpeg / browser needed):
# 1. Legacy -> SPA redirect sweep (+ the /api,/artwork never-redirect exemption).
# 2. Auth / CSRF / security-stamp flow (setup-claim, read-gate, CSRF gate, login, logout+revoke).
# 3. Library scan lifecycle status-code contract (404 / 202 / scan-status).
# 4. Optimistic-concurrency If-Match / 412 round-trip.
#
# Deliberately OUT of scope for this first cut (need seeded media + the scanner subprocess, or a
# browser, to be deterministic — tracked as ersatztv#299 follow-ups):
# - the 409 "already scanning" re-trigger (racy without a long-running scan),
# - the playout-build lock 409,
# - the genuinely UI-interactive Playwright flows.
#
# Exit status: 0 if every assertion passed, 1 if any failed. Assertions keep running after a
# failure so one run reports the full picture.
set -uo pipefail
BASE_URL="${1:-http://localhost:8409}"
CONFIG_DIR="${2:-}"
if [ -z "$CONFIG_DIR" ]; then
echo "error: CONFIG_DIR (arg 2) is required — needed to read the machine api key ($CONFIG_DIR/api.key)." >&2
echo "usage: scripts/e2e-functional.sh [BASE_URL] CONFIG_DIR" >&2
exit 2
fi
KEY_FILE="$CONFIG_DIR/api.key"
if [ ! -f "$KEY_FILE" ]; then
echo "error: $KEY_FILE not found. Is the instance running with ETV_CONFIG_FOLDER=$CONFIG_DIR?" >&2
exit 2
fi
API_KEY="$(cat "$KEY_FILE")"
PASS=0
FAIL=0
FAILURES=()
# ---- assertion helpers -------------------------------------------------------------------------
# ok/bad: record a single assertion result.
ok() { PASS=$((PASS + 1)); printf ' \033[32mPASS\033[0m %s\n' "$1"; }
bad() { FAIL=$((FAIL + 1)); FAILURES+=("$1"); printf ' \033[31mFAIL\033[0m %s\n' "$1"; }
# status_of METHOD URL [extra curl args...] -> prints the HTTP status code.
status_of() {
local method="$1" url="$2"; shift 2
curl -s -o /dev/null -w '%{http_code}' -X "$method" "$@" "$url"
}
# expect_status "desc" EXPECTED METHOD URL [extra curl args...]
expect_status() {
local desc="$1" expected="$2" method="$3" url="$4"; shift 4
local got; got="$(status_of "$method" "$url" "$@")"
if [ "$got" = "$expected" ]; then ok "$desc (=$expected)"; else bad "$desc — expected $expected, got $got [$method $url]"; fi
}
# api(): curl against the /api surface with the machine key + JSON content type.
api() { curl -s -H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' "$@"; }
# json_field FILE FIELD -> value of a top-level JSON field (via python3, always present here).
json_field() { python3 -c "import json,sys; print(json.load(open('$1')).get('$2',''))"; }
section() { printf '\n\033[1m== %s ==\033[0m\n' "$1"; }
echo "Functional E2E against $BASE_URL (config: $CONFIG_DIR)"
# ---- 1. Legacy -> SPA redirect sweep ------------------------------------------------------------
# The legacy branch 302s GET requests to their /app equivalent; /api, /artwork, /docs, /openapi are
# NEVER redirected (they 4xx from their own handlers/fallback). See ErsatzTV/LegacyUiRedirects.cs +
# Startup.cs MapFallback, and docs/blazor-route-parity.md.
section "Legacy -> SPA redirects"
# redirects_to "src" "expected /app location suffix"
redirects_to() {
local src="$1" want="$2"
local code loc
code="$(curl -s -o /dev/null -w '%{http_code}' "$BASE_URL$src")"
loc="$(curl -s -o /dev/null -w '%{redirect_url}' "$BASE_URL$src")"
if [ "$code" = "302" ] && [ "$loc" = "$BASE_URL$want" ]; then
ok "$src -> 302 $want"
else
bad "$src — expected 302 -> $BASE_URL$want, got $code -> ${loc:-<none>}"
fi
}
redirects_to "/" "/app"
redirects_to "/channels" "/app/channels"
redirects_to "/channels/5" "/app/edit-channel/5"
redirects_to "/settings/ffmpeg" "/app/settings/streaming"
redirects_to "/media/movies" "/app/media?kind=movies"
redirects_to "/media/sources/plex/3/libraries" "/app/libraries/plex/3/sync"
# The exempt prefixes must NOT redirect to /app — they 4xx from their own handlers (an unknown
# /api path 404s; /artwork rejects a non-artwork path 400). Assert only "did not 302 to /app".
never_redirects() {
local src="$1"
local code loc
code="$(curl -s -o /dev/null -w '%{http_code}' "$BASE_URL$src")"
loc="$(curl -s -o /dev/null -w '%{redirect_url}' "$BASE_URL$src")"
if [ "$code" != "302" ] && [ -z "$loc" ] && [ "${code:0:1}" = "4" ]; then
ok "$src -> not redirected (${code})"
else
bad "$src — expected a non-redirect 4xx, got $code -> ${loc:-<none>}"
fi
}
never_redirects "/api/does-not-exist"
never_redirects "/artwork/does-not-exist"
# ---- 2. Machine-key data-plane contracts (scan lifecycle + If-Match/412) ------------------------
# These use the machine key (X-Api-Key), which is independent of the session and does NOT claim the
# local admin — so the fresh-config setup-claim flow in section 3 still sees setupRequired:true.
section "Read gate (Api:RequireKeyForReads = true)"
expect_status "GET /api/v1/channels without a key -> 401" 401 GET "$BASE_URL/api/v1/channels"
expect_status "GET /api/v1/channels with the machine key -> 200" 200 GET "$BASE_URL/api/v1/channels" \
-H "X-Api-Key: $API_KEY"
section "Library scan lifecycle contract"
# Create an empty local library (no LibraryPath) — enough to exercise the enqueue/status contract
# without any media on disk (QueueLibraryScanByLibraryIdHandler never inspects paths).
LIB_JSON="$(mktemp)"
api -X POST "$BASE_URL/api/v1/libraries/local" -d '{"name":"E2E Functional Library"}' -o "$LIB_JSON" -w ''
LIB_ID="$(json_field "$LIB_JSON" id)"
if [ -n "$LIB_ID" ]; then ok "created local library id=$LIB_ID"; else bad "could not create local library (body: $(cat "$LIB_JSON"))"; fi
expect_status "POST /api/v1/libraries/$LIB_ID/scan -> 202 (queued)" 202 POST "$BASE_URL/api/v1/libraries/$LIB_ID/scan" \
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -d '{}'
expect_status "POST /api/v1/libraries/99999/scan -> 404 (unknown library)" 404 POST "$BASE_URL/api/v1/libraries/99999/scan" \
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -d '{}'
expect_status "GET /api/v1/libraries/scan-status -> 200" 200 GET "$BASE_URL/api/v1/libraries/scan-status" \
-H "X-Api-Key: $API_KEY"
section "Optimistic concurrency (If-Match / 412)"
# rerun-collections is the reference concurrency-aware editor: GET emits an ETag, PUT enforces
# If-Match. It needs an existing collection to target, so create one first.
COLL_JSON="$(mktemp)"
api -X POST "$BASE_URL/api/v1/collections" -d '{"name":"E2E Functional Collection"}' -o "$COLL_JSON" -w ''
COLL_ID="$(json_field "$COLL_JSON" id)"
RC_JSON="$(mktemp)"
api -X POST "$BASE_URL/api/v1/rerun-collections" -o "$RC_JSON" -w '' -d "{
\"name\":\"E2E Functional Rerun\",\"collectionType\":\"Collection\",\"selectedId\":$COLL_ID,
\"firstRunPlaybackOrder\":\"Chronological\",\"rerunPlaybackOrder\":\"Chronological\"}"
RC_ID="$(json_field "$RC_JSON" id)"
if [ -n "$RC_ID" ]; then ok "created rerun-collection id=$RC_ID (targets collection $COLL_ID)"; else bad "could not create rerun-collection (body: $(cat "$RC_JSON"))"; fi
# GET must surface the current version as a quoted-integer ETag.
ETAG="$(api -D - -o /dev/null "$BASE_URL/api/v1/rerun-collections/$RC_ID" | awk 'tolower($1)=="etag:"{print $2}' | tr -d '\r')"
if [ -n "$ETAG" ]; then ok "GET rerun-collection $RC_ID surfaces ETag $ETAG"; else bad "GET rerun-collection $RC_ID has no ETag header"; fi
RC_BODY="{\"name\":\"E2E Functional Rerun v2\",\"selectedId\":$COLL_ID,\"firstRunPlaybackOrder\":\"Chronological\",\"rerunPlaybackOrder\":\"Chronological\"}"
# A well-formed but non-current version must be rejected 412 (999999 can never be the live version here).
expect_status "PUT with a stale If-Match -> 412" 412 PUT "$BASE_URL/api/v1/rerun-collections/$RC_ID" \
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -H 'If-Match: "999999"' -d "$RC_BODY"
# The current ETag must be accepted.
expect_status "PUT with the current If-Match -> 200" 200 PUT "$BASE_URL/api/v1/rerun-collections/$RC_ID" \
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -H "If-Match: $ETAG" -d "$RC_BODY"
# A malformed (non-integer) If-Match is a client error.
expect_status "PUT with a malformed If-Match -> 400" 400 PUT "$BASE_URL/api/v1/rerun-collections/$RC_ID" \
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -H 'If-Match: not-a-version' -d "$RC_BODY"
# ---- 3. Auth / CSRF / security-stamp flow -------------------------------------------------------
# Runs last: setup-claim is a one-shot on a fresh config and logout revokes the session. All prior
# sections used the machine key, which does not claim the admin, so setupRequired is still true here.
section "Auth: setup-claim, CSRF gate, login, logout + stamp revocation"
CFG_JSON="$(mktemp)"
curl -s "$BASE_URL/api/v1/auth/config" -o "$CFG_JSON"
if [ "$(json_field "$CFG_JSON" setupRequired)" = "True" ]; then ok "fresh config: setupRequired=true"; else bad "expected setupRequired=true on a fresh config (body: $(cat "$CFG_JSON"))"; fi
JAR="$(mktemp)"
# Claim the local admin (setup is CSRF-gated like every mutation).
expect_status "POST /api/v1/auth/setup -> 200 (claims admin, issues session)" 200 POST "$BASE_URL/api/v1/auth/setup" \
-c "$JAR" -H 'Content-Type: application/json' -H 'X-CSRF: 1' -d '{"username":"admin","password":"e2e-pass-1234"}'
curl -s "$BASE_URL/api/v1/auth/config" -o "$CFG_JSON"
if [ "$(json_field "$CFG_JSON" setupRequired)" = "False" ]; then ok "after claim: setupRequired=false"; else bad "expected setupRequired=false after claim (body: $(cat "$CFG_JSON"))"; fi
expect_status "POST /api/v1/auth/setup again -> 409 (already configured)" 409 POST "$BASE_URL/api/v1/auth/setup" \
-H 'Content-Type: application/json' -H 'X-CSRF: 1' -d '{"username":"admin2","password":"e2e-pass-1234"}'
# CSRF gate on a general /api session mutation: a POST with the session cookie but no X-CSRF is 403.
expect_status "session mutation without X-CSRF -> 403" 403 POST "$BASE_URL/api/v1/collections" \
-b "$JAR" -H 'Content-Type: application/json' -d '{"name":"should-be-csrf-blocked"}'
# Login: wrong password 401, correct 200 (fresh cookie jar).
expect_status "login with wrong password -> 401" 401 POST "$BASE_URL/api/v1/auth/login" \
-H 'Content-Type: application/json' -H 'X-CSRF: 1' -d '{"username":"admin","password":"nope"}'
JAR2="$(mktemp)"
expect_status "login with correct password -> 200" 200 POST "$BASE_URL/api/v1/auth/login" \
-c "$JAR2" -H 'Content-Type: application/json' -H 'X-CSRF: 1' -d '{"username":"admin","password":"e2e-pass-1234"}'
# Logout is CSRF-gated; and after it, the same cookie is invalidated by the rotated security stamp.
expect_status "authed GET before logout -> 200" 200 GET "$BASE_URL/api/v1/auth/machine-key" -b "$JAR2"
expect_status "logout without X-CSRF -> 403" 403 POST "$BASE_URL/api/v1/auth/logout" -b "$JAR2"
expect_status "logout with X-CSRF -> 204" 204 POST "$BASE_URL/api/v1/auth/logout" -b "$JAR2" -H 'X-CSRF: 1'
expect_status "authed GET after logout (stamp revoked) -> 401" 401 GET "$BASE_URL/api/v1/auth/machine-key" -b "$JAR2"
# ---- summary ------------------------------------------------------------------------------------
printf '\n\033[1m== Summary ==\033[0m\n'
printf 'passed: %d failed: %d\n' "$PASS" "$FAIL"
if [ "$FAIL" -ne 0 ]; then
printf '\033[31mfailed assertions:\033[0m\n'
for f in "${FAILURES[@]}"; do printf ' - %s\n' "$f"; done
exit 1
fi
echo "all functional-E2E assertions passed."