test(444): deterministic functional-E2E for the playout-build lock 409 + isLocked projection #470
+16
-9
@@ -291,8 +291,8 @@ re-run by hand, turning them into a CI regression net (ersatztv#299). It is the
|
||||
**auth/CSRF/security-stamp** flow (setup-claim → read-gate 401/200 → re-claim 409 → CSRF 403 →
|
||||
login 401/200 → logout 403/204 → post-logout stamp-revocation 401), the **library-scan status
|
||||
contract** (404 unknown / 202 queued / `scan-status` 200), and the **If-Match/412** round-trip on
|
||||
`rerun-collections`. Since ersatztv#363 it also asserts two **lock-contention 409s** that aren't
|
||||
curl-only — see below. A `trap` kills the instance on step exit.
|
||||
`rerun-collections`. Since ersatztv#363 (extended by #444) it also asserts three **lock-contention
|
||||
409s** that aren't curl-only — see below. A `trap` kills the instance on step exit.
|
||||
|
||||
**Lock-contention 409s (ersatztv#363).** The harness now seeds rows the API can't create — a
|
||||
`LibraryPath` and a Jellyfin media-source — directly into the running instance's SQLite DB (via
|
||||
@@ -305,19 +305,26 @@ window is a strict subset of the scan lock's held window), then a second `POST .
|
||||
(deterministic bar a tiny residual TOCTOU gap the multi-second scan covers); (b) the
|
||||
**external-collections "already scanning" 409** — the per-family lock is taken
|
||||
synchronously *before* the 202, so the 202 proves it held, and pointing the seeded source at a
|
||||
non-routable address keeps the background sync hung so the window stays open. No new CI step or
|
||||
dependency: ffmpeg ships in the toolchain image, and python3 was already a harness dependency
|
||||
(`json_field`). The scan flow self-skips (advisory) if ffmpeg is ever absent.
|
||||
non-routable address keeps the background sync hung so the window stays open; and (c) the
|
||||
**playout-build "build in progress" 409 + `isLocked` projection** (#215/#444) — a build is enqueued
|
||||
onto the single-consumer `WorkerService` and the trigger returns *before* the handler locks, so poll
|
||||
`GET /playouts/{id}` until `isLocked:true` (not the accepted trigger), then fire. Seed a Classic Flood
|
||||
schedule over a few short episodes and crank `PlayoutDaysToBuild` (`playout.days_to_build`) so one
|
||||
build is wide enough to observe (~5 days ≈ 43k items ≈ ~1s locally, wider on slower CI); assert `PUT
|
||||
/playouts/{id}` 409, `POST .../playout/reset` 409, the list `isLocked:true`, then post-build the same
|
||||
`PUT` 200s. No new CI step or dependency: ffmpeg ships in the toolchain image, and python3 was already
|
||||
a harness dependency (`json_field`). The scan + build flows self-skip (advisory) if ffmpeg is ever
|
||||
absent, and the build flow also self-skips if the build is never observed locked (never asserts an
|
||||
unproven race).
|
||||
|
||||
**Advisory, by design** (the issue's "keep it a separate job so a functional-E2E flake can't block the
|
||||
unit-test gate"): it is **not** a `needs:` of `build` and **not (yet) a required check**, so a flake
|
||||
blocks nothing. Promote it to a required check / `build` dependency once it's proven reliable — the
|
||||
same staged rollout the `migrations` job used. SQLite is the default provider, so unlike `migrations`
|
||||
it needs **no** DB service container. Runs on PRs and on `main` (regression net); skipped for `v*` tag
|
||||
builds. Still out of scope (tracked as ersatztv#363 follow-ups): the **playout-build lock 409 +
|
||||
`isLocked` projection** (#215) — needs a seeded schedule/playout and a build window wide enough to
|
||||
observe without racing — and the genuinely **UI-interactive Playwright** flows (a separate CI
|
||||
browser-tooling lift).
|
||||
builds. Still out of scope (tracked as an ersatztv#363 follow-up): the genuinely **UI-interactive
|
||||
Playwright** flows (a separate CI browser-tooling lift). The playout-build lock 409 + `isLocked`
|
||||
projection (#215) landed in #444.
|
||||
|
||||
### Docs-only skip (ersatztv#416)
|
||||
|
||||
|
||||
+20
-6
@@ -148,7 +148,7 @@ kill "$PID"
|
||||
each assertion even after a failure and prints a pass/fail summary, exiting non-zero if any failed.
|
||||
|
||||
What it covers. Most assertions are curl-only (no seeded media / ffmpeg / browser); the
|
||||
**lock-contention** section (added by ersatztv#363) is the exception — it seeds DB rows + media.
|
||||
**lock-contention** section (added by ersatztv#363, extended by #444) is the exception — it seeds DB rows + media.
|
||||
- **Legacy→SPA redirects**: a sweep of representative `LegacyUiRedirects` routes 302→`/app/*`, plus
|
||||
the `/api` + `/artwork` never-redirect exemption (asserted as "did not 302 to `/app`", since those
|
||||
4xx from their own handlers — `/api/*` 404s, `/artwork/*` 400s).
|
||||
@@ -156,8 +156,8 @@ What it covers. Most assertions are curl-only (no seeded media / ffmpeg / browse
|
||||
`404`, `scan-status` `200`.
|
||||
- **Optimistic concurrency**: create a collection + a `rerun-collections` targeting it → `GET` emits
|
||||
an `ETag` → `PUT` with a stale `If-Match` `412`, current `200`, malformed `400`.
|
||||
- **Lock contention (409)** — the two `IEntityLocker` 409s, made **deterministic** by firing the
|
||||
racing request only once the lock is *provably* held (never a sleep-and-hope). Both seed rows the
|
||||
- **Lock contention (409)** — the three `IEntityLocker` 409s, made **deterministic** by firing the
|
||||
racing request only once the lock is *provably* held (never a sleep-and-hope). All seed rows the
|
||||
API can't create straight into the running instance's DB via python3's stdlib `sqlite3` (whose
|
||||
busy-timeout retry serializes behind the app's writer):
|
||||
- **library-scan "already scanning"**: seed ~60 tiny ffmpeg clips into the built-in Shows library
|
||||
@@ -172,15 +172,29 @@ What it covers. Most assertions are curl-only (no seeded media / ffmpeg / browse
|
||||
is taken synchronously *before* the `202`, so the `202` proves it held → `collections-scan-status`
|
||||
shows the `jellyfin` family → a second `POST .../scan-collections` is `409`; unknown source `404`.
|
||||
Needs neither ffmpeg nor the scanner.
|
||||
- **playout-build "build in progress"** (#215/#444, the hardest): a build is enqueued onto the
|
||||
single-consumer `WorkerService` channel and the trigger returns *before* `BuildPlayoutHandler`
|
||||
dequeues and `LockPlayout`s (released in its `finally`), so an accepted trigger does **not** prove
|
||||
the lock is held — poll `GET /playouts/{id}` until `isLocked:true`, then fire. Seed a few short
|
||||
ffmpeg episodes → a Collection → a **Classic Flood** schedule → a Classic playout, and crank
|
||||
`PlayoutDaysToBuild` (config `playout.days_to_build`) so a single build is wide enough to observe:
|
||||
5 days ≈ 43k playout items ≈ ~1s locally, wider on slower CI (sized by measurement — item count is
|
||||
`window / item-duration`, so the flood over a handful of short clips scales with the day count;
|
||||
going *too* wide is counter-productive — a 777k-item build saturates the single worker with its
|
||||
post-build gap/overlap jobs). While provably locked: `PUT /playouts/{id}` → 409, `POST
|
||||
.../playout/reset` → 409, and the `GET /playouts` list projection shows `isLocked:true`; after the
|
||||
build the lock clears (`isLocked:false`) and the same `PUT` now `200`s (proving the 409 is
|
||||
lock-specific). Restores the config default afterwards. Self-skips (advisory) without ffmpeg, or if
|
||||
the build is never observed locked (never asserts a race it can't prove it won).
|
||||
- **Auth/CSRF/security-stamp** (mirrors the #295 manual set; runs **last** — setup-claim is one-shot
|
||||
and logout revokes the session): fresh `auth/config` `setupRequired:true` → read-gate
|
||||
401-without-key / 200-with-key → setup-claim 200 → re-claim 409 → session mutation
|
||||
403-without-`X-CSRF` → login 401-then-200 → logout 403-without-CSRF / 204-with → post-logout the
|
||||
same cookie is 401 (rotated security stamp).
|
||||
|
||||
**Still deferred** (ersatztv#363 follow-ups): the **playout-build lock 409 + `isLocked` projection**
|
||||
(#215) — needs a seeded schedule/playout and a build window wide enough to observe without racing —
|
||||
and the genuinely **UI-interactive Playwright** flows (a separate CI browser-tooling lift). When you
|
||||
**Still deferred** (ersatztv#363 follow-up): the genuinely **UI-interactive Playwright** flows (a
|
||||
separate CI browser-tooling lift). The playout-build lock 409 + `isLocked` projection (#215) landed in
|
||||
#444 — see the lock-contention bullet above. When you
|
||||
extend the harness, add the observed contract here and keep the assertions deterministic (probe the
|
||||
real instance first — the initial cut caught that `/artwork/*` 400s where a guess said 404; the #363
|
||||
cut measured the real scan window before sizing the seed).
|
||||
|
||||
+168
-13
@@ -18,20 +18,18 @@
|
||||
# 1. Legacy -> SPA redirect sweep (+ the /api,/artwork never-redirect exemption). [curl-only]
|
||||
# 2. Machine-key data-plane: read-gate, library-scan lifecycle (404/202/scan-status),
|
||||
# optimistic-concurrency If-Match/412. [curl-only]
|
||||
# 3. Lock-contention 409s: scan-in-progress + external-collections-in-progress [seeds DB;
|
||||
# (ersatztv#363) — see that section's header for how each is made deterministic. scan=ffmpeg]
|
||||
# 3. Lock-contention 409s: scan-in-progress + external-collections-in-progress + playout-build [seeds DB;
|
||||
# (ersatztv#363, #444) — see that section's header for how each is made deterministic. scan=ffmpeg]
|
||||
# 4. Auth / CSRF / security-stamp (setup-claim, read-gate, CSRF gate, login, logout+revoke).
|
||||
#
|
||||
# Section 3 is the first that isn't curl-only: it seeds rows the API can't create (a LibraryPath and
|
||||
# a media-source) straight into the running instance's SQLite DB (via python3's stdlib sqlite3, whose
|
||||
# busy-timeout retry serializes behind the app's writer) and, for the scan flow, synthesizes media
|
||||
# with ffmpeg. Both 409s
|
||||
# are made DETERMINISTIC (no racy sleep-and-hope) — see the section header for how. The scan flow is
|
||||
# skipped (advisory) when ffmpeg is absent; the collections flow needs neither ffmpeg nor a scanner.
|
||||
# busy-timeout retry serializes behind the app's writer) and, for the scan/build flows, synthesizes
|
||||
# media with ffmpeg. All three 409s
|
||||
# are made DETERMINISTIC (no racy sleep-and-hope) — see the section header for how. The scan + build
|
||||
# flows are skipped (advisory) when ffmpeg is absent; the collections flow needs neither ffmpeg nor a scanner.
|
||||
#
|
||||
# Deliberately still OUT of scope (tracked as ersatztv#363 follow-ups):
|
||||
# - the playout-build lock 409 + isLocked projection (#215) — needs a seeded schedule/playout and a
|
||||
# build window wide enough to observe without racing,
|
||||
# - the genuinely UI-interactive Playwright (headless) flows — a separate CI browser-tooling lift.
|
||||
#
|
||||
# Exit status: 0 if every assertion passed, 1 if any failed. Assertions keep running after a
|
||||
@@ -179,8 +177,8 @@ expect_status "PUT with the current If-Match -> 200" 200 PUT "$BASE_URL/api/v1/r
|
||||
expect_status "PUT with a malformed If-Match -> 400" 400 PUT "$BASE_URL/api/v1/rerun-collections/$RC_ID" \
|
||||
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -H 'If-Match: not-a-version' -d "$RC_BODY"
|
||||
|
||||
# ---- 3. Lock contention (409): scan-in-progress + collections-in-progress -----------------------
|
||||
# The IEntityLocker 409 contracts the first cut deferred (ersatztv#299 -> #363). Both are made
|
||||
# ---- 3. Lock contention (409): scan-in-progress + collections-in-progress + playout-build --------
|
||||
# The IEntityLocker 409 contracts the first cut deferred (ersatztv#299 -> #363, #444). All three are made
|
||||
# DETERMINISTIC (no sleep-and-hope) by only firing the racing request once the lock is provably held:
|
||||
#
|
||||
# A. Library-scan "already scanning" 409. A real scan must be in flight, so we seed a Shows library
|
||||
@@ -197,12 +195,21 @@ expect_status "PUT with a malformed If-Match -> 400" 400 PUT "$BASE_URL/api/v1/r
|
||||
# address keeps the background sync hung so the window stays wide open. GET
|
||||
# /media-sources/collections-scan-status corroborates.
|
||||
#
|
||||
# C. Playout-build "build in progress" 409 + isLocked projection (#215). Hardest of the three: the
|
||||
# build is enqueued onto the single-consumer WorkerService channel and the trigger returns BEFORE
|
||||
# BuildPlayoutHandler dequeues + LockPlayout (released in its finally -- BuildPlayoutHandler.cs), so
|
||||
# an accepted trigger does NOT prove the lock is held. We POLL GET /playouts/{id} until isLocked:true,
|
||||
# then fire the racing mutations while it is provably held. The build is made observable by seeding a
|
||||
# Classic Flood schedule over a few short episodes and cranking PlayoutDaysToBuild (config
|
||||
# 'playout.days_to_build'): a single 5-day build is ~43k playout items ~= ~1s here, and CI runners are
|
||||
# slower so the window only widens (#444 measurement). Restores the config default afterwards.
|
||||
#
|
||||
# Seeding writes rows the API can't create (LibraryPath, media-source) directly into the instance's
|
||||
# SQLite DB via python3's stdlib sqlite3 (already a hard dep of this harness); the connection's
|
||||
# busy-timeout retry serializes this second writer behind the app. Runs on the machine key, before the
|
||||
# auth section (which claims the admin) -- these never claim it. Flow A is skipped (advisory) without
|
||||
# ffmpeg; Flow B needs neither ffmpeg nor a scanner.
|
||||
section "Lock contention (409): scan-in-progress + collections-in-progress"
|
||||
# auth section (which claims the admin) -- these never claim it. Flows A + C are skipped (advisory)
|
||||
# without ffmpeg; Flow B needs neither ffmpeg nor a scanner.
|
||||
section "Lock contention (409): scan-in-progress + collections-in-progress + playout-build"
|
||||
|
||||
DB="$CONFIG_DIR/ersatztv.sqlite3"
|
||||
|
||||
@@ -307,6 +314,154 @@ else
|
||||
else
|
||||
bad "could not seed a Jellyfin media source row for the collections-lock flow"
|
||||
fi
|
||||
|
||||
# --- Flow C: playout-build lock 409 + isLocked projection (#215/#444) -----------------------------
|
||||
# Unlike Flows A/B, a playout build is enqueued onto the single-consumer WorkerService channel and the
|
||||
# trigger returns BEFORE BuildPlayoutHandler dequeues and calls entityLocker.LockPlayout (released in its
|
||||
# finally). So an accepted trigger does NOT prove the lock is held -- there is a real enqueue->dequeue
|
||||
# race. We make it DETERMINISTIC by POLLING GET /playouts/{id} until isLocked:true (never sleep-and-hope),
|
||||
# then firing the racing mutations while the lock is provably held. The build is made wide enough to
|
||||
# observe by seeding a Classic Flood schedule over a few short episodes and cranking PlayoutDaysToBuild:
|
||||
# a single 5-day build is ~43k playout items ~= ~1s here (#444 measurement); CI runners are slower, so
|
||||
# the window only widens. Needs ffmpeg (to synthesize scannable media) -> advisory skip without it.
|
||||
playout_locked() { api "$BASE_URL/api/v1/playouts/$1" | python3 -c "import json,sys
|
||||
try: print(json.load(sys.stdin).get('isLocked'))
|
||||
except Exception: print('')"; }
|
||||
# isLocked for PLID as seen in the GET /playouts LIST projection (the other place #215 stamps it).
|
||||
list_locked() { api "$BASE_URL/api/v1/playouts?pageSize=100" | PLID="$1" python3 -c "import json,sys,os
|
||||
try:
|
||||
d=json.load(sys.stdin); pid=int(os.environ['PLID'])
|
||||
print(next((str(p.get('isLocked')) for p in d.get('page',[]) if p.get('id')==pid),'MISSING'))
|
||||
except Exception: print('ERR')"; }
|
||||
# PlayoutDaysToBuild is ConfigElementKey 'playout.days_to_build' (default 2); widen it to size the build.
|
||||
set_days_to_build() { python3 - "$DB" "$1" <<'PY'
|
||||
import sqlite3, sys
|
||||
con = sqlite3.connect(sys.argv[1], timeout=10) # busy-timeout retry serializes this second writer
|
||||
con.execute("DELETE FROM ConfigElement WHERE Key='playout.days_to_build'")
|
||||
if sys.argv[2] != 'CLEAR':
|
||||
con.execute("INSERT INTO ConfigElement (Key, Value) VALUES ('playout.days_to_build', ?)", (sys.argv[2],))
|
||||
con.commit()
|
||||
PY
|
||||
}
|
||||
# expect_409_while_locked "desc" METHOD URL [curl args...] -- assert a build-locked mutation returns 409,
|
||||
# but if the build RELEASED before our call landed (a 2xx AND the playout is now confirmed unlocked),
|
||||
# advisory-skip instead of failing: a lost race is not a bug, and this flow never reds on a race it can't
|
||||
# prove it won. A 2xx while the playout is STILL locked is a real defect and still fails. ($BL_PL in scope.)
|
||||
expect_409_while_locked() {
|
||||
local desc="$1" method="$2" url="$3"; shift 3
|
||||
local got; got="$(status_of "$method" "$url" "$@")"
|
||||
if [ "$got" = "409" ]; then
|
||||
ok "$desc (=409)"
|
||||
elif { [ "$got" = "200" ] || [ "$got" = "202" ]; } && [ "$(playout_locked "$BL_PL")" = "False" ]; then
|
||||
skip "$desc — build released before the racing call landed (got $got, now unlocked) -> advisory"
|
||||
else
|
||||
bad "$desc — expected 409, got $got"
|
||||
fi
|
||||
}
|
||||
|
||||
if ! command -v ffmpeg >/dev/null 2>&1; then
|
||||
skip "ffmpeg not found -> skipping the playout-build lock 409 flow (advisory; needs seeded media)"
|
||||
else
|
||||
BL_ROOT="$(mktemp -d)/tv"; BL_SEASON="$BL_ROOT/Build Lock E2E/Season 01"; mkdir -p "$BL_SEASON"
|
||||
BL_SRC="$(mktemp -d)/bl.mkv"
|
||||
if ffmpeg -y -loglevel error -f lavfi -i "testsrc=duration=10:size=160x120:rate=5" \
|
||||
-c:v libx264 -pix_fmt yuv420p "$BL_SRC" >/dev/null 2>&1; then
|
||||
# A few 10s episodes: the flood item repeats the collection to fill the whole build window, so item
|
||||
# count (hence build duration) is window/item-duration -- 3 distinct items are plenty to flood with.
|
||||
for n in 1 2 3; do cp "$BL_SRC" "$BL_SEASON/Build Lock E2E - s01e0$n.mkv"; done
|
||||
if [ -z "$(seed_library_path "$BL_ROOT" 2)" ]; then
|
||||
bad "could not seed a LibraryPath row for the build-lock flow (library 2)"
|
||||
else
|
||||
# Wait for any in-progress library-2 scan (Flow A's) to clear so our scan won't 409, then scan and
|
||||
# wait for OUR titled episodes to land (filtered by title, so Flow A's 'Show Lock' items don't leak in).
|
||||
for _ in $(seq 1 200); do
|
||||
api "$BASE_URL/api/v1/libraries/scan-status" | grep -q '"libraryId":2,' || break
|
||||
sleep 0.1
|
||||
done
|
||||
api -X POST "$BASE_URL/api/v1/libraries/2/scan" -H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -d '{}' -o /dev/null -w ''
|
||||
BL_EPIDS=""
|
||||
for _ in $(seq 1 200); do
|
||||
BL_EPIDS=$(api "$BASE_URL/api/v1/library/browse?mediaType=Episode&pageSize=200" | python3 -c "import json,sys
|
||||
d=json.load(sys.stdin)
|
||||
print(','.join(str(i['mediaItemId']) for i in d.get('page',[]) if 'Build Lock E2E' in (i.get('title') or '')))")
|
||||
[ -n "$BL_EPIDS" ] && break; sleep 0.15
|
||||
done
|
||||
if [ -z "$BL_EPIDS" ]; then
|
||||
skip "build-lock flow: seeded episodes never scanned in within ~30s -> not asserting (advisory)"
|
||||
else
|
||||
ok "seeded + scanned build-lock episodes (ids: $BL_EPIDS)"
|
||||
# Collection over the episodes -> a Classic Flood schedule item -> a Classic playout on a channel.
|
||||
BL_COLL_JSON="$(mktemp)"; api -X POST "$BASE_URL/api/v1/collections" -d '{"name":"E2E BuildLock Collection"}' -o "$BL_COLL_JSON" -w ''
|
||||
BL_COLL="$(json_field "$BL_COLL_JSON" id)"
|
||||
api -X POST "$BASE_URL/api/v1/collections/$BL_COLL/items" -d "{\"episodeIds\":[$BL_EPIDS]}" -o /dev/null -w ''
|
||||
BL_SCH_JSON="$(mktemp)"; api -X POST "$BASE_URL/api/v1/schedules" -d '{"name":"E2E BuildLock Schedule"}' -o "$BL_SCH_JSON" -w ''
|
||||
BL_SCH="$(json_field "$BL_SCH_JSON" id)"
|
||||
api -X PUT "$BASE_URL/api/v1/schedules/$BL_SCH/items" -o /dev/null -w '' -d "{\"items\":[{
|
||||
\"startType\":\"Dynamic\",\"playoutMode\":\"Flood\",\"collectionType\":\"Collection\",\"collectionId\":$BL_COLL,
|
||||
\"playbackOrder\":\"Chronological\",\"marathonGroupBy\":\"None\",\"marathonShuffleGroups\":false,
|
||||
\"marathonShuffleItems\":false,\"fillWithGroupMode\":\"None\",\"multipleMode\":\"Count\",\"multipleCount\":\"\",
|
||||
\"tailMode\":\"None\",\"customTitle\":\"\",\"guideMode\":\"Normal\",\"searchTitle\":\"\",\"searchQuery\":\"\",
|
||||
\"preferredAudioLanguageCode\":\"\",\"preferredAudioTitle\":\"\",\"preferredSubtitleLanguageCode\":\"\",
|
||||
\"watermarkIds\":[],\"graphicsElementIds\":[]}]}"
|
||||
BL_CH_JSON="$(mktemp)"; api -X POST "$BASE_URL/api/v1/channels" -o "$BL_CH_JSON" -w '' -d '{
|
||||
"name":"E2E BuildLock Channel","number":"79999","group":"E2E","categories":"","ffmpegProfileId":1,
|
||||
"streamSelectorMode":"Default","streamSelector":"","playoutSource":"Generated","playoutMode":"Continuous",
|
||||
"streamingMode":"TransportStream","subtitleMode":"None","musicVideoCreditsMode":"None","songVideoMode":"Default",
|
||||
"transcodeMode":"OnDemand","idleBehavior":"StopOnDisconnect","isEnabled":true,"showInEpg":true}'
|
||||
BL_CH="$(json_field "$BL_CH_JSON" id)"
|
||||
set_days_to_build 5
|
||||
# Creating a Classic playout enqueues a Reset build (CreateClassicPlayoutHandler) -- the build we observe.
|
||||
BL_PL_JSON="$(mktemp)"; api -X POST "$BASE_URL/api/v1/playouts" -o "$BL_PL_JSON" -w '' \
|
||||
-d "{\"channelId\":$BL_CH,\"scheduleKind\":\"Classic\",\"programScheduleId\":$BL_SCH}"
|
||||
BL_PL="$(json_field "$BL_PL_JSON" id)"
|
||||
if [ -z "$BL_PL" ]; then
|
||||
bad "could not create the Classic playout for the build-lock flow (body: $(cat "$BL_PL_JSON"))"
|
||||
else
|
||||
bl_locked=""
|
||||
for _ in $(seq 1 300); do
|
||||
[ "$(playout_locked "$BL_PL")" = "True" ] && { bl_locked=1; break; }
|
||||
sleep 0.05
|
||||
done
|
||||
if [ -z "$bl_locked" ]; then
|
||||
# Never observed locked (build finished faster than we sampled): don't assert a race we can't
|
||||
# prove we won -- advisory skip, matching Flow A's discipline.
|
||||
skip "playout build never observed locked within ~15s -> not asserting the 409 (advisory; build too fast)"
|
||||
else
|
||||
ok "playout $BL_PL build observed locked (GET /playouts/{id} isLocked:true projection)"
|
||||
# While the lock is provably held: every id-keyed mutation 409s and both isLocked projections
|
||||
# agree. Each racing call is guarded (expect_409_while_locked / the list re-check below) so a
|
||||
# build that finishes mid-flight degrades to an advisory skip, never a false red.
|
||||
expect_409_while_locked "PUT /playouts/$BL_PL while build in progress -> 409" PUT "$BASE_URL/api/v1/playouts/$BL_PL" \
|
||||
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -d '{"dailyRebuildTime":null,"scheduleFile":null}'
|
||||
expect_409_while_locked "POST /channels/$BL_CH/playout/reset while build in progress -> 409" POST "$BASE_URL/api/v1/channels/$BL_CH/playout/reset?mode=Reset" \
|
||||
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json'
|
||||
bl_list="$(list_locked "$BL_PL")"
|
||||
if [ "$bl_list" = "True" ]; then
|
||||
ok "GET /playouts list projection shows isLocked:true for $BL_PL"
|
||||
elif [ "$bl_list" = "False" ]; then
|
||||
skip "GET /playouts list projection already isLocked:false (build released mid-flight) -> advisory"
|
||||
else
|
||||
bad "GET /playouts list projection error for $BL_PL (got '$bl_list')"
|
||||
fi
|
||||
# Once the build completes the lock clears: isLocked flips false and the SAME PUT now succeeds,
|
||||
# proving the 409 above was lock-specific (not an always-failing call).
|
||||
for _ in $(seq 1 600); do [ "$(playout_locked "$BL_PL")" = "False" ] && break; sleep 0.05; done
|
||||
if [ "$(playout_locked "$BL_PL")" = "False" ]; then
|
||||
ok "playout $BL_PL isLocked:false after the build completes"
|
||||
else
|
||||
bad "playout $BL_PL never unlocked after the build"
|
||||
fi
|
||||
expect_status "PUT /playouts/$BL_PL after the build (unlocked) -> 200" 200 PUT "$BASE_URL/api/v1/playouts/$BL_PL" \
|
||||
-H "X-Api-Key: $API_KEY" -H 'Content-Type: application/json' -d '{"dailyRebuildTime":null,"scheduleFile":null}'
|
||||
fi
|
||||
fi
|
||||
set_days_to_build CLEAR # restore the default build horizon
|
||||
fi
|
||||
fi
|
||||
else
|
||||
bad "ffmpeg present but failed to synthesize the seed clip for the build-lock flow"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---- 4. Auth / CSRF / security-stamp flow -------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user