test(363): functional-E2E harness — deterministic scan-lock + collections-lock 409 flows #443

Merged
timothy merged 3 commits from feat/363-e2e-deferred-flows into main 2026-07-18 16:05:25 +02:00
Owner

Closes part of #363 (fixes #363; the two remaining flows are spun out to follow-ups below).

Extends scripts/e2e-functional.sh with two IEntityLocker 409 contracts the first cut (#299) deferred as "racy", made deterministic by firing the racing request only once the lock is provably held (no sleep-and-hope):

  • library-scan "already scanning" 409 — seed ~60 tiny ffmpeg clips into the built-in Shows library so the scanner subprocess runs a few seconds → POST .../scan?deep=true 202 → poll GET /libraries/scan-status until the library shows active (that window is a strict subset of the scan lock's held window: StartScan after LockLibrary, EndScan before UnlockLibrary) → second POST .../scan = guaranteed 409. Self-skips (advisory) if ffmpeg is absent.
  • external-collections "already scanning" 409 — seed a Jellyfin media-source row at a non-routable address so the sync hangs and the per-family lock stays held; the lock is taken synchronously before the 202, so the 202 proves it held. collections-scan-status corroborates; unknown source 404.

Seeding uses python3's stdlib sqlite3 (already a harness dep) to insert rows the API can't create (LibraryPath, media-source); WAL mode tolerates the second writer. No new CI step/dependency — ffmpeg ships in the toolchain image.

Verification: 4/4 fresh-instance harness runs green (38/38 assertions), lock section deterministic every time.

Deferred to follow-ups (the two harder flows): playout-build lock 409 + isLocked projection (#215), and the UI-interactive Playwright flows.

Docs updated same PR: docs/e2e-local.md, docs/ci-cd.md, and the functional-e2e job comment in .gitea/workflows/docker-build.yml.

No .cs touched (no BOM/format exposure); no /api/* change (no OpenAPI regen).

Closes part of #363 (fixes #363; the two remaining flows are spun out to follow-ups below). Extends `scripts/e2e-functional.sh` with two `IEntityLocker` 409 contracts the first cut (#299) deferred as "racy", made **deterministic** by firing the racing request only once the lock is *provably* held (no sleep-and-hope): - **library-scan "already scanning" 409** — seed ~60 tiny ffmpeg clips into the built-in Shows library so the scanner subprocess runs a few seconds → `POST .../scan?deep=true` 202 → poll `GET /libraries/scan-status` until the library shows active (that window is a strict subset of the scan lock's held window: `StartScan` after `LockLibrary`, `EndScan` before `UnlockLibrary`) → second `POST .../scan` = guaranteed 409. Self-skips (advisory) if ffmpeg is absent. - **external-collections "already scanning" 409** — seed a Jellyfin media-source row at a non-routable address so the sync hangs and the per-family lock stays held; the lock is taken synchronously *before* the 202, so the 202 proves it held. `collections-scan-status` corroborates; unknown source 404. Seeding uses python3's stdlib `sqlite3` (already a harness dep) to insert rows the API can't create (LibraryPath, media-source); WAL mode tolerates the second writer. **No new CI step/dependency** — ffmpeg ships in the toolchain image. **Verification:** 4/4 fresh-instance harness runs green (38/38 assertions), lock section deterministic every time. **Deferred to follow-ups** (the two harder flows): playout-build lock 409 + `isLocked` projection (#215), and the UI-interactive Playwright flows. **Docs updated same PR:** `docs/e2e-local.md`, `docs/ci-cd.md`, and the `functional-e2e` job comment in `.gitea/workflows/docker-build.yml`. No `.cs` touched (no BOM/format exposure); no `/api/*` change (no OpenAPI regen).
timothy added 1 commit 2026-07-18 15:18:30 +02:00
test(363): functional-E2E harness — add deterministic scan-lock + collections-lock 409 flows
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 10s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 11s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 28s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m24s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Failing after 3m45s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m16s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m46s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
8a85f9ddb5
Extends scripts/e2e-functional.sh with the two IEntityLocker 409 contracts the
first cut (ersatztv#299) deferred as "racy", made DETERMINISTIC by firing the
racing request only once the lock is provably held (no sleep-and-hope):

- library-scan "already scanning" 409: seed ~60 tiny ffmpeg clips into the
  built-in Shows library so the scanner subprocess runs a few seconds, poll
  GET /libraries/scan-status until the library is active (that window is a
  strict subset of the scan lock's held window — StartScan after LockLibrary,
  EndScan before UnlockLibrary), then a second POST .../scan is a guaranteed
  409. Self-skips (advisory) when ffmpeg is absent.
- external-collections "already scanning" 409: seed a Jellyfin media-source row
  pointing at a non-routable address so the background sync hangs and the
  per-family lock stays held; the lock is taken synchronously before the 202,
  so the 202 proves it held. collections-scan-status corroborates; unknown
  source 404.

Seeding uses python3's stdlib sqlite3 (already a harness dep) to insert rows the
API can't create (LibraryPath, media-source); WAL mode tolerates the second
writer. No new CI step/dependency — ffmpeg ships in the toolchain image.
Verified: 4/4 fresh-instance runs green (38/38), lock section deterministic.

Still deferred to #363 follow-ups: the playout-build lock 409 + isLocked
projection (#215) and the UI-interactive Playwright flows.

Docs updated same PR: docs/e2e-local.md, docs/ci-cd.md, the functional-e2e
job comment in .gitea/workflows/docker-build.yml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
timothy added 1 commit 2026-07-18 15:33:51 +02:00
review(363): robustness + wording fixes from cold review
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 6s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 11s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 9s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 20s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m47s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m11s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m48s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
4345180a56
- check seed_library_path succeeded (print lastrowid) so a silent seeding
  failure surfaces as a FAIL instead of degrading Flow A to an advisory skip
  with no diagnostic (Medium)
- anchor the scan-status match to '"libraryId":2,' so it can't substring-match
  "libraryId":20/23 if the suite ever creates more libraries (Low)
- drop the no-op ?deep=true (local scans always ForceScan; deep only affects
  Plex/Jellyfin/Emby) + note why (Low)
- soften "guaranteed 409" for the scan flow to note the tiny residual TOCTOU
  gap the multi-second scan covers; Flow B stays race-free by construction (Low)
- correct the "WAL tolerates a second writer" wording to the real reason (the
  busy-timeout retry serializes the writer) in the script + both docs (Nit)
- use TEST-NET-1 192.0.2.1 (RFC 5737) instead of RFC1918 10.255.255.1 for the
  non-routable Jellyfin address (Nit)

Re-verified: fresh-instance harness runs green (38/38), lock section
deterministic. Functional E2E CI job already green on the prior head.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
timothy added 1 commit 2026-07-18 15:44:51 +02:00
docs(363): sweep the scan-flow drift the re-review caught
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 6s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 8s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 8s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m12s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m36s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m18s
a1b75c1f2c
The fix commit softened the script's scan-409 wording + dropped ?deep=true but
left the docs describing the old behavior. Match them:
- docs/e2e-local.md: POST .../scan (no ?deep; note local scans always ForceScan)
  + "409, deterministic bar a tiny residual TOCTOU gap" (was "guaranteed 409")
- docs/ci-cd.md: same "guaranteed 409" -> "409 (deterministic bar ...)"

Docs-only; harness behavior unchanged (still 38/38 green, deterministic).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Author
Owner

Independent cold adversarial review (review-only agent, 2 rounds — base diff + fix commit), verified against source (QueueLibraryScanByLibraryIdHandler, ScannerProxyService, ScannerService, JellyfinMediaSourcesController, GetCollectionsScanStatusHandler, the LibraryScanStatusResponseModel DTO, and the raw-SQL default-library migration), not just against the PR's own comments.

Round 1 (base 8a85f9dd): MERGEABLE-WITH-NITS, no Blocker/High. Verified the load-bearing assumptions independently — LibraryId=2==Shows is seeded by raw SQL in the LibraryRework migration; Flow B is race-free by construction (lock taken synchronously before the 202; collections-scan-status reads the same locker, no TOCTOU); the TPT schema has no extra NOT NULL cols so the seed can't violate constraints; the hourly scheduled auto-scan can't fire within a CI job. Findings: 1 Medium (unchecked seed_library_path), 3 Low, 2 Nits.

Fixes (4345180a): all four actionable findings addressed — seed-failure now surfaces as a FAIL not a silent skip; scan-status match anchored to "libraryId":2, (DTO guarantees LibraryId is the first key); no-op ?deep=true dropped; "guaranteed 409" softened to note the tiny residual TOCTOU gap the multi-second scan covers; WAL wording corrected to the busy-timeout-retry reason; non-routable address → TEST-NET-1 192.0.2.1 (RFC 5737).

Round 2 (re-review of 4345180a): all findings confirmed soundly fixed, no new functional defects; flagged one leftover doc/code drift (3 doc lines still said ?deep=true / "guaranteed 409").

a1b75c1f = 4345180a + exactly that 3-line docs sweep — no unreviewed functional code.

Validation: functional-E2E CI job green on CI (Release/ubuntu/image-ffmpeg) on the reviewed script; harness 38/38 green across 4+ fresh-instance runs, lock section deterministic every time.

Review-verdict: MERGEABLE @ a1b75c1f2c

**Independent cold adversarial review** (review-only agent, 2 rounds — base diff + fix commit), verified against source (`QueueLibraryScanByLibraryIdHandler`, `ScannerProxyService`, `ScannerService`, `JellyfinMediaSourcesController`, `GetCollectionsScanStatusHandler`, the `LibraryScanStatusResponseModel` DTO, and the raw-SQL default-library migration), not just against the PR's own comments. **Round 1** (base 8a85f9dd): MERGEABLE-WITH-NITS, no Blocker/High. Verified the load-bearing assumptions independently — `LibraryId=2`==Shows is seeded by raw SQL in the `LibraryRework` migration; Flow B is race-free by construction (lock taken synchronously before the 202; `collections-scan-status` reads the same locker, no TOCTOU); the TPT schema has no extra NOT NULL cols so the seed can't violate constraints; the hourly scheduled auto-scan can't fire within a CI job. Findings: 1 Medium (unchecked `seed_library_path`), 3 Low, 2 Nits. **Fixes** (4345180a): all four actionable findings addressed — seed-failure now surfaces as a FAIL not a silent skip; scan-status match anchored to `"libraryId":2,` (DTO guarantees LibraryId is the first key); no-op `?deep=true` dropped; "guaranteed 409" softened to note the tiny residual TOCTOU gap the multi-second scan covers; WAL wording corrected to the busy-timeout-retry reason; non-routable address → TEST-NET-1 `192.0.2.1` (RFC 5737). **Round 2** (re-review of 4345180a): all findings confirmed soundly fixed, **no new functional defects**; flagged one leftover doc/code drift (3 doc lines still said `?deep=true` / "guaranteed 409"). **a1b75c1f** = 4345180a + exactly that 3-line docs sweep — no unreviewed functional code. Validation: functional-E2E CI job **green** on CI (Release/ubuntu/image-ffmpeg) on the reviewed script; harness **38/38 green across 4+ fresh-instance runs**, lock section deterministic every time. Review-verdict: MERGEABLE @ a1b75c1f2c3a1f1387af24e48631907fd7cc3dcd
timothy merged commit c4695b1095 into main 2026-07-18 16:05:25 +02:00
timothy deleted branch feat/363-e2e-deferred-flows 2026-07-18 16:05:25 +02:00
Sign in to join this conversation.