timothy
71706f3849
Second review round. Codex returned BLOCKED @ e50a2624 with three High findings;
all three were real and all three are fixed. One of them was severe and was
introduced by my OWN previous "fix" commit.
## HIGH 1 — a FAILING spec run exited 0, silently passing CI
`if ! npx playwright test; then status=$?; ... exit "$status"; fi`
Under `!` negation bash sets `$?` to the LOGICAL NEGATION of the command's
status, so inside the failure branch `$?` reads 0 — the script exited 0 on a
failing run. Verified: `if ! (exit 42); then echo $?; fi` prints 0.
A UI-E2E harness that reports success when its specs fail is worse than no
harness. My five green local runs could never have caught this: the bug lives
only on the failure path. Introduced by the log-tail improvement in e50a2624.
Fixed with `set +e` / read `$?` / `set -e`, then exit that status explicitly.
Verified: a deliberately-failing run (`--grep ZZZ_NOPE`) now exits 1.
## HIGH 2 + 3 — the pre-PID fallback needed lsof and only GUESSED ownership
The mid-boot fallback reaped "whatever LISTENS on $PORT", which was wrong twice:
- it needed `lsof`, which is ABSENT from the CI toolchain image (verified
directly in the published image) — so it silently no-opped precisely where
it was needed;
- it INFERRED ownership from the earlier pre-flight rather than proving it, so
a process that grabbed the port after the pre-flight — or a real instance on
a shared host — could be killed. Reaping someone else's server is worse than
the leak it was meant to fix.
Replaced with an opt-in `ETV_PIDFILE`: e2e-local.sh writes the PID the instant
it forks, BEFORE its readiness wait, which is exactly the window a mid-boot
signal lands in. A pidfile we asked for PROVES ownership, needs no external
tool, and works in CI. The port-based kill is gone; the lsof pre-flight remains
as a friendly local check only.
Verified: the pidfile is populated while still mid-boot (readiness not yet
reached), names the real `dotnet ErsatzTV` process (not a subshell — which also
re-confirms the `exec` fix), and killing that PID alone frees the port.
Also dropped the `seq` dependency inside the trap (shell arithmetic instead),
addressing the other reviewer's busybox concern.
## Docs-reviewer finding — my stated reasoning was wrong
I justified amending `testing.e2e-local-fresh-config-dir` rather than superseding
it partly on "renaming the heading trips CI". That's a true statement that does
NOT bear on the choice: a supersession relocates to `archive/` with the heading
INTACT (verified: archive/api.md keeps the #72 heading verbatim). Corrected to
the actual reasons — the Rule never reversed, and the key is cited from
docs/handoffs/chicorytv-issue-queue.md plus two docs/superpowers/ files, which a
supersession would aim at an archived, stale-labelled record.
## Gotcha found by accident, now documented
A flawed test of mine booted two e2e-local.sh instances concurrently and the
first mysteriously failed to become ready. Cause: every run `rm -rf`s and
re-copies the SAME build-output wwwroot, so a second run yanks the static files
out from under a still-starting first instance. Documented in both the script
header and docs/e2e-local.md, because the symptom (readiness timeout, or /app
404ing) looks nothing like a shared-directory race. CI is unaffected — its curl
and UI-E2E steps are sequential.
## Budget: filed, not shaved
This PR pushes the active decisions corpus 7 lines past its 5600-line soft
budget (main was under). I trimmed my records repeatedly and each rewrite
recovered ~1 line, because the content is load-bearing; continuing would have
meant deleting useful rationale from a new convention record to hit an arbitrary
cap. The validator's own remedy is "schedule a consolidation", so that is filed
as #595 rather than paid for by starving the record. Non-blocking warning.
Also filed #594 for the pre-existing `ci-image-pin` any-hex-length weakness.
## Verification
- failing run exits 1 (was 0); passing run still 3/3 green
- pidfile written mid-boot, names the real dotnet proc, reap frees the port
- SIGTERM mid-run: exit 143, no orphan listener or process
- curl harness unaffected: 45/45 PASS; ETV_PIDFILE unset => unchanged behaviour
- decisions validator OK; zero orphaned processes after the full gate
Refs #445 #533 #594 #595
396 lines
26 KiB
Markdown
396 lines
26 KiB
Markdown
# Local live-E2E recipe
|
||
|
||
Purpose: how to stand up a real, running instance of this fork locally (dotnet host + built SPA)
|
||
for manual or Playwright-MCP-driven end-to-end verification — no live Docker/prod dependency.
|
||
**Update this doc (and `scripts/e2e-local.sh`) in the same PR that changes any convention below.**
|
||
|
||
This is for interactive/agent-driven verification, not CI (`docs/ci-cd.md` covers the CI pipeline,
|
||
which never runs the app itself).
|
||
|
||
## When live-E2E is required (not optional)
|
||
|
||
A live-E2E pass through this recipe is a **required** step — not a nicety — for any PR that changes
|
||
an **API write-path handler**: a `POST`/`PUT`/`DELETE` `/api/*` command that mutates state and then
|
||
reloads it through the read path. Reason: this class has a **correlated blind spot** that unit and
|
||
characterization tests share. A green fixed-point test passed while a write-path returned a 500 in
|
||
production because the handler returned a *lazy* LanguageExt `Map` the test never enumerated (#229,
|
||
PR — see `docs/decisions.md` and `api-conventions.md` §7); the reload-through-read-path mechanics can
|
||
throw only when the result is actually materialised, which the SPA does and the test did not. Live
|
||
driving the real screen (or `curl`ing the real endpoint) is the only net that reliably catches it.
|
||
|
||
Concretely, for a write-path PR: stand up the instance (Steps below), exercise the changed
|
||
create/update/delete flow against the real endpoint or its SPA screen, and confirm the mutation
|
||
**round-trips through a subsequent read** (list/detail/browse) — not just that the write returned 2xx.
|
||
Pure-SPA/read-only or docs PRs don't need it. State in the PR/close comment that live-E2E ran (or,
|
||
for a non-write-path change, that it wasn't required) — the same auditable-exemption rule the review
|
||
skip rubric uses (kickoff workflow lore).
|
||
|
||
## Why the steps are in this order
|
||
|
||
- **`ErsatzTV/Startup.cs`'s `/app` SPA middleware resolves its static-file root once, at startup**
|
||
(`SpaStaticFileRoot()`, checked with `Directory.Exists` and swapped to a `NullFileProvider` if
|
||
missing — see `Startup.cs` around the `app.MapWhen(... "/app" ...)` block). If `wwwroot/app`
|
||
doesn't exist yet when the process starts, **the SPA will 404 forever until you restart the
|
||
process** — copying the built files in after the fact does nothing for an already-running
|
||
instance.
|
||
- The dotnet host and the SPA share **one port** (default 8409, `ETV_UI_PORT` / `SystemEnvironment.
|
||
UiPort` in `ErsatzTV.Core/SystemEnvironment.cs`) — there's no separate dev server/proxy in this
|
||
workflow; you're testing the actual production static-hosting path.
|
||
- A fresh config folder per run avoids state bleed (leftover channels/schedules/DB) between test
|
||
sessions corrupting your assertions.
|
||
|
||
## Steps
|
||
|
||
1. **Build the prerequisites** (once, or after any source change):
|
||
```bash
|
||
dotnet build ErsatzTV.sln
|
||
cd web && npm run build && cd .. # → ErsatzTV/wwwroot/app (vite.config.ts outDir)
|
||
```
|
||
|
||
2. **Copy `wwwroot` into the build output** (the `dotnet build` output directory does not
|
||
automatically pick up `web/`'s build artifacts placed directly into the source `wwwroot`):
|
||
```bash
|
||
rm -rf ErsatzTV/bin/Debug/net10.0/wwwroot
|
||
cp -R ErsatzTV/wwwroot ErsatzTV/bin/Debug/net10.0/wwwroot
|
||
```
|
||
The `rm` matters: if the destination dir already exists (any prior run), `cp -R src dst`
|
||
copies **into** it (`dst/wwwroot/...`) and the server silently keeps serving the previous
|
||
run's stale assets. `scripts/e2e-local.sh` does the rm+copy for you.
|
||
If you rebuild the SPA (`npm run build`) while the dotnet process from step 4 is already
|
||
running, **re-copy and then restart the process** — see the "why" note above; it will not pick
|
||
up new files live.
|
||
|
||
3. **Use a fresh scratch config folder** — never reuse one across test runs:
|
||
```bash
|
||
CONFIG_DIR=$(mktemp -d)
|
||
```
|
||
|
||
4. **Run the app**, pointed at the scratch folder:
|
||
```bash
|
||
cd ErsatzTV/bin/Debug/net10.0
|
||
ETV_CONFIG_FOLDER="$CONFIG_DIR" dotnet ErsatzTV.dll
|
||
```
|
||
Wait for the log line **`Done migrating search index`** (emitted by
|
||
`RebuildSearchIndexHandler` in `ErsatzTV.Application/Search/Commands/`, with a
|
||
`... in {Duration}` suffix) — that's the last long-running startup step; before that, requests
|
||
may 404/error. The UI and the entire `/api/*` surface are served on the **same port**, 8409 by
|
||
default (override with `ETV_UI_PORT`).
|
||
|
||
**On a reused config dir that line never appears.** `RebuildSearchIndexHandler` logs one of two
|
||
mutually-exclusive lines immediately before `SystemStartup.SearchIndexIsReady()`, and which one
|
||
depends on whether the index had anything to migrate:
|
||
|
||
| Config dir | Line logged |
|
||
| --- | --- |
|
||
| fresh | `Done migrating search index in {Duration}` |
|
||
| reused (index already current) | `Search index is already version {Version}` |
|
||
|
||
So when waiting on readiness, match **either** line — the handler's `if`/`else` is exhaustive, so
|
||
the pair covers every path to ready. Watching only for `Done migrating search index` against a
|
||
reused dir waits out the full timeout and then kills a perfectly healthy server; that cost one
|
||
session two failed launches before the cause was found (ersatztv#533).
|
||
|
||
5. **Seed data** as needed via the API. **The `/api` surface is fail-closed since #197** (writes always,
|
||
and reads because `Api:RequireKeyForReads` defaults true), so curl seeding must send the machine key the
|
||
server generates at startup: `-H "X-Api-Key: $(cat "$CONFIG_DIR/api.key")"`. (Mutations from a *session*
|
||
also need `-H 'X-CSRF: 1'`, but for scripted seeding the machine key is simpler and CSRF-immune.)
|
||
Examples:
|
||
- Create a channel: `POST /api/v1/channels` — check the current
|
||
`ErsatzTV/wwwroot/openapi/v1.json` (or `CreateChannelRequest.cs`) for the exact required field
|
||
list before assuming these are complete, but as of this writing it requires (among plain
|
||
fields) these enums: `PlayoutSource: "Generated"`, `PlayoutMode: "Continuous"`,
|
||
`SongVideoMode: "Default"`, `TranscodeMode: "OnDemand"`, `IdleBehavior: "StopOnDisconnect"`.
|
||
- Create a playout for an existing channel: `POST /api/v1/playouts` with
|
||
`{"channelId": <id>, "scheduleKind": "Block"}` (or `"Classic"`/`"Scripted"`/`"Sequential"` per
|
||
`ChannelPlayoutSource`/schedule-kind enums — check `v1.json` for the current set).
|
||
|
||
5b. **Browser flow (since #295 session auth)**: the first `/app` load hits a **boot gate**. On a fresh
|
||
config it shows **Setup** — claim the local admin (pick any username/password); that issues the session
|
||
cookie and drops you into the app. On a subsequent run of the *same* config it shows **Login** instead.
|
||
(To skip the browser claim in automation, set `Auth:LocalAdmin:Password` before launch — the env seed
|
||
provisions the admin and disables the browser setup-claim.) The machine key still works for curl seeding
|
||
regardless. Driving this with Playwright: fill the Setup/Login form before asserting any authed screen.
|
||
|
||
6. **Tear down**: kill the `dotnet ErsatzTV.dll` process **by the PID you captured at launch**, and
|
||
confirm the port is freed (`lsof -i :8409` should return nothing) before starting another run — a
|
||
stray process holding the port will make the next run's health check hang or fail confusingly.
|
||
|
||
> ### ⚠️ Kill by PID. Never `pkill -f` a pattern.
|
||
>
|
||
> This machine is shared by parallel sessions, and **several of them run this same binary at
|
||
> once**. `pkill -f "dotnet ErsatzTV.dll"` reaps every one of them, not just yours.
|
||
>
|
||
> ```bash
|
||
> kill "$PID" # ✅ the PID e2e-local.sh printed
|
||
> pkill -f "dotnet ErsatzTV.dll" # ❌ NEVER — reaps other sessions' servers
|
||
> ```
|
||
>
|
||
> **Filtering by port does not make a pattern kill safe** — `pkill -f` matches the *command line*,
|
||
> not the port, so it hits every instance whatever port each one chose. Nor are the ports actually
|
||
> separated: the CI `functional-e2e` step exports `ETV_UI_PORT=8409`, the **same** port local runs
|
||
> use. (8410 is the `ersatztv-test` *container* on jazz — a different thing; the ui-E2E step from
|
||
> #445 does use it.) Scope by PID, not by pattern and not by port.
|
||
>
|
||
> **If the port is already busy, that process is not yours — diagnose, don't reap.** Report it and
|
||
> move to another port; `scripts/e2e-local.sh` pre-flights this for you and prints the foreign PID:
|
||
> ```bash
|
||
> lsof -ti :8409 # who is holding it
|
||
> ETV_UI_PORT=8420 scripts/e2e-local.sh # your run, out of the way
|
||
> ```
|
||
>
|
||
> **Launching the DLL by hand? Set both ports.** `Program.cs` binds a second listener on
|
||
> `ETV_STREAMING_PORT`, which defaults to **8409 regardless of `ETV_UI_PORT`** — so
|
||
> `ETV_UI_PORT=8420 dotnet ErsatzTV.dll` still binds 8409 and dies against a foreign holder:
|
||
> ```bash
|
||
> ETV_UI_PORT=8420 ETV_STREAMING_PORT=8420 dotnet ErsatzTV.dll
|
||
> ```
|
||
> `scripts/e2e-local.sh` defaults `ETV_STREAMING_PORT` to whatever port you gave it, so through the
|
||
> script `ETV_UI_PORT=8420` alone is sufficient.
|
||
>
|
||
> Why this is a hard rule rather than a preference: killing another session's harness mid-run does
|
||
> **not** fail loudly. It truncates that run's output into plausible-looking-but-wrong data — the
|
||
> failure class that survives review. A near-miss of exactly this shape is recorded in
|
||
> `docs/decisions/workflow-process.md` → `testing.e2e-cleanup-scope-by-pid` (ersatztv#586).
|
||
>
|
||
> **Writing a brief for a delegated agent? Put this constraint in it.** The #586 incident was a
|
||
> delegation gap, not agent error: the brief specified a fresh config dir but said nothing about
|
||
> process cleanup, so the agent invented a reasonable-looking pattern kill. An omitted rule is not
|
||
> an unenforced rule — it is a rule replaced by whatever plausible default the agent reaches for.
|
||
|
||
## Playwright MCP screenshots
|
||
|
||
If you're driving the browser via the Playwright MCP server for visual verification, screenshots
|
||
land in **the MCP server process's own cwd** (this repo's root, not wherever you ran the dotnet
|
||
process from) — expect stray `*.png` files at the repo root after a session; this is tolerated,
|
||
not a bug to fix, but don't check them in.
|
||
|
||
## Script: `scripts/e2e-local.sh`
|
||
|
||
A copy of this script is included in this doc's directory; it is intended to land at
|
||
`scripts/e2e-local.sh` in the repo. It automates steps 2–4 above (build is assumed already done —
|
||
run `dotnet build` / `npm run build` yourself first, since rebuilding on every invocation is slow
|
||
and this script is meant to be re-run often during a debugging session).
|
||
|
||
Usage:
|
||
```bash
|
||
scripts/e2e-local.sh [CONFIG_DIR]
|
||
```
|
||
- `CONFIG_DIR` defaults to a fresh `mktemp -d` if omitted. Prefer a fresh dir per run for
|
||
**state-bleed** reasons (above); the readiness probe itself handles a reused dir since #533.
|
||
- ⚠️ **Never run two instances concurrently, even on different ports.** Each run `rm -rf`s and
|
||
re-copies the *same* `ErsatzTV/bin/<config>/net10.0/wwwroot`, so the second run pulls the static
|
||
files out from under a still-starting first instance. The symptom — a readiness timeout, or `/app`
|
||
404ing on the instance that *did* start — looks nothing like a shared-directory race. Run
|
||
sequentially; the CI job's curl and UI-E2E steps do.
|
||
- **`ETV_PIDFILE`** (optional): the server PID is written here the instant it forks, *before* the
|
||
readiness wait. `scripts/e2e-ui.sh` sets it so it can still reap the server if it is killed
|
||
mid-boot, before it has parsed `PID=` from stdout. Unset means unchanged behaviour.
|
||
- Copies `ErsatzTV/wwwroot` → `ErsatzTV/bin/<config>/net10.0/wwwroot`.
|
||
- Launches `dotnet ErsatzTV.dll` in the background with `ETV_CONFIG_FOLDER` set.
|
||
- Waits (up to 120s) for readiness — either `Done migrating search index` (fresh config) or
|
||
`Search index is already version` (reused config); see the step-4 table above.
|
||
- **Pre-flights both bound ports** (`ETV_UI_PORT` and `ETV_STREAMING_PORT`): if something is already
|
||
listening, it fails immediately naming the offending PID, rather than letting Kestrel fail its bind
|
||
a moment later and surface as `process N exited before becoming ready` plus a log tail — a framing
|
||
that reads like a broken build. It **reports, never reaps** — on this shared machine a foreign
|
||
listener is most likely another session's harness mid-run. Re-run with `ETV_UI_PORT=<other>`.
|
||
- **Defaults `ETV_STREAMING_PORT` to `ETV_UI_PORT`** so that re-run actually works: the app binds a
|
||
second listener that otherwise defaults to 8409 no matter what the UI port is. An explicit
|
||
`ETV_STREAMING_PORT` still wins, and CI (which uses 8409) is unaffected.
|
||
- Prints the PID and port, then **exits leaving the server running** — the caller owns that PID and
|
||
is responsible for killing it when done (`kill "$PID"`, never a `pkill -f` pattern — see the
|
||
teardown warning in step 6).
|
||
- **Does not trap-and-kill on exit, by design** — unlike `scripts/e2e-ui.sh`, which owns its
|
||
instance's whole lifecycle and kills the server from an `EXIT INT TERM` trap. The two scripts sit
|
||
on opposite sides of that contract on purpose: this one is the *launcher* and hands a running
|
||
instance to its caller (an `EXIT` trap here would kill the server the instant it returned, breaking
|
||
every caller including `e2e-ui.sh` and the CI `functional-e2e` step); `e2e-ui.sh` is a *lifecycle
|
||
owner* and traps. If you write a new harness that boots and then finishes on its own, follow
|
||
`e2e-ui.sh`: capture the PID and trap.
|
||
- **`ETV_BUILD_CONFIG`** selects which build output to launch (`Debug` default for local dev; the
|
||
CI `functional-e2e` job sets `Release`). It must match the `dotnet build --configuration` you ran
|
||
first — the script only copies `wwwroot` + launches; it does not build.
|
||
|
||
## Functional-E2E harness: `scripts/e2e-functional.sh`
|
||
|
||
The ad-hoc curl scenarios sessions run against a live instance (redirect sweeps, auth flows, the scan
|
||
status contract, `If-Match`/412) are codified into a single harness so they run identically by hand
|
||
and in CI (the advisory `functional-e2e` job — `docs/ci-cd.md`). It does **not** boot the app; pair it
|
||
with `scripts/e2e-local.sh` above (or point it at any running instance):
|
||
|
||
```bash
|
||
CFG=$(mktemp -d)
|
||
eval "$(scripts/e2e-local.sh "$CFG" | sed -n 's/^\(PID\|PORT\)=/\1=/p')" # launch, capture PID/PORT
|
||
scripts/e2e-functional.sh "http://localhost:${PORT}" "$CFG" # assert; exit 1 on any failure
|
||
kill "$PID"
|
||
```
|
||
|
||
`CONFIG_DIR` (arg 2) is **required** — the harness reads the instance's machine key from
|
||
`$CONFIG_DIR/api.key` and sends `X-Api-Key` on every `/api` call (the surface is fail-closed). It runs
|
||
each assertion even after a failure and prints a pass/fail summary, exiting non-zero if any failed.
|
||
|
||
What it covers. Most assertions are curl-only (no seeded media / ffmpeg / browser); the
|
||
**lock-contention** section (added by ersatztv#363, extended by #444) is the exception — it seeds DB rows + media.
|
||
- **Legacy→SPA redirects**: a sweep of representative `LegacyUiRedirects` routes 302→`/app/*`, plus
|
||
the `/api` + `/artwork` never-redirect exemption (asserted as "did not 302 to `/app`", since those
|
||
4xx from their own handlers — `/api/*` 404s, `/artwork/*` 400s).
|
||
- **Library-scan status contract**: create an empty local library → scan `202`, unknown-library scan
|
||
`404`, `scan-status` `200`.
|
||
- **Optimistic concurrency**: create a collection + a `rerun-collections` targeting it → `GET` emits
|
||
an `ETag` → `PUT` with a stale `If-Match` `412`, current `200`, malformed `400`.
|
||
- **Lock contention (409)** — the three `IEntityLocker` 409s, made **deterministic** by firing the
|
||
racing request only once the lock is *provably* held (never a sleep-and-hope). All seed rows the
|
||
API can't create straight into the running instance's DB via python3's stdlib `sqlite3` (whose
|
||
busy-timeout retry serializes behind the app's writer):
|
||
- **library-scan "already scanning"**: seed ~60 tiny ffmpeg clips into the built-in Shows library
|
||
(Id=2) so the scanner subprocess runs a few seconds → `POST .../scan` `202` (local scans always
|
||
ForceScan, so no `deep` needed) → poll `GET /libraries/scan-status` until library 2 shows active
|
||
(that window is a strict subset of the scan lock's held window — `ScannerProxyService.StartScan`
|
||
fires *after* `LockLibrary`, `EndScan` *before* `UnlockLibrary`) → a second `POST .../scan` is a
|
||
`409`, deterministic bar a tiny residual TOCTOU gap the multi-second scan covers. Self-skips
|
||
(advisory) if ffmpeg is absent.
|
||
- **external-collections "already scanning"**: seed a Jellyfin media-source row pointing at a
|
||
non-routable address (so the background sync hangs and the per-family lock stays held). The lock
|
||
is taken synchronously *before* the `202`, so the `202` proves it held → `collections-scan-status`
|
||
shows the `jellyfin` family → a second `POST .../scan-collections` is `409`; unknown source `404`.
|
||
Needs neither ffmpeg nor the scanner.
|
||
- **playout-build "build in progress"** (#215/#444, the hardest): a build is enqueued onto the
|
||
single-consumer `WorkerService` channel and the trigger returns *before* `BuildPlayoutHandler`
|
||
dequeues and `LockPlayout`s (released in its `finally`), so an accepted trigger does **not** prove
|
||
the lock is held — poll `GET /playouts/{id}` until `isLocked:true`, then fire. Seed a few short
|
||
ffmpeg episodes → a Collection → a **Classic Flood** schedule → a Classic playout, and crank
|
||
`PlayoutDaysToBuild` (config `playout.days_to_build`) so a single build is wide enough to observe:
|
||
5 days ≈ 43k playout items ≈ ~1s locally, wider on slower CI (sized by measurement — item count is
|
||
`window / item-duration`, so the flood over a handful of short clips scales with the day count;
|
||
going *too* wide is counter-productive — a 777k-item build saturates the single worker with its
|
||
post-build gap/overlap jobs). While provably locked: `PUT /playouts/{id}` → 409, `POST
|
||
.../playout/reset` → 409, and the `GET /playouts` list projection shows `isLocked:true`; after the
|
||
build the lock clears (`isLocked:false`) and the same `PUT` now `200`s (proving the 409 is
|
||
lock-specific). Restores the config default afterwards. Self-skips (advisory) without ffmpeg, or if
|
||
the build is never observed locked (never asserts a race it can't prove it won).
|
||
- **Auth/CSRF/security-stamp** (mirrors the #295 manual set; runs **last** — setup-claim is one-shot
|
||
and logout revokes the session): fresh `auth/config` `setupRequired:true` → read-gate
|
||
401-without-key / 200-with-key → setup-claim 200 → re-claim 409 → session mutation
|
||
403-without-`X-CSRF` → login 401-then-200 → logout 403-without-CSRF / 204-with → post-logout the
|
||
same cookie is 401 (rotated security stamp).
|
||
|
||
The playout-build lock 409 + `isLocked` projection (#215) landed in #444 — see the lock-contention
|
||
bullet above. The genuinely **UI-interactive Playwright** flows landed in #445 — see the next section.
|
||
When you extend the harness, add the observed contract here and keep the assertions deterministic (probe
|
||
the real instance first — the initial cut caught that `/artwork/*` 400s where a guess said 404; the #363
|
||
cut measured the real scan window before sizing the seed).
|
||
|
||
## UI-E2E harness: `scripts/e2e-ui.sh` + `web/e2e/*.spec.ts`
|
||
|
||
The handful of contracts that **cannot** be expressed as curl calls, driven by **headless Playwright**
|
||
(never claude-in-chrome, per project convention — ersatztv#445). Unlike `e2e-functional.sh`, this script
|
||
owns the whole lifecycle: it boots a fresh instance, runs the specs, and always kills the server (trap
|
||
on `EXIT`/`INT`/`TERM`), exiting with Playwright's status:
|
||
|
||
```bash
|
||
scripts/e2e-ui.sh # fresh config dir, all specs
|
||
scripts/e2e-ui.sh "$(mktemp -d)" --headed --debug # extra args pass through to `playwright test`
|
||
```
|
||
|
||
**A fresh config dir is required, not merely preferred** — the first spec asserts the one-shot Setup
|
||
(first-run admin claim) gate, which only exists while the server has no local admin. Pointing it at a
|
||
reused dir fails that spec by design.
|
||
|
||
**What it covers, and the rule for extending it.** Only assert here what curl *structurally cannot* —
|
||
the curl harness above already covers the auth HTTP contracts, so re-asserting them through a browser
|
||
buys nothing but flake surface. The four things that qualify:
|
||
|
||
| Contract | Why curl can't reach it |
|
||
| --- | --- |
|
||
| Setup card's confirm-password gate | Pure React state; makes no request, so there is no HTTP contract |
|
||
| `AuthGate` states (Setup vs Login vs app) | Assertion is about what *renders*, not a status code |
|
||
| Session cookie on the SPA's own `/api` XHRs | curl proves the cookie works *for curl*, not that the app sends it |
|
||
| Sign-out via the `UserMenu` | A DOM interaction chain, not a single endpoint |
|
||
|
||
**Determinism rules** (#445 asked for deterministic flows, so these are deliberate):
|
||
- `retries: 0`, in CI too — a retry would let a genuinely flaky flow merge looking green.
|
||
- `serial` + `workers: 1`: server state is shared and partly one-shot (the setup-claim).
|
||
- Each `test` still gets its own browser context (own cookie jar) — that is what gives the login specs a
|
||
genuinely signed-out browser without a logout dance. **Anything needing a session held across steps
|
||
must stay inside one `test`.**
|
||
- Address form fields by their unique **placeholders**, not labels: the shared `<Input>` wraps its
|
||
`<input>` in a `<label>`, so a field's accessible name absorbs its error text when invalid
|
||
(`"Confirm password Passwords do not match."`).
|
||
- A fresh config is **not** empty — `DbInitializer` seeds one default channel (`"ErsatzTV"`, number 1),
|
||
so the Channels *empty state* is unreachable there; assert the seeded row instead.
|
||
|
||
**The browser is baked into the CI toolchain image**, not installed per run (`docker/ci/Dockerfile`:
|
||
`chromium-headless-shell`, 267M vs 656M for full chromium). Two consequences:
|
||
- `web/package.json`'s `@playwright/test` pin is **EXACT** (no caret) because Playwright ties a browser
|
||
revision to the package version. Bumping it requires rebuilding the CI image — `scripts/e2e-ui.sh`
|
||
guards this by *launching* a browser up front and failing with that instruction. See `docs/ci-cd.md`
|
||
→ "CI toolchain image".
|
||
- A **headed** run inside the CI image would fail (headless shell only). Locally, run
|
||
`cd web && npx playwright install chromium` once and headed works from your own cache.
|
||
|
||
Vitest deliberately excludes `e2e/**` (`web/vite.config.ts`) — its default `**/*.spec.*` glob would
|
||
otherwise try to run these under jsdom.
|
||
|
||
## Seeding a local TV library for E2E
|
||
|
||
Channels/playouts are API-seedable (step 5 above), but a **local media library is not** — there
|
||
is no `/api/*` endpoint to add a local library folder. To exercise media-browse / search / detail
|
||
screens you need real scanned items. Recipe (used to verify the #220 episode-nav PR):
|
||
|
||
1. **Generate tiny media files on disk** — one show (one season, ~3 episodes), plus a second show
|
||
whose title *contains the first as a substring* (good substring-search sanity data), plus a
|
||
movie if you need a non-episode kind. Keep TV and movies under **separate roots** so each
|
||
library scans cleanly (a Shows library pointed at a folder that also contains movies will try
|
||
to parse the movies as shows). Each file is a 2-second `testsrc` clip:
|
||
```bash
|
||
MEDIA=/tmp/etv-media # any scratch path
|
||
mkdir -p "$MEDIA/tv/Show Alpha/Season 01" \
|
||
"$MEDIA/tv/Show Alpha Returns/Season 01" \
|
||
"$MEDIA/movies/Test Movie (2020)"
|
||
for n in 01 02 03; do
|
||
ffmpeg -y -f lavfi -i testsrc=duration=2:size=320x240:rate=10 -c:v libx264 -pix_fmt yuv420p \
|
||
"$MEDIA/tv/Show Alpha/Season 01/Show Alpha - s01e$n.mkv"
|
||
done
|
||
ffmpeg -y -f lavfi -i testsrc=duration=2:size=320x240:rate=10 -c:v libx264 -pix_fmt yuv420p \
|
||
"$MEDIA/tv/Show Alpha Returns/Season 01/Show Alpha Returns - s01e01.mkv"
|
||
ffmpeg -y -f lavfi -i testsrc=duration=2:size=320x240:rate=10 -c:v libx264 -pix_fmt yuv420p \
|
||
"$MEDIA/movies/Test Movie (2020)/Test Movie (2020).mkv"
|
||
```
|
||
|
||
2. **Attach the folders to the built-in local libraries via SQLite.** A fresh config DB already
|
||
has the seven default local libraries (`Library` rows for a single `LocalMediaSource`): `Movies`
|
||
is `Id=1`, `Shows` is `Id=2`. `LibraryPath` is just `(Path TEXT, LibraryId INT)` — insert one
|
||
row per root, pointing each at the matching library:
|
||
```bash
|
||
DB="$CONFIG_DIR/ersatztv.sqlite3" # CONFIG_DIR from the run above; server may be running
|
||
sqlite3 "$DB" "INSERT INTO LibraryPath (Path, LibraryId) VALUES ('$MEDIA/tv', 2);" # Shows
|
||
sqlite3 "$DB" "INSERT INTO LibraryPath (Path, LibraryId) VALUES ('$MEDIA/movies', 1);" # Movies
|
||
```
|
||
|
||
3. **Trigger a scan and wait for items to appear.** The scan endpoint takes an empty body:
|
||
```bash
|
||
curl -s -X POST http://localhost:8409/api/v1/libraries/2/scan -H 'Content-Type: application/json' -d '{}'
|
||
curl -s -X POST http://localhost:8409/api/v1/libraries/1/scan -H 'Content-Type: application/json' -d '{}'
|
||
# poll until episodes show up (scanner runs as a background subprocess):
|
||
curl -s "http://localhost:8409/api/v1/library/browse?mediaType=Episode&pageSize=50"
|
||
```
|
||
The scan runs even though `LibraryPath` was inserted after startup — the scan handler re-reads
|
||
the library from the DB. Browse (`/api/v1/library/browse`) reads straight from the DB, so items
|
||
appear there within a few seconds.
|
||
|
||
### Gotchas
|
||
|
||
- **Do NOT delete the `search-index/` folder to "reset" search.** On startup the app *recreates the
|
||
index empty* (`Search index failed to initialize; will delete and recreate` → `Migrating search
|
||
index to version N`) and that migration does **not** re-index from the DB — only a **scan**
|
||
writes documents into the Lucene index. The scanner subprocess writes the index while running; a
|
||
restart never rebuilds it from existing DB rows. If you wipe `search-index/`, a *rescan of
|
||
unchanged files won't repopulate it* (the scanner skips unchanged items), so search stays empty.
|
||
The clean recovery is a fresh `CONFIG_DIR`: launch → insert `LibraryPath` → scan **once** → leave
|
||
the index alone.
|
||
- **Search query relevance is field-scoped, not free-text.** The `/api/v1/search` default field does
|
||
**not** match bare title words: `Alpha` and `Show` return nothing for a "Show Alpha" title, while
|
||
`title:Alpha`, `Show*`, or `*Alpha*` all match. The SPA search box forwards the query verbatim, so
|
||
when driving search-result screens in E2E use a field/wildcard query (e.g. `title:Alpha`) to get
|
||
deterministic hits. (This is pre-existing ErsatzTV search behavior, independent of any SPA change.)
|