diff --git a/.gitea/workflows/docker-build.yml b/.gitea/workflows/docker-build.yml index fe7845d3f..8fa32cda0 100644 --- a/.gitea/workflows/docker-build.yml +++ b/.gitea/workflows/docker-build.yml @@ -29,7 +29,7 @@ name: Build ErsatzTV Image # cannot read the workflow `env` context. **Bump all five together**; see docs/ci-cd.md -> # "CI toolchain image" for the two-step procedure. # -# CI image pin: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7 +# CI image pin: 192.168.1.95:3000/timothy/ersatztv-ci:32747a0 # # DOCS-ONLY SKIP (ersatztv#416): a change that touches only docs/** or *.md has nothing for the # heavy jobs to validate. `test`, `migrations`, `functional-e2e` and `build` each run @@ -107,7 +107,7 @@ jobs: name: Build & test (.NET) runs-on: ubuntu-latest container: - image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7 + image: 192.168.1.95:3000/timothy/ersatztv-ci:32747a0 credentials: username: ${{ secrets.REGISTRY_USER }} password: ${{ secrets.REGISTRY_PASSWORD }} @@ -262,7 +262,7 @@ jobs: name: EF migration integrity (SQLite + MySql) runs-on: ubuntu-latest container: - image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7 + image: 192.168.1.95:3000/timothy/ersatztv-ci:32747a0 credentials: username: ${{ secrets.REGISTRY_USER }} password: ${{ secrets.REGISTRY_PASSWORD }} @@ -409,7 +409,7 @@ jobs: echo "::endgroup::" functional-e2e: - name: Functional E2E (curl contracts) + name: Functional E2E (curl + UI contracts) runs-on: ubuntu-latest # Advisory gate (ersatztv#299): boots the app from source and drives the manual live-E2E # flows (legacy->SPA redirects, auth/CSRF/security-stamp, library-scan status contract, @@ -421,7 +421,7 @@ jobs: # v* tag builds. if: github.event_name == 'pull_request' || github.ref == 'refs/heads/main' container: - image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7 + image: 192.168.1.95:3000/timothy/ersatztv-ci:32747a0 credentials: username: ${{ secrets.REGISTRY_USER }} password: ${{ secrets.REGISTRY_PASSWORD }} @@ -498,6 +498,27 @@ jobs: trap 'kill "$PID" 2>/dev/null || true' EXIT scripts/e2e-functional.sh "http://localhost:${ETV_UI_PORT}" "$CFG" + # ersatztv#445: the UI-interactive flows the curl harness structurally CANNOT express — + # client-side form validation, AuthGate's rendered states, the session cookie authenticating the + # SPA's own /api XHRs, and sign-out through the UserMenu. + # + # Why in THIS job rather than its own: the dominant cost here is `npm ci` + the Release build, + # which are already done above. A separate job would duplicate both to add ~5s of browser work. + # The browser itself is baked into the toolchain image (docker/ci/Dockerfile — + # chromium-headless-shell), so this step installs nothing. + # + # It boots its OWN fresh instance on a DIFFERENT port: the first spec asserts the one-shot Setup + # gate, which the curl harness's auth section has already claimed on its own config dir, and a + # separate port keeps this independent of the previous step's teardown timing. + - name: Run UI-E2E Playwright flows (headless) + if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true' + run: | + set -euo pipefail + export ETV_BUILD_CONFIG=Release ETV_UI_PORT=8410 + # e2e-ui.sh owns the whole lifecycle: fresh config dir, boot, run specs, always kill the + # server. Its exit status is Playwright's. + scripts/e2e-ui.sh + build: name: Build & push image (amd64) # Moved back off `small` (server-management#639). This is the one HEAVY job that @@ -675,7 +696,7 @@ jobs: # 48 GiB at capacity 4 + a bumblebee overflow slot), which fixes the queue at the source. runs-on: ubuntu-latest container: - image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7 + image: 192.168.1.95:3000/timothy/ersatztv-ci:32747a0 credentials: username: ${{ secrets.REGISTRY_USER }} password: ${{ secrets.REGISTRY_PASSWORD }} @@ -770,7 +791,7 @@ jobs: # move to a lighter lane is a server-management capacity call (#604). runs-on: ubuntu-latest container: - image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7 + image: 192.168.1.95:3000/timothy/ersatztv-ci:32747a0 credentials: username: ${{ secrets.REGISTRY_USER }} password: ${{ secrets.REGISTRY_PASSWORD }} diff --git a/.gitignore b/.gitignore index fd6ce72a2..9dd03dfa7 100644 --- a/.gitignore +++ b/.gitignore @@ -73,6 +73,10 @@ web/node_modules # E2E / screenshot scratch (from Playwright/live-E2E runs) — never committed /*.png .playwright-mcp/ +# UI-E2E run artifacts: traces/screenshots Playwright writes on failure (outputDir in +# web/playwright.config.ts), plus the report dir it would use if a reporter is ever added (#445). +web/e2e/.output/ +web/playwright-report/ # Per-session worktree-ownership marker (H7, ersatztv#303) — local, never committed .claude-worktree-owner diff --git a/docker/ci/Dockerfile b/docker/ci/Dockerfile index b3e43b265..334520dde 100644 --- a/docker/ci/Dockerfile +++ b/docker/ci/Dockerfile @@ -16,6 +16,14 @@ # Keep the pins below in sync with what CI actually needs; Renovate tracks the image pins # (see renovate.json). The two dotnet tool versions mirror what docker-build.yml used to # `dotnet tool install` per run — bump them here, not in the workflow. +# +# ⚠️ If a branch carrying a change to THIS file gets rebased, `ci-image-pin` goes red: the pin must +# equal the short sha of the commit that touched docker/ci, and a rebase rewrites it. Re-dispatching +# ci-image.yml does not help — it tags the branch HEAD, not this commit. It cost ersatztv#445 two +# recovery cycles because main moved twice mid-review. The pin is also length-checked (exactly 7 +# chars, ersatztv#594) so a locally-computed 8-char `--short` now fails loudly at the gate instead of +# at image-pull time. Recovery, and the reason to land a toolchain change on its OWN branch BEFORE +# the work that consumes it: docs/ci-cd.md -> "CI toolchain image". # Keep the ffmpeg tag on the FROM below equal to the one docker/Dockerfile pins, so CI's ffmpeg # stays prod-identical. Renovate manages both pins (dockerfile manager); mirror any bump here. @@ -75,6 +83,39 @@ ENV DOTNET_ROOT=/usr/share/dotnet \ DOTNET_NOLOGO=1 \ DOTNET_SKIP_FIRST_TIME_EXPERIENCE=1 +# Headless Chromium for the UI-E2E Playwright flows (ersatztv#445), baked so the `functional-e2e` +# job installs no browser at run time — the same "jobs install nothing" rule as the rest of this +# image. `--with-deps` also pulls the system libraries headless Chromium needs, which the ffmpeg +# runtime base does not ship. +# +# PLAYWRIGHT_VERSION **must equal web/package.json's `@playwright/test` pin** (which is deliberately +# EXACT, no caret). Playwright ties a browser REVISION to the package version and resolves a +# revision-specific path, so a mismatch means "Executable doesn't exist at ...". `scripts/e2e-ui.sh` +# fails early with that instruction rather than a raw Playwright error. Renovate bumps the npm pin; +# when it does, bump this ARG, let ci-image.yml publish the new :, then update the five +# container pins in docker-build.yml (docs/ci-cd.md -> "CI toolchain image"). +ARG PLAYWRIGHT_VERSION=1.62.0 + +# A fixed, world-readable location instead of root's HOME cache: the path is then stable for +# scripts/e2e-ui.sh to detect, and independent of which user a step runs as. +ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright + +# `chromium-headless-shell`, NOT `chromium` — measured on this exact base: the headless shell is +# 267M where full chromium is 656M, and `chromium.launch()` (headless is the default, and +# web/playwright.config.ts never asks for headed) resolves to the shell anyway. The tradeoff: a +# HEADED run inside this image would fail. That is fine for CI and intentional; a developer running +# headed locally uses their own `npx playwright install chromium`, not this image. +# +# `--with-deps` also installs the system libraries headless Chromium needs, which the ffmpeg runtime +# base does not ship. Verified on the real base (Ubuntu 24.04): Chromium launches as root in a +# container with NO `--no-sandbox`/`chromiumSandbox:false` opt-out, so the config needs no workaround. +RUN npm install -g "playwright@${PLAYWRIGHT_VERSION}" && \ + playwright install --with-deps chromium-headless-shell && \ + chmod -R a+rX /ms-playwright && \ + npm cache clean --force && \ + apt-get clean -y && \ + rm -rf /var/lib/apt/lists/* + # Global dotnet tools the jobs used to install per run (5s + ~4s each, every run, forever). # dotnet-ef — the migrations job (was: `dotnet tool install --global dotnet-ef --version 9.0.12`) # dotnet-reportgenerator-globaltool — the test job's coverage summary @@ -94,6 +135,10 @@ RUN set -eux; \ python3 --version; \ ffmpeg -version; \ ffprobe -version; \ + # Headless Chromium must not merely be present — it must LAUNCH. The interesting failure mode is + # a missing system library or a sandbox refusal at startup, neither of which a file test would + # catch. Fail the IMAGE build here rather than a CI run. + NODE_PATH="$(npm root -g)" node -e "require('playwright').chromium.launch().then(async b => { console.log('chromium ' + b.version()); await b.close(); })"; \ # reportgenerator has no clean version probe: `--version` prints the banner and then exits 1 # ("No report files specified"), so asserting on it would fail this build. Probe that the # dotnet-tool shim resolves on PATH instead. diff --git a/docs/README.md b/docs/README.md index 172b6dd13..1a3d6b8ef 100644 --- a/docs/README.md +++ b/docs/README.md @@ -25,6 +25,7 @@ doc below, or that changes which sections a task signal points to.** | Auth / security-surface work | `docs/decisions/api-auth-security.md` | | CI / release pipeline work | `docs/ci-cd.md` + `docs/decisions/release-ci-governance.md` | | Live local run / Playwright-MCP verification | `docs/e2e-local.md` + `scripts/e2e-local.sh` | +| Adding/changing a UI-E2E browser flow | `docs/e2e-local.md` → "UI-E2E harness" + `scripts/e2e-ui.sh` | | What does a test suite cover | `docs/testing.md` | | Legacy Blazor route lookup | `docs/blazor-route-parity.md` (historical #91 phase (b) inventory) | | "Why do we do X this way" / challenging a convention | **Catalog-first**: `docs/decisions/README.md` (active rows) → `docs/decisions.md` + `docs/decisions/*.md` for full rationale. `docs/decisions/archive/` only for "what did the rule used to be." | diff --git a/docs/ci-cd.md b/docs/ci-cd.md index 317ee684b..4bd9d4951 100644 --- a/docs/ci-cd.md +++ b/docs/ci-cd.md @@ -392,7 +392,7 @@ the image build. Boots the app **from source** and drives the manual live-E2E curl flows sessions have historically re-run by hand, turning them into a CI regression net (ersatztv#299). It is the automatable half of -`docs/e2e-local.md`; the two scripts it chains run identically locally and in CI: +`docs/e2e-local.md`; every script it chains runs identically locally and in CI: 1. `npm ci` + `npm run build` (SPA), `dotnet build ErsatzTV.sln -c Release`. (ffmpeg comes from the CI toolchain image; the old `apt-get install ffmpeg` step is gone — see below.) @@ -406,6 +406,8 @@ re-run by hand, turning them into a CI regression net (ersatztv#299). It is the contract** (404 unknown / 202 queued / `scan-status` 200), and the **If-Match/412** round-trip on `rerun-collections`. Since ersatztv#363 (extended by #444) it also asserts three **lock-contention 409s** that aren't curl-only — see below. A `trap` kills the instance on step exit. +4. `ETV_UI_PORT=8410 scripts/e2e-ui.sh` — boots a second, **fresh** instance and runs the headless + Playwright UI flows (ersatztv#445). See "UI-E2E step" below. **Lock-contention 409s (ersatztv#363).** The harness now seeds rows the API can't create — a `LibraryPath` and a Jellyfin media-source — directly into the running instance's SQLite DB (via @@ -435,9 +437,28 @@ unit-test gate"): it is **not** a `needs:` of `build` and **not (yet) a required blocks nothing. Promote it to a required check / `build` dependency once it's proven reliable — the same staged rollout the `migrations` job used. SQLite is the default provider, so unlike `migrations` it needs **no** DB service container. Runs on PRs and on `main` (regression net); skipped for `v*` tag -builds. Still out of scope (tracked as an ersatztv#363 follow-up): the genuinely **UI-interactive -Playwright** flows (a separate CI browser-tooling lift). The playout-build lock 409 + `isLocked` -projection (#215) landed in #444. +builds. The playout-build lock 409 + `isLocked` projection (#215) landed in #444. + +**UI-E2E step (ersatztv#445).** The last step of this same job (item 4 in the summary above; the +`Run UI-E2E Playwright flows (headless)` entry in the YAML) runs the headless-browser flows the curl +harness structurally cannot express — client-side form validation, `AuthGate`'s rendered states, the +session cookie authenticating the **SPA's own** `/api` XHRs, and sign-out via the `UserMenu`: + +``` +ETV_BUILD_CONFIG=Release ETV_UI_PORT=8410 scripts/e2e-ui.sh +``` + +`e2e-ui.sh` owns the lifecycle (fresh config dir → boot → `playwright test` → always kill the server) +and exits with Playwright's status. Three deliberate choices: +- **This job, not a new one.** The dominant cost here is `npm ci` + the Release build, both already + done; a separate job would duplicate them to add ~5s of browser work. The browser is baked into the + toolchain image, so the step installs nothing. +- **Its own fresh instance on port 8410.** The first spec asserts the one-shot **Setup** gate, which + step 3's auth section has already claimed on its config dir; a separate port also keeps this step + independent of step 3's teardown timing. +- **`retries: 0`, `serial`, single worker.** #445 asked for deterministic flows and a retry would let a + flaky flow merge looking green (measured: 4 consecutive clean runs, ~2s each). Full rationale and the + rule for extending the suite: `docs/e2e-local.md` → "UI-E2E harness". ### Docs-only skip (ersatztv#416) @@ -611,10 +632,29 @@ run **inside** a shared image via `container:` instead of installing their toolc and **no** `dotnet tool install`. **What it ships:** .NET 10 SDK, Node 22, prod-identical ffmpeg/ffprobe, `git`/`python3`/`jq`/`zstd`, -and the `dotnet-ef` + `dotnet-reportgenerator-globaltool` global tools (which the `migrations` and +the `dotnet-ef` + `dotnet-reportgenerator-globaltool` global tools (which the `migrations` and `test` jobs used to install on every run — **bump those versions in the Dockerfile, not the -workflow**). Project dependencies (NuGet/npm) are deliberately **not** baked in — they change per -commit and stay on `actions/cache` (`~/.nuget/packages`, `~/.npm`). +workflow**), and **headless Chromium** for the UI-E2E flows (below). Project dependencies (NuGet/npm) +are deliberately **not** baked in — they change per commit and stay on `actions/cache` +(`~/.nuget/packages`, `~/.npm`). + +**Headless Chromium for UI-E2E (ersatztv#445).** `PLAYWRIGHT_BROWSERS_PATH=/ms-playwright` holds +`chromium-headless-shell`, installed with `--with-deps` at image build time so the `functional-e2e` +job installs no browser per run. Measured on this exact base: the headless shell is **267M** where +full `chromium` is **656M**, and `chromium.launch()` resolves to the shell anyway because +`web/playwright.config.ts` never asks for headed — the accepted tradeoff being that a *headed* run +inside this image would fail. Also verified on the real base rather than assumed: Chromium launches +as **root inside a container with no `--no-sandbox`/`chromiumSandbox:false` opt-out**, so the config +carries no sandbox workaround. The Dockerfile's build-time smoke test actually *launches* the browser, +so a missing system library fails the image build rather than a CI run. + +⚠️ **`ARG PLAYWRIGHT_VERSION` must equal `web/package.json`'s `@playwright/test` pin**, which is +deliberately **EXACT** (no caret): Playwright ties a browser *revision* to the package version, so a +mismatch leaves no usable browser. Renovate bumps the npm pin but cannot know about this ARG — when it +does, bump the ARG, let `ci-image.yml` publish the new `:`, then update all five container pins. +`scripts/e2e-ui.sh` guards the drift by *launching* a browser up front and failing with exactly that +instruction (it probes by launch, not by path, because `chromium.executablePath()` reports the +full-chromium path that a headless-shell-only image deliberately lacks). **How it's layered:** `FROM ersatztv-ffmpeg:8.1.2` + `COPY --from=mcr.microsoft.com/dotnet/sdk:10.0-noble-amd64 /usr/share/dotnet` — the same pattern `docker/Dockerfile` uses for the prod image. Our ffmpeg base is @@ -676,6 +716,21 @@ registry. Renovate tracks the Dockerfile's image pins (`dockerfile` manager, see (`entrypoint=["/bin/sleep" "10800"]`), so this is belt-and-braces for anyone running the image by hand — unlike the two above, which are real. +⚠️ **A REBASE invalidates the pin.** The pin must equal the short sha of the commit that touched +`docker/ci/**`, and a rebase **rewrites that commit's sha** — so `ci-image-pin` goes red on a branch +that was green before, with a pin that still resolves to a real (now-orphaned) commit and an image +that still exists in the registry. Worse, the force-push usually does **not** rebuild: `ci-image.yml` +filters on `paths: docker/ci/**`, and a rebase that doesn't change the Dockerfile's *content* +produces no diff for that path, so nothing republishes. And you cannot simply re-dispatch it — +`ci-image.yml` tags `git rev-parse --short HEAD`, i.e. whatever the branch HEAD is when it runs, not +the commit that touched `docker/ci`. Those two coincide only when the `docker/ci` commit *is* HEAD. + +Recovery (ersatztv#445 hit this): make the `docker/ci` commit be HEAD again — push a commit that +really does change `docker/ci/**`, let `ci-image.yml` publish `:`, then bump the pin in +a follow-up commit. That is the same two-step below, just re-run after the rebase. The cheapest way to +avoid it entirely is to **land a toolchain-image change on its own, before** the work that consumes +it, so the consuming branch never carries the `docker/ci` commit through a rebase. + **Bumping the pin is enforced, not remembered.** The `ci-image-pin` job (blocking, PR-only; defined in `pr-checks.yml`, but it greps `docker-build.yml` where the pins live) fails if `docker-build.yml`'s pin isn't the short sha of the last commit to touch `docker/ci/**` or diff --git a/docs/decisions.md b/docs/decisions.md index e08c43f2c..b6d85a63d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -117,28 +117,56 @@ in-file entries. - [2026-07-17 — Seasonal / date-conditional scheduling already exists (alternate schedules / playout templates); #73 closed as implemented](#2026-07-17--seasonal--date-conditional-scheduling-already-exists-alternate-schedules--playout-templates-73-closed-as-implemented) - [2026-07-17 — Auto-Tune DetailPanel member list = live search-index roll-up, not EF enumeration (#384)](#2026-07-17--auto-tune-detailpanel-member-list--live-search-index-roll-up-not-ef-enumeration-384) - [2026-07-17 — No persistent compiler servers in CI; every `services:` container gets an explicit cap; #390's small-lane move reversed (#406)](#2026-07-17--no-persistent-compiler-servers-in-ci-every-services-container-gets-an-explicit-cap-390s-small-lane-move-reversed-406) -- [2026-07-17 — Channel health on the API = the raw `PlayoutCount` fact on the list DTO, not a derived status enum (#72)](#2026-07-17--channel-health-on-the-api--the-raw-playoutcount-fact-on-the-list-dto-not-a-derived-status-enum-72) - [2026-07-17 — Weighted / fair-share distribution is a new `WeightedShuffle` order; `ShuffleInOrder` is anti-clumping, not fair-share (#70)](#2026-07-17--weighted--fair-share-distribution-is-a-new-weightedshuffle-order-shuffleinorder-is-anti-clumping-not-fair-share-70) +- [2026-07-17 — Docs-only CI skip gates STEPS in always-running required jobs, never `if:`-skips them (#416)](#2026-07-17--docs-only-ci-skip-gates-steps-in-always-running-required-jobs-never-if-skips-them-416) +- [2026-07-17 — Docs-only detect must be shallow-checkout safe: FETCH_HEAD + two-dot, not origin/main + three-dot (#416 follow-up)](#2026-07-17--docs-only-detect-must-be-shallow-checkout-safe-fetch_head--two-dot-not-originmain--three-dot-416-follow-up) +- [2026-07-17 — Pre-push guard: don't push a file whose working-tree copy is uncommitted (H13, #416 session)](#2026-07-17--pre-push-guard-dont-push-a-file-whose-working-tree-copy-is-uncommitted-h13-416-session) - [2026-07-17 — Auto-Tune per-channel overrides reuse the Channel Builder advanced-options DTO; weights + bug-colour logo split out to #425 (#385)](#2026-07-17--auto-tune-per-channel-overrides-reuse-the-channel-builder-advanced-options-dto-weights--bug-colour-logo-split-out-to-425-385) - [2026-07-17 — Health-check remediation is server-declared `{Kind, Target}` on an additive DTO; the SPA acts on it (#164)](#2026-07-17--health-check-remediation-is-server-declared-kind-target-on-an-additive-dto-the-spa-acts-on-it-164) - [2026-07-18 — Auto-Tune DetailPanel SPA: reusable `SlideOver` + shared advanced-options model; decorative panes dropped to match the backend (#386)](#2026-07-18--auto-tune-detailpanel-spa-reusable-slideover--shared-advanced-options-model-decorative-panes-dropped-to-match-the-backend-386) - [2026-07-18 — Auto-Tune per-source weights ride #70's MultiCollection machinery; created at tune time, not a post-hoc PUT (#425)](#2026-07-18--auto-tune-per-source-weights-ride-70s-multicollection-machinery-created-at-tune-time-not-a-post-hoc-put-425) - [2026-07-18 — Search all-items is paged to cap DoS exposure; SPA add-all pages to completeness (#293)](#2026-07-18--search-all-items-is-paged-to-cap-dos-exposure-spa-add-all-pages-to-completeness-293) +- [2026-07-18 — Collapsible sidebar + nav-group accordions: two `ctv-sidebar-*` localStorage keys, labeled groups default-collapsed (#396)](#2026-07-18--collapsible-sidebar--nav-group-accordions-two-ctv-sidebar--localstorage-keys-labeled-groups-default-collapsed-396) - [2026-07-18 — Unsupported PlaybackOrder is loud at build time; a declared support matrix and tripwire test make new orders safe by construction (#403)](#2026-07-18--unsupported-playbackorder-is-loud-at-build-time-a-declared-support-matrix-and-tripwire-test-make-new-orders-safe-by-construction-403) - [2026-07-18 — CI build-once was measured and rejected; keep the #420 tree-skip](#2026-07-18--ci-build-once-was-measured-and-rejected-keep-the-420-tree-skip) - [2026-07-18 — Never-scanned `LastScan` surfaces as null at the API boundary, not the 0001-01-01 MinValue sentinel (#409)](#2026-07-18--never-scanned-lastscan-surfaces-as-null-at-the-api-boundary-not-the-0001-01-01-minvalue-sentinel-409) - [2026-07-19 — WeightedShuffle SPA: weights edited on the multi-collection, order offered only on classic MultiCollection schedule items; fair-share is a reset not a mode (#404)](#2026-07-19--weightedshuffle-spa-weights-edited-on-the-multi-collection-order-offered-only-on-classic-multicollection-schedule-items-fair-share-is-a-reset-not-a-mode-404) - [2026-07-19 — Health-check results are TTL-cached; `?refresh=true` forces a fresh run (#431)](#2026-07-19--health-check-results-are-ttl-cached-refreshtrue-forces-a-fresh-run-431) +- [2026-07-19 — The `format` gate runs `dotnet format whitespace . --folder`, not the full solution format (#469)](#2026-07-19--the-format-gate-runs-dotnet-format-whitespace----folder-not-the-full-solution-format-469) - [2026-07-19 — CI `test` job reports a sampled true peak-anon, not cache-inflated `memory.peak` (#412)](#2026-07-19--ci-test-job-reports-a-sampled-true-peak-anon-not-cache-inflated-memorypeak-412) +- [2026-07-19 — Media-server remote-stream URLs are probed before use: a redirected 404 fails closed, everything else fails open, no toggle (#473)](#2026-07-19--media-server-remote-stream-urls-are-probed-before-use-a-redirected-404-fails-closed-everything-else-fails-open-no-toggle-473) +- [2026-07-19 — A media-server library sweep refuses to flag when a successful fetch returns zero items, rather than nuking the whole library (#477)](#2026-07-19--a-media-server-library-sweep-refuses-to-flag-when-a-successful-fetch-returns-zero-items-rather-than-nuking-the-whole-library-477) +- [2026-07-20 — External-JSON playout channels now probe the remote-stream URL too, closing the #473 scope gap (#480)](#2026-07-20--external-json-playout-channels-now-probe-the-remote-stream-url-too-closing-the-473-scope-gap-480) +- [2026-07-20 — `ILibraryRepository.GetOrAddFolder` resolves the folder from the DB, not the caller's `LibraryPath.LibraryFolders` navigation (#488)](#2026-07-20--ilibraryrepositorygetoraddfolder-resolves-the-folder-from-the-db-not-the-callers-librarypathlibraryfolders-navigation-488) +- [2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)](#2026-07-20--jellyfinmusicvideolibraryscanner-reconciles-by-library-scoped-path-diff--hard-delete-not-server-itemid-soft-trash-494) +- [2026-07-20 (#489) — Jellyfin mixed-content libraries map to one library holding many kinds](#2026-07-20-489--jellyfin-mixed-content-libraries-map-to-one-library-holding-many-kinds) +- [2026-07-20 — One logo drives the bug via a shared ChannelLogo preset, not new schema (#67)](#2026-07-20--one-logo-drives-the-bug-via-a-shared-channellogo-preset-not-new-schema-67) +- [2026-07-22 — Channel-level graphics-element attachment + seeded On Now/Next text element (#74)](#2026-07-22--channel-level-graphics-element-attachment--seeded-on-nownext-text-element-74) +- [2026-07-20 (#498) — QSV decode is split from QSV encode via a single `QsvPreferNativeDecoder` bool](#2026-07-20-498--qsv-decode-is-split-from-qsv-encode-via-a-single-qsvprefernativedecoder-bool) +- [2026-07-20 — `runs-on: small` means git-only; the two `docker build` jobs move to `ubuntu-latest` (server-management#639)](#2026-07-20--runs-on-small-means-git-only-the-two-docker-build-jobs-move-to-ubuntu-latest-server-management639) - [2026-07-20 — External-URL channel logos pass through to the graphics engine; never `File.Exists`-gated, never ffmpeg-native (#502)](#2026-07-20--external-url-channel-logos-pass-through-to-the-graphics-engine-never-fileexists-gated-never-ffmpeg-native-502) +- [2026-07-20 — HLS cold start is fixed with `-readrate_initial_burst`, not by raising the work-ahead limit (#350)](#2026-07-20--hls-cold-start-is-fixed-with--readrate_initial_burst-not-by-raising-the-work-ahead-limit-350) +- [2026-07-20 — MCP server (`ErsatzTV.Mcp`) built fresh over frozen `/api/v1`: read + cautious writes (#58)](#2026-07-20--mcp-server-ersatztvmcp-built-fresh-over-frozen-apiv1-read--cautious-writes-58) - [2026-07-20 — Remote graphics-engine images are fetched through a bounded, pooled `IRemoteImageFetcher`; re-fetched per element init, not cached (#511)](#2026-07-20--remote-graphics-engine-images-are-fetched-through-a-bounded-pooled-iremoteimagefetcher-re-fetched-per-element-init-not-cached-511) - [2026-07-21 — Decision records carry a lifecycle schema, validated by a script; append-only-by-diff is retired (#521)](#2026-07-21--decision-records-carry-a-lifecycle-schema-validated-by-a-script-append-only-by-diff-is-retired-521) - [2026-07-21 — Parallel orientation + selection is the startup protocol; #237 retired (#520)](#2026-07-21--parallel-orientation--selection-is-the-startup-protocol-237-retired-520) - [2026-07-21 — External channel-logo URLs are downloaded and cached at save time; the render path never fetches a logo (#525)](#2026-07-21--external-channel-logo-urls-are-downloaded-and-cached-at-save-time-the-render-path-never-fetches-a-logo-525) +- [2026-07-21 — QSV hardware-frame headroom is a floor, not an operator preference (#529)](#2026-07-21--qsv-hardware-frame-headroom-is-a-floor-not-an-operator-preference-529) - [2026-07-21 — Check the worked issue before the decision corpus; a closed tracker's comments need no retrofit (#524)](#2026-07-21--check-the-worked-issue-before-the-decision-corpus-a-closed-trackers-comments-need-no-retrofit-524) - [2026-07-21 — Work-ahead slots are claimed atomically by the caller, released by the transcode it hands them to (#536)](#2026-07-21--work-ahead-slots-are-claimed-atomically-by-the-caller-released-by-the-transcode-it-hands-them-to-536) +- [2026-07-21 — An on-demand time shift rebuilds the channel's cached XMLTV so the guide can't lag playback (#68)](#2026-07-21--an-on-demand-time-shift-rebuilds-the-channels-cached-xmltv-so-the-guide-cant-lag-playback-68) - [2026-07-21 — Session end fast-forwards the shared checkout; a stale tree serves stale FILES (#541)](#2026-07-21--session-end-fast-forwards-the-shared-checkout-a-stale-tree-serves-stale-files-541) +- [2026-07-21 — `WorkAheadSlots.Release()` clamps before decrementing and reports unbalance in-band (#539)](#2026-07-21--workaheadslotsrelease-clamps-before-decrementing-and-reports-unbalance-in-band-539) +- [2026-07-21 — `from-lineup` advanced overrides express "clear to none" via a typed `clear` enum list (#135)](#2026-07-21--from-lineup-advanced-overrides-express-clear-to-none-via-a-typed-clear-enum-list-135) +- [2026-07-21 — Browser channel preview is a server-declared per-channel capability (#60)](#2026-07-21--browser-channel-preview-is-a-server-declared-per-channel-capability-60) +- [2026-07-22 — Sequential (YAML) playout gets a golden; Scripted is excluded from the golden net by construction (#381)](#2026-07-22--sequential-yaml-playout-gets-a-golden-scripted-is-excluded-from-the-golden-net-by-construction-381) +- [2026-07-22 — per-schedule clock-boundary padding is a synthetic content-less Pad over the existing per-episode machinery (#392)](#2026-07-22--per-schedule-clock-boundary-padding-is-a-synthetic-content-less-pad-over-the-existing-per-episode-machinery-392) +- [2026-07-23 — Channel origin is immutable creation-provenance, stamped at insert, not a health signal (#414)](#2026-07-23--channel-origin-is-immutable-creation-provenance-stamped-at-insert-not-a-health-signal-414) +- [2026-07-23 — Facet-value typeahead is a new endpoint, allow-listed to text fields, no caching (#434)](#2026-07-23--facet-value-typeahead-is-a-new-endpoint-allow-listed-to-text-fields-no-caching-434) +- [2026-07-23 — Relative-date rule builder operators are a frontend-only mapping onto existing Lucene macros (#435)](#2026-07-23--relative-date-rule-builder-operators-are-a-frontend-only-mapping-onto-existing-lucene-macros-435) +- [2026-07-23 — Channel health = a server-derived `health` object on the channel DTOs, built-timeline detection (#415)](#2026-07-23--channel-health--a-server-derived-health-object-on-the-channel-dtos-built-timeline-detection-415) - [2026-07-25 — Rule-builder group nesting is bounded-arbitrary depth (`MAX_GROUP_DEPTH`), not one level (#436)](#2026-07-25--rule-builder-group-nesting-is-bounded-arbitrary-depth-max_group_depth-not-one-level-436) +- [2026-07-25 — UI-E2E: headless Playwright flows in the existing `functional-e2e` job, browser baked into the CI image (#445)](#2026-07-25--ui-e2e-headless-playwright-flows-in-the-existing-functional-e2e-job-browser-baked-into-the-ci-image-445) --- @@ -1148,7 +1176,7 @@ non-existence-check, and the missing `Name=` on `PlayoutController` Create/Delet ## 2026-07-16 — Functional-E2E CI harness: advisory curl-contract job over an app booted from source (#299) `key: ci.functional-e2e-harness` · `status: active` · `since: 2026-07-16` · `supersedes: none` · `superseded-by: none` -**Rule:** The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs curl-only, deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. +**Rule:** The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. Originally curl-only; since #445 the same job carries a second, headless-browser step for the contracts curl cannot express — see `ci.ui-e2e-harness`. **Signals:** staged rollout precedent (`migrations` job), racy/interactive flows deferred · paths: `scripts/e2e-local.sh`, `scripts/e2e-functional.sh` · issues: #299 **Mechanics:** `scripts/e2e-functional.sh` @@ -1176,7 +1204,8 @@ auth/CSRF/security-stamp flow, the library-scan status contract (404/202/`scan-s empty local library still enqueues `202`; an empty collection drives the concurrency editor). The 409 "already-scanning" re-trigger (needs a long-running scan to be non-racy), the playout-build lock 409, and the genuinely UI-interactive Playwright flows are deferred as #299 follow-ups rather than shipped -flaky. Assertions were written against a real running instance, not the source — which caught that +flaky. (All three have since landed: the two 409s in #363/#444, the Playwright flows in #445 — +`ci.ui-e2e-harness` — as a second step of this same job.) Assertions were written against a real running instance, not the source — which caught that `/artwork/*` returns `400` (not the `404` a static read suggested); extend the harness the same way. ## 2026-07-16 — Optional advertised IPTV base URL (`iptv.base_url`) resolved centrally in the two generators (#340) @@ -3637,3 +3666,51 @@ so a regression is diagnosable without decoding a random seed. **No backend change.** Nesting depth is entirely a client-side model concern; the stored query string is still plain Lucene, so nothing about `/api/v1`, the search index or the MCP surface moves. +## 2026-07-25 — UI-E2E: headless Playwright flows in the existing `functional-e2e` job, browser baked into the CI image (#445) +`key: ci.ui-e2e-harness` · `status: active` · `since: 2026-07-25` · `supersedes: none` · `superseded-by: none` +**Rule:** The UI-interactive E2E flows run as headless Playwright specs (`web/e2e/*.spec.ts`, driven by `scripts/e2e-ui.sh`) in a **second step of the existing advisory `functional-e2e` job**, never their own job; the browser is `chromium-headless-shell` **baked into the CI toolchain image** (`docker/ci/Dockerfile`, `PLAYWRIGHT_VERSION` kept equal to `web/package.json`'s EXACT `@playwright/test` pin), never installed per run; specs are `serial` with `retries: 0` and assert only contracts the curl harness structurally cannot reach. +**Signals:** UI-E2E, Playwright headless, boot-gate/Setup/Login flows, chromium-headless-shell, PLAYWRIGHT_BROWSERS_PATH=/ms-playwright, browser baked not installed, e2e-ui.sh, vitest `e2e/**` exclude, port 8410, no retries · paths: `scripts/e2e-ui.sh`, `web/playwright.config.ts`, `web/e2e/boot-gate.spec.ts`, `docker/ci/Dockerfile`, `.gitea/workflows/docker-build.yml`, `docs/e2e-local.md` · issues: #445, #363, #299 +**Mechanics:** `scripts/e2e-ui.sh`; `docs/e2e-local.md` -> "UI-E2E harness"; `docs/ci-cd.md` -> "CI toolchain image" + +Completes the last #299/#363 follow-up — the UI-interactive flows both prior records deferred. + +**Only assert what curl structurally cannot.** The curl harness already covers the auth *HTTP* contracts +(setup-claim 200/409, login 401/200, CSRF 403, stamp rotation); re-asserting them through a browser buys +nothing but flake surface. Scope is the four things curl cannot express: client-side form validation +(the Setup confirm-password gate is pure React state, makes no request), `AuthGate`'s *rendered* states, +the session cookie authenticating the **SPA's own** `/api` XHRs (curl proves the cookie works for curl, +not that the app sends it), and sign-out via `UserMenu`. **This scoping rule is the durable part** — +extend the browser suite only when a contract fails that test. + +**Baked browser, not a per-run install.** `chromium-headless-shell` lands in `/ms-playwright` at image +build time, so the job installs nothing — the "jobs install nothing at run time" rule of the shared CI +toolchain image (#390, which has no standalone record — see `ci.runner-placement` and `docs/ci-cd.md`). +Measured on the real base: headless shell **267M** vs full `chromium` **656M** (+171M compressed pull), +and `chromium.launch()` resolves to the shell anyway; the tradeoff is that a *headed* run in the image +fails. Verified rather than assumed: Chromium runs as root in-container with **no** `--no-sandbox` +opt-out. The browser revision is tied to the npm package version, so that pin is EXACT and `e2e-ui.sh` +guards drift by **launching** a browser — not by path, since `executablePath()` reports the +full-chromium path such an image lacks. + +**Why that job and not a new one** (`docs/ci-cd.md` has the detail): its `npm ci` + Release build are +already done, so a separate job would duplicate the dominant cost to add ~5s of browser work. The UI step +boots its **own** fresh instance on port 8410 — the first spec asserts the one-shot Setup gate the curl +step has already claimed. + +**`serial` + `retries: 0`, and vitest must not collect these files.** Server state is shared and partly +one-shot (the setup-claim), so specs are serial/single-worker; each `test` still gets its own browser +context, which gives the login specs a signed-out browser without a logout dance. Retries are 0 even in +CI — a retry lets a flaky flow merge looking green. Coupling worth knowing: vitest's default `include` +glob would run `web/e2e/*.spec.ts` under jsdom, so `vite.config.ts` excludes `e2e/**` by spreading +`configDefaults.exclude` — not by narrowing `include` to `src/**`, which would silently stop collecting +the real vitest test under `web/scripts/`. + +**Lifecycle correctness in `scripts/e2e-ui.sh`** — all four found by adversarial review, **none by a +passing run**; that is the transferable lesson (green runs never exercise the failure/interrupt paths). +(a) Install the cleanup trap *before* boot and have `e2e-local.sh` publish its PID to an opt-in +`ETV_PIDFILE` as it forks, so a mid-boot signal can still reap the server — a "kill whatever LISTENS on +the port" fallback is wrong twice over: it needs `lsof` (absent from the CI image) and it *infers* +ownership instead of proving it. (b) Re-raise signals, so a cancelled run cannot exit 0. (c) Never +`if ! cmd; then status=$?` — under `!` bash sets `$?` to the logical negation, so it reads 0 and a +FAILING run exits 0, silently passing CI. (d) `exec` the app inside the backgrounded subshell, or `$!` +is the SUBSHELL on bash 3.2 (stock macOS) and every PID-based kill targets the wrong process. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index bf1d0a345..e5a693aab 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -40,7 +40,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.docs-only-detect-shallow-safe` | The docs-only detect script must diff against `FETCH_HEAD` (always resolves after `git fetch`, even shallow) using a two-dot tree diff — not `origin/` with three-dot — because a `fetch-depth: 1` shallow clone has no remote-tracking ref and no merge-base, which silently fails the original detect into `docs_only=false` (full matrix, no functional error). A CI-behavior change must be verified by measuring the effect (job durations), not just a green check. | 2026-07-17 | [link](../decisions.md#2026-07-17--docs-only-detect-must-be-shallow-checkout-safe-fetch_head--two-dot-not-originmain--three-dot-416-follow-up) | | `ci.docs-only-skip-steps` | A docs-only change must still run every required job (`test`, `migrations`) so their commit-status contexts always report; each heavy job runs `scripts/ci-detect-docs-only.sh` first and gates its real STEPS on `if: steps.detect.outputs.docs_only != 'true'`, never `if:`-skips the whole job (an `if:`-skipped job reports `skipped`, not `success`, which branch protection may never unblock on). Detection biases toward running more on any doubt. | 2026-07-17 | [link](../decisions.md#2026-07-17--docs-only-ci-skip-gates-steps-in-always-running-required-jobs-never-if-skips-them-416) | | `ci.format-gate-folder-mode` | The blocking `format` CI job (and matching pre-commit hook) runs `dotnet format whitespace . --folder --include ` instead of loading the full MSBuild/Roslyn solution, cutting the gate from ~480s to ~0.5s with unchanged whitespace/charset coverage. | 2026-07-19 | [link](../decisions.md#2026-07-19--the-format-gate-runs-dotnet-format-whitespace----folder-not-the-full-solution-format-469) | -| `ci.functional-e2e-harness` | The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs curl-only, deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. | 2026-07-16 | [link](../decisions.md#2026-07-16--functional-e2e-ci-harness-advisory-curl-contract-job-over-an-app-booted-from-source-299) | +| `ci.functional-e2e-harness` | The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. Originally curl-only; since #445 the same job carries a second, headless-browser step for the contracts curl cannot express — see `ci.ui-e2e-harness`. | 2026-07-16 | [link](../decisions.md#2026-07-16--functional-e2e-ci-harness-advisory-curl-contract-job-over-an-app-booted-from-source-299) | | `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](workflow-process.md#2026-07-21--giteas-milestones-issue-filter-silently-no-ops-on-names-containing--or--542) | | `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](workflow-process.md#2026-07-21--an-infra-shaped-red-under-host-load-is-not-a-code-failure-542) | | `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-killed-ci-job-reports-conclusion-failure-read-the-log-tail-before-diagnosing-the-diff-542) | @@ -50,6 +50,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `ci.root-screenshot-guard` | The Husky `pre-commit` hook refuses a staged root-level `*.png` (belt-and-suspenders with the `.gitignore` rule); nested `*.png` real assets are unaffected. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--root-screenshot-guard-pre-commit-refuses-root-level-png-303-h3) | | `ci.runner-placement` | No persistent Roslyn compiler server survives a CI build (`UseSharedCompilation=false` etc., runner env + Dockerfile `ENV`); every `services:` container gets its own explicit `--memory`/`--memory-swap`/`--cpus` cap (it does not inherit the job container's). | 2026-07-17 | [link](../decisions.md#2026-07-17--no-persistent-compiler-servers-in-ci-every-services-container-gets-an-explicit-cap-390s-small-lane-move-reversed-406) | | `ci.small-lane-git-only` | `runs-on: small` is defined by what a job does (git-only), not its usual runtime; the two `docker build` jobs (docker-build.yml, ci-image.yml) move to `ubuntu-latest` because their worst-case memory, not median runtime, was pinning the small lane's per-slot cap. | 2026-07-20 | [link](../decisions.md#2026-07-20--runs-on-small-means-git-only-the-two-docker-build-jobs-move-to-ubuntu-latest-server-management639) | +| `ci.ui-e2e-harness` | The UI-interactive E2E flows run as headless Playwright specs (`web/e2e/*.spec.ts`, driven by `scripts/e2e-ui.sh`) in a **second step of the existing advisory `functional-e2e` job**, never their own job; the browser is `chromium-headless-shell` **baked into the CI toolchain image** (`docker/ci/Dockerfile`, `PLAYWRIGHT_VERSION` kept equal to `web/package.json`'s EXACT `@playwright/test` pin), never installed per run; specs are `serial` with `retries: 0` and assert only contracts the curl harness structurally cannot reach. | 2026-07-25 | [link](../decisions.md#2026-07-25--ui-e2e-headless-playwright-flows-in-the-existing-functional-e2e-job-browser-baked-into-the-ci-image-445) | | `ci.verify-locally-ci-confirms` | Treat the local build/verify/review pass as the decision point and CI as confirmation — don't idle waiting on a run you have no reason to doubt. | 2026-07-21 | [link](workflow-process.md#2026-07-21--build-and-verify-locally-then-trust-it-ci-confirms-542) | | `ci.web-test-per-test-timeouts` | Give heavy-render web tests an explicit per-test vitest timeout (e.g. 15s); never raise the global default to fix one slow test. | 2026-07-21 | [link](workflow-process.md#2026-07-21--heavy-render-web-tests-need-explicit-per-test-vitest-timeouts-on-the-ci-vm-542) | | `concurrency.diff-scalar-fanout` | The frozen Block optimistic-concurrency recipe (api-conventions §7a) fans out to Collection/Playout×2/MultiCollection/RerunCollection, keeping a guard-returned `PreconditionFailedError` out of any handler's generic `catch(Exception)`→422 mapping, and preserving each aggregate's existing `SaveChangesAsync() > 0` gate semantics under the new unconditional `Version++`. | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--253-pr3-diff--scalar-concurrency-fan-out-collection--playout2--multicollection--reruncollection) | @@ -162,7 +163,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `spa.yaml-validator-textarea` | The YAML playout validator takes pasted YAML via a `