Round 13 found no blocker. Two things, both mine. WATCH MODE (residual 6), present since the first commit on this branch and missed by twelve rounds. The virtual module has no backing file, so Vite never invalidates it and `resolveTrackedSourceFiles` runs once per dev-server lifetime while the glob refreshes. MEASURED under `npx vitest --watch`: create an untracked scratch file mid-session and the cross-check reddens; `git add` does not clear it; `npx vitest run` on the same tree is green. That is the #778/#806 developer-red / CI-green symptom this change exists to remove, appearing under the invocation `docs/testing.md` documents as `npm test`. My first probe of this reported all-green and was WRONG — a filtered watcher never re-ran, so I was reading a stale first result. Re-measured with a forced re-run, the review's report reproduces exactly. The obvious fix is rejected on MEASUREMENT rather than taste, which is the part worth recording: invalidating the module from `configureServer` on the watcher's `add`/`unlink` does fix the scratch-file red — and then `git add`, which touches no file and so fires no watcher event, leaves the module holding its pre-staging answer. Measured with that hook in place, a tracked planted `pageSize: 100` goes UNSEEN, green. It trades a noisy red for a blind green. So it is named, with both measurements, in the guard, in `docs/guard-inventory.md` and beside the `npm test` line in `docs/testing.md`. Also: widening residual (3) to the plugin's three outputs last round did not carry to its DIRECTION paragraph, which still said "(3) is invisible to any CONSUMER by construction". Measured false for the `absentFromDisk` half — the assertion added last round catches it wherever it overlaps a walked file — and `docs/guard-inventory.md` already had the correct wording, so the claim had drifted between two copies again. Corrected, with a note not to delete that assertion as decoration. Two smaller ones: "an UNCONDITIONAL mispartition" became the wrong noun once (3) covered three outputs, and "the walk set and the index set are THE SAME SET" is imprecise — they agree on every in-scope key, but the index also carries `.css`. Refs: #819
128 lines
9.8 KiB
Markdown
128 lines
9.8 KiB
Markdown
# Testing map
|
|
|
|
Purpose: authoritative map of what each test project/suite covers and how to run it. Read this
|
|
before adding tests, not just `docs/contributing.md` §8 (which now just points here).
|
|
|
|
## Test projects
|
|
|
|
> **Font dependency (ersatztv#732).** `Infrastructure/Graphics/TextElementBackgroundBoxTests` is the
|
|
> only suite that rasterises text, so it needs at least one system font to lay anything out. The CI
|
|
> image installs none explicitly — fonts arrive via `playwright install --with-deps` in
|
|
> `docker/ci/Dockerfile` (351 present in the pinned image, measured 2026-08-26). This is a real
|
|
> dependency, declared here so a future slimming of that install produces a known cause rather than a
|
|
> mystery red. `Baseline_Renders_A_Non_Empty_Bitmap` exists to make that failure loud: without it a
|
|
> fontless host would render a 0x0 bitmap and every relative geometry assertion would pass vacuously.
|
|
|
|
| Project | Covers | Notes |
|
|
|---|---|---|
|
|
| `ErsatzTV.Tests` | API controllers + MediatR handlers, plus the SkiaSharp text-overlay rasteriser (`Infrastructure/Graphics/`) | In-memory SQLite fixture: a shared `SqliteConnection("Data Source=:memory:;Foreign Keys=False")` kept open + `EnsureCreatedAsync()` (**not** full migration replay) + `PRAGMA foreign_keys=OFF`, then seed; a tiny `IDbContextFactory` wraps `new TvContext(...)`. ~1,870 tests (approximate on purpose — an exact count goes stale on every PR that adds one; the previous hardcoded 828 was off by over a thousand). |
|
|
| `ErsatzTV.Core.Tests` | Domain logic, scheduling, IPTV/XMLTV generation | References `ErsatzTV.Application` directly — there is no separate `Application.Tests` project. ~650 tests + 1 skipped under `TZ=UTC` (the Block playout golden additionally skips under a non-UTC `TZ`; see Golden-file nets). |
|
|
| `ErsatzTV.Scanner.Tests` | Library scanning: scan handlers, folder scanners, NFO readers | Handler tests substitute the folder scanners + `ILibraryRepository` and assert the resulting repository writes (e.g. `ScanLocalLibraryHandlerTests` pins which `LastScan` levels a scan records — ersatztv#264). Fakes/`Testably` back the file-system-facing scanners. ~1,485 tests (approximate on purpose — an exact count goes stale on every PR that adds one). Additionally contains `Core/FFmpeg/TranscodingTests` — `[Explicit]` + `[Combinatorial]`, so it never runs in CI or a plain `dotnet test` (it needs real ffmpeg/hardware) and contributes 0 to that count; run it by name when touching the transcoding pipeline. |
|
|
| `ErsatzTV.Architecture.Tests` | Layering rules via NetArchTest.eNhancedEdition | Core↛Infra/App/EF; FFmpeg↛all; App↛concrete providers. 5 tests. See `docs/contributing.md` §1. |
|
|
| `ErsatzTV.FFmpeg.Tests` | FFmpeg command construction | Build a pipeline, assert the exact rendered arg string (`PipelineBuilderBaseTests.cs`). |
|
|
| `web/` (vitest) | React SPA unit tests | Run alongside typecheck + build (see below). Collects `src/**`, `web/scripts/**` *and* `web/vite-plugins/**`, but deliberately **excludes** `web/e2e/**` (the Playwright specs — vitest's default `**/*.spec.*` glob would otherwise run them under jsdom). Two files have git prerequisites since ersatztv#819. `web/src/api/pageSizeCallSites.guard.test.ts` needs a git **checkout** AND, through it, the **binary** — it derives its file population from `git ls-files` via `web/vite-plugins/trackedSourceFiles.ts` rather than a directory walk, and refuses rather than falling back. `web/vite-plugins/trackedSourceFiles.realgit.test.ts` needs the **binary** but no checkout: it builds its own temp repository to prove that derivation by executing it. So it is not checkout-versus-binary — supplying a `.git` alone would not let either run. Every other file runs fine with neither. `docker/Dockerfile`'s web-build stage runs the suite with exactly those two `--exclude`d, because its context carries no `.git` and `node:22-bookworm-slim` ships no git — and the two exclusions overlap rather than divide — both files need the binary. |
|
|
| `web/e2e/` (Playwright) | UI-interactive E2E flows against a **live** instance | Not a unit suite and **not** part of `npm test` — needs a running server, so it runs via `scripts/e2e-ui.sh` (boots its own fresh instance) and in CI as a step of the `functional-e2e` job. Headless Chromium, `serial`, `retries: 0`. Scope rule: assert only what the curl harness structurally cannot. See `docs/e2e-local.md` → "UI-E2E harness". |
|
|
|
|
## Golden-file nets
|
|
|
|
Three golden-file suites guard the highest-value, most-subtle output:
|
|
|
|
- **M3U**: `ErsatzTV.Core.Tests/Iptv/ChannelPlaylistGoldenTests.cs` (ersatztv#11) — env var `ETV_UPDATE_GOLDENS`
|
|
- **XMLTV**: `ChannelGuideGoldenTests` (ersatztv#28) — env var `ETV_UPDATE_GOLDENS`
|
|
- **Playout build**: `ErsatzTV.Core.Tests/Scheduling/Goldens/PlayoutBuildGoldenTests.cs` (ersatztv#163)
|
|
— env var **`ETV_UPDATE_PLAYOUT_GOLDENS`** (deliberately separate from `ETV_UPDATE_GOLDENS` so
|
|
regenerating one net can't silently rewrite the other). Snapshots the `PlayoutItem`s each builder
|
|
produces over a pinned build window. Covers the **Classic** (`PlaybackOrder.Chronological`),
|
|
**Block**, and **Sequential (YAML)** builders. The **Scripted** *end-to-end pipeline* is excluded from this
|
|
net — `ScriptedPlayoutBuilder` runs a user-authored external process that drives the engine over HTTP, which
|
|
the in-memory harness can't pin; that integration harness is tracked in ersatztv#563. The scheduling
|
|
*behavior* those scripts drive, though, lives in the in-process `SchedulingEngine` (the HTTP controller is a
|
|
1:1 pass-through) and IS directly testable — `SchedulingEngineTests` news it up with substitutes, and
|
|
`ContentEnumeratorBuilderTests` (ersatztv#395) is the direct regression net over the enumerator-construction
|
|
helper the Scripted and Sequential/YAML engines now share (decision:
|
|
`testing.scripted-playout-golden-deferred`). The **Sequential** case
|
|
(`Sequential_yaml`, ersatztv#381)
|
|
builds from a committed YAML fixture (`Goldens/Fixtures/sequential-schedule.yml`) instead of a
|
|
`ProgramSchedule`; it is TZ-independent (the `count`/`all`/`duration` handlers do UTC-only arithmetic —
|
|
it passes, not skips, under a non-UTC `TZ`) so needs no `Assume` guard. A third case,
|
|
`Classic_clock_padded` (ersatztv#77), locks clock-boundary padding: a `FillerMode.Pad` +
|
|
`PadToNearestMinute=15` PostRoll preset snaps content to `:15`, and the test both goldens the output and
|
|
asserts every content item after the first starts on a quarter-hour. Its EPG counterpart is
|
|
`ErsatzTV.Core.Tests/Channels/ChannelGuideProjectorClockPadTests.cs` (guide programmes stop on the padded
|
|
boundary). The build reads
|
|
no wall clock — time enters only via the caller-supplied `start` — so a pinned `start` is fully
|
|
deterministic. Snapshots the raw `PlayoutItem.Start`/`Finish` (UTC), **not** the `*Offset` properties
|
|
(those call `.ToLocalTime()` and would make the golden machine-TZ dependent). The **Block** case is
|
|
TZ-sensitive by construction (`BlockPlayoutBuilder` maps template times via `TimeZoneInfo.Local`), so
|
|
it is guarded with `Assume.That(TimeZoneInfo.Local.BaseUtcOffset == Zero)`: it runs under `TZ=UTC`
|
|
(CI) and reports **inconclusive** (a graceful skip, not a failure) under any other TZ. A real TZ seam
|
|
for the block builder is ersatztv#380's scope.
|
|
|
|
All three locate their golden files via `[CallerFilePath]`. A missing golden is a hard fail, not a
|
|
skip. Regenerate via `ETV_UPDATE_GOLDENS=1 dotnet test ...` (M3U/XMLTV) or
|
|
`ETV_UPDATE_PLAYOUT_GOLDENS=1 dotnet test ...` (playout build).
|
|
|
|
**Never set `ETV_UPDATE_GOLDENS` / `ETV_UPDATE_PLAYOUT_GOLDENS` in CI or from an agent.** A golden
|
|
diff during normal test runs means the code changed the output — regenerating to make the diff go
|
|
away hides the change instead of surfacing it. Only a human who has confirmed the change is
|
|
intentional should regenerate.
|
|
|
|
## Timezone independence
|
|
|
|
The suite is timezone-independent (ersatztv#24). When constructing test `PlayoutItem`s, always
|
|
set a real `Start` (e.g. `startState.CurrentTime.UtcDateTime`) — never rely on the default
|
|
`DateTime.MinValue`, which underflows `DateTimeOffset.MinValue` once a non-UTC local offset is
|
|
applied (`StartOffset` calls `ToLocalTime()`). CI runs in UTC; local runs may not.
|
|
|
|
## Running tests
|
|
|
|
Full .NET gate:
|
|
|
|
```bash
|
|
dotnet build ErsatzTV.sln
|
|
TZ=UTC dotnet test
|
|
```
|
|
|
|
CI adds `--blame-hang-timeout 2m` to catch hangs.
|
|
|
|
Fast subsets:
|
|
|
|
```bash
|
|
# single project
|
|
dotnet test ErsatzTV.Tests
|
|
|
|
# filtered
|
|
dotnet test ErsatzTV.Core.Tests --filter FullyQualifiedName~ChannelPlaylistGoldenTests
|
|
```
|
|
|
|
Web (`web/`):
|
|
|
|
```bash
|
|
npm test -- --run # vitest, single pass — use this for the SPA guards (see below)
|
|
npm test # vitest WATCH mode; `pageSizeCallSites.guard.test.ts` reads the git index once
|
|
# per dev-server lifetime, so a file created or staged mid-session is judged
|
|
# against a stale index and reddens misleadingly. Restart the watcher, or --run.
|
|
npm run typecheck # tsc -b --pretty false
|
|
npm run lint # eslint .
|
|
npm run build # tsc -b && vite build
|
|
```
|
|
|
|
UI-E2E (needs a built solution + built SPA; boots and tears down its own instance):
|
|
|
|
```bash
|
|
scripts/e2e-ui.sh # from the repo root, NOT web/
|
|
```
|
|
|
|
## Per-PR verification gate
|
|
|
|
Before opening a PR: build the solution, run `ErsatzTV.Tests` + `ErsatzTV.Core.Tests` (plus
|
|
`ErsatzTV.Scanner.Tests`, `ErsatzTV.Architecture.Tests` and `ErsatzTV.FFmpeg.Tests` if touched),
|
|
and run the web test/lint/typecheck/build steps above. All must be green. A golden-file diff or
|
|
an architecture-test failure is a hard stop — fix the code, don't regenerate/relax the test.
|
|
|
|
## See also
|
|
|
|
- `docs/ci-cd.md` — CI pipeline (test → migrations → build), versioning, dependency management.
|
|
- `docs/contributing.md` §1 — layering rules enforced by `ErsatzTV.Architecture.Tests`.
|
|
- `docs/contributing.md` §8 — short pointer back to this doc.
|