Files

T

timothy b1321a2144 fix(819): residual (6) is BLIND as well as noisy, and "restart the watcher" was wrong advice

Round 14 found no blocker in the mechanism — the Docker stage, all six residual
mutations and every gate reproduce exactly as documented — and one more false
direction claim from me. That is the fourth in a row, and the class has been the
only defect for six rounds: a coverage or direction sentence asserted without
measurement, in a change whose subject is the difference between a check and the
appearance of one.

Residual (6) was labelled fail-NOISY in four places. MEASURED here before fixing,
with a file that is already untracked when the watcher starts and `git add`ed
mid-session: watch mode is green across both phases, while `npx vitest run` on
that identical tree reports `UNREGISTERED (1) — screens/Blind819.tsx:literal:100`.
The stale module still answers `others`, so the file never enters the population
and its call sites are never scanned. Blind, not noisy — and a window that did
NOT exist before #819, since the population was then the walk, which always
contained the file.

Worse, the remedy I wrote into `docs/testing.md` — "restart the watcher" —
resolves the noisy case by putting the developer into the blind one, because
after a restart the file is untracked at session start. It now says what is
actually true: a watch-mode green is not authoritative for this guard, confirm
with `npm test -- --run`.

(6) is now stated in both directions with the sequence that produces each, and is
explicitly the one residual that belongs to neither the noisy nor the blind
bullet — previously the taxonomy silently covered five of six.

The `configureServer` rejection stands but was stated too broadly. It is not
"a noisy red traded for a blind green": the hook fixes the created-mid-session
red and ADDITIONALLY blinds the create-then-stage sequence, while the
already-untracked-then-staged sequence is blind either way, because `git add`
fires no watcher event in either design.

Also: `docs/guard-inventory.md` said FIVE residuals and then described a sixth,
and still carried "the walk set and the index set are the same set" — the exact
phrase the previous commit corrected in the guard file and did not mirror here.
Two-copy drift again, in the paragraph that warns about it.

Refs: #819

2026-08-29 09:36:39 +02:00

10 KiB

Raw Blame History

Testing map

Purpose: authoritative map of what each test project/suite covers and how to run it. Read this before adding tests, not just docs/contributing.md §8 (which now just points here).

Test projects

Font dependency (ersatztv#732). Infrastructure/Graphics/TextElementBackgroundBoxTests is the only suite that rasterises text, so it needs at least one system font to lay anything out. The CI image installs none explicitly — fonts arrive via playwright install --with-deps in docker/ci/Dockerfile (351 present in the pinned image, measured 2026-08-26). This is a real dependency, declared here so a future slimming of that install produces a known cause rather than a mystery red. Baseline_Renders_A_Non_Empty_Bitmap exists to make that failure loud: without it a fontless host would render a 0x0 bitmap and every relative geometry assertion would pass vacuously.

Project	Covers	Notes
`ErsatzTV.Tests`	API controllers + MediatR handlers, plus the SkiaSharp text-overlay rasteriser (`Infrastructure/Graphics/`)	In-memory SQLite fixture: a shared `SqliteConnection("Data Source=:memory:;Foreign Keys=False")` kept open + `EnsureCreatedAsync()` (not full migration replay) + `PRAGMA foreign_keys=OFF`, then seed; a tiny `IDbContextFactory` wraps `new TvContext(...)`. ~1,870 tests (approximate on purpose — an exact count goes stale on every PR that adds one; the previous hardcoded 828 was off by over a thousand).
`ErsatzTV.Core.Tests`	Domain logic, scheduling, IPTV/XMLTV generation	References `ErsatzTV.Application` directly — there is no separate `Application.Tests` project. ~650 tests + 1 skipped under `TZ=UTC` (the Block playout golden additionally skips under a non-UTC `TZ`; see Golden-file nets).
`ErsatzTV.Scanner.Tests`	Library scanning: scan handlers, folder scanners, NFO readers	Handler tests substitute the folder scanners + `ILibraryRepository` and assert the resulting repository writes (e.g. `ScanLocalLibraryHandlerTests` pins which `LastScan` levels a scan records — ersatztv#264). Fakes/`Testably` back the file-system-facing scanners. ~1,485 tests (approximate on purpose — an exact count goes stale on every PR that adds one). Additionally contains `Core/FFmpeg/TranscodingTests` — `[Explicit]` + `[Combinatorial]`, so it never runs in CI or a plain `dotnet test` (it needs real ffmpeg/hardware) and contributes 0 to that count; run it by name when touching the transcoding pipeline.
`ErsatzTV.Architecture.Tests`	Layering rules via NetArchTest.eNhancedEdition	Core↛Infra/App/EF; FFmpeg↛all; App↛concrete providers. 5 tests. See `docs/contributing.md` §1.
`ErsatzTV.FFmpeg.Tests`	FFmpeg command construction	Build a pipeline, assert the exact rendered arg string (`PipelineBuilderBaseTests.cs`).
`web/` (vitest)	React SPA unit tests	Run alongside typecheck + build (see below). Collects `src/`, `web/scripts/` and `web/vite-plugins/`, but deliberately excludes `web/e2e/` (the Playwright specs — vitest's default `*/.spec.` glob would otherwise run them under jsdom). Two files have git prerequisites since ersatztv#819. `web/src/api/pageSizeCallSites.guard.test.ts` needs a git checkout* AND, through it, the binary — it derives its file population from `git ls-files` via `web/vite-plugins/trackedSourceFiles.ts` rather than a directory walk, and refuses rather than falling back. `web/vite-plugins/trackedSourceFiles.realgit.test.ts` needs the binary but no checkout: it builds its own temp repository to prove that derivation by executing it. So it is not checkout-versus-binary — supplying a `.git` alone would not let either run. Every other file runs fine with neither. `docker/Dockerfile`'s web-build stage runs the suite with exactly those two `--exclude`d, because its context carries no `.git` and `node:22-bookworm-slim` ships no git — and the two exclusions overlap rather than divide — both files need the binary.
`web/e2e/` (Playwright)	UI-interactive E2E flows against a live instance	Not a unit suite and not part of `npm test` — needs a running server, so it runs via `scripts/e2e-ui.sh` (boots its own fresh instance) and in CI as a step of the `functional-e2e` job. Headless Chromium, `serial`, `retries: 0`. Scope rule: assert only what the curl harness structurally cannot. See `docs/e2e-local.md` → "UI-E2E harness".

Golden-file nets

Three golden-file suites guard the highest-value, most-subtle output:

M3U: ErsatzTV.Core.Tests/Iptv/ChannelPlaylistGoldenTests.cs (ersatztv#11) — env var ETV_UPDATE_GOLDENS
XMLTV: ChannelGuideGoldenTests (ersatztv#28) — env var ETV_UPDATE_GOLDENS
Playout build: ErsatzTV.Core.Tests/Scheduling/Goldens/PlayoutBuildGoldenTests.cs (ersatztv#163) — env var ETV_UPDATE_PLAYOUT_GOLDENS (deliberately separate from ETV_UPDATE_GOLDENS so regenerating one net can't silently rewrite the other). Snapshots the PlayoutItems each builder produces over a pinned build window. Covers the Classic (PlaybackOrder.Chronological), Block, and Sequential (YAML) builders. The Scripted end-to-end pipeline is excluded from this net — ScriptedPlayoutBuilder runs a user-authored external process that drives the engine over HTTP, which the in-memory harness can't pin; that integration harness is tracked in ersatztv#563. The scheduling behavior those scripts drive, though, lives in the in-process SchedulingEngine (the HTTP controller is a 1:1 pass-through) and IS directly testable — SchedulingEngineTests news it up with substitutes, and ContentEnumeratorBuilderTests (ersatztv#395) is the direct regression net over the enumerator-construction helper the Scripted and Sequential/YAML engines now share (decision: testing.scripted-playout-golden-deferred). The Sequential case (Sequential_yaml, ersatztv#381) builds from a committed YAML fixture (Goldens/Fixtures/sequential-schedule.yml) instead of a ProgramSchedule; it is TZ-independent (the count/all/duration handlers do UTC-only arithmetic — it passes, not skips, under a non-UTC TZ) so needs no Assume guard. A third case, Classic_clock_padded (ersatztv#77), locks clock-boundary padding: a FillerMode.Pad + PadToNearestMinute=15 PostRoll preset snaps content to :15, and the test both goldens the output and asserts every content item after the first starts on a quarter-hour. Its EPG counterpart is ErsatzTV.Core.Tests/Channels/ChannelGuideProjectorClockPadTests.cs (guide programmes stop on the padded boundary). The build reads no wall clock — time enters only via the caller-supplied start — so a pinned start is fully deterministic. Snapshots the raw PlayoutItem.Start/Finish (UTC), not the *Offset properties (those call .ToLocalTime() and would make the golden machine-TZ dependent). The Block case is TZ-sensitive by construction (BlockPlayoutBuilder maps template times via TimeZoneInfo.Local), so it is guarded with Assume.That(TimeZoneInfo.Local.BaseUtcOffset == Zero): it runs under TZ=UTC (CI) and reports inconclusive (a graceful skip, not a failure) under any other TZ. A real TZ seam for the block builder is ersatztv#380's scope.

All three locate their golden files via [CallerFilePath]. A missing golden is a hard fail, not a skip. Regenerate via ETV_UPDATE_GOLDENS=1 dotnet test ... (M3U/XMLTV) or ETV_UPDATE_PLAYOUT_GOLDENS=1 dotnet test ... (playout build).

Never set ETV_UPDATE_GOLDENS / ETV_UPDATE_PLAYOUT_GOLDENS in CI or from an agent. A golden diff during normal test runs means the code changed the output — regenerating to make the diff go away hides the change instead of surfacing it. Only a human who has confirmed the change is intentional should regenerate.

Timezone independence

The suite is timezone-independent (ersatztv#24). When constructing test PlayoutItems, always set a real Start (e.g. startState.CurrentTime.UtcDateTime) — never rely on the default DateTime.MinValue, which underflows DateTimeOffset.MinValue once a non-UTC local offset is applied (StartOffset calls ToLocalTime()). CI runs in UTC; local runs may not.

Running tests

Full .NET gate:

dotnet build ErsatzTV.sln
TZ=UTC dotnet test

CI adds --blame-hang-timeout 2m to catch hangs.

Fast subsets:

# single project
dotnet test ErsatzTV.Tests

# filtered
dotnet test ErsatzTV.Core.Tests --filter FullyQualifiedName~ChannelPlaylistGoldenTests

Web (web/):

npm test -- --run  # vitest, single pass — use this for the SPA guards (see below)
npm test           # vitest WATCH mode. `pageSizeCallSites.guard.test.ts` reads the git index ONCE
                   # per dev-server lifetime while the glob refreshes, so it drifts BOTH ways: a
                   # file created mid-session reddens it misleadingly, and a file that was already
                   # untracked when the watcher started stays invisible to it after `git add` — a
                   # green that is not authoritative. Restarting the watcher swaps the first
                   # problem for the second; confirm with `npm test -- --run`.
npm run typecheck  # tsc -b --pretty false
npm run lint       # eslint .
npm run build      # tsc -b && vite build

UI-E2E (needs a built solution + built SPA; boots and tears down its own instance):

scripts/e2e-ui.sh              # from the repo root, NOT web/

Per-PR verification gate

Before opening a PR: build the solution, run ErsatzTV.Tests + ErsatzTV.Core.Tests (plus ErsatzTV.Scanner.Tests, ErsatzTV.Architecture.Tests and ErsatzTV.FFmpeg.Tests if touched), and run the web test/lint/typecheck/build steps above. All must be green. A golden-file diff or an architecture-test failure is a hard stop — fix the code, don't regenerate/relax the test.

10 KiB Raw Blame History