Files
ersatztv/docs/testing.md
T
timothy 157d16df67 fix(819): the Docker stage needs the git BINARY excluded too, not just the index
Round 8 found a BLOCKER, and it was my verification that was wrong rather than
an argument. The previous commit re-armed `npm test` in the web-build stage and
excluded the one file that needs a git CHECKOUT. It did not exclude the file
added in round 6 that needs the git BINARY: `trackedSourceFiles`' real-repository
tests shell out to `git` to prove the derivation by executing it, and
`node:22-bookworm-slim` ships no git. So `docker build` would have failed on
every push to `main` and every `v*` tag — no image ships.

Why I did not catch it: every "gitless replica" I ran deleted `.git` and left the
git binary on `PATH`. That reproduces one of the two preconditions. Re-measured
with `PATH` stripped, the previous head gives `spawnSync git ENOENT`,
`Test Files 1 failed | 116 passed`. The Dockerfile comment stated the right
premise — "no `.git`, AND node:22-bookworm-slim ships no git binary" — two lines
above the line that only acted on the first half.

The real-git block is now its own file, `trackedSourceFiles.realgit.test.ts`, so
excluding it costs two tests rather than the thirteen injected-runner ones that
run fine with no git at all. Both exclusions are named in the Dockerfile with
their DIFFERENT reasons, and with the instruction to verify any change there with
the binary off `PATH`. MEASURED in a replica with neither `.git` nor `git`:
117 files / 1225 tests pass, lint, typecheck and build all exit 0.

Also from round 8:
- `docs/testing.md`'s row asserted the suite "cannot run against an exported tree
  with no index" and that the Dockerfile "no longer runs it" — both false at the
  previous head. It now distinguishes the checkout requirement from the binary
  requirement, which is the distinction the blocker turned on.
- Two clauses this PR added had no proof and MEASURED green when disarmed: the
  trailing-slash restriction in `unaccountedWalkedPaths` (without it every
  `others` entry becomes a prefix mask) and the plugin's dedupe. Both now have
  fixtures, and both mutations redden by name.
- The dedupe's stated rationale was wrong: duplicates appear on both sides of the
  ratio the floor computes, so they cancel. Re-stated as tidiness, with the
  measurement, and "three times" corrected to two-or-three (a both-added conflict
  has two stages).
- Three JSDoc blocks were stacked above one function, so two of them documented
  functions defined 15 and 50 lines below and attached to the wrong ones. Pure
  reordering, plus the blank lines that were detaching them.
- `vite-env.d.ts` named the wrong mechanism for its loud-failure claim: a renamed
  field yields an empty Set, caught by the anti-vacuity floor rather than by a
  throw at first read.
- The Dockerfile no longer hardcodes a test-file count that nothing reddens on.

Refs: #819
2026-08-29 05:12:23 +02:00

9.3 KiB

Testing map

Purpose: authoritative map of what each test project/suite covers and how to run it. Read this before adding tests, not just docs/contributing.md §8 (which now just points here).

Test projects

Font dependency (ersatztv#732). Infrastructure/Graphics/TextElementBackgroundBoxTests is the only suite that rasterises text, so it needs at least one system font to lay anything out. The CI image installs none explicitly — fonts arrive via playwright install --with-deps in docker/ci/Dockerfile (351 present in the pinned image, measured 2026-08-26). This is a real dependency, declared here so a future slimming of that install produces a known cause rather than a mystery red. Baseline_Renders_A_Non_Empty_Bitmap exists to make that failure loud: without it a fontless host would render a 0x0 bitmap and every relative geometry assertion would pass vacuously.

Project Covers Notes
ErsatzTV.Tests API controllers + MediatR handlers, plus the SkiaSharp text-overlay rasteriser (Infrastructure/Graphics/) In-memory SQLite fixture: a shared SqliteConnection("Data Source=:memory:;Foreign Keys=False") kept open + EnsureCreatedAsync() (not full migration replay) + PRAGMA foreign_keys=OFF, then seed; a tiny IDbContextFactory wraps new TvContext(...). ~1,870 tests (approximate on purpose — an exact count goes stale on every PR that adds one; the previous hardcoded 828 was off by over a thousand).
ErsatzTV.Core.Tests Domain logic, scheduling, IPTV/XMLTV generation References ErsatzTV.Application directly — there is no separate Application.Tests project. ~650 tests + 1 skipped under TZ=UTC (the Block playout golden additionally skips under a non-UTC TZ; see Golden-file nets).
ErsatzTV.Scanner.Tests Library scanning: scan handlers, folder scanners, NFO readers Handler tests substitute the folder scanners + ILibraryRepository and assert the resulting repository writes (e.g. ScanLocalLibraryHandlerTests pins which LastScan levels a scan records — ersatztv#264). Fakes/Testably back the file-system-facing scanners. ~1,485 tests (approximate on purpose — an exact count goes stale on every PR that adds one). Additionally contains Core/FFmpeg/TranscodingTests — [Explicit] + [Combinatorial], so it never runs in CI or a plain dotnet test (it needs real ffmpeg/hardware) and contributes 0 to that count; run it by name when touching the transcoding pipeline.
ErsatzTV.Architecture.Tests Layering rules via NetArchTest.eNhancedEdition Core↛Infra/App/EF; FFmpeg↛all; App↛concrete providers. 5 tests. See docs/contributing.md §1.
ErsatzTV.FFmpeg.Tests FFmpeg command construction Build a pipeline, assert the exact rendered arg string (PipelineBuilderBaseTests.cs).
web/ (vitest) React SPA unit tests Run alongside typecheck + build (see below). Collects src/**, web/scripts/** and web/vite-plugins/**, but deliberately excludes web/e2e/** (the Playwright specs — vitest's default **/*.spec.* glob would otherwise run them under jsdom). Two files have git prerequisites since ersatztv#819, and they differ: web/src/api/pageSizeCallSites.guard.test.ts needs a git checkout (it derives its file population from git ls-files via web/vite-plugins/trackedSourceFiles.ts rather than a directory walk, and refuses rather than falling back), while web/vite-plugins/trackedSourceFiles.realgit.test.ts needs the git binary (it builds a real temp repository to prove that derivation by executing it). Every other file runs fine with neither. docker/Dockerfile's web-build stage runs the suite with exactly those two --excluded, because its context carries no .git and node:22-bookworm-slim ships no git — and the two exclusions are there for those two different reasons.
web/e2e/ (Playwright) UI-interactive E2E flows against a live instance Not a unit suite and not part of npm test — needs a running server, so it runs via scripts/e2e-ui.sh (boots its own fresh instance) and in CI as a step of the functional-e2e job. Headless Chromium, serial, retries: 0. Scope rule: assert only what the curl harness structurally cannot. See docs/e2e-local.md → "UI-E2E harness".

Golden-file nets

Three golden-file suites guard the highest-value, most-subtle output:

  • M3U: ErsatzTV.Core.Tests/Iptv/ChannelPlaylistGoldenTests.cs (ersatztv#11) — env var ETV_UPDATE_GOLDENS
  • XMLTV: ChannelGuideGoldenTests (ersatztv#28) — env var ETV_UPDATE_GOLDENS
  • Playout build: ErsatzTV.Core.Tests/Scheduling/Goldens/PlayoutBuildGoldenTests.cs (ersatztv#163) — env var ETV_UPDATE_PLAYOUT_GOLDENS (deliberately separate from ETV_UPDATE_GOLDENS so regenerating one net can't silently rewrite the other). Snapshots the PlayoutItems each builder produces over a pinned build window. Covers the Classic (PlaybackOrder.Chronological), Block, and Sequential (YAML) builders. The Scripted end-to-end pipeline is excluded from this net — ScriptedPlayoutBuilder runs a user-authored external process that drives the engine over HTTP, which the in-memory harness can't pin; that integration harness is tracked in ersatztv#563. The scheduling behavior those scripts drive, though, lives in the in-process SchedulingEngine (the HTTP controller is a 1:1 pass-through) and IS directly testable — SchedulingEngineTests news it up with substitutes, and ContentEnumeratorBuilderTests (ersatztv#395) is the direct regression net over the enumerator-construction helper the Scripted and Sequential/YAML engines now share (decision: testing.scripted-playout-golden-deferred). The Sequential case (Sequential_yaml, ersatztv#381) builds from a committed YAML fixture (Goldens/Fixtures/sequential-schedule.yml) instead of a ProgramSchedule; it is TZ-independent (the count/all/duration handlers do UTC-only arithmetic — it passes, not skips, under a non-UTC TZ) so needs no Assume guard. A third case, Classic_clock_padded (ersatztv#77), locks clock-boundary padding: a FillerMode.Pad + PadToNearestMinute=15 PostRoll preset snaps content to :15, and the test both goldens the output and asserts every content item after the first starts on a quarter-hour. Its EPG counterpart is ErsatzTV.Core.Tests/Channels/ChannelGuideProjectorClockPadTests.cs (guide programmes stop on the padded boundary). The build reads no wall clock — time enters only via the caller-supplied start — so a pinned start is fully deterministic. Snapshots the raw PlayoutItem.Start/Finish (UTC), not the *Offset properties (those call .ToLocalTime() and would make the golden machine-TZ dependent). The Block case is TZ-sensitive by construction (BlockPlayoutBuilder maps template times via TimeZoneInfo.Local), so it is guarded with Assume.That(TimeZoneInfo.Local.BaseUtcOffset == Zero): it runs under TZ=UTC (CI) and reports inconclusive (a graceful skip, not a failure) under any other TZ. A real TZ seam for the block builder is ersatztv#380's scope.

All three locate their golden files via [CallerFilePath]. A missing golden is a hard fail, not a skip. Regenerate via ETV_UPDATE_GOLDENS=1 dotnet test ... (M3U/XMLTV) or ETV_UPDATE_PLAYOUT_GOLDENS=1 dotnet test ... (playout build).

Never set ETV_UPDATE_GOLDENS / ETV_UPDATE_PLAYOUT_GOLDENS in CI or from an agent. A golden diff during normal test runs means the code changed the output — regenerating to make the diff go away hides the change instead of surfacing it. Only a human who has confirmed the change is intentional should regenerate.

Timezone independence

The suite is timezone-independent (ersatztv#24). When constructing test PlayoutItems, always set a real Start (e.g. startState.CurrentTime.UtcDateTime) — never rely on the default DateTime.MinValue, which underflows DateTimeOffset.MinValue once a non-UTC local offset is applied (StartOffset calls ToLocalTime()). CI runs in UTC; local runs may not.

Running tests

Full .NET gate:

dotnet build ErsatzTV.sln
TZ=UTC dotnet test

CI adds --blame-hang-timeout 2m to catch hangs.

Fast subsets:

# single project
dotnet test ErsatzTV.Tests

# filtered
dotnet test ErsatzTV.Core.Tests --filter FullyQualifiedName~ChannelPlaylistGoldenTests

Web (web/):

npm test           # vitest (excludes web/e2e — those need a live server)
npm run typecheck  # tsc -b --pretty false
npm run lint       # eslint .
npm run build      # tsc -b && vite build

UI-E2E (needs a built solution + built SPA; boots and tears down its own instance):

scripts/e2e-ui.sh              # from the repo root, NOT web/

Per-PR verification gate

Before opening a PR: build the solution, run ErsatzTV.Tests + ErsatzTV.Core.Tests (plus ErsatzTV.Scanner.Tests, ErsatzTV.Architecture.Tests and ErsatzTV.FFmpeg.Tests if touched), and run the web test/lint/typecheck/build steps above. All must be green. A golden-file diff or an architecture-test failure is a hard stop — fix the code, don't regenerate/relax the test.

See also

  • docs/ci-cd.md — CI pipeline (test → migrations → build), versioning, dependency management.
  • docs/contributing.md §1 — layering rules enforced by ErsatzTV.Architecture.Tests.
  • docs/contributing.md §8 — short pointer back to this doc.