Compare commits

...
Author SHA1 Message Date
timothy 601d855b25 Merge pull request 'docs(release): v26.11.0 headline is the QSV VA-API decode fix (#498) + unified logo bug (#67)' (#507) from release/v26.11.0-notes into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build CI Toolchain Image / Build & push CI image (push) Successful in 21s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 7m13s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 19m34s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 7m53s
2026-07-20 20:22:04 +00:00
timothy a352d832ad docs(release): v26.11.0 headline is the QSV VA-API decode fix (#498) + unified logo bug (#67)
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 9s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 17s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 9s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 33s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 9s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 38s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 21s
The release range grew after the host-correction PR landed: #67 and #498 both
merged into main. #498 in particular flips a default (QsvPreferNativeDecoder is
ON), so it leads the row rather than sitting in a fixes list.
2026-07-20 22:20:52 +02:00
timothy 36499a96e0 Merge pull request 'feat(498): QSV profiles can decode via VA-API (Prefer native decoder)' (#506) from feat/498-qsv-native-decode into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
Build ErsatzTV Image / Build & test (.NET) (push) Has been cancelled
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Has been cancelled
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Has been cancelled
2026-07-20 20:16:36 +00:00
timothyandClaude Opus 4.8 374528250b fix(498): address final-review findings (Windows guard, Nut parity, docs, test)
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 14s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m19s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m19s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 12m49s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m39s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m17s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 8s
From the Fable whole-branch review:
- M1: guard native VA-API decode with !OperatingSystem.IsWindows() (no vaapi
  hwaccel on Windows; QSV caps over-reported there) — DRY'd into a preferNativeDecode local.
- L2: IsIntelVaapiOrQsv also matches decoder mode Vaapi, preserving Intel
  audio-dup parity on the (producerless) Nut-output branch.
- L1: replace the vacuous ShouldNotContain(" deinterlace_qsv") with an
  occurrence-count assertion that actually catches a second bare occurrence.
- N1/H1: decisions.md — correct the column to nullable-with-default (not NOT NULL),
  and record the accepted HDR software-tonemap trade-off + Linux-only guard, with
  the tonemap_qsv optimization tracked in #505.

H1 (HDR tonemap reroute) accepted-and-deferred per that decision; #505 filed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:54:13 +02:00
timothyandClaude Opus 4.8 264b17516d style(498): normalize pre-existing whitespace in touched files (#311 fix-as-you-touch)
Whitespace-only (git diff -w is empty); the #311 format job checks whole
touched files, and these two legacy files carried pre-existing violations
never caught before (no PR had touched them since the gate landed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:54:13 +02:00
timothy ed2bce4994 feat(498): add 'Prefer native decoder' checkbox to QSV FFmpeg profiles 2026-07-20 21:54:13 +02:00
timothy 9cbd1fdb5c chore(498): regenerate OpenAPI + SPA types for QsvPreferNativeDecoder 2026-07-20 21:54:13 +02:00
timothy be27d9ab8d feat(498): carry QsvPreferNativeDecoder through application + REST layer 2026-07-20 21:54:13 +02:00
timothy 993b6fd00c feat(498): QSV pipeline decodes via VA-API to software when preferring native decoder 2026-07-20 21:54:13 +02:00
timothy 5edc45ab76 feat(498): pass QsvPreferNativeDecoder from profile into FFmpegState 2026-07-20 21:54:13 +02:00
timothy 79c396d5ed docs(498): record QsvPreferNativeDecoder decision
Field-reference update skipped: docs/channels.md and docs/domain-model.md
have no per-field FFmpeg-profile catalogue (channels.md's Encoding bullet
is a one-line summary, not a field list; grep for QsvExtraHardwareFrames
or HardwareAcceleration finds no such list in either doc).
2026-07-20 21:54:13 +02:00
timothy 626d5e7365 feat(498): QsvHardwareAccelerationOption emits VA-API decode when preferring native decoder 2026-07-20 21:53:16 +02:00
timothy 46fdbd282f feat(498): thread QsvPreferNativeDecoder through FFmpegState 2026-07-20 21:53:16 +02:00
timothy 381a1c2029 feat(498): add QsvPreferNativeDecoder domain field + migration (default on) 2026-07-20 21:53:16 +02:00
timothyandClaude Opus 4.8 cdbb685d22 docs(498): fold Fable review fixes into plan + spec
- Blocker 1: VA-API decode to SOFTWARE frames (drop -hwaccel_output_format,
  new DecoderVaapiToSoftware) so the proven hwupload/vpp_qsv branch bridges to
  the QSV encoder — the naive hardware-surface path emits a bare vpp_qsv on
  VA-API frames and fails on ~all content.
- Blocker 2: bool? domain property + != false coercion (DeinterlaceVideo
  pattern) so create-with-false actually persists false.
- High 3: REST DTOs bool?=null + ?? true for /api/v1 additive-compat.
- Medium 4: correct Task 4 test scaffolding (DefaultHardwareCapabilities).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:53:16 +02:00
timothyandClaude Opus 4.8 369ba3332e docs(498): implementation plan — VA-API decode on the QSV pipeline
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:53:16 +02:00
timothyandClaude Opus 4.8 32593a2263 docs(498): design spec — VA-API decode on the QSV pipeline
Mirrors Jellyfin's default-on "prefer native decoder" hybrid via a new
QsvPreferNativeDecoder profile boolean. Reverified against code before
designing; records the rejected decode-family-enum alternative (Option C).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:53:16 +02:00
timothy 72dc1c9fba Merge pull request 'feat(67): one logo drives both the listing and the on-screen bug' (#504) from feat/67-unified-logo-bug into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 22s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 22s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m57s
2026-07-20 19:36:43 +00:00
timothy 893d4a6398 test(67): de-vacuify the template-stamping assertion; cover adopt-then-delete
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 11s
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 9s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 10s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 1m6s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 13m25s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 16m1s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 20m26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m19s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Whole-branch review found the stamping test was structurally vacuous: a bare
foreach over ChannelTemplates.Where(IsSystem) passes with zero iterations, so
the test would have stayed green if template seeding silently bailed out.
Assert the collection is non-empty first. Proven non-vacuous by a negative
control (forcing SeedChannelTemplates to bail makes exactly this test fail).

Also cover the ACTUAL production sequence -- adopt an existing hand-made row,
then delete it -- which the previous no-resurrect test did not exercise (it
covered seed-then-delete). The marker is written on the adopt path too, so
the deleted row must stay deleted.

docs: note that a deleted preset degrades to no default rather than failing,
and that the default applies to newly created channels, not retroactively.

Refs #67
2026-07-20 21:13:47 +02:00
timothy 5d0a70a20f merge(67): new channels default to the logo-bug preset (T6) 2026-07-20 21:06:21 +02:00
timothy c1bd8c067a feat(67): default new channels to the logo-bug watermark preset 2026-07-20 21:05:38 +02:00
timothy 0cad333653 merge(67): channel editor logo-bug toggle + preview (T5) 2026-07-20 21:03:04 +02:00
timothyandClaude Opus 4.8 2176b73e23 fix(67): cover the bug-preview geometry path, preserve a custom logo preset across toggling
- Add ChannelEditScreen tests exercising the geometry-fetch/preview path (previously
  untested because the fixture's blank logo path kept BugPreview from ever rendering):
  asserts the fetched watermark geometry lands on the preview image with asymmetric
  values, and that an external logo URL suppresses the preview.
- Fix a toggle-off/toggle-on data loss: re-enabling "use logo as on-screen bug" used to
  always rebind to the default 'Channel Bug' preset, discarding a channel's own
  per-geometry ChannelLogo preset. Remember the last referenced logo-driven watermark id
  in a ref and prefer it, falling back to the shared default.
- Strip the geometry cache's `id` key before it reaches BugPreview's props (was leaking
  via a spread).
- Drop the "(see #502)" issue reference from the visible help text; keep it in the code
  comment.
- Constrain findLogoBugWatermark's imageSource param to the generated
  ChannelWatermarkImageSource union instead of `string`, and add unit tests for its
  preference/fallback/none-found behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:02:16 +02:00
timothy d5da504651 feat(67): add Use logo as on-screen bug toggle with preview to the channel editor 2026-07-20 20:51:58 +02:00
timothy e847c9eef3 merge(67): watermarks screen uses BugPreview (T4) 2026-07-20 20:50:00 +02:00
timothy defe76b551 merge(67): documentation (T7) 2026-07-20 20:48:59 +02:00
timothy bb7f57d04d docs(67): record the shared-preset decision, watermark seeding, and the additive DTO field 2026-07-20 20:48:14 +02:00
timothyandClaude Opus 4.8 5d103ca6ff feat(67): show resolved bug geometry on the watermarks screen
Replace the raw <img> preview in the Watermarks editor's Image row with
the shared BugPreview component, so the on-screen bug's location/size/
margins/opacity render the same way as the other three preview spots.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 20:47:25 +02:00
timothy b0c06c6e7c merge(67): seed the shared Channel Bug watermark preset (T1) 2026-07-20 20:46:56 +02:00
timothy 520b113d66 merge(67): additive imageSource on the watermark picker DTO (T2) 2026-07-20 20:44:42 +02:00
timothy afd68b18fc merge(67): BugPreview component (T3) 2026-07-20 20:44:42 +02:00
timothy 4aeaecb1b2 feat(67): seed shared Channel Bug watermark preset, once per database 2026-07-20 20:43:27 +02:00
timothy ebaadc0656 feat(67): add imageSource to the watermark picker DTO (additive)
WatermarkResponseModel gains ImageSource so a client can identify
logo-driven presets generically instead of matching a user-editable name.
Additive under the frozen-additive /api/v1 contract (#286).

Adding a positional record parameter is source-breaking for existing
constructor call sites, so the two test files that built the DTO
positionally are updated. WatermarkHandlerTests now seeds its two rows with
DIFFERENT image sources so the round-trip assertion proves the field is
actually carried through the mapper rather than matching a constant on both.

Regenerated v1.json, endpoint-index.md and v1.d.ts; check:api clean.
Stripped the inherited UTF-8 BOM from Mapper.cs (#311 fix-as-you-touch).

Refs #67
2026-07-20 20:39:12 +02:00
timothy 5884bb2768 test(67): cover the remaining 4 bug-preview locations; note the MiddleCenter divergence (#503) 2026-07-20 20:23:38 +02:00
timothy 9ed933956b feat(67): add shared BugPreview component with pure geometry helper
Extracts on-screen bug (watermark) placement math into a pure,
DOM-free bugPreviewStyle(geometry) function plus a BugPreview
component that renders it inside a 16:9 frame. Consumed by the
channel and playout watermark editor screens (tasks 4/5) so users
can preview bug placement before saving.
2026-07-20 20:12:49 +02:00
timothy 71bb2c23f3 docs(67): revise plan+spec per independent review — fix toggle blocker, seed marker, real preview geometry
Fable review (cold, review-only) found a blocker in the plan's own code:
Task 5/6 selected the logo-bug preset by searching for the first
imageSource==='ChannelLogo' entry, but getWatermarks() sorts by name
(pickers.ts:14), so with a second logo-driven preset -- which this design
explicitly invites users to create -- the toggle would read OFF for a
logo-driven channel and REPOINT it on tick, from a control documented as a
pure reflection of stored state. Now resolved by id lookup, with the
tick-on target chosen by a shared findLogoBugWatermark() helper and pinned
by a two-preset regression test.

Also adopted:
- ConfigElement seed marker (watermark.channel_bug_seeded): ChannelWatermark
  has no IsSystem flag and Initialize runs every startup, so a name-only
  guard resurrected a deliberately deleted preset forever.
- Channel-editor preview now fetches the referenced preset's REAL geometry
  via the existing GET /api/v1/watermarks/{id} instead of hardcoding the
  seeded defaults -- which would have been wrong for exactly the users who
  tuned theirs.
- The 're-save untouched leaves watermarkId unchanged' test the spec
  promised and the plan had omitted, plus a create-path degrade test.
- Reversed the ChannelBuilder exclusion (operator decision): fresh installs
  stamp the preset onto the templates the seed creates; existing installs
  are untouched.
- External-URL logos never render a bug (File.Exists against a URL,
  WatermarkSelector.cs:269-286) -- verified, filed as #502, preview no
  longer promises it.
- Dropped Task 2's InternalsVisibleTo branch: already present
  (ErsatzTV.Application.csproj:30-32).

Refs #67 #502
2026-07-20 20:04:56 +02:00
timothy 9ff5580d49 docs(67): implementation plan + spec correction (ChannelBuilder inherits from template, excluded)
Self-review against the spec caught that the spec named both SPA creation
paths, but ChannelBuilder does not hardcode a watermark -- it inherits
watermarkId from the selected ChannelTemplate. Defaulting it would mean
mutating the seeded Standard/Music videos template rows, which the seed's
name-guard deliberately skips on existing installs, so it would no-op on
prod while changing fresh installs. Excluded in both documents with the
reason recorded.

Refs #67
2026-07-20 19:48:55 +02:00
timothy afbd50e3f0 docs(67): design spec — unified logo/bug via shared ChannelLogo watermark preset
Recon showed ChannelWatermarkImageSource.ChannelLogo already resolves the
channel's logo artwork at render time, and prod already runs that pattern
by hand (43 channels -> one 'Channel Bug' preset). So #67 is productizing
a proven mechanism, not building one: seed the preset idempotently in
DbInitializer (adopting any existing same-name row untouched), default the
SPA creation paths to it, and add a shared <BugPreview> that renders the
resolved geometry in both the channel editor and WatermarksScreen.

No schema change, no WatermarkSelector change. One additive DTO field
(WatermarkResponseModel.imageSource) under the frozen-additive /api/v1.

Refs #67
2026-07-20 19:39:37 +02:00
timothy d7d53f7ffe Merge pull request 'docs(release): prepare v26.11.0 promotion — correct the deploy host to jazz' (#501) from release/v26.11.0 into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 13s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 26s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 6s
2026-07-20 17:19:30 +00:00
timothy d6aec6de76 docs(release): prepare v26.11.0 promotion — correct the deploy host to jazz
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 6s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 11s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 12s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 33s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 33s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 32s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 34s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 7s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The media transcoders (Jellyfin, ersatztv, ersatztv-test) moved from bumblebee
to jazz (192.168.1.29) on 2026-07-20, server-management#633. Our docs still sent
the release operator to bumblebee for security-scan.sh and named it as the docker
host, which would have scanned the wrong box.

Also fixes the inverse error: `jazz` was an EARLIER name for the .99 host, so
three pre-migration references meaning today's bumblebee read as jazz. Those are
now bumblebee, and a Hosts table + name-reuse warning is added at the top of
ci-cd.md so the next session resolves hostnames by IP rather than re-breaking it
in either direction.

Version table had drifted (stopped at v26.9.0) — adds v26.10.0 + v26.11.0.
2026-07-20 19:18:03 +02:00
timothy c4ad4a73ba Merge pull request 'fix(497): reconcile music-video metadata collections on Jellyfin rescan' (#499) from fix/497-musicvideo-metadata-update into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 7m15s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 14m27s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 17m30s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m50s
2026-07-20 16:53:41 +00:00
timothyandtimothy 56dff2c49f docs(489): spec + implementation plan for Jellyfin mixed-content libraries (#490)
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 5m16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Has started running
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Has been cancelled
Design spec and bite-sized TDD implementation plan for #489, whose
implementation landed in #493.

Docs-only. Kept as the record of how the design was reached: that Jellyfin
classifies mixed-library items server-side via includeItemTypes (so no
inference is needed), that MediaItem is TPT keyed on LibraryPathId (so no
migration is needed), that MediaKind is dispatch + presentation only, and why
the feature is deliberately scoped to Jellyfin rather than local libraries.

Also records the open risk the plan carried -- the music-video scanner's
untraced reconciliation -- which #494 subsequently answered.

Refs #489

Co-authored-by: Timothy <timothy.look@gmail.com>
Co-committed-by: Timothy <timothy.look@gmail.com>
2026-07-20 16:36:02 +00:00
timothyandtimothy 2cf90fb44f feat(489): support Jellyfin mixed-content libraries (#493)
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Has started running
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Has started running
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Has been cancelled
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been cancelled
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been cancelled
Jellyfin libraries typed `mixed` were dropped by JellyfinApiClient.Project's
`_ => None` with no log line, so music and standup content could not be
ingested without a local-library workaround that bypassed Jellyfin entirely.

Adds LibraryMediaKind.Mixed, maps "mixed"/absent/blank CollectionType onto it,
and gives SynchronizeJellyfinLibraryByIdHandler a Mixed arm composing the three
existing per-kind scanners. Jellyfin classifies items server-side via
includeItemTypes, so the passes see disjoint sets; reconciliation is type-scoped
and cannot cross-delete. No new scanner and no DB migration -- MediaItem is TPT
keyed on LibraryPathId, so heterogeneous contents were already legal.

Segregation falls out of the model: a library is a place (one path <-> one
Jellyfin library <-> one ErsatzTV library), so music/standup cannot leak into
Movies or TV Shows.

Also removes the silent-success `_ => Unit.Default` from both scanner
dispatchers, which returned Right for an unhandled kind and stamped LastScan as
though a scan had run, and rejects Mixed for local libraries at the API.

Deliberately Jellyfin-only: local scanners share one video extension list and
would claim each other's files, and LibraryFolder etags are keyed by
LibraryPathId with no notion of kind.

Verified by live E2E against a real Jellyfin, including the interaction with
#494's reconciliation sweep. Four cold review rounds, all MERGEABLE.

fixes #489

Co-authored-by: Timothy <timothy.look@gmail.com>
Co-committed-by: Timothy <timothy.look@gmail.com>
2026-07-20 16:34:51 +00:00
timothyandClaude Opus 4.8 c633b89ad9 fix(497): reconcile music-video metadata collections on Jellyfin rescan
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m12s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 51s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m25s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 5m56s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m37s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m3s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
JellyfinMusicVideoLibraryScanner.UpdateMetadata copied only scalar fields
for an EXISTING music video, so genres/tags/studios/artists edited in
Jellyfin never reached ErsatzTV — the update path silently dropped every
child collection (only the Add path ever persisted them). Root cause is
inherited from upstream: unlike the movie/episode Jellyfin path, which
reconciles collections inside the tracked repository GetOrAdd,
MusicVideoRepository.GetOrAdd is AsNoTracking and the scanner never
reconciled the collections itself.

Fix mirrors PlexMovieLibraryScanner.UpdateMetadata's remove-stale +
add-new idiom, reconciling exactly the collections that BOTH the Add path
persists AND GetOrAdd eager-loads: Genres, Tags, Studios, Artists. Guids
(add-persisted but not eager-loaded — would duplicate) and Directors
(eager-loaded but not add-persisted for music videos) are deliberately
out of scope.

Movie/Episode paths do NOT have this gap (they reconcile in the tracked
repo GetOrAdd), so no separate fix is needed there.

Test is an interaction test (substituted repos, canned existing item)
verifying the exact reconcile calls; proven non-vacuous. The real-DB
double-scan approach can't drive this: the in-memory harness shares one
SQLite connection across contexts and mid-scan GetOrAdd's First()-nav
path predicate mis-resolves once the existing item carries metadata
children — a harness-only quirk (prod uses per-context pooled
connections).

fixes #497

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 18:32:29 +02:00
timothyandtimothy 5b0d8b5d09 fix(494): reconcile removed music videos in Jellyfin scanner (#495)
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 7m32s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 14m8s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 17m25s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m10s
fix(494): reconcile removed music videos in Jellyfin scanner (#495)

JellyfinMusicVideoLibraryScanner did add/update only, so a music video or
artist deleted on the Jellyfin side lingered forever. Add a library-scoped
trash sweep (TrashMissingMusicVideos) gated by the #477 empty-fetch guard.

fixes #494

Co-authored-by: Timothy <timothy.look@gmail.com>
Co-committed-by: Timothy <timothy.look@gmail.com>
2026-07-20 15:22:15 +00:00
timothy a696f06d57 Merge pull request 'fix(488): resolve LibraryFolder from DB so Jellyfin music-video scans stop crashing' (#492) from fix/488-getoraddfolder-null-libraryfolders into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 39s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 42s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 42s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m28s
2026-07-20 14:09:00 +00:00
timothyandClaude Opus 4.8 d3db2f6af1 fix(488): resolve LibraryFolder from DB so Jellyfin music-video scans stop NRE'ing
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 8s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 39s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m21s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m18s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m47s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m21s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
GetOrAddFolder read libraryPath.LibraryFolders, a navigation collection only
eager-loaded on the local scan path (via GetLibrary). The Jellyfin remote sync
path takes its LibraryPath off the JellyfinLibrary entity, where LibraryFolders
is null, so .Filter(null) threw ArgumentNullException('source') on the first
item of every Jellyfin music-video scan — a feature that had therefore never
run in prod, CI, or locally.

Look the folder up from the DB by (LibraryPathId, Path) instead, removing the
implicit eager-load contract entirely (correct for all nine callers) and
documenting it on ILibraryRepository. null != empty is preserved so a re-scan
does not insert duplicate LibraryFolder rows. No new hot-path cost: local
scanners already query GetParentFolderId per folder just before this call.

Tests:
- LibraryRepositoryTests: GetOrAddFolder with a null LibraryFolders (the exact
  remote-path shape) creates the folder, is idempotent on re-scan, and persists
  a supplied ParentId.
- JellyfinMusicVideoLibraryScannerTests: an end-to-end scan of one synthetic
  music video, wiring the REAL LibraryRepository/ArtistRepository/
  MusicVideoRepository against in-memory SQLite (the existing MediaServer*
  scanner tests mock every repo, which is why the bug escaped), asserts the scan
  completes and creates Artist + MusicVideo rows with a real LibraryFolder.

Both proven non-vacuous against the reverted fix (each reproduces the issue's
ArgumentNullException). decisions.md entry added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 01:16:16 +02:00
timothy 3c5f4e867a Merge pull request 'fix(480): probe external-JSON remote-stream URLs before ffmpeg' (#486) from fix/480-external-json-probe into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 13s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 25s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 26s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m49s
Renovate / Renovate (push) Failing after 14m57s
Build CI Toolchain Image / Build & push CI image (push) Successful in 1m20s
Dependency vulnerability scan / NuGet vulnerable packages (push) Successful in 2m15s
2026-07-19 22:37:29 +00:00
timothyandClaude Opus 4.8 4a9502cbb5 fix(480): probe external-JSON remote-stream URLs before handing them to ffmpeg
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 10s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m26s
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 6s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 6s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 5s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m54s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m23s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 18m47s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
External-JSON playout channels build their own /media/plex/{server}/{plexFile}
URL in ExternalJsonPlayoutItemProvider.StreamRemotely and the handler assigns it
without routing through ValidatePlayoutItemPath, so the #473 class survived here:
a media item gone from the server 404s under ffmpeg (exit 8) and the same dead
item is re-selected for its whole slot.

Route StreamRemotely through the same IRemoteStreamProber seam #473/PR #479 added
for the generated-playout path. Probe runs before the Plex metadata round-trip
(the URL needs only server.Id + plexFile), so a gone item skips it. An unavailable
stream returns PlayoutItemNotAvailableFromMediaServer, which the handler already
maps to a real-error card. The fail-open policy (redirected-404 only) lives inside
IRemoteStreamProber, so this second call site duplicates only the decision to probe.

Tests: ExternalJsonPlayoutItemProviderTests pins both directions; proven non-vacuous
by neutralizing the probe. docs/decisions.md gets a #480 entry closing the #473
scope gap (append-only: the old #473 entry is cross-referenced, not edited).

fixes #480

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 00:17:15 +02:00
timothy ab25658643 Merge pull request 'feat(472): sub-split the HLS cold-start startup phase' (#482) from feat/472-startup-subsplit into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 7m37s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 15m9s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 17m47s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m15s
2026-07-19 21:59:22 +00:00
timothy 55be64bd77 Merge pull request 'fix(477): guard media-server library sweeps against successful-but-empty fetches' (#483) from fix/477-scanner-antinuke-guard into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 24s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 32s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 35s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m24s
2026-07-19 21:53:37 +00:00
timothy 37674d6519 test(472): name the stale-playlist test for what it actually pins
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 9s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 46s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 10s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 18s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 14m22s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m25s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Third-round review caught that
Stale_Playlist_Guard_Should_Take_Precedence_Over_An_Otherwise_Valid_Three_Way
describes an impossible case: ThreeWay requires
processLaunched <= playlistExists, which is exactly the negation of the
guard condition, so the guard can never preempt a ThreeWay. What the
test really pins is precedence over the progress branches
(TwoWayLateProgress) — still the ordering that matters.

That is the same "rationale misstates the mechanism" defect the previous
commit existed to fix, landed inside the fix itself. Renaming rather
than leaving a test whose name teaches the next reader something false.

Also broadens the escape-hatch caveat: a stale playlist that slips past
the guard lands as TwoWay more often than ThreeWay, since FFmpeg has
usually not reported progress that early.

Test name and comments only; no logic change.
2026-07-19 23:33:28 +02:00
timothyandClaude Opus 4.8 aa32fd78dd fix(477): guard media-server library sweeps against successful-but-empty fetches
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 9s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 7s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m23s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m18s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
A successful fetch returning zero items made existing.Except([]) flag the ENTIRE
library FileNotFound in one scan — feeding EmptyTrashHandler's permanent delete
and emptying every affected collection (dead channels). Add a shared
MediaServerReconciliationGuard that skips (and logs a Warning) the sweep when
incoming==0 while items exist, wired into the three library-level sweeps
(Television shows / Movie / OtherVideo).

An empty incoming set is indistinguishable at scan time from a mid-restore /
emptied-upstream error (both report a zero total), so this deliberately overrides
#476's degenerate "last item removed => empty incoming => flag" case. #476's
cascade still fires for partial deletions (survivors present); its characterization
test moves from an empty incoming to a survivor+removed partial-deletion case.

Tests: policy table (MediaServerReconciliationGuardTests) + per-scanner integration
proving the wiring (empty incoming + non-empty existing flags/reindexes nothing).
Proven non-vacuous by neutralizing the guard. Nested TV season/episode sweeps left
unguarded (bounded blast radius); ratio-threshold + projection-failure detection
deferred to a follow-up. docs/decisions.md updated.

Fixes #477

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 23:28:27 +02:00
timothy 58b93d3e3d docs(472): correct the stale-playlist rationale; pin guard precedence
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 6s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 9s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 35s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 20s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 20s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 5m42s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m8s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m25s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Re-review of the fix commit returned MERGEABLE with one Medium: the
comment justifying the stale-playlist guard misstated the mechanism, in
three places. It claimed Run "warns about a non-empty transcode folder
but does not delete it" — but StartFFmpegSessionHandler.FolderMustBeEmpty
calls EmptyFolder BEFORE the worker spawns, and Run's finally empties it
again. Verified directly rather than taken on the reviewer's word.

The real residual path is EmptyFolder FAILING: it swallows every
exception into a LogWarning and continues. Say that instead.

On a PR whose entire value is that the numbers mean what they say, a
rationale comment that misstates the mechanism is the same class of
defect the PR exists to prevent, so it does not get to ship as a nit.

Also documents that the guard is best-effort rather than a proof (if the
wipe failed, before-or-after-launch is a scheduling race, so an unlucky
sample can still slip through as an implausibly fast ThreeWay), and adds
the test the reviewer noted was missing: guard precedence over an
otherwise-valid ThreeWay.

Comments and one test only; no logic change.
2026-07-19 23:21:03 +02:00
timothy 757fb76151 fix(472): review fixes — honest bucket boundaries, stale-playlist guard
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 13s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 10s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 26s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m25s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been cancelled
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Has been cancelled
Build ErsatzTV Image / Build & test (.NET) (pull_request) Has been cancelled
From the cold adversarial review of the initial diff. No blockers were
found; these address what the numbers MEAN, which is the whole point of
an instrumentation change.

- The buckets span the worker's Run entry, not the startup stopwatch, so
  prep overlaps the tail of `setup`. Rather than let the log imply an
  invariant it does not satisfy, say "spans runEntry" in the line, spell
  it out in the doc comment, and rename the test that had codified the
  false `sum == startup` claim.
- Guard `processLaunched > playlistExists` -> Unavailable: a stale
  live.m3u8 from a previous session (Run warns about a non-empty
  transcode folder but does not delete it) would otherwise yield a
  plausible-looking sample whose prep exceeds the measured phase.
- Split the two-way fallback into TwoWay vs TwoWayLateProgress. They are
  different stories about the pipeline and discriminating stories is
  what this issue is for.
- Document the 100ms playlist-poll quantization (it lands entirely in
  firstGop, the smallest bucket) and the first-process-failed case where
  ffmpegInit spans a retry.
- Short-circuit the per-line timestamp call; static readonly Unavailable.
- Tests for the new guard, progress-before-launch, and boundary equality
  (so tightening >= to > later cannot pass silently).
2026-07-19 23:05:14 +02:00
timothy d91d6ee1ed feat(472): sub-split the HLS cold-start startup phase
#350's measurement showed `startup` is 81% of tune-in latency and carries
100% of its variance, while remaining one opaque bucket spanning FFmpeg
spawn -> input open/probe -> encoder init -> first GOP. Two hypotheses
survive that measurement (NFS input open vs VAAPI init under contention)
and they need opposite fixes, so split before optimizing.

Adds `prep` (ErsatzTV-side work before FFmpeg exists) + `ffmpegInit`
(launch -> first `-progress` output) + `firstGop` (-> live.m3u8 exists)
to the existing Information-level cold-start line.

The pipeline runs `-loglevel error -nostats -hide_banner`, so a healthy
FFmpeg writes nothing to stderr; the `-progress` stream is the only
zero-cost milestone available and `ffmpegInit` therefore still lumps
input-open with encoder-init. That limit is documented rather than
papered over, and the split degrades to the two-way form #472 accepts
when no progress arrives before the playlist.

Log-only: no transcode behavior change, no new endpoint or config knob.

fixes #472
2026-07-19 22:49:07 +02:00
timothy de11eb01b3 Merge pull request 'fix(473): probe media-server remote streams before handing the URL to ffmpeg' (#479) from fix/473-missing-media-resilience into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 14m51s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 18m21s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 19m4s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m6s
2026-07-19 20:24:59 +00:00
timothy 0d70425572 Merge pull request 'fix(476): cascade FileNotFound from removed shows/seasons to descendants' (#481) from feat/476-cascade-filenotfound into main
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 14s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 33s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 34s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m9s
2026-07-19 20:19:37 +00:00
timothyandClaude Opus 4.8 1eff0b39b7 fix(473): gate the cancellation filter on the caller's token
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 6s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 5s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 16s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 18s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m17s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m10s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m28s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Third review pass: MERGEABLE WITH NITS. Taking the one finding it asked
for before merge, plus a doc nit.

The cancellation filter added last commit had no token check, and it spans
the whole Transcode body -- including every mediator send (ffprobe via
CliWrap, media-server API calls, subtitle extraction, song-video
generation). TaskCanceledException is also what HttpClient throws on its
OWN timeout, so a real timeout in any of those was being downgraded from
an ERROR with a stack trace to a routine "Terminating HLS session"
Information line. Behaviour was unchanged (both arms return false) but the
fault signal was lost, and this repo has been bitten before by
"empty log != the event didn't happen". Now filters on
cancellationToken.IsCancellationRequested, so only genuine caller
cancellation is treated as a graceful teardown.

Doc nit: the <exception> block said cancellation "is thrown"; it is only
thrown when the token trips while the probe is in flight -- cancelling
after it completes returns normally. Now says "may propagate".

Declined the reviewer's optional suggestion to drain until a 0-return
instead of reading exactly one byte: reading exactly one byte is what
makes the guard safe BY CONSTRUCTION, since a server or proxy that
answers 206 with a wider range than requested still cannot be drained
unboundedly. 206-only was confirmed correct rather than extended to short
200s, since deciding "short" from Content-Length would reopen the
unbounded path for a chunked or Content-Length-less response.

Also records the operator's standing rule in the handoff lore: a lone
`decisions.md append-only` red is a known infra flake -- do not
investigate, rebase, amend or push to clear it; the operator reruns that
job from the UI. I violated this earlier in this PR with a
tidy-but-wrong "my entry is no longer at EOF" theory, and the rebase did
not fix it -- the job went red again on a verified pure-append diff, which
is the proof the red was never about the diff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 22:02:15 +02:00
timothyandClaude Opus 4.8 61bd374494 fix(476): cascade FileNotFound from removed shows/seasons to descendants
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 7s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 24s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 5m52s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 5m36s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 9s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m14s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The media-server television scanner reconciles removed items at three levels
via existing.Except(incoming), but the season and episode sweeps live inside
the per-parent loops (ScanSeasons inside the show loop, ScanEpisodes inside the
season loop). Those loops only iterate parents present in the incoming list, so
a show (or season) that is gone from the media server is never visited and its
descendants are never swept — they keep their last state. On Jellyfin/Emby that
is RemoteOnly, which PlayoutBuilder does NOT skip even with PlayoutSkipMissingItems
on, so every orphaned episode keeps getting scheduled as a guaranteed tune-in
failure (the #473 reproduction; 717 stale prod rows across 10 removed shows).

Fix: cascade the flag by parent MediaItem.Id. Two provider-agnostic repo helpers
(Season.ShowId / Episode.SeasonId are on the base tables) flag descendants and the
scanner drives them after each parent sweep — show → seasons → episodes, and
season → episodes for the show-present case. Shared abstract base fixes
Jellyfin/Plex/Emby at once; Movie/OtherVideo are flat and have no such gap.

Tests: a Scanner.Tests case asserts the scanner cascades through the (substituted)
repository (non-vacuous — fails if the cascade calls are removed), and Integration
tests exercise the real cascade SQL against the schema, proving it flags only the
targeted subtree and no-ops on empty input.

fixes #476

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 21:49:33 +02:00
timothyandClaude Opus 4.8 dc5ceb5a14 fix(473): bound the drain, correct the interface contract, quiet graceful cancels
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 12s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
Build ErsatzTV Image / decisions.md append-only (pull_request) Failing after 12m6s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m47s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 18m54s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m46s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Second review pass returned BLOCKED on two findings introduced by the
first fix commit. Both were right.

BLOCKER 1 — the drain added for "return the connection to the pool" was
unbounded. `response.Content.ReadAsByteArrayAsync()` buffers the WHOLE
body, and it ran for every non-404 response. A server that ignores
`Range: bytes=0-0` answers 200 with the entire file, so this would
download at line rate into a byte[] on the streaming hot path for up to
the 2s timeout -- strictly worse than the aborted socket it replaced, and
it defeated the ResponseHeadersRead the probe deliberately uses. Now the
single byte is read only on 206 (where the server honoured the range and
the body really is one byte); any other status aborts the socket, which
is much the cheaper evil. Two tests pin both directions; verified
non-vacuous (restoring the unbounded drain fails the 200-with-body test).

BLOCKER 2 — IRemoteStreamProber's doc-comment still described pre-fix
behaviour. I had told the reviewer it was updated; it was not -- only the
implementation's <remarks> had been. It claimed `false` on any 404 (now
only a redirected one) and that every other outcome returns `true` (caller
cancellation throws). Both clauses corrected, and the throwing contract is
now documented with <exception>.

Also fixed the reviewer's own follow-on finding: the cancellation rethrow
it asked for reached HlsSessionWorker's catch-all, which logs a
channel-level ERROR with a stack trace. The graceful
TaskCanceledException/OperationCanceledException handler at :662 wraps only
the inner ffmpeg block, not the mediator sends, so every client disconnect
on a remote-streaming channel would have produced a spurious ERROR -- in
exactly the logs a #350 cold-start investigation reads. Added a
cancellation filter on the outer try that logs Information instead.

Nit: stale SeedAll doc-comment now mentions the emby case.

Deferred, per reviewer's explicit agreement: Plex-branch handler coverage
(follow-up), and HEAD-with-GET-fallback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 21:47:50 +02:00
timothyandClaude Opus 4.8 a10c0325eb fix(473): compare parsed Uris in the redirect check
Defense in depth on the redirect detector: Uri.Equals compares normalized
components, so an escaping/casing difference can't be mistaken for a
redirect and fail CLOSED -- the exact failure the check exists to prevent.
A plex key can contain spaces or unicode.

Honest note: this is NOT a fix for an observed bug. I wrote a test claiming
to pin it, then ran the negative control and the test passed against the
string comparison too -- Uri.ToString() unescapes, so both forms agree for
our machine-generated URLs. The test was vacuous as written. It is kept,
retitled and re-commented to describe what it actually guards (an
un-redirected 404 on an escaping-sensitive url fails open), and the code
comment says plainly that this is defense in depth rather than a repair.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 21:42:21 +02:00
timothyandClaude Opus 4.8 bad19f8d26 fix(473): review fixes — only a redirected 404 fails closed
Adversarial review of PR #479 found the stated fail-open contract was not
what the code measured, plus four smaller gaps. All fixed here as a
follow-up commit (no amend/force-push).

High — a 404 from ErsatzTV's OWN endpoint was treated as "media gone".
/media/{provider}/... is served by InternalController, which returns
NotFound when the media source is unconfigured or momentarily missing
(a media-source edit that deletes+reinserts connections, a restore, a
partially-configured server). Probing for "any 404" therefore failed
CLOSED for every item on that source -- exactly the case the fail-open
contract exists to prevent. A media-server 404 always arrives after a
redirect, so an un-redirected 404 is now treated as available.

Medium — the new switch label was untested and its benefit overstated.
maybeDuration/finish are computed before the switch, so `default:`
already sized the error card to the next playout item; the label only
changes the caption. The handler test asserted call counts only, so
deleting the label still passed. It now asserts the error message, and
removing the label fails the test (verified).

Medium — Plex/Emby branches changed but had no coverage. Added an Emby
handler test asserting the probe is called with the emby URL.

Low — caller cancellation was swallowed and pinned as desired behaviour.
A shutdown / client disconnect is a genuine signal, not a probe failure;
it now propagates, and only the probe's own 2s timeout fails open.

Low — the response stream was disposed unread, aborting the connection
instead of returning it to the pool. The one requested byte is drained.

Nit — fully-qualified RangeHeaderValue replaced with a using.

docs/decisions.md corrected where it overstated: the switch label's role,
the "fixes the class for all three media servers" claim (external-JSON
channels bypass ValidatePlayoutItemPath entirely -- filed as #480), and
the unmeasured latency assertion. Deferred HEAD-instead-of-GET recorded
with its reason rather than silently dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 21:42:20 +02:00
timothyandClaude Opus 4.8 90dc864ce5 fix(473): probe media-server remote streams before handing the URL to ffmpeg
Tuning a channel intermittently hard-failed with ffmpeg exit 8 and
`Server returned 404 Not Found` on /media/jellyfin/{itemId}.

Root cause: ValidatePlayoutItemPath checked `File.Exists` on the local
branch, but the three media-server remote-stream branches returned
`http://localhost:{port}/media/{plex,jellyfin,emby}/{id}` unconditionally.
When the media was gone from the media server too, validation "succeeded"
and ffmpeg was launched against a URL that 404s.

That bypassed the good error path the handler already had
(PlayoutItemDoesNotExistOnDisk renders an error card sized to run until
the NEXT playout item, so the dead item is skipped) and instead landed in
HlsSessionWorker's generic ffmpeg-failure path, which sizes its error card
to the failed 44s work-ahead chunk and then re-selects the SAME broken
item -- a repeating error card for the item's whole slot (~22 min).

Restore the method's own invariant: every PlayoutItemWithPath it returns
has been checked for existence. A definitive 404 now returns the new
PlayoutItemNotAvailableFromMediaServer error, handled in the same switch
arm as PlayoutItemDoesNotExistOnDisk.

The probe is deliberately fail-open: only a 404 reports the media gone.
A timeout, 5xx, auth error or transport failure reports available, so a
probe that cannot answer can never break a tune that would have worked.
That contract is pinned by tests so a later refactor cannot invert it.

Rejected alternatives (see docs/decisions.md): resizing the
HlsSessionWorker retry loop (cannot distinguish a dead item from a
transient transcoder failure -- prod has live VAAPI hwupload -22 failures
that must keep retrying), and writing MediaItemState from the streaming
path (breaks scanner ownership, and would not have fixed this: the item
is RemoteOnly, which PlayoutBuilder's skip does not exclude).

Scanner-side follow-ups filed separately: #476 (FileNotFound does not
cascade show -> episodes, the reason dead items keep being scheduled),

fixes #473

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 21:41:55 +02:00
timothy 48d7598bae Merge pull request 'ci(412): sample true peak-anon in the test job, not cache-inflated memory.peak' (#475) from ci/412-peak-anon-sampler into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 27s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 22s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m56s
2026-07-19 19:09:29 +00:00
timothy c5369b1d69 ci(412): sample true peak-anon in the test job, not cache-inflated memory.peak
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 7s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m15s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m10s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 6s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m12s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 13m47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m12s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The test-job memory instrument (#411) reported memory.peak — the high-water mark of
memory.current, which charges reclaimable page cache to the cgroup. A build does heavy
NuGet/npm/obj/bin/coverage I/O, so cache can dominate the peak, and page cache is reclaimed
under a tighter cap rather than OOM-killed. Sizing a per-job cap (server-management#604) off
memory.peak therefore inverts the decision. The OOM-forcing quantity is peak anon, which the
kernel exposes no counter for and which the end-of-job split misses (a job that peaks
mid-dotnet-test then frees reports a low anon).

New scripts/ci-peak-anon.sh: a `start` step (before Build/Test/Coverage) launches a detached
background sampler tracking the high-water mark of cgroup anon; a `report` step (last) stops it
and prints the sampled peak anon as the headline, keeping memory.peak + end-of-job split as a
cache-inflated ceiling and reference. Both continue-on-error + fail-open so they never redden a
build. Validated on bumblebee: survives step-boundary re-execs, catches a transient 2.5 GiB
anon spike the snapshot reports as 0, stops cleanly on kill, degrades gracefully.

Compiler-server A/B (swap-off, sampled peak-anon, n=2 interleaved): OFF (CI config) ~5.84 GiB
consistent; ON (defaults) 6.3-7.6 GiB, always higher, + a ~3 GiB resident VBCSCompiler.
Disabling the servers is worth it, but OFF sits right at 6 GiB for the build phase alone and the
test job adds test+coverage, so #406's "budget loosens well under 6 GiB" premise is not
supported. Size the cap off the live test-job sampler.

Docs: ci-cd.md instrument section rewritten (peak-anon headline + A/B table + premise verdict);
decisions.md entry added. No .cs touched.

fixes #412
2026-07-19 20:49:08 +02:00
timothy 48d632d50a Merge pull request 'perf(469): format gate uses dotnet format whitespace --folder (~480s → ~0.5s)' (#471) from ci/469-format-speed into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 33s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 34s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 35s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m4s
2026-07-19 17:59:33 +00:00
timothyandClaude Opus 4.8 33657b4753 docs(469): sweep remaining stale format-job cost/memory claims (review)
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 8s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 9s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 23s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 20s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 5m49s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 18m28s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m33s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 5s
Cold adversarial review (MERGEABLE) flagged two spots still describing the
pre-#469 heavy `format` job as current, plus a wording imprecision:
- ci-cd.md CI-lane table row: annotate `format` runtime 37s → ~0.5s (#469).
- ci-cd.md #406 memory narrative: note the 3.95 GiB Roslyn heap is now moot
  (folder mode loads no workspace); api-docs remains the lane's real consumer.
- Precise the coverage mechanism in ci-cd.md + decisions.md: the naming rule
  passes the full gate because naming violations have no `dotnet format` batch
  code-fixer (so `--verify-no-changes` sees no change), not merely a severity
  floor.

Docs-only; no workflow/hook logic change from the reviewed commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 19:34:43 +02:00
timothyandClaude Opus 4.8 eafb2e39e2 perf(469): format gate uses dotnet format whitespace --folder (~480s → ~0.5s)
Build ErsatzTV Image / decisions.md append-only (pull_request) Waiting to run
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 6s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m15s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m28s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Has been cancelled
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been cancelled
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Has been cancelled
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Has been cancelled
The blocking `format` CI job and the Husky pre-commit hook verified changed .cs
files with `dotnet format ErsatzTV.sln --no-restore --verify-no-changes
--include <files>`. `--include` only narrows *which* files are checked, never
what gets loaded: the full recipe loaded the ~10-project MSBuild workspace and
built a Roslyn compilation per project before checking a single line (~480s
locally, whole-solution). Switch both to `dotnet format whitespace . --folder
--verify-no-changes --include <files>`, which treats the tree as a plain folder
of files, skips MSBuild/Roslyn entirely (~0.5s), and needs no `dotnet restore`
(NuGet-cache + Restore steps removed).

Coverage is unchanged: folder mode reads .editorconfig and enforces exactly the
gate's purpose — whitespace + charset (BOM). Proven non-vacuous (error
WHITESPACE on a trailing-space line, error CHARSET on a prepended BOM, exit 0
clean). The full gate never enforced the style/analyzer pass either — a
warning-severity naming violation passes the full solution format (exit 0) — and
the analyzers that must block (NU1904, S3981) are enforced at compile via
WarningsAsErrors, not by this job.

Docs: ci-cd.md Formatting section + the obsolete #406 memory note; decisions.md.

fixes #469

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 18:33:31 +02:00
timothy e8d730359a Merge pull request 'test(444): deterministic functional-E2E for the playout-build lock 409 + isLocked projection' (#470) from feat/444-buildlock-e2e into main
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 16s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 27s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m43s
2026-07-19 15:57:41 +00:00
timothyandClaude Opus 4.8 038703fe67 test(444): deterministic functional-E2E for the playout-build lock 409 + isLocked projection
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 10s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 14s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 9s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 33s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m24s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m4s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m25s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 18m37s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Adds "Flow C" to scripts/e2e-functional.sh, the last deferred lock-contention flow from #363.
A playout build is enqueued onto the single-consumer WorkerService channel and the trigger
returns before BuildPlayoutHandler acquires the lock, so an accepted trigger does not prove the
lock is held. Flow C makes it deterministic: seed a Classic Flood schedule over a few short
ffmpeg episodes, crank PlayoutDaysToBuild=5 (~43k items ~= ~1s build), then POLL GET
/playouts/{id} until isLocked:true before firing. Asserts PUT /playouts/{id} -> 409, reset ->
409, and the list-projection isLocked:true while locked; then isLocked:false + PUT -> 200 after
the build (proving the 409 is lock-specific). Each racing assertion is guarded so a build that
finishes mid-flight degrades to an advisory skip, never a false red; the whole flow self-skips
without ffmpeg or if the build is never observed locked.

Sized by measurement on a fresh instance -- going wider is counter-productive (a 777k-item build
saturates the single worker with post-build gap/overlap jobs). Verified green across 6
fresh-instance runs; cold adversarial review MERGEABLE.

Docs: docs/e2e-local.md + docs/ci-cd.md updated to describe Flow C and drop it from the
"deferred" lists.

fixes #444

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 17:37:50 +02:00
timothy 481a14675f Merge pull request 'feat(431): TTL-cache health-check results; ?refresh=true forces a fresh run' (#468) from feat/431-health-ttl-cache into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 34s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 33s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 33s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m37s
2026-07-19 13:43:11 +00:00
timothyandClaude Opus 4.8 ca54853b60 style(431): dotnet format the touched troubleshooting handler (#311 fix-as-you-touch)
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 29s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m22s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 13m0s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m35s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m46s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 37m33s
The Formatting CI gate flagged legacy stacked-foreach whitespace in
GetTroubleshootingInfoHandler.cs (lines ~121-137) — pulled into scope
because this PR touches the file (fix-as-you-touch, ersatztv#311). Pure
`dotnet format` reindent, zero logic change (git diff -w is empty).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 15:03:48 +02:00
timothyandClaude Opus 4.8 be25df670e feat(431): TTL-cache health-check results; ?refresh=true forces a fresh run
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 6s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 13m16s
Build ErsatzTV Image / decisions.md append-only (pull_request) Failing after 12m23s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Failing after 14m6s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m44s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m19s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m28s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
GET /api/v1/health re-ran all 14 health checks on every request, 4 of
which shell out to ffmpeg/ffprobe via CliWrap — so each poll spawned ~4
subprocesses. The existing HealthCheckSummary cache was write-only.

Cache the full result list for 30s inside HealthCheckService keyed on a
new "healthcheck.results" entry; a non-forced call returns it on a hit,
skipping the checks and the (subscriber-less) summary publish. Add a
`bool forceRefresh` first parameter to IHealthCheckService.PerformHealthChecks:
the API poll path reads the cache, while startup (RunHealthChecksService)
and the troubleshooting support bundle force a fresh run.

Refresh surface: GET /api/v1/health gains an optional `[FromQuery] bool
refresh` (additive, follows the ?deep= exemplar); the SPA "Refresh health"
button calls /api/v1/health?refresh=true, the initial/poll load does not.

Tests: HealthCheckService cache-hit vs force-bypass (mutually opposing,
non-vacuous), handler+controller refresh-flag threading, SPA refresh URL.
Docs: decisions.md 2026-07-19 (#431), api-conventions §2; regenerated v1.json.

fixes #431

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 14:41:12 +02:00
timothy 67dc9ca4c0 Merge pull request 'fix(464): render real channel logos in guide grid + channels list' (#467) from fix/464-guide-logos into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 7m28s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 13m10s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 17m20s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m42s
2026-07-19 12:10:28 +00:00
timothyandClaude Opus 4.8 1aa1f0cf5b fix(464): render real channel logos in guide grid + channels list
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 8s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 12s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m33s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 13m11s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m45s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m31s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 36m5s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 6s
The guide/EPG grid (/app/guide) and the channels list (/app/channels) always
drew the generated initials "bug" because the browse DTOs never carried a logo
URL — GuideScreen/ChannelsScreen rendered <ChannelLogo> with no src. The logo
data existed (it round-trips through the channel editor) but never reached these
views.

Add a rooted, directly-usable Logo URL to ChannelGuideChannelResponseModel and
ChannelResponseModel, populated by a single Channels.Mapper.GetLogoUrl helper
(#181 artwork convention): /iptv/logos/{file} for an uploaded logo, the absolute
URL passed through for an external one, null when unset so the SPA keeps its
generated-initials fallback. The guide query now includes Channel.Artwork.

Regenerated OpenAPI + v1.d.ts; updated api-conventions.md + domain-model.md.

fixes #464

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 13:28:00 +02:00
timothy a6c300730b Merge pull request 'docs(404): mirror the weight UI into the MultiCollections design prototype' (#466) from docs/404-designsync-mirror into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 17s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 27s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m7s
2026-07-19 11:25:26 +00:00
timothyandClaude Opus 4.8 586b075cc1 docs(404): mirror the weight UI into the MultiCollections design prototype
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 14s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m18s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 41s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m43s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m6s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m34s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Design-sync back-mirror of the #404 weight UI (PR #462) into the Claude Design
prototype so repo + Claude Design stay byte-identical (docs/design-sync.md step 7).
Pushed to the ChicoryTV Design System project via DesignSync in the same session.

The prototype's EditItemRow now shows a per-source Weight input (1..1000) + computed
% share, and the editor gains a "Reset to fair share" footer — a faithful mockup of
the shipped screen. Prototype-only (design-system/), no shipped code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 13:05:07 +02:00
timothy cc2968de15 Merge pull request 'fix(463): bound the schedules "Active schedule" selector so it stops overlapping the header' (#465) from fix/463-schedule-select-width into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 7m23s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 14m3s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 17m14s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m57s
2026-07-19 10:21:00 +00:00
timothy 86cbf79b48 Merge pull request 'feat(404): weighted-distribution SPA — per-source weight inputs + WeightedShuffle order' (#462) from feat/404-weight-ui into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 11s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 13s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 24s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m13s
2026-07-19 10:01:41 +00:00
timothyandClaude Opus 4.8 df617a64c7 fix(463): bound the schedules "Active schedule" selector so it stops overlapping the header
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 14s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 9s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 13s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m24s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m38s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m56s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The header selector rendered at the Select default `fullWidth=true`, which applies
`.ctv-field-full { width: 100% }`. In the flex `.ctv-schedule-header` that made it
demand the whole row, overlapping/distorting the title block and Add/Edit/Delete
buttons. Pass `fullWidth={false}` (sizes to content) + bound it to 150–260px so a
long schedule name can't re-widen it; the native select's value truncates within
the frame (`.ctv-select { min-width: 0 }`).

Pure-SPA/CSS, no API/DB change. Regression test asserts the selector wrapper is not
`.ctv-field-full`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 11:58:22 +02:00
timothyandClaude Opus 4.8 d2d678aae8 test/docs(404): add unrelated-edit round-trip test; correct MultiCollection-only rationale
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 7s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 12s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 23s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m20s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m48s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m10s
Review + live-E2E follow-ups (no behavior change):
- Add a test proving the canonical silent-reset trap directly: a rename-only save
  round-trips both weights untouched (the prior test only edited the weight it
  asserted). Cold review nit.
- Correct the rationale in itemRules.ts + decisions.md: WeightedShuffle is
  MultiCollection-only in the SPA for *meaningfulness* (per-source weights need
  2+ sources), NOT because the classic write path rejects it — live-E2E confirmed
  the classic engine ACCEPTS it on a plain Collection (200) and degrades to
  fair-share. The rejection is on the separate playlist/block write paths, whose
  editors keep their own order lists and already omit it.

Live-E2E (real API): weighted multi-collection create + read round-trips weights;
rename-only PUT preserves them (no silent reset); WeightedShuffle persists on a
classic MultiCollection schedule item. Ratio itself is pinned by the existing
PlayoutBuildGoldenTests.Classic_weighted (3:1), untouched by this SPA change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 11:39:29 +02:00
timothyandClaude Opus 4.8 d639946b5c feat(404): weighted-distribution SPA — per-source weight inputs + WeightedShuffle order
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 11s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 7s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 7s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m39s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 16m37s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 24m50s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The UI half of #70 (backend + API shipped in PR #402). Pure SPA + docs — no new
endpoint/DTO (`weight` was already on MultiCollectionItemRequest/Response, and
`WeightedShuffle` already in the PlaybackOrder enum).

- Multi-collection editor (`/app/multi-collections`): per-source weight input
  (1..1000, mirroring the API validator), a computed % share (3:1 shows 75/25),
  and a "Reset to fair share" action (fair-share = all weights 1, decisions.md
  2026-07-17 — a reset, not a separate mode). Weight round-trips through the
  draft (read in itemsFromMultiCollection, written in toItemRequest) so the
  replace-all PUT never silently resets it.
- Classic schedule editor: `WeightedShuffle` offered as a Playback Order ONLY
  for MultiCollection sources (itemRules `MULTI_COLLECTION_ORDERS`) — it needs
  per-source weights and the write path rejects it elsewhere. Excluded from
  fillWithGroup like ShuffleInOrder (PlayoutBuilder schedules fill-groups
  per-group, incompatible with whole-collection weighted share).
- `Input` gained min/max/inputMode/onBlur passthroughs for bounded numeric
  fields (reusable by #425's weight UI); weight held as a string for smooth
  editing, clamped on blur and at save so an out-of-range value never 400s.
- Docs: domain-model + spa-conventions (replace-all round-trip trap, bounded
  numeric input pattern) + decisions.md entry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 11:28:12 +02:00
timothy 996c545161 Merge pull request 'feat(queue): deterministic select-queue.sh — stop re-deriving the selector by hand' (#461) from feat/select-queue-script into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 15s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 16s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 8s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m25s
2026-07-19 08:46:18 +00:00
timothyandClaude Opus 4.8 708e602197 feat(queue): deterministic scripts/select-queue.sh — stop re-deriving the selector by hand
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 9s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 10s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m23s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m52s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 18m23s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m48s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The kickoff dispatches a cheap model to rank the backlog, and the lore then made the
orchestrator re-derive its mechanical claims (deps, milestone tiering, priority order,
in-progress state) because a small model kept getting them wrong. This pays that tax down:
the mechanical checks are now CODE — GET /dependencies exclusion, LOCAL
.milestone.state/review/priority tiering (never the no-op ?milestones= filter),
(tier,priority,issue#) ordering, in-progress/parked/PR exclusion — leaving only two
JUDGMENT flags (CLAIM?, UMBRELLA?) for a human/model to resolve.

- scripts/select-queue.sh: fail-open on no-creds/unreachable; ~1.3s; shellcheck clean;
  UMBRELLA? computed in the jq pass (no per-candidate body fetch).
- handoff kickoff: run the script FIRST; trust its deps/tiering/ordering, recheck only flags.
- handoff lore: the three 'cheap selector unreliable → re-derive by hand' bullets kept as the
  EVIDENCE for why the script exists; the prescription is redirected to 'run the script'.

Operator-requested this session: 'rather than have the lore make us redo the selector's work,
improve the selector.'

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 01:09:57 +02:00
timothyandtimothy df8c5202d6 fix(409): report never-scanned LastScan as null for API/MCP parity (migration + read coercion) (#459)
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build CI Toolchain Image / Build & push CI image (push) Failing after 10m41s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 18m20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 18m31s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 3m58s
Co-authored-by: Timothy <timothy.look@gmail.com>
Co-committed-by: Timothy <timothy.look@gmail.com>
2026-07-18 19:39:25 +00:00
151 changed files with 39402 additions and 439 deletions
+47 -84
View File
@@ -183,6 +183,16 @@ jobs:
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: sed -i '/Scanner/d' ErsatzTV/ErsatzTV.csproj
# Start the true peak-anon sampler just before the memory-heavy dotnet Build/Test/Coverage so
# its high-water mark spans them (SPA build/test above are comparatively light). Paired with the
# "Report peak container memory" step below. continue-on-error + a fail-open script => this
# instrumentation never reddens a build. Why anon and not memory.peak: ersatztv#412 /
# scripts/ci-peak-anon.sh header / docs/ci-cd.md "CI build memory".
- name: Start peak-anon sampler (ersatztv#412)
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
continue-on-error: true
run: scripts/ci-peak-anon.sh start
- name: Build
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: dotnet build --configuration Release --no-restore
@@ -223,71 +233,26 @@ jobs:
cat coverage/report/SummaryGithub.md >> "$GITHUB_STEP_SUMMARY"
fi
# Memory of THIS job container, reported every run (ersatztv#406, server-management#604).
# #604 sizes the runners' per-job caps on these numbers, and until now they were inherited
# rather than measured: the 10g cap traces back to server-management#570 observing the image
# build peg 5.999/6 GiB, which is a different job entirely.
# Memory of THIS job container, reported every run (ersatztv#406/#412, server-management#604).
# #604 sizes the runners' per-job caps on these numbers. The headline is the TRUE PEAK ANON
# sampled by the "Start peak-anon sampler" step above — NOT `memory.peak`, which is the
# high-water mark of memory.current and charges reclaimable page cache to the cgroup (a build
# job does heavy NuGet/npm/obj/bin/coverage I/O, so cache can dominate the peak). Page cache is
# reclaimed under a tighter cap, not OOM-killed, so sizing a cap off `memory.peak` inverts the
# decision. peak anon is the OOM-forcing number. Full rationale + the bumblebee demo:
# scripts/ci-peak-anon.sh header and docs/ci-cd.md "CI build memory".
#
# ⚠️ READ THE BREAKDOWN, NOT JUST THE PEAK. `memory.peak` is the high-water mark of
# `memory.current`, which charges **page cache** to the cgroup as well as anonymous memory —
# it is NOT "peak RSS", and for a build job (NuGet/npm/obj/bin/coverage I/O) the cache
# dominates. Demonstrated on bumblebee: a container with anon=0 that merely reads an 800 MB
# file reports memory.peak=826 MiB, of which file=800 MiB. This matters because the naive
# reading inverts the decision: page cache is **reclaimed** under a tighter cap, not
# OOM-killed, so a large peak that is mostly `file` is NOT evidence that the cap must stay
# high. `anon` (+ a little kernel/sock) is the part that actually forces an OOM.
#
# The split below is read at end-of-job, so it is the *current* composition rather than the
# composition at the peak instant — indicative, not exact. Sizing a cap off one run is still
# wrong; take a few runs, and treat anon as the floor and peak as the (cache-inflated)
# ceiling. Refining this into a true peak-anon sample is ersatztv#412.
#
# Runs LAST on purpose: memory.peak read at step N reports the peak only up to N, so this
# sits after Coverage summary to include reportgenerator, the job's last real workload.
# cgroup v2 first, v1 fallback.
#
# Skipped on docs-only runs (ersatztv#416): nothing ran, so there is nothing to measure.
# Runs LAST on purpose (after Coverage summary / reportgenerator, the job's last real workload)
# and stops the sampler. `always()` so a failed Build/Test still gets a peak reading; the split
# is read here (end-of-job = composition then, not at the peak instant — that is exactly why the
# sampler exists). Skipped on docs-only/already-validated runs (nothing ran to measure).
- name: Report peak container memory
# `always()` controls whether this step RUNS, not whether its failure fails the job — and
# `defaults.run.shell: bash` means `-e -o pipefail` is on, so a failed `cat`/redirect here
# would redden a green test job. `continue-on-error` is what actually makes it advisory,
# the same guarantee the Coverage summary step above uses.
# `always()` controls whether this step RUNS, not whether its failure fails the job. With
# `defaults.run.shell: bash` (`-e -o pipefail`) a stray non-zero here would redden a green
# test job, so `continue-on-error` makes it advisory — the same guarantee Coverage summary uses.
if: ${{ always() && steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true' }}
continue-on-error: true
run: |
mib() { echo "$(( ${1:-0} / 1048576 ))"; }
peak=""; src=""
for f in /sys/fs/cgroup/memory.peak /sys/fs/cgroup/memory/memory.max_usage_in_bytes; do
if [ -r "$f" ]; then peak=$(cat "$f" 2>/dev/null || echo ""); src="$f"; break; fi
done
if [ -z "$peak" ]; then
echo "No cgroup peak-memory file readable in this container -- skipping."
exit 0
fi
anon=""; file=""
if [ -r /sys/fs/cgroup/memory.stat ]; then
anon=$(awk '/^anon /{print $2}' /sys/fs/cgroup/memory.stat 2>/dev/null || echo "")
file=$(awk '/^file /{print $2}' /sys/fs/cgroup/memory.stat 2>/dev/null || echo "")
fi
echo "::group::Container memory (ersatztv#406 / server-management#604)"
printf 'peak (incl. page cache): %s MiB [%s bytes, %s]\n' "$(mib "$peak")" "$peak" "$src"
if [ -n "$anon" ]; then
printf 'end-of-job anon (the part that OOMs): %s MiB\n' "$(mib "$anon")"
printf 'end-of-job file (page cache, reclaimable): %s MiB\n' "$(mib "${file:-0}")"
echo 'NOTE: peak counts reclaimable page cache. Size caps on anon, not on peak.'
else
echo 'NOTE: no memory.stat breakdown available; peak includes reclaimable page cache.'
fi
echo "::endgroup::"
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
{
printf '**Container memory (test job):** peak %s MiB *(incl. reclaimable page cache)*' \
"$(mib "$peak")"
[ -n "$anon" ] && printf ' · end-of-job anon %s MiB · file %s MiB' \
"$(mib "$anon")" "$(mib "${file:-0}")"
printf '\n'
} >> "$GITHUB_STEP_SUMMARY" || true
fi
run: scripts/ci-peak-anon.sh report
migrations:
name: EF migration integrity (SQLite + MySql)
@@ -885,16 +850,26 @@ jobs:
echo "Generated API artifacts are in sync."
# Formatting-as-you-touch gate (ersatztv#311): verify the .cs files THIS PR changed conform to
# .editorconfig (style + charset=utf-8, i.e. no UTF-8 BOM). Scoped to changed files so it enforces
# "normalize a legacy file when you touch it" WITHOUT a big-bang reformat of the ~2500 pre-existing
# BOM files. A PR that touches no .cs skips the expensive steps and passes trivially (always reports
# a status, so it is safe as a required check).
# .editorconfig whitespace + charset=utf-8 (i.e. no UTF-8 BOM). Scoped to changed files so it
# enforces "normalize a legacy file when you touch it" WITHOUT a big-bang reformat of the ~2500
# pre-existing BOM files. A PR that touches no .cs skips the check and passes trivially (always
# reports a status, so it is safe as a required check).
#
# ersatztv#469: uses `dotnet format whitespace . --folder`, NOT the full `dotnet format <sln>`.
# `--folder` treats the tree as a plain folder of files and skips the MSBuild/Roslyn workspace load
# + per-project compilation that dominated the old recipe (~8 min locally on a whole-solution run) —
# `--include` only ever narrowed *which* files were checked, never what got loaded. Folder mode
# reads .editorconfig and still flags WHITESPACE (indent/EOL/trailing/final-newline) and CHARSET
# (BOM) violations — exactly what this gate exists to catch — in ~0.5s with no `dotnet restore`.
# What it drops is the style/analyzer pass (naming/`var`/qualification), which this gate never
# meaningfully enforced: those .editorconfig rules are :suggestion/:none severity. Full rationale +
# non-vacuity evidence: docs/ci-cd.md → Formatting; docs/decisions.md.
format:
name: Formatting (changed .cs conform to .editorconfig)
# Was on the `small` lane (ersatztv#390) to dodge a ~29 min queue; reverted to `ubuntu-latest`
# in ersatztv#406 — `dotnet format` needs the .NET SDK and real memory, so it does not belong
# in a lane sized for seconds-long shell jobs. See the api-docs job above for the full
# rationale; server-management#604 grew this lane so the queue it was dodging is gone.
# Folder-mode whitespace is now a seconds-long, low-memory job (no Roslyn workspace, unlike the
# 3.95 GiB full `dotnet format` measured in #406), so it no longer needs the memory headroom that
# kept it on `ubuntu-latest`. Left here to avoid re-touching the lane/memory-cap accounting; a
# move to a lighter lane is a server-management capacity call (#604).
runs-on: ubuntu-latest
container:
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
@@ -924,26 +899,14 @@ jobs:
echo "No .cs change -> skipping format verify (job passes)."
fi
- name: Cache NuGet packages
if: steps.detect.outputs.cs_changed == 'true'
uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: nuget-${{ runner.os }}-${{ hashFiles('Directory.Packages.props', 'global.json') }}
restore-keys: nuget-${{ runner.os }}-
- name: Restore
if: steps.detect.outputs.cs_changed == 'true'
run: dotnet restore
- name: Verify formatting of changed .cs files
if: steps.detect.outputs.cs_changed == 'true'
shell: bash
run: |
mapfile -t files < /tmp/changed-cs.txt
echo "Verifying ${#files[@]} changed .cs file(s) against .editorconfig..."
if ! dotnet format ErsatzTV.sln --no-restore --verify-no-changes --include "${files[@]}"; then
echo "::error::One or more .cs files this PR touches don't conform to .editorconfig (formatting or a UTF-8 BOM). Run 'dotnet format ErsatzTV.sln --include <files>' and commit the result in THIS PR — the fix-as-you-touch convention (docs/contributing.md §7; ersatztv#311). Legacy files you did NOT touch are unaffected."
echo "Verifying ${#files[@]} changed .cs file(s) against .editorconfig (whitespace + charset)..."
if ! dotnet format whitespace . --folder --verify-no-changes --include "${files[@]}"; then
echo "::error::One or more .cs files this PR touches don't conform to .editorconfig (whitespace or a UTF-8 BOM). Run 'dotnet format whitespace . --folder --include <files>' (or the full 'dotnet format ErsatzTV.sln --include <files>') and commit the result in THIS PR — the fix-as-you-touch convention (docs/contributing.md §7; ersatztv#311). Legacy files you did NOT touch are unaffected."
exit 1
fi
echo "All changed .cs files conform to .editorconfig."
+8 -6
View File
@@ -11,15 +11,17 @@ if [ -n "$root_png" ]; then
exit 1
fi
# dotnet format on staged .cs files (repo root). Scoped to the staged files so we
# don't pay the full-tree cost; skip entirely when no .cs is staged (avoids the
# ~20-40s sln load for web-only commits).
# dotnet format on staged .cs files (repo root). Uses `whitespace . --folder` — same recipe as
# the CI `format` job (ersatztv#469): folder mode checks .editorconfig whitespace + charset (BOM)
# without the MSBuild/Roslyn workspace load, so it runs in ~0.5s instead of the old ~20-40s sln
# load. Keeping this identical to CI avoids a local hook that blocks on rules CI no longer enforces.
# Skip entirely when no .cs is staged (avoids any cost for web-only commits).
cs_files=$(git diff --cached --name-only --diff-filter=ACM -- '*.cs')
if [ -n "$cs_files" ]; then
echo "husky - dotnet format (verify) on staged .cs files"
echo "husky - dotnet format (whitespace verify) on staged .cs files"
# shellcheck disable=SC2086
dotnet format ErsatzTV.sln --verify-no-changes --include $cs_files || {
echo "husky - dotnet format found issues in staged .cs files; run 'dotnet format ErsatzTV.sln --include <files>' to fix"
dotnet format whitespace . --folder --verify-no-changes --include $cs_files || {
echo "husky - dotnet format found whitespace/BOM issues in staged .cs files; run 'dotnet format whitespace . --folder --include <files>' to fix"
exit 1
}
fi
+2 -2
View File
@@ -35,8 +35,8 @@ Custom IPTV channel server for Jellyfin. Forked from [ErsatzTV/ErsatzTV](https:/
## Deployment
- **Docker host**: bumblebee (192.168.1.99), container `ersatztv`, port 8409
- **Config volume**: `~/downloadswarm/ersatztv/` on bumblebee`/config` in container
- **Docker host**: **jazz (192.168.1.29)**, container `ersatztv`, port 8409. Media transcoders (Jellyfin, `ersatztv`, `ersatztv-test`) moved here from bumblebee on 2026-07-20 (server-management#633); bumblebee (192.168.1.99) still hosts the **CI runners** and the rest of the stacks. **Name-reuse trap**: `jazz` was an *earlier* name for the .99 host, so pre-2026-07-20 docs/commits saying "jazz" mean today's **bumblebee** — go by the IP, not the name.
- **Config volume**: `~/downloadswarm/ersatztv/` on jazz`/config` in container
- **SQLite DB**: `/config/ersatztv.sqlite3` (WAL mode, root-owned)
- **Images** (our fork, built by `.gitea/workflows/docker-build.yml``192.168.1.95:3000/timothy/ersatztv`): push to `main``:latest` + `:<sha>` (test image); push `v*` tag → `:prod` + `:<version>` + `:<sha>`. Prod's **Komodo GitOps** `media-servers` stack follows floating `:prod`; after the immutable `:<version>` candidate passes the release scans, manually deploy the stack (Global Auto Update is the daily fallback). Both paths run the fail-closed pre-deploy backup and prod-copy migration smoke before recreation. Test tracks `:latest`. Pipeline details: `docs/ci-cd.md`.
+27 -1
View File
@@ -109,7 +109,8 @@ internal static class Mapper
GetStreamingMode(channel),
channel.IsEnabled,
channel.ShowInEpg,
playoutCount);
playoutCount,
GetLogoUrl(channel));
internal static ResolutionViewModel ProjectToViewModel(Resolution resolution) =>
new(resolution.Height, resolution.Width);
@@ -123,6 +124,31 @@ internal static class Mapper
channel.FFmpegProfile.VideoProfile,
channel.FFmpegProfile.AudioFormat);
// Rooted, directly-usable channel-logo URL for the SPA's <img src> on browse surfaces (guide grid +
// channels list), following the #181 artwork convention (docs/api-conventions.md §4): the SPA does no
// client-side path building. External logo URLs pass through as-is; an uploaded logo ("iptv/logos/{file}")
// is rooted with a leading slash so it resolves against the site root regardless of the current SPA route.
// Returns null when the channel has no logo, so the SPA falls back to the generated initials "bug".
#nullable enable
internal static string? GetLogoUrl(Channel channel)
{
// Browse surfaces must not crash the whole list over a missing Artwork include; GetLogo assumes
// the caller included Channel.Artwork (GetAll + the guide query do), but stay defensive here.
if (channel.Artwork is null)
{
return null;
}
ArtworkContentTypeModel logo = GetLogo(channel);
if (string.IsNullOrWhiteSpace(logo.Path))
{
return null;
}
return logo.IsExternalUrl || logo.Path.StartsWith('/') ? logo.Path : $"/{logo.Path}";
}
#nullable restore
private static ArtworkContentTypeModel GetLogo(Channel channel)
{
Option<Artwork> maybeArtwork = channel.Artwork
@@ -47,6 +47,7 @@ public class GetChannelGuideDataHandler(
List<Channel> channels = await dbContext.Channels
.AsNoTracking()
.Where(c => c.ShowInEpg)
.Include(c => c.Artwork)
.Include(c => c.MirrorSourceChannel)
.ToListAsync(cancellationToken);
@@ -121,6 +122,7 @@ public class GetChannelGuideDataHandler(
new ChannelGuideChannelResponseModel(
channel.Number,
channel.Name,
Mapper.GetLogoUrl(channel),
programmes.OrderBy(p => p.Start).ToList()));
}
@@ -1,4 +1,4 @@
using ErsatzTV.Core;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
@@ -34,4 +34,5 @@ public record CreateFFmpegProfile(
int AudioSampleRate,
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo) : IRequest<Either<BaseError, CreateFFmpegProfileResult>>;
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder) : IRequest<Either<BaseError, CreateFFmpegProfileResult>>;
@@ -1,4 +1,4 @@
using ErsatzTV.Core;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Interfaces.Search;
@@ -105,7 +105,8 @@ public class CreateFFmpegProfileHandler :
AudioSampleRate = request.AudioSampleRate,
NormalizeFramerate = request.NormalizeFramerate,
NormalizeColors = request.NormalizeColors,
DeinterlaceVideo = request.DeinterlaceVideo
DeinterlaceVideo = request.DeinterlaceVideo,
QsvPreferNativeDecoder = request.QsvPreferNativeDecoder
};
});
@@ -1,4 +1,4 @@
using ErsatzTV.Core;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
@@ -35,4 +35,5 @@ public record UpdateFFmpegProfile(
int AudioSampleRate,
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo) : IRequest<Either<BaseError, UpdateFFmpegProfileResult>>;
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder) : IRequest<Either<BaseError, UpdateFFmpegProfileResult>>;
@@ -102,6 +102,7 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
p.NormalizeFramerate = update.NormalizeFramerate;
p.NormalizeColors = update.NormalizeColors;
p.DeinterlaceVideo = update.DeinterlaceVideo;
p.QsvPreferNativeDecoder = update.QsvPreferNativeDecoder;
// don't save invalid preset
ICollection<string> presets = FFmpegLibraryHelper.PresetsForFFmpegProfile(
@@ -1,4 +1,4 @@
using ErsatzTV.Application.Resolutions;
using ErsatzTV.Application.Resolutions;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
@@ -35,4 +35,5 @@ public record FFmpegProfileViewModel(
int AudioSampleRate,
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo);
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder);
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Api.FFmpegProfiles;
using ErsatzTV.Core.Api.FFmpegProfiles;
using ErsatzTV.Core.Domain;
namespace ErsatzTV.Application.FFmpegProfiles;
@@ -37,7 +37,8 @@ internal static class Mapper
profile.AudioSampleRate,
profile.NormalizeFramerate,
profile.NormalizeColors,
profile.DeinterlaceVideo == true);
profile.DeinterlaceVideo == true,
profile.QsvPreferNativeDecoder != false);
internal static FFmpegProfileResponseModel ProjectToResponseModel(FFmpegProfile ffmpegProfile) =>
new(
@@ -80,5 +81,6 @@ internal static class Mapper
ffmpegProfile.AudioSampleRate,
ffmpegProfile.NormalizeFramerate,
ffmpegProfile.NormalizeColors,
ffmpegProfile.DeinterlaceVideo == true);
ffmpegProfile.DeinterlaceVideo == true,
ffmpegProfile.QsvPreferNativeDecoder != false);
}
@@ -2,4 +2,4 @@ using ErsatzTV.Core.Api.Health;
namespace ErsatzTV.Application.Health;
public record GetAllHealthCheckResultsForApi : IRequest<List<HealthCheckResponseModel>>;
public record GetAllHealthCheckResultsForApi(bool Refresh = false) : IRequest<List<HealthCheckResponseModel>>;
@@ -18,7 +18,8 @@ public class GetAllHealthCheckResultsForApiHandler
{
try
{
List<HealthCheckResult> results = await _healthCheckService.PerformHealthChecks(cancellationToken);
List<HealthCheckResult> results =
await _healthCheckService.PerformHealthChecks(request.Refresh, cancellationToken);
return results
.Filter(r => r.Status != HealthCheckStatus.NotApplicable)
.Map(ProjectToResponseModel)
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Health;
using ErsatzTV.Core.Health;
namespace ErsatzTV.Application.Health;
@@ -15,7 +15,7 @@ public class GetAllHealthCheckResultsHandler : IRequestHandler<GetAllHealthCheck
{
try
{
List<HealthCheckResult> results = await _healthCheckService.PerformHealthChecks(cancellationToken);
List<HealthCheckResult> results = await _healthCheckService.PerformHealthChecks(false, cancellationToken);
return results.Filter(r => r.Status != HealthCheckStatus.NotApplicable).ToList();
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException)
@@ -1,4 +1,4 @@
using System.IO.Abstractions;
using System.IO.Abstractions;
using System.Threading.Channels;
using ErsatzTV.Application.MediaSources;
using ErsatzTV.Core;
@@ -70,9 +70,23 @@ public class CreateLocalLibraryHandler : LocalLibraryHandlerBase,
CreateLocalLibrary request) =>
MediaSourceMustExist(dbContext, request)
.BindT(localLibrary => NameMustBeValid(request, localLibrary))
.BindT(MediaKindMustBeSupportedLocally)
.BindT(localLibrary => PathsMustBeValid(dbContext, localLibrary))
.BindT(localLibrary => NewPathsMustExist(fileSystem, localLibrary));
/// <summary>
/// Mixed is only ever produced for remote (Jellyfin) libraries, where the media server classifies
/// each item for us. No local folder scanner handles it, so a local Mixed library would fail every
/// scan forever. The API takes a raw LibraryMediaKind, so this must be enforced here rather than
/// left to the SPA's media-kind options.
/// </summary>
private static Validation<BaseError, LocalLibrary> MediaKindMustBeSupportedLocally(
LocalLibrary localLibrary) =>
localLibrary.MediaKind is LibraryMediaKind.Mixed
? BaseError.New(
"Local libraries cannot use the Mixed media kind; it is only valid for Jellyfin libraries.")
: localLibrary;
private static Task<Validation<BaseError, LocalLibrary>> MediaSourceMustExist(
TvContext dbContext,
CreateLocalLibrary request) =>
@@ -10,6 +10,16 @@ public class GetAllMediaSourcesForApiHandler(
IDbContextFactory<TvContext> dbContextFactory)
: IRequestHandler<GetAllMediaSourcesForApi, List<MediaSourceResponseModel>>
{
// A never-scanned library has a null LastScan at runtime, but historical DB rows still carry the
// 0001-01-01 MinValue sentinel written by the old Reset_* migrations. Coerce any such residual
// sentinel to null so the API/MCP surface reports "never scanned" as null (parity with the UI),
// regardless of DB history or provider. Belt-and-suspenders alongside the NullOutNeverScannedLastScan
// data migration.
private static readonly DateTime NeverScannedThreshold = new(2000, 1, 1);
private static DateTime? NormalizeLastScan(DateTime? lastScan) =>
lastScan is { } value && value < NeverScannedThreshold ? null : lastScan;
public async Task<List<MediaSourceResponseModel>> Handle(
GetAllMediaSourcesForApi request,
CancellationToken cancellationToken)
@@ -36,7 +46,7 @@ public class GetAllMediaSourcesForApiHandler(
l.Id,
l.Name,
l.MediaKind,
l.LastScan,
NormalizeLastScan(l.LastScan),
itemCountsByLibrary.TryGetValue(l.Id, out int count) ? count : 0))
.ToList();
@@ -131,9 +131,19 @@ public class StartFFmpegSessionHandler : IRequestHandler<StartFFmpegSession, Eit
long startupMs = (long)segments.ProcessStartup.TotalMilliseconds;
long fillMs = (long)segments.SegmentFill.TotalMilliseconds;
long setupMs = Math.Max(0, totalMs - startupMs - fillMs);
// #472 sub-splits the startup work (81% of total, all of the variance) into the ErsatzTV-side
// prep before FFmpeg is launched, FFmpeg's own init (input open+probe and decoder/encoder
// init), and the wait for the playlist once FFmpeg is reporting progress. splitKind says how
// much of that was actually observable for this sample. NOTE these buckets span the worker's
// Run entry rather than the startup stopwatch, so they do NOT sum to startupMs — prep overlaps
// the tail of setup. The log says "spans runEntry" so a reader can't miss it.
// See ColdStartStartupSplit for the full set of caveats.
ColdStartStartupSplit split = segments.StartupSplit;
_logger.LogInformation(
"HLS cold-start channel {Channel} mode {Mode}: total {TotalMs}ms " +
"(setup {SetupMs}ms + startup {ProcessStartupMs}ms + fill {SegmentFillMs}ms), " +
"startup split {SplitKind} spans runEntry (prep {PrepMs}ms + ffmpegInit {FFmpegInitMs}ms " +
"+ firstGop {FirstGopMs}ms), " +
"segments {SegmentsReached}/{InitialSegmentCount}, " +
"deadlineExpired {DeadlineExpired}, subtitleBurnIn {SubtitleBurnIn}, hwaccel {HwAccel}",
request.ChannelNumber,
@@ -142,6 +152,10 @@ public class StartFFmpegSessionHandler : IRequestHandler<StartFFmpegSession, Eit
setupMs,
startupMs,
fillMs,
split.Kind,
(long)split.Prep.TotalMilliseconds,
(long)split.FFmpegInit.TotalMilliseconds,
(long)split.FirstGop.TotalMilliseconds,
segments.SegmentsReached,
segments.InitialSegmentCount,
segments.DeadlineExpired,
@@ -61,6 +61,14 @@ public class HlsSessionWorker : IHlsSessionWorker
// segments cannot exist until this process ran) — volatile for cross-thread visibility.
private volatile string _coldStartFFmpegArguments;
// Stopwatch timestamps of the cold-start milestones used to sub-split the "startup" phase (#472).
// Each is written once on the sequential Run loop and read on the handler thread from
// WaitForPlaylistSegments; long fields cannot be volatile, so access goes through Volatile/
// Interlocked. Zero means "never reached", which ColdStartStartupSplit degrades gracefully on.
private long _coldStartRunTicks;
private long _coldStartProcessLaunchedTicks;
private long _coldStartFirstProgressTicks;
public HlsSessionWorker(
IServiceScopeFactory serviceScopeFactory,
IGraphicsEngine graphicsEngine,
@@ -187,6 +195,10 @@ public class HlsSessionWorker : IHlsSessionWorker
{
_cancellationTokenSource = CancellationTokenSource.CreateLinkedTokenSource(incomingCancellationToken);
// anchor for the cold-start startup sub-split (#472); this runs before any later milestone,
// so every sub-phase derived from it is non-negative by construction
Volatile.Write(ref _coldStartRunTicks, Stopwatch.GetTimestamp());
try
{
_channelNumber = channelNumber;
@@ -314,6 +326,7 @@ public class HlsSessionWorker : IHlsSessionWorker
var sw = Stopwatch.StartNew();
var processStartup = TimeSpan.Zero;
var startupSplit = ColdStartStartupSplit.Unavailable;
var segmentCount = 0;
try
{
@@ -329,6 +342,13 @@ public class HlsSessionWorker : IHlsSessionWorker
_logger.LogDebug("Playlist exists");
processStartup = sw.Elapsed;
// #472: sub-split the phase that #350 measured as 81% of cold-start and all of its variance
startupSplit = ColdStartStartupSplit.FromTimestamps(
Volatile.Read(ref _coldStartRunTicks),
Volatile.Read(ref _coldStartProcessLaunchedTicks),
Volatile.Read(ref _coldStartFirstProgressTicks),
Stopwatch.GetTimestamp());
// start the segment-wait deadline only after the playlist file appears,
// so slow pipeline setup (e.g. h264 profile probing) doesn't consume the budget
DateTimeOffset finish = DateTimeOffset.Now.AddSeconds(8);
@@ -362,7 +382,8 @@ public class HlsSessionWorker : IHlsSessionWorker
segmentCount,
initialSegmentCount,
segmentCount < initialSegmentCount,
ColdStartFeatures.FromFFmpegArguments(_coldStartFFmpegArguments));
ColdStartFeatures.FromFFmpegArguments(_coldStartFFmpegArguments),
startupSplit);
}
finally
{
@@ -576,10 +597,30 @@ public class HlsSessionWorker : IHlsSessionWorker
var progressParser = new FFmpegProgress();
// #472: the first -progress line is the only cold-start milestone FFmpeg gives us
// for free (the pipeline runs -loglevel error -nostats, so stderr stays silent on a
// healthy run). It means the input is open and probed and the decoder/encoder are
// initialized. Record-once, so only the session's first process is measured.
void ParseProgressLine(string line)
{
// the read short-circuits the timestamp call for every line after the first,
// which is every line for the life of the session
if (Volatile.Read(ref _coldStartFirstProgressTicks) == 0)
{
Interlocked.CompareExchange(ref _coldStartFirstProgressTicks, Stopwatch.GetTimestamp(), 0);
}
progressParser.ParseLine(line);
}
// everything before this point is ErsatzTV-side "prep" (playout item resolution,
// pipeline build, graphics engine spawn); FFmpeg's own clock starts here
Interlocked.CompareExchange(ref _coldStartProcessLaunchedTicks, Stopwatch.GetTimestamp(), 0);
CommandResult commandResult = await processWithPipe
.WithWorkingDirectory(_workingDirectory)
.WithStandardErrorPipe(PipeTarget.ToStringBuilder(stdErrBuffer))
.WithStandardOutputPipe(PipeTarget.ToDelegate(progressParser.ParseLine))
.WithStandardOutputPipe(PipeTarget.ToDelegate(ParseProgressLine))
.WithValidation(CommandResultValidation.None)
.ExecuteAsync(linkedCts.Token);
@@ -673,6 +714,20 @@ public class HlsSessionWorker : IHlsSessionWorker
}
}
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException
&& cancellationToken.IsCancellationRequested)
{
// a cancellation anywhere in this method (including inside the mediator sends, which sit
// outside the inner ffmpeg try below) is a shutdown or a client disconnect, not a fault.
// Without this it reaches the catch-all and logs a channel-level ERROR with a stack
// trace on every graceful teardown. The token check is load-bearing: TaskCanceledException
// is also what HttpClient throws on ITS OWN timeout, and a real timeout inside ffprobe, a
// media-server call or subtitle extraction must keep its ERROR-level signal rather than
// being downgraded to a routine teardown. (ersatztv#473 review)
_logger.LogInformation("Terminating HLS session for channel {Channel}", _channelNumber);
return false;
}
catch (Exception ex)
{
_logger.LogError(ex, "Error transcoding channel {Channel} - {Message}", _channelNumber, ex.Message);
@@ -1,4 +1,4 @@
using System.IO.Abstractions;
using System.IO.Abstractions;
using CliWrap;
using Dapper;
using ErsatzTV.Application.Playouts;
@@ -42,6 +42,7 @@ public class GetPlayoutItemProcessByChannelNumberHandler : FFmpegProcessHandler<
private readonly IGraphicsElementSelector _graphicsElementSelector;
private readonly IDecoSelector _decoSelector;
private readonly IPlexPathReplacementService _plexPathReplacementService;
private readonly IRemoteStreamProber _remoteStreamProber;
private readonly ISongVideoGenerator _songVideoGenerator;
private readonly ITelevisionRepository _televisionRepository;
private readonly bool _isDebugNoSync;
@@ -62,9 +63,11 @@ public class GetPlayoutItemProcessByChannelNumberHandler : FFmpegProcessHandler<
IWatermarkSelector watermarkSelector,
IGraphicsElementSelector graphicsElementSelector,
IDecoSelector decoSelector,
IRemoteStreamProber remoteStreamProber,
ILogger<GetPlayoutItemProcessByChannelNumberHandler> logger)
: base(dbContextFactory)
{
_remoteStreamProber = remoteStreamProber;
_ffmpegProcessService = ffmpegProcessService;
_fileSystem = fileSystem;
_externalJsonPlayoutItemProvider = externalJsonPlayoutItemProvider;
@@ -549,6 +552,7 @@ public class GetPlayoutItemProcessByChannelNumberHandler : FFmpegProcessHandler<
Optional(channel.PlayoutOffset),
!request.HlsRealtime);
case PlayoutItemDoesNotExistOnDisk:
case PlayoutItemNotAvailableFromMediaServer:
Command doesNotExistProcess = await _ffmpegProcessService.ForError(
ffmpegPath,
channel,
@@ -850,9 +854,15 @@ public class GetPlayoutItemProcessByChannelNumberHandler : FFmpegProcessHandler<
pmf.Path,
pmf.Key);
return new PlayoutItemWithPath(
playoutItem,
$"http://localhost:{Settings.StreamingPort}/media/plex/{plexMediaSourceId}/{pmf.Key}");
var plexUrl =
$"http://localhost:{Settings.StreamingPort}/media/plex/{plexMediaSourceId}/{pmf.Key}";
if (!await _remoteStreamProber.IsAvailable(plexUrl, cancellationToken))
{
return new PlayoutItemNotAvailableFromMediaServer(plexUrl);
}
return new PlayoutItemWithPath(playoutItem, plexUrl);
}
break;
@@ -868,9 +878,14 @@ public class GetPlayoutItemProcessByChannelNumberHandler : FFmpegProcessHandler<
foreach (string itemId in jellyfinItemId)
{
return new PlayoutItemWithPath(
playoutItem,
$"http://localhost:{Settings.StreamingPort}/media/jellyfin/{itemId}");
var jellyfinUrl = $"http://localhost:{Settings.StreamingPort}/media/jellyfin/{itemId}";
if (!await _remoteStreamProber.IsAvailable(jellyfinUrl, cancellationToken))
{
return new PlayoutItemNotAvailableFromMediaServer(jellyfinUrl);
}
return new PlayoutItemWithPath(playoutItem, jellyfinUrl);
}
// attempt to remotely stream emby
@@ -883,9 +898,14 @@ public class GetPlayoutItemProcessByChannelNumberHandler : FFmpegProcessHandler<
foreach (string itemId in embyItemId)
{
return new PlayoutItemWithPath(
playoutItem,
$"http://localhost:{Settings.StreamingPort}/media/emby/{itemId}");
var embyUrl = $"http://localhost:{Settings.StreamingPort}/media/emby/{itemId}";
if (!await _remoteStreamProber.IsAvailable(embyUrl, cancellationToken))
{
return new PlayoutItemNotAvailableFromMediaServer(embyUrl);
}
return new PlayoutItemWithPath(playoutItem, embyUrl);
}
return new PlayoutItemDoesNotExistOnDisk(path);
@@ -45,7 +45,8 @@ public class GetTroubleshootingInfoHandler : IRequestHandler<GetTroubleshootingI
public async Task<TroubleshootingInfo> Handle(GetTroubleshootingInfo request, CancellationToken cancellationToken)
{
List<HealthCheckResult> healthCheckResults = await _healthCheckService.PerformHealthChecks(cancellationToken);
// Support bundle wants current state, so force a fresh run rather than serving the poll cache.
List<HealthCheckResult> healthCheckResults = await _healthCheckService.PerformHealthChecks(true, cancellationToken);
string version = Assembly.GetEntryAssembly()?
.GetCustomAttribute<AssemblyInformationalVersionAttribute>()?
@@ -118,22 +119,22 @@ public class GetTroubleshootingInfoHandler : IRequestHandler<GetTroubleshootingI
{ VaapiDriver.iHD, VaapiDriver.i965, VaapiDriver.RadeonSI, VaapiDriver.Nouveau };
foreach (string display in vaapiDisplays)
foreach (VaapiDriver activeDriver in allDrivers)
foreach (string vaapiDevice in vaapiDevices)
{
foreach (string output in await _hardwareCapabilitiesFactory.GetVaapiOutput(
display,
Optional(GetDriverName(activeDriver)),
vaapiDevice))
{
vaapiCapabilities.AppendLine(
CultureInfo.InvariantCulture,
$"Checking display [{display}] driver [{activeDriver}] device [{vaapiDevice}]{Environment.NewLine}");
vaapiCapabilities.AppendLine();
vaapiCapabilities.AppendLine(output);
vaapiCapabilities.AppendLine();
}
}
foreach (VaapiDriver activeDriver in allDrivers)
foreach (string vaapiDevice in vaapiDevices)
{
foreach (string output in await _hardwareCapabilitiesFactory.GetVaapiOutput(
display,
Optional(GetDriverName(activeDriver)),
vaapiDevice))
{
vaapiCapabilities.AppendLine(
CultureInfo.InvariantCulture,
$"Checking display [{display}] driver [{activeDriver}] device [{vaapiDevice}]{Environment.NewLine}");
vaapiCapabilities.AppendLine();
vaapiCapabilities.AppendLine(output);
vaapiCapabilities.AppendLine();
}
}
}
if (_runtimeInfo.IsOSPlatform(OSPlatform.OSX))
+2 -2
View File
@@ -1,4 +1,4 @@
using ErsatzTV.Application.Artworks;
using ErsatzTV.Application.Artworks;
using ErsatzTV.Core.Api.Watermarks;
using ErsatzTV.Core.Domain;
@@ -7,7 +7,7 @@ namespace ErsatzTV.Application.Watermarks;
internal static class Mapper
{
internal static WatermarkResponseModel ProjectToResponseModel(ChannelWatermark watermark) =>
new(watermark.Id, watermark.Name);
new(watermark.Id, watermark.Name, watermark.ImageSource);
internal static WatermarkFullResponseModel ProjectToFullResponseModel(ChannelWatermark watermark) =>
new(
@@ -0,0 +1,171 @@
using System.Diagnostics;
using ErsatzTV.Core.FFmpeg;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Core.Tests.FFmpeg;
[TestFixture]
public class ColdStartStartupSplitTests
{
// milestones are Stopwatch.GetTimestamp() values; build them from a base + millisecond offsets
private const long Base = 1_000_000_000;
private static long At(double milliseconds) =>
Base + (long)(milliseconds / 1000.0 * Stopwatch.Frequency);
[Test]
public void Should_Split_Three_Ways_When_All_Milestones_Present()
{
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
At(0),
At(150),
At(1200),
At(1600));
split.Kind.ShouldBe(ColdStartStartupSplitKind.ThreeWay);
split.Prep.TotalMilliseconds.ShouldBe(150, 1);
split.FFmpegInit.TotalMilliseconds.ShouldBe(1050, 1);
split.FirstGop.TotalMilliseconds.ShouldBe(400, 1);
}
[Test]
public void Sub_Phases_Should_Sum_To_Run_Entry_Through_Playlist()
{
// deliberately NOT "should sum to startup": the buckets span the worker's Run entry, which
// begins before the request thread's startup stopwatch, so prep overlaps the tail of setup
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
At(0),
At(150),
At(1200),
At(1600));
(split.Prep + split.FFmpegInit + split.FirstGop).TotalMilliseconds.ShouldBe(1600, 1);
}
[Test]
public void Should_Be_Unavailable_When_The_Playlist_Predates_The_Process_Launch()
{
// a stale live.m3u8 survives when the handler's pre-session folder wipe fails (EmptyFolder
// swallows the failure into a warning). Every bucket would be meaningless, so report nothing
// rather than a plausible-looking sample with a prep that exceeds the whole measured phase
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
At(0),
At(1600),
0,
At(150));
split.ShouldBe(ColdStartStartupSplit.Unavailable);
}
[Test]
public void Stale_Playlist_Guard_Should_Take_Precedence_Over_The_Progress_Branches()
{
// without the guard, this input would be classified TwoWayLateProgress; the guard must be
// evaluated first. (It can never preempt a ThreeWay: that requires processLaunched <=
// playlistExists, which is exactly the negation of the guard condition.)
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
At(0),
At(1600),
At(1700),
At(150));
split.ShouldBe(ColdStartStartupSplit.Unavailable);
}
[Test]
public void Should_Fall_Back_To_Two_Way_Split_When_Progress_Predates_The_Process_Launch()
{
// a progress timestamp older than the launch cannot belong to this process
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
At(100),
At(150),
At(120),
At(1600));
split.Kind.ShouldBe(ColdStartStartupSplitKind.TwoWay);
split.FFmpegInit.TotalMilliseconds.ShouldBe(1450, 1);
split.FirstGop.ShouldBe(TimeSpan.Zero);
}
[Test]
public void Should_Stay_Three_Way_When_Progress_Coincides_With_A_Boundary()
{
ColdStartStartupSplit atLaunch = ColdStartStartupSplit.FromTimestamps(At(0), At(150), At(150), At(1600));
atLaunch.Kind.ShouldBe(ColdStartStartupSplitKind.ThreeWay);
atLaunch.FFmpegInit.ShouldBe(TimeSpan.Zero);
atLaunch.FirstGop.TotalMilliseconds.ShouldBe(1450, 1);
ColdStartStartupSplit atPlaylist = ColdStartStartupSplit.FromTimestamps(At(0), At(150), At(1600), At(1600));
atPlaylist.Kind.ShouldBe(ColdStartStartupSplitKind.ThreeWay);
atPlaylist.FFmpegInit.TotalMilliseconds.ShouldBe(1450, 1);
atPlaylist.FirstGop.ShouldBe(TimeSpan.Zero);
}
[Test]
public void Should_Fall_Back_To_Two_Way_Split_When_FFmpeg_Never_Reported_Progress()
{
// no -progress output before the playlist appeared: ffmpegInit must absorb the remainder
// rather than the split inventing a firstGop boundary that was never observed
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
At(0),
At(150),
0,
At(1600));
split.Kind.ShouldBe(ColdStartStartupSplitKind.TwoWay);
split.Prep.TotalMilliseconds.ShouldBe(150, 1);
split.FFmpegInit.TotalMilliseconds.ShouldBe(1450, 1);
split.FirstGop.ShouldBe(TimeSpan.Zero);
}
[Test]
public void Should_Report_Late_Progress_Distinctly_When_Progress_Arrived_After_The_Playlist()
{
// the playlist is observed on the request thread while progress is recorded on the worker
// thread; a progress milestone outside the phase must not produce a negative bucket
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
At(0),
At(150),
At(1800),
At(1600));
split.Kind.ShouldBe(ColdStartStartupSplitKind.TwoWayLateProgress);
split.FFmpegInit.TotalMilliseconds.ShouldBe(1450, 1);
split.FirstGop.ShouldBe(TimeSpan.Zero);
}
[TestCase(0L, 150L, 1200L, 1600L, TestName = "Run never started")]
[TestCase(100L, 0L, 0L, 1600L, TestName = "Process never launched")]
[TestCase(100L, 150L, 1200L, 0L, TestName = "Playlist never appeared")]
public void Should_Be_Unavailable_When_A_Required_Milestone_Is_Missing(
long runStarted,
long processLaunched,
long firstProgress,
long playlistExists)
{
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
runStarted == 0 ? 0 : At(runStarted),
processLaunched == 0 ? 0 : At(processLaunched),
firstProgress == 0 ? 0 : At(firstProgress),
playlistExists == 0 ? 0 : At(playlistExists));
split.ShouldBe(ColdStartStartupSplit.Unavailable);
}
[Test]
public void Should_Clamp_Rather_Than_Report_A_Negative_Prep()
{
// defensive: launch cannot precede Run entry, but telemetry must never show a negative
ColdStartStartupSplit split = ColdStartStartupSplit.FromTimestamps(
At(500),
At(150),
At(1200),
At(1600));
split.Prep.ShouldBe(TimeSpan.Zero);
split.Kind.ShouldBe(ColdStartStartupSplitKind.ThreeWay);
split.FFmpegInit.TotalMilliseconds.ShouldBe(1050, 1);
split.FirstGop.TotalMilliseconds.ShouldBe(400, 1);
}
}
@@ -16,6 +16,9 @@ public record ChannelGuideProgrammeResponseModel(
public record ChannelGuideChannelResponseModel(
string Number,
string Name,
// Rooted, directly-usable logo URL for the SPA's <img src>; null when the channel has no logo
// (SPA then renders the generated initials fallback). See ErsatzTV.Application Channels.Mapper.GetLogoUrl.
string? Logo,
List<ChannelGuideProgrammeResponseModel> Programmes);
/// <summary>The JSON channel-guide response: the resolved window plus per-channel programme arrays.</summary>
@@ -16,4 +16,7 @@ public record ChannelResponseModel(
string StreamingMode,
bool IsEnabled,
bool ShowInEpg,
int PlayoutCount);
int PlayoutCount,
// Rooted, directly-usable logo URL for the SPA's <img src>; null when the channel has no logo
// (SPA then renders the generated initials fallback). See ErsatzTV.Application Channels.Mapper.GetLogoUrl.
string? Logo);
@@ -36,4 +36,5 @@ public record FFmpegFullProfileResponseModel(
int AudioSampleRate,
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo);
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder);
@@ -1,4 +1,8 @@
#nullable enable
using ErsatzTV.Core.Domain;
namespace ErsatzTV.Core.Api.Watermarks;
public record WatermarkResponseModel(int Id, string Name);
// ImageSource lets a client identify logo-driven presets (the seeded "Channel Bug") without
// matching a user-editable name. Additive under the frozen /api/v1 contract (#286).
public record WatermarkResponseModel(int Id, string Name, ChannelWatermarkImageSource ImageSource);
+1
View File
@@ -24,6 +24,7 @@ public class ConfigElementKey
public static ConfigElementKey FFmpegGlobalWatermarkId => new("ffmpeg.global_watermark_id");
public static ConfigElementKey FFmpegGlobalFallbackFillerId => new("ffmpeg.global_fallback_filler_id");
public static ConfigElementKey ChannelTemplatesDefaultTemplateId => new("channel_templates.default_template_id");
public static ConfigElementKey WatermarkChannelBugSeeded => new("watermark.channel_bug_seeded");
public static ConfigElementKey FFmpegSegmenterTimeout => new("ffmpeg.segmenter.timeout_seconds");
public static ConfigElementKey FFmpegWorkAheadSegmenters => new("ffmpeg.segmenter.work_ahead_limit");
public static ConfigElementKey FFmpegInitialSegmentCount => new("ffmpeg.segmenter.initial_segment_count");
+3 -1
View File
@@ -1,4 +1,4 @@
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.FFmpeg;
namespace ErsatzTV.Core.Domain;
@@ -14,6 +14,7 @@ public record FFmpegProfile
public VaapiDriver VaapiDriver { get; set; }
public string VaapiDevice { get; set; }
public int? QsvExtraHardwareFrames { get; set; }
public bool? QsvPreferNativeDecoder { get; set; }
public int ResolutionId { get; set; }
public Resolution Resolution { get; set; }
public ScalingBehavior ScalingBehavior { get; set; }
@@ -63,6 +64,7 @@ public record FFmpegProfile
NormalizeFramerate = false,
HardwareAcceleration = HardwareAccelerationKind.None,
QsvExtraHardwareFrames = 64,
QsvPreferNativeDecoder = true,
NormalizeAudio = true,
NormalizeVideo = true,
NormalizeColors = true
@@ -1,4 +1,4 @@
namespace ErsatzTV.Core.Domain;
namespace ErsatzTV.Core.Domain;
public enum LibraryMediaKind
{
@@ -8,5 +8,13 @@ public enum LibraryMediaKind
OtherVideos = 4,
Songs = 5,
Images = 6,
RemoteStreams = 7
RemoteStreams = 7,
/// <summary>
/// A library whose contents are heterogeneous - movies, shows and music videos together.
/// Only produced for remote (Jellyfin) libraries whose collection type is "mixed", where the
/// media server classifies each item for us. A local library is never Mixed: the local folder
/// scanners all share one video extension list and would claim each other's files.
/// </summary>
Mixed = 8
}
@@ -0,0 +1,9 @@
namespace ErsatzTV.Core.Errors;
public class PlayoutItemNotAvailableFromMediaServer : BaseError
{
public PlayoutItemNotAvailableFromMediaServer(string url) : base(
$"Playout item is not available from media server\n{url}")
{
}
}
@@ -0,0 +1,146 @@
using System.Diagnostics;
namespace ErsatzTV.Core.FFmpeg;
/// <summary>
/// How finely a cold-start's startup work could be broken down (#472).
/// </summary>
public enum ColdStartStartupSplitKind
{
/// <summary>
/// No split available: the FFmpeg process was never launched, the playlist never appeared, or the
/// playlist was observed before FFmpeg was launched (a stale playlist left behind because the
/// pre-session transcode-folder wipe failed — it logs a warning and continues).
/// </summary>
Unavailable = 0,
/// <summary>
/// Two-way split: <c>prep</c> + <c>ffmpegInit</c>, because FFmpeg emitted no progress output at all
/// before the playlist appeared. <c>ffmpegInit</c> therefore runs to the playlist.
/// </summary>
TwoWay = 1,
/// <summary>
/// Two-way split, distinguished because FFmpeg <em>did</em> report progress but only after the
/// playlist was observed. Same buckets as <see cref="TwoWay"/>; kept separate because it means the
/// playlist appeared before the first progress report rather than FFmpeg being silent, which is a
/// different story about the pipeline (and is also what the 100ms playlist poll can manufacture).
/// </summary>
TwoWayLateProgress = 2,
/// <summary>Three-way split: <c>prep</c> + <c>ffmpegInit</c> + <c>firstGop</c>.</summary>
ThreeWay = 3
}
/// <summary>
/// Sub-split of the HLS cold-start startup work (#472), which #350's measurement showed to be 81% of
/// tune-in latency and to carry 100% of its variance while remaining a single opaque bucket.
/// <para>
/// <see cref="Prep"/> is ErsatzTV-side work before FFmpeg exists: playout-item resolution, pipeline
/// build, graphics-engine spawn. <see cref="FFmpegInit"/> is FFmpeg from launch until it first reports
/// progress — input open + probe (the NFS hypothesis) plus decoder/encoder init (the VAAPI-contention
/// hypothesis). <see cref="FirstGop"/> is from that first progress report until <c>live.m3u8</c> exists.
/// </para>
/// <para>
/// <b>These buckets span the session worker's <c>Run</c> entry to the playlist appearing, which is NOT
/// exactly the logged <c>startup</c> phase</b>: the worker is launched fire-and-forget slightly before
/// the request thread starts the <c>startup</c> stopwatch, so <see cref="Prep"/> overlaps the tail of
/// the logged <c>setup</c> bucket (in practice one config read). Do not expect
/// <c>prep + ffmpegInit + firstGop</c> to equal <c>startup</c> — it is a superset by that overlap.
/// </para>
/// <para>
/// Because the pipeline runs <c>-loglevel error -nostats -hide_banner</c>, a healthy FFmpeg writes
/// nothing to stderr, so input-open and encoder-init cannot be separated from each other without
/// changing the FFmpeg command — which this instrumentation deliberately does not do. The
/// <c>-progress</c> stream on stdout is therefore the only zero-cost milestone available, and
/// <see cref="FFmpegInit"/> necessarily lumps those two candidates together. #472 accepts this: a
/// large <see cref="Prep"/> vs a large <see cref="FFmpegInit"/> is itself the first discrimination,
/// and it is honest about what it cannot yet see.
/// </para>
/// <para>
/// Three further caveats when reading these numbers. The playlist is detected by a 100ms poll, so its
/// timestamp is up to 100ms late and that error lands entirely in <see cref="FirstGop"/> — the
/// smallest bucket — and can also flip a sample between <see cref="ColdStartStartupSplitKind.ThreeWay"/>
/// and <see cref="ColdStartStartupSplitKind.TwoWayLateProgress"/>. And if the session's first FFmpeg
/// process fails and a second one produces the playlist, <see cref="FFmpegInit"/> spans the first
/// process's whole lifetime plus the retry while still being labelled as one process's init. And the
/// stale-playlist guard below is best-effort rather than a proof: if the folder wipe failed, whether
/// the stale playlist is observed before or after the launch milestone is a scheduling race, so an
/// unlucky sample could still slip through as an implausibly fast one (most often
/// <see cref="ColdStartStartupSplitKind.TwoWay"/>, since FFmpeg has usually not reported progress
/// that early).
/// </para>
/// </summary>
public readonly record struct ColdStartStartupSplit(
TimeSpan Prep,
TimeSpan FFmpegInit,
TimeSpan FirstGop,
ColdStartStartupSplitKind Kind)
{
public static readonly ColdStartStartupSplit Unavailable =
new(TimeSpan.Zero, TimeSpan.Zero, TimeSpan.Zero, ColdStartStartupSplitKind.Unavailable);
/// <summary>
/// Builds the split from four <see cref="Stopwatch.GetTimestamp"/> milestones; <c>0</c> means the
/// milestone never happened. Milestones are recorded on different threads (the session worker
/// records the launch and progress ones; the request thread observes the playlist), so ordering is
/// validated rather than assumed: any out-of-order or missing milestone degrades the result to a
/// coarser <see cref="ColdStartStartupSplitKind"/> instead of producing a negative or invented bucket.
/// </summary>
public static ColdStartStartupSplit FromTimestamps(
long runStarted,
long processLaunched,
long firstProgress,
long playlistExists)
{
if (runStarted <= 0 || processLaunched <= 0 || playlistExists <= 0)
{
return Unavailable;
}
if (processLaunched > playlistExists)
{
// the playlist was observed before FFmpeg was even launched, so it is a stale file: the
// handler wipes the transcode folder before starting the session, but that wipe swallows
// its failures into a warning (LocalFileSystem.EmptyFolder) and continues.
// Every bucket would be meaningless; report nothing rather than a plausible-looking sample
return Unavailable;
}
// the worker's Run entry strictly precedes every later milestone; clamp anyway so a clock
// oddity can never surface as a negative duration in telemetry
TimeSpan prep = Elapsed(runStarted, processLaunched);
if (firstProgress <= 0 || firstProgress < processLaunched)
{
// FFmpeg reported no usable progress before the playlist appeared: fall back to the two-way
// split #472 explicitly accepts, rather than inventing a boundary that was never observed
return new ColdStartStartupSplit(
prep,
Elapsed(processLaunched, playlistExists),
TimeSpan.Zero,
ColdStartStartupSplitKind.TwoWay);
}
if (firstProgress > playlistExists)
{
return new ColdStartStartupSplit(
prep,
Elapsed(processLaunched, playlistExists),
TimeSpan.Zero,
ColdStartStartupSplitKind.TwoWayLateProgress);
}
return new ColdStartStartupSplit(
prep,
Elapsed(processLaunched, firstProgress),
Elapsed(firstProgress, playlistExists),
ColdStartStartupSplitKind.ThreeWay);
}
private static TimeSpan Elapsed(long from, long to)
{
TimeSpan elapsed = Stopwatch.GetElapsedTime(from, to);
return elapsed < TimeSpan.Zero ? TimeSpan.Zero : elapsed;
}
}
@@ -1,4 +1,4 @@
using System.Collections.Immutable;
using System.Collections.Immutable;
using System.Text;
using CliWrap;
using CliWrap.Buffered;
@@ -174,7 +174,7 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
foreach (Subtitle subtitle in maybeSubtitle)
{
if (subtitle.SubtitleKind == SubtitleKind.Sidecar || subtitle is
{ SubtitleKind: SubtitleKind.Embedded, IsImage: false, IsExtracted: true })
{ SubtitleKind: SubtitleKind.Embedded, IsImage: false, IsExtracted: true })
{
// proxy to avoid dealing with escaping
subtitle.Path = $"http://localhost:{Settings.StreamingPort}/media/subtitle/{subtitle.Id}";
@@ -287,20 +287,20 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
outputFormat = OutputFormatKind.Hls;
break;
case StreamingMode.HttpLiveStreamingDirect:
{
// use mpeg-ts by default
outputFormat = OutputFormatKind.MpegTs;
// override with setting if applicable
Option<OutputFormatKind> maybeOutputFormat = await _configElementRepository
.GetValue<OutputFormatKind>(ConfigElementKey.FFmpegHlsDirectOutputFormat, cancellationToken);
foreach (OutputFormatKind of in maybeOutputFormat)
{
outputFormat = of;
}
// use mpeg-ts by default
outputFormat = OutputFormatKind.MpegTs;
break;
}
// override with setting if applicable
Option<OutputFormatKind> maybeOutputFormat = await _configElementRepository
.GetValue<OutputFormatKind>(ConfigElementKey.FFmpegHlsDirectOutputFormat, cancellationToken);
foreach (OutputFormatKind of in maybeOutputFormat)
{
outputFormat = of;
}
break;
}
}
Option<string> subtitleLanguage = Option<string>.None;
@@ -445,7 +445,7 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
Option<string> hlsInitTemplate = outputFormat switch
{
OutputFormatKind.HlsMp4 => $"{nowSeconds}_init.mp4",
_ => Option<string>.None
_ => Option<string>.None
};
Option<string> hlsSegmentOptions = Option<string>.None;
@@ -587,7 +587,8 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
videoVersion.MediaVersion is BackgroundImageMediaVersion { IsSongWithProgress: true },
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel);
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
channel.FFmpegProfile.QsvPreferNativeDecoder != false);
_logger.LogDebug("FFmpeg desired state {FrameState}", desiredState);
@@ -6,6 +6,7 @@ namespace ErsatzTV.Core.FFmpeg;
/// FFmpeg process spawn + probe + libass/encoder init + first GOP, since the wait begins right
/// after the fire-and-forget worker is launched); <see cref="SegmentFill"/> is Phase B (playlist
/// exists -&gt; the requested number of segments are present, or the 8s deadline).
/// <see cref="StartupSplit"/> breaks Phase A down further (#472).
/// </summary>
public readonly record struct PlaylistSegmentsResult(
TimeSpan ProcessStartup,
@@ -13,4 +14,5 @@ public readonly record struct PlaylistSegmentsResult(
int SegmentsReached,
int InitialSegmentCount,
bool DeadlineExpired,
ColdStartFeatures Features);
ColdStartFeatures Features,
ColdStartStartupSplit StartupSplit);
+2 -2
View File
@@ -1,7 +1,7 @@
namespace ErsatzTV.Core.Health;
namespace ErsatzTV.Core.Health;
public interface IHealthCheckService
{
Task<List<HealthCheckResult>> PerformHealthChecks(CancellationToken cancellationToken);
Task<List<HealthCheckResult>> PerformHealthChecks(bool forceRefresh, CancellationToken cancellationToken);
HealthCheckSummary GetHealthCheckSummary();
}
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
namespace ErsatzTV.Core.Interfaces.Repositories;
@@ -23,6 +23,17 @@ public interface ILibraryRepository
Task SetEtag(LibraryPath libraryPath, Option<LibraryFolder> knownFolder, string path, string etag);
Task CleanEtagsForLibraryPath(LibraryPath libraryPath);
Task<Option<int>> GetParentFolderId(LibraryPath libraryPath, string folder, CancellationToken cancellationToken);
/// <summary>
/// Returns the <see cref="LibraryFolder" /> at <paramref name="folder" /> under
/// <paramref name="libraryPath" />, creating it if it does not yet exist.
/// </summary>
/// <remarks>
/// The existing folder is looked up from the database by <c>(LibraryPathId, Path)</c>. Callers do
/// <b>not</b> need to eager-load <see cref="LibraryPath.LibraryFolders" /> — the remote (Jellyfin)
/// sync path never does, and relying on that navigation collection here previously NRE'd every
/// Jellyfin music-video scan (ersatztv#488).
/// </remarks>
Task<LibraryFolder> GetOrAddFolder(LibraryPath libraryPath, Option<int> maybeParentFolder, string folder);
Task UpdateLibraryFolderId(MediaFile mediaFile, int libraryFolderId);
Task UpdatePath(LibraryPath libraryPath, string normalizedLibraryPath);
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Metadata;
namespace ErsatzTV.Core.Interfaces.Repositories;
@@ -47,6 +47,16 @@ public interface IMediaServerTelevisionRepository<in TLibrary, TShow, TSeason, T
TLibrary library,
List<string> episodeItemIds,
CancellationToken cancellationToken);
// Cascade helpers (#476): when a parent is swept to FileNotFound because it is gone from the media
// server, the per-parent loop never visits it, so its descendants are never reconciled. These flag
// the descendants by parent MediaItem.Id (Season.ShowId / Episode.SeasonId are on the base tables).
Task<List<int>> FlagFileNotFoundSeasonsForShows(
List<int> showIds,
CancellationToken cancellationToken);
Task<List<int>> FlagFileNotFoundEpisodesForSeasons(
List<int> seasonIds,
CancellationToken cancellationToken);
Task<Option<int>> FlagUnavailable(TLibrary library, TEpisode episode, CancellationToken cancellationToken);
Task<Option<int>> FlagRemoteOnly(TLibrary library, TEpisode episode, CancellationToken cancellationToken);
}
@@ -0,0 +1,28 @@
namespace ErsatzTV.Core.Interfaces.Streaming;
/// <summary>
/// Checks whether a media-server remote-stream URL still resolves to playable media.
/// </summary>
public interface IRemoteStreamProber
{
/// <summary>
/// Probes <paramref name="url" />, following redirects as ffmpeg would.
/// </summary>
/// <returns>
/// <c>false</c> only when the media server itself reported the media gone — i.e. a 404 that
/// arrived <em>after</em> ErsatzTV's own <c>/media/{provider}/...</c> endpoint redirected.
/// Every other outcome returns <c>true</c> (fail-open), including an un-redirected 404: that
/// one came from ErsatzTV's own endpoint, which also 404s when the media source is
/// unconfigured or momentarily missing, and honouring it would blank every item on that
/// source. Timeouts, transport failures and all other status codes likewise return
/// <c>true</c>, so a probe that cannot answer never prevents a tune that would have worked.
/// </returns>
/// <exception cref="OperationCanceledException">
/// May propagate when <paramref name="cancellationToken" /> is cancelled while the probe is
/// in flight. Caller cancellation is a genuine signal (shutdown / client disconnect), not a
/// probe failure, so it is not absorbed by the fail-open behaviour above. Cancelling after
/// the probe has already completed returns normally. The prober's own internal timeout does
/// <em>not</em> throw — it fails open.
/// </exception>
Task<bool> IsAvailable(string url, CancellationToken cancellationToken);
}
@@ -35,4 +35,20 @@ public class QsvHardwareAccelerationOptionTests
"-filter_hw_device", "hw"
]);
}
[Test]
public void GlobalOptions_WithHardwareDecode_AndPreferNative_ShouldUseVaapiDecodeToSoftware()
{
var option = new QsvHardwareAccelerationOption("/dev/dri/renderD128", FFmpegCapability.Hardware, preferNativeDecoder: true);
option.GlobalOptions.ShouldBe(
[
"-hwaccel", "vaapi",
"-init_hw_device", "vaapi=va:/dev/dri/renderD128",
"-init_hw_device", "qsv=hw@va",
"-filter_hw_device", "hw"
]);
// must NOT keep frames on the GPU as VA-API surfaces
option.GlobalOptions.ShouldNotContain("-hwaccel_output_format");
}
}
@@ -0,0 +1,236 @@
using System;
using System.Collections.Generic;
using ErsatzTV.FFmpeg.Capabilities;
using ErsatzTV.FFmpeg.Format;
using ErsatzTV.FFmpeg.OutputFormat;
using ErsatzTV.FFmpeg.Pipeline;
using ErsatzTV.FFmpeg.Preset;
using ErsatzTV.FFmpeg.State;
using LanguageExt;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using static LanguageExt.Prelude;
namespace ErsatzTV.FFmpeg.Tests.Pipeline;
[TestFixture]
public class QsvPipelineBuilderTests
{
private readonly ILogger _logger = Substitute.For<ILogger>();
[Test]
public void Qsv_PreferNativeDecoder_Should_Decode_Via_Vaapi_To_Software_Then_Qsv_Encode()
{
string command = BuildAndPrint(preferNativeDecoder: true);
// VA-API decode, frames downloaded to software (NO hwaccel_output_format)
command.ShouldContain("-hwaccel vaapi");
command.ShouldNotContain("-hwaccel_output_format");
command.ShouldNotContain("-hwaccel qsv");
// no QSV *decoder* input option (decoder input options sit directly before "-readrate"/"-i";
// "-c:v h264_qsv -" alone would also match the encoder's "-c:v h264_qsv -low_power ..." output option)
command.ShouldNotContain("-c:v h264_qsv -readrate");
// derived-device chain retained for the QSV encoder
command.ShouldContain("-init_hw_device vaapi=va:/dev/dri/renderD128");
command.ShouldContain("-init_hw_device qsv=hw@va");
// software frames re-uploaded before QSV filters/encoder (proves NO bare vpp_qsv on VA-API frames)
command.ShouldContain("hwupload=extra_hw_frames");
// QSV encoder still used
command.ShouldContain("h264_qsv");
}
[Test]
public void Qsv_Default_Should_Decode_And_Encode_With_Qsv()
{
string command = BuildAndPrint(preferNativeDecoder: false);
command.ShouldContain("-hwaccel qsv");
command.ShouldContain("-hwaccel_output_format qsv");
command.ShouldContain("h264_qsv");
command.ShouldNotContain("-hwaccel vaapi");
}
[Test]
public void Qsv_PreferNativeDecoder_Interlaced_Should_Hwupload_Before_Deinterlace_Qsv()
{
(VideoInputFile videoInputFile, AudioInputFile audioInputFile, FFmpegState ffmpegState, FrameState desiredState) =
BuildQsvH264Pipeline(preferNativeDecoder: true, scanKind: ScanKind.Interlaced, deinterlace: true);
var builder = new QsvPipelineBuilder(
new DefaultFFmpegCapabilities(),
new DefaultHardwareCapabilities(),
HardwareAccelerationMode.Qsv,
videoInputFile,
audioInputFile,
None,
None,
None,
Option<GraphicsEngineInput>.None,
"",
"",
_logger);
FFmpegPipeline result = builder.Build(ffmpegState, desiredState);
string command = PrintCommand(videoInputFile, audioInputFile, None, None, None, result);
// VA-API decode, software frames
command.ShouldContain("-hwaccel vaapi");
command.ShouldNotContain("-hwaccel_output_format");
// software frames re-uploaded BEFORE deinterlace_qsv (never a bare deinterlace_qsv on VA-API frames)
command.ShouldContain("hwupload=extra_hw_frames");
command.ShouldContain("hwupload=extra_hw_frames=64,deinterlace_qsv");
// exactly one deinterlace_qsv, and (assertion above) it is preceded by hwupload — so
// there is no second, bare deinterlace_qsv running on VA-API frames
(command.Split("deinterlace_qsv").Length - 1).ShouldBe(1);
command.ShouldContain("h264_qsv");
}
private string BuildAndPrint(bool preferNativeDecoder)
{
(VideoInputFile videoInputFile, AudioInputFile audioInputFile, FFmpegState ffmpegState, FrameState desiredState) =
BuildQsvH264Pipeline(preferNativeDecoder, ScanKind.Progressive, false);
var builder = new QsvPipelineBuilder(
new DefaultFFmpegCapabilities(),
new DefaultHardwareCapabilities(),
HardwareAccelerationMode.Qsv,
videoInputFile,
audioInputFile,
None,
None,
None,
Option<GraphicsEngineInput>.None,
"",
"",
_logger);
FFmpegPipeline result = builder.Build(ffmpegState, desiredState);
return PrintCommand(videoInputFile, audioInputFile, None, None, None, result);
}
private static (VideoInputFile, AudioInputFile, FFmpegState, FrameState) BuildQsvH264Pipeline(
bool preferNativeDecoder,
ScanKind scanKind,
bool deinterlace)
{
var videoInputFile = new VideoInputFile(
"/tmp/whatever.mkv",
new List<VideoStream>
{
new(
0,
VideoFormat.H264,
VideoProfile.Main,
new PixelFormatYuv420P(),
ColorParams.Default,
new FrameSize(1920, 1080),
"1:1",
"16:9",
FrameRate.DefaultFrameRate,
false,
scanKind)
});
var audioInputFile = new AudioInputFile(
"/tmp/whatever.mkv",
new List<AudioStream> { new(1, AudioFormat.Aac, 2) },
new AudioState(
AudioFormat.Aac,
2,
320,
640,
48,
false,
AudioFilter.None,
Option<double>.None));
var desiredState = new FrameState(
true,
false,
VideoFormat.H264,
VideoProfile.Main,
VideoPreset.Unset,
false,
new PixelFormatYuv420P(),
new FrameSize(1280, 720),
new FrameSize(1280, 720),
Option<FrameSize>.None,
FFmpegFilterMode.Software,
false,
Option<FrameRate>.None,
2000,
4000,
90_000,
false,
deinterlace);
var ffmpegState = new FFmpegState(
false,
HardwareAccelerationMode.Qsv,
HardwareAccelerationMode.Qsv,
Option<string>.None,
"/dev/dri/renderD128",
Option<TimeSpan>.None,
Option<TimeSpan>.None,
false,
Option<string>.None,
Option<string>.None,
Option<string>.None,
Option<string>.None,
Option<string>.None,
OutputFormatKind.MpegTs,
Option<string>.None,
Option<string>.None,
Option<string>.None,
Option<string>.None,
TimeSpan.Zero,
Option<int>.None,
Option<int>.None,
false,
false,
"linear",
false,
preferNativeDecoder);
return (videoInputFile, audioInputFile, ffmpegState, desiredState);
}
private static string PrintCommand(
Option<VideoInputFile> videoInputFile,
Option<AudioInputFile> audioInputFile,
Option<WatermarkInputFile> watermarkInputFile,
Option<ConcatInputFile> concatInputFile,
Option<GraphicsEngineInput> graphicsEngineInput,
FFmpegPipeline pipeline)
{
IList<string> arguments = CommandGenerator.GenerateArguments(
videoInputFile,
audioInputFile,
watermarkInputFile,
concatInputFile,
graphicsEngineInput,
pipeline.PipelineSteps,
pipeline.IsIntelVaapiOrQsv);
var command = string.Join(" ", arguments);
Console.WriteLine($"Generated command: ffmpeg {string.Join(" ", arguments)}");
return command;
}
public class DefaultFFmpegCapabilities() : FFmpegCapabilities(
string.Empty,
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>());
}
@@ -0,0 +1,19 @@
using ErsatzTV.FFmpeg.Format;
namespace ErsatzTV.FFmpeg.Decoder;
// VA-API-accelerated decode that downloads frames to system memory (no
// -hwaccel_output_format). Pairs with `-hwaccel vaapi` from
// QsvHardwareAccelerationOption on the "prefer native decoder" QSV path: the
// error-tolerant VA-API decoder feeds software frames into the QSV builder's
// format=nv12,hwupload,vpp_qsv branch, which re-uploads for the QSV encoder.
public class DecoderVaapiToSoftware : DecoderBase
{
protected override FrameDataLocation OutputFrameDataLocation => FrameDataLocation.Software;
public override string Name => "implicit_vaapi";
// no -c:v (implicit decoder; `-hwaccel vaapi` drives VA-API) and no
// -hwaccel_output_format (frames download to software)
public override string[] InputOptions(InputFile inputFile) => [];
}
+3 -2
View File
@@ -1,4 +1,4 @@
using ErsatzTV.FFmpeg.OutputFormat;
using ErsatzTV.FFmpeg.OutputFormat;
namespace ErsatzTV.FFmpeg;
@@ -27,7 +27,8 @@ public record FFmpegState(
bool IsSongWithProgress,
bool IsHdrTonemap,
string TonemapAlgorithm,
bool IsTroubleshooting)
bool IsTroubleshooting,
bool QsvPreferNativeDecoder = false)
{
public int QsvExtraHardwareFrames => MaybeQsvExtraHardwareFrames.IfNone(64);
@@ -1,9 +1,12 @@
using ErsatzTV.FFmpeg.Capabilities;
using ErsatzTV.FFmpeg.Capabilities;
using ErsatzTV.FFmpeg.Format;
namespace ErsatzTV.FFmpeg.GlobalOption.HardwareAcceleration;
public class QsvHardwareAccelerationOption(Option<string> device, FFmpegCapability decodeCapability) : GlobalOption
public class QsvHardwareAccelerationOption(
Option<string> device,
FFmpegCapability decodeCapability,
bool preferNativeDecoder = false) : GlobalOption
{
// TODO: read this from ffmpeg output
private readonly List<string> _supportedFFmpegFormats = new()
@@ -16,15 +19,18 @@ public class QsvHardwareAccelerationOption(Option<string> device, FFmpegCapabili
{
get
{
var result = new List<string>
{
"-hwaccel", "qsv",
"-hwaccel_output_format", "qsv"
};
var result = new List<string>();
if (decodeCapability is not FFmpegCapability.Hardware)
if (decodeCapability is FFmpegCapability.Hardware)
{
result.Clear();
// native path: decode with the error-tolerant VA-API decoder and let ffmpeg
// download frames to system memory (no -hwaccel_output_format), so the QSV
// filter graph's software->hwupload branch bridges them to the QSV encoder.
// default path: decode (and keep frames) on QSV.
result.AddRange(
preferNativeDecoder
? ["-hwaccel", "vaapi"]
: ["-hwaccel", "qsv", "-hwaccel_output_format", "qsv"]);
}
var deviceConfigured = false;
+16 -5
View File
@@ -51,7 +51,8 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
}
protected override bool IsIntelVaapiOrQsv(FFmpegState ffmpegState) =>
ffmpegState.DecoderHardwareAccelerationMode is HardwareAccelerationMode.Qsv ||
ffmpegState.DecoderHardwareAccelerationMode is HardwareAccelerationMode.Qsv
or HardwareAccelerationMode.Vaapi ||
ffmpegState.EncoderHardwareAccelerationMode is HardwareAccelerationMode.Qsv;
protected override FFmpegState SetAccelState(
@@ -102,13 +103,22 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
// give a bogus value so no cuda devices are visible to ffmpeg
pipelineSteps.Add(new CudaVisibleDevicesVariable("999"));
pipelineSteps.Add(new QsvHardwareAccelerationOption(ffmpegState.VaapiDevice, decodeCapability));
// native (VA-API) decode is a Linux-only path: ffmpeg has no vaapi hwaccel on
// Windows, where QSV capabilities are also over-reported, so keep QSV decode there
bool preferNativeDecode = ffmpegState.QsvPreferNativeDecoder != false && !OperatingSystem.IsWindows();
pipelineSteps.Add(new QsvHardwareAccelerationOption(
ffmpegState.VaapiDevice,
decodeCapability,
preferNativeDecode));
// disable hw accel if decoder/encoder isn't supported
return ffmpegState with
{
DecoderHardwareAccelerationMode = decodeCapability == FFmpegCapability.Hardware
? HardwareAccelerationMode.Qsv
? preferNativeDecode
? HardwareAccelerationMode.Vaapi
: HardwareAccelerationMode.Qsv
: HardwareAccelerationMode.None,
EncoderHardwareAccelerationMode = encodeCapability == FFmpegCapability.Hardware
? HardwareAccelerationMode.Qsv
@@ -130,6 +140,7 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
(HardwareAccelerationMode.Qsv, VideoFormat.Vc1) => new DecoderVc1Qsv(),
(HardwareAccelerationMode.Qsv, VideoFormat.Vp9) => new DecoderVp9Qsv(),
(HardwareAccelerationMode.Qsv, VideoFormat.Av1) => new DecoderAv1Qsv(),
(HardwareAccelerationMode.Vaapi, _) => new DecoderVaapiToSoftware(),
_ => GetSoftwareDecoder(videoStream)
};
@@ -202,7 +213,7 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
// need to download for any sort of overlay (and always for setpts)
if (currentState.FrameDataLocation == FrameDataLocation.Hardware) //&&
//(context.HasSubtitleOverlay || context.HasWatermark || context.HasGraphicsEngine))
//(context.HasSubtitleOverlay || context.HasWatermark || context.HasGraphicsEngine))
{
var hardwareDownload = new HardwareDownloadFilter(currentState);
currentState = hardwareDownload.NextState(currentState);
@@ -629,7 +640,7 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
DecoderHardwareAccelerationMode: HardwareAccelerationMode.None,
EncoderHardwareAccelerationMode: HardwareAccelerationMode.None
} && context is
{ HasGraphicsEngine: false, HasWatermark: false, HasSubtitleOverlay: false, ShouldDeinterlace: false };
{ HasGraphicsEngine: false, HasWatermark: false, HasSubtitleOverlay: false, ShouldDeinterlace: false };
// auto_scale filter seems to muck up 10-bit software decode => hardware scale, so use software scale in that case
useSoftwareFilter = useSoftwareFilter ||
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,23 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.MySql.Migrations
{
/// <inheritdoc />
public partial class NullOutNeverScannedLastScan : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql("UPDATE Library SET LastScan = NULL WHERE LastScan IS NOT NULL AND LastScan < '2000-01-01'");
migrationBuilder.Sql("UPDATE LibraryPath SET LastScan = NULL WHERE LastScan IS NOT NULL AND LastScan < '2000-01-01'");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
// irreversible data migration; there is no way to recover the original sentinel values
}
}
}
@@ -0,0 +1,29 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.MySql.Migrations
{
/// <inheritdoc />
public partial class Add_FFmpegProfile_QsvPreferNativeDecoder : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<bool>(
name: "QsvPreferNativeDecoder",
table: "FFmpegProfile",
type: "tinyint(1)",
nullable: true,
defaultValue: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "QsvPreferNativeDecoder",
table: "FFmpegProfile");
}
}
}
@@ -906,6 +906,11 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
b.Property<int?>("QsvExtraHardwareFrames")
.HasColumnType("int");
b.Property<bool?>("QsvPreferNativeDecoder")
.ValueGeneratedOnAdd()
.HasColumnType("tinyint(1)")
.HasDefaultValue(true);
b.Property<int>("ResolutionId")
.HasColumnType("int");
@@ -0,0 +1,23 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
{
/// <inheritdoc />
public partial class NullOutNeverScannedLastScan : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql("UPDATE Library SET LastScan = NULL WHERE LastScan IS NOT NULL AND LastScan < '2000-01-01'");
migrationBuilder.Sql("UPDATE LibraryPath SET LastScan = NULL WHERE LastScan IS NOT NULL AND LastScan < '2000-01-01'");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
// irreversible data migration; there is no way to recover the original sentinel values
}
}
}
@@ -0,0 +1,29 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
{
/// <inheritdoc />
public partial class Add_FFmpegProfile_QsvPreferNativeDecoder : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<bool>(
name: "QsvPreferNativeDecoder",
table: "FFmpegProfile",
type: "INTEGER",
nullable: true,
defaultValue: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "QsvPreferNativeDecoder",
table: "FFmpegProfile");
}
}
}
@@ -873,6 +873,11 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
b.Property<int?>("QsvExtraHardwareFrames")
.HasColumnType("INTEGER");
b.Property<bool?>("QsvPreferNativeDecoder")
.ValueGeneratedOnAdd()
.HasColumnType("INTEGER")
.HasDefaultValue(true);
b.Property<int>("ResolutionId")
.HasColumnType("INTEGER");
@@ -53,6 +53,141 @@ public class JellyfinApiClientTests
libraries[0].ShouldSyncItems.ShouldBeFalse();
libraries[0].Paths.Single().Path.ShouldBe("jellyfin://library-1");
}
[Test]
public async Task Should_Project_Mixed_Libraries()
{
const string response = """
[
{
"Name": "Music Videos",
"CollectionType": "mixed",
"ItemId": "library-9",
"LibraryOptions": {
"PathInfos": []
}
}
]
""";
var client = new JellyfinApiClient(
new MemoryCache(new MemoryCacheOptions()),
Substitute.For<IJellyfinPathReplacementService>(),
Substitute.For<IFallbackMetadataProvider>(),
new SingleResponseHttpClientFactory(response),
Substitute.For<ILogger<JellyfinApiClient>>());
Either<BaseError, List<JellyfinLibrary>> result =
await client.GetLibraries("http://jellyfin.example", "MediaBrowser Token=abc");
result.IsRight.ShouldBeTrue();
List<JellyfinLibrary> libraries = result.RightToSeq().Single();
libraries.Count.ShouldBe(1);
libraries[0].Name.ShouldBe("Music Videos");
libraries[0].ItemId.ShouldBe("library-9");
libraries[0].MediaKind.ShouldBe(LibraryMediaKind.Mixed);
libraries[0].ShouldSyncItems.ShouldBeFalse();
libraries[0].Paths.Single().Path.ShouldBe("jellyfin://library-9");
}
[Test]
public async Task Should_Project_Libraries_With_No_CollectionType_As_Mixed()
{
const string response = """
[
{
"Name": "Standup",
"ItemId": "library-10",
"LibraryOptions": {
"PathInfos": []
}
}
]
""";
var client = new JellyfinApiClient(
new MemoryCache(new MemoryCacheOptions()),
Substitute.For<IJellyfinPathReplacementService>(),
Substitute.For<IFallbackMetadataProvider>(),
new SingleResponseHttpClientFactory(response),
Substitute.For<ILogger<JellyfinApiClient>>());
Either<BaseError, List<JellyfinLibrary>> result =
await client.GetLibraries("http://jellyfin.example", "MediaBrowser Token=abc");
result.IsRight.ShouldBeTrue();
List<JellyfinLibrary> libraries = result.RightToSeq().Single();
libraries.Count.ShouldBe(1);
libraries[0].Name.ShouldBe("Standup");
libraries[0].MediaKind.ShouldBe(LibraryMediaKind.Mixed);
}
// Jellyfin serializes "no content type" as absent, empty or whitespace depending on version;
// all three mean mixed content, so all three must project identically.
[TestCase("\"CollectionType\": \"\",")]
[TestCase("\"CollectionType\": \" \",")]
public async Task Should_Project_Libraries_With_Blank_CollectionType_As_Mixed(string collectionTypeLine)
{
string response = $$"""
[
{
"Name": "Standup",
{{collectionTypeLine}}
"ItemId": "library-12",
"LibraryOptions": {
"PathInfos": []
}
}
]
""";
var client = new JellyfinApiClient(
new MemoryCache(new MemoryCacheOptions()),
Substitute.For<IJellyfinPathReplacementService>(),
Substitute.For<IFallbackMetadataProvider>(),
new SingleResponseHttpClientFactory(response),
Substitute.For<ILogger<JellyfinApiClient>>());
Either<BaseError, List<JellyfinLibrary>> result =
await client.GetLibraries("http://jellyfin.example", "MediaBrowser Token=abc");
result.IsRight.ShouldBeTrue();
List<JellyfinLibrary> libraries = result.RightToSeq().Single();
libraries.Count.ShouldBe(1);
libraries[0].MediaKind.ShouldBe(LibraryMediaKind.Mixed);
}
// Guard: mixed must not become a catch-all. Jellyfin "music" (audio) libraries have no
// supported scanner, so they must keep falling through to None.
[Test]
public async Task Should_Not_Project_Unknown_CollectionTypes()
{
const string response = """
[
{
"Name": "Explo Discovery",
"CollectionType": "music",
"ItemId": "library-11",
"LibraryOptions": {
"PathInfos": []
}
}
]
""";
var client = new JellyfinApiClient(
new MemoryCache(new MemoryCacheOptions()),
Substitute.For<IJellyfinPathReplacementService>(),
Substitute.For<IFallbackMetadataProvider>(),
new SingleResponseHttpClientFactory(response),
Substitute.For<ILogger<JellyfinApiClient>>());
Either<BaseError, List<JellyfinLibrary>> result =
await client.GetLibraries("http://jellyfin.example", "MediaBrowser Token=abc");
result.IsRight.ShouldBeTrue();
result.RightToSeq().Single().ShouldBeEmpty();
}
}
private sealed class SingleResponseHttpClientFactory(string response) : IHttpClientFactory
@@ -0,0 +1,247 @@
using System.Net;
using ErsatzTV.Infrastructure.Streaming;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Infrastructure.Tests.Streaming;
[TestFixture]
public class HttpRemoteStreamProberTests
{
private const string Url = "http://localhost:8409/media/jellyfin/abc123";
[Test]
public async Task Should_Report_Unavailable_On_404_From_The_Media_Server()
{
// a media-server 404 arrives after our /media/... endpoint redirected, so the response's
// final request uri is the media server's, not the probe url
HttpRemoteStreamProber prober = ProberReturning(
HttpStatusCode.NotFound,
finalUri: "http://jellyfin:8096/Videos/abc123/stream?static=true");
bool result = await prober.IsAvailable(Url, CancellationToken.None);
result.ShouldBeFalse();
}
// ersatztv#473 review finding: our OWN /media/{provider}/... endpoint 404s when the media source
// is unconfigured or momentarily missing. Failing closed there would blank every item on that
// source, which is exactly what the fail-open contract exists to prevent.
[Test]
public async Task Should_Fail_Open_On_404_That_Was_Not_Redirected()
{
HttpRemoteStreamProber prober = ProberReturning(HttpStatusCode.NotFound, finalUri: Url);
bool result = await prober.IsAvailable(Url, CancellationToken.None);
result.ShouldBeTrue();
}
// a plex key can contain spaces/unicode; pin that an un-redirected 404 on such a url still fails
// OPEN. (This passes against a naive string comparison too - Uri.ToString() unescapes - so it
// guards the behaviour, not the implementation choice.)
[Test]
public async Task Should_Fail_Open_On_404_For_An_Unredirected_Url_Needing_Escaping()
{
const string plexUrl = "http://localhost:8409/media/plex/1/library/parts/1/a file.mkv";
HttpRemoteStreamProber prober = ProberReturning(HttpStatusCode.NotFound, finalUri: plexUrl);
bool result = await prober.IsAvailable(plexUrl, CancellationToken.None);
result.ShouldBeTrue();
}
[TestCase(HttpStatusCode.OK)]
[TestCase(HttpStatusCode.PartialContent)]
[TestCase(HttpStatusCode.NoContent)]
public async Task Should_Report_Available_On_Success(HttpStatusCode statusCode)
{
HttpRemoteStreamProber prober = ProberReturning(statusCode);
bool result = await prober.IsAvailable(Url, CancellationToken.None);
result.ShouldBeTrue();
}
// the fail-open contract: a probe that cannot answer must never block a tune that would
// otherwise have worked. these cases exist so a future refactor can't silently invert it.
[TestCase(HttpStatusCode.InternalServerError)]
[TestCase(HttpStatusCode.BadGateway)]
[TestCase(HttpStatusCode.Unauthorized)]
[TestCase(HttpStatusCode.Forbidden)]
public async Task Should_Fail_Open_On_Other_Status_Codes(HttpStatusCode statusCode)
{
HttpRemoteStreamProber prober = ProberReturning(statusCode);
bool result = await prober.IsAvailable(Url, CancellationToken.None);
result.ShouldBeTrue();
}
// a server that ignores `Range: bytes=0-0` answers 200 with the WHOLE FILE. The probe must not
// read it -- buffering a video on the streaming hot path would be far worse than the aborted
// socket the drain was added to avoid. (Review finding against the first fix commit.)
[Test]
public async Task Should_Not_Read_The_Body_When_The_Server_Ignores_The_Range_Request()
{
var body = new TrackingStream(64 * 1024 * 1024);
var response = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StreamContent(body) };
var prober = new HttpRemoteStreamProber(
new StubHttpClientFactory(new FixedResponseHttpMessageHandler(response)),
Substitute.For<ILogger<HttpRemoteStreamProber>>());
bool result = await prober.IsAvailable(Url, CancellationToken.None);
result.ShouldBeTrue();
body.BytesRead.ShouldBe(0);
}
// the counterpart: when the server DID honour the range, the one byte is read so the connection
// goes back to the pool rather than being aborted
[Test]
public async Task Should_Drain_The_Single_Byte_When_The_Server_Honours_The_Range_Request()
{
var body = new TrackingStream(1);
var response = new HttpResponseMessage(HttpStatusCode.PartialContent)
{
Content = new StreamContent(body)
};
var prober = new HttpRemoteStreamProber(
new StubHttpClientFactory(new FixedResponseHttpMessageHandler(response)),
Substitute.For<ILogger<HttpRemoteStreamProber>>());
bool result = await prober.IsAvailable(Url, CancellationToken.None);
result.ShouldBeTrue();
body.BytesRead.ShouldBe(1);
}
[Test]
public async Task Should_Fail_Open_On_Transport_Failure()
{
var prober = new HttpRemoteStreamProber(
new StubHttpClientFactory(new ThrowingHttpMessageHandler(new HttpRequestException("no route to host"))),
Substitute.For<ILogger<HttpRemoteStreamProber>>());
bool result = await prober.IsAvailable(Url, CancellationToken.None);
result.ShouldBeTrue();
}
[Test]
public async Task Should_Fail_Open_On_Timeout()
{
var prober = new HttpRemoteStreamProber(
new StubHttpClientFactory(new ThrowingHttpMessageHandler(new TaskCanceledException("timed out"))),
Substitute.For<ILogger<HttpRemoteStreamProber>>());
bool result = await prober.IsAvailable(Url, CancellationToken.None);
result.ShouldBeTrue();
}
// caller cancellation (shutdown / client disconnect) is a genuine signal, NOT a probe failure --
// swallowing it would let the handler go on building an ffmpeg command on a dead token.
[Test]
public async Task Should_Propagate_Caller_Cancellation()
{
HttpRemoteStreamProber prober = ProberReturning(HttpStatusCode.OK);
using var cts = new CancellationTokenSource();
await cts.CancelAsync();
await Should.ThrowAsync<OperationCanceledException>(() => prober.IsAvailable(Url, cts.Token));
}
private static HttpRemoteStreamProber ProberReturning(HttpStatusCode statusCode, string finalUri = null) =>
new(
new StubHttpClientFactory(new StatusCodeHttpMessageHandler(statusCode, finalUri)),
Substitute.For<ILogger<HttpRemoteStreamProber>>());
private sealed class StubHttpClientFactory(HttpMessageHandler handler) : IHttpClientFactory
{
public HttpClient CreateClient(string name) => new(handler, disposeHandler: false);
}
private sealed class StatusCodeHttpMessageHandler(HttpStatusCode statusCode, string finalUri = null)
: HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
// HttpClient rewrites RequestMessage.RequestUri to the final hop when it follows a
// redirect; finalUri lets a test stand in for "the media server answered this".
if (finalUri is not null)
{
request.RequestUri = new Uri(finalUri);
}
return Task.FromResult(new HttpResponseMessage(statusCode) { RequestMessage = request });
}
}
private sealed class FixedResponseHttpMessageHandler(HttpResponseMessage response) : HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
response.RequestMessage = request;
return Task.FromResult(response);
}
}
/// <summary>A readable stream that records how many bytes were actually pulled from it.</summary>
private sealed class TrackingStream(long length) : Stream
{
public int BytesRead { get; private set; }
public override bool CanRead => true;
public override bool CanSeek => false;
public override bool CanWrite => false;
public override long Length => length;
public override long Position
{
get => BytesRead;
set => throw new NotSupportedException();
}
public override int Read(byte[] buffer, int offset, int count)
{
if (BytesRead >= length)
{
return 0;
}
int toRead = (int)Math.Min(count, length - BytesRead);
Array.Clear(buffer, offset, toRead);
BytesRead += toRead;
return toRead;
}
public override void Flush()
{
}
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}
private sealed class ThrowingHttpMessageHandler(Exception exception) : HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken) =>
Task.FromException<HttpResponseMessage>(exception);
}
}
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
@@ -24,5 +24,8 @@ public class FFmpegProfileConfiguration : IEntityTypeConfiguration<FFmpegProfile
builder.Property(p => p.NormalizeColors)
.HasDefaultValue(true);
builder.Property(p => p.QsvPreferNativeDecoder)
.HasDefaultValue(true);
}
}
+70 -7
View File
@@ -1,8 +1,9 @@
using System.Globalization;
using System.Globalization;
using System.Reflection;
using Dapper;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Scheduling;
using ErsatzTV.FFmpeg.State;
using Microsoft.EntityFrameworkCore;
namespace ErsatzTV.Infrastructure.Data;
@@ -131,13 +132,17 @@ public static class DbInitializer
await context.SaveChangesAsync(cancellationToken);
}
await SeedChannelTemplates(context, cancellationToken);
int? channelBugWatermarkId = await SeedChannelBugWatermark(context, cancellationToken);
await SeedChannelTemplates(context, cancellationToken, channelBugWatermarkId);
// TODO: create looping static image that mentions configuring via web
return Unit.Default;
}
private static async Task SeedChannelTemplates(TvContext context, CancellationToken cancellationToken)
private static async Task SeedChannelTemplates(
TvContext context,
CancellationToken cancellationToken,
int? channelBugWatermarkId)
{
if (await context.ChannelTemplates.AnyAsync(t => t.Name == "Standard", cancellationToken) &&
await context.ChannelTemplates.AnyAsync(t => t.Name == "Music videos", cancellationToken))
@@ -162,7 +167,8 @@ public static class DbInitializer
ChannelMusicVideoCreditsMode.None,
ChannelSongVideoMode.Default,
shuffleScheduleItems: false,
randomStartPoint: false),
randomStartPoint: false,
channelBugWatermarkId),
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
}
@@ -177,7 +183,8 @@ public static class DbInitializer
ChannelMusicVideoCreditsMode.GenerateSubtitles,
ChannelSongVideoMode.WithProgress,
shuffleScheduleItems: true,
randomStartPoint: true),
randomStartPoint: true,
channelBugWatermarkId),
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
}
@@ -185,6 +192,60 @@ public static class DbInitializer
await EnsureDefaultChannelTemplateConfig(context, cancellationToken);
}
// A single shared preset is all that's needed: ImageSource.ChannelLogo resolves each channel's
// own logo artwork at render time (WatermarkSelector), so one row makes every channel use its
// own logo as its on-screen bug.
//
// Guarded by a ConfigElement marker rather than by name alone: ChannelWatermark has no IsSystem
// flag, and Initialize runs on every startup, so a name-only guard would resurrect the row
// forever after a deliberate delete. Adopting an existing same-name row (an operator's tuned
// one) also sets the marker — adopt, never overwrite.
private static async Task<int?> SeedChannelBugWatermark(
TvContext context,
CancellationToken cancellationToken)
{
string seededKey = ConfigElementKey.WatermarkChannelBugSeeded.Key;
bool alreadySeeded = await context.ConfigElements
.AnyAsync(c => c.Key == seededKey, cancellationToken);
ChannelWatermark existing = await context.ChannelWatermarks
.FirstOrDefaultAsync(w => w.Name == "Channel Bug", cancellationToken);
if (alreadySeeded)
{
return existing?.Id;
}
if (existing is null)
{
existing = new ChannelWatermark
{
Name = "Channel Bug",
Mode = ChannelWatermarkMode.Permanent,
ImageSource = ChannelWatermarkImageSource.ChannelLogo,
Image = null,
Location = WatermarkLocation.TopLeft,
Size = WatermarkSize.Scaled,
WidthPercent = 5.0,
HorizontalMarginPercent = 1.0,
VerticalMarginPercent = 1.0,
FrequencyMinutes = 0,
DurationSeconds = 0,
Opacity = 80,
PlaceWithinSourceContent = false,
ZIndex = 0
};
await context.ChannelWatermarks.AddAsync(existing, cancellationToken);
}
await context.ConfigElements.AddAsync(
new ConfigElement { Key = seededKey, Value = "true" },
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
return existing.Id;
}
private static async Task<FFmpegProfile> GetDefaultFFmpegProfile(
TvContext context,
CancellationToken cancellationToken)
@@ -237,7 +298,8 @@ public static class DbInitializer
ChannelMusicVideoCreditsMode musicVideoCreditsMode,
ChannelSongVideoMode songVideoMode,
bool shuffleScheduleItems,
bool randomStartPoint) =>
bool randomStartPoint,
int? watermarkId) =>
new()
{
Name = name,
@@ -260,6 +322,7 @@ public static class DbInitializer
IdleBehavior = ChannelIdleBehavior.StopOnDisconnect,
ShuffleScheduleItems = shuffleScheduleItems,
RandomStartPoint = randomStartPoint,
FixedStartTimeBehavior = FixedStartTimeBehavior.Flexible
FixedStartTimeBehavior = FixedStartTimeBehavior.Flexible,
WatermarkId = watermarkId
};
}
@@ -1,4 +1,4 @@
using Dapper;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Emby;
@@ -387,6 +387,53 @@ public class EmbyTelevisionRepository(
return ids;
}
// #476: provider-agnostic — Season.ShowId is on the base Season table, so a cascade from the
// already-scoped show ids needs no provider join.
public async Task<List<int>> FlagFileNotFoundSeasonsForShows(
List<int> showIds,
CancellationToken cancellationToken) =>
await FlagFileNotFoundByParent(
"SELECT Id FROM Season WHERE ShowId IN @ParentIds",
showIds,
cancellationToken);
// #476: Episode.SeasonId is on the base Episode table.
public async Task<List<int>> FlagFileNotFoundEpisodesForSeasons(
List<int> seasonIds,
CancellationToken cancellationToken) =>
await FlagFileNotFoundByParent(
"SELECT Id FROM Episode WHERE SeasonId IN @ParentIds",
seasonIds,
cancellationToken);
private async Task<List<int>> FlagFileNotFoundByParent(
string selectSql,
List<int> parentIds,
CancellationToken cancellationToken)
{
if (parentIds.Count == 0)
{
return [];
}
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
List<int> ids = await dbContext.Connection.QueryAsync<int>(
new CommandDefinition(
selectSql,
parameters: new { ParentIds = parentIds },
cancellationToken: cancellationToken))
.Map(result => result.ToList());
await dbContext.Connection.ExecuteAsync(
new CommandDefinition(
"UPDATE MediaItem SET State = 1 WHERE Id IN @Ids AND State != 1",
parameters: new { Ids = ids },
cancellationToken: cancellationToken));
return ids;
}
public async Task<List<int>> FlagFileNotFoundEpisodes(
EmbyLibrary library,
List<string> episodeItemIds,
@@ -1,4 +1,4 @@
using Dapper;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
@@ -198,7 +198,7 @@ public class JellyfinTelevisionRepository : IJellyfinTelevisionRepository
await UpdateEpisode(dbContext, existing, item, cancellationToken);
result = new MediaItemScanResult<JellyfinEpisode>(existing)
{ IsAdded = false, IsUpdated = true };
{ IsAdded = false, IsUpdated = true };
}
else
{
@@ -421,6 +421,53 @@ public class JellyfinTelevisionRepository : IJellyfinTelevisionRepository
return ids;
}
// #476: provider-agnostic — Season.ShowId is on the base Season table, so a cascade from the
// already-scoped show ids needs no provider join.
public async Task<List<int>> FlagFileNotFoundSeasonsForShows(
List<int> showIds,
CancellationToken cancellationToken) =>
await FlagFileNotFoundByParent(
"SELECT Id FROM Season WHERE ShowId IN @ParentIds",
showIds,
cancellationToken);
// #476: Episode.SeasonId is on the base Episode table.
public async Task<List<int>> FlagFileNotFoundEpisodesForSeasons(
List<int> seasonIds,
CancellationToken cancellationToken) =>
await FlagFileNotFoundByParent(
"SELECT Id FROM Episode WHERE SeasonId IN @ParentIds",
seasonIds,
cancellationToken);
private async Task<List<int>> FlagFileNotFoundByParent(
string selectSql,
List<int> parentIds,
CancellationToken cancellationToken)
{
if (parentIds.Count == 0)
{
return [];
}
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync(cancellationToken);
List<int> ids = await dbContext.Connection.QueryAsync<int>(
new CommandDefinition(
selectSql,
parameters: new { ParentIds = parentIds },
cancellationToken: cancellationToken))
.Map(result => result.ToList());
await dbContext.Connection.ExecuteAsync(
new CommandDefinition(
"UPDATE MediaItem SET State = 1 WHERE Id IN @Ids AND State != 1",
parameters: new { Ids = ids },
cancellationToken: cancellationToken));
return ids;
}
public async Task<List<int>> FlagFileNotFoundEpisodes(
JellyfinLibrary library,
List<string> episodeItemIds,
@@ -1,4 +1,4 @@
using System.IO.Abstractions;
using System.IO.Abstractions;
using Dapper;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Repositories;
@@ -169,11 +169,16 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync();
// load from db or create new folder
LibraryFolder knownFolder = await libraryPath.LibraryFolders
.Filter(f => f.Path == folder && f.LibraryPathId == libraryPath.Id)
.HeadOrNone()
.IfNoneAsync(CreateNewFolder(libraryPath, maybeParentFolder, folder));
// load from db or create new folder. Look the folder up by (LibraryPathId, Path) rather than
// reading libraryPath.LibraryFolders: that navigation collection is only eager-loaded on the
// local scan path (via GetLibrary) and is null on the remote (Jellyfin) sync path, which used
// to NRE every Jellyfin music-video scan here (ersatztv#488). The local scanners already hit
// the db once per folder via GetParentFolderId, so this adds no new query pattern.
LibraryFolder knownFolder = await dbContext.LibraryFolders
.AsNoTracking()
.Filter(f => f.LibraryPathId == libraryPath.Id && f.Path == folder)
.FirstOrDefaultAsync()
?? CreateNewFolder(libraryPath, maybeParentFolder, folder);
// update parent folder if not present
foreach (int parentFolder in maybeParentFolder)
@@ -1,4 +1,4 @@
using System.Globalization;
using System.Globalization;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
@@ -465,6 +465,53 @@ public class PlexTelevisionRepository : IPlexTelevisionRepository
return ids;
}
// #476: provider-agnostic — Season.ShowId is on the base Season table, so a cascade from the
// already-scoped show ids needs no provider join.
public async Task<List<int>> FlagFileNotFoundSeasonsForShows(
List<int> showIds,
CancellationToken cancellationToken) =>
await FlagFileNotFoundByParent(
"SELECT Id FROM Season WHERE ShowId IN @ParentIds",
showIds,
cancellationToken);
// #476: Episode.SeasonId is on the base Episode table.
public async Task<List<int>> FlagFileNotFoundEpisodesForSeasons(
List<int> seasonIds,
CancellationToken cancellationToken) =>
await FlagFileNotFoundByParent(
"SELECT Id FROM Episode WHERE SeasonId IN @ParentIds",
seasonIds,
cancellationToken);
private async Task<List<int>> FlagFileNotFoundByParent(
string selectSql,
List<int> parentIds,
CancellationToken cancellationToken)
{
if (parentIds.Count == 0)
{
return [];
}
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync(cancellationToken);
List<int> ids = await dbContext.Connection.QueryAsync<int>(
new CommandDefinition(
selectSql,
parameters: new { ParentIds = parentIds },
cancellationToken: cancellationToken))
.Map(result => result.ToList());
await dbContext.Connection.ExecuteAsync(
new CommandDefinition(
"UPDATE MediaItem SET State = 1 WHERE Id IN @Ids AND State != 1",
parameters: new { Ids = ids },
cancellationToken: cancellationToken));
return ids;
}
public async Task<List<int>> FlagFileNotFoundEpisodes(
PlexLibrary library,
List<string> episodeItemIds,
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Health;
using ErsatzTV.Core.Health;
using ErsatzTV.Core.Health.Checks;
using MediatR;
using Microsoft.Extensions.Caching.Memory;
@@ -8,7 +8,14 @@ namespace ErsatzTV.Infrastructure.Health;
public class HealthCheckService : IHealthCheckService
{
private const string CacheKey = "healthcheck.summary";
private const string SummaryCacheKey = "healthcheck.summary";
private const string ResultsCacheKey = "healthcheck.results";
// Health checks shell out to ffmpeg/ffprobe (4 of the 14 checks) on every run, so a bare
// GET /api/v1/health spawns ~4 subprocesses per request. Cache the full result list for a
// short window so repeated polls (a status widget, an MCP client, monitoring) reuse it; an
// explicit refresh (forceRefresh) bypasses and repopulates. See docs/decisions.md 2026-07-19 (#431).
private static readonly TimeSpan CacheTtl = TimeSpan.FromSeconds(30);
private readonly List<IHealthCheck> _checks; // ReSharper disable SuggestBaseTypeForParameterInConstructor
private readonly IMemoryCache _memoryCache;
@@ -56,8 +63,13 @@ public class HealthCheckService : IHealthCheckService
];
}
public async Task<List<HealthCheckResult>> PerformHealthChecks(CancellationToken cancellationToken)
public async Task<List<HealthCheckResult>> PerformHealthChecks(bool forceRefresh, CancellationToken cancellationToken)
{
if (!forceRefresh && _memoryCache.TryGetValue(ResultsCacheKey, out List<HealthCheckResult> cached) && cached is not null)
{
return cached;
}
List<HealthCheckResult> result = await _checks.Map(c =>
{
var failedResult = new HealthCheckResult(
@@ -75,7 +87,8 @@ public class HealthCheckService : IHealthCheckService
result.Count(x => x.Status is HealthCheckStatus.Warning),
result.Count(x => x.Status is HealthCheckStatus.Fail));
_memoryCache.Set(CacheKey, summary);
_memoryCache.Set(ResultsCacheKey, result, CacheTtl);
_memoryCache.Set(SummaryCacheKey, summary);
await _mediator.Publish(summary, cancellationToken);
@@ -83,7 +96,7 @@ public class HealthCheckService : IHealthCheckService
}
public HealthCheckSummary GetHealthCheckSummary() =>
_memoryCache.Get<HealthCheckSummary>(CacheKey) ?? new HealthCheckSummary(0, 0);
_memoryCache.Get<HealthCheckSummary>(SummaryCacheKey) ?? new HealthCheckSummary(0, 0);
private HealthCheckResult LogAndReturn(Exception ex, HealthCheckResult failedResult)
{
@@ -1,4 +1,4 @@
using System.Globalization;
using System.Globalization;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Jellyfin;
@@ -435,7 +435,9 @@ public class JellyfinApiClient : IJellyfinApiClient
}
private Option<JellyfinLibrary> Project(JellyfinLibraryResponse response) =>
response.CollectionType?.ToLowerInvariant() switch
// normalize "no content type" to null: Jellyfin serializes a mixed library's collection type
// as absent, empty or whitespace depending on server version, and all three mean the same thing
(string.IsNullOrWhiteSpace(response.CollectionType) ? null : response.CollectionType.ToLowerInvariant()) switch
{
"tvshows" => new JellyfinLibrary
{
@@ -466,6 +468,21 @@ public class JellyfinApiClient : IJellyfinApiClient
},
// TODO: ??? for music libraries
"boxsets" => CacheCollectionLibraryId(response.ItemId),
// A "mixed content" library. Jellyfin reports these as either the literal "mixed" or with
// no collection type at all, depending on server version. Its items are read per type via
// includeItemTypes, so the mix is resolved authoritatively by Jellyfin rather than guessed.
"mixed" or null => new JellyfinLibrary
{
ItemId = response.ItemId,
Name = response.Name,
MediaKind = LibraryMediaKind.Mixed,
ShouldSyncItems = false,
Paths = new List<LibraryPath> { new() { Path = $"jellyfin://{response.ItemId}" } },
PathInfos = GetPathInfos(response)
},
// anything else (notably "music" audio libraries) stays unsupported
_ => None
};
@@ -26,6 +26,7 @@ public class ExternalJsonPlayoutItemProvider : IExternalJsonPlayoutItemProvider
private readonly IPlexPathReplacementService _plexPathReplacementService;
private readonly IPlexSecretStore _plexSecretStore;
private readonly IPlexServerApiClient _plexServerApiClient;
private readonly IRemoteStreamProber _remoteStreamProber;
public ExternalJsonPlayoutItemProvider(
IDbContextFactory<TvContext> dbContextFactory,
@@ -34,6 +35,7 @@ public class ExternalJsonPlayoutItemProvider : IExternalJsonPlayoutItemProvider
IPlexServerApiClient plexServerApiClient,
IPlexSecretStore plexSecretStore,
ILocalStatisticsProvider localStatisticsProvider,
IRemoteStreamProber remoteStreamProber,
ILogger<ExternalJsonPlayoutItemProvider> logger)
{
_dbContextFactory = dbContextFactory;
@@ -42,6 +44,7 @@ public class ExternalJsonPlayoutItemProvider : IExternalJsonPlayoutItemProvider
_plexServerApiClient = plexServerApiClient;
_plexSecretStore = plexSecretStore;
_localStatisticsProvider = localStatisticsProvider;
_remoteStreamProber = remoteStreamProber;
_logger = logger;
}
@@ -217,15 +220,29 @@ public class ExternalJsonPlayoutItemProvider : IExternalJsonPlayoutItemProvider
foreach (PlexServerAuthToken token in maybeToken)
{
var plexUrl =
$"http://localhost:{Settings.StreamingPort}/media/plex/{server.Id}/{program.PlexFile}";
// #480: probe the remote-stream URL before handing it to ffmpeg, exactly as the
// generated-playout path does in
// GetPlayoutItemProcessByChannelNumberHandler.ValidatePlayoutItemPath (#473). Without
// this, an item that is gone from the media server 404s under ffmpeg (exit 8) and the
// same dead item is re-selected for its whole slot. The fail-open contract (only a
// *redirected* 404 fails closed) lives inside IRemoteStreamProber, so this call site
// only owns the decision to probe, not the policy. Probing first also skips the Plex
// metadata round-trip when the item is already gone.
if (!await _remoteStreamProber.IsAvailable(plexUrl, cancellationToken))
{
return new PlayoutItemNotAvailableFromMediaServer(plexUrl);
}
MediaItem mediaItem = program.Type switch
{
"episode" => await GetPlexEpisode(server, connection, token, program),
_ => await GetPlexMovie(server, connection, token, program)
};
return new PlayoutItemWithPath(
GetPlayoutItem(startTime, mediaItem, program),
$"http://localhost:{Settings.StreamingPort}/media/plex/{server.Id}/{program.PlexFile}");
return new PlayoutItemWithPath(GetPlayoutItem(startTime, mediaItem, program), plexUrl);
}
}
}
@@ -0,0 +1,109 @@
using System.Net;
using System.Net.Http.Headers;
using ErsatzTV.Core.Interfaces.Streaming;
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Infrastructure.Streaming;
/// <summary>
/// Probes a media-server remote-stream URL over HTTP.
/// </summary>
/// <remarks>
/// Deliberately fail-open: the only outcome that reports the media as gone is a 404 that came
/// from the media server itself (i.e. arrived after our <c>/media/{provider}/...</c> endpoint
/// redirected). A timeout, a transport failure, any other status, or a 404 raised by ErsatzTV's
/// own endpoint all report available, so a probe that cannot answer never turns a tune that
/// would have worked into an error card. (ersatztv#473)
/// </remarks>
public class HttpRemoteStreamProber(
IHttpClientFactory httpClientFactory,
ILogger<HttpRemoteStreamProber> logger) : IRemoteStreamProber
{
private static readonly TimeSpan ProbeTimeout = TimeSpan.FromSeconds(2);
public async Task<bool> IsAvailable(string url, CancellationToken cancellationToken)
{
try
{
using var timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeoutCts.CancelAfter(ProbeTimeout);
using var request = new HttpRequestMessage(HttpMethod.Get, url);
// ask for a single byte; media servers vary in their HEAD support, and this exercises the
// same redirect chain ffmpeg will follow
request.Headers.Range = new RangeHeaderValue(0, 0);
using HttpClient client = httpClientFactory.CreateClient();
using HttpResponseMessage response = await client.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead,
timeoutCts.Token);
if (response.StatusCode is HttpStatusCode.NotFound)
{
// only the MEDIA SERVER's 404 is evidence that the item is gone. our own
// /media/{provider}/... endpoint also returns 404 when the media source is
// unconfigured or momentarily missing (InternalController maps a failed
// connection-parameter lookup to NotFound), and treating that as "gone" would fail
// CLOSED for every item on that source. A media-server 404 always arrives after a
// redirect, so an un-redirected 404 came from us and must fail open.
if (WasRedirected(response, url))
{
logger.LogWarning("Media server reported 404 for remote stream {Url}", url);
return false;
}
logger.LogDebug(
"Probe of {Url} returned 404 without redirecting to a media server; assuming the "
+ "item is available rather than failing closed on our own endpoint",
url);
return true;
}
// return the connection to the pool instead of aborting it by disposing an unread
// stream - but ONLY where the server honoured the range, i.e. the body really is one
// byte. A server that ignores `Range` answers 200 with the WHOLE FILE, and draining that
// would download at line rate into memory on the streaming hot path, defeating the
// ResponseHeadersRead above. There, abort the socket - much the cheaper evil.
if (response.StatusCode is HttpStatusCode.PartialContent)
{
var singleByte = new byte[1];
Stream body = await response.Content.ReadAsStreamAsync(timeoutCts.Token);
await body.ReadAsync(singleByte, timeoutCts.Token);
}
return true;
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
// the CALLER cancelled (shutdown / client disconnect). that is a genuine signal, not a
// probe failure, so it must propagate rather than be swallowed as fail-open.
throw;
}
catch (Exception ex)
{
// fail open - a probe failure is not evidence that the media is gone
logger.LogDebug(ex, "Unable to probe remote stream {Url}; assuming it is available", url);
return true;
}
}
private static bool WasRedirected(HttpResponseMessage response, string probeUrl)
{
Uri finalUri = response.RequestMessage?.RequestUri;
if (finalUri is null || !Uri.TryCreate(probeUrl, UriKind.Absolute, out Uri requestedUri))
{
// can't tell where the 404 came from; fail open rather than guess
return false;
}
// compare parsed Uris rather than strings. Uri.Equals compares normalized components, so it
// can't mistake an escaping/casing difference for a redirect and fail CLOSED - the exact
// failure this check exists to prevent. (A string compare on Uri.ToString() happens to agree
// for our machine-generated URLs, since ToString unescapes; this is defense in depth, not a
// fix for an observed bug.)
return !Uri.Equals(finalUri, requestedUri);
}
}
@@ -165,14 +165,276 @@ public class SynchronizeJellyfinLibraryByIdHandlerTests
// a user-initiated cancellation is not a failure and must not be logged at ERROR (#410)
await libraryRepository.DidNotReceive().UpdateLastScan(Arg.Any<Library>());
// prefix match, matching the mixed-library test below: equality would still be exact for
// this single-kind path, but a prefix cannot be quietly defeated by a reworded error
logger.ReceivedCalls()
.Any(call => call.GetArguments().ElementAtOrDefault(2)?.ToString()
== "Error synchronizing jellyfin library: Scan was canceled")
?.StartsWith("Error synchronizing jellyfin library:", StringComparison.Ordinal) == true)
.ShouldBeFalse();
logger.ReceivedCalls()
.Any(call => call.GetArguments().ElementAtOrDefault(2)?.ToString()
== "Scan of jellyfin library Concerts was canceled")
.ShouldBeTrue();
}
[Test]
public async Task Should_Scan_All_Three_Kinds_For_Mixed_Libraries()
{
var scannerProxy = Substitute.For<IScannerProxy>();
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
var jellyfinSecretStore = Substitute.For<IJellyfinSecretStore>();
var jellyfinMovieLibraryScanner = Substitute.For<IJellyfinMovieLibraryScanner>();
var jellyfinTelevisionLibraryScanner = Substitute.For<IJellyfinTelevisionLibraryScanner>();
var jellyfinMusicVideoLibraryScanner = Substitute.For<IJellyfinMusicVideoLibraryScanner>();
var libraryRepository = Substitute.For<ILibraryRepository>();
var configElementRepository = Substitute.For<IConfigElementRepository>();
var library = new JellyfinLibrary
{
Id = 42,
Name = "Music Videos",
MediaKind = LibraryMediaKind.Mixed,
MediaSourceId = 7
};
var mediaSource = new JellyfinMediaSource
{
Id = 7,
Connections =
[
new JellyfinConnection
{
Address = "http://jellyfin.example",
JellyfinMediaSourceId = 7
}
]
};
mediaSourceRepository.GetJellyfinByLibraryId(library.Id).Returns(Some(mediaSource).AsTask());
mediaSourceRepository.GetJellyfinLibrary(library.Id).Returns(Some(library).AsTask());
jellyfinSecretStore.ReadSecrets().Returns(new JellyfinSecrets
{
Address = "http://jellyfin.example",
ApiKey = "abc"
});
configElementRepository.GetValue<int>(
Arg.Is<ConfigElementKey>(key => key.Key == ConfigElementKey.LibraryRefreshInterval.Key),
Arg.Any<CancellationToken>())
.Returns(Task.FromResult<Option<int>>(Some(0)));
jellyfinMovieLibraryScanner.ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>())
.Returns(Right<BaseError, Unit>(Unit.Default).AsTask());
jellyfinTelevisionLibraryScanner.ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>())
.Returns(Right<BaseError, Unit>(Unit.Default).AsTask());
jellyfinMusicVideoLibraryScanner.ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>())
.Returns(Right<BaseError, Unit>(Unit.Default).AsTask());
var handler = new SynchronizeJellyfinLibraryByIdHandler(
scannerProxy,
mediaSourceRepository,
jellyfinSecretStore,
jellyfinMovieLibraryScanner,
jellyfinTelevisionLibraryScanner,
jellyfinMusicVideoLibraryScanner,
libraryRepository,
configElementRepository,
Substitute.For<ILogger<SynchronizeJellyfinLibraryByIdHandler>>());
Either<BaseError, string> result = await handler.Handle(
new SynchronizeJellyfinLibraryById("http://ersatztv.example", library.Id, true, true),
CancellationToken.None);
result.LeftToSeq().ShouldBeEmpty();
result.IsRight.ShouldBeTrue();
result.RightToSeq().Single().ShouldBe("Music Videos");
await jellyfinMovieLibraryScanner.Received(1).ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>());
await jellyfinTelevisionLibraryScanner.Received(1).ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>());
await jellyfinMusicVideoLibraryScanner.Received(1).ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>());
await libraryRepository.Received(1).UpdateLastScan(library);
}
[Test]
public async Task Should_Run_Remaining_Scanners_When_One_Fails_For_Mixed_Libraries()
{
var scannerProxy = Substitute.For<IScannerProxy>();
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
var jellyfinSecretStore = Substitute.For<IJellyfinSecretStore>();
var jellyfinMovieLibraryScanner = Substitute.For<IJellyfinMovieLibraryScanner>();
var jellyfinTelevisionLibraryScanner = Substitute.For<IJellyfinTelevisionLibraryScanner>();
var jellyfinMusicVideoLibraryScanner = Substitute.For<IJellyfinMusicVideoLibraryScanner>();
var libraryRepository = Substitute.For<ILibraryRepository>();
var configElementRepository = Substitute.For<IConfigElementRepository>();
var library = new JellyfinLibrary
{
Id = 42,
Name = "Music Videos",
MediaKind = LibraryMediaKind.Mixed,
MediaSourceId = 7
};
var mediaSource = new JellyfinMediaSource
{
Id = 7,
Connections =
[
new JellyfinConnection
{
Address = "http://jellyfin.example",
JellyfinMediaSourceId = 7
}
]
};
mediaSourceRepository.GetJellyfinByLibraryId(library.Id).Returns(Some(mediaSource).AsTask());
mediaSourceRepository.GetJellyfinLibrary(library.Id).Returns(Some(library).AsTask());
jellyfinSecretStore.ReadSecrets().Returns(new JellyfinSecrets
{
Address = "http://jellyfin.example",
ApiKey = "abc"
});
configElementRepository.GetValue<int>(
Arg.Is<ConfigElementKey>(key => key.Key == ConfigElementKey.LibraryRefreshInterval.Key),
Arg.Any<CancellationToken>())
.Returns(Task.FromResult<Option<int>>(Some(0)));
jellyfinMovieLibraryScanner.ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>())
.Returns(Left<BaseError, Unit>(BaseError.New("movie scan blew up")).AsTask());
jellyfinTelevisionLibraryScanner.ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>())
.Returns(Right<BaseError, Unit>(Unit.Default).AsTask());
jellyfinMusicVideoLibraryScanner.ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>())
.Returns(Right<BaseError, Unit>(Unit.Default).AsTask());
var handler = new SynchronizeJellyfinLibraryByIdHandler(
scannerProxy,
mediaSourceRepository,
jellyfinSecretStore,
jellyfinMovieLibraryScanner,
jellyfinTelevisionLibraryScanner,
jellyfinMusicVideoLibraryScanner,
libraryRepository,
configElementRepository,
Substitute.For<ILogger<SynchronizeJellyfinLibraryByIdHandler>>());
Either<BaseError, string> result = await handler.Handle(
new SynchronizeJellyfinLibraryById("http://ersatztv.example", library.Id, true, true),
CancellationToken.None);
// the movie arm failed, so the library as a whole failed...
result.IsLeft.ShouldBeTrue();
// ...but the other two kinds were still ingested
await jellyfinTelevisionLibraryScanner.Received(1).ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>());
await jellyfinMusicVideoLibraryScanner.Received(1).ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>());
// and LastScan is not stamped, because the scan was not fully successful
await libraryRepository.DidNotReceive().UpdateLastScan(Arg.Any<Library>());
}
[Test]
public async Task Should_Stop_Scanning_Mixed_Library_When_Canceled()
{
var scannerProxy = Substitute.For<IScannerProxy>();
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
var jellyfinSecretStore = Substitute.For<IJellyfinSecretStore>();
var jellyfinMovieLibraryScanner = Substitute.For<IJellyfinMovieLibraryScanner>();
var jellyfinTelevisionLibraryScanner = Substitute.For<IJellyfinTelevisionLibraryScanner>();
var jellyfinMusicVideoLibraryScanner = Substitute.For<IJellyfinMusicVideoLibraryScanner>();
var libraryRepository = Substitute.For<ILibraryRepository>();
var configElementRepository = Substitute.For<IConfigElementRepository>();
var library = new JellyfinLibrary
{
Id = 42,
Name = "Music Videos",
MediaKind = LibraryMediaKind.Mixed,
MediaSourceId = 7
};
var mediaSource = new JellyfinMediaSource
{
Id = 7,
Connections =
[
new JellyfinConnection
{
Address = "http://jellyfin.example",
JellyfinMediaSourceId = 7
}
]
};
mediaSourceRepository.GetJellyfinByLibraryId(library.Id).Returns(Some(mediaSource).AsTask());
mediaSourceRepository.GetJellyfinLibrary(library.Id).Returns(Some(library).AsTask());
jellyfinSecretStore.ReadSecrets().Returns(new JellyfinSecrets
{
Address = "http://jellyfin.example",
ApiKey = "abc"
});
configElementRepository.GetValue<int>(
Arg.Is<ConfigElementKey>(key => key.Key == ConfigElementKey.LibraryRefreshInterval.Key),
Arg.Any<CancellationToken>())
.Returns(Task.FromResult<Option<int>>(Some(0)));
var logger = Substitute.For<ILogger<SynchronizeJellyfinLibraryByIdHandler>>();
jellyfinMovieLibraryScanner.ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(), library, true, Arg.Any<CancellationToken>())
.Returns(Left<BaseError, Unit>(new ScanCanceled()).AsTask());
var handler = new SynchronizeJellyfinLibraryByIdHandler(
scannerProxy,
mediaSourceRepository,
jellyfinSecretStore,
jellyfinMovieLibraryScanner,
jellyfinTelevisionLibraryScanner,
jellyfinMusicVideoLibraryScanner,
libraryRepository,
configElementRepository,
logger);
Either<BaseError, string> result = await handler.Handle(
new SynchronizeJellyfinLibraryById("http://ersatztv.example", library.Id, true, true),
CancellationToken.None);
result.IsLeft.ShouldBeTrue();
// a user-initiated cancellation aborts the whole library; later kinds must not run
await jellyfinTelevisionLibraryScanner.DidNotReceive().ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(),
Arg.Any<JellyfinLibrary>(),
Arg.Any<bool>(),
Arg.Any<CancellationToken>());
await jellyfinMusicVideoLibraryScanner.DidNotReceive().ScanLibrary(
Arg.Any<JellyfinConnectionParameters>(),
Arg.Any<JellyfinLibrary>(),
Arg.Any<bool>(),
Arg.Any<CancellationToken>());
await libraryRepository.DidNotReceive().UpdateLastScan(Arg.Any<Library>());
// ScanMixedLibrary must return the ScanCanceled INSTANCE unwrapped, not folded into an
// aggregate BaseError -- otherwise `error is ScanCanceled` in the caller fails and a user
// cancellation is demoted to an ERROR log (#410).
logger.ReceivedCalls()
.Any(call => call.GetArguments().ElementAtOrDefault(2)?.ToString()
== "Scan of jellyfin library Music Videos was canceled")
.ShouldBeTrue();
// prefix match, not equality: if ScanCanceled were folded into the aggregate error the
// rendered message becomes "...: Mixed library X had 1 scan error(s): Scan was canceled",
// which an equality assertion would NOT catch -- making the check vacuous
logger.ReceivedCalls()
.Any(call => call.GetArguments().ElementAtOrDefault(2)?.ToString()
?.StartsWith("Error synchronizing jellyfin library:", StringComparison.Ordinal) == true)
.ShouldBeFalse();
}
}
}
@@ -0,0 +1,110 @@
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Jellyfin;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Scanner.Core.Interfaces;
using ErsatzTV.Scanner.Core.Interfaces.Metadata;
using ErsatzTV.Scanner.Core.Metadata;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Scanner.Tests.Core.Metadata;
// #477: a successful-but-empty media-server fetch must not flag a non-empty movie library FileNotFound
// (which EmptyTrash could then permanently delete). Assert the sweep is skipped when nothing came in.
public class MediaServerMovieLibraryScannerTests
{
[TestFixture]
public class CleanupFileNotFoundItems
{
[Test]
public async Task Empty_Incoming_With_Existing_Movies_Does_Not_Flag()
{
var movieRepository = Substitute.For<IJellyfinMovieRepository>();
var scannerProxy = Substitute.For<IScannerProxy>();
var library = new JellyfinLibrary { Id = 3, Name = "Movies" };
movieRepository.GetExistingMovies(library)
.Returns(new List<JellyfinItemEtag>
{
new() { ItemId = "movie-1", State = MediaItemState.Normal },
new() { ItemId = "movie-2", State = MediaItemState.Normal }
});
var scanner = new TestMovieLibraryScanner(scannerProxy);
Either<BaseError, Unit> result = await scanner.Scan(
movieRepository,
new JellyfinConnectionParameters("http://jellyfin.example", "abc", 7),
library);
result.IsRight.ShouldBeTrue();
await movieRepository.DidNotReceive().FlagFileNotFound(
Arg.Any<JellyfinLibrary>(), Arg.Any<List<string>>());
await scannerProxy.DidNotReceive().ReindexMediaItems(
Arg.Any<int[]>(), Arg.Any<CancellationToken>());
}
}
// Minimal concrete subclass exposing the cleanup path. Driven with an empty incoming list, so none of
// the per-item metadata members below are ever invoked — they exist only to satisfy the contract.
private sealed class TestMovieLibraryScanner : MediaServerMovieLibraryScanner<
JellyfinConnectionParameters, JellyfinLibrary, JellyfinMovie, JellyfinItemEtag>
{
public TestMovieLibraryScanner(IScannerProxy scannerProxy)
: base(
scannerProxy,
Substitute.For<IFileSystem>(),
Substitute.For<ILocalChaptersProvider>(),
Substitute.For<IMetadataRepository>(),
Substitute.For<ILogger>())
{
}
public Task<Either<BaseError, Unit>> Scan(
IJellyfinMovieRepository movieRepository,
JellyfinConnectionParameters connectionParameters,
JellyfinLibrary library) =>
ScanLibrary(
movieRepository,
connectionParameters,
library,
_ => string.Empty,
false,
CancellationToken.None);
protected override IAsyncEnumerable<Tuple<JellyfinMovie, int>> GetMovieLibraryItems(
JellyfinConnectionParameters connectionParameters, JellyfinLibrary library) => EmptyMovies();
protected override string MediaServerItemId(JellyfinMovie movie) => movie.ItemId;
protected override string MediaServerEtag(JellyfinMovie movie) => movie.Etag;
protected override Task<Option<MovieMetadata>> GetFullMetadata(
JellyfinConnectionParameters connectionParameters, JellyfinLibrary library,
MediaItemScanResult<JellyfinMovie> result, JellyfinMovie incoming, bool deepScan) =>
throw new NotSupportedException();
protected override Task<Option<Tuple<MovieMetadata, MediaVersion>>> GetFullMetadataAndStatistics(
JellyfinConnectionParameters connectionParameters, JellyfinLibrary library,
MediaItemScanResult<JellyfinMovie> result, JellyfinMovie incoming) =>
throw new NotSupportedException();
protected override Task<Either<BaseError, MediaItemScanResult<JellyfinMovie>>> UpdateMetadata(
MediaItemScanResult<JellyfinMovie> result, MovieMetadata fullMetadata,
CancellationToken cancellationToken) =>
throw new NotSupportedException();
private static async IAsyncEnumerable<Tuple<JellyfinMovie, int>> EmptyMovies()
{
await Task.CompletedTask;
yield break;
}
}
}
@@ -0,0 +1,108 @@
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Core.Plex;
using ErsatzTV.Scanner.Core.Interfaces;
using ErsatzTV.Scanner.Core.Interfaces.Metadata;
using ErsatzTV.Scanner.Core.Metadata;
using ErsatzTV.Scanner.Core.Plex;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Scanner.Tests.Core.Metadata;
// #477: a successful-but-empty media-server fetch must not flag a non-empty other-video library
// FileNotFound (which EmptyTrash could then permanently delete). Assert the sweep is skipped.
public class MediaServerOtherVideoLibraryScannerTests
{
[TestFixture]
public class CleanupFileNotFoundItems
{
[Test]
public async Task Empty_Incoming_With_Existing_OtherVideos_Does_Not_Flag()
{
var otherVideoRepository = Substitute.For<IPlexOtherVideoRepository>();
var scannerProxy = Substitute.For<IScannerProxy>();
var library = new PlexLibrary { Id = 9, Name = "Other Videos" };
otherVideoRepository.GetExistingOtherVideos(library)
.Returns(new List<PlexItemEtag>
{
new() { Key = "ov-1", State = MediaItemState.Normal },
new() { Key = "ov-2", State = MediaItemState.Normal }
});
var scanner = new TestOtherVideoLibraryScanner(scannerProxy);
Either<BaseError, Unit> result = await scanner.Scan(otherVideoRepository, library);
result.IsRight.ShouldBeTrue();
await otherVideoRepository.DidNotReceive().FlagFileNotFound(
Arg.Any<PlexLibrary>(), Arg.Any<List<string>>());
await scannerProxy.DidNotReceive().ReindexMediaItems(
Arg.Any<int[]>(), Arg.Any<CancellationToken>());
}
}
// Minimal concrete subclass exposing the cleanup path. Driven with an empty incoming list, so none of
// the per-item metadata members below are ever invoked — they exist only to satisfy the contract. The
// connection parameters are likewise never dereferenced on the empty path (passed null! below).
private sealed class TestOtherVideoLibraryScanner : MediaServerOtherVideoLibraryScanner<
PlexConnectionParameters, PlexLibrary, PlexOtherVideo, PlexItemEtag>
{
public TestOtherVideoLibraryScanner(IScannerProxy scannerProxy)
: base(
scannerProxy,
Substitute.For<IFileSystem>(),
Substitute.For<ILocalChaptersProvider>(),
Substitute.For<IMetadataRepository>(),
Substitute.For<ILogger>())
{
}
public Task<Either<BaseError, Unit>> Scan(
IPlexOtherVideoRepository otherVideoRepository,
PlexLibrary library) =>
ScanLibrary(
otherVideoRepository,
null!,
library,
_ => string.Empty,
false,
CancellationToken.None);
protected override IAsyncEnumerable<Tuple<PlexOtherVideo, int>> GetOtherVideoLibraryItems(
PlexConnectionParameters connectionParameters, PlexLibrary library) => EmptyOtherVideos();
protected override string MediaServerItemId(PlexOtherVideo otherVideo) => otherVideo.Key;
protected override string MediaServerEtag(PlexOtherVideo otherVideo) => otherVideo.Etag;
protected override Task<Option<OtherVideoMetadata>> GetFullMetadata(
PlexConnectionParameters connectionParameters, PlexLibrary library,
MediaItemScanResult<PlexOtherVideo> result, PlexOtherVideo incoming, bool deepScan) =>
throw new NotSupportedException();
protected override Task<Option<Tuple<OtherVideoMetadata, MediaVersion>>> GetFullMetadataAndStatistics(
PlexConnectionParameters connectionParameters, PlexLibrary library,
MediaItemScanResult<PlexOtherVideo> result, PlexOtherVideo incoming) =>
throw new NotSupportedException();
protected override Task<Either<BaseError, MediaItemScanResult<PlexOtherVideo>>> UpdateMetadata(
MediaItemScanResult<PlexOtherVideo> result, OtherVideoMetadata fullMetadata,
CancellationToken cancellationToken) =>
throw new NotSupportedException();
private static async IAsyncEnumerable<Tuple<PlexOtherVideo, int>> EmptyOtherVideos()
{
await Task.CompletedTask;
yield break;
}
}
}
@@ -0,0 +1,69 @@
using ErsatzTV.Scanner.Core.Metadata;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Scanner.Tests.Core.Metadata;
// #477: the deterministic policy behind the media-server anti-nuke guard. An empty incoming set with
// existing items present is the only case that skips the sweep (and logs); every other combination
// reconciles normally.
public class MediaServerReconciliationGuardTests
{
[Test]
public void Empty_Incoming_With_Existing_Items_Skips_And_Warns()
{
var logger = Substitute.For<ILogger>();
bool shouldFlag = MediaServerReconciliationGuard.ShouldFlagMissing(logger, "Movies", 0, 5);
shouldFlag.ShouldBeFalse();
logger.Received(1).Log(
LogLevel.Warning,
Arg.Any<EventId>(),
Arg.Any<object>(),
Arg.Any<Exception>(),
Arg.Any<Func<object, Exception?, string>>());
}
[Test]
public void Partial_Deletion_Still_Flags()
{
var logger = Substitute.For<ILogger>();
MediaServerReconciliationGuard.ShouldFlagMissing(logger, "Movies", 3, 5).ShouldBeTrue();
logger.DidNotReceive().Log(
LogLevel.Warning,
Arg.Any<EventId>(),
Arg.Any<object>(),
Arg.Any<Exception>(),
Arg.Any<Func<object, Exception?, string>>());
}
[Test]
public void Empty_Incoming_With_No_Existing_Items_Is_A_Noop_Sweep()
{
var logger = Substitute.For<ILogger>();
// nothing exists, so an empty incoming set flags nothing either way — allow the (empty) sweep
// rather than special-casing it, and do not emit the scary warning.
MediaServerReconciliationGuard.ShouldFlagMissing(logger, "Movies", 0, 0).ShouldBeTrue();
logger.DidNotReceive().Log(
LogLevel.Warning,
Arg.Any<EventId>(),
Arg.Any<object>(),
Arg.Any<Exception>(),
Arg.Any<Func<object, Exception?, string>>());
}
[Test]
public void Full_Fetch_Into_Empty_Library_Still_Flags()
{
var logger = Substitute.For<ILogger>();
MediaServerReconciliationGuard.ShouldFlagMissing(logger, "Movies", 3, 0).ShouldBeTrue();
}
}
@@ -0,0 +1,238 @@
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Jellyfin;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Scanner.Core.Interfaces;
using ErsatzTV.Scanner.Core.Interfaces.Metadata;
using ErsatzTV.Scanner.Core.Metadata;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Scanner.Tests.Core.Metadata;
// #476: a show/season gone from the media server is absent from the incoming list, so the per-parent
// loop never visits it and the descendant sweeps never run for it. These tests assert the scanner
// cascades the FileNotFound flag to descendants via the repository, using a substituted repository.
// #477: an empty incoming list is treated as a suspect (mid-restore / emptied) fetch and the sweep is
// skipped instead of nuking the whole library — so the #476 cascade is now exercised with a survivor
// present (a genuine partial deletion), and the empty case asserts nothing is flagged.
public class MediaServerTelevisionLibraryScannerTests
{
[TestFixture]
public class CleanupFileNotFoundItems
{
[Test]
public async Task Removed_Show_Cascades_FileNotFound_To_Seasons_And_Episodes()
{
var televisionRepository = Substitute.For<IJellyfinTelevisionRepository>();
var scannerProxy = Substitute.For<IScannerProxy>();
var library = new JellyfinLibrary { Id = 42, Name = "TV Shows" };
// two shows exist; the survivor is still in the (non-empty) incoming list, so this is a
// genuine partial deletion — "show-6366" is gone upstream and must be flagged + cascaded.
// (#477: an EMPTY incoming would instead skip the sweep — see the guard test below.)
televisionRepository.GetExistingShows(library, Arg.Any<CancellationToken>())
.Returns(new List<JellyfinItemEtag>
{
new() { ItemId = "show-keep", State = MediaItemState.Normal },
new() { ItemId = "show-6366", State = MediaItemState.FileNotFound }
});
// the survivor short-circuits to Left so the per-item metadata path (unsupported in this
// harness) is never entered; it is still recorded as incoming, so it is not swept.
televisionRepository.GetOrAdd(library, Arg.Any<JellyfinShow>(), Arg.Any<CancellationToken>())
.Returns(Left<BaseError, MediaItemScanResult<JellyfinShow>>(BaseError.New("skip metadata in test")));
// the per-item loop reports progress; an unstubbed substitute returns false => ScanCanceled
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
televisionRepository
.FlagFileNotFoundShows(library, Arg.Is<List<string>>(l => l.Contains("show-6366")),
Arg.Any<CancellationToken>())
.Returns(new List<int> { 100 });
televisionRepository
.FlagFileNotFoundSeasonsForShows(Arg.Is<List<int>>(l => l.SequenceEqual(new[] { 100 })),
Arg.Any<CancellationToken>())
.Returns(new List<int> { 200, 201 });
televisionRepository
.FlagFileNotFoundEpisodesForSeasons(Arg.Is<List<int>>(l => l.SequenceEqual(new[] { 200, 201 })),
Arg.Any<CancellationToken>())
.Returns(new List<int> { 300, 301, 302 });
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
var scanner = new TestTelevisionLibraryScanner(scannerProxy);
Either<BaseError, Unit> result = await scanner.Scan(
televisionRepository,
new JellyfinConnectionParameters("http://jellyfin.example", "abc", 7),
library,
Shows(new JellyfinShow
{
ItemId = "show-keep",
ShowMetadata = new List<ShowMetadata> { new() { Title = "Keeper" } }
}));
result.IsRight.ShouldBeTrue();
await televisionRepository.Received(1).FlagFileNotFoundShows(
library,
Arg.Is<List<string>>(l => l.Count == 1 && l.Contains("show-6366")),
Arg.Any<CancellationToken>());
await televisionRepository.Received(1).FlagFileNotFoundSeasonsForShows(
Arg.Is<List<int>>(l => l.SequenceEqual(new[] { 100 })),
Arg.Any<CancellationToken>());
await televisionRepository.Received(1).FlagFileNotFoundEpisodesForSeasons(
Arg.Is<List<int>>(l => l.SequenceEqual(new[] { 200, 201 })),
Arg.Any<CancellationToken>());
// every affected item (show + seasons + episodes) is reindexed so search reflects the new state
await scannerProxy.Received(1).ReindexMediaItems(
Arg.Is<int[]>(a => new[] { 100, 200, 201, 300, 301, 302 }.All(id => a.Contains(id))),
Arg.Any<CancellationToken>());
}
[Test]
public async Task Empty_Incoming_With_Existing_Shows_Does_Not_Flag()
{
var televisionRepository = Substitute.For<IJellyfinTelevisionRepository>();
var scannerProxy = Substitute.For<IScannerProxy>();
var library = new JellyfinLibrary { Id = 42, Name = "TV Shows" };
// shows exist locally, but a successful fetch returned ZERO items (server mid-restore or the
// library was emptied upstream). #477: flagging here would nuke the entire library, so the
// sweep must be skipped and nothing flagged or reindexed.
televisionRepository.GetExistingShows(library, Arg.Any<CancellationToken>())
.Returns(new List<JellyfinItemEtag>
{
new() { ItemId = "show-1", State = MediaItemState.Normal },
new() { ItemId = "show-2", State = MediaItemState.Normal }
});
var scanner = new TestTelevisionLibraryScanner(scannerProxy);
Either<BaseError, Unit> result = await scanner.Scan(
televisionRepository,
new JellyfinConnectionParameters("http://jellyfin.example", "abc", 7),
library,
EmptyShows());
result.IsRight.ShouldBeTrue();
await televisionRepository.DidNotReceive().FlagFileNotFoundShows(
Arg.Any<JellyfinLibrary>(), Arg.Any<List<string>>(), Arg.Any<CancellationToken>());
await televisionRepository.DidNotReceive().FlagFileNotFoundSeasonsForShows(
Arg.Any<List<int>>(), Arg.Any<CancellationToken>());
await televisionRepository.DidNotReceive().FlagFileNotFoundEpisodesForSeasons(
Arg.Any<List<int>>(), Arg.Any<CancellationToken>());
await scannerProxy.DidNotReceive().ReindexMediaItems(
Arg.Any<int[]>(), Arg.Any<CancellationToken>());
}
private static async IAsyncEnumerable<Tuple<JellyfinShow, int>> EmptyShows()
{
await Task.CompletedTask;
yield break;
}
private static async IAsyncEnumerable<Tuple<JellyfinShow, int>> Shows(params JellyfinShow[] shows)
{
await Task.CompletedTask;
foreach (JellyfinShow show in shows)
{
yield return new Tuple<JellyfinShow, int>(show, shows.Length);
}
}
}
// Minimal concrete subclass that exposes the abstract scanner's cleanup path. The incoming show list
// is supplied directly (empty = "all shows removed"), so none of the per-item metadata members below
// are ever invoked — they exist only to satisfy the abstract contract.
private sealed class TestTelevisionLibraryScanner : MediaServerTelevisionLibraryScanner<
JellyfinConnectionParameters, JellyfinLibrary, JellyfinShow, JellyfinSeason, JellyfinEpisode,
JellyfinItemEtag>
{
public TestTelevisionLibraryScanner(IScannerProxy scannerProxy)
: base(
scannerProxy,
Substitute.For<IFileSystem>(),
Substitute.For<ILocalChaptersProvider>(),
Substitute.For<IMetadataRepository>(),
Substitute.For<ILogger>())
{
}
public Task<Either<BaseError, Unit>> Scan(
IMediaServerTelevisionRepository<JellyfinLibrary, JellyfinShow, JellyfinSeason, JellyfinEpisode,
JellyfinItemEtag> televisionRepository,
JellyfinConnectionParameters connectionParameters,
JellyfinLibrary library,
IAsyncEnumerable<Tuple<JellyfinShow, int>> showEntries) =>
ScanLibrary(
televisionRepository,
connectionParameters,
library,
_ => string.Empty,
showEntries,
false,
CancellationToken.None);
protected override IAsyncEnumerable<Tuple<JellyfinShow, int>> GetShowLibraryItems(
JellyfinConnectionParameters connectionParameters, JellyfinLibrary library) =>
throw new NotSupportedException();
protected override string MediaServerItemId(JellyfinShow show) => show.ItemId;
protected override string MediaServerItemId(JellyfinSeason season) => season.ItemId;
protected override string MediaServerItemId(JellyfinEpisode episode) => episode.ItemId;
protected override string MediaServerEtag(JellyfinShow show) => show.Etag;
protected override string MediaServerEtag(JellyfinSeason season) => season.Etag;
protected override string MediaServerEtag(JellyfinEpisode episode) => episode.Etag;
protected override IAsyncEnumerable<Tuple<JellyfinSeason, int>> GetSeasonLibraryItems(
JellyfinLibrary library, JellyfinConnectionParameters connectionParameters, JellyfinShow show) =>
throw new NotSupportedException();
protected override IAsyncEnumerable<Tuple<JellyfinEpisode, int>> GetEpisodeLibraryItems(
JellyfinLibrary library, JellyfinConnectionParameters connectionParameters, JellyfinShow show,
JellyfinSeason season, bool isNewSeason) =>
throw new NotSupportedException();
protected override Task<Option<ShowMetadata>> GetFullMetadata(
JellyfinConnectionParameters connectionParameters, JellyfinLibrary library,
MediaItemScanResult<JellyfinShow> result, JellyfinShow incoming, bool deepScan) =>
throw new NotSupportedException();
protected override Task<Option<SeasonMetadata>> GetFullMetadata(
JellyfinConnectionParameters connectionParameters, JellyfinLibrary library,
MediaItemScanResult<JellyfinSeason> result, JellyfinSeason incoming, bool deepScan) =>
throw new NotSupportedException();
protected override Task<Option<EpisodeMetadata>> GetFullMetadata(
JellyfinConnectionParameters connectionParameters, JellyfinLibrary library,
MediaItemScanResult<JellyfinEpisode> result, JellyfinEpisode incoming, bool deepScan) =>
throw new NotSupportedException();
protected override Task<Option<Tuple<EpisodeMetadata, MediaVersion>>> GetFullMetadataAndStatistics(
JellyfinConnectionParameters connectionParameters, JellyfinLibrary library,
MediaItemScanResult<JellyfinEpisode> result, JellyfinEpisode incoming) =>
throw new NotSupportedException();
protected override Task<Either<BaseError, MediaItemScanResult<JellyfinShow>>> UpdateMetadata(
MediaItemScanResult<JellyfinShow> result, ShowMetadata fullMetadata) =>
throw new NotSupportedException();
protected override Task<Either<BaseError, MediaItemScanResult<JellyfinSeason>>> UpdateMetadata(
MediaItemScanResult<JellyfinSeason> result, SeasonMetadata fullMetadata) =>
throw new NotSupportedException();
protected override Task<Either<BaseError, MediaItemScanResult<JellyfinEpisode>>> UpdateMetadata(
MediaItemScanResult<JellyfinEpisode> result, EpisodeMetadata fullMetadata,
CancellationToken cancellationToken) =>
throw new NotSupportedException();
}
}
@@ -85,7 +85,10 @@ public class
parameters.Library,
parameters.DeepScan,
cancellationToken),
_ => Unit.Default
LibraryMediaKind.Mixed =>
await ScanMixedLibrary(parameters, cancellationToken),
_ => BaseError.New(
$"Jellyfin library {parameters.Library.Name} has unsupported media kind {parameters.Library.MediaKind}")
};
if (result.IsRight)
@@ -116,6 +119,73 @@ public class
return parameters.Library.Name;
}
/// <summary>
/// Scans a mixed-content library by running each per-kind scanner against it in turn. Jellyfin
/// resolves the mix server-side - each scanner queries with its own includeItemTypes - so the
/// passes see disjoint item sets, and their reconciliation is type-scoped and cannot
/// cross-delete.
/// </summary>
private async Task<Either<BaseError, Unit>> ScanMixedLibrary(
RequestParameters parameters,
CancellationToken cancellationToken)
{
_logger.LogInformation(
"Scanning mixed-content jellyfin library {Name}",
parameters.Library.Name);
var scans = new Func<Task<Either<BaseError, Unit>>>[]
{
() => _jellyfinMovieLibraryScanner.ScanLibrary(
parameters.ConnectionParameters,
parameters.Library,
parameters.DeepScan,
cancellationToken),
() => _jellyfinTelevisionLibraryScanner.ScanLibrary(
parameters.ConnectionParameters,
parameters.Library,
parameters.DeepScan,
cancellationToken),
() => _jellyfinMusicVideoLibraryScanner.ScanLibrary(
parameters.ConnectionParameters,
parameters.Library,
parameters.DeepScan,
cancellationToken)
};
var errors = new List<BaseError>();
foreach (Func<Task<Either<BaseError, Unit>>> scan in scans)
{
Either<BaseError, Unit> result = await scan();
foreach (BaseError error in result.LeftToSeq())
{
// a cancellation aborts the whole library immediately; it is not one kind failing
if (error is ScanCanceled)
{
return error;
}
// one kind failing must not stop the others from being ingested
_logger.LogWarning(
"Error scanning one media kind of mixed jellyfin library {Name}: {Error}",
parameters.Library.Name,
error.Value);
errors.Add(error);
}
}
if (errors.Count > 0)
{
return BaseError.New(
$"Mixed library {parameters.Library.Name} had {errors.Count} scan error(s): " +
string.Join("; ", errors.Map(e => e.Value)));
}
return Unit.Default;
}
private async Task<Validation<BaseError, RequestParameters>> Validate(
SynchronizeJellyfinLibraryById request,
CancellationToken cancellationToken) =>
@@ -48,7 +48,8 @@ public class
RequestParameters parameters,
CancellationToken cancellationToken)
{
if (parameters.Library.MediaKind != LibraryMediaKind.Shows)
// a mixed library legitimately contains shows alongside movies and music videos
if (parameters.Library.MediaKind is not (LibraryMediaKind.Shows or LibraryMediaKind.Mixed))
{
return BaseError.New($"Library {parameters.Library.Name} is not a TV show library");
}
@@ -139,7 +139,11 @@ public class ScanLocalLibraryHandler : IRequestHandler<ScanLocalLibrary, Either<
progressMin,
progressMax,
cancellationToken),
_ => Unit.Default
// returning success here would stamp LastScan as though the library had been
// scanned; a local library has no scanner for Mixed and never should
_ => BaseError.New(
$"Local library {localLibrary.Name} has unsupported media kind {localLibrary.MediaKind}")
};
if (result.IsRight)
@@ -1,4 +1,4 @@
using ErsatzTV.Core;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Extensions;
@@ -7,6 +7,7 @@ using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Jellyfin;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Scanner.Core.Interfaces;
using ErsatzTV.Scanner.Core.Metadata;
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Scanner.Core.Jellyfin;
@@ -77,6 +78,7 @@ public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanne
await _mediaSourceRepository.GetJellyfinPathReplacements(library.MediaSourceId);
var processed = 0;
var incomingPaths = new List<string>();
await foreach ((MusicVideo incoming, int totalCount) in _jellyfinApiClient
.GetMusicVideoLibraryItems(
connectionParameters.Address,
@@ -96,6 +98,8 @@ public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanne
return new ScanCanceled();
}
incomingPaths.Add(GetLocalPath(pathReplacements, incoming));
Either<BaseError, MediaItemScanResult<MusicVideo>> maybeMusicVideo =
await ProcessMusicVideo(library, libraryPath, pathReplacements, incoming, cancellationToken);
@@ -114,9 +118,62 @@ public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanne
}
}
await TrashMissingMusicVideos(library, libraryPath, incomingPaths, cancellationToken);
return Unit.Default;
}
// ersatztv#494: remove music videos (and now-empty artists) that Jellyfin no longer reports.
//
// Identity is LIBRARY-SCOPED by (LibraryPathId, path): FindMusicVideoPaths and DeleteByPath both filter
// LibraryPathId AND join the concrete MusicVideo table, so this can never touch a Movie/Show that shares
// the same LibraryPath (a mixed library) — the cross-delete safety is a property of those queries, not of
// the media kind. Unlike the MediaServer{Movie,Television,OtherVideo} base scanners, music videos carry no
// server ItemId/Etag (there is no JellyfinMusicVideo entity), so we diff on the local path instead of the
// server item id, and hard-delete rather than soft-trash (there is no per-item FileNotFound seam here).
//
// Known limitation: MusicVideoRepository.GetOrAdd dedups a path GLOBALLY (no LibraryPathId predicate), so a
// file served by two libraries with overlapping local paths is a single row owned by whichever library
// scanned it first. If that owning library later stops reporting the file while another library still
// serves it, this sweep removes the shared row. A proper fix needs per-library music-video identity (a
// JellyfinMusicVideo etag entity + migration) — the issue's deferred "option 2"; tracked as a follow-up.
private async Task TrashMissingMusicVideos(
JellyfinLibrary library,
LibraryPath libraryPath,
List<string> incomingPaths,
CancellationToken cancellationToken)
{
var existingPaths = (await _musicVideoRepository.FindMusicVideoPaths(libraryPath)).ToList();
// #477: refuse the sweep when a successful fetch returned zero items but rows exist locally — an empty
// incoming set is indistinguishable from a transient error and would otherwise wipe the whole library.
if (!MediaServerReconciliationGuard.ShouldFlagMissing(
_logger,
library.Name,
incomingPaths.Count,
existingPaths.Count))
{
return;
}
foreach (string path in existingPaths.Except(incomingPaths))
{
List<int> musicVideoIds = await _musicVideoRepository.DeleteByPath(libraryPath, path);
if (musicVideoIds.Count > 0 &&
!await _scannerProxy.RemoveMediaItems(musicVideoIds.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to remove media items from scanner process");
}
}
List<int> artistIds = await _artistRepository.DeleteEmptyArtists(libraryPath);
if (artistIds.Count > 0 &&
!await _scannerProxy.RemoveMediaItems(artistIds.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to remove empty artists from scanner process");
}
}
private async Task<Either<BaseError, MediaItemScanResult<MusicVideo>>> ProcessMusicVideo(
JellyfinLibrary library,
LibraryPath libraryPath,
@@ -206,11 +263,112 @@ public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanne
existing.DateUpdated = DateTime.UtcNow;
existing.MetadataKind = MetadataKind.External;
return await _metadataRepository.Update(existing);
bool updated = await _metadataRepository.Update(existing);
// ersatztv#497: the scalar Update above marks only the metadata row Modified; it does NOT touch
// child collections, and MusicVideoRepository.GetOrAdd loads them AsNoTracking — so tag/genre/studio/
// artist edits made in Jellyfin never reached an EXISTING music video (only the Add path persisted
// them). Reconcile the collections that BOTH the Add path persists AND GetOrAdd eager-loads:
// Genres, Tags, Studios, Artists. (Guids are add-persisted but not eager-loaded here — reconciling
// them would see an empty `existing` and duplicate-insert every scan; Directors are eager-loaded
// but not add-persisted for music videos — both are deliberately out of scope.) Mirrors the
// remove-stale + add-new idiom PlexMovieLibraryScanner.UpdateMetadata uses.
updated = await ReconcileGenres(existing, incoming) || updated;
updated = await ReconcileTags(existing, incoming) || updated;
updated = await ReconcileStudios(existing, incoming) || updated;
updated = await ReconcileArtists(existing, incoming) || updated;
return updated;
}
incoming.MusicVideoId = musicVideo.Id;
musicVideo.MusicVideoMetadata = [incoming];
return await _metadataRepository.Add(incoming);
}
private async Task<bool> ReconcileGenres(MusicVideoMetadata existing, MusicVideoMetadata incoming)
{
existing.Genres ??= [];
List<Genre> incomingGenres = incoming.Genres ?? [];
var updated = false;
foreach (Genre genre in existing.Genres.Filter(g => incomingGenres.All(g2 => g2.Name != g.Name)).ToList())
{
existing.Genres.Remove(genre);
updated = await _metadataRepository.RemoveGenre(genre) || updated;
}
foreach (Genre genre in incomingGenres.Filter(g => existing.Genres.All(g2 => g2.Name != g.Name)).ToList())
{
existing.Genres.Add(genre);
updated = await _musicVideoRepository.AddGenre(existing, genre) || updated;
}
return updated;
}
private async Task<bool> ReconcileTags(MusicVideoMetadata existing, MusicVideoMetadata incoming)
{
existing.Tags ??= [];
List<Tag> incomingTags = incoming.Tags ?? [];
var updated = false;
foreach (Tag tag in existing.Tags.Filter(t => incomingTags.All(t2 => t2.Name != t.Name)).ToList())
{
existing.Tags.Remove(tag);
updated = await _metadataRepository.RemoveTag(tag) || updated;
}
foreach (Tag tag in incomingTags.Filter(t => existing.Tags.All(t2 => t2.Name != t.Name)).ToList())
{
existing.Tags.Add(tag);
updated = await _musicVideoRepository.AddTag(existing, tag) || updated;
}
return updated;
}
private async Task<bool> ReconcileStudios(MusicVideoMetadata existing, MusicVideoMetadata incoming)
{
existing.Studios ??= [];
List<Studio> incomingStudios = incoming.Studios ?? [];
var updated = false;
foreach (Studio studio in existing.Studios.Filter(s => incomingStudios.All(s2 => s2.Name != s.Name)).ToList())
{
existing.Studios.Remove(studio);
updated = await _metadataRepository.RemoveStudio(studio) || updated;
}
foreach (Studio studio in incomingStudios.Filter(s => existing.Studios.All(s2 => s2.Name != s.Name)).ToList())
{
existing.Studios.Add(studio);
updated = await _musicVideoRepository.AddStudio(existing, studio) || updated;
}
return updated;
}
private async Task<bool> ReconcileArtists(MusicVideoMetadata existing, MusicVideoMetadata incoming)
{
existing.Artists ??= [];
List<MusicVideoArtist> incomingArtists = incoming.Artists ?? [];
var updated = false;
foreach (MusicVideoArtist artist in existing.Artists
.Filter(a => incomingArtists.All(a2 => a2.Name != a.Name)).ToList())
{
existing.Artists.Remove(artist);
updated = await _musicVideoRepository.RemoveArtist(artist) || updated;
}
foreach (MusicVideoArtist artist in incomingArtists
.Filter(a => existing.Artists.All(a2 => a2.Name != a.Name)).ToList())
{
existing.Artists.Add(artist);
updated = await _musicVideoRepository.AddArtist(existing, artist) || updated;
}
return updated;
}
}
@@ -1,4 +1,4 @@
using System.Collections.Immutable;
using System.Collections.Immutable;
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
@@ -203,11 +203,15 @@ public abstract class MediaServerMovieLibraryScanner<TConnectionParameters, TLib
}
// trash movies that are no longer present on the media server
var fileNotFoundItemIds = existingMovies.Keys.Except(incomingItemIds).ToList();
List<int> ids = await movieRepository.FlagFileNotFound(library, fileNotFoundItemIds);
if (!await _scannerProxy.ReindexMediaItems(ids.ToArray(), cancellationToken))
if (MediaServerReconciliationGuard.ShouldFlagMissing(
_logger, library.Name, incomingItemIds.Count, existingMovies.Count))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
var fileNotFoundItemIds = existingMovies.Keys.Except(incomingItemIds).ToList();
List<int> ids = await movieRepository.FlagFileNotFound(library, fileNotFoundItemIds);
if (!await _scannerProxy.ReindexMediaItems(ids.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
return Unit.Default;
@@ -210,11 +210,15 @@ public abstract class MediaServerOtherVideoLibraryScanner<TConnectionParameters,
}
// trash OtherVideo that are no longer present on the media server
var fileNotFoundItemIds = existingOtherVideos.Keys.Except(incomingItemIds).ToList();
List<int> ids = await otherVideoRepository.FlagFileNotFound(library, fileNotFoundItemIds);
if (!await _scannerProxy.ReindexMediaItems(ids.ToArray(), cancellationToken))
if (MediaServerReconciliationGuard.ShouldFlagMissing(
_logger, library.Name, incomingItemIds.Count, existingOtherVideos.Count))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
var fileNotFoundItemIds = existingOtherVideos.Keys.Except(incomingItemIds).ToList();
List<int> ids = await otherVideoRepository.FlagFileNotFound(library, fileNotFoundItemIds);
if (!await _scannerProxy.ReindexMediaItems(ids.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
return Unit.Default;
@@ -0,0 +1,40 @@
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Scanner.Core.Metadata;
// #477: a media-server library sweep computes "gone upstream" as existing.Except(incoming) and flags the
// result FileNotFound. If a successful fetch returns ZERO items (the server is up but mid-restore /
// mid-rebuild, or the library was emptied upstream) then existing.Except([]) is EVERY existing item, so
// the whole library is flagged FileNotFound in one pass. That is data-loss-adjacent: EmptyTrashHandler
// deletes state:FileNotFound rows permanently, and PlayoutSkipMissingItems empties every affected
// collection. An empty incoming set is indistinguishable at scan time from a transient error (both report
// zero), so the safe policy is to refuse the sweep — logged loudly — rather than nuke the library.
//
// This deliberately overrides the degenerate "last item removed => empty incoming => flag" case that a
// partial-deletion sweep would otherwise handle (see #476's cascade, which still fires for the common
// case where survivors are present and only some items are gone). The cost of not flagging a genuinely
// emptied library (stale rows persist until an item returns or the library is removed) is far smaller
// than a one-scan permanent wipe. Ratio-thresholds and projection-failure detection are deferred — see
// docs/decisions.md and the #477 follow-up.
internal static class MediaServerReconciliationGuard
{
public static bool ShouldFlagMissing(
ILogger logger,
string libraryName,
int incomingCount,
int existingCount)
{
if (incomingCount == 0 && existingCount > 0)
{
logger.LogWarning(
"Media server library {Library} returned zero items but {ExistingCount} exist locally; "
+ "skipping the file-not-found sweep to avoid flagging the entire library as missing "
+ "(expected if the server is mid-restore or the library was emptied upstream)",
libraryName,
existingCount);
return false;
}
return true;
}
}
@@ -1,4 +1,4 @@
using System.IO.Abstractions;
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.MediaServer;
@@ -170,12 +170,23 @@ public abstract class MediaServerTelevisionLibraryScanner<TConnectionParameters,
ScanProfiler.LogStatistics(s => _logger.LogInformation("{Profile}", s));
}
if (cleanupFileNotFoundItems)
if (cleanupFileNotFoundItems &&
MediaServerReconciliationGuard.ShouldFlagMissing(
_logger, library.Name, incomingItemIds.Count, existingShows.Count))
{
// trash shows that are no longer present on the media server
var fileNotFoundItemIds = existingShows.Map(s => s.MediaServerItemId).Except(incomingItemIds).ToList();
List<int> ids = await televisionRepository.FlagFileNotFoundShows(library, fileNotFoundItemIds, cancellationToken);
if (!await _scannerProxy.ReindexMediaItems(ids.ToArray(), cancellationToken))
// #476: a show gone from the media server is absent from the incoming list, so the per-show
// loop never visits it and the season/episode sweeps below never run for it. Cascade the flag
// to its descendants so they don't linger (RemoteOnly on Jellyfin/Emby) and keep getting
// scheduled as guaranteed tune-in failures.
List<int> seasonIds = await televisionRepository.FlagFileNotFoundSeasonsForShows(ids, cancellationToken);
List<int> episodeIds = await televisionRepository.FlagFileNotFoundEpisodesForSeasons(seasonIds, cancellationToken);
var reindexIds = ids.Concat(seasonIds).Concat(episodeIds).ToArray();
if (!await _scannerProxy.ReindexMediaItems(reindexIds, cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
@@ -369,7 +380,13 @@ public abstract class MediaServerTelevisionLibraryScanner<TConnectionParameters,
// trash seasons that are no longer present on the media server
var fileNotFoundItemIds = existingSeasons.Map(s => s.MediaServerItemId).Except(incomingItemIds).ToList();
List<int> ids = await televisionRepository.FlagFileNotFoundSeasons(library, fileNotFoundItemIds, cancellationToken);
if (!await _scannerProxy.ReindexMediaItems(ids.ToArray(), cancellationToken))
// #476: a season gone from the media server (while its show remains) is absent from the incoming
// list, so the per-season loop never visits it and the episode sweep in ScanEpisodes never runs
// for it. Cascade the flag to its episodes.
List<int> episodeIds = await televisionRepository.FlagFileNotFoundEpisodesForSeasons(ids, cancellationToken);
if (!await _scannerProxy.ReindexMediaItems(ids.Concat(episodeIds).ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
@@ -49,5 +49,53 @@ public class GetAllChannelsForApiHandlerTests
channel.StreamingMode.ShouldBe("HLS Segmenter");
channel.IsEnabled.ShouldBeFalse();
channel.ShowInEpg.ShouldBeFalse();
// No logo artwork -> null so the SPA renders its generated initials fallback.
channel.Logo.ShouldBeNull();
}
[Test]
public async Task Should_Root_Uploaded_Logo_Url()
{
IChannelRepository repository = Substitute.For<IChannelRepository>();
repository.GetAll(Arg.Any<CancellationToken>())
.Returns([NewChannel(new Artwork { ArtworkKind = ArtworkKind.Logo, Path = "abc123.png" })]);
var handler = new GetAllChannelsForApiHandler(repository);
List<ChannelResponseModel> result = await handler.Handle(new GetAllChannelsForApi(), CancellationToken.None);
// Uploaded logos are addressed as "iptv/logos/{file}"; the browse DTO roots it (leading slash) so
// the SPA's <img src> resolves against the site root regardless of the current SPA route.
result.ShouldHaveSingleItem().Logo.ShouldBe("/iptv/logos/abc123.png");
}
[Test]
public async Task Should_Pass_Through_External_Logo_Url()
{
IChannelRepository repository = Substitute.For<IChannelRepository>();
repository.GetAll(Arg.Any<CancellationToken>())
.Returns([NewChannel(new Artwork { ArtworkKind = ArtworkKind.Logo, Path = "https://example.com/logo.png" })]);
var handler = new GetAllChannelsForApiHandler(repository);
List<ChannelResponseModel> result = await handler.Handle(new GetAllChannelsForApi(), CancellationToken.None);
// An absolute external URL is directly usable and must pass through unchanged (no leading slash added).
result.ShouldHaveSingleItem().Logo.ShouldBe("https://example.com/logo.png");
}
private static Channel NewChannel(params Artwork[] artwork) =>
new(Guid.NewGuid())
{
Id = 7,
Number = "7.1",
SortNumber = 7.1,
Name = "Retro Cartoons",
Group = "Kids",
Categories = "animation",
FFmpegProfile = new FFmpegProfile { Name = "HLS 720p" },
PreferredAudioLanguageCode = "eng",
StreamingMode = StreamingMode.HttpLiveStreamingSegmenter,
IsEnabled = false,
ShowInEpg = false,
Artwork = [.. artwork]
};
}
@@ -240,6 +240,28 @@ public class GetChannelGuideDataHandlerTests
sourceItem.Finish.ShouldBe(BaseTime.AddHours(1));
}
[Test]
public async Task Handle_Should_Root_Uploaded_Logo_And_Null_When_Absent()
{
await using (TvContext context = _db.CreateContext())
{
DomainChannel withLogo = NewChannel("2", "WithLogo", showInEpg: true);
withLogo.Artwork = [new Artwork { ArtworkKind = ArtworkKind.Logo, Path = "abc123.png" }];
DomainChannel withoutLogo = NewChannel("3", "NoLogo", showInEpg: true);
context.Channels.AddRange(withLogo, withoutLogo);
await context.SaveChangesAsync();
}
ChannelGuideResponseModel result = await MakeHandler().Handle(
new GetChannelGuideData(BaseTime, BaseTime.AddDays(1)),
CancellationToken.None);
// Uploaded logo is rooted for the SPA's <img src>; a channel with no logo returns null so the SPA
// renders its generated initials fallback.
result.Channels.Single(c => c.Number == "2").Logo.ShouldBe("/iptv/logos/abc123.png");
result.Channels.Single(c => c.Number == "3").Logo.ShouldBeNull();
}
[Test]
public async Task Handle_Should_Return_Empty_Programmes_For_Channel_Without_Playout()
{
@@ -98,6 +98,45 @@ public class FFmpegProfileHandlerTests
LeftOf(result).ShouldBeAssignableTo<BaseError>();
}
[Test]
public async Task Create_Should_Persist_QsvPreferNativeDecoder_False()
{
// Guards the EF nullable-bool gotcha: QsvPreferNativeDecoder is `bool?` on the domain
// entity with null-means-ON semantics, so an explicit `false` must not get coerced back to
// null/true anywhere between the command and the persisted row.
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result =
await handler.Handle(MakeCreate(1, qsvPreferNativeDecoder: false), CancellationToken.None);
CreateFFmpegProfileResult created = RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.QsvPreferNativeDecoder.ShouldBe(false);
}
[Test]
public async Task Update_Should_Persist_QsvPreferNativeDecoder_False()
{
await SeedProfile(1);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, UpdateFFmpegProfileResult> result =
await handler.Handle(MakeUpdate(1, qsvPreferNativeDecoder: false), CancellationToken.None);
RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.QsvPreferNativeDecoder.ShouldBe(false);
}
private static TR RightOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => throw new AssertionException($"Expected a Right result, got {e}"), Right: r => r);
private static BaseError LeftOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => e, Right: _ => throw new AssertionException("Expected a Left result"));
@@ -145,7 +184,7 @@ public class FFmpegProfileHandlerTests
await context.SaveChangesAsync();
}
private static CreateFFmpegProfile MakeCreate(int resolutionId) =>
private static CreateFFmpegProfile MakeCreate(int resolutionId, bool qsvPreferNativeDecoder = true) =>
new(
"Default",
1,
@@ -176,9 +215,13 @@ public class FFmpegProfileHandlerTests
48_000,
false,
false,
false);
false,
qsvPreferNativeDecoder);
private static UpdateFFmpegProfile MakeUpdate(int id, int resolutionId = 1) =>
private static UpdateFFmpegProfile MakeUpdate(
int id,
int resolutionId = 1,
bool qsvPreferNativeDecoder = true) =>
new(
id,
"Default",
@@ -210,5 +253,6 @@ public class FFmpegProfileHandlerTests
48_000,
false,
false,
false);
false,
qsvPreferNativeDecoder);
}
@@ -32,7 +32,7 @@ public class GetAllHealthCheckResultsForApiHandlerTests
new("Info Check", HealthCheckStatus.Info, "fyi", "info", Option<HealthCheckLink>.None)
};
_healthCheckService.PerformHealthChecks(Arg.Any<CancellationToken>()).Returns(results);
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>()).Returns(results);
List<HealthCheckResponseModel> response =
await _handler.Handle(new GetAllHealthCheckResultsForApi(), CancellationToken.None);
@@ -50,7 +50,7 @@ public class GetAllHealthCheckResultsForApiHandlerTests
new("NA Check", HealthCheckStatus.NotApplicable, "skip", "skip", Option<HealthCheckLink>.None)
};
_healthCheckService.PerformHealthChecks(Arg.Any<CancellationToken>()).Returns(results);
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>()).Returns(results);
List<HealthCheckResponseModel> response =
await _handler.Handle(new GetAllHealthCheckResultsForApi(), CancellationToken.None);
@@ -72,7 +72,7 @@ public class GetAllHealthCheckResultsForApiHandlerTests
Option<HealthCheckLink>.Some(HealthCheckLink.ExternalDoc("https://example.com/docs")))
};
_healthCheckService.PerformHealthChecks(Arg.Any<CancellationToken>()).Returns(results);
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>()).Returns(results);
List<HealthCheckResponseModel> response =
await _handler.Handle(new GetAllHealthCheckResultsForApi(), CancellationToken.None);
@@ -99,7 +99,7 @@ public class GetAllHealthCheckResultsForApiHandlerTests
Option<HealthCheckLink>.Some(HealthCheckLink.AppRoute("/app/trash")))
};
_healthCheckService.PerformHealthChecks(Arg.Any<CancellationToken>()).Returns(results);
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>()).Returns(results);
List<HealthCheckResponseModel> response =
await _handler.Handle(new GetAllHealthCheckResultsForApi(), CancellationToken.None);
@@ -118,7 +118,7 @@ public class GetAllHealthCheckResultsForApiHandlerTests
new("No Link Check", HealthCheckStatus.Pass, "detail", "brief", Option<HealthCheckLink>.None)
};
_healthCheckService.PerformHealthChecks(Arg.Any<CancellationToken>()).Returns(results);
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>()).Returns(results);
List<HealthCheckResponseModel> response =
await _handler.Handle(new GetAllHealthCheckResultsForApi(), CancellationToken.None);
@@ -135,7 +135,7 @@ public class GetAllHealthCheckResultsForApiHandlerTests
new("Blank Brief", HealthCheckStatus.Pass, "detail", string.Empty, Option<HealthCheckLink>.None)
};
_healthCheckService.PerformHealthChecks(Arg.Any<CancellationToken>()).Returns(results);
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>()).Returns(results);
List<HealthCheckResponseModel> response =
await _handler.Handle(new GetAllHealthCheckResultsForApi(), CancellationToken.None);
@@ -146,7 +146,7 @@ public class GetAllHealthCheckResultsForApiHandlerTests
[Test]
public async Task Should_Return_Empty_List_On_Cancellation()
{
_healthCheckService.PerformHealthChecks(Arg.Any<CancellationToken>())
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>())
.Returns<Task<List<HealthCheckResult>>>(_ => throw new TaskCanceledException());
List<HealthCheckResponseModel> response =
@@ -154,4 +154,26 @@ public class GetAllHealthCheckResultsForApiHandlerTests
response.ShouldBeEmpty();
}
[Test]
public async Task Should_Not_Force_Refresh_By_Default()
{
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>())
.Returns(new List<HealthCheckResult>());
await _handler.Handle(new GetAllHealthCheckResultsForApi(), CancellationToken.None);
await _healthCheckService.Received(1).PerformHealthChecks(false, Arg.Any<CancellationToken>());
}
[Test]
public async Task Should_Force_Refresh_When_Requested()
{
_healthCheckService.PerformHealthChecks(Arg.Any<bool>(), Arg.Any<CancellationToken>())
.Returns(new List<HealthCheckResult>());
await _handler.Handle(new GetAllHealthCheckResultsForApi(Refresh: true), CancellationToken.None);
await _healthCheckService.Received(1).PerformHealthChecks(true, Arg.Any<CancellationToken>());
}
}
@@ -42,6 +42,28 @@ public class CreateLocalLibraryHandlerTests
result.IfLeft(error => error.Value.ShouldContain("/media/movies"));
}
// LibraryMediaKind.Mixed exists only for remote (Jellyfin) libraries, where the media server
// classifies each item. No local folder scanner handles it, so a local Mixed library would fail
// every scan forever and log at ERROR on every scheduler tick. The API takes a raw
// LibraryMediaKind, so hiding it from the SPA dropdown is not enforcement (#489 review M1).
[Test]
public async Task Handle_Should_Reject_The_Mixed_Media_Kind_For_Local_Libraries()
{
await SeedLocalMediaSource();
var fileSystem = new MockFileSystem();
fileSystem.Directory.CreateDirectory("/media/music");
CreateLocalLibraryHandler handler = CreateHandler(fileSystem);
Either<BaseError, LocalLibraryViewModel> result = await handler.Handle(
new CreateLocalLibrary("Music", LibraryMediaKind.Mixed, ["/media/music"]),
CancellationToken.None);
result.IsLeft.ShouldBeTrue();
result.IfLeft(error => error.Value.ShouldContain("Mixed"));
}
[Test]
public async Task Handle_Should_List_Only_The_Missing_Paths_When_Mixed()
{
@@ -1,4 +1,5 @@
using ErsatzTV.Application.MediaSources;
using ErsatzTV.Core.Api.MediaSources;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
@@ -66,6 +67,54 @@ public class GetAllMediaSourcesForApiHandlerTests
result.ShouldBeEmpty();
}
[Test]
public async Task Handle_Should_Report_Sentinel_And_Null_LastScan_As_Null()
{
await using (TvContext context = _db.CreateContext())
{
var source = new LocalMediaSource
{
Libraries =
[
new LocalLibrary
{
Name = "Never Scanned Sentinel",
MediaKind = LibraryMediaKind.Movies,
LastScan = new DateTime(1, 1, 1),
Paths = [MakePath("/media/sentinel", 0)]
},
new LocalLibrary
{
Name = "Never Scanned Null",
MediaKind = LibraryMediaKind.Movies,
LastScan = null,
Paths = [MakePath("/media/nullscan", 0)]
},
new LocalLibrary
{
Name = "Really Scanned",
MediaKind = LibraryMediaKind.Movies,
LastScan = new DateTime(2026, 7, 1, 0, 0, 0, DateTimeKind.Utc),
Paths = [MakePath("/media/scanned", 0)]
}
]
};
context.MediaSources.Add(source);
await context.SaveChangesAsync();
}
var handler = new GetAllMediaSourcesForApiHandler(_db.Factory);
var result = await handler.Handle(new GetAllMediaSourcesForApi(), CancellationToken.None);
MediaSourceLibraryResponseModel[] libraries = result.Single().Libraries.ToArray();
libraries.Single(l => l.Name == "Never Scanned Sentinel").LastScan.ShouldBeNull();
libraries.Single(l => l.Name == "Never Scanned Null").LastScan.ShouldBeNull();
libraries.Single(l => l.Name == "Really Scanned").LastScan
.ShouldBe(new DateTime(2026, 7, 1, 0, 0, 0, DateTimeKind.Utc));
}
[Test]
public async Task Handle_Should_Exclude_Unconfigured_And_Not_Synced_Libraries()
{
@@ -0,0 +1,278 @@
using System.IO.Abstractions;
using CliWrap;
using ErsatzTV.Application.Streaming;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Emby;
using ErsatzTV.Core.Interfaces.FFmpeg;
using ErsatzTV.Core.Interfaces.Jellyfin;
using ErsatzTV.Core.Interfaces.Plex;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Interfaces.Streaming;
using ErsatzTV.FFmpeg;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using NSubstitute.Core;
using NSubstitute.Extensions;
using NUnit.Framework;
using Shouldly;
using DomainChannel = ErsatzTV.Core.Domain.Channel;
namespace ErsatzTV.Tests.Application.Streaming;
/// <summary>
/// ersatztv#473: a media-server item that is missing from disk used to fall back to the
/// remote-stream URL unconditionally. It must now be probed first, and an unavailable
/// remote stream must render an error card instead of a real playout process.
/// </summary>
[TestFixture]
public class GetPlayoutItemProcessByChannelNumberHandlerTests
{
private const string ChannelNumber = "1";
private const string JellyfinItemId = "abc123";
private const string EmbyItemId = "def456";
private readonly List<string> _tempFiles = [];
private InMemoryTvContext _db = null!;
private IFFmpegProcessService _ffmpegProcessService = null!;
private IRemoteStreamProber _remoteStreamProber = null!;
[SetUp]
public async Task SetUp()
{
_db = await InMemoryTvContext.CreateAsync();
_remoteStreamProber = Substitute.For<IRemoteStreamProber>();
_ffmpegProcessService = Substitute.For<IFFmpegProcessService>();
// both ForError and ForPlayoutItem must hand back a real value; the handler dereferences them
_ffmpegProcessService.ReturnsForAll(Task.FromResult(Cli.Wrap("ffmpeg")));
_ffmpegProcessService.ReturnsForAll(
Task.FromResult(
new PlayoutItemResult(
Cli.Wrap("ffmpeg"),
Option<GraphicsEngineContext>.None,
Option<int>.None)));
}
[TearDown]
public async Task TearDown()
{
await _db.DisposeAsync();
foreach (string tempFile in _tempFiles)
{
if (File.Exists(tempFile))
{
File.Delete(tempFile);
}
}
}
[Test]
public async Task Should_Render_Error_Card_When_Remote_Stream_Is_Unavailable()
{
DateTimeOffset now = await SeedAll();
_remoteStreamProber.IsAvailable(Arg.Any<string>(), Arg.Any<CancellationToken>()).Returns(false);
Either<BaseError, PlayoutItemProcessModel> result =
await CreateHandler().Handle(Request(now), CancellationToken.None);
result.IsRight.ShouldBeTrue();
await _remoteStreamProber.Received(1)
.IsAvailable($"http://localhost:{Settings.StreamingPort}/media/jellyfin/{JellyfinItemId}", Arg.Any<CancellationToken>());
List<ICall> errorCalls = CallsTo(nameof(IFFmpegProcessService.ForError));
errorCalls.Count.ShouldBe(1);
CallsTo(nameof(IFFmpegProcessService.ForPlayoutItem)).Count.ShouldBe(0);
// the `case PlayoutItemNotAvailableFromMediaServer:` arm exists to surface the real error
// text; without it the error falls to `default:` and the card says "Channel is Offline".
// Assert the message, or the case label is untested dead weight.
object?[] arguments = errorCalls[0].GetArguments();
string errorMessage = arguments.OfType<string>().Single(a => a.Contains("not available"));
errorMessage.ShouldContain($"/media/jellyfin/{JellyfinItemId}");
}
[Test]
public async Task Should_Stream_Remotely_When_Remote_Stream_Is_Available()
{
DateTimeOffset now = await SeedAll();
_remoteStreamProber.IsAvailable(Arg.Any<string>(), Arg.Any<CancellationToken>()).Returns(true);
Either<BaseError, PlayoutItemProcessModel> result =
await CreateHandler().Handle(Request(now), CancellationToken.None);
result.IsRight.ShouldBeTrue();
CallsTo(nameof(IFFmpegProcessService.ForError)).Count.ShouldBe(0);
List<ICall> playoutCalls = CallsTo(nameof(IFFmpegProcessService.ForPlayoutItem));
playoutCalls.Count.ShouldBe(1);
// videoPath is the 7th parameter of ForPlayoutItem
object?[] arguments = playoutCalls[0].GetArguments();
arguments[6].ShouldBe($"http://localhost:{Settings.StreamingPort}/media/jellyfin/{JellyfinItemId}");
}
// the fix changed all three remote-stream branches; Jellyfin above covers one, this pins that a
// second provider is probed too rather than the fix being Jellyfin-only (ersatztv#473 review)
[Test]
public async Task Should_Render_Error_Card_When_Emby_Remote_Stream_Is_Unavailable()
{
DateTimeOffset now = await SeedAll(emby: true);
_remoteStreamProber.IsAvailable(Arg.Any<string>(), Arg.Any<CancellationToken>()).Returns(false);
Either<BaseError, PlayoutItemProcessModel> result =
await CreateHandler().Handle(Request(now), CancellationToken.None);
result.IsRight.ShouldBeTrue();
await _remoteStreamProber.Received(1)
.IsAvailable(
$"http://localhost:{Settings.StreamingPort}/media/emby/{EmbyItemId}",
Arg.Any<CancellationToken>());
CallsTo(nameof(IFFmpegProcessService.ForError)).Count.ShouldBe(1);
CallsTo(nameof(IFFmpegProcessService.ForPlayoutItem)).Count.ShouldBe(0);
}
private List<ICall> CallsTo(string methodName) =>
_ffmpegProcessService.ReceivedCalls()
.Where(c => c.GetMethodInfo().Name == methodName)
.ToList();
private static GetPlayoutItemProcessByChannelNumber Request(DateTimeOffset now) =>
new(
ChannelNumber,
StreamingMode.TransportStream,
now,
StartAtZero: false,
HlsRealtime: true,
ChannelStart: now,
PtsOffset: TimeSpan.Zero,
TargetFramerate: Option<FrameRate>.None,
IsTroubleshooting: false,
FFmpegProfileId: Option<int>.None);
private GetPlayoutItemProcessByChannelNumberHandler CreateHandler()
{
var fileSystem = Substitute.For<IFileSystem>();
fileSystem.File.Exists(Arg.Any<string>()).Returns(false);
return new GetPlayoutItemProcessByChannelNumberHandler(
_db.Factory,
_ffmpegProcessService,
fileSystem,
Substitute.For<IExternalJsonPlayoutItemProvider>(),
Substitute.For<IPlexPathReplacementService>(),
Substitute.For<IJellyfinPathReplacementService>(),
Substitute.For<IEmbyPathReplacementService>(),
Substitute.For<IMediaCollectionRepository>(),
Substitute.For<ITelevisionRepository>(),
Substitute.For<IArtistRepository>(),
Substitute.For<ISongVideoGenerator>(),
Substitute.For<IMusicVideoCreditsGenerator>(),
Substitute.For<IWatermarkSelector>(),
Substitute.For<IGraphicsElementSelector>(),
Substitute.For<IDecoSelector>(),
_remoteStreamProber,
NullLogger<GetPlayoutItemProcessByChannelNumberHandler>.Instance);
}
/// <summary>
/// Seeds ffmpeg/ffprobe config, an ffmpeg profile, a channel, a playout and a playout item
/// covering "now" whose media item is a jellyfin episode missing from disk — or, when
/// <paramref name="emby" /> is set, an emby episode.
/// </summary>
/// <returns>The "now" the request should use.</returns>
private async Task<DateTimeOffset> SeedAll(bool emby = false)
{
string ffmpeg = Path.GetTempFileName();
string ffprobe = Path.GetTempFileName();
_tempFiles.Add(ffmpeg);
_tempFiles.Add(ffprobe);
var now = new DateTimeOffset(2026, 1, 1, 12, 0, 0, TimeSpan.Zero);
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(new ConfigElement { Key = ConfigElementKey.FFmpegPath.Key, Value = ffmpeg });
context.ConfigElements.Add(new ConfigElement { Key = ConfigElementKey.FFprobePath.Key, Value = ffprobe });
var profile = new FFmpegProfile
{
Name = "Test",
Resolution = new Resolution { Name = "1080p", Width = 1920, Height = 1080 }
};
context.FFmpegProfiles.Add(profile);
await context.SaveChangesAsync();
var channel = new DomainChannel(Guid.NewGuid())
{
Number = ChannelNumber,
Name = "Test",
Group = "ErsatzTV",
Categories = string.Empty,
FFmpegProfileId = profile.Id,
StreamSelector = string.Empty,
PreferredAudioLanguageCode = string.Empty,
PreferredAudioTitle = string.Empty,
PreferredSubtitleLanguageCode = string.Empty,
MusicVideoCreditsTemplate = string.Empty,
StreamingMode = StreamingMode.TransportStream,
PlayoutSource = ChannelPlayoutSource.Generated,
PlayoutMode = ChannelPlayoutMode.Continuous
};
context.Channels.Add(channel);
List<EpisodeMetadata> metadata =
[new EpisodeMetadata { Title = "Missing", SortTitle = "Missing", Subtitles = [] }];
List<MediaVersion> versions =
[
new MediaVersion
{
Name = "Main",
Duration = TimeSpan.FromMinutes(30),
MediaFiles = [new MediaFile { Path = "/gone/episode.mkv", PathHash = "gone" }],
Streams = []
}
];
Episode episode = emby
? new EmbyEpisode { ItemId = EmbyItemId, EpisodeMetadata = metadata, MediaVersions = versions }
: new JellyfinEpisode { ItemId = JellyfinItemId, EpisodeMetadata = metadata, MediaVersions = versions };
context.AddRange(episode);
await context.SaveChangesAsync();
var playout = new Playout
{
ChannelId = channel.Id,
ScheduleKind = PlayoutScheduleKind.Classic
};
context.Playouts.Add(playout);
await context.SaveChangesAsync();
context.PlayoutItems.Add(
new PlayoutItem
{
PlayoutId = playout.Id,
MediaItemId = episode.Id,
Start = now.AddMinutes(-5).UtcDateTime,
Finish = now.AddMinutes(25).UtcDateTime,
InPoint = TimeSpan.FromMinutes(5),
OutPoint = TimeSpan.FromMinutes(30),
Watermarks = [],
PlayoutItemWatermarks = [],
GraphicsElements = [],
PlayoutItemGraphicsElements = []
});
await context.SaveChangesAsync();
return now;
}
}
@@ -24,7 +24,7 @@ public class WatermarkHandlerTests
public async Task GetAllWatermarksForApi_Should_Return_All_Watermarks()
{
await SeedWatermark(1, "Bug");
await SeedWatermark(2, "Logo");
await SeedWatermark(2, "Logo", ChannelWatermarkImageSource.ChannelLogo);
var handler = new GetAllWatermarksForApiHandler(_db.Factory);
@@ -32,8 +32,10 @@ public class WatermarkHandlerTests
await handler.Handle(new GetAllWatermarksForApi(), CancellationToken.None);
result.Count.ShouldBe(2);
result.ShouldContain(new WatermarkResponseModel(1, "Bug"));
result.ShouldContain(new WatermarkResponseModel(2, "Logo"));
// Distinct image sources so the assertion proves ImageSource is actually carried through
// the mapper, rather than matching a constant on both rows.
result.ShouldContain(new WatermarkResponseModel(1, "Bug", ChannelWatermarkImageSource.Custom));
result.ShouldContain(new WatermarkResponseModel(2, "Logo", ChannelWatermarkImageSource.ChannelLogo));
}
[Test]
@@ -47,7 +49,10 @@ public class WatermarkHandlerTests
result.ShouldBeEmpty();
}
private async Task SeedWatermark(int id, string name)
private async Task SeedWatermark(
int id,
string name,
ChannelWatermarkImageSource imageSource = ChannelWatermarkImageSource.Custom)
{
await using TvContext context = _db.CreateContext();
context.ChannelWatermarks.Add(new ChannelWatermark
@@ -55,7 +60,7 @@ public class WatermarkHandlerTests
Id = id,
Name = name,
Mode = ChannelWatermarkMode.Permanent,
ImageSource = ChannelWatermarkImageSource.Custom,
ImageSource = imageSource,
Image = "watermark.png",
Location = WatermarkLocation.BottomRight,
Size = WatermarkSize.Scaled,
@@ -0,0 +1,28 @@
using ErsatzTV.Core.Api.Watermarks;
using ErsatzTV.Core.Domain;
using NUnit.Framework;
using Shouldly;
using static ErsatzTV.Application.Watermarks.Mapper;
namespace ErsatzTV.Tests.Application.Watermarks;
[TestFixture]
public class WatermarkMapperTests
{
[Test]
public void ProjectToResponseModel_Should_Carry_ImageSource()
{
var watermark = new ChannelWatermark
{
Id = 7,
Name = "Channel Bug",
ImageSource = ChannelWatermarkImageSource.ChannelLogo
};
WatermarkResponseModel result = ProjectToResponseModel(watermark);
result.Id.ShouldBe(7);
result.Name.ShouldBe("Channel Bug");
result.ImageSource.ShouldBe(ChannelWatermarkImageSource.ChannelLogo);
}
}
@@ -216,7 +216,8 @@ public class FFmpegProfileControllerTests
48_000,
false,
true,
false);
false,
true);
private static CreateFFmpegProfileRequest MakeCreateRequest() =>
new(
@@ -53,7 +53,7 @@ public class HealthControllerTests
_mediator.Send(Arg.Any<GetAllHealthCheckResultsForApi>(), Arg.Any<CancellationToken>())
.Returns(expected);
List<HealthCheckResponseModel> result = await _controller.GetAll(CancellationToken.None);
List<HealthCheckResponseModel> result = await _controller.GetAll(false, CancellationToken.None);
result.ShouldBe(expected);
}
@@ -64,8 +64,21 @@ public class HealthControllerTests
_mediator.Send(Arg.Any<GetAllHealthCheckResultsForApi>(), Arg.Any<CancellationToken>())
.Returns([]);
List<HealthCheckResponseModel> result = await _controller.GetAll(CancellationToken.None);
List<HealthCheckResponseModel> result = await _controller.GetAll(false, CancellationToken.None);
result.ShouldBeEmpty();
}
[Test]
public async Task GetAll_Should_Forward_Refresh_Flag_To_Query()
{
_mediator.Send(Arg.Any<GetAllHealthCheckResultsForApi>(), Arg.Any<CancellationToken>())
.Returns([]);
await _controller.GetAll(true, CancellationToken.None);
await _mediator.Received(1).Send(
Arg.Is<GetAllHealthCheckResultsForApi>(q => q.Refresh),
Arg.Any<CancellationToken>());
}
}
@@ -152,7 +152,8 @@ public class OpenApiSerializerContractTests
"TransportStream",
true,
true,
2);
2,
"/iptv/logos/logo.png");
private static string FindOpenApiDocument()
{
@@ -63,8 +63,8 @@ public class WatermarkControllerTests
{
List<WatermarkResponseModel> models =
[
new WatermarkResponseModel(1, "Corner Logo"),
new WatermarkResponseModel(2, "Ticker")
new WatermarkResponseModel(1, "Corner Logo", ChannelWatermarkImageSource.Custom),
new WatermarkResponseModel(2, "Ticker", ChannelWatermarkImageSource.ChannelLogo)
];
_mediator.Send(Arg.Any<GetAllWatermarksForApi>(), Arg.Any<CancellationToken>())
.Returns(models);
@@ -0,0 +1,151 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Infrastructure;
[TestFixture]
public class DbInitializerChannelBugWatermarkTests
{
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
[Test]
public async Task Initialize_Should_Seed_Channel_Bug_Watermark()
{
await using TvContext context = _db.CreateContext();
await DbInitializer.Initialize(context, CancellationToken.None);
ChannelWatermark watermark = context.ChannelWatermarks.Single(w => w.Name == "Channel Bug");
watermark.Mode.ShouldBe(ChannelWatermarkMode.Permanent);
watermark.ImageSource.ShouldBe(ChannelWatermarkImageSource.ChannelLogo);
watermark.Image.ShouldBeNull();
watermark.Location.ShouldBe(WatermarkLocation.TopLeft);
watermark.Size.ShouldBe(WatermarkSize.Scaled);
watermark.WidthPercent.ShouldBe(5.0);
watermark.HorizontalMarginPercent.ShouldBe(1.0);
watermark.VerticalMarginPercent.ShouldBe(1.0);
watermark.Opacity.ShouldBe(80);
watermark.ZIndex.ShouldBe(0);
watermark.PlaceWithinSourceContent.ShouldBeFalse();
}
// The production instance already has a hand-made "Channel Bug" row with tuned geometry.
// Seeding must adopt it untouched, never overwrite it and never duplicate it.
[Test]
public async Task Initialize_Should_Adopt_Existing_Channel_Bug_Watermark_Untouched()
{
await using TvContext context = _db.CreateContext();
await context.ChannelWatermarks.AddAsync(
new ChannelWatermark
{
Name = "Channel Bug",
Mode = ChannelWatermarkMode.Intermittent,
ImageSource = ChannelWatermarkImageSource.ChannelLogo,
Location = WatermarkLocation.BottomRight,
Size = WatermarkSize.Scaled,
WidthPercent = 12,
HorizontalMarginPercent = 3,
VerticalMarginPercent = 4,
Opacity = 55,
FrequencyMinutes = 10,
DurationSeconds = 20
});
await context.SaveChangesAsync();
await DbInitializer.Initialize(context, CancellationToken.None);
ChannelWatermark watermark = context.ChannelWatermarks.Single(w => w.Name == "Channel Bug");
watermark.Mode.ShouldBe(ChannelWatermarkMode.Intermittent);
watermark.Location.ShouldBe(WatermarkLocation.BottomRight);
watermark.WidthPercent.ShouldBe(12);
watermark.Opacity.ShouldBe(55);
}
[Test]
public async Task Initialize_Should_Be_Idempotent()
{
await using TvContext context = _db.CreateContext();
await DbInitializer.Initialize(context, CancellationToken.None);
await DbInitializer.Initialize(context, CancellationToken.None);
context.ChannelWatermarks.Count(w => w.Name == "Channel Bug").ShouldBe(1);
}
// No IsSystem flag exists on ChannelWatermark, and Initialize runs on every startup, so without
// a seed marker a deliberate delete would be undone forever.
[Test]
public async Task Initialize_Should_Not_Resurrect_A_Deleted_Preset()
{
await using TvContext context = _db.CreateContext();
await DbInitializer.Initialize(context, CancellationToken.None);
ChannelWatermark seeded = context.ChannelWatermarks.Single(w => w.Name == "Channel Bug");
context.ChannelWatermarks.Remove(seeded);
await context.SaveChangesAsync();
await DbInitializer.Initialize(context, CancellationToken.None);
context.ChannelWatermarks.Any(w => w.Name == "Channel Bug").ShouldBeFalse();
}
// The production sequence is ADOPT (an existing hand-made row) and then, possibly, delete —
// not seed-then-delete. The marker is written on the adopt path too, so this must not resurrect.
[Test]
public async Task Initialize_Should_Not_Resurrect_An_Adopted_Preset_After_Deletion()
{
await using TvContext context = _db.CreateContext();
await context.ChannelWatermarks.AddAsync(
new ChannelWatermark
{
Name = "Channel Bug",
Mode = ChannelWatermarkMode.Permanent,
ImageSource = ChannelWatermarkImageSource.ChannelLogo,
Location = WatermarkLocation.BottomRight,
Size = WatermarkSize.Scaled,
WidthPercent = 12,
Opacity = 55
});
await context.SaveChangesAsync();
// First run adopts the existing row and records the marker.
await DbInitializer.Initialize(context, CancellationToken.None);
ChannelWatermark adopted = context.ChannelWatermarks.Single(w => w.Name == "Channel Bug");
context.ChannelWatermarks.Remove(adopted);
await context.SaveChangesAsync();
await DbInitializer.Initialize(context, CancellationToken.None);
context.ChannelWatermarks.Any(w => w.Name == "Channel Bug").ShouldBeFalse();
}
[Test]
public async Task Initialize_Should_Stamp_The_Preset_On_Freshly_Seeded_Templates()
{
await using TvContext context = _db.CreateContext();
await DbInitializer.Initialize(context, CancellationToken.None);
ChannelWatermark watermark = context.ChannelWatermarks.Single(w => w.Name == "Channel Bug");
List<ChannelTemplate> templates = context.ChannelTemplates.Where(t => t.IsSystem).ToList();
// Assert the collection is non-empty FIRST: a bare foreach over zero rows passes vacuously,
// so this test would stay green if template seeding silently bailed out.
templates.Count.ShouldBe(2);
foreach (ChannelTemplate template in templates)
{
template.WatermarkId.ShouldBe(watermark.Id);
}
}
}
@@ -0,0 +1,107 @@
using ErsatzTV.Core.Health;
using ErsatzTV.Core.Health.Checks;
using ErsatzTV.Infrastructure.Health;
using LanguageExt;
using MediatR;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Infrastructure.Health;
[TestFixture]
public class HealthCheckServiceTests
{
private IFFmpegVersionHealthCheck _representativeCheck = null!;
private IMemoryCache _memoryCache = null!;
private IMediator _mediator = null!;
private HealthCheckService _service = null!;
[SetUp]
public void SetUp()
{
_memoryCache = new MemoryCache(new MemoryCacheOptions());
_mediator = Substitute.For<IMediator>();
_representativeCheck = PassCheck<IFFmpegVersionHealthCheck>();
// All 14 checks run together on a single PerformHealthChecks call, so the representative
// check's invocation count equals the number of actual (non-cached) runs.
_service = new HealthCheckService(
PassCheck<IMacOsConfigFolderHealthCheck>(),
_representativeCheck,
PassCheck<IFFmpegCapabilitiesHealthCheck>(),
PassCheck<IFFmpegReportsHealthCheck>(),
PassCheck<IHardwareAccelerationHealthCheck>(),
PassCheck<IMovieMetadataHealthCheck>(),
PassCheck<IEpisodeMetadataHealthCheck>(),
PassCheck<IZeroDurationHealthCheck>(),
PassCheck<IFileNotFoundHealthCheck>(),
PassCheck<IUnavailableHealthCheck>(),
PassCheck<IVaapiDriverHealthCheck>(),
PassCheck<IUnifiedDockerHealthCheck>(),
PassCheck<IDowngradeHealthCheck>(),
PassCheck<IEmptyScheduleHealthCheck>(),
_memoryCache,
_mediator,
NullLogger<HealthCheckService>.Instance);
}
[TearDown]
public void TearDown() => (_memoryCache as MemoryCache)?.Dispose();
[Test]
public async Task PerformHealthChecks_Should_Serve_Cache_Within_Window()
{
await _service.PerformHealthChecks(false, CancellationToken.None);
await _service.PerformHealthChecks(false, CancellationToken.None);
// Second call is served from cache: checks are not re-run, summary is not re-published.
await _representativeCheck.Received(1).Check(Arg.Any<CancellationToken>());
await _mediator.Received(1).Publish(Arg.Any<HealthCheckSummary>(), Arg.Any<CancellationToken>());
}
[Test]
public async Task PerformHealthChecks_Should_Return_Cached_Instance_Within_Window()
{
List<HealthCheckResult> first = await _service.PerformHealthChecks(false, CancellationToken.None);
List<HealthCheckResult> second = await _service.PerformHealthChecks(false, CancellationToken.None);
second.ShouldBeSameAs(first);
}
[Test]
public async Task PerformHealthChecks_Should_Bypass_Cache_On_Force_Refresh()
{
await _service.PerformHealthChecks(false, CancellationToken.None);
await _service.PerformHealthChecks(true, CancellationToken.None);
// Force refresh re-runs the checks and re-publishes, regardless of the warm cache.
await _representativeCheck.Received(2).Check(Arg.Any<CancellationToken>());
await _mediator.Received(2).Publish(Arg.Any<HealthCheckSummary>(), Arg.Any<CancellationToken>());
}
[Test]
public async Task Force_Refresh_Should_Repopulate_Cache_For_Later_Reads()
{
// A forced run should warm the cache so a following non-forced read is a hit.
await _service.PerformHealthChecks(true, CancellationToken.None);
await _service.PerformHealthChecks(false, CancellationToken.None);
await _representativeCheck.Received(1).Check(Arg.Any<CancellationToken>());
}
private static T PassCheck<T>() where T : class, IHealthCheck
{
var check = Substitute.For<T>();
check.Check(Arg.Any<CancellationToken>())
.Returns(_ => new HealthCheckResult(
typeof(T).Name,
HealthCheckStatus.Pass,
"ok",
"ok",
Option<HealthCheckLink>.None));
return check;
}
}
@@ -0,0 +1,220 @@
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.Core.Interfaces.Plex;
using ErsatzTV.Core.Interfaces.Streaming;
using ErsatzTV.Core.Plex;
using ErsatzTV.Core.Streaming;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming;
using ErsatzTV.Tests.Support;
using LanguageExt;
using Newtonsoft.Json;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using static LanguageExt.Prelude;
using DomainChannel = ErsatzTV.Core.Domain.Channel;
namespace ErsatzTV.Tests.Infrastructure.Streaming;
/// <summary>
/// ersatztv#480: external-JSON playout channels build their own <c>/media/plex/...</c> URL and used
/// to hand it to ffmpeg unprobed, so the #473 class survived there — a media item gone from the
/// server 404s under ffmpeg (exit 8) and the same dead item is re-selected for its whole slot. The
/// remote path must now probe the URL via <see cref="IRemoteStreamProber" /> first, failing closed
/// only on a media-server (redirected) 404. The fail-open contract itself is pinned by
/// <c>HttpRemoteStreamProberTests</c>; these tests pin that the external-JSON path routes through it.
/// </summary>
[TestFixture]
public class ExternalJsonPlayoutItemProviderTests
{
private const string ServerKey = "server1";
private const string ClientIdentifier = "client1";
private const string PlexFile = "shows/example/s01e01.mkv";
private const string ScheduleFile = "/config/externaljson/channel1.json";
private const string LocalPath = "/plex/shows/example/s01e01.mkv";
private readonly DateTimeOffset _now = new(2026, 1, 1, 12, 0, 0, TimeSpan.Zero);
private InMemoryTvContext _db = null!;
private IFileSystem _fileSystem = null!;
private ILocalStatisticsProvider _localStatisticsProvider = null!;
private IPlexPathReplacementService _plexPathReplacementService = null!;
private IPlexSecretStore _plexSecretStore = null!;
private IPlexServerApiClient _plexServerApiClient = null!;
private IRemoteStreamProber _remoteStreamProber = null!;
private DomainChannel _channel = null!;
private int _serverId;
[SetUp]
public async Task SetUp()
{
_db = await InMemoryTvContext.CreateAsync();
_fileSystem = Substitute.For<IFileSystem>();
_localStatisticsProvider = Substitute.For<ILocalStatisticsProvider>();
_plexPathReplacementService = Substitute.For<IPlexPathReplacementService>();
_plexSecretStore = Substitute.For<IPlexSecretStore>();
_plexServerApiClient = Substitute.For<IPlexServerApiClient>();
_remoteStreamProber = Substitute.For<IRemoteStreamProber>();
await SeedAsync();
// the JSON schedule exists, but the resolved local file does not — so the provider takes the
// remote-stream branch, which is the one #480 fixes
_fileSystem.File.Exists(ScheduleFile).Returns(true);
_fileSystem.File.Exists(LocalPath).Returns(false);
_fileSystem.File.ReadAllTextAsync(ScheduleFile, Arg.Any<CancellationToken>()).Returns(ScheduleJson());
_plexPathReplacementService
.GetReplacementPlexPath(Arg.Any<int>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(LocalPath);
_plexSecretStore.GetServerAuthToken(Arg.Any<string>())
.Returns(Option<PlexServerAuthToken>.Some(new PlexServerAuthToken(ClientIdentifier, "token")));
}
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private string ExpectedUrl => $"http://localhost:{Settings.StreamingPort}/media/plex/{_serverId}/{PlexFile}";
[Test]
public async Task Should_Return_Not_Available_Error_When_Remote_Stream_Is_Unavailable()
{
_remoteStreamProber.IsAvailable(Arg.Any<string>(), Arg.Any<CancellationToken>()).Returns(false);
Either<BaseError, PlayoutItemWithPath> result =
await CreateProvider().CheckForExternalJson(_channel, _now, "/ffprobe", CancellationToken.None);
// fails closed with the error the handler renders as a real "not available" card (not exit 8)
result.IsLeft.ShouldBeTrue();
result.LeftToSeq().Head.ShouldBeOfType<PlayoutItemNotAvailableFromMediaServer>();
await _remoteStreamProber.Received(1).IsAvailable(ExpectedUrl, Arg.Any<CancellationToken>());
// probing first means a gone item never pays for the plex metadata round-trip
await _plexServerApiClient.DidNotReceive().GetEpisodeMetadataAndStatistics(
Arg.Any<int>(),
Arg.Any<string>(),
Arg.Any<PlexConnection>(),
Arg.Any<PlexServerAuthToken>());
}
[Test]
public async Task Should_Return_Playout_Item_With_Url_When_Remote_Stream_Is_Available()
{
_remoteStreamProber.IsAvailable(Arg.Any<string>(), Arg.Any<CancellationToken>()).Returns(true);
_plexServerApiClient.GetEpisodeMetadataAndStatistics(
Arg.Any<int>(),
Arg.Any<string>(),
Arg.Any<PlexConnection>(),
Arg.Any<PlexServerAuthToken>())
.Returns(
Right<BaseError, Tuple<EpisodeMetadata, MediaVersion>>(
Tuple(new EpisodeMetadata(), new MediaVersion { Name = "Main", Streams = [] })));
Either<BaseError, PlayoutItemWithPath> result =
await CreateProvider().CheckForExternalJson(_channel, _now, "/ffprobe", CancellationToken.None);
// fail-open: an available remote stream still yields the playable /media/plex URL
result.IsRight.ShouldBeTrue();
result.RightToSeq().Head.Path.ShouldBe(ExpectedUrl);
await _remoteStreamProber.Received(1).IsAvailable(ExpectedUrl, Arg.Any<CancellationToken>());
}
private ExternalJsonPlayoutItemProvider CreateProvider() =>
new(
_db.Factory,
_fileSystem,
_plexPathReplacementService,
_plexServerApiClient,
_plexSecretStore,
_localStatisticsProvider,
_remoteStreamProber,
Microsoft.Extensions.Logging.Abstractions.NullLogger<ExternalJsonPlayoutItemProvider>.Instance);
private static string ScheduleJson() =>
JsonConvert.SerializeObject(
new ExternalJsonChannel
{
// one program, a 1h window starting 1 minute before "now", so it is the current item
StartTime = "2026-01-01T11:59:00Z",
Programs =
[
new ExternalJsonProgram
{
Type = "episode",
Duration = 3_600_000,
PlexFile = PlexFile,
File = "/plexserver/shows/example/s01e01.mkv",
ServerKey = ServerKey,
RatingKey = "12345",
Title = "Example",
ShowTitle = "Example Show",
Season = 1,
Episode = 1
}
]
});
private async Task SeedAsync()
{
await using TvContext context = _db.CreateContext();
var source = new PlexMediaSource
{
ServerName = ServerKey,
ClientIdentifier = ClientIdentifier,
ProductVersion = "1",
Platform = "Linux",
PlatformVersion = "1",
PathReplacements = [],
Connections = [new PlexConnection { IsActive = true, Uri = "http://plex:32400" }],
Libraries =
[
new PlexLibrary
{
Name = "Plex Movies",
MediaKind = LibraryMediaKind.Movies,
Key = "1",
ShouldSyncItems = true,
Paths = [new LibraryPath { Path = "/plex" }]
}
]
};
await context.PlexMediaSources.AddAsync(source);
await context.SaveChangesAsync();
_serverId = source.Id;
_channel = new DomainChannel(Guid.NewGuid())
{
Number = "1",
Name = "External JSON",
Group = "ErsatzTV",
Categories = string.Empty,
StreamSelector = string.Empty,
PreferredAudioLanguageCode = string.Empty,
PreferredAudioTitle = string.Empty,
PreferredSubtitleLanguageCode = string.Empty,
MusicVideoCreditsTemplate = string.Empty,
StreamingMode = StreamingMode.TransportStream,
PlayoutSource = ChannelPlayoutSource.Generated,
PlayoutMode = ChannelPlayoutMode.Continuous
};
context.Channels.Add(_channel);
await context.SaveChangesAsync();
context.Playouts.Add(
new Playout
{
ChannelId = _channel.Id,
ScheduleKind = PlayoutScheduleKind.ExternalJson,
ScheduleFile = ScheduleFile
});
await context.SaveChangesAsync();
}
}
@@ -0,0 +1,573 @@
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Jellyfin;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Jellyfin;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Data.Repositories;
using ErsatzTV.Scanner.Core.Interfaces;
using ErsatzTV.Scanner.Core.Jellyfin;
using ErsatzTV.Tests.Support;
using LanguageExt;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using IFileSystem = System.IO.Abstractions.IFileSystem;
using Unit = LanguageExt.Unit;
using static LanguageExt.Prelude;
namespace ErsatzTV.Tests.Integration;
// End-to-end regression for ersatztv#488. Unlike the existing MediaServer*LibraryScanner tests (which
// substitute every repository, and therefore could never have exhibited the null-navigation bug — see the
// mocked GetOrAddFolder in MovieFolderScannerTests), this test wires the REAL LibraryRepository /
// ArtistRepository / MusicVideoRepository against in-memory SQLite so the actual crash path runs. The
// deviation from the mock-and-verify house style is deliberate and required: a substituted
// ILibraryRepository cannot exhibit the defect this issue is about.
[TestFixture]
public class JellyfinMusicVideoLibraryScannerTests
{
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
[Test]
public async Task ScanLibrary_Should_Complete_And_Create_Artist_And_MusicVideo_Rows()
{
int libraryPathId = await SeedLibraryPath("/data/music");
// the remote-path shape that used to crash: Paths is populated, LibraryFolders is null
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
var library = new JellyfinLibrary
{
Id = 42,
MediaSourceId = 1,
ItemId = "lib15",
ShouldSyncItems = true,
Paths = new List<LibraryPath> { libraryPath }
};
const string VideoPath = "/data/music/artist1/song1.mkv";
MusicVideo incoming = BuildIncoming(VideoPath, artistName: "Artist 1", title: "Song 1");
var apiClient = Substitute.For<IJellyfinApiClient>();
apiClient.GetMusicVideoLibraryItems(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<JellyfinLibrary>())
.Returns(OneItem(incoming));
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>();
pathReplacement
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>())
.Returns(ci => ci.ArgAt<string>(1));
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
var scanner = new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
pathReplacement,
mediaSourceRepository,
new ArtistRepository(_db.Factory),
new MusicVideoRepository(_db.Factory),
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
Substitute.For<IMetadataRepository>(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
Either<BaseError, Unit> result =
await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None);
// the scan runs to completion instead of crashing on GetOrAddFolder
result.IsRight.ShouldBeTrue(result.Match(Right: _ => "", Left: e => e.Value));
await using TvContext context = _db.CreateContext();
// Artist row created from the incoming metadata
List<Artist> artists = await context.Artists
.Include(a => a.ArtistMetadata)
.Where(a => a.LibraryPathId == libraryPathId)
.ToListAsync();
artists.Count.ShouldBe(1);
artists[0].ArtistMetadata.Single().Title.ShouldBe("Artist 1");
// MusicVideo row created, wired to a real LibraryFolder (proving GetOrAddFolder returned a persisted row)
List<MusicVideo> musicVideos = await context.MusicVideos
.Include(mv => mv.MediaVersions)
.ThenInclude(v => v.MediaFiles)
.Where(mv => mv.LibraryPathId == libraryPathId)
.ToListAsync();
musicVideos.Count.ShouldBe(1);
MediaFile file = musicVideos[0].MediaVersions.Single().MediaFiles.Single();
file.Path.ShouldBe(VideoPath);
file.LibraryFolderId.ShouldNotBeNull();
// the folder was created by the fixed GetOrAddFolder
LibraryFolder folder = await context.LibraryFolders.SingleAsync(f => f.Id == file.LibraryFolderId);
folder.Path.ShouldBe("/data/music/artist1");
folder.LibraryPathId.ShouldBe(libraryPathId);
}
// ersatztv#494: a music video removed from Jellyfin must be removed from ErsatzTV on the next scan.
[Test]
public async Task ScanLibrary_Should_Remove_MusicVideo_Missing_From_Jellyfin()
{
int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// first scan seeds two music videos
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"),
() => BuildIncoming("/data/music/artist2/gone.mkv", "Artist 2", "Gone")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await MusicVideoPaths(id)).Count.ShouldBe(2);
int goneId = await MusicVideoId("/data/music/artist2/gone.mkv");
// second scan: only "keep" is still present upstream
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
List<string> paths = await MusicVideoPaths(id);
paths.ShouldBe(new[] { "/data/music/artist1/keep.mkv" });
await scannerProxy.Received().RemoveMediaItems(
Arg.Is<int[]>(ids => ids.Contains(goneId)),
Arg.Any<CancellationToken>());
}
// ersatztv#494: an artist left with zero music videos is cleaned up.
[Test]
public async Task ScanLibrary_Should_Cleanup_Empty_Artist()
{
int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"),
() => BuildIncoming("/data/music/artist2/gone.mkv", "Artist 2", "Gone")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await ArtistTitles(id)).Count.ShouldBe(2);
// "Artist 2" loses its only music video
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await ArtistTitles(id)).ShouldBe(new[] { "Artist 1" });
}
// ersatztv#494 (Done-when 3): the music-video sweep must never delete a Movie or Show that shares the
// same LibraryPath — the cross-delete risk is a LibraryPathId property, not a Mixed-library property.
[Test]
public async Task ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath()
{
int id = await SeedLibraryPath("/data/mixed");
await using (TvContext seedContext = _db.CreateContext())
{
seedContext.Add(new Movie { LibraryPathId = id, MovieMetadata = new List<MovieMetadata>() });
seedContext.Add(new Show { LibraryPathId = id, ShowMetadata = new List<ShowMetadata>() });
await seedContext.SaveChangesAsync();
}
JellyfinLibrary library = BuildLibrary(id, "/data/mixed");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// seed one music video under the same LibraryPath
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/mixed/song.mkv", "Artist 1", "Song")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
// next scan drops "song.mkv" and adds "song2.mkv" — the sweep removes song.mkv
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/mixed/song2.mkv", "Artist 1", "Song 2")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
(await context.Movies.CountAsync(m => m.LibraryPathId == id)).ShouldBe(1);
(await context.Shows.CountAsync(s => s.LibraryPathId == id)).ShouldBe(1);
(await MusicVideoPaths(id)).ShouldBe(new[] { "/data/mixed/song2.mkv" });
}
// ersatztv#477 guard: a successful-but-empty fetch must NOT wipe the library. Negative control — if the
// sweep were ungated, an empty incoming set would remove every existing music video.
[Test]
public async Task ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items()
{
int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
// Jellyfin returns zero items (mid-restore / transient) — the sweep must be skipped
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi());
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await MusicVideoPaths(id)).ShouldBe(new[] { "/data/music/artist1/keep.mkv" });
await scannerProxy.DidNotReceive().RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>());
}
// #493 session field data: a MIXED Jellyfin library runs the music-video arm with a legitimately EMPTY
// incoming set on every scan while Movies/Shows exist under the same LibraryPath (the real "Standup" case).
// If existing were computed by LibraryPathId alone, existing.Except([]) would wipe the movies/episodes; the
// MusicVideo-joined FindMusicVideoPaths (not just the empty-fetch guard) is what protects them. Seed a Movie
// that even carries a MediaFile path, to prove the sweep never touches a non-music-video row.
[Test]
public async Task ScanLibrary_Should_Not_Touch_Movies_Or_Shows_When_No_MusicVideos_Present()
{
int id = await SeedLibraryPath("/data/standup");
await using (TvContext seedContext = _db.CreateContext())
{
seedContext.Add(new Movie
{
LibraryPathId = id,
MovieMetadata = new List<MovieMetadata>(),
MediaVersions = new List<MediaVersion>
{
new()
{
MediaFiles = new List<MediaFile>
{
new() { Path = "/data/standup/show.mkv", PathHash = "hash-standup" }
},
Streams = new List<MediaStream>()
}
}
});
seedContext.Add(new Show { LibraryPathId = id, ShowMetadata = new List<ShowMetadata>() });
await seedContext.SaveChangesAsync();
}
JellyfinLibrary library = BuildLibrary(id, "/data/standup");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// the music-video arm returns zero incoming — steady state for a mixed library
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi());
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
(await context.Movies.CountAsync(m => m.LibraryPathId == id)).ShouldBe(1);
(await context.Shows.CountAsync(s => s.LibraryPathId == id)).ShouldBe(1);
await scannerProxy.DidNotReceive().RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>());
}
// ersatztv#497: metadata edits (genres/tags/studios/artists) made in Jellyfin to an EXISTING music video
// must reach ErsatzTV on the next scan. Before the fix, UpdateMetadata copied only scalar fields, so the
// update path silently dropped every child collection — add-new AND remove-stale. This is an interaction
// test: the repositories are substituted and GetOrAdd returns a canned existing item so we can verify the
// scanner issues the exact reconcile calls. (The real-DB double-scan approach can't drive this here: the
// in-memory harness shares ONE SQLite connection across contexts, and mid-scan GetOrAdd's
// `MediaVersions.First().MediaFiles.First().Path` predicate mis-resolves once the existing item carries
// metadata children — a harness-only quirk; prod uses per-context pooled connections and looks music videos
// up by that predicate only because they carry no server ItemId. See the #497 close comment.)
// Non-vacuous: reverting the Reconcile* calls in UpdateMetadata drops every Received() below.
[Test]
public async Task ScanLibrary_Should_Reconcile_Metadata_Collections_On_Rescan_Of_Existing_Item()
{
JellyfinLibrary library = BuildLibrary(1, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
const string VideoPath = "/data/music/artist1/song1.mkv";
// the EXISTING item already in ErsatzTV, with the collections Jellyfin first gave it
var existing = new MusicVideo
{
Id = 7,
ArtistId = 3,
MediaVersions = new List<MediaVersion>
{
new() { MediaFiles = new List<MediaFile> { new() { Path = VideoPath } }, Streams = new List<MediaStream>() }
},
MusicVideoMetadata = new List<MusicVideoMetadata>
{
new()
{
Id = 11,
Genres = new List<Genre> { new() { Name = "Synthwave" }, new() { Name = "Retro" } },
Tags = new List<Tag> { new() { Name = "KeepTag" }, new() { Name = "DropTag" } },
Studios = new List<Studio> { new() { Name = "OldStudio" } },
Artists = new List<MusicVideoArtist> { new() { Name = "Artist 1" }, new() { Name = "Featured X" } }
}
}
};
var artistRepository = Substitute.For<IArtistRepository>();
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>())
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() }));
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>());
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository
.GetOrAdd(Arg.Any<Artist>(), Arg.Any<LibraryPath>(), Arg.Any<LibraryFolder>(), Arg.Any<string>())
.Returns(Right<BaseError, MediaItemScanResult<MusicVideo>>(
new MediaItemScanResult<MusicVideo>(existing) { IsAdded = false }));
musicVideoRepository.FindMusicVideoPaths(Arg.Any<LibraryPath>())
.Returns(new List<string> { VideoPath }.AsEnumerable());
musicVideoRepository.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Any<Genre>()).Returns(true);
musicVideoRepository.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Any<Tag>()).Returns(true);
musicVideoRepository.AddStudio(Arg.Any<MusicVideoMetadata>(), Arg.Any<Studio>()).Returns(true);
musicVideoRepository.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Any<MusicVideoArtist>()).Returns(true);
musicVideoRepository.RemoveArtist(Arg.Any<MusicVideoArtist>()).Returns(true);
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>()).Returns(false);
metadataRepository.RemoveGenre(Arg.Any<Genre>()).Returns(true);
metadataRepository.RemoveTag(Arg.Any<Tag>()).Returns(true);
metadataRepository.RemoveStudio(Arg.Any<Studio>()).Returns(true);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
JellyfinMusicVideoLibraryScanner scanner = BuildScannerWith(
FakeApi(() => BuildIncoming(
VideoPath, "Artist 1", "Song 1",
genres: new[] { "Synthwave", "Vaporwave" },
tags: new[] { "KeepTag", "NewTag" },
studios: new[] { "NewStudio" },
artists: new[] { "Artist 1", "Featured Y" })),
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository);
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
// remove-stale: Retro / DropTag / OldStudio / "Featured X" gone; kept items are NOT removed
await metadataRepository.Received(1).RemoveGenre(Arg.Is<Genre>(g => g.Name == "Retro"));
await metadataRepository.DidNotReceive().RemoveGenre(Arg.Is<Genre>(g => g.Name == "Synthwave"));
await metadataRepository.Received(1).RemoveTag(Arg.Is<Tag>(t => t.Name == "DropTag"));
await metadataRepository.DidNotReceive().RemoveTag(Arg.Is<Tag>(t => t.Name == "KeepTag"));
await metadataRepository.Received(1).RemoveStudio(Arg.Is<Studio>(s => s.Name == "OldStudio"));
await musicVideoRepository.Received(1).RemoveArtist(Arg.Is<MusicVideoArtist>(a => a.Name == "Featured X"));
await musicVideoRepository.DidNotReceive().RemoveArtist(Arg.Is<MusicVideoArtist>(a => a.Name == "Artist 1"));
// add-new: Vaporwave / NewTag / NewStudio / "Featured Y" added; kept items are NOT re-added
await musicVideoRepository.Received(1)
.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Is<Genre>(g => g.Name == "Vaporwave"));
await musicVideoRepository.DidNotReceive()
.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Is<Genre>(g => g.Name == "Synthwave"));
await musicVideoRepository.Received(1)
.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Is<Tag>(t => t.Name == "NewTag"));
await musicVideoRepository.DidNotReceive()
.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Is<Tag>(t => t.Name == "KeepTag"));
await musicVideoRepository.Received(1)
.AddStudio(Arg.Any<MusicVideoMetadata>(), Arg.Is<Studio>(s => s.Name == "NewStudio"));
await musicVideoRepository.Received(1)
.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Is<MusicVideoArtist>(a => a.Name == "Featured Y"));
await musicVideoRepository.DidNotReceive()
.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Is<MusicVideoArtist>(a => a.Name == "Artist 1"));
}
private JellyfinMusicVideoLibraryScanner BuildScannerWith(
IJellyfinApiClient apiClient,
IArtistRepository artistRepository,
IMusicVideoRepository musicVideoRepository,
ILibraryRepository libraryRepository,
IMetadataRepository metadataRepository)
{
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>();
pathReplacement
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>())
.Returns(ci => ci.ArgAt<string>(1));
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
return new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
pathReplacement,
mediaSourceRepository,
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository,
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
}
private async Task<int> MusicVideoId(string path)
{
await using TvContext context = _db.CreateContext();
List<MusicVideo> musicVideos = await context.MusicVideos
.Include(mv => mv.MediaVersions)
.ThenInclude(v => v.MediaFiles)
.ToListAsync();
return musicVideos
.Single(mv => mv.MediaVersions.Single().MediaFiles.Single().Path == path)
.Id;
}
private async Task<List<string>> MusicVideoPaths(int libraryPathId)
{
await using TvContext context = _db.CreateContext();
List<MusicVideo> musicVideos = await context.MusicVideos
.Include(mv => mv.MediaVersions)
.ThenInclude(v => v.MediaFiles)
.Where(mv => mv.LibraryPathId == libraryPathId)
.ToListAsync();
return musicVideos
.Select(mv => mv.MediaVersions.Single().MediaFiles.Single().Path)
.OrderBy(p => p)
.ToList();
}
private async Task<List<string>> ArtistTitles(int libraryPathId)
{
await using TvContext context = _db.CreateContext();
List<Artist> artists = await context.Artists
.Include(a => a.ArtistMetadata)
.Where(a => a.LibraryPathId == libraryPathId)
.ToListAsync();
return artists.Select(a => a.ArtistMetadata.Single().Title).OrderBy(t => t).ToList();
}
private (JellyfinMusicVideoLibraryScanner Scanner, IScannerProxy ScannerProxy) BuildScanner(
IJellyfinApiClient apiClient)
{
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>();
pathReplacement
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>())
.Returns(ci => ci.ArgAt<string>(1));
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
var scanner = new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
pathReplacement,
mediaSourceRepository,
new ArtistRepository(_db.Factory),
new MusicVideoRepository(_db.Factory),
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
Substitute.For<IMetadataRepository>(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
return (scanner, scannerProxy);
}
// Each Func builds a fresh MusicVideo so the async stream can be re-enumerated across ScanLibrary calls
// (an IAsyncEnumerable iterator is single-use, and the scanner mutates the incoming item).
private static IJellyfinApiClient FakeApi(params Func<MusicVideo>[] items)
{
var apiClient = Substitute.For<IJellyfinApiClient>();
apiClient.GetMusicVideoLibraryItems(Arg.Any<string>(), Arg.Any<string>(), Arg.Any<JellyfinLibrary>())
.Returns(_ => Items(items));
return apiClient;
}
private static async IAsyncEnumerable<Tuple<MusicVideo, int>> Items(Func<MusicVideo>[] items)
{
foreach (Func<MusicVideo> item in items)
{
yield return new Tuple<MusicVideo, int>(item(), items.Length);
}
await Task.CompletedTask;
}
private static JellyfinLibrary BuildLibrary(int libraryPathId, string path)
{
var libraryPath = new LibraryPath { Id = libraryPathId, Path = path, LibraryFolders = null };
return new JellyfinLibrary
{
Id = 42,
MediaSourceId = 1,
ItemId = "lib15",
Name = "Music",
ShouldSyncItems = true,
Paths = new List<LibraryPath> { libraryPath }
};
}
private static MusicVideo BuildIncoming(
string path,
string artistName,
string title,
IEnumerable<string> genres = null,
IEnumerable<string> tags = null,
IEnumerable<string> studios = null,
IEnumerable<string> artists = null) =>
new()
{
MediaVersions = new List<MediaVersion>
{
new()
{
MediaFiles = new List<MediaFile> { new() { Path = path } },
Streams = new List<MediaStream>(),
Chapters = new List<MediaChapter>()
}
},
MusicVideoMetadata = new List<MusicVideoMetadata>
{
new()
{
Title = title,
Genres = (genres ?? Enumerable.Empty<string>()).Select(g => new Genre { Name = g }).ToList(),
Tags = (tags ?? Enumerable.Empty<string>()).Select(t => new Tag { Name = t }).ToList(),
Studios = (studios ?? Enumerable.Empty<string>()).Select(s => new Studio { Name = s }).ToList(),
Artists = (artists ?? new[] { artistName })
.Select(a => new MusicVideoArtist { Name = a }).ToList()
}
}
};
private static async IAsyncEnumerable<Tuple<MusicVideo, int>> OneItem(MusicVideo musicVideo)
{
yield return new Tuple<MusicVideo, int>(musicVideo, 1);
await Task.CompletedTask;
}
private async Task<int> SeedLibraryPath(string path)
{
await using TvContext context = _db.CreateContext();
var libraryPath = new LibraryPath { Path = path };
await context.LibraryPaths.AddAsync(libraryPath);
await context.SaveChangesAsync();
return libraryPath.Id;
}
}
@@ -0,0 +1,116 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Data.Repositories;
using ErsatzTV.Tests.Support;
using LanguageExt;
using Microsoft.EntityFrameworkCore;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using IFileSystem = System.IO.Abstractions.IFileSystem;
namespace ErsatzTV.Tests.Integration;
[TestFixture]
public class LibraryRepositoryTests
{
private InMemoryTvContext _db = null!;
private LibraryRepository _repository = null!;
[SetUp]
public async Task SetUp()
{
_db = await InMemoryTvContext.CreateAsync();
_repository = new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory);
}
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
// Regression for ersatztv#488: the Jellyfin (remote) sync path takes its LibraryPath straight off the
// JellyfinLibrary entity, so LibraryPath.LibraryFolders is never eager-loaded (null). GetOrAddFolder
// used to read that navigation collection directly and threw ArgumentNullException on the very first
// item of every Jellyfin music-video scan. The repository must resolve the folder from the database
// instead, so an unloaded collection is not a precondition.
[Test]
public async Task GetOrAddFolder_Should_Create_Folder_When_LibraryFolders_Not_Loaded()
{
int libraryPathId = await SeedLibraryPath("/data/music");
// mimic the Jellyfin path: Paths is populated, but LibraryFolders was never included
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
LibraryFolder result =
await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
result.ShouldNotBeNull();
result.Id.ShouldBeGreaterThan(0);
result.Path.ShouldBe("/data/music/artist1");
result.LibraryPathId.ShouldBe(libraryPathId);
await using TvContext context = _db.CreateContext();
List<LibraryFolder> folders = await context.LibraryFolders
.Where(f => f.LibraryPathId == libraryPathId)
.ToListAsync();
folders.Count.ShouldBe(1);
folders[0].Path.ShouldBe("/data/music/artist1");
}
// Re-scanning must be idempotent: a second GetOrAddFolder for the same path returns the existing row
// rather than inserting a duplicate LibraryFolder (there is no unique constraint behind it).
[Test]
public async Task GetOrAddFolder_Should_Be_Idempotent_On_Rescan()
{
int libraryPathId = await SeedLibraryPath("/data/music");
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
LibraryFolder first =
await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
LibraryFolder second =
await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
second.Id.ShouldBe(first.Id);
await using TvContext context = _db.CreateContext();
int count = await context.LibraryFolders.CountAsync(f => f.LibraryPathId == libraryPathId);
count.ShouldBe(1);
}
// Covers the maybeParentFolder = Some(...) branch: on a folder already in the db, the parent id is
// persisted through the raw Dapper UPDATE against the no-tracking entity (not change tracking), and the
// returned object reflects it. Guards the AsNoTracking + raw-UPDATE interaction the DB-lookup fix relies on.
[Test]
public async Task GetOrAddFolder_Should_Persist_ParentId_On_Existing_Folder()
{
int libraryPathId = await SeedLibraryPath("/data/music");
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
LibraryFolder parent =
await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music");
// first pass creates the child with no parent
LibraryFolder child =
await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
child.ParentId.ShouldBeNull();
// second pass supplies the parent — the existing row must be updated, not duplicated
LibraryFolder updated =
await _repository.GetOrAddFolder(libraryPath, Option<int>.Some(parent.Id), "/data/music/artist1");
updated.Id.ShouldBe(child.Id);
updated.ParentId.ShouldBe(parent.Id);
await using TvContext context = _db.CreateContext();
LibraryFolder persisted = await context.LibraryFolders.SingleAsync(f => f.Id == child.Id);
persisted.ParentId.ShouldBe(parent.Id);
int count = await context.LibraryFolders.CountAsync(f => f.Path == "/data/music/artist1");
count.ShouldBe(1);
}
private async Task<int> SeedLibraryPath(string path)
{
await using TvContext context = _db.CreateContext();
var libraryPath = new LibraryPath { Path = path };
await context.LibraryPaths.AddAsync(libraryPath);
await context.SaveChangesAsync();
return libraryPath.Id;
}
}
@@ -0,0 +1,156 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Data.Repositories;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
// #476: exercises the actual cascade SQL against the real schema. When a show/season is gone from the
// media server the per-parent scan loop never visits it, so its descendants must be swept to
// FileNotFound by parent MediaItem.Id. These prove the queries flip the right rows and only those.
[TestFixture]
public class TelevisionRepositoryCascadeTests
{
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
[Test]
public async Task FlagFileNotFoundSeasonsForShows_Flags_Only_The_Targeted_Shows_Seasons()
{
await SeedTwoShows();
var repository = new JellyfinTelevisionRepository(_db.Factory, NullLogger<JellyfinTelevisionRepository>.Instance);
// cascade from show 20 only
List<int> flagged = await repository.FlagFileNotFoundSeasonsForShows([20], CancellationToken.None);
flagged.ShouldBe([30, 31], ignoreOrder: true);
(await StateOf(30)).ShouldBe(MediaItemState.FileNotFound);
(await StateOf(31)).ShouldBe(MediaItemState.FileNotFound);
// sibling show 50's season is untouched
(await StateOf(60)).ShouldBe(MediaItemState.RemoteOnly);
}
[Test]
public async Task FlagFileNotFoundEpisodesForSeasons_Flags_Only_The_Targeted_Seasons_Episodes()
{
await SeedTwoShows();
var repository = new JellyfinTelevisionRepository(_db.Factory, NullLogger<JellyfinTelevisionRepository>.Instance);
// cascade from show 20's seasons only
List<int> flagged = await repository.FlagFileNotFoundEpisodesForSeasons([30, 31], CancellationToken.None);
flagged.ShouldBe([40, 41, 42], ignoreOrder: true);
(await StateOf(40)).ShouldBe(MediaItemState.FileNotFound);
(await StateOf(41)).ShouldBe(MediaItemState.FileNotFound);
(await StateOf(42)).ShouldBe(MediaItemState.FileNotFound);
// sibling show 50's episode is untouched
(await StateOf(70)).ShouldBe(MediaItemState.RemoteOnly);
}
[Test]
public async Task Cascade_Helpers_No_Op_On_Empty_Input()
{
await SeedTwoShows();
var repository = new JellyfinTelevisionRepository(_db.Factory, NullLogger<JellyfinTelevisionRepository>.Instance);
(await repository.FlagFileNotFoundSeasonsForShows([], CancellationToken.None)).ShouldBeEmpty();
(await repository.FlagFileNotFoundEpisodesForSeasons([], CancellationToken.None)).ShouldBeEmpty();
// nothing changed
(await StateOf(30)).ShouldBe(MediaItemState.RemoteOnly);
(await StateOf(40)).ShouldBe(MediaItemState.RemoteOnly);
}
private async Task<MediaItemState> StateOf(int id)
{
await using TvContext context = _db.CreateContext();
MediaItem item = await context.MediaItems.AsNoTracking().SingleAsync(m => m.Id == id);
return item.State;
}
// show 20 → seasons 30,31 → episodes 40,41 (season 30), 42 (season 31)
// sibling show 50 → season 60 → episode 70 (must never be touched by a cascade from show 20)
private async Task SeedTwoShows()
{
await using TvContext context = _db.CreateContext();
var library = new LocalLibrary { Id = 1, Name = "TV", MediaKind = LibraryMediaKind.Shows, Paths = [] };
var path = new LibraryPath { Id = 1, Path = "/media", Library = library, LibraryFolders = [], MediaItems = [] };
library.Paths.Add(path);
Show show20 = MakeShow(20, path);
Season season30 = MakeSeason(30, path, show20);
Season season31 = MakeSeason(31, path, show20);
Episode ep40 = MakeEpisode(40, path, season30);
Episode ep41 = MakeEpisode(41, path, season30);
Episode ep42 = MakeEpisode(42, path, season31);
Show show50 = MakeShow(50, path);
Season season60 = MakeSeason(60, path, show50);
Episode ep70 = MakeEpisode(70, path, season60);
path.MediaItems.AddRange([show20, season30, season31, ep40, ep41, ep42, show50, season60, ep70]);
context.LocalLibraries.Add(library);
context.Shows.AddRange(show20, show50);
context.Seasons.AddRange(season30, season31, season60);
context.Episodes.AddRange(ep40, ep41, ep42, ep70);
await context.SaveChangesAsync();
}
private static Show MakeShow(int id, LibraryPath path) => new()
{
Id = id,
LibraryPath = path,
State = MediaItemState.FileNotFound, // the show itself is already swept; children lag behind
Collections = [],
CollectionItems = [],
TraktListItems = [],
Seasons = [],
ShowMetadata = []
};
private static Season MakeSeason(int id, LibraryPath path, Show show)
{
var season = new Season
{
Id = id,
LibraryPath = path,
Show = show,
State = MediaItemState.RemoteOnly,
Collections = [],
CollectionItems = [],
TraktListItems = [],
Episodes = [],
SeasonMetadata = []
};
show.Seasons.Add(season);
return season;
}
private static Episode MakeEpisode(int id, LibraryPath path, Season season)
{
var episode = new Episode
{
Id = id,
LibraryPath = path,
Season = season,
State = MediaItemState.RemoteOnly,
Collections = [],
CollectionItems = [],
TraktListItems = [],
EpisodeMetadata = [],
MediaVersions = []
};
season.Episodes.Add(episode);
return episode;
}
}
+6 -2
View File
@@ -11,8 +11,12 @@ public class HealthController(IMediator mediator) : ControllerBase
[HttpGet("/api/v1/health", Name = "GetHealthChecks")]
[Tags("Health")]
[EndpointSummary("Get health check results")]
[EndpointDescription(
"Results are cached briefly; pass refresh=true to force a fresh run (re-executes ffmpeg-backed checks).")]
[EndpointGroupName("general")]
[ProducesResponseType(typeof(List<HealthCheckResponseModel>), StatusCodes.Status200OK)]
public async Task<List<HealthCheckResponseModel>> GetAll(CancellationToken cancellationToken) =>
await mediator.Send(new GetAllHealthCheckResultsForApi(), cancellationToken);
public async Task<List<HealthCheckResponseModel>> GetAll(
[FromQuery] bool refresh,
CancellationToken cancellationToken) =>
await mediator.Send(new GetAllHealthCheckResultsForApi(refresh), cancellationToken);
}
@@ -55,6 +55,10 @@ public class LocalLibrariesController(
[HttpPost("/api/v1/libraries/local")]
[Tags("Libraries")]
[EndpointSummary("Create a local library")]
[EndpointDescription(
"The shared LibraryMediaKind enum includes Mixed, but it is rejected here with 422. Mixed exists "
+ "only for Jellyfin libraries, where the media server classifies each item; no local folder "
+ "scanner handles it.")]
[ProducesResponseType(typeof(LocalLibraryResponseModel), StatusCodes.Status201Created)]
[ProducesResponseType(typeof(ProblemDetails), StatusCodes.Status422UnprocessableEntity)]
public async Task<IActionResult> Create(
@@ -34,7 +34,8 @@ public record CreateFFmpegProfileRequest(
int AudioSampleRate,
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo)
bool DeinterlaceVideo,
bool? QsvPreferNativeDecoder = null)
{
public CreateFFmpegProfile ToCommand() =>
new(
@@ -67,5 +68,6 @@ public record CreateFFmpegProfileRequest(
AudioSampleRate,
NormalizeFramerate,
NormalizeColors,
DeinterlaceVideo);
DeinterlaceVideo,
QsvPreferNativeDecoder ?? true);
}

Some files were not shown because too many files have changed in this diff Show More