Commit Graph
27 Commits
Author SHA1 Message Date
timothy 6f4497e1ce fix(484): guard the nested TV season and episode sweeps against projection failures
Review finding 1 (blocking). ScanSeasons' FlagFileNotFoundSeasons and ScanEpisodes'
FlagFileNotFoundEpisodes had no guard at all — neither #477's nor #484's — so ProjectToSeason /
ProjectToEpisode returning Failed() was computed and discarded.

#477 scoped those out because "the blast radius is one show's seasons / one season's episodes",
which holds for a per-parent EMPTY fetch but not for a projection failure: that is systematic by
construction. One bad code path fires on every parent, so every season enumerates zero episodes,
existing.Except([]) is the whole episode library, and EmptyTrashHandler deletes it permanently.

Threads the counter into GetSeasonLibraryItems / GetEpisodeLibraryItems(WithoutPeople) for
Jellyfin and Emby using the same optional-trailing-param shape, and guards both sweeps with
MediaServerReconciliationGuard.ShouldFlagMissingDescendants — the same class and the same private
failure predicate as ShouldFlagMissing, deliberately WITHOUT #477's empty-fetch branch so
per-parent empty behaviour (and #476's cascade, which depends on it) is unchanged.

Also from the review:
- finding 3: tests now pin the same-instance JOIN at every level (movie, show, season, episode,
  music video) by driving the real ScanLibrary entry point and recording the failure from inside
  the enumeration, so a refactor handing the api client a fresh counter goes red.
- finding 4: the missing-library Failed() branch is documented as defensive and unreachable.
- finding 2: the mass-Skip residual (Emby's response-shape-dependent MediaSources guard, Plex's
  pre-projection filter) is stated as a known limitation in the decision record.
- finding 5: the log-contract change (only the #484 message when both refusals apply) is noted.

fixes #484
2026-07-25 16:36:35 +02:00
timothy e3645a2840 feat(484): suppress the media-server sweep on projection failures; reject the ratio threshold
Extends MediaServerReconciliationGuard (#477) with a second deterministic refusal: when the
enumeration that produced the incoming set silently dropped items whose projection THREW, the
file-not-found sweep is refused. A dropped item the server did return is indistinguishable
from a deletion at the reconcile step, so a projection regression could otherwise mass-flag a
healthy library FileNotFound (which EmptyTrash then deletes permanently).

Deliberate guard-clause skips (STRM files, virtual items, unsupported types) are explicitly NOT
failures and never suppress a sweep — counting them would permanently disable reconciliation for
any library holding a single STRM file.

The ratio / missing-fraction threshold is REJECTED, not deferred: it is a two-sided heuristic
with no tunable default and no telemetry, and the failure it approximates is exactly observable
via the projection-failure count (a genuine bulk deletion produces zero failures).

Seam is deliberately narrow — the private ProjectTo* contract inside each api client changed from
Option<T> to MediaServerProjectionResult<T> (projected/skipped/failed), the paged helper counts
IsFailure in one place, and the scanner reads it through an optional trailing
MediaServerProjectionFailureCounter on only the five library-level methods that feed a sweep.
The counter is per-enumeration state created by the scanner, never a field on an api client.

fixes #484
2026-07-25 16:36:35 +02:00
timothyandClaude Opus 4.8 aa32fd78dd fix(477): guard media-server library sweeps against successful-but-empty fetches
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 9s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 7s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m23s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m18s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
A successful fetch returning zero items made existing.Except([]) flag the ENTIRE
library FileNotFound in one scan — feeding EmptyTrashHandler's permanent delete
and emptying every affected collection (dead channels). Add a shared
MediaServerReconciliationGuard that skips (and logs a Warning) the sweep when
incoming==0 while items exist, wired into the three library-level sweeps
(Television shows / Movie / OtherVideo).

An empty incoming set is indistinguishable at scan time from a mid-restore /
emptied-upstream error (both report a zero total), so this deliberately overrides
#476's degenerate "last item removed => empty incoming => flag" case. #476's
cascade still fires for partial deletions (survivors present); its characterization
test moves from an empty incoming to a survivor+removed partial-deletion case.

Tests: policy table (MediaServerReconciliationGuardTests) + per-scanner integration
proving the wiring (empty incoming + non-empty existing flags/reindexes nothing).
Proven non-vacuous by neutralizing the guard. Nested TV season/episode sweeps left
unguarded (bounded blast radius); ratio-threshold + projection-failure detection
deferred to a follow-up. docs/decisions.md updated.

Fixes #477

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 23:28:27 +02:00
timothyandClaude Opus 4.8 61bd374494 fix(476): cascade FileNotFound from removed shows/seasons to descendants
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 7s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 24s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 5m52s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 5m36s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 9s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m14s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The media-server television scanner reconciles removed items at three levels
via existing.Except(incoming), but the season and episode sweeps live inside
the per-parent loops (ScanSeasons inside the show loop, ScanEpisodes inside the
season loop). Those loops only iterate parents present in the incoming list, so
a show (or season) that is gone from the media server is never visited and its
descendants are never swept — they keep their last state. On Jellyfin/Emby that
is RemoteOnly, which PlayoutBuilder does NOT skip even with PlayoutSkipMissingItems
on, so every orphaned episode keeps getting scheduled as a guaranteed tune-in
failure (the #473 reproduction; 717 stale prod rows across 10 removed shows).

Fix: cascade the flag by parent MediaItem.Id. Two provider-agnostic repo helpers
(Season.ShowId / Episode.SeasonId are on the base tables) flag descendants and the
scanner drives them after each parent sweep — show → seasons → episodes, and
season → episodes for the show-present case. Shared abstract base fixes
Jellyfin/Plex/Emby at once; Movie/OtherVideo are flat and have no such gap.

Tests: a Scanner.Tests case asserts the scanner cascades through the (substituted)
repository (non-vacuous — fails if the cascade calls are removed), and Integration
tests exercise the real cascade SQL against the schema, proving it flags only the
targeted subtree and no-ops on empty input.

fixes #476

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 21:49:33 +02:00
Jason DoveandGitHub 7c82ecdfff use separate load queries for jf episode inserts and updates (#2751) 2026-01-02 16:28:19 -06:00
Jason DoveandGitHub 80f6e468eb collect and print timings during jellyfin show library scans (#2748)
* collect and print timings during jellyfin show library scans

* update p99
2026-01-01 20:02:24 -06:00
Jason DoveandGitHub 474e647d6d more jellyfin performance improvements (#2747)
* fix slow db and api logging so it also works in scanner project

* don't request people from jellyfin by default
2025-12-31 22:28:15 -06:00
Jason DoveandGitHub ec0d8ea6ac work around sequential schedule validation limit (#2655)
* remove readalltext

* remove unused method

* remove fileexists

* remove folderexists

* remove readalllines

* remove fake local file system

* show playlist name in playout build errors

* add basic sequential schedule validator tests

* work around sequential schedule validation limit
2025-11-24 12:08:43 -06:00
Jason DoveandGitHub 626048f9c3 change how scanner and main process communicate (#2555)
* report scanner progress using api

* process scanner search index updates through api

* update changelog

* update dependencies
2025-10-21 15:08:49 -05:00
Jason DoveandGitHub 9462156148 fix mysql playout builds (#2352)
* more cancellation tokens and fixes

* so much cancellation token

* fix mysql playout builds
2025-08-27 18:09:56 +00:00
Jason DoveandGitHub 1c07df5bc3 use cancellation tokens in many places (#2350)
* use cancellation tokens everywhere

* more cancellation tokens
2025-08-27 03:20:35 +00:00
Jason DoveandGitHub 5d081ceeff fix editorconfig and run code cleanup (#2324)
* fix formatting rules

* reformat ersatztv

* reformat ersatztv.application

* reformat ersatztv.core

* refactor ersatztv.core.tests

* reformat ersatztv.ffmpeg

* reformat ersatztv.ffmpeg.tests

* reformat ersatztv.infrastructure

* cleanup infra mysql

* cleanup infra sqlite

* cleanup infra tests

* cleanup ersatztv.scanner

* cleanup ersatztv.scanner.tests

* sln cleanup

* update dependencies
2025-08-16 14:44:48 +00:00
d0af507bef add ability to deep scan just a single tv show for Plex, Emby, and Jellyfin (#2318)
* add ability to deep scan just a single tv show for Plex, Emby, and Jellyfin

Including "/api/libraries/{id:int}/scan-show" REST API endpoint to
trigger.

* restrict plex search results to the intended library

* restrict scanning to media server libraries that are marked to sync with etv

* fix previous commit

* also guard library scan api

* add scan buttons to show ui

* scan single plex show by id

* scan jellyfin and emby single shows by id

* update changelog

---------

Co-authored-by: Jeff Slutter <MrMustard@gmail.com>
Co-authored-by: Jason Dove <1695733+jasongdove@users.noreply.github.com>
2025-08-14 16:07:56 +00:00
f626954eb7 add external chapter file scanning (#2317)
* add external chapter file scanning

Support Matroska chapter xml files next to media file with extension .xml or .chapters

* only update chapters in db

---------

Co-authored-by: Jeff Slutter <MrMustard@gmail.com>
Co-authored-by: Jason Dove <1695733+jasongdove@users.noreply.github.com>
2025-08-14 03:34:47 +00:00
Jason DoveandGitHub 7e30444857 dependencies and code cleanup (#2117)
* fix validation in new form layout

* pin mediatr to last oss version

* update dependencies

* cleanup code in core

* cleanup code in ffmpeg

* cleanup code in infra

* cleanup code in scanner

* cleanup code in application

* cleanup main code

* cleanup test code

* solution-wide code cleanup
2025-07-06 15:56:17 +00:00
Jason DoveandGitHub f41fa669be update media server scanning and paging (#1770)
* update media server scanning and paging

* remove unused types
2024-07-02 13:12:09 -05:00
Jason DoveandGitHub 6e7880386b update plex show title (#1677)
* update plex show title when changed in plex

* fix test build
2024-04-12 09:21:11 -05:00
Jason DoveandGitHub abed22b560 prevent unnecessary search index updates (#1592)
* add sub_language and sub_language_tag fields to search index

* prevent unnecessary search index updates

* update changelog

* update dependencies
2024-02-02 06:33:06 -06:00
Jason DoveandGitHub 8f5b181372 mysql media server library scan fixes (#1491)
* fix some mysql movie library updates

* fix some mysql show library updates

* update dependencies
2023-10-30 06:45:00 -05:00
Jason DoveandGitHub 03df2a6c8a overdue code cleanup (#1271) 2023-05-10 13:18:18 -05:00
Jason DoveandGitHub 126304bb8a fix subtitles from media server libraries (#1233)
* fix embedded subtitles from media servers

* fix plex external subtitles

* fix artwork bug, delete orphaned subtitles

* jellyfin subtitles work again

* emby subtitles work

* rescan all media server libraries
2023-04-07 13:22:45 -05:00
Jason DoveandGitHub 7e3436e68f direct stream content from emby as needed (#1168) 2023-02-19 20:29:09 -06:00
Jason DoveandGitHub b751f1054b direct stream content from jellyfin if needed (#1167)
* redirect to jellyfin stream as needed

* get jellyfin playback info

* sync chapters from jellyfin

* update changelog

* cleanup
2023-02-19 10:37:48 -06:00
Jason DoveandGitHub 900e9e75f3 sync chapter markers from plex (#1166) 2023-02-18 14:51:51 -06:00
Jason DoveandGitHub 62c28d9f51 direct stream content from plex if needed (#1165)
* start to stream directly from plex

* update metadata and statistics with one plex api call

* stream movies from plex

* scanning bug fix; update changelog
2023-02-18 10:40:05 -06:00
Jason DoveandGitHub 13e21bbcce sync episode tags and genres (#1155)
* sync episode tags and genres

* update dependencies

* property update local episode genres and tags

* fix test
2023-02-12 09:53:20 -06:00
Jason DoveandGitHub 9479806cb0 scanner refactoring and other cleanup (#1082)
* move subtitles provider into scanner

* move more stuff into scanner

* move nfo into scanner

* add scan subcommand

* fix a bunch of nfo build warnings

* more subcommands

* fix warnings

* cleanup logging

* remove unused code

* cleanup old ffmpeg stuff

* rename complex filter

* refactor wrapped segmenter
2022-12-31 10:57:20 -06:00