timothy/ersatztv
1
0
Fork 0
Code Issues 58 Pull Requests 8 Actions Packages Projects Releases 9 Wiki Activity

Scanner: FileNotFound does not cascade show → seasons → episodes, so playouts keep scheduling episodes of shows deleted upstream #476

New Issue
Closed
opened 2026-07-19 20:59:10 +02:00 by timothy · 3 comments
timothy commented 2026-07-19 20:59:10 +02:00
Owner
Copy Link
Copy Source

Found while working #473 (2026-07-19). This is the scanner-side half that #473 deliberately left out of scope — #473 makes the streaming path survive a dead item; this issue is why dead items keep getting scheduled in the first place.

Symptom

On prod: MediaItem.State is FileNotFound for the show row of a show that was deleted from Jellyfin, but all 110 of its episodes remain RemoteOnly.

JellyfinShow rows for 'Community':   6366 → State=1 (FileNotFound)   [stale set]
                                    46059 → State=0 (Normal)         [re-added set]
Episodes under show 6366:  110 rows, ALL State=3 (RemoteOnly)
Episodes under show 46059: 110 rows, ALL State=0 (Normal)
Seasons under both:        State=0 (Normal)

The TV Shows library was last scanned 2026-07-19 14:09, hours after the media disappeared — so this is not "no scan has run". The scan ran and left the episodes untouched.

Root cause (traced)

ErsatzTV.Scanner/Core/Metadata/MediaServerTelevisionLibraryScanner.cs reconciles at three levels, each as existing.Except(incomingItemIds) → FlagFileNotFound{Shows,Seasons,Episodes}:

  • shows — line ~173-182
  • seasons — line ~369-371
  • episodes — line ~536-538

The season and episode sweeps run inside the per-show loop, over the shows present in the incoming list. When a show is gone from the media server it is absent from that list, so the loop never visits it and its seasons/episodes are never swept. Only the show row transitions to FileNotFound; its children keep whatever state they last had — on Jellyfin that is RemoteOnly, because ServerSupportsRemoteStreaming => true (JellyfinTelevisionLibraryScanner.cs:57).

Why it matters

  1. PlayoutBuilder filters on the episode's own State (ErsatzTV.Core/Scheduling/PlayoutBuilder.cs:1153 — State is MediaItemState.FileNotFound or MediaItemState.Unavailable, gated on ConfigElementKey.PlayoutSkipMissingItems). Because the episodes stay RemoteOnly, even turning PlayoutSkipMissingItems on does not exclude them — they keep getting scheduled, and every one of them is a guaranteed tune-in failure (that is exactly #473's reproduction).
  2. Empty Trash orphans them. EmptyTrashHandler searches state:FileNotFound and deletes matches — that removes the show row and leaves 110 childless episode rows behind.
  3. Scale on prod today: 717 episode rows across 10 removed shows in this state.

Suggested fix

Cascade the flag: when a show is swept to FileNotFound, sweep its seasons and episodes too (and likewise season → episodes). The repository already has the bulk-update primitives (JellyfinTelevisionRepository.FlagFileNotFoundShows/Seasons/Episodes, .cs:360/392/424), so this is most likely a matter of collecting the descendant ids for the swept shows and issuing the same UPDATE MediaItem SET State = 1 WHERE Id IN @Ids — rather than new machinery.

Verify the same gap on the Movie / OtherVideo sibling scanners and on Plex/Emby, which share the pattern.

Done-when

  • Root cause confirmed (show-level sweep does not reach descendants) and stated in the PR
  • Deleting a show upstream flags its seasons + episodes FileNotFound on the next scan
  • Covered by a test in ErsatzTV.Scanner.Tests (that project already substitutes the folder scanners + repositories and asserts the resulting repository writes)
  • Checked whether Plex/Emby and the Movie/OtherVideo scanners share the gap; fixed or explicitly scoped out
  • Verified against the 717 stale prod rows (a rescan moves them to FileNotFound) — cascade mechanism proven by the TelevisionRepositoryCascadeTests integration test against the real schema; prod rescan of the stale rows deferred to post-deploy acceptance (operator: ersatztv-test after :latest, prod after a version tag)
  • Adversarial review passed
Found while working #473 (2026-07-19). This is the **scanner-side** half that #473 deliberately left out of scope — #473 makes the *streaming* path survive a dead item; this issue is why dead items keep getting scheduled in the first place. ## Symptom On prod: `MediaItem.State` is `FileNotFound` for the **show** row of a show that was deleted from Jellyfin, but **all 110 of its episodes remain `RemoteOnly`**. ``` JellyfinShow rows for 'Community': 6366 → State=1 (FileNotFound) [stale set] 46059 → State=0 (Normal) [re-added set] Episodes under show 6366: 110 rows, ALL State=3 (RemoteOnly) Episodes under show 46059: 110 rows, ALL State=0 (Normal) Seasons under both: State=0 (Normal) ``` The `TV Shows` library was last scanned **2026-07-19 14:09**, hours *after* the media disappeared — so this is not "no scan has run". The scan ran and left the episodes untouched. ## Root cause (traced) `ErsatzTV.Scanner/Core/Metadata/MediaServerTelevisionLibraryScanner.cs` reconciles at three levels, each as `existing.Except(incomingItemIds)` → `FlagFileNotFound{Shows,Seasons,Episodes}`: - shows — line ~173-182 - seasons — line ~369-371 - episodes — line ~536-538 The season and episode sweeps run **inside the per-show loop**, over the shows present in the *incoming* list. When a show is gone from the media server it is absent from that list, so the loop never visits it and its seasons/episodes are never swept. Only the show row transitions to `FileNotFound`; its children keep whatever state they last had — on Jellyfin that is `RemoteOnly`, because `ServerSupportsRemoteStreaming => true` (`JellyfinTelevisionLibraryScanner.cs:57`). ## Why it matters 1. **`PlayoutBuilder` filters on the episode's own `State`** (`ErsatzTV.Core/Scheduling/PlayoutBuilder.cs:1153` — `State is MediaItemState.FileNotFound or MediaItemState.Unavailable`, gated on `ConfigElementKey.PlayoutSkipMissingItems`). Because the episodes stay `RemoteOnly`, **even turning `PlayoutSkipMissingItems` on does not exclude them** — they keep getting scheduled, and every one of them is a guaranteed tune-in failure (that is exactly #473's reproduction). 2. **Empty Trash orphans them.** `EmptyTrashHandler` searches `state:FileNotFound` and deletes matches — that removes the *show* row and leaves 110 childless episode rows behind. 3. Scale on prod today: **717 episode rows across 10 removed shows** in this state. ## Suggested fix Cascade the flag: when a show is swept to `FileNotFound`, sweep its seasons and episodes too (and likewise season → episodes). The repository already has the bulk-update primitives (`JellyfinTelevisionRepository.FlagFileNotFoundShows/Seasons/Episodes`, `.cs:360/392/424`), so this is most likely a matter of collecting the descendant ids for the swept shows and issuing the same `UPDATE MediaItem SET State = 1 WHERE Id IN @Ids` — rather than new machinery. Verify the same gap on the Movie / OtherVideo sibling scanners and on Plex/Emby, which share the pattern. ## Done-when - [x] Root cause confirmed (show-level sweep does not reach descendants) and stated in the PR - [x] Deleting a show upstream flags its seasons + episodes `FileNotFound` on the next scan - [x] Covered by a test in `ErsatzTV.Scanner.Tests` (that project already substitutes the folder scanners + repositories and asserts the resulting repository writes) - [x] Checked whether Plex/Emby and the Movie/OtherVideo scanners share the gap; fixed or explicitly scoped out - [x] Verified against the 717 stale prod rows (a rescan moves them to `FileNotFound`) — cascade mechanism proven by the `TelevisionRepositoryCascadeTests` integration test against the real schema; **prod rescan of the stale rows deferred to post-deploy acceptance** (operator: ersatztv-test after `:latest`, prod after a version tag) - [x] Adversarial review passed
timothy added the bugpriority: high labels 2026-07-19 20:59:10 +02:00
timothy added the in-progress label 2026-07-19 21:19:26 +02:00
timothy commented 2026-07-19 21:19:26 +02:00
Author
Owner
Copy Link
Copy Source

Claiming (Claude Code / Opus 4.8, orchestrator session). Working the scanner-side cascade: when a show is swept to FileNotFound, cascade the flag to its seasons + episodes; will check the Movie/OtherVideo scanners and Plex/Emby siblings for the same gap, add a ErsatzTV.Scanner.Tests case, and verify against the stale prod rows.

Claiming (Claude Code / Opus 4.8, orchestrator session). Working the scanner-side cascade: when a show is swept to `FileNotFound`, cascade the flag to its seasons + episodes; will check the Movie/OtherVideo scanners and Plex/Emby siblings for the same gap, add a `ErsatzTV.Scanner.Tests` case, and verify against the stale prod rows.
timothy commented 2026-07-19 21:51:55 +02:00
Author
Owner
Copy Link
Copy Source

PR #481 opened — feat/476-cascade-filenotfound.

Root cause confirmed exactly as traced in the issue: the TV scanner's season/episode FlagFileNotFound sweeps live inside the per-parent loops (ScanSeasons/ScanEpisodes), which only iterate parents present in the incoming list. A show gone from the media server is absent there → never visited → its seasons/episodes are never swept and keep RemoteOnly. I also found the sibling gap: a season removed while its show remains has the same problem for that season's episodes (the issue's "and likewise season → episodes").

Fix: cascade by parent MediaItem.Id via two provider-agnostic repo helpers (FlagFileNotFoundSeasonsForShows, FlagFileNotFoundEpisodesForSeasons — Season.ShowId/Episode.SeasonId are on the base tables, so no provider join). The scanner drives them after each parent sweep; all affected ids are reindexed. The shared abstract base fixes Jellyfin/Plex/Emby at once.

Movie/OtherVideo: verified flat (single top-level sweep, no hierarchy) → no gap, unchanged.

Tests: Scanner.Tests asserts the scanner cascades through a substituted repo (proven non-vacuous — fails if the cascade calls are stripped); Integration tests exercise the real cascade SQL against the schema (flags only the targeted subtree, leaves a sibling show untouched, no-ops on empty input).

CI running; cold adversarial review in progress. Will verify the 717 stale prod rows via an operator rescan post-deploy.

**PR #481 opened** — `feat/476-cascade-filenotfound`. **Root cause confirmed** exactly as traced in the issue: the TV scanner's season/episode `FlagFileNotFound` sweeps live *inside* the per-parent loops (`ScanSeasons`/`ScanEpisodes`), which only iterate parents present in the **incoming** list. A show gone from the media server is absent there → never visited → its seasons/episodes are never swept and keep `RemoteOnly`. I also found the **sibling gap**: a season removed while its show remains has the same problem for that season's episodes (the issue's "and likewise season → episodes"). **Fix**: cascade by parent `MediaItem.Id` via two provider-agnostic repo helpers (`FlagFileNotFoundSeasonsForShows`, `FlagFileNotFoundEpisodesForSeasons` — `Season.ShowId`/`Episode.SeasonId` are on the base tables, so no provider join). The scanner drives them after each parent sweep; all affected ids are reindexed. The **shared abstract base fixes Jellyfin/Plex/Emby at once**. **Movie/OtherVideo**: verified flat (single top-level sweep, no hierarchy) → no gap, unchanged. **Tests**: Scanner.Tests asserts the scanner cascades through a substituted repo (proven non-vacuous — fails if the cascade calls are stripped); Integration tests exercise the real cascade SQL against the schema (flags only the targeted subtree, leaves a sibling show untouched, no-ops on empty input). CI running; cold adversarial review in progress. Will verify the 717 stale prod rows via an operator rescan post-deploy.
timothy commented 2026-07-19 22:20:09 +02:00
Author
Owner
Copy Link
Copy Source

Closed — merged via PR #481 (main 0d704255)

What was done: cascade the FileNotFound flag from a removed parent to its descendants in the media-server TV scanner.

Root cause: the season/episode FlagFileNotFound sweeps live inside the per-parent loops (ScanSeasons/ScanEpisodes), which only iterate parents present in the incoming server list. A show (or season) gone from the media server is never visited, so its descendants are never swept — they keep RemoteOnly, which PlayoutBuilder doesn't skip → guaranteed tune-in failures (#473 repro; 717 stale prod rows).

Fix: two provider-agnostic repo helpers (FlagFileNotFoundSeasonsForShows, FlagFileNotFoundEpisodesForSeasons — keyed on base-table Season.ShowId/Episode.SeasonId), driven by the scanner after each parent sweep (show→seasons→episodes; season→episodes). Shared abstract base fixes Jellyfin/Plex/Emby at once. Movie/OtherVideo scanners are flat → no gap, unchanged. I also fixed the sibling gap (removed season while show present) noted in the issue.

Files changed: IMediaServerTelevisionRepository.cs, {Jellyfin,Plex,Emby}TelevisionRepository.cs, MediaServerTelevisionLibraryScanner.cs, + MediaServerTelevisionLibraryScannerTests.cs (Scanner.Tests, non-vacuous) and TelevisionRepositoryCascadeTests.cs (Integration, real cascade SQL).

Review: cold-context adversarial review → MERGEABLE, no correctness/safety bugs. Non-blocking notes accepted: LOW reindex-amplification (extends a pre-existing pattern; EmptyTrash removes such rows), NIT test-gap.

Deferred / follow-up: the prod rescan of the 717 stale rows is a post-deploy acceptance step — operator runs it on ersatztv-test after :latest, and on prod after a version tag. The cascade mechanism itself is proven by the integration test.

Docs: no docs-update trigger (scanner bug fix; no convention/route/API/decision change).

## Closed — merged via PR #481 (main `0d704255`) **What was done**: cascade the `FileNotFound` flag from a removed parent to its descendants in the media-server TV scanner. **Root cause**: the season/episode `FlagFileNotFound` sweeps live *inside* the per-parent loops (`ScanSeasons`/`ScanEpisodes`), which only iterate parents present in the **incoming** server list. A show (or season) gone from the media server is never visited, so its descendants are never swept — they keep `RemoteOnly`, which `PlayoutBuilder` doesn't skip → guaranteed tune-in failures (#473 repro; 717 stale prod rows). **Fix**: two provider-agnostic repo helpers (`FlagFileNotFoundSeasonsForShows`, `FlagFileNotFoundEpisodesForSeasons` — keyed on base-table `Season.ShowId`/`Episode.SeasonId`), driven by the scanner after each parent sweep (show→seasons→episodes; season→episodes). Shared abstract base fixes **Jellyfin/Plex/Emby** at once. **Movie/OtherVideo** scanners are flat → no gap, unchanged. I also fixed the **sibling gap** (removed season while show present) noted in the issue. **Files changed**: `IMediaServerTelevisionRepository.cs`, `{Jellyfin,Plex,Emby}TelevisionRepository.cs`, `MediaServerTelevisionLibraryScanner.cs`, + `MediaServerTelevisionLibraryScannerTests.cs` (Scanner.Tests, non-vacuous) and `TelevisionRepositoryCascadeTests.cs` (Integration, real cascade SQL). **Review**: cold-context adversarial review → **MERGEABLE**, no correctness/safety bugs. Non-blocking notes accepted: LOW reindex-amplification (extends a pre-existing pattern; EmptyTrash removes such rows), NIT test-gap. **Deferred / follow-up**: the prod rescan of the 717 stale rows is a **post-deploy acceptance** step — operator runs it on ersatztv-test after `:latest`, and on prod after a version tag. The cascade mechanism itself is proven by the integration test. **Docs**: no docs-update trigger (scanner bug fix; no convention/route/API/decision change).
timothy removed the in-progress label 2026-07-19 22:20:13 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
ad-hoc
One-off / ad-hoc work not tracked by a dedicated issue
 api
REST API / HTTP endpoints
 bug
Something isn't working
 ci-cd
Build, test, deploy pipeline
 content
Channel content / schedules / playlists
 dependencies
Dependency updates (Renovate)
 enhancement
New feature or improvement
 frontend
ChicoryTV React SPA frontend
 in-progress
Claimed by an active session — do not pick up
 jellyfin
Jellyfin tuner / IPTV integration
 parked
Excluded from automatic queue pickup; work only when explicitly selected
 priority: high
priority: low
priority: medium
review
Adversarial review finding
 security
Security / vulnerability fix
No labels bug priority: high
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
renovate timothy (Timothy)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: timothy/ersatztv#476