fix(477): guard media-server library sweeps against successful-but-empty fetches #483

Merged

timothy merged 1 commits from fix/477-scanner-antinuke-guard into main 2026-07-19 23:53:38 +02:00

Conversation 1 Commits 1 Files Changed 9

+460 -12

timothy commented 2026-07-19 23:29:27 +02:00

Owner

Copy Link

Copy Source

Fixes #477.

Problem

Every media-server scanner reconciles "gone upstream" as existing.Except(incoming) → flag FileNotFound. A successful but empty fetch (server up but mid-restore / mid-rebuild, or library emptied upstream) makes incomingItemIds == [], so existing.Except([]) is every item — the whole library is flagged FileNotFound in one scan. That is data-loss-adjacent: EmptyTrashHandler permanently deletes state:FileNotFound rows, and PlayoutSkipMissingItems empties every affected collection.

Fix

A shared MediaServerReconciliationGuard.ShouldFlagMissing(logger, library, incoming, existing) skips the sweep (logging a Warning) only when incoming == 0 && existing > 0; every other combination reconciles normally. Wired into the three library-level sweeps: Television (shows), Movie, OtherVideo.

Interaction with #476 (deliberate, documented)

An empty incoming set is indistinguishable at scan time from a transient error (both report a zero total), so this overrides #476's degenerate "last item removed ⇒ empty incoming ⇒ flag" case. #476's cascade still fires for the common partial-deletion path (survivors present, incoming non-empty); its characterization test was moved from an empty incoming to a survivor+removed partial deletion so it still exercises the cascade without tripping the guard.

Scope / deferrals

• Library-level sweeps only. The nested TV season/episode sweeps are left unguarded: bounded blast radius (one parent's children), a per-parent empty is a more plausible legitimate state, and #476's descendant cascade already handles a fully-removed parent.
• Deferred to a follow-up: ratio-threshold guard + projection-failure detection (needs a dropped-count threaded out of JellyfinApiClient; a ratio risks suppressing legitimate bulk deletions). The deterministic zero-count guard has no false positives and covers the reported catastrophic case.

Tests

• MediaServerReconciliationGuardTests — policy table (only (0, N>0) skips-and-warns; (0,0), (3,5), (3,0) sweep).
• MediaServer{Television,Movie,OtherVideo}LibraryScannerTests — per-scanner integration: empty incoming + non-empty existing flags/reindexes nothing; TV also keeps the #476 partial-deletion cascade.
• Proven non-vacuous by neutralizing the guard (all four anti-nuke assertions fail; the (0,0) no-op stays green). Full ErsatzTV.Scanner.Tests: 1481 passed. Full solution build: 0 errors.

Docs

docs/decisions.md entry added (policy, the #476 override, scope, deferrals).

Note (out of scope)

While committing I hit a latent trap: .gitignore line 49 (core) matches the Core/ directory case-insensitively on macOS, so new files under any */Core/ path are silently ignored (already-tracked files are unaffected). Force-added here; filing a follow-up.

Fixes #477. ## Problem Every media-server scanner reconciles "gone upstream" as `existing.Except(incoming)` → flag `FileNotFound`. A **successful but empty** fetch (server up but mid-restore / mid-rebuild, or library emptied upstream) makes `incomingItemIds == []`, so `existing.Except([])` is **every** item — the whole library is flagged `FileNotFound` in one scan. That is data-loss-adjacent: `EmptyTrashHandler` permanently deletes `state:FileNotFound` rows, and `PlayoutSkipMissingItems` empties every affected collection. ## Fix A shared `MediaServerReconciliationGuard.ShouldFlagMissing(logger, library, incoming, existing)` skips the sweep (logging a Warning) **only** when `incoming == 0 && existing > 0`; every other combination reconciles normally. Wired into the three **library-level** sweeps: Television (shows), Movie, OtherVideo. ## Interaction with #476 (deliberate, documented) An empty incoming set is indistinguishable at scan time from a transient error (both report a zero total), so this **overrides #476's degenerate "last item removed ⇒ empty incoming ⇒ flag" case**. #476's cascade still fires for the common partial-deletion path (survivors present, incoming non-empty); its characterization test was moved from an empty incoming to a survivor+removed partial deletion so it still exercises the cascade without tripping the guard. ## Scope / deferrals - **Library-level sweeps only.** The nested TV season/episode sweeps are left unguarded: bounded blast radius (one parent's children), a per-parent empty is a more plausible legitimate state, and #476's descendant cascade already handles a fully-removed parent. - **Deferred to a follow-up:** ratio-threshold guard + projection-failure detection (needs a dropped-count threaded out of `JellyfinApiClient`; a ratio risks suppressing legitimate bulk deletions). The deterministic zero-count guard has no false positives and covers the reported catastrophic case. ## Tests - `MediaServerReconciliationGuardTests` — policy table (only `(0, N>0)` skips-and-warns; `(0,0)`, `(3,5)`, `(3,0)` sweep). - `MediaServer{Television,Movie,OtherVideo}LibraryScannerTests` — per-scanner integration: empty incoming + non-empty existing flags/reindexes nothing; TV also keeps the #476 partial-deletion cascade. - Proven **non-vacuous** by neutralizing the guard (all four anti-nuke assertions fail; the `(0,0)` no-op stays green). Full `ErsatzTV.Scanner.Tests`: 1481 passed. Full solution build: 0 errors. ## Docs `docs/decisions.md` entry added (policy, the #476 override, scope, deferrals). ## Note (out of scope) While committing I hit a latent trap: `.gitignore` line 49 (`core`) matches the `Core/` directory case-insensitively on macOS, so new files under any `*/Core/` path are silently ignored (already-tracked files are unaffected). Force-added here; filing a follow-up.

timothy added 1 commit 2026-07-19 23:29:28 +02:00

fix(477): guard media-server library sweeps against successful-but-empty fetches ... aa32fd78dd

A successful fetch returning zero items made existing.Except([]) flag the ENTIRE
library FileNotFound in one scan — feeding EmptyTrashHandler's permanent delete
and emptying every affected collection (dead channels). Add a shared
MediaServerReconciliationGuard that skips (and logs a Warning) the sweep when
incoming==0 while items exist, wired into the three library-level sweeps
(Television shows / Movie / OtherVideo).

An empty incoming set is indistinguishable at scan time from a mid-restore /
emptied-upstream error (both report a zero total), so this deliberately overrides
#476's degenerate "last item removed => empty incoming => flag" case. #476's
cascade still fires for partial deletions (survivors present); its characterization
test moves from an empty incoming to a survivor+removed partial-deletion case.

Tests: policy table (MediaServerReconciliationGuardTests) + per-scanner integration
proving the wiring (empty incoming + non-empty existing flags/reindexes nothing).
Proven non-vacuous by neutralizing the guard. Nested TV season/episode sweeps left
unguarded (bounded blast radius); ratio-threshold + projection-failure detection
deferred to a follow-up. docs/decisions.md updated.

Fixes #477

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

timothy referenced this pull request 2026-07-19 23:31:23 +02:00

Media-server scanner: ratio-threshold + projection-failure detection for the empty-fetch guard #484

timothy referenced this pull request 2026-07-19 23:31:24 +02:00

Build hygiene: `.gitignore` `core` entry silently ignores new files under any `Core/` directory (case-insensitive on macOS) #485

timothy referenced this pull request 2026-07-19 23:31:49 +02:00

Scanner has no anti-nuke guard: a successful-but-empty media-server fetch flags an entire library FileNotFound #477

timothy commented 2026-07-19 23:32:58 +02:00

Author

Owner

Copy Link

Copy Source

Independent cold-context adversarial review (Claude, review-only) — read all three scanner base classes in full, the guard, all four test files, grepped every FlagFileNotFound/.Except site in ErsatzTV.Scanner, and ran the 8 new tests (all pass).

Confirmed: guard wiring correct in all three library-level sweeps (no half-done work / wrong return on skip); the TV guard correctly gates the #476 descendant cascade; the #476 survivor short-circuit is real (incomingItemIds.Add runs before GetOrAdd→Left) so the rewritten test exercises the real cascade, not a stub; no missed whole-library sweep site (local folder scanners use a per-file pattern not exposed to this failure mode); tests non-vacuous (negative control real, UpdateProgress correctly stubbed so no ScanCanceled masking). One non-blocking, already-deferred scope note: nested TV season/episode sweeps remain unguarded (bounded blast radius; tracked for #477 follow-up).

Review-verdict: MERGEABLE @ aa32fd78dd

Independent cold-context adversarial review (Claude, review-only) — read all three scanner base classes in full, the guard, all four test files, grepped every FlagFileNotFound/.Except site in ErsatzTV.Scanner, and ran the 8 new tests (all pass). Confirmed: guard wiring correct in all three library-level sweeps (no half-done work / wrong return on skip); the TV guard correctly gates the #476 descendant cascade; the #476 survivor short-circuit is real (incomingItemIds.Add runs before GetOrAdd→Left) so the rewritten test exercises the real cascade, not a stub; no missed whole-library sweep site (local folder scanners use a per-file pattern not exposed to this failure mode); tests non-vacuous (negative control real, UpdateProgress correctly stubbed so no ScanCanceled masking). One non-blocking, already-deferred scope note: nested TV season/episode sweeps remain unguarded (bounded blast radius; tracked for #477 follow-up). Review-verdict: MERGEABLE @ aa32fd78dd68bf476df6d0ff464e15fa2ce85e5d

timothy merged commit 55be64bd77 into main 2026-07-19 23:53:38 +02:00

timothy deleted branch fix/477-scanner-antinuke-guard 2026-07-19 23:53:38 +02:00

timothy referenced this issue from a commit 2026-07-19 23:53:40 +02:00

Merge pull request 'fix(477): guard media-server library sweeps against successful-but-empty fetches' (#483) from fix/477-scanner-antinuke-guard into main

timothy referenced this pull request 2026-07-19 23:54:19 +02:00

Scanner has no anti-nuke guard: a successful-but-empty media-server fetch flags an entire library FileNotFound #477

timothy referenced this pull request 2026-07-19 23:54:42 +02:00

ChicoryTV rewrite — queue tracker (goal, arc, session protocol) #237

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

ad-hoc

One-off / ad-hoc work not tracked by a dedicated issue

api

REST API / HTTP endpoints

bug

Something isn't working

ci-cd

Build, test, deploy pipeline

content

Channel content / schedules / playlists

dependencies

Dependency updates (Renovate)

enhancement

New feature or improvement

frontend

ChicoryTV React SPA frontend

in-progress

Claimed by an active session — do not pick up

jellyfin

Jellyfin tuner / IPTV integration

parked

Excluded from automatic queue pickup; work only when explicitly selected

priority: high

priority: low

priority: medium

review

Adversarial review finding

security

Security / vulnerability fix

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

renovate timothy (Timothy)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: timothy/ersatztv#483