Review finding 1 (blocking). ScanSeasons' FlagFileNotFoundSeasons and ScanEpisodes' FlagFileNotFoundEpisodes had no guard at all — neither #477's nor #484's — so ProjectToSeason / ProjectToEpisode returning Failed() was computed and discarded. #477 scoped those out because "the blast radius is one show's seasons / one season's episodes", which holds for a per-parent EMPTY fetch but not for a projection failure: that is systematic by construction. One bad code path fires on every parent, so every season enumerates zero episodes, existing.Except([]) is the whole episode library, and EmptyTrashHandler deletes it permanently. Threads the counter into GetSeasonLibraryItems / GetEpisodeLibraryItems(WithoutPeople) for Jellyfin and Emby using the same optional-trailing-param shape, and guards both sweeps with MediaServerReconciliationGuard.ShouldFlagMissingDescendants — the same class and the same private failure predicate as ShouldFlagMissing, deliberately WITHOUT #477's empty-fetch branch so per-parent empty behaviour (and #476's cascade, which depends on it) is unchanged. Also from the review: - finding 3: tests now pin the same-instance JOIN at every level (movie, show, season, episode, music video) by driving the real ScanLibrary entry point and recording the failure from inside the enumeration, so a refactor handing the api client a fresh counter goes red. - finding 4: the missing-library Failed() branch is documented as defensive and unreachable. - finding 2: the mass-Skip residual (Emby's response-shape-dependent MediaSources guard, Plex's pre-projection filter) is stated as a known limitation in the decision record. - finding 5: the log-contract change (only the #484 message when both refusals apply) is noted. fixes #484
121 lines
6.2 KiB
C#
121 lines
6.2 KiB
C#
using Microsoft.Extensions.Logging;
|
|
|
|
namespace ErsatzTV.Scanner.Core.Metadata;
|
|
|
|
// #477: a media-server library sweep computes "gone upstream" as existing.Except(incoming) and flags the
|
|
// result FileNotFound. If a successful fetch returns ZERO items (the server is up but mid-restore /
|
|
// mid-rebuild, or the library was emptied upstream) then existing.Except([]) is EVERY existing item, so
|
|
// the whole library is flagged FileNotFound in one pass. That is data-loss-adjacent: EmptyTrashHandler
|
|
// deletes state:FileNotFound rows permanently, and PlayoutSkipMissingItems empties every affected
|
|
// collection. An empty incoming set is indistinguishable at scan time from a transient error (both report
|
|
// zero), so the safe policy is to refuse the sweep — logged loudly — rather than nuke the library.
|
|
//
|
|
// This deliberately overrides the degenerate "last item removed => empty incoming => flag" case that a
|
|
// partial-deletion sweep would otherwise handle (see #476's cascade, which still fires for the common
|
|
// case where survivors are present and only some items are gone). The cost of not flagging a genuinely
|
|
// emptied library (stale rows persist until an item returns or the library is removed) is far smaller
|
|
// than a one-scan permanent wipe.
|
|
//
|
|
// #484 extended the policy with a SECOND deterministic refusal and rejected the ratio threshold:
|
|
//
|
|
// - projection failures (implemented). A media-server API client maps every item the server returned
|
|
// through a private projection whose `catch` swallows the exception and drops the item. A dropped
|
|
// item the server DID return is indistinguishable from a deletion here, so one projection
|
|
// regression could mass-flag a healthy library. When the enumeration that produced `incomingCount`
|
|
// reports any such failure, the sweep is refused. Deliberate guard-clause skips (STRM files,
|
|
// virtual items, unsupported types) are NOT failures and never suppress the sweep — see
|
|
// MediaServerProjectionResult; counting them would permanently disable reconciliation for any
|
|
// library holding a single STRM file.
|
|
// - ratio / missing-fraction threshold (REJECTED). It is a two-sided heuristic: set low it silently
|
|
// suppresses legitimate bulk deletions, set high it misses the partial fetch it exists for, and
|
|
// there is no telemetry to tune it with. The failure it approximates is exactly observable via the
|
|
// projection-failure count above, and a genuine bulk deletion produces zero projection failures, so
|
|
// the deterministic signal has no false positives where the heuristic has unbounded ones.
|
|
//
|
|
// #484 also applies the projection-failure refusal to the NESTED per-show season and per-season episode
|
|
// sweeps, which #477 deliberately left unguarded. #477's reasoning ("blast radius is one show's seasons /
|
|
// one season's episodes") holds for a per-parent EMPTY fetch — a plausible legitimate state — but not for
|
|
// a projection failure, which is systematic by construction: one bad code path fires on every parent, so
|
|
// every season enumerates zero episodes and the whole episode library is swept in a single scan. The
|
|
// nested sweeps therefore get ShouldFlagMissingDescendants: the SAME failure predicate from the SAME
|
|
// class, deliberately WITHOUT #477's empty-fetch branch, so the per-parent empty behaviour (and #476's
|
|
// cascade, which depends on it) is unchanged.
|
|
//
|
|
// See docs/decisions.md `scan.projection-failure-sweep-guard`.
|
|
internal static class MediaServerReconciliationGuard
|
|
{
|
|
public static bool ShouldFlagMissing(
|
|
ILogger logger,
|
|
string libraryName,
|
|
int incomingCount,
|
|
int existingCount,
|
|
int projectionFailureCount = 0)
|
|
{
|
|
if (RefuseForProjectionFailures(
|
|
logger,
|
|
$"library {libraryName}",
|
|
incomingCount,
|
|
existingCount,
|
|
projectionFailureCount))
|
|
{
|
|
return false;
|
|
}
|
|
|
|
if (incomingCount == 0 && existingCount > 0)
|
|
{
|
|
logger.LogWarning(
|
|
"Media server library {Library} returned zero items but {ExistingCount} exist locally; "
|
|
+ "skipping the file-not-found sweep to avoid flagging the entire library as missing "
|
|
+ "(expected if the server is mid-restore or the library was emptied upstream)",
|
|
libraryName,
|
|
existingCount);
|
|
return false;
|
|
}
|
|
|
|
return true;
|
|
}
|
|
|
|
/// <summary>
|
|
/// The nested per-show season / per-season episode sweeps. Applies ONLY the #484 projection-failure
|
|
/// refusal — a per-parent empty fetch is a plausible legitimate state at this level and #477
|
|
/// deliberately left it unguarded, so importing that branch here would silently change #476's
|
|
/// per-parent cascade behaviour.
|
|
/// </summary>
|
|
/// <param name="scope">
|
|
/// Names the parent whose descendants are being swept, e.g. <c>show "Sesame Street" seasons</c>.
|
|
/// </param>
|
|
public static bool ShouldFlagMissingDescendants(
|
|
ILogger logger,
|
|
string scope,
|
|
int incomingCount,
|
|
int existingCount,
|
|
int projectionFailureCount) =>
|
|
!RefuseForProjectionFailures(logger, scope, incomingCount, existingCount, projectionFailureCount);
|
|
|
|
// the one place the projection-failure predicate lives, so the library-level and descendant-level
|
|
// sweeps can never drift apart on what counts as a failure or when it matters.
|
|
private static bool RefuseForProjectionFailures(
|
|
ILogger logger,
|
|
string scope,
|
|
int incomingCount,
|
|
int existingCount,
|
|
int projectionFailureCount)
|
|
{
|
|
if (projectionFailureCount <= 0 || existingCount <= 0)
|
|
{
|
|
return false;
|
|
}
|
|
|
|
logger.LogWarning(
|
|
"Media server {Scope} silently dropped {FailureCount} item(s) that failed to project during "
|
|
+ "this scan ({IncomingCount} usable, {ExistingCount} exist locally); skipping the "
|
|
+ "file-not-found sweep because a dropped item is indistinguishable from a deletion and "
|
|
+ "would be flagged as missing",
|
|
scope,
|
|
projectionFailureCount,
|
|
incomingCount,
|
|
existingCount);
|
|
return true;
|
|
}
|
|
}
|