From 1f7b763a7b09f45647d7ec74d15b348f04e765f5 Mon Sep 17 00:00:00 2001 From: Timothy Date: Thu, 22 Oct 2026 06:24:18 +0000 Subject: [PATCH] fix(924): media-server music videos read their statistics from the server's playback info (#928) Jellyfin's library-items projection builds every music video's MediaVersion with Streams = [] and no width/height; the server reports those only through per-item PlaybackInfo. MediaServerMusicVideoLibraryScanner.UpdateStatistics wrote that empty projection straight through (also removing any streams a stored row already had), so every Jellyfin music video was stored without streams and FFmpegStreamSelector.SelectVideoStream threw "Sequence contains no matching element" on play. Prod 2026-10-21: the Jellyfin "Music Videos" library (id 16), 1520/1520 versions with zero MediaStream rows. - UpdateStatistics mirrors the movie scanner: on deep scan, new item, changed etag or no stored streams it asks the abstract GetMediaServerStatistics, and writes nothing when the server cannot answer. - JellyfinMusicVideoLibraryScanner implements it via GetPlaybackInfo; chapters come from the incoming projection. - SelectVideoStream fails with the version id, file and remediation (deep scan, or remove a file that has no video stream) instead of a bare First() throw. - Other-video and TV scanners checked: no such gap. - Record scan.mediaserver-statistics-from-server. Existing rows heal only on a DEEP scan (a regular scan skips unchanged etags): POST /api/v1/libraries/16/scan?deep=true after deploy. fixes #924 Co-Authored-By: Claude Opus 5.5 Claude-Session: 99d1dedb-f027-4e05-baab-2111f3a7d951 Co-authored-by: Timothy --- .claude/skills/ersatztv/SKILL.md | 7 +- .../FFmpeg/FFmpegStreamSelectorTests.cs | 61 ++++ ErsatzTV.Core/FFmpeg/FFmpegStreamSelector.cs | 31 +- .../JellyfinMusicVideoLibraryScanner.cs | 37 +++ .../MediaServerMusicVideoLibraryScanner.cs | 46 ++- .../JellyfinMusicVideoLibraryScannerTests.cs | 301 +++++++++++++++++- docs/decisions/README.md | 1 + .../mediaserver-statistics-from-server.md | 43 +++ 8 files changed, 515 insertions(+), 12 deletions(-) create mode 100644 docs/decisions/records/scan/mediaserver-statistics-from-server.md diff --git a/.claude/skills/ersatztv/SKILL.md b/.claude/skills/ersatztv/SKILL.md index 5bf559753..8fe27b353 100644 --- a/.claude/skills/ersatztv/SKILL.md +++ b/.claude/skills/ersatztv/SKILL.md @@ -176,7 +176,7 @@ docker compose -f $D/compose.yaml up -d --no-deps ersatztv-test ### Mutation Endpoints (POST) ```bash -# Library scan +# Library scan (?deep=true also rescans items a media server reports unchanged) POST /api/v1/libraries/{id}/scan # Scan single show @@ -472,6 +472,11 @@ docker start ersatztv ### Common Mistakes (check every time) - **Playout not building**: Three things must all be correct: (1) `ProgramScheduleOneItem` row exists for the schedule item, (2) `PlaybackOrder=3` (Shuffle), (3) `ScheduleKind=1` on Playout. Missing any one results in 0 playout items — this is the most common issue. - **Collection queries fail**: `CollectionItem` has no `Id` column — use `rowid` for counting. Content lookup goes through `MediaVersion.MovieId` → `MediaFile.Path` (not a generic MediaItemId join). +- **Channel won't start — "has no video stream on record"** (images before v26.16.0, which carries the ersatztv#924 fix, log a bare + `Sequence contains no matching element` at `FFmpegStreamSelector.SelectVideoStream`): the playing item's + `MediaVersion` has no `MediaStream` rows. Deep-scan its library — `POST /api/v1/libraries/{id}/scan?deep=true`; + a regular scan skips items whose server etag is unchanged and cannot heal them. If the file itself has no video + stream, no scan heals it. Jellyfin music videos scanned before that fix were all stored without streams (`scan.mediaserver-statistics-from-server`). - **Channel logos forgotten**: After creating a channel, add an Artwork row (ArtworkKind=2) + logo file, then run `fix_logos.py` to push to Jellyfin. Without this, the channel shows no logo in the EPG. - **Playout reset required**: After any schedule/collection change, run `POST /api/v1/channels/{id}/playout/reset`. Wait 5-10s for the playout to build before verifying item count. diff --git a/ErsatzTV.Core.Tests/FFmpeg/FFmpegStreamSelectorTests.cs b/ErsatzTV.Core.Tests/FFmpeg/FFmpegStreamSelectorTests.cs index 07a2cba0a..dbcde233a 100644 --- a/ErsatzTV.Core.Tests/FFmpeg/FFmpegStreamSelectorTests.cs +++ b/ErsatzTV.Core.Tests/FFmpeg/FFmpegStreamSelectorTests.cs @@ -191,4 +191,65 @@ public class FFmpegStreamSelectorTests } } } + + // ersatztv#924: a version stored without a video stream (Jellyfin music videos whose statistics were never + // read) used to surface First()'s bare "Sequence contains no matching element" from HLS session start. + [TestFixture] + public class SelectVideoStream + { + private static FFmpegStreamSelector Selector() => + new( + new ScriptEngine(Substitute.For>()), + Substitute.For(), + Substitute.For(), + new MockFileSystem(), + Substitute.For(), + Substitute.For>()); + + [Test] + public async Task Should_Select_The_Video_Stream() + { + var version = new MediaVersion + { + Id = 12, + Streams = + [ + new MediaStream { Index = 0, MediaStreamKind = MediaStreamKind.Audio }, + new MediaStream { Index = 1, MediaStreamKind = MediaStreamKind.Video, Codec = "h264" } + ] + }; + + MediaStream stream = await Selector().SelectVideoStream(version); + + stream.Index.ShouldBe(1); + } + + [Test] + public async Task Should_Name_The_Version_And_File_When_There_Is_No_Video_Stream() + { + var version = new MediaVersion + { + Id = 1520, + MediaFiles = [new MediaFile { Path = "/data/music/artist1/song1.mkv" }], + Streams = [new MediaStream { Index = 0, MediaStreamKind = MediaStreamKind.Audio }] + }; + + InvalidOperationException ex = + await Should.ThrowAsync(() => Selector().SelectVideoStream(version)); + + ex.Message.ShouldContain("Media version 1520 (/data/music/artist1/song1.mkv) has no video stream"); + ex.Message.ShouldContain("deep scan"); + } + + [Test] + public async Task Should_Fail_Clearly_When_Streams_And_Files_Were_Never_Loaded() + { + var version = new MediaVersion { Id = 7, MediaFiles = null, Streams = null }; + + InvalidOperationException ex = + await Should.ThrowAsync(() => Selector().SelectVideoStream(version)); + + ex.Message.ShouldContain("Media version 7 ((no media file)) has no video stream"); + } + } } diff --git a/ErsatzTV.Core/FFmpeg/FFmpegStreamSelector.cs b/ErsatzTV.Core/FFmpeg/FFmpegStreamSelector.cs index 401650649..4023a2df7 100644 --- a/ErsatzTV.Core/FFmpeg/FFmpegStreamSelector.cs +++ b/ErsatzTV.Core/FFmpeg/FFmpegStreamSelector.cs @@ -1,4 +1,4 @@ -using System.Collections.Immutable; +using System.Collections.Immutable; using System.Diagnostics; using System.Diagnostics.CodeAnalysis; using System.Globalization; @@ -38,8 +38,31 @@ public class FFmpegStreamSelector : IFFmpegStreamSelector _logger = logger; } - public Task SelectVideoStream(MediaVersion version) => - version.Streams.First(s => s.MediaStreamKind == MediaStreamKind.Video).AsTask(); + // ersatztv#924: a version with no video stream on record (its statistics were never read, or were stored + // empty) cannot be played. Name the version and its file and say how the statistics get populated, instead + // of surfacing First()'s bare "Sequence contains no matching element". + public Task SelectVideoStream(MediaVersion version) + { + foreach (MediaStream videoStream in Optional(version.Streams).Flatten() + .Filter(s => s.MediaStreamKind == MediaStreamKind.Video) + .HeadOrNone()) + { + return videoStream.AsTask(); + } + + string path = Optional(version.MediaFiles).Flatten() + .HeadOrNone() + .Map(f => f.Path) + .IfNone("(no media file)"); + + return Task.FromException( + new InvalidOperationException( + $"Media version {version.Id} ({path}) has no video stream on record, so it cannot be played. " + + "If its media statistics were never read, rescan the item's library to read them (for a " + + "Jellyfin/Emby/Plex library, a deep scan, since a regular scan skips items the server reports " + + "unchanged). If the file itself has no video stream, a rescan cannot help: remove it from " + + "video channels.")); + } public async Task> SelectAudioStream( MediaItemAudioVersion version, @@ -115,7 +138,7 @@ public class FFmpegStreamSelector : IFFmpegStreamSelector } break; - // let default fall through + // let default fall through } } catch (Exception ex) diff --git a/ErsatzTV.Scanner/Core/Jellyfin/JellyfinMusicVideoLibraryScanner.cs b/ErsatzTV.Scanner/Core/Jellyfin/JellyfinMusicVideoLibraryScanner.cs index 58a899ff0..b552572b2 100644 --- a/ErsatzTV.Scanner/Core/Jellyfin/JellyfinMusicVideoLibraryScanner.cs +++ b/ErsatzTV.Scanner/Core/Jellyfin/JellyfinMusicVideoLibraryScanner.cs @@ -19,6 +19,7 @@ public class JellyfinMusicVideoLibraryScanner : { private readonly IJellyfinApiClient _jellyfinApiClient; private readonly IJellyfinMusicVideoRepository _jellyfinMusicVideoRepository; + private readonly ILogger _logger; private readonly IMediaSourceRepository _mediaSourceRepository; private readonly IMetadataRepository _metadataRepository; private readonly IMusicVideoRepository _musicVideoRepository; @@ -50,6 +51,7 @@ public class JellyfinMusicVideoLibraryScanner : _mediaSourceRepository = mediaSourceRepository; _musicVideoRepository = musicVideoRepository; _metadataRepository = metadataRepository; + _logger = logger; } public async Task> ScanLibrary( @@ -97,6 +99,41 @@ public class JellyfinMusicVideoLibraryScanner : library, projectionFailures); + protected override async Task> GetMediaServerStatistics( + JellyfinConnectionParameters connectionParameters, + JellyfinLibrary library, + MediaItemScanResult result, + JellyfinMusicVideo incoming) + { + _logger.LogDebug("Refreshing {Attribute} for {Path}", "Jellyfin Statistics", result.LocalPath); + + Either maybeVersion = + await _jellyfinApiClient.GetPlaybackInfo( + connectionParameters.Address, + connectionParameters.AuthorizationHeader, + library, + incoming.ItemId); + + foreach (BaseError error in maybeVersion.LeftToSeq()) + { + _logger.LogWarning( + "Failed to get music video statistics from Jellyfin for {Path}: {Error}", + result.LocalPath, + error.ToString()); + } + + // Chapters arrive with the library-items projection, not with playback info. They are taken from the + // INCOMING projection, not from result.Item as JellyfinMovieLibraryScanner does: the music-video + // repository's GetByItemId does not eager-load chapters, so on an existing row result.Item's head version + // carries none to write. + foreach (MediaVersion version in maybeVersion.RightToSeq()) + { + version.Chapters = incoming.GetHeadVersion().Chapters ?? []; + } + + return maybeVersion.ToOption(); + } + protected override async Task>> UpdateMetadata( MediaItemScanResult result, JellyfinMusicVideo incoming, diff --git a/ErsatzTV.Scanner/Core/Metadata/MediaServerMusicVideoLibraryScanner.cs b/ErsatzTV.Scanner/Core/Metadata/MediaServerMusicVideoLibraryScanner.cs index 8eb9dbfa5..40b34a685 100644 --- a/ErsatzTV.Scanner/Core/Metadata/MediaServerMusicVideoLibraryScanner.cs +++ b/ErsatzTV.Scanner/Core/Metadata/MediaServerMusicVideoLibraryScanner.cs @@ -131,6 +131,7 @@ public abstract class MediaServerMusicVideoLibraryScanner> maybeMusicVideo = await ProcessMusicVideo( musicVideoRepository, + connectionParameters, library, libraryPath, incoming, @@ -250,6 +251,7 @@ public abstract class MediaServerMusicVideoLibraryScanner>> ProcessMusicVideo( IMediaServerMusicVideoRepository musicVideoRepository, + TConnectionParameters connectionParameters, TLibrary library, LibraryPath libraryPath, TMusicVideo incoming, @@ -279,7 +281,7 @@ public abstract class MediaServerMusicVideoLibraryScanner UpdateMetadata(result, incoming, cancellationToken)) - .BindT(result => UpdateStatistics(result, incoming)); + .BindT(result => UpdateStatistics(connectionParameters, library, result, incoming, deepScan)); } private async Task> GetOrAddArtist(LibraryPath libraryPath, TMusicVideo musicVideo) @@ -315,13 +317,40 @@ public abstract class MediaServerMusicVideoLibraryScanner scanResult.Item); } + // ersatztv#924: the library-items projection carries NO streams and no width/height (the server only reports + // those through its per-item playback info), so the projection's version must never be written as the + // statistics — doing so stored a version FFmpegStreamSelector cannot play, and, because + // IMetadataRepository.UpdateStatistics reconciles streams against the version it is given, removed every + // stream an existing row already had. Statistics come from the media server, under the same condition + // MediaServerMovieLibraryScanner.UpdateStatistics uses; when the server cannot supply them nothing is written, + // so stored streams are kept rather than replaced by the projection's empty list. (`result.IsAdded` is implied + // by the empty-streams arm while the projection carries no streams; it is kept so the condition stays the + // movie scanner's.) private async Task>> UpdateStatistics( + TConnectionParameters connectionParameters, + TLibrary library, MediaItemScanResult result, - TMusicVideo incoming) + TMusicVideo incoming, + bool deepScan) { - if (await _metadataRepository.UpdateStatistics(result.Item, incoming.GetHeadVersion())) + TMusicVideo existing = result.Item; + + if (deepScan || result.IsAdded || MediaServerEtag(existing) != MediaServerEtag(incoming) || + existing.GetHeadVersion().Streams is not { Count: > 0 }) { - result.IsUpdated = true; + Option maybeMediaVersion = await GetMediaServerStatistics( + connectionParameters, + library, + result, + incoming); + + foreach (MediaVersion mediaVersion in maybeMediaVersion) + { + if (await _metadataRepository.UpdateStatistics(result.Item, mediaVersion)) + { + result.IsUpdated = true; + } + } } return result; @@ -405,4 +434,13 @@ public abstract class MediaServerMusicVideoLibraryScanner result, TMusicVideo incoming, CancellationToken cancellationToken); + + // ersatztv#924: abstract rather than the movie scanner's virtual-returning-None, because a None default is + // exactly the defect — an implementation that forgot to override it would scan music videos with no streams. + // The returned version carries the chapters to store alongside the statistics. + protected abstract Task> GetMediaServerStatistics( + TConnectionParameters connectionParameters, + TLibrary library, + MediaItemScanResult result, + TMusicVideo incoming); } diff --git a/ErsatzTV.Tests/Integration/JellyfinMusicVideoLibraryScannerTests.cs b/ErsatzTV.Tests/Integration/JellyfinMusicVideoLibraryScannerTests.cs index 552ae640e..b5d6b64e0 100644 --- a/ErsatzTV.Tests/Integration/JellyfinMusicVideoLibraryScannerTests.cs +++ b/ErsatzTV.Tests/Integration/JellyfinMusicVideoLibraryScannerTests.cs @@ -726,6 +726,285 @@ public class JellyfinMusicVideoLibraryScannerTests metadata.Artists.Count.ShouldBe(1); } + // ersatztv#924: Jellyfin's library-items projection carries no streams and no width/height — those come only + // from the per-item playback info. These run against the REAL MetadataRepository so the assertions read what + // was persisted. Non-vacuous (measured 2026-10-21): restoring the scanner's former body, + // `UpdateStatistics(result.Item, incoming.GetHeadVersion())`, reddens all five tests below. + [Test] + public async Task ScanLibrary_Should_Store_PlaybackInfo_Statistics_For_A_New_MusicVideo() + { + int libraryPathId = await SeedLibraryPath("/data/music"); + const string VideoPath = "/data/music/artist1/song1.mkv"; + + JellyfinMusicVideo Incoming() + { + JellyfinMusicVideo musicVideo = BuildIncoming(VideoPath, "Artist 1", "Song 1"); + musicVideo.MediaVersions[0].Chapters = [Chapter(0, "Intro")]; + return musicVideo; + } + + var playbackCalls = new List(); + IJellyfinApiClient api = FakeApiWithPlaybackInfo( + itemId => + { + playbackCalls.Add(itemId); + return PlaybackInfo(1920, 1080); + }, + Incoming); + + (JellyfinMusicVideoLibraryScanner scanner, _) = + BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory)); + + (await scanner.ScanLibrary( + ConnectionParameters, + BuildLibrary(libraryPathId, "/data/music"), + deepScan: false, + CancellationToken.None)).IsRight.ShouldBeTrue(); + + playbackCalls.ShouldBe([ItemIdFor(VideoPath)]); + + MediaVersion version = await HeadVersionOf(VideoPath); + version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Video).ShouldBe(1); + version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Audio).ShouldBe(1); + version.Width.ShouldBe(1920); + version.Height.ShouldBe(1080); + + // the projection's chapters are stored alongside the statistics, not dropped by them + version.Chapters.Select(c => c.Title).ShouldBe(["Intro"]); + } + + // The remediation path for rows already stored without streams: a regular scan skips an item whose etag is + // unchanged (so it cannot heal them), and a deep scan re-reads the statistics. + [Test] + public async Task ScanLibrary_Deep_Scan_Should_Read_Statistics_For_An_Unchanged_MusicVideo_Without_Streams() + { + int libraryPathId = await SeedLibraryPath("/data/music"); + JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music"); + const string VideoPath = "/data/music/artist1/song1.mkv"; + + var playbackCalls = 0; + var serverAnswers = false; + (int Width, int Height) size = (1280, 720); + IJellyfinApiClient api = FakeApiWithPlaybackInfo( + _ => + { + playbackCalls++; + return serverAnswers + ? PlaybackInfo(size.Width, size.Height) + : BaseError.New("playback info unavailable"); + }, + () => BuildIncoming(VideoPath, "Artist 1", "Song 1")); + + (JellyfinMusicVideoLibraryScanner scanner, _) = + BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory)); + + // 1. the item is added while the server cannot supply statistics: stored with no streams + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + (await HeadVersionOf(VideoPath)).Streams.ShouldBeEmpty(); + playbackCalls.ShouldBe(1); + + // 2. a regular scan of the unchanged item does not look at it again + serverAnswers = true; + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + playbackCalls.ShouldBe(1); + (await HeadVersionOf(VideoPath)).Streams.ShouldBeEmpty(); + + // 3. a deep scan reads the statistics + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: true, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + playbackCalls.ShouldBe(2); + MediaVersion version = await HeadVersionOf(VideoPath); + version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Video).ShouldBe(1); + version.Width.ShouldBe(1280); + version.Height.ShouldBe(720); + + // 4. a deep scan re-reads them even when streams are already stored and the etag is unchanged + size = (1920, 1080); + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: true, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + playbackCalls.ShouldBe(3); + (await HeadVersionOf(VideoPath)).Width.ShouldBe(1920); + } + + // The "no stored streams" arm of the refresh condition: an Unavailable item whose file is back is scanned with + // an UNCHANGED etag and no deep scan, so only the empty stream list says its statistics must be read. + [Test] + public async Task ScanLibrary_Should_Read_Statistics_For_A_Returning_Unavailable_MusicVideo_Without_Streams() + { + int libraryPathId = await SeedLibraryPath("/data/music"); + JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music"); + const string VideoPath = "/data/music/artist1/song1.mkv"; + + var serverAnswers = false; + IJellyfinApiClient api = FakeApiWithPlaybackInfo( + _ => serverAnswers ? PlaybackInfo(1280, 720) : BaseError.New("playback info unavailable"), + () => BuildIncoming(VideoPath, "Artist 1", "Song 1")); + + (JellyfinMusicVideoLibraryScanner scanner, _) = + BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory)); + + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + + int musicVideoId = await MusicVideoId(VideoPath); + await using (TvContext context = _db.CreateContext()) + { + MediaItem mediaItem = await context.MediaItems.SingleAsync(mi => mi.Id == musicVideoId); + mediaItem.State = MediaItemState.Unavailable; + await context.SaveChangesAsync(); + } + + serverAnswers = true; + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + + MediaVersion version = await HeadVersionOf(VideoPath); + version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Video).ShouldBe(1); + version.Width.ShouldBe(1280); + (await MediaItemStateOf(musicVideoId)).ShouldBe(MediaItemState.Normal); + } + + // When the server cannot supply statistics on a rescan, nothing is written: the projection's empty stream + // list must not replace the streams already stored (IMetadataRepository.UpdateStatistics removes every + // stored stream the version it is given does not carry). + [Test] + public async Task ScanLibrary_Should_Keep_Stored_Statistics_When_PlaybackInfo_Fails_On_Rescan() + { + int libraryPathId = await SeedLibraryPath("/data/music"); + JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music"); + const string VideoPath = "/data/music/artist1/song1.mkv"; + + var title = "Song 1"; + IJellyfinApiClient api = FakeApiWithPlaybackInfo( + _ => BaseError.New("playback info unavailable"), + () => BuildIncoming(VideoPath, "Artist 1", title)); + + (JellyfinMusicVideoLibraryScanner scanner, _) = + BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory)); + + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + + // the row holds statistics from an earlier successful read + int versionId = (await HeadVersionOf(VideoPath)).Id; + await using (TvContext context = _db.CreateContext()) + { + MediaVersion stored = await context.MediaVersions.SingleAsync(v => v.Id == versionId); + stored.Width = 1920; + stored.Height = 1080; + await context.MediaStreams.AddAsync( + new MediaStream + { + MediaVersionId = versionId, + Index = 0, + MediaStreamKind = MediaStreamKind.Video, + Codec = "h264" + }); + await context.SaveChangesAsync(); + } + + // the server reports the item changed (BuildIncoming derives the etag from the title), but playback + // info fails + title = "Song 1 (Remastered)"; + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + + MediaVersion version = await HeadVersionOf(VideoPath); + version.Streams.Select(s => (s.Index, s.MediaStreamKind, s.Codec)) + .ShouldBe([(0, MediaStreamKind.Video, "h264")]); + version.Width.ShouldBe(1920); + version.Height.ShouldBe(1080); + } + + // Refreshed statistics take their chapters from the INCOMING projection: the repository does not eager-load + // chapters on an existing row, so JellyfinMovieLibraryScanner's source (the stored item) would have none to + // write. + [Test] + public async Task ScanLibrary_Should_Refresh_Statistics_And_Projection_Chapters_When_The_Etag_Changes() + { + int libraryPathId = await SeedLibraryPath("/data/music"); + JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music"); + const string VideoPath = "/data/music/artist1/song1.mkv"; + + var title = "Song 1"; + List chapters = [Chapter(0, "Intro")]; + (int Width, int Height) size = (1280, 720); + + JellyfinMusicVideo Incoming() + { + JellyfinMusicVideo musicVideo = BuildIncoming(VideoPath, "Artist 1", title); + musicVideo.MediaVersions[0].Chapters = chapters.Select(c => Chapter(c.ChapterId, c.Title)).ToList(); + return musicVideo; + } + + IJellyfinApiClient api = FakeApiWithPlaybackInfo(_ => PlaybackInfo(size.Width, size.Height), Incoming); + (JellyfinMusicVideoLibraryScanner scanner, _) = + BuildScanner(api, metadataRepository: new MetadataRepository(_db.Factory)); + + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + + title = "Song 1 (Remastered)"; + chapters = [Chapter(0, "Intro"), Chapter(1, "Chorus")]; + size = (1920, 1080); + (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None)) + .IsRight.ShouldBeTrue(); + + MediaVersion version = await HeadVersionOf(VideoPath); + version.Width.ShouldBe(1920); + version.Height.ShouldBe(1080); + version.Streams.Count(s => s.MediaStreamKind == MediaStreamKind.Video).ShouldBe(1); + version.Chapters.OrderBy(c => c.ChapterId).Select(c => c.Title).ShouldBe(["Intro", "Chorus"]); + } + + private static MediaChapter Chapter(long chapterId, string title) => + new() + { + ChapterId = chapterId, + Title = title, + StartTime = TimeSpan.FromSeconds(chapterId * 30), + EndTime = TimeSpan.FromSeconds((chapterId + 1) * 30) + }; + + // What JellyfinApiClient.GetPlaybackInfo returns for a playable file: a version carrying the streams and the + // dimensions. Built fresh per call, since UpdateStatistics attaches the stream objects to the stored version. + private static Either PlaybackInfo(int width, int height) => + new MediaVersion + { + Duration = TimeSpan.FromMinutes(4), + Width = width, + Height = height, + SampleAspectRatio = "1:1", + DisplayAspectRatio = "16:9", + RFrameRate = "23.976", + VideoScanKind = VideoScanKind.Progressive, + Streams = + [ + new MediaStream { Index = 0, MediaStreamKind = MediaStreamKind.Video, Codec = "h264" }, + new MediaStream { Index = 1, MediaStreamKind = MediaStreamKind.Audio, Codec = "aac", Channels = 2 } + ], + Chapters = [] + }; + + private async Task HeadVersionOf(string path) + { + await using TvContext context = _db.CreateContext(); + List musicVideos = await context.MusicVideos + .Include(mv => mv.MediaVersions) + .ThenInclude(v => v.MediaFiles) + .Include(mv => mv.MediaVersions) + .ThenInclude(v => v.Streams) + .Include(mv => mv.MediaVersions) + .ThenInclude(v => v.Chapters) + .ToListAsync(); + return musicVideos + .Single(mv => mv.MediaVersions.Single().MediaFiles.Single().Path == path) + .MediaVersions + .Single(); + } + private static JellyfinConnectionParameters ConnectionParameters => new("http://jellyfin", "api-key", 1); @@ -810,7 +1089,8 @@ public class JellyfinMusicVideoLibraryScannerTests private (JellyfinMusicVideoLibraryScanner Scanner, IScannerProxy ScannerProxy) BuildScanner( IJellyfinApiClient apiClient, - Func replacePath = null) + Func replacePath = null, + IMetadataRepository metadataRepository = null) { IScannerProxy scannerProxy = BuildScannerProxy(); @@ -823,7 +1103,7 @@ public class JellyfinMusicVideoLibraryScannerTests new ArtistRepository(_db.Factory), new MusicVideoRepository(_db.Factory), new LibraryRepository(Substitute.For(), _db.Factory), - Substitute.For(), + metadataRepository ?? Substitute.For(), BuildFileSystem(), NullLogger.Instance); @@ -912,7 +1192,14 @@ public class JellyfinMusicVideoLibraryScannerTests // Each Func builds a fresh music video so the async stream can be re-enumerated across ScanLibrary calls // (an IAsyncEnumerable iterator is single-use, and the scanner mutates the incoming item). - private static IJellyfinApiClient FakeApi(params Func[] items) + // Playback info succeeds by default (ersatztv#924: the scanner reads statistics from it for every scanned item). + private static IJellyfinApiClient FakeApi(params Func[] items) => + FakeApiWithPlaybackInfo(_ => PlaybackInfo(1920, 1080), items); + + // `playbackInfo` answers GetPlaybackInfo per item id, standing in for the server's per-item playback info. + private static IJellyfinApiClient FakeApiWithPlaybackInfo( + Func> playbackInfo, + params Func[] items) { var apiClient = Substitute.For(); apiClient.GetMusicVideoLibraryItems( @@ -921,9 +1208,16 @@ public class JellyfinMusicVideoLibraryScannerTests Arg.Any(), Arg.Any()) .Returns(_ => Items(items)); + StubPlaybackInfo(apiClient, playbackInfo); return apiClient; } + private static void StubPlaybackInfo( + IJellyfinApiClient apiClient, + Func> playbackInfo) => + apiClient.GetPlaybackInfo(Arg.Any(), Arg.Any(), Arg.Any(), Arg.Any()) + .Returns(ci => Task.FromResult(playbackInfo(ci.ArgAt(3)))); + // Same as FakeApi, but the enumeration also reports `failures` swallowed projection exceptions into the // counter the scanner handed in — what the real JellyfinApiClient does from ProjectToMusicVideo's catch. private static IJellyfinApiClient FakeApiWithProjectionFailures( @@ -940,6 +1234,7 @@ public class JellyfinMusicVideoLibraryScannerTests items, ci.ArgAt(3), failures)); + StubPlaybackInfo(apiClient, _ => PlaybackInfo(1920, 1080)); return apiClient; } diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 8d8fab068..04940d387 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -160,6 +160,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `scan.getoraddfolder-db-lookup` | `ILibraryRepository.GetOrAddFolder` resolves the existing folder via a DB query on `(LibraryPathId, Path)`, not the caller's `LibraryPath.LibraryFolders` in-memory navigation, since that navigation is only eager-loaded on the local scan path and is null on remote (Jellyfin) callers. | 2026-07-20 | [link](records/scan/getoraddfolder-db-lookup.md) | | `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](records/scan/jellyfin-mixed-content-library.md) | | `scan.libraryfolder-unique-identity` | `LibraryFolder` uniqueness per `(LibraryPathId, Path)` is enforced by a database unique index over a SHA-256 `PathHash` (Path is unbounded and not portably indexable), and `LibraryRepository.GetOrAddFolder`/`SetEtag` tolerate the constraint violation by re-reading and adopting the winner's row. | 2026-07-25 | [link](records/scan/libraryfolder-unique-identity.md) | +| `scan.mediaserver-statistics-from-server` | Every media-server scanner writes a head `MediaVersion`'s statistics (streams, width/height, aspect, frame rate) from what the server reports per item — `GetMediaServerStatistics` (Jellyfin/Emby playback info) or the metadata call that carries them (`ServerReturnsStatisticsWithMetadata`, Plex) — and NEVER writes the library-items projection's version, whose `Streams` list is empty. When the server cannot supply them, nothing is written. In `MediaServerMusicVideoLibraryScanner` the hook is `abstract`, not a virtual returning `None`. | 2026-10-21 | [link](records/scan/mediaserver-statistics-from-server.md) | | `scan.musicvideo-server-identity` | Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added. | 2026-07-25 | [link](records/scan/musicvideo-server-identity.md) | | `scan.projection-failure-sweep-guard` | `MediaServerReconciliationGuard` takes a per-enumeration projection-failure count and refuses the file-not-found sweep (logged loudly) whenever it is non-zero against a non-empty existing set — at all six sweeps, including the nested per-show season and per-season episode ones #477 left unguarded (via `ShouldFlagMissingDescendants`, which applies the failure refusal but not #477's empty-fetch branch). Deliberate guard-clause skips (STRM, virtual, unsupported type) are explicitly **not** failures and never suppress a sweep. The missing-fraction / ratio threshold floated by #477 is **rejected**, not deferred. | 2026-07-25 | [link](records/scan/projection-failure-sweep-guard.md) | | `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](records/scan/zero-item-fetch-guard.md) | diff --git a/docs/decisions/records/scan/mediaserver-statistics-from-server.md b/docs/decisions/records/scan/mediaserver-statistics-from-server.md new file mode 100644 index 000000000..8ea334881 --- /dev/null +++ b/docs/decisions/records/scan/mediaserver-statistics-from-server.md @@ -0,0 +1,43 @@ +--- +key: scan.mediaserver-statistics-from-server +title: 2026-10-21 — A media-server scanner reads a version's statistics from the server, never from the library-items projection (#924) +status: active +since: '2026-10-21' +supersedes: none +superseded-by: none +rule: 'Every media-server scanner writes a head `MediaVersion`''s statistics (streams, width/height, aspect, frame rate) from what the server reports per item — `GetMediaServerStatistics` (Jellyfin/Emby playback info) or the metadata call that carries them (`ServerReturnsStatisticsWithMetadata`, Plex) — and NEVER writes the library-items projection''s version, whose `Streams` list is empty. When the server cannot supply them, nothing is written. In `MediaServerMusicVideoLibraryScanner` the hook is `abstract`, not a virtual returning `None`.' +signals: 'zero MediaStream rows, Width/Height 0, Sequence contains no matching element, SelectVideoStream, playback info, GetPlaybackInfo, projection Streams = [], deep scan to heal, music videos will not play · paths: `MediaServerMusicVideoLibraryScanner.UpdateStatistics`, `JellyfinMusicVideoLibraryScanner.GetMediaServerStatistics`, `MediaServerMovieLibraryScanner.UpdateStatistics`, `IMetadataRepository.UpdateStatistics`, `FFmpegStreamSelector.SelectVideoStream` · issues: #924, #496' +mechanics: '`JellyfinMusicVideoLibraryScannerTests` (`ScanLibrary_Should_Store_PlaybackInfo_Statistics_For_A_New_MusicVideo`, `..._Deep_Scan_Should_Read_Statistics_For_An_Unchanged_MusicVideo_Without_Streams`, `..._Should_Read_Statistics_For_A_Returning_Unavailable_MusicVideo_Without_Streams`, `..._Should_Keep_Stored_Statistics_When_PlaybackInfo_Fails_On_Rescan`, `..._Should_Refresh_Statistics_And_Projection_Chapters_When_The_Etag_Changes`, against the real `MetadataRepository`); `FFmpegStreamSelectorTests.SelectVideoStream`' +--- + +The Jellyfin and Emby library-items projections build every movie, episode and music-video version with +`Streams = []` and no dimensions; only the per-item playback info carries them. The movie and TV scanners fetch +statistics when the item is new, changed, deep-scanned or has no stored streams. The music-video scanner wrote +the projection's version instead, so every Jellyfin music video was stored without streams and +`FFmpegStreamSelector.SelectVideoStream` threw on play. Measured on prod 2026-10-21 (#924): the Jellyfin "Music Videos" library (id 16), 1520 of +1520 music-video versions with zero `MediaStream` rows (movies 0, episodes 4). + +- **Writing the projection is worse than writing nothing.** `IMetadataRepository.UpdateStatistics` reconciles + the stored streams against the version it is handed, so an empty stream list REMOVES every stream an existing + row has. A failed statistics read therefore leaves the row untouched, matching the movie scanner. +- **The refresh condition is the movie scanner's**: `deepScan || IsAdded || etag changed || no stored streams`. + For music videos `IsAdded` is implied by the empty-streams arm while the projection carries no streams; it is + kept so the two conditions read the same. +- **`abstract`, not virtual-`None`.** The movie base defaults `GetMediaServerStatistics` to `None`; Jellyfin and + Emby override it, and Plex takes statistics from its metadata call (`ServerReturnsStatisticsWithMetadata`). + Music videos have one implementation, and a `None` default is the defect itself, so a future Emby/Plex + music-video scanner must answer the question to compile. +- **Chapters come from the incoming projection**, not from the stored item as in + `JellyfinMovieLibraryScanner`: `JellyfinMusicVideoRepository.GetByItemId` does not eager-load chapters, so the + stored item has none to write. +- **Healing stored rows takes a deep scan.** A regular scan skips an item whose etag is unchanged before the + refresh condition is reached, so rows already stored without streams stay that way until their library is + deep-scanned or the server changes the item. On a live instance that scan is a channel operation + (media-management), not part of shipping the fix. +- **The player names the gap.** `SelectVideoStream` on a version with no video stream fails with the version id, + its file and the remediation (rescan; deep scan for a media-server library) instead of `First()`'s bare + `Sequence contains no matching element`. +- **Known residual, shared with movies:** when the server cannot answer, the item's etag is still recorded, so a + transient playback-info failure on a new or changed item leaves it without streams until the next deep scan or + server-side change. Not changed here: skipping the etag write would retry the item on every scan, and the same + trade-off already holds for movies and episodes.