fix(496): give music videos a per-library server identity; itemId diff + soft trash

Music videos carried no server identity, so JellyfinMusicVideoLibraryScanner had to
reconcile by a (LibraryPathId, path) diff and HARD-delete the remainder. A file served
by two libraries with overlapping local paths is a single row owned by whichever library
scanned it first, so that owner's sweep destroyed a row another library still served —
taking collection membership and playout references with it, irreversibly.

This is #494's deferred "option 2":

- New JellyfinMusicVideo : MusicVideo (ItemId/Etag), mirroring JellyfinMovie — TPT table,
  varchar(36), ItemId index. Dual-provider migration Add_JellyfinMusicVideo.
- New IMediaServerMusicVideoRepository + JellyfinMusicVideoRepository: itemId-keyed
  existing-set/lookup and Flag{Normal,Unavailable,FileNotFound} seams, all scoped per
  library via LibraryPath.LibraryId.
- New MediaServerMusicVideoLibraryScanner base; JellyfinMusicVideoLibraryScanner folds
  onto it and keeps the #177/#488/#497/#500 metadata-reconcile logic verbatim.
- The sweep now soft-trashes (FileNotFound) instead of deleting, so removal is reversible
  and EmptyTrash-governed. DeleteEmptyArtists consequently no longer fires from a sweep.
- Pre-identity rows are ADOPTED in place: the identity row is inserted against the same
  MediaItem id, scoped to the scanned library's own LibraryPath, so collection membership
  survives and a local/second-library row is never hijacked.
- AddMusicVideo normalizes Path/PathHash to the path-REPLACED local path; the projection
  fills them from the server-reported path, which would break every later PathHash lookup.

Docs: scan.musicvideo-reconciliation relocated to docs/decisions/archive/scan.md as
superseded; new active record scan.musicvideo-server-identity.

fixes #496
This commit is contained in:
2026-07-25 17:20:53 +02:00
parent eb4de0cc2b
commit 64decd492e
22 changed files with 15996 additions and 580 deletions
@@ -0,0 +1,7 @@
namespace ErsatzTV.Core.Domain;
public class JellyfinMusicVideo : MusicVideo
{
public string ItemId { get; set; }
public string Etag { get; set; }
}
@@ -21,7 +21,7 @@ public interface IJellyfinApiClient
JellyfinLibrary library, JellyfinLibrary library,
MediaServerProjectionFailureCounter projectionFailures = null); MediaServerProjectionFailureCounter projectionFailures = null);
IAsyncEnumerable<Tuple<MusicVideo, int>> GetMusicVideoLibraryItems( IAsyncEnumerable<Tuple<JellyfinMusicVideo, int>> GetMusicVideoLibraryItems(
string address, string address,
string authorizationHeader, string authorizationHeader,
JellyfinLibrary library, JellyfinLibrary library,
@@ -0,0 +1,10 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Jellyfin;
namespace ErsatzTV.Core.Interfaces.Repositories;
public interface
IJellyfinMusicVideoRepository : IMediaServerMusicVideoRepository<JellyfinLibrary, JellyfinMusicVideo,
JellyfinItemEtag>
{
}
@@ -0,0 +1,26 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Metadata;
namespace ErsatzTV.Core.Interfaces.Repositories;
public interface IMediaServerMusicVideoRepository<in TLibrary, TMusicVideo, TEtag> where TLibrary : Library
where TMusicVideo : MusicVideo
where TEtag : MediaServerItemEtag
{
Task<List<TEtag>> GetExistingMusicVideos(TLibrary library);
Task<Option<int>> FlagNormal(TLibrary library, TMusicVideo musicVideo);
Task<Option<int>> FlagUnavailable(TLibrary library, TMusicVideo musicVideo);
Task<List<int>> FlagFileNotFound(TLibrary library, List<string> musicVideoItemIds);
// Unlike the movie/other-video seams, GetOrAdd takes the resolved Artist and LibraryFolder: a music video is
// owned by an Artist (FK, not null) and ersatztv#488 requires the media file to carry its LibraryFolderId.
Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> GetOrAdd(
TLibrary library,
Artist artist,
LibraryFolder libraryFolder,
TMusicVideo item,
bool deepScan,
CancellationToken cancellationToken);
Task<Unit> SetEtag(TMusicVideo musicVideo, string etag);
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,48 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.MySql.Migrations
{
/// <inheritdoc />
public partial class Add_JellyfinMusicVideo : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "JellyfinMusicVideo",
columns: table => new
{
Id = table.Column<int>(type: "int", nullable: false),
ItemId = table.Column<string>(type: "varchar(36)", unicode: false, maxLength: 36, nullable: true)
.Annotation("MySql:CharSet", "utf8mb4"),
Etag = table.Column<string>(type: "varchar(36)", unicode: false, maxLength: 36, nullable: true)
.Annotation("MySql:CharSet", "utf8mb4")
},
constraints: table =>
{
table.PrimaryKey("PK_JellyfinMusicVideo", x => x.Id);
table.ForeignKey(
name: "FK_JellyfinMusicVideo_MusicVideo_Id",
column: x => x.Id,
principalTable: "MusicVideo",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
})
.Annotation("MySql:CharSet", "utf8mb4");
migrationBuilder.CreateIndex(
name: "IX_JellyfinMusicVideo_ItemId",
table: "JellyfinMusicVideo",
column: "ItemId");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "JellyfinMusicVideo");
}
}
}
@@ -4452,6 +4452,25 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
b.ToTable("PlexMovie", (string)null); b.ToTable("PlexMovie", (string)null);
}); });
modelBuilder.Entity("ErsatzTV.Core.Domain.JellyfinMusicVideo", b =>
{
b.HasBaseType("ErsatzTV.Core.Domain.MusicVideo");
b.Property<string>("Etag")
.HasMaxLength(36)
.IsUnicode(false)
.HasColumnType("varchar(36)");
b.Property<string>("ItemId")
.HasMaxLength(36)
.IsUnicode(false)
.HasColumnType("varchar(36)");
b.HasIndex("ItemId");
b.ToTable("JellyfinMusicVideo", (string)null);
});
modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b => modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b =>
{ {
b.HasBaseType("ErsatzTV.Core.Domain.OtherVideo"); b.HasBaseType("ErsatzTV.Core.Domain.OtherVideo");
@@ -6730,6 +6749,15 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
.IsRequired(); .IsRequired();
}); });
modelBuilder.Entity("ErsatzTV.Core.Domain.JellyfinMusicVideo", b =>
{
b.HasOne("ErsatzTV.Core.Domain.MusicVideo", null)
.WithOne()
.HasForeignKey("ErsatzTV.Core.Domain.JellyfinMusicVideo", "Id")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b => modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b =>
{ {
b.HasOne("ErsatzTV.Core.Domain.OtherVideo", null) b.HasOne("ErsatzTV.Core.Domain.OtherVideo", null)
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,46 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
{
/// <inheritdoc />
public partial class Add_JellyfinMusicVideo : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "JellyfinMusicVideo",
columns: table => new
{
Id = table.Column<int>(type: "INTEGER", nullable: false)
.Annotation("Sqlite:Autoincrement", true),
ItemId = table.Column<string>(type: "TEXT", unicode: false, maxLength: 36, nullable: true),
Etag = table.Column<string>(type: "TEXT", unicode: false, maxLength: 36, nullable: true)
},
constraints: table =>
{
table.PrimaryKey("PK_JellyfinMusicVideo", x => x.Id);
table.ForeignKey(
name: "FK_JellyfinMusicVideo_MusicVideo_Id",
column: x => x.Id,
principalTable: "MusicVideo",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateIndex(
name: "IX_JellyfinMusicVideo_ItemId",
table: "JellyfinMusicVideo",
column: "ItemId");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "JellyfinMusicVideo");
}
}
}
@@ -4277,6 +4277,25 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
b.ToTable("PlexMovie", (string)null); b.ToTable("PlexMovie", (string)null);
}); });
modelBuilder.Entity("ErsatzTV.Core.Domain.JellyfinMusicVideo", b =>
{
b.HasBaseType("ErsatzTV.Core.Domain.MusicVideo");
b.Property<string>("Etag")
.HasMaxLength(36)
.IsUnicode(false)
.HasColumnType("TEXT");
b.Property<string>("ItemId")
.HasMaxLength(36)
.IsUnicode(false)
.HasColumnType("TEXT");
b.HasIndex("ItemId");
b.ToTable("JellyfinMusicVideo", (string)null);
});
modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b => modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b =>
{ {
b.HasBaseType("ErsatzTV.Core.Domain.OtherVideo"); b.HasBaseType("ErsatzTV.Core.Domain.OtherVideo");
@@ -6555,6 +6574,15 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
.IsRequired(); .IsRequired();
}); });
modelBuilder.Entity("ErsatzTV.Core.Domain.JellyfinMusicVideo", b =>
{
b.HasOne("ErsatzTV.Core.Domain.MusicVideo", null)
.WithOne()
.HasForeignKey("ErsatzTV.Core.Domain.JellyfinMusicVideo", "Id")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b => modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b =>
{ {
b.HasOne("ErsatzTV.Core.Domain.OtherVideo", null) b.HasOne("ErsatzTV.Core.Domain.OtherVideo", null)
@@ -0,0 +1,23 @@
using ErsatzTV.Core.Domain;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
namespace ErsatzTV.Infrastructure.Data.Configurations;
public class JellyfinMusicVideoConfiguration : IEntityTypeConfiguration<JellyfinMusicVideo>
{
public void Configure(EntityTypeBuilder<JellyfinMusicVideo> builder)
{
builder.ToTable("JellyfinMusicVideo");
builder.Property(mv => mv.Etag)
.HasMaxLength(36)
.IsUnicode(false);
builder.Property(mv => mv.ItemId)
.HasMaxLength(36)
.IsUnicode(false);
builder.HasIndex(mv => mv.ItemId);
}
}
@@ -0,0 +1,348 @@
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Extensions;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Jellyfin;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Infrastructure.Extensions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Infrastructure.Data.Repositories;
public class JellyfinMusicVideoRepository : IJellyfinMusicVideoRepository
{
private readonly IDbContextFactory<TvContext> _dbContextFactory;
private readonly ILogger<JellyfinMusicVideoRepository> _logger;
public JellyfinMusicVideoRepository(
IDbContextFactory<TvContext> dbContextFactory,
ILogger<JellyfinMusicVideoRepository> logger)
{
_dbContextFactory = dbContextFactory;
_logger = logger;
}
public async Task<List<JellyfinItemEtag>> GetExistingMusicVideos(JellyfinLibrary library)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
return await dbContext.Connection.QueryAsync<JellyfinItemEtag>(
@"SELECT ItemId, Etag, MI.State FROM JellyfinMusicVideo
INNER JOIN MusicVideo M on JellyfinMusicVideo.Id = M.Id
INNER JOIN MediaItem MI on M.Id = MI.Id
INNER JOIN LibraryPath LP on MI.LibraryPathId = LP.Id
WHERE LP.LibraryId = @LibraryId",
new { LibraryId = library.Id })
.Map(result => result.ToList());
}
public async Task<Option<int>> FlagNormal(JellyfinLibrary library, JellyfinMusicVideo musicVideo)
{
if (musicVideo.State is MediaItemState.Normal)
{
return Option<int>.None;
}
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
musicVideo.State = MediaItemState.Normal;
Option<int> maybeId = await dbContext.Connection.ExecuteScalarAsync<int>(
@"SELECT JellyfinMusicVideo.Id FROM JellyfinMusicVideo
INNER JOIN MediaItem MI ON MI.Id = JellyfinMusicVideo.Id
INNER JOIN LibraryPath LP on MI.LibraryPathId = LP.Id AND LibraryId = @LibraryId
WHERE JellyfinMusicVideo.ItemId = @ItemId",
new { LibraryId = library.Id, musicVideo.ItemId });
foreach (int id in maybeId)
{
return await dbContext.Connection.ExecuteAsync(
"UPDATE MediaItem SET State = 0 WHERE Id = @Id AND State != 0",
new { Id = id }).Map(count => count > 0 ? Some(id) : None);
}
return None;
}
public async Task<Option<int>> FlagUnavailable(JellyfinLibrary library, JellyfinMusicVideo musicVideo)
{
if (musicVideo.State is MediaItemState.Unavailable)
{
return Option<int>.None;
}
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
musicVideo.State = MediaItemState.Unavailable;
Option<int> maybeId = await dbContext.Connection.ExecuteScalarAsync<int>(
@"SELECT JellyfinMusicVideo.Id FROM JellyfinMusicVideo
INNER JOIN MediaItem MI ON MI.Id = JellyfinMusicVideo.Id
INNER JOIN LibraryPath LP on MI.LibraryPathId = LP.Id AND LibraryId = @LibraryId
WHERE JellyfinMusicVideo.ItemId = @ItemId",
new { LibraryId = library.Id, musicVideo.ItemId });
foreach (int id in maybeId)
{
return await dbContext.Connection.ExecuteAsync(
"UPDATE MediaItem SET State = 2 WHERE Id = @Id AND State != 2",
new { Id = id }).Map(count => count > 0 ? Some(id) : None);
}
return None;
}
public async Task<List<int>> FlagFileNotFound(JellyfinLibrary library, List<string> musicVideoItemIds)
{
if (musicVideoItemIds.Count == 0)
{
return [];
}
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
List<int> ids = await dbContext.Connection.QueryAsync<int>(
@"SELECT M.Id
FROM MediaItem M
INNER JOIN JellyfinMusicVideo ON JellyfinMusicVideo.Id = M.Id
INNER JOIN LibraryPath LP on M.LibraryPathId = LP.Id AND LP.LibraryId = @LibraryId
WHERE JellyfinMusicVideo.ItemId IN @MusicVideoItemIds",
new { LibraryId = library.Id, MusicVideoItemIds = musicVideoItemIds })
.Map(result => result.ToList());
await dbContext.Connection.ExecuteAsync(
"UPDATE MediaItem SET State = 1 WHERE Id IN @Ids AND State != 1",
new { Ids = ids });
return ids;
}
public async Task<Either<BaseError, MediaItemScanResult<JellyfinMusicVideo>>> GetOrAdd(
JellyfinLibrary library,
Artist artist,
LibraryFolder libraryFolder,
JellyfinMusicVideo item,
bool deepScan,
CancellationToken cancellationToken)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync(cancellationToken);
string path = item.GetHeadVersion().MediaFiles.Head().Path;
Option<JellyfinMusicVideo> maybeExisting = await GetByItemId(dbContext, item.ItemId, cancellationToken);
// ersatztv#496: rows created before this library gained per-item identity (and rows created by the
// pre-#496 path-keyed scanner) carry no JellyfinMusicVideo row, so they can never be found by ItemId.
// Adopt them IN PLACE — insert the identity row against the SAME MediaItem id — rather than deleting and
// re-adding: the MediaItem id is referenced by CollectionItem, playouts, artwork and the search index, so
// a delete/re-add would silently drop collection membership and break built playouts.
//
// Adoption is scoped to THIS library's own library path, which is the whole point of the issue: a music
// video owned by a local library (MusicVideoFolderScanner uses the same MusicVideo table) or by a second
// Jellyfin library must never be hijacked into this library's identity.
if (maybeExisting.IsNone)
{
Option<int> maybeAdopted = await AdoptExistingMusicVideo(
dbContext,
library.Paths.Head().Id,
path,
item.ItemId);
if (maybeAdopted.IsSome)
{
_logger.LogDebug(
"ADOPT: existing music video {Path} now carries Jellyfin item id {ItemId}",
path,
item.ItemId);
// the identity row is written with an empty etag, so the existing-item path below always sees an
// etag mismatch and refreshes the adopted row exactly like any other changed item
maybeExisting = await GetByItemId(dbContext, item.ItemId, cancellationToken);
}
}
foreach (JellyfinMusicVideo existing in maybeExisting)
{
var result = new MediaItemScanResult<JellyfinMusicVideo>(existing) { IsAdded = false };
// identity is now the server item id, so the file behind it can move; keep the local path, owning
// folder and artist in sync when the server says the item changed
if (existing.Etag != item.Etag || deepScan)
{
await UpdateMusicVideoFile(dbContext, existing.Id, artist.Id, libraryFolder.Id, path);
existing.ArtistId = artist.Id;
existing.Artist = artist;
result.IsUpdated = true;
}
else if (existing.ArtistId != artist.Id)
{
await dbContext.Connection.ExecuteAsync(
"UPDATE MusicVideo SET ArtistId = @ArtistId WHERE Id = @Id",
new { Id = existing.Id, ArtistId = artist.Id });
existing.ArtistId = artist.Id;
existing.Artist = artist;
result.IsUpdated = true;
}
return result;
}
return await AddMusicVideo(dbContext, library, artist, libraryFolder, item, path, cancellationToken);
}
public async Task<Unit> SetEtag(JellyfinMusicVideo musicVideo, string etag)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
return await dbContext.Connection.ExecuteAsync(
"UPDATE JellyfinMusicVideo SET Etag = @Etag WHERE Id = @Id",
new { Etag = etag, musicVideo.Id }).Map(_ => Unit.Default);
}
// Mirrors MusicVideoRepository.GetOrAdd's eager-load set: the scanner's metadata reconcile (ersatztv#497)
// only reconciles collections that are loaded here, so this list is load-bearing, not cosmetic.
private static async Task<Option<JellyfinMusicVideo>> GetByItemId(
TvContext dbContext,
string itemId,
CancellationToken cancellationToken) =>
await dbContext.JellyfinMusicVideos
.AsNoTracking()
.Include(mv => mv.Artist)
.ThenInclude(a => a.ArtistMetadata)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Artwork)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Artists)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Genres)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Tags)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Studios)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Directors)
.Include(mv => mv.LibraryPath)
.ThenInclude(lp => lp.Library)
.Include(mv => mv.MediaVersions)
.ThenInclude(mv => mv.MediaFiles)
.Include(mv => mv.MediaVersions)
.ThenInclude(mv => mv.Streams)
.Include(mv => mv.TraktListItems)
.ThenInclude(tli => tli.TraktList)
.SelectOneAsync(mv => mv.ItemId, mv => mv.ItemId == itemId, cancellationToken);
private static async Task<Option<int>> AdoptExistingMusicVideo(
TvContext dbContext,
int libraryPathId,
string path,
string itemId)
{
Option<int> maybeId = await dbContext.Connection.QuerySingleOrDefaultAsync<int?>(
@"SELECT M.Id
FROM MusicVideo M
INNER JOIN MediaItem MI on M.Id = MI.Id
INNER JOIN MediaVersion MV on M.Id = MV.MusicVideoId
INNER JOIN MediaFile MF on MV.Id = MF.MediaVersionId
WHERE MI.LibraryPathId = @LibraryPathId
AND MF.PathHash = @PathHash
AND NOT EXISTS (SELECT 1 FROM JellyfinMusicVideo J WHERE J.Id = M.Id)",
new { LibraryPathId = libraryPathId, PathHash = PathUtils.GetPathHash(path) })
.Map(Optional);
foreach (int id in maybeId)
{
await dbContext.Connection.ExecuteAsync(
"INSERT INTO JellyfinMusicVideo (Id, ItemId, Etag) VALUES (@Id, @ItemId, '')",
new { Id = id, ItemId = itemId });
return id;
}
return None;
}
private static async Task UpdateMusicVideoFile(
TvContext dbContext,
int musicVideoId,
int artistId,
int libraryFolderId,
string path)
{
await dbContext.Connection.ExecuteAsync(
"UPDATE MusicVideo SET ArtistId = @ArtistId WHERE Id = @Id",
new { Id = musicVideoId, ArtistId = artistId });
await dbContext.Connection.ExecuteAsync(
@"UPDATE MediaFile SET Path = @Path, PathHash = @PathHash, LibraryFolderId = @LibraryFolderId
WHERE Id IN (SELECT MF.Id
FROM MediaFile MF
INNER JOIN MediaVersion MV on MF.MediaVersionId = MV.Id
WHERE MV.MusicVideoId = @Id)",
new
{
Id = musicVideoId,
Path = path,
PathHash = PathUtils.GetPathHash(path),
LibraryFolderId = libraryFolderId
});
}
private async Task<Either<BaseError, MediaItemScanResult<JellyfinMusicVideo>>> AddMusicVideo(
TvContext dbContext,
JellyfinLibrary library,
Artist artist,
LibraryFolder libraryFolder,
JellyfinMusicVideo musicVideo,
string path,
CancellationToken cancellationToken)
{
try
{
if (await MediaItemRepository.MediaFileAlreadyExists(
path,
library.Paths.Head().Id,
dbContext,
_logger,
cancellationToken))
{
return new MediaFileAlreadyExists();
}
// blank out etag for initial save in case other updates fail
string etag = musicVideo.Etag;
musicVideo.Etag = string.Empty;
musicVideo.ArtistId = artist.Id;
musicVideo.LibraryPathId = library.Paths.Head().Id;
// Music videos store the PATH-REPLACED local path (they always have — the pre-#496 scanner passed the
// replaced path straight into GetOrAdd), while the projection fills Path/PathHash from the path the
// media server reported. Normalize both here, or the stored hash would be the hash of the server-side
// path and every later lookup by PathHash — MediaFileAlreadyExists, and #496's adoption probe — would
// miss the row it is meant to find.
MediaFile file = musicVideo.GetHeadVersion().MediaFiles.Head();
file.Path = path;
file.PathHash = PathUtils.GetPathHash(path);
file.LibraryFolderId = libraryFolder.Id;
await dbContext.AddAsync(musicVideo, cancellationToken);
await dbContext.SaveChangesAsync(cancellationToken);
// restore etag
musicVideo.Etag = etag;
await dbContext.Entry(musicVideo).Reference(mv => mv.Artist).LoadAsync(cancellationToken);
await dbContext.Entry(musicVideo.Artist).Collection(a => a.ArtistMetadata).LoadAsync(cancellationToken);
await dbContext.Entry(musicVideo).Reference(mv => mv.LibraryPath).LoadAsync(cancellationToken);
await dbContext.Entry(musicVideo.LibraryPath).Reference(lp => lp.Library).LoadAsync(cancellationToken);
return new MediaItemScanResult<JellyfinMusicVideo>(musicVideo) { IsAdded = true };
}
catch (Exception ex)
{
return BaseError.New(ex.ToString());
}
}
}
@@ -92,6 +92,7 @@ public class TvContext : DbContext
public DbSet<PlexEpisode> PlexEpisodes { get; set; } public DbSet<PlexEpisode> PlexEpisodes { get; set; }
public DbSet<PlexCollection> PlexCollections { get; set; } public DbSet<PlexCollection> PlexCollections { get; set; }
public DbSet<JellyfinMovie> JellyfinMovies { get; set; } public DbSet<JellyfinMovie> JellyfinMovies { get; set; }
public DbSet<JellyfinMusicVideo> JellyfinMusicVideos { get; set; }
public DbSet<JellyfinShow> JellyfinShows { get; set; } public DbSet<JellyfinShow> JellyfinShows { get; set; }
public DbSet<JellyfinSeason> JellyfinSeasons { get; set; } public DbSet<JellyfinSeason> JellyfinSeasons { get; set; }
public DbSet<JellyfinEpisode> JellyfinEpisodes { get; set; } public DbSet<JellyfinEpisode> JellyfinEpisodes { get; set; }
@@ -95,7 +95,7 @@ public class JellyfinApiClient : IJellyfinApiClient
(maybeLibrary, item) => WithLibrary(maybeLibrary, lib => ProjectToMovie(lib, item)), (maybeLibrary, item) => WithLibrary(maybeLibrary, lib => ProjectToMovie(lib, item)),
projectionFailures); projectionFailures);
public IAsyncEnumerable<Tuple<MusicVideo, int>> GetMusicVideoLibraryItems( public IAsyncEnumerable<Tuple<JellyfinMusicVideo, int>> GetMusicVideoLibraryItems(
string address, string address,
string authorizationHeader, string authorizationHeader,
JellyfinLibrary library, JellyfinLibrary library,
@@ -701,7 +701,7 @@ public class JellyfinApiClient : IJellyfinApiClient
return metadata; return metadata;
} }
private MediaServerProjectionResult<MusicVideo> ProjectToMusicVideo( private MediaServerProjectionResult<JellyfinMusicVideo> ProjectToMusicVideo(
JellyfinLibrary library, JellyfinLibrary library,
JellyfinLibraryItemResponse item) JellyfinLibraryItemResponse item)
{ {
@@ -709,13 +709,13 @@ public class JellyfinApiClient : IJellyfinApiClient
{ {
if (item.LocationType != "FileSystem") if (item.LocationType != "FileSystem")
{ {
return MediaServerProjectionResult<MusicVideo>.Skipped(); return MediaServerProjectionResult<JellyfinMusicVideo>.Skipped();
} }
if (Path.GetExtension(item.Path)?.ToLowerInvariant() == ".strm") if (Path.GetExtension(item.Path)?.ToLowerInvariant() == ".strm")
{ {
_logger.LogInformation("STRM files are not supported; skipping {Path}", item.Path); _logger.LogInformation("STRM files are not supported; skipping {Path}", item.Path);
return MediaServerProjectionResult<MusicVideo>.Skipped(); return MediaServerProjectionResult<JellyfinMusicVideo>.Skipped();
} }
string path = item.Path ?? string.Empty; string path = item.Path ?? string.Empty;
@@ -752,8 +752,10 @@ public class JellyfinApiClient : IJellyfinApiClient
MusicVideoMetadata metadata = ProjectToMusicVideoMetadata(item); MusicVideoMetadata metadata = ProjectToMusicVideoMetadata(item);
var musicVideo = new MusicVideo var musicVideo = new JellyfinMusicVideo
{ {
ItemId = item.Id,
Etag = item.Etag,
MediaVersions = [version], MediaVersions = [version],
MusicVideoMetadata = [metadata], MusicVideoMetadata = [metadata],
TraktListItems = [] TraktListItems = []
@@ -764,7 +766,7 @@ public class JellyfinApiClient : IJellyfinApiClient
catch (Exception ex) catch (Exception ex)
{ {
_logger.LogWarning(ex, "Error projecting Jellyfin music video"); _logger.LogWarning(ex, "Error projecting Jellyfin music video");
return MediaServerProjectionResult<MusicVideo>.Failed(); return MediaServerProjectionResult<JellyfinMusicVideo>.Failed();
} }
} }
@@ -1,6 +1,6 @@
using System.IO.Abstractions;
using ErsatzTV.Core; using ErsatzTV.Core;
using ErsatzTV.Core.Domain; using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Extensions; using ErsatzTV.Core.Extensions;
using ErsatzTV.Core.Interfaces.Jellyfin; using ErsatzTV.Core.Interfaces.Jellyfin;
using ErsatzTV.Core.Interfaces.Repositories; using ErsatzTV.Core.Interfaces.Repositories;
@@ -12,40 +12,44 @@ using Microsoft.Extensions.Logging;
namespace ErsatzTV.Scanner.Core.Jellyfin; namespace ErsatzTV.Scanner.Core.Jellyfin;
public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanner public class JellyfinMusicVideoLibraryScanner :
MediaServerMusicVideoLibraryScanner<JellyfinConnectionParameters, JellyfinLibrary, JellyfinMusicVideo,
JellyfinItemEtag>,
IJellyfinMusicVideoLibraryScanner
{ {
private const string UnknownArtist = "Unknown Artist";
private readonly IArtistRepository _artistRepository;
private readonly IJellyfinApiClient _jellyfinApiClient; private readonly IJellyfinApiClient _jellyfinApiClient;
private readonly IJellyfinPathReplacementService _pathReplacementService; private readonly IJellyfinMusicVideoRepository _jellyfinMusicVideoRepository;
private readonly ILibraryRepository _libraryRepository;
private readonly ILogger<JellyfinMusicVideoLibraryScanner> _logger;
private readonly IMediaSourceRepository _mediaSourceRepository; private readonly IMediaSourceRepository _mediaSourceRepository;
private readonly IMetadataRepository _metadataRepository; private readonly IMetadataRepository _metadataRepository;
private readonly IMusicVideoRepository _musicVideoRepository; private readonly IMusicVideoRepository _musicVideoRepository;
private readonly IScannerProxy _scannerProxy; private readonly IJellyfinPathReplacementService _pathReplacementService;
public JellyfinMusicVideoLibraryScanner( public JellyfinMusicVideoLibraryScanner(
IScannerProxy scannerProxy, IScannerProxy scannerProxy,
IJellyfinApiClient jellyfinApiClient, IJellyfinApiClient jellyfinApiClient,
IJellyfinMusicVideoRepository jellyfinMusicVideoRepository,
IJellyfinPathReplacementService pathReplacementService, IJellyfinPathReplacementService pathReplacementService,
IMediaSourceRepository mediaSourceRepository, IMediaSourceRepository mediaSourceRepository,
IArtistRepository artistRepository, IArtistRepository artistRepository,
IMusicVideoRepository musicVideoRepository, IMusicVideoRepository musicVideoRepository,
ILibraryRepository libraryRepository, ILibraryRepository libraryRepository,
IMetadataRepository metadataRepository, IMetadataRepository metadataRepository,
IFileSystem fileSystem,
ILogger<JellyfinMusicVideoLibraryScanner> logger) ILogger<JellyfinMusicVideoLibraryScanner> logger)
: base(
scannerProxy,
fileSystem,
artistRepository,
libraryRepository,
metadataRepository,
logger)
{ {
_scannerProxy = scannerProxy;
_jellyfinApiClient = jellyfinApiClient; _jellyfinApiClient = jellyfinApiClient;
_jellyfinMusicVideoRepository = jellyfinMusicVideoRepository;
_pathReplacementService = pathReplacementService; _pathReplacementService = pathReplacementService;
_mediaSourceRepository = mediaSourceRepository; _mediaSourceRepository = mediaSourceRepository;
_artistRepository = artistRepository;
_musicVideoRepository = musicVideoRepository; _musicVideoRepository = musicVideoRepository;
_libraryRepository = libraryRepository;
_metadataRepository = metadataRepository; _metadataRepository = metadataRepository;
_logger = logger;
} }
public async Task<Either<BaseError, Unit>> ScanLibrary( public async Task<Either<BaseError, Unit>> ScanLibrary(
@@ -53,212 +57,50 @@ public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanne
JellyfinLibrary library, JellyfinLibrary library,
bool deepScan, bool deepScan,
CancellationToken cancellationToken) CancellationToken cancellationToken)
{
try
{ {
Option<LibraryPath> maybeLibraryPath = Optional(library.Paths).Flatten().HeadOrNone(); Option<LibraryPath> maybeLibraryPath = Optional(library.Paths).Flatten().HeadOrNone();
return await maybeLibraryPath.Match( foreach (LibraryPath libraryPath in maybeLibraryPath)
libraryPath => ScanLibrary(connectionParameters, library, libraryPath, cancellationToken),
() => Task.FromResult<Either<BaseError, Unit>>(
BaseError.New($"Jellyfin library {library.Id} has no library path")));
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException)
{
return new ScanCanceled();
}
}
private async Task<Either<BaseError, Unit>> ScanLibrary(
JellyfinConnectionParameters connectionParameters,
JellyfinLibrary library,
LibraryPath libraryPath,
CancellationToken cancellationToken)
{ {
List<JellyfinPathReplacement> pathReplacements = List<JellyfinPathReplacement> pathReplacements =
await _mediaSourceRepository.GetJellyfinPathReplacements(library.MediaSourceId); await _mediaSourceRepository.GetJellyfinPathReplacements(library.MediaSourceId);
var processed = 0; string GetLocalPath(JellyfinMusicVideo musicVideo) =>
var incomingPaths = new List<string>();
// #484: one counter per enumeration, created here and read only after the enumeration completes.
// It is never a field on the (singleton) api client, so concurrent scans of different libraries
// cannot leak failures into each other's sweep decision.
var projectionFailures = new MediaServerProjectionFailureCounter();
await foreach ((MusicVideo incoming, int totalCount) in _jellyfinApiClient
.GetMusicVideoLibraryItems(
connectionParameters.Address,
connectionParameters.AuthorizationHeader,
library,
projectionFailures)
.WithCancellation(cancellationToken))
{
if (cancellationToken.IsCancellationRequested)
{
return new ScanCanceled();
}
processed++;
decimal percentCompletion = totalCount == 0 ? 1 : Math.Clamp((decimal)processed / totalCount, 0, 1);
if (!await _scannerProxy.UpdateProgress(percentCompletion, cancellationToken))
{
return new ScanCanceled();
}
incomingPaths.Add(GetLocalPath(pathReplacements, incoming));
Either<BaseError, MediaItemScanResult<MusicVideo>> maybeMusicVideo =
await ProcessMusicVideo(library, libraryPath, pathReplacements, incoming, cancellationToken);
foreach (BaseError error in maybeMusicVideo.LeftToSeq())
{
_logger.LogWarning("Error processing Jellyfin music video: {Error}", error.Value);
}
foreach (MediaItemScanResult<MusicVideo> result in maybeMusicVideo.RightToSeq()
.Filter(result => result.IsAdded || result.IsUpdated))
{
if (!await _scannerProxy.ReindexMediaItems([result.Item.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
}
await TrashMissingMusicVideos(
library,
libraryPath,
incomingPaths,
projectionFailures,
cancellationToken);
return Unit.Default;
}
// ersatztv#494: remove music videos (and now-empty artists) that Jellyfin no longer reports.
//
// Identity is LIBRARY-SCOPED by (LibraryPathId, path): FindMusicVideoPaths and DeleteByPath both filter
// LibraryPathId AND join the concrete MusicVideo table, so this can never touch a Movie/Show that shares
// the same LibraryPath (a mixed library) — the cross-delete safety is a property of those queries, not of
// the media kind. Unlike the MediaServer{Movie,Television,OtherVideo} base scanners, music videos carry no
// server ItemId/Etag (there is no JellyfinMusicVideo entity), so we diff on the local path instead of the
// server item id, and hard-delete rather than soft-trash (there is no per-item FileNotFound seam here).
//
// Known limitation: MusicVideoRepository.GetOrAdd dedups a path GLOBALLY (no LibraryPathId predicate), so a
// file served by two libraries with overlapping local paths is a single row owned by whichever library
// scanned it first. If that owning library later stops reporting the file while another library still
// serves it, this sweep removes the shared row. A proper fix needs per-library music-video identity (a
// JellyfinMusicVideo etag entity + migration) — the issue's deferred "option 2"; tracked as a follow-up.
private async Task TrashMissingMusicVideos(
JellyfinLibrary library,
LibraryPath libraryPath,
List<string> incomingPaths,
MediaServerProjectionFailureCounter projectionFailures,
CancellationToken cancellationToken)
{
var existingPaths = (await _musicVideoRepository.FindMusicVideoPaths(libraryPath)).ToList();
// #477: refuse the sweep when a successful fetch returned zero items but rows exist locally — an empty
// incoming set is indistinguishable from a transient error and would otherwise wipe the whole library.
// #484: also refuse when the api client silently dropped items whose projection threw — those are
// items Jellyfin DID return, so treating them as deletions would hard-delete healthy rows.
if (!MediaServerReconciliationGuard.ShouldFlagMissing(
_logger,
library.Name,
incomingPaths.Count,
existingPaths.Count,
projectionFailures.Count))
{
return;
}
foreach (string path in existingPaths.Except(incomingPaths))
{
List<int> musicVideoIds = await _musicVideoRepository.DeleteByPath(libraryPath, path);
if (musicVideoIds.Count > 0 &&
!await _scannerProxy.RemoveMediaItems(musicVideoIds.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to remove media items from scanner process");
}
}
List<int> artistIds = await _artistRepository.DeleteEmptyArtists(libraryPath);
if (artistIds.Count > 0 &&
!await _scannerProxy.RemoveMediaItems(artistIds.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to remove empty artists from scanner process");
}
}
private async Task<Either<BaseError, MediaItemScanResult<MusicVideo>>> ProcessMusicVideo(
JellyfinLibrary library,
LibraryPath libraryPath,
List<JellyfinPathReplacement> pathReplacements,
MusicVideo incoming,
CancellationToken cancellationToken)
{
string localPath = GetLocalPath(pathReplacements, incoming);
string folder = Path.GetDirectoryName(localPath) ?? libraryPath.Path;
Option<int> maybeParentFolder = await _libraryRepository.GetParentFolderId(libraryPath, folder, cancellationToken);
LibraryFolder libraryFolder = await _libraryRepository.GetOrAddFolder(libraryPath, maybeParentFolder, folder);
return await GetOrAddArtist(libraryPath, incoming)
.BindT(artist => _musicVideoRepository.GetOrAdd(artist, libraryPath, libraryFolder, localPath))
.BindT(result => UpdateMusicVideo(result, incoming, localPath, cancellationToken));
}
private string GetLocalPath(List<JellyfinPathReplacement> pathReplacements, MusicVideo musicVideo) =>
_pathReplacementService.GetReplacementJellyfinPath( _pathReplacementService.GetReplacementJellyfinPath(
pathReplacements, pathReplacements,
musicVideo.GetHeadVersion().MediaFiles.Head().Path, musicVideo.GetHeadVersion().MediaFiles.Head().Path,
false); false);
private async Task<Either<BaseError, Artist>> GetOrAddArtist(LibraryPath libraryPath, MusicVideo musicVideo) return await ScanLibrary(
{ _jellyfinMusicVideoRepository,
string artistName = musicVideo.MusicVideoMetadata connectionParameters,
.HeadOrNone() library,
.Bind(metadata => Optional(metadata.Artists).Flatten().HeadOrNone()) libraryPath,
.Map(artist => artist.Name) GetLocalPath,
.IfNone(UnknownArtist); deepScan,
cancellationToken);
var metadata = new ArtistMetadata
{
MetadataKind = MetadataKind.External,
Title = artistName,
SortTitle = SortTitle.GetSortTitle(artistName),
DateAdded = DateTime.UtcNow,
Genres = [],
Styles = [],
Moods = [],
Artwork = [],
Guids = [],
Subtitles = []
};
Option<Artist> maybeArtist = await _artistRepository.GetArtistByMetadata(libraryPath.Id, metadata);
foreach (Artist artist in maybeArtist)
{
return artist;
} }
Either<BaseError, MediaItemScanResult<Artist>> result = return BaseError.New($"Jellyfin library {library.Id} has no library path");
await _artistRepository.AddArtist(libraryPath.Id, artistName, metadata);
return result.Map(scanResult => scanResult.Item);
} }
private async Task<Either<BaseError, MediaItemScanResult<MusicVideo>>> UpdateMusicVideo( protected override string MediaServerItemId(JellyfinMusicVideo musicVideo) => musicVideo.ItemId;
MediaItemScanResult<MusicVideo> result,
MusicVideo incoming, protected override string MediaServerEtag(JellyfinMusicVideo musicVideo) => musicVideo.Etag;
string localPath,
protected override IAsyncEnumerable<Tuple<JellyfinMusicVideo, int>> GetMusicVideoLibraryItems(
JellyfinConnectionParameters connectionParameters,
JellyfinLibrary library) =>
_jellyfinApiClient.GetMusicVideoLibraryItems(
connectionParameters.Address,
connectionParameters.AuthorizationHeader,
library);
protected override async Task<Either<BaseError, MediaItemScanResult<JellyfinMusicVideo>>> UpdateMetadata(
MediaItemScanResult<JellyfinMusicVideo> result,
JellyfinMusicVideo incoming,
CancellationToken cancellationToken) CancellationToken cancellationToken)
{ {
result.LocalPath = localPath; if (await UpdateMetadata(result.Item, incoming.MusicVideoMetadata.Head()))
MusicVideoMetadata incomingMetadata = incoming.MusicVideoMetadata.Head();
bool updated = await UpdateMetadata(result.Item, incomingMetadata);
updated = await _metadataRepository.UpdateStatistics(result.Item, incoming.GetHeadVersion()) || updated;
if (updated)
{ {
result.IsUpdated = true; result.IsUpdated = true;
} }
@@ -289,7 +131,7 @@ public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanne
bool updated = await _metadataRepository.Update(existing); bool updated = await _metadataRepository.Update(existing);
// ersatztv#497: the scalar Update above marks only the metadata row Modified; it does NOT touch // ersatztv#497: the scalar Update above marks only the metadata row Modified; it does NOT touch
// child collections, and MusicVideoRepository.GetOrAdd loads them AsNoTracking — so tag/genre/studio/ // child collections, and the repository's GetOrAdd loads them AsNoTracking — so tag/genre/studio/
// artist edits made in Jellyfin never reached an EXISTING music video (only the Add path persisted // artist edits made in Jellyfin never reached an EXISTING music video (only the Add path persisted
// them). Reconcile the collections that BOTH the Add path persists AND GetOrAdd eager-loads: // them). Reconcile the collections that BOTH the Add path persists AND GetOrAdd eager-loads:
// Genres, Tags, Studios, Artists. (Guids are add-persisted but not eager-loaded here — reconciling // Genres, Tags, Studios, Artists. (Guids are add-persisted but not eager-loaded here — reconciling
@@ -0,0 +1,376 @@
using System.Collections.Immutable;
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.MediaServer;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Extensions;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Scanner.Core.Interfaces;
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Scanner.Core.Metadata;
// ersatztv#496: music videos used to be reconciled by a LIBRARY-SCOPED local-path diff plus hard delete, because
// they carried no server identity. They now carry one (JellyfinMusicVideo.ItemId/Etag), so this scanner is the
// music-video sibling of MediaServer{Movie,Television,OtherVideo}LibraryScanner: it diffs on the server item id
// and soft-trashes (FileNotFound) instead of deleting, which is reversible and EmptyTrash-governed.
public abstract class MediaServerMusicVideoLibraryScanner<TConnectionParameters, TLibrary, TMusicVideo, TEtag>
where TConnectionParameters : MediaServerConnectionParameters
where TLibrary : Library
where TMusicVideo : MusicVideo
where TEtag : MediaServerItemEtag
{
private const string UnknownArtist = "Unknown Artist";
private readonly IArtistRepository _artistRepository;
private readonly IFileSystem _fileSystem;
private readonly ILibraryRepository _libraryRepository;
private readonly ILogger _logger;
private readonly IMetadataRepository _metadataRepository;
private readonly IScannerProxy _scannerProxy;
protected MediaServerMusicVideoLibraryScanner(
IScannerProxy scannerProxy,
IFileSystem fileSystem,
IArtistRepository artistRepository,
ILibraryRepository libraryRepository,
IMetadataRepository metadataRepository,
ILogger logger)
{
_scannerProxy = scannerProxy;
_fileSystem = fileSystem;
_artistRepository = artistRepository;
_libraryRepository = libraryRepository;
_metadataRepository = metadataRepository;
_logger = logger;
}
protected async Task<Either<BaseError, Unit>> ScanLibrary(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TConnectionParameters connectionParameters,
TLibrary library,
LibraryPath libraryPath,
Func<TMusicVideo, string> getLocalPath,
bool deepScan,
CancellationToken cancellationToken)
{
try
{
return await ScanLibrary(
musicVideoRepository,
connectionParameters,
library,
libraryPath,
getLocalPath,
GetMusicVideoLibraryItems(connectionParameters, library),
deepScan,
cancellationToken);
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException)
{
return new ScanCanceled();
}
}
private async Task<Either<BaseError, Unit>> ScanLibrary(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TConnectionParameters connectionParameters,
TLibrary library,
LibraryPath libraryPath,
Func<TMusicVideo, string> getLocalPath,
IAsyncEnumerable<Tuple<TMusicVideo, int>> musicVideoEntries,
bool deepScan,
CancellationToken cancellationToken)
{
var incomingItemIds = new List<string>();
var existingMusicVideos = (await musicVideoRepository.GetExistingMusicVideos(library))
.ToImmutableDictionary(e => e.MediaServerItemId, e => e);
await foreach ((TMusicVideo incoming, int totalMusicVideoCount) in musicVideoEntries.WithCancellation(
cancellationToken))
{
if (cancellationToken.IsCancellationRequested)
{
return new ScanCanceled();
}
incomingItemIds.Add(MediaServerItemId(incoming));
decimal percentCompletion = totalMusicVideoCount == 0
? 1
: Math.Clamp((decimal)incomingItemIds.Count / totalMusicVideoCount, 0, 1);
if (!await _scannerProxy.UpdateProgress(percentCompletion, cancellationToken))
{
return new ScanCanceled();
}
string localPath = getLocalPath(incoming);
if (!await ShouldScanItem(
musicVideoRepository,
library,
existingMusicVideos,
incoming,
localPath,
deepScan))
{
continue;
}
Either<BaseError, MediaItemScanResult<TMusicVideo>> maybeMusicVideo =
await ProcessMusicVideo(
musicVideoRepository,
library,
libraryPath,
incoming,
localPath,
deepScan,
cancellationToken);
if (maybeMusicVideo.IsLeft)
{
foreach (BaseError error in maybeMusicVideo.LeftToSeq())
{
_logger.LogWarning("Error processing music video: {Error}", error.Value);
}
continue;
}
foreach (MediaItemScanResult<TMusicVideo> result in maybeMusicVideo.RightToSeq())
{
await musicVideoRepository.SetEtag(result.Item, MediaServerEtag(incoming));
if (_fileSystem.File.Exists(result.LocalPath))
{
Option<int> flagResult = await musicVideoRepository.FlagNormal(library, result.Item);
if (flagResult.IsSome)
{
result.IsUpdated = true;
}
}
else
{
Option<int> flagResult = await musicVideoRepository.FlagUnavailable(library, result.Item);
if (flagResult.IsSome)
{
result.IsUpdated = true;
}
}
if (result.IsAdded || result.IsUpdated)
{
if (!await _scannerProxy.ReindexMediaItems([result.Item.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
}
}
await TrashMissingMusicVideos(
musicVideoRepository,
library,
libraryPath,
incomingItemIds,
existingMusicVideos,
cancellationToken);
return Unit.Default;
}
// Soft-trash music videos the media server no longer reports. Identity is the SERVER ITEM ID, scoped to this
// library by GetExistingMusicVideos/FlagFileNotFound (both join LibraryPath.LibraryId), so a file served by two
// libraries with overlapping local paths is two independent identities and one library's sweep can no longer
// reach the other's row — the ersatztv#496 cross-library false-trash this replaced.
private async Task TrashMissingMusicVideos(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TLibrary library,
LibraryPath libraryPath,
List<string> incomingItemIds,
ImmutableDictionary<string, TEtag> existingMusicVideos,
CancellationToken cancellationToken)
{
// ersatztv#477: refuse the sweep when a successful fetch returned zero items but rows exist locally — an
// empty incoming set is indistinguishable from a transient error and would otherwise flag a whole library.
if (!MediaServerReconciliationGuard.ShouldFlagMissing(
_logger,
library.Name,
incomingItemIds.Count,
existingMusicVideos.Count))
{
return;
}
var fileNotFoundItemIds = existingMusicVideos.Keys.Except(incomingItemIds).ToList();
List<int> ids = await musicVideoRepository.FlagFileNotFound(library, fileNotFoundItemIds);
if (ids.Count > 0 && !await _scannerProxy.ReindexMediaItems(ids.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
// Trashed music videos are still rows, so an artist only becomes empty once the trash is emptied (or its
// items are removed some other way); this keeps ersatztv#494's empty-artist cleanup without the delete.
List<int> artistIds = await _artistRepository.DeleteEmptyArtists(libraryPath);
if (artistIds.Count > 0 && !await _scannerProxy.RemoveMediaItems(artistIds.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to remove empty artists from scanner process");
}
}
private async Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> ProcessMusicVideo(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TLibrary library,
LibraryPath libraryPath,
TMusicVideo incoming,
string localPath,
bool deepScan,
CancellationToken cancellationToken)
{
// ersatztv#488: resolve the owning folder from the DB, never from libraryPath.LibraryFolders (that
// navigation is only eager-loaded on the local scan path and is null for media-server callers).
string folder = Path.GetDirectoryName(localPath) ?? libraryPath.Path;
Option<int> maybeParentFolder =
await _libraryRepository.GetParentFolderId(libraryPath, folder, cancellationToken);
LibraryFolder libraryFolder = await _libraryRepository.GetOrAddFolder(libraryPath, maybeParentFolder, folder);
return await GetOrAddArtist(libraryPath, incoming)
.BindT(artist => musicVideoRepository.GetOrAdd(
library,
artist,
libraryFolder,
incoming,
deepScan,
cancellationToken))
.MapT(result =>
{
result.LocalPath = localPath;
return result;
})
.BindT(result => UpdateMetadata(result, incoming, cancellationToken))
.BindT(result => UpdateStatistics(result, incoming));
}
private async Task<Either<BaseError, Artist>> GetOrAddArtist(LibraryPath libraryPath, TMusicVideo musicVideo)
{
string artistName = musicVideo.MusicVideoMetadata
.HeadOrNone()
.Bind(metadata => Optional(metadata.Artists).Flatten().HeadOrNone())
.Map(artist => artist.Name)
.IfNone(UnknownArtist);
var metadata = new ArtistMetadata
{
MetadataKind = MetadataKind.External,
Title = artistName,
SortTitle = SortTitle.GetSortTitle(artistName),
DateAdded = DateTime.UtcNow,
Genres = [],
Styles = [],
Moods = [],
Artwork = [],
Guids = [],
Subtitles = []
};
Option<Artist> maybeArtist = await _artistRepository.GetArtistByMetadata(libraryPath.Id, metadata);
foreach (Artist artist in maybeArtist)
{
return artist;
}
Either<BaseError, MediaItemScanResult<Artist>> result =
await _artistRepository.AddArtist(libraryPath.Id, artistName, metadata);
return result.Map(scanResult => scanResult.Item);
}
private async Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> UpdateStatistics(
MediaItemScanResult<TMusicVideo> result,
TMusicVideo incoming)
{
if (await _metadataRepository.UpdateStatistics(result.Item, incoming.GetHeadVersion()))
{
result.IsUpdated = true;
}
return result;
}
private async Task<bool> ShouldScanItem(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TLibrary library,
ImmutableDictionary<string, TEtag> existingMusicVideos,
TMusicVideo incoming,
string localPath,
bool deepScan)
{
// deep scan will always pull every music video
if (deepScan)
{
return true;
}
string existingEtag = string.Empty;
MediaItemState existingState = MediaItemState.Normal;
if (existingMusicVideos.TryGetValue(MediaServerItemId(incoming), out TEtag? existingEntry))
{
existingEtag = existingEntry.Etag;
existingState = existingEntry.State;
}
if (existingState is MediaItemState.Unavailable or MediaItemState.FileNotFound &&
existingEtag == MediaServerEtag(incoming))
{
// skip scanning unavailable/file not found items that are unchanged and still don't exist locally
if (!_fileSystem.File.Exists(localPath))
{
return false;
}
}
else if (existingEtag == MediaServerEtag(incoming))
{
// item is unchanged, but file does not exist
// don't scan, but mark as unavailable
if (!_fileSystem.File.Exists(localPath))
{
if (existingState is not MediaItemState.Unavailable)
{
foreach (int id in await musicVideoRepository.FlagUnavailable(library, incoming))
{
if (!await _scannerProxy.ReindexMediaItems([id], CancellationToken.None))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
}
}
return false;
}
if (existingEntry is null)
{
_logger.LogDebug("INSERT: new music video {Path}", localPath);
}
else
{
_logger.LogDebug("UPDATE: Etag has changed for music video {Path}", localPath);
}
return true;
}
protected abstract string MediaServerItemId(TMusicVideo musicVideo);
protected abstract string MediaServerEtag(TMusicVideo musicVideo);
protected abstract IAsyncEnumerable<Tuple<TMusicVideo, int>> GetMusicVideoLibraryItems(
TConnectionParameters connectionParameters,
TLibrary library);
protected abstract Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> UpdateMetadata(
MediaItemScanResult<TMusicVideo> result,
TMusicVideo incoming,
CancellationToken cancellationToken);
}
+1
View File
@@ -241,6 +241,7 @@ public class Program
services.AddScoped<IJellyfinApiClient, JellyfinApiClient>(); services.AddScoped<IJellyfinApiClient, JellyfinApiClient>();
services.AddScoped<IJellyfinCollectionRepository, JellyfinCollectionRepository>(); services.AddScoped<IJellyfinCollectionRepository, JellyfinCollectionRepository>();
services.AddScoped<IJellyfinMovieRepository, JellyfinMovieRepository>(); services.AddScoped<IJellyfinMovieRepository, JellyfinMovieRepository>();
services.AddScoped<IJellyfinMusicVideoRepository, JellyfinMusicVideoRepository>();
services.AddScoped<IJellyfinTelevisionRepository, JellyfinTelevisionRepository>(); services.AddScoped<IJellyfinTelevisionRepository, JellyfinTelevisionRepository>();
services.AddScoped<IJellyfinPathReplacementService, JellyfinPathReplacementService>(); services.AddScoped<IJellyfinPathReplacementService, JellyfinPathReplacementService>();
@@ -24,9 +24,13 @@ namespace ErsatzTV.Tests.Integration;
// End-to-end regression for ersatztv#488. Unlike the existing MediaServer*LibraryScanner tests (which // End-to-end regression for ersatztv#488. Unlike the existing MediaServer*LibraryScanner tests (which
// substitute every repository, and therefore could never have exhibited the null-navigation bug — see the // substitute every repository, and therefore could never have exhibited the null-navigation bug — see the
// mocked GetOrAddFolder in MovieFolderScannerTests), this test wires the REAL LibraryRepository / // mocked GetOrAddFolder in MovieFolderScannerTests), this test wires the REAL LibraryRepository /
// ArtistRepository / MusicVideoRepository against in-memory SQLite so the actual crash path runs. The // ArtistRepository / JellyfinMusicVideoRepository against in-memory SQLite so the actual crash path runs. The
// deviation from the mock-and-verify house style is deliberate and required: a substituted // deviation from the mock-and-verify house style is deliberate and required: a substituted
// ILibraryRepository cannot exhibit the defect this issue is about. // ILibraryRepository cannot exhibit the defect this issue is about.
//
// ersatztv#496 reshaped this suite: music videos now carry a per-library server identity
// (JellyfinMusicVideo.ItemId/Etag), so reconciliation diffs on the item id and SOFT-trashes (FileNotFound)
// instead of hard-deleting by path. The #494 "row disappears" assertions became "row is flagged" assertions.
[TestFixture] [TestFixture]
public class JellyfinMusicVideoLibraryScannerTests public class JellyfinMusicVideoLibraryScannerTests
{ {
@@ -44,55 +48,18 @@ public class JellyfinMusicVideoLibraryScannerTests
int libraryPathId = await SeedLibraryPath("/data/music"); int libraryPathId = await SeedLibraryPath("/data/music");
// the remote-path shape that used to crash: Paths is populated, LibraryFolders is null // the remote-path shape that used to crash: Paths is populated, LibraryFolders is null
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null }; JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music");
var library = new JellyfinLibrary
{
Id = 42,
MediaSourceId = 1,
ItemId = "lib15",
ShouldSyncItems = true,
Paths = new List<LibraryPath> { libraryPath }
};
const string VideoPath = "/data/music/artist1/song1.mkv"; const string VideoPath = "/data/music/artist1/song1.mkv";
MusicVideo incoming = BuildIncoming(VideoPath, artistName: "Artist 1", title: "Song 1");
var apiClient = Substitute.For<IJellyfinApiClient>(); (JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
apiClient.GetMusicVideoLibraryItems( () => BuildIncoming(VideoPath, "Artist 1", "Song 1")));
Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<JellyfinLibrary>(),
Arg.Any<MediaServerProjectionFailureCounter>())
.Returns(OneItem(incoming));
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>(); Either<BaseError, Unit> result = await scanner.ScanLibrary(
pathReplacement ConnectionParameters,
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>()) library,
.Returns(ci => ci.ArgAt<string>(1)); deepScan: false,
CancellationToken.None);
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
var scanner = new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
pathReplacement,
mediaSourceRepository,
new ArtistRepository(_db.Factory),
new MusicVideoRepository(_db.Factory),
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
Substitute.For<IMetadataRepository>(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
Either<BaseError, Unit> result =
await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None);
// the scan runs to completion instead of crashing on GetOrAddFolder // the scan runs to completion instead of crashing on GetOrAddFolder
result.IsRight.ShouldBeTrue(result.Match(Right: _ => "", Left: e => e.Value)); result.IsRight.ShouldBeTrue(result.Match(Right: _ => "", Left: e => e.Value));
@@ -122,21 +89,27 @@ public class JellyfinMusicVideoLibraryScannerTests
LibraryFolder folder = await context.LibraryFolders.SingleAsync(f => f.Id == file.LibraryFolderId); LibraryFolder folder = await context.LibraryFolders.SingleAsync(f => f.Id == file.LibraryFolderId);
folder.Path.ShouldBe("/data/music/artist1"); folder.Path.ShouldBe("/data/music/artist1");
folder.LibraryPathId.ShouldBe(libraryPathId); folder.LibraryPathId.ShouldBe(libraryPathId);
// ersatztv#496: the row carries the server item id, so it is no longer identified by its path
List<JellyfinMusicVideo> identities = await context.JellyfinMusicVideos.ToListAsync();
identities.Count.ShouldBe(1);
identities[0].ItemId.ShouldBe(ItemIdFor(VideoPath));
} }
// ersatztv#494: a music video removed from Jellyfin must be removed from ErsatzTV on the next scan. // ersatztv#494 as reshaped by ersatztv#496: a music video removed from Jellyfin is SOFT-trashed
// (State = FileNotFound), not hard-deleted, matching the movie/TV/other-video scanners. The row survives, so
// collection membership and playout references survive with it and EmptyTrash governs the real removal.
[Test] [Test]
public async Task ScanLibrary_Should_Remove_MusicVideo_Missing_From_Jellyfin() public async Task ScanLibrary_Should_Flag_FileNotFound_For_MusicVideo_Missing_From_Jellyfin()
{ {
int id = await SeedLibraryPath("/data/music"); int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music"); JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// first scan seeds two music videos // first scan seeds two music videos
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi( (JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"), () => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"),
() => BuildIncoming("/data/music/artist2/gone.mkv", "Artist 2", "Gone"))); () => BuildIncoming("/data/music/artist2/gone.mkv", "Artist 2", "Gone")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await seed.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
(await MusicVideoPaths(id)).Count.ShouldBe(2); (await MusicVideoPaths(id)).Count.ShouldBe(2);
int goneId = await MusicVideoId("/data/music/artist2/gone.mkv"); int goneId = await MusicVideoId("/data/music/artist2/gone.mkv");
@@ -144,44 +117,155 @@ public class JellyfinMusicVideoLibraryScannerTests
// second scan: only "keep" is still present upstream // second scan: only "keep" is still present upstream
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi( (JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"))); () => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
List<string> paths = await MusicVideoPaths(id); // both rows still exist — the missing one is flagged, not deleted
paths.ShouldBe(new[] { "/data/music/artist1/keep.mkv" }); (await MusicVideoPaths(id)).Count.ShouldBe(2);
await scannerProxy.Received().RemoveMediaItems( (await MediaItemStateOf(goneId)).ShouldBe(MediaItemState.FileNotFound);
(await MediaItemStateOf(await MusicVideoId("/data/music/artist1/keep.mkv")))
.ShouldBe(MediaItemState.Normal);
await scannerProxy.Received().ReindexMediaItems(
Arg.Is<int[]>(ids => ids.Contains(goneId)),
Arg.Any<CancellationToken>());
await scannerProxy.DidNotReceive().RemoveMediaItems(
Arg.Is<int[]>(ids => ids.Contains(goneId)), Arg.Is<int[]>(ids => ids.Contains(goneId)),
Arg.Any<CancellationToken>()); Arg.Any<CancellationToken>());
} }
// ersatztv#494: an artist left with zero music videos is cleaned up. // ersatztv#496: the artist of a soft-trashed music video is NOT emptied, because the music video row is still
// there. This is the deliberate consequence of replacing #494's hard delete — artist cleanup now happens only
// once the trash is actually emptied.
[Test] [Test]
public async Task ScanLibrary_Should_Cleanup_Empty_Artist() public async Task ScanLibrary_Should_Keep_Artist_Of_Trashed_MusicVideo()
{ {
int id = await SeedLibraryPath("/data/music"); int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music"); JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi( (JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"), () => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"),
() => BuildIncoming("/data/music/artist2/gone.mkv", "Artist 2", "Gone"))); () => BuildIncoming("/data/music/artist2/gone.mkv", "Artist 2", "Gone")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await seed.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
(await ArtistTitles(id)).Count.ShouldBe(2); (await ArtistTitles(id)).Count.ShouldBe(2);
// "Artist 2" loses its only music video // "Artist 2" loses its only music video from the server's point of view
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi( (JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"))); () => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
(await ArtistTitles(id)).ShouldBe(new[] { "Artist 1" }); (await ArtistTitles(id)).ShouldBe(new[] { "Artist 1", "Artist 2" });
} }
// ersatztv#494 (Done-when 3): the music-video sweep must never delete a Movie or Show that shares the // ersatztv#496 (the issue's Done-when 2): one library's sweep must never reach another library's music
// same LibraryPath — the cross-delete risk is a LibraryPathId property, not a Mixed-library property. // videos. Identity is now (server item id, owning library), so library B reporting nothing cannot flag a row
// owned by library A — even though both libraries point at the SAME local path.
//
// Under the pre-#496 path diff this was the live hazard: the sweep resolved rows by (LibraryPathId, path) and
// HARD-deleted them, so the row a second library still served was destroyed outright.
[Test] [Test]
public async Task ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath() public async Task ScanLibrary_Should_Not_Flag_MusicVideos_Owned_By_Another_Library()
{
int pathA = await SeedLibraryPath("/data/music", libraryId: 42);
int pathB = await SeedLibraryPath("/data/music-overlap", libraryId: 43);
JellyfinLibrary libraryA = BuildLibrary(pathA, "/data/music", libraryId: 42);
JellyfinLibrary libraryB = BuildLibrary(pathB, "/data/music-overlap", libraryId: 43);
// library A owns the row
(JellyfinMusicVideoLibraryScanner seedA, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/song.mkv", "Artist 1", "Song")));
(await seedA.ScanLibrary(ConnectionParameters, libraryA, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
int ownedId = await MusicVideoId("/data/music/artist1/song.mkv");
// library B has its own, different item and reports only that one
(JellyfinMusicVideoLibraryScanner seedB, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music-overlap/artist9/other.mkv", "Artist 9", "Other")));
(await seedB.ScanLibrary(ConnectionParameters, libraryB, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
// library B now reports nothing but its own item disappearing — A's row must be untouched
(JellyfinMusicVideoLibraryScanner scannerB, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music-overlap/artist9/other2.mkv", "Artist 9", "Other 2")));
(await scannerB.ScanLibrary(ConnectionParameters, libraryB, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await MediaItemStateOf(ownedId)).ShouldBe(MediaItemState.Normal);
(await MediaItemStateOf(await MusicVideoId("/data/music-overlap/artist9/other.mkv")))
.ShouldBe(MediaItemState.FileNotFound);
}
// ersatztv#496: rows that predate per-item identity (created by the path-keyed scanner, or by the local
// MusicVideoFolderScanner) must be ADOPTED in place — the same MediaItem id gains a JellyfinMusicVideo
// identity row. A delete-and-re-add would silently drop collection membership, which is exactly what a
// populated music collection depends on.
[Test]
public async Task ScanLibrary_Should_Adopt_PreExisting_MusicVideo_Preserving_Identity_And_Collections()
{
int libraryPathId = await SeedLibraryPath("/data/music");
const string VideoPath = "/data/music/artist1/song1.mkv";
int existingId = await SeedPlainMusicVideo(libraryPathId, VideoPath, "Artist 1");
int collectionId = await SeedCollectionWith(existingId);
JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music");
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming(VideoPath, "Artist 1", "Song 1")));
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
// no duplicate row was created
(await context.MusicVideos.CountAsync(mv => mv.LibraryPathId == libraryPathId)).ShouldBe(1);
// the SAME row now carries the server identity
List<JellyfinMusicVideo> identities = await context.JellyfinMusicVideos.ToListAsync();
identities.Count.ShouldBe(1);
identities[0].Id.ShouldBe(existingId);
identities[0].ItemId.ShouldBe(ItemIdFor(VideoPath));
// and collection membership survived the adoption
List<CollectionItem> items = await context.CollectionItems
.Where(ci => ci.CollectionId == collectionId)
.ToListAsync();
items.Count.ShouldBe(1);
items[0].MediaItemId.ShouldBe(existingId);
}
// ersatztv#496: adoption is scoped to the scanned library's own library path, so a music video owned by a
// LOCAL library (MusicVideoFolderScanner writes the same MusicVideo table) is never hijacked into a Jellyfin
// library's identity.
[Test]
public async Task ScanLibrary_Should_Not_Adopt_A_MusicVideo_Owned_By_Another_LibraryPath()
{
int localPathId = await SeedLibraryPath("/data/local-music", libraryId: 99);
int jellyfinPathId = await SeedLibraryPath("/data/music", libraryId: 42);
const string VideoPath = "/data/shared/song1.mkv";
int localId = await SeedPlainMusicVideo(localPathId, VideoPath, "Artist 1");
JellyfinLibrary library = BuildLibrary(jellyfinPathId, "/data/music");
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming(VideoPath, "Artist 1", "Song 1")));
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
// the local row was left alone: no identity row, still owned by the local library path
(await context.JellyfinMusicVideos.CountAsync()).ShouldBe(0);
MusicVideo local = await context.MusicVideos.SingleAsync(mv => mv.Id == localId);
local.LibraryPathId.ShouldBe(localPathId);
}
// ersatztv#494 (Done-when 3): the music-video sweep must never touch a Movie or Show that shares the
// same LibraryPath — the cross-flag risk is a LibraryPathId property, not a Mixed-library property.
[Test]
public async Task ScanLibrary_Should_Not_CrossFlag_Movie_Or_Show_Sharing_The_LibraryPath()
{ {
int id = await SeedLibraryPath("/data/mixed"); int id = await SeedLibraryPath("/data/mixed");
@@ -193,54 +277,54 @@ public class JellyfinMusicVideoLibraryScannerTests
} }
JellyfinLibrary library = BuildLibrary(id, "/data/mixed"); JellyfinLibrary library = BuildLibrary(id, "/data/mixed");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// seed one music video under the same LibraryPath // seed one music video under the same LibraryPath
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi( (JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/mixed/song.mkv", "Artist 1", "Song"))); () => BuildIncoming("/data/mixed/song.mkv", "Artist 1", "Song")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await seed.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
// next scan drops "song.mkv" and adds "song2.mkv" — the sweep removes song.mkv // next scan drops "song.mkv" and adds "song2.mkv" — the sweep flags song.mkv only
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi( (JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/mixed/song2.mkv", "Artist 1", "Song 2"))); () => BuildIncoming("/data/mixed/song2.mkv", "Artist 1", "Song 2")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext(); await using TvContext context = _db.CreateContext();
(await context.Movies.CountAsync(m => m.LibraryPathId == id)).ShouldBe(1); (await context.Movies.CountAsync(m => m.LibraryPathId == id)).ShouldBe(1);
(await context.Shows.CountAsync(s => s.LibraryPathId == id)).ShouldBe(1); (await context.Shows.CountAsync(s => s.LibraryPathId == id)).ShouldBe(1);
(await MusicVideoPaths(id)).ShouldBe(new[] { "/data/mixed/song2.mkv" }); (await context.MediaItems.CountAsync(mi => mi.LibraryPathId == id && mi.State != MediaItemState.Normal))
.ShouldBe(1);
(await MediaItemStateOf(await MusicVideoId("/data/mixed/song.mkv")))
.ShouldBe(MediaItemState.FileNotFound);
} }
// ersatztv#477 guard: a successful-but-empty fetch must NOT wipe the library. Negative control — if the // ersatztv#477 guard: a successful-but-empty fetch must NOT flag the library. Negative control — if the
// sweep were ungated, an empty incoming set would remove every existing music video. // sweep were ungated, an empty incoming set would flag every existing music video.
[Test] [Test]
public async Task ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items() public async Task ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items()
{ {
int id = await SeedLibraryPath("/data/music"); int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music"); JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi( (JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"))); () => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await seed.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
int keepId = await MusicVideoId("/data/music/artist1/keep.mkv");
// Jellyfin returns zero items (mid-restore / transient) — the sweep must be skipped // Jellyfin returns zero items (mid-restore / transient) — the sweep must be skipped
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi()); (JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi());
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
(await MusicVideoPaths(id)).ShouldBe(new[] { "/data/music/artist1/keep.mkv" }); (await MusicVideoPaths(id)).ShouldBe(new[] { "/data/music/artist1/keep.mkv" });
await scannerProxy.DidNotReceive().RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()); (await MediaItemStateOf(keepId)).ShouldBe(MediaItemState.Normal);
} }
// #493 session field data: a MIXED Jellyfin library runs the music-video arm with a legitimately EMPTY // #493 session field data: a MIXED Jellyfin library runs the music-video arm with a legitimately EMPTY
// incoming set on every scan while Movies/Shows exist under the same LibraryPath (the real "Standup" case). // incoming set on every scan while Movies/Shows exist under the same LibraryPath (the real "Standup" case).
// If existing were computed by LibraryPathId alone, existing.Except([]) would wipe the movies/episodes; the // Identity is now per-item and per-library, so a movie can never enter the music-video diff at all.
// MusicVideo-joined FindMusicVideoPaths (not just the empty-fetch guard) is what protects them. Seed a Movie
// that even carries a MediaFile path, to prove the sweep never touches a non-music-video row.
[Test] [Test]
public async Task ScanLibrary_Should_Not_Touch_Movies_Or_Shows_When_No_MusicVideos_Present() public async Task ScanLibrary_Should_Not_Touch_Movies_Or_Shows_When_No_MusicVideos_Present()
{ {
@@ -269,16 +353,17 @@ public class JellyfinMusicVideoLibraryScannerTests
} }
JellyfinLibrary library = BuildLibrary(id, "/data/standup"); JellyfinLibrary library = BuildLibrary(id, "/data/standup");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// the music-video arm returns zero incoming — steady state for a mixed library // the music-video arm returns zero incoming — steady state for a mixed library
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi()); (JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi());
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue(); .IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext(); await using TvContext context = _db.CreateContext();
(await context.Movies.CountAsync(m => m.LibraryPathId == id)).ShouldBe(1); (await context.Movies.CountAsync(m => m.LibraryPathId == id)).ShouldBe(1);
(await context.Shows.CountAsync(s => s.LibraryPathId == id)).ShouldBe(1); (await context.Shows.CountAsync(s => s.LibraryPathId == id)).ShouldBe(1);
(await context.MediaItems.CountAsync(mi => mi.LibraryPathId == id && mi.State != MediaItemState.Normal))
.ShouldBe(0);
await scannerProxy.DidNotReceive().RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()); await scannerProxy.DidNotReceive().RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>());
} }
@@ -286,24 +371,20 @@ public class JellyfinMusicVideoLibraryScannerTests
// must reach ErsatzTV on the next scan. Before the fix, UpdateMetadata copied only scalar fields, so the // must reach ErsatzTV on the next scan. Before the fix, UpdateMetadata copied only scalar fields, so the
// update path silently dropped every child collection — add-new AND remove-stale. This is an interaction // update path silently dropped every child collection — add-new AND remove-stale. This is an interaction
// test: the repositories are substituted and GetOrAdd returns a canned existing item so we can verify the // test: the repositories are substituted and GetOrAdd returns a canned existing item so we can verify the
// scanner issues the exact reconcile calls. (The real-DB double-scan approach can't drive this here: the // scanner issues the exact reconcile calls.
// in-memory harness shares ONE SQLite connection across contexts, and mid-scan GetOrAdd's
// `MediaVersions.First().MediaFiles.First().Path` predicate mis-resolves once the existing item carries
// metadata children — a harness-only quirk; prod uses per-context pooled connections and looks music videos
// up by that predicate only because they carry no server ItemId. See the #497 close comment.)
// Non-vacuous: reverting the Reconcile* calls in UpdateMetadata drops every Received() below. // Non-vacuous: reverting the Reconcile* calls in UpdateMetadata drops every Received() below.
[Test] [Test]
public async Task ScanLibrary_Should_Reconcile_Metadata_Collections_On_Rescan_Of_Existing_Item() public async Task ScanLibrary_Should_Reconcile_Metadata_Collections_On_Rescan_Of_Existing_Item()
{ {
JellyfinLibrary library = BuildLibrary(1, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
const string VideoPath = "/data/music/artist1/song1.mkv"; const string VideoPath = "/data/music/artist1/song1.mkv";
// the EXISTING item already in ErsatzTV, with the collections Jellyfin first gave it // the EXISTING item already in ErsatzTV, with the collections Jellyfin first gave it
var existing = new MusicVideo var existing = new JellyfinMusicVideo
{ {
Id = 7, Id = 7,
ArtistId = 3, ArtistId = 3,
ItemId = ItemIdFor(VideoPath),
Etag = "old-etag",
MediaVersions = new List<MediaVersion> MediaVersions = new List<MediaVersion>
{ {
new() { MediaFiles = new List<MediaFile> { new() { Path = VideoPath } }, Streams = new List<MediaStream>() } new() { MediaFiles = new List<MediaFile> { new() { Path = VideoPath } }, Streams = new List<MediaStream>() }
@@ -321,51 +402,19 @@ public class JellyfinMusicVideoLibraryScannerTests
} }
}; };
var artistRepository = Substitute.For<IArtistRepository>(); (JellyfinMusicVideoLibraryScanner scanner, IMusicVideoRepository musicVideoRepository,
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>()) IMetadataRepository metadataRepository) =
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() })); BuildScannerWithSubstitutes(
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>()); existing,
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository
.GetOrAdd(Arg.Any<Artist>(), Arg.Any<LibraryPath>(), Arg.Any<LibraryFolder>(), Arg.Any<string>())
.Returns(Right<BaseError, MediaItemScanResult<MusicVideo>>(
new MediaItemScanResult<MusicVideo>(existing) { IsAdded = false }));
musicVideoRepository.FindMusicVideoPaths(Arg.Any<LibraryPath>())
.Returns(new List<string> { VideoPath }.AsEnumerable());
musicVideoRepository.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Any<Genre>()).Returns(true);
musicVideoRepository.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Any<Tag>()).Returns(true);
musicVideoRepository.AddStudio(Arg.Any<MusicVideoMetadata>(), Arg.Any<Studio>()).Returns(true);
musicVideoRepository.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Any<MusicVideoArtist>()).Returns(true);
musicVideoRepository.RemoveArtist(Arg.Any<MusicVideoArtist>()).Returns(true);
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>()).Returns(false);
metadataRepository.RemoveGenre(Arg.Any<Genre>()).Returns(true);
metadataRepository.RemoveTag(Arg.Any<Tag>()).Returns(true);
metadataRepository.RemoveStudio(Arg.Any<Studio>()).Returns(true);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
JellyfinMusicVideoLibraryScanner scanner = BuildScannerWith(
FakeApi(() => BuildIncoming( FakeApi(() => BuildIncoming(
VideoPath, "Artist 1", "Song 1", VideoPath, "Artist 1", "Song 1",
genres: new[] { "Synthwave", "Vaporwave" }, genres: new[] { "Synthwave", "Vaporwave" },
tags: new[] { "KeepTag", "NewTag" }, tags: new[] { "KeepTag", "NewTag" },
studios: new[] { "NewStudio" }, studios: new[] { "NewStudio" },
artists: new[] { "Artist 1", "Featured Y" })), artists: new[] { "Artist 1", "Featured Y" })));
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository);
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await scanner.ScanLibrary(ConnectionParameters, BuildLibrary(1, "/data/music"), deepScan: false,
.IsRight.ShouldBeTrue(); CancellationToken.None)).IsRight.ShouldBeTrue();
// remove-stale: Retro / DropTag / OldStudio / "Featured X" gone; kept items are NOT removed // remove-stale: Retro / DropTag / OldStudio / "Featured X" gone; kept items are NOT removed
await metadataRepository.Received(1).RemoveGenre(Arg.Is<Genre>(g => g.Name == "Retro")); await metadataRepository.Received(1).RemoveGenre(Arg.Is<Genre>(g => g.Name == "Retro"));
@@ -478,8 +527,6 @@ public class JellyfinMusicVideoLibraryScannerTests
[Test] [Test]
public async Task ScanLibrary_Should_Update_Album_And_Track_On_Rescan_Of_Existing_Item() public async Task ScanLibrary_Should_Update_Album_And_Track_On_Rescan_Of_Existing_Item()
{ {
JellyfinLibrary library = BuildLibrary(1, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
const string VideoPath = "/data/music/artist1/song1.mkv"; const string VideoPath = "/data/music/artist1/song1.mkv";
var existingMetadata = new MusicVideoMetadata var existingMetadata = new MusicVideoMetadata
@@ -493,10 +540,12 @@ public class JellyfinMusicVideoLibraryScannerTests
Artists = [new MusicVideoArtist { Name = "Artist 1" }] Artists = [new MusicVideoArtist { Name = "Artist 1" }]
}; };
var existing = new MusicVideo var existing = new JellyfinMusicVideo
{ {
Id = 7, Id = 7,
ArtistId = 3, ArtistId = 3,
ItemId = ItemIdFor(VideoPath),
Etag = "old-etag",
MediaVersions = MediaVersions =
[ [
new MediaVersion new MediaVersion
@@ -508,43 +557,15 @@ public class JellyfinMusicVideoLibraryScannerTests
MusicVideoMetadata = [existingMetadata] MusicVideoMetadata = [existingMetadata]
}; };
var artistRepository = Substitute.For<IArtistRepository>(); JellyfinMusicVideo incoming = BuildIncoming(VideoPath, "Artist 1", "Song 1");
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>())
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() }));
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>());
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository
.GetOrAdd(Arg.Any<Artist>(), Arg.Any<LibraryPath>(), Arg.Any<LibraryFolder>(), Arg.Any<string>())
.Returns(Right<BaseError, MediaItemScanResult<MusicVideo>>(
new MediaItemScanResult<MusicVideo>(existing) { IsAdded = false }));
musicVideoRepository.FindMusicVideoPaths(Arg.Any<LibraryPath>())
.Returns(new List<string> { VideoPath }.AsEnumerable());
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>())
.Returns(false);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
MusicVideo incoming = BuildIncoming(VideoPath, "Artist 1", "Song 1");
incoming.MusicVideoMetadata[0].Album = "Corrected Album"; incoming.MusicVideoMetadata[0].Album = "Corrected Album";
incoming.MusicVideoMetadata[0].Track = 4; incoming.MusicVideoMetadata[0].Track = 4;
JellyfinMusicVideoLibraryScanner scanner = BuildScannerWith( (JellyfinMusicVideoLibraryScanner scanner, _, _) =
FakeApi(() => incoming), BuildScannerWithSubstitutes(existing, FakeApi(() => incoming));
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository);
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await scanner.ScanLibrary(ConnectionParameters, BuildLibrary(1, "/data/music"), deepScan: false,
.IsRight.ShouldBeTrue(); CancellationToken.None)).IsRight.ShouldBeTrue();
existingMetadata.Album.ShouldBe("Corrected Album"); existingMetadata.Album.ShouldBe("Corrected Album");
existingMetadata.Track.ShouldBe(4); existingMetadata.Track.ShouldBe(4);
@@ -558,15 +579,15 @@ public class JellyfinMusicVideoLibraryScannerTests
[Test] [Test]
public async Task ScanLibrary_Should_Not_Double_Insert_Duplicate_Named_Incoming_Collection_Entries() public async Task ScanLibrary_Should_Not_Double_Insert_Duplicate_Named_Incoming_Collection_Entries()
{ {
JellyfinLibrary library = BuildLibrary(1, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
const string VideoPath = "/data/music/artist1/song1.mkv"; const string VideoPath = "/data/music/artist1/song1.mkv";
// the existing item carries NONE of the incoming names, so every incoming entry is an "add" // the existing item carries NONE of the incoming names, so every incoming entry is an "add"
var existing = new MusicVideo var existing = new JellyfinMusicVideo
{ {
Id = 7, Id = 7,
ArtistId = 3, ArtistId = 3,
ItemId = ItemIdFor(VideoPath),
Etag = "old-etag",
MediaVersions = MediaVersions =
[ [
new MediaVersion new MediaVersion
@@ -588,49 +609,19 @@ public class JellyfinMusicVideoLibraryScannerTests
] ]
}; };
var artistRepository = Substitute.For<IArtistRepository>();
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>())
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() }));
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>());
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository
.GetOrAdd(Arg.Any<Artist>(), Arg.Any<LibraryPath>(), Arg.Any<LibraryFolder>(), Arg.Any<string>())
.Returns(Right<BaseError, MediaItemScanResult<MusicVideo>>(
new MediaItemScanResult<MusicVideo>(existing) { IsAdded = false }));
musicVideoRepository.FindMusicVideoPaths(Arg.Any<LibraryPath>())
.Returns(new List<string> { VideoPath }.AsEnumerable());
musicVideoRepository.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Any<Genre>()).Returns(true);
musicVideoRepository.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Any<Tag>()).Returns(true);
musicVideoRepository.AddStudio(Arg.Any<MusicVideoMetadata>(), Arg.Any<Studio>()).Returns(true);
musicVideoRepository.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Any<MusicVideoArtist>()).Returns(true);
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>())
.Returns(false);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
// Jellyfin reports each name TWICE for the same item // Jellyfin reports each name TWICE for the same item
JellyfinMusicVideoLibraryScanner scanner = BuildScannerWith( (JellyfinMusicVideoLibraryScanner scanner, IMusicVideoRepository musicVideoRepository, _) =
BuildScannerWithSubstitutes(
existing,
FakeApi(() => BuildIncoming( FakeApi(() => BuildIncoming(
VideoPath, "Artist 1", "Song 1", VideoPath, "Artist 1", "Song 1",
genres: ["Synthwave", "Synthwave"], genres: ["Synthwave", "Synthwave"],
tags: ["DupTag", "DupTag"], tags: ["DupTag", "DupTag"],
studios: ["DupStudio", "DupStudio"], studios: ["DupStudio", "DupStudio"],
artists: ["Artist 1", "Artist 1"])), artists: ["Artist 1", "Artist 1"])));
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository);
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None)) (await scanner.ScanLibrary(ConnectionParameters, BuildLibrary(1, "/data/music"), deepScan: false,
.IsRight.ShouldBeTrue(); CancellationToken.None)).IsRight.ShouldBeTrue();
await musicVideoRepository.Received(1) await musicVideoRepository.Received(1)
.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Is<Genre>(g => g.Name == "Synthwave")); .AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Is<Genre>(g => g.Name == "Synthwave"));
@@ -649,37 +640,131 @@ public class JellyfinMusicVideoLibraryScannerTests
metadata.Artists.Count.ShouldBe(1); metadata.Artists.Count.ShouldBe(1);
} }
private JellyfinMusicVideoLibraryScanner BuildScannerWith( private static JellyfinConnectionParameters ConnectionParameters =>
IJellyfinApiClient apiClient, new("http://jellyfin", "api-key", 1);
IArtistRepository artistRepository,
IMusicVideoRepository musicVideoRepository, // A stable per-file server item id, so a re-scan of the same file is the same identity.
ILibraryRepository libraryRepository, private static string ItemIdFor(string path) => $"item-{PathUtils.GetPathHash(path)}";
IMetadataRepository metadataRepository)
private (JellyfinMusicVideoLibraryScanner Scanner, IMusicVideoRepository MusicVideoRepository,
IMetadataRepository MetadataRepository) BuildScannerWithSubstitutes(
JellyfinMusicVideo existing,
IJellyfinApiClient apiClient)
{
var artistRepository = Substitute.For<IArtistRepository>();
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>())
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() }));
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>());
var jellyfinMusicVideoRepository = Substitute.For<IJellyfinMusicVideoRepository>();
jellyfinMusicVideoRepository.GetOrAdd(
Arg.Any<JellyfinLibrary>(),
Arg.Any<Artist>(),
Arg.Any<LibraryFolder>(),
Arg.Any<JellyfinMusicVideo>(),
Arg.Any<bool>(),
Arg.Any<CancellationToken>())
.Returns(Right<BaseError, MediaItemScanResult<JellyfinMusicVideo>>(
new MediaItemScanResult<JellyfinMusicVideo>(existing) { IsAdded = false }));
jellyfinMusicVideoRepository.GetExistingMusicVideos(Arg.Any<JellyfinLibrary>())
.Returns(new List<JellyfinItemEtag>
{
new() { ItemId = existing.ItemId, Etag = existing.Etag, State = MediaItemState.Normal }
});
jellyfinMusicVideoRepository.FlagFileNotFound(Arg.Any<JellyfinLibrary>(), Arg.Any<List<string>>())
.Returns(new List<int>());
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Any<Genre>()).Returns(true);
musicVideoRepository.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Any<Tag>()).Returns(true);
musicVideoRepository.AddStudio(Arg.Any<MusicVideoMetadata>(), Arg.Any<Studio>()).Returns(true);
musicVideoRepository.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Any<MusicVideoArtist>()).Returns(true);
musicVideoRepository.RemoveArtist(Arg.Any<MusicVideoArtist>()).Returns(true);
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>())
.Returns(false);
metadataRepository.RemoveGenre(Arg.Any<Genre>()).Returns(true);
metadataRepository.RemoveTag(Arg.Any<Tag>()).Returns(true);
metadataRepository.RemoveStudio(Arg.Any<Studio>()).Returns(true);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
var scanner = new JellyfinMusicVideoLibraryScanner(
BuildScannerProxy(),
apiClient,
jellyfinMusicVideoRepository,
BuildPathReplacement(),
BuildMediaSourceRepository(),
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository,
BuildFileSystem(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
return (scanner, musicVideoRepository, metadataRepository);
}
private (JellyfinMusicVideoLibraryScanner Scanner, IScannerProxy ScannerProxy) BuildScanner(
IJellyfinApiClient apiClient)
{
IScannerProxy scannerProxy = BuildScannerProxy();
var scanner = new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
new JellyfinMusicVideoRepository(_db.Factory, NullLogger<JellyfinMusicVideoRepository>.Instance),
BuildPathReplacement(),
BuildMediaSourceRepository(),
new ArtistRepository(_db.Factory),
new MusicVideoRepository(_db.Factory),
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
Substitute.For<IMetadataRepository>(),
BuildFileSystem(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
return (scanner, scannerProxy);
}
private static IScannerProxy BuildScannerProxy()
{
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
return scannerProxy;
}
private static IJellyfinPathReplacementService BuildPathReplacement()
{ {
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>(); var pathReplacement = Substitute.For<IJellyfinPathReplacementService>();
pathReplacement pathReplacement
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>()) .GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>())
.Returns(ci => ci.ArgAt<string>(1)); .Returns(ci => ci.ArgAt<string>(1));
return pathReplacement;
}
private static IMediaSourceRepository BuildMediaSourceRepository()
{
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>(); var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>()) mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>())); .Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
return mediaSourceRepository;
}
var scannerProxy = Substitute.For<IScannerProxy>(); // every scanned file "exists" locally, so scanned items settle in Normal and a FileNotFound state can only
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true); // come from the sweep under test
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true); private static IFileSystem BuildFileSystem()
scannerProxy.RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true); {
var fileSystem = Substitute.For<IFileSystem>();
return new JellyfinMusicVideoLibraryScanner( fileSystem.File.Exists(Arg.Any<string>()).Returns(true);
scannerProxy, return fileSystem;
apiClient,
pathReplacement,
mediaSourceRepository,
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository,
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
} }
private async Task<int> MusicVideoId(string path) private async Task<int> MusicVideoId(string path)
@@ -694,6 +779,13 @@ public class JellyfinMusicVideoLibraryScannerTests
.Id; .Id;
} }
private async Task<MediaItemState> MediaItemStateOf(int mediaItemId)
{
await using TvContext context = _db.CreateContext();
MediaItem mediaItem = await context.MediaItems.SingleAsync(mi => mi.Id == mediaItemId);
return mediaItem.State;
}
private async Task<List<string>> MusicVideoPaths(int libraryPathId) private async Task<List<string>> MusicVideoPaths(int libraryPathId)
{ {
await using TvContext context = _db.CreateContext(); await using TvContext context = _db.CreateContext();
@@ -718,78 +810,9 @@ public class JellyfinMusicVideoLibraryScannerTests
return artists.Select(a => a.ArtistMetadata.Single().Title).OrderBy(t => t).ToList(); return artists.Select(a => a.ArtistMetadata.Single().Title).OrderBy(t => t).ToList();
} }
private (JellyfinMusicVideoLibraryScanner Scanner, IScannerProxy ScannerProxy) BuildScanner( // Each Func builds a fresh music video so the async stream can be re-enumerated across ScanLibrary calls
IJellyfinApiClient apiClient)
{
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>();
pathReplacement
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>())
.Returns(ci => ci.ArgAt<string>(1));
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
var scanner = new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
pathReplacement,
mediaSourceRepository,
new ArtistRepository(_db.Factory),
new MusicVideoRepository(_db.Factory),
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
Substitute.For<IMetadataRepository>(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
return (scanner, scannerProxy);
}
// #484 finding 3: the api client records projection failures into the counter the SCANNER created and
// handed it. This fake does the same, so the test exercises the same-instance join that
// JellyfinMusicVideoLibraryScanner.ScanLibrary makes between GetMusicVideoLibraryItems and the sweep.
private static IJellyfinApiClient FakeApiWithProjectionFailures(
int projectionFailureCount,
params Func<MusicVideo>[] items)
{
var apiClient = Substitute.For<IJellyfinApiClient>();
apiClient.GetMusicVideoLibraryItems(
Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<JellyfinLibrary>(),
Arg.Any<MediaServerProjectionFailureCounter>())
.Returns(ci => ItemsWithFailures(
items,
projectionFailureCount,
ci.ArgAt<MediaServerProjectionFailureCounter>(3)));
return apiClient;
}
private static async IAsyncEnumerable<Tuple<MusicVideo, int>> ItemsWithFailures(
Func<MusicVideo>[] items,
int projectionFailureCount,
MediaServerProjectionFailureCounter projectionFailures)
{
for (var i = 0; i < projectionFailureCount; i++)
{
projectionFailures.RecordFailure();
}
foreach (Func<MusicVideo> item in items)
{
yield return new Tuple<MusicVideo, int>(item(), items.Length + projectionFailureCount);
}
await Task.CompletedTask;
}
// Each Func builds a fresh MusicVideo so the async stream can be re-enumerated across ScanLibrary calls
// (an IAsyncEnumerable iterator is single-use, and the scanner mutates the incoming item). // (an IAsyncEnumerable iterator is single-use, and the scanner mutates the incoming item).
private static IJellyfinApiClient FakeApi(params Func<MusicVideo>[] items) private static IJellyfinApiClient FakeApi(params Func<JellyfinMusicVideo>[] items)
{ {
var apiClient = Substitute.For<IJellyfinApiClient>(); var apiClient = Substitute.For<IJellyfinApiClient>();
apiClient.GetMusicVideoLibraryItems( apiClient.GetMusicVideoLibraryItems(
@@ -801,31 +824,31 @@ public class JellyfinMusicVideoLibraryScannerTests
return apiClient; return apiClient;
} }
private static async IAsyncEnumerable<Tuple<MusicVideo, int>> Items(Func<MusicVideo>[] items) private static async IAsyncEnumerable<Tuple<JellyfinMusicVideo, int>> Items(Func<JellyfinMusicVideo>[] items)
{ {
foreach (Func<MusicVideo> item in items) foreach (Func<JellyfinMusicVideo> item in items)
{ {
yield return new Tuple<MusicVideo, int>(item(), items.Length); yield return new Tuple<JellyfinMusicVideo, int>(item(), items.Length);
} }
await Task.CompletedTask; await Task.CompletedTask;
} }
private static JellyfinLibrary BuildLibrary(int libraryPathId, string path) private static JellyfinLibrary BuildLibrary(int libraryPathId, string path, int libraryId = 42)
{ {
var libraryPath = new LibraryPath { Id = libraryPathId, Path = path, LibraryFolders = null }; var libraryPath = new LibraryPath { Id = libraryPathId, Path = path, LibraryFolders = null };
return new JellyfinLibrary return new JellyfinLibrary
{ {
Id = 42, Id = libraryId,
MediaSourceId = 1, MediaSourceId = 1,
ItemId = "lib15", ItemId = $"lib{libraryId}",
Name = "Music", Name = "Music",
ShouldSyncItems = true, ShouldSyncItems = true,
Paths = new List<LibraryPath> { libraryPath } Paths = new List<LibraryPath> { libraryPath }
}; };
} }
private static MusicVideo BuildIncoming( private static JellyfinMusicVideo BuildIncoming(
string path, string path,
string artistName, string artistName,
string title, string title,
@@ -835,6 +858,8 @@ public class JellyfinMusicVideoLibraryScannerTests
IEnumerable<string> artists = null) => IEnumerable<string> artists = null) =>
new() new()
{ {
ItemId = ItemIdFor(path),
Etag = $"etag-{title}",
MediaVersions = new List<MediaVersion> MediaVersions = new List<MediaVersion>
{ {
new() new()
@@ -858,18 +883,76 @@ public class JellyfinMusicVideoLibraryScannerTests
} }
}; };
private static async IAsyncEnumerable<Tuple<MusicVideo, int>> OneItem(MusicVideo musicVideo) // Seeds a real JellyfinLibrary row owning the LibraryPath. ersatztv#496 made this load-bearing: identity
{ // lookups and the sweep are scoped per LIBRARY (GetExistingMusicVideos / FlagFileNotFound both join
yield return new Tuple<MusicVideo, int>(musicVideo, 1); // LibraryPath.LibraryId), so a dangling LibraryPath with no owning library would silently match nothing.
await Task.CompletedTask; private async Task<int> SeedLibraryPath(string path, int libraryId = 42)
}
private async Task<int> SeedLibraryPath(string path)
{ {
await using TvContext context = _db.CreateContext(); await using TvContext context = _db.CreateContext();
var libraryPath = new LibraryPath { Path = path }; var libraryPath = new LibraryPath { Path = path };
await context.LibraryPaths.AddAsync(libraryPath); var library = new JellyfinLibrary
{
Id = libraryId,
MediaSourceId = 1,
ItemId = $"lib{libraryId}",
Name = "Music",
ShouldSyncItems = true,
Paths = new List<LibraryPath> { libraryPath }
};
await context.Libraries.AddAsync(library);
await context.SaveChangesAsync(); await context.SaveChangesAsync();
return libraryPath.Id; return libraryPath.Id;
} }
// a MusicVideo row with NO JellyfinMusicVideo identity — what the pre-#496 scanner (and the local
// MusicVideoFolderScanner) leaves behind
private async Task<int> SeedPlainMusicVideo(int libraryPathId, string path, string artistName)
{
await using TvContext context = _db.CreateContext();
var artist = new Artist
{
LibraryPathId = libraryPathId,
ArtistMetadata = new List<ArtistMetadata> { new() { Title = artistName } }
};
await context.Artists.AddAsync(artist);
await context.SaveChangesAsync();
var musicVideo = new MusicVideo
{
ArtistId = artist.Id,
LibraryPathId = libraryPathId,
MusicVideoMetadata = new List<MusicVideoMetadata> { new() { Title = "Song 1" } },
MediaVersions = new List<MediaVersion>
{
new()
{
MediaFiles = new List<MediaFile>
{
new() { Path = path, PathHash = PathUtils.GetPathHash(path) }
},
Streams = new List<MediaStream>()
}
}
};
await context.MusicVideos.AddAsync(musicVideo);
await context.SaveChangesAsync();
return musicVideo.Id;
}
private async Task<int> SeedCollectionWith(int mediaItemId)
{
await using TvContext context = _db.CreateContext();
var collection = new Collection
{
Name = "Vaporwave",
CollectionItems = new List<CollectionItem> { new() { MediaItemId = mediaItemId } }
};
await context.Collections.AddAsync(collection);
await context.SaveChangesAsync();
return collection.Id;
}
} }
+1
View File
@@ -1127,6 +1127,7 @@ public class Startup
services.AddScoped<IJellyfinTelevisionRepository, JellyfinTelevisionRepository>(); services.AddScoped<IJellyfinTelevisionRepository, JellyfinTelevisionRepository>();
services.AddScoped<IJellyfinCollectionRepository, JellyfinCollectionRepository>(); services.AddScoped<IJellyfinCollectionRepository, JellyfinCollectionRepository>();
services.AddScoped<IJellyfinMovieRepository, JellyfinMovieRepository>(); services.AddScoped<IJellyfinMovieRepository, JellyfinMovieRepository>();
services.AddScoped<IJellyfinMusicVideoRepository, JellyfinMusicVideoRepository>();
services.AddScoped<IEmbyApiClient, EmbyApiClient>(); services.AddScoped<IEmbyApiClient, EmbyApiClient>();
services.AddScoped<IEmbyPathReplacementService, EmbyPathReplacementService>(); services.AddScoped<IEmbyPathReplacementService, EmbyPathReplacementService>();
services.AddScoped<IEmbyTelevisionRepository, EmbyTelevisionRepository>(); services.AddScoped<IEmbyTelevisionRepository, EmbyTelevisionRepository>();
+38 -41
View File
@@ -2530,47 +2530,6 @@ remote scanner tripped it, and the feature had never run in prod, CI, or locally
still reads `libraryPath.LibraryFolders` directly, but it is only reached on the local (eager-loaded) path, so still reads `libraryPath.LibraryFolders` directly, but it is only reached on the local (eager-loaded) path, so
it is not affected; left as-is (out of #488 scope). it is not affected; left as-is (out of #488 scope).
## 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)
`key: scan.musicvideo-reconciliation` · `status: active` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: none`
**Rule:** `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity.
**Signals:** music-video trash sweep, path-based identity, cross-kind safety, path-keyed identity, empty-fetch guard reuse, remove-stale+add-new dedup · paths: `JellyfinMusicVideoLibraryScanner.TrashMissingMusicVideos`, `FindMusicVideoPaths`/`DeleteByPath`, `IMusicVideoRepository`, `MediaServerReconciliationGuard` · issues: #494, #477, #488, #496, #500
**Mechanics:** `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`; `ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items`; integration tests extending the #488 harness. #500 — when mirroring the remove-stale + add-new idiom, dedup the incoming set on **the same key its add filter compares** (the filter is materialized before the loop mutates `existing`, so duplicates both pass): `Name`, `Guid` for guids, and for Plex `Actors` an artwork-preferring dedup shared with the remove filter (whose key is `(Name, artwork-presence)`). Remaining un-deduped copies of the idiom: #600.
The Jellyfin music-video scanner did add/update only — a music video removed on the Jellyfin side lingered in
ErsatzTV forever and could still be scheduled. It now runs a trash sweep at the end of `ScanLibrary`
(`TrashMissingMusicVideos`), mirroring the `MediaServer{Movie,Television,OtherVideo}LibraryScanner` "gone
upstream ⇒ remove" pattern but with a deliberately different identity function, because music videos lack the
media-server identity those base scanners rely on.
- **Identity is (LibraryPathId, path), not server itemId.** The base scanners diff `GetExisting*` (keyed by
`MediaServerItemId`) against the incoming server item ids, then soft-trash via `FlagFileNotFound`. Music videos
have **no `JellyfinMusicVideo` entity and no `ItemId`/`Etag`** — the scanner is a standalone
`IJellyfinMusicVideoLibraryScanner` that injects the *local* `IMusicVideoRepository`, which offers no
itemId-keyed existing-set or flag seam. So the sweep diffs the **local path** set instead: existing =
`FindMusicVideoPaths(libraryPath)` `.Except` the incoming items' replaced local paths, then hard-deletes the
remainder with `DeleteByPath` + `IScannerProxy.RemoveMediaItems`, and cleans now-empty artists with
`IArtistRepository.DeleteEmptyArtists`. Hard delete (not soft `FileNotFound` trash) because there is no
per-item FileNotFound seam on this path and the issue's Done-when is "removed".
- **Cross-kind safety is a property of the queries, not the media kind.** `MediaItem` is TPT with `LibraryPathId`
on the abstract base, so a Movie, Show and MusicVideo can share one `LibraryPath` (a mixed Jellyfin library).
Both `FindMusicVideoPaths` and `DeleteByPath` filter `LibraryPathId` **and** join the concrete `MusicVideo`
table, so the sweep can only ever see/delete music videos — a Movie/Show under the same `LibraryPath` is
invisible to it. Pinned by `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`.
- **Reuses the #477 empty-fetch guard.** The sweep is gated by `MediaServerReconciliationGuard.ShouldFlagMissing`
— a successful fetch that returns zero items (server mid-restore / transient) is indistinguishable from a real
emptying, so the whole-library wipe is refused and logged. Pinned as a negative control by
`ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items` (removing the guard flips it red).
- **Known limitation (deferred to per-library identity).** `MusicVideoRepository.GetOrAdd` dedups a path
**globally** (no `LibraryPathId` predicate), so a file served by two libraries with overlapping local paths is
a single row owned by whichever library scanned it first. If that owner later stops reporting the file while
another library still serves it, this sweep removes the shared row. A proper fix needs per-library music-video
identity (a `JellyfinMusicVideo` etag entity + migration) — the issue's "option 2 / fold into the base
scanner" refactor — tracked as #496.
- **Tests.** Integration tests (real `ArtistRepository`/`MusicVideoRepository`/`LibraryRepository` over in-memory
SQLite, extending the #488 harness) pin removal, empty-artist cleanup, cross-kind safety, and the empty-fetch
guard. Proven non-vacuous: all four fail against the pre-fix scanner except the guard control, which only
earns its keep once the sweep exists.
## 2026-07-20 (#489) — Jellyfin mixed-content libraries map to one library holding many kinds ## 2026-07-20 (#489) — Jellyfin mixed-content libraries map to one library holding many kinds
`key: scan.jellyfin-mixed-content-library` · `status: active` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: none` `key: scan.jellyfin-mixed-content-library` · `status: active` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: none`
**Rule:** A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. **Rule:** A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind.
@@ -3895,3 +3854,41 @@ This is a **second, independent** refusal on the same guard, plus a decision not
Sweep`, the two nested TV cases, and `JellyfinMusicVideoLibraryScannerTests Sweep`, the two nested TV cases, and `JellyfinMusicVideoLibraryScannerTests
.MusicVideo_Sweep_Respects_Projection_Failures` all drive the real `ScanLibrary` entry point and record .MusicVideo_Sweep_Respects_Projection_Failures` all drive the real `ScanLibrary` entry point and record
the failure from *inside* the enumeration, so same-instance wiring is what makes them pass. the failure from *inside* the enumeration, so same-instance wiring is what makes them pass.
## 2026-07-25 — Music videos carry a per-library server identity; reconciliation is an itemId diff + soft trash (#496)
`key: scan.musicvideo-server-identity` · `status: active` · `since: 2026-07-25` · `supersedes: scan.musicvideo-reconciliation@2026-07-20` · `superseded-by: none`
**Rule:** Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added.
**Signals:** music-video server identity, JellyfinMusicVideo ItemId/Etag, itemId diff, soft FileNotFound trash, adoption of pre-identity rows, cross-library false-trash, path-replaced PathHash · paths: `JellyfinMusicVideo`, `JellyfinMusicVideoRepository`, `IMediaServerMusicVideoRepository`, `MediaServerMusicVideoLibraryScanner`, `JellyfinMusicVideoLibraryScanner` · issues: #496, #494, #477, #488, #497, #500
**Mechanics:** dual-provider migration `Add_JellyfinMusicVideo`; adoption probe joins `MediaFile.PathHash` + `NOT EXISTS (JellyfinMusicVideo)` filtered to the library's `LibraryPath`; `AddMusicVideo` normalizes `Path`/`PathHash` to the path-REPLACED local path; `ScanLibrary_Should_Adopt_PreExisting_MusicVideo_Preserving_Identity_And_Collections`, `ScanLibrary_Should_Not_Adopt_A_MusicVideo_Owned_By_Another_LibraryPath`, `ScanLibrary_Should_Not_Flag_MusicVideos_Owned_By_Another_Library`
#494 gave music videos a trash sweep but had to key it on `(LibraryPathId, path)` and hard-delete, because
music videos carried no server identity. That left the known limitation this issue is named for: a file served
by two libraries with overlapping local paths is one row owned by whichever library scanned it first, and that
owner's sweep **destroyed** the row the other library still served. This is the deferred "option 2".
- **Identity is the server item id, per library.** `JellyfinMusicVideo` mirrors `JellyfinMovie` exactly (TPT
table, `ItemId`/`Etag` `varchar(36)`, index on `ItemId`). `GetExistingMusicVideos` and `FlagFileNotFound` both
join `LibraryPath.LibraryId`, so the existing-set and the flag set are scoped to the scanning library — one
library's sweep can no longer resolve, let alone remove, another library's row.
- **Soft trash replaces hard delete.** The sweep now flags `FileNotFound` (`State = 1`) like the
movie/TV/other-video base scanners. The row survives, so collection membership, playout references and
artwork survive with it, and removal is `EmptyTrash`-governed and reversible. The visible consequence is that
`DeleteEmptyArtists` no longer fires from a sweep — a trashed music video still belongs to its artist.
- **Pre-identity rows are ADOPTED, not re-added.** Every music video on an existing install has no
`JellyfinMusicVideo` row and so can never be found by item id. Deleting and re-adding would mint a new
`MediaItem` id and silently drop `CollectionItem` membership; `MediaItemRepository.MediaFileAlreadyExists`
would in fact block the re-add outright and the item would error on every scan forever. So `GetOrAdd` probes
for an identity-less `MusicVideo` at the same `PathHash` **within the scanned library's own `LibraryPath`**
and inserts the identity row against that same id. The scoping is the point: the local
`MusicVideoFolderScanner` writes the same `MusicVideo` table, and a local (or second-library) row must never
be hijacked into this library's identity.
- **The adopted row is written with an empty etag** so the ordinary "etag changed ⇒ refresh" path picks it up
once, rather than needing a second adoption-specific update path.
- **`Path`/`PathHash` are normalized to the path-REPLACED local path on add.** The projection fills them from
the path Jellyfin reported, but music videos have always stored the replaced local path. Storing the
projection's hash would break every later `PathHash` lookup — `MediaFileAlreadyExists` and the adoption probe
above. Caught by the #488 integration test failing on a `NOT NULL`/`UNIQUE` `PathHash` constraint.
- **Scope honesty: this is parity, not a total fix.** One file path is still one `MediaItem` row globally
(`MediaFileAlreadyExists` is a global path-hash guard), so a second library serving the same file still gets
no row of its own — exactly as for movies/TV. What changes is that the first library's sweep now *flags*
rather than *destroys* that shared row. The unrecoverable data loss is gone; the shared-row limitation is a
whole-app property, not a music-video one.
+1
View File
@@ -115,6 +115,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](../decisions.md#2026-07-20-489--jellyfin-mixed-content-libraries-map-to-one-library-holding-many-kinds) | | `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](../decisions.md#2026-07-20-489--jellyfin-mixed-content-libraries-map-to-one-library-holding-many-kinds) |
| `scan.musicvideo-reconciliation` | `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity. | 2026-07-20 | [link](../decisions.md#2026-07-20--jellyfinmusicvideolibraryscanner-reconciles-by-library-scoped-path-diff--hard-delete-not-server-itemid-soft-trash-494) | | `scan.musicvideo-reconciliation` | `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity. | 2026-07-20 | [link](../decisions.md#2026-07-20--jellyfinmusicvideolibraryscanner-reconciles-by-library-scoped-path-diff--hard-delete-not-server-itemid-soft-trash-494) |
| `scan.projection-failure-sweep-guard` | `MediaServerReconciliationGuard` takes a per-enumeration projection-failure count and refuses the file-not-found sweep (logged loudly) whenever it is non-zero against a non-empty existing set — at all six sweeps, including the nested per-show season and per-season episode ones #477 left unguarded (via `ShouldFlagMissingDescendants`, which applies the failure refusal but not #477's empty-fetch branch). Deliberate guard-clause skips (STRM, virtual, unsupported type) are explicitly **not** failures and never suppress a sweep. The missing-fraction / ratio threshold floated by #477 is **rejected**, not deferred. | 2026-07-25 | [link](../decisions.md#2026-07-25--a-media-server-sweep-also-refuses-when-the-api-client-silently-dropped-items-whose-projection-threw-the-ratio-threshold-is-rejected-484) | | `scan.projection-failure-sweep-guard` | `MediaServerReconciliationGuard` takes a per-enumeration projection-failure count and refuses the file-not-found sweep (logged loudly) whenever it is non-zero against a non-empty existing set — at all six sweeps, including the nested per-show season and per-season episode ones #477 left unguarded (via `ShouldFlagMissingDescendants`, which applies the failure refusal but not #477's empty-fetch branch). Deliberate guard-clause skips (STRM, virtual, unsupported type) are explicitly **not** failures and never suppress a sweep. The missing-fraction / ratio threshold floated by #477 is **rejected**, not deferred. | 2026-07-25 | [link](../decisions.md#2026-07-25--a-media-server-sweep-also-refuses-when-the-api-client-silently-dropped-items-whose-projection-threw-the-ratio-threshold-is-rejected-484) |
| `scan.musicvideo-server-identity` | Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added. | 2026-07-25 | [link](../decisions.md#2026-07-25--music-videos-carry-a-per-library-server-identity-reconciliation-is-an-itemid-diff--soft-trash-496) |
| `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](../decisions.md#2026-07-19--a-media-server-library-sweep-refuses-to-flag-when-a-successful-fetch-returns-zero-items-rather-than-nuking-the-whole-library-477) | | `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](../decisions.md#2026-07-19--a-media-server-library-sweep-refuses-to-flag-when-a-successful-fetch-returns-zero-items-rather-than-nuking-the-whole-library-477) |
| `sched.auto-tune-foundation` | Auto-tune preview enumeration uses EF distinct+count queries for exact counts, while each created channel is persisted as a live SmartCollection; coexistence with existing channels/numbers is additive-only, never mutating. | 2026-07-16 | [link](../decisions.md#2026-07-16--auto-tuning-enumerates-via-ef-persists-via-smartcollection-additive-coexistence-69) | | `sched.auto-tune-foundation` | Auto-tune preview enumeration uses EF distinct+count queries for exact counts, while each created channel is persisted as a live SmartCollection; coexistence with existing channels/numbers is additive-only, never mutating. | 2026-07-16 | [link](../decisions.md#2026-07-16--auto-tuning-enumerates-via-ef-persists-via-smartcollection-additive-coexistence-69) |
| `sched.autotune-detailpanel-members` | The Auto-Tune DetailPanel's per-channel content-source list is a live `ISearchIndex.Search` roll-up through the server-owned `AutoTuneAxisMap.GenerateQuery`, not an EF distinct+count query, so the preview matches exactly what the built channel's SmartCollection will contain. | 2026-07-17 | [link](../decisions.md#2026-07-17--auto-tune-detailpanel-member-list--live-search-index-roll-up-not-ef-enumeration-384) | | `sched.autotune-detailpanel-members` | The Auto-Tune DetailPanel's per-channel content-source list is a live `ISearchIndex.Search` roll-up through the server-owned `AutoTuneAxisMap.GenerateQuery`, not an EF distinct+count query, so the preview matches exactly what the built channel's SmartCollection will contain. | 2026-07-17 | [link](../decisions.md#2026-07-17--auto-tune-detailpanel-member-list--live-search-index-roll-up-not-ef-enumeration-384) |
+49
View File
@@ -0,0 +1,49 @@
# Archive — library scanning / media-server reconciliation
Superseded/retired records for the media-server library scanners and their reconciliation
strategies. See `docs/decisions/archive/README.md` for the archive's general rules (rationale kept
verbatim, never in the active read-path). Active successor for music-video reconciliation:
`scan.musicvideo-server-identity` in `docs/decisions.md`.
---
## 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)
`key: scan.musicvideo-reconciliation` · `status: superseded` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: scan.musicvideo-server-identity@2026-07-25`
**Rule:** (superseded) `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity.
**Signals:** music-video trash sweep, path-based identity, cross-kind safety, path-keyed identity, empty-fetch guard reuse, remove-stale+add-new dedup · paths: `JellyfinMusicVideoLibraryScanner.TrashMissingMusicVideos`, `FindMusicVideoPaths`/`DeleteByPath`, `IMusicVideoRepository`, `MediaServerReconciliationGuard` · issues: #494, #477, #488, #496, #500
**Mechanics:** `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`; `ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items`; integration tests extending the #488 harness. #500 — when mirroring the remove-stale + add-new idiom, dedup the incoming set on **the same key its add filter compares** (the filter is materialized before the loop mutates `existing`, so duplicates both pass): `Name`, `Guid` for guids, and for Plex `Actors` an artwork-preferring dedup shared with the remove filter (whose key is `(Name, artwork-presence)`). Remaining un-deduped copies of the idiom: #600. Superseded by `scan.musicvideo-server-identity` (ersatztv#496): music videos gained a `JellyfinMusicVideo` ItemId/Etag identity, so the path diff + hard delete became an itemId diff + soft `FileNotFound` trash.
The Jellyfin music-video scanner did add/update only — a music video removed on the Jellyfin side lingered in
ErsatzTV forever and could still be scheduled. It now runs a trash sweep at the end of `ScanLibrary`
(`TrashMissingMusicVideos`), mirroring the `MediaServer{Movie,Television,OtherVideo}LibraryScanner` "gone
upstream ⇒ remove" pattern but with a deliberately different identity function, because music videos lack the
media-server identity those base scanners rely on.
- **Identity is (LibraryPathId, path), not server itemId.** The base scanners diff `GetExisting*` (keyed by
`MediaServerItemId`) against the incoming server item ids, then soft-trash via `FlagFileNotFound`. Music videos
have **no `JellyfinMusicVideo` entity and no `ItemId`/`Etag`** — the scanner is a standalone
`IJellyfinMusicVideoLibraryScanner` that injects the *local* `IMusicVideoRepository`, which offers no
itemId-keyed existing-set or flag seam. So the sweep diffs the **local path** set instead: existing =
`FindMusicVideoPaths(libraryPath)` `.Except` the incoming items' replaced local paths, then hard-deletes the
remainder with `DeleteByPath` + `IScannerProxy.RemoveMediaItems`, and cleans now-empty artists with
`IArtistRepository.DeleteEmptyArtists`. Hard delete (not soft `FileNotFound` trash) because there is no
per-item FileNotFound seam on this path and the issue's Done-when is "removed".
- **Cross-kind safety is a property of the queries, not the media kind.** `MediaItem` is TPT with `LibraryPathId`
on the abstract base, so a Movie, Show and MusicVideo can share one `LibraryPath` (a mixed Jellyfin library).
Both `FindMusicVideoPaths` and `DeleteByPath` filter `LibraryPathId` **and** join the concrete `MusicVideo`
table, so the sweep can only ever see/delete music videos — a Movie/Show under the same `LibraryPath` is
invisible to it. Pinned by `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`.
- **Reuses the #477 empty-fetch guard.** The sweep is gated by `MediaServerReconciliationGuard.ShouldFlagMissing`
— a successful fetch that returns zero items (server mid-restore / transient) is indistinguishable from a real
emptying, so the whole-library wipe is refused and logged. Pinned as a negative control by
`ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items` (removing the guard flips it red).
- **Known limitation (deferred to per-library identity).** `MusicVideoRepository.GetOrAdd` dedups a path
**globally** (no `LibraryPathId` predicate), so a file served by two libraries with overlapping local paths is
a single row owned by whichever library scanned it first. If that owner later stops reporting the file while
another library still serves it, this sweep removes the shared row. A proper fix needs per-library music-video
identity (a `JellyfinMusicVideo` etag entity + migration) — the issue's "option 2 / fold into the base
scanner" refactor — tracked as #496.
- **Tests.** Integration tests (real `ArtistRepository`/`MusicVideoRepository`/`LibraryRepository` over in-memory
SQLite, extending the #488 harness) pin removal, empty-artist cleanup, cross-kind safety, and the empty-fetch
guard. Proven non-vacuous: all four fail against the pre-fix scanner except the guard control, which only
earns its keep once the sweep exists.