fix(496): give music videos a per-library server identity; itemId diff + soft trash

Music videos carried no server identity, so JellyfinMusicVideoLibraryScanner had to
reconcile by a (LibraryPathId, path) diff and HARD-delete the remainder. A file served
by two libraries with overlapping local paths is a single row owned by whichever library
scanned it first, so that owner's sweep destroyed a row another library still served —
taking collection membership and playout references with it, irreversibly.

This is #494's deferred "option 2":

- New JellyfinMusicVideo : MusicVideo (ItemId/Etag), mirroring JellyfinMovie — TPT table,
  varchar(36), ItemId index. Dual-provider migration Add_JellyfinMusicVideo.
- New IMediaServerMusicVideoRepository + JellyfinMusicVideoRepository: itemId-keyed
  existing-set/lookup and Flag{Normal,Unavailable,FileNotFound} seams, all scoped per
  library via LibraryPath.LibraryId.
- New MediaServerMusicVideoLibraryScanner base; JellyfinMusicVideoLibraryScanner folds
  onto it and keeps the #177/#488/#497/#500 metadata-reconcile logic verbatim.
- The sweep now soft-trashes (FileNotFound) instead of deleting, so removal is reversible
  and EmptyTrash-governed. DeleteEmptyArtists consequently no longer fires from a sweep.
- Pre-identity rows are ADOPTED in place: the identity row is inserted against the same
  MediaItem id, scoped to the scanned library's own LibraryPath, so collection membership
  survives and a local/second-library row is never hijacked.
- AddMusicVideo normalizes Path/PathHash to the path-REPLACED local path; the projection
  fills them from the server-reported path, which would break every later PathHash lookup.

Docs: scan.musicvideo-reconciliation relocated to docs/decisions/archive/scan.md as
superseded; new active record scan.musicvideo-server-identity.

fixes #496
This commit is contained in:
2026-07-25 17:20:53 +02:00
parent eb4de0cc2b
commit 64decd492e
22 changed files with 15996 additions and 580 deletions
@@ -0,0 +1,7 @@
namespace ErsatzTV.Core.Domain;
public class JellyfinMusicVideo : MusicVideo
{
public string ItemId { get; set; }
public string Etag { get; set; }
}
@@ -21,7 +21,7 @@ public interface IJellyfinApiClient
JellyfinLibrary library,
MediaServerProjectionFailureCounter projectionFailures = null);
IAsyncEnumerable<Tuple<MusicVideo, int>> GetMusicVideoLibraryItems(
IAsyncEnumerable<Tuple<JellyfinMusicVideo, int>> GetMusicVideoLibraryItems(
string address,
string authorizationHeader,
JellyfinLibrary library,
@@ -0,0 +1,10 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Jellyfin;
namespace ErsatzTV.Core.Interfaces.Repositories;
public interface
IJellyfinMusicVideoRepository : IMediaServerMusicVideoRepository<JellyfinLibrary, JellyfinMusicVideo,
JellyfinItemEtag>
{
}
@@ -0,0 +1,26 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Metadata;
namespace ErsatzTV.Core.Interfaces.Repositories;
public interface IMediaServerMusicVideoRepository<in TLibrary, TMusicVideo, TEtag> where TLibrary : Library
where TMusicVideo : MusicVideo
where TEtag : MediaServerItemEtag
{
Task<List<TEtag>> GetExistingMusicVideos(TLibrary library);
Task<Option<int>> FlagNormal(TLibrary library, TMusicVideo musicVideo);
Task<Option<int>> FlagUnavailable(TLibrary library, TMusicVideo musicVideo);
Task<List<int>> FlagFileNotFound(TLibrary library, List<string> musicVideoItemIds);
// Unlike the movie/other-video seams, GetOrAdd takes the resolved Artist and LibraryFolder: a music video is
// owned by an Artist (FK, not null) and ersatztv#488 requires the media file to carry its LibraryFolderId.
Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> GetOrAdd(
TLibrary library,
Artist artist,
LibraryFolder libraryFolder,
TMusicVideo item,
bool deepScan,
CancellationToken cancellationToken);
Task<Unit> SetEtag(TMusicVideo musicVideo, string etag);
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,48 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.MySql.Migrations
{
/// <inheritdoc />
public partial class Add_JellyfinMusicVideo : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "JellyfinMusicVideo",
columns: table => new
{
Id = table.Column<int>(type: "int", nullable: false),
ItemId = table.Column<string>(type: "varchar(36)", unicode: false, maxLength: 36, nullable: true)
.Annotation("MySql:CharSet", "utf8mb4"),
Etag = table.Column<string>(type: "varchar(36)", unicode: false, maxLength: 36, nullable: true)
.Annotation("MySql:CharSet", "utf8mb4")
},
constraints: table =>
{
table.PrimaryKey("PK_JellyfinMusicVideo", x => x.Id);
table.ForeignKey(
name: "FK_JellyfinMusicVideo_MusicVideo_Id",
column: x => x.Id,
principalTable: "MusicVideo",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
})
.Annotation("MySql:CharSet", "utf8mb4");
migrationBuilder.CreateIndex(
name: "IX_JellyfinMusicVideo_ItemId",
table: "JellyfinMusicVideo",
column: "ItemId");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "JellyfinMusicVideo");
}
}
}
@@ -4452,6 +4452,25 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
b.ToTable("PlexMovie", (string)null);
});
modelBuilder.Entity("ErsatzTV.Core.Domain.JellyfinMusicVideo", b =>
{
b.HasBaseType("ErsatzTV.Core.Domain.MusicVideo");
b.Property<string>("Etag")
.HasMaxLength(36)
.IsUnicode(false)
.HasColumnType("varchar(36)");
b.Property<string>("ItemId")
.HasMaxLength(36)
.IsUnicode(false)
.HasColumnType("varchar(36)");
b.HasIndex("ItemId");
b.ToTable("JellyfinMusicVideo", (string)null);
});
modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b =>
{
b.HasBaseType("ErsatzTV.Core.Domain.OtherVideo");
@@ -6730,6 +6749,15 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
.IsRequired();
});
modelBuilder.Entity("ErsatzTV.Core.Domain.JellyfinMusicVideo", b =>
{
b.HasOne("ErsatzTV.Core.Domain.MusicVideo", null)
.WithOne()
.HasForeignKey("ErsatzTV.Core.Domain.JellyfinMusicVideo", "Id")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b =>
{
b.HasOne("ErsatzTV.Core.Domain.OtherVideo", null)
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,46 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
{
/// <inheritdoc />
public partial class Add_JellyfinMusicVideo : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable(
name: "JellyfinMusicVideo",
columns: table => new
{
Id = table.Column<int>(type: "INTEGER", nullable: false)
.Annotation("Sqlite:Autoincrement", true),
ItemId = table.Column<string>(type: "TEXT", unicode: false, maxLength: 36, nullable: true),
Etag = table.Column<string>(type: "TEXT", unicode: false, maxLength: 36, nullable: true)
},
constraints: table =>
{
table.PrimaryKey("PK_JellyfinMusicVideo", x => x.Id);
table.ForeignKey(
name: "FK_JellyfinMusicVideo_MusicVideo_Id",
column: x => x.Id,
principalTable: "MusicVideo",
principalColumn: "Id",
onDelete: ReferentialAction.Cascade);
});
migrationBuilder.CreateIndex(
name: "IX_JellyfinMusicVideo_ItemId",
table: "JellyfinMusicVideo",
column: "ItemId");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(
name: "JellyfinMusicVideo");
}
}
}
@@ -4277,6 +4277,25 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
b.ToTable("PlexMovie", (string)null);
});
modelBuilder.Entity("ErsatzTV.Core.Domain.JellyfinMusicVideo", b =>
{
b.HasBaseType("ErsatzTV.Core.Domain.MusicVideo");
b.Property<string>("Etag")
.HasMaxLength(36)
.IsUnicode(false)
.HasColumnType("TEXT");
b.Property<string>("ItemId")
.HasMaxLength(36)
.IsUnicode(false)
.HasColumnType("TEXT");
b.HasIndex("ItemId");
b.ToTable("JellyfinMusicVideo", (string)null);
});
modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b =>
{
b.HasBaseType("ErsatzTV.Core.Domain.OtherVideo");
@@ -6555,6 +6574,15 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
.IsRequired();
});
modelBuilder.Entity("ErsatzTV.Core.Domain.JellyfinMusicVideo", b =>
{
b.HasOne("ErsatzTV.Core.Domain.MusicVideo", null)
.WithOne()
.HasForeignKey("ErsatzTV.Core.Domain.JellyfinMusicVideo", "Id")
.OnDelete(DeleteBehavior.Cascade)
.IsRequired();
});
modelBuilder.Entity("ErsatzTV.Core.Domain.PlexOtherVideo", b =>
{
b.HasOne("ErsatzTV.Core.Domain.OtherVideo", null)
@@ -0,0 +1,23 @@
using ErsatzTV.Core.Domain;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
namespace ErsatzTV.Infrastructure.Data.Configurations;
public class JellyfinMusicVideoConfiguration : IEntityTypeConfiguration<JellyfinMusicVideo>
{
public void Configure(EntityTypeBuilder<JellyfinMusicVideo> builder)
{
builder.ToTable("JellyfinMusicVideo");
builder.Property(mv => mv.Etag)
.HasMaxLength(36)
.IsUnicode(false);
builder.Property(mv => mv.ItemId)
.HasMaxLength(36)
.IsUnicode(false);
builder.HasIndex(mv => mv.ItemId);
}
}
@@ -0,0 +1,348 @@
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Extensions;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Jellyfin;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Infrastructure.Extensions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Infrastructure.Data.Repositories;
public class JellyfinMusicVideoRepository : IJellyfinMusicVideoRepository
{
private readonly IDbContextFactory<TvContext> _dbContextFactory;
private readonly ILogger<JellyfinMusicVideoRepository> _logger;
public JellyfinMusicVideoRepository(
IDbContextFactory<TvContext> dbContextFactory,
ILogger<JellyfinMusicVideoRepository> logger)
{
_dbContextFactory = dbContextFactory;
_logger = logger;
}
public async Task<List<JellyfinItemEtag>> GetExistingMusicVideos(JellyfinLibrary library)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
return await dbContext.Connection.QueryAsync<JellyfinItemEtag>(
@"SELECT ItemId, Etag, MI.State FROM JellyfinMusicVideo
INNER JOIN MusicVideo M on JellyfinMusicVideo.Id = M.Id
INNER JOIN MediaItem MI on M.Id = MI.Id
INNER JOIN LibraryPath LP on MI.LibraryPathId = LP.Id
WHERE LP.LibraryId = @LibraryId",
new { LibraryId = library.Id })
.Map(result => result.ToList());
}
public async Task<Option<int>> FlagNormal(JellyfinLibrary library, JellyfinMusicVideo musicVideo)
{
if (musicVideo.State is MediaItemState.Normal)
{
return Option<int>.None;
}
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
musicVideo.State = MediaItemState.Normal;
Option<int> maybeId = await dbContext.Connection.ExecuteScalarAsync<int>(
@"SELECT JellyfinMusicVideo.Id FROM JellyfinMusicVideo
INNER JOIN MediaItem MI ON MI.Id = JellyfinMusicVideo.Id
INNER JOIN LibraryPath LP on MI.LibraryPathId = LP.Id AND LibraryId = @LibraryId
WHERE JellyfinMusicVideo.ItemId = @ItemId",
new { LibraryId = library.Id, musicVideo.ItemId });
foreach (int id in maybeId)
{
return await dbContext.Connection.ExecuteAsync(
"UPDATE MediaItem SET State = 0 WHERE Id = @Id AND State != 0",
new { Id = id }).Map(count => count > 0 ? Some(id) : None);
}
return None;
}
public async Task<Option<int>> FlagUnavailable(JellyfinLibrary library, JellyfinMusicVideo musicVideo)
{
if (musicVideo.State is MediaItemState.Unavailable)
{
return Option<int>.None;
}
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
musicVideo.State = MediaItemState.Unavailable;
Option<int> maybeId = await dbContext.Connection.ExecuteScalarAsync<int>(
@"SELECT JellyfinMusicVideo.Id FROM JellyfinMusicVideo
INNER JOIN MediaItem MI ON MI.Id = JellyfinMusicVideo.Id
INNER JOIN LibraryPath LP on MI.LibraryPathId = LP.Id AND LibraryId = @LibraryId
WHERE JellyfinMusicVideo.ItemId = @ItemId",
new { LibraryId = library.Id, musicVideo.ItemId });
foreach (int id in maybeId)
{
return await dbContext.Connection.ExecuteAsync(
"UPDATE MediaItem SET State = 2 WHERE Id = @Id AND State != 2",
new { Id = id }).Map(count => count > 0 ? Some(id) : None);
}
return None;
}
public async Task<List<int>> FlagFileNotFound(JellyfinLibrary library, List<string> musicVideoItemIds)
{
if (musicVideoItemIds.Count == 0)
{
return [];
}
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
List<int> ids = await dbContext.Connection.QueryAsync<int>(
@"SELECT M.Id
FROM MediaItem M
INNER JOIN JellyfinMusicVideo ON JellyfinMusicVideo.Id = M.Id
INNER JOIN LibraryPath LP on M.LibraryPathId = LP.Id AND LP.LibraryId = @LibraryId
WHERE JellyfinMusicVideo.ItemId IN @MusicVideoItemIds",
new { LibraryId = library.Id, MusicVideoItemIds = musicVideoItemIds })
.Map(result => result.ToList());
await dbContext.Connection.ExecuteAsync(
"UPDATE MediaItem SET State = 1 WHERE Id IN @Ids AND State != 1",
new { Ids = ids });
return ids;
}
public async Task<Either<BaseError, MediaItemScanResult<JellyfinMusicVideo>>> GetOrAdd(
JellyfinLibrary library,
Artist artist,
LibraryFolder libraryFolder,
JellyfinMusicVideo item,
bool deepScan,
CancellationToken cancellationToken)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync(cancellationToken);
string path = item.GetHeadVersion().MediaFiles.Head().Path;
Option<JellyfinMusicVideo> maybeExisting = await GetByItemId(dbContext, item.ItemId, cancellationToken);
// ersatztv#496: rows created before this library gained per-item identity (and rows created by the
// pre-#496 path-keyed scanner) carry no JellyfinMusicVideo row, so they can never be found by ItemId.
// Adopt them IN PLACE — insert the identity row against the SAME MediaItem id — rather than deleting and
// re-adding: the MediaItem id is referenced by CollectionItem, playouts, artwork and the search index, so
// a delete/re-add would silently drop collection membership and break built playouts.
//
// Adoption is scoped to THIS library's own library path, which is the whole point of the issue: a music
// video owned by a local library (MusicVideoFolderScanner uses the same MusicVideo table) or by a second
// Jellyfin library must never be hijacked into this library's identity.
if (maybeExisting.IsNone)
{
Option<int> maybeAdopted = await AdoptExistingMusicVideo(
dbContext,
library.Paths.Head().Id,
path,
item.ItemId);
if (maybeAdopted.IsSome)
{
_logger.LogDebug(
"ADOPT: existing music video {Path} now carries Jellyfin item id {ItemId}",
path,
item.ItemId);
// the identity row is written with an empty etag, so the existing-item path below always sees an
// etag mismatch and refreshes the adopted row exactly like any other changed item
maybeExisting = await GetByItemId(dbContext, item.ItemId, cancellationToken);
}
}
foreach (JellyfinMusicVideo existing in maybeExisting)
{
var result = new MediaItemScanResult<JellyfinMusicVideo>(existing) { IsAdded = false };
// identity is now the server item id, so the file behind it can move; keep the local path, owning
// folder and artist in sync when the server says the item changed
if (existing.Etag != item.Etag || deepScan)
{
await UpdateMusicVideoFile(dbContext, existing.Id, artist.Id, libraryFolder.Id, path);
existing.ArtistId = artist.Id;
existing.Artist = artist;
result.IsUpdated = true;
}
else if (existing.ArtistId != artist.Id)
{
await dbContext.Connection.ExecuteAsync(
"UPDATE MusicVideo SET ArtistId = @ArtistId WHERE Id = @Id",
new { Id = existing.Id, ArtistId = artist.Id });
existing.ArtistId = artist.Id;
existing.Artist = artist;
result.IsUpdated = true;
}
return result;
}
return await AddMusicVideo(dbContext, library, artist, libraryFolder, item, path, cancellationToken);
}
public async Task<Unit> SetEtag(JellyfinMusicVideo musicVideo, string etag)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
return await dbContext.Connection.ExecuteAsync(
"UPDATE JellyfinMusicVideo SET Etag = @Etag WHERE Id = @Id",
new { Etag = etag, musicVideo.Id }).Map(_ => Unit.Default);
}
// Mirrors MusicVideoRepository.GetOrAdd's eager-load set: the scanner's metadata reconcile (ersatztv#497)
// only reconciles collections that are loaded here, so this list is load-bearing, not cosmetic.
private static async Task<Option<JellyfinMusicVideo>> GetByItemId(
TvContext dbContext,
string itemId,
CancellationToken cancellationToken) =>
await dbContext.JellyfinMusicVideos
.AsNoTracking()
.Include(mv => mv.Artist)
.ThenInclude(a => a.ArtistMetadata)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Artwork)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Artists)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Genres)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Tags)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Studios)
.Include(mv => mv.MusicVideoMetadata)
.ThenInclude(mvm => mvm.Directors)
.Include(mv => mv.LibraryPath)
.ThenInclude(lp => lp.Library)
.Include(mv => mv.MediaVersions)
.ThenInclude(mv => mv.MediaFiles)
.Include(mv => mv.MediaVersions)
.ThenInclude(mv => mv.Streams)
.Include(mv => mv.TraktListItems)
.ThenInclude(tli => tli.TraktList)
.SelectOneAsync(mv => mv.ItemId, mv => mv.ItemId == itemId, cancellationToken);
private static async Task<Option<int>> AdoptExistingMusicVideo(
TvContext dbContext,
int libraryPathId,
string path,
string itemId)
{
Option<int> maybeId = await dbContext.Connection.QuerySingleOrDefaultAsync<int?>(
@"SELECT M.Id
FROM MusicVideo M
INNER JOIN MediaItem MI on M.Id = MI.Id
INNER JOIN MediaVersion MV on M.Id = MV.MusicVideoId
INNER JOIN MediaFile MF on MV.Id = MF.MediaVersionId
WHERE MI.LibraryPathId = @LibraryPathId
AND MF.PathHash = @PathHash
AND NOT EXISTS (SELECT 1 FROM JellyfinMusicVideo J WHERE J.Id = M.Id)",
new { LibraryPathId = libraryPathId, PathHash = PathUtils.GetPathHash(path) })
.Map(Optional);
foreach (int id in maybeId)
{
await dbContext.Connection.ExecuteAsync(
"INSERT INTO JellyfinMusicVideo (Id, ItemId, Etag) VALUES (@Id, @ItemId, '')",
new { Id = id, ItemId = itemId });
return id;
}
return None;
}
private static async Task UpdateMusicVideoFile(
TvContext dbContext,
int musicVideoId,
int artistId,
int libraryFolderId,
string path)
{
await dbContext.Connection.ExecuteAsync(
"UPDATE MusicVideo SET ArtistId = @ArtistId WHERE Id = @Id",
new { Id = musicVideoId, ArtistId = artistId });
await dbContext.Connection.ExecuteAsync(
@"UPDATE MediaFile SET Path = @Path, PathHash = @PathHash, LibraryFolderId = @LibraryFolderId
WHERE Id IN (SELECT MF.Id
FROM MediaFile MF
INNER JOIN MediaVersion MV on MF.MediaVersionId = MV.Id
WHERE MV.MusicVideoId = @Id)",
new
{
Id = musicVideoId,
Path = path,
PathHash = PathUtils.GetPathHash(path),
LibraryFolderId = libraryFolderId
});
}
private async Task<Either<BaseError, MediaItemScanResult<JellyfinMusicVideo>>> AddMusicVideo(
TvContext dbContext,
JellyfinLibrary library,
Artist artist,
LibraryFolder libraryFolder,
JellyfinMusicVideo musicVideo,
string path,
CancellationToken cancellationToken)
{
try
{
if (await MediaItemRepository.MediaFileAlreadyExists(
path,
library.Paths.Head().Id,
dbContext,
_logger,
cancellationToken))
{
return new MediaFileAlreadyExists();
}
// blank out etag for initial save in case other updates fail
string etag = musicVideo.Etag;
musicVideo.Etag = string.Empty;
musicVideo.ArtistId = artist.Id;
musicVideo.LibraryPathId = library.Paths.Head().Id;
// Music videos store the PATH-REPLACED local path (they always have — the pre-#496 scanner passed the
// replaced path straight into GetOrAdd), while the projection fills Path/PathHash from the path the
// media server reported. Normalize both here, or the stored hash would be the hash of the server-side
// path and every later lookup by PathHash — MediaFileAlreadyExists, and #496's adoption probe — would
// miss the row it is meant to find.
MediaFile file = musicVideo.GetHeadVersion().MediaFiles.Head();
file.Path = path;
file.PathHash = PathUtils.GetPathHash(path);
file.LibraryFolderId = libraryFolder.Id;
await dbContext.AddAsync(musicVideo, cancellationToken);
await dbContext.SaveChangesAsync(cancellationToken);
// restore etag
musicVideo.Etag = etag;
await dbContext.Entry(musicVideo).Reference(mv => mv.Artist).LoadAsync(cancellationToken);
await dbContext.Entry(musicVideo.Artist).Collection(a => a.ArtistMetadata).LoadAsync(cancellationToken);
await dbContext.Entry(musicVideo).Reference(mv => mv.LibraryPath).LoadAsync(cancellationToken);
await dbContext.Entry(musicVideo.LibraryPath).Reference(lp => lp.Library).LoadAsync(cancellationToken);
return new MediaItemScanResult<JellyfinMusicVideo>(musicVideo) { IsAdded = true };
}
catch (Exception ex)
{
return BaseError.New(ex.ToString());
}
}
}
@@ -92,6 +92,7 @@ public class TvContext : DbContext
public DbSet<PlexEpisode> PlexEpisodes { get; set; }
public DbSet<PlexCollection> PlexCollections { get; set; }
public DbSet<JellyfinMovie> JellyfinMovies { get; set; }
public DbSet<JellyfinMusicVideo> JellyfinMusicVideos { get; set; }
public DbSet<JellyfinShow> JellyfinShows { get; set; }
public DbSet<JellyfinSeason> JellyfinSeasons { get; set; }
public DbSet<JellyfinEpisode> JellyfinEpisodes { get; set; }
@@ -95,7 +95,7 @@ public class JellyfinApiClient : IJellyfinApiClient
(maybeLibrary, item) => WithLibrary(maybeLibrary, lib => ProjectToMovie(lib, item)),
projectionFailures);
public IAsyncEnumerable<Tuple<MusicVideo, int>> GetMusicVideoLibraryItems(
public IAsyncEnumerable<Tuple<JellyfinMusicVideo, int>> GetMusicVideoLibraryItems(
string address,
string authorizationHeader,
JellyfinLibrary library,
@@ -701,7 +701,7 @@ public class JellyfinApiClient : IJellyfinApiClient
return metadata;
}
private MediaServerProjectionResult<MusicVideo> ProjectToMusicVideo(
private MediaServerProjectionResult<JellyfinMusicVideo> ProjectToMusicVideo(
JellyfinLibrary library,
JellyfinLibraryItemResponse item)
{
@@ -709,13 +709,13 @@ public class JellyfinApiClient : IJellyfinApiClient
{
if (item.LocationType != "FileSystem")
{
return MediaServerProjectionResult<MusicVideo>.Skipped();
return MediaServerProjectionResult<JellyfinMusicVideo>.Skipped();
}
if (Path.GetExtension(item.Path)?.ToLowerInvariant() == ".strm")
{
_logger.LogInformation("STRM files are not supported; skipping {Path}", item.Path);
return MediaServerProjectionResult<MusicVideo>.Skipped();
return MediaServerProjectionResult<JellyfinMusicVideo>.Skipped();
}
string path = item.Path ?? string.Empty;
@@ -752,8 +752,10 @@ public class JellyfinApiClient : IJellyfinApiClient
MusicVideoMetadata metadata = ProjectToMusicVideoMetadata(item);
var musicVideo = new MusicVideo
var musicVideo = new JellyfinMusicVideo
{
ItemId = item.Id,
Etag = item.Etag,
MediaVersions = [version],
MusicVideoMetadata = [metadata],
TraktListItems = []
@@ -764,7 +766,7 @@ public class JellyfinApiClient : IJellyfinApiClient
catch (Exception ex)
{
_logger.LogWarning(ex, "Error projecting Jellyfin music video");
return MediaServerProjectionResult<MusicVideo>.Failed();
return MediaServerProjectionResult<JellyfinMusicVideo>.Failed();
}
}
@@ -1,6 +1,6 @@
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Extensions;
using ErsatzTV.Core.Interfaces.Jellyfin;
using ErsatzTV.Core.Interfaces.Repositories;
@@ -12,40 +12,44 @@ using Microsoft.Extensions.Logging;
namespace ErsatzTV.Scanner.Core.Jellyfin;
public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanner
public class JellyfinMusicVideoLibraryScanner :
MediaServerMusicVideoLibraryScanner<JellyfinConnectionParameters, JellyfinLibrary, JellyfinMusicVideo,
JellyfinItemEtag>,
IJellyfinMusicVideoLibraryScanner
{
private const string UnknownArtist = "Unknown Artist";
private readonly IArtistRepository _artistRepository;
private readonly IJellyfinApiClient _jellyfinApiClient;
private readonly IJellyfinPathReplacementService _pathReplacementService;
private readonly ILibraryRepository _libraryRepository;
private readonly ILogger<JellyfinMusicVideoLibraryScanner> _logger;
private readonly IJellyfinMusicVideoRepository _jellyfinMusicVideoRepository;
private readonly IMediaSourceRepository _mediaSourceRepository;
private readonly IMetadataRepository _metadataRepository;
private readonly IMusicVideoRepository _musicVideoRepository;
private readonly IScannerProxy _scannerProxy;
private readonly IJellyfinPathReplacementService _pathReplacementService;
public JellyfinMusicVideoLibraryScanner(
IScannerProxy scannerProxy,
IJellyfinApiClient jellyfinApiClient,
IJellyfinMusicVideoRepository jellyfinMusicVideoRepository,
IJellyfinPathReplacementService pathReplacementService,
IMediaSourceRepository mediaSourceRepository,
IArtistRepository artistRepository,
IMusicVideoRepository musicVideoRepository,
ILibraryRepository libraryRepository,
IMetadataRepository metadataRepository,
IFileSystem fileSystem,
ILogger<JellyfinMusicVideoLibraryScanner> logger)
: base(
scannerProxy,
fileSystem,
artistRepository,
libraryRepository,
metadataRepository,
logger)
{
_scannerProxy = scannerProxy;
_jellyfinApiClient = jellyfinApiClient;
_jellyfinMusicVideoRepository = jellyfinMusicVideoRepository;
_pathReplacementService = pathReplacementService;
_mediaSourceRepository = mediaSourceRepository;
_artistRepository = artistRepository;
_musicVideoRepository = musicVideoRepository;
_libraryRepository = libraryRepository;
_metadataRepository = metadataRepository;
_logger = logger;
}
public async Task<Either<BaseError, Unit>> ScanLibrary(
@@ -54,211 +58,49 @@ public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanne
bool deepScan,
CancellationToken cancellationToken)
{
try
Option<LibraryPath> maybeLibraryPath = Optional(library.Paths).Flatten().HeadOrNone();
foreach (LibraryPath libraryPath in maybeLibraryPath)
{
Option<LibraryPath> maybeLibraryPath = Optional(library.Paths).Flatten().HeadOrNone();
return await maybeLibraryPath.Match(
libraryPath => ScanLibrary(connectionParameters, library, libraryPath, cancellationToken),
() => Task.FromResult<Either<BaseError, Unit>>(
BaseError.New($"Jellyfin library {library.Id} has no library path")));
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException)
{
return new ScanCanceled();
List<JellyfinPathReplacement> pathReplacements =
await _mediaSourceRepository.GetJellyfinPathReplacements(library.MediaSourceId);
string GetLocalPath(JellyfinMusicVideo musicVideo) =>
_pathReplacementService.GetReplacementJellyfinPath(
pathReplacements,
musicVideo.GetHeadVersion().MediaFiles.Head().Path,
false);
return await ScanLibrary(
_jellyfinMusicVideoRepository,
connectionParameters,
library,
libraryPath,
GetLocalPath,
deepScan,
cancellationToken);
}
return BaseError.New($"Jellyfin library {library.Id} has no library path");
}
private async Task<Either<BaseError, Unit>> ScanLibrary(
protected override string MediaServerItemId(JellyfinMusicVideo musicVideo) => musicVideo.ItemId;
protected override string MediaServerEtag(JellyfinMusicVideo musicVideo) => musicVideo.Etag;
protected override IAsyncEnumerable<Tuple<JellyfinMusicVideo, int>> GetMusicVideoLibraryItems(
JellyfinConnectionParameters connectionParameters,
JellyfinLibrary library,
LibraryPath libraryPath,
JellyfinLibrary library) =>
_jellyfinApiClient.GetMusicVideoLibraryItems(
connectionParameters.Address,
connectionParameters.AuthorizationHeader,
library);
protected override async Task<Either<BaseError, MediaItemScanResult<JellyfinMusicVideo>>> UpdateMetadata(
MediaItemScanResult<JellyfinMusicVideo> result,
JellyfinMusicVideo incoming,
CancellationToken cancellationToken)
{
List<JellyfinPathReplacement> pathReplacements =
await _mediaSourceRepository.GetJellyfinPathReplacements(library.MediaSourceId);
var processed = 0;
var incomingPaths = new List<string>();
// #484: one counter per enumeration, created here and read only after the enumeration completes.
// It is never a field on the (singleton) api client, so concurrent scans of different libraries
// cannot leak failures into each other's sweep decision.
var projectionFailures = new MediaServerProjectionFailureCounter();
await foreach ((MusicVideo incoming, int totalCount) in _jellyfinApiClient
.GetMusicVideoLibraryItems(
connectionParameters.Address,
connectionParameters.AuthorizationHeader,
library,
projectionFailures)
.WithCancellation(cancellationToken))
{
if (cancellationToken.IsCancellationRequested)
{
return new ScanCanceled();
}
processed++;
decimal percentCompletion = totalCount == 0 ? 1 : Math.Clamp((decimal)processed / totalCount, 0, 1);
if (!await _scannerProxy.UpdateProgress(percentCompletion, cancellationToken))
{
return new ScanCanceled();
}
incomingPaths.Add(GetLocalPath(pathReplacements, incoming));
Either<BaseError, MediaItemScanResult<MusicVideo>> maybeMusicVideo =
await ProcessMusicVideo(library, libraryPath, pathReplacements, incoming, cancellationToken);
foreach (BaseError error in maybeMusicVideo.LeftToSeq())
{
_logger.LogWarning("Error processing Jellyfin music video: {Error}", error.Value);
}
foreach (MediaItemScanResult<MusicVideo> result in maybeMusicVideo.RightToSeq()
.Filter(result => result.IsAdded || result.IsUpdated))
{
if (!await _scannerProxy.ReindexMediaItems([result.Item.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
}
await TrashMissingMusicVideos(
library,
libraryPath,
incomingPaths,
projectionFailures,
cancellationToken);
return Unit.Default;
}
// ersatztv#494: remove music videos (and now-empty artists) that Jellyfin no longer reports.
//
// Identity is LIBRARY-SCOPED by (LibraryPathId, path): FindMusicVideoPaths and DeleteByPath both filter
// LibraryPathId AND join the concrete MusicVideo table, so this can never touch a Movie/Show that shares
// the same LibraryPath (a mixed library) — the cross-delete safety is a property of those queries, not of
// the media kind. Unlike the MediaServer{Movie,Television,OtherVideo} base scanners, music videos carry no
// server ItemId/Etag (there is no JellyfinMusicVideo entity), so we diff on the local path instead of the
// server item id, and hard-delete rather than soft-trash (there is no per-item FileNotFound seam here).
//
// Known limitation: MusicVideoRepository.GetOrAdd dedups a path GLOBALLY (no LibraryPathId predicate), so a
// file served by two libraries with overlapping local paths is a single row owned by whichever library
// scanned it first. If that owning library later stops reporting the file while another library still
// serves it, this sweep removes the shared row. A proper fix needs per-library music-video identity (a
// JellyfinMusicVideo etag entity + migration) — the issue's deferred "option 2"; tracked as a follow-up.
private async Task TrashMissingMusicVideos(
JellyfinLibrary library,
LibraryPath libraryPath,
List<string> incomingPaths,
MediaServerProjectionFailureCounter projectionFailures,
CancellationToken cancellationToken)
{
var existingPaths = (await _musicVideoRepository.FindMusicVideoPaths(libraryPath)).ToList();
// #477: refuse the sweep when a successful fetch returned zero items but rows exist locally — an empty
// incoming set is indistinguishable from a transient error and would otherwise wipe the whole library.
// #484: also refuse when the api client silently dropped items whose projection threw — those are
// items Jellyfin DID return, so treating them as deletions would hard-delete healthy rows.
if (!MediaServerReconciliationGuard.ShouldFlagMissing(
_logger,
library.Name,
incomingPaths.Count,
existingPaths.Count,
projectionFailures.Count))
{
return;
}
foreach (string path in existingPaths.Except(incomingPaths))
{
List<int> musicVideoIds = await _musicVideoRepository.DeleteByPath(libraryPath, path);
if (musicVideoIds.Count > 0 &&
!await _scannerProxy.RemoveMediaItems(musicVideoIds.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to remove media items from scanner process");
}
}
List<int> artistIds = await _artistRepository.DeleteEmptyArtists(libraryPath);
if (artistIds.Count > 0 &&
!await _scannerProxy.RemoveMediaItems(artistIds.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to remove empty artists from scanner process");
}
}
private async Task<Either<BaseError, MediaItemScanResult<MusicVideo>>> ProcessMusicVideo(
JellyfinLibrary library,
LibraryPath libraryPath,
List<JellyfinPathReplacement> pathReplacements,
MusicVideo incoming,
CancellationToken cancellationToken)
{
string localPath = GetLocalPath(pathReplacements, incoming);
string folder = Path.GetDirectoryName(localPath) ?? libraryPath.Path;
Option<int> maybeParentFolder = await _libraryRepository.GetParentFolderId(libraryPath, folder, cancellationToken);
LibraryFolder libraryFolder = await _libraryRepository.GetOrAddFolder(libraryPath, maybeParentFolder, folder);
return await GetOrAddArtist(libraryPath, incoming)
.BindT(artist => _musicVideoRepository.GetOrAdd(artist, libraryPath, libraryFolder, localPath))
.BindT(result => UpdateMusicVideo(result, incoming, localPath, cancellationToken));
}
private string GetLocalPath(List<JellyfinPathReplacement> pathReplacements, MusicVideo musicVideo) =>
_pathReplacementService.GetReplacementJellyfinPath(
pathReplacements,
musicVideo.GetHeadVersion().MediaFiles.Head().Path,
false);
private async Task<Either<BaseError, Artist>> GetOrAddArtist(LibraryPath libraryPath, MusicVideo musicVideo)
{
string artistName = musicVideo.MusicVideoMetadata
.HeadOrNone()
.Bind(metadata => Optional(metadata.Artists).Flatten().HeadOrNone())
.Map(artist => artist.Name)
.IfNone(UnknownArtist);
var metadata = new ArtistMetadata
{
MetadataKind = MetadataKind.External,
Title = artistName,
SortTitle = SortTitle.GetSortTitle(artistName),
DateAdded = DateTime.UtcNow,
Genres = [],
Styles = [],
Moods = [],
Artwork = [],
Guids = [],
Subtitles = []
};
Option<Artist> maybeArtist = await _artistRepository.GetArtistByMetadata(libraryPath.Id, metadata);
foreach (Artist artist in maybeArtist)
{
return artist;
}
Either<BaseError, MediaItemScanResult<Artist>> result =
await _artistRepository.AddArtist(libraryPath.Id, artistName, metadata);
return result.Map(scanResult => scanResult.Item);
}
private async Task<Either<BaseError, MediaItemScanResult<MusicVideo>>> UpdateMusicVideo(
MediaItemScanResult<MusicVideo> result,
MusicVideo incoming,
string localPath,
CancellationToken cancellationToken)
{
result.LocalPath = localPath;
MusicVideoMetadata incomingMetadata = incoming.MusicVideoMetadata.Head();
bool updated = await UpdateMetadata(result.Item, incomingMetadata);
updated = await _metadataRepository.UpdateStatistics(result.Item, incoming.GetHeadVersion()) || updated;
if (updated)
if (await UpdateMetadata(result.Item, incoming.MusicVideoMetadata.Head()))
{
result.IsUpdated = true;
}
@@ -289,7 +131,7 @@ public class JellyfinMusicVideoLibraryScanner : IJellyfinMusicVideoLibraryScanne
bool updated = await _metadataRepository.Update(existing);
// ersatztv#497: the scalar Update above marks only the metadata row Modified; it does NOT touch
// child collections, and MusicVideoRepository.GetOrAdd loads them AsNoTracking — so tag/genre/studio/
// child collections, and the repository's GetOrAdd loads them AsNoTracking — so tag/genre/studio/
// artist edits made in Jellyfin never reached an EXISTING music video (only the Add path persisted
// them). Reconcile the collections that BOTH the Add path persists AND GetOrAdd eager-loads:
// Genres, Tags, Studios, Artists. (Guids are add-persisted but not eager-loaded here — reconciling
@@ -0,0 +1,376 @@
using System.Collections.Immutable;
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.MediaServer;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Extensions;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Scanner.Core.Interfaces;
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Scanner.Core.Metadata;
// ersatztv#496: music videos used to be reconciled by a LIBRARY-SCOPED local-path diff plus hard delete, because
// they carried no server identity. They now carry one (JellyfinMusicVideo.ItemId/Etag), so this scanner is the
// music-video sibling of MediaServer{Movie,Television,OtherVideo}LibraryScanner: it diffs on the server item id
// and soft-trashes (FileNotFound) instead of deleting, which is reversible and EmptyTrash-governed.
public abstract class MediaServerMusicVideoLibraryScanner<TConnectionParameters, TLibrary, TMusicVideo, TEtag>
where TConnectionParameters : MediaServerConnectionParameters
where TLibrary : Library
where TMusicVideo : MusicVideo
where TEtag : MediaServerItemEtag
{
private const string UnknownArtist = "Unknown Artist";
private readonly IArtistRepository _artistRepository;
private readonly IFileSystem _fileSystem;
private readonly ILibraryRepository _libraryRepository;
private readonly ILogger _logger;
private readonly IMetadataRepository _metadataRepository;
private readonly IScannerProxy _scannerProxy;
protected MediaServerMusicVideoLibraryScanner(
IScannerProxy scannerProxy,
IFileSystem fileSystem,
IArtistRepository artistRepository,
ILibraryRepository libraryRepository,
IMetadataRepository metadataRepository,
ILogger logger)
{
_scannerProxy = scannerProxy;
_fileSystem = fileSystem;
_artistRepository = artistRepository;
_libraryRepository = libraryRepository;
_metadataRepository = metadataRepository;
_logger = logger;
}
protected async Task<Either<BaseError, Unit>> ScanLibrary(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TConnectionParameters connectionParameters,
TLibrary library,
LibraryPath libraryPath,
Func<TMusicVideo, string> getLocalPath,
bool deepScan,
CancellationToken cancellationToken)
{
try
{
return await ScanLibrary(
musicVideoRepository,
connectionParameters,
library,
libraryPath,
getLocalPath,
GetMusicVideoLibraryItems(connectionParameters, library),
deepScan,
cancellationToken);
}
catch (Exception ex) when (ex is TaskCanceledException or OperationCanceledException)
{
return new ScanCanceled();
}
}
private async Task<Either<BaseError, Unit>> ScanLibrary(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TConnectionParameters connectionParameters,
TLibrary library,
LibraryPath libraryPath,
Func<TMusicVideo, string> getLocalPath,
IAsyncEnumerable<Tuple<TMusicVideo, int>> musicVideoEntries,
bool deepScan,
CancellationToken cancellationToken)
{
var incomingItemIds = new List<string>();
var existingMusicVideos = (await musicVideoRepository.GetExistingMusicVideos(library))
.ToImmutableDictionary(e => e.MediaServerItemId, e => e);
await foreach ((TMusicVideo incoming, int totalMusicVideoCount) in musicVideoEntries.WithCancellation(
cancellationToken))
{
if (cancellationToken.IsCancellationRequested)
{
return new ScanCanceled();
}
incomingItemIds.Add(MediaServerItemId(incoming));
decimal percentCompletion = totalMusicVideoCount == 0
? 1
: Math.Clamp((decimal)incomingItemIds.Count / totalMusicVideoCount, 0, 1);
if (!await _scannerProxy.UpdateProgress(percentCompletion, cancellationToken))
{
return new ScanCanceled();
}
string localPath = getLocalPath(incoming);
if (!await ShouldScanItem(
musicVideoRepository,
library,
existingMusicVideos,
incoming,
localPath,
deepScan))
{
continue;
}
Either<BaseError, MediaItemScanResult<TMusicVideo>> maybeMusicVideo =
await ProcessMusicVideo(
musicVideoRepository,
library,
libraryPath,
incoming,
localPath,
deepScan,
cancellationToken);
if (maybeMusicVideo.IsLeft)
{
foreach (BaseError error in maybeMusicVideo.LeftToSeq())
{
_logger.LogWarning("Error processing music video: {Error}", error.Value);
}
continue;
}
foreach (MediaItemScanResult<TMusicVideo> result in maybeMusicVideo.RightToSeq())
{
await musicVideoRepository.SetEtag(result.Item, MediaServerEtag(incoming));
if (_fileSystem.File.Exists(result.LocalPath))
{
Option<int> flagResult = await musicVideoRepository.FlagNormal(library, result.Item);
if (flagResult.IsSome)
{
result.IsUpdated = true;
}
}
else
{
Option<int> flagResult = await musicVideoRepository.FlagUnavailable(library, result.Item);
if (flagResult.IsSome)
{
result.IsUpdated = true;
}
}
if (result.IsAdded || result.IsUpdated)
{
if (!await _scannerProxy.ReindexMediaItems([result.Item.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
}
}
await TrashMissingMusicVideos(
musicVideoRepository,
library,
libraryPath,
incomingItemIds,
existingMusicVideos,
cancellationToken);
return Unit.Default;
}
// Soft-trash music videos the media server no longer reports. Identity is the SERVER ITEM ID, scoped to this
// library by GetExistingMusicVideos/FlagFileNotFound (both join LibraryPath.LibraryId), so a file served by two
// libraries with overlapping local paths is two independent identities and one library's sweep can no longer
// reach the other's row — the ersatztv#496 cross-library false-trash this replaced.
private async Task TrashMissingMusicVideos(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TLibrary library,
LibraryPath libraryPath,
List<string> incomingItemIds,
ImmutableDictionary<string, TEtag> existingMusicVideos,
CancellationToken cancellationToken)
{
// ersatztv#477: refuse the sweep when a successful fetch returned zero items but rows exist locally — an
// empty incoming set is indistinguishable from a transient error and would otherwise flag a whole library.
if (!MediaServerReconciliationGuard.ShouldFlagMissing(
_logger,
library.Name,
incomingItemIds.Count,
existingMusicVideos.Count))
{
return;
}
var fileNotFoundItemIds = existingMusicVideos.Keys.Except(incomingItemIds).ToList();
List<int> ids = await musicVideoRepository.FlagFileNotFound(library, fileNotFoundItemIds);
if (ids.Count > 0 && !await _scannerProxy.ReindexMediaItems(ids.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
// Trashed music videos are still rows, so an artist only becomes empty once the trash is emptied (or its
// items are removed some other way); this keeps ersatztv#494's empty-artist cleanup without the delete.
List<int> artistIds = await _artistRepository.DeleteEmptyArtists(libraryPath);
if (artistIds.Count > 0 && !await _scannerProxy.RemoveMediaItems(artistIds.ToArray(), cancellationToken))
{
_logger.LogWarning("Failed to remove empty artists from scanner process");
}
}
private async Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> ProcessMusicVideo(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TLibrary library,
LibraryPath libraryPath,
TMusicVideo incoming,
string localPath,
bool deepScan,
CancellationToken cancellationToken)
{
// ersatztv#488: resolve the owning folder from the DB, never from libraryPath.LibraryFolders (that
// navigation is only eager-loaded on the local scan path and is null for media-server callers).
string folder = Path.GetDirectoryName(localPath) ?? libraryPath.Path;
Option<int> maybeParentFolder =
await _libraryRepository.GetParentFolderId(libraryPath, folder, cancellationToken);
LibraryFolder libraryFolder = await _libraryRepository.GetOrAddFolder(libraryPath, maybeParentFolder, folder);
return await GetOrAddArtist(libraryPath, incoming)
.BindT(artist => musicVideoRepository.GetOrAdd(
library,
artist,
libraryFolder,
incoming,
deepScan,
cancellationToken))
.MapT(result =>
{
result.LocalPath = localPath;
return result;
})
.BindT(result => UpdateMetadata(result, incoming, cancellationToken))
.BindT(result => UpdateStatistics(result, incoming));
}
private async Task<Either<BaseError, Artist>> GetOrAddArtist(LibraryPath libraryPath, TMusicVideo musicVideo)
{
string artistName = musicVideo.MusicVideoMetadata
.HeadOrNone()
.Bind(metadata => Optional(metadata.Artists).Flatten().HeadOrNone())
.Map(artist => artist.Name)
.IfNone(UnknownArtist);
var metadata = new ArtistMetadata
{
MetadataKind = MetadataKind.External,
Title = artistName,
SortTitle = SortTitle.GetSortTitle(artistName),
DateAdded = DateTime.UtcNow,
Genres = [],
Styles = [],
Moods = [],
Artwork = [],
Guids = [],
Subtitles = []
};
Option<Artist> maybeArtist = await _artistRepository.GetArtistByMetadata(libraryPath.Id, metadata);
foreach (Artist artist in maybeArtist)
{
return artist;
}
Either<BaseError, MediaItemScanResult<Artist>> result =
await _artistRepository.AddArtist(libraryPath.Id, artistName, metadata);
return result.Map(scanResult => scanResult.Item);
}
private async Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> UpdateStatistics(
MediaItemScanResult<TMusicVideo> result,
TMusicVideo incoming)
{
if (await _metadataRepository.UpdateStatistics(result.Item, incoming.GetHeadVersion()))
{
result.IsUpdated = true;
}
return result;
}
private async Task<bool> ShouldScanItem(
IMediaServerMusicVideoRepository<TLibrary, TMusicVideo, TEtag> musicVideoRepository,
TLibrary library,
ImmutableDictionary<string, TEtag> existingMusicVideos,
TMusicVideo incoming,
string localPath,
bool deepScan)
{
// deep scan will always pull every music video
if (deepScan)
{
return true;
}
string existingEtag = string.Empty;
MediaItemState existingState = MediaItemState.Normal;
if (existingMusicVideos.TryGetValue(MediaServerItemId(incoming), out TEtag? existingEntry))
{
existingEtag = existingEntry.Etag;
existingState = existingEntry.State;
}
if (existingState is MediaItemState.Unavailable or MediaItemState.FileNotFound &&
existingEtag == MediaServerEtag(incoming))
{
// skip scanning unavailable/file not found items that are unchanged and still don't exist locally
if (!_fileSystem.File.Exists(localPath))
{
return false;
}
}
else if (existingEtag == MediaServerEtag(incoming))
{
// item is unchanged, but file does not exist
// don't scan, but mark as unavailable
if (!_fileSystem.File.Exists(localPath))
{
if (existingState is not MediaItemState.Unavailable)
{
foreach (int id in await musicVideoRepository.FlagUnavailable(library, incoming))
{
if (!await _scannerProxy.ReindexMediaItems([id], CancellationToken.None))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
}
}
return false;
}
if (existingEntry is null)
{
_logger.LogDebug("INSERT: new music video {Path}", localPath);
}
else
{
_logger.LogDebug("UPDATE: Etag has changed for music video {Path}", localPath);
}
return true;
}
protected abstract string MediaServerItemId(TMusicVideo musicVideo);
protected abstract string MediaServerEtag(TMusicVideo musicVideo);
protected abstract IAsyncEnumerable<Tuple<TMusicVideo, int>> GetMusicVideoLibraryItems(
TConnectionParameters connectionParameters,
TLibrary library);
protected abstract Task<Either<BaseError, MediaItemScanResult<TMusicVideo>>> UpdateMetadata(
MediaItemScanResult<TMusicVideo> result,
TMusicVideo incoming,
CancellationToken cancellationToken);
}
+1
View File
@@ -241,6 +241,7 @@ public class Program
services.AddScoped<IJellyfinApiClient, JellyfinApiClient>();
services.AddScoped<IJellyfinCollectionRepository, JellyfinCollectionRepository>();
services.AddScoped<IJellyfinMovieRepository, JellyfinMovieRepository>();
services.AddScoped<IJellyfinMusicVideoRepository, JellyfinMusicVideoRepository>();
services.AddScoped<IJellyfinTelevisionRepository, JellyfinTelevisionRepository>();
services.AddScoped<IJellyfinPathReplacementService, JellyfinPathReplacementService>();
@@ -24,9 +24,13 @@ namespace ErsatzTV.Tests.Integration;
// End-to-end regression for ersatztv#488. Unlike the existing MediaServer*LibraryScanner tests (which
// substitute every repository, and therefore could never have exhibited the null-navigation bug — see the
// mocked GetOrAddFolder in MovieFolderScannerTests), this test wires the REAL LibraryRepository /
// ArtistRepository / MusicVideoRepository against in-memory SQLite so the actual crash path runs. The
// ArtistRepository / JellyfinMusicVideoRepository against in-memory SQLite so the actual crash path runs. The
// deviation from the mock-and-verify house style is deliberate and required: a substituted
// ILibraryRepository cannot exhibit the defect this issue is about.
//
// ersatztv#496 reshaped this suite: music videos now carry a per-library server identity
// (JellyfinMusicVideo.ItemId/Etag), so reconciliation diffs on the item id and SOFT-trashes (FileNotFound)
// instead of hard-deleting by path. The #494 "row disappears" assertions became "row is flagged" assertions.
[TestFixture]
public class JellyfinMusicVideoLibraryScannerTests
{
@@ -44,55 +48,18 @@ public class JellyfinMusicVideoLibraryScannerTests
int libraryPathId = await SeedLibraryPath("/data/music");
// the remote-path shape that used to crash: Paths is populated, LibraryFolders is null
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
var library = new JellyfinLibrary
{
Id = 42,
MediaSourceId = 1,
ItemId = "lib15",
ShouldSyncItems = true,
Paths = new List<LibraryPath> { libraryPath }
};
JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music");
const string VideoPath = "/data/music/artist1/song1.mkv";
MusicVideo incoming = BuildIncoming(VideoPath, artistName: "Artist 1", title: "Song 1");
var apiClient = Substitute.For<IJellyfinApiClient>();
apiClient.GetMusicVideoLibraryItems(
Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<JellyfinLibrary>(),
Arg.Any<MediaServerProjectionFailureCounter>())
.Returns(OneItem(incoming));
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming(VideoPath, "Artist 1", "Song 1")));
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>();
pathReplacement
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>())
.Returns(ci => ci.ArgAt<string>(1));
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
var scanner = new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
pathReplacement,
mediaSourceRepository,
new ArtistRepository(_db.Factory),
new MusicVideoRepository(_db.Factory),
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
Substitute.For<IMetadataRepository>(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
Either<BaseError, Unit> result =
await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None);
Either<BaseError, Unit> result = await scanner.ScanLibrary(
ConnectionParameters,
library,
deepScan: false,
CancellationToken.None);
// the scan runs to completion instead of crashing on GetOrAddFolder
result.IsRight.ShouldBeTrue(result.Match(Right: _ => "", Left: e => e.Value));
@@ -122,21 +89,27 @@ public class JellyfinMusicVideoLibraryScannerTests
LibraryFolder folder = await context.LibraryFolders.SingleAsync(f => f.Id == file.LibraryFolderId);
folder.Path.ShouldBe("/data/music/artist1");
folder.LibraryPathId.ShouldBe(libraryPathId);
// ersatztv#496: the row carries the server item id, so it is no longer identified by its path
List<JellyfinMusicVideo> identities = await context.JellyfinMusicVideos.ToListAsync();
identities.Count.ShouldBe(1);
identities[0].ItemId.ShouldBe(ItemIdFor(VideoPath));
}
// ersatztv#494: a music video removed from Jellyfin must be removed from ErsatzTV on the next scan.
// ersatztv#494 as reshaped by ersatztv#496: a music video removed from Jellyfin is SOFT-trashed
// (State = FileNotFound), not hard-deleted, matching the movie/TV/other-video scanners. The row survives, so
// collection membership and playout references survive with it and EmptyTrash governs the real removal.
[Test]
public async Task ScanLibrary_Should_Remove_MusicVideo_Missing_From_Jellyfin()
public async Task ScanLibrary_Should_Flag_FileNotFound_For_MusicVideo_Missing_From_Jellyfin()
{
int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// first scan seeds two music videos
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"),
() => BuildIncoming("/data/music/artist2/gone.mkv", "Artist 2", "Gone")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(await seed.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await MusicVideoPaths(id)).Count.ShouldBe(2);
int goneId = await MusicVideoId("/data/music/artist2/gone.mkv");
@@ -144,44 +117,155 @@ public class JellyfinMusicVideoLibraryScannerTests
// second scan: only "keep" is still present upstream
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
List<string> paths = await MusicVideoPaths(id);
paths.ShouldBe(new[] { "/data/music/artist1/keep.mkv" });
await scannerProxy.Received().RemoveMediaItems(
// both rows still exist — the missing one is flagged, not deleted
(await MusicVideoPaths(id)).Count.ShouldBe(2);
(await MediaItemStateOf(goneId)).ShouldBe(MediaItemState.FileNotFound);
(await MediaItemStateOf(await MusicVideoId("/data/music/artist1/keep.mkv")))
.ShouldBe(MediaItemState.Normal);
await scannerProxy.Received().ReindexMediaItems(
Arg.Is<int[]>(ids => ids.Contains(goneId)),
Arg.Any<CancellationToken>());
await scannerProxy.DidNotReceive().RemoveMediaItems(
Arg.Is<int[]>(ids => ids.Contains(goneId)),
Arg.Any<CancellationToken>());
}
// ersatztv#494: an artist left with zero music videos is cleaned up.
// ersatztv#496: the artist of a soft-trashed music video is NOT emptied, because the music video row is still
// there. This is the deliberate consequence of replacing #494's hard delete — artist cleanup now happens only
// once the trash is actually emptied.
[Test]
public async Task ScanLibrary_Should_Cleanup_Empty_Artist()
public async Task ScanLibrary_Should_Keep_Artist_Of_Trashed_MusicVideo()
{
int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep"),
() => BuildIncoming("/data/music/artist2/gone.mkv", "Artist 2", "Gone")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(await seed.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await ArtistTitles(id)).Count.ShouldBe(2);
// "Artist 2" loses its only music video
// "Artist 2" loses its only music video from the server's point of view
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await ArtistTitles(id)).ShouldBe(new[] { "Artist 1" });
(await ArtistTitles(id)).ShouldBe(new[] { "Artist 1", "Artist 2" });
}
// ersatztv#494 (Done-when 3): the music-video sweep must never delete a Movie or Show that shares the
// same LibraryPath — the cross-delete risk is a LibraryPathId property, not a Mixed-library property.
// ersatztv#496 (the issue's Done-when 2): one library's sweep must never reach another library's music
// videos. Identity is now (server item id, owning library), so library B reporting nothing cannot flag a row
// owned by library A — even though both libraries point at the SAME local path.
//
// Under the pre-#496 path diff this was the live hazard: the sweep resolved rows by (LibraryPathId, path) and
// HARD-deleted them, so the row a second library still served was destroyed outright.
[Test]
public async Task ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath()
public async Task ScanLibrary_Should_Not_Flag_MusicVideos_Owned_By_Another_Library()
{
int pathA = await SeedLibraryPath("/data/music", libraryId: 42);
int pathB = await SeedLibraryPath("/data/music-overlap", libraryId: 43);
JellyfinLibrary libraryA = BuildLibrary(pathA, "/data/music", libraryId: 42);
JellyfinLibrary libraryB = BuildLibrary(pathB, "/data/music-overlap", libraryId: 43);
// library A owns the row
(JellyfinMusicVideoLibraryScanner seedA, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/song.mkv", "Artist 1", "Song")));
(await seedA.ScanLibrary(ConnectionParameters, libraryA, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
int ownedId = await MusicVideoId("/data/music/artist1/song.mkv");
// library B has its own, different item and reports only that one
(JellyfinMusicVideoLibraryScanner seedB, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music-overlap/artist9/other.mkv", "Artist 9", "Other")));
(await seedB.ScanLibrary(ConnectionParameters, libraryB, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
// library B now reports nothing but its own item disappearing — A's row must be untouched
(JellyfinMusicVideoLibraryScanner scannerB, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music-overlap/artist9/other2.mkv", "Artist 9", "Other 2")));
(await scannerB.ScanLibrary(ConnectionParameters, libraryB, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await MediaItemStateOf(ownedId)).ShouldBe(MediaItemState.Normal);
(await MediaItemStateOf(await MusicVideoId("/data/music-overlap/artist9/other.mkv")))
.ShouldBe(MediaItemState.FileNotFound);
}
// ersatztv#496: rows that predate per-item identity (created by the path-keyed scanner, or by the local
// MusicVideoFolderScanner) must be ADOPTED in place — the same MediaItem id gains a JellyfinMusicVideo
// identity row. A delete-and-re-add would silently drop collection membership, which is exactly what a
// populated music collection depends on.
[Test]
public async Task ScanLibrary_Should_Adopt_PreExisting_MusicVideo_Preserving_Identity_And_Collections()
{
int libraryPathId = await SeedLibraryPath("/data/music");
const string VideoPath = "/data/music/artist1/song1.mkv";
int existingId = await SeedPlainMusicVideo(libraryPathId, VideoPath, "Artist 1");
int collectionId = await SeedCollectionWith(existingId);
JellyfinLibrary library = BuildLibrary(libraryPathId, "/data/music");
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming(VideoPath, "Artist 1", "Song 1")));
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
// no duplicate row was created
(await context.MusicVideos.CountAsync(mv => mv.LibraryPathId == libraryPathId)).ShouldBe(1);
// the SAME row now carries the server identity
List<JellyfinMusicVideo> identities = await context.JellyfinMusicVideos.ToListAsync();
identities.Count.ShouldBe(1);
identities[0].Id.ShouldBe(existingId);
identities[0].ItemId.ShouldBe(ItemIdFor(VideoPath));
// and collection membership survived the adoption
List<CollectionItem> items = await context.CollectionItems
.Where(ci => ci.CollectionId == collectionId)
.ToListAsync();
items.Count.ShouldBe(1);
items[0].MediaItemId.ShouldBe(existingId);
}
// ersatztv#496: adoption is scoped to the scanned library's own library path, so a music video owned by a
// LOCAL library (MusicVideoFolderScanner writes the same MusicVideo table) is never hijacked into a Jellyfin
// library's identity.
[Test]
public async Task ScanLibrary_Should_Not_Adopt_A_MusicVideo_Owned_By_Another_LibraryPath()
{
int localPathId = await SeedLibraryPath("/data/local-music", libraryId: 99);
int jellyfinPathId = await SeedLibraryPath("/data/music", libraryId: 42);
const string VideoPath = "/data/shared/song1.mkv";
int localId = await SeedPlainMusicVideo(localPathId, VideoPath, "Artist 1");
JellyfinLibrary library = BuildLibrary(jellyfinPathId, "/data/music");
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming(VideoPath, "Artist 1", "Song 1")));
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
// the local row was left alone: no identity row, still owned by the local library path
(await context.JellyfinMusicVideos.CountAsync()).ShouldBe(0);
MusicVideo local = await context.MusicVideos.SingleAsync(mv => mv.Id == localId);
local.LibraryPathId.ShouldBe(localPathId);
}
// ersatztv#494 (Done-when 3): the music-video sweep must never touch a Movie or Show that shares the
// same LibraryPath — the cross-flag risk is a LibraryPathId property, not a Mixed-library property.
[Test]
public async Task ScanLibrary_Should_Not_CrossFlag_Movie_Or_Show_Sharing_The_LibraryPath()
{
int id = await SeedLibraryPath("/data/mixed");
@@ -193,54 +277,54 @@ public class JellyfinMusicVideoLibraryScannerTests
}
JellyfinLibrary library = BuildLibrary(id, "/data/mixed");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// seed one music video under the same LibraryPath
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/mixed/song.mkv", "Artist 1", "Song")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(await seed.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
// next scan drops "song.mkv" and adds "song2.mkv" — the sweep removes song.mkv
// next scan drops "song.mkv" and adds "song2.mkv" — the sweep flags song.mkv only
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/mixed/song2.mkv", "Artist 1", "Song 2")));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
(await context.Movies.CountAsync(m => m.LibraryPathId == id)).ShouldBe(1);
(await context.Shows.CountAsync(s => s.LibraryPathId == id)).ShouldBe(1);
(await MusicVideoPaths(id)).ShouldBe(new[] { "/data/mixed/song2.mkv" });
(await context.MediaItems.CountAsync(mi => mi.LibraryPathId == id && mi.State != MediaItemState.Normal))
.ShouldBe(1);
(await MediaItemStateOf(await MusicVideoId("/data/mixed/song.mkv")))
.ShouldBe(MediaItemState.FileNotFound);
}
// ersatztv#477 guard: a successful-but-empty fetch must NOT wipe the library. Negative control — if the
// sweep were ungated, an empty incoming set would remove every existing music video.
// ersatztv#477 guard: a successful-but-empty fetch must NOT flag the library. Negative control — if the
// sweep were ungated, an empty incoming set would flag every existing music video.
[Test]
public async Task ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items()
{
int id = await SeedLibraryPath("/data/music");
JellyfinLibrary library = BuildLibrary(id, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
(JellyfinMusicVideoLibraryScanner seed, _) = BuildScanner(FakeApi(
() => BuildIncoming("/data/music/artist1/keep.mkv", "Artist 1", "Keep")));
(await seed.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(await seed.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
int keepId = await MusicVideoId("/data/music/artist1/keep.mkv");
// Jellyfin returns zero items (mid-restore / transient) — the sweep must be skipped
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi());
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(JellyfinMusicVideoLibraryScanner scanner, _) = BuildScanner(FakeApi());
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await MusicVideoPaths(id)).ShouldBe(new[] { "/data/music/artist1/keep.mkv" });
await scannerProxy.DidNotReceive().RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>());
(await MediaItemStateOf(keepId)).ShouldBe(MediaItemState.Normal);
}
// #493 session field data: a MIXED Jellyfin library runs the music-video arm with a legitimately EMPTY
// incoming set on every scan while Movies/Shows exist under the same LibraryPath (the real "Standup" case).
// If existing were computed by LibraryPathId alone, existing.Except([]) would wipe the movies/episodes; the
// MusicVideo-joined FindMusicVideoPaths (not just the empty-fetch guard) is what protects them. Seed a Movie
// that even carries a MediaFile path, to prove the sweep never touches a non-music-video row.
// Identity is now per-item and per-library, so a movie can never enter the music-video diff at all.
[Test]
public async Task ScanLibrary_Should_Not_Touch_Movies_Or_Shows_When_No_MusicVideos_Present()
{
@@ -269,16 +353,17 @@ public class JellyfinMusicVideoLibraryScannerTests
}
JellyfinLibrary library = BuildLibrary(id, "/data/standup");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
// the music-video arm returns zero incoming — steady state for a mixed library
(JellyfinMusicVideoLibraryScanner scanner, IScannerProxy scannerProxy) = BuildScanner(FakeApi());
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
(await scanner.ScanLibrary(ConnectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
(await context.Movies.CountAsync(m => m.LibraryPathId == id)).ShouldBe(1);
(await context.Shows.CountAsync(s => s.LibraryPathId == id)).ShouldBe(1);
(await context.MediaItems.CountAsync(mi => mi.LibraryPathId == id && mi.State != MediaItemState.Normal))
.ShouldBe(0);
await scannerProxy.DidNotReceive().RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>());
}
@@ -286,24 +371,20 @@ public class JellyfinMusicVideoLibraryScannerTests
// must reach ErsatzTV on the next scan. Before the fix, UpdateMetadata copied only scalar fields, so the
// update path silently dropped every child collection — add-new AND remove-stale. This is an interaction
// test: the repositories are substituted and GetOrAdd returns a canned existing item so we can verify the
// scanner issues the exact reconcile calls. (The real-DB double-scan approach can't drive this here: the
// in-memory harness shares ONE SQLite connection across contexts, and mid-scan GetOrAdd's
// `MediaVersions.First().MediaFiles.First().Path` predicate mis-resolves once the existing item carries
// metadata children — a harness-only quirk; prod uses per-context pooled connections and looks music videos
// up by that predicate only because they carry no server ItemId. See the #497 close comment.)
// scanner issues the exact reconcile calls.
// Non-vacuous: reverting the Reconcile* calls in UpdateMetadata drops every Received() below.
[Test]
public async Task ScanLibrary_Should_Reconcile_Metadata_Collections_On_Rescan_Of_Existing_Item()
{
JellyfinLibrary library = BuildLibrary(1, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
const string VideoPath = "/data/music/artist1/song1.mkv";
// the EXISTING item already in ErsatzTV, with the collections Jellyfin first gave it
var existing = new MusicVideo
var existing = new JellyfinMusicVideo
{
Id = 7,
ArtistId = 3,
ItemId = ItemIdFor(VideoPath),
Etag = "old-etag",
MediaVersions = new List<MediaVersion>
{
new() { MediaFiles = new List<MediaFile> { new() { Path = VideoPath } }, Streams = new List<MediaStream>() }
@@ -321,51 +402,19 @@ public class JellyfinMusicVideoLibraryScannerTests
}
};
var artistRepository = Substitute.For<IArtistRepository>();
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>())
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() }));
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>());
(JellyfinMusicVideoLibraryScanner scanner, IMusicVideoRepository musicVideoRepository,
IMetadataRepository metadataRepository) =
BuildScannerWithSubstitutes(
existing,
FakeApi(() => BuildIncoming(
VideoPath, "Artist 1", "Song 1",
genres: new[] { "Synthwave", "Vaporwave" },
tags: new[] { "KeepTag", "NewTag" },
studios: new[] { "NewStudio" },
artists: new[] { "Artist 1", "Featured Y" })));
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository
.GetOrAdd(Arg.Any<Artist>(), Arg.Any<LibraryPath>(), Arg.Any<LibraryFolder>(), Arg.Any<string>())
.Returns(Right<BaseError, MediaItemScanResult<MusicVideo>>(
new MediaItemScanResult<MusicVideo>(existing) { IsAdded = false }));
musicVideoRepository.FindMusicVideoPaths(Arg.Any<LibraryPath>())
.Returns(new List<string> { VideoPath }.AsEnumerable());
musicVideoRepository.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Any<Genre>()).Returns(true);
musicVideoRepository.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Any<Tag>()).Returns(true);
musicVideoRepository.AddStudio(Arg.Any<MusicVideoMetadata>(), Arg.Any<Studio>()).Returns(true);
musicVideoRepository.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Any<MusicVideoArtist>()).Returns(true);
musicVideoRepository.RemoveArtist(Arg.Any<MusicVideoArtist>()).Returns(true);
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>()).Returns(false);
metadataRepository.RemoveGenre(Arg.Any<Genre>()).Returns(true);
metadataRepository.RemoveTag(Arg.Any<Tag>()).Returns(true);
metadataRepository.RemoveStudio(Arg.Any<Studio>()).Returns(true);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
JellyfinMusicVideoLibraryScanner scanner = BuildScannerWith(
FakeApi(() => BuildIncoming(
VideoPath, "Artist 1", "Song 1",
genres: new[] { "Synthwave", "Vaporwave" },
tags: new[] { "KeepTag", "NewTag" },
studios: new[] { "NewStudio" },
artists: new[] { "Artist 1", "Featured Y" })),
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository);
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await scanner.ScanLibrary(ConnectionParameters, BuildLibrary(1, "/data/music"), deepScan: false,
CancellationToken.None)).IsRight.ShouldBeTrue();
// remove-stale: Retro / DropTag / OldStudio / "Featured X" gone; kept items are NOT removed
await metadataRepository.Received(1).RemoveGenre(Arg.Is<Genre>(g => g.Name == "Retro"));
@@ -478,8 +527,6 @@ public class JellyfinMusicVideoLibraryScannerTests
[Test]
public async Task ScanLibrary_Should_Update_Album_And_Track_On_Rescan_Of_Existing_Item()
{
JellyfinLibrary library = BuildLibrary(1, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
const string VideoPath = "/data/music/artist1/song1.mkv";
var existingMetadata = new MusicVideoMetadata
@@ -493,10 +540,12 @@ public class JellyfinMusicVideoLibraryScannerTests
Artists = [new MusicVideoArtist { Name = "Artist 1" }]
};
var existing = new MusicVideo
var existing = new JellyfinMusicVideo
{
Id = 7,
ArtistId = 3,
ItemId = ItemIdFor(VideoPath),
Etag = "old-etag",
MediaVersions =
[
new MediaVersion
@@ -508,43 +557,15 @@ public class JellyfinMusicVideoLibraryScannerTests
MusicVideoMetadata = [existingMetadata]
};
var artistRepository = Substitute.For<IArtistRepository>();
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>())
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() }));
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>());
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository
.GetOrAdd(Arg.Any<Artist>(), Arg.Any<LibraryPath>(), Arg.Any<LibraryFolder>(), Arg.Any<string>())
.Returns(Right<BaseError, MediaItemScanResult<MusicVideo>>(
new MediaItemScanResult<MusicVideo>(existing) { IsAdded = false }));
musicVideoRepository.FindMusicVideoPaths(Arg.Any<LibraryPath>())
.Returns(new List<string> { VideoPath }.AsEnumerable());
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>())
.Returns(false);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
MusicVideo incoming = BuildIncoming(VideoPath, "Artist 1", "Song 1");
JellyfinMusicVideo incoming = BuildIncoming(VideoPath, "Artist 1", "Song 1");
incoming.MusicVideoMetadata[0].Album = "Corrected Album";
incoming.MusicVideoMetadata[0].Track = 4;
JellyfinMusicVideoLibraryScanner scanner = BuildScannerWith(
FakeApi(() => incoming),
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository);
(JellyfinMusicVideoLibraryScanner scanner, _, _) =
BuildScannerWithSubstitutes(existing, FakeApi(() => incoming));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await scanner.ScanLibrary(ConnectionParameters, BuildLibrary(1, "/data/music"), deepScan: false,
CancellationToken.None)).IsRight.ShouldBeTrue();
existingMetadata.Album.ShouldBe("Corrected Album");
existingMetadata.Track.ShouldBe(4);
@@ -558,15 +579,15 @@ public class JellyfinMusicVideoLibraryScannerTests
[Test]
public async Task ScanLibrary_Should_Not_Double_Insert_Duplicate_Named_Incoming_Collection_Entries()
{
JellyfinLibrary library = BuildLibrary(1, "/data/music");
var connectionParameters = new JellyfinConnectionParameters("http://jellyfin", "api-key", 1);
const string VideoPath = "/data/music/artist1/song1.mkv";
// the existing item carries NONE of the incoming names, so every incoming entry is an "add"
var existing = new MusicVideo
var existing = new JellyfinMusicVideo
{
Id = 7,
ArtistId = 3,
ItemId = ItemIdFor(VideoPath),
Etag = "old-etag",
MediaVersions =
[
new MediaVersion
@@ -588,49 +609,19 @@ public class JellyfinMusicVideoLibraryScannerTests
]
};
var artistRepository = Substitute.For<IArtistRepository>();
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>())
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() }));
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>());
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository
.GetOrAdd(Arg.Any<Artist>(), Arg.Any<LibraryPath>(), Arg.Any<LibraryFolder>(), Arg.Any<string>())
.Returns(Right<BaseError, MediaItemScanResult<MusicVideo>>(
new MediaItemScanResult<MusicVideo>(existing) { IsAdded = false }));
musicVideoRepository.FindMusicVideoPaths(Arg.Any<LibraryPath>())
.Returns(new List<string> { VideoPath }.AsEnumerable());
musicVideoRepository.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Any<Genre>()).Returns(true);
musicVideoRepository.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Any<Tag>()).Returns(true);
musicVideoRepository.AddStudio(Arg.Any<MusicVideoMetadata>(), Arg.Any<Studio>()).Returns(true);
musicVideoRepository.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Any<MusicVideoArtist>()).Returns(true);
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>())
.Returns(false);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
// Jellyfin reports each name TWICE for the same item
JellyfinMusicVideoLibraryScanner scanner = BuildScannerWith(
FakeApi(() => BuildIncoming(
VideoPath, "Artist 1", "Song 1",
genres: ["Synthwave", "Synthwave"],
tags: ["DupTag", "DupTag"],
studios: ["DupStudio", "DupStudio"],
artists: ["Artist 1", "Artist 1"])),
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository);
(JellyfinMusicVideoLibraryScanner scanner, IMusicVideoRepository musicVideoRepository, _) =
BuildScannerWithSubstitutes(
existing,
FakeApi(() => BuildIncoming(
VideoPath, "Artist 1", "Song 1",
genres: ["Synthwave", "Synthwave"],
tags: ["DupTag", "DupTag"],
studios: ["DupStudio", "DupStudio"],
artists: ["Artist 1", "Artist 1"])));
(await scanner.ScanLibrary(connectionParameters, library, deepScan: false, CancellationToken.None))
.IsRight.ShouldBeTrue();
(await scanner.ScanLibrary(ConnectionParameters, BuildLibrary(1, "/data/music"), deepScan: false,
CancellationToken.None)).IsRight.ShouldBeTrue();
await musicVideoRepository.Received(1)
.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Is<Genre>(g => g.Name == "Synthwave"));
@@ -649,37 +640,131 @@ public class JellyfinMusicVideoLibraryScannerTests
metadata.Artists.Count.ShouldBe(1);
}
private JellyfinMusicVideoLibraryScanner BuildScannerWith(
IJellyfinApiClient apiClient,
IArtistRepository artistRepository,
IMusicVideoRepository musicVideoRepository,
ILibraryRepository libraryRepository,
IMetadataRepository metadataRepository)
private static JellyfinConnectionParameters ConnectionParameters =>
new("http://jellyfin", "api-key", 1);
// A stable per-file server item id, so a re-scan of the same file is the same identity.
private static string ItemIdFor(string path) => $"item-{PathUtils.GetPathHash(path)}";
private (JellyfinMusicVideoLibraryScanner Scanner, IMusicVideoRepository MusicVideoRepository,
IMetadataRepository MetadataRepository) BuildScannerWithSubstitutes(
JellyfinMusicVideo existing,
IJellyfinApiClient apiClient)
{
var artistRepository = Substitute.For<IArtistRepository>();
artistRepository.GetArtistByMetadata(Arg.Any<int>(), Arg.Any<ArtistMetadata>())
.Returns(Some(new Artist { Id = 3, ArtistMetadata = new List<ArtistMetadata>() }));
artistRepository.DeleteEmptyArtists(Arg.Any<LibraryPath>()).Returns(new List<int>());
var jellyfinMusicVideoRepository = Substitute.For<IJellyfinMusicVideoRepository>();
jellyfinMusicVideoRepository.GetOrAdd(
Arg.Any<JellyfinLibrary>(),
Arg.Any<Artist>(),
Arg.Any<LibraryFolder>(),
Arg.Any<JellyfinMusicVideo>(),
Arg.Any<bool>(),
Arg.Any<CancellationToken>())
.Returns(Right<BaseError, MediaItemScanResult<JellyfinMusicVideo>>(
new MediaItemScanResult<JellyfinMusicVideo>(existing) { IsAdded = false }));
jellyfinMusicVideoRepository.GetExistingMusicVideos(Arg.Any<JellyfinLibrary>())
.Returns(new List<JellyfinItemEtag>
{
new() { ItemId = existing.ItemId, Etag = existing.Etag, State = MediaItemState.Normal }
});
jellyfinMusicVideoRepository.FlagFileNotFound(Arg.Any<JellyfinLibrary>(), Arg.Any<List<string>>())
.Returns(new List<int>());
var musicVideoRepository = Substitute.For<IMusicVideoRepository>();
musicVideoRepository.AddGenre(Arg.Any<MusicVideoMetadata>(), Arg.Any<Genre>()).Returns(true);
musicVideoRepository.AddTag(Arg.Any<MusicVideoMetadata>(), Arg.Any<Tag>()).Returns(true);
musicVideoRepository.AddStudio(Arg.Any<MusicVideoMetadata>(), Arg.Any<Studio>()).Returns(true);
musicVideoRepository.AddArtist(Arg.Any<MusicVideoMetadata>(), Arg.Any<MusicVideoArtist>()).Returns(true);
musicVideoRepository.RemoveArtist(Arg.Any<MusicVideoArtist>()).Returns(true);
var metadataRepository = Substitute.For<IMetadataRepository>();
metadataRepository.Update(Arg.Any<ErsatzTV.Core.Domain.Metadata>()).Returns(true);
metadataRepository.UpdateStatistics(Arg.Any<MediaItem>(), Arg.Any<MediaVersion>(), Arg.Any<bool>())
.Returns(false);
metadataRepository.RemoveGenre(Arg.Any<Genre>()).Returns(true);
metadataRepository.RemoveTag(Arg.Any<Tag>()).Returns(true);
metadataRepository.RemoveStudio(Arg.Any<Studio>()).Returns(true);
var libraryRepository = Substitute.For<ILibraryRepository>();
libraryRepository.GetParentFolderId(Arg.Any<LibraryPath>(), Arg.Any<string>(), Arg.Any<CancellationToken>())
.Returns(Option<int>.None);
libraryRepository.GetOrAddFolder(Arg.Any<LibraryPath>(), Arg.Any<Option<int>>(), Arg.Any<string>())
.Returns(new LibraryFolder { Id = 5, Path = "/data/music/artist1" });
var scanner = new JellyfinMusicVideoLibraryScanner(
BuildScannerProxy(),
apiClient,
jellyfinMusicVideoRepository,
BuildPathReplacement(),
BuildMediaSourceRepository(),
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository,
BuildFileSystem(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
return (scanner, musicVideoRepository, metadataRepository);
}
private (JellyfinMusicVideoLibraryScanner Scanner, IScannerProxy ScannerProxy) BuildScanner(
IJellyfinApiClient apiClient)
{
IScannerProxy scannerProxy = BuildScannerProxy();
var scanner = new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
new JellyfinMusicVideoRepository(_db.Factory, NullLogger<JellyfinMusicVideoRepository>.Instance),
BuildPathReplacement(),
BuildMediaSourceRepository(),
new ArtistRepository(_db.Factory),
new MusicVideoRepository(_db.Factory),
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
Substitute.For<IMetadataRepository>(),
BuildFileSystem(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
return (scanner, scannerProxy);
}
private static IScannerProxy BuildScannerProxy()
{
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
return scannerProxy;
}
private static IJellyfinPathReplacementService BuildPathReplacement()
{
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>();
pathReplacement
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>())
.Returns(ci => ci.ArgAt<string>(1));
return pathReplacement;
}
private static IMediaSourceRepository BuildMediaSourceRepository()
{
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
return mediaSourceRepository;
}
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
return new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
pathReplacement,
mediaSourceRepository,
artistRepository,
musicVideoRepository,
libraryRepository,
metadataRepository,
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
// every scanned file "exists" locally, so scanned items settle in Normal and a FileNotFound state can only
// come from the sweep under test
private static IFileSystem BuildFileSystem()
{
var fileSystem = Substitute.For<IFileSystem>();
fileSystem.File.Exists(Arg.Any<string>()).Returns(true);
return fileSystem;
}
private async Task<int> MusicVideoId(string path)
@@ -694,6 +779,13 @@ public class JellyfinMusicVideoLibraryScannerTests
.Id;
}
private async Task<MediaItemState> MediaItemStateOf(int mediaItemId)
{
await using TvContext context = _db.CreateContext();
MediaItem mediaItem = await context.MediaItems.SingleAsync(mi => mi.Id == mediaItemId);
return mediaItem.State;
}
private async Task<List<string>> MusicVideoPaths(int libraryPathId)
{
await using TvContext context = _db.CreateContext();
@@ -718,78 +810,9 @@ public class JellyfinMusicVideoLibraryScannerTests
return artists.Select(a => a.ArtistMetadata.Single().Title).OrderBy(t => t).ToList();
}
private (JellyfinMusicVideoLibraryScanner Scanner, IScannerProxy ScannerProxy) BuildScanner(
IJellyfinApiClient apiClient)
{
var pathReplacement = Substitute.For<IJellyfinPathReplacementService>();
pathReplacement
.GetReplacementJellyfinPath(Arg.Any<List<JellyfinPathReplacement>>(), Arg.Any<string>(), Arg.Any<bool>())
.Returns(ci => ci.ArgAt<string>(1));
var mediaSourceRepository = Substitute.For<IMediaSourceRepository>();
mediaSourceRepository.GetJellyfinPathReplacements(Arg.Any<int>())
.Returns(Task.FromResult(new List<JellyfinPathReplacement>()));
var scannerProxy = Substitute.For<IScannerProxy>();
scannerProxy.UpdateProgress(Arg.Any<decimal>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.ReindexMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
scannerProxy.RemoveMediaItems(Arg.Any<int[]>(), Arg.Any<CancellationToken>()).Returns(true);
var scanner = new JellyfinMusicVideoLibraryScanner(
scannerProxy,
apiClient,
pathReplacement,
mediaSourceRepository,
new ArtistRepository(_db.Factory),
new MusicVideoRepository(_db.Factory),
new LibraryRepository(Substitute.For<IFileSystem>(), _db.Factory),
Substitute.For<IMetadataRepository>(),
NullLogger<JellyfinMusicVideoLibraryScanner>.Instance);
return (scanner, scannerProxy);
}
// #484 finding 3: the api client records projection failures into the counter the SCANNER created and
// handed it. This fake does the same, so the test exercises the same-instance join that
// JellyfinMusicVideoLibraryScanner.ScanLibrary makes between GetMusicVideoLibraryItems and the sweep.
private static IJellyfinApiClient FakeApiWithProjectionFailures(
int projectionFailureCount,
params Func<MusicVideo>[] items)
{
var apiClient = Substitute.For<IJellyfinApiClient>();
apiClient.GetMusicVideoLibraryItems(
Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<JellyfinLibrary>(),
Arg.Any<MediaServerProjectionFailureCounter>())
.Returns(ci => ItemsWithFailures(
items,
projectionFailureCount,
ci.ArgAt<MediaServerProjectionFailureCounter>(3)));
return apiClient;
}
private static async IAsyncEnumerable<Tuple<MusicVideo, int>> ItemsWithFailures(
Func<MusicVideo>[] items,
int projectionFailureCount,
MediaServerProjectionFailureCounter projectionFailures)
{
for (var i = 0; i < projectionFailureCount; i++)
{
projectionFailures.RecordFailure();
}
foreach (Func<MusicVideo> item in items)
{
yield return new Tuple<MusicVideo, int>(item(), items.Length + projectionFailureCount);
}
await Task.CompletedTask;
}
// Each Func builds a fresh MusicVideo so the async stream can be re-enumerated across ScanLibrary calls
// Each Func builds a fresh music video so the async stream can be re-enumerated across ScanLibrary calls
// (an IAsyncEnumerable iterator is single-use, and the scanner mutates the incoming item).
private static IJellyfinApiClient FakeApi(params Func<MusicVideo>[] items)
private static IJellyfinApiClient FakeApi(params Func<JellyfinMusicVideo>[] items)
{
var apiClient = Substitute.For<IJellyfinApiClient>();
apiClient.GetMusicVideoLibraryItems(
@@ -801,31 +824,31 @@ public class JellyfinMusicVideoLibraryScannerTests
return apiClient;
}
private static async IAsyncEnumerable<Tuple<MusicVideo, int>> Items(Func<MusicVideo>[] items)
private static async IAsyncEnumerable<Tuple<JellyfinMusicVideo, int>> Items(Func<JellyfinMusicVideo>[] items)
{
foreach (Func<MusicVideo> item in items)
foreach (Func<JellyfinMusicVideo> item in items)
{
yield return new Tuple<MusicVideo, int>(item(), items.Length);
yield return new Tuple<JellyfinMusicVideo, int>(item(), items.Length);
}
await Task.CompletedTask;
}
private static JellyfinLibrary BuildLibrary(int libraryPathId, string path)
private static JellyfinLibrary BuildLibrary(int libraryPathId, string path, int libraryId = 42)
{
var libraryPath = new LibraryPath { Id = libraryPathId, Path = path, LibraryFolders = null };
return new JellyfinLibrary
{
Id = 42,
Id = libraryId,
MediaSourceId = 1,
ItemId = "lib15",
ItemId = $"lib{libraryId}",
Name = "Music",
ShouldSyncItems = true,
Paths = new List<LibraryPath> { libraryPath }
};
}
private static MusicVideo BuildIncoming(
private static JellyfinMusicVideo BuildIncoming(
string path,
string artistName,
string title,
@@ -835,6 +858,8 @@ public class JellyfinMusicVideoLibraryScannerTests
IEnumerable<string> artists = null) =>
new()
{
ItemId = ItemIdFor(path),
Etag = $"etag-{title}",
MediaVersions = new List<MediaVersion>
{
new()
@@ -858,18 +883,76 @@ public class JellyfinMusicVideoLibraryScannerTests
}
};
private static async IAsyncEnumerable<Tuple<MusicVideo, int>> OneItem(MusicVideo musicVideo)
{
yield return new Tuple<MusicVideo, int>(musicVideo, 1);
await Task.CompletedTask;
}
private async Task<int> SeedLibraryPath(string path)
// Seeds a real JellyfinLibrary row owning the LibraryPath. ersatztv#496 made this load-bearing: identity
// lookups and the sweep are scoped per LIBRARY (GetExistingMusicVideos / FlagFileNotFound both join
// LibraryPath.LibraryId), so a dangling LibraryPath with no owning library would silently match nothing.
private async Task<int> SeedLibraryPath(string path, int libraryId = 42)
{
await using TvContext context = _db.CreateContext();
var libraryPath = new LibraryPath { Path = path };
await context.LibraryPaths.AddAsync(libraryPath);
var library = new JellyfinLibrary
{
Id = libraryId,
MediaSourceId = 1,
ItemId = $"lib{libraryId}",
Name = "Music",
ShouldSyncItems = true,
Paths = new List<LibraryPath> { libraryPath }
};
await context.Libraries.AddAsync(library);
await context.SaveChangesAsync();
return libraryPath.Id;
}
// a MusicVideo row with NO JellyfinMusicVideo identity — what the pre-#496 scanner (and the local
// MusicVideoFolderScanner) leaves behind
private async Task<int> SeedPlainMusicVideo(int libraryPathId, string path, string artistName)
{
await using TvContext context = _db.CreateContext();
var artist = new Artist
{
LibraryPathId = libraryPathId,
ArtistMetadata = new List<ArtistMetadata> { new() { Title = artistName } }
};
await context.Artists.AddAsync(artist);
await context.SaveChangesAsync();
var musicVideo = new MusicVideo
{
ArtistId = artist.Id,
LibraryPathId = libraryPathId,
MusicVideoMetadata = new List<MusicVideoMetadata> { new() { Title = "Song 1" } },
MediaVersions = new List<MediaVersion>
{
new()
{
MediaFiles = new List<MediaFile>
{
new() { Path = path, PathHash = PathUtils.GetPathHash(path) }
},
Streams = new List<MediaStream>()
}
}
};
await context.MusicVideos.AddAsync(musicVideo);
await context.SaveChangesAsync();
return musicVideo.Id;
}
private async Task<int> SeedCollectionWith(int mediaItemId)
{
await using TvContext context = _db.CreateContext();
var collection = new Collection
{
Name = "Vaporwave",
CollectionItems = new List<CollectionItem> { new() { MediaItemId = mediaItemId } }
};
await context.Collections.AddAsync(collection);
await context.SaveChangesAsync();
return collection.Id;
}
}
+1
View File
@@ -1127,6 +1127,7 @@ public class Startup
services.AddScoped<IJellyfinTelevisionRepository, JellyfinTelevisionRepository>();
services.AddScoped<IJellyfinCollectionRepository, JellyfinCollectionRepository>();
services.AddScoped<IJellyfinMovieRepository, JellyfinMovieRepository>();
services.AddScoped<IJellyfinMusicVideoRepository, JellyfinMusicVideoRepository>();
services.AddScoped<IEmbyApiClient, EmbyApiClient>();
services.AddScoped<IEmbyPathReplacementService, EmbyPathReplacementService>();
services.AddScoped<IEmbyTelevisionRepository, EmbyTelevisionRepository>();
+38 -41
View File
@@ -2530,47 +2530,6 @@ remote scanner tripped it, and the feature had never run in prod, CI, or locally
still reads `libraryPath.LibraryFolders` directly, but it is only reached on the local (eager-loaded) path, so
it is not affected; left as-is (out of #488 scope).
## 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)
`key: scan.musicvideo-reconciliation` · `status: active` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: none`
**Rule:** `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity.
**Signals:** music-video trash sweep, path-based identity, cross-kind safety, path-keyed identity, empty-fetch guard reuse, remove-stale+add-new dedup · paths: `JellyfinMusicVideoLibraryScanner.TrashMissingMusicVideos`, `FindMusicVideoPaths`/`DeleteByPath`, `IMusicVideoRepository`, `MediaServerReconciliationGuard` · issues: #494, #477, #488, #496, #500
**Mechanics:** `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`; `ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items`; integration tests extending the #488 harness. #500 — when mirroring the remove-stale + add-new idiom, dedup the incoming set on **the same key its add filter compares** (the filter is materialized before the loop mutates `existing`, so duplicates both pass): `Name`, `Guid` for guids, and for Plex `Actors` an artwork-preferring dedup shared with the remove filter (whose key is `(Name, artwork-presence)`). Remaining un-deduped copies of the idiom: #600.
The Jellyfin music-video scanner did add/update only — a music video removed on the Jellyfin side lingered in
ErsatzTV forever and could still be scheduled. It now runs a trash sweep at the end of `ScanLibrary`
(`TrashMissingMusicVideos`), mirroring the `MediaServer{Movie,Television,OtherVideo}LibraryScanner` "gone
upstream ⇒ remove" pattern but with a deliberately different identity function, because music videos lack the
media-server identity those base scanners rely on.
- **Identity is (LibraryPathId, path), not server itemId.** The base scanners diff `GetExisting*` (keyed by
`MediaServerItemId`) against the incoming server item ids, then soft-trash via `FlagFileNotFound`. Music videos
have **no `JellyfinMusicVideo` entity and no `ItemId`/`Etag`** — the scanner is a standalone
`IJellyfinMusicVideoLibraryScanner` that injects the *local* `IMusicVideoRepository`, which offers no
itemId-keyed existing-set or flag seam. So the sweep diffs the **local path** set instead: existing =
`FindMusicVideoPaths(libraryPath)` `.Except` the incoming items' replaced local paths, then hard-deletes the
remainder with `DeleteByPath` + `IScannerProxy.RemoveMediaItems`, and cleans now-empty artists with
`IArtistRepository.DeleteEmptyArtists`. Hard delete (not soft `FileNotFound` trash) because there is no
per-item FileNotFound seam on this path and the issue's Done-when is "removed".
- **Cross-kind safety is a property of the queries, not the media kind.** `MediaItem` is TPT with `LibraryPathId`
on the abstract base, so a Movie, Show and MusicVideo can share one `LibraryPath` (a mixed Jellyfin library).
Both `FindMusicVideoPaths` and `DeleteByPath` filter `LibraryPathId` **and** join the concrete `MusicVideo`
table, so the sweep can only ever see/delete music videos — a Movie/Show under the same `LibraryPath` is
invisible to it. Pinned by `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`.
- **Reuses the #477 empty-fetch guard.** The sweep is gated by `MediaServerReconciliationGuard.ShouldFlagMissing`
— a successful fetch that returns zero items (server mid-restore / transient) is indistinguishable from a real
emptying, so the whole-library wipe is refused and logged. Pinned as a negative control by
`ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items` (removing the guard flips it red).
- **Known limitation (deferred to per-library identity).** `MusicVideoRepository.GetOrAdd` dedups a path
**globally** (no `LibraryPathId` predicate), so a file served by two libraries with overlapping local paths is
a single row owned by whichever library scanned it first. If that owner later stops reporting the file while
another library still serves it, this sweep removes the shared row. A proper fix needs per-library music-video
identity (a `JellyfinMusicVideo` etag entity + migration) — the issue's "option 2 / fold into the base
scanner" refactor — tracked as #496.
- **Tests.** Integration tests (real `ArtistRepository`/`MusicVideoRepository`/`LibraryRepository` over in-memory
SQLite, extending the #488 harness) pin removal, empty-artist cleanup, cross-kind safety, and the empty-fetch
guard. Proven non-vacuous: all four fail against the pre-fix scanner except the guard control, which only
earns its keep once the sweep exists.
## 2026-07-20 (#489) — Jellyfin mixed-content libraries map to one library holding many kinds
`key: scan.jellyfin-mixed-content-library` · `status: active` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: none`
**Rule:** A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind.
@@ -3895,3 +3854,41 @@ This is a **second, independent** refusal on the same guard, plus a decision not
Sweep`, the two nested TV cases, and `JellyfinMusicVideoLibraryScannerTests
.MusicVideo_Sweep_Respects_Projection_Failures` all drive the real `ScanLibrary` entry point and record
the failure from *inside* the enumeration, so same-instance wiring is what makes them pass.
## 2026-07-25 — Music videos carry a per-library server identity; reconciliation is an itemId diff + soft trash (#496)
`key: scan.musicvideo-server-identity` · `status: active` · `since: 2026-07-25` · `supersedes: scan.musicvideo-reconciliation@2026-07-20` · `superseded-by: none`
**Rule:** Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added.
**Signals:** music-video server identity, JellyfinMusicVideo ItemId/Etag, itemId diff, soft FileNotFound trash, adoption of pre-identity rows, cross-library false-trash, path-replaced PathHash · paths: `JellyfinMusicVideo`, `JellyfinMusicVideoRepository`, `IMediaServerMusicVideoRepository`, `MediaServerMusicVideoLibraryScanner`, `JellyfinMusicVideoLibraryScanner` · issues: #496, #494, #477, #488, #497, #500
**Mechanics:** dual-provider migration `Add_JellyfinMusicVideo`; adoption probe joins `MediaFile.PathHash` + `NOT EXISTS (JellyfinMusicVideo)` filtered to the library's `LibraryPath`; `AddMusicVideo` normalizes `Path`/`PathHash` to the path-REPLACED local path; `ScanLibrary_Should_Adopt_PreExisting_MusicVideo_Preserving_Identity_And_Collections`, `ScanLibrary_Should_Not_Adopt_A_MusicVideo_Owned_By_Another_LibraryPath`, `ScanLibrary_Should_Not_Flag_MusicVideos_Owned_By_Another_Library`
#494 gave music videos a trash sweep but had to key it on `(LibraryPathId, path)` and hard-delete, because
music videos carried no server identity. That left the known limitation this issue is named for: a file served
by two libraries with overlapping local paths is one row owned by whichever library scanned it first, and that
owner's sweep **destroyed** the row the other library still served. This is the deferred "option 2".
- **Identity is the server item id, per library.** `JellyfinMusicVideo` mirrors `JellyfinMovie` exactly (TPT
table, `ItemId`/`Etag` `varchar(36)`, index on `ItemId`). `GetExistingMusicVideos` and `FlagFileNotFound` both
join `LibraryPath.LibraryId`, so the existing-set and the flag set are scoped to the scanning library — one
library's sweep can no longer resolve, let alone remove, another library's row.
- **Soft trash replaces hard delete.** The sweep now flags `FileNotFound` (`State = 1`) like the
movie/TV/other-video base scanners. The row survives, so collection membership, playout references and
artwork survive with it, and removal is `EmptyTrash`-governed and reversible. The visible consequence is that
`DeleteEmptyArtists` no longer fires from a sweep — a trashed music video still belongs to its artist.
- **Pre-identity rows are ADOPTED, not re-added.** Every music video on an existing install has no
`JellyfinMusicVideo` row and so can never be found by item id. Deleting and re-adding would mint a new
`MediaItem` id and silently drop `CollectionItem` membership; `MediaItemRepository.MediaFileAlreadyExists`
would in fact block the re-add outright and the item would error on every scan forever. So `GetOrAdd` probes
for an identity-less `MusicVideo` at the same `PathHash` **within the scanned library's own `LibraryPath`**
and inserts the identity row against that same id. The scoping is the point: the local
`MusicVideoFolderScanner` writes the same `MusicVideo` table, and a local (or second-library) row must never
be hijacked into this library's identity.
- **The adopted row is written with an empty etag** so the ordinary "etag changed ⇒ refresh" path picks it up
once, rather than needing a second adoption-specific update path.
- **`Path`/`PathHash` are normalized to the path-REPLACED local path on add.** The projection fills them from
the path Jellyfin reported, but music videos have always stored the replaced local path. Storing the
projection's hash would break every later `PathHash` lookup — `MediaFileAlreadyExists` and the adoption probe
above. Caught by the #488 integration test failing on a `NOT NULL`/`UNIQUE` `PathHash` constraint.
- **Scope honesty: this is parity, not a total fix.** One file path is still one `MediaItem` row globally
(`MediaFileAlreadyExists` is a global path-hash guard), so a second library serving the same file still gets
no row of its own — exactly as for movies/TV. What changes is that the first library's sweep now *flags*
rather than *destroys* that shared row. The unrecoverable data loss is gone; the shared-row limitation is a
whole-app property, not a music-video one.
+1
View File
@@ -115,6 +115,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](../decisions.md#2026-07-20-489--jellyfin-mixed-content-libraries-map-to-one-library-holding-many-kinds) |
| `scan.musicvideo-reconciliation` | `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity. | 2026-07-20 | [link](../decisions.md#2026-07-20--jellyfinmusicvideolibraryscanner-reconciles-by-library-scoped-path-diff--hard-delete-not-server-itemid-soft-trash-494) |
| `scan.projection-failure-sweep-guard` | `MediaServerReconciliationGuard` takes a per-enumeration projection-failure count and refuses the file-not-found sweep (logged loudly) whenever it is non-zero against a non-empty existing set — at all six sweeps, including the nested per-show season and per-season episode ones #477 left unguarded (via `ShouldFlagMissingDescendants`, which applies the failure refusal but not #477's empty-fetch branch). Deliberate guard-clause skips (STRM, virtual, unsupported type) are explicitly **not** failures and never suppress a sweep. The missing-fraction / ratio threshold floated by #477 is **rejected**, not deferred. | 2026-07-25 | [link](../decisions.md#2026-07-25--a-media-server-sweep-also-refuses-when-the-api-client-silently-dropped-items-whose-projection-threw-the-ratio-threshold-is-rejected-484) |
| `scan.musicvideo-server-identity` | Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added. | 2026-07-25 | [link](../decisions.md#2026-07-25--music-videos-carry-a-per-library-server-identity-reconciliation-is-an-itemid-diff--soft-trash-496) |
| `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](../decisions.md#2026-07-19--a-media-server-library-sweep-refuses-to-flag-when-a-successful-fetch-returns-zero-items-rather-than-nuking-the-whole-library-477) |
| `sched.auto-tune-foundation` | Auto-tune preview enumeration uses EF distinct+count queries for exact counts, while each created channel is persisted as a live SmartCollection; coexistence with existing channels/numbers is additive-only, never mutating. | 2026-07-16 | [link](../decisions.md#2026-07-16--auto-tuning-enumerates-via-ef-persists-via-smartcollection-additive-coexistence-69) |
| `sched.autotune-detailpanel-members` | The Auto-Tune DetailPanel's per-channel content-source list is a live `ISearchIndex.Search` roll-up through the server-owned `AutoTuneAxisMap.GenerateQuery`, not an EF distinct+count query, so the preview matches exactly what the built channel's SmartCollection will contain. | 2026-07-17 | [link](../decisions.md#2026-07-17--auto-tune-detailpanel-member-list--live-search-index-roll-up-not-ef-enumeration-384) |
+49
View File
@@ -0,0 +1,49 @@
# Archive — library scanning / media-server reconciliation
Superseded/retired records for the media-server library scanners and their reconciliation
strategies. See `docs/decisions/archive/README.md` for the archive's general rules (rationale kept
verbatim, never in the active read-path). Active successor for music-video reconciliation:
`scan.musicvideo-server-identity` in `docs/decisions.md`.
---
## 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)
`key: scan.musicvideo-reconciliation` · `status: superseded` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: scan.musicvideo-server-identity@2026-07-25`
**Rule:** (superseded) `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity.
**Signals:** music-video trash sweep, path-based identity, cross-kind safety, path-keyed identity, empty-fetch guard reuse, remove-stale+add-new dedup · paths: `JellyfinMusicVideoLibraryScanner.TrashMissingMusicVideos`, `FindMusicVideoPaths`/`DeleteByPath`, `IMusicVideoRepository`, `MediaServerReconciliationGuard` · issues: #494, #477, #488, #496, #500
**Mechanics:** `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`; `ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items`; integration tests extending the #488 harness. #500 — when mirroring the remove-stale + add-new idiom, dedup the incoming set on **the same key its add filter compares** (the filter is materialized before the loop mutates `existing`, so duplicates both pass): `Name`, `Guid` for guids, and for Plex `Actors` an artwork-preferring dedup shared with the remove filter (whose key is `(Name, artwork-presence)`). Remaining un-deduped copies of the idiom: #600. Superseded by `scan.musicvideo-server-identity` (ersatztv#496): music videos gained a `JellyfinMusicVideo` ItemId/Etag identity, so the path diff + hard delete became an itemId diff + soft `FileNotFound` trash.
The Jellyfin music-video scanner did add/update only — a music video removed on the Jellyfin side lingered in
ErsatzTV forever and could still be scheduled. It now runs a trash sweep at the end of `ScanLibrary`
(`TrashMissingMusicVideos`), mirroring the `MediaServer{Movie,Television,OtherVideo}LibraryScanner` "gone
upstream ⇒ remove" pattern but with a deliberately different identity function, because music videos lack the
media-server identity those base scanners rely on.
- **Identity is (LibraryPathId, path), not server itemId.** The base scanners diff `GetExisting*` (keyed by
`MediaServerItemId`) against the incoming server item ids, then soft-trash via `FlagFileNotFound`. Music videos
have **no `JellyfinMusicVideo` entity and no `ItemId`/`Etag`** — the scanner is a standalone
`IJellyfinMusicVideoLibraryScanner` that injects the *local* `IMusicVideoRepository`, which offers no
itemId-keyed existing-set or flag seam. So the sweep diffs the **local path** set instead: existing =
`FindMusicVideoPaths(libraryPath)` `.Except` the incoming items' replaced local paths, then hard-deletes the
remainder with `DeleteByPath` + `IScannerProxy.RemoveMediaItems`, and cleans now-empty artists with
`IArtistRepository.DeleteEmptyArtists`. Hard delete (not soft `FileNotFound` trash) because there is no
per-item FileNotFound seam on this path and the issue's Done-when is "removed".
- **Cross-kind safety is a property of the queries, not the media kind.** `MediaItem` is TPT with `LibraryPathId`
on the abstract base, so a Movie, Show and MusicVideo can share one `LibraryPath` (a mixed Jellyfin library).
Both `FindMusicVideoPaths` and `DeleteByPath` filter `LibraryPathId` **and** join the concrete `MusicVideo`
table, so the sweep can only ever see/delete music videos — a Movie/Show under the same `LibraryPath` is
invisible to it. Pinned by `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`.
- **Reuses the #477 empty-fetch guard.** The sweep is gated by `MediaServerReconciliationGuard.ShouldFlagMissing`
— a successful fetch that returns zero items (server mid-restore / transient) is indistinguishable from a real
emptying, so the whole-library wipe is refused and logged. Pinned as a negative control by
`ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items` (removing the guard flips it red).
- **Known limitation (deferred to per-library identity).** `MusicVideoRepository.GetOrAdd` dedups a path
**globally** (no `LibraryPathId` predicate), so a file served by two libraries with overlapping local paths is
a single row owned by whichever library scanned it first. If that owner later stops reporting the file while
another library still serves it, this sweep removes the shared row. A proper fix needs per-library music-video
identity (a `JellyfinMusicVideo` etag entity + migration) — the issue's "option 2 / fold into the base
scanner" refactor — tracked as #496.
- **Tests.** Integration tests (real `ArtistRepository`/`MusicVideoRepository`/`LibraryRepository` over in-memory
SQLite, extending the #488 harness) pin removal, empty-artist cleanup, cross-kind safety, and the empty-fetch
guard. Proven non-vacuous: all four fail against the pre-fix scanner except the guard control, which only
earns its keep once the sweep exists.