fix(496): give music videos a per-library server identity; itemId diff + soft trash
Music videos carried no server identity, so JellyfinMusicVideoLibraryScanner had to reconcile by a (LibraryPathId, path) diff and HARD-delete the remainder. A file served by two libraries with overlapping local paths is a single row owned by whichever library scanned it first, so that owner's sweep destroyed a row another library still served — taking collection membership and playout references with it, irreversibly. This is #494's deferred "option 2": - New JellyfinMusicVideo : MusicVideo (ItemId/Etag), mirroring JellyfinMovie — TPT table, varchar(36), ItemId index. Dual-provider migration Add_JellyfinMusicVideo. - New IMediaServerMusicVideoRepository + JellyfinMusicVideoRepository: itemId-keyed existing-set/lookup and Flag{Normal,Unavailable,FileNotFound} seams, all scoped per library via LibraryPath.LibraryId. - New MediaServerMusicVideoLibraryScanner base; JellyfinMusicVideoLibraryScanner folds onto it and keeps the #177/#488/#497/#500 metadata-reconcile logic verbatim. - The sweep now soft-trashes (FileNotFound) instead of deleting, so removal is reversible and EmptyTrash-governed. DeleteEmptyArtists consequently no longer fires from a sweep. - Pre-identity rows are ADOPTED in place: the identity row is inserted against the same MediaItem id, scoped to the scanned library's own LibraryPath, so collection membership survives and a local/second-library row is never hijacked. - AddMusicVideo normalizes Path/PathHash to the path-REPLACED local path; the projection fills them from the server-reported path, which would break every later PathHash lookup. Docs: scan.musicvideo-reconciliation relocated to docs/decisions/archive/scan.md as superseded; new active record scan.musicvideo-server-identity. fixes #496
This commit is contained in:
+38
-41
@@ -2530,47 +2530,6 @@ remote scanner tripped it, and the feature had never run in prod, CI, or locally
|
||||
still reads `libraryPath.LibraryFolders` directly, but it is only reached on the local (eager-loaded) path, so
|
||||
it is not affected; left as-is (out of #488 scope).
|
||||
|
||||
## 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)
|
||||
`key: scan.musicvideo-reconciliation` · `status: active` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity.
|
||||
**Signals:** music-video trash sweep, path-based identity, cross-kind safety, path-keyed identity, empty-fetch guard reuse, remove-stale+add-new dedup · paths: `JellyfinMusicVideoLibraryScanner.TrashMissingMusicVideos`, `FindMusicVideoPaths`/`DeleteByPath`, `IMusicVideoRepository`, `MediaServerReconciliationGuard` · issues: #494, #477, #488, #496, #500
|
||||
**Mechanics:** `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`; `ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items`; integration tests extending the #488 harness. #500 — when mirroring the remove-stale + add-new idiom, dedup the incoming set on **the same key its add filter compares** (the filter is materialized before the loop mutates `existing`, so duplicates both pass): `Name`, `Guid` for guids, and for Plex `Actors` an artwork-preferring dedup shared with the remove filter (whose key is `(Name, artwork-presence)`). Remaining un-deduped copies of the idiom: #600.
|
||||
|
||||
The Jellyfin music-video scanner did add/update only — a music video removed on the Jellyfin side lingered in
|
||||
ErsatzTV forever and could still be scheduled. It now runs a trash sweep at the end of `ScanLibrary`
|
||||
(`TrashMissingMusicVideos`), mirroring the `MediaServer{Movie,Television,OtherVideo}LibraryScanner` "gone
|
||||
upstream ⇒ remove" pattern but with a deliberately different identity function, because music videos lack the
|
||||
media-server identity those base scanners rely on.
|
||||
|
||||
- **Identity is (LibraryPathId, path), not server itemId.** The base scanners diff `GetExisting*` (keyed by
|
||||
`MediaServerItemId`) against the incoming server item ids, then soft-trash via `FlagFileNotFound`. Music videos
|
||||
have **no `JellyfinMusicVideo` entity and no `ItemId`/`Etag`** — the scanner is a standalone
|
||||
`IJellyfinMusicVideoLibraryScanner` that injects the *local* `IMusicVideoRepository`, which offers no
|
||||
itemId-keyed existing-set or flag seam. So the sweep diffs the **local path** set instead: existing =
|
||||
`FindMusicVideoPaths(libraryPath)` `.Except` the incoming items' replaced local paths, then hard-deletes the
|
||||
remainder with `DeleteByPath` + `IScannerProxy.RemoveMediaItems`, and cleans now-empty artists with
|
||||
`IArtistRepository.DeleteEmptyArtists`. Hard delete (not soft `FileNotFound` trash) because there is no
|
||||
per-item FileNotFound seam on this path and the issue's Done-when is "removed".
|
||||
- **Cross-kind safety is a property of the queries, not the media kind.** `MediaItem` is TPT with `LibraryPathId`
|
||||
on the abstract base, so a Movie, Show and MusicVideo can share one `LibraryPath` (a mixed Jellyfin library).
|
||||
Both `FindMusicVideoPaths` and `DeleteByPath` filter `LibraryPathId` **and** join the concrete `MusicVideo`
|
||||
table, so the sweep can only ever see/delete music videos — a Movie/Show under the same `LibraryPath` is
|
||||
invisible to it. Pinned by `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`.
|
||||
- **Reuses the #477 empty-fetch guard.** The sweep is gated by `MediaServerReconciliationGuard.ShouldFlagMissing`
|
||||
— a successful fetch that returns zero items (server mid-restore / transient) is indistinguishable from a real
|
||||
emptying, so the whole-library wipe is refused and logged. Pinned as a negative control by
|
||||
`ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items` (removing the guard flips it red).
|
||||
- **Known limitation (deferred to per-library identity).** `MusicVideoRepository.GetOrAdd` dedups a path
|
||||
**globally** (no `LibraryPathId` predicate), so a file served by two libraries with overlapping local paths is
|
||||
a single row owned by whichever library scanned it first. If that owner later stops reporting the file while
|
||||
another library still serves it, this sweep removes the shared row. A proper fix needs per-library music-video
|
||||
identity (a `JellyfinMusicVideo` etag entity + migration) — the issue's "option 2 / fold into the base
|
||||
scanner" refactor — tracked as #496.
|
||||
- **Tests.** Integration tests (real `ArtistRepository`/`MusicVideoRepository`/`LibraryRepository` over in-memory
|
||||
SQLite, extending the #488 harness) pin removal, empty-artist cleanup, cross-kind safety, and the empty-fetch
|
||||
guard. Proven non-vacuous: all four fail against the pre-fix scanner except the guard control, which only
|
||||
earns its keep once the sweep exists.
|
||||
|
||||
## 2026-07-20 (#489) — Jellyfin mixed-content libraries map to one library holding many kinds
|
||||
`key: scan.jellyfin-mixed-content-library` · `status: active` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind.
|
||||
@@ -3895,3 +3854,41 @@ This is a **second, independent** refusal on the same guard, plus a decision not
|
||||
Sweep`, the two nested TV cases, and `JellyfinMusicVideoLibraryScannerTests
|
||||
.MusicVideo_Sweep_Respects_Projection_Failures` all drive the real `ScanLibrary` entry point and record
|
||||
the failure from *inside* the enumeration, so same-instance wiring is what makes them pass.
|
||||
## 2026-07-25 — Music videos carry a per-library server identity; reconciliation is an itemId diff + soft trash (#496)
|
||||
`key: scan.musicvideo-server-identity` · `status: active` · `since: 2026-07-25` · `supersedes: scan.musicvideo-reconciliation@2026-07-20` · `superseded-by: none`
|
||||
**Rule:** Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added.
|
||||
**Signals:** music-video server identity, JellyfinMusicVideo ItemId/Etag, itemId diff, soft FileNotFound trash, adoption of pre-identity rows, cross-library false-trash, path-replaced PathHash · paths: `JellyfinMusicVideo`, `JellyfinMusicVideoRepository`, `IMediaServerMusicVideoRepository`, `MediaServerMusicVideoLibraryScanner`, `JellyfinMusicVideoLibraryScanner` · issues: #496, #494, #477, #488, #497, #500
|
||||
**Mechanics:** dual-provider migration `Add_JellyfinMusicVideo`; adoption probe joins `MediaFile.PathHash` + `NOT EXISTS (JellyfinMusicVideo)` filtered to the library's `LibraryPath`; `AddMusicVideo` normalizes `Path`/`PathHash` to the path-REPLACED local path; `ScanLibrary_Should_Adopt_PreExisting_MusicVideo_Preserving_Identity_And_Collections`, `ScanLibrary_Should_Not_Adopt_A_MusicVideo_Owned_By_Another_LibraryPath`, `ScanLibrary_Should_Not_Flag_MusicVideos_Owned_By_Another_Library`
|
||||
|
||||
#494 gave music videos a trash sweep but had to key it on `(LibraryPathId, path)` and hard-delete, because
|
||||
music videos carried no server identity. That left the known limitation this issue is named for: a file served
|
||||
by two libraries with overlapping local paths is one row owned by whichever library scanned it first, and that
|
||||
owner's sweep **destroyed** the row the other library still served. This is the deferred "option 2".
|
||||
|
||||
- **Identity is the server item id, per library.** `JellyfinMusicVideo` mirrors `JellyfinMovie` exactly (TPT
|
||||
table, `ItemId`/`Etag` `varchar(36)`, index on `ItemId`). `GetExistingMusicVideos` and `FlagFileNotFound` both
|
||||
join `LibraryPath.LibraryId`, so the existing-set and the flag set are scoped to the scanning library — one
|
||||
library's sweep can no longer resolve, let alone remove, another library's row.
|
||||
- **Soft trash replaces hard delete.** The sweep now flags `FileNotFound` (`State = 1`) like the
|
||||
movie/TV/other-video base scanners. The row survives, so collection membership, playout references and
|
||||
artwork survive with it, and removal is `EmptyTrash`-governed and reversible. The visible consequence is that
|
||||
`DeleteEmptyArtists` no longer fires from a sweep — a trashed music video still belongs to its artist.
|
||||
- **Pre-identity rows are ADOPTED, not re-added.** Every music video on an existing install has no
|
||||
`JellyfinMusicVideo` row and so can never be found by item id. Deleting and re-adding would mint a new
|
||||
`MediaItem` id and silently drop `CollectionItem` membership; `MediaItemRepository.MediaFileAlreadyExists`
|
||||
would in fact block the re-add outright and the item would error on every scan forever. So `GetOrAdd` probes
|
||||
for an identity-less `MusicVideo` at the same `PathHash` **within the scanned library's own `LibraryPath`**
|
||||
and inserts the identity row against that same id. The scoping is the point: the local
|
||||
`MusicVideoFolderScanner` writes the same `MusicVideo` table, and a local (or second-library) row must never
|
||||
be hijacked into this library's identity.
|
||||
- **The adopted row is written with an empty etag** so the ordinary "etag changed ⇒ refresh" path picks it up
|
||||
once, rather than needing a second adoption-specific update path.
|
||||
- **`Path`/`PathHash` are normalized to the path-REPLACED local path on add.** The projection fills them from
|
||||
the path Jellyfin reported, but music videos have always stored the replaced local path. Storing the
|
||||
projection's hash would break every later `PathHash` lookup — `MediaFileAlreadyExists` and the adoption probe
|
||||
above. Caught by the #488 integration test failing on a `NOT NULL`/`UNIQUE` `PathHash` constraint.
|
||||
- **Scope honesty: this is parity, not a total fix.** One file path is still one `MediaItem` row globally
|
||||
(`MediaFileAlreadyExists` is a global path-hash guard), so a second library serving the same file still gets
|
||||
no row of its own — exactly as for movies/TV. What changes is that the first library's sweep now *flags*
|
||||
rather than *destroys* that shared row. The unrecoverable data loss is gone; the shared-row limitation is a
|
||||
whole-app property, not a music-video one.
|
||||
|
||||
@@ -115,6 +115,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
| `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](../decisions.md#2026-07-20-489--jellyfin-mixed-content-libraries-map-to-one-library-holding-many-kinds) |
|
||||
| `scan.musicvideo-reconciliation` | `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity. | 2026-07-20 | [link](../decisions.md#2026-07-20--jellyfinmusicvideolibraryscanner-reconciles-by-library-scoped-path-diff--hard-delete-not-server-itemid-soft-trash-494) |
|
||||
| `scan.projection-failure-sweep-guard` | `MediaServerReconciliationGuard` takes a per-enumeration projection-failure count and refuses the file-not-found sweep (logged loudly) whenever it is non-zero against a non-empty existing set — at all six sweeps, including the nested per-show season and per-season episode ones #477 left unguarded (via `ShouldFlagMissingDescendants`, which applies the failure refusal but not #477's empty-fetch branch). Deliberate guard-clause skips (STRM, virtual, unsupported type) are explicitly **not** failures and never suppress a sweep. The missing-fraction / ratio threshold floated by #477 is **rejected**, not deferred. | 2026-07-25 | [link](../decisions.md#2026-07-25--a-media-server-sweep-also-refuses-when-the-api-client-silently-dropped-items-whose-projection-threw-the-ratio-threshold-is-rejected-484) |
|
||||
| `scan.musicvideo-server-identity` | Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added. | 2026-07-25 | [link](../decisions.md#2026-07-25--music-videos-carry-a-per-library-server-identity-reconciliation-is-an-itemid-diff--soft-trash-496) |
|
||||
| `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](../decisions.md#2026-07-19--a-media-server-library-sweep-refuses-to-flag-when-a-successful-fetch-returns-zero-items-rather-than-nuking-the-whole-library-477) |
|
||||
| `sched.auto-tune-foundation` | Auto-tune preview enumeration uses EF distinct+count queries for exact counts, while each created channel is persisted as a live SmartCollection; coexistence with existing channels/numbers is additive-only, never mutating. | 2026-07-16 | [link](../decisions.md#2026-07-16--auto-tuning-enumerates-via-ef-persists-via-smartcollection-additive-coexistence-69) |
|
||||
| `sched.autotune-detailpanel-members` | The Auto-Tune DetailPanel's per-channel content-source list is a live `ISearchIndex.Search` roll-up through the server-owned `AutoTuneAxisMap.GenerateQuery`, not an EF distinct+count query, so the preview matches exactly what the built channel's SmartCollection will contain. | 2026-07-17 | [link](../decisions.md#2026-07-17--auto-tune-detailpanel-member-list--live-search-index-roll-up-not-ef-enumeration-384) |
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# Archive — library scanning / media-server reconciliation
|
||||
|
||||
Superseded/retired records for the media-server library scanners and their reconciliation
|
||||
strategies. See `docs/decisions/archive/README.md` for the archive's general rules (rationale kept
|
||||
verbatim, never in the active read-path). Active successor for music-video reconciliation:
|
||||
`scan.musicvideo-server-identity` in `docs/decisions.md`.
|
||||
|
||||
---
|
||||
|
||||
## 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)
|
||||
`key: scan.musicvideo-reconciliation` · `status: superseded` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: scan.musicvideo-server-identity@2026-07-25`
|
||||
**Rule:** (superseded) `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity.
|
||||
**Signals:** music-video trash sweep, path-based identity, cross-kind safety, path-keyed identity, empty-fetch guard reuse, remove-stale+add-new dedup · paths: `JellyfinMusicVideoLibraryScanner.TrashMissingMusicVideos`, `FindMusicVideoPaths`/`DeleteByPath`, `IMusicVideoRepository`, `MediaServerReconciliationGuard` · issues: #494, #477, #488, #496, #500
|
||||
**Mechanics:** `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`; `ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items`; integration tests extending the #488 harness. #500 — when mirroring the remove-stale + add-new idiom, dedup the incoming set on **the same key its add filter compares** (the filter is materialized before the loop mutates `existing`, so duplicates both pass): `Name`, `Guid` for guids, and for Plex `Actors` an artwork-preferring dedup shared with the remove filter (whose key is `(Name, artwork-presence)`). Remaining un-deduped copies of the idiom: #600. Superseded by `scan.musicvideo-server-identity` (ersatztv#496): music videos gained a `JellyfinMusicVideo` ItemId/Etag identity, so the path diff + hard delete became an itemId diff + soft `FileNotFound` trash.
|
||||
|
||||
The Jellyfin music-video scanner did add/update only — a music video removed on the Jellyfin side lingered in
|
||||
ErsatzTV forever and could still be scheduled. It now runs a trash sweep at the end of `ScanLibrary`
|
||||
(`TrashMissingMusicVideos`), mirroring the `MediaServer{Movie,Television,OtherVideo}LibraryScanner` "gone
|
||||
upstream ⇒ remove" pattern but with a deliberately different identity function, because music videos lack the
|
||||
media-server identity those base scanners rely on.
|
||||
|
||||
- **Identity is (LibraryPathId, path), not server itemId.** The base scanners diff `GetExisting*` (keyed by
|
||||
`MediaServerItemId`) against the incoming server item ids, then soft-trash via `FlagFileNotFound`. Music videos
|
||||
have **no `JellyfinMusicVideo` entity and no `ItemId`/`Etag`** — the scanner is a standalone
|
||||
`IJellyfinMusicVideoLibraryScanner` that injects the *local* `IMusicVideoRepository`, which offers no
|
||||
itemId-keyed existing-set or flag seam. So the sweep diffs the **local path** set instead: existing =
|
||||
`FindMusicVideoPaths(libraryPath)` `.Except` the incoming items' replaced local paths, then hard-deletes the
|
||||
remainder with `DeleteByPath` + `IScannerProxy.RemoveMediaItems`, and cleans now-empty artists with
|
||||
`IArtistRepository.DeleteEmptyArtists`. Hard delete (not soft `FileNotFound` trash) because there is no
|
||||
per-item FileNotFound seam on this path and the issue's Done-when is "removed".
|
||||
- **Cross-kind safety is a property of the queries, not the media kind.** `MediaItem` is TPT with `LibraryPathId`
|
||||
on the abstract base, so a Movie, Show and MusicVideo can share one `LibraryPath` (a mixed Jellyfin library).
|
||||
Both `FindMusicVideoPaths` and `DeleteByPath` filter `LibraryPathId` **and** join the concrete `MusicVideo`
|
||||
table, so the sweep can only ever see/delete music videos — a Movie/Show under the same `LibraryPath` is
|
||||
invisible to it. Pinned by `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`.
|
||||
- **Reuses the #477 empty-fetch guard.** The sweep is gated by `MediaServerReconciliationGuard.ShouldFlagMissing`
|
||||
— a successful fetch that returns zero items (server mid-restore / transient) is indistinguishable from a real
|
||||
emptying, so the whole-library wipe is refused and logged. Pinned as a negative control by
|
||||
`ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items` (removing the guard flips it red).
|
||||
- **Known limitation (deferred to per-library identity).** `MusicVideoRepository.GetOrAdd` dedups a path
|
||||
**globally** (no `LibraryPathId` predicate), so a file served by two libraries with overlapping local paths is
|
||||
a single row owned by whichever library scanned it first. If that owner later stops reporting the file while
|
||||
another library still serves it, this sweep removes the shared row. A proper fix needs per-library music-video
|
||||
identity (a `JellyfinMusicVideo` etag entity + migration) — the issue's "option 2 / fold into the base
|
||||
scanner" refactor — tracked as #496.
|
||||
- **Tests.** Integration tests (real `ArtistRepository`/`MusicVideoRepository`/`LibraryRepository` over in-memory
|
||||
SQLite, extending the #488 harness) pin removal, empty-artist cleanup, cross-kind safety, and the empty-fetch
|
||||
guard. Proven non-vacuous: all four fail against the pre-fix scanner except the guard control, which only
|
||||
earns its keep once the sweep exists.
|
||||
Reference in New Issue
Block a user