Remove-stale + add-new reconcile idiom: 8 further copies still double-insert on duplicate names (#500 follow-up) #600

Open
opened 2026-07-25 13:03:59 +02:00 by timothy · 0 comments
Owner

Follow-up carved out of #500 (surfaced by that PR's cold review).

What

#500 fixed the duplicate-name double-insert in the two paths its issue named — PlexMovieLibraryScanner.UpdateMetadata and JellyfinMusicVideoLibraryScanner.Reconcile{Genres,Tags,Studios,Artists} — by deduping the incoming set on the same key the add filter compares.

The same materialized-add-filter-then-mutate shape exists verbatim in 8 more places, none of which dedup upstream:

File Collections
ErsatzTV.Scanner/Core/Plex/PlexOtherVideoLibraryScanner.cs genres, studios, actors, directors, writers, guids, tags
ErsatzTV.Scanner/Core/Plex/PlexTelevisionLibraryScanner.cs show / season / episode
ErsatzTV.Scanner/Core/Jellyfin/JellyfinTelevisionLibraryScanner.cs show (actors) + episode (actors/directors/writers) — no season-level loop here; Jellyfin season reconcile lives in JellyfinTelevisionRepository below
ErsatzTV.Scanner/Core/Metadata/LocalMetadataProvider.cs NFO path — genres/styles/moods/artists/directors/writers/guids/tags/actors
ErsatzTV.Infrastructure/Data/Repositories/JellyfinMovieRepository.cs metadata collections
ErsatzTV.Infrastructure/Data/Repositories/JellyfinTelevisionRepository.cs metadata collections
ErsatzTV.Infrastructure/Data/Repositories/EmbyMovieRepository.cs metadata collections
ErsatzTV.Infrastructure/Data/Repositories/EmbyTelevisionRepository.cs metadata collections

The four repository copies add straight into EF-tracked collections, so a duplicate becomes a row on SaveChanges exactly as in the scanner copies.

Why it matters

Same low likelihood as #500 (a media server emitting two identically-named genres/tags/studios for one item is unusual, and no occurrence has been observed) — but the class of bug now survives in the majority of its instances, and the next copy-paste of the idiom will be from an unfixed copy. This is the "verify a shared primitive covers ALL variants" trap.

Fix direction

Prefer extracting the idiom into one shared helper over an 8-way copy of DistinctBy, so the dedup rule cannot be dropped by the next mirrorer. Note the per-collection key differs: Name for most, Guid for guids, and Plex Actors need an artwork-preferring dedup shared with the remove filter — a bare DistinctBy(a => a.Name) there keeps the first duplicate and can permanently discard actor artwork (see the #500 review; that scanner's remove filter is keyed on (Name, artwork-presence), which is the mechanism that upgrades an artwork-less actor). Any Actors copy in the list above has the same hazard.

Also worth folding in: JellyfinMusicVideoLibraryScanner.ReconcileTags lacks the .Filter(g => g.ExternalCollectionId is null) guard its Plex/Jellyfin-movie counterparts have. Harmless today (JellyfinCollectionRepository never tags MusicVideoMetadata) but latent if music-video collection support is added.

Recorded under key: scan.musicvideo-reconciliation Mechanics in docs/decisions.md.

Done-when

  • Every remaining copy of the idiom cannot double-insert a duplicate-named incoming entry
  • Actors copies use the artwork-preferring dedup (no silent artwork/Role loss)
  • Preferably one shared helper rather than 8 inline dedups; tests cover at least the shared seam + one Actors case, non-vacuous
  • Adversarial review passed

Ref: #500, #497.

Follow-up carved out of #500 (surfaced by that PR's cold review). ## What #500 fixed the duplicate-name double-insert in the two paths its issue named — `PlexMovieLibraryScanner.UpdateMetadata` and `JellyfinMusicVideoLibraryScanner.Reconcile{Genres,Tags,Studios,Artists}` — by deduping the incoming set on the same key the add filter compares. The **same** materialized-add-filter-then-mutate shape exists verbatim in 8 more places, none of which dedup upstream: | File | Collections | |---|---| | `ErsatzTV.Scanner/Core/Plex/PlexOtherVideoLibraryScanner.cs` | genres, studios, actors, directors, writers, guids, tags | | `ErsatzTV.Scanner/Core/Plex/PlexTelevisionLibraryScanner.cs` | show / season / episode | | `ErsatzTV.Scanner/Core/Jellyfin/JellyfinTelevisionLibraryScanner.cs` | show (actors) + episode (actors/directors/writers) — **no season-level loop here**; Jellyfin season reconcile lives in `JellyfinTelevisionRepository` below | | `ErsatzTV.Scanner/Core/Metadata/LocalMetadataProvider.cs` | NFO path — genres/styles/moods/artists/directors/writers/guids/tags/actors | | `ErsatzTV.Infrastructure/Data/Repositories/JellyfinMovieRepository.cs` | metadata collections | | `ErsatzTV.Infrastructure/Data/Repositories/JellyfinTelevisionRepository.cs` | metadata collections | | `ErsatzTV.Infrastructure/Data/Repositories/EmbyMovieRepository.cs` | metadata collections | | `ErsatzTV.Infrastructure/Data/Repositories/EmbyTelevisionRepository.cs` | metadata collections | The four repository copies add straight into EF-tracked collections, so a duplicate becomes a row on `SaveChanges` exactly as in the scanner copies. ## Why it matters Same low likelihood as #500 (a media server emitting two identically-named genres/tags/studios for one item is unusual, and no occurrence has been observed) — but the class of bug now survives in the **majority** of its instances, and the next copy-paste of the idiom will be from an unfixed copy. This is the "verify a shared primitive covers ALL variants" trap. ## Fix direction Prefer extracting the idiom into one shared helper over an 8-way copy of `DistinctBy`, so the dedup rule cannot be dropped by the next mirrorer. Note the per-collection key differs: `Name` for most, `Guid` for guids, and **Plex `Actors` need an artwork-preferring dedup shared with the remove filter** — a bare `DistinctBy(a => a.Name)` there keeps the first duplicate and can permanently discard actor artwork (see the #500 review; that scanner's remove filter is keyed on `(Name, artwork-presence)`, which is the mechanism that upgrades an artwork-less actor). Any `Actors` copy in the list above has the same hazard. Also worth folding in: `JellyfinMusicVideoLibraryScanner.ReconcileTags` lacks the `.Filter(g => g.ExternalCollectionId is null)` guard its Plex/Jellyfin-movie counterparts have. Harmless today (`JellyfinCollectionRepository` never tags `MusicVideoMetadata`) but latent if music-video collection support is added. Recorded under `key: scan.musicvideo-reconciliation` Mechanics in `docs/decisions.md`. ## Done-when - [ ] Every remaining copy of the idiom cannot double-insert a duplicate-named incoming entry - [ ] Actors copies use the artwork-preferring dedup (no silent artwork/Role loss) - [ ] Preferably one shared helper rather than 8 inline dedups; tests cover at least the shared seam + one Actors case, non-vacuous - [ ] Adversarial review passed Ref: #500, #497.
timothy added the bugpriority: low labels 2026-07-25 13:03:59 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: timothy/ersatztv#600