Commit Graph
327 Commits
Author SHA1 Message Date
timothyandtimothy 08cd3a002d fix(690,758): count the same query a paged handler pages (#833)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 14s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m53s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m18s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m40s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m46s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 07:39:23 +00:00
timothyandClaude Opus 5 dd7b58232c fix(691): revert entity-level null guard, guard read sites instead
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 59s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m54s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m4s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 11s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 8s
review-verdict/h10 Review-verdict: MERGEABLE @ dd7b582 (base: main)
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 22m12s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Review verdict / Set review-verdict status (pull_request) Successful in 5s
PR Gates / decisions lifecycle (pull_request) Successful in 13s
PR Gates / Script tests (pytest) (pull_request) Successful in 46s
The prior commit (a4700185b) made SongMetadata.Artists/AlbumArtists
coalesce null to [] via backing-field getters, reasoning that EF
Core's PreferField access mode never observes the getter. Adversarial
review disproved this on real TvContext/SQLite: a single read of
.Artists on a TRACKED entity mutates the backing field through the
getter, flips the entity to Modified, and the next SaveChanges writes
[] over what was a NULL column -- silent data loss waiting on the
first tracked reader (today all readers happen to be AsNoTracking).

This also reversed docs/decisions/records/api/selection-projection-include-chain.md
(#671) without the doc update CLAUDE.md requires; #691 is that
record's own "sweep by FIELD" follow-up, so it should follow the
record, not contradict it.

Revert SongMetadata.cs to plain auto-properties (byte-identical to
origin/main, BOM still stripped per the #311 gate). Guard the read
sites instead, per the #671 convention (Optional(...).Flatten(),
matching Playouts/Mapper.cs and MediaItems/Mapper.cs):

- SongVideoGenerator.cs: hoist `artists`/`albumArtists` locals once
  near the top of the metadata loop instead of repeating the guard at
  each of the six former call sites.
- MediaCollectionRepository.cs (GroupIntoFakeCollections): guard the
  two AlbumArtists reads at lines ~1147/~1160 that #691 never named --
  dropping the entity-level fix without these would trade one bug for
  two.

Verified RED per guard by removing only the Optional(...).Flatten()
clause (not the whole file): the artists local throws
ArgumentNullException at SongVideoGenerator.cs:88, the albumArtists
local at :89 (List.ToList() on a null IList<string> source -- same
loaded-gun shape the review demonstrated, precise exception type is
ArgumentNullException rather than NullReferenceException since the
throw site is Enumerable.ToList's null-source check). Restored both;
existing SongVideoGeneratorTests still pass. Full ErsatzTV.Core.Tests:
685 passed (1 pre-existing skip), ErsatzTV.Tests: 1996 passed (4
pre-existing skips), 0 failures in each. No EF model drift
(`dotnet ef migrations has-pending-model-changes` reports none).
`dotnet format --verify-no-changes` on the three touched files exits
0.

Refs #691

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 22:39:52 +02:00
timothy 83cd36e0de test(491): run the dedupe fixture against MySql in CI; correct the collation claim
The dedupe DML had zero automated coverage on MySql: the migrations job only
applies migrations to a fresh EMPTY database, so no dedupe row ever executed
there. Two MySql-only collation defects escaped that gate in this session and
were caught only by hand-run containers.

Parameterize LibraryFolderDedupeMigrationTests over both providers from ONE
fixture body - same seeded rows, same expected survivors - rather than adding a
MySql-only copy that would drift and recreate the gap. Assertions no longer use
WHERE Path = '...', which is itself collation-dependent and would quietly mean
something different per provider; rows are read once and compared ordinally in
memory. A new step in the existing migrations job runs it against that job's
mysql:8.4 service, on a per-test database of its own.

Proven red when the collation is wrong: restoring COLLATE utf8mb4_bin fails the
MySql half with survivors [1,4,5,6,7,9] - the trailing-space sibling deleted -
while SQLite stays green. Proven non-skippable: without
ETV_TEST_MYSQL_CONNECTION the fixture ignores visibly, and with
ETV_REQUIRE_MYSQL_TESTS=1 (which CI sets) that skip becomes a hard failure, so
it cannot pass having connected to nothing. Local runs need no MySql.

Also correct an overstated comment. The schema pins only the utf8mb4 charset,
never a collation, so the effective comparison is the server default: always
case-insensitive, but PAD SPACE only on utf8mb4_general_ci - 8.4's default
utf8mb4_0900_ai_ci is NO PAD, verified on the real column. The migration bug was
independent of that because the old code applied an EXPLICIT utf8mb4_bin, which
is PAD SPACE everywhere; the runtime simply tolerates both.

Refs #488 #308
fix #491
2026-07-25 21:13:31 +02:00
timothy 48d41b9235 fix(491): make the MySql dedupe byte-exact, not just case-exact (PAD SPACE)
Cross-family review of 1b4dd6d6 found that utf8mb4_bin - chosen to keep the
dedupe case-exact - is a PAD SPACE collation, so trailing spaces are
insignificant under it. Verified on MySQL 8.4: '/media/Foo' = '/media/Foo ' is
TRUE, while case correctly compares unequal. Two distinct legal directories
therefore grouped together and the second was DELETED irreversibly, even though
PathUtils.GetPathHash hashes them differently and the unique index about to be
created would have accepted both. The dedupe destroyed data the constraint
never required it to destroy.

Group and join on CONVERT(Path USING binary) instead - NO PAD and byte-exact,
matching the hash. utf8mb4_0900_bin is also NO PAD but carries a server-version
floor. This is the only path comparison in either migration (every other
predicate keys off an integer id), so there is no mix of padded and unpadded
comparisons across the keeper-selection, repoint and delete steps.

SQLite's = on TEXT is byte-exact with no padding, so that migration was already
correct - which is exactly why a SQLite-only test could not see the divergence.
The two providers are now semantically equivalent, and the dedupe fixture is
shared: same rows, same expected survivors (1,4,5,6,7,9,10), asserted by the
SQLite test and reproduced by hand on MySQL 8.4.

Runtime was never affected, and this is now stated and tested rather than
assumed: GetFolder's SQL equality is a superset narrowing (both collation quirks
make it more permissive, never less, so it cannot miss a byte-exact match) and
ResolveExact settles identity with StringComparison.Ordinal, which compares
length first. Added ResolveExact coverage for the trailing-space axis.

Refs #488 #308
fix #491
2026-07-25 21:13:31 +02:00
timothy 50eff83628 docs(491): reattach the case-exact lookup rationale to GetFolder
Review follow-up. ResolveExact was inserted between GetFolder's summary
and GetFolder itself, leaving ResolveExact with two summaries and
GetFolder with none -- so the H1 rationale (SQL equality is only a
narrowing filter; OrderBy(Id) for determinism) described neither of the
things ResolveExact does. On a fix that turns on exactly that reasoning,
a misattached explanation is what misleads the next reader.
2026-07-25 21:13:31 +02:00
timothy ecb763ea58 fix(491): review polish — heal cannot abort a scan, ordinal settle unit-tested
Final low-severity items from the re-review of ee10f932.

L2: DbUpdateConcurrencyException derives from DbUpdateException but carries no
provider exception, so IsUniqueConstraintViolation does not classify it. A row
deleted by a concurrent library edit between the heal's read and its save would
propagate and fail the scan, contradicting the invariant stated directly above
it. Admit it in the filter.

L1: lift the in-memory ordinal settle into LibraryRepository.ResolveExact and
unit-test it with both spellings in the candidate list. No SQLite-backed test
can exercise it (SQLite's = on TEXT is already binary), so this converts the
half that rested on hand-run MySQL evidence into automated coverage. The
end-to-end companion test's comment no longer claims to be provider-independent.

L4: assert PRAGMA foreign_keys is 1 before migrating, so the enforcement guard
cannot silently degrade into the weak pre-fix form it was added to replace.

L3: detach the failed heal, matching the insert path.
N3: the heal's inner predicate now matches its IsNullOrEmpty outer guard, so a
PathHash = '' row cannot enter the branch and silently never heal.
N4: record that GetFolder returning null for a case-differing spelling makes
MySQL insert a second row where it used to reuse one — correct, and now matching
SQLite, but a real behaviour change on a case-insensitive filesystem.

Refs #488 #308
fix #491
2026-07-25 21:13:30 +02:00
timothy 14e9b03433 fix(491): wire the unique-violation classifier in the scanner; make folder lookup case-exact
Review of 491f5099 found the fix inert in the only process that runs it, plus
a MySQL collation defect in the lookup.

B1 — TvContext.IsUniqueConstraintViolation was assigned only in ErsatzTV/
Startup.cs, but ErsatzTV.Scanner is a separate executable and every production
caller of GetOrAddFolder/SetEtag lives there. The classifier kept its '_ =>
false' default, so the catch never ran and the DbUpdateException failed the
whole scan - worse than the duplicate row it replaced. Wire both provider
branches in ErsatzTV.Scanner/Program.cs, and add ProviderStaticsWiringTests
(architecture) asserting the scanner assigns every TvContext static the host
assigns, with IsSqlite documented as the one exemption.

H1 — GetFolder's 'Path == folder' is case-insensitive on MySQL while PathHash
is case-sensitive, and FirstOrDefault was unordered: a scan of '/x/foo' could
resolve the '/x/Foo' row and stamp the wrong hash onto it (verified on MySQL
8.4: the WHERE matches both, LIMIT 1 returns the wrong one). Treat the SQL
equality as a narrowing filter, order by Id, and settle identity ordinally.
Route the heal through EF and drop a classified violation, so an opportunistic
maintenance write can never abort a scan.

Also: run the dedupe migration test with foreign keys ON (matching prod), clear
the keeper's etag, null out a self-parent, and document the cleanup's limits
(NULL paths excluded, Down does not restore deleted rows, CI's fresh-DB apply
covers none of the data mutation).

Refs #488 #308
fix #491
2026-07-25 21:13:30 +02:00
timothy 9a4f3e832d fix(491): unique index on LibraryFolder(LibraryPathId, PathHash) + tolerate concurrent insert
GetOrAddFolder was a check-then-insert with no unique constraint behind it,
so two callers racing the same folder could both miss the lookup and both
insert. Enforce identity in the schema and make the loser adopt the winner.

- LibraryFolder gains a SHA-256 PathHash (the MediaFile.Path/PathHash
  precedent): Path is MySQL longtext, which cannot be indexed without a
  prefix length and collates case-insensitively, so the unique index is on
  (LibraryPathId, PathHash) instead.
- GetOrAddFolder and SetEtag catch a classified unique violation via the
  existing TvContext.IsUniqueConstraintViolation seam (#308) and re-read.
- Dual-provider migration audits and collapses pre-existing duplicates
  (repointing MediaFile, ParentId and ImageFolderDuration) before creating
  the index; legacy rows keep a null hash and heal on the next scan.
- Tests: deterministic cross-connection race, 8x10 barrier stress with an
  insert-attempt vacuity guard, classifier-inversion negative control, and
  a real-migration dedupe test.

Refs #488 #308
fix #491
2026-07-25 21:13:22 +02:00
timothy 147166b053 fix(496): harden the legacy path sweep against MySQL collation (re-review Low)
Re-review returned MERGEABLE with one Low, MySQL-collation-dependent edge in
FlagFileNotFoundByPaths: the C#-side Except diff is ordinal, but `MF.Path IN @LocalPaths`
runs under MySQL's case-insensitive default collation, so a still-reported identified row
differing only in case from an absent legacy row could be matched and flagged missing.

Match on the indexed PathHash instead of collated Path text, and re-state the
NOT EXISTS (JellyfinMusicVideo) guard so the identity pass and the legacy pass are disjoint
by construction rather than by the caller's diff being correct. SQLite was unaffected.
2026-07-25 17:20:53 +02:00
timothy 6bd1d954bd fix(496): review fixes — thread the replaced local path, scope identity per library, sweep legacy rows [decisions-edit]
Independent cold review (Codex) returned BLOCKED. Findings 1, 3 and 4 are fixed here;
each has a regression test proven non-vacuous by a negative control.

1. Blocker — the replaced local path was discarded. The scanner computed localPath but
   GetOrAdd only received `incoming`, so the repository re-derived the path from the
   UNREPLACED projection. On any install with path replacements, adoption hashed the
   server-side path, missed the existing row, ALSO slipped past MediaFileAlreadyExists
   (which hashes that same wrong string) and inserted a duplicate row under a server path,
   leaving the original collection-linked row identity-less forever. The test harness hid
   this because its path-replacement stub was an identity function.
   → GetOrAdd now takes localPath explicitly and never reads the projection's path;
     BuildPathReplacement takes a real mapping and the new test genuinely replaces.

3. Medium — GetByItemId matched on ItemId alone, so two media sources presenting the same
   item id (cloned Jellyfin DB) resolved to each other's row, letting one library repoint
   another's. → filtered by LibraryPath.LibraryId.

4. Medium — a row predating the identity that the server had ALREADY stopped reporting was
   never adopted (adoption only runs for an incoming item) and carried no identity, so the
   itemId diff could not see it either: it sat Normal and schedulable forever, strictly
   worse than the hard delete it replaced. → GetExistingLegacyMusicVideoPaths +
   FlagFileNotFoundByPaths reconcile legacy rows by local path, and they are counted into
   the #477 empty-fetch guard (on the first scan after this ships they ARE the whole
   library, so a guard counting only identity rows would sweep all of them on a transient
   empty fetch).

Finding 2 (the issue's Done-when #2) is a scope question, not a defect, and is unchanged:
one file path is still one MediaItem row globally, so this lands music videos at parity
with movies rather than eliminating shared-row trashing. Recorded honestly in the decision
record; raised for an explicit call before the issue is closed.

fixes #496
2026-07-25 17:20:53 +02:00
timothy 64decd492e fix(496): give music videos a per-library server identity; itemId diff + soft trash
Music videos carried no server identity, so JellyfinMusicVideoLibraryScanner had to
reconcile by a (LibraryPathId, path) diff and HARD-delete the remainder. A file served
by two libraries with overlapping local paths is a single row owned by whichever library
scanned it first, so that owner's sweep destroyed a row another library still served —
taking collection membership and playout references with it, irreversibly.

This is #494's deferred "option 2":

- New JellyfinMusicVideo : MusicVideo (ItemId/Etag), mirroring JellyfinMovie — TPT table,
  varchar(36), ItemId index. Dual-provider migration Add_JellyfinMusicVideo.
- New IMediaServerMusicVideoRepository + JellyfinMusicVideoRepository: itemId-keyed
  existing-set/lookup and Flag{Normal,Unavailable,FileNotFound} seams, all scoped per
  library via LibraryPath.LibraryId.
- New MediaServerMusicVideoLibraryScanner base; JellyfinMusicVideoLibraryScanner folds
  onto it and keeps the #177/#488/#497/#500 metadata-reconcile logic verbatim.
- The sweep now soft-trashes (FileNotFound) instead of deleting, so removal is reversible
  and EmptyTrash-governed. DeleteEmptyArtists consequently no longer fires from a sweep.
- Pre-identity rows are ADOPTED in place: the identity row is inserted against the same
  MediaItem id, scoped to the scanned library's own LibraryPath, so collection membership
  survives and a local/second-library row is never hijacked.
- AddMusicVideo normalizes Path/PathHash to the path-REPLACED local path; the projection
  fills them from the server-reported path, which would break every later PathHash lookup.

Docs: scan.musicvideo-reconciliation relocated to docs/decisions/archive/scan.md as
superseded; new active record scan.musicvideo-server-identity.

fixes #496
2026-07-25 17:20:53 +02:00
timothy 890745d1d4 fix(460): write null LastScan on disable-sync instead of the MinValue sentinel [decisions-edit]
MediaSourceRepository's Plex/Jellyfin/Emby remove-and-recreate (disable-sync)
flows stamped SystemTime.MinValueUtc into library.LastScan, so normal use kept
minting 0001-01-01 sentinel rows. #409 fixed the READ side (the API coerces the
sentinel to null and a migration cleaned the historical residue), so the wire
contract was already correct — this stops new sentinel rows being written.

Safe because every remaining LastScan reader either coalesces
(LastScan ?? SystemTime.MinValueUtc) for its own non-nullable scan-comparison
needs, or is the API read-boundary coercion itself — audited every reference
under ErsatzTV{,.Application,.Core,.Infrastructure,.Scanner,.Mcp} plus web/.
There is no Where/OrderBy/GroupBy on LastScan anywhere, so the SQL
null-ordering divergence between SQLite and MySQL has no surface here.
Scan-comparison behavior is unchanged.

Deliberately ships NO second cleanup migration: rows written between #409's
NullOutNeverScannedLastScan and this change keep the sentinel at rest, and the
permanent read coercion — not a migration — is what keeps the contract honest
for them (as it must be anyway for a restored or hand-edited DB).
LibraryPath.LastScan is likewise left untouched: the flows re-add Paths with
their original values, and its only readers are the local-library scan
handlers, so it has no API surface and no remote-scan effect.

Regression test per provider (MediaSourceRepositoryDisableSyncTests), proven
non-vacuous: restoring the MinValue writes fails all three.

[decisions-edit] — the media.lastscan-null-boundary record documented this
write as an ONGOING sentinel source and rested its "the coercion is permanent"
argument on it, so the rationale prose is corrected in the same PR per
docs.decision-lifecycle. The Rule line is unchanged, so the generated
decisions/README.md catalog is byte-identical.

MediaSourceRepository.cs also loses its UTF-8 BOM (fix-as-you-touch, #311).

fixes #460
2026-07-25 13:46:09 +02:00
timothyandtimothy 65c0e09179 feat(415): per-channel fault detection — server-derived health object + Problems filter (#581)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 15m30s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 15m48s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 7m32s
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
Closes #415. Server-derived health object on the channel list + detail DTOs (built-timeline detection, kind-agnostic across all 5 PlayoutScheduleKind; assessable gate keyed to the owning channel's mode), single "Problems" SPA filter with per-fault badges. Supersedes #72's api.channel-health-signal decision.

Co-authored-by: Timothy <timothy.look@gmail.com>
Co-committed-by: Timothy <timothy.look@gmail.com>
2026-07-23 20:45:19 +00:00
timothy 7757814766 feat(74): channel graphicsElementIds + graphics builtIn; regen OpenAPI 2026-07-22 22:11:46 +02:00
timothyandClaude Opus 4.8 d3db2f6af1 fix(488): resolve LibraryFolder from DB so Jellyfin music-video scans stop NRE'ing
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 8s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 39s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m21s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m18s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m47s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m21s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
GetOrAddFolder read libraryPath.LibraryFolders, a navigation collection only
eager-loaded on the local scan path (via GetLibrary). The Jellyfin remote sync
path takes its LibraryPath off the JellyfinLibrary entity, where LibraryFolders
is null, so .Filter(null) threw ArgumentNullException('source') on the first
item of every Jellyfin music-video scan — a feature that had therefore never
run in prod, CI, or locally.

Look the folder up from the DB by (LibraryPathId, Path) instead, removing the
implicit eager-load contract entirely (correct for all nine callers) and
documenting it on ILibraryRepository. null != empty is preserved so a re-scan
does not insert duplicate LibraryFolder rows. No new hot-path cost: local
scanners already query GetParentFolderId per folder just before this call.

Tests:
- LibraryRepositoryTests: GetOrAddFolder with a null LibraryFolders (the exact
  remote-path shape) creates the folder, is idempotent on re-scan, and persists
  a supplied ParentId.
- JellyfinMusicVideoLibraryScannerTests: an end-to-end scan of one synthetic
  music video, wiring the REAL LibraryRepository/ArtistRepository/
  MusicVideoRepository against in-memory SQLite (the existing MediaServer*
  scanner tests mock every repo, which is why the bug escaped), asserts the scan
  completes and creates Artist + MusicVideo rows with a real LibraryFolder.

Both proven non-vacuous against the reverted fix (each reproduces the issue's
ArgumentNullException). decisions.md entry added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 01:16:16 +02:00
timothyandClaude Opus 4.8 61bd374494 fix(476): cascade FileNotFound from removed shows/seasons to descendants
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 7s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 24s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 5m52s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 5m36s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 9s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m14s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The media-server television scanner reconciles removed items at three levels
via existing.Except(incoming), but the season and episode sweeps live inside
the per-parent loops (ScanSeasons inside the show loop, ScanEpisodes inside the
season loop). Those loops only iterate parents present in the incoming list, so
a show (or season) that is gone from the media server is never visited and its
descendants are never swept — they keep their last state. On Jellyfin/Emby that
is RemoteOnly, which PlayoutBuilder does NOT skip even with PlayoutSkipMissingItems
on, so every orphaned episode keeps getting scheduled as a guaranteed tune-in
failure (the #473 reproduction; 717 stale prod rows across 10 removed shows).

Fix: cascade the flag by parent MediaItem.Id. Two provider-agnostic repo helpers
(Season.ShowId / Episode.SeasonId are on the base tables) flag descendants and the
scanner drives them after each parent sweep — show → seasons → episodes, and
season → episodes for the show-present case. Shared abstract base fixes
Jellyfin/Plex/Emby at once; Movie/OtherVideo are flat and have no such gap.

Tests: a Scanner.Tests case asserts the scanner cascades through the (substituted)
repository (non-vacuous — fails if the cascade calls are removed), and Integration
tests exercise the real cascade SQL against the schema, proving it flags only the
targeted subtree and no-ops on empty input.

fixes #476

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-19 21:49:33 +02:00
0295a8a9f6 style(70): strip UTF-8 BOM from the .cs files this PR touches
CI's Formatting job failed: 19 touched files carried a BOM, which .editorconfig
forbids (charset=utf-8). Pure encoding change — one byte per file, no semantic
diff (verified: every hunk is `-namespace` -> `+namespace`).

Self-inflicted. The patches that edited these legacy files wrote them back as
utf-8-sig to "preserve the existing style", but the #311 fix-as-you-touch gate
requires a file to be normalized when you touch it — that is the whole point of
scoping the gate to changed files instead of reformatting the ~2500 legacy BOM
files at once. dotnet format leaves the EF-generated Designer/snapshot files
alone as generated code, and its verify skips them the same way, so they stay as
ef emitted them.

Two corrections to what I believed going in:
- `dotnet format --include` does NOT no-op here. It reported `error CHARSET` for
  each file and exit 2, reproducing CI exactly, and fixed them in place. The note
  claiming otherwise is wrong for this invocation.
- My first BOM check reported all files clean. The od pattern was wrong; reading
  the first three bytes directly found 19. A detector that can only say "ok" is
  worse than no detector.

Core.Tests 565, ErsatzTV.Tests 1673, Architecture.Tests 5 — all passed. API
artifacts still in sync.

Refs #70

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 17:00:46 +00:00
68be319403 feat(70): WeightedShuffle playback order — weighted / fair-share distribution
Adds PlaybackOrder.WeightedShuffle = 9 (Classic engine only) plus the
per-source weight it distributes by.

Why a new order rather than making ShuffleInOrder weight-aware: ShuffleInOrder's
balanced shuffle pads sources to equal length with Option.None spacers, and
spacers emit nothing — so one cycle plays every item exactly once and airtime
stays proportional to collection size. It prevents *clumping*, not *domination*.
Retrofitting weights onto it would silently change shipped users' output.

WeightedShuffleCollectionEnumerator picks a source by smooth weighted
round-robin (acc += weight; richest wins; pays the total), then takes that
source's next item. Weights 3:1 emit A A B A; ties break to the earliest source
in list order. Fair-share is the equal-weights default, so one mechanism covers
both behaviors in the issue.

One rotation is sized so the source needing the most picks works through all its
items at its share; smaller sources loop within it — that looping is what makes
equal weights mean equal airtime regardless of library size. The sequence stays a
pure function of (Seed, Index), so it restores by replay like its siblings and
needs no per-source persisted counters.

It consumes ShuffleSourceBuilder.GetCollectionItemsForShuffleInOrder unchanged —
the schedule-entity-free entry point #380 reserved for this issue. Source
identity is destroyed on the Shuffle path (MultiCollectionGrouper collapses to
GroupedMediaItem + Distinct), so the weight is only reachable via
CollectionWithItems on the ShuffleInOrder-shaped path.

Weight is persisted on MultiCollectionItem AND MultiCollectionSmartItem — the
mirror is mandatory; omitting the smart item silently un-weights smart-collection
members. Both carry a DB default of 1: without it existing rows would migrate to
0, and a 0-weight source is dropped by the enumerator.

Refs #70

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 17:00:46 +00:00
timothy 1e35248edf style(72): de-BOM the six touched .cs files (#311 formatting gate)
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 5s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 4s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 5s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 5m11s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 4m49s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 14m47s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 18m36s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 36m51s
CI's "Formatting (changed .cs conform to .editorconfig)" job failed with
`error CHARSET: Fix file encoding` on all six pre-existing BOM'd files this PR
touches. The #311 gate is fix-as-you-touch: any .cs a PR touches must conform to
.editorconfig (charset=utf-8), and these carried BOMs inherited from upstream.

BOM removal only — six files, one line each, zero content change. The three
files already without a BOM (GetChannelByIdForApiHandler + both new/changed
Tests files) needed nothing.

Note for the next person: this cannot be validated locally on this Mac —
`dotnet format --include` silently no-ops here, so the gate is only observable
in CI. Check `head -c3 <file> | xxd -p` for `efbbbf` on every touched .cs before
pushing instead of trusting a local format run.

Refs #72
2026-07-17 17:51:43 +02:00
timothy 1a40b4a8e6 fix(72): report a real per-channel playout count on the channels API
The channels API could not answer "will this channel play?", which #72 needs
to flag a broken channel in the lineup at a glance.

Two defects, one root cause each:

1. `ChannelRepository.GetChannel` never included `Playouts`. The read is
   AsNoTracking with no lazy-loading proxies, so the navigation came back
   empty and `GetChannelByIdForApiHandler`'s `channel.Playouts?.Count ?? 0`
   could only ever evaluate to 0 — `GET /api/v1/channels/{id}` reported
   `playoutCount: 0` for every channel on the system. That silently disabled
   the channel editor's playout-source guard (ChannelEditScreen:820, gated on
   `playoutCount > 0`), so the "Cannot be changed once a generated channel has
   a playout" control was always live. The server still enforces the invariant
   (UpdateChannelHandler coerces Mirror back to Generated), so nothing was
   corrupted — but the user's change was silently discarded. That silent
   coercion is filed separately as #401.

2. The detail path counted only the channel's own playouts, never the mirror
   source's, so a working Mirror channel would read as "no playout" even once
   the include landed.

Both call sites now share `Mapper.GetPlayoutsCount` (previously private to
GetAllChannelsHandler), which handles the Mirror case. `ChannelResponseModel`
gains `PlayoutCount` so the list — #72's actual surface — can render it; the
count is free there, since `GetAll` already includes `Playouts` and
`MirrorSourceChannel.Playouts` and simply discarded them.

Tests run the real repository against a real context on purpose: a handler
test with a substituted IChannelRepository populates `Playouts` itself, so it
passes whether or not the query includes them. Proven non-vacuous — removing
the include again turns the 2-playout and mirror cases red (0 CS errors, so
no stale-dll false pass).

Refs #72
2026-07-17 17:51:26 +02:00
timothyandClaude Opus 4.8 5f89bfc1a0 feat(api): #288/#197 wrap ChannelViewModel + re-key playout/reset to {id}
GetById/Create/Update return ChannelResponseModel via new GetChannelByIdForApi
read-side query; POST /api/channels/{id:int}/playout/reset (new
GetPlayoutIdByChannelId; by-number kept for HlsSessionWorker broadcast).

Refs #288 #197

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 02:08:12 +02:00
timothy ee104164db Merge branch 'feat/202-s3-jfemby' into feat/202-media-sources 2026-07-11 15:48:07 +02:00
timothyandClaude Opus 4.8 c617a01e83 feat(api): add Jellyfin/Emby media-source write API (#202 slice S3)
New JellyfinMediaSourcesController (/api/media-sources/jellyfin, J1-J9) and
EmbyMediaSourcesController (/api/media-sources/emby, E1-E9), wrapping the
existing Jellyfin/Emby MediatR commands per the #202 design doc §A.3/§A.4.

Secure connection contract (§C3/§B, finding 1): the connection GET returns
only { address, hasApiKey } — the API key never crosses the wire. The PUT
retains the existing key when the incoming key is blank, sets a new one when
non-blank, and 422s "API key is required" on a blank first connect.

Finding 7 (lock-release discipline): DisconnectJellyfinHandler and
DisconnectEmbyHandler now wrap their work in try/finally so a throw from any
awaited dependency (repo delete, search-index commit, secret store) still
releases the family lock instead of wedging every future disconnect at 409.

Findings 2c/8 (path-replacement cross-source guard): UpdateJellyfinPathReplacementsHandler
and UpdateEmbyPathReplacementsHandler now reject, before any write, an incoming
positive Id that isn't owned by the route's media source, a null item, or a
blank RemotePath/LocalPath — all 422 with no partial mutation. Defense-in-depth
repo fix: the Jellyfin/Emby path-replacement UPDATE SQL in MediaSourceRepository
now scopes by {Jellyfin,Emby}MediaSourceId (was previously unscoped by Id alone,
allowing a PUT to one source to silently overwrite another source's row). The
Plex path-replacement method (~line 397) is untouched — that's slice S2's file.

Library preferences (§C4a): the controller validates the incoming id set
against the source's known libraries (reject foreign ids, require full
coverage, no Id=0) before dispatch, then — for §C7 — LockLibrary + enqueues
the SynchronizeXLibraries/SynchronizeXLibraryByIdIfNeeded pair per enabled
library (compensating unlock if the enqueue throws), and returns the reloaded
list (ids are not stable across a disable).

404s on id-taking endpoints come from a controller pre-check (GetXMediaSourceById
is None), not a handler NotFoundError, since Either.Apply/ToEitherAsync join any
NotFoundError into a flat 422 (finding 9).

Tests: controller route/404/409/422 tests for both families; disconnect
fault-injection tests proving the lock releases even when a dependency throws;
path-replacement handler tests for cross-source-id/blank/null-item rejection
and correct add/update/delete merge; a repository-level test proving the SQL
fix stops a same-family cross-source path-replacement overwrite.

No new commands, no DB migration, no OpenAPI regen (gated until S1-S3 merge
per the design doc's build-slice plan).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:47:42 +02:00
timothy 26205b8ccd Merge branch 'feat/202-s1-local' into feat/202-media-sources 2026-07-11 15:43:38 +02:00
timothy f5cf23c952 feat(api): local libraries REST endpoints (#202 slice S1)
Adds LocalLibrariesController (L1-L7: list/get/create/update/delete/move-path/
path-exists) wrapping the existing local-library MediatR commands, mapping to
the shared S0 response DTOs. Per design #202 §A.1/§C5/§C6:

- 404 for L4/L5/L6 comes from a controller pre-check (GetLocalLibraryById is
  None), not the handler -- .Apply/.ToEitherAsync both .Join() a NotFoundError
  into a plain 422, so relying on the handler would be dead code. This is
  check-then-act; a delete racing the pre-check falls through to the handler's
  422, documented in the controller.
- L4/L5 409 via IEntityLocker.IsLibraryLocked(id); L6 resolves the source
  library from the path id (new ILibraryRepository.GetLibraryIdForPath) before
  its own lock check.
- MoveLocalLibraryPathHandler gains same-MediaKind and different-library
  validation (finding 3) -- Blazor only filtered these client-side in the move
  dialog, so an API/MCP client could bypass them.
- CreateLocalLibraryHandler/UpdateLocalLibraryHandler gain a shared
  NewPathsMustExist validation (LocalLibraryHandlerBase) that Directory.Exists-
  checks only new paths (Id < 1); existing rows stay exempt so an unmounted
  share doesn't block a rename. L7 (path-exists) is a controller-local
  IFileSystem check with no command.

Tests: controller route/mediator-arg tests incl. 404-pre-check vs
fall-through-422 and 409-lock cases; handler tests for the move-path
cross-kind/same-library 422s, new-path 422 (missing/mixed), and a lossless
round-trip proving local paths are identified by normalized path string, not
id. Full solution test suite (Scanner/Core/Architecture/Tests/Infrastructure)
green, 0 regressions.

Deviations: none from the S1 slice description. Did not touch
MediaSourceRepository.cs or any Plex/Jellyfin/Emby file (S2/S3 scope). Did not
run the OpenAPI regen scripts (separate gate after S1-S3 merge per design §E).
2026-07-11 15:43:10 +02:00
timothy 4852c36268 feat(api): Plex media-source write API + lock-lifecycle fixes (#202 slice S2)
New PlexMediaSourcesController (/api/media-sources/plex) P1-P8 wrapping
existing MediatR commands: state GET, pin-flow, sign-out, per-server
libraries/path-replacements GET+PUT, and refresh — VMs projected to the
S0 shared DTOs, ApiResults mapping, 404 controller pre-checks, #215-style
409 lock guards, [EndpointGroupName("general")].

Lock-lifecycle hardening (the tricky part):
- TryCompletePlexPinFlowHandler now releases the Plex lock ONLY on its
  non-handoff exits (timeout-throw, poll exception, enqueue exception, the
  dead return-false) via try/catch — NOT an unconditional finally. On
  success the lock is handed off to SynchronizePlexMediaSources (the sole
  releaser after discovery); a finally would double-release and release
  before discovery, re-opening the finding-5 poll race. Fixes the latent
  leak where an abandoned pin flow wedged Plex locked until restart.
- StartPlexPinFlow controller compensates UnlockPlex on the Left branch AND
  any thrown dispatch/enqueue; only the Right/200 path holds the lock.
- SignOutOfPlexHandler wraps its work in try/finally { UnlockPlex() } — a
  terminal handler with no handoff, so unconditional release is correct.
- Post-save library sync enqueues SynchronizePlexLibraryByIdIfNeeded
  (Unlock:false) then SynchronizePlexNetworks (Unlock:true) — one lock, one
  release on the last message, compensating-unlock if the 2nd enqueue throws
  (corrects the Blazor Unlock-ordering bug, finding 6).

Data-integrity hardening:
- UpdatePlexPathReplacementsHandler rejects (422, no mutation) any positive
  Id not owned by the route source, blank RemotePath/LocalPath, and null
  list/items (findings 2c/8).
- MediaSourceRepository Plex path-replacement UPDATE gains
  AND PlexMediaSourceId = @id (Jellyfin/Emby untouched — slice S3).
- ReplaceLibraryPreferences controller validates the id set against the
  source's libraries (rejects unowned + Id=0), returns the reloaded list
  (ids change on disable).

Tests (NUnit/Shouldly/NSubstitute), 34 new, all green: pin-flow lock
released on thrown-cancellation/poll-throw/enqueue-throw AND held on
success (no double-release); sign-out finally-release under a throwing
dependency; cross-source/nonblank/null path-replacement 422s; library-prefs
id-not-owned 422; post-save enqueue exact messages + Unlock flags; full
controller route/404/409/422 coverage.

Refs #202
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 15:42:40 +02:00
timothyandClaude Fable 5 8ddf0ae169 fix(api): resolve per-show scan by exact show id, not substring title (fixes #219)
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m11s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 12s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 9m49s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
POST /api/libraries/{id}/scan-show resolved the target show via
GetShowIdByTitle, an EF.Functions.Like "%title%" substring match with
no OrderBy - non-deterministic under duplicate/overlapping titles and
capable of scanning the wrong show. The Blazor UI never had this bug
(it always passed the exact show id); this endpoint shipped days ago
in PR #216 with no external consumers, so the contract break is safe.

BREAKING CHANGE: ScanShowRequest now takes `showId: int` instead of
`showTitle: string`. Replaced ITelevisionRepository.GetShowIdByTitle
with GetShowTitle(libraryId, showId), which also enforces the show
belongs to the given library. LibrariesController.ScanShow now returns
a genuine 404 ProblemDetails (via ApiResults.NotFoundProblem, the
established pre-check pattern from TemplateController.DeleteGroup)
when the show id doesn't exist in that library, then queues
QueueShowScanByLibraryId with the DB-resolved title.

SPA: libraries.ts ScanShowParams.showId replaces showTitle;
MediaDetailScreen.tsx passes show.id. Extended
ApiErrorResponseMetadataTests and OpenApiErrorResponseContractTests
with the new 404 contract for ScanShow. Regenerated v1.json / v1.d.ts
via scripts/update-openapi.sh + npm run generate:api.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 23:28:09 +02:00
Jason DoveandGitHub 0d301df5e8 remove external dependencies (bugsnag, trakt) (#2840)
* remove bugsnag

* remove trakt client id (that will expire)
2026-02-26 10:43:48 -06:00
Jason DoveandGitHub f47134d2d0 log warnings when transcoding speed is potentially insufficient (#2808)
* refactor parsing ffmpeg progress/speed

* log warnings when transcoding speed is potentially insufficient

* dont log progress on hls direct; fix tests
2026-02-03 08:49:07 -06:00
Jason DoveandGitHub e80f687612 add marathon group by director (#2800) 2026-01-26 09:10:01 -06:00
Jason DoveandGitHub f96be8f99f update plex episode metadata during scan (#2795) 2026-01-21 16:59:01 -06:00
Jason DoveandGitHub f217ba185b sync jf and emby library name and type changes (#2784) 2026-01-17 06:14:45 -06:00
Jason DoveandGitHub e925bd6913 sync plex library name changes (#2783)
* sync plex library name changes

* feedback
2026-01-16 19:45:34 -06:00
Jason DoveandGitHub a2d023ee69 local scanner artwork cleanup (#2778)
* move plex artwork removal to its own repository

* clean up old local movie artwork

* clien up old music video/artist artwork

* clean up old remote stream artwork

* clean up old song artwork

* clean up old show artwork; properly update season artwork
2026-01-16 10:23:26 -06:00
Jason DoveandGitHub 36f44f14bb fix other video artwork in xmltv (#2777) 2026-01-15 22:42:16 -06:00
Jason DoveandGitHub aa08ad5765 optimize check for orphaned artwork (#2760) 2026-01-07 16:46:17 -06:00
Jason DoveandGitHub effb96a2c2 alternate schedule and template consistency (#2757)
* refactor classic and block schedules to use same alternate schedule selector

* handle start year and end year

* add migrations for classic and block schedules

* allow editing block template start and end year

* add tests that include years

* add date range editing to classic (alternate) schedules

* fix running tests locally

* restore media files load; needed for local folder scanners

* update changelog

* feedback
2026-01-06 12:51:07 -06:00
Jason DoveandGitHub a96ef328a5 more scanning improvements related to media file table (#2754) 2026-01-04 19:16:06 -06:00
Jason DoveandGitHub 89bb3759de more jf episode scanning improvements (#2753)
* more jf episode scanning improvements

* bump log level on important message

* add transaction
2026-01-03 12:43:26 -06:00
Jason DoveandGitHub 12f2583c96 add timings for jf episode loading and saving during update (#2752) 2026-01-02 20:16:30 -06:00
Jason DoveandGitHub 7c82ecdfff use separate load queries for jf episode inserts and updates (#2751) 2026-01-02 16:28:19 -06:00
Jason DoveandGitHub bcd2ea7db3 db optimizations around names and case-sensitivity (#2749)
* generate (case-insensitive) unique names for fields that should be unique

* move name case-insensitivity down to schema level

* update changelog
2026-01-02 09:25:23 -06:00
Jason DoveandGitHub 80f6e468eb collect and print timings during jellyfin show library scans (#2748)
* collect and print timings during jellyfin show library scans

* update p99
2026-01-01 20:02:24 -06:00
Jason DoveandGitHub 474e647d6d more jellyfin performance improvements (#2747)
* fix slow db and api logging so it also works in scanner project

* don't request people from jellyfin by default
2025-12-31 22:28:15 -06:00
Jason DoveandGitHub 6bd49ffcec add remote stream metadata (#2690)
* add remote stream metadata

* use ifilesystem for last write time

* clean up unused inheritance

* optimize channel data query

* revert removeartwork change

* properly handle remote stream artwork in orphaned artwork check
2025-12-01 12:48:03 -06:00
Jason DoveandGitHub 1f31beab5b fix plex other video library detection (#2679) 2025-11-30 08:35:04 -06:00
Jason DoveandGitHub ec0d8ea6ac work around sequential schedule validation limit (#2655)
* remove readalltext

* remove unused method

* remove fileexists

* remove folderexists

* remove readalllines

* remove fake local file system

* show playlist name in playout build errors

* add basic sequential schedule validator tests

* work around sequential schedule validation limit
2025-11-24 12:08:43 -06:00
Jason DoveandGitHub d88e721d2f optimize database calls related to search index (#2645) 2025-11-13 13:27:37 -06:00
Jason DoveandGitHub 8b18f2a304 expose arbitrary epg data to graphics engine (#2633) 2025-11-11 12:41:45 -06:00
Jason DoveandGitHub d0505cd5c5 add better check for interlaced content (#2620) 2025-11-10 06:39:04 -06:00