Both issues are #701 deferrals, and they land together because both rewrite
the same decision record.
#823 -- can a null reach one of the six collection-valued scalar columns?
MEASURED against a real TvContext on BOTH providers (SQLite, and MySQL 8.4
on an ephemeral server), because the reasoning available beforehand pointed
the wrong way. The two converters differ on their read side --
IntCollectionValueConverter maps null-or-blank to Array.Empty<int>(), while
EnumCollectionJsonValueConverter would dereference the result of
JsonConvert.DeserializeObject -- so the expectation was that a NULL row
behaves differently per column. NEITHER RUNS: EF does not invoke a value
converter for a NULL column at all. All six materialize as CLR null, the
int converter's null-to-empty branch is dead on this path, and unguarded
each .Contains in AlternateScheduleSelector throws NullReferenceException.
A NULL reads as UNRESTRICTED -- the All*() sets -- not as empty. This is
the whole semantic question and the first draft got it backwards. It is
decided by the one NULL reachable WITHOUT any code writing one: Sqlite's
20240113140741_Add_PlayoutTemplate_DaysOfMonth adds the column with
nullable:true and NO defaultValue, so a PlayoutTemplate row inserted before
it holds NULL and by construction had no day-of-month restriction. Reading
that as empty INVERTS the row's meaning and silently stops the template
applying at all. All*() preserves it, and is how "no restriction recorded"
is already represented (GetPlayoutAlternateSchedulesHandler,
PreviewBlockPlayoutHandler). What does NOT decide it, and was wrongly cited
in the first draft: the API request records normalize an omitted field with
`?? []`, but that is a client omitting a field on a WRITE and says nothing
about what a legacy database NULL meant.
Two read sites, not one. Guarding only the selector would have left the
entity->DTO mappers unguarded, and those feed the SPA: PlayoutScheduleEditors
spreads the collection (`[...template.daysOfMonth]` -> TypeError on a JSON
null) and playoutTemplateCalendar's appliesToDate -- an exact port of
GetScheduleForDate -- calls .includes on it. Both mappers now substitute the
SAME defaults, so the preview agrees with what is actually scheduled. Neither
guard is assigned back onto the entity, which is the
media.nullable-primitive-collection-mutation mechanism.
Reachability, stated precisely rather than overclaimed. All six are
nullable:true on both providers, but a nullable column does not produce a
NULL row: five of the six were present at CreateTable, so a NULL there still
needs code to write one, and on MySQL there is NO code-path-free NULL for any
of the six. The write path ACCEPTS a null (SaveChanges succeeds, stores SQL
NULL) but no caller supplies one today -- every production construction of the
two commands goes through the request records. That is a property of the code,
not a live caller; claiming otherwise would be the banned "it's AsNoTracking
today" argument pointed the other way.
#824 -- ElasticSearchIndex.UpdateSong had no regression test
Issue option 1 (a non-network transport) shipped, and needed no new package:
Elastic.Transport.InMemoryRequestInvoker is public in the pinned version and
ElasticsearchClientSettings(NodePool, IRequestInvoker) accepts it, injected
into the private _client the way #701 injects the Lucene IndexWriter.
UpdateItems never runs `_client ??= CreateClient()`, so the injected instance
is the one used.
Two traps there are load-bearing, both measured: the canned response must
carry an `X-Elastic-Product: Elasticsearch` header or the client's product
check throws UnsupportedProductException INTO UpdateSong's catch, and an empty
body fails to deserialize the same way. Either turns the fixture into a green
measurement of the error path -- which is how it first failed here, caught by
the ThrowOnWarningLogger. The document id is asserted as the LAST PATH SEGMENT,
not by substring: the index name carries digits, so ShouldContain would stop
discriminating for a song whose id collided with one.
Six mutations executed, each disarming ITS OWN clause alone:
- `??=` restored in ElasticSearchIndex only -> the Elastic fixture reddens on
"metadata.Artists should be null but was []" while the LUCENE fixture stays
GREEN. The #824 hole demonstrated, not described.
- DaysOfWeek guard disarmed in the selector -> 4 red, 3 green (DaysOfMonth and
MonthsOfYear unaffected). Each clause is independently load-bearing.
- DaysOfMonth guard disarmed in Playouts.Mapper -> 1 red, 2 green.
- Elastic dropped from the covered set / mapped to the SAME fixture as Lucene /
mapped to a class with no [Test] -> SearchIndexMutationCoverageTests reddens
on each.
That coverage guard is the boundary fix the issue asked for: the covered set is
compared against an ISearchIndex population DERIVED FROM THE ASSEMBLY. Its claim
stops where the check does -- no static check can establish that a named fixture
actually DRIVES its indexer, so it forces a human to look rather than proving
coverage. ThrowOnWarningLogger moved to ErsatzTV.Tests/Support so both fixtures
share it; the Lucene fixture's assertions are otherwise untouched, since it is a
witnessed proof artifact.
No production change in ElasticSearchIndex.cs -- #824 is coverage only.
Docs: testing.md gains a "Provider-parity fixtures" section naming all THREE
opt-in-MySQL fixtures and recording that CI runs none of them (#627);
docs/README.md gains the matching task signal; guard-inventory.md's
hand-written C# guard list goes from five files to six. Scheduling/Mapper.cs
loses the UTF-8 BOM it inherited, per #311 fix-as-you-touch.
Local gate (with the MySQL lane armed): ErsatzTV.Tests 2096 passed / 0 skipped,
Core.Tests 693/1, Infrastructure.Tests 114, Architecture.Tests 7, Scanner.Tests
1504 -- 0 failures in each. scripts/tests 1228 passed / 2 skipped. dotnet format
whitespace --verify-no-changes clean; BOM check over the touched set with the
population COUNT asserted, because a bare zsh loop silently checks one
concatenated filename. decisions_validate OK.
Fixes#823Fixes#824
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
The prior commit (a4700185b) made SongMetadata.Artists/AlbumArtists
coalesce null to [] via backing-field getters, reasoning that EF
Core's PreferField access mode never observes the getter. Adversarial
review disproved this on real TvContext/SQLite: a single read of
.Artists on a TRACKED entity mutates the backing field through the
getter, flips the entity to Modified, and the next SaveChanges writes
[] over what was a NULL column -- silent data loss waiting on the
first tracked reader (today all readers happen to be AsNoTracking).
This also reversed docs/decisions/records/api/selection-projection-include-chain.md
(#671) without the doc update CLAUDE.md requires; #691 is that
record's own "sweep by FIELD" follow-up, so it should follow the
record, not contradict it.
Revert SongMetadata.cs to plain auto-properties (byte-identical to
origin/main, BOM still stripped per the #311 gate). Guard the read
sites instead, per the #671 convention (Optional(...).Flatten(),
matching Playouts/Mapper.cs and MediaItems/Mapper.cs):
- SongVideoGenerator.cs: hoist `artists`/`albumArtists` locals once
near the top of the metadata loop instead of repeating the guard at
each of the six former call sites.
- MediaCollectionRepository.cs (GroupIntoFakeCollections): guard the
two AlbumArtists reads at lines ~1147/~1160 that #691 never named --
dropping the entity-level fix without these would trade one bug for
two.
Verified RED per guard by removing only the Optional(...).Flatten()
clause (not the whole file): the artists local throws
ArgumentNullException at SongVideoGenerator.cs:88, the albumArtists
local at :89 (List.ToList() on a null IList<string> source -- same
loaded-gun shape the review demonstrated, precise exception type is
ArgumentNullException rather than NullReferenceException since the
throw site is Enumerable.ToList's null-source check). Restored both;
existing SongVideoGeneratorTests still pass. Full ErsatzTV.Core.Tests:
685 passed (1 pre-existing skip), ErsatzTV.Tests: 1996 passed (4
pre-existing skips), 0 failures in each. No EF model drift
(`dotnet ef migrations has-pending-model-changes` reports none).
`dotnet format --verify-no-changes` on the three touched files exits
0.
Refs #691
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SongMetadata.Artists and .AlbumArtists are nullable EF primitive
collections that FallbackMetadataProvider.GetSongMetadata never
assigns, so untagged songs persist them as NULL. SongVideoGenerator
dereferenced both unguarded (metadata.Artists.Count, string.Join,
AlbumArtists.Filter(...Artists.Contains...)), throwing NRE/ANE during
song-video generation on the playback path.
Rather than enumerating and guarding each read site (the same mistake
that left these unswept after #671), add backing fields to the two
properties whose getters coalesce null to an empty list. EF Core's
default PreferField access mode reads/writes the raw backing field
during materialization and change-tracking (confirmed by running the
full ErsatzTV.Tests suite, including SongMetadata round-trip tests,
unchanged), while every other caller -- SongVideoGenerator,
MediaCollectionRepository's rerun-collection artist grouping, and any
future reader -- goes through the property getter and always sees a
non-null list. This subsumes the ad hoc `metadata.Artists ??= []`
guards already hand-applied in LuceneSearchIndex/ElasticSearchIndex
and the `?? []` in LibraryBrowseItemMapper, which remain but are now
redundant.
Adds SongVideoGeneratorTests covering an untagged song (null Artists/
AlbumArtists) through GenerateSongVideo; verified RED (NRE at
SongMetadata.cs's Artists getter) by reverting only the `??= []`
clause, not the file.
Strips the pre-existing UTF-8 BOM from SongMetadata.cs per the #311
formatting gate (touching a legacy-BOM file makes stripping it ours
to do).
Refs #691
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Re-review of the previous fix commit found that two tests added to close
round-1 findings could not fail. Both verified before fixing:
1. Missing_But_Named_Custom_Playout_Item_Watermark_Should_Not_Fall_Through
gave the channel-level fallback the SAME missing custom path as the
playout-item watermark, so a wrongly-widened guard would have fallen
through to a fallback that also resolved to None -- the assertion held
either way. The fallback is now an independently resolvable ChannelLogo
whose cached file exists, so a fall-through returns it and fails the test.
Added the matching positive control (blank -> falls through and DOES
return that logo), so the pair shows the guard distinguishes blank from
unresolvable instead of both landing on None.
2. Deco_With_One_Valid_And_One_Missing_Watermark... asserted a filtered list
length while the routing claim the decision record cited it for lives in
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark, which the test
never called. It now calls the real predicate.
Also, three wrong claims of my own:
3. The Resource arm comment said "nothing in the app writes a Resource
watermark to the database". False -- CreateWatermarkHandler and
UpdateWatermarkHandler persist whatever ImageSource the request names, so
a Resource watermark IS creatable through the API, always with
Image = null. That is precisely why the new null guard is load-bearing,
so the comment was arguing for its own removal.
4. "One resolver and no per-caller policy" contradicted the surviving
playout-item blank-Custom fall-through documented a few lines later.
Reworded in both the record and the XML docs: one resolver, and exactly
one piece of per-caller policy which lives in the CALLER.
5. The record's "12 of 18 new tests fail pre-fix" was stale. Re-measured
against the final fixture: 19 of 29. The other 10 pass both ways by
design because they pin preserved behavior, which the record now says
explicitly rather than leaving the gap to be read as weakness.
Removed the vacuous generated-URL test rather than keeping it with an honest
comment -- an empty list trivially contains no URL, so it implied coverage it
never had. Its assertion is folded into the sibling test that has a real
arrangement.
Gates: 2772 tests green across 5 projects, dotnet format exit 0, no BOMs,
decisions-validate OK, live-E2E re-run against this binary (0 changed pixels,
nameplate absent, warning emitted).
refs #510
Two independent reviews (cross-family Codex + cold Opus) both returned
BLOCKED. Findings, all verified against source before acting:
1. Resource arm could throw ArgumentNullException (Codex, Medium). Making the
channel/global Resource arm reachable exposed that CreateWatermarkHandler
and UpdateWatermarkHandler write `Image = null` for EVERY non-Custom
watermark, so an API-created Resource watermark reached
Path.Combine(folder, null). Added the blank/null guard the arm never had.
This was live at the playout-item level too, not just newly-reachable code.
2. "Routing is unaffected" was false (Codex, Low but sharp). The predicate is
unchanged, but CanUseFFmpegNativeWatermark also tests Count == 1, and
dropping an unresolvable watermark shortens the list. A deco with one valid
and one missing permanent watermark now routes ffmpeg-native where it
previously routed to the graphics engine. Intended, but observable -- so it
is documented and pinned by a test rather than claimed away.
3. "Exactly one resolver" over-claimed (Opus, High). True of the selector, not
the application: the song-progress overlay is built as a WatermarkOptions
directly by the streaming and troubleshooting handlers, unchecked, and can
still hand ffmpeg a nonexistent -i. Pre-existing; scoped the claim in the
record and channels.md and filed #653.
4. Undeclared crash->degrade change (Opus, Medium). Channel/global Custom had
no blank-image guard, so a cleared image hit ImageCache's fileName[..2] and
threw out of stream startup. Now declared in the record and tested.
5. Contradictory rule text (Opus, Medium) -- the catalog one-liner said
"always no bug" while the body documents the playout-item fall-through
exception. Qualified; catalog regenerated.
6. History was wrong in both the record and the XML docs: the three precedence
levels did NOT all check every source -- channel/global had no Resource arm
and threw. Corrected.
Tests: 30 in the fixture now (was 18). New coverage for the preserved
blank-Custom fall-through (to channel AND to global), the complement case
(missing-but-named must NOT fall through), null/blank Resource, and the
valid+missing routing case. 17 of 24 failed against the pre-fix resolver
before this round; the fixture stays mutation-sensitive.
Also: hoisted the mock-filesystem Initialize() out of its loop so a
multi-file case cannot silently seed only the last file, and marked the
generated-URL test honestly as redundant-by-construction rather than
claiming independent coverage.
The decision record is now 81 prose lines, over the 60-line ceiling. Declared
as a legitimate decline per docs.corpus-size-signal: the length is the review
findings above, each a distinct fact, not redundancy.
refs #510#652#653
WatermarkSelector resolved watermarks in two places with two policies. The
three precedence levels (playout item, channel, global) existence-checked
every image source and degraded to None; the deco path had its own copy of
the same switch that returned whatever path it computed, unchecked. So one
channel could disagree with itself about whether an on-screen bug rendered,
based only on how the watermark was attached.
#502 deferred this here but scoped it to ChannelLogo. It was never
ChannelLogo-only: the deco path skipped the existence check for Custom and
Resource too. Extract one ResolveWatermark used by all four sites.
Severity is not cosmetic. A dead LOCAL path is not harmlessly skipped --
CanUseFFmpegNativeWatermark hands a single permanent watermark to ffmpeg as
a bare -i argument and excludes only URLs, so the deco path could hand
ffmpeg a nonexistent input file.
The generated-initials nameplate was real: a live-E2E on a real transcoded
frame confirmed it composited via the deco path (/iptv/logos/gen is on
ArtworkController, which has no auth filter, so the container-internal
self-fetch succeeded). The #502-era comment claiming "it has never rendered
here" was wrong, and the new record says so. It is still removed: serving it
means an HTTP fetch inside stream startup, which graphics.channel-logo-caching
(#525) eliminated for logos, and it depends on #1's hardcoded localhost.
Reviving it by caching the image instead is #652.
Measured blast radius on prod: 0 Deco rows, 0 DecoWatermark rows, all 43
channels have logo artwork -- no rendered output changes.
Preserved deliberately: a playout-item Custom watermark with a blank image
still falls THROUGH to the channel/global watermark; unifying resolution must
not change which watermark wins. Routing is untouched.
Strict improvement: the channel and global arms previously threw
NotSupportedException on a Resource watermark; they now resolve it. The
default arm still throws so a new image source fails loudly.
Tests: 18 new cases including a positive control and 8 deco-vs-channel parity
cases. 12 of the 18 fail against the pre-fix resolver, which is what proves
they are load-bearing rather than vacuous.
fixes#510
#616 filed three MCP/API paging traps. Two were real; one was not, and one was
already half-fixed on main. Verified each against the code before changing it.
REAL — pageNum documented as 1-based. `ToolCatalog.Page()` described pageNum as
"1-based page number" while every paged controller defaults it to 0, floors it
with `Math.Max(0, pageNum)`, and skips `PageNum * PageSize`. A caller that
trusted the description started at page 1 and silently lost the first page: no
error, just a short set that reads as data loss rather than an off-by-one (it
cost #487 a verification pass). Fixed in the description rather than by making
the MCP layer 1-based: /api/v1 is additive-only post-freeze, 0-based is
load-bearing in a dozen controllers and the SPA, and a 1-based wrapper over a
0-based API would make the same parameter name mean two different things on two
surfaces a reader reads together.
NOT REAL — "pageSize caps the page but the offset honors the requested value".
Not reproducible on any endpoint. Every controller clamps before passing, every
handler skips by the clamped size, and GetCollectionItemsHandler re-clamps
defensively. The reported observation (pageSize=500&pageNum=2 on a 204-item
collection returning 4 items) is exactly correct 0-based behaviour at the
clamped width of 100 — page 2 is items 201-204. The issue's own trap-1 table
states this. Pinned by test rather than "fixed".
ALREADY FIXED — playout LIST rows gained channelId in #297 (2026-07-22), three
days before #616 was filed; the report was measured against prod, which runs an
older :prod image. The DETAIL response (PlayoutResponseModel) genuinely still
lacked it, so channelId is added there (additive) and the reset_channel_playout
argument now names the trap: the id spaces overlap numerically, so passing a
playout id silently resets a different channel and returns a plausible 202.
Tests, both mutation-verified (each fails when its fix is reverted):
- ToolCatalogTests pins "0-based" on EVERY paged tool's pageNum description,
with a non-empty guard so it can't pass vacuously over an empty tool set.
- GetCollectionItemsHandlerTests pins 0-based page boundaries and proves the
offset derives from the clamped pageSize (page 1 at pageSize=500 returns
items 101-150; the mutation that honors 500 returns an empty page).
Docs: new decision record api.paging-zero-based (catalog regenerated), the
api-conventions paging bullet, and a Paging section in docs/mcp.md. OpenAPI
v1.json + web/src/api/generated/v1.d.ts regenerated for the added field.
fixes#616
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GetOrAddFolder was a check-then-insert with no unique constraint behind it,
so two callers racing the same folder could both miss the lookup and both
insert. Enforce identity in the schema and make the loser adopt the winner.
- LibraryFolder gains a SHA-256 PathHash (the MediaFile.Path/PathHash
precedent): Path is MySQL longtext, which cannot be indexed without a
prefix length and collates case-insensitively, so the unique index is on
(LibraryPathId, PathHash) instead.
- GetOrAddFolder and SetEtag catch a classified unique violation via the
existing TvContext.IsUniqueConstraintViolation seam (#308) and re-read.
- Dual-provider migration audits and collapses pre-existing duplicates
(repointing MediaFile, ParentId and ImageFolderDuration) before creating
the index; legacy rows keep a null hash and heal on the next scan.
- Tests: deterministic cross-connection race, 8x10 barrier stress with an
insert-attempt vacuity guard, classifier-inversion negative control, and
a real-migration dedupe test.
Refs #488#308fix#491
Independent cold review (Codex) returned BLOCKED. Findings 1, 3 and 4 are fixed here;
each has a regression test proven non-vacuous by a negative control.
1. Blocker — the replaced local path was discarded. The scanner computed localPath but
GetOrAdd only received `incoming`, so the repository re-derived the path from the
UNREPLACED projection. On any install with path replacements, adoption hashed the
server-side path, missed the existing row, ALSO slipped past MediaFileAlreadyExists
(which hashes that same wrong string) and inserted a duplicate row under a server path,
leaving the original collection-linked row identity-less forever. The test harness hid
this because its path-replacement stub was an identity function.
→ GetOrAdd now takes localPath explicitly and never reads the projection's path;
BuildPathReplacement takes a real mapping and the new test genuinely replaces.
3. Medium — GetByItemId matched on ItemId alone, so two media sources presenting the same
item id (cloned Jellyfin DB) resolved to each other's row, letting one library repoint
another's. → filtered by LibraryPath.LibraryId.
4. Medium — a row predating the identity that the server had ALREADY stopped reporting was
never adopted (adoption only runs for an incoming item) and carried no identity, so the
itemId diff could not see it either: it sat Normal and schedulable forever, strictly
worse than the hard delete it replaced. → GetExistingLegacyMusicVideoPaths +
FlagFileNotFoundByPaths reconcile legacy rows by local path, and they are counted into
the #477 empty-fetch guard (on the first scan after this ships they ARE the whole
library, so a guard counting only identity rows would sweep all of them on a transient
empty fetch).
Finding 2 (the issue's Done-when #2) is a scope question, not a defect, and is unchanged:
one file path is still one MediaItem row globally, so this lands music videos at parity
with movies rather than eliminating shared-row trashing. Recorded honestly in the decision
record; raised for an explicit call before the issue is closed.
fixes#496
Music videos carried no server identity, so JellyfinMusicVideoLibraryScanner had to
reconcile by a (LibraryPathId, path) diff and HARD-delete the remainder. A file served
by two libraries with overlapping local paths is a single row owned by whichever library
scanned it first, so that owner's sweep destroyed a row another library still served —
taking collection membership and playout references with it, irreversibly.
This is #494's deferred "option 2":
- New JellyfinMusicVideo : MusicVideo (ItemId/Etag), mirroring JellyfinMovie — TPT table,
varchar(36), ItemId index. Dual-provider migration Add_JellyfinMusicVideo.
- New IMediaServerMusicVideoRepository + JellyfinMusicVideoRepository: itemId-keyed
existing-set/lookup and Flag{Normal,Unavailable,FileNotFound} seams, all scoped per
library via LibraryPath.LibraryId.
- New MediaServerMusicVideoLibraryScanner base; JellyfinMusicVideoLibraryScanner folds
onto it and keeps the #177/#488/#497/#500 metadata-reconcile logic verbatim.
- The sweep now soft-trashes (FileNotFound) instead of deleting, so removal is reversible
and EmptyTrash-governed. DeleteEmptyArtists consequently no longer fires from a sweep.
- Pre-identity rows are ADOPTED in place: the identity row is inserted against the same
MediaItem id, scoped to the scanned library's own LibraryPath, so collection membership
survives and a local/second-library row is never hijacked.
- AddMusicVideo normalizes Path/PathHash to the path-REPLACED local path; the projection
fills them from the server-reported path, which would break every later PathHash lookup.
Docs: scan.musicvideo-reconciliation relocated to docs/decisions/archive/scan.md as
superseded; new active record scan.musicvideo-server-identity.
fixes#496
Review follow-up. The comments still said three (Jellyfin) / two (Emby)
library-level enumerations after the nested season and episode ones were
counted, which understates the reach of the very safety property this
branch establishes -- a future auditor reading them would conclude the
nested sweeps are unprotected.
Review finding 1 (blocking). ScanSeasons' FlagFileNotFoundSeasons and ScanEpisodes'
FlagFileNotFoundEpisodes had no guard at all — neither #477's nor #484's — so ProjectToSeason /
ProjectToEpisode returning Failed() was computed and discarded.
#477 scoped those out because "the blast radius is one show's seasons / one season's episodes",
which holds for a per-parent EMPTY fetch but not for a projection failure: that is systematic by
construction. One bad code path fires on every parent, so every season enumerates zero episodes,
existing.Except([]) is the whole episode library, and EmptyTrashHandler deletes it permanently.
Threads the counter into GetSeasonLibraryItems / GetEpisodeLibraryItems(WithoutPeople) for
Jellyfin and Emby using the same optional-trailing-param shape, and guards both sweeps with
MediaServerReconciliationGuard.ShouldFlagMissingDescendants — the same class and the same private
failure predicate as ShouldFlagMissing, deliberately WITHOUT #477's empty-fetch branch so
per-parent empty behaviour (and #476's cascade, which depends on it) is unchanged.
Also from the review:
- finding 3: tests now pin the same-instance JOIN at every level (movie, show, season, episode,
music video) by driving the real ScanLibrary entry point and recording the failure from inside
the enumeration, so a refactor handing the api client a fresh counter goes red.
- finding 4: the missing-library Failed() branch is documented as defensive and unreachable.
- finding 2: the mass-Skip residual (Emby's response-shape-dependent MediaSources guard, Plex's
pre-projection filter) is stated as a known limitation in the decision record.
- finding 5: the log-contract change (only the #484 message when both refusals apply) is noted.
fixes#484
Extends MediaServerReconciliationGuard (#477) with a second deterministic refusal: when the
enumeration that produced the incoming set silently dropped items whose projection THREW, the
file-not-found sweep is refused. A dropped item the server did return is indistinguishable
from a deletion at the reconcile step, so a projection regression could otherwise mass-flag a
healthy library FileNotFound (which EmptyTrash then deletes permanently).
Deliberate guard-clause skips (STRM files, virtual items, unsupported types) are explicitly NOT
failures and never suppress a sweep — counting them would permanently disable reconciliation for
any library holding a single STRM file.
The ratio / missing-fraction threshold is REJECTED, not deferred: it is a two-sided heuristic
with no tunable default and no telemetry, and the failure it approximates is exactly observable
via the projection-failure count (a genuine bulk deletion produces zero failures).
Seam is deliberately narrow — the private ProjectTo* contract inside each api client changed from
Option<T> to MediaServerProjectionResult<T> (projected/skipped/failed), the paged helper counts
IsFailure in one place, and the scanner reads it through an optional trailing
MediaServerProjectionFailureCounter on only the five library-level methods that feed a sweep.
The counter is per-enumeration state created by the scanner, never a field on an api client.
fixes#484
Closes#415. Server-derived health object on the channel list + detail DTOs (built-timeline detection, kind-agnostic across all 5 PlayoutScheduleKind; assessable gate keyed to the owning channel's mode), single "Problems" SPA filter with per-fault badges. Supersedes #72's api.channel-health-signal decision.
Co-authored-by: Timothy <timothy.look@gmail.com>
Co-committed-by: Timothy <timothy.look@gmail.com>
The GetFieldValues additions were reverted in the DB-sourcing rework; these two
files had only an incidental BOM strip left, which pulled unrelated pre-existing
whitespace debt into the scoped format gate. Restore byte-identical to origin/main.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ISearchIndex.cs / LuceneSearchIndex.cs still carried a BOM after the prior
commit — the strip ran after `git add`, so the staged (BOM'd) content was
what got committed. No content change beyond the BOM.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Lucene term dictionary stores lowercased word tokens for analyzed text
fields ("Science Fiction" -> science/fiction), so the typeahead was
suggesting fragments instead of whole values. GetSearchFieldValuesHandler
now injects IDbContextFactory<TvContext> and resolves an explicit
per-field-name distinct-values query (genre/studio/director/writer/actor/
artist/tag/network/collection/video_codec/album), with state and
video_dynamic_range computed in memory and content_rating split on '/' to
match what search actually matches on. title/show_title/album_artist have
no distinct source and now correctly 404 (free-text fallback), same as
before. Reverts the GetFieldValues additions to ISearchIndex/
LuceneSearchIndex/ElasticSearchIndex back to their pre-#434 state (BOM
stripped per #311, otherwise byte-identical). Endpoint shape, DTO,
controller, and OpenAPI are unchanged (no diff from
./scripts/update-openapi.sh).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds GET /api/v1/search/fields/{name}/values?q=&limit= — the backend slice of the
visual rule builder's facet-value typeahead (#434). Enumerates distinct Lucene term
values for a text field via MultiFields.GetTerms + TermsEnum, filtered by a
case-insensitive prefix, limit clamped to [1,50]. 404s when the field is absent from
SearchFieldCatalog or is not type "text". ElasticSearchIndex (the optional external
backend) throws NotSupportedException for this method — its text fields are analyzed,
not keyword-mapped, so a terms aggregation isn't safe to guess at without verifying
against a live cluster.
Regenerated OpenAPI trio (v1.json, v1.d.ts, endpoint-index.md).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds parameterized invariant tests (Schedule_clock_padded_offline_multimode)
exercising schedule-level clock pad + offline advance across a 2-day window
(two midnight crossings) through Flood, Duration, and Multiple — previously
only PlayoutModeSchedulerOne had any coverage. Fixture uses sub-15-min content
so each padded item occupies one :15 slot and Duration's fill-the-block
contract tiles exactly (no off-boundary packing).
Part 2 (precision): replaces the day-boundary anchor-clamp magnitude heuristic
(overrun <= one pad interval on a padded schedule) with a precise signal — the
last scheduler now reports the exact offline-pad target it advanced CurrentTime
to (transient PlayoutSchedulerResult.ClockPadOfflineTarget, never persisted),
and the clamp exempts only when CurrentTime equals that target exactly. A
non-offline overrun (Duration/Flood/Multiple ending short of its natural end, a
hard-stop, a tail advance) changes CurrentTime away from the target and still
clamps, so persisted NextStart no longer shifts by up to an interval. No
persisted-schema/migration change. Output byte-identical for all existing
goldens.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the byte-identical PlaybackOrder -> IMediaCollectionEnumerator switch
shared by SchedulingEngine.EnumeratorForContent (Scripted) and
EnumeratorCache.GetEnumeratorForContent (Sequential/YAML) into one static
per-family seam, mirroring #380's ShuffleSourceBuilder. Each engine keeps its own
"not supported" warning on the None branch, so the per-engine message is unchanged.
Adds ContentEnumeratorBuilderTests pinning the block-shuffle-not-classic trap and
the unsupported-order -> None (#70) contract across all 8 unsupported orders.
Corrects the testing.scripted-playout-golden-deferred decision record: Scripted's
external-process + HTTP pipeline is integration-only (deferred to #563), but the
in-process SchedulingEngine it drives IS unit-testable (ScriptedScheduleController
is a 1:1 pass-through) -- the earlier "un-golden-able by construction" framing
conflated transport with engine. docs/testing.md reframed to match.
[decisions-edit]
fixes#395
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Added ChannelId to PlayoutNameViewModel and all 6 construction sites
(Mapper, GetPlayoutByIdHandler, and the Update{,Scripted,ExternalJson,Sequential}
PlayoutHandler commands), plus the list DTO PlayoutListItemResponseModel and the
PlayoutController list projection. Regenerated OpenAPI (v1.json) and the TS client
(v1.d.ts); endpoint-index.md unchanged (no endpoint/operation delta). Simplified
PlayoutsScreen resetSelectedChannel to key directly on selectedSummary.channelId
instead of resolving via channelStates. Updated controller + SPA tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GetPreview keyed only on StreamingMode + JWT, so a disabled channel or one
with no playout was declared Available and then failed confusingly (404 from
IptvController, or an indefinitely-blocking manifest request). Extend it to
take isEnabled + playoutCount and check JWT, then disabled, then no-playout,
before falling back to the existing mode-based rules; order is documented
in a comment. Also corrects a stale comment in ChannelPreviewResponseModel.cs
that claimed a C# string generates a TypeScript union (it does not).
Adds ChannelPreviewResponseModel + ChannelPreviewAvailability constants
and Mapper.GetPreview(streamingMode, channelNumber, iptvJwtEnabled),
threaded through ProjectToResponseModel's new iptvJwtEnabled parameter
and GetAllChannelsForApi's new IptvJwtEnabled property.
Also adds ErsatzTV.Core.Tests to ErsatzTV.Application's
InternalsVisibleTo list (was ErsatzTV.Tests only) so the new
ChannelPreviewTests can call the internal Mapper methods it's testing.
ChannelController.GetAll is left failing to compile (Task 2 wires
JwtHelper.IsEnabled in at the controller).
CreateChannelFromLineupHandler resolved every advanced override with
advanced.X ?? template.X, so null always meant INHERIT and a channel could
not drop a template-set watermark / filler / preferred language. Add an
optional typed `clear` enum list to CreateChannelFromLineupAdvancedOptions:
omitted/null still inherits (byte-stable for existing clients), a field named
in `clear` is forced to none. Set+clear of the same field is a 422.
The enum (CreateChannelFromLineupClearField) lives in ErsatzTV.Core so the
OpenAPI string-enum scan renders it as a string enum, matching every sibling
advanced-options enum. Handler resolves clearable fields once via
ResolveClearable and validates set/clear conflicts via ValidateClear;
reference validation skips existence checks for cleared (null) refs.
SPA: the shared advancedOptions model re-adds a real "None" option to the five
id selects (watermark + fillers) in both the Channel Builder and the Auto-Tune
DetailPanel, routed through a CLEAR overrides sentinel that applyOverridesToRequest
folds into advanced.clear (never leaking onto the wire as a field value). The
backend enum also covers the preferred audio/subtitle language strings for
machine clients; the SPA text inputs keep "empty = inherit" (tri-state deferred).
Docs: api-conventions.md §2, spa-conventions.md §11, decisions.md record
api.from-lineup-clear-to-none; v1.json + generated TS regenerated.
fixes#135
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An on-demand channel (`PlayoutMode.OnDemand`) already is the "resume where I
left off" feature: `Playout.OnDemandCheckpoint` persists the viewer's position
and `PlayoutTimeShifter.TimeShift` slides the materialized timeline forward on
tune-in so the paused item is active again. Because it rewrites `GuideStart`/
`GuideFinish` alongside `Start`/`Finish`, guide and playback freeze together —
structurally avoiding the free-running-wall-clock desync #68 was filed about.
The one gap: `TimeShift` rewrote the stored `PlayoutItem` rows but the XMLTV
guide is served from a cached fragment that only `RefreshChannelData` rebuilds,
and the tune-in path never enqueued it. So an external EPG client polling after
a thaw could see a stale timeline until the next incidental rebuild.
Fix: `IPlayoutTimeShifter.TimeShift` now returns the channel numbers whose cached
guide is stale — the shifted channel plus any channels that mirror it (the same
fan-out `BuildPlayoutHandler` already does) — and `TimeShiftOnDemandPlayoutHandler`
enqueues a `RefreshChannelData` for each on `CancellationToken.None` (post-commit
side effect must not be abandoned if the session token cancels).
Tests: handler enqueues a rebuild per stale channel (+ mirror + no-shift cases);
`PlayoutTimeShifter` reports source+mirrors on a shift, empty on Continuous /
zero-offset / active-unforced, and correctly seeds+shifts a never-watched playout.
Non-vacuity of the enqueue proven by a compiling negative control.
Docs: channels.md (On-demand resume section), domain-model.md, decisions.md
(scheduling.ondemand-guide-refresh-on-thaw). Per-viewer resume is out of scope
(single per-channel checkpoint; #68 says per-channel suffices).
fixes#68
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`ImageElementBase.LoadImage` fetched http(s) images with a throwaway
`new HttpClient()` + `GetStreamAsync`: no timeout override (the 100s
default), no size cap, unbounded redirects, no pooling — all inside
stream startup, while ffmpeg waits on the pipe. #502 routed ordinary
channel-logo watermarks onto that path, widening a pre-existing weakness.
Introduce `IRemoteImageFetcher` / `HttpRemoteImageFetcher`, modelled on
the neighbouring `IRemoteStreamProber`:
- deadline covers headers AND body (linked CTS + `CancelAfter`, client
`Timeout = InfiniteTimeSpan`) — under `ResponseHeadersRead` the body
read falls outside `HttpClient.Timeout` (the #289 lesson)
- 10 MiB cap enforced during the copy; `Content-Length` is only a cheap
early reject, since it can be absent or a lie
- permissive content-type check (rejects an HTML error page, allows a
missing type and octet-stream)
- pooled via `IHttpClientFactory`; redirects capped at 3, not 50
A byte cap does NOT bound decoding, so `DecodeRemoteImage` additionally
reads declared dimensions + frame count from the header and rejects
before `Image.LoadAsync` allocates (50 MP / 600 frames). A 4 KB PNG
declaring 30000x30000 costs ~3.6 GB to decode and passes every wire-size
check — caught by adversarial review of the first version of this change,
which capped bytes and wrongly claimed that was decode-bomb protection.
Not cached and SSRF not mitigated — both deliberate, with the reasoning
recorded in docs/decisions.md.
fixes#511
Adversarial review found a documentation defect, not a code one: both
docs/decisions.md and the WatermarkSelector comment asserted the deco path
was unaffected by this change. That is true of the *resolution* half and
false of the *routing* half. SelectWatermarks puts deco-derived options
into the same list the routing guard filters, so a deco watermark whose
resolved path is a URL is rerouted to the graphics engine too — including
the generated-initials localhost URL, which only the deco path still emits
and which plausibly rendered through ffmpeg before.
That reroute is intended (routing by what the path is beats routing by
provenance, which would drift), so the fix is to say so accurately rather
than to narrow the guard. Also records the accepted per-frame cost
asymmetry the entry previously argued on correctness grounds alone.
The guard is extracted as CanUseFFmpegNativeWatermark so it can be tested
directly — review's highest-value gap was that the half of the fix which
decides whether pixels appear had no automated coverage, only the one-off
live E2E. Nine cases pin it, including the localhost-fallback reroute.
Both deferrals now point at real issues instead of an unverifiable
"tracked separately": #510 (deco vs precedence-level missing-logo policy)
and #511 (remote-fetch hardening — timeout, size cap, redirects, pooling,
caching, SSRF).
Also pins scheme-case insensitivity in the selector.
A channel whose logo is an external URL never rendered a watermark, even
with an ImageSource=ChannelLogo watermark attached. WatermarkSelector
resolved the URL correctly and then existence-checked it on the
filesystem — File.Exists("https://…") is always false — so all three
precedence levels (playout item, channel, global) logged "Channel logo
no longer exists" and returned None. The channel editor advertises the
URL as winning over an uploaded logo, which was true for the guide
listing and silently false for the bug.
External artwork passes through rather than being downloaded into the
image cache: that is already the convention everywhere else (M3U, XMLTV,
SPA JSON all emit the raw URL), no fetch->SaveArtworkToCache glue exists,
and the render path does not need it — ImageElementBase.LoadImage already
fetches an http(s) path with HttpClient and decodes it for real pixel
dimensions.
A remote-URL watermark is therefore forced onto the graphics engine
instead of the ffmpeg-native shortcut, which would otherwise hand the URL
to ffprobe and ffmpeg as a bare -i argument, putting an unbounded network
fetch inside stream startup.
The three gated precedence levels now share one ChannelLogoWatermarkOptions
helper — the triplicated block is what let the defect exist three times
over. Scope held narrow: the generated-initials localhost fallback (#1)
stays disabled behind an explicit comment and a scope-guard test, and the
deco path keeps its own long-standing unchecked policy.
Verified by live-E2E against a real channel playout with an external-URL
logo: origin/main renders 0 logo pixels and logs the "no longer exists"
warning verbatim; this branch renders the logo in the expected region.
Whitespace-only reformatting in FFmpegLibraryProcessService.cs is the
fix-as-you-touch format gate on pre-existing violations, plus a BOM strip.
fixes#502
Whitespace-only (git diff -w is empty); the #311 format job checks whole
touched files, and these two legacy files carried pre-existing violations
never caught before (no PR had touched them since the gate landed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
timothyandClaude Opus 5