Commit Graph
134 Commits
Author SHA1 Message Date
timothyandClaude Opus 5 b5dee26202 fix(701): guard SongMetadata's nullable primitive collections at the read site
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 8s
PR Gates / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 19s
review-verdict/h10 Review-verdict: MERGEABLE @ b5dee26 (base: main)
PR Gates / decisions lifecycle (pull_request) Successful in 20s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 15s
Review verdict / Set review-verdict status (pull_request_target) Successful in 6s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 4m32s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m44s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m18s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m53s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Both search indexers opened UpdateSong with

    metadata.AlbumArtists ??= [];
    metadata.Artists ??= [];

Artists/AlbumArtists hold the whole list in ONE COLUMN rather than
being navigations. So unlike the same `??= []` idiom on
Genres/Tags/Artwork all around them, the property IS the column value:
assigning it on a TRACKED entity flips the entry to Modified and the
next SaveChanges writes [] over a NULL column. This is the mechanism an
adversarial review demonstrated in #691, which is why that issue's
entity-level guard was reverted in favour of guarding at the read site.

Measured rather than reasoned about, per the issue's first done-when
box. Restoring ONLY the `??= []` clause (the real predecessor lines,
not a hand-written mutant) reddens the new fixture on
`metadata.Artists should be null but was []`; a probe variant with the
first two assertions replaced by prints reports STATE=Modified and the
raw column moving from NULL to "[]". Today's two feeds are both
AsNoTracking (SearchRepository.GetItemToIndex and GetAllSongs), so no
shipped caller loses data -- but that is a property of two callers, not
of the indexer, and #691 already recorded it as a loaded gun. The
fixture pins the indexer's own contract instead.

Removing the assignment is not sufficient alone: it was load-bearing
for the four reads below it, and deleting it by itself converts a
silent write into a live throw on every untagged song. Measured by
deleting only those two lines from the real predecessor file:
NullReferenceException, thrown at the foreach (cited by symbol: a line
number in a mutant that exists in no committed tree is unreproducible
by construction). The
exception type follows the read FORM, not the field -- foreach yields
NRE, string.Join/ToList yield ArgumentNullException -- and this PR
contains two of each, which is why no single exception-name grep
characterises the class. So each site moves together with its reads:

- LuceneSearchIndex.UpdateSong / ElasticSearchIndex.UpdateSong: hoist
  Optional(...).Flatten().ToList() locals and read those.
- RefreshChannelDataHandler: the Scriban context took the raw nullable
  lists (the issue's second item). The shipped _song.sbntxt only does
  array.join, but a custom template is free to do anything.

The population was derived from the MODEL rather than from the issue's
file list, and the obvious derivation is wrong: "the IList<string>
properties under ErsatzTV.Core/Domain" returns two of eight. It misses
the six value-converted collections (ProgramScheduleAlternate and
PlayoutTemplate each carrying DaysOfMonth, MonthsOfYear, DaysOfWeek),
declared as plain ICollection<T> and made single columns only in
Data/Configurations -- and their storage differs (comma-separated text
for the int converter, JSON for the enum one), so the shared property
is "one scalar column", not the serialization. No site applies `??=`
to any of the six, so this defect has no instance there; whether a null
can REACH one at runtime is unverified and is filed as #823 rather than
asserted either way. Only the SongMetadata pair is left NULL in
practice, by FallbackMetadataProvider. Every site touching either field
was then swept; the remaining readers were already guarded by #691.

The fixture carries two anti-vacuity guards, both witnessed:

- A POSITIVE CONTROL (`writer.NumDocs.ShouldBe(1)`). Every other
  assertion says something did NOT happen, so all of them hold
  vacuously if UpdateSong never runs -- and it silently stops running
  if a future refactor gates UpdateItems on `_initialized`, which this
  fixture bypasses by injecting the writer. Verified BOTH directions:
  with that gate added the control fails `NumDocs should be 1 but was
  0`, and with the control removed the whole test PASSES while the code
  under test is unreachable.
- A capturing logger, because UpdateSong wraps its body in a catch that
  assigns metadata.Song = null -- severing a required relationship and
  cascading the metadata to Deleted. Without it the probe silently
  measures the error path; on the first run it did exactly that (a bare
  ILanguageCodeService substitute NPEs inside AddLanguages). The raw
  column helper also fails loudly on a missing row, since ExecuteScalar
  returns CLR null for both "NULL column" and "no such row".

ElasticSearchIndex has no equivalent fixture -- it needs a stubbed
transport -- so its change is by inspection against the Lucene one, and
the gap is filed as #824 rather than covered by a source-text guard.

The whitespace-only churn in ElasticSearchIndex.cs is the #311
fix-as-you-touch format gate: it scopes to whole changed FILES.
`git diff -w` over that file shows only the two hunks above.

Local gate: ErsatzTV.Tests 2006 passed / 4 pre-existing skips,
Core.Tests 685/1 skip, Infrastructure.Tests 114, Architecture.Tests 7,
Scanner.Tests 1504 -- 0 failures in each. scripts/tests 874 passed / 2
skipped. dotnet format whitespace --verify-no-changes clean on the four
touched files, no BOM on any. decisions_validate OK.

Fixes #701

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 00:49:57 +02:00
Jason DoveandGitHub 0d301df5e8 remove external dependencies (bugsnag, trakt) (#2840)
* remove bugsnag

* remove trakt client id (that will expire)
2026-02-26 10:43:48 -06:00
Jason DoveandGitHub 36147b9e9c fix indexing collections in elasticsearch (#2806)
* fix indexing collections in elasticsearch

* more safety
2026-01-29 18:23:07 -06:00
Jason DoveandGitHub 974020a98f optimize searching for shows, seasons and movies (#2768)
* load search logging level on startup

* optimize searching for shows, seasons and movies

* use season metadata directly
2026-01-12 19:42:49 -06:00
Jason DoveandGitHub d88e721d2f optimize database calls related to search index (#2645) 2025-11-13 13:27:37 -06:00
Jason DoveandGitHub b6d509b9cd add search query collection type to block schedules (#2644)
* add search query collection type to block schedules

* fix history
2025-11-13 10:48:04 -06:00
Jason DoveandGitHub c6245bae0c fix indexing songs with null artists/album artists (#2592) 2025-11-02 09:25:46 -06:00
Jason DoveandGitHub 727a978689 fix block history and smart_collection search bugs (#2579)
* fix block history when using mirror offset

* fix smart_collection search crashes
2025-10-29 14:01:14 -05:00
Jason DoveandGitHub 1c07df5bc3 use cancellation tokens in many places (#2350)
* use cancellation tokens everywhere

* more cancellation tokens
2025-08-27 03:20:35 +00:00
Jason DoveandGitHub 5d081ceeff fix editorconfig and run code cleanup (#2324)
* fix formatting rules

* reformat ersatztv

* reformat ersatztv.application

* reformat ersatztv.core

* refactor ersatztv.core.tests

* reformat ersatztv.ffmpeg

* reformat ersatztv.ffmpeg.tests

* reformat ersatztv.infrastructure

* cleanup infra mysql

* cleanup infra sqlite

* cleanup infra tests

* cleanup ersatztv.scanner

* cleanup ersatztv.scanner.tests

* sln cleanup

* update dependencies
2025-08-16 14:44:48 +00:00
Jason DoveandGitHub ab2b926de0 add searching log category (#2251) 2025-08-04 14:51:13 +00:00
Jason DoveandGitHub dd9a6d5a06 add chapters to search index (#2199) 2025-07-24 21:03:58 +00:00
Jason DoveandGitHub 5c43ae47b1 add basic remote stream library (#2175)
* initial remote stream library support; scanning seems to work ok

* flood schedule remote streams kind of works

* switch remote stream definitions to yaml files

* implement remote stream script playback

* update changelog
2025-07-20 16:10:32 +00:00
Jason DoveandGitHub c29788bc3f add movie nfo country to search index (#2173) 2025-07-19 21:56:13 +00:00
Jason DoveandGitHub 451c534062 allow block items to disable watermarks (#2139)
* allow block items to disable watermarks

* fix test
2025-07-12 03:55:18 +00:00
Jason DoveandGitHub 7e30444857 dependencies and code cleanup (#2117)
* fix validation in new form layout

* pin mediatr to last oss version

* update dependencies

* cleanup code in core

* cleanup code in ffmpeg

* cleanup code in infra

* cleanup code in scanner

* cleanup code in application

* cleanup main code

* cleanup test code

* solution-wide code cleanup
2025-07-06 15:56:17 +00:00
Jason DoveandGitHub 0f795e4e2f add plex network metadata (#2085)
* initial plumbing

* scan for plex networks

* save network contents to db as tags

* eliminate network tag id churn

* add network and show_network to search index

* update last networks scan

* show networks on tv show page

* update changelog
2025-06-28 12:49:26 +00:00
Jason DoveandGitHub c4c164df6a detect cycles in yaml sequence definitions (#2060) 2025-06-23 02:55:35 +00:00
Jason DoveandGitHub b90463e3af detect cycles in smart collection queries (#2059) 2025-06-23 01:45:17 +00:00
Jason DoveandGitHub 06f233e5bd upgrade to dotnet 9 (#2056) 2025-06-22 19:04:39 +00:00
Jason DoveandGitHub 9917774671 allow searching by smart collection (#2055) 2025-06-22 15:43:12 +00:00
Jason DoveandGitHub 5be929da18 add collection (name) to search index (#2054) 2025-06-22 14:51:17 +00:00
Jason DoveandGitHub 140a663da4 update dependencies (#2003)
* update dependencies

* more dependency updates
2025-04-28 17:20:45 +00:00
Jason DoveandGitHub 17a77694a0 add tag_full field to search index (#1956) 2024-12-09 13:55:47 -06:00
Jason DoveandGitHub 7bba422880 fix emptying trash (#1905) 2024-09-25 10:27:34 -05:00
Jason DoveandGitHub 4f393d7b06 fix two letter language code stream selection (#1798) 2024-07-22 06:54:08 -05:00
Jason DoveandGitHub 109960c457 fix some more search queries (#1763) 2024-06-26 22:08:59 -05:00
Jason DoveandGitHub beee038be3 fix video_bit_depth search field for plex media (#1745) 2024-06-10 15:46:23 -05:00
Jason DoveandGitHub a87ec2d75d cleanup (#1708)
* fix blazor naming

* code cleanup

* update dependencies
2024-05-06 17:00:52 -05:00
Jason DoveandGitHub 706a2d14a2 add playlists (#1690)
* update dependencies

* add playlists

* add playlist support to schedules

* playout builder (flood) supports playlists

* update changelog
2024-04-27 19:58:40 -05:00
Jason DoveandGitHub 2e2523c380 another wildcard search fix (#1667) 2024-04-05 09:12:46 -05:00
Jason DoveandGitHub d91d991251 always interpret asterisk as wildcard search (#1657) 2024-03-29 13:27:18 -05:00
Jason DoveandGitHub 60bb369d0c add search query parser (#1652)
* add search query parser

* add some search tests; use consistent analyzers for indexing and searching
2024-03-22 09:41:08 -05:00
Jason DoveandGitHub aae704f3a5 fix searching id fields (#1651) 2024-03-21 19:33:50 -05:00
Jason DoveandGitHub a45583d77a fix type and tag searches (#1649) 2024-03-20 22:52:16 -05:00
Jason DoveandGitHub 923b36604c fix many search queries (#1648) 2024-03-20 22:29:51 -05:00
Jason DoveandGitHub b21d16b0f1 add show_content_rating to search index (#1647) 2024-03-20 19:46:19 -05:00
Jason DoveandGitHub 087901d177 adjust block unique constraint (#1634)
* upgrade dependencies

* allow blocks with same name in different groups

* code cleanup
2024-03-05 10:39:06 -06:00
Jason DoveandGitHub 707292c50f add configurable image duration (#1617) 2024-02-16 13:05:28 -06:00
Jason DoveandGitHub f8c4f44216 add basic image library support (#1608)
* add basic image library support

* add image page

* update changelog
2024-02-11 11:24:19 -06:00
Jason DoveandGitHub a15854d0ad more guide templates; new song metadata library (#1603)
* refactor template processing

* use template for song xmltv entries

* use template for other video xmltv entries

* update changelog
2024-02-09 11:27:44 -06:00
Jason DoveandGitHub c743d07425 include external subtitles in search index (#1602) 2024-02-07 13:28:43 -06:00
Jason DoveandGitHub abed22b560 prevent unnecessary search index updates (#1592)
* add sub_language and sub_language_tag fields to search index

* prevent unnecessary search index updates

* update changelog

* update dependencies
2024-02-02 06:33:06 -06:00
Jason DoveandGitHub 7d5cd229d4 add show_studio search field (#1584) 2024-01-30 16:50:57 -06:00
Jason DoveandGitHub 12234c3e21 allow shuffling block items (#1571)
* allow shuffling block items

* fix drop down search results
2024-01-23 22:42:28 -06:00
Jason DoveandGitHub 56bc58fce9 reorganize to fix build (#1540) 2024-01-08 20:00:35 -06:00
Jason DoveandGitHub c18be5559b fix delete old segments (#1536)
* code cleanup

* ignore errors deleting old hls segments
2024-01-04 10:42:04 -06:00
Jason DoveandGitHub 9471cb55dd upgrade from dotnet 7 to dotnet 8 (#1529)
* upgrade sdk

* fix warnings in ersatztv.ffmpeg

* fix warnings in ersatztv.core

* fix warnings in ersatztv.infrastructure

* fix warnings in ersatztv.application

* disable analysis for migrations projects

* fix warnings in ersatztv.scanner

* fix warnings in ersatztv

* upgrade project framework

* update github actions and dockerfiles
2023-12-30 13:29:57 -06:00
Jason DoveandGitHub 948b3735bd fix file not found music videos (#1502)
* fix indexing music videos in file not found state

* update dependencies
2023-11-14 05:50:51 -06:00
Jason DoveandGitHub 714f68a887 add language_tag and seconds fields to search index (#1479)
* add `language_tag` and `seconds` fields to search index

* simplify
2023-10-10 20:36:50 -05:00