968 Commits

Include renames

Author	SHA1	Message	Date
timothyandClaude Fable 5.1	a7d91bf15a	fix(876): sweep session narrative out of hooks, workflows, scripts, tests and code comments; grow the detector to the process corpus ... `docs.no-session-narrative` reaches every durable artifact, but its detector scanned only `docs/**/*.md` and root markdown, and nothing had ever swept the rest. The issue named four sites from one grep and called them a floor. Deriving the population instead — a whitespace-joined sweep over every tracked file outside the detector, for the detector's own phrasings plus the attribution and review-round class #812 found — gave 453 sites in 108 files at `fb5592971`, and a second pass for phrasings the first list missed (hyphenated `round-N`, "an earlier version", "the reviewer proved") added residuals in the same files. Every site was classified with #812's three dispositions (CUT / SEVER / KEEP with its sub-kind) under the who-benefits test; the per-site manifests are on the PR. The rejected designs, tested-and-rejected fixtures, measurements and traps stay; the attribution of who found them and the round in which they were found go. The detector's population grows to `.claude/`, `.gitea/`, `.husky/` and `scripts/` regardless of extension, minus the detector and its own test (whose fixtures ARE the phrasings) and minus `scripts/tests/fixtures/` (test data, including decision-record copies — the same reasoning as the records' own exemption, and what keeps the record's depth measurement true), and `--all` lists tracked REGULAR files only — a symlink's content is its target and a gitlink has none. The #812 argument for leaving `docs/superpowers/**` in the population runs the other way here: `--diff` sees only ADDED lines, and 287 of the 453 sites were under 30 days old — this corpus is where narrative is being added, so the advisory nudge has reach. Density agrees: 56 line-mode hits over the 113 regular files the predicate admits, against 9 over 66 docs files before #812. `web/` and C# stay out on the same measurement (3 of 74 PATTERNS-matching sites, ~4,600 files). The predicate did not grow: PATTERNS matched 74 of 453 sites, and widening the word list to the attribution class is the treadmill the withdrawn parity test ran on. The population oracle is restated over segments with the new arms, the synthetic cross product gains the process heads and non-markdown extensions, a fixture witnesses that a tracked symlink is neither scanned nor counted, a `.py.bak` axis separates a by-name exemption from a `startswith` over the same tuple, and eight mutants (drop the process arm, drop the by-name exemption, exempt by `startswith`, drop or add a prefix, drop the fixtures exemption, list only markdown, drop the symlink filter, test the mode per row instead of per path) each redden it. A pre-existing silent drop in `--diff` goes with it: git tab-terminates a `+++` filename that contains a space, and the kept tab made `is_scanned_path` refuse the file with no notice — fixed, with a positive control and its own mutant. Code is unchanged by construction, measured per file type against `origin/main`: Python modules are AST-equal with docstrings stripped, except `#` lines inside the embedded fixture programs (string literals) of three test modules; workflows differ only in `#` lines inside `run:` block scalars; shell, C#, TypeScript and jq are equal with comment lines stripped. The stated exceptions: the detector and its test, 26 vitest titles that carried review-round or severity labels or a reviewer attribution (call sites whose title changed — every changed title line walked back to its `it(` / `it.each(...)(` anchor, so a `' + '` concatenation counts once), two registry note strings and the mutation manifest's prose fields. scripts/tests: 1565 passed. Web: lint, typecheck, 1319 tests green. Closes #876. Decisions-Edit: yes Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEcBoFw7ctrf3Nb7R7x7wk	2026-09-03 20:51:39 +02:00
timothyandClaude Opus 5	528383cf3a	fix(880): an absent recurrence array means unrestricted, an explicit [] is rejected (#892) ... The three recurrence arrays are read CONJUNCTIVELY by AlternateScheduleSelector.GetScheduleForDate, so an empty set matches no date. `?? []` on an omitted array therefore returned HTTP 200 while storing an alternate-schedule or template item that could never apply, silently -- while the read side (#823) already read a NULL column as the All*() sets. Absent and explicitly-empty are two different requests and get two answers: ABSENT (missing, or explicit null) normalizes to AlternateScheduleSelector.All*(), the same symbols the read side substitutes; EXPLICIT [] is rejected with a 422 naming the consequence, via RecurrenceSetBounds called from both replace handlers. The rejection lives in the handlers, not the controller, because api.ffmpeg-profile-numeric-bounds' "accept an UNCHANGED bad value" rule binds hardest here: both PUT paths are whole-list replaces, so rejecting a pre-existing empty set would make every OTHER item in the list uneditable. That comparison needs the stored row. The validated set is derived from `incoming`, so the highest-Index catch-all -- whose recurrence the handler discards -- is excluded by construction. Verified: full ErsatzTV.Tests suite green; three mutation proofs with disjoint reddened sets; live-E2E against a real instance confirmed an OMITTED property round-trips as unrestricted (the Newtonsoft missing-property chain unit tests cannot reach), an explicit [] returns the 422, and [] on the catch-all is accepted. Cross-family cold review BLOCKED the first implementation with 3 findings, all real and all fixed; re-review returned MERGEABLE. Follow-up #894 filed: the SPA can still build the empty state the server rejects. fixes #880 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-30 09:29:02 +00:00
timothyandClaude Opus 5	95b2700f09	fix(823,824): a scheduling NULL collection reads as UNRESTRICTED and is guarded at both read sites; the Elastic indexer gets its own mutation proof ... Both issues are #701 deferrals, and they land together because both rewrite the same decision record. #823 -- can a null reach one of the six collection-valued scalar columns? MEASURED against a real TvContext on BOTH providers (SQLite, and MySQL 8.4 on an ephemeral server), because the reasoning available beforehand pointed the wrong way. The two converters differ on their read side -- IntCollectionValueConverter maps null-or-blank to Array.Empty<int>(), while EnumCollectionJsonValueConverter would dereference the result of JsonConvert.DeserializeObject -- so the expectation was that a NULL row behaves differently per column. NEITHER RUNS: EF does not invoke a value converter for a NULL column at all. All six materialize as CLR null, the int converter's null-to-empty branch is dead on this path, and unguarded each .Contains in AlternateScheduleSelector throws NullReferenceException. A NULL reads as UNRESTRICTED -- the All*() sets -- not as empty. This is the whole semantic question and the first draft got it backwards. It is decided by the one NULL reachable WITHOUT any code writing one: Sqlite's 20240113140741_Add_PlayoutTemplate_DaysOfMonth adds the column with nullable:true and NO defaultValue, so a PlayoutTemplate row inserted before it holds NULL and by construction had no day-of-month restriction. Reading that as empty INVERTS the row's meaning and silently stops the template applying at all. All*() preserves it, and is how "no restriction recorded" is already represented (GetPlayoutAlternateSchedulesHandler, PreviewBlockPlayoutHandler). What does NOT decide it, and was wrongly cited in the first draft: the API request records normalize an omitted field with `?? []`, but that is a client omitting a field on a WRITE and says nothing about what a legacy database NULL meant. Two read sites, not one. Guarding only the selector would have left the entity->DTO mappers unguarded, and those feed the SPA: PlayoutScheduleEditors spreads the collection (`[...template.daysOfMonth]` -> TypeError on a JSON null) and playoutTemplateCalendar's appliesToDate -- an exact port of GetScheduleForDate -- calls .includes on it. Both mappers now substitute the SAME defaults, so the preview agrees with what is actually scheduled. Neither guard is assigned back onto the entity, which is the media.nullable-primitive-collection-mutation mechanism. Reachability, stated precisely rather than overclaimed. All six are nullable:true on both providers, but a nullable column does not produce a NULL row: five of the six were present at CreateTable, so a NULL there still needs code to write one, and on MySQL there is NO code-path-free NULL for any of the six. The write path ACCEPTS a null (SaveChanges succeeds, stores SQL NULL) but no caller supplies one today -- every production construction of the two commands goes through the request records. That is a property of the code, not a live caller; claiming otherwise would be the banned "it's AsNoTracking today" argument pointed the other way. #824 -- ElasticSearchIndex.UpdateSong had no regression test Issue option 1 (a non-network transport) shipped, and needed no new package: Elastic.Transport.InMemoryRequestInvoker is public in the pinned version and ElasticsearchClientSettings(NodePool, IRequestInvoker) accepts it, injected into the private _client the way #701 injects the Lucene IndexWriter. UpdateItems never runs `_client ??= CreateClient()`, so the injected instance is the one used. Two traps there are load-bearing, both measured: the canned response must carry an `X-Elastic-Product: Elasticsearch` header or the client's product check throws UnsupportedProductException INTO UpdateSong's catch, and an empty body fails to deserialize the same way. Either turns the fixture into a green measurement of the error path -- which is how it first failed here, caught by the ThrowOnWarningLogger. The document id is asserted as the LAST PATH SEGMENT, not by substring: the index name carries digits, so ShouldContain would stop discriminating for a song whose id collided with one. Six mutations executed, each disarming ITS OWN clause alone: - `??=` restored in ElasticSearchIndex only -> the Elastic fixture reddens on "metadata.Artists should be null but was []" while the LUCENE fixture stays GREEN. The #824 hole demonstrated, not described. - DaysOfWeek guard disarmed in the selector -> 4 red, 3 green (DaysOfMonth and MonthsOfYear unaffected). Each clause is independently load-bearing. - DaysOfMonth guard disarmed in Playouts.Mapper -> 1 red, 2 green. - Elastic dropped from the covered set / mapped to the SAME fixture as Lucene / mapped to a class with no [Test] -> SearchIndexMutationCoverageTests reddens on each. That coverage guard is the boundary fix the issue asked for: the covered set is compared against an ISearchIndex population DERIVED FROM THE ASSEMBLY. Its claim stops where the check does -- no static check can establish that a named fixture actually DRIVES its indexer, so it forces a human to look rather than proving coverage. ThrowOnWarningLogger moved to ErsatzTV.Tests/Support so both fixtures share it; the Lucene fixture's assertions are otherwise untouched, since it is a witnessed proof artifact. No production change in ElasticSearchIndex.cs -- #824 is coverage only. Docs: testing.md gains a "Provider-parity fixtures" section naming all THREE opt-in-MySQL fixtures and recording that CI runs none of them (#627); docs/README.md gains the matching task signal; guard-inventory.md's hand-written C# guard list goes from five files to six. Scheduling/Mapper.cs loses the UTF-8 BOM it inherited, per #311 fix-as-you-touch. Local gate (with the MySQL lane armed): ErsatzTV.Tests 2096 passed / 0 skipped, Core.Tests 693/1, Infrastructure.Tests 114, Architecture.Tests 7, Scanner.Tests 1504 -- 0 failures in each. scripts/tests 1228 passed / 2 skipped. dotnet format whitespace --verify-no-changes clean; BOM check over the touched set with the population COUNT asserted, because a bare zsh loop silently checks one concatenated filename. decisions_validate OK. Fixes #823 Fixes #824 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW	2026-08-29 20:02:52 +02:00
timothyandtimothy	ed8b602445	feat(735): bound the numeric FFmpeg profile fields with a 422, and expose readrate pacing (#847) ... Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>	2026-08-26 22:05:38 +00:00
timothyandtimothy	ba6a4b08aa	feat(732): On Now / Next gets a background box, and is on by default (#843) ... Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>	2026-08-26 19:28:25 +00:00
timothyandtimothy	08cd3a002d	fix(690,758): count the same query a paged handler pages (#833) ... Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>	2026-08-26 07:39:23 +00:00
timothyandClaude Opus 5	b5dee26202	fix(701): guard SongMetadata's nullable primitive collections at the read site ... Both search indexers opened UpdateSong with metadata.AlbumArtists ??= []; metadata.Artists ??= []; Artists/AlbumArtists hold the whole list in ONE COLUMN rather than being navigations. So unlike the same `??= []` idiom on Genres/Tags/Artwork all around them, the property IS the column value: assigning it on a TRACKED entity flips the entry to Modified and the next SaveChanges writes [] over a NULL column. This is the mechanism an adversarial review demonstrated in #691, which is why that issue's entity-level guard was reverted in favour of guarding at the read site. Measured rather than reasoned about, per the issue's first done-when box. Restoring ONLY the `??= []` clause (the real predecessor lines, not a hand-written mutant) reddens the new fixture on `metadata.Artists should be null but was []`; a probe variant with the first two assertions replaced by prints reports STATE=Modified and the raw column moving from NULL to "[]". Today's two feeds are both AsNoTracking (SearchRepository.GetItemToIndex and GetAllSongs), so no shipped caller loses data -- but that is a property of two callers, not of the indexer, and #691 already recorded it as a loaded gun. The fixture pins the indexer's own contract instead. Removing the assignment is not sufficient alone: it was load-bearing for the four reads below it, and deleting it by itself converts a silent write into a live throw on every untagged song. Measured by deleting only those two lines from the real predecessor file: NullReferenceException, thrown at the foreach (cited by symbol: a line number in a mutant that exists in no committed tree is unreproducible by construction). The exception type follows the read FORM, not the field -- foreach yields NRE, string.Join/ToList yield ArgumentNullException -- and this PR contains two of each, which is why no single exception-name grep characterises the class. So each site moves together with its reads: - LuceneSearchIndex.UpdateSong / ElasticSearchIndex.UpdateSong: hoist Optional(...).Flatten().ToList() locals and read those. - RefreshChannelDataHandler: the Scriban context took the raw nullable lists (the issue's second item). The shipped _song.sbntxt only does array.join, but a custom template is free to do anything. The population was derived from the MODEL rather than from the issue's file list, and the obvious derivation is wrong: "the IList<string> properties under ErsatzTV.Core/Domain" returns two of eight. It misses the six value-converted collections (ProgramScheduleAlternate and PlayoutTemplate each carrying DaysOfMonth, MonthsOfYear, DaysOfWeek), declared as plain ICollection<T> and made single columns only in Data/Configurations -- and their storage differs (comma-separated text for the int converter, JSON for the enum one), so the shared property is "one scalar column", not the serialization. No site applies `??=` to any of the six, so this defect has no instance there; whether a null can REACH one at runtime is unverified and is filed as #823 rather than asserted either way. Only the SongMetadata pair is left NULL in practice, by FallbackMetadataProvider. Every site touching either field was then swept; the remaining readers were already guarded by #691. The fixture carries two anti-vacuity guards, both witnessed: - A POSITIVE CONTROL (`writer.NumDocs.ShouldBe(1)`). Every other assertion says something did NOT happen, so all of them hold vacuously if UpdateSong never runs -- and it silently stops running if a future refactor gates UpdateItems on `_initialized`, which this fixture bypasses by injecting the writer. Verified BOTH directions: with that gate added the control fails `NumDocs should be 1 but was 0`, and with the control removed the whole test PASSES while the code under test is unreachable. - A capturing logger, because UpdateSong wraps its body in a catch that assigns metadata.Song = null -- severing a required relationship and cascading the metadata to Deleted. Without it the probe silently measures the error path; on the first run it did exactly that (a bare ILanguageCodeService substitute NPEs inside AddLanguages). The raw column helper also fails loudly on a missing row, since ExecuteScalar returns CLR null for both "NULL column" and "no such row". ElasticSearchIndex has no equivalent fixture -- it needs a stubbed transport -- so its change is by inspection against the Lucene one, and the gap is filed as #824 rather than covered by a source-text guard. The whitespace-only churn in ElasticSearchIndex.cs is the #311 fix-as-you-touch format gate: it scopes to whole changed FILES. `git diff -w` over that file shows only the two hunks above. Local gate: ErsatzTV.Tests 2006 passed / 4 pre-existing skips, Core.Tests 685/1 skip, Infrastructure.Tests 114, Architecture.Tests 7, Scanner.Tests 1504 -- 0 failures in each. scripts/tests 874 passed / 2 skipped. dotnet format whitespace --verify-no-changes clean on the four touched files, no BOM on any. decisions_validate OK. Fixes #701 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-23 00:49:57 +02:00
timothyandClaude Opus 5	2249a806c9	fix(671): review round 4 -- fix the chapter-title entity interpolation ... Final cold review of 34eee753b: the commit's own changes were confirmed correct, but it flagged a real pre-existing bug in the exact block I had just edited, and I was adding the first-ever tests for that method without covering it. `Playouts/Mapper.GetDisplayTitle`'s Song arm interpolated `{s}` -- the `case Song s` ENTITY -- into its chapter-title branch instead of `{t}`, the composed title. Song has no ToString() override, so a chaptered song rendered as the literal "ErsatzTV.Core.Domain.Song (Chapter 3)" in the playout guide, troubleshooting, media-item info and channel states. The sibling MusicVideo and OtherVideo arms are correct only because they happen to name their lambda `s`. Pre-existing on main; fixed here because it is one token inside the block this branch already touches. Two tests pin it; reverting renders the type name. Also: completed the guard on that arm (`Optional(s.SongMetadata).Flatten()`, the other half of the sibling pattern I claimed to have copied), added the new mechanism to the record's `mechanics:`, added the symptom tokens a future session would actually search for (ArgumentNullException, Artists, primitive collection, chaptered song) to `signals:`, restored the remedy sentence an earlier trim dropped, and trimmed to 59 prose lines for margin under the 60-line ceiling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 20:27:08 +02:00
timothyandClaude Opus 5	34eee753b2	fix(671): review round 3 -- sweep Artists by FIELD, correct the archaeology ... Cold review of 572737a29. Findings taken; two are corrections to my own claims. CORRECTION: "a regression this branch INTRODUCED" was wrong, and I verified the reviewer's counter-claim against origin/main before accepting it. That handler already included SongMetadata AND already routed Song there, so GET /api/v1/playlists/{id}/items was ALREADY a live 500 for a null-Artists song. This branch only made the same throw reachable on a second path. The record said so twice; both are fixed, because a wrong explanation outlives a wrong line. SWEEP: fixing one site left the mirror standing -- Playouts/Mapper.GetDisplayTitle had the identical unguarded join on a path that also eager-loads SongMetadata, so it too was live, feeding the playout guide, troubleshooting, media-item info and channel states. Guarded, with a unit test; reverting it reproduces ArgumentNullException. LibraryBrowseItemMapper already wrote `Artists ?? []`, so the nullability was known in-tree and these sites were simply unswept. Filed #691 for the remaining SongVideoGenerator dereferences on the playback path. Also: documented that the shared matrix is the RERUN predicate used as a superset for playlists (the playlist write path rejects RemoteStream today); noted the third, inert consumer ReplacePlaylistItemsHandler; added the new mechanisms to the record's `mechanics:` field; and trimmed the record under the 60-line prose ceiling -- it was tipping the corpus p90 above the ceiling and reddening the calibration test in scripts/tests, which passes again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 20:09:07 +02:00
timothyandClaude Opus 5	572737a29e	fix(671): review round 2 -- guard Song.Artists, cover the second consumer ... Cold independent review of 017ef988d. Three findings taken, one filed out. The important one is a regression this branch INTRODUCED. `SongMetadata.Artists` is a nullable EF primitive collection (JSON in one column, not a navigation) that FallbackMetadataProvider leaves unassigned when a song's tags fail to read, and `string.Join` throws ArgumentNullException on a null sequence. The rerun list previously did not load SongMetadata at all, so the throw was unreachable there; adding the include promoted it to a live 500 that would have failed the whole page. Confirmed by reverting the guard: ArgumentNullException, parameter 'values'. The file header claiming every member was guarded was false. The empty case is filtered too, so an artist-less song loses its bare " - ". Second: `GetPlaylistItemsHandler` had no handler-level test at all (its controller tests stub the mediator), so the RemoteStream include added last round was discharged by inspection -- the same method that produced #671. It now runs the same 13-type matrix via a shared SelectionSeedData; removing the include fails that matrix. Third: dropped the dead `(i as Season).SeasonMetadata` include leg -- the Season projection reads Show.ShowMetadata and the scalar SeasonNumber, never SeasonMetadata. Filed #690 for the pre-existing, out-of-scope finding: the paged TotalCount ignores the search query, so the SPA renders empty pages. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 19:33:52 +02:00
timothyandClaude Opus 5	017ef988d0	fix(671): review round 1 -- pin exact names, complete the playlist include ... Cross-family (Codex) adversarial review of 8523088ce. All three findings taken: - The name assertion only checked "not a placeholder", so it could not see a missing NESTED include leg: dropping Episode -> Season -> Show still renders "s00e04 - Selected episode", which contains no placeholder marker and passed. Now every type pins its whole expected string; re-removing that leg fails, as verified before restoring it. - Widening the shared switch with a RemoteStream arm put `GetPlaylistItemsHandler` one include short -- it loaded metadata for the other nine types, so playlist RemoteStream names alone would have degraded to "???". - `?? 0` rendered an unloaded Season as "s00", which conventionally means Specials and so fabricated plausible-looking real data; it now renders "s??". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 19:00:08 +02:00
timothyandClaude Opus 5	8523088ceb	fix(671): resolve rerun-collection selections through one shared include chain ... The paged list handler eager-loaded nothing, so `ProjectToViewModel` read four unloaded navigations and every row of every collection type projected a null selection. Because the selected id and the display name are read off the SAME navigation, this dropped the id too -- the harm is not an unlabelled badge but an editor that round-trips a null and clears the user's stored selection. The by-id handler loaded metadata for only four of the ten selectable media types: Song/OtherVideo/Image/RemoteStream returned a null-ish selection and Episode/MusicVideo threw an NRE that surfaced as a 500. Fixed at the boundary rather than per call site: - `RerunCollectionQueryExtensions.IncludeSelectionDetails()` is now the single include chain, called by both handlers, joining the existing `ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails()` precedent (#229). Artwork legs are deliberately omitted -- this projection reads only ids and titles. - The media-item switch was duplicated verbatim for RerunCollection and PlaylistItem; both now call one `ProjectMediaItemToViewModel`, which handles `RemoteStream` (via a new `ProjectToNamedViewModel`, since the existing `ProjectToViewModel(RemoteStream)` returns an unrelated type) and never falls through to null -- an unknown subtype keeps its id and takes a conspicuous name, because throwing would fail a whole paged GET over one bad row. - Every metadata navigation in `MediaItems.Mapper` is now read through `Optional(...).Flatten()`, so an un-included nav degrades to "???" instead of being a latent 500 for whichever caller loads least. Tests enumerate all 13 supported CollectionTypes for both handlers, with the matrix derived from `IsSupportedSelectionType` so a newly-supported type joins it automatically, plus a completeness guard on the set. Each mechanism was removed in turn and confirmed red first. fixes #671 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-28 18:48:24 +02:00
timothy	dda98efcc4	fix(668): review round 3 -- MySQL does NOT over-match; correct the claim everywhere ... Review BLOCKED on a false CI claim I copied from the sibling fixture ("CI sets ETV_REQUIRE_MYSQL_TESTS=1"). Nothing sets it; the doc now says plainly that CI does not arm this lane and points at ersatztv#627. That was the blocker. Chasing the reviewer's second finding then overturned something bigger. It predicted that seeding an unaccented "Edith" would make the in-memory ordinal filter load-bearing on MySQL, since utf8mb4_0900_ai_ci treats é as e. Mutation test says otherwise: with the filter deleted the MySQL test stays GREEN. Measured against a live 8.4 to find out why: LOWER(Name) LIKE 'é%' (literal) -> Édith AND Edith LOWER(Name) LIKE @v (ai_ci variable) -> Édith AND Edith LOWER(Name) LIKE @v COLLATE _bin -> Édith only the EF query, executed -> Édith only The driver binds the pattern with a BINARY collation, so the executed comparison is accent-SENSITIVE and MySQL does not over-match at all. MySQL's correctness rests on its Unicode-aware LOWER(), not on the collation. My earlier probe used a LITERAL pattern -- a different query from the one the code runs -- and I wrote its result into the handler comment, the decision record and the PR body. All three now say what actually happens, and the record carries the lesson: measure the query the CODE runs, not one you type. The "Edith" row stays as a near-miss control, with a docstring that says what it does and does not prove rather than the over-match story it was added for. Also moved EnsureCreatedAsync out of [SetUp]: NUnit skips [TearDown] when [SetUp] throws, so a mid-create failure would strand the database. Decisions-Edit: yes	2026-07-27 21:52:16 +02:00
timothy	7ca058f83b	fix(668): review round 2 -- remove a second false comment, harden a vacuous assertion ... Re-review of the round-1 fix commit returned MERGEABLE with three findings, all about claims rather than behaviour. All three applied. 1. A stale FALSE parenthetical survived round 1. The docstring on Unicode_Fold_Agrees_With_The_Ordinal_Filter still claimed it catches "one that stops filtering the extras out". It does not. Mutation-verified: delete the Where in FilterSortTake and all EIGHT cases stay green, because each is either a positive SQL alone returns or an ASCII-query negative SQL alone rejects. The same mutation turns the new over-match test RED, so the pair does cover both directions -- but only the corrected wording says so. This is the same species of error round 1 fixed, one paragraph above it; swept by subject this time. 2. Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter asserts an EMPTY result, so it passes vacuously if the fold never runs. Its premises are now asserted explicitly (the query is non-ASCII, and ToUpperInvariant maps ſ to S), so a fold that quietly stopped mapping ſ would fail rather than go green for the wrong reason. 3. The comment on IsSqlite overstated its enforcement. ProviderStaticsWiringTests parses the composition roots for ASSIGNMENTS only; nothing mechanically stops a read of TvContext.IsSqlite here. The real reason stands -- such a read would falsify that test's prose exemption while the test stayed green -- so the comment now says that instead of implying a guard that does not exist. Decisions-Edit: yes	2026-07-27 21:02:24 +02:00
timothy	ac67c9ee74	fix(668): review round 1 -- make two guards actually guard ... Independent review found both new guard tests weaker than they read. 1. A false docstring. It claimed the SQL stage "genuinely returns 'ſweet' for q='S'". It does not: 'S' is ASCII, so ContainsNonAscii is false and the fold branch is SKIPPED. Those three negative cases exercise the ASCII fast path, which is worth pinning but is not what the comment said -- and the consequence was that NO test drove a row through the fold for the ordinal filter to discard, i.e. the harmless over-match direction the whole design rests on was untested. Comment corrected and Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter added (stored "Sword", q="ſ" -> fold runs, SQL pattern S%, SQLite returns the row, filter drops it, response empty). 2. Unicode_Fold_Escapes_Like_Wildcards could not fail if the %/_ escaping it names were deleted -- the in-memory filter masks the over-match, so the counts stay right. The escaping's real role is preventing LIMIT crowding, so Unicode_Fold_Escaping_Prevents_Limit_Crowding pins that instead. Verified by mutation: with the %/_ replaces removed the new test fails while the original two still pass. Also: the crowding residual in the decision record was attributed to MySQL alone; the SQLite fold shares it in principle, so "no accepted loss" is narrowed to mean no unreachable VALUE rather than a guaranteed count. And a comment says why the provider check is derived per-context instead of reading TvContext.IsSqlite (that static is scoped host-only by ProviderStaticsWiringTests, and reading it here would falsify the exemption). Refs #668 Decisions-Edit: yes	2026-07-27 20:50:48 +02:00
timothy	05542946ad	fix(668): reach accented facet values via a registered Unicode fold on SQLite ... SQLite's LOWER() folds ASCII only -- lower('Édith') is 'Édith' unchanged -- so the EF-sourced facet fields UNDER-matched any stored value whose prefix carries an uppercase non-ASCII character. An under-match is unrecoverable: no later stage can reintroduce a row SQL never returned. Adds a SECOND, ADDITIVE query taken only when the provider is SQLite and q contains a non-ASCII character: raw Dapper SQL folding through etv_upper(), a SqliteConnection.CreateFunction scalar implementing ToUpperInvariant. Every other case -- all-ASCII q, and MySQL for all q -- runs the existing EF query byte-identically. MySQL needed no change and gets none: verified on MySQL 8.4 that its LOWER() is Unicode-aware and its ci collation makes the predicate OVER-match, which the existing ordinal filter already discards. The fold is ToUpperInvariant because OrdinalIgnoreCase equality is a strict SUBSET of invariant-uppercase equality, so the SQL stage yields a superset of the final filter's matches and can never under-match. Note OrdinalIgnoreCase is NOT "invariant-upper then ordinal": ToUpperInvariant('ſ') is 'S', yet "ſweet".StartsWith("S", OrdinalIgnoreCase) is false. Tests pin that. No migration, no model change; both provider snapshots are untouched. Refs #668 Decisions-Edit: yes	2026-07-27 20:36:28 +02:00
timothy	fc3ede09bc	docs(578): the retracted claim survived in 9 places, including the record title and rule ... Comment- and docs-only; verified no non-comment line changed in any .cs. I reported last round that I had "classified every surviving hit". That was false, and the false confidence is the expensive part: a confidently-stated "I checked everything" stops anyone else checking. The retracted wording survived in nine places, two of them the record's title and rule: — and the catalog copies rule: verbatim, so the generated entry point and the record disagreed semantically while docs/decisions.md said the correct thing. Root cause of the miss, because it will recur otherwise: I built the sweep term list from the DELETED MECHANISM's vocabulary (LIKE, superset, keyspace, anchor, over-match) and never added the RETRACTED CLAIM's own words. "no predicate", "bound on work", "index entries", "no gap" and "holds in memory" were never grepped. After a retraction the subject list has to include the words of the thing being retracted, not just the thing already deleted. Second, worse: my first attempt at this round's sweep printed nothing for every term and I nearly read that as "all clear". zsh does not word-split an unquoted $FILES, so grep received one giant non-existent path — and the `|| echo "(none)"` never fired because the pipeline's exit status was sed's. Same failure shape as the bug arc itself: a check reporting success while examining nothing. Re-run with a proper array plus a control term ("SongMetadata" -> 42 hits) so an empty result is distinguishable from a broken grep. Fixed all nine, replacing "no predicate" with the seekable-cursor-vs-residual distinction already written correctly elsewhere: - handler: the "real bound on work" claim, the short-page rationale - SearchFieldValuesQueryShapeTests: "ANY predicate" + "reads exactly n index entries", and added what the test can and cannot pin (a SQL string, not a plan / visibility work / payload I/O) - GetSearchFieldValuesHandlerTests: "no gap between what the engine looks at and what it hands back", and the current-behaviour comment - record title, rule:, attempt-5 table row; api-conventions - regenerated docs/decisions/README.md so catalog and record agree again Tenth item, the same overclaim one level down and it survived the first retraction: the row bound was said to cap what the process holds in memory. It does not — payload width is unrestricted and one JSON array can contain arbitrarily many strings, each of which may enter the in-memory distinct set. It caps logical rows returned/materialized and round-trip count, nothing about bytes. Added as a third struck-through bullet next to the other two retractions.	2026-07-27 03:10:36 +02:00
timothy	5f73cd4482	docs(578): scope the bound to LOGICAL ROWS — physical work is not bounded, and I claimed it was ... Comment- and docs-only. Verified: the diff for both .cs files contains no non-comment line. HIGH (claim). "LIMIT n reads exactly n index entries and n rows" is false, and the bounded-physical- work reading it implied is false with it. Two mechanisms, both retracted explicitly in the record rather than quietly reworded: - MySQL purge lag. Deleted clustered-index records survive until purge runs and a range scan still traverses them, so returning 2,000 VISIBLE rows can touch far more index records. Deletion history therefore STILL affects physical work — precisely what attempt 4's failure was supposed to have made irrelevant. Attempt 5 removes the LOGICAL dependence on Id distribution; it does not make physical work independent of deletion history. - Unbounded row width. Artists/AlbumArtists are unrestricted TEXT/longtext and both SQLite and InnoDB spill to overflow pages, so a row count implies neither a byte count nor a page-read count. The guarantee is now stated as exactly what it is: at most 20,000 LOGICAL rows returned/materialized, and at most 10 round trips (11 for artist). The 392 KiB measurement is labelled one data point on one library whose credits average ~20 B/row, with an instruction to re-measure rather than extrapolate for long credit lists or MySQL over a network. Also noted: the query-shape test pins the SQL STRING — it cannot pin a plan, MVCC visibility work or payload I/O, and on MySQL using the index to satisfy ORDER BY is an optimizer choice, not a semantic guarantee. Precision fix, and the reviewer is right that the sloppy version misleads: "any predicate defeats LIMIT" is wrong, since the query's own cursor is a predicate. The distinction is a SEEKABLE PREDICATE ON THE ORDERING KEY (positions the scan, never discards a row) versus a RESIDUAL predicate (discards rows the engine already produced, so LIMIT bounds survivors and says nothing about how many were produced). Restated in the handler, the record and api-conventions. MEDIUM. docs/decisions.md still advertised "a superset LIKE pre-filter that may over-match but never under-match" for list-valued columns. That is the documented entry point for convention lookups, so a maintainer starting there was told to preserve exactly what round 4 deleted, in direct contradiction of the linked active record. Fixed. LOW. Two test comments read as current: "however many non-matching rows" (false past the ceiling — now bounded and cross-referenced to the test that pins that boundary) and the ampersand case "widens to the bare anchor" (no prefix predicate exists; kept because it is the input shape that broke the old scheme, now labelled as such). Fifth consecutive round of stale text, so this sweep was done by grepping the subject across handler, tests, record, decisions.md, api-conventions.md and the endpoint description: LIKE, ESCAPE, pre-filter, superset, over-match, under-match, anchor, keyspace, window, candidate, row cap, index entries, 392, and every deleted constant name. Every surviving hit is now either current-and-correct or explicitly framed as history.	2026-07-27 03:10:36 +02:00
timothy	373956fcee	fix(578): delete the SQL predicate — LIMIT only bounds work when there is nothing to discard ... BLOCKER. Attempt 4 bounded the Id KEYSPACE, and keyspace is not rows. Delete 20,000 historical rows, put one song at Id 20001, query artist?q=que: the walk burned all ten windows on empty ranges and returned [] for a table containing exactly one row. Capacity fell linearly with deletion ratio and no ratio was safe — one placed gap hides the next match. My record called that "heavily fragmented" and the endpoint description said loss happens "on a very large library"; the one-row example disproves both. Option A. The query now carries NO predicate at all — no LIKE, no LOWER, not even IS NOT NULL: SELECT Id, Artists AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch That is the whole fix, and it is the point. LIMIT truncates what survives a predicate, so with any predicate present it bounds the OUTPUT and says nothing about the WORK; the engine may evaluate and discard arbitrarily many rows first. Stripped to a bare primary-key range, LIMIT n reads exactly n index entries and n rows — independent of sparsity, deletion history or where the gaps fall. All selectivity moves into memory. A short page can now only mean exhaustion, which is precisely what it could not mean while a predicate was present. Four attempts, four wrong quantities: the result (a fixed budget the over-matching pre-filter starved), candidates returned (a no-match query must evaluate every eligible row before returning an empty page), keyspace width (above), and finally actual rows. The record carries the table; it is worth more than the code. Deleting the predicate deletes a whole bug family with it: the JSON-escape reasoning, the narrow-only-on-verbatim-ASCII rule, the exhaustive Unicode sweep that proved it sound, the ESCAPE '/' portability workaround, and the may-over-match-never-under-match invariant that turned out to be conditional on something untrue. SearchFieldValuesPrefilterSupersetTests is deleted entirely; the one assertion worth keeping — that the SQL has no predicate — moved to the query-shape suite, which pins the SQL string exactly so "just a cheap filter" fails a test instead of silently unbounding the walk. Measured cost of no server-side narrowing, on a seeded 20,000-song library (in-memory SQLite): worst case (no match, full walk) 20,000 rows / 10 round trips / 391.9 KiB / 119ms SQL, ~40ms warm end-to-end. Empty q, dense and non-ASCII prefixes all stop on page 1 at ~39 KiB and ~40ms. Judged acceptable for a debounced typeahead against a local file. If it ever is not, the answer is #669, not reintroducing selectivity — the record says so explicitly. Also fixed: - Round-trip count was advertised as 10; it is at most 10 for album_artist and 11 for artist, which also runs its EF query. The MAX(Id) probe is gone with the keyspace scheme, so there is no extra scalar call. - The duplicated-formula ceiling test is deleted rather than rewritten. It re-implemented the loop's arithmetic and would have passed through an off-by-one or a stall in the real loop; the dense integration tests carry that coverage. Its MaxVisited >= Window assertion was a style constraint in correctness clothing. - Stale text swept by grepping the mechanism nouns rather than re-reading: candidate/keyspace/ pre-filter/superset/row cap/LIKE/ESCAPE and the removed constant names, across handler, tests, record, api-conventions and the endpoint description. The two surviving "pre-filter" mentions are deliberate history. Test comments that rendered escaped non-ASCII as literal characters (which contradicted the raw-storage assertion in the same file) now show the escape text. New test List_Valued_Walk_Reads_Live_Rows_Regardless_Of_Id_Density reproduces the one-row killer and fails against attempt 4.	2026-07-27 03:10:36 +02:00
timothy	fbc7b2a1dd	fix(578): bound the Id KEYSPACE — LIMIT cannot bound a query that matches nothing ... BLOCKER. Round 3's ceiling counted LIKE-positive candidates, which is not the quantity that needed bounding. To return an empty page the engine must first evaluate every eligible row, so a no-match query came back with rows.Count == 0, ended the walk having counted zero against the ceiling, and had already inspected the entire table. Round trips and materialized rows were bounded; database work was not. Worse, a dense widened prefix could materialize 20x the candidates and make 10x the round trips of round 1 — a regression dressed as a bound. Third time bounding the wrong quantity: revisions 1-2 bounded the RESULT (a fixed LIMIT budget the widened pattern starved), revision 3 bounded the CANDIDATES, and neither bounds what the database LOOKS AT. Now the Id range is closed on both sides — `Id > @AfterId AND Id <= @AfterId + @Window` — so each round trip is a primary-key range scan of known width. The LIKE still decides what comes back; it no longer decides how much gets looked at. The walk advances by the WINDOW, never by what returned (an empty page means "nothing matched in this stretch", not "exhausted"), and there is deliberately no LIMIT in the SQL — the window caps the row count, and a LIMIT would only restore the illusion that it is doing the bounding. One indexed SELECT MAX(Id) up front stops the walk burning windows on empty keyspace. Result: at most 10 round trips and 20,000 rows inspected for any q, matching or not. New test `List_Valued_Walk_Cannot_Inspect_Past_The_Bound_When_Nothing_Matches` covers the case every earlier bound test missed — they all gave the pre-filter something to return. With a narrow pattern and a needle at row 30,000, revision 3 returns the needle (right answer, unbounded work) and this revision returns []. Paired with a positive control at row 20,000 inside the bound. The ceiling's remaining false negative (20,000 "zzz" rows then "éclair" at 20,001 -> []) is kept and still pinned: it is the documented bounded-best-effort contract at a rarer threshold, not a defect. Stale/false comments, several of them repeat offenders: - The handler still used "Zulu"/"Éclair" as the truncation example, where both orderings pick "Zulu". I had corrected the test and left the comment. Now "Zulu"/"apple" in both. - "leans entirely on the row cap" described a mechanism that no longer exists. - The escape example rendered as literal Édith/É/é rather than the escape TEXT (Édith, É, é) in both the handler and the record, contradicting the explanation it was giving. - "every accented artist was unsuggestable" overstated it: exactly three of the nine pinned cases fail 1b78dc9e (those where query and stored casing differ, so the escape texts diverge); the other six pass. The record now says what the test comment already said. - The record claimed the ordinal switch left response SETS unchanged. False — ordering runs before Take(limit), so with "Zulu"/"apple", empty q and limit=1 the comparer changes which value survives. - The_Scan_Ceiling_Is_A_Whole_Number_Of_Batches was a style preference wearing a correctness costume (a 20,500 ceiling is perfectly safe — the final window clamps). Replaced with the invariant that actually matters: the walk's own arithmetic terminates and lands exactly on the ceiling.	2026-07-27 03:10:35 +02:00
timothy	a37847e509	fix(578): a fixed row budget turns an over-matching prefilter into a false negative ... BLOCKER 1. The invariant a883e5f0 established — "the pre-filter may over-match, it must never under-match" — is only sound while the candidate set is not truncated. It was truncated. A non-ASCII or JSON-escaped prefix collapses the pattern to the bare `%"%` anchor, so every row becomes a candidate, and `ORDER BY Id LIMIT 1000` then spent the whole budget on rows that could not match. Seed 1000 songs by "zzz", put the only "éclair" in row 1001, ask for album_artist?q=é: a883e5f0 returns [], while 1b78dc9e returned "éclair" because its (separately broken) tighter pattern kept the candidate set small. Neither revision was correct — the old one under-matched at the pattern, the new one under-matched at the cap. Widening a predicate under a fixed budget starves it. So the budget is gone. Candidate rows are now walked keyset-paged on Id (`Id > @AfterId … ORDER BY Id LIMIT @Batch`), continuing past non-matching candidates and stopping on the first of: enough distinct exact matches for `limit`, a short page (source exhausted), or a 20,000-candidate-row ceiling in 2,000-row batches. The bound is on effort; it no longer silently decides the result, and the lossy case needs 20,000 rows that already passed the pre-filter before it bites. BLOCKER 2. The endpoint description and the record's rule claimed ordinal matching/dedup/ordering endpoint-wide. False for EF-backed fields: the database runs LOWER/DISTINCT/ORDER BY/LIMIT before any ordinal code, so `genre?q=é` still misses a stored "Éclair" on SQLite. Both are now scoped to the final in-memory stages, and the underlying gap is referenced as #668 rather than described as fixed. #669 (normalized SongArtist table) is referenced as the follow-up for the scan cost. Accuracy corrections to my own claims, all verified by re-running the mutations: - Only THREE of the nine Unicode cases fail 1b78dc9e (é/édith/BJÖRK — where query and stored casing differ, so the escape texts diverge); the other six pass it. The comment said all nine. They stay as continuity coverage, now labelled as such rather than as regression guards. - Ordering_Is_Best_Effort used "Zulu"/"Éclair", where DB and ordinal orderings BOTH pick "Zulu" — it could not demonstrate the divergence it claimed. Now "Zulu"/"apple", which actually diverges: ordinal ranks "Zulu" first, the DB ranks "apple" first, and limit=1 returns ["apple"]. The record sentence was false and is corrected. - The record printed literal "Édith"/"é" where it needed to show the escape TEXT (Édith, é), contradicting the very explanation it was giving. - Corrected the cost claim: the leading wildcard forces scan ACCESS, but each page stops once it has filled @Batch, so a dense query finishes early — it is not necessarily a full table scan. - The Unicode sweep is labelled a PROOF OBLIGATION: it is revision-independent and passes every revision, which is correct for what it is but must not read as regression coverage. One process note: the new record's frontmatter had a lone apostrophe inside a single-quoted YAML scalar ("SQLite's"). decisions_validate.py's hand parser accepted it; scripts/tests caught it.	2026-07-27 03:10:35 +02:00
timothy	1641ca8305	fix(578): the LIKE prefilter under-matched every accented artist; make the superset provable ... Review of 1b78dc9e found the pre-filter's correctness claim was false, and the claim was in the decision record as well as the code. F1 (high). The pattern JSON-encoded the whole query prefix on the reasoning that the stored text escapes non-ASCII, so encoding the prefix the same way would line up. It does not: SQL LOWER() lowercases the *escape text* (`É` -> `é`); it cannot case-fold the codepoint that escape denotes. So `q=é` built `%"é%`, the stored `Édith Piaf` never matched, and the row was discarded before the in-memory filter could accept it. Every accented artist — Beyoncé, Björk, Sigur Rós, Édith Piaf — was silently unsuggestable, which in a music library is the common case. The invariant that was missing, now stated in the code: the SQL pre-filter is an OPTIMIZATION. It may over-match; it must never under-match. Correctness lives in the in-memory filter. So the pattern now narrows only on the leading run of characters the JSON writer stores verbatim and stops at the first character it cannot prove — `q=Beyoncé` still narrows on `beyonc`, `q=é` narrows on nothing and leans on the row cap. Soundness rests on two facts now asserted by exhaustive computation rather than argued: no non-ASCII codepoint in U+0080..U+10FFFF OrdinalIgnoreCase-equals a printable ASCII character (false for InvariantCultureIgnoreCase, which folds ~190 — the choice of Ordinal is load-bearing), and the exact set of ASCII the encoder escapes. F1b. `UseRequestLocalization` honours Accept-Language, so the culture was caller-controlled and `ToLower()` plus the default linguistic `StartsWith(string)` let a header change the answer. Comparison is now OrdinalIgnoreCase and ordering StringComparer.Ordinal throughout — including the shared FilterSortTake that state/video_dynamic_range/content_rating also use. Sets unchanged, order now ordinal rather than culture-dependent. F2. The merge comment asserted an exactness the code does not have: sources truncate by their own ordering (DB collation / primary key), not the merge's, so a dropped value can outrank a survivor. Comment and record now say best-effort, exact only below the truncation points. F3/F4. The cap now rides `ORDER BY Id` rather than the JSON column: MySQL sorts TEXT by only max_sort_length bytes, so the old ordering was not deterministic there, and sorting the whole matching set was avoidable work. What the cap still does NOT bound is the scan — a leading-wildcard LIKE cannot seek an index — so that cost is now documented as accepted, with a normalized `SongArtist` table named as the follow-up candidate rather than left implicit. Every clause above is covered by a test verified to FAIL when that clause is mutated (old pattern builder: 5 red; culture chain: 3 red; cap=3 / cap=limit / ORDER BY json / no cap: red each). F5. Converted to a proper supersession. The old record did not merely hold a stale fact — it recorded song/music-video credits as an "intentionally-uncovered gap" and album_artist as unsupported, and this reverses that call, which `docs.decision-lifecycle` says is never a line-edit. `api.search-field-values` is archived with its original prose restored, and `api.search-field-values-sources` replaces it carrying the whole endpoint contract.	2026-07-27 03:10:35 +02:00
timothy	cd6f36185c	feat(578): artist typeahead covers music-video and song credits; album_artist stops 404ing ... `GET /api/v1/search/fields/{name}/values` sourced `artist` from `ArtistMetadata.Title` only — entity artists — so the free-text credits that `LuceneSearchIndex` also writes to the `artist` field (`MusicVideoArtist.Name`, `SongMetadata.Artists`) produced no suggestions, and `album_artist` 404'd outright. `MusicVideoArtist` turned out to be a real entity table, so it just joins the existing server-side pipeline as a `Concat` — one bounded `UNION ALL` + `LOWER(...) LIKE ... LIMIT` on both providers. `SongMetadata.Artists`/`AlbumArtists` are the hard case: EF 9 maps them as primitive collections (one JSON array per row in a single column), and neither provider can project the elements server-side — SQLite needs the SQL APPLY operator it lacks, Pomelo MySQL 9.0.0 has no primitive-collection support at all. Both failures are now pinned by a test, so a provider upgrade that fixes them shows up as a red rather than as a stale workaround. For those columns the handler pre-filters on the raw JSON (`LOWER(col) LIKE '%"<encoded-prefix>%' ESCAPE '/'` — a deliberate superset, since it matches a row and not an element), caps the rows at 1000 with an `ORDER BY` that makes the truncation deterministic, then splits and exact-filters in memory. Provider portability is by construction rather than by trusting `LIKE`: the prefix is JSON-encoded before matching (which is how it is stored, and which makes the pattern pure ASCII, so SQLite's ASCII-only `lower()` and MySQL's Unicode-aware `LOWER()` agree) and lowercased in C#, so the match is correct under a case-sensitive MySQL collation as well as a case-insensitive one. The escape character is `/`, never `\`: `ESCAPE '\'` is not a portable SQL literal. An empty `q` stays supported for these fields — the row cap already bounds it, and a non-empty-`q` carve-out would make one group of fields behave differently for the same client code. Docs: new `api.search-field-values-list-columns` record (additive sibling — the base record's rule still reads true; only its body's "intentionally-uncovered gap" claim was stale), plus `api-conventions.md`, `spa-conventions.md` §12 and the regenerated `v1.json`. Decisions-Edit: yes	2026-07-27 03:10:35 +02:00
timothyandtimothy	65c0e09179	feat(415): per-channel fault detection — server-derived health object + Problems filter (#581) ... Closes #415. Server-derived health object on the channel list + detail DTOs (built-timeline detection, kind-agnostic across all 5 PlayoutScheduleKind; assessable gate keyed to the owning channel's mode), single "Problems" SPA filter with per-fault badges. Supersedes #72's api.channel-health-signal decision. Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-23 20:45:19 +00:00
timothyandClaude Opus 4.8	7e6861a928	fix(434): accurate endpoint description (DB-sourced) + tests for case-insensitivity, tag exclusion, distinct ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:55:32 +02:00
timothyandClaude Opus 4.8	fb5f609cef	rework(434): source facet typeahead from DB distinct values, not Lucene analyzed tokens ... The Lucene term dictionary stores lowercased word tokens for analyzed text fields ("Science Fiction" -> science/fiction), so the typeahead was suggesting fragments instead of whole values. GetSearchFieldValuesHandler now injects IDbContextFactory<TvContext> and resolves an explicit per-field-name distinct-values query (genre/studio/director/writer/actor/ artist/tag/network/collection/video_codec/album), with state and video_dynamic_range computed in memory and content_rating split on '/' to match what search actually matches on. title/show_title/album_artist have no distinct source and now correctly 404 (free-text fallback), same as before. Reverts the GetFieldValues additions to ISearchIndex/ LuceneSearchIndex/ElasticSearchIndex back to their pre-#434 state (BOM stripped per #311, otherwise byte-identical). Endpoint shape, DTO, controller, and OpenAPI are unchanged (no diff from ./scripts/update-openapi.sh). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:44:15 +02:00
timothyandClaude Opus 4.8	b4ae0bde18	feat(434): distinct-values search endpoint (text fields, Lucene term enumeration) ... Adds GET /api/v1/search/fields/{name}/values?q=&limit= — the backend slice of the visual rule builder's facet-value typeahead (#434). Enumerates distinct Lucene term values for a text field via MultiFields.GetTerms + TermsEnum, filtered by a case-insensitive prefix, limit clamped to [1,50]. 404s when the field is absent from SearchFieldCatalog or is not type "text". ElasticSearchIndex (the optional external backend) throws NotSupportedException for this method — its text fields are analyzed, not keyword-mapped, so a terms aggregation isn't safe to guess at without verifying against a live cluster. Regenerated OpenAPI trio (v1.json, v1.d.ts, endpoint-index.md). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-23 20:43:27 +02:00
timothyandtimothy	8b9a7ed541	feat(414): stamp immutable Channel.Origin (auto-tuned vs user-created) and surface it (#575) ... Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-23 18:12:25 +00:00
timothyandtimothy	7a7c611267	fix(458): reject duplicate names on playlist &amp; playlist-group rename (#576) ... Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-23 18:07:43 +00:00
timothyandtimothy	0c063c23fb	harden(421,559): percent-encode access_token in IPTV URLs, redact from logs, no-store on tokened manifests (#574) ... Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-23 16:30:59 +00:00
timothy	9ea2750cbf	feat(392): expose ProgramSchedule.padToNearestMinute on the REST API	2026-07-23 08:03:31 +02:00
timothy	7757814766	feat(74): channel graphicsElementIds + graphics builtIn; regen OpenAPI	2026-07-22 22:11:46 +02:00
timothy	d6652dbe13	feat(74): selector emits channel-level graphics elements as a base layer	2026-07-22 22:11:46 +02:00
timothyandClaude Opus 4.8	1a3c8e277f	feat(297): add channelId to PlayoutListItemResponseModel; SPA reset keys directly ... Added ChannelId to PlayoutNameViewModel and all 6 construction sites (Mapper, GetPlayoutByIdHandler, and the Update{,Scripted,ExternalJson,Sequential} PlayoutHandler commands), plus the list DTO PlayoutListItemResponseModel and the PlayoutController list projection. Regenerated OpenAPI (v1.json) and the TS client (v1.d.ts); endpoint-index.md unchanged (no endpoint/operation delta). Simplified PlayoutsScreen resetSelectedChannel to key directly on selectedSummary.channelId instead of resolving via channelStates. Updated controller + SPA tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-22 18:57:54 +02:00
timothyandClaude Opus 4.8	f8ae4d62ab	fix(552): mint a short-lived JWT so the SPA reaches /iptv/* under JWT auth ... Under a JWT-enabled deployment (JWT:IssuerSigningKey set), /iptv/* is gated by ConditionalIptvAuthorizeFilter and the "jwt" scheme does not accept the SPA's ctv-session cookie, and nothing minted a JWT for the browser. So the #60 channel preview was declared Unavailable and could not run at all. Add GET /api/v1/auth/iptv-token (session-gated, on the [IgnoreApi] AuthController): mints a short-lived global token via JwtHelper.GenerateBrowserToken (60 min default, JWT:BrowserTokenLifetimeMinutes override), 204 when JWT is disabled. The SPA's new withIptvToken(url) helper appends it as ?access_token= to the manifest URL (a no-op when JWT is off), used by the channel-preview panel and the troubleshooting screen. Mapper.GetPreview drops its iptvJwtEnabled -> Unavailable guard; preview is now JWT-agnostic. Live-E2E under JWT: /iptv manifest 401s without a token and passes with a valid one (garbage token -> 401); token endpoint 401s anonymous, mints with a session. Honest finding: the issue's point 2 (troubleshooting screen broken under JWT) does not reproduce -- its live.m3u8 is static-served (UseStaticFiles at /iptv/session), outside the JWT filter, so it was never gated. The withIptvToken call there is a harmless defensive no-op. Docs: security.iptv-browser-token (api-auth-security.md), amended api.channel-preview-capability, spa-conventions §5b. No OpenAPI change (IgnoreApi + unchanged Preview schema). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-22 17:21:05 +02:00
timothy	6000356739	fix(60): declare disabled-channel and no-playout as Unavailable preview causes ... GetPreview keyed only on StreamingMode + JWT, so a disabled channel or one with no playout was declared Available and then failed confusingly (404 from IptvController, or an indefinitely-blocking manifest request). Extend it to take isEnabled + playoutCount and check JWT, then disabled, then no-playout, before falling back to the existing mode-based rules; order is documented in a comment. Also corrects a stale comment in ChannelPreviewResponseModel.cs that claimed a C# string generates a TypeScript union (it does not).	2026-07-21 23:13:27 +02:00
timothy	26330cf04c	fix(60): strip UTF-8 BOM from GetAllChannelsForApi.cs ... Trips the CI Formatting gate (CHARSET). Refs #60	2026-07-21 23:13:27 +02:00
timothy	e781bd03be	fix(60): relocate ChannelPreviewTests to ErsatzTV.Tests, fix call sites ... Review findings on the Task 1 commit (f6ff3b63): 1. Mapper.GetPreview is internal to ErsatzTV.Application, granted only to ErsatzTV.Tests by convention (one-assembly-one-test-project). Move ChannelPreviewTests.cs from ErsatzTV.Core.Tests to ErsatzTV.Tests/Application/Channels, and revert the second InternalsVisibleTo entry added to ErsatzTV.Application.csproj for ErsatzTV.Core.Tests. 2. GetAllChannelsForApiHandlerTests.cs constructed `new GetAllChannelsForApi()` at three sites, which no longer compiles now that the record requires IptvJwtEnabled. Pass IptvJwtEnabled: false at each site (all three tests are about plain channel listing/logo mapping, not JWT). ChannelController.GetAll's missing argument remains, deliberately, for a later task.	2026-07-21 23:13:07 +02:00
timothy	e4aa28b127	feat(60): server-declared channel preview capability ... Adds ChannelPreviewResponseModel + ChannelPreviewAvailability constants and Mapper.GetPreview(streamingMode, channelNumber, iptvJwtEnabled), threaded through ProjectToResponseModel's new iptvJwtEnabled parameter and GetAllChannelsForApi's new IptvJwtEnabled property. Also adds ErsatzTV.Core.Tests to ErsatzTV.Application's InternalsVisibleTo list (was ErsatzTV.Tests only) so the new ChannelPreviewTests can call the internal Mapper methods it's testing. ChannelController.GetAll is left failing to compile (Task 2 wires JwtHelper.IsEnabled in at the controller).	2026-07-21 23:13:07 +02:00
timothyandClaude Opus 4.8	928784ba48	fix(135): from-lineup advanced overrides can express "clear to none" ... CreateChannelFromLineupHandler resolved every advanced override with advanced.X ?? template.X, so null always meant INHERIT and a channel could not drop a template-set watermark / filler / preferred language. Add an optional typed `clear` enum list to CreateChannelFromLineupAdvancedOptions: omitted/null still inherits (byte-stable for existing clients), a field named in `clear` is forced to none. Set+clear of the same field is a 422. The enum (CreateChannelFromLineupClearField) lives in ErsatzTV.Core so the OpenAPI string-enum scan renders it as a string enum, matching every sibling advanced-options enum. Handler resolves clearable fields once via ResolveClearable and validates set/clear conflicts via ValidateClear; reference validation skips existence checks for cleared (null) refs. SPA: the shared advancedOptions model re-adds a real "None" option to the five id selects (watermark + fillers) in both the Channel Builder and the Auto-Tune DetailPanel, routed through a CLEAR overrides sentinel that applyOverridesToRequest folds into advanced.clear (never leaking onto the wire as a field value). The backend enum also covers the preferred audio/subtitle language strings for machine clients; the SPA text inputs keep "empty = inherit" (tri-state deferred). Docs: api-conventions.md §2, spa-conventions.md §11, decisions.md record api.from-lineup-clear-to-none; v1.json + generated TS regenerated. fixes #135 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 22:25:58 +02:00
timothyandClaude Opus 4.8	1ce5743bc1	fix(539): WorkAheadSlots.Release clamps before decrementing, reports unbalance in-band ... Three Low findings from the #536 clamp re-review, unreachable today (one guarded release site) but filed against the day a second release site is added. - §1: Release() now reads the count and CAS-decrements only when current > 0, so it never publishes a negative count even transiently. The prior decrement-first-then-clamp shape dipped to -1, which a concurrent TryAcquire could read as phantom room and over-admit at the limit (re-opening the #529 QSV pool exhaustion). It records the unbalanced release synchronously on the offending thread rather than blaming a later innocent release. - §3: Release() returns bool; HlsSessionWorker logs a warning on the false (unbalanced) return — the one in-band signal a future second release site would need. WorkAheadSlots stays logger-free by design. - §2: UnbalancedReleases doc-comment corrected — it can under-count (an over-release while count > 0 cancels a coexisting leak and goes unrecorded); no false positives, but zero does not prove correctness. Test: Release_Unbalanced_NeverPublishesNegativeCount (2M unbalanced releases vs 4 count-samplers) with a documented, verified negative control (reverting to the decrement-first body makes readers observe the transient -1). Adds a decisions.md entry (ffmpeg.work-ahead-slot-release-never-negative). fixes #539 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 20:59:32 +02:00
timothyandClaude Opus 4.8	dfed9a393b	fix(68): rebuild on-demand channel guide (and mirrors) on thaw ... An on-demand channel (`PlayoutMode.OnDemand`) already is the "resume where I left off" feature: `Playout.OnDemandCheckpoint` persists the viewer's position and `PlayoutTimeShifter.TimeShift` slides the materialized timeline forward on tune-in so the paused item is active again. Because it rewrites `GuideStart`/ `GuideFinish` alongside `Start`/`Finish`, guide and playback freeze together — structurally avoiding the free-running-wall-clock desync #68 was filed about. The one gap: `TimeShift` rewrote the stored `PlayoutItem` rows but the XMLTV guide is served from a cached fragment that only `RefreshChannelData` rebuilds, and the tune-in path never enqueued it. So an external EPG client polling after a thaw could see a stale timeline until the next incidental rebuild. Fix: `IPlayoutTimeShifter.TimeShift` now returns the channel numbers whose cached guide is stale — the shifted channel plus any channels that mirror it (the same fan-out `BuildPlayoutHandler` already does) — and `TimeShiftOnDemandPlayoutHandler` enqueues a `RefreshChannelData` for each on `CancellationToken.None` (post-commit side effect must not be abandoned if the session token cancels). Tests: handler enqueues a rebuild per stale channel (+ mirror + no-shift cases); `PlayoutTimeShifter` reports source+mirrors on a shift, empty on Continuous / zero-offset / active-unforced, and correctly seeds+shifts a never-watched playout. Non-vacuity of the enqueue proven by a compiling negative control. Docs: channels.md (On-demand resume section), domain-model.md, decisions.md (scheduling.ondemand-guide-refresh-on-thaw). Per-viewer resume is out of scope (single per-channel checkpoint; #68 says per-channel suffices). fixes #68 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 19:12:42 +02:00
timothy	5db0836d41	fix(536): clamp an unbalanced work-ahead release instead of going negative ... Pre-push adversarial review, Low finding. The pool is process-wide and lives for the life of the app, so a `Release()` not matched by a successful `TryAcquire` would drive the count to -1 permanently: with a limit of 1 that silently admits two unthrottled transcodes forever, which is exactly the #529 QSV pool exhaustion with nothing in the logs to find it by. Clamp at zero and record the breakage in `UnbalancedReleases` rather than throwing — the sole caller releases from a `finally`, where a throw would swallow the real exception. The hammer tests now also assert `UnbalancedReleases == 0`, so the clamp cannot mask drift it was added to survive. Also moves the #536 index line to the end of the in-file decisions index (it was inserted in the 2026-07-11 block while its body appends at the end) — review nit, anchors were already correct. refs #536	2026-07-21 18:23:12 +02:00
timothy	c0d3dab190	fix(536): enforce workAheadSegmenterLimit with an atomic slot claim [decisions-edit] ... The slot check and its increment straddled an await: `Run` compared `Volatile.Read(ref _workAheadCount)` against a DB-backed limit, and the increment happened later inside `Transcode`. Every simultaneous tune-in therefore observed `0 < limit` and started unthrottled — three concurrent tunes on prod with a limit of 1 all ran with no `-readrate`. `Interlocked` on the write side alone buys nothing when the read side is a separate, earlier load (same class as #231/#250). Extract the counter into a `WorkAheadSlots` pool whose `TryAcquire(limit)` claims via compare-exchange, so the count never even transiently exceeds the limit that the QSV hardware-frame pool sizing (#529) is derived from. `Run` claims the slot and passes ownership in; `Transcode(bool ownsWorkAheadSlot, ...)` derives `realtime` from it and releases it in its existing `finally`, keeping acquire/release one-for-one. Acquisition stays in the caller because `Run` sets `_state` from the outcome and `Transcode` reads that state on entry to pick the item start time. Tests hammer 8 threads x 20k rounds (a single Barrier round does not collide on this hardware); the documented negative control reinstates the check-then-act body and produces 15912 over-claiming rounds of 20000. Also annotates the #350 decision record, whose "every concurrent tune-in falls back to the throttled path" bullet described the intent rather than the behaviour. fixes #536	2026-07-21 18:16:13 +02:00
timothy	b27c950943	fix(529): address review — decision schema, #350 forward pointer, warning log, save-time normalization [decisions-edit] ... Cold adversarial review returned BLOCKED on the documentation half. Addressed: - The new decision record carried no lifecycle metadata block, taking the repo from 82/82 to 83/82 and making it invisible to the by-key catalog lookup that #521 established the same day. Added key/status/since/supersedes/superseded-by (ffmpeg.qsv-extra-hw-frames-floor) and regenerated docs/decisions/README.md; decisions_validate.py now reports OK with no legacy-unmigrated notice. - The entry claimed to correct the #350 record but left that record untouched, so the stale "the burst is bounded" claim stayed authoritative for anyone resolving ffmpeg.hls-cold-start-burst. Added a forward-pointing correction note there (hence the [decisions-edit] token on this commit). - The floor was applied silently. QsvPipelineBuilder.SetAccelState now logs a warning naming both the configured and applied value, because raising a deliberately small pool costs additional surfaces (64 NV12 1080p surfaces is roughly 190 MiB, 760 MiB at 4K) on memory-constrained iGPUs. - Narrowed an overstated claim in the entry: 1..63 are untested, not known-bad. We raise them because the risk is a channel serving nothing, not because asking for less is illegitimate. Recorded as a deliberate over-reach with a stated cost. - Corrected a factual error: SubtitleScaleQsvFilter also formats extra_hw_frames but is dead code with no construction site, so it is NOT covered by the guard. - Config-vs-behavior mismatch: Create/UpdateFFmpegProfileHandler now normalize on save so stored rows converge on what the pipeline runs, and the SPA field carries min=64 rather than defaulting the display to 0. Render-time flooring is kept as the net that fixes existing deployments with no migration; the remaining gap for un-resaved rows is recorded as an accepted residual. - Tests strengthened: pinned to the literal measured 64 rather than to the constant (so lowering the floor cannot quietly satisfy them), added a negative-value case, added a deinterlace-upload case, and replaced the narrow ShouldNotContain with a regex asserting EVERY extra_hw_frames occurrence in the command is >= the minimum. Negative control re-run against the strengthened tests: reverting the floor fails 5, with the build verified succeeded first. Full suite green (4086 .NET, 891 web). Review finding that needed no change: the "single point" claim was independently verified — no bypass exists, every FFmpegState construction routes through MaybeQsvExtraHardwareFrames. Refs #350, #516, #519.	2026-07-21 16:06:35 +02:00
timothy	8930972a0a	fix(525): address final-review findings (migration host-crash guard, multi-error 400s) ... - Medium-1: wrap ExternalLogoMigratorService.ExecuteAsync in try/catch — a DB exception (e.g. a channel deleted mid-migration -> DbUpdateConcurrencyException) no longer trips BackgroundServiceExceptionBehavior.StopHost and kills the app; it logs and self-heals on the next boot. Caller-cancel path handled separately. - Low-2: CreateChannelHandler/UpdateChannelHandler validation failure now returns errors.Join() (all accumulated errors) not errors.Head (first only), restoring the repo-wide convention; regression test added. - Low-4: corrected the Startup registration comment (migrator self-awaits WaitForDatabase; order is not load-bearing). Final whole-branch review: MERGEABLE @ 6d5f6b24 (fable). Carried Minors adjudicated acceptable-defer.	2026-07-21 13:18:48 +02:00
timothy	a5b4783b13	feat(525): budget-check direct artwork uploads (close the upload gap)	2026-07-21 13:13:21 +02:00
timothyandClaude Opus 4.8	93188eeb5e	feat(525): download external-url logo on channel create + create-from-lineup ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 13:13:21 +02:00
timothy	1f21f0f30e	feat(525): download external-url logo on channel update	2026-07-21 13:13:21 +02:00
timothy	be27d9ab8d	feat(498): carry QsvPreferNativeDecoder through application + REST layer	2026-07-20 21:54:13 +02:00