39 Commits

Include renames

Author	SHA1	Message	Date
timothyandClaude Fable 5.1	a7d91bf15a	fix(876): sweep session narrative out of hooks, workflows, scripts, tests and code comments; grow the detector to the process corpus ... `docs.no-session-narrative` reaches every durable artifact, but its detector scanned only `docs/**/*.md` and root markdown, and nothing had ever swept the rest. The issue named four sites from one grep and called them a floor. Deriving the population instead — a whitespace-joined sweep over every tracked file outside the detector, for the detector's own phrasings plus the attribution and review-round class #812 found — gave 453 sites in 108 files at `fb5592971`, and a second pass for phrasings the first list missed (hyphenated `round-N`, "an earlier version", "the reviewer proved") added residuals in the same files. Every site was classified with #812's three dispositions (CUT / SEVER / KEEP with its sub-kind) under the who-benefits test; the per-site manifests are on the PR. The rejected designs, tested-and-rejected fixtures, measurements and traps stay; the attribution of who found them and the round in which they were found go. The detector's population grows to `.claude/`, `.gitea/`, `.husky/` and `scripts/` regardless of extension, minus the detector and its own test (whose fixtures ARE the phrasings) and minus `scripts/tests/fixtures/` (test data, including decision-record copies — the same reasoning as the records' own exemption, and what keeps the record's depth measurement true), and `--all` lists tracked REGULAR files only — a symlink's content is its target and a gitlink has none. The #812 argument for leaving `docs/superpowers/**` in the population runs the other way here: `--diff` sees only ADDED lines, and 287 of the 453 sites were under 30 days old — this corpus is where narrative is being added, so the advisory nudge has reach. Density agrees: 56 line-mode hits over the 113 regular files the predicate admits, against 9 over 66 docs files before #812. `web/` and C# stay out on the same measurement (3 of 74 PATTERNS-matching sites, ~4,600 files). The predicate did not grow: PATTERNS matched 74 of 453 sites, and widening the word list to the attribution class is the treadmill the withdrawn parity test ran on. The population oracle is restated over segments with the new arms, the synthetic cross product gains the process heads and non-markdown extensions, a fixture witnesses that a tracked symlink is neither scanned nor counted, a `.py.bak` axis separates a by-name exemption from a `startswith` over the same tuple, and eight mutants (drop the process arm, drop the by-name exemption, exempt by `startswith`, drop or add a prefix, drop the fixtures exemption, list only markdown, drop the symlink filter, test the mode per row instead of per path) each redden it. A pre-existing silent drop in `--diff` goes with it: git tab-terminates a `+++` filename that contains a space, and the kept tab made `is_scanned_path` refuse the file with no notice — fixed, with a positive control and its own mutant. Code is unchanged by construction, measured per file type against `origin/main`: Python modules are AST-equal with docstrings stripped, except `#` lines inside the embedded fixture programs (string literals) of three test modules; workflows differ only in `#` lines inside `run:` block scalars; shell, C#, TypeScript and jq are equal with comment lines stripped. The stated exceptions: the detector and its test, 26 vitest titles that carried review-round or severity labels or a reviewer attribution (call sites whose title changed — every changed title line walked back to its `it(` / `it.each(...)(` anchor, so a `' + '` concatenation counts once), two registry note strings and the mutation manifest's prose fields. scripts/tests: 1565 passed. Web: lint, typecheck, 1319 tests green. Closes #876. Decisions-Edit: yes Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PEcBoFw7ctrf3Nb7R7x7wk	2026-09-03 20:51:39 +02:00
timothyandClaude Opus 5	9685132ee0	fix(823): three cleanups from the coherence pass — a wrong witness, a wrong because, a duplicated paths: ... Round-five review confirmed the decision record is coherent with no third survivor of the empty reading, and returned three LOW findings. All are in prose I wrote in the last two commits. - The comment defending `(IsAbstract && !IsSealed)` cited AlternateScheduleSelectorTests as an in-repo static-fixture witness. That class IS static, but it merely NESTS its [TestFixture]es and declares no test of its own, so it would fail the sibling "declares no runnable test" assertion rather than demonstrating the point. The rule is right and the witness was wrong, which is the worse of the two failures because a wrong example is what a reader checks the rule against. No witness is cited now, and why is stated. - A mis-bound `because` in `rule:`: "assigning a null and calling SaveChanges SUCCEEDS ... because only the HTTP request records normalize with `?? []`". The `?? []` clause explains how a null could REACH the entity; what makes the save succeed is the column being nullable. A right observation with a wrong cause attached. Split into the two claims. - `signals:` carried the literal token `paths:` twice, an artifact of appending the #823 path list to the existing one. It degrades the field the discovery surface parses. Also recorded from that review, and NOT changed: `MonthsOfYear ?? AllDaysOfMonth()` survives the selector fixture and no date can kill it -- 1..31 contains every valid month, so it is an EQUIVALENT mutant there rather than a coverage gap. Its non-equivalent twin at the DTO boundary is pinned per-dimension by RecurrenceLimitsMapperNullTests. Left alone deliberately: chasing an equivalent mutant with a contrived date would buy nothing and cost the fixture's readability. Local gate: ErsatzTV.Tests 2091 passed / 6 skipped, Core.Tests 697/1 -- 0 failures. Format clean, no BOM. decisions_validate OK. Refs #823 Refs #824 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW	2026-08-29 21:14:27 +02:00
timothyandClaude Opus 5	3951fcf516	fix(823): stop patching the record by grep — a second ?? [] survived, and my "swept it" claim was false ... Round-four review. One HIGH, again in the decision record, and the previous commit message asserted this exact class was cleared. It was not. THE HIGH, and the reason it recurred. A second sentence still described the rejected reading: "The guard form is `?? []` into a local rather than this record's Optional(x).Flatten(), a STATED deviation". The shipped guard is `?? AllDaysOfWeek()`. That sentence is the one that dictates guard FORM to the next implementer, so it would have taught the `[]` reading the same record spends a paragraph calling data corruption -- and it had already propagated into docs/decisions/README.md, the mandated entry point, which carries `rule:` verbatim. The mechanism, not the sentence, is the defect. I swept with a regex keyed on "null" plus a reading word; this sentence talks about guard FORM and contains neither, so it could not match. That is grepping the retracted WORDING instead of sweeping the CONCEPT, which is exactly what this corpus warns about -- and three rounds in a row have now found a defect introduced by the previous round's targeted string edit. So the fix is not another targeted edit: the whole `rule:` field was split into its 39 sentences and read back one by one against the code. Everything below came out of that pass rather than a grep. Its secondary damage is worth recording because it is the shape of a rationale that outlives its claim: the deviation was justified by ".ToList() allocates for nothing", which is now BOTH irrelevant to the choice AND false about the shipped code, since AllDaysOfMonth()/AllMonthsOfYear() are themselves Enumerable.Range(...).ToList() on exactly the null path it describes. - The opening sentence of `rule:` prescribed Optional(x).Flatten() as THE read-site form. It is the sentence most likely to be read in isolation, and it is wrong for six of the eight columns. It now separates the universal half (a LOCAL, never assigned back) from the half that is not (the substituted value), and names where each applies. - `signals:` had never been touched, so roughly 60% of `rule:` was unreachable by the discovery surface built for it -- no AlternateScheduleSelector, no mapper, no "unrestricted", and its paths: list named none of the files this work touched. It also advertised "Optional Flatten hoisted local" as the form, which is precisely what the six do NOT use. - The body prose was still entirely about SongMetadata while `rule:` had grown a whole second subject. Added the two results that contradicted the prior reasoning, in prose, where a reader meets them. A REAL BUG in my own guard, not just prose: fixture.IsAbstract.ShouldBeFalse(...) A C# `static class` compiles to `abstract sealed`, and NUnit runs tests declared in one -- this repo already has such a fixture (AlternateScheduleSelectorTests is `public static class`). So the check I added one commit ago to reject an un-runnable fixture would have falsely reddened a perfectly good static one. Now rejects an abstract BASE (abstract and NOT sealed), which is the case NUnit actually cannot instantiate. A SURVIVING MUTANT the added controls did not kill: AnyDate was 2024-03-06. With a day <= 12 a CROSS-WIRED substitution survives the whole fixture -- `DaysOfMonth ?? AllMonthsOfYear()` hands back 1..12, which still contains day 6, so every assertion passes while the guard substitutes the wrong set. Moved to 2024-03-20, still a Wednesday in March, outside 1..12. Measured both ways rather than reasoned: the cross-wire mutant passes the old fixture and FAILS 2 of 11 on the new one. Also re-witnessed, because I had modified that file and never re-proved it: restoring `??=` in LuceneSearchIndex reddens the LUCENE fixture (1 red, 1 green) -- the exact mirror of the Elastic mutation. Extracting ThrowOnWarningLogger did not cost #701 its proof, and the two fixtures are independently load-bearing in both directions. The record is now 73 prose lines, over the 60-line WARNING ceiling. Stated rather than trimmed: it is 42nd of 42 records over that line, and the added content is distinct findings (a second subject, a migration analysis and three residuals), not redundancy against a sibling. Local gate: ErsatzTV.Tests 2091 passed / 6 skipped (the three fixtures' MySQL halves), Core.Tests 697/1 -- 0 failures. Format clean, no BOM. decisions validate OK. Refs #823 Refs #824 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW	2026-08-29 21:00:22 +02:00
timothyandClaude Opus 5	ea888011aa	fix(823): the decision record argued BOTH readings — and the per-dimension mutant that survived the fixture ... Round-three review findings. One HIGH, and it was in the durable artifact rather than the code. THE HIGH: the record stated the shipped reading and its inverse. The semantic reversal (empty -> unrestricted) rewrote the residual and the write-half of `media.nullable-primitive-collection-mutation` but left the ORIGINAL reasoning standing two sentences earlier: "A null reads as EMPTY, so the item matches nothing"; "the REJECTED alternative was the All*() set"; "SKIPPING the row is the conservative repair". The shipped code is `?? AllDaysOfWeek()` -- precisely the alternative that passage calls rejected. The previous commit then inserted residual (1), which reasons entirely FROM the All*() reading, two sentences after the sentence denying it. That is worse than a stale comment. A session resolving this key -- or reading the MemPalace mirror, which carries `rule:` verbatim -- would have been told to write the guard the other way, i.e. talked into the `[]` reading that the same record elsewhere argues is data corruption one save later. Replaced the whole passage, then swept the record for every other mention of the empty reading rather than trusting the one replacement: the only survivor is the new sentence that records EMPTY as the rejected alternative, which is the direction that stops it being re-adopted. THE MEDIUM: one arrangement did not close the hole it claimed to. The discriminating control added last commit nulls DaysOfWeek against a restrictive MonthsOfYear. It excludes "any NULL matches unconditionally" only for that dimension. The review supplied the surviving mutant -- `if (item.MonthsOfYear is null) { return item; }` ahead of the checks -- and traced it green through all nine tests. Verified by EXECUTION, not by reading: applied to the previous fixture it passes; applied now it FAILS 1 of 11. Each of the three dimensions is now nulled against a restriction on a different dimension. The rest, all from the same round: - The coverage guard's test detection listed attribute TYPES, and each list falsely reddened whatever it omitted: TestAttribute alone missed [TestCase], and the three-type replacement missed [Theory]. Now decided by NUnit's own ITestBuilder/ISimpleTestBuilder interfaces, which cannot fall behind the vocabulary. It also dropped BindingFlags.Static (GetMethods() defaults to including it), which would have falsely reddened a static test method. - The same guard accepted an ABSTRACT fixture -- NUnit never instantiates one. The indexer population already filtered IsAbstract; the fixture side now mirrors it. - The record's `mechanics:` still described the old `[Test]`-only clause, in the same file the change edited. - An <inheritdoc> made the ProgramScheduleAlternate empty-case test inherit a docstring written from the PlayoutTemplate test's viewpoint. Two more mutations executed: - `if (item.MonthsOfYear is null) return item;` -> 1 red, 10 green. This is the mutant that survived the previous head; it no longer does. - an abstract type named in the covered set -> coverage guard red. Local gate: ErsatzTV.Tests 2091 passed / 6 skipped (the three fixtures' MySQL halves, skipping visibly without ETV_TEST_MYSQL_CONNECTION), Core.Tests 697/1 -- 0 failures. Format clean, no BOM on the touched set. decisions_validate OK. Refs #823 Refs #824 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW	2026-08-29 20:43:07 +02:00
timothyandClaude Opus 5	6200713965	fix(823,824): close the review round's findings — a discriminating control, the second mapper's empty case, and honest test detection ... Follow-up commit (the branch is pushed, so not an amend). Two more cold reviews landed on the previous head; both reported 0 Blocker and 0 High, and these are their Mediums and Lows. Each fix carries its own witnessed mutation. 1. The selector fixture could not tell the fix from a much broader one. Every null test set a NULL and expected the item SELECTED, so all of them pass equally under "NULL means unrestricted" and under "any NULL makes this item match unconditionally" -- a refactor short-circuiting the whole date check on any null kept them green. Added the discriminating control: a NULL DaysOfWeek paired with MonthsOfYear = [1] against a MARCH date must be None. Only the narrow reading passes. 2. A_Null_Item_Does_Not_Disturb_Selection_Of_A_Later_Item never measured its own docstring. The nulled item was unrestricted and at Index 0, so it always won and the second item was never evaluated -- the stated invariant ("a null on the first item must not decide the second") went unmeasured while the test passed. Split into two: one where the nulled item genuinely does not match, which measures that the loop CONTINUES; and one that pins the index-order win separately. 3. The empty-preservation control existed for one of two identical mappers. The anti-mutant test for "empty or null becomes All*" covered only Playouts.Mapper; Scheduling.Mapper is a byte-identical triple in another file and had none, so a defensive edit to it alone would have rewritten a deliberately-empty user selection to 1..31 with the suite green. That is the one-helper-two-callers shape this repo has been bitten by. Added the matching test. 4. The coverage guard's [Test] clause did not check what its message claimed. GetMethods() without BindingFlags returns INHERITED methods, so a fixture that merely subclasses another satisfied it while driving the wrong indexer -- and Values.Distinct() cannot catch that, since the two Types differ. It also matched TestAttribute alone, so a future fixture written as [TestCase] would have falsely reddened, and it accepted an [Explicit]/[Ignore]d fixture that never runs, which is the "wired is not running" failure the guard exists to prevent. Now DeclaredOnly, the full test-method vocabulary, and Explicit/Ignore rejected at both method and fixture level. 5. Three residuals recorded on media.nullable-primitive-collection-mutation that the previous head asserted nothing about: - the LOUDNESS change, worst for an all-three-NULL ProgramScheduleAlternate, which now matches unconditionally and shadows the default schedule where it previously threw. Unreachable today, and a choice over an unreachable state rather than a measured requirement -- said plainly. - the normalization is ONE-WAY and WHOLE-LIST: both PUT paths are full replaces, so editing any row persists All*() over EVERY NULL row in that playout, and afterwards "the operator selected all 31" and "this is a legacy row" are indistinguishable. An ordinary user action closes that door. - the WRITE side disagrees with the READ side about what ABSENCE means: an omitted daysOfWeek normalizes to [] ("never applies") while a NULL column reads as unrestricted, so an API client gets HTTP 200 and a row that silently never fires. Filed as #880 rather than folded in here, because a client omitting a field on a write is a different question from what a legacy NULL meant. Two more mutations executed, both witnessed: - DaysOfWeek guard disarmed in Scheduling.Mapper -> 1 red, 3 green. - A fixture with no DECLARED test named in the covered set -> coverage red. Local gate (MySQL lane armed): ErsatzTV.Tests 2097 passed / 0 skipped, Core.Tests 695/1 -- 0 failures. Format clean, no BOM on the touched set with the population count asserted. decisions_validate OK. Refs #823 Refs #824 Refs #880 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW	2026-08-29 20:27:33 +02:00
timothyandClaude Opus 5	95b2700f09	fix(823,824): a scheduling NULL collection reads as UNRESTRICTED and is guarded at both read sites; the Elastic indexer gets its own mutation proof ... Both issues are #701 deferrals, and they land together because both rewrite the same decision record. #823 -- can a null reach one of the six collection-valued scalar columns? MEASURED against a real TvContext on BOTH providers (SQLite, and MySQL 8.4 on an ephemeral server), because the reasoning available beforehand pointed the wrong way. The two converters differ on their read side -- IntCollectionValueConverter maps null-or-blank to Array.Empty<int>(), while EnumCollectionJsonValueConverter would dereference the result of JsonConvert.DeserializeObject -- so the expectation was that a NULL row behaves differently per column. NEITHER RUNS: EF does not invoke a value converter for a NULL column at all. All six materialize as CLR null, the int converter's null-to-empty branch is dead on this path, and unguarded each .Contains in AlternateScheduleSelector throws NullReferenceException. A NULL reads as UNRESTRICTED -- the All*() sets -- not as empty. This is the whole semantic question and the first draft got it backwards. It is decided by the one NULL reachable WITHOUT any code writing one: Sqlite's 20240113140741_Add_PlayoutTemplate_DaysOfMonth adds the column with nullable:true and NO defaultValue, so a PlayoutTemplate row inserted before it holds NULL and by construction had no day-of-month restriction. Reading that as empty INVERTS the row's meaning and silently stops the template applying at all. All*() preserves it, and is how "no restriction recorded" is already represented (GetPlayoutAlternateSchedulesHandler, PreviewBlockPlayoutHandler). What does NOT decide it, and was wrongly cited in the first draft: the API request records normalize an omitted field with `?? []`, but that is a client omitting a field on a WRITE and says nothing about what a legacy database NULL meant. Two read sites, not one. Guarding only the selector would have left the entity->DTO mappers unguarded, and those feed the SPA: PlayoutScheduleEditors spreads the collection (`[...template.daysOfMonth]` -> TypeError on a JSON null) and playoutTemplateCalendar's appliesToDate -- an exact port of GetScheduleForDate -- calls .includes on it. Both mappers now substitute the SAME defaults, so the preview agrees with what is actually scheduled. Neither guard is assigned back onto the entity, which is the media.nullable-primitive-collection-mutation mechanism. Reachability, stated precisely rather than overclaimed. All six are nullable:true on both providers, but a nullable column does not produce a NULL row: five of the six were present at CreateTable, so a NULL there still needs code to write one, and on MySQL there is NO code-path-free NULL for any of the six. The write path ACCEPTS a null (SaveChanges succeeds, stores SQL NULL) but no caller supplies one today -- every production construction of the two commands goes through the request records. That is a property of the code, not a live caller; claiming otherwise would be the banned "it's AsNoTracking today" argument pointed the other way. #824 -- ElasticSearchIndex.UpdateSong had no regression test Issue option 1 (a non-network transport) shipped, and needed no new package: Elastic.Transport.InMemoryRequestInvoker is public in the pinned version and ElasticsearchClientSettings(NodePool, IRequestInvoker) accepts it, injected into the private _client the way #701 injects the Lucene IndexWriter. UpdateItems never runs `_client ??= CreateClient()`, so the injected instance is the one used. Two traps there are load-bearing, both measured: the canned response must carry an `X-Elastic-Product: Elasticsearch` header or the client's product check throws UnsupportedProductException INTO UpdateSong's catch, and an empty body fails to deserialize the same way. Either turns the fixture into a green measurement of the error path -- which is how it first failed here, caught by the ThrowOnWarningLogger. The document id is asserted as the LAST PATH SEGMENT, not by substring: the index name carries digits, so ShouldContain would stop discriminating for a song whose id collided with one. Six mutations executed, each disarming ITS OWN clause alone: - `??=` restored in ElasticSearchIndex only -> the Elastic fixture reddens on "metadata.Artists should be null but was []" while the LUCENE fixture stays GREEN. The #824 hole demonstrated, not described. - DaysOfWeek guard disarmed in the selector -> 4 red, 3 green (DaysOfMonth and MonthsOfYear unaffected). Each clause is independently load-bearing. - DaysOfMonth guard disarmed in Playouts.Mapper -> 1 red, 2 green. - Elastic dropped from the covered set / mapped to the SAME fixture as Lucene / mapped to a class with no [Test] -> SearchIndexMutationCoverageTests reddens on each. That coverage guard is the boundary fix the issue asked for: the covered set is compared against an ISearchIndex population DERIVED FROM THE ASSEMBLY. Its claim stops where the check does -- no static check can establish that a named fixture actually DRIVES its indexer, so it forces a human to look rather than proving coverage. ThrowOnWarningLogger moved to ErsatzTV.Tests/Support so both fixtures share it; the Lucene fixture's assertions are otherwise untouched, since it is a witnessed proof artifact. No production change in ElasticSearchIndex.cs -- #824 is coverage only. Docs: testing.md gains a "Provider-parity fixtures" section naming all THREE opt-in-MySQL fixtures and recording that CI runs none of them (#627); docs/README.md gains the matching task signal; guard-inventory.md's hand-written C# guard list goes from five files to six. Scheduling/Mapper.cs loses the UTF-8 BOM it inherited, per #311 fix-as-you-touch. Local gate (with the MySQL lane armed): ErsatzTV.Tests 2096 passed / 0 skipped, Core.Tests 693/1, Infrastructure.Tests 114, Architecture.Tests 7, Scanner.Tests 1504 -- 0 failures in each. scripts/tests 1228 passed / 2 skipped. dotnet format whitespace --verify-no-changes clean; BOM check over the touched set with the population COUNT asserted, because a bare zsh loop silently checks one concatenated filename. decisions_validate OK. Fixes #823 Fixes #824 Decisions-Edit: yes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW	2026-08-29 20:02:52 +02:00
timothyandClaude Opus 5	b5dee26202	fix(701): guard SongMetadata's nullable primitive collections at the read site ... Both search indexers opened UpdateSong with metadata.AlbumArtists ??= []; metadata.Artists ??= []; Artists/AlbumArtists hold the whole list in ONE COLUMN rather than being navigations. So unlike the same `??= []` idiom on Genres/Tags/Artwork all around them, the property IS the column value: assigning it on a TRACKED entity flips the entry to Modified and the next SaveChanges writes [] over a NULL column. This is the mechanism an adversarial review demonstrated in #691, which is why that issue's entity-level guard was reverted in favour of guarding at the read site. Measured rather than reasoned about, per the issue's first done-when box. Restoring ONLY the `??= []` clause (the real predecessor lines, not a hand-written mutant) reddens the new fixture on `metadata.Artists should be null but was []`; a probe variant with the first two assertions replaced by prints reports STATE=Modified and the raw column moving from NULL to "[]". Today's two feeds are both AsNoTracking (SearchRepository.GetItemToIndex and GetAllSongs), so no shipped caller loses data -- but that is a property of two callers, not of the indexer, and #691 already recorded it as a loaded gun. The fixture pins the indexer's own contract instead. Removing the assignment is not sufficient alone: it was load-bearing for the four reads below it, and deleting it by itself converts a silent write into a live throw on every untagged song. Measured by deleting only those two lines from the real predecessor file: NullReferenceException, thrown at the foreach (cited by symbol: a line number in a mutant that exists in no committed tree is unreproducible by construction). The exception type follows the read FORM, not the field -- foreach yields NRE, string.Join/ToList yield ArgumentNullException -- and this PR contains two of each, which is why no single exception-name grep characterises the class. So each site moves together with its reads: - LuceneSearchIndex.UpdateSong / ElasticSearchIndex.UpdateSong: hoist Optional(...).Flatten().ToList() locals and read those. - RefreshChannelDataHandler: the Scriban context took the raw nullable lists (the issue's second item). The shipped _song.sbntxt only does array.join, but a custom template is free to do anything. The population was derived from the MODEL rather than from the issue's file list, and the obvious derivation is wrong: "the IList<string> properties under ErsatzTV.Core/Domain" returns two of eight. It misses the six value-converted collections (ProgramScheduleAlternate and PlayoutTemplate each carrying DaysOfMonth, MonthsOfYear, DaysOfWeek), declared as plain ICollection<T> and made single columns only in Data/Configurations -- and their storage differs (comma-separated text for the int converter, JSON for the enum one), so the shared property is "one scalar column", not the serialization. No site applies `??=` to any of the six, so this defect has no instance there; whether a null can REACH one at runtime is unverified and is filed as #823 rather than asserted either way. Only the SongMetadata pair is left NULL in practice, by FallbackMetadataProvider. Every site touching either field was then swept; the remaining readers were already guarded by #691. The fixture carries two anti-vacuity guards, both witnessed: - A POSITIVE CONTROL (`writer.NumDocs.ShouldBe(1)`). Every other assertion says something did NOT happen, so all of them hold vacuously if UpdateSong never runs -- and it silently stops running if a future refactor gates UpdateItems on `_initialized`, which this fixture bypasses by injecting the writer. Verified BOTH directions: with that gate added the control fails `NumDocs should be 1 but was 0`, and with the control removed the whole test PASSES while the code under test is unreachable. - A capturing logger, because UpdateSong wraps its body in a catch that assigns metadata.Song = null -- severing a required relationship and cascading the metadata to Deleted. Without it the probe silently measures the error path; on the first run it did exactly that (a bare ILanguageCodeService substitute NPEs inside AddLanguages). The raw column helper also fails loudly on a missing row, since ExecuteScalar returns CLR null for both "NULL column" and "no such row". ElasticSearchIndex has no equivalent fixture -- it needs a stubbed transport -- so its change is by inspection against the Lucene one, and the gap is filed as #824 rather than covered by a source-text guard. The whitespace-only churn in ElasticSearchIndex.cs is the #311 fix-as-you-touch format gate: it scopes to whole changed FILES. `git diff -w` over that file shows only the two hunks above. Local gate: ErsatzTV.Tests 2006 passed / 4 pre-existing skips, Core.Tests 685/1 skip, Infrastructure.Tests 114, Architecture.Tests 7, Scanner.Tests 1504 -- 0 failures in each. scripts/tests 874 passed / 2 skipped. dotnet format whitespace --verify-no-changes clean on the four touched files, no BOM on any. decisions_validate OK. Fixes #701 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-23 00:49:57 +02:00
timothy	f2d9c0dc8e	fix(668): review round 5 -- three prose nits, including an off-by-one I filed ... Final sweep confirmed the retracted MySQL over-match claim survives in no file on the branch (only in two immutable commit messages, which stay -- rewriting history would invalidate every sha-bound review verdict). Three nits remained. - The fixture's class docstring said the on-MySQL claim "rests on the server's collation", which is the one thing the decision record says it does NOT rest on. It rests on Unicode-aware LOWER(); the executed comparison bypasses the collation entirely. Reworded. - The record's `rule:` enumerated the covered fields but omitted show_genre, which GetSource and the fold both handle ("genre" or "show_genre"). Added. - My own #688 write-up was wrong twice: the 60-line ceiling warning is NON-blocking by design, and the calibration pytest reds at >=61, not >=60 -- main's p90 is 59, so a 60-line record makes p90 == ceiling and PASSES. The bullet even contradicted itself, since the next sentence relies on 60 passing. Corrected in the PR body and in the issue. Decisions-Edit: yes	2026-07-27 22:21:55 +02:00
timothy	07723e418b	fix(668): review round 4 -- sweep the retracted claim by SUBJECT, not by memory ... Round 3 said "correct the claim everywhere" and missed two places, which is the same mistake in a new coat: I fixed the spots I remembered instead of grepping for the subject. - The class-level summary of the very file round 3 edited still said MySQL's "ci collation OVER-matches instead", contradicting the method docstring forty lines below it. Reworded. - The PR body still carried the retracted over-match story -- and round 3's commit message claimed it had been corrected. It had not. Now corrected, with the measurement table and the retraction stated openly. This time the sweep was `grep -i over-match` across every file the branch touches; the remaining hits are the SQLite-fold invariant and #578 history, which are correct and stay. Also softened two overclaims the reviewer flagged. Round 3 deleted the predecessor's "configuration-incidental, not designed" hedge and replaced it with a firmer statement than the evidence supports: that MySQL cannot over-match is contingent on MySqlConnector fixing the connection collation to utf8mb4_bin, not a property of MySQL. A driver, protocol or prepared-statement change could restore it. The record and the fixture docstring now say "driver-contingent, not a law"; the code is safe either way because the ordinal filter stays regardless. Filed #689 for the source of the copied falsehood -- LibraryFolderDedupeMigrationTests on main carries the same "CI sets ETV_REQUIRE_MYSQL_TESTS=1" sentence. Decisions-Edit: yes	2026-07-27 22:09:26 +02:00
timothy	dda98efcc4	fix(668): review round 3 -- MySQL does NOT over-match; correct the claim everywhere ... Review BLOCKED on a false CI claim I copied from the sibling fixture ("CI sets ETV_REQUIRE_MYSQL_TESTS=1"). Nothing sets it; the doc now says plainly that CI does not arm this lane and points at ersatztv#627. That was the blocker. Chasing the reviewer's second finding then overturned something bigger. It predicted that seeding an unaccented "Edith" would make the in-memory ordinal filter load-bearing on MySQL, since utf8mb4_0900_ai_ci treats é as e. Mutation test says otherwise: with the filter deleted the MySQL test stays GREEN. Measured against a live 8.4 to find out why: LOWER(Name) LIKE 'é%' (literal) -> Édith AND Edith LOWER(Name) LIKE @v (ai_ci variable) -> Édith AND Edith LOWER(Name) LIKE @v COLLATE _bin -> Édith only the EF query, executed -> Édith only The driver binds the pattern with a BINARY collation, so the executed comparison is accent-SENSITIVE and MySQL does not over-match at all. MySQL's correctness rests on its Unicode-aware LOWER(), not on the collation. My earlier probe used a LITERAL pattern -- a different query from the one the code runs -- and I wrote its result into the handler comment, the decision record and the PR body. All three now say what actually happens, and the record carries the lesson: measure the query the CODE runs, not one you type. The "Edith" row stays as a near-miss control, with a docstring that says what it does and does not prove rather than the over-match story it was added for. Also moved EnsureCreatedAsync out of [SetUp]: NUnit skips [TearDown] when [SetUp] throws, so a mid-create failure would strand the database. Decisions-Edit: yes	2026-07-27 21:52:16 +02:00
timothy	1f6802bb62	test(668): execute the accented-value claim on a REAL MySQL server ... The PR's central claim is "reachable on BOTH providers", and on MySQL that rests on the server's collation rather than on any code this repo owns -- exactly the kind of assumption worth executing rather than asserting. Adds SearchFieldValuesProviderTests, parameterized over Sqlite and MySql on the LibraryFolderDedupeMigrationTests contract: opt-in via ETV_TEST_MYSQL_CONNECTION, a VISIBLE skip without it, and a hard failure instead of a skip when ETV_REQUIRE_MYSQL_TESTS is set, so the lane cannot degrade into "connected to nothing and passed". Fresh never-reused database per test, dropped with its pool cleared. Verified for real, not just written: run against a live mysql:8.4 it reports 4 passed / 0 skipped and the stored 'Édith' is returned for both q=é and q=É. Without the connection string it skips (2 skipped); with REQUIRE set and no connection it FAILS. All three paths exercised. The MySql half wires RegisterUnicodeCaseFunctions to an explicit no-op, so the test proves MySQL reaches the value through its own Unicode-aware LOWER() and not through SQLite's custom fold. Known and deliberate: CI does NOT arm this lane, so it will skip there. Per the note in docker-build.yml, running MySQL fixtures against the live service was implemented and removed as non-deterministic (ersatztv#627) on the grounds that an intermittently-red gate is worse than none. Re-arming it is that issue's job, not this PR's -- so this fixture is opt-in exactly like its sibling.	2026-07-27 21:31:36 +02:00
timothy	04ea2abb40	test(491): restore per-test databases; keep the pool clear (fixture determinism) ... Second CI failure of this fixture, and my previous fix caused it: collapsing to one shared database traded isolation for a wipe that has to succeed, and when it silently did not, the second test seeded onto the first's rows and failed with 'Duplicate entry 1 for key LibraryPath.PRIMARY' in 539ms - too fast to have re-run the migration chain, i.e. MigrateAsync no-opped against an already-current __EFMigrationsHistory. The coordinator's read was right and I verified it rather than assuming: the leak came from never clearing pools, not from names being unique. Harness against a real 8.4 server, unique database name per iteration WITH ClearPoolAsync on that connection string: 0 leaked threads over 30 iterations. So isolation costs nothing and the shared name was solving a problem pool-clearing already solved. Restore a fresh etv491_<guid> database per test, never created out of band (the test's own MigrateAsync(PreviousMigration) creates it, keeping EF the single owner of the schema), dropped in TearDown via the guarded EnsureDeletedAsync and followed by ClearPoolAsync on that exact connection string. The stale-session hazard from the first failure needs the connection string to be REUSED after the drop, which a never-repeated name makes impossible; clearing the pool is the belt to that brace and closes the leak. Verified: 10 consecutive runs 10/10 green, server threads flat at 2 and zero leftover schemas throughout; each test run ALONE twice; both tests with the order REVERSED - order-independence being the evidence this failure would have needed, since contamination is invisible when a test runs first. Negative control re-confirmed after the change: restoring COLLATE utf8mb4_bin still fails with survivors [1,4,5,6,7,9] vs [1,4,5,6,7,9,10], then restored and green again. No Retry anywhere. One shared fixture body across providers; still fail-closed under ETV_REQUIRE_MYSQL_TESTS. Refs #488 #308 fix #491	2026-07-25 21:13:31 +02:00
timothy	cdb67e26f2	test(491): make the MySql dedupe fixture deterministic (pool + database lifetime) ... The new MySql coverage failed intermittently in CI with 'Unknown database etv491_...' at connect time, and passed on the immediately preceding commit - a latent race in the fixture, not a regression. Diagnosed against a real 8.4 server rather than by inspection; two measured mechanisms: 1. A pooled session outlives DROP DATABASE. Reopening the dropped database's connection string SUCCEEDS, because MySqlConnector hands back the still-alive session whose default schema no longer exists. Whether a caller sees success or 1049 then depends on whether the pool reuses that session or opens a fresh one - a fresh handshake names the dropped schema and fails. That is the non-determinism, and it is invisible in any single green run. 2. A per-test database name leaks a pool per test. MySqlConnector keys pools by connection string, so each test built a new pool that was never cleared, leaking ~1 server thread per iteration (measured); hammering it exhausted max_connections outright, and a saturated server also inflated this fixture from ~1m20s to 13 minutes. Fix is structural, not a retry: ONE fixed database name (hence one pool), EF owns the lifecycle via EnsureDeletedAsync (guarded, unlike a raw DROP), the pool is cleared after every drop, and the drop moves to the START of a test so no test leaves a live pool pointing at a database it just destroyed. ServerVersion is resolved once per SetUp instead of twice. Determinism verified: 10 consecutive runs, 10/10 green, server threads flat at 2 throughout. Negative control re-confirmed after the change - restoring COLLATE utf8mb4_bin still fails with survivors [1,4,5,6,7,9] vs [1,4,5,6,7,9,10]. Both providers still share one fixture body; still fail-closed under ETV_REQUIRE_MYSQL_TESTS. Refs #488 #308 fix #491	2026-07-25 21:13:31 +02:00
timothy	b587f80f43	test(491): clear the nullable warnings in the dedupe fixture ... Review follow-up. The new MySql-half members introduced CS8600/8604/8625/8618 and were inconsistent with this same file's established late-init pattern (_databasePath = null!). Nullable-annotate the genuinely-nullable ones and use null! for the Dapper-populated row properties.	2026-07-25 21:13:31 +02:00
timothy	83cd36e0de	test(491): run the dedupe fixture against MySql in CI; correct the collation claim ... The dedupe DML had zero automated coverage on MySql: the migrations job only applies migrations to a fresh EMPTY database, so no dedupe row ever executed there. Two MySql-only collation defects escaped that gate in this session and were caught only by hand-run containers. Parameterize LibraryFolderDedupeMigrationTests over both providers from ONE fixture body - same seeded rows, same expected survivors - rather than adding a MySql-only copy that would drift and recreate the gap. Assertions no longer use WHERE Path = '...', which is itself collation-dependent and would quietly mean something different per provider; rows are read once and compared ordinally in memory. A new step in the existing migrations job runs it against that job's mysql:8.4 service, on a per-test database of its own. Proven red when the collation is wrong: restoring COLLATE utf8mb4_bin fails the MySql half with survivors [1,4,5,6,7,9] - the trailing-space sibling deleted - while SQLite stays green. Proven non-skippable: without ETV_TEST_MYSQL_CONNECTION the fixture ignores visibly, and with ETV_REQUIRE_MYSQL_TESTS=1 (which CI sets) that skip becomes a hard failure, so it cannot pass having connected to nothing. Local runs need no MySql. Also correct an overstated comment. The schema pins only the utf8mb4 charset, never a collation, so the effective comparison is the server default: always case-insensitive, but PAD SPACE only on utf8mb4_general_ci - 8.4's default utf8mb4_0900_ai_ci is NO PAD, verified on the real column. The migration bug was independent of that because the old code applied an EXPLICIT utf8mb4_bin, which is PAD SPACE everywhere; the runtime simply tolerates both. Refs #488 #308 fix #491	2026-07-25 21:13:31 +02:00
timothy	48d41b9235	fix(491): make the MySql dedupe byte-exact, not just case-exact (PAD SPACE) ... Cross-family review of 1b4dd6d6 found that utf8mb4_bin - chosen to keep the dedupe case-exact - is a PAD SPACE collation, so trailing spaces are insignificant under it. Verified on MySQL 8.4: '/media/Foo' = '/media/Foo ' is TRUE, while case correctly compares unequal. Two distinct legal directories therefore grouped together and the second was DELETED irreversibly, even though PathUtils.GetPathHash hashes them differently and the unique index about to be created would have accepted both. The dedupe destroyed data the constraint never required it to destroy. Group and join on CONVERT(Path USING binary) instead - NO PAD and byte-exact, matching the hash. utf8mb4_0900_bin is also NO PAD but carries a server-version floor. This is the only path comparison in either migration (every other predicate keys off an integer id), so there is no mix of padded and unpadded comparisons across the keeper-selection, repoint and delete steps. SQLite's = on TEXT is byte-exact with no padding, so that migration was already correct - which is exactly why a SQLite-only test could not see the divergence. The two providers are now semantically equivalent, and the dedupe fixture is shared: same rows, same expected survivors (1,4,5,6,7,9,10), asserted by the SQLite test and reproduced by hand on MySQL 8.4. Runtime was never affected, and this is now stated and tested rather than assumed: GetFolder's SQL equality is a superset narrowing (both collation quirks make it more permissive, never less, so it cannot miss a byte-exact match) and ResolveExact settles identity with StringComparison.Ordinal, which compares length first. Added ResolveExact coverage for the trailing-space axis. Refs #488 #308 fix #491	2026-07-25 21:13:31 +02:00
timothy	ecb763ea58	fix(491): review polish — heal cannot abort a scan, ordinal settle unit-tested ... Final low-severity items from the re-review of ee10f932. L2: DbUpdateConcurrencyException derives from DbUpdateException but carries no provider exception, so IsUniqueConstraintViolation does not classify it. A row deleted by a concurrent library edit between the heal's read and its save would propagate and fail the scan, contradicting the invariant stated directly above it. Admit it in the filter. L1: lift the in-memory ordinal settle into LibraryRepository.ResolveExact and unit-test it with both spellings in the candidate list. No SQLite-backed test can exercise it (SQLite's = on TEXT is already binary), so this converts the half that rested on hand-run MySQL evidence into automated coverage. The end-to-end companion test's comment no longer claims to be provider-independent. L4: assert PRAGMA foreign_keys is 1 before migrating, so the enforcement guard cannot silently degrade into the weak pre-fix form it was added to replace. L3: detach the failed heal, matching the insert path. N3: the heal's inner predicate now matches its IsNullOrEmpty outer guard, so a PathHash = '' row cannot enter the branch and silently never heal. N4: record that GetFolder returning null for a case-differing spelling makes MySQL insert a second row where it used to reuse one — correct, and now matching SQLite, but a real behaviour change on a case-insensitive filesystem. Refs #488 #308 fix #491	2026-07-25 21:13:30 +02:00
timothy	14e9b03433	fix(491): wire the unique-violation classifier in the scanner; make folder lookup case-exact ... Review of 491f5099 found the fix inert in the only process that runs it, plus a MySQL collation defect in the lookup. B1 — TvContext.IsUniqueConstraintViolation was assigned only in ErsatzTV/ Startup.cs, but ErsatzTV.Scanner is a separate executable and every production caller of GetOrAddFolder/SetEtag lives there. The classifier kept its '_ => false' default, so the catch never ran and the DbUpdateException failed the whole scan - worse than the duplicate row it replaced. Wire both provider branches in ErsatzTV.Scanner/Program.cs, and add ProviderStaticsWiringTests (architecture) asserting the scanner assigns every TvContext static the host assigns, with IsSqlite documented as the one exemption. H1 — GetFolder's 'Path == folder' is case-insensitive on MySQL while PathHash is case-sensitive, and FirstOrDefault was unordered: a scan of '/x/foo' could resolve the '/x/Foo' row and stamp the wrong hash onto it (verified on MySQL 8.4: the WHERE matches both, LIMIT 1 returns the wrong one). Treat the SQL equality as a narrowing filter, order by Id, and settle identity ordinally. Route the heal through EF and drop a classified violation, so an opportunistic maintenance write can never abort a scan. Also: run the dedupe migration test with foreign keys ON (matching prod), clear the keeper's etag, null out a self-parent, and document the cleanup's limits (NULL paths excluded, Down does not restore deleted rows, CI's fresh-DB apply covers none of the data mutation). Refs #488 #308 fix #491	2026-07-25 21:13:30 +02:00
timothy	9a4f3e832d	fix(491): unique index on LibraryFolder(LibraryPathId, PathHash) + tolerate concurrent insert ... GetOrAddFolder was a check-then-insert with no unique constraint behind it, so two callers racing the same folder could both miss the lookup and both insert. Enforce identity in the schema and make the loser adopt the winner. - LibraryFolder gains a SHA-256 PathHash (the MediaFile.Path/PathHash precedent): Path is MySQL longtext, which cannot be indexed without a prefix length and collates case-insensitively, so the unique index is on (LibraryPathId, PathHash) instead. - GetOrAddFolder and SetEtag catch a classified unique violation via the existing TvContext.IsUniqueConstraintViolation seam (#308) and re-read. - Dual-provider migration audits and collapses pre-existing duplicates (repointing MediaFile, ParentId and ImageFolderDuration) before creating the index; legacy rows keep a null hash and heal on the next scan. - Tests: deterministic cross-connection race, 8x10 barrier stress with an insert-attempt vacuity guard, classifier-inversion negative control, and a real-migration dedupe test. Refs #488 #308 fix #491	2026-07-25 21:13:22 +02:00
timothy	b5b6e7f636	test(496,484): record projection failures during enumeration, not eagerly [decisions-edit] ... Re-review Low. The substitute incremented the failure counter inside .Returns(...), i.e. when the enumerable was handed out, while the real paginator records from ProjectToMusicVideo's catch DURING enumeration. A refactor that snapshotted Count before the enumeration completed would then break production while both replacement tests kept passing — exactly the regression the guard exists to prevent. Moves the recording into an async iterator, and corrects the decision record to describe #484's removed music-video test accurately and name its two replacements.	2026-07-25 17:36:34 +02:00
timothy	4bdad4bf52	fix(496,484): thread #484's projection-failure guard through the new music-video scanner [decisions-edit] ... Rebasing onto #612 exposed a silent gap rather than a conflict. #612 added a `projectionFailureCount` parameter to MediaServerReconciliationGuard.ShouldFlagMissing that refuses the sweep when the enumeration reported swallowed projection exceptions — but the parameter is OPTIONAL with a default of 0, so this scanner compiled unchanged while opting out of the protection entirely. ProjectToMusicVideo has exactly the swallowing catch #484 exists to defend against, so the music-video sweep would have been the only one unguarded. - MediaServerMusicVideoLibraryScanner creates one MediaServerProjectionFailureCounter per enumeration (never a field on the singleton api client, so concurrent scans of different libraries can't leak failures into each other's sweep decision), passes it to GetMusicVideoLibraryItems, and feeds its Count to ShouldFlagMissing. - The single guard call also gates the #496 legacy path diff, which is more exposed: a legacy row has no etag to fall back on. - ProjectToMusicVideo now returns MediaServerProjectionResult<JellyfinMusicVideo>, combining #612's Skipped/Failed distinction with #496's identity type. - main's own #484 music-video test targeted the pre-#496 hard-delete scanner (FindMusicVideoPaths /DeleteByPath) and no longer applies; it is replaced by two tests in the new architecture covering the identity sweep and the legacy sweep. Both proven non-vacuous — removing projectionFailures.Count from the guard call fails exactly those two. 4,277 tests green; format/BOM clean; decisions validator OK.	2026-07-25 17:27:50 +02:00
timothy	6bd1d954bd	fix(496): review fixes — thread the replaced local path, scope identity per library, sweep legacy rows [decisions-edit] ... Independent cold review (Codex) returned BLOCKED. Findings 1, 3 and 4 are fixed here; each has a regression test proven non-vacuous by a negative control. 1. Blocker — the replaced local path was discarded. The scanner computed localPath but GetOrAdd only received `incoming`, so the repository re-derived the path from the UNREPLACED projection. On any install with path replacements, adoption hashed the server-side path, missed the existing row, ALSO slipped past MediaFileAlreadyExists (which hashes that same wrong string) and inserted a duplicate row under a server path, leaving the original collection-linked row identity-less forever. The test harness hid this because its path-replacement stub was an identity function. → GetOrAdd now takes localPath explicitly and never reads the projection's path; BuildPathReplacement takes a real mapping and the new test genuinely replaces. 3. Medium — GetByItemId matched on ItemId alone, so two media sources presenting the same item id (cloned Jellyfin DB) resolved to each other's row, letting one library repoint another's. → filtered by LibraryPath.LibraryId. 4. Medium — a row predating the identity that the server had ALREADY stopped reporting was never adopted (adoption only runs for an incoming item) and carried no identity, so the itemId diff could not see it either: it sat Normal and schedulable forever, strictly worse than the hard delete it replaced. → GetExistingLegacyMusicVideoPaths + FlagFileNotFoundByPaths reconcile legacy rows by local path, and they are counted into the #477 empty-fetch guard (on the first scan after this ships they ARE the whole library, so a guard counting only identity rows would sweep all of them on a transient empty fetch). Finding 2 (the issue's Done-when #2) is a scope question, not a defect, and is unchanged: one file path is still one MediaItem row globally, so this lands music videos at parity with movies rather than eliminating shared-row trashing. Recorded honestly in the decision record; raised for an explicit call before the issue is closed. fixes #496	2026-07-25 17:20:53 +02:00
timothy	64decd492e	fix(496): give music videos a per-library server identity; itemId diff + soft trash ... Music videos carried no server identity, so JellyfinMusicVideoLibraryScanner had to reconcile by a (LibraryPathId, path) diff and HARD-delete the remainder. A file served by two libraries with overlapping local paths is a single row owned by whichever library scanned it first, so that owner's sweep destroyed a row another library still served — taking collection membership and playout references with it, irreversibly. This is #494's deferred "option 2": - New JellyfinMusicVideo : MusicVideo (ItemId/Etag), mirroring JellyfinMovie — TPT table, varchar(36), ItemId index. Dual-provider migration Add_JellyfinMusicVideo. - New IMediaServerMusicVideoRepository + JellyfinMusicVideoRepository: itemId-keyed existing-set/lookup and Flag{Normal,Unavailable,FileNotFound} seams, all scoped per library via LibraryPath.LibraryId. - New MediaServerMusicVideoLibraryScanner base; JellyfinMusicVideoLibraryScanner folds onto it and keeps the #177/#488/#497/#500 metadata-reconcile logic verbatim. - The sweep now soft-trashes (FileNotFound) instead of deleting, so removal is reversible and EmptyTrash-governed. DeleteEmptyArtists consequently no longer fires from a sweep. - Pre-identity rows are ADOPTED in place: the identity row is inserted against the same MediaItem id, scoped to the scanned library's own LibraryPath, so collection membership survives and a local/second-library row is never hijacked. - AddMusicVideo normalizes Path/PathHash to the path-REPLACED local path; the projection fills them from the server-reported path, which would break every later PathHash lookup. Docs: scan.musicvideo-reconciliation relocated to docs/decisions/archive/scan.md as superseded; new active record scan.musicvideo-server-identity. fixes #496	2026-07-25 17:20:53 +02:00
timothy	6f4497e1ce	fix(484): guard the nested TV season and episode sweeps against projection failures ... Review finding 1 (blocking). ScanSeasons' FlagFileNotFoundSeasons and ScanEpisodes' FlagFileNotFoundEpisodes had no guard at all — neither #477's nor #484's — so ProjectToSeason / ProjectToEpisode returning Failed() was computed and discarded. #477 scoped those out because "the blast radius is one show's seasons / one season's episodes", which holds for a per-parent EMPTY fetch but not for a projection failure: that is systematic by construction. One bad code path fires on every parent, so every season enumerates zero episodes, existing.Except([]) is the whole episode library, and EmptyTrashHandler deletes it permanently. Threads the counter into GetSeasonLibraryItems / GetEpisodeLibraryItems(WithoutPeople) for Jellyfin and Emby using the same optional-trailing-param shape, and guards both sweeps with MediaServerReconciliationGuard.ShouldFlagMissingDescendants — the same class and the same private failure predicate as ShouldFlagMissing, deliberately WITHOUT #477's empty-fetch branch so per-parent empty behaviour (and #476's cascade, which depends on it) is unchanged. Also from the review: - finding 3: tests now pin the same-instance JOIN at every level (movie, show, season, episode, music video) by driving the real ScanLibrary entry point and recording the failure from inside the enumeration, so a refactor handing the api client a fresh counter goes red. - finding 4: the missing-library Failed() branch is documented as defensive and unreachable. - finding 2: the mass-Skip residual (Emby's response-shape-dependent MediaSources guard, Plex's pre-projection filter) is stated as a known limitation in the decision record. - finding 5: the log-contract change (only the #484 message when both refusals apply) is noted. fixes #484	2026-07-25 16:36:35 +02:00
timothy	e3645a2840	feat(484): suppress the media-server sweep on projection failures; reject the ratio threshold ... Extends MediaServerReconciliationGuard (#477) with a second deterministic refusal: when the enumeration that produced the incoming set silently dropped items whose projection THREW, the file-not-found sweep is refused. A dropped item the server did return is indistinguishable from a deletion at the reconcile step, so a projection regression could otherwise mass-flag a healthy library FileNotFound (which EmptyTrash then deletes permanently). Deliberate guard-clause skips (STRM files, virtual items, unsupported types) are explicitly NOT failures and never suppress a sweep — counting them would permanently disable reconciliation for any library holding a single STRM file. The ratio / missing-fraction threshold is REJECTED, not deferred: it is a two-sided heuristic with no tunable default and no telemetry, and the failure it approximates is exactly observable via the projection-failure count (a genuine bulk deletion produces zero failures). Seam is deliberately narrow — the private ProjectTo* contract inside each api client changed from Option<T> to MediaServerProjectionResult<T> (projected/skipped/failed), the paged helper counts IsFailure in one place, and the scanner reads it through an optional trailing MediaServerProjectionFailureCounter on only the five library-level methods that feed a sweep. The counter is per-enumeration state created by the scanner, never a field on an api client. fixes #484	2026-07-25 16:36:35 +02:00
timothy	0081bed4b1	docs(460): drop the "clean at rest" overstatement from the test header ... Follow-up to the cold review of 615e00a. The record was corrected to say only that no NEW sentinel rows are created, but the test file's header still carried the disclaimed "the data is clean at rest for every provider" claim — sitting on the very test that supposedly proved it. It now states what the tests actually pin, and names what they cannot speak to (rows written before this change, which keep the sentinel and are covered only by the read coercion). Also corrects docs/testing.md's ErsatzTV.Core.Tests count (543 -> ~650 measured; the neighbouring Scanner.Tests cell is corrected in #602 instead, to keep the two open PRs off the same line).	2026-07-25 13:46:41 +02:00
timothy	890745d1d4	fix(460): write null LastScan on disable-sync instead of the MinValue sentinel [decisions-edit] ... MediaSourceRepository's Plex/Jellyfin/Emby remove-and-recreate (disable-sync) flows stamped SystemTime.MinValueUtc into library.LastScan, so normal use kept minting 0001-01-01 sentinel rows. #409 fixed the READ side (the API coerces the sentinel to null and a migration cleaned the historical residue), so the wire contract was already correct — this stops new sentinel rows being written. Safe because every remaining LastScan reader either coalesces (LastScan ?? SystemTime.MinValueUtc) for its own non-nullable scan-comparison needs, or is the API read-boundary coercion itself — audited every reference under ErsatzTV{,.Application,.Core,.Infrastructure,.Scanner,.Mcp} plus web/. There is no Where/OrderBy/GroupBy on LastScan anywhere, so the SQL null-ordering divergence between SQLite and MySQL has no surface here. Scan-comparison behavior is unchanged. Deliberately ships NO second cleanup migration: rows written between #409's NullOutNeverScannedLastScan and this change keep the sentinel at rest, and the permanent read coercion — not a migration — is what keeps the contract honest for them (as it must be anyway for a restored or hand-edited DB). LibraryPath.LastScan is likewise left untouched: the flows re-add Paths with their original values, and its only readers are the local-library scan handlers, so it has no API surface and no remote-scan effect. Regression test per provider (MediaSourceRepositoryDisableSyncTests), proven non-vacuous: restoring the MinValue writes fails all three. [decisions-edit] — the media.lastscan-null-boundary record documented this write as an ONGOING sentinel source and rested its "the coercion is permanent" argument on it, so the rationale prose is corrected in the same PR per docs.decision-lifecycle. The Rule line is unchanged, so the generated decisions/README.md catalog is byte-identical. MediaSourceRepository.cs also loses its UTF-8 BOM (fix-as-you-touch, #311). fixes #460	2026-07-25 13:46:09 +02:00
timothy	2f2bcca681	fix(500): dedup incoming metadata collections so a duplicate name inserts once ... The remove-stale + add-new reconcile idiom materializes its add set with .ToList() BEFORE the loop mutates the existing collection, so the add filter (`incoming.All(x2 => x2.Name != x.Name)`) is evaluated against a snapshot. Two identically-named incoming entries whose name is not yet on the existing item therefore BOTH passed the filter and BOTH inserted — a duplicate row. Deduplicate the incoming set on the same key the filter compares (Name; Guid for Guids), in both copies of the idiom: - PlexMovieLibraryScanner.UpdateMetadata (the original) — genres, studios, actors, directors, writers, guids, tags. - JellyfinMusicVideoLibraryScanner.Reconcile{Genres,Tags,Studios,Artists} (added in #497, mirrors the Plex pattern verbatim). Plex ACTORS are the exception and get an artwork-preferring dedup hoisted out and shared with the remove filter, because that filter is keyed on (Name, artwork-presence) — it is the mechanism that drops an artwork-less actor so the add loop can re-add it WITH artwork. A bare DistinctBy(a => a.Name) there keeps the FIRST duplicate, so Plex listing the artwork-less copy first discarded the artwork; worse, the remove filter would still see the artwork-less duplicate, making its upgrade clause false, so the stale row was never removed and the artwork never arrived on ANY later scan either. Actor also carries Role/Order, which first-wins would silently drop too. Caught by the cold review of the first version of this commit. For the Jellyfin scanner the dedup sits at the incoming-list declaration, which also covers the remove filter — safe because all four of those filters only ask "is this name present at all", an answer duplicates cannot change. Tests: duplicate-collapse for both paths, the two Actors cases above, and a POSITIVE CONTROL proving distinct entries are still all added and stale ones still removed (without it, a mis-keyed dedup that collapsed genuinely different entries would pass every other assertion). Each proven non-vacuous. The Plex tests drive the protected UpdateMetadata through a minimal test-only subclass, as MediaServerMovieLibraryScannerTests already does. Low likelihood in practice (a media server emitting two identically-named genres for one item is unusual); this is defensive, with no observed occurrence. The same idiom is copied into 8 further scanners/repositories that this change deliberately does not touch (the issue scoped it to two paths) — filed as #600 so the class of bug is tracked rather than silently left in the majority of its instances. The dedup rule is recorded under scan.musicvideo-reconciliation. fixes #500	2026-07-25 13:04:38 +02:00
timothy	b45dcc7190	fix(177): map Album/Track in the Jellyfin music video projection ... Jellyfin-sourced music videos rendered weaker MTV-style credits than local NFO libraries: the Scriban credits templates expose Album/Track, and MusicVideoNfoReader has always mapped both, but the Jellyfin projection never did. ChronologicalMediaComparer orders music videos by the same two fields, so they were also ordering worse. Verified against the live server (1437 MusicVideo items): Album comes back on 111 and IndexNumber on 4, both WITHOUT being named in the `fields` query param -- Album is a plain BaseItemDto property, not an ItemFields value, so no Refit `fields` change is needed (and adding one would be wrong). ParentIndexNumber is deliberately NOT used for Track: on live data, where both are present ParentIndexNumber is 1 while IndexNumber carries the real ordinal, and where only ParentIndexNumber is present it is a collection/disc grouping that tracks the Album ("Glastonbury: 2022" -> 230). The fix is two layers, not one. The projection alone would only ever reach music videos ADDED after it -- UpdateMetadata copies scalars field by field, so an existing item whose album/track is set or corrected in Jellyfin would keep a stale value forever. That is the same class of bug #497 fixed for child collections, one layer up. Also strips a pre-existing UTF-8 BOM from JellyfinLibraryItemResponse.cs, which the format gate flags once the file is touched (format-as-you-touch). fixes #177	2026-07-21 23:21:10 +02:00
timothyandClaude Opus 4.8	93188eeb5e	feat(525): download external-url logo on channel create + create-from-lineup ... Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-21 13:13:21 +02:00
timothyandClaude Opus 4.8	c633b89ad9	fix(497): reconcile music-video metadata collections on Jellyfin rescan ... JellyfinMusicVideoLibraryScanner.UpdateMetadata copied only scalar fields for an EXISTING music video, so genres/tags/studios/artists edited in Jellyfin never reached ErsatzTV — the update path silently dropped every child collection (only the Add path ever persisted them). Root cause is inherited from upstream: unlike the movie/episode Jellyfin path, which reconciles collections inside the tracked repository GetOrAdd, MusicVideoRepository.GetOrAdd is AsNoTracking and the scanner never reconciled the collections itself. Fix mirrors PlexMovieLibraryScanner.UpdateMetadata's remove-stale + add-new idiom, reconciling exactly the collections that BOTH the Add path persists AND GetOrAdd eager-loads: Genres, Tags, Studios, Artists. Guids (add-persisted but not eager-loaded — would duplicate) and Directors (eager-loaded but not add-persisted for music videos) are deliberately out of scope. Movie/Episode paths do NOT have this gap (they reconcile in the tracked repo GetOrAdd), so no separate fix is needed there. Test is an interaction test (substituted repos, canned existing item) verifying the exact reconcile calls; proven non-vacuous. The real-DB double-scan approach can't drive this: the in-memory harness shares one SQLite connection across contexts and mid-scan GetOrAdd's First()-nav path predicate mis-resolves once the existing item carries metadata children — a harness-only quirk (prod uses per-context pooled connections). fixes #497 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-20 18:32:29 +02:00
timothyandtimothy	5b0d8b5d09	fix(494): reconcile removed music videos in Jellyfin scanner (#495) ... fix(494): reconcile removed music videos in Jellyfin scanner (#495) JellyfinMusicVideoLibraryScanner did add/update only, so a music video or artist deleted on the Jellyfin side lingered forever. Add a library-scoped trash sweep (TrashMissingMusicVideos) gated by the #477 empty-fetch guard. fixes #494 Co-authored-by: Timothy <timothy.look@gmail.com> Co-committed-by: Timothy <timothy.look@gmail.com>	2026-07-20 15:22:15 +00:00
timothyandClaude Opus 4.8	d3db2f6af1	fix(488): resolve LibraryFolder from DB so Jellyfin music-video scans stop NRE'ing ... GetOrAddFolder read libraryPath.LibraryFolders, a navigation collection only eager-loaded on the local scan path (via GetLibrary). The Jellyfin remote sync path takes its LibraryPath off the JellyfinLibrary entity, where LibraryFolders is null, so .Filter(null) threw ArgumentNullException('source') on the first item of every Jellyfin music-video scan — a feature that had therefore never run in prod, CI, or locally. Look the folder up from the DB by (LibraryPathId, Path) instead, removing the implicit eager-load contract entirely (correct for all nine callers) and documenting it on ILibraryRepository. null != empty is preserved so a re-scan does not insert duplicate LibraryFolder rows. No new hot-path cost: local scanners already query GetParentFolderId per folder just before this call. Tests: - LibraryRepositoryTests: GetOrAddFolder with a null LibraryFolders (the exact remote-path shape) creates the folder, is idempotent on re-scan, and persists a supplied ParentId. - JellyfinMusicVideoLibraryScannerTests: an end-to-end scan of one synthetic music video, wiring the REAL LibraryRepository/ArtistRepository/ MusicVideoRepository against in-memory SQLite (the existing MediaServer* scanner tests mock every repo, which is why the bug escaped), asserts the scan completes and creates Artist + MusicVideo rows with a real LibraryFolder. Both proven non-vacuous against the reverted fix (each reproduces the issue's ArgumentNullException). decisions.md entry added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-20 01:16:16 +02:00
timothyandClaude Opus 4.8	61bd374494	fix(476): cascade FileNotFound from removed shows/seasons to descendants ... The media-server television scanner reconciles removed items at three levels via existing.Except(incoming), but the season and episode sweeps live inside the per-parent loops (ScanSeasons inside the show loop, ScanEpisodes inside the season loop). Those loops only iterate parents present in the incoming list, so a show (or season) that is gone from the media server is never visited and its descendants are never swept — they keep their last state. On Jellyfin/Emby that is RemoteOnly, which PlayoutBuilder does NOT skip even with PlayoutSkipMissingItems on, so every orphaned episode keeps getting scheduled as a guaranteed tune-in failure (the #473 reproduction; 717 stale prod rows across 10 removed shows). Fix: cascade the flag by parent MediaItem.Id. Two provider-agnostic repo helpers (Season.ShowId / Episode.SeasonId are on the base tables) flag descendants and the scanner drives them after each parent sweep — show → seasons → episodes, and season → episodes for the show-present case. Shared abstract base fixes Jellyfin/Plex/Emby at once; Movie/OtherVideo are flat and have no such gap. Tests: a Scanner.Tests case asserts the scanner cascades through the (substituted) repository (non-vacuous — fails if the cascade calls are removed), and Integration tests exercise the real cascade SQL against the schema, proving it flags only the targeted subtree and no-ops on empty input. fixes #476 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-19 21:49:33 +02:00
timothyandClaude Opus 4.8	02a493e95e	feat(#259): id-based reconcile for schedule-items replace (backend + DTO) ... Add optional `int? Id` to ScheduleItemRequest/ReplaceProgramScheduleItem so a client can round-trip each existing item's server id. When ids are present, ReplaceProgramScheduleItemsHandler reconciles by id (not array position), so an item's persisted fill-group/shuffle state (PlayoutScheduleItemFillGroupIndex, FK OnDelete Cascade) follows the logical item across reorders/inserts instead of being inherited by whatever previously occupied its new slot (#259, split from #252/#253). A fully id-less payload keeps the verbatim positional fallback. Guards (inside PersistItems, after CheckVersion so 412 precedes 422): duplicate id -> 422; id not in this schedule -> 422 (a stale id under Phase-1 force-write is a live lost-update signal, not a new item). Index stays array-position derived. Regenerated v1.json + TS client. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-11 21:43:52 +02:00
timothy	b53b06c615	feat(api): add playout REST endpoints ... fixes #37	2026-06-29 20:31:18 +02:00
timothy	b552905799	feat(api): add schedule REST endpoints ... Refs #36	2026-06-29 08:13:14 +02:00
timothy	fb3f2856da	feat(api): add collections REST endpoints ... Refs #35	2026-06-27 21:43:11 +02:00
timothyandClaude Opus 4.8	8f7d240fc8	feat(api): REST foundation + Channels CRUD (#34) ... First slice of the REST API (ersatztv#2). Idiomatic REST over existing MediatR handlers; design in docs/rest-api.md. Foundation: NotFoundError : BaseError (Core) to split 404 from 422; ApiResults mapping helpers (Either/Option -> 201+Location / 200 / 204 / 404 / 422), additive (existing *ToActionResult untouched); ApiKeyAuthorizationFilter (optional X-Api-Key on mutating actions, gated by Api:WriteKey, no-op when unset) — independent of the IPTV JWT toggle so Jellyfin/Dispatcharr reads are unaffected, applied at controller-class level so every mutating action incl. ResetPlayout is covered fail-safe. Channels: POST /api/channels (201+Location), PUT/{id} (200/404), DELETE/{id} (204/404), GET/{id} (200/404). Request DTOs -> existing commands; responses reuse ChannelViewModel. Ported page-only validations into handlers (ShowInEpg-when-disabled, external-logo-URL, Group NotEmpty) + FFmpegProfile/Watermark/Filler existence on update for create-parity. Fixed a latent 500: the .Filter(c>0) existence pattern threw TaskCanceledException on not-found; rewritten to AnyAsync. Tests (NUnit, TZ=UTC): handler success/404/422, ApiResults mapping, API-key filter, controller status/Location/mapping, a controller-security regression net, and a create->read->delete EF integration test on a new in-memory SQLite harness. ErsatzTV.Tests 46/46, Architecture 5/5. Refs #34 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-27 15:51:06 +02:00