Closes two separately-filed bugs that turned out to be one mechanism, and the sweep for that
mechanism found it at nine sites rather than the two reported.
The defect
A paged handler computed CountAsync over the whole DbSet and then paged a differently-filtered IQueryable, so TotalCount reported the size of the table whenever a filter applied. The page is
right and the count is wrong, which is silent and shaped like working software: the SPA paginates on TotalCount and renders pages that can never fill, and an API caller paging toward a completeness
target reads a total its own page cannot reach.
Scope: derived by shape, not by the issues' file lists
Six handlers in ErsatzTV.Application, of which #690 and #758 named two: GetPagedCollections, GetPagedMultiCollections, GetPagedRerunCollections (#690), GetPagedSmartCollections, GetPagedPlayouts (#758), GetPagedProgramSchedules.
GetPagedMultiCollections and GetPagedSmartCollections were in the half-state that argues for a
structural fix rather than a patched predicate: their counts carried the OwnedByChannelId == null
clause faithfully and silently omitted the Query clause added later. So both are derived from one IQueryable and cannot drift; includes move to the page chain, where a COUNT never needed them. GetPagedPlayouts additionally gains its Channel != null clause in the count.
Three more sites in ErsatzTV.Application/MediaCards, found only by deriving the population from the
SHAPE instead of the GetPaged* name — there the count and the page are two repository methods, so
the structural fix has nothing to attach to and a test pins their agreement instead:
Method
Was
Now
GetSeasonCount
counts ShowId == showId
expands to the same Title+Year show set GetPagedSeasons pages
GetEpisodeCount
counts Episode
counts EpisodeMetadata, the table the page reads
GetMusicVideoCount
counts MusicVideo
counts MusicVideoMetadata, the table the page reads
GetPagedFillerPresets and GetPagedTraktLists had no bug (no Query parameter) but now take the
same shape, so the first filter added cannot leave their counts behind.
Verification
11 new tests across two fixtures; expected values are pinned literals, never re-derived from the
handler's own predicate. One case pins the property the issues are actually about: pageSize 1 over
2 matching rows, asserting a real second page and an empty page past the last match.
Mutation proof. Restoring the pre-fix clause at all six handler sites turns all six PagedQueryTotalCountTests red, and each test constructs exactly one handler, so each is shown to
detect its own site. Restoring all three pre-fix repository counts turns all four MediaCardsCountMatchesPageTests red.
ErsatzTV.Tests 2017 passed / 4 skipped; ErsatzTV.Core.Tests 685; ErsatzTV.Scanner.Tests 1504; ErsatzTV.Infrastructure.Tests 114; scripts/tests 956. Full solution builds; format + BOM gates
pass (pre-existing BOMs stripped from the five touched files, per #311 fix-as-you-touch).
Docs (same PR, per the docs-update rule)
New record api.paged-count-matches-page-query + regenerated catalog, the docs/api-conventions.md paged-GET exemplar, and a docs/README.md task-signal row resolving by key.
Deferred, with the reasons recorded rather than assumed
#832 — the three MediaCards handlers page 1-based against api.paging-zero-based, and all three are
unreachable (Blazor leftovers, zero call sites), so delete-vs-keep should be decided before migrating
their paging. It also records a gap this PR does not close: an include chain can filter more
narrowly than the count, so "count the table the page reads" is necessary and not sufficient there.
Closes two separately-filed bugs that turned out to be one mechanism, and the sweep for that
mechanism found it at nine sites rather than the two reported.
## The defect
A paged handler computed `CountAsync` over the whole `DbSet` and then paged a differently-filtered
`IQueryable`, so `TotalCount` reported the size of the table whenever a filter applied. The page is
right and the count is wrong, which is silent and shaped like working software: the SPA paginates on
`TotalCount` and renders pages that can never fill, and an API caller paging toward a completeness
target reads a total its own page cannot reach.
## Scope: derived by shape, not by the issues' file lists
Six handlers in `ErsatzTV.Application`, of which #690 and #758 named two:
`GetPagedCollections`, `GetPagedMultiCollections`, `GetPagedRerunCollections` (#690),
`GetPagedSmartCollections`, `GetPagedPlayouts` (#758), `GetPagedProgramSchedules`.
`GetPagedMultiCollections` and `GetPagedSmartCollections` were in the half-state that argues for a
structural fix rather than a patched predicate: their counts carried the `OwnedByChannelId == null`
clause faithfully and silently omitted the `Query` clause added later. So both are derived from **one**
`IQueryable` and cannot drift; includes move to the page chain, where a `COUNT` never needed them.
`GetPagedPlayouts` additionally gains its `Channel != null` clause in the count.
Three more sites in `ErsatzTV.Application/MediaCards`, found only by deriving the population from the
SHAPE instead of the `GetPaged*` name — there the count and the page are two *repository methods*, so
the structural fix has nothing to attach to and a test pins their agreement instead:
| Method | Was | Now |
|---|---|---|
| `GetSeasonCount` | counts `ShowId == showId` | expands to the same Title+Year show set `GetPagedSeasons` pages |
| `GetEpisodeCount` | counts `Episode` | counts `EpisodeMetadata`, the table the page reads |
| `GetMusicVideoCount` | counts `MusicVideo` | counts `MusicVideoMetadata`, the table the page reads |
`GetPagedFillerPresets` and `GetPagedTraktLists` had no bug (no `Query` parameter) but now take the
same shape, so the first filter added cannot leave their counts behind.
## Verification
- 11 new tests across two fixtures; expected values are pinned literals, never re-derived from the
handler's own predicate. One case pins the property the issues are actually about: `pageSize` 1 over
2 matching rows, asserting a real second page and an empty page past the last match.
- **Mutation proof.** Restoring the pre-fix clause at all six handler sites turns all six
`PagedQueryTotalCountTests` red, and each test constructs exactly one handler, so each is shown to
detect its own site. Restoring all three pre-fix repository counts turns all four
`MediaCardsCountMatchesPageTests` red.
- `ErsatzTV.Tests` 2017 passed / 4 skipped; `ErsatzTV.Core.Tests` 685; `ErsatzTV.Scanner.Tests` 1504;
`ErsatzTV.Infrastructure.Tests` 114; `scripts/tests` 956. Full solution builds; format + BOM gates
pass (pre-existing BOMs stripped from the five touched files, per #311 fix-as-you-touch).
## Docs (same PR, per the docs-update rule)
New record `api.paged-count-matches-page-query` + regenerated catalog, the
`docs/api-conventions.md` paged-GET exemplar, and a `docs/README.md` task-signal row resolving by key.
## Deferred, with the reasons recorded rather than assumed
#832 — the three MediaCards handlers page 1-based against `api.paging-zero-based`, and all three are
unreachable (Blazor leftovers, zero call sites), so delete-vs-keep should be decided before migrating
their paging. It also records a gap this PR does **not** close: an include chain can filter more
narrowly than the count, so "count the table the page reads" is necessary and not sufficient there.
fixes #690
fixes #758
refs #832
`GetPagedRerunCollections` (#690) and `GetPagedPlayouts` (#758) were filed as
separate bugs; they are one mechanism, and the sweep found it at six sites.
Each handler computed `CountAsync` over the whole `DbSet` and then paged a
differently-filtered `IQueryable`, so `TotalCount` reported the size of the
table whenever a search query was supplied. The page is right and the count is
wrong, which is silent: the SPA paginates on `TotalCount` and renders pages
that can never contain anything, and an MCP agent paging to a completeness
target reads a total its own page can never reach.
The two named handlers were two of six. Sweeping `ErsatzTV.Application` for the
shape rather than auditing the issues' file lists also found
`GetPagedCollections`, `GetPagedMultiCollections`, `GetPagedSmartCollections`
and `GetPagedProgramSchedules`. `GetPagedMultiCollections` and
`GetPagedSmartCollections` were in the half-state that makes the structural fix
the right one: their counts carried the `OwnedByChannelId == null` clause
faithfully and silently omitted the `Query` clause added later.
So the fix is not "add the missing Where to the count" — both are derived from
one `IQueryable` and cannot drift. Includes move to the page chain, where they
already belonged: a COUNT does not materialize the graph, and #671's bound on
the rerun selection graph is unaffected. `GetPagedPlayouts` additionally gains
its `Channel != null` filter in the count, which the DbSet count never had.
`GetPagedFillerPresets` and `GetPagedTraktLists` take no `Query` parameter, so
their DbSet counts already count what they page and are deliberately unchanged.
`GetBlockPlayoutHistory`, `GetFuturePlayoutItemsById` and `GetLibraryBrowseItems`
already used this idiom.
Mutation proof: restoring the pre-fix `CountAsync` clause at all six sites turns
all six new tests red; each test constructs exactly one handler, so every test
is shown to detect its own site. Expected counts are pinned literals, never
recomputed by re-applying the handler's own predicate.
BOMs stripped from the three touched handlers that carried one (#311
fix-as-you-touch; `dotnet format whitespace` reported CHARSET on them).
fixes#690fixes#758
Round 1 of the cold adversarial review found no correctness defect in the six
handler fixes, and three real defects in what the change CLAIMED. Answering
them here rather than in the doc, per docs.no-session-narrative.
1. The record said the population was "eight `GetPaged*Handler` types, fully
enumerated". That is a NAME-derived population — the exact method the record
itself condemns two paragraphs earlier. Deriving it by SHAPE from the git
index (CountAsync / .Skip( / TotalCount) finds three more count+page
producers in ErsatzTV.Application/MediaCards with the same drift:
GetTelevisionSeasonCards (counts ShowId == showId, pages every show sharing
a Title+Year), GetTelevisionEpisodeCards and GetMusicVideoCards (count the
item table, page the metadata table). All three are unreachable dead code
left by blazor.ui-removed — zero call sites outside their own Queries folder
— and page 1-based against api.paging-zero-based, so they are DEFERRED to
#832 and recorded, not folded in. An unrecorded site reads as a clean tree.
2. docs/api-conventions.md was not updated. CLAUDE.md's docs table makes that a
same-PR hard rule, and its "Paged GET with clamped params" exemplar is where
a controller author actually looks. It now carries the count rule.
3. The `Channel != null` claim was a mechanism I did not measure. Playout.ChannelId
is non-nullable with DeleteBehavior.Cascade and both production connection
strings set foreign keys=true, so the orphan state is unreachable while the FK
holds. The clause is defensive; the record no longer describes an inflated
total anyone saw. Same correction for blast radius: only rerun-collections,
multi-collections and playouts reach a controller — the other three had no
production caller and were latent, not live.
Also: GetPagedFillerPresets and GetPagedTraktLists now derive count and page
from one query too. Their DbSet counts were not wrong (they take no Query
parameter), but leaving them counting one expression while paging another
preserves the drift for whoever adds the first filter — which is the argument
the record makes, so exempting them contradicted it.
And a test that pins the property the issues are actually about: pageSize 1
over 2 matching rows, asserting a real second page and an empty page past the
last match. Every previous case fit on one page, so none exercised the count
driving pagination at all.
refs #832
Both cold reviews (different model families) independently rated the deferral
MEDIUM, and the deciding fact I had not checked when I deferred: each of the six
repository methods involved has exactly ONE caller — the dead handler. Zero
blast radius, so "cheap and safe" beat "dead code, leave it", and leaving a
known instance of the mechanism this PR's own record establishes is what rots.
- GetSeasonCount expanded to the same Title+Year show set GetPagedSeasons pages
(it counted only the requested ShowId, so the same show present in two
libraries under-reported: count 2 against a 5-row page).
- GetEpisodeCount and GetMusicVideoCount now count the METADATA table their
pages are taken from, not the item table. An item whose metadata row was lost
to a scanner failure — a state FindOrphanPaths and RemoveMetadata both model —
inflated the total and produced the permanently-empty-page symptom.
The 1-based pageNumber on these three is a separate defect against
api.paging-zero-based and stays open in #832, which is now narrowed to that plus
the delete-or-keep question.
Mutation proof: restoring all three pre-fix counts turns all four new
MediaCardsCountMatchesPageTests red.
Two findings that only appeared by writing those tests, both now pinned or
recorded rather than left as folklore:
- GetSeasonCount has a third answer — with no ShowMetadata row there is nothing
to expand from, so GetPagedSeasons returns nothing and the count must be 0,
not the show's season total. Pinned as its own case.
- The episode page can be narrower than even the metadata count: its include
chain walks Episode -> Season -> Show -> ShowMetadata and a missing Show drops
every row. So "count the table the page reads" is necessary and NOT sufficient
wherever an include chain can filter. Recorded in the record and #832 rather
than claimed closed.
Also from review: docs/README.md's row now resolves the rule by KEY through the
catalog rather than hard-linking record paths, per the catalog-first convention
it was violating.
refs #832
Codex round 2 found the code and tests sound and two defects in the record —
both staleness the previous commit introduced by changing the rule without
re-reading the whole record. That is the failure mode this repo already has on
file as "sweep by subject, not verb": I edited the rule text and left two other
places asserting the superseded version.
- The `signals:` line still carried "an unfiltered paged handler is not an
instance", which now contradicts the rule: GetPagedFillerPresets and
GetPagedTraktLists were changed to take the shape precisely so the first
filter added cannot leave their counts behind. A later shape audit following
the stale signal would exclude exactly the handlers the change included.
- The no-detector paragraph still cited GetPagedFillerPresets as "counts a
DbSet and is correct" — it counts the shared query now. Replaced with two
real false positives (GetPlayoutWarningsCount, DeleteFFmpegProfileHandler's
usage check, both verified filtered-count-with-no-page) and the observation
that a same-file detector cannot see the MediaCards sites at all.
Also scoped the include-chain gap the same way the Channel != null clause is
scoped, rather than leaving it reading as a live open defect: Season.ShowId is
non-nullable with DeleteBehavior.Cascade and production enforces the FK, so
count 1 / page 0 is corruption-only. It is stated because it makes "count the
table the page reads" necessary and not sufficient in general, not because
something reachable is being left unfixed.
Round 2's second reviewer returned MERGEABLE with no correctness defect, and
reproduced the mutation proof rather than trusting it. Three accuracy findings,
each backed by a probe I re-verified against the entity definitions.
The include-chain gap is not one pair, it is all three, and I had named the
wrong link. A REQUIRED reference Include is emitted as an INNER JOIN, so
GetPagedSeasons (Include(s => s.Show)) and GetPagedMusicVideos
(ThenInclude(mv => mv.Artist)) have the identical gap, not just GetPagedEpisodes.
And the link I blamed was inert: Show.ShowMetadata is a List<>, a COLLECTION nav
and therefore a LEFT JOIN, which drops nothing — the reviewer's probe seeds the
full graph minus ShowMetadata and still pages 2 rows. The load-bearing legs are
Season and Show. The test comment and the record both narrated a chain one link
longer than the one that actually filters, and the seeded ShowMetadata row is
incidental; it is now labelled as such rather than presented as the reason the
seed exists.
That distinction is the mechanism, so the record now states it: "an include
filters" is false as a blanket claim and true only of required references.
Frontmatter was still describing the pre-MediaCards change: paths listed six
handlers and one test file, omitting both repositories, the second test fixture
and the two unfiltered handlers, and mechanics named only PagedQueryTotalCountTests
though the record's own point (3) makes the MediaCards test a second enforcement
mode. Retrieval by those signals would have reproduced exactly the name-derived
population this record condemns.
Not changed, with the reason recorded: TelevisionRepository.GetSeasonCount passes
CancellationToken.None because ITelevisionRepository.GetSeasonCount(int) has no
token parameter, so there is nothing to thread. Threading one means an interface
change touching FakeTelevisionRepository for a dead caller. The asymmetry (the
count is uncancellable while its sibling GetPagedSeasons is cancellable, and it
went from 1 round trip to 3) is real and has zero impact while the caller is dead
code; #832 owns the delete-or-keep decision that settles it.
Self-review pass over the whole record after two fix rounds, because both of my
last two defects were introduced BY a fix round rather than caught in the
original — editing a rule and leaving other sentences asserting the old one.
- 'Two corollaries' introduced a third and was never updated.
- The no-detector paragraph attributed the three MediaCards sites to #832 as if
still open. They are fixed in this PR; #832 carries only the 1-based paging and
the delete-or-keep question. Left as written it would have told a reader the
tree still has three live instances.
- The mutation-proof sentence described only the handler mode and 'all six tests'
after a seventh case was added; it now states both modes and their real counts.
- 'one test per filtering instance' became 'at least one' once the second-page
case landed.
A bounded cold review of the docs-only commits (which no reviewer had seen)
found the same failure a third time — the previous commit fixed the body wording
and left `mechanics:` asserting the superseded version — and one finding that
was not prose at all.
`mechanics:` claimed both fixtures "assert LITERAL expected counts and expected
page contents". False for MediaCardsCountMatchesPageTests: all four cases
asserted only `count.ShouldBe(n)` and `page.Count.ShouldBe(n)`. A count and a
page can agree on the WRONG SET and stay green, so the fixture pinned agreement
without pinning correctness. The honest repair is the test, not the sentence:
each case now pins the identities on the page (season ids, episode ids, music
video titles).
Mutation proving those assertions are load-bearing rather than decorative:
changing GetPagedSeasons' predicate to `s.Id != 11` pages a SAME-SIZE wrong set
(5 rows, [12,13,14,15,19] instead of [11..15]). Both count assertions still
pass; the identity assertion fails. Executed, witnessed red, reverted.
Remaining prose corrections from the same review, each checked against the tree:
- `mechanics:` "one test per filtering paged handler" -> "at least one" (7 cases
over 6 handlers since the second-page test landed) — the exact residual the
previous commit set out to remove, one field over.
- "Each seeds five rows of which exactly two match Alpha" was untrue of the
handlers carrying a non-search clause, which seed a sixth row that clause must
exclude from both sides. Stated as such.
- The corruption-only scoping enumerated Season.ShowId and MusicVideo.ArtistId
but not Episode.SeasonId, the first inner join in the episode chain.
- The title described only enforcement mode 1, so a retrieval hit on the title
alone under-described the record.
- api-conventions.md said "beside a filtered page query", which reads as
permission to count the DbSet while no filter exists yet — the opposite of
corollary (2).
Four cold review rounds, two model families. Rounds 1-3 findings all fixed; round 4 (bounded, on the final commit) MERGEABLE with three cosmetic nits accepted rather than triggering another CI cycle: a stray blank line and two unwrapped doc lines, format gate green. Code and tests cleared by two independent reviewers; mutation proofs executed in both enforcement modes and independently reproduced.
Review-verdict: MERGEABLE @ f392879
Four cold review rounds, two model families. Rounds 1-3 findings all fixed; round 4 (bounded, on the final commit) MERGEABLE with three cosmetic nits accepted rather than triggering another CI cycle: a stray blank line and two unwrapped doc lines, format gate green. Code and tests cleared by two independent reviewers; mutation proofs executed in both enforcement modes and independently reproduced.
timothy
merged commit 08cd3a002d into main2026-08-26 09:39:25 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes two separately-filed bugs that turned out to be one mechanism, and the sweep for that
mechanism found it at nine sites rather than the two reported.
The defect
A paged handler computed
CountAsyncover the wholeDbSetand then paged a differently-filteredIQueryable, soTotalCountreported the size of the table whenever a filter applied. The page isright and the count is wrong, which is silent and shaped like working software: the SPA paginates on
TotalCountand renders pages that can never fill, and an API caller paging toward a completenesstarget reads a total its own page cannot reach.
Scope: derived by shape, not by the issues' file lists
Six handlers in
ErsatzTV.Application, of which #690 and #758 named two:GetPagedCollections,GetPagedMultiCollections,GetPagedRerunCollections(#690),GetPagedSmartCollections,GetPagedPlayouts(#758),GetPagedProgramSchedules.GetPagedMultiCollectionsandGetPagedSmartCollectionswere in the half-state that argues for astructural fix rather than a patched predicate: their counts carried the
OwnedByChannelId == nullclause faithfully and silently omitted the
Queryclause added later. So both are derived from oneIQueryableand cannot drift; includes move to the page chain, where aCOUNTnever needed them.GetPagedPlayoutsadditionally gains itsChannel != nullclause in the count.Three more sites in
ErsatzTV.Application/MediaCards, found only by deriving the population from theSHAPE instead of the
GetPaged*name — there the count and the page are two repository methods, sothe structural fix has nothing to attach to and a test pins their agreement instead:
GetSeasonCountShowId == showIdGetPagedSeasonspagesGetEpisodeCountEpisodeEpisodeMetadata, the table the page readsGetMusicVideoCountMusicVideoMusicVideoMetadata, the table the page readsGetPagedFillerPresetsandGetPagedTraktListshad no bug (noQueryparameter) but now take thesame shape, so the first filter added cannot leave their counts behind.
Verification
handler's own predicate. One case pins the property the issues are actually about:
pageSize1 over2 matching rows, asserting a real second page and an empty page past the last match.
PagedQueryTotalCountTestsred, and each test constructs exactly one handler, so each is shown todetect its own site. Restoring all three pre-fix repository counts turns all four
MediaCardsCountMatchesPageTestsred.ErsatzTV.Tests2017 passed / 4 skipped;ErsatzTV.Core.Tests685;ErsatzTV.Scanner.Tests1504;ErsatzTV.Infrastructure.Tests114;scripts/tests956. Full solution builds; format + BOM gatespass (pre-existing BOMs stripped from the five touched files, per #311 fix-as-you-touch).
Docs (same PR, per the docs-update rule)
New record
api.paged-count-matches-page-query+ regenerated catalog, thedocs/api-conventions.mdpaged-GET exemplar, and adocs/README.mdtask-signal row resolving by key.Deferred, with the reasons recorded rather than assumed
#832 — the three MediaCards handlers page 1-based against
api.paging-zero-based, and all three areunreachable (Blazor leftovers, zero call sites), so delete-vs-keep should be decided before migrating
their paging. It also records a gap this PR does not close: an include chain can filter more
narrowly than the count, so "count the table the page reads" is necessary and not sufficient there.
fixes #690
fixes #758
refs #832
Review-verdict: MERGEABLE @
f392879Four cold review rounds, two model families. Rounds 1-3 findings all fixed; round 4 (bounded, on the final commit) MERGEABLE with three cosmetic nits accepted rather than triggering another CI cycle: a stray blank line and two unwrapped doc lines, format gate green. Code and tests cleared by two independent reviewers; mutation proofs executed in both enforcement modes and independently reproduced.