Two more list loads truncate at the cap — but at-cap, so #644's over-cap grep could not see them #650

Closed
opened 2026-07-26 20:49:10 +02:00 by timothy · 2 comments
Owner

Found by the cold review of the #644 PR, both confirmed by reading the code. Same defect class as #634/#644 — "took result.page as the whole list with no paging UI" — but these request exactly the server cap rather than exceeding it, so #644's completeness grep (pageSize: above an endpoint's cap) structurally could not find them.

That is the reusable lesson: the grep guard keys on the symptom (an inflated pageSize), not the defect (treating one page as the whole list). An at-cap request is just as truncating.

The two sites

1. web/src/screens/TraktListsScreen.tsx:297 — the screen contradicts itself on screen

getTraktLists({ pageSize: 100 }); TraktController.cs:24 clamps at MaxPageSize = 100. It then does setLists(paged.page ?? []) and setTotalCount(paged.totalCount ?? 0), and line 500 renders {totalCount} list{s}.

So with 101 Trakt lists the screen displays the text "101 lists" above a table containing 100 rows. There is no paging UI. The truncation is not merely silent, it is visibly self-contradictory, which at least makes it diagnosable.

2. web/src/builder/ChannelBuilder.tsx:513-521 — canLoadMore can never fire

loadCollections fans out getLibraryBrowseItems({ query, mediaType, pageSize: 100 }) per kind, then returns { page: merged, totalCount: merged.length }. Reporting the truncated page length as the total means any canLoadMore-style comparison is merged.length >= merged.length — permanently false.

The sibling loadLibraryItems immediately below it carries the comment "preserves real paging (each kind keeps its own pageNum/totalCount, summed across kinds) so canLoadMore stays meaningful for large libraries" — so the correct pattern already exists in the same file, three lines away. loadCollections simply never got it.

Fix

Sum the real per-kind totalCounts in loadCollections (mirror loadLibraryItems). For Trakt, either page to completeness via the shared loadAllPages helper added in #644 (the list is small and admin-created, so this is the safe class) or show the truncation explicitly.

Do not simply raise the caps — api.search-allitems-paging is that the client pages and the server stays bounded.

Done-when

  • TraktListsScreen can no longer display a totalCount larger than the rows it renders
  • loadCollections reports a real total, and canLoadMore demonstrably fires past 100 merged items
  • The #644 completeness guard is extended to catch the AT-cap shape, not only the over-cap one (otherwise the next instance is invisible again)
  • Tests pin the exact expected set at the boundary, not a non-empty check
  • Adversarial review passed

Resolution notes (PR #675)

There was no guard to extend. #644's box 4 was a manual grep, not codified. So box 3 was satisfied by building one — and deliberately not as another value-pattern match, since that repeats the mistake this issue names. web/src/api/pageSizeScan.ts walks the TypeScript AST, keying identity on exact line:column with multiset comparison against a registry. Comments, strings, template interpolation, JSX containers, destructuring, type literals, as/satisfies, and quoted/computed keys become node kinds rather than lexer edge cases. An interim hand-rolled scanner treated the // in 'https://…' as a comment and reported zero sites for that whole file — the dangerous failure direction, and the reason the AST rewrite was worth it.

A third at-cap site turned up that this issue didn't name: ChannelBuilder.tsx's SeasonsDialog (TelevisionSeason). Judged correctly Class B — no show has 100 seasons — so it keeps one bounded page, but its previously-unread totalCount now drives a truncation hint.

Deferred to #665: cross-page sort order and an overclaimed totalCount permitting no-op "Load more" — both pre-existing in the sibling loadLibraryItems path, made newly reachable here.

Found by the cold review of the #644 PR, both confirmed by reading the code. Same defect class as #634/#644 — *"took `result.page` as the whole list with no paging UI"* — but these request **exactly** the server cap rather than exceeding it, so #644's completeness grep (`pageSize:` above an endpoint's cap) structurally could not find them. That is the reusable lesson: the grep guard keys on the *symptom* (an inflated `pageSize`), not the *defect* (treating one page as the whole list). An at-cap request is just as truncating. ## The two sites ### 1. `web/src/screens/TraktListsScreen.tsx:297` — the screen contradicts itself on screen `getTraktLists({ pageSize: 100 })`; `TraktController.cs:24` clamps at `MaxPageSize = 100`. It then does `setLists(paged.page ?? [])` **and** `setTotalCount(paged.totalCount ?? 0)`, and line 500 renders `{totalCount} list{s}`. So with 101 Trakt lists the screen displays the text **"101 lists"** above a table containing **100** rows. There is no paging UI. The truncation is not merely silent, it is visibly self-contradictory, which at least makes it diagnosable. ### 2. `web/src/builder/ChannelBuilder.tsx:513-521` — `canLoadMore` can never fire `loadCollections` fans out `getLibraryBrowseItems({ query, mediaType, pageSize: 100 })` per kind, then returns `{ page: merged, totalCount: merged.length }`. Reporting the **truncated page length as the total** means any `canLoadMore`-style comparison is `merged.length >= merged.length` — permanently false. The sibling `loadLibraryItems` immediately below it carries the comment *"preserves real paging (each kind keeps its own pageNum/totalCount, summed across kinds) so canLoadMore stays meaningful for large libraries"* — so the correct pattern already exists in the same file, three lines away. `loadCollections` simply never got it. ## Fix Sum the real per-kind `totalCount`s in `loadCollections` (mirror `loadLibraryItems`). For Trakt, either page to completeness via the shared `loadAllPages` helper added in #644 (the list is small and admin-created, so this is the safe class) or show the truncation explicitly. **Do not** simply raise the caps — `api.search-allitems-paging` is that the client pages and the server stays bounded. ## Done-when - [x] `TraktListsScreen` can no longer display a `totalCount` larger than the rows it renders - [x] `loadCollections` reports a real total, and `canLoadMore` demonstrably fires past 100 merged items - [x] The #644 completeness guard is extended to catch the AT-cap shape, not only the over-cap one (otherwise the next instance is invisible again) - [x] Tests pin the exact expected set at the boundary, not a non-empty check - [x] Adversarial review passed --- ## Resolution notes (PR #675) **There was no guard to extend.** #644's box 4 was a *manual grep*, not codified. So box 3 was satisfied by building one — and deliberately not as another value-pattern match, since that repeats the mistake this issue names. `web/src/api/pageSizeScan.ts` walks the TypeScript AST, keying identity on exact `line:column` with multiset comparison against a registry. Comments, strings, template interpolation, JSX containers, destructuring, type literals, `as`/`satisfies`, and quoted/computed keys become node kinds rather than lexer edge cases. An interim hand-rolled scanner treated the `//` in `'https://…'` as a comment and reported **zero** sites for that whole file — the dangerous failure direction, and the reason the AST rewrite was worth it. **A third at-cap site** turned up that this issue didn't name: `ChannelBuilder.tsx`'s `SeasonsDialog` (`TelevisionSeason`). Judged correctly Class B — no show has 100 seasons — so it keeps one bounded page, but its previously-unread `totalCount` now drives a truncation hint. **Deferred to #665**: cross-page sort order and an overclaimed `totalCount` permitting no-op "Load more" — both pre-existing in the sibling `loadLibraryItems` path, made newly reachable here.
timothy added the bugfrontendpriority: medium labels 2026-07-26 20:49:10 +02:00
timothy added the in-progress label 2026-07-26 22:57:09 +02:00
Author
Owner

Claiming — worked as part of a three-issue frontend bundle this session: #650 + #651 + #578 (all downstream of #644's cold review; #578 folds in because #651 puts us in the same combobox/SearchPicker code).

#650 runs in its own worktree (TraktListsScreen + ChannelBuilder.loadCollections + extending the #644 completeness guard to the at-cap shape), disjoint from #651's picker work.

Claiming — worked as part of a three-issue `frontend` bundle this session: **#650 + #651 + #578** (all downstream of #644's cold review; #578 folds in because #651 puts us in the same combobox/`SearchPicker` code). #650 runs in its own worktree (TraktListsScreen + ChannelBuilder.loadCollections + extending the #644 completeness guard to the at-cap shape), disjoint from #651's picker work.
Author
Owner

Closing record

Outcome: Merged as PR #675. TraktListsScreen pages to completeness via loadAllPages (checking complete, aborting on unmount, with a distinct incomplete state); ChannelBuilder.loadCollections sums the real per-kind totalCount instead of reporting the truncated page length, so canLoadMore can fire. A third at-cap site the issue didn't name — SeasonsDialog — was judged correctly Class B but now surfaces its previously-unread totalCount.

Root cause: #644's guard keyed on the symptom (an inflated pageSize) rather than the defect (treating one page as the whole list), so an at-cap request was structurally invisible to it. Compounding that, there was no codified guard to extend — #644's box 4 was a manual grep.

Decisions/conventions changed: None reversed. The new guard is a mechanism, not a convention change.

Reusable knowledge:

  • A hand-rolled lexer fails one input class at a time. Two rounds of patching the scanner (identity, then shorthand) were immediately followed by findings for comments-in-strings, template literals, ternary/?? context, JSX containers and same-line duplicates. Rewriting on the TypeScript AST closed all of them at once, because they become node kinds rather than edge cases. The dangerous one: stripComments treated the // in 'https://…' as a comment and reported zero sites for that whole file.
  • npx tsc --noEmit checks ZERO files here — the root tsconfig is solution-style ("files": [] + references). It was used for four review rounds and hid six real type errors. The gate is npm run typecheck (tsc -b).
  • Concurrency: eliminate the interleaving, don't enumerate it. Rounds 1–3 each fixed one ordering and the next found its mirror (page-0-first → page-1-first loses a page; duplication → a silently skipped page). Single-flight made the class unreachable and was smaller than what it replaced.
  • A test can encode the defect. The round-2 regression test asserted the next request was page 3 while page 1 was missing — it would have defended the data loss against the next person to fix it.

Verification: npm run lint clean · npm run typecheck clean · npx vitest run 110 files / 1078 tests. Four cold cross-family review rounds (read-only sandbox); round 4 found no runtime defect, round 5 MERGEABLE. Verified independently of the implementer: planted at-cap defects (duplicate in a registered file, shorthand, two same-line ternary branches, a URL-string file, as const/satisfies/quoted/computed keys, a .mts file) — all caught at distinct line:columns; and removing the single-flight guard fails the hook-level test with expected [ +0, 1 ] to deeply equal [ +0 ].

Deferred: #665 — cross-page sort order, and an overclaimed totalCount permitting no-op "Load more". Both pre-existing in the sibling loadLibraryItems path, made newly reachable here.

Docs updated: None required — no route, endpoint, convention or decision changed. The guard's contract and its documented residual gaps live in web/src/api/pageSizeScan.ts and pageSizeCallSites.guard.test.ts.

## Closing record **Outcome:** Merged as PR #675. `TraktListsScreen` pages to completeness via `loadAllPages` (checking `complete`, aborting on unmount, with a distinct incomplete state); `ChannelBuilder.loadCollections` sums the real per-kind `totalCount` instead of reporting the truncated page length, so `canLoadMore` can fire. A third at-cap site the issue didn't name — `SeasonsDialog` — was judged correctly Class B but now surfaces its previously-unread `totalCount`. **Root cause:** #644's guard keyed on the *symptom* (an inflated `pageSize`) rather than the *defect* (treating one page as the whole list), so an at-cap request was structurally invisible to it. Compounding that, there was no codified guard to extend — #644's box 4 was a manual grep. **Decisions/conventions changed:** None reversed. The new guard is a mechanism, not a convention change. **Reusable knowledge:** - **A hand-rolled lexer fails one input class at a time.** Two rounds of patching the scanner (identity, then shorthand) were immediately followed by findings for comments-in-strings, template literals, ternary/`??` context, JSX containers and same-line duplicates. Rewriting on the TypeScript AST closed all of them at once, because they become node kinds rather than edge cases. The dangerous one: `stripComments` treated the `//` in `'https://…'` as a comment and reported **zero** sites for that whole file. - **`npx tsc --noEmit` checks ZERO files here** — the root tsconfig is solution-style (`"files": []` + `references`). It was used for four review rounds and hid six real type errors. The gate is `npm run typecheck` (`tsc -b`). - **Concurrency: eliminate the interleaving, don't enumerate it.** Rounds 1–3 each fixed one ordering and the next found its mirror (page-0-first → page-1-first loses a page; duplication → a silently skipped page). Single-flight made the class unreachable and was smaller than what it replaced. - **A test can encode the defect.** The round-2 regression test asserted the next request was page 3 while page 1 was missing — it would have defended the data loss against the next person to fix it. **Verification:** `npm run lint` clean · `npm run typecheck` clean · `npx vitest run` 110 files / 1078 tests. Four cold cross-family review rounds (read-only sandbox); round 4 found no runtime defect, round 5 MERGEABLE. Verified independently of the implementer: planted at-cap defects (duplicate in a registered file, shorthand, two same-line ternary branches, a URL-string file, `as const`/`satisfies`/quoted/computed keys, a `.mts` file) — all caught at distinct line:columns; and removing the single-flight guard fails the hook-level test with `expected [ +0, 1 ] to deeply equal [ +0 ]`. **Deferred:** **#665** — cross-page sort order, and an overclaimed `totalCount` permitting no-op "Load more". Both pre-existing in the sibling `loadLibraryItems` path, made newly reachable here. **Docs updated:** None required — no route, endpoint, convention or decision changed. The guard's contract and its documented residual gaps live in `web/src/api/pageSizeScan.ts` and `pageSizeCallSites.guard.test.ts`.
timothy removed the in-progress label 2026-07-27 08:11:04 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: timothy/ersatztv#650