fix(650): two at-cap list truncations, and a completeness guard that keys on the defect #675
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: timothy/ersatztv#675
Reference in New Issue
Block a user
fixes #650
Two SPA list loads truncated at the server cap while requesting exactly the cap, so #644's over-cap
pageSize:grep structurally could not see them. The reusable lesson is in the issue title: that grep keyed on the symptom (an inflatedpageSize), not the defect (treating one page as the whole list).The two sites
TraktListsScreendisplayed{totalCount} listsabove a table holdingpage— "101 lists" over 100 rows. Trakt lists are bounded-by-construction (Class A), so this now pages to completeness vialoadAllPages, checks thecompleteflag, aborts on unmount, and renders a distinct incomplete state. A genuinely empty account still gets "No Trakt lists yet." (totalCount: 0→complete: true).ChannelBuilder.loadCollectionsreturned{ page: merged, totalCount: merged.length }— the truncated page length as the total, makingcanLoadMore'sitems.length < totalCountpermanently false. It now sums the real per-kindtotalCounts, mirroring the siblingloadLibraryItems. Thesource === 'library'gate oncanLoadMoreis removed because it only existed to compensate for the fake total.A third at-cap site found during the work —
SeasonsDialog(TelevisionSeason) — is judged correctly Class B (no show has 100 seasons) and keeps one bounded page, but its previously-unreadtotalCountnow drives a truncation hint.The guard (issue box 3)
There was no codified guard to extend: #644's box 4 was a manual grep. So this builds one, and deliberately not as another value-pattern match.
web/src/api/pageSizeScan.tswalks the TypeScript AST (ts.createSourceFile) collectingpageSizeproperty/shorthand assignments inside call-argument object literals, with exactline:columnidentity and multiset comparison against a registry. Source discovery uses Vite'simport.meta.glob, filtered through the named, testedisScannableSourceFileNamepredicate.Going via the AST closes an entire class rather than enumerating cases — comments, strings, template interpolation, JSX containers, destructuring, type literals,
as/satisfies, quoted and computed keys are node kinds, not lexer edge cases. An earlier hand-rolled scanner treated the//in'https://…'as a comment and reported zero sites for the whole file.pageSizeScan.test.tspins each case against inline fixture strings with no repo dependency; anti-vacuity floors guard the file and site counts so a broken glob or regex fails rather than passing empty.Concurrency fixes (found by review; newly reachable because of the above)
useLibraryBrowseallowed a page-0 refresh and an append to be in flight together. Successive review rounds each fixed one interleaving and found its mirror (page-0-first fixed → page-1-first lost a page; duplication fixed → a page was silently skipped). Rather than continue enumerating orderings, single-flight now makes overlap impossible: a synchronousbusyRefguard means a secondloadMore()is ignored, not queued, and the append-failure rollback is unconditional because nothing else can move the cursor.Also: an append failure no longer wipes already-loaded rows, and a superseded append no longer strands the loading spinner.
Review
Four adversarial rounds (cold, cross-family, read-only sandbox). Rounds 1–3 returned BLOCKED; round 4 found no runtime correctness defect and round 5 returned MERGEABLE. Two findings deferred as pre-existing in the sibling code path → #665 (cross-page sort order; an overclaimed
totalCountpermitting no-op "Load more").Verified independently of the implementer: planted at-cap defects (a duplicate in an already-registered file, a shorthand, two same-line ternary branches, a URL-string file,
as const/satisfies/quoted/computed keys, a.mtsfile) — all caught at distinct line:columns; and the single-flight guard removed → the hook-level test fails withexpected [ +0, 1 ] to deeply equal [ +0 ].Verification
npm run lintclean ·npm run typecheck(tsc -b— the real gate) clean ·npx vitest run110 files / 1078 tests passed. No.cstouched, so the BOM/format gate is not applicable.Cold cross-family (Codex) review of 9763fdca found real defects; addressed as a follow-up rather than amending that commit. MUST FIX, addressed: - F5: pageSizeCallSites.guard.test.ts collapsed call-site identity to `file:value`, so a SECOND at-cap call in an already-registered file was invisible (verified: appending a duplicate `getLibraryBrowseItems({ mediaType: 'Movie', pageSize: 100 })` to ChannelBuilder.tsx passed all 4 guard tests before this fix). Identity is now `file:line:kind:value` — a bracket/quote-tracked scan resolves each occurrence's exact line, so a duplicate on a new line is a new, unregistered identity. - F6: the guard now also detects the ES6 shorthand property form (`{ ..., pageSize }`), not just `pageSize: <value>`. Implemented as a bracket-stack scan that distinguishes an object-literal `{` (real risk) from a block-statement `{` or an array `[` (false positives from things like `useCallback` dependency arrays and `const pageSize = 100;` inside a function body) by inspecting the token immediately preceding each `{`. Six real shorthand sites are now registered (the two inside loadAllPages itself, ChannelBuilder's two per-kind fan-outs, and two genuine user-adjustable pagers in LogsScreen/BlockPlayoutTroubleshootingScreen). Object SPREAD and positional-argument pageSize (api/search.ts's api.search-allitems-paging precedent) remain a documented residual gap, written down in the test file's own header comment, not silently absent. - F2 (TraktListsScreen.tsx): an incomplete load with zero accumulated rows rendered BOTH "List may be incomplete" and the unsupported "No Trakt lists yet." claim. The zero-row empty state now branches on `incomplete` first. - F3 (ChannelBuilder.tsx useLibraryBrowse): changing the query/library while a "Load more" append was in flight stranded the button in its loading/disabled state forever (the stale append's own `finally` no longer matched the current request id, and the superseding fresh fetch never cleared `loadingMore` either). `finally` now clears `loadingMore` whenever the settling request is still the CURRENT one, regardless of whether that particular request was an append. - F4 (ChannelBuilder.tsx useLibraryBrowse): one rejected per-kind request in an append's `Promise.all` wiped every already-loaded row via `items: []` with no way back. Append failures now preserve state, surface the error inline next to a still-present "Load more" button, and roll the page cursor back so a retry re-requests the same page instead of skipping it. Both new UI fixes are pre-existing defects in the 'library' source that 9763fdca's loadCollections fix newly made reachable from 'collections' too. Verified all four fixes against negative controls: reverted each in turn and confirmed its dedicated test fails with the expected message, then restored. DO NOT FIX (filed as timothy/ersatztv#665 instead, bug+frontend+priority:low): - F1: loadCollections/loadLibraryItems sort each fetched page independently, so appended pages are only locally sorted, not globally sorted across the accumulated list. - F7: an overclaiming totalCount can leave "Load more" clickable after every kind is actually exhausted (no auto-loop; a user click is still required each time). Not touched (reviewer confirmed correct as-is): Trakt sequence/abort/unmount handling, the Class-A vs Class-B incomplete-copy distinction, and the `lists.length` footer count.Second cold cross-family (Codex, BLOCKED) re-review of b90f8a3b found the regex/bracket-tracking guard scanner still defeated in five ways, and two new High-severity races introduced by the F3/F4 fixes. Addressed as a further follow-up (b90f8a3b left untouched). GUARD REWRITE (per the review's explicit direction — stop patching the regex, use the compiler): - New `web/src/api/pageSizeScan.ts`: `scanPageSizeSites` parses each file with `ts.createSourceFile` and walks the real AST for `pageSize` PropertyAssignment/ShorthandPropertyAssignment nodes inside an ObjectLiteralExpression. This eliminates categorically (not case-by-case): - M-3: comments and string/template CONTENTS are never revisited as code, so a `'https://...'` string can't be misread as an unterminated string that swallows the rest of the file. - M-4: an object literal nested in a ternary, `??`, or JSX expression container is still found — the walk visits every descendant node regardless of the syntactic context above the ObjectLiteralExpression. - M-5: template-literal interpolations are real AST children, not opaque text. - L-7: a type literal (`type P = { pageSize: 100 }`), an interface PropertySignature, and a destructuring ObjectBindingPattern (parameter or nested) are structurally different node kinds from ObjectLiteralExpression — excluded by kind, not by a preceding-character heuristic a stray `{`/`(`/`,` could fool. `getLineAndCharacterOfPosition` gives exact line+column (fixes M-6 identity granularity) instead of the prior line-only identity. - `pageSizeCallSites.guard.test.ts` now imports the shared scanner; identity is `file:line:column:kind:value`, compared as a MULTISET (count, not membership) in both directions. - Both directions (unregistered / stale) are computed and folded into ONE thrown Error so a failure always shows the complete picture in one run, addressing the line-churn "second direction never renders" concern. - New `pageSizeScan.test.ts`: a FIXTURE test (inline source strings, no repo scan) pinning the exact discovered set for every case the review named — comment-in-string, string containing the literal text `pageSize: 100`, template interpolation, ternary, `??`, JSX container, same-line duplicates, parameter/nested destructuring, a type literal, an interface property, a forwarded call expression, a React dependency array. This is what actually protects the scanner going forward — the guard test alone only ever proved today's snapshot of real call sites, never the scanner's handling of input classes it hadn't happened to encounter yet. - Re-verified both original plants (a duplicate at-cap call in an already-registered file, and a new file with both a literal and a shorthand site) against the rewritten scanner; both still fail with the new combined-direction message. Also verified a run with BOTH directions simultaneously non-empty renders both in one report. HIGH-1 (ChannelBuilder.tsx useLibraryBrowse, now web/src/builder/libraryBrowse.ts): `reqId` identifies a query GENERATION, not an individual fetch — a page-0 refresh and a "Load more" append can be outstanding simultaneously under the same reqId (query changes while an append is in flight for the new generation). Whichever settled first used to clear `loadingMore`, letting a second click fire an out-of-order/duplicate page fetch. Fixed with a per-fetch `fetchId` plus a `loadingFetchIdRef`/`loadingFetchReqIdRef` pair: only the fetch that OWNS the currently-displayed spinner can clear it; a same-generation page-0 refresh leaves a same-generation append's spinner alone, while a page-0 refresh for a NEW generation still retires an abandoned OLDER-generation append's spinner (preserving the original #650 F3 fix). Reproduced the exact interleaving from the review in a new test (gate B's page-0 and page-1 fetches independently, click "Load more" while B's page-0 is still in flight) and confirmed it fails without the fix (button re-enables while the append is still pending). HIGH-2 (same file): the append-failure rollback mutated whatever `pageRef.current` currently held, rather than the specific page THIS fetch requested — under an overlapping-append race, a later page's success followed by an earlier page's failure could roll the cursor back past already-appended progress, corrupting a retry into refetching a duplicate. Fixed with a compare-and-set guard (`if (pageRef.current === pageNum)`) so the rollback only fires when nothing has advanced the cursor since. Since this overlap is UI-unreachable once HIGH-1's single-flight disabling is wired up (verified empirically: two synchronous fireEvent.click calls in RTL only produce one request, since act() flushes the disabling render between them), the regression test drives `useLibraryBrowse` directly via `renderHook` (now exported) to force the exact interleaving and confirms it fails without the fix (page 2 gets duplicated, page 3 never requested). Extracted `useLibraryBrowse` (plus `loadCollections`/`loadLibraryItems`/ `BrowseState`/the media-type const arrays) into a new non-JSX module `web/src/builder/libraryBrowse.ts` — exporting a hook from a .tsx file tripped `react-refresh/only-export-components`; this also makes the hook importable by `renderHook` without pulling in the whole screen component. No server-side/C# change. Full local gate: lint clean, tsc clean, full vitest run 110 files / 1051 tests passed (one LibrariesScreen.test.tsx flake reproduced under full-suite parallel load, confirmed pre-existing and unrelated — passes in isolation, never touched that file).Fourth cold cross-family review: no runtime correctness finding this round (single-flight held against synchronous throws, generation overlap, StrictMode, unmount, and separate hook instances). Two test-gap findings remained, both the same shape as prior rounds' review feedback: asserting on something ADJACENT to the mechanism rather than the mechanism itself. 1. ChannelBuilder.test.tsx's page-0 single-flight test asserted on the disabled BUTTON — during a page-0 refresh, `loadingMore` disables the native button, so `fireEvent.click` never reaches `loadMore` at all. Deleting `busyRef.current = true` at libraryBrowse.ts's generation-change effect would leave that test green while direct hook calls could still overlap page 0 and page 1. Added a hook-level test that calls `result.current.loadMore()` directly (via `renderHook`, bypassing the button/DOM layer) while a gated page-0 fetch is in flight, and asserts no page-1 request is ever issued. Verified: removing `busyRef.current = true` at that line makes the new test fail with `expected [ +0, 1 ] to deeply equal [ +0 ]` (a page-1 request was issued when the guard should have ignored the direct call); restored, green again. 2. pageSizeScan.test.ts's `.mts`/`.cts` tests exercised the SCANNER'S PARSING, not the guard's file-discovery glob — reverting the glob back to `.ts`/`.tsx` left both those tests green (the scanner still parses a `.mts`/`.cts` filename as plain TS regardless of extension) AND the real-source guard green (this repo has no committed `.mts`/`.cts` file for the reverted glob to miss). A prior verification planted a real `.mts` file and watched the guard notice it, which proved the behavior exists today but pinned nothing going forward. Extracted the inline glob condition into a standalone, exported `isScannableSourceFileName` predicate and added 14 parametrized cases asserting it BY FILENAME (no filesystem involved) — `.ts`/`.tsx`/`.mts`/`.cts` accepted, their `.test.*` and `.guard.test.ts` variants excluded, non-TS extensions rejected. Verified: reverting the predicate's regex back to `.ts`/`.tsx` only makes 2 of the 14 cases fail (`isScannableSourceFileName('*.mts')` and `'*.cts'` both wrongly return `false`); restored, green again. Nothing else changed — the reviewer confirmed no further findings (loader async/throw handling, generation sequencing, the unconditional rollback, StrictMode/unmount behavior, the disabled-during-refresh UX, type-only node exclusion, computed-name exactness, wrapped-forwarded-call rejection, and all existing test labelling were all confirmed correct as-is). Full local gate: lint clean, tsc clean, full vitest run 110 files / 1078 tests passed, re-run twice for stability (no flakes).npm run typecheck (tsc -b --pretty false, the real gate — npx tsc --noEmit was silently checking nothing meaningful due to the root tsconfig.json's empty `files: []`) flagged 3 errors: `let x: (() => void) | null = null` gate-release helpers, reassigned only inside a Promise executor, narrowed a later `x?.()` call to `never` under tsc -b's project-mode control-flow analysis. Not reproducible under a bare `tsc --noEmit` invocation. Fixed by adopting the no-op-initializer pattern already established elsewhere in this repo (api/libraries.test.ts's `releasePost`): declare as `let x = () => {};` instead of `(() => void) | null = null`, dropping the now-unnecessary optional chain at each call site. Same runtime behavior, no `as any`/`@ts-expect-error`, no change to what any test asserts. 3 call sites fixed: releaseStrandedAppend (F3 test), and releaseB0 in both the inverse-settlement-order test and the round-4 hook-level single-flight test. Verified: `npm run typecheck` no longer reports these 3; `npm run lint` clean; `npx vitest run` 110 files / 1078 tests still pass. 3 more typecheck errors remain (node:fs/node:path/node:url unresolvable in src/api/pageSizeCallSites.guard.test.ts, the only file under src that needs real Node fs access) — deliberately NOT fixed here. @types/node is already a devDependency and physically installed, but tsconfig.app.json (covering all of src) has no "node" in its `types` array, and a file-local `/// <reference types="node" />` was tried and reverted: under tsc -b's single-program compilation, it leaked Node's ambient `setTimeout` (returning NodeJS.Timeout) into the whole app project, breaking 3 unrelated window.setTimeout mocks expecting the DOM signature (confirmed by trying it — TS2345 in ChannelsScreen.test.tsx/LibrariesScreen.test.tsx/ PlayoutsScreen.test.tsx). The clean fix is a real project-config change (either widen tsconfig.app.json's types, accepting Node globals become type-visible in browser app code too, or move this one file into tsconfig.node.json's project alongside the e2e specs, which would also need a "references" wire-up for its cross-project import of pageSizeScan.ts) — left for the coordinator to choose per their explicit instruction not to make this call unilaterally.The gate is npm run typecheck (tsc -b --pretty false, project-mode) — the prior fix for the 3 node:*-unresolvable errors deferred a decision rather than resolving it, per instruction not to change tsconfig unilaterally. Evaluated and rejected: adding "node" to tsconfig.app.json's `types` (makes Node globals type-visible to production browser code, dissolving a deliberate separation the repo documents) and a file-local `/// <reference types="node" />` (empirically leaks Node's ambient `setTimeout` into the whole tsc -b program, breaking 3 unrelated window.setTimeout mocks — confirmed by trying it). Adopted instead: Vite's `import.meta.glob('/src/**/*.{ts,tsx,mts,cts}', { query: '?raw', import: 'default', eager: true })`, resolved natively by Vite/vitest at transform time — no node:fs, no node:path, no node:url, no tsconfig change, no cross-project references, and the guard test stays exactly where it is (`src/api/`). `vite/client` types (hence `import.meta.glob`) were already wired in via `src/vite-env.d.ts`. `isScannableSourceFileName` is unchanged and still the single place the glob's results are filtered through — the extension set moved into the glob literal, but discovery still runs every matched file through this same named, tested predicate rather than a second copy of the logic. Verified the discovery mechanism swap changes nothing observable: a side-by-side comparison test (old fs-based walk vs new glob-based walk, both run against the current repo, not committed — temporary) found byte-identical results — 19 call sites, 136 scannable files, zero set difference in either direction. Re-ran both required plants against the new mechanism: 1. Duplicate at-cap literal appended to builder/ChannelBuilder.tsx (an already-registered file): caught — `UNREGISTERED (1) ... + builder/ChannelBuilder.tsx:2058:54:literal:100`. 2. New file with a `https://` string (no false match), a shorthand `{ pageSize }`, and two literal pageSize values in one ternary's two branches on a single line: all 3 real sites caught at distinct columns — `+ screens/_fakeDefectPlant.ts:4:30:shorthand:pageSize`, `+ screens/_fakeDefectPlant.ts:8:22:literal:111`, `+ screens/_fakeDefectPlant.ts:8:48:literal:222`. Also re-verified the .mts/.cts discovery pin still fails when the extension set is narrowed back to `.ts`/`.tsx` only (2 of 14 predicate cases fail, as before). All three plants reverted after verification; registry/predicate otherwise untouched. Full local gate: `npm run lint` clean, `npm run typecheck` (tsc -b) clean — zero errors, including the 3 node:* ones this commit resolves — `npx vitest run` 110 files / 1078 tests passed (re-run three times; two runs hit transient vitest worker-pool timeouts from overlapping background processes on this machine, unrelated to the code — a clean sequential run passed in full both before and after).Review-verdict: MERGEABLE @
17c25e7Four cold cross-family review rounds (read-only sandbox); round 4 found no runtime defect, round 5 clean. Guard verified by orchestrator-planted defects the implementer never saw. Full gate green: lint, npm run typecheck (tsc -b), 110 files / 1078 tests.