timothy
373956fcee
BLOCKER. Attempt 4 bounded the Id KEYSPACE, and keyspace is not rows. Delete 20,000 historical rows, put one song at Id 20001, query artist?q=que: the walk burned all ten windows on empty ranges and returned [] for a table containing exactly one row. Capacity fell linearly with deletion ratio and no ratio was safe — one placed gap hides the next match. My record called that "heavily fragmented" and the endpoint description said loss happens "on a very large library"; the one-row example disproves both. Option A. The query now carries NO predicate at all — no LIKE, no LOWER, not even IS NOT NULL: SELECT Id, Artists AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch That is the whole fix, and it is the point. LIMIT truncates what survives a predicate, so with any predicate present it bounds the OUTPUT and says nothing about the WORK; the engine may evaluate and discard arbitrarily many rows first. Stripped to a bare primary-key range, LIMIT n reads exactly n index entries and n rows — independent of sparsity, deletion history or where the gaps fall. All selectivity moves into memory. A short page can now only mean exhaustion, which is precisely what it could not mean while a predicate was present. Four attempts, four wrong quantities: the result (a fixed budget the over-matching pre-filter starved), candidates returned (a no-match query must evaluate every eligible row before returning an empty page), keyspace width (above), and finally actual rows. The record carries the table; it is worth more than the code. Deleting the predicate deletes a whole bug family with it: the JSON-escape reasoning, the narrow-only-on-verbatim-ASCII rule, the exhaustive Unicode sweep that proved it sound, the ESCAPE '/' portability workaround, and the may-over-match-never-under-match invariant that turned out to be conditional on something untrue. SearchFieldValuesPrefilterSupersetTests is deleted entirely; the one assertion worth keeping — that the SQL has no predicate — moved to the query-shape suite, which pins the SQL string exactly so "just a cheap filter" fails a test instead of silently unbounding the walk. Measured cost of no server-side narrowing, on a seeded 20,000-song library (in-memory SQLite): worst case (no match, full walk) 20,000 rows / 10 round trips / 391.9 KiB / 119ms SQL, ~40ms warm end-to-end. Empty q, dense and non-ASCII prefixes all stop on page 1 at ~39 KiB and ~40ms. Judged acceptable for a debounced typeahead against a local file. If it ever is not, the answer is #669, not reintroducing selectivity — the record says so explicitly. Also fixed: - Round-trip count was advertised as 10; it is at most 10 for album_artist and 11 for artist, which also runs its EF query. The MAX(Id) probe is gone with the keyspace scheme, so there is no extra scalar call. - The duplicated-formula ceiling test is deleted rather than rewritten. It re-implemented the loop's arithmetic and would have passed through an off-by-one or a stall in the real loop; the dense integration tests carry that coverage. Its MaxVisited >= Window assertion was a style constraint in correctness clothing. - Stale text swept by grepping the mechanism nouns rather than re-reading: candidate/keyspace/ pre-filter/superset/row cap/LIKE/ESCAPE and the removed constant names, across handler, tests, record, api-conventions and the endpoint description. The two surviving "pre-filter" mentions are deliberate history. Test comments that rendered escaped non-ASCII as literal characters (which contradicted the raw-storage assertion in the same file) now show the escape text. New test List_Valued_Walk_Reads_Live_Rows_Regardless_Of_Id_Density reproduces the one-row killer and fails against attempt 4.
docs/ — task-signal map
Purpose: route a fresh contributor/agent to the minimal set of docs for the task at hand, instead of a mandatory front-to-back read. Update this doc in the same PR that adds, removes, or retitles a doc below, or that changes which sections a task signal points to.
Start here, always
CLAUDE.md(repo root) — project intro: architecture, layout, dev commands, conventions, Task Completion Protocol.docs/contributing.md— established code patterns (CQRS/MediatR, LanguageExt, the ChicoryTV SPA, EF Core dual-provider migrations, FFmpeg pipeline, analyzers, testing). Read before any non-trivial change.
Task signal → minimal sections
| Signal | Read |
|---|---|
| Session startup / "what's next" (no issue named) | docs/handoffs/chicorytv-issue-queue.md (standing kickoff — two concurrent tracks: orientation ‖ scripts/select-queue.sh 5) |
| Named-issue pickup | Skip queue selection; go straight to focused retrieval — see "Knowledge retrieval" below, then the issue body |
Adding/changing a /api/* endpoint |
docs/api-conventions.md checklist + docs/endpoint-index.md |
| Adding a ChicoryTV SPA screen | docs/spa-conventions.md |
| Scheduling / playout engine work | docs/domain-model.md + decisions catalog rows keyed sched.* (docs/decisions/README.md) |
| Concurrency / optimistic-locking work | docs/api-conventions.md §7a/b/c + docs/decisions/optimistic-concurrency.md |
| Auth / security-surface work | docs/decisions/api-auth-security.md |
| CI / release pipeline work | docs/ci-cd.md + docs/decisions/release-ci-governance.md |
| Live local run / Playwright-MCP verification | docs/e2e-local.md + scripts/e2e-local.sh |
| Adding/changing a UI-E2E browser flow | docs/e2e-local.md → "UI-E2E harness" + scripts/e2e-ui.sh |
| What does a test suite cover | docs/testing.md |
| Legacy Blazor route lookup | docs/blazor-route-parity.md (historical #91 phase (b) inventory) |
| "Why do we do X this way" / challenging a convention | Catalog-first: docs/decisions/README.md (active rows) → follow the row's link to docs/decisions/records/<area>/<topic>.md for full rationale. docs/decisions/archive/<area>/ only for "what did the rule used to be." |
Knowledge retrieval (MemPalace + catalog + Gitea)
These four rules are the seam agreed with server-management#642 (the Gitea→MemPalace exporter). They apply whether the question comes up via MemPalace, a grep, or a stale comment:
- Current conventions/decisions → catalog-first. Start at
docs/decisions/README.md; discover via theErsatzTV-Decisionswing (active) /ErsatzTV-Decisions-Archive(superseded/retired). Resolve by topic/key, never by chasing a file path. - Issue history → evidence, not authority. The
Gitea-ErsatzTVwing is historical narrative that may be stale; it never overrides current Markdown. - The breadcrumb rule (the crux behavior change). A file path named inside a historical issue
comment (e.g. "grep
docs/decisions.md2026-07-17", "see …") is a breadcrumb, not a live pointer. Find the current rule via the catalog / active wing by concept; do not treat the named path as current. (Why it's safe: still-current → in the active wing, breadcrumb resolves; superseded → the active wing returns the successor and a literal follow lands on a record that announces its ownstatus: superseded; retired → the active wing returns nothing, which is itself the signal. The validator-enforced move-to-archive/is what prevents the catastrophic "superseded rule read as current" case.) - Fallback when MemPalace is stale/down:
docs/decisions/README.mdcatalog, thenrg '^`key: <dotted.key>`' docs/decisions/. MemPalace is never authority nor sole fallback.
MemPalace is candidate discovery only — every passage is verified against its cited Markdown/Gitea
source before use. Never derive live queue state from MemPalace, #237, or historical comments; queue
state is live Gitea state, retrieved via scripts/select-queue.sh (see
docs/handoffs/chicorytv-issue-queue.md). Full retrieval contract (altitude/precedence, staleness
bounds, what's mined per issue): docs/handoffs/chicorytv-issue-queue.md → "Knowledge retrieval".
Also present in docs/
docs/domain-model.md— what the app IS: entity glossary, channel→playout→schedule/block concept map, where each concept is edited in the SPA.docs/api-conventions.md— checklist for adding/changing a/api/*endpoint (controllers, DTOs, error mapping, auth, OpenAPI regen, tests).docs/spa-conventions.md— playbook for adding a screen to the ChicoryTV React SPA.docs/e2e-local.md(+scripts/e2e-local.sh) — how to run a live local instance for manual or Playwright-MCP verification.docs/testing.md— testing map: what each*.Testsproject /websuite covers, golden-file nets, the timezone-independence rule, how to run subsets, the per-PR verification gate.docs/blazor-route-parity.md— historical record of the completed #91 phase (b) cutover: the Blazor Server UI is removed and every legacy route now 302-redirects to its SPA equivalent (or falls through to the catch-all →/app). Read it for the full legacy→SPA route inventory.docs/decisions/records/<area>/<topic>.md— one active decision record per file, YAML frontmatter (key/title/status/since/supersedes/superseded-by, plus optionalstale-after/sources— ersatztv#603), rationale prose in the body. The filename is the key, so one-active-record-per-key is a filesystem property (ersatztv#610).docs/decisions.mdand the topic files remain as the lifecycle-schema narrative plus a "Records formerly in this file" index, which is what keeps older date-based pointers resolvable. Generated active view:docs/decisions/README.md(catalog / task router) — start there. Superseded/retired records live indocs/decisions/archive/and are read only for history, never for "what is the current rule."docs/ci-cd.md— build/test/release pipeline, versioning, dependency management.docs/rest-api.md— REST API design doc for ersatztv#2 (goals, conventions, per-slice plan). Largely superseded day-to-day bydocs/api-conventions.md; read this for the original rationale.docs/mcp.md— theErsatzTV.Mcpstdio JSON-RPC MCP server (#58): how it wraps/api/v1as read + cautious-write tools, its config/env vars, auth, security posture, and the tool catalog.docs/channels.md— Channel entity field reference.docs/m3u-xmltv.md— M3U/XMLTV generation overview (ChannelPlaylist,GetChannelGuideHandler).docs/fork-strategy.md— divergence policy vs upstream ErsatzTV.docs/design-sync.md— Claude Design ↔ repo screen workflow (#92).docs/endpoint-index.md— generated REST endpoint index (method/path/operationId/summary per OpenAPI tag). Do not edit by hand; regenerated byscripts/generate-endpoint-index.py/scripts/update-openapi.sh.docs/handoffs/chicorytv-issue-queue.md— static session kickoff prompt + workflow lore. Queue state is live Gitea state, retrieved each session viascripts/select-queue.sh— see that file's standing kickoff for the two concurrent tracks (orientation ‖ selection). ersatztv#237 is a closed, archival historical tracker (superseded bystartup.parallel-orientationindocs/decisions.md) — not a live pointer.docs/tracker-retrofit-triage-237.md— audit trail for the #524 triage of ersatztv#237's 111 comments (method, per-comment classification, totals). Evidence for thedocs.tracker-comment-retrofitdecision; read it only when triaging another over-cap tracker.docs/handoffs/rest-api.md— original handoff prompt for kicking off the REST API work (#2).