From fc3ede09bcf1dc582a6efc729423bf2ec96ed5a5 Mon Sep 17 00:00:00 2001 From: Timothy Date: Mon, 27 Jul 2026 03:09:32 +0200 Subject: [PATCH] docs(578): the retracted claim survived in 9 places, including the record title and rule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Comment- and docs-only; verified no non-comment line changed in any .cs. I reported last round that I had "classified every surviving hit". That was false, and the false confidence is the expensive part: a confidently-stated "I checked everything" stops anyone else checking. The retracted wording survived in nine places, two of them the record's title and rule: — and the catalog copies rule: verbatim, so the generated entry point and the record disagreed semantically while docs/decisions.md said the correct thing. Root cause of the miss, because it will recur otherwise: I built the sweep term list from the DELETED MECHANISM's vocabulary (LIKE, superset, keyspace, anchor, over-match) and never added the RETRACTED CLAIM's own words. "no predicate", "bound on work", "index entries", "no gap" and "holds in memory" were never grepped. After a retraction the subject list has to include the words of the thing being retracted, not just the thing already deleted. Second, worse: my first attempt at this round's sweep printed nothing for every term and I nearly read that as "all clear". zsh does not word-split an unquoted $FILES, so grep received one giant non-existent path — and the `|| echo "(none)"` never fired because the pipeline's exit status was sed's. Same failure shape as the bug arc itself: a check reporting success while examining nothing. Re-run with a proper array plus a control term ("SongMetadata" -> 42 hits) so an empty result is distinguishable from a broken grep. Fixed all nine, replacing "no predicate" with the seekable-cursor-vs-residual distinction already written correctly elsewhere: - handler: the "real bound on work" claim, the short-page rationale - SearchFieldValuesQueryShapeTests: "ANY predicate" + "reads exactly n index entries", and added what the test can and cannot pin (a SQL string, not a plan / visibility work / payload I/O) - GetSearchFieldValuesHandlerTests: "no gap between what the engine looks at and what it hands back", and the current-behaviour comment - record title, rule:, attempt-5 table row; api-conventions - regenerated docs/decisions/README.md so catalog and record agree again Tenth item, the same overclaim one level down and it survived the first retraction: the row bound was said to cap what the process holds in memory. It does not — payload width is unrestricted and one JSON array can contain arbitrarily many strings, each of which may enter the in-memory distinct set. It caps logical rows returned/materialized and round-trip count, nothing about bytes. Added as a third struck-through bullet next to the other two retractions. --- .../Queries/GetSearchFieldValuesHandler.cs | 20 +++++++++++-------- .../GetSearchFieldValuesHandlerTests.cs | 9 +++++---- .../SearchFieldValuesQueryShapeTests.cs | 13 ++++++++---- docs/api-conventions.md | 5 +++-- docs/decisions/README.md | 2 +- .../api/search-field-values-sources.md | 16 +++++++++------ 6 files changed, 40 insertions(+), 25 deletions(-) diff --git a/ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs b/ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs index 0dc0b188d..e37d7ed00 100644 --- a/ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs +++ b/ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs @@ -52,7 +52,9 @@ public class GetSearchFieldValuesHandler(IDbContextFactory dbContextF /// /// /// The logical-row bound is still worth having — it is what makes the walk terminate and what caps - /// what this process holds in memory — but do not restate it as bounded I/O. + /// the number of rows and round trips — but do not restate it as bounded I/O, and do not restate it + /// as bounded MEMORY either: payload width is unrestricted and a single JSON array can hold + /// arbitrarily many strings, every one of which may enter the in-memory set. /// /// /// The trade is real and deliberate: no server-side narrowing, so a query with few matches transfers @@ -192,9 +194,10 @@ public class GetSearchFieldValuesHandler(IDbContextFactory dbContextF /// /// /// So the rows are walked in primary-key order, keyset-paged by row position, and split + - /// exact-filtered in memory. All selectivity is in memory — the query carries no predicate at all, - /// which is what makes its LIMIT a real bound on work. See - /// for the four revisions it took to get that right. + /// exact-filtered in memory. All selectivity is in memory — the query's only condition is the + /// cursor, a seek on the ordering key that never discards a row, so its LIMIT bounds the + /// LOGICAL ROWS returned. See for the four revisions it took to + /// get that right, and for what that bound does and does not cover. /// /// private static async Task> GetSongListValuedValues( @@ -241,10 +244,11 @@ public class GetSearchFieldValuesHandler(IDbContextFactory dbContextF if (rows.Count < batch) { - // With no predicate in the query, a short page can only mean the table is exhausted -- it can - // never mean "this stretch happened to match nothing", which is precisely why the predicate had - // to go. Advancing from the last returned Id is likewise safe: nothing was filtered out behind - // it, so no row can be skipped. + // With no RESIDUAL predicate -- only the cursor, which selects a range rather than discarding + // rows from it -- a short page can only mean the table is exhausted. It can never mean "this + // stretch happened to match nothing", which is precisely why the residual predicate had to go. + // Advancing from the last returned Id is safe for the same reason: nothing was filtered out + // behind it, so no row can be skipped. break; } } diff --git a/ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs b/ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs index ad9415ea9..4a8ae8677 100644 --- a/ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs +++ b/ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs @@ -331,8 +331,9 @@ public class GetSearchFieldValuesHandlerTests public async Task List_Valued_Walk_Reads_At_Most_20000_Rows() { // Pinned in both directions so the ceiling itself is nailed down: a match in row 20000 is read, the same - // match in row 20001 is not. The query carries no predicate, so "rows read" is literally what LIMIT - // returns -- there is no gap between what the engine looks at and what it hands back. + // match in row 20001 is not. The query has no RESIDUAL predicate -- only the cursor -- so "rows read" is + // what LIMIT returns. That bounds LOGICAL rows, not physical work: the engine may still traverse more + // index records than it returns (MySQL purge lag), and row width is unbounded. const string needle = "\u00E9clair-the-needle"; await using (TvContext context = _db.CreateContext()) @@ -429,8 +430,8 @@ public class GetSearchFieldValuesHandlerTests // (\u00C9) and SQL LOWER() folds the escape TEXT, not the codepoint it denotes. THREE of these nine // cases fail against that revision (the ones where query and stored casing differ, so \u00e9 and // \u00C9 diverge); the other six pass it, because when the casings agree the escape texts line up. - // The SQL predicate is gone entirely now -- matching happens in memory, where a string is just a - // string -- so these cases pin current behaviour rather than guard that revision. + // The SQL now has no residual predicate at all -- matching happens in memory, where a string is just + // a string -- so these cases pin current behaviour rather than guard that revision. await using (TvContext context = _db.CreateContext()) { context.SongMetadata.AddRange( diff --git a/ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs b/ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs index 4417d3f1b..601e3b6d6 100644 --- a/ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs +++ b/ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs @@ -82,10 +82,15 @@ public class SearchFieldValuesQueryShapeTests public void List_Valued_Page_Query_Has_No_Predicate_Beyond_The_Keyset_Cursor() { // This is the whole basis of the row bound, so it is asserted rather than assumed. LIMIT truncates what - // survives a predicate — so with ANY predicate present it bounds the output, not the work, and the - // engine may evaluate and discard arbitrarily many rows first. That is how four successive revisions - // scanned past their own bound. Stripped to a bare primary-key range, LIMIT n reads exactly n index - // entries and n rows, whatever the data looks like. + // survives a RESIDUAL predicate — one that discards rows the engine already produced — so with such a + // predicate present it bounds the output rather than the row count, and the engine may produce and + // discard arbitrarily many rows first. That is how four successive revisions scanned past their own + // bound. The cursor `Id > @AfterId` is NOT such a predicate: it is a seek on the ordering key, which + // positions the scan without discarding anything, so LIMIT n yields n logical rows. + // + // What this test can and cannot do: it pins the SQL STRING. It cannot pin an execution plan, MVCC + // visibility work or payload I/O -- physical work is NOT bounded (see the record: MySQL traverses + // deleted-but-unpurged index records, and TEXT payloads spill to overflow pages). string sql = GetSearchFieldValuesHandler.ListValuedSql("Artists"); sql.ShouldBe( diff --git a/docs/api-conventions.md b/docs/api-conventions.md index 9580f5591..f60514eb0 100644 --- a/docs/api-conventions.md +++ b/docs/api-conventions.md @@ -456,8 +456,9 @@ No server-side caching. Powers the visual rule builder's value-input combobox fo or partly) by `SongMetadata.Artists`/`AlbumArtists`, which EF maps as **primitive collections** — one JSON array per row in a single column, with no server-side projection on either provider. `album_artist` therefore no longer 404s, and `artist` now also covers free-text music-video (`MusicVideoArtist`) and -song credits, not only entity artists. Those rows are read by a keyset page that carries **no predicate -at all** and filtered in memory, bounded at 20,000 rows per request — so on a larger library the +song credits, not only entity artists. Those rows are read by a keyset page whose only condition is the +**cursor** — no residual predicate that could discard a row — and filtered in memory, bounded at 20,000 +logical rows per request; so on a larger library the response may be a bounded subset of the matches — bounded in LOGICAL ROWS, which is not the same as bounded work or bytes. Say so in the `[EndpointDescription]` of any endpoint that adopts this shape. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 61c366005..9f498ca36 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -28,7 +28,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera | `api.schedule-item-flat-dto` | Schedule-item GET/POST/PUT use a flat, non-polymorphic `ScheduleItemResponseModel` (every subtype field promoted to a nullable top-level member) instead of the polymorphic Application VM hierarchy, with mutation field names matching `ScheduleItemRequest` 1:1 for a lossless round-trip. | 2026-07-10 | [link](records/api/schedule-item-flat-dto.md) | | `api.scheduling-hardening` | Create/Replace handlers guard against null/whitespace `name` (`IsNullOrWhiteSpace`, not just `Length`) to prevent NRE-500s, template-item overlap validation compares by index (not record value-equality) to catch exact-duplicate items, and unreachable 404 `ProducesResponseType` attributes on create-only actions are trimmed. | 2026-07-13 | [link](records/api/scheduling-hardening.md) | | `api.search-allitems-paging` | `GET /api/v1/search/all-items` is paginated (capped page size, `Totals` field) to bound DoS exposure; the SPA add-all flow pages to completeness instead of relying on an unbounded response. | 2026-07-18 | [link](records/api/search-allitems-paging.md) | -| `api.search-field-values-sources` | `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite's `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404'd, as bounded best-effort, and its rows are read by a keyset page carrying **NO predicate whatsoever** — `SELECT Id, AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL` — because `LIMIT` only truncates what survives a predicate, so any predicate at all makes it bound the OUTPUT rather than the WORK. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling. | 2026-07-26 | [link](records/api/search-field-values-sources.md) | +| `api.search-field-values-sources` | `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite's `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404'd, as bounded best-effort, and its rows are read by a keyset page carrying **NO RESIDUAL predicate** — `SELECT Id, AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL`. The cursor is itself a predicate, but a SEEKABLE one on the ordering key: it positions the scan and never discards a row. A RESIDUAL predicate discards rows the engine already produced, and `LIMIT` truncates only the survivors — so with one present it bounds the OUTPUT rather than the row count. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling. | 2026-07-26 | [link](records/api/search-field-values-sources.md) | | `api.search-paging-cap` | Search stays capped at 100 items per media kind; an overflowing kind's "See all" reuses library-browse paging instead of adding new API surface. | 2026-07-11 | [link](records/api/search-paging-cap.md) | | `api.versioning-v1` | The entire `/api` surface is versioned to `/api/v1` uniformly (no unversioned corner); legacy unversioned callers are rewritten in-pipeline (not redirected) with Deprecation/Link/Sunset headers, and post-freeze `/api/v1` is additive-only — a breaking change requires `/api/v2`. | 2026-07-13 | [link](records/api/versioning-v1.md) | | `blazor.rollback-tag` | The commit immediately preceding the Blazor-removal merge is tagged `blazor-final` (not a `v*` tag, so it doesn't trigger a prod release build) as the documented rollback/restore path. | 2026-07-11 | [link](records/blazor/rollback-tag.md) | diff --git a/docs/decisions/records/api/search-field-values-sources.md b/docs/decisions/records/api/search-field-values-sources.md index ca0806f3a..a19693afd 100644 --- a/docs/decisions/records/api/search-field-values-sources.md +++ b/docs/decisions/records/api/search-field-values-sources.md @@ -1,12 +1,12 @@ --- key: api.search-field-values-sources -title: '2026-07-26 — Facet-value typeahead, restated: every artist-bearing source is covered, and the JSON-column source is paged by ROW POSITION with no SQL predicate at all (#578)' +title: '2026-07-26 — Facet-value typeahead, restated: every artist-bearing source is covered, and the JSON-column source is paged by ROW POSITION with no RESIDUAL SQL predicate (#578)' status: active since: '2026-07-26' supersedes: api.search-field-values@2026-07-23 superseded-by: none -rule: '`GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite''s `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404''d, as bounded best-effort, and its rows are read by a keyset page carrying **NO predicate whatsoever** — `SELECT Id, AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL` — because `LIMIT` only truncates what survives a predicate, so any predicate at all makes it bound the OUTPUT rather than the WORK. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling.' -signals: 'artist typeahead free-text credits, album_artist 404, artist suggestions missing music videos, SongMetadata.Artists, SongMetadata.AlbumArtists, MusicVideoArtist, EF primitive collection, PrimitiveCollection JSON column, SelectMany requires APPLY on SQLite, Pomelo primitive collections not enabled, LIMIT bounds output not work, seekable cursor vs residual predicate, MySQL purge lag traverses deleted index records, TEXT overflow pages, logical rows not physical work, keyspace is not rows, page by row position, density-independent paging, deleted rows leave Id gaps, ListValuedBatchRows, ListValuedMaxRowsRead, bounded best-effort facet values, OrdinalIgnoreCase vs InvariantCultureIgnoreCase folding, Accept-Language tr-TR dotless i, content_rating split, text field allow-list · paths: `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `ErsatzTV/Controllers/Api/SearchController.cs`, `ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs`, `ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs`, `web/src/api/search.ts` · issues: #578, #434, #176, #668, #669' +rule: '`GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite''s `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404''d, as bounded best-effort, and its rows are read by a keyset page carrying **NO RESIDUAL predicate** — `SELECT Id, AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL`. The cursor is itself a predicate, but a SEEKABLE one on the ordering key: it positions the scan and never discards a row. A RESIDUAL predicate discards rows the engine already produced, and `LIMIT` truncates only the survivors — so with one present it bounds the OUTPUT rather than the row count. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling.' +signals: 'artist typeahead free-text credits, album_artist 404, artist suggestions missing music videos, SongMetadata.Artists, SongMetadata.AlbumArtists, MusicVideoArtist, EF primitive collection, PrimitiveCollection JSON column, SelectMany requires APPLY on SQLite, Pomelo primitive collections not enabled, LIMIT bounds output not work, seekable cursor vs residual predicate, cursor is a predicate too, MySQL purge lag traverses deleted index records, TEXT overflow pages, logical rows not physical work, keyspace is not rows, page by row position, density-independent paging, deleted rows leave Id gaps, ListValuedBatchRows, ListValuedMaxRowsRead, bounded best-effort facet values, OrdinalIgnoreCase vs InvariantCultureIgnoreCase folding, Accept-Language tr-TR dotless i, content_rating split, text field allow-list · paths: `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `ErsatzTV/Controllers/Api/SearchController.cs`, `ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs`, `ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs`, `web/src/api/search.ts` · issues: #578, #434, #176, #668, #669' mechanics: '`GetSearchFieldValuesHandler` (`GetSource`, `GetSongListValuedColumn`, `GetSongListValuedValues`, `ListValuedSql`, `ParseElements`, `FilterSortTake`); `SearchFieldValuesQueryShapeTests.List_Valued_Page_Query_Has_No_Predicate_Beyond_The_Keyset_Cursor`; api-conventions.md; spa-conventions.md §12' --- @@ -99,7 +99,7 @@ and each was caught only by someone constructing the adversarial case rather tha | 1–2 | the **result** — fixed `LIMIT 1000` on pre-filtered rows | the pre-filter was deliberately allowed to over-match, so a widened pattern (any non-ASCII or JSON-escaped prefix collapses it to `%"%`) filled the budget with rows that could not match. 1,000 `"zzz"` songs, `"éclair"` at row 1,001, `q=é` → `[]` | | 3 | **candidates returned** — keyset paging + `LIMIT` | a query matching nothing must evaluate every eligible row before it can return an empty page, so the first empty page ended the walk having counted **zero** against the ceiling. Rows returned bounded, rows inspected unbounded | | 4 | **keyspace width** — closed `Id` range per page | keyspace is not rows. Delete 20,000 historical rows, put one song at `Id` 20001, `q=que` → `[]`. **One row in the table, zero rows inspected.** Capacity fell linearly with deletion ratio and no ratio was safe: one placed gap hides the next match | -| 5 | **actual rows** — keyset page by row position, **no predicate** | — | +| 5 | **logical rows returned** — keyset page by row position, cursor only, **no residual predicate** | — (physical work still unbounded; see below) | The through-line: **`LIMIT` truncates what survives a RESIDUAL predicate.** The distinction is not "predicate vs no predicate" — attempt 5's query still has `Id > @AfterId`. It is: @@ -133,8 +133,8 @@ more dangerous than the code ever was. What holds: - **at most `ListValuedMaxRowsRead` logical rows returned and materialized per request**, and - **at most 10 (or 11) round trips.** -That is the whole guarantee. It is what makes the walk terminate and what caps what this process -holds in memory. **Explicitly retracted**, having been asserted here in an earlier revision: +That is the whole guarantee. It is what makes the walk terminate and what caps the number of rows and +round trips. **Explicitly retracted**, having been asserted here in earlier revisions: - ~~"`LIMIT n` reads exactly `n` index entries"~~ — **false on MySQL.** Deleted clustered-index records survive until purge runs, and a range scan still traverses them. Hold an old InnoDB @@ -146,6 +146,10 @@ holds in memory. **Explicitly retracted**, having been asserted here in an earli - ~~bounded physical work / bounded I/O~~ — row width is unbounded. `Artists`/`AlbumArtists` are unrestricted `TEXT`/`longtext`, and both SQLite and InnoDB spill large payloads to overflow pages, so a row count implies neither a byte count nor a page-read count. +- ~~"caps what this process holds in memory"~~ — the same overclaim one level down, and it survived + the first retraction. A row count bounds neither bytes buffered nor set size: payload width is + unrestricted, and one JSON array can contain arbitrarily many strings, every one of which may enter + the in-memory distinct set. Nor can the query-shape test carry more than it does: it pins the SQL **string**. It cannot pin an execution plan, MVCC visibility work, or payload I/O — and on MySQL, using the index to satisfy