docs(578): the retracted claim survived in 9 places, including the record title and rule
PR Gates / Script tests (pytest) (pull_request) Successful in 51s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m38s
Review verdict / Set review-verdict status (pull_request) Successful in 1m18s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 4m51s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m52s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m10s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 25m44s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ fc3ede0 (base: main)
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 7s
PR Gates / Docs update reminder (pull_request) Successful in 8s
PR Gates / decisions lifecycle (pull_request) Successful in 21s
PR Gates / Script tests (pytest) (pull_request) Successful in 51s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m38s
Review verdict / Set review-verdict status (pull_request) Successful in 1m18s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 4m51s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m52s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m10s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 25m44s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ fc3ede0 (base: main)
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 7s
PR Gates / Docs update reminder (pull_request) Successful in 8s
PR Gates / decisions lifecycle (pull_request) Successful in 21s
Comment- and docs-only; verified no non-comment line changed in any .cs.
I reported last round that I had "classified every surviving hit". That was false, and the false
confidence is the expensive part: a confidently-stated "I checked everything" stops anyone else
checking. The retracted wording survived in nine places, two of them the record's title and rule: —
and the catalog copies rule: verbatim, so the generated entry point and the record disagreed
semantically while docs/decisions.md said the correct thing.
Root cause of the miss, because it will recur otherwise: I built the sweep term list from the
DELETED MECHANISM's vocabulary (LIKE, superset, keyspace, anchor, over-match) and never added the
RETRACTED CLAIM's own words. "no predicate", "bound on work", "index entries", "no gap" and
"holds in memory" were never grepped. After a retraction the subject list has to include the words
of the thing being retracted, not just the thing already deleted.
Second, worse: my first attempt at this round's sweep printed nothing for every term and I nearly
read that as "all clear". zsh does not word-split an unquoted $FILES, so grep received one giant
non-existent path — and the `|| echo "(none)"` never fired because the pipeline's exit status was
sed's. Same failure shape as the bug arc itself: a check reporting success while examining nothing.
Re-run with a proper array plus a control term ("SongMetadata" -> 42 hits) so an empty result is
distinguishable from a broken grep.
Fixed all nine, replacing "no predicate" with the seekable-cursor-vs-residual distinction already
written correctly elsewhere:
- handler: the "real bound on work" claim, the short-page rationale
- SearchFieldValuesQueryShapeTests: "ANY predicate" + "reads exactly n index entries", and added what
the test can and cannot pin (a SQL string, not a plan / visibility work / payload I/O)
- GetSearchFieldValuesHandlerTests: "no gap between what the engine looks at and what it hands back",
and the current-behaviour comment
- record title, rule:, attempt-5 table row; api-conventions
- regenerated docs/decisions/README.md so catalog and record agree again
Tenth item, the same overclaim one level down and it survived the first retraction: the row bound was
said to cap what the process holds in memory. It does not — payload width is unrestricted and one
JSON array can contain arbitrarily many strings, each of which may enter the in-memory distinct set.
It caps logical rows returned/materialized and round-trip count, nothing about bytes. Added as a
third struck-through bullet next to the other two retractions.
This commit is contained in:
@@ -52,7 +52,9 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
|
||||
/// </item>
|
||||
/// </list>
|
||||
/// The logical-row bound is still worth having — it is what makes the walk terminate and what caps
|
||||
/// what this process holds in memory — but do not restate it as bounded I/O.
|
||||
/// the number of rows and round trips — but do not restate it as bounded I/O, and do not restate it
|
||||
/// as bounded MEMORY either: payload width is unrestricted and a single JSON array can hold
|
||||
/// arbitrarily many strings, every one of which may enter the in-memory set.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// The trade is real and deliberate: no server-side narrowing, so a query with few matches transfers
|
||||
@@ -192,9 +194,10 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// So the rows are walked in primary-key order, keyset-paged by row position, and split +
|
||||
/// exact-filtered in memory. All selectivity is in memory — the query carries no predicate at all,
|
||||
/// which is what makes its <c>LIMIT</c> a real bound on work. See <see cref="ListValuedBatchRows" />
|
||||
/// for the four revisions it took to get that right.
|
||||
/// exact-filtered in memory. All selectivity is in memory — the query's only condition is the
|
||||
/// cursor, a seek on the ordering key that never discards a row, so its <c>LIMIT</c> bounds the
|
||||
/// LOGICAL ROWS returned. See <see cref="ListValuedBatchRows" /> for the four revisions it took to
|
||||
/// get that right, and for what that bound does and does not cover.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private static async Task<List<string>> GetSongListValuedValues(
|
||||
@@ -241,10 +244,11 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
|
||||
|
||||
if (rows.Count < batch)
|
||||
{
|
||||
// With no predicate in the query, a short page can only mean the table is exhausted -- it can
|
||||
// never mean "this stretch happened to match nothing", which is precisely why the predicate had
|
||||
// to go. Advancing from the last returned Id is likewise safe: nothing was filtered out behind
|
||||
// it, so no row can be skipped.
|
||||
// With no RESIDUAL predicate -- only the cursor, which selects a range rather than discarding
|
||||
// rows from it -- a short page can only mean the table is exhausted. It can never mean "this
|
||||
// stretch happened to match nothing", which is precisely why the residual predicate had to go.
|
||||
// Advancing from the last returned Id is safe for the same reason: nothing was filtered out
|
||||
// behind it, so no row can be skipped.
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -331,8 +331,9 @@ public class GetSearchFieldValuesHandlerTests
|
||||
public async Task List_Valued_Walk_Reads_At_Most_20000_Rows()
|
||||
{
|
||||
// Pinned in both directions so the ceiling itself is nailed down: a match in row 20000 is read, the same
|
||||
// match in row 20001 is not. The query carries no predicate, so "rows read" is literally what LIMIT
|
||||
// returns -- there is no gap between what the engine looks at and what it hands back.
|
||||
// match in row 20001 is not. The query has no RESIDUAL predicate -- only the cursor -- so "rows read" is
|
||||
// what LIMIT returns. That bounds LOGICAL rows, not physical work: the engine may still traverse more
|
||||
// index records than it returns (MySQL purge lag), and row width is unbounded.
|
||||
const string needle = "\u00E9clair-the-needle";
|
||||
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
@@ -429,8 +430,8 @@ public class GetSearchFieldValuesHandlerTests
|
||||
// (\u00C9) and SQL LOWER() folds the escape TEXT, not the codepoint it denotes. THREE of these nine
|
||||
// cases fail against that revision (the ones where query and stored casing differ, so \u00e9 and
|
||||
// \u00C9 diverge); the other six pass it, because when the casings agree the escape texts line up.
|
||||
// The SQL predicate is gone entirely now -- matching happens in memory, where a string is just a
|
||||
// string -- so these cases pin current behaviour rather than guard that revision.
|
||||
// The SQL now has no residual predicate at all -- matching happens in memory, where a string is just
|
||||
// a string -- so these cases pin current behaviour rather than guard that revision.
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.AddRange(
|
||||
|
||||
@@ -82,10 +82,15 @@ public class SearchFieldValuesQueryShapeTests
|
||||
public void List_Valued_Page_Query_Has_No_Predicate_Beyond_The_Keyset_Cursor()
|
||||
{
|
||||
// This is the whole basis of the row bound, so it is asserted rather than assumed. LIMIT truncates what
|
||||
// survives a predicate — so with ANY predicate present it bounds the output, not the work, and the
|
||||
// engine may evaluate and discard arbitrarily many rows first. That is how four successive revisions
|
||||
// scanned past their own bound. Stripped to a bare primary-key range, LIMIT n reads exactly n index
|
||||
// entries and n rows, whatever the data looks like.
|
||||
// survives a RESIDUAL predicate — one that discards rows the engine already produced — so with such a
|
||||
// predicate present it bounds the output rather than the row count, and the engine may produce and
|
||||
// discard arbitrarily many rows first. That is how four successive revisions scanned past their own
|
||||
// bound. The cursor `Id > @AfterId` is NOT such a predicate: it is a seek on the ordering key, which
|
||||
// positions the scan without discarding anything, so LIMIT n yields n logical rows.
|
||||
//
|
||||
// What this test can and cannot do: it pins the SQL STRING. It cannot pin an execution plan, MVCC
|
||||
// visibility work or payload I/O -- physical work is NOT bounded (see the record: MySQL traverses
|
||||
// deleted-but-unpurged index records, and TEXT payloads spill to overflow pages).
|
||||
string sql = GetSearchFieldValuesHandler.ListValuedSql("Artists");
|
||||
|
||||
sql.ShouldBe(
|
||||
|
||||
@@ -456,8 +456,9 @@ No server-side caching. Powers the visual rule builder's value-input combobox fo
|
||||
or partly) by `SongMetadata.Artists`/`AlbumArtists`, which EF maps as **primitive collections** — one
|
||||
JSON array per row in a single column, with no server-side projection on either provider. `album_artist`
|
||||
therefore no longer 404s, and `artist` now also covers free-text music-video (`MusicVideoArtist`) and
|
||||
song credits, not only entity artists. Those rows are read by a keyset page that carries **no predicate
|
||||
at all** and filtered in memory, bounded at 20,000 rows per request — so on a larger library the
|
||||
song credits, not only entity artists. Those rows are read by a keyset page whose only condition is the
|
||||
**cursor** — no residual predicate that could discard a row — and filtered in memory, bounded at 20,000
|
||||
logical rows per request; so on a larger library the
|
||||
response may be a bounded subset of the matches — bounded in LOGICAL ROWS, which is not the same as
|
||||
bounded work or bytes. Say so in the `[EndpointDescription]` of any endpoint that adopts this shape.
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
| `api.schedule-item-flat-dto` | Schedule-item GET/POST/PUT use a flat, non-polymorphic `ScheduleItemResponseModel` (every subtype field promoted to a nullable top-level member) instead of the polymorphic Application VM hierarchy, with mutation field names matching `ScheduleItemRequest` 1:1 for a lossless round-trip. | 2026-07-10 | [link](records/api/schedule-item-flat-dto.md) |
|
||||
| `api.scheduling-hardening` | Create/Replace handlers guard against null/whitespace `name` (`IsNullOrWhiteSpace`, not just `Length`) to prevent NRE-500s, template-item overlap validation compares by index (not record value-equality) to catch exact-duplicate items, and unreachable 404 `ProducesResponseType` attributes on create-only actions are trimmed. | 2026-07-13 | [link](records/api/scheduling-hardening.md) |
|
||||
| `api.search-allitems-paging` | `GET /api/v1/search/all-items` is paginated (capped page size, `Totals` field) to bound DoS exposure; the SPA add-all flow pages to completeness instead of relying on an unbounded response. | 2026-07-18 | [link](records/api/search-allitems-paging.md) |
|
||||
| `api.search-field-values-sources` | `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite's `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404'd, as bounded best-effort, and its rows are read by a keyset page carrying **NO predicate whatsoever** — `SELECT Id, <col> AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL` — because `LIMIT` only truncates what survives a predicate, so any predicate at all makes it bound the OUTPUT rather than the WORK. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling. | 2026-07-26 | [link](records/api/search-field-values-sources.md) |
|
||||
| `api.search-field-values-sources` | `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite's `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404'd, as bounded best-effort, and its rows are read by a keyset page carrying **NO RESIDUAL predicate** — `SELECT Id, <col> AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL`. The cursor is itself a predicate, but a SEEKABLE one on the ordering key: it positions the scan and never discards a row. A RESIDUAL predicate discards rows the engine already produced, and `LIMIT` truncates only the survivors — so with one present it bounds the OUTPUT rather than the row count. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling. | 2026-07-26 | [link](records/api/search-field-values-sources.md) |
|
||||
| `api.search-paging-cap` | Search stays capped at 100 items per media kind; an overflowing kind's "See all" reuses library-browse paging instead of adding new API surface. | 2026-07-11 | [link](records/api/search-paging-cap.md) |
|
||||
| `api.versioning-v1` | The entire `/api` surface is versioned to `/api/v1` uniformly (no unversioned corner); legacy unversioned callers are rewritten in-pipeline (not redirected) with Deprecation/Link/Sunset headers, and post-freeze `/api/v1` is additive-only — a breaking change requires `/api/v2`. | 2026-07-13 | [link](records/api/versioning-v1.md) |
|
||||
| `blazor.rollback-tag` | The commit immediately preceding the Blazor-removal merge is tagged `blazor-final` (not a `v*` tag, so it doesn't trigger a prod release build) as the documented rollback/restore path. | 2026-07-11 | [link](records/blazor/rollback-tag.md) |
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
---
|
||||
key: api.search-field-values-sources
|
||||
title: '2026-07-26 — Facet-value typeahead, restated: every artist-bearing source is covered, and the JSON-column source is paged by ROW POSITION with no SQL predicate at all (#578)'
|
||||
title: '2026-07-26 — Facet-value typeahead, restated: every artist-bearing source is covered, and the JSON-column source is paged by ROW POSITION with no RESIDUAL SQL predicate (#578)'
|
||||
status: active
|
||||
since: '2026-07-26'
|
||||
supersedes: api.search-field-values@2026-07-23
|
||||
superseded-by: none
|
||||
rule: '`GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite''s `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404''d, as bounded best-effort, and its rows are read by a keyset page carrying **NO predicate whatsoever** — `SELECT Id, <col> AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL` — because `LIMIT` only truncates what survives a predicate, so any predicate at all makes it bound the OUTPUT rather than the WORK. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling.'
|
||||
signals: 'artist typeahead free-text credits, album_artist 404, artist suggestions missing music videos, SongMetadata.Artists, SongMetadata.AlbumArtists, MusicVideoArtist, EF primitive collection, PrimitiveCollection JSON column, SelectMany requires APPLY on SQLite, Pomelo primitive collections not enabled, LIMIT bounds output not work, seekable cursor vs residual predicate, MySQL purge lag traverses deleted index records, TEXT overflow pages, logical rows not physical work, keyspace is not rows, page by row position, density-independent paging, deleted rows leave Id gaps, ListValuedBatchRows, ListValuedMaxRowsRead, bounded best-effort facet values, OrdinalIgnoreCase vs InvariantCultureIgnoreCase folding, Accept-Language tr-TR dotless i, content_rating split, text field allow-list · paths: `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `ErsatzTV/Controllers/Api/SearchController.cs`, `ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs`, `ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs`, `web/src/api/search.ts` · issues: #578, #434, #176, #668, #669'
|
||||
rule: '`GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite''s `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404''d, as bounded best-effort, and its rows are read by a keyset page carrying **NO RESIDUAL predicate** — `SELECT Id, <col> AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL`. The cursor is itself a predicate, but a SEEKABLE one on the ordering key: it positions the scan and never discards a row. A RESIDUAL predicate discards rows the engine already produced, and `LIMIT` truncates only the survivors — so with one present it bounds the OUTPUT rather than the row count. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling.'
|
||||
signals: 'artist typeahead free-text credits, album_artist 404, artist suggestions missing music videos, SongMetadata.Artists, SongMetadata.AlbumArtists, MusicVideoArtist, EF primitive collection, PrimitiveCollection JSON column, SelectMany requires APPLY on SQLite, Pomelo primitive collections not enabled, LIMIT bounds output not work, seekable cursor vs residual predicate, cursor is a predicate too, MySQL purge lag traverses deleted index records, TEXT overflow pages, logical rows not physical work, keyspace is not rows, page by row position, density-independent paging, deleted rows leave Id gaps, ListValuedBatchRows, ListValuedMaxRowsRead, bounded best-effort facet values, OrdinalIgnoreCase vs InvariantCultureIgnoreCase folding, Accept-Language tr-TR dotless i, content_rating split, text field allow-list · paths: `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `ErsatzTV/Controllers/Api/SearchController.cs`, `ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs`, `ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs`, `web/src/api/search.ts` · issues: #578, #434, #176, #668, #669'
|
||||
mechanics: '`GetSearchFieldValuesHandler` (`GetSource`, `GetSongListValuedColumn`, `GetSongListValuedValues`, `ListValuedSql`, `ParseElements`, `FilterSortTake`); `SearchFieldValuesQueryShapeTests.List_Valued_Page_Query_Has_No_Predicate_Beyond_The_Keyset_Cursor`; api-conventions.md; spa-conventions.md §12'
|
||||
---
|
||||
|
||||
@@ -99,7 +99,7 @@ and each was caught only by someone constructing the adversarial case rather tha
|
||||
| 1–2 | the **result** — fixed `LIMIT 1000` on pre-filtered rows | the pre-filter was deliberately allowed to over-match, so a widened pattern (any non-ASCII or JSON-escaped prefix collapses it to `%"%`) filled the budget with rows that could not match. 1,000 `"zzz"` songs, `"éclair"` at row 1,001, `q=é` → `[]` |
|
||||
| 3 | **candidates returned** — keyset paging + `LIMIT` | a query matching nothing must evaluate every eligible row before it can return an empty page, so the first empty page ended the walk having counted **zero** against the ceiling. Rows returned bounded, rows inspected unbounded |
|
||||
| 4 | **keyspace width** — closed `Id` range per page | keyspace is not rows. Delete 20,000 historical rows, put one song at `Id` 20001, `q=que` → `[]`. **One row in the table, zero rows inspected.** Capacity fell linearly with deletion ratio and no ratio was safe: one placed gap hides the next match |
|
||||
| 5 | **actual rows** — keyset page by row position, **no predicate** | — |
|
||||
| 5 | **logical rows returned** — keyset page by row position, cursor only, **no residual predicate** | — (physical work still unbounded; see below) |
|
||||
|
||||
The through-line: **`LIMIT` truncates what survives a RESIDUAL predicate.** The distinction is not
|
||||
"predicate vs no predicate" — attempt 5's query still has `Id > @AfterId`. It is:
|
||||
@@ -133,8 +133,8 @@ more dangerous than the code ever was. What holds:
|
||||
- **at most `ListValuedMaxRowsRead` logical rows returned and materialized per request**, and
|
||||
- **at most 10 (or 11) round trips.**
|
||||
|
||||
That is the whole guarantee. It is what makes the walk terminate and what caps what this process
|
||||
holds in memory. **Explicitly retracted**, having been asserted here in an earlier revision:
|
||||
That is the whole guarantee. It is what makes the walk terminate and what caps the number of rows and
|
||||
round trips. **Explicitly retracted**, having been asserted here in earlier revisions:
|
||||
|
||||
- ~~"`LIMIT n` reads exactly `n` index entries"~~ — **false on MySQL.** Deleted clustered-index
|
||||
records survive until purge runs, and a range scan still traverses them. Hold an old InnoDB
|
||||
@@ -146,6 +146,10 @@ holds in memory. **Explicitly retracted**, having been asserted here in an earli
|
||||
- ~~bounded physical work / bounded I/O~~ — row width is unbounded. `Artists`/`AlbumArtists` are
|
||||
unrestricted `TEXT`/`longtext`, and both SQLite and InnoDB spill large payloads to overflow pages,
|
||||
so a row count implies neither a byte count nor a page-read count.
|
||||
- ~~"caps what this process holds in memory"~~ — the same overclaim one level down, and it survived
|
||||
the first retraction. A row count bounds neither bytes buffered nor set size: payload width is
|
||||
unrestricted, and one JSON array can contain arbitrarily many strings, every one of which may enter
|
||||
the in-memory distinct set.
|
||||
|
||||
Nor can the query-shape test carry more than it does: it pins the SQL **string**. It cannot pin an
|
||||
execution plan, MVCC visibility work, or payload I/O — and on MySQL, using the index to satisfy
|
||||
|
||||
Reference in New Issue
Block a user