fix(578): the LIKE prefilter under-matched every accented artist; make the superset provable

Review of 1b78dc9e found the pre-filter's correctness claim was false, and the claim was in the
decision record as well as the code.

F1 (high). The pattern JSON-encoded the whole query prefix on the reasoning that the stored text
escapes non-ASCII, so encoding the prefix the same way would line up. It does not: SQL LOWER()
lowercases the *escape text* (`É` -> `é`); it cannot case-fold the codepoint that escape
denotes. So `q=é` built `%"é%`, the stored `Édith Piaf` never matched, and the row was
discarded before the in-memory filter could accept it. Every accented artist — Beyoncé, Björk,
Sigur Rós, Édith Piaf — was silently unsuggestable, which in a music library is the common case.

The invariant that was missing, now stated in the code: the SQL pre-filter is an OPTIMIZATION. It
may over-match; it must never under-match. Correctness lives in the in-memory filter. So the pattern
now narrows only on the leading run of characters the JSON writer stores verbatim and stops at the
first character it cannot prove — `q=Beyoncé` still narrows on `beyonc`, `q=é` narrows on nothing
and leans on the row cap. Soundness rests on two facts now asserted by exhaustive computation rather
than argued: no non-ASCII codepoint in U+0080..U+10FFFF OrdinalIgnoreCase-equals a printable ASCII
character (false for InvariantCultureIgnoreCase, which folds ~190 — the choice of Ordinal is
load-bearing), and the exact set of ASCII the encoder escapes.

F1b. `UseRequestLocalization` honours Accept-Language, so the culture was caller-controlled and
`ToLower()` plus the default linguistic `StartsWith(string)` let a header change the answer.
Comparison is now OrdinalIgnoreCase and ordering StringComparer.Ordinal throughout — including the
shared FilterSortTake that state/video_dynamic_range/content_rating also use. Sets unchanged,
order now ordinal rather than culture-dependent.

F2. The merge comment asserted an exactness the code does not have: sources truncate by their own
ordering (DB collation / primary key), not the merge's, so a dropped value can outrank a survivor.
Comment and record now say best-effort, exact only below the truncation points.

F3/F4. The cap now rides `ORDER BY Id` rather than the JSON column: MySQL sorts TEXT by only
max_sort_length bytes, so the old ordering was not deterministic there, and sorting the whole
matching set was avoidable work. What the cap still does NOT bound is the scan — a leading-wildcard
LIKE cannot seek an index — so that cost is now documented as accepted, with a normalized
`SongArtist` table named as the follow-up candidate rather than left implicit.

Every clause above is covered by a test verified to FAIL when that clause is mutated (old pattern
builder: 5 red; culture chain: 3 red; cap=3 / cap=limit / ORDER BY json / no cap: red each).

F5. Converted to a proper supersession. The old record did not merely hold a stale fact — it
recorded song/music-video credits as an "intentionally-uncovered gap" and album_artist as
unsupported, and this reverses that call, which `docs.decision-lifecycle` says is never a
line-edit. `api.search-field-values` is archived with its original prose restored, and
`api.search-field-values-sources` replaces it carrying the whole endpoint contract.

docs/decisions/records/api/search-field-values-sources.md

@@ -0,0 +1,116 @@
+---
+key: api.search-field-values-sources
+title: '2026-07-26 — Facet-value typeahead, restated: every artist-bearing source is covered, list-valued columns via a superset LIKE pre-filter that may over-match but never under-match (#578)'
+status: active
+since: '2026-07-26'
+supersedes: api.search-field-values@2026-07-23
+superseded-by: none
+rule: '`GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). Prefix matching, dedup and ordering are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled. A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404''d, as bounded best-effort: SQL narrows on the raw JSON with a `LIKE` pattern built ONLY from the leading run of characters the JSON writer stores verbatim (printable ASCII minus `"&''+<>\`), stopping at the first character it cannot prove — this pre-filter is an optimization that MAY over-match and MUST NEVER under-match, with correctness living in the in-memory exact filter; the rows are capped at 1000 on `ORDER BY Id` (a unique integer key, not the JSON column) and then split, filtered, deduped, sorted and `limit`ed in memory. Ordering across merged sources is best-effort, exact only below the truncation points.'
+signals: 'artist typeahead free-text credits, album_artist 404, artist suggestions missing music videos, SongMetadata.Artists, SongMetadata.AlbumArtists, MusicVideoArtist, EF primitive collection, PrimitiveCollection JSON column, SelectMany requires APPLY on SQLite, Pomelo primitive collections not enabled, raw JSON LIKE pre-filter, superset never under-match, accented artist unsuggestable, LOWER cannot fold a \u00XX escape, OrdinalIgnoreCase vs InvariantCultureIgnoreCase folding, Accept-Language tr-TR dotless i, ListValuedRowCap, ORDER BY Id not TEXT, max_sort_length, content_rating split, text field allow-list · paths: `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `ErsatzTV/Controllers/Api/SearchController.cs`, `ErsatzTV.Tests/Application/Search/SearchFieldValuesPrefilterSupersetTests.cs`, `ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs`, `web/src/api/search.ts` · issues: #578, #434, #176'
+mechanics: '`GetSearchFieldValuesHandler` (`GetSource`, `GetSongListValuedColumn`, `GetSongListValuedValues`, `ListValuedSql`, `JsonElementPrefixPattern`, `FilterSortTake`); `SearchFieldValuesPrefilterSupersetTests` (the superset proof); api-conventions.md; spa-conventions.md §12'
+---
+
+Supersedes `api.search-field-values` (#434). That record did not merely carry a stale implementation
+detail — it recorded a **call**: free-text music-video/song artist credits were "a known,
+intentionally-uncovered gap" and `album_artist` was unsupported. #578 reverses that call, so this is
+a supersession, not a line-edit. Everything #434 settled that still holds is restated here rather
+than left in the archive: enum fields ship their values inline on `GET /api/v1/search/fields` and
+need no lookup; text fields need a live one; the source is the database and never the search index;
+`content_rating` splits its compound `"PG-13/TV-14"` strings in memory; there is no result cache.
+
+## The three `artist` sources are three different problems
+
+`LuceneSearchIndex` writes the `artist` field from three places, and only two are ordinary columns:
+
+- `ArtistMetadata.Title` — a plain column. Already worked.
+- `MusicVideoArtist.Name` — also a real entity table (`MusicVideoMetadata.HasMany(m => m.Artists)`),
+  so the free-text music-video credits are directly `SELECT DISTINCT`-able. It just joins the
+  existing server-side pipeline as a `Concat`, emitted as one bounded `UNION ALL` +
+  `LOWER(...) LIKE ... LIMIT` on both providers.
+- `SongMetadata.Artists` (and, for `album_artist`, `AlbumArtists`) — an `IList<string>` EF 9 maps as
+  a **primitive collection**: no `HasConversion` anywhere, one JSON array per row in a single
+  `TEXT`/`longtext` column, with no server-side projection at all. Verified against both providers:
+  SQLite reports *"Translating this query requires the SQL APPLY operation, which is not supported on
+  SQLite"*, Pomelo MySQL 9.0.0 reports *"Primitive collections support has not been enabled"*. Both
+  failures are pinned by a test, so a provider upgrade that fixes them surfaces as a red rather than
+  leaving a workaround in place forever.
+
+## The pre-filter is an optimization; it may over-match, it must never under-match
+
+This is the load-bearing invariant, and the first implementation got it wrong in a way that no test
+of the returned values could see. It JSON-encoded the whole query prefix, reasoning that since the
+stored text escapes non-ASCII (`Édith` is on disk as `Édith`), encoding the prefix the same way
+would line up. **It does not.** SQL `LOWER()` lowercases the *escape text* — `É` — it cannot
+case-fold the codepoint that escape denotes. So `q=é` produced a pattern of `é` that never
+matched, the row was discarded before the in-memory filter ever saw it, and **every accented artist
+was silently unsuggestable** — in a music library, the common case, not an edge case.
+
+The fix is to narrow only on ground that can be proved: the leading run of characters the JSON
+writer stores **verbatim**, stopping at the first non-ASCII or JSON-escaped character. A query
+starting with such a character narrows to the bare element-opening anchor `%"%` and leans on the row
+cap and the in-memory filter. `q=Beyoncé` still narrows on `beyonc`; `q=é` narrows on nothing.
+
+Soundness rests on two facts, both **asserted by exhaustive computation** in
+`SearchFieldValuesPrefilterSupersetTests` rather than argued:
+
+1. **No non-ASCII codepoint in all of Unicode `OrdinalIgnoreCase`-equals a printable ASCII
+   character** (swept over `U+0080`–`U+10FFFF`). So for a run character `c`, every element character
+   that can match `c` is itself ASCII, hence stored verbatim, hence foldable by both providers'
+   `LOWER()`. This is *false* for `InvariantCultureIgnoreCase`, which folds ~190 codepoints
+   (`U+00AA`→`a`, `U+017F`→`s`, the modifier letters) onto ASCII letters — **the choice of Ordinal is
+   load-bearing, not stylistic**.
+2. **The exact set of printable-ASCII characters the JSON writer escapes** is `"`, `&`, `'`, `+`,
+   `<`, `>`, `\`, `` ` ``, pinned against the encoder itself so an encoder change fails the build
+   instead of quietly shrinking the superset.
+
+## Ordinal everywhere, because the culture is caller-controlled
+
+`UseRequestLocalization` honours `Accept-Language`, so a caller can select `tr-TR` and turn `q=I`
+into `ı`. The old chain used `ToLower()` plus the default *linguistic* `StartsWith(string)`, making
+the same library answer differently per caller. Comparison is now `OrdinalIgnoreCase` and ordering
+`StringComparer.Ordinal` throughout, including the shared `FilterSortTake` that `state`,
+`video_dynamic_range` and `content_rating` also use. That is a deliberate change to shared behaviour:
+response *sets* are unchanged, response *order* is now ordinal rather than culture-dependent.
+
+## Ordering is best-effort, and the code says so
+
+Merging sources does **not** yield the exact first `limit` of the union. Each source truncates using
+its own ordering — the EF source by the database collation (SQLite's is ASCII-only), the list source
+by primary key — and neither is the ordinal ordering the merge applies. With `"Zulu"` and `"Éclair"`
+and `limit=1`, the EF source keeps `"Zulu"` and the merge never sees `"Éclair"`, which it would have
+ranked first. Below the truncation points — the normal typeahead case — the result is exact. An
+earlier comment claimed exactness the code does not have; do not restore it.
+
+## What the row cap does and does not bound
+
+`ListValuedRowCap = 1000` bounds **rows fetched**, not values returned, because the pre-filter
+matches rows whole. The cap rides `ORDER BY Id` — a unique integer primary key both providers can
+walk — rather than the JSON column: ordering on `TEXT` would not be deterministic on MySQL, which
+sorts using only the first `max_sort_length` (default 1024) bytes, so long rows sharing a prefix
+would tie arbitrarily; and it avoids sorting the whole matching set.
+
+**It does not bound database work, and that is a known cost.** `LOWER(col) LIKE '%…'` has a leading
+wildcard, so no index can be sought and every `SongMetadata` row is scanned before the cap applies.
+An empty `q` matches nearly every row. Every `artist` request now scans the song table's JSON column
+in addition to its previous work, once per debounced keystroke; the endpoint is authenticated by
+default but `Api:RequireKeyForReads=false` makes it anonymous. Accepted for now because the scan is
+bounded in *memory* and the affected table is one of the smaller ones, and because the alternative is
+the migration below. **Follow-up candidate: a normalized `SongArtist` join table** (the shape
+`MusicVideoArtist` already has) would make this exact, indexable and seekable — at the cost of a
+dual-provider schema migration plus data backfill, changes to every scanner write path populating
+`SongMetadata.Artists`, changes to the Lucene indexer, and two representations of the same fact free
+to drift. Rejected for #578 on blast radius, not on merit.
+
+## `album_artist` 404 → 200 is additive
+
+Nothing consumes the 404 as a signal: the SPA's `getSearchFieldValues` (`web/src/api/search.ts`)
+treats any non-200 as "no suggestions, fall back to a free-text input", which it will now do less
+often. Per `api.versioning-v1`, widening which fields return values adds capability without removing
+any, so no `/api/v2`.
+
+## Known limitation inherited, not introduced
+
+The **EF-sourced** fields (`genre`, `studio`, `artist`'s entity half, …) still prefix-match through
+SQL `LOWER()`, which on SQLite is ASCII-only — so `q=é` does not match a stored `Édith` for those
+fields either. That predates #578 and is unchanged by it; fixing it would mean filtering those tables
+client-side. Noted here so the next reader does not mistake the list-valued fix for a global one.