BLOCKER 1. The invariant a883e5f0 established — "the pre-filter may over-match, it must never
under-match" — is only sound while the candidate set is not truncated. It was truncated. A non-ASCII
or JSON-escaped prefix collapses the pattern to the bare `%"%` anchor, so every row becomes a
candidate, and `ORDER BY Id LIMIT 1000` then spent the whole budget on rows that could not match.
Seed 1000 songs by "zzz", put the only "éclair" in row 1001, ask for album_artist?q=é: a883e5f0
returns [], while 1b78dc9e returned "éclair" because its (separately broken) tighter pattern kept
the candidate set small. Neither revision was correct — the old one under-matched at the pattern,
the new one under-matched at the cap. Widening a predicate under a fixed budget starves it.
So the budget is gone. Candidate rows are now walked keyset-paged on Id (`Id > @AfterId … ORDER BY
Id LIMIT @Batch`), continuing past non-matching candidates and stopping on the first of: enough
distinct exact matches for `limit`, a short page (source exhausted), or a 20,000-candidate-row
ceiling in 2,000-row batches. The bound is on effort; it no longer silently decides the result, and
the lossy case needs 20,000 rows that already passed the pre-filter before it bites.
BLOCKER 2. The endpoint description and the record's rule claimed ordinal matching/dedup/ordering
endpoint-wide. False for EF-backed fields: the database runs LOWER/DISTINCT/ORDER BY/LIMIT before
any ordinal code, so `genre?q=é` still misses a stored "Éclair" on SQLite. Both are now scoped to
the final in-memory stages, and the underlying gap is referenced as #668 rather than described as
fixed. #669 (normalized SongArtist table) is referenced as the follow-up for the scan cost.
Accuracy corrections to my own claims, all verified by re-running the mutations:
- Only THREE of the nine Unicode cases fail 1b78dc9e (é/édith/BJÖRK — where query and stored casing
differ, so the escape texts diverge); the other six pass it. The comment said all nine. They stay
as continuity coverage, now labelled as such rather than as regression guards.
- Ordering_Is_Best_Effort used "Zulu"/"Éclair", where DB and ordinal orderings BOTH pick "Zulu" — it
could not demonstrate the divergence it claimed. Now "Zulu"/"apple", which actually diverges:
ordinal ranks "Zulu" first, the DB ranks "apple" first, and limit=1 returns ["apple"]. The record
sentence was false and is corrected.
- The record printed literal "Édith"/"é" where it needed to show the escape TEXT (Édith,
é), contradicting the very explanation it was giving.
- Corrected the cost claim: the leading wildcard forces scan ACCESS, but each page stops once it has
filled @Batch, so a dense query finishes early — it is not necessarily a full table scan.
- The Unicode sweep is labelled a PROOF OBLIGATION: it is revision-independent and passes every
revision, which is correct for what it is but must not read as regression coverage.
One process note: the new record's frontmatter had a lone apostrophe inside a single-quoted YAML
scalar ("SQLite's"). decisions_validate.py's hand parser accepted it; scripts/tests caught it.
timothy
a37847e509