Compare commits

...
Author SHA1 Message Date
renovate 6b41340abe chore(deps): update dependency sqlitepclraw.bundle_e_sqlite3 to 3.0.5
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 9s
PR Gates / Docs update reminder (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 34s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m21s
review-verdict/h10 Exempt: authored by the 'renovate' bot account, touches no protected path, and changes only dependency manifests
Review verdict / Set review-verdict status (pull_request_target) Successful in 34s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m18s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m59s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m35s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
2026-08-03 17:03:35 +00:00
timothy 9928be805f Merge pull request 'chore(deps): batch three Renovate patch bumps (supersedes #679, #680, #681)' (#714) from chore/renovate-batch-2026-07-30 into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 16s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 27s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 28s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 10m54s
Build CI Toolchain Image / Build & push CI image (push) Successful in 9m5s
Dependency vulnerability scan / NuGet vulnerable packages (push) Successful in 1m10s
Renovate / Renovate (push) Successful in 2m20s
2026-07-30 21:31:32 +00:00
timothy fbbdaeca3c chore(deps): batch three Renovate patch bumps
PR Gates / Docs update reminder (pull_request) Successful in 22s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 25s
PR Gates / decisions lifecycle (pull_request) Successful in 32s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 58s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 57s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m8s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m23s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m9s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m47s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ fbbdaec (base: main)
Review verdict / Set review-verdict status (pull_request_target) Successful in 5s
Consolidates the three open Renovate PRs into one change so they land on a
single CI pipeline instead of three. They all edit Directory.Packages.props,
so merging them individually would force a rebase + full re-run for each
remaining PR.

  CliWrap                        3.10.2  -> 3.10.3
  Meziantou.Analyzer             3.0.115 -> 3.0.129
  SQLitePCLRaw.bundle_e_sqlite3  3.0.3   -> 3.0.4

Also refreshes the #8 security-pin comment, which named 3.0.3 explicitly.
Renovate only rewrites the version attribute, so its own PR would have left
that comment contradicting the line directly beneath it. The pin's intent is
unchanged: stay on the 3.x line that ships the patched native SQLite
(GHSA-2m69-gcr7-jv3q), and 3.0.4 still satisfies Microsoft.Data.Sqlite's
`>= 2.1.10`.

All three original PRs went red on 2026-07-27, but none of the failures
reached any code. The runner host had exhausted its disk at ~03:20 UTC:
tar cache-restore failing with "No space left on device", SQLite Error 13
"database or disk is full", and ErsatzTV refusing to boot for want of 128 MB
of free space. #681's migration job in particular died during cache restore,
before either the SQLite or MySQL half ran, so the native-bundle bump was
never actually exercised there.

Verified locally on this combined change:
  - dotnet build -c Release: 0 errors; zero MA/S/CA analyzer diagnostics, so
    the Meziantou 3.0.115 -> 3.0.129 jump introduces no new rules that bite
  - full test suite: 4440 passed, 0 failed across all 7 test projects
  - SQLite model drift clean + all migrations applied to a fresh DB, which is
    the exact job that was red on #681

Supersedes #679, #680, #681.
2026-07-30 22:58:01 +02:00
timothy f9cbd152bc Merge pull request 'chore: ignore .codex/, and stop shipping a plaintext credential in docs' (#712) from chore/codex-ignore-and-credential-redaction into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 16m45s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 20m5s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 22m4s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m18s
2026-07-30 20:27:33 +00:00
timothyandClaude Opus 5 980da6db00 chore: ignore .codex/, and stop shipping a plaintext credential in docs
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 10s
PR Gates / Docs update reminder (pull_request) Successful in 16s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 15s
PR Gates / decisions lifecycle (pull_request) Successful in 22s
Review verdict / Set review-verdict status (pull_request_target) Successful in 21s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m15s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 17m25s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m3s
review-verdict/h10 Review-verdict: MERGEABLE @ 980da6d (base: main)
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m44s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
.codex/ is generated by `codex exec` as a machine-local mirror of the .claude hooks.
It is deliberately NOT tracked even though .claude/ is (17 files): its config.toml
embeds a plaintext Gitea credential and absolute /Users paths, so committing it would
leak the credential and would not be portable anyway. Ignoring it also unblocks
scripts/refresh-shared-checkout.sh, which refuses on a dirty tree.

Separately, docs/handoffs/rest-api.md carried the same credential inline; it now
references $ETV_GITEA_BASICAUTH like every other doc. NOTE this does not purge git
history — the literal appears in 12 earlier commits and is still recoverable there.

Refs: #698
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 21:59:27 +02:00
timothy 81be685df9 Merge pull request 'docs(698): correct the Renovate auto-pass rule in CLAUDE.md' (#710) from docs/698-claudemd-renovate-rule into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 38s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 38s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 39s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 10s
2026-07-30 19:58:03 +00:00
timothyandClaude Opus 5 1eca9b0c11 docs(698): correct the Renovate auto-pass rule in CLAUDE.md
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 18s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19s
PR Gates / decisions lifecycle (pull_request) Successful in 20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 39s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 38s
review-verdict/h10 Exempt: docs-only change (no code, no protected path)
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 38s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 38s
Review verdict / Set review-verdict status (pull_request_target) Successful in 24s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
PR Gates / Script tests (pytest) (pull_request) Successful in 1m0s
CLAUDE.md still described the bot exemption as identity-only (auto-passed unless a
protected path is touched). Since #698 it also requires EVERY changed path to be a
dependency manifest — a bot account does not attribute the code at a head. CLAUDE.md is
loaded every session, so a stale rule here is worse than a stale doc.

Refs: #698
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 21:56:15 +02:00
timothy a3458e6e2c Merge pull request 'fix(698): bind the base, constrain the bot exemption by content, re-derive unattributable successes' (#705) from fix/698-exemption-provenance into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 37s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 37s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 38s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 16m20s
2026-07-30 19:22:01 +00:00
timothyandClaude Opus 5 57e33f9937 chore(698): drop a trailing blank line at EOF
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 24s
Review verdict / Set review-verdict status (pull_request_target) Successful in 12s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 8s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m31s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m3s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ 57e33f9 (base: main)
Nit from review round 6 (git diff --check). No behaviour change.

Refs: #698
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 21:02:41 +02:00
timothyandClaude Opus 5 fe00e0d71f fix(698): compare the recorded base exactly, never parse it out
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 37s
PR Gates / Docs update reminder (pull_request) Successful in 40s
PR Gates / decisions lifecycle (pull_request) Successful in 45s
review-verdict/h10 Awaiting review verdict for fe00e0d
Review verdict / Set review-verdict status (pull_request_target) Successful in 14s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m23s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m19s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m34s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m12s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 21m4s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 24m7s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Review round 5 returned BLOCKED with one High, and it needed no forgery and no #697 —
just a branch name.

`main)evil` IS A VALID GIT BRANCH NAME (`git check-ref-format --branch 'main)evil'`
succeeds). A genuine human verdict earned while head H targeted it is written
`(base: main)evil)`. Truncating at the first `)` yields exactly `main`, which matches a
PR that has since been retargeted onto `main`, so the verdict is inherited over a
completely different diff.

I had asserted the opposite in a code comment one commit earlier — that a `)` in a
branch name "mismatches — safe direction". That was generalised from `feat/foo)bar`,
which does mismatch, and is false for EVERY branch whose name starts with the target
base. Two attempts at extracting this value have now been defeated (`##` last-marker by
an appended marker, `#` first-marker by this), so the lesson is the shape, not the
off-by-one: do not parse a value out of user- or attacker-influenced text when you can
compare against the exact expected literal instead.

The description must now END with the literal `(base: <this PR's base>)` AND contain
exactly ONE marker — the marker count kills the append trick without having to decide
which occurrence is authoritative. Pure shell (`${#}` arithmetic), no truncation to
abuse. Verified across all six shapes, including a PR that legitimately targets
`main)evil` (accepted) and `(base: )` (rejected). Absent markers remain accepted, since
verdicts predating #632 carry none.

Mutation-verified: restoring the truncating parse reddens only the new paren test, while
the appended-marker, matching-base and legacy tests stay green.

385 tests pass. Note for the record: pytest has never executed inside the review sandbox
in any of the five rounds, so the suite has only ever been run here.

Refs: #698
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 20:51:57 +02:00
timothyandClaude Opus 5 ef92b46dd2 fix(698): parse the recorded base at its FIRST occurrence, not its last
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 17s
PR Gates / Docs update reminder (pull_request) Successful in 19s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
review-verdict/h10 Awaiting review verdict for ef92b46
Review verdict / Set review-verdict status (pull_request_target) Successful in 23s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m21s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m31s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m31s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m40s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m35s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m29s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Self-found while writing the round-5 review brief, by asking what an attacker who can
influence the status description (#697) could do to the parse I had just added.

`${ex_desc##*"(base: "}` is greedy, so it reads the LAST occurrence. A description of
`Review-verdict: MERGEABLE @ abc1234 (base: probe/scratch) (base: main)` therefore parsed
as `main`, matched the PR's base, and the verdict was inherited — reopening the exact hole
the base check was added to close, one commit earlier. Measured both forms before choosing:
first-match yields `probe/scratch`, mismatches, and fails closed.

Two adjacent cases confirmed to fail in the safe direction: a `)` inside a branch name
truncates the value (mismatch), and an empty `(base: )` is present-but-different (mismatch),
so neither is waved through by the legacy-absent-base allowance.

Tests for both, and the appended-base test is mutation-verified: restoring `##` reddens it.

384 tests pass.

Refs: #698
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 22:11:08 +02:00
timothyandClaude Opus 5 e7bae06385 fix(698): review round 5 — a human verdict formed against ANOTHER base is no longer inherited
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / Docs update reminder (pull_request) Successful in 21s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 23s
review-verdict/h10 Awaiting review verdict for e7bae06
PR Gates / decisions lifecycle (pull_request) Successful in 31s
Review verdict / Set review-verdict status (pull_request_target) Successful in 21s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m5s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m27s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m23s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m13s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Round-4 cross-family review returned BLOCKED with a single Medium; the three round-3
items were confirmed actually fixed.

THE SHA-BINDING WAS ESCAPABLE THROUGH THE HUMAN PATH, not the exemption path. The
short-circuit identified a human verdict by creator + `Review-verdict:` prefix and then
exited before looking at the base. So: earn a GENUINE `success` on head H while it
targets a scratch base with a benign diff, then retarget H onto `main`, where its diff
carries unreviewed code. Creator real, prefix real, status inherited — a green required
check over code nobody reviewed. `post-review-verdict.sh` has recorded the reviewed base
in the description since #632; this gate simply never read it. The merge-consent hook
did compare it, but that is advisory and covers only its own path: a merge through the
Gitea UI or API sees nothing but the status.

The gate now rejects a verdict whose recorded base differs from the PR's. An ABSENT base
is deliberately NOT a mismatch — verdicts predating #632 carry none, and re-deriving over
one would un-approve a genuinely reviewed head. Only present-and-different is rejected,
which is exactly the escape.

Tests: the mismatch case, plus two positive controls (matching base still short-circuits;
a legacy no-base verdict still short-circuits) so the check cannot pass by blanket
rejection. Mutation-verified: removing the check reddens only the mismatch test.

Also from round 4: sharpened the docstring of test_the_classify_step_runs_without_SHELL_ERRORS.
It catches guards that die NOISILY; it is not a general liveness check, since a clean
mutation like hardcoding n_protected=0 emits nothing. The branch-discriminator test is the
actual liveness guard. Claiming otherwise would have made a cheap net look like a strong one.

And fixed a dangling decision key I had just introduced: the base-in-description convention
belongs to `release.verdict-status-check`, not the `ci.verdict-records-base` I invented —
the breadcrumb hazard our own retrieval rules warn about.

382 tests pass.

Refs: #698
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 21:44:44 +02:00
timothyandClaude Opus 5 d4c600149d fix(698): review round 4 — the PROTECTED guard was DEAD; define before use, fail closed, fix prescriptive docs
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 19s
PR Gates / decisions lifecycle (pull_request) Successful in 22s
review-verdict/h10 Awaiting review verdict for d4c6001
Review verdict / Set review-verdict status (pull_request_target) Successful in 13s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m23s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 22s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m38s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m30s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m18s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Round-3 cross-family review returned BLOCKED with 3 Mediums. The first was serious
and self-inflicted.

THE PROTECTED GUARD WAS A NO-OP. Round 3's `count_matching` / `count_not_matching`
helpers were defined AFTER the classification chain that calls them, so
`count_matching` was `command not found` on every run, `$( )` yielded an empty string,
`[ "" -gt 0 ]` errored, and the `elif` was simply skipped — the protected-path check
never executed at all. Confirmed by direct execution before fixing.

Three "protected path" tests stayed GREEN throughout, because a protected path is also
not a manifest and not docs-only, so the job still reached `pending` down a different
route. Asserting the STATE could not distinguish a working guard from a dead one. The
mutation battery missed it too: I had mutated the predicates, not their reachability.

Fixed three ways:
  * helpers are defined immediately after `gh()`, before any use;
  * the three counts are evaluated ONCE at TOP LEVEL and validated numeric, because
    `exit 1` inside `$( )` leaves only the subshell and, with the substitution sitting
    in a conditional, `set -e` never fires either — so a grep error had been silently
    reading as "no match". A non-numeric result now aborts with nothing posted, and an
    absent required check blocks the merge;
  * the helpers return a non-numeric sentinel instead of trying to `exit`.

Verified: an invalid regex now exits 2 and posts NOTHING (previously it classified and
posted). Renaming the helper at its definition turns six tests red.

TESTS, aimed at the failure mode rather than the symptom:
  * assert the DISCRIMINATOR (the job's `Decision:` reason line), not the outcome —
    when several branches yield the same verdict, the verdict cannot tell you which ran.
    A first draft of this test asserted the status description and failed against a
    WORKING guard, because for `pending` the description is constant;
  * a cheap stderr sweep for `command not found` / `integer expression expected` /
    `unbound variable` across four representative PR shapes. Each of those makes an `if`
    condition merely false while the job exits 0 and posts a plausible status, so this
    catches a whole family of silently-skipped guards.

DOCS. The record and ci-cd.md still PRESCRIBED the here-string that round 3 removed —
following them would have reintroduced the temp-storage failure. Both now prescribe
counting, define-before-use, top-level evaluation and numeric validation. The workflow's
measurement paragraph still said the npm manifests "are included" three lines above the
note saying they are excluded; corrected.

379 tests pass.

Refs: #698
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 21:13:56 +02:00
timothyandClaude Opus 5 d8bd1dcba9 fix(698): review round 3 — count instead of matching, re-read before the POST, fix stale docs
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 18s
PR Gates / Docs update reminder (pull_request) Successful in 23s
PR Gates / decisions lifecycle (pull_request) Successful in 31s
review-verdict/h10 Awaiting review verdict for d8bd1dc
Review verdict / Set review-verdict status (pull_request_target) Successful in 36s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m13s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 19s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 16s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m13s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m28s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m59s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Round-2 cross-family review returned BLOCKED: 2 High + 3 Medium.

HIGH — here-strings traded one fail-open for another. `grep -q… <<< "$data"` fixes
the SIGPIPE inversion, but bash materialises a large here-string via temporary
storage, so it fails when temp space is full or unwritable — and since these sit
inside `if`/`!`, that failure flips the predicate exactly as SIGPIPE did. It did NOT
reproduce on my bash 3.2, DID on the reviewer's Linux bash 5.x, and CI is Linux; the
disagreement is itself the argument for a construct that cannot fail either way.

Path predicates now COUNT with `grep -c`, which drains stdin (no early exit, no
SIGPIPE) over an ordinary pipe (no temp file), and grep's status is read honestly:
exit 1 means "zero matches", a legitimate answer, while >1 is a real error that FAILS
THE JOB rather than silently reading as "no match". `set -e` does not catch these on
its own — they sit in command substitution inside a conditional. Verified correct
under 171KB input AND an unwritable TMPDIR. The description test became a `case`
prefix match, removing another pipeline from a security predicate. New record
`ci.grep-q-pipefail-inversion` covers the whole class.

HIGH — a human verdict landing mid-run was still overwritten, and the code claimed
otherwise. The job read statuses once, classified over several round-trips, then
posted: a reviewer posting BLOCKED in between had it replaced by an exemption
`success`, turning an explicit rejection into a merge. Added a re-read immediately
before the POST which refuses to write over a human verdict found then. The heading
no longer says "never overwrite" — it cannot promise that, since there is no
compare-and-set on Gitea's status API. Remainder tracked as #706.

MEDIUM — documentation was stale in three places, all mine. The record's frontmatter
`rule:` still listed the npm manifests (I fixed the body and forgot the frontmatter,
so the canonical rule AND the generated catalog were wrong); docs/ci-cd.md still said
`edited` was absent from `types:`, contradicting a section I had just updated; and the
workflow header still implied the `edited` re-run settles the ABA race. All corrected
to say detection, not atomicity.

TESTS. 373 pass. New: a mid-run human verdict via a status stub that returns nothing
on the first read and BLOCKED on the re-read, and large-input regression tests for the
ADVISORY hook, which had none — the copy with less authority is the one that quietly
keeps a bug (#649's whole point). Mutation-verified: reverting the hook predicate,
the workflow predicates, or the pre-POST re-read turns exactly the intended tests red
while every positive control stays green.

Refs: #698
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 20:53:36 +02:00
timothyandClaude Opus 5 bafb487eaa fix(698): review round 2 — close a PRE-EXISTING SIGPIPE inversion, narrow the bot allow-list, correct two overclaims
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 24s
PR Gates / decisions lifecycle (pull_request) Successful in 33s
review-verdict/h10 Awaiting review verdict for bafb487
PR Gates / Script tests (pytest) (pull_request) Successful in 59s
Review verdict / Set review-verdict status (pull_request_target) Successful in 34s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m30s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m36s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m9s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m6s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Cross-family adversarial review returned BLOCKED with 3 Blockers + 2 Highs. Two
were verified independently before acting on them.

BLOCKER 1 (pre-existing on main, not introduced here). `producer | grep -q…`
inverts under `set -o pipefail`: grep exits at its first match, the producer takes
SIGPIPE and exits 141 once the list exceeds the pipe buffer, so a MATCH is reported
as a FAILED pipeline. Reproduced at 1901 paths / ~171KB, inside the enumerator's
2000-file cap:
  * `A.cs` + 1900 docs paths  -> docs_only=yes, status 141 (a C# PR exempted)
  * `.gitea/…` + 1900 docs    -> the PROTECTED guard MISSES entirely
The second reaches a green required status on a PR rewriting the gate, with no
retarget, no bot account and no forged status. This construct shipped with the
original docs-only check (review-verdict.yml:257 on main), so it was live. Fixed in
both the enforced gate and the advisory hook by using here-strings, which have no
writer to signal. New record `ci.grep-q-pipefail-inversion`.

BLOCKER 2 (mine). `web/package.json` / `web/package-lock.json` are removed from
BOT_MANIFESTS. renovate.json enables only nuget/github-actions/dockerfile, so npm is
unmanaged here and the entry bought nothing — while package.json `scripts` are
EXECUTED by CI (npm ci, npm run build). It widened an exemption onto a
code-execution path for no benefit.

BLOCKER 3 + HIGH (documentation was wrong, code unchanged). The claim that `edited`
made the retarget residual "non-durable" is retracted: runs are not serialized, so a
stale run can post `success` after the reclassifying run posts `pending`. The ABA
transition is narrowed and observable, NOT closed. Likewise the provenance check
asks "posted by a user credential", not "posted by a reviewer" — ETV_STATUS_AUTH is
basic auth, so a #697 forgery gets a non-null creator AND an attacker-chosen
description and is preserved as human. Both now stated at full strength.

TESTS. 4 large-input cases crossing the pipe buffer, each paired with a large-input
POSITIVE control so "large lists now fail closed" (a deadlock) cannot pass as a fix.
Verified by mutation: reverting the here-strings turns all three negatives red while
the control stays green. Two of my own weak tests fixed — the "base advances" case
called head_moves_to(SHA) with the already-current sha (a duplicate positive control,
now a structural assertion that the comparator is .base.ref and never .base.sha), and
the arity test counted five arguments without checking the fifth was the base.

The whole class was invisible because every previous test used a handful of short
paths: a guard whose behaviour depends on a buffer threshold needs a test that
crosses it.

Refs: #698
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 20:35:29 +02:00
timothyandClaude Opus 5 f523fc535d fix(698): bind the base, constrain the bot exemption by content, re-derive unattributable successes
PR Gates / decisions lifecycle (pull_request) Successful in 24s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 27s
PR Gates / Docs update reminder (pull_request) Successful in 28s
Review verdict / Set review-verdict status (pull_request_target) Successful in 17s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m1s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m9s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m8s
review-verdict/h10 Review-verdict: BLOCKED @ f523fc5 (base: main)
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 23m34s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The `review-verdict/h10` exemption path decided from mutable or unattributed PR
state, and a machine-written `success` was never revalidated. Three routes, one
root cause, so one change.

Route 1 (reproduced live as probe PR #703, closed unmerged): `/pulls/{n}/files`
diffs against the PR's LIVE base, so retargeting moves the answer without moving
the head sha. A PR opened into `main` and retargeted mid-run enumerated docs-only
and was granted `h10=success` while its diff against `main` carried a C# file;
retargeting back reclassified nothing. `scripts/pr-changed-files.sh` now takes the
expected base branch as a REQUIRED 5th argument (optional would be a silent
opt-out) and checks it before and after paging; the workflow passes it from the
`pull_request_target` payload, which a retarget cannot rewrite, and `edited` is in
`types:` so a retarget reclassifies.

A pinned two-sha diff would close route 1 outright but Gitea 1.25.4 cannot serve
one: `compare/{base}...{head}` returns no `files`, and a `--depth=1` fetch of the
two shas has no merge base. Measured, not assumed. The residual window is stated
in the code and the record rather than papered over.

Route 2: `pull_request.user.login` is the PR's immutable CREATOR while its head is
not, so pushing code onto an open Renovate branch kept the exemption. The bot
exemption now also requires EVERY path to be a dependency manifest — a set measured
across all 11 Renovate PRs this repo has had, not guessed.

Route 3: the never-overwrite short-circuit exited on ANY `success`, so a forgery
obtained once was inherited forever. It now fires only for a status positively
identified as a human verdict (non-null `.creator.login` AND a `Review-verdict:`
description — measured: user-posted statuses carry a creator, Actions-posted ones
carry null). Written in the positive direction so an unrecognised shape is
re-derived rather than trusted.

The two exemptions are composed, not chained: as an `elif` chain a Renovate
docs-only PR lost the docs-only exemption. Caught before commit and pinned by a
test.

Tests: 17 new cases in scripts/tests/test_pr_changed_files.py, each verified by
mutating the clause it covers (8 mutations, 8 kills). Both records trimmed under
the 60-line prose ceiling so the corpus tail-boundary check stays calibrated.

Does NOT close the class: anyone who can POST a status directly can still
impersonate a verdict — that is #697, deliberately left open.

Refs: #698
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 19:52:00 +02:00
timothy 0c492defac Merge pull request 'fix(672): trigger the verdict gate on pull_request_target scoped to main' (#699) from fix/672-review-verdict-head-resolution into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 16m19s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 20m21s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 23m5s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 9m39s
2026-07-28 21:44:19 +00:00
timothy 4e2ea61674 Merge pull request 'fix(691): guard the nullable SongMetadata.Artists/AlbumArtists at their read sites' (#700) from fix/691-song-artists-null-guard into main
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 17s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 31s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 33s
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
2026-07-28 21:43:29 +00:00
timothy ceef16081d docs(672): make the self-test gap discoverable (signals + section pointer)
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Successful in 20s
PR Gates / Script tests (pytest) (pull_request) Successful in 51s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 20s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m50s
review-verdict/h10 Review-verdict: MERGEABLE @ ceef160 (base: main)
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m57s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m17s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Re-review of the round-2 head returned MERGEABLE with one LOW: the record's
`signals:` field did not mention the self-test gap. That field is the semantic
discovery surface -- it is what the MemPalace mirror matches on -- so the most
operationally dangerous property of this change ("a gate edit goes live only on
merge, having never run") was unreachable by anyone searching for it. Someone
asking "how do I test a change to review-verdict.yml" would have found nothing.

Adds three signal phrases and points the record's one-line reference at a
section rather than at a ~1050-line file.

No behaviour change; frontmatter and prose only. Verified the frontmatter still
parses under PyYAML rather than the validator's hand parser, per #674 -- an
apostrophe in a single-quoted scalar is exactly what that hand parser cannot see.

Refs: #672
Decisions-Edit: yes
2026-07-28 23:17:14 +02:00
timothy 20b7171fba fix(672): review round 2 -- correct a stale rule: field, document the self-test gap
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 19s
PR Gates / decisions lifecycle (pull_request) Successful in 26s
PR Gates / Script tests (pytest) (pull_request) Successful in 52s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 20s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m22s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m49s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Re-review of the fix commits returned MERGEABLE (nine trigger mutations all
caught, every prior finding verified against independent sources) with four
low-severity findings. All four are addressed here.

F1: `release.verdict-status-check`'s `rule:` frontmatter still said "A
`pull_request` workflow auto-passes the two exempt classes". Round 1 past-tensed
that record's BODY and left its `rule:` stale -- which is the exact failure mode
the previous commit cites as the reason to put limitations in `rule:` in the
first place. The catalog row mirrors this field verbatim and it mirrors again
per-`key:` into MemPalace, so a stale `rule:` propagates further than a stale
paragraph.

F2: same record, "is what makes the rollout self-hosting" -> past tense. It
described #630 and now reads as a live property.

F3, the one that matters operationally: base resolution cuts BOTH ways. A change
to `review-verdict.yml` is no longer exercised by its own PR -- the PR runs the
version already on `main` -- so an edit goes live only ON MERGE, repo-wide,
having never run. A broken edit merges green and then breaks the gate for every
subsequent PR, and the PR that would repair it is gated by the same broken
workflow. The recipe for verifying one safely (scratch base + throwaway PR +
probe-named context) now lives in docs/ci-cd.md, which is where an operator
looks, rather than in the record.

F4: the sibling-workflow guard globbed `*.yml`, so a workflow added as `.yaml`
would be silently unscanned. Latent today, which is when it is cheap.

The record lost its meta-justification paragraph to the 60-line prose ceiling.
Fifth trim this session; the operational recipe moving to ci-cd.md is better
placement anyway, but it was forced rather than chosen. ersatztv#688.

Refs: #672
Decisions-Edit: yes
2026-07-28 23:03:15 +02:00
timothy b2a5c72bfe docs(672): widen the residual to the real inventory (#697, #698)
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 17s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 19s
PR Gates / Docs update reminder (pull_request) Successful in 21s
PR Gates / decisions lifecycle (pull_request) Successful in 26s
PR Gates / Script tests (pytest) (pull_request) Successful in 43s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m52s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m59s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 24m4s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Cross-family review established the residual is materially larger than the
previous commit said, and that saying "docker-build.yml / ETV_STATUS_AUTH"
understates it.

Gitea injects `GITEA_TOKEN` into EVERY job automatically, defaulting to
read/write. So the set of workflows that can POST `review-verdict/h10` is not a
short list to audit -- it is all of them, plus `workflow_dispatch` (1.24+ loads
the definition from the selected branch) and `push`-triggered ones. A
collaborator's own write-scoped API token is a route with no workflow at all,
because branch protection binds the required CONTEXT, not its issuer. Recorded
in #697.

The same review found three defects in the exemption path itself, none of them
introduced here and none closed here: a retarget race that enumerates a
docs-only diff against a scratch base while the enumerator revalidates only
head.sha, a Renovate-PR hijack (the exemption reads the immutable PR creator,
not who pushed the head), and an inherited `success` that short-circuits before
any PR/base/author/file check. Filed together as #698, since they share one root
cause -- the gate trusts state it cannot attribute.

Also drops the claim that the three properties are "pinned by tests". The tests
pin the workflow's SHAPE; no in-repository test can establish status-authority
isolation, and the sibling-workflow guard added in the previous commit catches
only a workflow that names the context in plain text.

Trimmed to the 60-line prose ceiling for the third time in this session. That is
ersatztv#688 -- the ceiling is now deleting rationale two reviewers asked for.

Refs: #672
Refs: #697
Refs: #698
Decisions-Edit: yes
2026-07-28 22:41:48 +02:00
timothyandClaude Opus 5 dd7b58232c fix(691): revert entity-level null guard, guard read sites instead
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 59s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m54s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m4s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 11s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 8s
review-verdict/h10 Review-verdict: MERGEABLE @ dd7b582 (base: main)
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 22m12s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Review verdict / Set review-verdict status (pull_request) Successful in 5s
PR Gates / decisions lifecycle (pull_request) Successful in 13s
PR Gates / Script tests (pytest) (pull_request) Successful in 46s
The prior commit (a4700185b) made SongMetadata.Artists/AlbumArtists
coalesce null to [] via backing-field getters, reasoning that EF
Core's PreferField access mode never observes the getter. Adversarial
review disproved this on real TvContext/SQLite: a single read of
.Artists on a TRACKED entity mutates the backing field through the
getter, flips the entity to Modified, and the next SaveChanges writes
[] over what was a NULL column -- silent data loss waiting on the
first tracked reader (today all readers happen to be AsNoTracking).

This also reversed docs/decisions/records/api/selection-projection-include-chain.md
(#671) without the doc update CLAUDE.md requires; #691 is that
record's own "sweep by FIELD" follow-up, so it should follow the
record, not contradict it.

Revert SongMetadata.cs to plain auto-properties (byte-identical to
origin/main, BOM still stripped per the #311 gate). Guard the read
sites instead, per the #671 convention (Optional(...).Flatten(),
matching Playouts/Mapper.cs and MediaItems/Mapper.cs):

- SongVideoGenerator.cs: hoist `artists`/`albumArtists` locals once
  near the top of the metadata loop instead of repeating the guard at
  each of the six former call sites.
- MediaCollectionRepository.cs (GroupIntoFakeCollections): guard the
  two AlbumArtists reads at lines ~1147/~1160 that #691 never named --
  dropping the entity-level fix without these would trade one bug for
  two.

Verified RED per guard by removing only the Optional(...).Flatten()
clause (not the whole file): the artists local throws
ArgumentNullException at SongVideoGenerator.cs:88, the albumArtists
local at :89 (List.ToList() on a null IList<string> source -- same
loaded-gun shape the review demonstrated, precise exception type is
ArgumentNullException rather than NullReferenceException since the
throw site is Enumerable.ToList's null-source check). Restored both;
existing SongVideoGeneratorTests still pass. Full ErsatzTV.Core.Tests:
685 passed (1 pre-existing skip), ErsatzTV.Tests: 1996 passed (4
pre-existing skips), 0 failures in each. No EF model drift
(`dotnet ef migrations has-pending-model-changes` reports none).
`dotnet format --verify-no-changes` on the three touched files exits
0.

Refs #691

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 22:39:52 +02:00
timothy 35a8ea8aef fix(672): review round 1 -- pin the trigger set exactly, sweep the stale claims
Cold review found the first cut of the test satisfiable by a still-vulnerable
config, and two prose claims that outran the evidence.

The test asserted "pull_request_target present, pull_request absent". Adding
`workflow_dispatch:` or `push:` ALONGSIDE it kept that green, and both are
ref-resolved with secrets, so either one restores an equivalent
self-supplied-definition path. Enumerating those two would have the same hole one
trigger later, so the assertion now pins the whole set: exactly
{pull_request_target}, nothing else. Verified by mutation -- adding
`workflow_dispatch` now reds.

Adds the guard that would have caught the residual below rather than only the
instance: no workflow OTHER than review-verdict.yml may reference
`review-verdict/h10` in executable lines. Scoped honestly in its docstring as a
drift guard, not a security boundary -- a workflow can still write the status
through an indirection a text scan cannot see.

`release.verdict-status-check` item 4 still asserted, in the present tense, that
a PR editing review-verdict.yml is judged by its own edited copy. That is now
false for this workflow, and it is the record a reader resolving the gate from
the catalog actually lands on. Past-tensed, with the surviving residual named.

The probe count said three. There were four; the omitted one is the only one with
a negative result, which is what turns an honest partial into an overclaim.
Corrected in both the record and ci-cd.md, along with what was NOT measured
(`reopened`/`ready_for_review` firing under the new trigger).

Also records two operational consequences a maintainer will otherwise hit cold:
retargeting a PR onto `main` leaves it statusless until its next push (`edited`
is not in `types:`), and the required contexts carry a literal `(pull_request)`
suffix, so repeating this fix on docker-build.yml would rename them and deadlock
merges unless branch protection is edited in the same operation.

Trimmed the record back under the 60-line prose ceiling -- for the second time
this session, which is ersatztv#688 reproducing, not a defect here.

Refs: #672
Refs: #697
Decisions-Edit: yes
2026-07-28 21:49:22 +02:00
timothy 8b73234d78 docs(672): record that the fix closes the route, not the class (#697)
Probing rather than reasoning turned up a second instance of the same
vulnerability class while this fix was in review. `docker-build.yml` also
triggers on `pull_request`, so it is head-resolved too, and it carries
`ETV_STATUS_AUTH` (`REGISTRY_USER:REGISTRY_PASSWORD`) for the #420 revalidation
read. Basic auth is not scoped: an account that can read commit statuses can
write them. Confirmed with a scratch PR that POSTed a probe-named context using
those credentials and succeeded — so a PR rewriting `docker-build.yml` can still
post `review-verdict/h10=success` for its own head.

That workflow cannot take the same fix. It builds and tests the PR's code, so it
must resolve from the head; `pull_request_target` there would be the real
footgun. It needs a read-only status identity instead. Filed as #697.

The `rule:` field carries the limitation, not just the body, because the
predecessor record's documented failure was exactly a reassuring sentence in the
position a catalog reader stops at.

Also trims the record to the 60-line prose ceiling. Adding it at 62 lines pushed
p90 past the ceiling and reddened the blocking `script-tests` job — which is
ersatztv#688 reproducing live, not a defect in this change.

Refs: #672
Refs: #697
Decisions-Edit: yes
2026-07-28 21:41:26 +02:00
timothyandClaude Opus 5 a4700185b2 fix(691): guard SongMetadata.Artists/AlbumArtists at the domain boundary
SongMetadata.Artists and .AlbumArtists are nullable EF primitive
collections that FallbackMetadataProvider.GetSongMetadata never
assigns, so untagged songs persist them as NULL. SongVideoGenerator
dereferenced both unguarded (metadata.Artists.Count, string.Join,
AlbumArtists.Filter(...Artists.Contains...)), throwing NRE/ANE during
song-video generation on the playback path.

Rather than enumerating and guarding each read site (the same mistake
that left these unswept after #671), add backing fields to the two
properties whose getters coalesce null to an empty list. EF Core's
default PreferField access mode reads/writes the raw backing field
during materialization and change-tracking (confirmed by running the
full ErsatzTV.Tests suite, including SongMetadata round-trip tests,
unchanged), while every other caller -- SongVideoGenerator,
MediaCollectionRepository's rerun-collection artist grouping, and any
future reader -- goes through the property getter and always sees a
non-null list. This subsumes the ad hoc `metadata.Artists ??= []`
guards already hand-applied in LuceneSearchIndex/ElasticSearchIndex
and the `?? []` in LibraryBrowseItemMapper, which remain but are now
redundant.

Adds SongVideoGeneratorTests covering an untagged song (null Artists/
AlbumArtists) through GenerateSongVideo; verified RED (NRE at
SongMetadata.cs's Artists getter) by reverting only the `??= []`
clause, not the file.

Strips the pre-existing UTF-8 BOM from SongMetadata.cs per the #311
formatting gate (touching a legacy-BOM file makes stripping it ours
to do).

Refs #691

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 21:30:06 +02:00
timothy cf907f0988 fix(672): trigger the verdict gate on pull_request_target scoped to main
`review-verdict.yml` writes `review-verdict/h10`, the status branch protection
requires on `main`. It triggered on `pull_request`, and Gitea resolves a
`pull_request` workflow DEFINITION from the PR's own head commit — so a PR
editing this file ran its own rewritten copy and could post `h10=success` for
itself. `PROTECTED` did not help (the rewrite defines it too), and neither did
branch protection, which requires the context rather than an author and carries
`required_approvals: 0`.

The #649 base-ref checkout binds the SCRIPTS this job runs to the base. It
cannot bind the job definition. Only the trigger can.

Measured on this instance (Gitea 1.25.4) with three throwaway scratch PRs
rather than inferred from GitHub, since the premise is a claim about someone
else's software:

  A. A PR rewriting the workflow posted its own probe context and the real
     `review-verdict/h10` was never written — the base definition never ran.
  B. Under `pull_request_target` the identical rewrite was ignored: the BASE
     definition ran and posted `h10=pending`, on `opened` and `synchronize`
     alike, with `secrets` still available.
  C. With `branches: [main]`, a PR into a non-main base produced no run and no
     status at all.

The probes only ever posted probe-named contexts, never a forged `h10`.

`branches: [main]` is half the fix, not a refinement: base resolution means the
BASE branch supplies the definition, so without it the rewrite simply moves to
an attacker-pushed base — and a status forged there is inherited by any later
real PR with the same head sha (#663).

`pull_request_target` is safe here only because this job never checks out or
executes head-supplied code; the base-ref checkout is what makes the trigger
usable, so the two are one decision.

Rejected `required_approvals: 1` as the cheaper fix: Gitea forbids approving
your own PR and this is effectively a single-maintainer repo, so it would
deadlock every PR rather than gate the dangerous ones.

Three mutations confirm the new test discriminates rather than merely passing:
reverting to `pull_request`, dropping the `branches` filter, and re-adding
`pull_request` alongside the safe trigger each go red with a distinct message.
It parses the YAML instead of substring-matching because `pull_request` is a
prefix of `pull_request_target`.

Refs: #672
Decisions-Edit: yes
2026-07-28 21:28:38 +02:00
timothy 036bcfc5a0 Merge pull request 'fix(671): resolve rerun-collection selections through one shared include chain' (#692) from fix/671-rerun-collection-selection into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 16s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 27s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 28s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m13s
2026-07-28 18:56:13 +00:00
timothyandClaude Opus 5 2249a806c9 fix(671): review round 4 -- fix the chapter-title entity interpolation
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 22s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 29s
PR Gates / Docs update reminder (pull_request) Successful in 41s
PR Gates / decisions lifecycle (pull_request) Successful in 41s
Review verdict / Set review-verdict status (pull_request) Successful in 15s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m17s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 20m1s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 22m49s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 25m35s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ 2249a80 (base: main)
Final cold review of 34eee753b: the commit's own changes were confirmed correct,
but it flagged a real pre-existing bug in the exact block I had just edited, and
I was adding the first-ever tests for that method without covering it.

`Playouts/Mapper.GetDisplayTitle`'s Song arm interpolated `{s}` -- the
`case Song s` ENTITY -- into its chapter-title branch instead of `{t}`, the
composed title. Song has no ToString() override, so a chaptered song rendered as
the literal "ErsatzTV.Core.Domain.Song (Chapter 3)" in the playout guide,
troubleshooting, media-item info and channel states. The sibling MusicVideo and
OtherVideo arms are correct only because they happen to name their lambda `s`.
Pre-existing on main; fixed here because it is one token inside the block this
branch already touches. Two tests pin it; reverting renders the type name.

Also: completed the guard on that arm (`Optional(s.SongMetadata).Flatten()`, the
other half of the sibling pattern I claimed to have copied), added the new
mechanism to the record's `mechanics:`, added the symptom tokens a future session
would actually search for (ArgumentNullException, Artists, primitive collection,
chaptered song) to `signals:`, restored the remedy sentence an earlier trim
dropped, and trimmed to 59 prose lines for margin under the 60-line ceiling.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 20:27:08 +02:00
timothyandClaude Opus 5 34eee753b2 fix(671): review round 3 -- sweep Artists by FIELD, correct the archaeology
Cold review of 572737a29. Findings taken; two are corrections to my own claims.

CORRECTION: "a regression this branch INTRODUCED" was wrong, and I verified the
reviewer's counter-claim against origin/main before accepting it. That handler
already included SongMetadata AND already routed Song there, so
GET /api/v1/playlists/{id}/items was ALREADY a live 500 for a null-Artists song.
This branch only made the same throw reachable on a second path. The record said
so twice; both are fixed, because a wrong explanation outlives a wrong line.

SWEEP: fixing one site left the mirror standing -- Playouts/Mapper.GetDisplayTitle
had the identical unguarded join on a path that also eager-loads SongMetadata, so
it too was live, feeding the playout guide, troubleshooting, media-item info and
channel states. Guarded, with a unit test; reverting it reproduces
ArgumentNullException. LibraryBrowseItemMapper already wrote `Artists ?? []`, so
the nullability was known in-tree and these sites were simply unswept. Filed #691
for the remaining SongVideoGenerator dereferences on the playback path.

Also: documented that the shared matrix is the RERUN predicate used as a superset
for playlists (the playlist write path rejects RemoteStream today); noted the
third, inert consumer ReplacePlaylistItemsHandler; added the new mechanisms to the
record's `mechanics:` field; and trimmed the record under the 60-line prose
ceiling -- it was tipping the corpus p90 above the ceiling and reddening the
calibration test in scripts/tests, which passes again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 20:09:07 +02:00
timothyandClaude Opus 5 572737a29e fix(671): review round 2 -- guard Song.Artists, cover the second consumer
Cold independent review of 017ef988d. Three findings taken, one filed out.

The important one is a regression this branch INTRODUCED. `SongMetadata.Artists`
is a nullable EF primitive collection (JSON in one column, not a navigation)
that FallbackMetadataProvider leaves unassigned when a song's tags fail to read,
and `string.Join` throws ArgumentNullException on a null sequence. The rerun
list previously did not load SongMetadata at all, so the throw was unreachable
there; adding the include promoted it to a live 500 that would have failed the
whole page. Confirmed by reverting the guard: ArgumentNullException, parameter
'values'. The file header claiming every member was guarded was false.
The empty case is filtered too, so an artist-less song loses its bare " - ".

Second: `GetPlaylistItemsHandler` had no handler-level test at all (its
controller tests stub the mediator), so the RemoteStream include added last
round was discharged by inspection -- the same method that produced #671. It
now runs the same 13-type matrix via a shared SelectionSeedData; removing the
include fails that matrix.

Third: dropped the dead `(i as Season).SeasonMetadata` include leg -- the Season
projection reads Show.ShowMetadata and the scalar SeasonNumber, never
SeasonMetadata.

Filed #690 for the pre-existing, out-of-scope finding: the paged TotalCount
ignores the search query, so the SPA renders empty pages.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 19:33:52 +02:00
timothyandClaude Opus 5 017ef988d0 fix(671): review round 1 -- pin exact names, complete the playlist include
Cross-family (Codex) adversarial review of 8523088ce. All three findings taken:

- The name assertion only checked "not a placeholder", so it could not see a
  missing NESTED include leg: dropping Episode -> Season -> Show still renders
  "s00e04 - Selected episode", which contains no placeholder marker and passed.
  Now every type pins its whole expected string; re-removing that leg fails, as
  verified before restoring it.
- Widening the shared switch with a RemoteStream arm put `GetPlaylistItemsHandler`
  one include short -- it loaded metadata for the other nine types, so playlist
  RemoteStream names alone would have degraded to "???".
- `?? 0` rendered an unloaded Season as "s00", which conventionally means
  Specials and so fabricated plausible-looking real data; it now renders "s??".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 19:00:08 +02:00
timothyandClaude Opus 5 8523088ceb fix(671): resolve rerun-collection selections through one shared include chain
The paged list handler eager-loaded nothing, so `ProjectToViewModel` read four
unloaded navigations and every row of every collection type projected a null
selection. Because the selected id and the display name are read off the SAME
navigation, this dropped the id too -- the harm is not an unlabelled badge but
an editor that round-trips a null and clears the user's stored selection.

The by-id handler loaded metadata for only four of the ten selectable media
types: Song/OtherVideo/Image/RemoteStream returned a null-ish selection and
Episode/MusicVideo threw an NRE that surfaced as a 500.

Fixed at the boundary rather than per call site:

- `RerunCollectionQueryExtensions.IncludeSelectionDetails()` is now the single
  include chain, called by both handlers, joining the existing
  `ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails()` precedent
  (#229). Artwork legs are deliberately omitted -- this projection reads only
  ids and titles.
- The media-item switch was duplicated verbatim for RerunCollection and
  PlaylistItem; both now call one `ProjectMediaItemToViewModel`, which handles
  `RemoteStream` (via a new `ProjectToNamedViewModel`, since the existing
  `ProjectToViewModel(RemoteStream)` returns an unrelated type) and never falls
  through to null -- an unknown subtype keeps its id and takes a conspicuous
  name, because throwing would fail a whole paged GET over one bad row.
- Every metadata navigation in `MediaItems.Mapper` is now read through
  `Optional(...).Flatten()`, so an un-included nav degrades to "???" instead of
  being a latent 500 for whichever caller loads least.

Tests enumerate all 13 supported CollectionTypes for both handlers, with the
matrix derived from `IsSupportedSelectionType` so a newly-supported type joins
it automatically, plus a completeness guard on the set. Each mechanism was
removed in turn and confirmed red first.

fixes #671

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:48:24 +02:00
timothy d4ea1584c0 Merge pull request 'fix(668): reach accented facet values via a registered Unicode fold on SQLite' (#687) from fix/668-accented-facet-values into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 33s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 33s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 34s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m17s
2026-07-27 21:08:28 +00:00
timothy f2d9c0dc8e fix(668): review round 5 -- three prose nits, including an off-by-one I filed
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 14s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 21s
Review verdict / Set review-verdict status (pull_request) Successful in 15s
PR Gates / decisions lifecycle (pull_request) Successful in 26s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 15s
PR Gates / Script tests (pytest) (pull_request) Successful in 53s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 17m8s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m0s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 14m33s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ f2d9c0d (base: main)
Final sweep confirmed the retracted MySQL over-match claim survives in no file
on the branch (only in two immutable commit messages, which stay -- rewriting
history would invalidate every sha-bound review verdict). Three nits remained.

- The fixture's class docstring said the on-MySQL claim "rests on the server's
  collation", which is the one thing the decision record says it does NOT rest
  on. It rests on Unicode-aware LOWER(); the executed comparison bypasses the
  collation entirely. Reworded.
- The record's `rule:` enumerated the covered fields but omitted show_genre,
  which GetSource and the fold both handle ("genre" or "show_genre"). Added.
- My own #688 write-up was wrong twice: the 60-line ceiling warning is
  NON-blocking by design, and the calibration pytest reds at >=61, not >=60 --
  main's p90 is 59, so a 60-line record makes p90 == ceiling and PASSES. The
  bullet even contradicted itself, since the next sentence relies on 60 passing.
  Corrected in the PR body and in the issue.

Decisions-Edit: yes
2026-07-27 22:21:55 +02:00
timothy 07723e418b fix(668): review round 4 -- sweep the retracted claim by SUBJECT, not by memory
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / Docs update reminder (pull_request) Successful in 21s
PR Gates / decisions lifecycle (pull_request) Successful in 25s
PR Gates / Script tests (pytest) (pull_request) Successful in 44s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 34s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 19s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m55s
Review verdict / Set review-verdict status (pull_request) Failing after 10m59s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m54s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m38s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Round 3 said "correct the claim everywhere" and missed two places, which is
the same mistake in a new coat: I fixed the spots I remembered instead of
grepping for the subject.

- The class-level summary of the very file round 3 edited still said MySQL's
  "ci collation OVER-matches instead", contradicting the method docstring
  forty lines below it. Reworded.
- The PR body still carried the retracted over-match story -- and round 3's
  commit message claimed it had been corrected. It had not. Now corrected,
  with the measurement table and the retraction stated openly.

This time the sweep was `grep -i over-match` across every file the branch
touches; the remaining hits are the SQLite-fold invariant and #578 history,
which are correct and stay.

Also softened two overclaims the reviewer flagged. Round 3 deleted the
predecessor's "configuration-incidental, not designed" hedge and replaced it
with a firmer statement than the evidence supports: that MySQL cannot
over-match is contingent on MySqlConnector fixing the connection collation to
utf8mb4_bin, not a property of MySQL. A driver, protocol or prepared-statement
change could restore it. The record and the fixture docstring now say
"driver-contingent, not a law"; the code is safe either way because the ordinal
filter stays regardless.

Filed #689 for the source of the copied falsehood -- LibraryFolderDedupeMigrationTests
on main carries the same "CI sets ETV_REQUIRE_MYSQL_TESTS=1" sentence.

Decisions-Edit: yes
2026-07-27 22:09:26 +02:00
timothy dda98efcc4 fix(668): review round 3 -- MySQL does NOT over-match; correct the claim everywhere
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 13s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
PR Gates / Docs update reminder (pull_request) Successful in 17s
review-verdict/h10 Awaiting review verdict for dda98ef
PR Gates / decisions lifecycle (pull_request) Successful in 25s
Review verdict / Set review-verdict status (pull_request) Successful in 13s
PR Gates / Script tests (pytest) (pull_request) Successful in 55s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m32s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m18s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m5s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Review BLOCKED on a false CI claim I copied from the sibling fixture ("CI sets
ETV_REQUIRE_MYSQL_TESTS=1"). Nothing sets it; the doc now says plainly that CI
does not arm this lane and points at ersatztv#627. That was the blocker.

Chasing the reviewer's second finding then overturned something bigger. It
predicted that seeding an unaccented "Edith" would make the in-memory ordinal
filter load-bearing on MySQL, since utf8mb4_0900_ai_ci treats é as e. Mutation
test says otherwise: with the filter deleted the MySQL test stays GREEN.

Measured against a live 8.4 to find out why:

    LOWER(Name) LIKE 'é%'          (literal)          -> Édith AND Edith
    LOWER(Name) LIKE @v            (ai_ci variable)   -> Édith AND Edith
    LOWER(Name) LIKE @v COLLATE _bin                  -> Édith only
    the EF query, executed                            -> Édith only

The driver binds the pattern with a BINARY collation, so the executed
comparison is accent-SENSITIVE and MySQL does not over-match at all. MySQL's
correctness rests on its Unicode-aware LOWER(), not on the collation.

My earlier probe used a LITERAL pattern -- a different query from the one the
code runs -- and I wrote its result into the handler comment, the decision
record and the PR body. All three now say what actually happens, and the record
carries the lesson: measure the query the CODE runs, not one you type.

The "Edith" row stays as a near-miss control, with a docstring that says what it
does and does not prove rather than the over-match story it was added for.

Also moved EnsureCreatedAsync out of [SetUp]: NUnit skips [TearDown] when
[SetUp] throws, so a mid-create failure would strand the database.

Decisions-Edit: yes
2026-07-27 21:52:16 +02:00
timothy 1f6802bb62 test(668): execute the accented-value claim on a REAL MySQL server
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / Docs update reminder (pull_request) Successful in 16s
review-verdict/h10 Awaiting review verdict for 1f6802b
PR Gates / decisions lifecycle (pull_request) Successful in 27s
PR Gates / Script tests (pytest) (pull_request) Successful in 45s
Review verdict / Set review-verdict status (pull_request) Successful in 17s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m28s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m30s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m43s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 15m48s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m23s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The PR's central claim is "reachable on BOTH providers", and on MySQL that
rests on the server's collation rather than on any code this repo owns --
exactly the kind of assumption worth executing rather than asserting.

Adds SearchFieldValuesProviderTests, parameterized over Sqlite and MySql on
the LibraryFolderDedupeMigrationTests contract: opt-in via
ETV_TEST_MYSQL_CONNECTION, a VISIBLE skip without it, and a hard failure
instead of a skip when ETV_REQUIRE_MYSQL_TESTS is set, so the lane cannot
degrade into "connected to nothing and passed". Fresh never-reused database
per test, dropped with its pool cleared.

Verified for real, not just written: run against a live mysql:8.4 it reports
4 passed / 0 skipped and the stored 'Édith' is returned for both q=é and q=É.
Without the connection string it skips (2 skipped); with REQUIRE set and no
connection it FAILS. All three paths exercised.

The MySql half wires RegisterUnicodeCaseFunctions to an explicit no-op, so the
test proves MySQL reaches the value through its own Unicode-aware LOWER() and
not through SQLite's custom fold.

Known and deliberate: CI does NOT arm this lane, so it will skip there. Per the
note in docker-build.yml, running MySQL fixtures against the live service was
implemented and removed as non-deterministic (ersatztv#627) on the grounds that
an intermittently-red gate is worse than none. Re-arming it is that issue's job,
not this PR's -- so this fixture is opt-in exactly like its sibling.
2026-07-27 21:31:36 +02:00
timothy 7ca058f83b fix(668): review round 2 -- remove a second false comment, harden a vacuous assertion
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 19s
PR Gates / Docs update reminder (pull_request) Successful in 20s
PR Gates / decisions lifecycle (pull_request) Successful in 26s
PR Gates / Script tests (pytest) (pull_request) Successful in 46s
Review verdict / Set review-verdict status (pull_request) Successful in 1m6s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 21s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 19s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m32s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 15m45s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m27s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ 7ca058f (base: main)
Re-review of the round-1 fix commit returned MERGEABLE with three findings,
all about claims rather than behaviour. All three applied.

1. A stale FALSE parenthetical survived round 1. The docstring on
   Unicode_Fold_Agrees_With_The_Ordinal_Filter still claimed it catches "one
   that stops filtering the extras out". It does not. Mutation-verified:
   delete the Where in FilterSortTake and all EIGHT cases stay green, because
   each is either a positive SQL alone returns or an ASCII-query negative SQL
   alone rejects. The same mutation turns the new over-match test RED, so the
   pair does cover both directions -- but only the corrected wording says so.
   This is the same species of error round 1 fixed, one paragraph above it;
   swept by subject this time.

2. Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter asserts an EMPTY
   result, so it passes vacuously if the fold never runs. Its premises are now
   asserted explicitly (the query is non-ASCII, and ToUpperInvariant maps ſ to
   S), so a fold that quietly stopped mapping ſ would fail rather than go green
   for the wrong reason.

3. The comment on IsSqlite overstated its enforcement. ProviderStaticsWiringTests
   parses the composition roots for ASSIGNMENTS only; nothing mechanically stops
   a read of TvContext.IsSqlite here. The real reason stands -- such a read would
   falsify that test's prose exemption while the test stayed green -- so the
   comment now says that instead of implying a guard that does not exist.

Decisions-Edit: yes
2026-07-27 21:02:24 +02:00
timothy ce215be590 chore(668): arm the Decisions-Edit trailer, which the earlier commits voided
The two preceding commits both END with:

    Refs #668
    Decisions-Edit: yes

`Refs #668` has no colon, so git does not recognise it as a trailer -- and a
single non-trailer line in the final paragraph voids the WHOLE block, taking
the valid `Decisions-Edit: yes` with it. Confirmed with interpret-trailers:
the pair parses to nothing, while `Decisions-Edit: yes` alone (or `Refs: #668`
with a colon) parses fine. `%(trailers:key=Decisions-Edit,valueonly,unfold)`
-- exactly what scripts/decisions_validate.py reads -- returned empty, so the
`decisions lifecycle` job failed for a real reason, not the known flake.

The branch is already pushed, so amending is out (process.pr-routine-sequence).
The guard accepts the trailer on ANY non-merge commit in the merge-base range,
so this empty commit carries a well-formed one. It is deliberately empty rather
than bundled onto an invented change.

Decisions-Edit: yes
2026-07-27 20:55:37 +02:00
timothy ac67c9ee74 fix(668): review round 1 -- make two guards actually guard
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 17s
review-verdict/h10 Awaiting review verdict for ac67c9e
PR Gates / decisions lifecycle (pull_request) Failing after 30s
PR Gates / Script tests (pytest) (pull_request) Successful in 43s
Review verdict / Set review-verdict status (pull_request) Successful in 43s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m31s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m32s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m57s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 15m44s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m38s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Independent review found both new guard tests weaker than they read.

1. A false docstring. It claimed the SQL stage "genuinely returns 'ſweet'
   for q='S'". It does not: 'S' is ASCII, so ContainsNonAscii is false and
   the fold branch is SKIPPED. Those three negative cases exercise the
   ASCII fast path, which is worth pinning but is not what the comment
   said -- and the consequence was that NO test drove a row through the
   fold for the ordinal filter to discard, i.e. the harmless over-match
   direction the whole design rests on was untested. Comment corrected and
   Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter added
   (stored "Sword", q="ſ" -> fold runs, SQL pattern S%, SQLite returns the
   row, filter drops it, response empty).

2. Unicode_Fold_Escapes_Like_Wildcards could not fail if the %/_ escaping
   it names were deleted -- the in-memory filter masks the over-match, so
   the counts stay right. The escaping's real role is preventing LIMIT
   crowding, so Unicode_Fold_Escaping_Prevents_Limit_Crowding pins that
   instead. Verified by mutation: with the %/_ replaces removed the new
   test fails while the original two still pass.

Also: the crowding residual in the decision record was attributed to MySQL
alone; the SQLite fold shares it in principle, so "no accepted loss" is
narrowed to mean no unreachable VALUE rather than a guaranteed count. And
a comment says why the provider check is derived per-context instead of
reading TvContext.IsSqlite (that static is scoped host-only by
ProviderStaticsWiringTests, and reading it here would falsify the
exemption).

Refs #668
Decisions-Edit: yes
2026-07-27 20:50:48 +02:00
timothy 05542946ad fix(668): reach accented facet values via a registered Unicode fold on SQLite
SQLite's LOWER() folds ASCII only -- lower('Édith') is 'Édith' unchanged --
so the EF-sourced facet fields UNDER-matched any stored value whose prefix
carries an uppercase non-ASCII character. An under-match is unrecoverable:
no later stage can reintroduce a row SQL never returned.

Adds a SECOND, ADDITIVE query taken only when the provider is SQLite and q
contains a non-ASCII character: raw Dapper SQL folding through etv_upper(),
a SqliteConnection.CreateFunction scalar implementing ToUpperInvariant.
Every other case -- all-ASCII q, and MySQL for all q -- runs the existing
EF query byte-identically.

MySQL needed no change and gets none: verified on MySQL 8.4 that its LOWER()
is Unicode-aware and its ci collation makes the predicate OVER-match, which
the existing ordinal filter already discards.

The fold is ToUpperInvariant because OrdinalIgnoreCase equality is a strict
SUBSET of invariant-uppercase equality, so the SQL stage yields a superset of
the final filter's matches and can never under-match. Note OrdinalIgnoreCase
is NOT "invariant-upper then ordinal": ToUpperInvariant('ſ') is 'S', yet
"ſweet".StartsWith("S", OrdinalIgnoreCase) is false. Tests pin that.

No migration, no model change; both provider snapshots are untouched.

Refs #668
Decisions-Edit: yes
2026-07-27 20:36:28 +02:00
timothy 61aa8a902a test(668): red-first pin for accented values on EF-sourced facet fields
Stored 'Édith' is unreachable from q=é and q=É on SQLite, because SQL
LOWER() folds ASCII only. The stored-LOWERCASE pair is pinned alongside
it and passes today, so the fix must supplement that path, not replace it.

Red: both uppercase cases return []. Refs #668
2026-07-27 20:08:39 +02:00
timothy aa1f504e02 Merge pull request 'fix(684): key the pageSize guard registry on identity, not source position' (#686) from fix/pagesize-guard-line-churn into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 35s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 34s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 35s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 6m6s
2026-07-27 17:39:39 +00:00
timothyandClaude Opus 5 689451161e fix(684): review round 2 -- drop a false exhaustiveness claim I introduced
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / Docs update reminder (pull_request) Successful in 16s
Review verdict / Set review-verdict status (pull_request) Successful in 7s
PR Gates / decisions lifecycle (pull_request) Successful in 20s
PR Gates / Script tests (pytest) (pull_request) Successful in 51s
review-verdict/h10 Review-verdict: MERGEABLE @ 6894511 (base: main)
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m33s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 27s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 18s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m38s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The L1 fix (name every class-b shape) collided with the M1 fix (move
SmartCollectionDialog into class-b) in the same commit: the comment claimed
"exactly three shapes, which is the whole list" while the registry 80 lines
below already held four. That is the same false-exhaustiveness defect this PR
exists to remove -- a reader adding a fifth class-b site would conclude theirs
cannot be class-b despite rendering a real totalCount hint, and reach for
search-bounded or deviation.

The operative rule is now stated first and the shape list is explicitly
illustrative: a site qualifies by RENDERING a totalCount-backed hint, not by
resembling a listed shape.

Also, both non-blocking review items:
- the deviation prose said the tracking issue goes "in the note", while
  enforcement is on the structural `issue` field -- it now points at the
  mechanism that actually binds, and says why the note scrape was rejected;
- the UNREGISTERED report prints every position sharing an identity, so it now
  says "identity seen at:" rather than implying all of them are unregistered. A
  positionless key cannot know which occurrence is excess; the candidate set is
  the honest answer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 19:17:07 +02:00
timothyandClaude Opus 5 fc8353c75c fix(684): key the pageSize guard registry on identity, not source position
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 21s
review-verdict/h10 Awaiting review verdict for fc8353c
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 25s
PR Gates / decisions lifecycle (pull_request) Successful in 34s
Review verdict / Set review-verdict status (pull_request) Successful in 31s
PR Gates / Script tests (pytest) (pull_request) Successful in 44s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m42s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m38s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m16s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m38s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The #650 guard pinned every pageSize call site by absolute line:column, making
the registry a function of every OTHER file in the repo.

The guard was BORN RED. #651 moved AutoTuneScreen.tsx up ten lines and
FillerPresetsScreen.tsx down seventy-two, and merged to main BEFORE the guard's
own PR (#675) did -- so the registry, authored against a pre-#651 base, was
stale the instant it landed. Its own merge run was CANCELLED, so nothing
reported it; the red first surfaced on the next push (#676's merge, which
touches no web/src file and is not the cause). One ordering accident, not a
recurring pattern -- but the exposure is general, because every PR is green
against its own base and the breakage exists only in the merge result.

Identity is now (file, kind, value). New/removed/changed sites all still fail.
The MULTISET comparison is preserved, so a shared identity must be discovered
exactly as many times as it is registered. The scanner's positional
pageSizeSiteId is untouched: pageSizeScan.test.ts asserts real AST positions
against fixed fixtures, the opposite case, with no churn to remove.

The one case this costs is stated rather than implied: a same-identity
substitution within one file (delete a registered site, add a different
unreviewed one with the same kind and value token) now passes. Narrow, and
caught only incidentally before. Named in the guard and the record because
"costs no coverage" is a claim that outlives whoever made it.

Failure reports still print the discovered line:column -- identity and
diagnostics need not share a format, and a bare id was useless in a file
holding two such sites.

Registry reconciled with #651: Playlists/RerunCollections lost their bounded
windows to the shared searchLibraryPickerOptions, now registered in its place.
Adds a 'search-bounded' class for that shape.

Adds a 'deviation' class rather than laundering a live defect into a
compliant-looking label. Reconciling the registry surfaced a §3b violation
(#685, filed): CollectionsScreen's AddItemsDialog degrades to an unfiltered
whole-type window on an empty query and surfaces nothing. Both existing labels
would have been false, and either would have made the guard vouch for behaviour
that does not exist. Deviation entries must name a tracking issue, enforced by
a structural field -- a #\d+ scrape of the note passed with the reference
deleted, because notes legitimately cite historical issues.

Corrects SmartCollectionDialog to class-b: it does render a totalCount badge,
which is class-b's defining evidence.

fixes #684

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 19:08:26 +02:00
timothy ac0f65c743 Merge pull request 'docs(649): narrow the base-ref headline to what the checkout actually binds' (#683) from docs/649-narrow-base-ref-headline into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 16s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 1m4s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 9s
2026-07-27 16:33:23 +00:00
timothy c794a48462 docs(649): narrow the base-ref headline to what the checkout actually binds
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Successful in 24s
review-verdict/h10 Exempt: docs-only change (no code, no protected path)
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 29s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 30s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 14s
Review verdict / Set review-verdict status (pull_request) Successful in 17s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 13s
PR Gates / Script tests (pytest) (pull_request) Successful in 52s
The record's bold sentence said the base-ref checkout means the workflow "cannot be
rewritten by that same PR to weaken its own judgment". That is true of the SCRIPTS the job
executes and false of the workflow itself: Gitea resolves a `pull_request` workflow
definition from the PR's own head, so a PR editing `review-verdict.yml` runs its rewritten
copy — which can delete the checkout outright, or just post `review-verdict/h10=success`
and stop. Branch protection requires the context, not an author, and carries
`required_approvals: 0` (#672).

The scoping already existed further down, under "What is deliberately NOT claimed". That
is not good enough for this particular sentence: it is bold, it is the paragraph a reader
resolving this record from the catalog lands on, and someone who stops there leaves with
the opposite of the truth. A caveat only works if it is reached.

So the headline now says what the checkout binds (the scripts, from the already-reviewed
base) and a following paragraph states the head-resolution hole directly, with the
superseded claim quoted so a reader who remembers it can see it was retracted rather than
wonder whether two records disagree. The later paragraph loses its duplicated opener and
points at it instead.

Docs only; no behaviour change. Same failure class the rest of #649 kept turning up — a
claim stronger than the code — reached this time through prose rather than a test.

Refs #649, #672

Decisions-Edit: yes
2026-07-27 18:30:44 +02:00
timothy aeff810cad Merge pull request 'test(649): cover the review-verdict status read and the bot-path guards' (#673) from test/649-workflow-body-coverage into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Failing after 1m41s
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Has been cancelled
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Has been cancelled
2026-07-27 16:22:47 +00:00
timothy cb7da865b6 Merge pull request 'docs: permit subagents explicitly, and make claiming an issue a check rather than a label' (#682) from docs/claim-protocol-and-subagents into main
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 29s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 43s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 11s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
2026-07-27 16:04:13 +00:00
timothy 1d76a088c6 Merge pull request 'feat(578): artist typeahead covers music-video and song credits; album_artist stops 404ing' (#676) from feat/578-artist-typeahead-source into main
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m58s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 19m9s
Build ErsatzTV Image / Build & test (.NET) (push) Failing after 3m39s
Build ErsatzTV Image / Build & push image (amd64) (push) Has been skipped
2026-07-27 06:09:30 +00:00
timothy d751f5e01d Merge pull request 'fix(650): two at-cap list truncations, and a completeness guard that keys on the defect' (#675) from fix/650-at-cap-truncation into main
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Has been cancelled
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been cancelled
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been cancelled
Build ErsatzTV Image / Build & test (.NET) (push) Has been cancelled
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Has been cancelled
2026-07-27 06:08:13 +00:00
timothy 400e30a278 docs: record the Decisions-Edit trailer for the parallel-session-claim rationale
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 24s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / Docs update reminder (pull_request) Successful in 15s
PR Gates / decisions lifecycle (pull_request) Successful in 16s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18s
review-verdict/h10 Exempt: docs-only change (no code, no protected path)
Review verdict / Set review-verdict status (pull_request) Successful in 9s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
PR Gates / Script tests (pytest) (pull_request) Successful in 39s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 19s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The previous commit rewrote the rationale prose of process.parallel-session-claim (adding the
ersatztv#649 double-implementation incident) without this trailer, and CI's decisions-lifecycle
gate correctly rejected it.

Worth recording WHY it passed locally and failed in CI: I ran decisions_validate.py BEFORE
`git commit` in the same command chain, so it inspected the working tree. The trailer check reads
COMMITTED history, so the one rule that can only fail after committing was the one I validated
before committing. Run the decisions validator after the commit, not before it.

Decisions-Edit: yes
2026-07-27 08:03:18 +02:00
timothy 7ed0a59c56 test(649): cold-review fixes — the never-overwrite test skipped the case its docstring called sharpest
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
PR Gates / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 23s
Review verdict / Set review-verdict status (pull_request) Successful in 11s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 11s
PR Gates / Script tests (pytest) (pull_request) Successful in 52s
review-verdict/h10 Review-verdict: MERGEABLE @ 7ed0a59 (base: main)
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m13s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 23m21s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m34s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Four gaps, all found by mutation rather than reading.

The never-overwrite test used only a NON-EXEMPT file list, so "an exemption posted over a human
BLOCKED verdict" — the scenario its own docstring named as the sharp one — was asserted nowhere.
Moving the short-circuit to after classification, gated on non-exempt, survived the whole suite
while turning a human rejection into a green required status for both a docs-only PR and a bot PR.
Now parametrised over non-exempt, docs-only-exempt and bot-exempt file lists.

The structural emptiness pin is REPLACED by a real jq-1.6 behavioural test. Its stated
justification — "no behavioural test can catch this on a dev machine" — was simply false: this
file already imports _JQ16_SHIM for pr-changed-files.sh, so the runner's quirk is reproducible
here. The structural version was also weaker than it looked, stripping only FULL-LINE comments, so
leaving the literal as a trailing comment on the surviving guard satisfied it while the real check
was gone. The behavioural test catches that mutant and needs no comment-stripping.

The status-read stub now returns DECOY contexts either side of the verdict row, so dropping
`select(.context == $c)` is caught. First attempt gave the decoys `status: success`, which triggers
the same short-circuit as a real verdict — the mutation still produced an identical outcome and
survived. `pending` decoys make mis-selection observable.

DOCS_ONLY's `^` anchor is now covered alongside its `$`: losing it exempts ErsatzTV/docs/Evil.cs,
a C# file, and is fail-OPEN.

Two remaining survivors are documented in-file as behaviourally equivalent, not gaps: `first` ->
`last` (the combined endpoint returns one row per context by contract, so a two-row fixture would
test a fiction), and the garbage-response test defending the type guard only by redundancy.
2026-07-27 08:01:36 +02:00
timothy b83e965994 docs: make subagent use explicit, and turn "claim an issue" into a check rather than a label
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 28s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 20s
PR Gates / Docs update reminder (pull_request) Successful in 24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 29s
PR Gates / decisions lifecycle (pull_request) Failing after 24s
review-verdict/h10 Exempt: docs-only change (no code, no protected path)
Review verdict / Set review-verdict status (pull_request) Successful in 7s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 26s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 25s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
PR Gates / Script tests (pytest) (pull_request) Successful in 54s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 15s
Two rules that were implied but not enforceable, both demonstrated by ersatztv#649 being
implemented TWICE in parallel to completion on the same day.

Subagents. The kickoff's HARD CONSTRAINTS already require parallel disjoint slices, mandatory
independent review from a cold brief, and a named model/effort per dispatch — none of which is
satisfiable without delegation. But nothing said so outright, and a generic client preamble of the
form "do not use the Agent tool unless the user requested it" reads as a prohibition. Now stated
plainly in both CLAUDE.md (always loaded) and the kickoff (pasted per session), with what to
delegate and what to keep inline.

Claiming. `in-progress` prevents duplicate PICKUP, not duplicate WORK — the record already said so,
but step 3 told you to apply the label and nothing else. It now requires four checks first: an open
PR whose body says `fixes #N`, a remote branch naming the number, a claiming comment predating the
label (exactly what select-queue.sh's CLAIM? flag raises and deliberately leaves unresolved), and a
fresh git fetch. Each fails differently; all four are cheap.

Staleness. The second half of the #649 collision was reading origin/main once, at branch time, and
not again across hours and four review rounds. A branch on a stale base computes its diff against
that base, so `git diff origin/main` shows other sessions' merged work as DELETIONS and pushing it
reverts them. Re-fetch before every push, rebase when it moved.

process.parallel-session-claim carries the incident, including what worked: the merged
implementation was better in one respect and the discarded branch's test coverage was salvageable,
so diff the two before discarding yours.
2026-07-27 07:44:25 +02:00
timothy 2a2dcacd58 test(649): cover the review-verdict status read, and the guards that only fire on the bot path
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 12s
PR Gates / decisions lifecycle (pull_request) Successful in 25s
Review verdict / Set review-verdict status (pull_request) Successful in 24s
PR Gates / Script tests (pytest) (pull_request) Successful in 50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m35s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m24s
review-verdict/h10 Review-verdict: MERGEABLE @ 2a2dcac (base: main)
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m26s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 23m38s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Additive tests for properties #666 shipped correctly but left unguarded. No behaviour change.

The stub's status read was hardcoded to "no verdict yet", so two whole branches of the classify
step were unreachable from any test. Four mutations survived the full suite because of it —
including re-introducing the literal ersatztv#647 fail-open, and overwriting an existing human
verdict. The stub now models a transport error, a garbage body, and an existing verdict.

`test_an_EMPTY_enumeration_is_not_exempt_even_for_a_BOT` needs the bot author to test anything:
with a non-bot, the blank line an empty list produces already fails DOCS_ONLY, so the
`count -eq 0` guard never decides the outcome. On the bot path it is the ONLY thing between an
enumeration that read nothing and an unattended success. Verified by mutation — `grep -c .` ->
`grep -c ''` grants a bot PR success while every other test stays green. Same short-circuit
shape as the PROTECTED/DOCS_ONLY disjointness this file already documents.

Two anchors were also unguarded: `grep -qxF` (author `ova` is a substring of `renovate`) and
DOCS_ONLY's `$` (`evil.mdx` reads as docs-only).

Five of the six mutations are caught behaviourally. The sixth — dropping the shell emptiness
check — cannot be caught locally: `jq -e` over empty input exits 4 on jq 1.8 so the guard still
fires on a dev Mac, and 0 on the runner's 1.6 where it is the actual bug. A structural assertion
closes that gap, with comments stripped first, since a raw substring search is satisfiable by
moving the guard into a comment while deleting the real one — verified.

refs #649, #672
2026-07-27 07:40:00 +02:00
timothy 31f2a927a2 Merge pull request 'feat(651): library-browse pickers resolve by search, not a 100-row window' (#678) from feat/651-searchable-pickers into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 16s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 32s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 32s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 12m22s
Build CI Toolchain Image / Build & push CI image (push) Successful in 1m44s
Dependency vulnerability scan / NuGet vulnerable packages (push) Successful in 47s
2026-07-27 03:47:06 +00:00
timothyandClaude Opus 5 66c8500e94 fix(651): pre-merge asks — an empty filter loop asserts nothing; fix "1 item need"
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 17s
PR Gates / Docs update reminder (pull_request) Successful in 23s
PR Gates / decisions lifecycle (pull_request) Successful in 25s
Review verdict / Set review-verdict status (pull_request) Successful in 7s
PR Gates / Script tests (pytest) (pull_request) Successful in 46s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 18s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 15s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m51s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m40s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m57s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ 66c8500 (base: main)
FIX 1: `PlaylistsScreen.test.tsx`'s group-id test filtered POSTs and asserted inside a
`for` loop over the result. On the fixed build that array is empty — the unmatched value
leaves the select at '', so Create is disabled and jsdom won't dispatch its onClick — so
ZERO assertions ran and the title claimed more than the body proved. A future change that
re-enabled Create and POSTed `playlistGroupId: null` would still have passed. Added the
unconditional `expect(posts).toHaveLength(0)` before the loop.

Worth noting this is the ninth instance of the shape on this branch, and the sibling
strengthening in the *same commit* got it right (`RerunCollectionsScreen.test.tsx` uses an
unconditional `toHaveLength(0)`) — so the lesson didn't generalize even one file over. The
rule is: an assertion inside a loop over a filtered collection proves nothing until the
collection's length is asserted.

FIX 2: "1 item need a selection" — the noun was pluralized, the verb wasn't, and singular is
the common case. My test used `/need a selection/i`, which matches both the right and wrong
grammar, so nothing could catch it; it now asserts the exact string '1 item needs a
selection'.

FIX 3: two comments about clicking an already-disabled button read as contradictory policy.
They're not — on the parent the Playlists button was ENABLED, so there the click genuinely
discriminates, while the rerun button was disabled on both sides, making it a restatement of
`toBeDisabled()`. Both comments now say which case they are and why.

Added to #677: the row label falls back to "(no X selected)" on empty `selectedName`
regardless of `selectedId` (the mirror image of the bug fixed here), and an all-unbindable
group list disables Create with no reason shown.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 05:19:49 +02:00
timothyandClaude Opus 5 27867e03cf fix(651): make the stated invariant true on Playlists; pin the predicate's endpoints
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / Docs update reminder (pull_request) Successful in 18s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 22s
review-verdict/h10 Awaiting review verdict for 27867e0
PR Gates / decisions lifecycle (pull_request) Successful in 23s
Review verdict / Set review-verdict status (pull_request) Successful in 24s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m24s
PR Gates / Script tests (pytest) (pull_request) Failing after 13m19s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 17m7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m30s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
The review's headline finding was in my prose, not my code: spa-conventions and the round-8
commit both claimed an unbindable id "surfaces as 'no selection' with Save disabled" and that
regressions "assert zero writes are reachable". True on RerunCollections and FillerPresets.
False on PlaylistsScreen in all three respects — `draftFromItem` nulled the id but KEPT
`selectedName`, so the row read "Cool Movie" over a null draft; Save had no selection check;
and clicking it did issue the PUT with `mediaItemId: null`. Only the server's
`ReplacePlaylistItemsHandler` 422 stood there, and the DB would have persisted it
(`PlaylistItemConfiguration` marks all four FKs `IsRequired(false)`).

Rather than weaken the claim, made it true: a dropped id now clears its label, and Save is
gated on every item having a selection, with a visible count as the reason.

`playlistGroupId` was the same class on the same screen — seeded from the wire into
`AddPlaylistDialog`, re-parsed with a bare `Number()`, and POSTed as an entity reference — so
"every path by which an id from the wire becomes editor state" was not literally true. Now
filtered from the group options and normalized on submit.

Added `selectionId.test.ts`. The predicate had become the single point of failure for eleven
call sites across three screens while being exercised only indirectly; nothing pinned the
inclusive endpoints, so a `>` for `>=` slip passed the entire suite. Verified by mutating
each comparison. Also documented why `0` and negatives are accepted — the contract is
bindability, not existence — because every other id check in this repo uses `id > 0` and the
next reader would otherwise "fix" the inconsistency.

Two of my assertions were vacuous, the eighth of that shape on this branch: one clicked a
button it had just asserted disabled (a restatement of `toBeDisabled()`), and one asserted a
POST count on a path that never attempted a save. The first is deleted; the second now
actually attempts the write, which makes it fail against the unguarded parent.

Corrected claim: all five round-8 regressions do fail against their parent, but on their
load-bearing assertions (`getByText('A selection is required')`,
`queryByText('Bogus Collection')`) — not on the write-count ones, which were passengers.

Follow-up filed as #677 (ScheduleItemInspector's unguarded ingresses; list-backed pickers
dropping malformed options silently).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 39c4e8df0a fix(651): review round 8 — put the selection-id predicate at the boundary, not the site
Round 7 added the int32 check inside `isSearchPickerOption` — the place the defect was found
— which left every other door into editor state open. A malformed successful response
carrying `1.5` or `2147483648` still entered `draft` through list-backed options and through
the selection restored from the detail read, keeping Save enabled and sending a value the API
cannot bind, while the identical value arriving via SearchPicker was correctly rejected.

The predicate now lives once, in `web/src/api/selectionId.ts`, and sits on every path by
which an id from the wire becomes editor state. The class crosses all three screens, not just
the one the finding named, so all three are covered:
- RerunCollectionsScreen: `toPickerOptions` (3 list branches) + `draftFromRerun`
- PlaylistsScreen: `toPickerOptions` (3 list branches) + `draftFromItem` (4 id fields)
- FillerPresetsScreen: `draftFromPreset` (5 id fields) + the collection-family browse options
- pickers.tsx: `isSearchPickerOption` now delegates rather than carrying its own copy

An unbindable id is treated as ABSENT, never coerced — rounding 1.5 to 1 would submit a
DIFFERENT record — so it surfaces as "no selection" with Save disabled and a visible reason;
an option that cannot be selected safely is dropped rather than rendered. Five regressions
assert zero writes are reachable via each previously-unguarded path.

Also corrects two of my own test descriptions, per the review: the padded-ETag test is a
regression guard rather than a round-7 defect demonstration (Headers strips outer whitespace
before the app sees it), and the late-settlement test guards the abort/race COMPOSITION —
what it actually fails is an abort-only implementation whose fetch ignores its signal, which
is why its stub ignores `init.signal`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 e605e4006a fix(651): review round 7 — treat "no usable token" as one class, not three values
HIGH: the fail-closed gate rejected `null` but not the adjacent values. `Headers.get('ETag')`
returns `''` for an empty or whitespace-only header, which PASSED the gate and produced an
editable draft; `updateRerunCollection`'s `ifMatch ? … : undefined` then dropped the empty
string as falsy and sent no `If-Match`, silently overwriting a collaborator — the exact class
the gate exists to make unreachable, reached through the value next door. Absent, empty and
whitespace are now one case ("no usable concurrency token"), normalized by a single
`usableEtag` helper that returns the TRIMMED token or null, so `etagRef` can only ever hold
something that will actually be sent. Tested across four blank shapes asserting zero PUTs are
reachable, plus a padded ETag that must be trimmed and USED rather than dropped.

MEDIUM: the deadline abandoned the wait without cancelling the work, so each Retry stacked
another live connection. It now aborts via an AbortSignal (threaded through
`getRerunCollectionWithMeta`) AND clears its timer on settlement and unmount. Both halves are
kept deliberately: aborting cancels the work, while the rejected race stops the UI waiting
even if the abort never propagates — cancellation and giving-up are not the same guarantee,
which the late-settlement test proves by using a stub that ignores its signal.

MEDIUM: `Number.isFinite` accepted ids the API cannot bind — `1.5` and values outside int32
rendered, committed through `onSelect`, and would fail server-side on `selectedId`. Validated
as an int32 integer.

MEDIUM: a malformed or failed page was reported as "No matches", telling the user the library
is empty when the request actually failed and giving no hint that reopening retries. Failures
now surface as a distinct alert.

MEDIUM: `spa-conventions.md` still mandated the deleted "never let a refresh clear an id it
failed to name" guard and said "the client guard stays" — contradicting the initialize-once
bullet 20 lines below it. Rewritten to state that the guard is gone and must not be rebuilt,
with the reason (it only ever preserved a list-seeded value that is null in production).
Grepping the DELETED TERMS across all docs — the lesson from round 6's stale `rule:` — also
caught two stale `signals:` tokens on the record that the rule fix had missed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 a973fc48e2 fix(651): review round 6 — fail closed on a missing ETag; validate elements, not containers
HIGH: "the draft is only created alongside the ETag" was not the invariant I claimed. The
response can OMIT the header, in which case the draft was still created and the subsequent
PUT carried no `If-Match` — the round-4 force-write hole in a new form. The editor now fails
closed: no concurrency token, no editable draft (error + Retry/Back). Six tests were leaning
on ETag-less detail mocks, which is exactly how this stayed invisible; every single-record
GET mock now goes through a `detailResponse` helper that always sets one, and the absent
case is tested explicitly — asserting zero PUTs are even reachable.

MEDIUM: the detail GET had neither deadline nor recovery affordance, so a never-settling
request left a bare spinner with no way out. It is now raced against a 15s deadline, the
loading view carries a Back control, and the error view offers Retry.

MEDIUM: the malformed-body guard checked the container, not the elements. `[null]` passes
`Array.isArray`, reaches `setResults`, and throws on `option.id` during render; a wrong-typed
`id` would commit an invalid value through `onSelect`. Each element is now validated, and a
malformed payload is treated as a failed attempt so it stays retryable rather than cached.

MEDIUM: the decision record's `rule:` — the authoritative string, copied verbatim into the
catalog that is the documented entry point — still mandated the machinery round 5 deleted:
touched-field hydration, `replaceDraft`, conflict reconciliation. Anyone following it would
have rebuilt the rejected design. Rewritten to the initialize-once policy and the catalog
regenerated; historical prose no longer says `replaceDraft` "is now" separate.

MEDIUM: the replacement Reload test resolved its second GET immediately and returned a
non-null selection, so it observed neither a pending reload nor the dirty-selection discard —
removing `setDraft(null)` could leave it green. It now holds the reload open, asserts the
form is ABSENT while pending, and returns `selectedId: null` to pin the case round 3 showed
could resurrect a dirty id over a collaborator's change.

Checklist item taken from this: when a mechanism is deleted, the decision record's `rule:` is
the single most likely thing left stale, and the one string that propagates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 78cd9e0ebf fix(651): review round 5 — delete the draft-reconciliation layer instead of extending it
Took the coordinator's proposal. Rounds 2-4 built and rebuilt a layer that merged a late
detail response into a draft the user was already editing; it produced a HIGH finding in
three consecutive rounds, three of them cross-user lost updates. Round 5's finding was
unfixable in kind: `identityConflicts` had no immutable baseline, so it could not tell "the
user changed this" from "the server changed this" — giving both a missed conflict (same
type, different id: v2 ETag installed over the user's id, third lost update) and a false one
(local type switch: dialog opens spuriously, and "Keep editing" leaves `etagRef` null so the
next PUT is a silent FORCE-WRITE).

The race is removed rather than refereed. `RerunCollectionEditor` now initializes its draft
EXACTLY ONCE from the detail GET and withholds the form until it lands; conflicts are
detected at save time by the `If-Match` -> 412 -> Reload path that already existed. Deleted:
`touchedRef`, `hydrateDraft`, `hydrateSelection`, `hydrateIdentity`, `identityOf`,
`identityConflicts`, `replaceDraft`, `replacePending` and the hydrate/replace load mode.
Reload simply sets the draft back to null and re-runs the same load, so the form is
unmounted while the replacement is in flight — round 4's MEDIUM-4 becomes structural.

Two facts make this lossless rather than a regression. The list row could never have helped:
`GetPagedRerunCollectionsHandler` applies ZERO `.Include()`s where
`GetRerunCollectionByIdHandler` applies fourteen, and both project through the same mapper,
so the list response is a strict SUBSET of the detail one — the id round 1 preserved from it
is null in production for every row (#671), and existed only in test fixtures. And
FillerPresetsScreen/PlaylistsScreen already worked this way; RerunCollections was the
outlier, which is why nearly every finding in rounds 3-5 traced to it.

The ETag is now written in the same callback that sets the draft, so `draft != null` implies
an ETag and a PUT without `If-Match` is unreachable by construction.

MEDIUM-2: a failed search retried every debounce forever — a fresh `{ok:false}` re-ran the
effect and the success guard declined it. `ok` (is the held answer authoritative) is now
separate from `attemptRef` (have we already tried this exact source+query); only an explicit
reopen/focus/edit re-arms a retry.

MEDIUM-3: a malformed 2xx body resolves as `undefined` (client.ts swallows the SyntaxError),
and `setResults(undefined)` threw on the next render. A non-array is now treated as a failed
attempt, and `search` is raced against a 10s deadline since a caller-supplied promise carries
no abort signal.

Two tests were hiding findings and are fixed: the failed-search test pressed Escape before
the unintended retry could fire, and the round-4 conflict test claimed to select id 9 while
firing the already-selected id 5. Fourteen tests of the deleted machinery are gone with it,
replaced by six asserting the new invariants.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 f601d957a6 fix(651): review round 4 — an id never travels without its namespace
Round 4's through-line: rounds 2-3 made HYDRATION treat {collectionType, selectedId,
selectedName} as one value, but the fix was applied to the structure that was named rather
than to every structure that carries an id. Three more instances of the same class, plus
two independent findings.

HIGH-1 + HIGH-2 (one change, per the structural directive): every result, option and
cached result set now carries its SOURCE, and identity is compared as (type, id).
- `SearchPicker` takes a REQUIRED `source` prop and caches results against (source, query),
  not query text. Keying on text let the round-3 re-query guard SUPPRESS the new source's
  request and leave the previous namespace's hit clickable under the new label — a
  Collection id stored as a SmartCollection id. Results from another source are now hidden
  outright rather than dimmed: they are not stale, they are wrong. Required rather than
  defaulted, because a default would silently opt every caller out.
- `pickerFor` tags list-backed options with the type they were loaded for, on both
  RerunCollectionsScreen and PlaylistsScreen, so the previous type's rows stop being
  selectable the instant the active type changes rather than lingering through the
  replacement load.

HIGH-3: a touched identity contradicting the server's type is a CONFLICT. Pinning the
user's edit was right; adopting the response's newest ETag alongside it authorized a Save
that silently overwrote the collaborator's type change with no 412. The conflict dialog is
raised and the stale ETag kept, so even a forced Save 412s. With round 3's Reload defect
this is the second cross-user lost update, so it is recorded as a category: never install a
save-authorizing ETag over a local edit the server contradicts.

MEDIUM-4: the editor is inert while a Reload is pending. The dialog closes immediately, so
an edit typed before the replacement landed was silently erased along with the touched set
that protected it.

MEDIUM-5: cached search provenance records `ok`, so a transient 500 is retried instead of
being cached as an authoritative "No matches" that reopening can never clear.

Also: `npx tsc --noEmit` typechecks NOTHING in web/ — the root tsconfig is solution-style
("files": [] + references), so it resolves to zero inputs and exits 0. The real gate, and
what CI runs, is `npm run typecheck` (tsc -b). Verified by planting a deliberate type error:
--noEmit stayed green, -b caught it. Running the real gate surfaced four genuine errors in
tests written earlier this branch (a missing required prop and three `never has no call
signatures` from closure-assigned mock variables), fixed with the repo's existing
holder-object pattern rather than casts. Recorded in spa-conventions so the next session
doesn't repeat it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 5b0ba08aab fix(651): review round 3 — cover the class, not the named instance
Round 3 found four defects that are all one mistake: each earlier fix enumerated one
instance of a class instead of covering the class. Fixed by asking what else is in each
class rather than patching the named case.

CRITICAL — cross-user lost update. Conflict "Reload" ran through the refresh policy with a
touched-set reset. Because a reloaded RemoteStream reports `selectedId: null` under the
#671 server gap, the keep-ours-when-the-id-is-missing fallback restored the user's DIRTY
selection, the fresh ETag was installed, and the next Save silently overwrote the
collaborator's change — with the very edits the user had explicitly asked to discard.
"Keep ours when the server omits the id" is a REFRESH policy; a reload is a REPLACE.
`replaceDraft` is now a separate function and the mode travels with the load, so the two
cannot be confused at the call site.

HIGH — `collectionType` and the selection still hydrated apart. `collectionType` says which
table an id indexes, so it is part of the same indivisible value as the id and its name;
splitting it is the identical bug to splitting id from name. A record retyped server-side
mid-load hydrated the new type while retaining the old id, displaying and saving a
Collection id as a RemoteStream id. All three fields now resolve as one `Identity` unit:
either half touched pins the whole thing, a differing type takes the response's unit whole
(null selection included), and only once both sides agree on the type does the id/name rule
apply.

HIGH — stale results were still committable by pointer. Enter was gated and `onClick` was
not: the same defect in another modality. The guard moved into the single `choose()` sink
so every commit path is covered, including any added later, and a superseded list is now
genuinely inert (`aria-disabled` + dimmed) rather than looking normal and silently
no-opping.

MEDIUM — reopening after Escape re-queried an already-current result set; the duplicate
response reset the cursor the user had since moved, leaving Enter doing nothing. The effect
now skips the search when the cached results match the trimmed query, and reopening places
the cursor per the ARIA APG instead of swallowing the keypress.

Also corrects an overstated justification in FillerPresetsScreen: the render-time id check
was dropped because there is no reachable path TODAY, not because "every writer sets both"
— the initial load writes the id alone, and a stale resolver can repopulate the label after
a clear. Same enumeration error as the findings above; the comment now says what is
actually true and what to do if a path appears.

Note: an apostrophe I introduced inside the single-quoted `rule:` scalar broke PyYAML while
`decisions_validate.py` (hand parser) stayed green — the same trap class as the unquoted
`#`, caught only by scripts/tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 ba52219a9a fix(651): review round 2 — hydrate untouched fields, never merge a selection field-by-field
Re-review of a04d9f0b came back BLOCKED: the F1 merge fixed a visible data loss and
introduced a silent one. Four blockers, all web-only.

BLOCKER 1: `selectedId` and `selectedName` were coalesced independently, but they are one
value. Against a Song response (id resolves, name does not), a user selecting a different
song while the detail GET was in flight got the NEW name paired with the STORED id — chip
read "New Song", Save wrote 42, no error and no visual cue. Strictly worse than the
original defect, which at least cleared the field visibly. The same merge resurrected a
deliberate clear and reverted a type switch.

Replaced with two rules. The id/name pair resolves atomically (no id in the response ->
keep ours whole; same id -> its name may fill ours in; different id -> take theirs whole).
And hydration loses every race against the user: a `touchedRef`, fed by a single `edit()`
funnel so "touched" cannot drift from "changed", limits the refresh to fields the user has
not edited; an explicit conflict-reload clears it, since discarding local edits is its
entire purpose. Three interleaving tests hold the detail response open, act as the user,
then release it.

BLOCKER 2: Enter could commit a result from the previous query — highlight Alpha for "Al",
retype "Be", press Enter before the debounce. The highlight now drops on input change
rather than when the next response happens to arrive, and every keyboard action is gated
on the results matching what is typed. The stale list stays visible (hiding it flickers on
each keystroke), it just stops being actionable.

BLOCKER 3: Escape closed the popup while focus stayed in the input, where `onFocus` can
never re-arm it — the picker was dead until the user blurred and refocused. Typing and
ArrowDown now both reopen it.

BLOCKER 4: the LCG boundary test recomputed the divisor instead of exercising `lcg`, so
all three tests passed with the old `/ 0xffffffff`. Since the recurrence is a bijection
mod 2^32, the seed whose first step lands on 0xffffffff is solvable exactly (653637408);
the tests now drive the real generator into that state.

Also: the rerun #id-degrade and re-save tests did not await the refresh, so they were
satisfiable from the initial draft; they now await it and re-read live textContent. The
Episode/MusicVideo cases are relabelled as error-path guards — a 500 never reaches the
hydrate. FillerPresets' render-time id comparison is dropped as unreachable (every writer
sets label and id together, and the one async writer refuses a mismatched id); an
unreachable guard is an untested one.

#671 is referenced from spa-conventions §3b as the server-side root cause the client guard
defends against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 e7e425fa25 fix(651): review round 1 — never clear an unnamed id, complete the Lucene escaping, keyboard-operable picker
Cold cross-family review of 57aefcdf. Six findings, all web-only.

F1 (HIGH, data-loss shaped): RerunCollectionsController.ProjectToResponseModel derives
BOTH selectedId and selectedName from the same eager-loaded navigation, and
GetRerunCollectionByIdHandler loads media metadata only for Show/Season/Artist/Movie
while MediaCollections/Mapper maps RemoteStream through `_ => null`. So opening a
RemoteStream rerun collection returned HTTP 200 with a null selection and the edit-load
refresh CLEARED a stored id, leaving Save permanently disabled. The refresh now merges
instead of replacing, so no path can clear an id it merely failed to name; the label
degrades to `#id`. Covered per affected type — RemoteStream, Episode, MusicVideo, Song,
OtherVideo, Image — plus a re-save assertion. The read-model gaps themselves are
server-side and are NOT touched here.

F2: `&` and `|` were missing from the escaped set, so `Rock && Roll` compiled with the
boolean operator live. Pre-existing in Auto-Tune's original helper, but propagated to
three more pickers — and now fixed for Auto-Tune too, since the helper is shared. The
test that claimed to cover "every Lucene special" carried its own hand-copied sample and
could not see its own omissions; it is now driven per-character off an exported
LIBRARY_PICKER_LUCENE_SPECIALS.

F3: a slow edit-load name resolution could relabel a newer selection. The label is now
keyed to the id it was resolved for AND refuses to overwrite a label naming a different
id — keying the render alone stops the mislabelling but discards the correct new label.

F4: searchLibraryPickerOptions clamps pageSize instead of merely defaulting it. A bound a
caller can exceed is not a bound.

F6: replacing a native <select> with an input+listbox dropped keyboard operability. Full
ARIA combobox pattern added — role/aria-expanded/aria-controls/aria-autocomplete,
Arrow/Home/End over aria-activedescendant, Enter to commit, Escape to dismiss, options as
non-tab-stops, cursor reset on each new result set.

F7: both is-mounted tests were unsound. React 19 no longer warns on setState-after-unmount
and an unmounted tree renders nothing either way, so the DOM assertion could not fail; the
hook re-arm test used rerender rather than an effect cleanup. Now: a hook-module mock
proving SearchPicker actually reads the guard and sees false, and a StrictMode
double-invoke for the re-arm. Both verified by removing the mechanism and watching them
fail. Same for the LCG divisor, which now has a direct boundary test.

F5 (FillerPresets collection-family names) is filed as #670, not fixed here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothyandClaude Opus 5 fad6805b91 feat(651): media-library pickers resolve by search instead of a bounded window
The three `getLibraryBrowseItems` pickers (RerunCollectionsScreen, PlaylistsScreen,
FillerPresetsScreen) populated a native <select> from a 100-row window over media-library
tables that can hold tens of thousands of rows. #644 made that truncation visible; it did
not make the picker usable, and paging to completeness would have been worse than the bug
(~200 serial requests, each more expensive than the last).

They now resolve by SEARCH through the shared `SearchPicker` over a new
`searchLibraryPickerOptions` helper: zero requests on mount or on a type switch, at most ONE
bounded request (25 rows) per settled query, nothing below 2 characters. Typed text is
compiled via the now-shared `titleContainsQuery` (`title:*<escaped>*`) rather than forwarded
raw, since the index's default field does not match bare title words. The current selection
renders from the owning record — `selectedName` for rerun collections and playlist items, and
for filler presets (which store only an id) a single by-id detail read — so editing an
existing record can never lose or fail to name its selection.

Class A stays put: bounded-by-construction admin lists still page to completeness via
`loadAllPages`, and the collection-family filler-preset types keep their bounded single page
(their `query` is a SQL LIKE, which a compiled Lucene query would not match). No server-side
cap is raised; this is a web-only change.

Folded in from #578: the rule-builder facet typeahead arms on focus rather than on mount (an
N-rule tree fired N unrequested lookups), both typeaheads pair their `seqRef` guard with a
shared `useIsMountedRef`, and the roundtrip test's LCG divides by 2^32 so `pick()` can no
longer index one past the end.

Decision record `spa.list-completeness-vs-bounded-pickers` is archived as superseded by the
new `spa.library-pickers-resolve-by-search`; spa-conventions §3b rewritten to match.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 04:25:14 +02:00
timothy fc3ede09bc docs(578): the retracted claim survived in 9 places, including the record title and rule
PR Gates / Script tests (pytest) (pull_request) Successful in 51s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m38s
Review verdict / Set review-verdict status (pull_request) Successful in 1m18s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 4m51s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m52s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m10s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 25m44s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ fc3ede0 (base: main)
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 7s
PR Gates / Docs update reminder (pull_request) Successful in 8s
PR Gates / decisions lifecycle (pull_request) Successful in 21s
Comment- and docs-only; verified no non-comment line changed in any .cs.

I reported last round that I had "classified every surviving hit". That was false, and the false
confidence is the expensive part: a confidently-stated "I checked everything" stops anyone else
checking. The retracted wording survived in nine places, two of them the record's title and rule: —
and the catalog copies rule: verbatim, so the generated entry point and the record disagreed
semantically while docs/decisions.md said the correct thing.

Root cause of the miss, because it will recur otherwise: I built the sweep term list from the
DELETED MECHANISM's vocabulary (LIKE, superset, keyspace, anchor, over-match) and never added the
RETRACTED CLAIM's own words. "no predicate", "bound on work", "index entries", "no gap" and
"holds in memory" were never grepped. After a retraction the subject list has to include the words
of the thing being retracted, not just the thing already deleted.

Second, worse: my first attempt at this round's sweep printed nothing for every term and I nearly
read that as "all clear". zsh does not word-split an unquoted $FILES, so grep received one giant
non-existent path — and the `|| echo "(none)"` never fired because the pipeline's exit status was
sed's. Same failure shape as the bug arc itself: a check reporting success while examining nothing.
Re-run with a proper array plus a control term ("SongMetadata" -> 42 hits) so an empty result is
distinguishable from a broken grep.

Fixed all nine, replacing "no predicate" with the seekable-cursor-vs-residual distinction already
written correctly elsewhere:
- handler: the "real bound on work" claim, the short-page rationale
- SearchFieldValuesQueryShapeTests: "ANY predicate" + "reads exactly n index entries", and added what
  the test can and cannot pin (a SQL string, not a plan / visibility work / payload I/O)
- GetSearchFieldValuesHandlerTests: "no gap between what the engine looks at and what it hands back",
  and the current-behaviour comment
- record title, rule:, attempt-5 table row; api-conventions
- regenerated docs/decisions/README.md so catalog and record agree again

Tenth item, the same overclaim one level down and it survived the first retraction: the row bound was
said to cap what the process holds in memory. It does not — payload width is unrestricted and one
JSON array can contain arbitrarily many strings, each of which may enter the in-memory distinct set.
It caps logical rows returned/materialized and round-trip count, nothing about bytes. Added as a
third struck-through bullet next to the other two retractions.
2026-07-27 03:10:36 +02:00
timothy 5f73cd4482 docs(578): scope the bound to LOGICAL ROWS — physical work is not bounded, and I claimed it was
Comment- and docs-only. Verified: the diff for both .cs files contains no non-comment line.

HIGH (claim). "LIMIT n reads exactly n index entries and n rows" is false, and the bounded-physical-
work reading it implied is false with it. Two mechanisms, both retracted explicitly in the record
rather than quietly reworded:

- MySQL purge lag. Deleted clustered-index records survive until purge runs and a range scan still
  traverses them, so returning 2,000 VISIBLE rows can touch far more index records. Deletion history
  therefore STILL affects physical work — precisely what attempt 4's failure was supposed to have
  made irrelevant. Attempt 5 removes the LOGICAL dependence on Id distribution; it does not make
  physical work independent of deletion history.
- Unbounded row width. Artists/AlbumArtists are unrestricted TEXT/longtext and both SQLite and InnoDB
  spill to overflow pages, so a row count implies neither a byte count nor a page-read count.

The guarantee is now stated as exactly what it is: at most 20,000 LOGICAL rows returned/materialized,
and at most 10 round trips (11 for artist). The 392 KiB measurement is labelled one data point on one
library whose credits average ~20 B/row, with an instruction to re-measure rather than extrapolate
for long credit lists or MySQL over a network. Also noted: the query-shape test pins the SQL STRING —
it cannot pin a plan, MVCC visibility work or payload I/O, and on MySQL using the index to satisfy
ORDER BY is an optimizer choice, not a semantic guarantee.

Precision fix, and the reviewer is right that the sloppy version misleads: "any predicate defeats
LIMIT" is wrong, since the query's own cursor is a predicate. The distinction is a SEEKABLE PREDICATE
ON THE ORDERING KEY (positions the scan, never discards a row) versus a RESIDUAL predicate (discards
rows the engine already produced, so LIMIT bounds survivors and says nothing about how many were
produced). Restated in the handler, the record and api-conventions.

MEDIUM. docs/decisions.md still advertised "a superset LIKE pre-filter that may over-match but never
under-match" for list-valued columns. That is the documented entry point for convention lookups, so a
maintainer starting there was told to preserve exactly what round 4 deleted, in direct contradiction
of the linked active record. Fixed.

LOW. Two test comments read as current: "however many non-matching rows" (false past the ceiling —
now bounded and cross-referenced to the test that pins that boundary) and the ampersand case
"widens to the bare anchor" (no prefix predicate exists; kept because it is the input shape that
broke the old scheme, now labelled as such).

Fifth consecutive round of stale text, so this sweep was done by grepping the subject across handler,
tests, record, decisions.md, api-conventions.md and the endpoint description: LIKE, ESCAPE,
pre-filter, superset, over-match, under-match, anchor, keyspace, window, candidate, row cap, index
entries, 392, and every deleted constant name. Every surviving hit is now either current-and-correct
or explicitly framed as history.
2026-07-27 03:10:36 +02:00
timothy b93a7d33ff docs(578): record the update-openapi.sh incremental-skip trap that made my own check vacuous
Verifying the regenerated OpenAPI artifacts, I re-ran the pipeline against an already-built tree and
got a clean git diff — which I nearly reported as "artifacts confirmed". It was a no-op.

When the project is already built and unchanged, MSBuild skips the document-generation work but
still runs RenameOpenApiFiles (AfterTargets), whose Move then fails with MSB3680 "ErsatzTV.json does
not exist" — nothing produced it. The script exits non-zero correctly, but I had piped it
(`./scripts/update-openapi.sh 2>&1 | tail -2`), so the shell reported tail's 0 and the failure was
invisible. A clean diff after a regeneration that never regenerated proves nothing.

Caught it with a positive control: tamper all three artifacts, re-run, see which get restored.
v1.d.ts came back (npm run generate:api is unconditional) while v1.json and endpoint-index.md stayed
tampered. A `touch` on a compiled source then made the real regeneration run and restore all three
byte-exact, which is the verification that actually means something.

CI is unaffected — the api-docs job restores into a clean tree, so generation never skips. This is a
local-dev hazard only, and it is the same shape as the bug arc this branch is about: a check that
reports success without examining anything, exactly what LIMIT was doing to the row bound.
2026-07-27 03:10:36 +02:00
timothy 373956fcee fix(578): delete the SQL predicate — LIMIT only bounds work when there is nothing to discard
BLOCKER. Attempt 4 bounded the Id KEYSPACE, and keyspace is not rows. Delete 20,000 historical rows,
put one song at Id 20001, query artist?q=que: the walk burned all ten windows on empty ranges and
returned [] for a table containing exactly one row. Capacity fell linearly with deletion ratio and no
ratio was safe — one placed gap hides the next match. My record called that "heavily fragmented" and
the endpoint description said loss happens "on a very large library"; the one-row example disproves
both.

Option A. The query now carries NO predicate at all — no LIKE, no LOWER, not even IS NOT NULL:

  SELECT Id, Artists AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch

That is the whole fix, and it is the point. LIMIT truncates what survives a predicate, so with any
predicate present it bounds the OUTPUT and says nothing about the WORK; the engine may evaluate and
discard arbitrarily many rows first. Stripped to a bare primary-key range, LIMIT n reads exactly n
index entries and n rows — independent of sparsity, deletion history or where the gaps fall. All
selectivity moves into memory. A short page can now only mean exhaustion, which is precisely what it
could not mean while a predicate was present.

Four attempts, four wrong quantities: the result (a fixed budget the over-matching pre-filter
starved), candidates returned (a no-match query must evaluate every eligible row before returning an
empty page), keyspace width (above), and finally actual rows. The record carries the table; it is
worth more than the code.

Deleting the predicate deletes a whole bug family with it: the JSON-escape reasoning, the
narrow-only-on-verbatim-ASCII rule, the exhaustive Unicode sweep that proved it sound, the ESCAPE '/'
portability workaround, and the may-over-match-never-under-match invariant that turned out to be
conditional on something untrue. SearchFieldValuesPrefilterSupersetTests is deleted entirely; the one
assertion worth keeping — that the SQL has no predicate — moved to the query-shape suite, which pins
the SQL string exactly so "just a cheap filter" fails a test instead of silently unbounding the walk.

Measured cost of no server-side narrowing, on a seeded 20,000-song library (in-memory SQLite):
worst case (no match, full walk) 20,000 rows / 10 round trips / 391.9 KiB / 119ms SQL, ~40ms warm
end-to-end. Empty q, dense and non-ASCII prefixes all stop on page 1 at ~39 KiB and ~40ms. Judged
acceptable for a debounced typeahead against a local file. If it ever is not, the answer is #669, not
reintroducing selectivity — the record says so explicitly.

Also fixed:
- Round-trip count was advertised as 10; it is at most 10 for album_artist and 11 for artist, which
  also runs its EF query. The MAX(Id) probe is gone with the keyspace scheme, so there is no extra
  scalar call.
- The duplicated-formula ceiling test is deleted rather than rewritten. It re-implemented the loop's
  arithmetic and would have passed through an off-by-one or a stall in the real loop; the dense
  integration tests carry that coverage. Its MaxVisited >= Window assertion was a style constraint in
  correctness clothing.
- Stale text swept by grepping the mechanism nouns rather than re-reading: candidate/keyspace/
  pre-filter/superset/row cap/LIKE/ESCAPE and the removed constant names, across handler, tests,
  record, api-conventions and the endpoint description. The two surviving "pre-filter" mentions are
  deliberate history. Test comments that rendered escaped non-ASCII as literal characters (which
  contradicted the raw-storage assertion in the same file) now show the escape text.

New test List_Valued_Walk_Reads_Live_Rows_Regardless_Of_Id_Density reproduces the one-row killer and
fails against attempt 4.
2026-07-27 03:10:36 +02:00
timothy fbc7b2a1dd fix(578): bound the Id KEYSPACE — LIMIT cannot bound a query that matches nothing
BLOCKER. Round 3's ceiling counted LIKE-positive candidates, which is not the quantity that needed
bounding. To return an empty page the engine must first evaluate every eligible row, so a no-match
query came back with rows.Count == 0, ended the walk having counted zero against the ceiling, and had
already inspected the entire table. Round trips and materialized rows were bounded; database work was
not. Worse, a dense widened prefix could materialize 20x the candidates and make 10x the round trips
of round 1 — a regression dressed as a bound.

Third time bounding the wrong quantity: revisions 1-2 bounded the RESULT (a fixed LIMIT budget the
widened pattern starved), revision 3 bounded the CANDIDATES, and neither bounds what the database
LOOKS AT. Now the Id range is closed on both sides — `Id > @AfterId AND Id <= @AfterId + @Window` —
so each round trip is a primary-key range scan of known width. The LIKE still decides what comes
back; it no longer decides how much gets looked at. The walk advances by the WINDOW, never by what
returned (an empty page means "nothing matched in this stretch", not "exhausted"), and there is
deliberately no LIMIT in the SQL — the window caps the row count, and a LIMIT would only restore the
illusion that it is doing the bounding. One indexed SELECT MAX(Id) up front stops the walk burning
windows on empty keyspace. Result: at most 10 round trips and 20,000 rows inspected for any q,
matching or not.

New test `List_Valued_Walk_Cannot_Inspect_Past_The_Bound_When_Nothing_Matches` covers the case every
earlier bound test missed — they all gave the pre-filter something to return. With a narrow pattern
and a needle at row 30,000, revision 3 returns the needle (right answer, unbounded work) and this
revision returns []. Paired with a positive control at row 20,000 inside the bound.

The ceiling's remaining false negative (20,000 "zzz" rows then "éclair" at 20,001 -> []) is kept and
still pinned: it is the documented bounded-best-effort contract at a rarer threshold, not a defect.

Stale/false comments, several of them repeat offenders:
- The handler still used "Zulu"/"Éclair" as the truncation example, where both orderings pick "Zulu".
  I had corrected the test and left the comment. Now "Zulu"/"apple" in both.
- "leans entirely on the row cap" described a mechanism that no longer exists.
- The escape example rendered as literal Édith/É/é rather than the escape TEXT (Édith,
  É, é) in both the handler and the record, contradicting the explanation it was giving.
- "every accented artist was unsuggestable" overstated it: exactly three of the nine pinned cases
  fail 1b78dc9e (those where query and stored casing differ, so the escape texts diverge); the other
  six pass. The record now says what the test comment already said.
- The record claimed the ordinal switch left response SETS unchanged. False — ordering runs before
  Take(limit), so with "Zulu"/"apple", empty q and limit=1 the comparer changes which value survives.
- The_Scan_Ceiling_Is_A_Whole_Number_Of_Batches was a style preference wearing a correctness costume
  (a 20,500 ceiling is perfectly safe — the final window clamps). Replaced with the invariant that
  actually matters: the walk's own arithmetic terminates and lands exactly on the ceiling.
2026-07-27 03:10:35 +02:00
timothy a37847e509 fix(578): a fixed row budget turns an over-matching prefilter into a false negative
BLOCKER 1. The invariant a883e5f0 established — "the pre-filter may over-match, it must never
under-match" — is only sound while the candidate set is not truncated. It was truncated. A non-ASCII
or JSON-escaped prefix collapses the pattern to the bare `%"%` anchor, so every row becomes a
candidate, and `ORDER BY Id LIMIT 1000` then spent the whole budget on rows that could not match.
Seed 1000 songs by "zzz", put the only "éclair" in row 1001, ask for album_artist?q=é: a883e5f0
returns [], while 1b78dc9e returned "éclair" because its (separately broken) tighter pattern kept
the candidate set small. Neither revision was correct — the old one under-matched at the pattern,
the new one under-matched at the cap. Widening a predicate under a fixed budget starves it.

So the budget is gone. Candidate rows are now walked keyset-paged on Id (`Id > @AfterId … ORDER BY
Id LIMIT @Batch`), continuing past non-matching candidates and stopping on the first of: enough
distinct exact matches for `limit`, a short page (source exhausted), or a 20,000-candidate-row
ceiling in 2,000-row batches. The bound is on effort; it no longer silently decides the result, and
the lossy case needs 20,000 rows that already passed the pre-filter before it bites.

BLOCKER 2. The endpoint description and the record's rule claimed ordinal matching/dedup/ordering
endpoint-wide. False for EF-backed fields: the database runs LOWER/DISTINCT/ORDER BY/LIMIT before
any ordinal code, so `genre?q=é` still misses a stored "Éclair" on SQLite. Both are now scoped to
the final in-memory stages, and the underlying gap is referenced as #668 rather than described as
fixed. #669 (normalized SongArtist table) is referenced as the follow-up for the scan cost.

Accuracy corrections to my own claims, all verified by re-running the mutations:
- Only THREE of the nine Unicode cases fail 1b78dc9e (é/édith/BJÖRK — where query and stored casing
  differ, so the escape texts diverge); the other six pass it. The comment said all nine. They stay
  as continuity coverage, now labelled as such rather than as regression guards.
- Ordering_Is_Best_Effort used "Zulu"/"Éclair", where DB and ordinal orderings BOTH pick "Zulu" — it
  could not demonstrate the divergence it claimed. Now "Zulu"/"apple", which actually diverges:
  ordinal ranks "Zulu" first, the DB ranks "apple" first, and limit=1 returns ["apple"]. The record
  sentence was false and is corrected.
- The record printed literal "Édith"/"é" where it needed to show the escape TEXT (Édith,
  é), contradicting the very explanation it was giving.
- Corrected the cost claim: the leading wildcard forces scan ACCESS, but each page stops once it has
  filled @Batch, so a dense query finishes early — it is not necessarily a full table scan.
- The Unicode sweep is labelled a PROOF OBLIGATION: it is revision-independent and passes every
  revision, which is correct for what it is but must not read as regression coverage.

One process note: the new record's frontmatter had a lone apostrophe inside a single-quoted YAML
scalar ("SQLite's"). decisions_validate.py's hand parser accepted it; scripts/tests caught it.
2026-07-27 03:10:35 +02:00
timothy 1641ca8305 fix(578): the LIKE prefilter under-matched every accented artist; make the superset provable
Review of 1b78dc9e found the pre-filter's correctness claim was false, and the claim was in the
decision record as well as the code.

F1 (high). The pattern JSON-encoded the whole query prefix on the reasoning that the stored text
escapes non-ASCII, so encoding the prefix the same way would line up. It does not: SQL LOWER()
lowercases the *escape text* (`É` -> `é`); it cannot case-fold the codepoint that escape
denotes. So `q=é` built `%"é%`, the stored `Édith Piaf` never matched, and the row was
discarded before the in-memory filter could accept it. Every accented artist — Beyoncé, Björk,
Sigur Rós, Édith Piaf — was silently unsuggestable, which in a music library is the common case.

The invariant that was missing, now stated in the code: the SQL pre-filter is an OPTIMIZATION. It
may over-match; it must never under-match. Correctness lives in the in-memory filter. So the pattern
now narrows only on the leading run of characters the JSON writer stores verbatim and stops at the
first character it cannot prove — `q=Beyoncé` still narrows on `beyonc`, `q=é` narrows on nothing
and leans on the row cap. Soundness rests on two facts now asserted by exhaustive computation rather
than argued: no non-ASCII codepoint in U+0080..U+10FFFF OrdinalIgnoreCase-equals a printable ASCII
character (false for InvariantCultureIgnoreCase, which folds ~190 — the choice of Ordinal is
load-bearing), and the exact set of ASCII the encoder escapes.

F1b. `UseRequestLocalization` honours Accept-Language, so the culture was caller-controlled and
`ToLower()` plus the default linguistic `StartsWith(string)` let a header change the answer.
Comparison is now OrdinalIgnoreCase and ordering StringComparer.Ordinal throughout — including the
shared FilterSortTake that state/video_dynamic_range/content_rating also use. Sets unchanged,
order now ordinal rather than culture-dependent.

F2. The merge comment asserted an exactness the code does not have: sources truncate by their own
ordering (DB collation / primary key), not the merge's, so a dropped value can outrank a survivor.
Comment and record now say best-effort, exact only below the truncation points.

F3/F4. The cap now rides `ORDER BY Id` rather than the JSON column: MySQL sorts TEXT by only
max_sort_length bytes, so the old ordering was not deterministic there, and sorting the whole
matching set was avoidable work. What the cap still does NOT bound is the scan — a leading-wildcard
LIKE cannot seek an index — so that cost is now documented as accepted, with a normalized
`SongArtist` table named as the follow-up candidate rather than left implicit.

Every clause above is covered by a test verified to FAIL when that clause is mutated (old pattern
builder: 5 red; culture chain: 3 red; cap=3 / cap=limit / ORDER BY json / no cap: red each).

F5. Converted to a proper supersession. The old record did not merely hold a stale fact — it
recorded song/music-video credits as an "intentionally-uncovered gap" and album_artist as
unsupported, and this reverses that call, which `docs.decision-lifecycle` says is never a
line-edit. `api.search-field-values` is archived with its original prose restored, and
`api.search-field-values-sources` replaces it carrying the whole endpoint contract.
2026-07-27 03:10:35 +02:00
timothy cd6f36185c feat(578): artist typeahead covers music-video and song credits; album_artist stops 404ing
`GET /api/v1/search/fields/{name}/values` sourced `artist` from `ArtistMetadata.Title` only —
entity artists — so the free-text credits that `LuceneSearchIndex` also writes to the `artist`
field (`MusicVideoArtist.Name`, `SongMetadata.Artists`) produced no suggestions, and
`album_artist` 404'd outright.

`MusicVideoArtist` turned out to be a real entity table, so it just joins the existing server-side
pipeline as a `Concat` — one bounded `UNION ALL` + `LOWER(...) LIKE ... LIMIT` on both providers.

`SongMetadata.Artists`/`AlbumArtists` are the hard case: EF 9 maps them as primitive collections
(one JSON array per row in a single column), and neither provider can project the elements
server-side — SQLite needs the SQL APPLY operator it lacks, Pomelo MySQL 9.0.0 has no
primitive-collection support at all. Both failures are now pinned by a test, so a provider upgrade
that fixes them shows up as a red rather than as a stale workaround. For those columns the handler
pre-filters on the raw JSON (`LOWER(col) LIKE '%"<encoded-prefix>%' ESCAPE '/'` — a deliberate
superset, since it matches a row and not an element), caps the rows at 1000 with an `ORDER BY` that
makes the truncation deterministic, then splits and exact-filters in memory.

Provider portability is by construction rather than by trusting `LIKE`: the prefix is JSON-encoded
before matching (which is how it is stored, and which makes the pattern pure ASCII, so SQLite's
ASCII-only `lower()` and MySQL's Unicode-aware `LOWER()` agree) and lowercased in C#, so the match
is correct under a case-sensitive MySQL collation as well as a case-insensitive one. The escape
character is `/`, never `\`: `ESCAPE '\'` is not a portable SQL literal.

An empty `q` stays supported for these fields — the row cap already bounds it, and a non-empty-`q`
carve-out would make one group of fields behave differently for the same client code.

Docs: new `api.search-field-values-list-columns` record (additive sibling — the base record's rule
still reads true; only its body's "intentionally-uncovered gap" claim was stale), plus
`api-conventions.md`, `spa-conventions.md` §12 and the regenerated `v1.json`.

Decisions-Edit: yes
2026-07-27 03:10:35 +02:00
timothy 17c25e75fa fix(650): replace node:fs/path/url with import.meta.glob in the pageSize guard
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 30s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 33s
PR Gates / Docs update reminder (pull_request) Successful in 37s
Review verdict / Set review-verdict status (pull_request) Successful in 36s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m33s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m7s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 21m13s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 24m42s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 27m7s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ 17c25e7 (base: main)
PR Gates / decisions lifecycle (pull_request) Successful in 13s
The gate is npm run typecheck (tsc -b --pretty false, project-mode) — the
prior fix for the 3 node:*-unresolvable errors deferred a decision rather
than resolving it, per instruction not to change tsconfig unilaterally.

Evaluated and rejected: adding "node" to tsconfig.app.json's `types` (makes
Node globals type-visible to production browser code, dissolving a
deliberate separation the repo documents) and a file-local
`/// <reference types="node" />` (empirically leaks Node's ambient
`setTimeout` into the whole tsc -b program, breaking 3 unrelated
window.setTimeout mocks — confirmed by trying it).

Adopted instead: Vite's `import.meta.glob('/src/**/*.{ts,tsx,mts,cts}',
{ query: '?raw', import: 'default', eager: true })`, resolved natively by
Vite/vitest at transform time — no node:fs, no node:path, no node:url, no
tsconfig change, no cross-project references, and the guard test stays
exactly where it is (`src/api/`). `vite/client` types (hence
`import.meta.glob`) were already wired in via `src/vite-env.d.ts`.

`isScannableSourceFileName` is unchanged and still the single place the
glob's results are filtered through — the extension set moved into the glob
literal, but discovery still runs every matched file through this same
named, tested predicate rather than a second copy of the logic.

Verified the discovery mechanism swap changes nothing observable: a
side-by-side comparison test (old fs-based walk vs new glob-based walk, both
run against the current repo, not committed — temporary) found byte-identical
results — 19 call sites, 136 scannable files, zero set difference in either
direction. Re-ran both required plants against the new mechanism:
  1. Duplicate at-cap literal appended to builder/ChannelBuilder.tsx (an
     already-registered file): caught —
     `UNREGISTERED (1) ... + builder/ChannelBuilder.tsx:2058:54:literal:100`.
  2. New file with a `https://` string (no false match), a shorthand
     `{ pageSize }`, and two literal pageSize values in one ternary's two
     branches on a single line: all 3 real sites caught at distinct columns —
     `+ screens/_fakeDefectPlant.ts:4:30:shorthand:pageSize`,
     `+ screens/_fakeDefectPlant.ts:8:22:literal:111`,
     `+ screens/_fakeDefectPlant.ts:8:48:literal:222`.
Also re-verified the .mts/.cts discovery pin still fails when the extension
set is narrowed back to `.ts`/`.tsx` only (2 of 14 predicate cases fail, as
before). All three plants reverted after verification; registry/predicate
otherwise untouched.

Full local gate: `npm run lint` clean, `npm run typecheck` (tsc -b) clean —
zero errors, including the 3 node:* ones this commit resolves — `npx vitest
run` 110 files / 1078 tests passed (re-run three times; two runs hit
transient vitest worker-pool timeouts from overlapping background processes
on this machine, unrelated to the code — a clean sequential run passed in
full both before and after).
2026-07-27 02:40:31 +02:00
timothy 5b46214774 fix(650): fix 3 tsc -b never-callable errors in gate helpers (project-mode typecheck)
npm run typecheck (tsc -b --pretty false, the real gate — npx tsc --noEmit
was silently checking nothing meaningful due to the root tsconfig.json's
empty `files: []`) flagged 3 errors: `let x: (() => void) | null = null`
gate-release helpers, reassigned only inside a Promise executor, narrowed a
later `x?.()` call to `never` under tsc -b's project-mode control-flow
analysis. Not reproducible under a bare `tsc --noEmit` invocation.

Fixed by adopting the no-op-initializer pattern already established
elsewhere in this repo (api/libraries.test.ts's `releasePost`): declare as
`let x = () => {};` instead of `(() => void) | null = null`, dropping the
now-unnecessary optional chain at each call site. Same runtime behavior,
no `as any`/`@ts-expect-error`, no change to what any test asserts.

3 call sites fixed: releaseStrandedAppend (F3 test), and releaseB0 in both
the inverse-settlement-order test and the round-4 hook-level single-flight
test.

Verified: `npm run typecheck` no longer reports these 3; `npm run lint`
clean; `npx vitest run` 110 files / 1078 tests still pass.

3 more typecheck errors remain (node:fs/node:path/node:url unresolvable in
src/api/pageSizeCallSites.guard.test.ts, the only file under src that needs
real Node fs access) — deliberately NOT fixed here. @types/node is already
a devDependency and physically installed, but tsconfig.app.json (covering
all of src) has no "node" in its `types` array, and a file-local
`/// <reference types="node" />` was tried and reverted: under tsc -b's
single-program compilation, it leaked Node's ambient `setTimeout` (returning
NodeJS.Timeout) into the whole app project, breaking 3 unrelated
window.setTimeout mocks expecting the DOM signature (confirmed by trying
it — TS2345 in ChannelsScreen.test.tsx/LibrariesScreen.test.tsx/
PlayoutsScreen.test.tsx). The clean fix is a real project-config change
(either widen tsconfig.app.json's types, accepting Node globals become
type-visible in browser app code too, or move this one file into
tsconfig.node.json's project alongside the e2e specs, which would also need
a "references" wire-up for its cross-project import of pageSizeScan.ts) —
left for the coordinator to choose per their explicit instruction not to
make this call unilaterally.
2026-07-27 02:08:19 +02:00
timothy 937ee92a3f fix(650): close two test-adjacency gaps — hook-level single-flight pin, predicate-level glob pin
Fourth cold cross-family review: no runtime correctness finding this round
(single-flight held against synchronous throws, generation overlap,
StrictMode, unmount, and separate hook instances). Two test-gap findings
remained, both the same shape as prior rounds' review feedback: asserting on
something ADJACENT to the mechanism rather than the mechanism itself.

1. ChannelBuilder.test.tsx's page-0 single-flight test asserted on the
   disabled BUTTON — during a page-0 refresh, `loadingMore` disables the
   native button, so `fireEvent.click` never reaches `loadMore` at all.
   Deleting `busyRef.current = true` at libraryBrowse.ts's generation-change
   effect would leave that test green while direct hook calls could still
   overlap page 0 and page 1. Added a hook-level test that calls
   `result.current.loadMore()` directly (via `renderHook`, bypassing the
   button/DOM layer) while a gated page-0 fetch is in flight, and asserts no
   page-1 request is ever issued. Verified: removing `busyRef.current = true`
   at that line makes the new test fail with
   `expected [ +0, 1 ] to deeply equal [ +0 ]` (a page-1 request was issued
   when the guard should have ignored the direct call); restored, green
   again.

2. pageSizeScan.test.ts's `.mts`/`.cts` tests exercised the SCANNER'S
   PARSING, not the guard's file-discovery glob — reverting the glob back to
   `.ts`/`.tsx` left both those tests green (the scanner still parses a
   `.mts`/`.cts` filename as plain TS regardless of extension) AND the
   real-source guard green (this repo has no committed `.mts`/`.cts` file
   for the reverted glob to miss). A prior verification planted a real
   `.mts` file and watched the guard notice it, which proved the behavior
   exists today but pinned nothing going forward. Extracted the inline glob
   condition into a standalone, exported `isScannableSourceFileName`
   predicate and added 14 parametrized cases asserting it BY FILENAME (no
   filesystem involved) — `.ts`/`.tsx`/`.mts`/`.cts` accepted, their
   `.test.*` and `.guard.test.ts` variants excluded, non-TS extensions
   rejected. Verified: reverting the predicate's regex back to `.ts`/`.tsx`
   only makes 2 of the 14 cases fail (`isScannableSourceFileName('*.mts')`
   and `'*.cts'` both wrongly return `false`); restored, green again.

Nothing else changed — the reviewer confirmed no further findings (loader
async/throw handling, generation sequencing, the unconditional rollback,
StrictMode/unmount behavior, the disabled-during-refresh UX, type-only node
exclusion, computed-name exactness, wrapped-forwarded-call rejection, and
all existing test labelling were all confirmed correct as-is).

Full local gate: lint clean, tsc clean, full vitest run 110 files / 1078
tests passed, re-run twice for stability (no flakes).
2026-07-27 01:29:15 +02:00
timothy 1c86a1c1fc fix(650): enforce single-flight in useLibraryBrowse; close scanner false negatives
Third cold cross-family review (BLOCKED) found the append/page-0-refresh
races were being fixed one interleaving at a time — round 1 fixed
page-0-settles-first, round 2's compare-and-set rollback fixed the
duplicate-append case but introduced a permanently-skipped page, and the
reviewer found the exact mirror of round 1's fix (page-1-settles-first,
erasing page 1 with no cursor reset). Direction from the review: stop
enumerating orderings, make the overlap structurally impossible.

SINGLE-FLIGHT (web/src/builder/libraryBrowse.ts): a new `busyRef` guard is
true from the moment ANY fetch (a page-0 refresh OR an append) for the
current query generation is issued until it settles. `loadMore` checks it
SYNCHRONOUSLY and returns immediately (ignored, not queued) if a fetch is
already in flight — including a page-0 refresh, not just a prior append, so
a "Load more" click that lands while a query change is still resolving is a
no-op rather than starting a second, overlapping request. With overlapping
fetches eliminated by construction, the append-failure rollback no longer
needs the round-2 compare-and-set: single-flight guarantees nothing else
could have moved `pageRef` since a given fetch started, so it now always
rolls back and retries the exact page that failed, unconditionally.

Visual feedback (the button showing loading/disabled during a page-0
refresh, not just an append) is set via `queueMicrotask(() => setLoadingMore
(true))` rather than a bare synchronous call in the generation-change effect
— `react-hooks/set-state-in-effect` flags the latter; a microtask-deferred
call resolves before any human-perceptible input, satisfies the lint rule
(the same reason `.then()` callbacks elsewhwere in this hook aren't flagged),
and keeps the actual correctness guarantee (the ref check) perfectly
synchronous regardless.

TESTS REWRITTEN, not just added — the round-2 "HIGH-2" hook test explicitly
asserted the NEXT request after a failed page 1 (following an overlapping
page 2 success) should be page 3, i.e. it blessed page 1's permanent loss.
Replaced with two hook-level tests: single-flight ignores a synchronous
double `loadMore()` call (only one fetch issued), and a failed page is
retried as the SAME page number. Replaced the round-2 component-level
"HIGH-1" test (which drove the now-impossible overlap through the DOM) with
one asserting the click during a pending page-0 refresh is ignored, and that
once free, the correct page-1-then-page-2 sequence completes with both
pages' rows present. Verified all three new/rewritten tests against the
prior committed hook (7b1ae48b0): the two single-flight-specific tests fail
as expected (`[0, 1]` requested when only `[0]` should have been); the
retry-semantics test happens to pass against 7b1ae48b0 too (compare-and-set
and unconditional rollback coincide in the non-overlapping case) but is kept
because it is the correct "retry as page 1, not page 3" pin the review asked
for, replacing the one that asserted the wrong thing.

SCANNER (pageSizeScan.ts) — closed three documented false-negative classes:
- Transparent TS wrappers around the initializer (`pageSize: 100 as const`,
  `100 satisfies number`, parenthesized) are now unwrapped before the
  NumericLiteral/Identifier check.
- Non-Identifier property names: a quoted string key (`'pageSize': 100`) or
  a statically-resolvable computed key (`['pageSize']: 100`) are now
  accepted; a computed key that isn't a literal correctly stays unresolved.
- `.mts`/`.cts` are no longer silently excluded from the guard's file
  discovery glob (tsconfig.app.json's `include` covers all of `src`; no such
  files exist in the repo today, but the glob shouldn't hide one if it ever
  does).
10 new fixture tests in pageSizeScan.test.ts pin each case (plus a rejection
test confirming a forwarded call wrapped in `as` still doesn't match, and
one confirming an unresolvable computed key stays unmatched).

TEST LABELLING: relabeled the URL/M-3 and `??`/M-4 fixtures as CONTRACT
fixtures rather than regression pins — a round-3 review found round 1's
plain literal regex already handled those two exact inputs correctly on its
own; only the combined multi-case fixture (and the string-contains-text,
template-interpolation, same-line-identity, JSX, and destructuring fixtures)
actually fail against round 1. Labeled the guard test's 4 tests as BASELINE
assertions (they all pass on clean b90f8a3b) rather than implying they prove
this round's specific fixes — pageSizeScan.test.ts's fixtures are what
actually regression-pin the scanner.

No server-side/C# change. Full local gate: lint clean, tsc clean, full
vitest run 110 files / 1063 tests passed (re-run twice, stable).
2026-07-27 01:29:15 +02:00
timothy ca99bedb1a fix(650): rewrite the pageSize guard on the TS compiler API; fix two append-ownership races
Second cold cross-family (Codex, BLOCKED) re-review of b90f8a3b found the
regex/bracket-tracking guard scanner still defeated in five ways, and two new
High-severity races introduced by the F3/F4 fixes. Addressed as a further
follow-up (b90f8a3b left untouched).

GUARD REWRITE (per the review's explicit direction — stop patching the regex,
use the compiler):

- New `web/src/api/pageSizeScan.ts`: `scanPageSizeSites` parses each file with
  `ts.createSourceFile` and walks the real AST for `pageSize`
  PropertyAssignment/ShorthandPropertyAssignment nodes inside an
  ObjectLiteralExpression. This eliminates categorically (not case-by-case):
    - M-3: comments and string/template CONTENTS are never revisited as code,
      so a `'https://...'` string can't be misread as an unterminated string
      that swallows the rest of the file.
    - M-4: an object literal nested in a ternary, `??`, or JSX expression
      container is still found — the walk visits every descendant node
      regardless of the syntactic context above the ObjectLiteralExpression.
    - M-5: template-literal interpolations are real AST children, not opaque
      text.
    - L-7: a type literal (`type P = { pageSize: 100 }`), an interface
      PropertySignature, and a destructuring ObjectBindingPattern (parameter
      or nested) are structurally different node kinds from
      ObjectLiteralExpression — excluded by kind, not by a
      preceding-character heuristic a stray `{`/`(`/`,` could fool.
  `getLineAndCharacterOfPosition` gives exact line+column (fixes M-6 identity
  granularity) instead of the prior line-only identity.
- `pageSizeCallSites.guard.test.ts` now imports the shared scanner; identity
  is `file:line:column:kind:value`, compared as a MULTISET (count, not
  membership) in both directions.
- Both directions (unregistered / stale) are computed and folded into ONE
  thrown Error so a failure always shows the complete picture in one run,
  addressing the line-churn "second direction never renders" concern.
- New `pageSizeScan.test.ts`: a FIXTURE test (inline source strings, no repo
  scan) pinning the exact discovered set for every case the review named —
  comment-in-string, string containing the literal text `pageSize: 100`,
  template interpolation, ternary, `??`, JSX container, same-line duplicates,
  parameter/nested destructuring, a type literal, an interface property, a
  forwarded call expression, a React dependency array. This is what actually
  protects the scanner going forward — the guard test alone only ever proved
  today's snapshot of real call sites, never the scanner's handling of input
  classes it hadn't happened to encounter yet.
- Re-verified both original plants (a duplicate at-cap call in an
  already-registered file, and a new file with both a literal and a
  shorthand site) against the rewritten scanner; both still fail with the
  new combined-direction message. Also verified a run with BOTH directions
  simultaneously non-empty renders both in one report.

HIGH-1 (ChannelBuilder.tsx useLibraryBrowse, now web/src/builder/libraryBrowse.ts):
`reqId` identifies a query GENERATION, not an individual fetch — a page-0
refresh and a "Load more" append can be outstanding simultaneously under the
same reqId (query changes while an append is in flight for the new
generation). Whichever settled first used to clear `loadingMore`, letting a
second click fire an out-of-order/duplicate page fetch. Fixed with a
per-fetch `fetchId` plus a `loadingFetchIdRef`/`loadingFetchReqIdRef` pair:
only the fetch that OWNS the currently-displayed spinner can clear it; a
same-generation page-0 refresh leaves a same-generation append's spinner
alone, while a page-0 refresh for a NEW generation still retires an
abandoned OLDER-generation append's spinner (preserving the original #650 F3
fix). Reproduced the exact interleaving from the review in a new test
(gate B's page-0 and page-1 fetches independently, click "Load more" while
B's page-0 is still in flight) and confirmed it fails without the fix
(button re-enables while the append is still pending).

HIGH-2 (same file): the append-failure rollback mutated whatever
`pageRef.current` currently held, rather than the specific page THIS fetch
requested — under an overlapping-append race, a later page's success
followed by an earlier page's failure could roll the cursor back past
already-appended progress, corrupting a retry into refetching a duplicate.
Fixed with a compare-and-set guard (`if (pageRef.current === pageNum)`) so
the rollback only fires when nothing has advanced the cursor since. Since
this overlap is UI-unreachable once HIGH-1's single-flight disabling is
wired up (verified empirically: two synchronous fireEvent.click calls in RTL
only produce one request, since act() flushes the disabling render between
them), the regression test drives `useLibraryBrowse` directly via
`renderHook` (now exported) to force the exact interleaving and confirms it
fails without the fix (page 2 gets duplicated, page 3 never requested).

Extracted `useLibraryBrowse` (plus `loadCollections`/`loadLibraryItems`/
`BrowseState`/the media-type const arrays) into a new non-JSX module
`web/src/builder/libraryBrowse.ts` — exporting a hook from a .tsx file
tripped `react-refresh/only-export-components`; this also makes the hook
importable by `renderHook` without pulling in the whole screen component.

No server-side/C# change. Full local gate: lint clean, tsc clean, full
vitest run 110 files / 1051 tests passed (one LibrariesScreen.test.tsx
flake reproduced under full-suite parallel load, confirmed pre-existing and
unrelated — passes in isolation, never touched that file).
2026-07-27 01:29:15 +02:00
timothy 2d049e9a28 fix(650): follow-up — per-occurrence guard identity, shorthand pageSize detection, and three UI defects
Cold cross-family (Codex) review of 9763fdca found real defects; addressed as a
follow-up rather than amending that commit.

MUST FIX, addressed:

- F5: pageSizeCallSites.guard.test.ts collapsed call-site identity to
  `file:value`, so a SECOND at-cap call in an already-registered file was
  invisible (verified: appending a duplicate `getLibraryBrowseItems({
  mediaType: 'Movie', pageSize: 100 })` to ChannelBuilder.tsx passed all 4
  guard tests before this fix). Identity is now `file:line:kind:value` — a
  bracket/quote-tracked scan resolves each occurrence's exact line, so a
  duplicate on a new line is a new, unregistered identity.

- F6: the guard now also detects the ES6 shorthand property form (`{ ...,
  pageSize }`), not just `pageSize: <value>`. Implemented as a bracket-stack
  scan that distinguishes an object-literal `{` (real risk) from a
  block-statement `{` or an array `[` (false positives from things like
  `useCallback` dependency arrays and `const pageSize = 100;` inside a
  function body) by inspecting the token immediately preceding each `{`.
  Six real shorthand sites are now registered (the two inside loadAllPages
  itself, ChannelBuilder's two per-kind fan-outs, and two genuine
  user-adjustable pagers in LogsScreen/BlockPlayoutTroubleshootingScreen).
  Object SPREAD and positional-argument pageSize (api/search.ts's
  api.search-allitems-paging precedent) remain a documented residual gap,
  written down in the test file's own header comment, not silently absent.

- F2 (TraktListsScreen.tsx): an incomplete load with zero accumulated rows
  rendered BOTH "List may be incomplete" and the unsupported "No Trakt lists
  yet." claim. The zero-row empty state now branches on `incomplete` first.

- F3 (ChannelBuilder.tsx useLibraryBrowse): changing the query/library while
  a "Load more" append was in flight stranded the button in its
  loading/disabled state forever (the stale append's own `finally` no longer
  matched the current request id, and the superseding fresh fetch never
  cleared `loadingMore` either). `finally` now clears `loadingMore` whenever
  the settling request is still the CURRENT one, regardless of whether that
  particular request was an append.

- F4 (ChannelBuilder.tsx useLibraryBrowse): one rejected per-kind request in
  an append's `Promise.all` wiped every already-loaded row via
  `items: []` with no way back. Append failures now preserve state, surface
  the error inline next to a still-present "Load more" button, and roll the
  page cursor back so a retry re-requests the same page instead of skipping
  it.

Both new UI fixes are pre-existing defects in the 'library' source that
9763fdca's loadCollections fix newly made reachable from 'collections' too.

Verified all four fixes against negative controls: reverted each in turn and
confirmed its dedicated test fails with the expected message, then restored.

DO NOT FIX (filed as timothy/ersatztv#665 instead, bug+frontend+priority:low):
- F1: loadCollections/loadLibraryItems sort each fetched page independently,
  so appended pages are only locally sorted, not globally sorted across the
  accumulated list.
- F7: an overclaiming totalCount can leave "Load more" clickable after every
  kind is actually exhausted (no auto-loop; a user click is still required
  each time).

Not touched (reviewer confirmed correct as-is): Trakt sequence/abort/unmount
handling, the Class-A vs Class-B incomplete-copy distinction, and the
`lists.length` footer count.
2026-07-27 01:29:15 +02:00
timothy ad4ac6c7e0 fix(650): page Trakt lists to completeness and report real totals in loadCollections
Two SPA list loads requested EXACTLY the server's pageSize cap (100), truncating
identically to #634/#644's over-cap defect but invisible to that fix's manual
"pageSize above the cap" grep:

- TraktListsScreen requested pageSize:100 and rendered BOTH the truncated page
  AND the real totalCount, so 101 lists showed as "101 lists" over a 100-row
  table. Trakt lists are bounded-by-construction (Class A), so this now pages
  to completeness via the shared loadAllPages helper, surfaces an "incomplete"
  badge if a page ever comes back short of totalCount, and passes an
  AbortSignal from the effect cleanup.

- ChannelBuilder's loadCollections (fanning out per collection kind) reported
  the truncated merged.length as totalCount, so canLoadMore's
  `items.length < totalCount` comparison was permanently false and "load more"
  could never fire. It now sums the real per-kind totalCount, mirroring the
  existing loadLibraryItems pattern in the same file, and canLoadMore is no
  longer gated to the 'library' source only.

Also found and fixed a third at-cap site not named in #650: ChannelBuilder's
SeasonsDialog (TelevisionSeason browse scoped to one show) reads pageSize:100
but never read the response's totalCount. No real show has 100+ seasons, so
this stays a single bounded page (Class B) rather than paging to completeness,
but now surfaces a "Showing the first N of M seasons" hint instead of silently
truncating if a show somehow exceeds the cap.

Codifies the missing completeness guard as an enumerating allow-list vitest
test (web/src/api/pageSizeCallSites.guard.test.ts): scans every `pageSize:`
call site in the SPA and diffs it against a hand-reviewed registry in both
directions (unregistered site = new defect risk, stale entry = registry rot),
with anti-vacuity floors on files-scanned and sites-discovered. Verified the
guard actually fails on a planted defect and a planted stale entry before
finalizing it.

No server-side change: the client pages, the server stays bounded
(api.search-allitems-paging precedent).
2026-07-27 01:29:15 +02:00
timothy 8de02d5bde Merge pull request 'fix(649): point the ENFORCED review-verdict gate at the shared PR-file enumeration' (#666) from fix/649-enforced-verdict-guard into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 16m56s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 17m36s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 22m9s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 14m40s
Renovate / Renovate (push) Successful in 5m16s
2026-07-26 22:04:06 +00:00
timothy 8dcd4f3602 Merge pull request 'fix(632): bind a review verdict to its BASE branch, not only to its head sha' (#667) from fix/632-verdict-base-ref into main
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 18s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 33s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 36s
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
2026-07-26 21:58:49 +00:00
timothy e960d5b918 test(649): make the POST-wiring assertion unable to opt out or accept the wrong host
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
PR Gates / Script tests (pytest) (pull_request) Successful in 36s
Review verdict / Set review-verdict status (pull_request) Successful in 40s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
review-verdict/h10 Review-verdict: MERGEABLE @ e960d5b
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m28s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 23m3s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 24m2s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Re-review found the verifier could disable itself two ways, both of which look like
coverage:

- it was guarded by `if url_file.exists()`, so deleting the recorder in the stub turned
  the whole assertion into a no-op and every test stayed green;
- it compared only the URL SUFFIX, so a POST to the right path on the wrong HOST or the
  wrong REPO passed — which is exactly the class the assertion was added to catch.

It now requires the URL to have been recorded whenever a status was posted, and compares
the full URL against the env the job was given. Mutation-verified three ways: wrong host,
wrong repo, and deleting the recorder each redden the suite.

Refs #649
2026-07-26 23:38:25 +02:00
timothy ed8de77e10 fix(632): validate status ROWS, not just the top-level array — the same swallow one level down
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 19s
Review verdict / Set review-verdict status (pull_request) Successful in 6s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
PR Gates / Script tests (pytest) (pull_request) Successful in 49s
review-verdict/h10 Review-verdict: MERGEABLE @ ed8de77 (base: main)
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m14s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m1s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 8s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 16m46s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Re-review caught my previous fix claiming more than it delivered. "Every unreadable input
asks" was false: validating only that `.statuses` is an array left `{"statuses":[1]}`
passing the guard, after which `.context` on a number errors and the `|| true` on the
extraction turned that error into an empty description — straight back onto the
graceful-adoption path the guard exists to distinguish from. The identical
swallow-the-error shape I had just fixed a few lines up, surviving one level deeper.

The validation domain now matches the CONSUMPTION domain: every row must be an object
with a string `.context` and a `.description` that is absent or a string. The extraction
drops its `|| true` and asks explicitly instead, since a swallowed error there is
indistinguishable from a benign "no base recorded".

Both guards are load-bearing, for DIFFERENT shapes — established by mutating them
together and separately rather than assuming the pair was redundant:

- a non-string `.description` is caught ONLY by the row validation (jq -r renders the
  object as JSON, the sed finds no `(base: …)`, and it silently reads as a legacy verdict);
- a scalar row is caught by EITHER, so with the validation weakened the extraction guard
  is what still asks.

Also noted rather than changed: this is the third read of the same status endpoint in a
worst-case hook run. Sharing one snapshot would close a narrow same-run disagreement
window, but the other two branches derive different decisions from a failed read, so
threading a shared response through them changes pre-existing logic rather than #632's.
Recorded in place so it is not rediscovered as an oversight — every `decide` exits
immediately, so the reads cannot produce one self-contradictory message.

Refs #632
2026-07-26 23:38:14 +02:00
timothy 3885fd6aea docs(649): narrow the enumeration's stated guarantees to what it actually proves
Two limitations the cold review surfaced are now written where the guarantees are
described, rather than living only in a review transcript. Both are pre-existing and
tracked separately (#663, #664); neither is fixed here.

- Head-sha binding detects ONE-WAY movement. An A->B->A force-push round trip restores
  the expected sha, so the binding holds while the pages came from two states. The record
  previously read as though the race were closed.
- A commit status is repo-GLOBAL, so a success earned on one PR is inherited by any other
  PR with the same head. Same property that makes the per-sha binding work, read from the
  other end.

Refs #649

Decisions-Edit: yes
2026-07-26 23:29:44 +02:00
timothy d51255a8ef fix(632): "could not check" is a third outcome, not a quiet synonym for "nothing to check"
Cold review's substantive finding. The first draft collapsed an unreadable status
response into the graceful-adoption path: `vdesc` came back empty, so `recorded_base` was
empty, so the comparison was skipped IN SILENCE — and a later, successful status read
could then auto-grant, emitting "merge gate: satisfied" for a comparison that never
happened. A transient Gitea hiccup is not evidence that the base is unchanged.

The unreadable status response and a PR with no resolvable `.base.ref` now both fall
through to a human `ask`, leaving exactly one benign silent case: a verdict that predates
#632 and could not have carried the field. The emptiness check is done in SHELL before jq
sees it, same jq-1.6 rule as the rest of this file.

Also from review: the graceful-adoption test asserted only that the decision lacked the
issue tag, so it would have passed for a base-specific ask or deny whose wording omitted
it — the failure mode most likely to appear when someone edits these messages. It now
asserts on the word "base".

Recorded rather than fixed, because fixing it would be worse: docs-only PRs exit before
this check, since that carve-out short-circuits the gate earlier. It does not auto-grant
— it passes through to an ordinary permission prompt — so the exposure is a missing
warning on a merge a human is already confirming, not a silent merge. The record now says
so instead of implying the deny is unconditional.

Mutation-verified: collapsing the unreadable case back into graceful adoption, skipping
the check on a missing live base, and dropping the mismatch deny each redden their own
test and nothing else.

Refs #632
2026-07-26 23:27:22 +02:00
timothy 322dd43d10 fix(649): close the test-isolation gaps cold review found, and make the job's Gitea config authoritative
Four findings acted on; two more are real but pre-existing and are being filed rather
than fixed here (see below).

**The job's Gitea config was not authoritative.** `pr-changed-files.sh` resolves
`ETV_GITEA_URL` BEFORE `GITEA_BASE_URL` (and `ETV_GITEA_TOKEN` before `GITEA_TOKEN`),
because its other caller is a developer Mac using the ETV_* convention. Setting only the
GITEA_* names meant a runner exporting a stale ETV_GITEA_URL would enumerate a DIFFERENT
Gitea instance and this job would post a verdict here from a diff read there. Both names
are now set to the same value, so precedence cannot matter.

**Three guards passed their tests for the wrong reason.** Each was confirmed by deleting
the clause and watching the suite stay green — the reviewer asserted it, mutation proved
it:

- The explicit empty-response clause was uncovered on jq 1.8, because jq 1.8 rejects
  empty input by itself. jq 1.6 does not, and THE RUNNER SHIPS 1.6 — so the one
  environment where the clause is load-bearing had no coverage. That is the #643/#647
  failure class reproduced inside the suite meant to prevent it. Now covered by importing
  the existing jq-1.6 shim (imported, not copied — a second quirk emulator is the same
  drift problem one level down), with a verify-the-verifier test and a positive control.
- `type == "array"` needed a body whose VALUES are valid rows. Two earlier attempts
  failed for a third reason: `jq`'s `all(.[]; …)` iterates an object's values, so
  `{"message":"…"}` and a single flat row are both rejected by `.filename` erroring on a
  string. Only `{"0": {…valid row…}}` reaches the fail-open, where a non-array body
  enumerates as a complete docs-only list.
- `.filename | ok` is now isolated by a row carrying a valid `.status` and no filename,
  removing the closed-allow-list as a second reason to reject.

**Two assertions proved less than their names claimed.** `"jq-preflight.sh" in code` also
matched the `[ -x … ]` presence guard, so deleting the invocation left it green; it now
requires an invoking line. `_run_classify` accepted every POST, so a status aimed at the
wrong endpoint or sha would not have been noticed; it now asserts the POST lands on
`/statuses/<full head sha>`.

**One test name overclaimed** and is narrowed rather than left implying coverage it does
not have: the head-movement test proves "final head != expected sha", not movement
*during* enumeration.

Deferred, both pre-existing and neither introduced here — filed as follow-ups:
- A commit status is repo-global, so a `review-verdict/h10=success` obtained for head H
  on one PR is inherited by any other PR with the same head, including one opened against
  a different base. Same class as #632, reached by a third route.
- The A->B->A force-push race: paging is several round-trips and the head is re-read once
  at the end, so a restore to the original sha passes the binding while the pages came
  from two states. Inherent to enumerating a mutable list over an API with no
  commit-pinned files endpoint.

Refs #649
2026-07-26 23:23:52 +02:00
timothy f0f8708a6e fix(632): fail closed when the head/base re-read itself fails
Self-review of the previous commit. Folding the head and base re-reads into one
`prjson_now=$(api_get ... || true)` swallowed a guard that used to be implicit: the old
`sha_now=$(api_get ... | jq ...)` aborted under `set -e` + `pipefail` when the GET
failed, before any status was written. With `|| true`, both `sha_now` and `base_now`
come back empty, both `[ -n ... ]` guards no-op, and the status is written having
confirmed nothing about either the head or the base — a fail-open regression introduced
by the refactor itself.

Confirmed the old behaviour empirically rather than by reading it: a failed piped command
substitution under `set -euo pipefail` exits with curl's status.

The refusal is now explicit, and pinned by a test — nothing asserted it before, which is
exactly why the refactor could drop it silently. Mutation-verified: restoring `|| true`
reddens that test alone.

Refs #632
2026-07-26 23:16:35 +02:00
timothy 00e623c066 fix(632): bind a review verdict to its BASE branch, not only to its head sha
#622 made `review-verdict/h10` a per-sha required status, so a new commit cannot
inherit an old verdict — the required context is simply absent on the new head.
Retargeting a PR's base reaches the same end from the opposite direction: the head sha
and the status both hold still while the merge-base, and therefore the effective diff
the verdict was formed against, changes underneath them. #622's record claimed the
invariant holds "by construction"; this was the documented exception, and an unrecorded
exception is how a guarantee degrades into a habit.

`post-review-verdict.sh` now records the base branch in the status description as a
trailing `(base: <ref>)`, and refuses to write a status at all if the base moved between
reading the PR and posting — the same TOCTOU window the head check already covers, which
the head check cannot see because retargeting does not move the head.
`pretooluse-merge-consent.sh` reads the field back and denies when it no longer matches
the PR's live `base.ref`.

Two choices are load-bearing, and each is pinned by a test rather than left to a comment:

- The comparator is `base.ref`, NOT `base.sha`. `base.sha` tracks the base branch's tip,
  which moves whenever anything merges to `main` — comparing it would invalidate every
  open verdict on every unrelated merge, converting a rare-event guard into a permanent
  merge deadlock. A base that merely advances is out of scope by design: rebasing onto
  it moves the head sha, which the per-sha binding already covers.
- The field goes in the status DESCRIPTION, not the verdict comment. The comment body is
  parsed by `scripts/check-review-verdict.sh`, whose grammar had three false-opens in its
  history (#629); nothing parses the description, so this adds a field without reopening
  that surface.

Scope is stated honestly rather than overclaimed: this is DETECTION on the hook path
only. A commit status carries no base of its own, so the server-side required check
cannot see a retarget, and a merge driven through the Gitea UI or API is unaffected. That
is the accepted exposure — base changes are rare, manual, and this is a two-account repo
— but it now fails loud in the one place that evaluates consent, instead of living only
in a doc.

Verdicts posted before this change carry no `(base: …)` and get NO opinion rather than a
deny; denying would block every in-flight PR the day it lands, and the window closes on
its own since verdicts are per-head and short-lived.

Verified by mutation, six mutants, each killed by its intended test: remove the hook's
deny; compare base.sha instead of base.ref; drop graceful adoption; stop recording the
base; drop the TOCTOU guard; accept a PR with no resolvable base. The positive controls
matter more than usual here — the test PR is deliberately non-docs (a docs-only PR
short-circuits the whole gate and would never reach the base check) and the rest of the
gate is unstubbed, so "the hook denied" alone proves nothing.

Refs #632

Decisions-Edit: yes
2026-07-26 23:13:50 +02:00
timothy 9114a7e8af fix(649): point the ENFORCED review-verdict gate at the shared PR-file enumeration
#658 landed the shared implementation, `scripts/pr-changed-files.sh`, and rewired the
ADVISORY hook onto it. The ENFORCED copy — the one that writes the branch-protection-
required `review-verdict/h10` status — was left byte-identical to main, so its
fail-closed behaviour on a malformed or empty response stayed INCIDENTAL: an empty `n`
erroring `[ "$n" -lt 50 ]` to false. That is #649's second Done-when box, and the whole
point of the issue was that the gate with real authority was weaker than the gate with
none.

`review-verdict.yml` now:

- checks out the PR's BASE ref (`base.sha`, `persist-credentials: false`), never the
  head, so a PR cannot supply the code that judges it;
- runs `scripts/jq-preflight.sh` in FLOOR-ONLY mode — `--expect` here would deadlock
  every merge on `main` the day the runner's jq changes;
- calls `scripts/pr-changed-files.sh` and reads its EXIT STATUS, never its stdout on a
  failure path. The env trap flagged in review is handled: the script reads
  GITEA_BASE_URL and takes owner/repo as two separate arguments, so passing BASE_URL and
  a combined `owner/repo` would have silently fallen back to the hardcoded LAN default.

The ~40 lines of inline enumeration are deleted, so the two copies can no longer drift.
A base ref predating #658 has no such script; that posts `pending` with the reason
rather than dying with no status at all.

The drift guard is re-tightened from "the hook uses the shared script" to "BOTH callers
do", and the workflow's own preconditions are pinned by parsing the YAML rather than
substring-matching it — `head.sha` for `base.sha` is a nine-character diff.

Verified by mutation, six mutants, each killed by its intended test: ignore the exit
status; check out the head; drop `persist-credentials`; add `--expect`; re-inline a
`pulls/N/files?` fetch; delete the PROTECTED clause.

That last one initially MISSED, and the miss was the useful finding. The test used a
docs-only-plus-protected file list and passed with the clause deleted, because
PROTECTED (`.claude/ .gitea/ .husky/ scripts/ docker/ci/`) and DOCS_ONLY (`docs/`, root
`*.md`) are disjoint — on the docs-only path that clause can never fire, and DOCS_ONLY
was doing all the work. PROTECTED is load-bearing only on the BOT path, so the test now
covers a Renovate PR editing the shared script, with a positive control proving the bot
exemption fires at all.

The caller contract is tested by EXECUTING the workflow's `run:` block against a stubbed
enumeration that fails while emitting a perfectly docs-only list — the one combination
the "every failure path also happens to print nothing" redundancy cannot absorb, and the
exact mutation that survived the whole suite last round.

Docs: both "Landing note" blocks removed, and the record's base-ref paragraph converted
from a future-tense requirement to present-tense fact with its staging rationale kept as
history.

Refs #649

Decisions-Edit: yes
2026-07-26 23:05:14 +02:00
timothy 8103e34fff Merge pull request 'fix(648): an explicit jq version contract + one shared PR-file enumeration' (#658) from fix/648-649-jq-gates into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m41s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 16m49s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 18m37s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m11s
2026-07-26 20:46:04 +00:00
timothy 256cb0221b Merge pull request 'docs(ersatztv skill): record the #510 no-logo-no-bug policy and deco seeding recipe' (#659) from docs/510-skill-logo-bug-policy into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 15s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 29s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 30s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 19s
2026-07-26 20:41:55 +00:00
timothy 3684fd7ef6 Merge pull request 'docs(505): retire the stale "do NOT set QSV on jazz" rule in the ersatztv skill' (#657) from docs/505-skill-qsv-stale into main
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 13s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 24s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 27s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 15s
2026-07-26 20:34:21 +00:00
timothy b255b7ffdc test(648): close the mutation gaps round 5 found — two tests passed for the wrong reason
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Successful in 17s
Review verdict / Set review-verdict status (pull_request) Successful in 31s
PR Gates / Script tests (pytest) (pull_request) Successful in 35s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m59s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 16m24s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 22m27s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ b255b7f
Round 5 could not break the predicate itself: 28,930 real runs of the script across
14,465 crafted --version strings on bash 3.2.57 and 5.3.15 produced zero fail-opens, and
`{1,9}` is honoured on bash 3.2, so round 4's bound is not void on the authoring Macs.

What it did find is that two of round 4's changes were unpinned, and the tests that
looked like they covered them did not.

Reverting BOTH the first-line slice and `[[:blank:]]`→`[[:space:]]` together left the whole
suite green. The four filler cases are all killed by the SEPARATOR restriction alone, so
they attributed the fix to the wrong layer. Added three cases carrying the literal word
`version` (`jq\nversion\n9.9` and friends), which satisfy the separator rule and can only be
stopped by confining the parse to line one with a newline-free blank class.

The CR-strip test was worse: vacuous through two independent mechanisms. `str.splitlines()`
also splits on `\r`, so a per-line view dropped the stray CR; and `subprocess.run(text=True)`
translates `\r` to `\n` outright, so even a raw-string check on stdout was unfalsifiable.
The mutant demonstrably emits `... = jq-1.6<CR> (parsed 1.6; ...)` at the byte level while
the test reported green. Added `run_bytes()` and a bytes comparison.

Both gaps are now mutation-verified: reverting either change reddens exactly its own test.

Also records the operational edge this parser acquires in the follow-up: it is strictly
fail-closed by design, so once the floor mode gates the required check, a jq wrapper that
prints a banner line would deadlock merges. The fix there is to widen the accepted forms,
never to relax fail-closed.

Decisions-Edit: yes
2026-07-26 22:21:07 +02:00
timothy 807ebbd38e fix(648): round 4 — the round-1 fail-open was still reachable, via an over-long number
Round 4 found the round-1 MECHANISM alive in round 3's regex. The pattern guaranteed the
operands were digits but not that they fit `test`'s integer range, so a 23-digit major made
`[ "$major" -lt "$min_major" ]` error with "integer expression expected" — and `set -e`
exempts a failing command in an `if` condition, so the conditional read false and the floor
was never asserted. Exit 0. That is precisely what the empty string did in round 1: same
shape, third occurrence, same predicate.

Bounding the runs with {1,9} was not sufficient on its own. The pattern is unanchored at the
end, so `jq-1.99999999999999999999999` simply matched the first 9 digits of the minor and
compared THAT — a mis-parse that passes the floor rather than an error that skips it. The
trailing `([^0-9]|$)` is what actually closes it.

Second hole: `[[:space:]]` matches NEWLINES, so round 3's "anchor" still scanned the whole
output. `jq\n2.34: cannot load shared library` matched `jq`, crossed the newline as
separator, and parsed 2.34. Now the first line only, with `[[:blank:]]`.

Third: the separator class `[-[:blank:]]{1,4}` could be walked across filler —
`jq -- 2.34 (real jq-1.6)` parsed as 2.34, `jq<TAB><TAB>9.9` as 9.9. It is now one of the two
forms real jq emits: `jq-1.6` or `jq version 1.6` (a blank separator REQUIRES the literal
word `version`).

Verified across a 20-case matrix: every legitimate form still parses to the right numbers
(jq-1.6, jq version 1.6, jq-1.7.1, jq-1.6-dirty, jq-1.10 numerically, jq-1.6 (Debian 1.6-2.1),
jq-v1.6, JQ-1.6, jq-1.6.0, CRLF), and every constructed attack fails closed. Four mutations,
each reddening exactly its own tests. The real jq 1.8.2 on this machine still reports cleanly.

Also: the log line now interpolates the first line, so a multi-line --version cannot split the
single grep-able line the no-arg mode exists to emit.

None of these are reachable from a real jq build. They are recorded and fixed because the
guard's own stated invariant — never assert a floor against something it did not parse — was
still violable three rounds in, and the follow-up PR moves this exact code into the
branch-protection-required check.
2026-07-26 22:21:07 +02:00
timothy 4e094637c6 fix(648): the version parser was fail-OPEN on a jq that cannot start
Round 3, and it found that round 2's fix was a REGRESSION on the case that matters most.

`raw=$(jq --version 2>&1 || true)` did two wrong things at once: folded stderr into the
parse input and discarded the exit status. Combined with a pattern that matched the first
<digits>.<digits> ANYWHERE, a jq broken by a glibc mismatch — which exits 127 and writes
"version `GLIBC_2.34' not found" to stderr — parsed as version 2.34 and PASSED the floor.
The strip-based parse this replaced failed CLOSED there. So the fix for a fail-open bug
introduced a worse fail-open bug, in the one script whose entire purpose is to refuse to
certify a version it did not parse.

Same mechanism, second symptom: an unanchored match let a prefix outrank the real version.
`2026.07.26 jq-1.6` parsed as 2026.07; a leading warning line carrying any number won too.

Now: jq's exit status is captured explicitly (`$?` inside `if ! cmd` is the NEGATED status,
so that needed care too), stderr is kept out of the parse, and the pattern is anchored to
the leading `jq` token. Every legitimate form still parses — `jq-1.6`, `jq version 1.6`,
`jq-1.7.1`, `jq-1.6-dirty`, `jq-1.6 (Debian 1.6-2.1)`, `jq-1.10` (numeric compare, so the
two-digit minor is not read lexically).

The tests could not have caught any of this: the shim always exited 0 and never wrote to
stderr, so every case it could express was clean. It now takes stderr and an exit code, and
the four new cases turn red under the exact mutation.

Also: the drift guard now strips comment lines before matching. A future comment citing
`pulls/$pr/files?limit=100` as an example of what not to do would otherwise have reddened
script-tests — which, per this branch's own correction, blocks merges.

And the record no longer over-corrects: the combined-status read is guarded by
`if [ "$mwcs" != "true" ]`, so a red script-tests blocks the hook-mediated merge path, not
literally every merge.

Decisions-Edit: yes
2026-07-26 22:21:07 +02:00
timothy 5e7623b8d5 fix(648,649): security-review round 2 — close the version-parse hole and the untested caller contract
Two real defects, and three docs claims that were simply wrong.

jq-preflight.sh parsed the version by stripping around the first `-` and `.`, which
assumed the format is exactly `jq-X.Y`. A build printing `jq version 1.6` left major
empty; the sanity check concatenated major+minor into "6", which is non-empty and
all-digits, so it PASSED. The floor comparison then ran `[ "" -lt 1 ]`, which errors —
and `set -e` exempts a failing command in an `if` condition, so the conditional read
false and the script exited 0 having asserted nothing, after printing a plausible
"parsed" line. The silently-untested-axis failure this script exists to eliminate,
reproduced inside the script itself. Now parsed by explicit regex, failing closed with a
diagnosis when there is no <digits>.<digits> match. Also: `--expect` with no value exited
1 with empty output on both streams.

The hook's exit-status check was pinned by nothing: mutating `if files=$(...)` into
`files=$(...) || true; files_complete=yes` left the ENTIRE suite green. It survived only
by redundancy — the script writes stdout once, right before exit 0, so failures also
happen to yield empty stdout and `[ -n "$files" ]` catches it. Safe by accident, which is
the exact criticism this branch levels at the old code. Four tests now pin it, with a
stub that FAILS while emitting a docs-only list (the one case redundancy cannot absorb)
plus a positive control proving the harness can see the difference. Verified: the
mutation now turns exactly those tests red.

Docs corrections. The record claimed the --expect pin was safe because script-tests is
"advisory, not a required check" — false. The merge-consent hook reads the COMBINED
status (ci.advisory-red-blocks-the-merge-gate, #598), so firing the tripwire blocks every
non-docs-only merge until someone re-pins. Kept anyway, for a stated reason, but no
longer described as free. The record also asserted in the present tense that
review-verdict.yml checks out the base ref; it has no checkout step at all, so that is
now a future-tense requirement on the follow-up. And the documented .status allow-list
named GitHub's `removed`, which the code rejects.

The drift-guard regex anchored on `?limit=`, so a re-inlined copy written
`files?page=1&limit=50` would have walked past it.

Decisions-Edit: yes
2026-07-26 22:21:07 +02:00
timothy 2c10f057b8 fix(648,649): stage the enforced-gate wiring behind the scripts it calls
Splits the review-verdict.yml rewiring out of this PR. That workflow checks out the
PR's BASE ref — deliberately, so a PR cannot rewrite the gate that judges it — and the
base is main, which does not yet contain scripts/pr-changed-files.sh or
scripts/jq-preflight.sh. Wiring it here would make the job exit 127 on its own PR and
block the merge gate through the combined status, which reads red jobs as blocking.

So this PR lands the scripts, their tests, the hook rewiring and the script-tests jq
pin; the follow-up points review-verdict.yml at them once they exist on main.

The two tests that asserted on review-verdict.yml are scoped accordingly, each carrying
the reason. test_review_verdict_never_pins_a_jq_version is asserted NOW rather than in
the follow-up, so the no-pin constraint on the required check is already enforced when
the wiring lands.

Decisions-Edit: yes
2026-07-26 22:21:07 +02:00
timothy 63fa81fbb5 docs(648,649): the jq contract + the shared PR-file enumeration record
Adds docs/ci-cd.md "The jq contract" (1.6 floor, the three divergent constructs,
and the deliberate pin-vs-floor asymmetry with its merge-deadlock reason), plus two
decision records: ci.jq-version-contract and ci.shared-pr-file-enumeration.
ci.script-tests-job stops restating the three jq rules and points at the new record.

Also corrects the script-tests preflight description: it is now two steps (git
presence, then jq VERSION via scripts/jq-preflight.sh --expect 1.6), not one.

A literal NUL byte had crept into the ci-cd.md paragraph describing jq 1.6's NUL
truncation — which git treats as a binary file. Replaced with the literal text.

Decisions-Edit: yes
2026-07-26 22:21:07 +02:00
timothy 2fd798cccf fix(648,649): one shared PR-file enumeration + an explicit jq version contract
#649 — the enforced review-verdict.yml guard had drifted strictly WEAKER than the
advisory merge-consent hook: four rounds of #643 hardening landed on the copy whose
failures produce a human prompt, and never reached the copy that writes the
branch-protection-required review-verdict/h10 status. Its fail-closed behaviour on a
garbage response was also incidental (an empty `n` erroring a bash conditional to
false), not designed.

Extract scripts/pr-changed-files.sh as the single implementation both call. Shared
MECHANISM, not policy: the two docs-only allow-lists differ deliberately and stay
separate. review-verdict.yml now checks out the BASE ref, never the PR head, so a PR
cannot rewrite the gate that judges it.

#648 — baking jq into docker/ci/Dockerfile provably cannot cover the gate that broke:
review-verdict.yml is runs-on:small with no toolchain pin, so it gets the host's jq 1.6
(checked, not assumed). Add scripts/jq-preflight.sh: floor+observable everywhere, and a
--expect tripwire on script-tests only — pinning the required merge check would deadlock
every merge on a jq bump.

Verified by mutation: six guards individually broken, each turning exactly its own test
red, then restored byte-identical.

fixes #648
fixes #649
2026-07-26 22:21:07 +02:00
timothy 06e8181dee docs(505): retire the stale "do NOT set QSV on jazz" rule in the ersatztv skill
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Successful in 19s
Review verdict / Set review-verdict status (pull_request) Successful in 3s
PR Gates / Script tests (pytest) (pull_request) Successful in 37s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 8s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 8s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 8s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
review-verdict/h10 Review-verdict: MERGEABLE @ 06e8181
The skill told sessions to keep jazz on HardwareAcceleration=3 (Vaapi) and "do
NOT set QSV", citing the 2026-07-20 cold-start regression and the fact that one
column governed both decode and encode. #498 fixed exactly that by adding
QsvPreferNativeDecoder (VA-API decode + QSV encode, the Jellyfin split), and
prod has run that way since.

Verified live on jazz 2026-07-26: the single FFmpegProfile used by all 43
channels is HardwareAcceleration=1 (Qsv), QsvPreferNativeDecoder=1,
QsvExtraHardwareFrames=64. A session following the old note would have
"corrected" a working prod profile back to VAAPI.

Also records the two QSV traps already paid for in code so they are not
re-derived: the extra_hw_frames=0 zero-segment failure (#523/#529) and the
vpp_qsv=tonemap silent no-op (#505), including that the same tonemap trap
applies to Jellyfin's EnableVppTonemapping on this host.

refs #505
2026-07-26 22:02:54 +02:00
95 changed files with 12068 additions and 831 deletions
+121 -103
View File
@@ -85,110 +85,58 @@ sha=$(printf '%s' "$prjson" | jq -r '.head.sha // ""' 2>/dev/null || true)
body=$(printf '%s' "$prjson" | jq -r '.body // ""' 2>/dev/null || true)
# --- Docs-only exemption: if every changed file is docs/process, skip the gate. ---
# The file list must be enumerated EXHAUSTIVELY or the exemption is unsafe. Gitea caps this
# endpoint at 50 rows per page and silently ignores a larger `limit` (verified: PR #619 has 194
# changed files and `?limit=100` returns exactly 50), so the previous single-page read could see 50
# docs files, miss the code in positions 51+, and exempt a PR that is not remotely docs-only.
# Page until a short page proves the end; anything else leaves `files_complete=no`, which withholds
# the exemption and falls through to the full gate (ersatztv#622).
files=""; files_complete=no; page=1
while [ "$page" -le 40 ]; do
raw=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=50&page=$page")
# A transport/parse failure must not look like a legitimate short final page: `gq` returns empty
# on any error, which counts as zero rows and would set files_complete=yes over a PARTIAL list —
# failing OPEN into the exemption.
#
# Checking only the top-level type leaves the same hole one level down: `[{}]` is a valid array
# whose rows carry no `filename`, so it yields no paths, looks like a short page, and completes
# the enumeration from a partial list. Require every row to carry a non-empty string `filename`
# (an empty array is still valid — that is a genuine end-of-pagination). This also rejects arrays
# of scalars, which would otherwise make the `.filename` extraction below fail under `set -e`.
#
# An EMPTY body is rejected EXPLICITLY here rather than left to jq's exit status, because that
# status is not portable: `jq -e` over empty input exits 4 on jq >= 1.7 but **0 on jq 1.6**
# (verified against both binaries — ersatztv#631). Relying on it made this guard fail OPEN on any
# host with the older jq, including the CI runner, which ships jq 1.6. The chain: a transport
# failure makes `gq` return empty, the jq guard wrongly passes, `n` is empty so `[ "$n" -lt 50 ]`
# errors into false, the loop walks PAST the failed page, the NEXT page legitimately returns `[]`,
# and `files_complete=yes` is set over a PARTIAL list — exempting a PR whose unread pages may be
# pure code. That is the very defect the paragraph above describes, reintroduced one layer down.
if [ -z "${raw//[[:space:]]/}" ]; then
files_complete=no; break
fi
#
# CR/LF in a path is REJECTED outright (ersatztv#643 review). `chunk` below flattens paths into
# newline-delimited text, so a filename containing a newline splits into TWO lines that are each
# matched against the allow-list separately: `"safe.md\ndocs/Program.cs"` yields `safe.md` and
# `docs/Program.cs`, both of which pass, while the actual single path ends in `.cs`. Git permits
# newlines in filenames, so this is reachable, and it was reproduced against this hook. Failing
# closed on control characters is the cheap fix; no decision/docs path ever contains one.
#
# VALIDATE EVERY FIELD THE EXTRACTION BELOW CONSUMES. `chunk` emits `(.previous_filename //
# empty)` for EVERY row regardless of `.status`, so validating that field only on `renamed` rows
# left a hole one predicate wide: a row with `status: "modified"` (or Gitea's distinct `copied`)
# carrying a newline in `previous_filename` was reproducibly exempted. The rule this encodes:
# the validation domain must match the CONSUMPTION domain, not the domain the field is
# semantically "supposed to" appear in. The `renamed` => REQUIRED clause is kept on top of the
# unconditional if-present check.
#
# `..` is rejected for the same reason: the allow-list anchors `^docs/`, so
# `docs/../ErsatzTV/Program.cs` matches it. Git will not produce such a path, but this guard's
# whole job is to fail closed on unexpected 2xx shapes rather than to assume a well-behaved peer.
#
# `.status` is checked against a CLOSED set, verified against live Gitea 1.25.4 output:
# added|deleted|changed|renamed|copied. Without it, the `renamed => previous_filename REQUIRED`
# clause could be dodged by any other value — `"Renamed"` with a capital R, or an absent status —
# letting a `git mv ErsatzTV/Program.cs -> docs/a.md` drop its source path and read as docs-only.
# An unknown status now fails closed rather than silently taking the `else true` branch.
#
# `modified` is accepted ALONGSIDE `changed` deliberately. Live Gitea 1.25.4 emits `changed`, but
# a closed allow-list built from the wrong vocabulary is a worse failure than the hole it closes:
# it would gate every genuine docs-only PR, on every version that spells it differently. The
# security property here is "reject values we do not recognise", not "enumerate one version
# exactly", so the set errs toward accepting plausible synonyms.
if ! printf '%s' "$raw" \
| jq -e 'def ok: type == "string" and length > 0
and (test("[\\r\\n]") | not)
and (split("/") | index("..") | not);
type == "array" and all(.[];
(.filename | ok)
and (.previous_filename == null or (.previous_filename | ok))
and ((.status // "") as $s | ($s | type) == "string"
and (["added","deleted","changed","modified","renamed","copied"] | index($s)) != null)
and (if .status == "renamed"
then (.previous_filename | type == "string" and length > 0)
else true end))' \
>/dev/null 2>&1; then
files_complete=no; break
fi
# BOTH sides of a rename: Gitea reports a `git mv` as ONE row whose `filename` is the DESTINATION,
# with the source in `previous_filename`. Reading only `filename` would let a PR move code into
# docs/ and claim the docs-only exemption. Page size is measured in ROWS, not paths — one renamed
# row is one row but two paths.
n=$(printf '%s' "$raw" | jq -r 'length')
chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)')
[ -n "$chunk" ] && files=$(printf '%s\n%s' "$files" "$chunk")
# Terminate ONLY on an explicitly validated EMPTY page — never on a merely SHORT one
# (ersatztv#643 review). "Fewer than 50 rows means last page" assumes the server's page size is
# the 50 we asked for, but Gitea caps `limit` at the server-wide `MAX_RESPONSE_ITEMS` (default 50,
# configurable) and is free to return fewer. A 30-row page followed by a page of code would set
# files_complete=yes over a PARTIAL list — the same fail-open, reached without any transport error.
# Costs one extra request per enumeration; the `page <= 40` cap still fails closed.
if [ "$n" -eq 0 ]; then files_complete=yes; break; fi
page=$((page + 1))
done
files=$(printf '%s\n' "$files" | grep -v '^$' || true)
# Bind the enumeration to ONE head (ersatztv#643 review). Paging is several round-trips; a
# force-push between them means page 1 came from head A and page 2 from head B, so the assembled
# list belongs to no single commit — B's code page can be skipped entirely while B's docs page
# reads as a clean short tail. Re-read the head and refuse the exemption if it moved.
if [ "$files_complete" = yes ]; then
sha_after=$(printf '%s' "$(gq "repos/$owner/$repo/pulls/$pr")" | jq -r '.head.sha // ""' 2>/dev/null || true)
if [ -z "$sha_after" ] || [ "$sha_after" != "$sha" ]; then
files_complete=no
fi
# The file list must be enumerated EXHAUSTIVELY, validated row by row, and bound to ONE head, or the
# exemption is unsafe. ALL of that now lives in scripts/pr-changed-files.sh — the single shared
# implementation, also called by .gitea/workflows/review-verdict.yml (ersatztv#649).
#
# Why it moved: this logic was written twice. This copy is ADVISORY (a failure produces a human
# prompt); the workflow's copy is ENFORCED (it writes the branch-protection-required
# `review-verdict/h10` status). Four rounds of ersatztv#643 hardening landed here and never reached
# there, leaving the copy with real authority strictly weaker than the copy without — and its safe
# behaviour resting on a bash arithmetic error rather than an intentional guard. Two copies of a
# security predicate drift; one cannot.
#
# What is NOT shared, deliberately: the docs-only allow-list below. This one also lets .claude/,
# .gitea/ and .husky/ through, which is safe HERE only because a match falls through to a human
# prompt rather than auto-granting. The workflow's list is narrower for exactly that reason. Sharing
# the enumeration fixes the drift; sharing the classification would erase an intended difference.
#
# A non-zero exit means "could not tell" and MUST withhold the exemption — never read stdout without
# checking the status. An empty `$sha` (unparseable PR JSON) reaches the script as an empty argument
# and is rejected there, so that path also fails closed.
#
# The 5th argument binds the enumeration to a base branch (ersatztv#698 route 1), because
# `/pulls/{n}/files` diffs against the PR's LIVE base and retargeting moves that without moving the
# head. Be precise about what it buys HERE, which is less than what it buys in the workflow: the
# workflow passes the base from a `pull_request_target` event payload, fixed at event time and beyond
# a retarget's reach, so it detects a retarget outright. This hook has no such trusted snapshot — it
# passes the base it just read from the live PR, so what it asserts is that the base did not move
# between that read and the enumeration. Narrower, and still worth having: without it the hook cannot
# tell a mid-flight retarget from an honest read at all. An empty/unparseable `.base.ref` reaches the
# script as an empty argument and is rejected there, so that path fails closed too.
base_ref=$(printf '%s' "$prjson" | jq -r '.base.ref // ""' 2>/dev/null || true)
repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
files=""; files_complete=no
if files=$("$repo_root/scripts/pr-changed-files.sh" "$owner" "$repo" "$pr" "$sha" "$base_ref" 2>/dev/null); then
files_complete=yes
fi
if [ "$files_complete" = yes ] && [ -n "$files" ] && ! printf '%s\n' "$files" | grep -qvE '^(docs/|\.claude/|\.husky/|\.gitea/|.*\.md$)'; then
# HOW THIS PREDICATE IS EVALUATED, matching the enforced gate (ersatztv#698,
# `ci.grep-q-pipefail-inversion`). `printf … | grep -q` INVERTS under `set -o pipefail`: grep -q exits
# at its first match, printf then takes SIGPIPE (141), and a MATCH is reported as a failed pipeline —
# so this negated test would grant a spurious docs-only exemption for any PR whose path list exceeds
# the pipe buffer. A here-string fixes that but is materialised via temporary storage for large inputs,
# so it can fail when temp space is full or unwritable and flip the predicate the same way. Counting
# with `grep -c` drains stdin (no SIGPIPE) over an ordinary pipe (no temp file); `grep -c` exits 1 for
# a zero count, which is a legitimate answer, so only a status >1 is a real error and is treated as
# "cannot tell" -> no exemption.
# Advisory here, so the blast radius is a missing prompt rather than a green required check; the
# construct is identical on purpose, because the two copies drifting is what ersatztv#649 was about.
docs_nonmatching=$(printf '%s\n' "$files" | grep -cvE '^(docs/|\.claude/|\.husky/|\.gitea/|.*\.md$)') || docs_grep_status=$?
if [ "${docs_grep_status:-0}" -gt 1 ]; then
docs_nonmatching=1 # grep itself failed: cannot tell, so withhold the exemption
fi
if [ "$files_complete" = yes ] && [ -n "$files" ] && [ "${docs_nonmatching:-1}" -eq 0 ]; then
# Docs/process-only PR: the Done-when + review-verdict gate doesn't apply — but this exemption is a
# file-TYPE bypass, NOT the a+b+c "provably reviewed & ready" proof, so it does NOT auto-grant. It
# passes through to normal permissioning (one prompt). This deliberately keeps a human in the loop for
@@ -198,6 +146,76 @@ if [ "$files_complete" = yes ] && [ -n "$files" ] && ! printf '%s\n' "$files" |
decide allow "" # passthrough (exit 0 → normal prompt), NOT grant
fi
# --- Base-change detection: a verdict is bound to a head AND to a base (ersatztv#632). ---
# `review-verdict/h10` is per-sha, which makes "the head moved under a fixed verdict" impossible by
# construction. Retargeting a PR's base is the mirror case and slips through: it changes neither the
# head sha nor the status, so a verdict formed while the PR targeted `main` still reads green after
# the PR is pointed at a branch with a very different merge-base. The diff moves while the verdict
# and the head both hold still.
#
# DETECTION, NOT PREVENTION, and only on this path. A commit status carries no base, so the
# server-side required check cannot see this; a merge driven through the Gitea UI or API is
# unaffected. That is the accepted exposure — base changes are rare, manual, and this is a
# two-account repo — but it is now recorded in a place that fails LOUD rather than only in a doc.
#
# GRACEFUL ADOPTION, mirroring (b) and (c): a description with no `(base: …)` field is a verdict
# posted before ersatztv#632 and gets NO opinion, rather than denying every in-flight PR the day
# this lands. The window closes on its own — verdicts are per-head and short-lived, so every verdict
# posted after this carries the field.
# "Could not check" is a THIRD outcome, distinct from both "matches" and "no base recorded". Cold
# review found the first draft collapsing it into the latter: an unreadable status response yielded
# an empty `recorded_base`, which took the graceful-adoption path and skipped validation silently —
# after which a later, successful status read could still auto-grant. A transient failure would then
# have produced a "merge gate: satisfied" message for a comparison that never happened. Every
# unreadable input here therefore falls through to a human (`ask`), never to silence.
live_base=$(printf '%s' "$prjson" | jq -r '.base.ref // ""' 2>/dev/null || true)
if [ -z "$live_base" ]; then
decide ask "H10 merge gate: PR #$pr reports no base branch (.base.ref), so the verdict cannot be checked against the branch it was formed for (ersatztv#632). Confirm the PR still targets the branch it was reviewed against before merging."
fi
if [ -n "$sha" ]; then
# This is the THIRD read of this endpoint in a worst-case hook run (the ordinary-CI branch and the
# scheduled-auto-merge branch each do their own). Sharing one snapshot would close a narrow
# same-run window where two reads disagree, but the later branches derive different decisions from
# a failed read than this one does, so threading a shared response through them is a change to
# pre-existing logic rather than to ersatztv#632's. Left deliberately, noted so it is not
# rediscovered as an oversight: every `decide` exits immediately, so the reads cannot produce a
# single self-contradictory message — only a later decision made on a fresher snapshot.
vjson_base=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100")
# Same jq-1.6 rule as everywhere else in this file: check emptiness in SHELL first, never via
# `jq -e`'s exit status over empty input.
# VALIDATE EVERY FIELD THE EXTRACTION CONSUMES, on EVERY row — the same rule the file-enumeration
# guard learned the hard way. Checking only that `.statuses` is an array left a hole one level
# down: `{"statuses":[1]}` passes a top-level type check, then `.context` on a number errors, and
# a `|| true` on the extraction turned that error into an empty `vdesc` — i.e. straight back onto
# the graceful-adoption path this block exists to distinguish from. That is the identical
# swallow-the-error shape fixed a few lines up, surviving one level deeper.
if [ -z "${vjson_base//[[:space:]]/}" ] \
|| ! printf '%s' "$vjson_base" \
| jq -e '.statuses | type == "array"
and all(.[]; type == "object"
and (.context | type == "string")
and (.description == null or (.description | type == "string")))' \
>/dev/null 2>&1; then
decide ask "H10 merge gate: could not read the commit statuses for PR #$pr head ${sha:0:7}, so the verdict could not be checked against the PR's base branch (ersatztv#632). Confirm the review covered the branch this PR currently targets ('$live_base') before merging."
fi
# No `|| true` here. The validation above makes an error unreachable, but a swallowed error would
# be indistinguishable from "no base recorded" — the exact confusion this block removes — so the
# failure is handled explicitly rather than left to a fallback that reads as a benign result.
if ! vdesc=$(printf '%s' "$vjson_base" \
| jq -r '[.statuses[] | select(.context == "review-verdict/h10")] | first | .description // ""' \
2>/dev/null); then
decide ask "H10 merge gate: the commit statuses for PR #$pr head ${sha:0:7} could not be parsed to find the review verdict, so it could not be checked against the PR's base branch (ersatztv#632). Confirm the review covered the branch this PR currently targets ('$live_base') before merging."
fi
# The field is written by scripts/post-review-verdict.sh as a trailing `(base: <ref>)`. Its
# ABSENCE is the one benign case: a verdict posted before ersatztv#632 could not have carried it,
# and denying those would block every in-flight PR the day this lands. The window closes on its
# own, since verdicts are per-head and short-lived.
recorded_base=$(printf '%s' "$vdesc" | sed -n 's/.*(base: \(.*\))$/\1/p')
if [ -n "$recorded_base" ] && [ "$recorded_base" != "$live_base" ]; then
decide deny "H10 merge gate: BLOCKED — the review verdict on head ${sha:0:7} was formed while PR #$pr targeted '$recorded_base', but it now targets '$live_base'. Retargeting a base does not move the head sha, so the per-sha verdict status still reads green even though the effective diff has changed (ersatztv#632). Re-review against the new base and run: scripts/post-review-verdict.sh $pr MERGEABLE"
fi
fi
# --- Linked issue: Gitea auto-close keywords in the PR body. ---
issues=$(printf '%s' "$body" | grep -ioE '(close[sd]?|fix(e[sd])?|resolve[sd]?) +#[0-9]+' | grep -oE '[0-9]+' | sort -u || true)
[ -n "$issues" ] || decide ask "H6 merge gate: PR #$pr has no linked issue (no 'fixes #N' / 'closes #N' in its body), so there is no Done-when checklist to derive consent from. Confirm the work is complete + reviewed, then approve."
+6 -3
View File
@@ -331,12 +331,15 @@ docker start ersatztv
## FFmpeg & Hardware
- **VAAPI on Intel (iHD)** hardware acceleration — ErsatzTV runs on **jazz** (i7-10700K, Intel iGPU) since #633. The single `FFmpegProfile` row (`Id = 1`, referenced by all 43 channels) has `HardwareAcceleration = 3` (Vaapi), `VaapiDevice = /dev/dri/renderD128`, `VaapiDriver = 0` (auto → iHD), `VaapiDisplay = drm`.
- **Do NOT set QSV (1) here, despite the Intel hardware.** It was tried on 2026-07-20 and **regressed**: QSV's *decoder* is far stricter than VAAPI about malformed NAL units and failed **3 of 6 channel cold-starts** (`Error splitting the input into NAL units` → `dec:h264_qsv Error while opening decoder: Invalid data found`). ErsatzTV has a **single** `HardwareAcceleration` column governing *both* decode and encode, so it cannot express Jellyfin's working combination of VAAPI-decode + QSV-encode. Tracked upstream: timothy/ersatztv#498. Jellyfin **does** use QSV successfully, because it splits the two.
- **QSV encode + VA-API decode on Intel (iHD)** — ErsatzTV runs on **jazz** (i7-10700K, Intel iGPU) since #633. The single `FFmpegProfile` row (`Id = 1`, referenced by all 43 channels) has `HardwareAcceleration = 1` (**Qsv**), `QsvPreferNativeDecoder = 1` (ON), `QsvExtraHardwareFrames = 64`, `VaapiDevice = /dev/dri/renderD128`. Verified live 2026-07-26. The profile is still *named* "1080p VAAPI h264 aac" — cosmetic, ignore the name.
- **The old "do NOT set QSV" rule is RETIRED — #498 fixed the blocker it was based on.** The 2026-07-20 regression was real (QSV's *decoder* is far stricter than VAAPI about malformed NAL units and failed 3 of 6 cold-starts: `Error splitting the input into NAL units`), and the stated cause was that one `HardwareAcceleration` column governed both decode and encode. **#498 added `QsvPreferNativeDecoder` (default ON, Linux-only)**, which splits them exactly like Jellyfin: decode with the tolerant VA-API decoder, encode with QSV. That is what prod runs now. Do not "fix" prod back to `3` (Vaapi) on the strength of the old note.
- **Two QSV traps already paid for, both fixed in code — don't re-derive them:**
- `QsvExtraHardwareFrames` must never be `0`: the software→QSV `hwupload` bridge has no headroom and the transcode writes **zero segments** on any unthrottled read (#523/#529). Code now floors it at 64 (`ffmpeg.qsv-extra-hw-frames-floor`).
- **HDR tonemapping never uses `vpp_qsv=tonemap`** — on this Gen9.5 iGPU that filter is a *silent no-op* (byte-identical output, exit 0, no warning), so it looked like GPU tonemapping while doing nothing. ErsatzTV now tonemaps via VA-API→OpenCL (#505, `ffmpeg.qsv-hdr-tonemap-opencl`). Same trap applies to Jellyfin's `EnableVppTonemapping` on this host — keep it off.
- Fallback if VAAPI also misbehaves (see #631, VAAPI `hwupload -22` on 10-bit): `HardwareAcceleration = 0` (software). jazz has 16 threads at load ~2, so it is affordable and maximally tolerant of imperfect sources.
- Resolution: 1920x1080, H264, AAC stereo
- Device: `/dev/dri` passed through (`renderD128`)
- HardwareAccelerationKind: 0=None, 1=Qsv, 2=Nvenc, 3=Vaapi, 4=VideoToolbox, 5=Amf — **use 3 (Vaapi)** on jazz (not Qsv — see above)
- HardwareAccelerationKind: 0=None, 1=Qsv, 2=Nvenc, 3=Vaapi, 4=VideoToolbox, 5=Amf — **jazz uses 1 (Qsv)** with `QsvPreferNativeDecoder` ON (see above)
- jazz's iGPU is shared with Jellyfin only (Frigate stayed on bumblebee); render GID is 992 on both hosts, so `group_add: '992'` carried over unchanged
## Jellyfin Integration
+17 -7
View File
@@ -239,14 +239,24 @@ jobs:
# as ~20 opaque assertion errors — this turns that into one actionable line.
- name: Preflight external tools
run: |
missing=()
for t in jq git; do command -v "$t" >/dev/null 2>&1 || missing+=("$t"); done
if [ ${#missing[@]} -gt 0 ]; then
echo "::error::script-tests needs these on PATH but they are absent: ${missing[*]}." \
"The suite execs real shell scripts that use them. Bake them into the runner" \
"image rather than apt-get installing here (see ersatztv#390)."
if ! command -v git >/dev/null 2>&1; then
echo "::error::script-tests needs git on PATH but it is absent. The suite execs real" \
"shell scripts that use it. Bake it into the runner image rather than apt-get" \
"installing here (see ersatztv#390)."
exit 1
fi
echo "Preflight OK: $(jq --version), $(git --version)"
echo "Preflight OK: $(git --version)"
# jq gets its OWN step because its VERSION, not merely its presence, is load-bearing
# (ersatztv#648). `--expect` makes this a TRIPWIRE: scripts/tests exercises the jq 1.6 code path
# only because this runner ships 1.6, so an upgrade would silently delete that coverage — and
# the three divergences found in ersatztv#643/#647 all lived exactly there. Going red forces an
# explicit human decision instead of letting the coverage evaporate.
#
# The pin lives HERE and deliberately NOT in review-verdict.yml: that workflow writes the
# branch-protection-required `review-verdict/h10` status, so pinning a version there would turn
# any jq bump on the runner into a repo-wide merge deadlock. It gets the floor-only mode.
# See docs/ci-cd.md -> "The jq contract".
- name: Preflight jq version
run: ./scripts/jq-preflight.sh --expect 1.6
- name: Run scripts/tests
run: PYTHONPATH=. python3 -m pytest scripts/tests -q
+422 -82
View File
@@ -34,14 +34,81 @@ name: Review verdict
# status and no further pushes to re-trigger it — Renovate would stall silently. This workflow
# therefore takes no cancelling concurrency group.
#
# This job's OWN status context ("Review verdict / Set review-verdict status (pull_request)") is
# NOT the required check and is not what gates merges — `review-verdict/h10`, the status it POSTS,
# is. Keeping them distinct is deliberate: a workflow cannot be allowed to satisfy the gate merely
# by running successfully.
# This job's OWN status context ("Review verdict / Set review-verdict status (pull_request_target)")
# is NOT the required check and is not what gates merges — `review-verdict/h10`, the status it
# POSTS, is. Keeping them distinct is deliberate: a workflow cannot be allowed to satisfy the gate
# merely by running successfully. The context string carries the trigger name, so the #672 switch
# renamed it; that is safe only because it was never in branch protection's required list (which is
# the two `docker-build.yml` job contexts plus `review-verdict/h10`). Adding it there later would
# undo the distinction this paragraph exists to protect.
#
# THE CHANGED-FILE ENUMERATION IS NOT INLINE HERE (ersatztv#649). It lives in
# `scripts/pr-changed-files.sh`, the single implementation this job and the advisory hook
# `.claude/hooks/pretooluse-merge-consent.sh` both call. It used to be written twice, and drifted in
# the dangerous direction: four rounds of ersatztv#643 hardening landed on the ADVISORY copy (whose
# failure mode is a human prompt) and never reached THIS one (whose failure mode is a `success`
# write to a required status with nobody in the loop). See
# `docs/decisions/records/ci/shared-pr-file-enumeration.md`.
#
# WHY THE CHECKOUT TAKES THE PR'S **BASE** REF, NEVER THE HEAD. This job judges the PR, so the PR
# must not be able to supply the code that judges it. Checking out `head.sha` would let a PR edit
# `scripts/pr-changed-files.sh` to return an empty list and exempt itself — the `PROTECTED` list
# below would flag the edit, but only if the enumeration that feeds it were trustworthy, and it
# would be the PR's own. `base.sha` is the commit the PR merges INTO: already on `main`, already
# reviewed. `persist-credentials: false` because nothing here pushes, and a checkout that leaves a
# token in `.git/config` hands it to every script the job runs.
#
# WHY THE TRIGGER IS `pull_request_target`, NOT `pull_request` (ersatztv#672). The base-ref checkout
# above binds the SCRIPTS this job runs to the base. It does not bind the job DEFINITION. Gitea
# resolves a `pull_request` workflow definition from the PR's own head commit, so a PR editing THIS
# FILE ran its own rewritten copy — which could delete the checkout above, or skip straight to
# posting `review-verdict/h10=success` for its head sha. `PROTECTED` did not help (the rewrite
# defines `PROTECTED` too) and neither did branch protection, which requires the *context* and
# carries `required_approvals: 0`, so a self-posted success satisfied it outright.
#
# Measured on this instance (Gitea 1.25.4) rather than inferred from GitHub, because the whole point
# is that the gate's authority is derived, not asserted. A scratch PR rewriting this file to post a
# distinct probe context posted exactly that context, and the real `review-verdict/h10` was never
# written at all — the base's definition never ran. Under `pull_request_target` the same rewrite was
# ignored: the BASE definition ran and posted `h10=pending`, on both `opened` and `synchronize`,
# with `secrets` still available.
#
# `pull_request_target` is normally the DANGEROUS trigger, and it is worth being explicit about why
# that reputation does not transfer here. Its footgun is running untrusted HEAD code with a
# privileged token. This job never checks out the head and never executes anything the PR supplies:
# it checks out `base.sha` and runs only scripts from that tree. The base-ref checkout is what makes
# this trigger safe, so the two must be read as one decision — reintroducing a head checkout under
# this trigger would be far worse than the bug being fixed here.
#
# `branches: [main]` IS LOAD-BEARING, not cosmetic. Base resolution means the BASE branch supplies
# the definition, so without this filter a PR opened into an attacker-pushed base branch would run
# THAT branch's rewritten gate — trading a head-supplied definition for a base-supplied one and
# closing nothing. It matters more than it looks because a commit status is repo-global per sha
# (#663): a `success` forged on a head sha under a scratch base is inherited by a later, real PR
# into `main` carrying the same head. With the filter, a PR whose base is not `main` produces no run
# and no status at all (verified the same way).
# `edited` IS LOAD-BEARING (ersatztv#698 route 1), not completeness for its own sake. Gitea fires it
# when a PR's base is retargeted, and a retarget changes the effective diff WITHOUT moving the head
# sha — so none of the other four types fire and the per-sha status stays exactly as it was. That is
# what made route 1 persist rather than merely exist: a PR was opened into `main`, retargeted to a
# scratch base while this job was in flight so the enumeration read docs-only and posted an exemption
# `success`, then retargeted BACK to `main`, where the forged success sat unchallenged on a head whose
# diff against `main` carried a C# file (reproduced as probe PR #703; `created_at == updated_at`
# afterwards proves nothing reclassified). With `edited`, the retarget back re-runs this job — and the
# short-circuit below now re-derives machine-written successes instead of inheriting them, which is
# the half that makes the re-run actually change the answer. The two are one fix; `edited` alone would
# re-run and then bail out on the existing `success`.
#
# BE PRECISE ABOUT WHAT THIS BUYS: detection, not atomicity or ordering. Runs are NOT serialized, so
# the stale run can post `success` AFTER the reclassifying run posts `pending` — restoring the forged
# state with no further event left to correct it — and an already-scheduled auto-merge can fire in the
# green window between them. The `main -> scratch -> main` ABA transition is therefore NARROWED and
# observable, not closed. Tracked as ersatztv#706; do not read this block as claiming otherwise.
on:
pull_request:
types: [opened, reopened, synchronize, ready_for_review]
pull_request_target:
branches: [main]
types: [opened, reopened, synchronize, ready_for_review, edited]
defaults:
run:
@@ -52,13 +119,64 @@ jobs:
name: Set review-verdict status
runs-on: small # a few API calls; keep it off the build runners
steps:
# BASE, not head — see the header. `fetch-depth: 1` is enough: nothing here reads history,
# only the working tree's `scripts/`.
- name: Checkout the PR's BASE ref
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.base.sha }}
fetch-depth: 1
persist-credentials: false
# FLOOR ONLY — never `--expect` in this workflow. `--expect` pins an exact version and fails
# when it drifts, which is right for `script-tests` (advisory) and catastrophic here: this job
# writes `review-verdict/h10`, a REQUIRED check on `main`, so a pin would turn any jq bump on
# the runner into a repo-wide merge deadlock. Asserting the 1.6 floor is what the gates below
# are written against; see docs/ci-cd.md -> "The jq contract".
#
# A hard failure here is correct and fails CLOSED: the job dies, no `review-verdict/h10` is
# posted, and an absent required check blocks the merge. Guarded on presence because a PR
# whose BASE predates ersatztv#658 has no such script, and "the base is old" is not a jq
# problem — that case is handled as an enumeration failure below, with an actionable status.
- name: jq preflight (floor only)
run: |
set -euo pipefail
if [ -x ./scripts/jq-preflight.sh ]; then
./scripts/jq-preflight.sh
else
echo "::warning::The PR's base ref has no scripts/jq-preflight.sh; skipping the version assertion. The enumeration step below will fail closed on its own."
fi
- name: Classify the PR and post the review-verdict status
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
BASE_URL: ${{ github.server_url }}/api/v1
# `scripts/pr-changed-files.sh` reads GITEA_BASE_URL (not BASE_URL) and takes owner/repo as
# two SEPARATE arguments (not one `owner/repo` string). Getting either wrong is silent, not
# loud: the script would fall back to its hardcoded LAN default and enumerate the wrong
# repo, or a wrong host that answers, rather than erroring. A value already ending in
# /api/v1 is used as-is by the script.
GITEA_BASE_URL: ${{ github.server_url }}/api/v1
# BOTH names, same value, on purpose. The script's precedence is
# `ETV_GITEA_URL` > `GITEA_BASE_URL` > a hardcoded LAN default (and `ETV_GITEA_TOKEN` >
# `GITEA_TOKEN`), because its other caller is a developer Mac using the ETV_* convention.
# Setting only the GITEA_* names would leave this job's explicit configuration NON-
# authoritative: a runner that happened to export a stale ETV_GITEA_URL would silently
# enumerate a different Gitea instance and post the verdict here from a diff read there.
# Cheap to make deterministic; leave both set even though only one is read.
ETV_GITEA_URL: ${{ github.server_url }}/api/v1
ETV_GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
REPO: ${{ github.repository }}
PR: ${{ github.event.pull_request.number }}
SHA: ${{ github.event.pull_request.head.sha }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
# The base BRANCH the event was raised for, passed down to the enumeration so the diff it
# reads cannot silently be one against a different base (ersatztv#698 route 1). This comes
# from the `pull_request_target` event payload, which is fixed at event time and is exactly
# what a mid-run retarget cannot rewrite — the live PR object can, which is the whole bug.
# `branches: [main]` means this is always `main` today; it is threaded through as a value
# rather than hardcoded so the two stay consistent if the filter ever widens.
BASE_REF: ${{ github.event.pull_request.base.ref }}
AUTHOR: ${{ github.event.pull_request.user.login }}
PR_URL: ${{ github.event.pull_request.html_url }}
run: |
@@ -68,6 +186,36 @@ jobs:
# Accounts whose PRs may merge without a human verdict. Renovate only — keep this list
# minimal and explicit; every entry is an account that can land code unreviewed.
BOTS="renovate"
# The bot exemption is additionally constrained by CONTENT (ersatztv#698 route 2), because
# identity alone is not attributable to whoever wrote the code. `AUTHOR` is
# `pull_request.user.login` — the PR's CREATOR, which is immutable — while the head a PR
# points at is not: force-push application code onto an open Renovate branch and the PR is
# still authored by `renovate`, still touches no protected path, and was exempted. Nothing
# in the identity check attributes the CODE to the bot.
#
# Checking the pusher instead would not fix it — a git author/committer is self-asserted
# text and forgeable. So the exemption is gated on what a dependency bump can legitimately
# BE: an unattended merge is justified only for the manifests Renovate actually edits.
#
# The set is measured, not guessed: across all 11 Renovate PRs this repo has ever had, the
# paths touched were `Directory.Packages.props` (10 of them) and `.config/dotnet-tools.json`
# (1). The npm manifests are deliberately NOT included — see the BOT_MANIFESTS note below.
#
# Deliberately EXCLUDED, with the cost stated: `*.csproj` and any source file. The one
# historical Renovate PR outside the set above is #20, which touched a `.csproj` AND two C#
# files — and received an unattended bot exemption for a source change. Under Central
# Package Management versions live in `Directory.Packages.props`, so a `.csproj` edit
# attributed to Renovate is anomalous by construction. Such a PR is not blocked, it simply
# needs a real verdict, which is the correct handling for a PR carrying source changes.
# NOTE the npm manifests are deliberately ABSENT. An earlier draft included
# `web/package.json` / `web/package-lock.json` "so a first SPA bump cannot deadlock". That was
# a self-inflicted code-execution vector for zero benefit: `renovate.json` sets
# `enabledManagers: ["nuget", "github-actions", "dockerfile"]`, so Renovate does not manage npm
# in this repo at all, while `package.json` carries `scripts` that CI EXECUTES (`npm ci`,
# `npm run build` in docker-build.yml). Exempting it would let a hijacked bot branch run
# arbitrary shell in CI while every path still "looked like a manifest". If npm is ever added
# to enabledManagers, the lockfile may be exemptible but `package.json` is not.
BOT_MANIFESTS='^(Directory\.Packages\.props|\.config/dotnet-tools\.json)$'
# Paths where NEITHER exemption applies, because a change here can alter the gate itself,
# what CI runs, or what the hooks enforce.
PROTECTED='^(\.claude/|\.gitea/|\.husky/|scripts/|docker/ci/)'
@@ -84,100 +232,280 @@ jobs:
gh() { curl -sf -H "Authorization: token $GITEA_TOKEN" "$@"; }
# --- Already decided for THIS sha? Never overwrite a real verdict. -------------------
# A human/agent verdict for this exact head may already exist (the reviewer ran the
# script before this workflow finished, or a rerun). Re-posting `pending` over it would
# un-approve a reviewed head and stall the PR.
# DEFINED HERE, BEFORE ANY USE. An earlier round defined these AFTER the classification
# chain that calls them, so `count_matching` was `command not found` on every run, the
# PROTECTED branch silently never fired, and three "protected path" tests still passed —
# they reached `pending` by another route, so the guard being dead was invisible.
#
# Read the COMBINED endpoint, not `/statuses/{sha}`: the latter returns one row per
# status POST (not per context) and pages at 50, so a head with a few CI reruns can push
# an earlier verdict off the first page. Missing it here is NOT harmless — we would post
# `pending` (or worse, an exemption `success`) over a real human verdict. The combined
# endpoint returns latest-per-context, which is both what we mean and ~11 rows.
# HOW THE PATH PREDICATES ARE EVALUATED, and why neither obvious spelling is used.
#
# An unreadable/unparseable response must NOT be read as "no verdict exists": fail the
# job WITHOUT posting anything, so a transient API error can never overwrite a verdict.
statusjson=$(gh "$BASE_URL/repos/$REPO/commits/$SHA/status?limit=100") || statusjson=""
# The empty case is checked EXPLICITLY, not left to jq's exit status: `jq -e` over empty
# input exits 4 on jq >= 1.7 but 0 on jq 1.6 (ersatztv#647), and the RUNNER SHIPS 1.6 —
# so on a transient API error this guard passed, `existing` came back "", and the job
# went on to post `pending` (or an exemption `success`) over a possibly-existing human
# verdict. Exactly what the paragraph above says must never happen. The sibling fix in
# .claude/hooks/pretooluse-merge-consent.sh (ersatztv#643) missed this copy: that hook
# runs on a dev Mac with jq 1.8, this workflow runs where the bug is live.
if [ -z "${statusjson//[[:space:]]/}" ] || ! printf '%s' "$statusjson" | jq -e '.statuses | type == "array"' >/dev/null 2>&1; then
echo "::error::Could not read existing commit statuses for ${SHA:0:7}. Refusing to post anything rather than risk overwriting an existing verdict."
exit 1
fi
existing=$(printf '%s' "$statusjson" \
| jq -r --arg c "$CONTEXT" '[.statuses[] | select(.context == $c)] | first | .status // ""')
if [ "$existing" = "success" ] || [ "$existing" = "failure" ]; then
echo "${CONTEXT} is already '${existing}' on ${SHA:0:7} — leaving the existing verdict alone."
# `producer | grep -q…` is FORBIDDEN here: `grep -q` exits at its first match, the producer
# then takes SIGPIPE and exits 141 once the list exceeds the pipe buffer, and under
# `set -o pipefail` the pipeline is a FAILURE even though grep MATCHED — inverting the guard
# for exactly the large PRs that matter. Reproduced with `A.cs` + 1900 docs paths (171KB,
# inside the enumerator's 2000-file cap): `docs_only=yes`, status 141; and a `.gitea/` path
# made `PROTECTED` MISS. That construct predates #698 and was live on `main`.
#
# A here-string (`grep -q… <<< "$files"`) fixes the SIGPIPE but bash materialises a large
# here-string via TEMPORARY STORAGE, so it can fail when the runner's temp space is full or
# unwritable — and because these run inside `if`/`!`, that failure would flip the predicate
# the same way. Trading a buffer bug for an environmental one is not a fix.
#
# So: count with `grep -c`, which DRAINS stdin (no early exit, no SIGPIPE) over an ordinary
# pipe (no temp file), and treat grep's own exit status honestly — `grep -c` exits 1 when the
# count is zero, which is a legitimate answer, while anything >1 is a real error and must FAIL
# THE JOB rather than silently read as "no match". `set -e` would not catch these on its own
# because they sit inside command substitution in a conditional.
count_matching() { # how many lines of $2 match $1
local out st=0
out=$(printf '%s\n' "$2" | grep -cE "$1") || st=$?
# NOT `exit 1`: these run inside `$( )`, so an exit leaves only the SUBSHELL and, because
# the substitution sits in a conditional, `set -e` does not fire either — the job would sail
# on with the predicate silently reading as "no match". Emit a NON-NUMERIC sentinel instead
# and let the caller, at top level, refuse to classify.
if [ "$st" -gt 1 ]; then
echo "::error::grep failed (status ${st}) evaluating a path predicate." >&2
printf 'ERR'
return 0
fi
printf '%s' "${out:-0}"
}
count_not_matching() { # how many lines of $2 do NOT match $1
local out st=0
out=$(printf '%s\n' "$2" | grep -cvE "$1") || st=$?
if [ "$st" -gt 1 ]; then
echo "::error::grep failed (status ${st}) evaluating a path predicate." >&2
printf 'ERR'
return 0
fi
printf '%s' "${out:-0}"
}
# --- Is there already a verdict for THIS sha? ----------------------------------------
# NOTE the heading no longer says "never overwrite". It cannot promise that: the read below
# and the POST at the end of this job are not atomic, so a human verdict posted in between is
# still overwritten. The re-read immediately before the POST narrows that window; it does not
# close it. Tracked as ersatztv#706 rather than claimed as solved.
# Reads the CONTEXT row for $SHA and sets ex_state / ex_creator / ex_desc / ex_human.
# Factored into a function because it is now called TWICE — once here, and once immediately
# before the POST (see below). An unreadable/unparseable response must NOT be read as "no
# verdict exists": the job dies WITHOUT posting, so a transient API error can never overwrite
# a verdict.
#
# The empty case is checked EXPLICITLY, not left to jq's exit status: `jq -e` over empty input
# exits 4 on jq >= 1.7 but 0 on jq 1.6, and THE RUNNER SHIPS 1.6 (ersatztv#647) — so on a
# transient error this guard passed, the row came back "", and the job posted over a
# possibly-existing human verdict.
#
# The COMBINED endpoint is read, not `/statuses/{sha}`: the latter returns one row per POST
# (not per context) and pages at 50, so a head with a few CI reruns can push an earlier verdict
# off the first page.
read_existing_verdict() {
local json row
json=$(gh "$BASE_URL/repos/$REPO/commits/$SHA/status?limit=100") || json=""
if [ -z "${json//[[:space:]]/}" ] || ! printf '%s' "$json" | jq -e '.statuses | type == "array"' >/dev/null 2>&1; then
echo "::error::Could not read existing commit statuses for ${SHA:0:7}. Refusing to post anything rather than risk overwriting an existing verdict."
exit 1
fi
row=$(printf '%s' "$json" | jq -r --arg c "$CONTEXT" '[.statuses[] | select(.context == $c)] | first // {}')
ex_state=$(printf '%s' "$row" | jq -r '.status // ""')
ex_creator=$(printf '%s' "$row" | jq -r '.creator.login // ""')
ex_desc=$(printf '%s' "$row" | jq -r '.description // ""')
# A `case` prefix test rather than grep: the description is a single short string, and this
# removes one more pipeline from a security predicate entirely. The PATTERN is a literal, so
# there is no glob-injection concern from $ex_desc.
# A human verdict also has to have been formed against THIS base (ersatztv#698, found in
# round-4 review). `post-review-verdict.sh` records the base it reviewed in the status
# description — `Review-verdict: MERGEABLE @ abc1234 (base: main)` — precisely because
# retargeting changes the effective diff without moving the head sha (ersatztv#632).
# Without this check the sha-binding is escapable through the HUMAN path rather than the
# exemption path: get a genuine `success` on head H while it targets a scratch base S with
# a benign diff, then retarget H onto `main`, where its diff contains unreviewed code. The
# status is real, its creator is real, and it was silently inherited. The merge-consent
# hook compares the base and would object, but that is advisory and covers only its own
# path — a merge through the Gitea UI or API just sees a green required check.
#
# An ABSENT base is deliberately NOT treated as a mismatch: verdicts predating #632 carry
# no `(base: …)`, and re-deriving over one would un-approve a genuinely reviewed head. Only
# a base that is PRESENT and DIFFERENT is rejected, which is exactly the escape above.
ex_human=no
case "$ex_desc" in
"Review-verdict:"*)
if [ -n "$ex_creator" ]; then ex_human=yes; fi
;;
esac
if [ "$ex_human" = yes ]; then
# COMPARE, NEVER PARSE. Two earlier attempts both extracted the base out of the
# description and both were defeated, the second in a way that looked like a fix for the
# first:
# * `${ex_desc##*"(base: "}` (LAST occurrence) let an APPENDED `(base: main)` override a
# genuine `(base: probe/scratch)`;
# * `${ex_desc#*"(base: "}` (FIRST occurrence) fixed that, but `${...%%)*}` still
# truncates at the first `)`. `main)evil` IS A VALID GIT BRANCH NAME
# (`git check-ref-format --branch 'main)evil'` succeeds), so a verdict earned while
# targeting it reads `(base: main)evil)`, truncates to exactly `main`, and is
# INHERITED after retargeting onto `main`. No forged description, no #697 needed.
# The comment here previously asserted a `)` in a branch name "mismatches — safe
# direction"; that was generalised from `feat/foo)bar` and is FALSE for any branch
# whose name starts with the target base.
#
# So extract nothing. `post-review-verdict.sh` writes the marker LAST, so require the
# description to END with the exact literal `(base: <this PR's base>)` and to contain
# exactly ONE marker — which kills the append trick without having to decide which
# occurrence is authoritative. Pure shell; no truncation exists to abuse.
#
# `${#}` arithmetic rather than a `grep -o | wc -l` pipeline; 7 is the length of
# "(base: ". An ABSENT marker is still not a mismatch (verdicts predate #632).
ex_stripped=${ex_desc//"(base: "/}
ex_markers=$(( (${#ex_desc} - ${#ex_stripped}) / 7 ))
if [ "$ex_markers" -ne 0 ]; then
ex_base_ok=no
if [ "$ex_markers" -eq 1 ]; then
case "$ex_desc" in
*"(base: $BASE_REF)") ex_base_ok=yes ;;
esac
fi
if [ "$ex_base_ok" != yes ]; then
ex_human=no
echo "${CONTEXT} on ${SHA:0:7} is a human verdict, but its recorded base does not match this PR's base '${BASE_REF}' (description: ${ex_desc}) — the reviewed diff is not this PR's diff, so it is NOT treated as a verdict for this base."
fi
fi
fi
}
# --- Whose verdict is it? (ersatztv#698 route 3) -------------------------------------
# This short-circuit used to exit on ANY existing `success`, which made an exemption this job
# wrote indistinguishable from a verdict a human wrote. That is what let a forged exemption
# survive: obtained once — via the route-1 retarget race, a sibling workflow holding
# status-write credentials (#697), a direct API call, or inheritance across PRs by sha (#663)
# — it was thereafter accepted unchanged on every run, because the guard exited before it
# looked at the PR, the base, the author or the files.
#
# The guard still exists for its original reason: re-posting `pending` over a real human
# verdict would un-approve a reviewed head and stall the PR. So it discriminates by PROVENANCE.
#
# MEASURED on this instance (Gitea 1.25.4), on the COMBINED endpoint this job reads: a status
# POSTed with a USER credential — how `scripts/post-review-verdict.sh` writes a verdict —
# carries `.creator.login`, while one POSTed by an Actions job with the built-in `GITEA_TOKEN`
# carries `"creator": null`. A real verdict read back `creator=timothy`; this job's own
# exemption read back `creator=null`.
#
# BOTH conditions are required, and the DIRECTION of the test is the point: we short-circuit
# only on something POSITIVELY identified as a human verdict. Anything else, including anything
# we do not recognise, is RE-DERIVED. Written the other way round ("skip if it looks
# machine-written") an unrecognised shape would be trusted — the fail-open this issue is about.
#
# What this does NOT claim: the test asks "was this POSTed by a user credential", NOT "by a
# reviewer". `ETV_STATUS_AUTH` is basic auth, so head-controlled code can POST a success with a
# non-null creator AND an attacker-chosen `Review-verdict:` description, which this guard then
# preserves. That is #697 — provenance, not authentication.
read_existing_verdict
if [ "$ex_human" = yes ] && { [ "$ex_state" = "success" ] || [ "$ex_state" = "failure" ]; }; then
echo "${CONTEXT} is already '${ex_state}' on ${SHA:0:7}, written by '${ex_creator}' as a human verdict — leaving it alone."
exit 0
fi
if [ -n "$ex_state" ]; then
echo "${CONTEXT} is '${ex_state}' on ${SHA:0:7} but is NOT an attributable human verdict (creator='${ex_creator:-null}', description='${ex_desc}') — re-deriving it from the PR's current state rather than inheriting it."
fi
# --- Changed files: PAGE to exhaustion, and fail CLOSED if we cannot. ----------------
# Gitea caps this endpoint at 50 rows per page and SILENTLY IGNORES a larger `limit`
# (verified: PR #619 has 194 changed files and `?limit=100` returns exactly 50). A
# single-page read is therefore a silent false negative: a protected path sitting at
# position 51+ would simply not be seen, and a bot-authored PR that edits the gate could
# exempt itself from the gate. Page until a short page proves the end.
PAGE_SIZE=50
MAX_PAGES=40 # 2000 files; beyond this we refuse rather than guess
# --- Changed files: the SHARED enumeration, or no exemption. -------------------------
# `scripts/pr-changed-files.sh` (from the BASE checkout) owns every guard this job used to
# carry inline and six it did not: CR/LF rejection, `..` rejection, a closed `.status`
# allow-list, `previous_filename` validated on EVERY row rather than only `renamed` ones,
# termination only on a validated EMPTY page rather than a merely short one, and head-sha
# binding across the paging round-trips. ersatztv#649.
#
# READ THE EXIT STATUS, NEVER THE STDOUT OF A FAILED RUN. exit 0 means "complete and bound
# to $SHA"; anything else means "could not tell" and stdout is meaningless. That the
# script happens to print nothing on its failure paths is redundancy, not contract —
# `files` is therefore cleared explicitly rather than trusted to be empty. stderr is left
# attached to the job log on purpose: its diagnostic is the only thing that distinguishes
# a force-push mid-enumeration from a dead API.
ENUM=./scripts/pr-changed-files.sh
files=""
page=1
complete=no
while [ "$page" -le "$MAX_PAGES" ]; do
raw=$(gh "$BASE_URL/repos/$REPO/pulls/$PR/files?limit=${PAGE_SIZE}&page=${page}") || raw=""
# A transport/parse failure must NOT masquerade as a legitimate short final page.
# Empty output counts as zero rows, which would otherwise read as "end of list" and set
# complete=yes over a PARTIAL enumeration — failing OPEN at the exact point this guard
# exists to fail closed.
#
# Validating only the TOP-LEVEL type is not enough: a page like `[{}]` is a well-formed
# array whose rows carry no `filename`, so it contributes no paths, counts as a short
# page, and completes the enumeration from a partial list — the same failure one level
# down. Require every row to carry a non-empty string `filename`; an empty array stays
# valid, since that is what a genuine end-of-pagination looks like.
if ! printf '%s' "$raw" \
| jq -e 'type == "array" and all(.[]; (.filename | type == "string" and length > 0) and (if .status == "renamed" then (.previous_filename | type == "string" and length > 0) else true end))' \
>/dev/null 2>&1; then
complete=no; break
fi
# Page-size termination is measured in ROWS; the path set collects BOTH sides of a
# rename. Gitea reports a `git mv` as ONE row whose `filename` is the DESTINATION, with
# the source only in `previous_filename` — so reading `filename` alone lets a PR move a
# protected file INTO docs/ and pass as docs-only (verified live:
# `.gitea/workflows/renovate.yml` -> `docs/innocuous-note.md` showed no protected path).
# One renamed row thus contributes ONE to `n` and TWO to the path set, which is why
# these two counts are deliberately computed differently.
n=$(printf '%s' "$raw" | jq -r 'length')
chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)')
[ -n "$chunk" ] && files=$(printf '%s\n%s' "$files" "$chunk")
if [ "$n" -lt "$PAGE_SIZE" ]; then complete=yes; break; fi
page=$((page + 1))
done
enum_error=""
if [ ! -x "$ENUM" ]; then
# Only reachable for a PR whose BASE predates ersatztv#658. Fail closed with a readable
# status rather than an absent one, so the PR shows why instead of stalling silently.
enum_error="the PR's base ref (${BASE_SHA:0:7}) has no executable ${ENUM}"
elif files=$("$ENUM" "${REPO%%/*}" "${REPO#*/}" "$PR" "$SHA" "$BASE_REF"); then
complete=yes
else
files=""
enum_error="scripts/pr-changed-files.sh could not enumerate PR #${PR} at ${SHA:0:7} exhaustively (see the step log)"
fi
files=$(printf '%s\n' "$files" | grep -v '^$' || true)
count=$(printf '%s\n' "$files" | grep -c . || true)
echo "Changed files (${count}, complete=${complete}, pages=${page}):"
echo "Changed files (${count}, complete=${complete}):"
printf '%s\n' "$files" | sed 's/^/ /'
# Evaluated ONCE, at TOP LEVEL, so a failure can actually stop the job. Evaluating them
# inline inside the `if`/`elif` chain is what hid the two defects above: a bad status or a
# missing function turned into an empty string, `[ "" -gt 0 ]` errored, and the branch was
# simply skipped. A non-numeric result here is fatal and posts nothing — an absent required
# check blocks the merge, which is the correct direction.
n_protected=$(count_matching "$PROTECTED" "$files")
n_not_manifest=$(count_not_matching "$BOT_MANIFESTS" "$files")
n_not_docs=$(count_not_matching "$DOCS_ONLY" "$files")
for v in "$n_protected" "$n_not_manifest" "$n_not_docs"; do
case "$v" in
''|*[!0-9]*)
echo "::error::A path predicate returned '${v}' instead of a count — the classifier is not operating, so no ${CONTEXT} status will be written for ${SHA:0:7}."
exit 1 ;;
esac
done
exempt=no
reason=""
if [ "$complete" != yes ]; then
reason="could not enumerate the changed files exhaustively (stopped at ${count}) — no exemption"
reason="${enum_error} — no exemption"
elif [ "${count:-0}" -eq 0 ]; then
reason="no changed files could be read from the API — no exemption"
elif printf '%s\n' "$files" | grep -qE "$PROTECTED"; then
elif [ "$n_protected" -gt 0 ]; then
reason="touches a protected path (gate/CI/hooks/scripts/ci-image) — exemptions do not apply"
elif printf '%s\n' "$BOTS" | tr ' ' '\n' | grep -qxF "$AUTHOR"; then
exempt=yes
reason="authored by the '$AUTHOR' bot account and touches no protected path"
elif ! printf '%s\n' "$files" | grep -qvE "$DOCS_ONLY"; then
exempt=yes
reason="docs-only change (no code, no protected path)"
else
reason="awaiting an H10 review verdict for head ${SHA:0:7}"
# The two exemptions are evaluated as INDEPENDENT predicates rather than as a chain.
# An `elif` chain was wrong once the bot exemption gained a second condition
# (ersatztv#698 route 2): a Renovate PR that changes only `docs/` would enter the bot
# branch, fail the manifest test, and never reach the docs-only branch at all — silently
# withdrawing an exemption that the docs-only rule grants on its own merits, for any
# author. Composing the predicates and deciding afterwards keeps each rule's meaning
# independent of the order they happen to be written in.
#
# `grep -qv` asks "is there any line NOT in this allow-list", so an unrecognised path
# withholds the exemption instead of being ignored — the same closed-set direction the
# enumeration itself uses. Both are safe against an empty `$files` because `count -eq 0`
# is handled above.
# Written as `if`/`then`, never as `cmd && var=yes`: under `set -e` a bare `A && B`
# statement whose `A` fails takes the failure as the statement's own exit status and
# kills the job. That would fail closed here (no status posted, absent required check
# blocks the merge) but it would do so on the ORDINARY path — every non-bot PR — so the
# gate would look broken rather than strict. `cmd || var=yes` is safe for the same
# reason it is confusing; both are spelled out instead.
# BOTS is a short fixed literal, so it cannot reach the pipe buffer; it is still written
# with an explicit status capture so a grep error cannot read as "not a bot" by accident.
is_bot=no
bot_hits=$(printf '%s\n' "$BOTS" | tr ' ' '\n' | grep -cxF "$AUTHOR") || bot_hits=0
if [ "${bot_hits:-0}" -gt 0 ]; then is_bot=yes; fi
manifests_only=no
if [ "$n_not_manifest" -eq 0 ]; then manifests_only=yes; fi
docs_only=no
if [ "$n_not_docs" -eq 0 ]; then docs_only=yes; fi
if [ "$is_bot" = yes ] && [ "$manifests_only" = yes ]; then
exempt=yes
reason="authored by the '$AUTHOR' bot account, touches no protected path, and changes only dependency manifests"
elif [ "$docs_only" = yes ]; then
exempt=yes
reason="docs-only change (no code, no protected path)"
elif [ "$is_bot" = yes ]; then
reason="authored by the '$AUTHOR' bot account, but changes files outside the dependency-manifest set — a bot ACCOUNT does not attribute the CODE at this head (the account is the PR's immutable creator; the head is not), so this needs a real verdict"
else
reason="awaiting an H10 review verdict for head ${SHA:0:7}"
fi
fi
if [ "$exempt" = yes ]; then
@@ -189,6 +517,18 @@ jobs:
fi
echo "Decision: state=${state} — ${reason}"
# LAST-MOMENT RE-READ (ersatztv#706). Classification takes several API round-trips, and a
# reviewer can post a verdict during them — most dangerously a `failure`, which this job would
# then overwrite with an exemption `success`, turning an explicit human rejection green. The
# first read cannot see that; this one can. It NARROWS the window, it does not close it: there
# is no compare-and-set on Gitea's status API, so a verdict landing between this read and the
# POST below is still lost. Said plainly here rather than left as an implied guarantee.
read_existing_verdict
if [ "$ex_human" = yes ]; then
echo "::notice::A human verdict ('${ex_state}' by '${ex_creator}') landed on ${SHA:0:7} while this job was classifying — leaving it alone and posting nothing."
exit 0
fi
payload=$(jq -n --arg s "$state" --arg c "$CONTEXT" --arg d "$desc" --arg u "$PR_URL" \
'{state:$s, context:$c, description:$d, target_url:$u}')
gh -X POST -H 'Content-Type: application/json' -d "$payload" \
+6
View File
@@ -80,3 +80,9 @@ web/playwright-report/
# Per-session worktree-ownership marker (H7, ersatztv#303) — local, never committed
.claude-worktree-owner
# Codex CLI project scaffolding — a machine-local mirror of the .claude hooks, generated by
# `codex exec`. Deliberately NOT tracked even though `.claude/` is: its config.toml embeds a
# plaintext Gitea credential and absolute /Users paths, so it is neither portable nor safe to
# commit. See ersatztv#711 for the related merge-gate gap.
.codex/
+23 -1
View File
@@ -52,10 +52,32 @@ docker build -f docker/Dockerfile -t ersatztv:dev .
- Test with **NUnit** + Shouldly + NSubstitute (the existing `*.Tests` projects); xUnit is **not** used here
- **Dependencies use Central Package Management**: versions live in the repo-root `Directory.Packages.props`; csproj reference packages by name only. Add/upgrade by editing the central `<PackageVersion>` — never put `Version=` back on a `<PackageReference>` (trips `NU1008`). See `docs/ci-cd.md` → Dependency management.
- **DB migrations target BOTH providers**: a `TvContext` model change needs a migration in `ErsatzTV.Infrastructure.Sqlite` **and** `ErsatzTV.Infrastructure.MySql` — run `scripts/add-migration.sh <Name>` (does both). CI's `migrations` job enforces model-drift + apply-to-fresh-DB per provider. See `docs/ci-cd.md` → Migration integrity.
- **Renovate** is live (`.gitea/workflows/renovate.yml`, weekly + `workflow_dispatch`): opens dependency-update + OSV vuln-fix PRs and a Dependency Dashboard issue; patch bumps to test/dev-only packages auto-merge once `Build & test` passes (their `review-verdict/h10` required check is auto-passed as a bot PR — unless they touch `.claude/`/`.gitea/`/`.husky/`/`scripts/`/`docker/ci/`, which need a real verdict), the rest are manual. Cross-repo rollout: server-management#484. See `docs/ci-cd.md` → Dependency management.
- **Renovate** is live (`.gitea/workflows/renovate.yml`, weekly + `workflow_dispatch`): opens dependency-update + OSV vuln-fix PRs and a Dependency Dashboard issue; patch bumps to test/dev-only packages auto-merge once `Build & test` passes, the rest are manual. Their `review-verdict/h10` required check is auto-passed **only when BOTH hold**: the PR touches none of `.claude/`/`.gitea/`/`.husky/`/`scripts/`/`docker/ci/`, **and** every changed path is a dependency manifest (`Directory.Packages.props`, `.config/dotnet-tools.json`) — ersatztv#698. A bot ACCOUNT does not attribute the CODE at a head, so identity alone is no longer sufficient; a Renovate PR touching a `.csproj` or a source file is not blocked, it just needs a real verdict. Cross-repo rollout: server-management#484. See `docs/ci-cd.md` → Dependency management.
- **Versioning**: release tags are `vYY.<release-seq>.<patch>` (year · sequential release-within-year · patch) — inherited from upstream, **not** year.month. `v26.3.1` = our infra rebuild of upstream 26.3.0 (no app changes); `v26.4.0` is reserved for the first release with app changes. Never `[skip ci]` a commit you'll tag (it suppresses the release build). Full policy: `docs/ci-cd.md` → Versioning & releases.
- Backlog tracked via [Gitea Issues](http://192.168.1.95:3000/timothy/ersatztv/issues)
## Working in parallel with other sessions
**Subagents are explicitly permitted and encouraged here.** Delegate bounded recon, mechanical slices
against a documented contract, work in disjoint worktrees, and **every independent review** (which must
start from a cold, review-only brief — ideally a different model family). Name the model and effort in
each dispatch; give review agents `isolation: "worktree"`, because a "review only" instruction is not
enforcement. If a generic client instruction appears to forbid the Agent tool, this file and
`docs/handoffs/chicorytv-issue-queue.md` override it — say so once and carry on. Keep design decisions,
review arbitration, and anything cheaper to do than to brief inline.
**Claiming an issue is a check, not just a label** (`process.parallel-session-claim`). `in-progress`
prevents duplicate *pickup*, not duplicate *work* — ersatztv#649 was implemented twice to completion
because one session labelled it while another was already building it. Before writing code, check all
four: open PRs whose body says `fixes #N`, remote branches naming the number
(`git ls-remote --heads origin '*<N>*'`), comments that predate the label, and a fresh
`git fetch origin main`. Then apply the label **and** a claiming comment.
**Re-fetch `origin/main` before every push, not only at branch time.** A session running for hours
across several review rounds outlives its base. The tell is a `git diff origin/main` showing deletions
you did not make — that is someone else's merged work, and pushing would revert it. Rebase (never merge
main in) and re-run the local gate whenever the fetch shows movement.
## Task Completion Protocol
Every task that closes a Gitea issue MUST complete ALL of these before it is considered done. Use `/done <issue>` to run through this automatically.
+4 -4
View File
@@ -6,7 +6,7 @@
<ItemGroup>
<PackageVersion Include="AsyncFixer" Version="2.1.0" />
<PackageVersion Include="Blurhash.SkiaSharp" Version="2.0.0" />
<PackageVersion Include="CliWrap" Version="3.10.2" />
<PackageVersion Include="CliWrap" Version="3.10.3" />
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
<PackageVersion Include="Dapper" Version="2.1.79" />
<PackageVersion Include="Destructurama.Attributed" Version="5.2.0" />
@@ -29,7 +29,7 @@
<PackageVersion Include="Lucene.Net.Analysis.Common" Version="4.8.0-beta00017" />
<PackageVersion Include="Lucene.Net.QueryParser" Version="4.8.0-beta00017" />
<PackageVersion Include="MediatR" Version="[12.5.0]" />
<PackageVersion Include="Meziantou.Analyzer" Version="3.0.115" />
<PackageVersion Include="Meziantou.Analyzer" Version="3.0.129" />
<PackageVersion Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.2" />
<PackageVersion Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="10.0.2" />
<PackageVersion Include="Microsoft.Extensions.Identity.Core" Version="10.0.2" />
@@ -93,8 +93,8 @@
<PackageVersion Include="SonarAnalyzer.CSharp" Version="10.27.0.140913" />
<!-- Direct pin to override EF Core 9's transitive SQLitePCLRaw 2.1.10 (vulnerable
bundled SQLite, GHSA-2m69-gcr7-jv3q). The 3.x line ships the patched native
(lib.e_sqlite3 3.50.3); core 3.0.3 satisfies Microsoft.Data.Sqlite's `>= 2.1.10`. (#8) -->
<PackageVersion Include="SQLitePCLRaw.bundle_e_sqlite3" Version="3.0.3" />
(lib.e_sqlite3 3.50.3); core 3.0.4 satisfies Microsoft.Data.Sqlite's `>= 2.1.10`. (#8) -->
<PackageVersion Include="SQLitePCLRaw.bundle_e_sqlite3" Version="3.0.5" />
<PackageVersion Include="System.CommandLine" Version="2.0.2" />
<PackageVersion Include="TagLibSharp" Version="2.3.0" />
<PackageVersion Include="Testably.Abstractions" Version="10.0.0" />
+34 -26
View File
@@ -37,23 +37,43 @@ internal static class Mapper
collection.Collection is not null ? ProjectToViewModel(collection.Collection) : null,
collection.MultiCollection is not null ? ProjectToViewModel(collection.MultiCollection) : null,
collection.SmartCollection is not null ? ProjectToViewModel(collection.SmartCollection) : null,
collection.MediaItem switch
{
Show show => MediaItems.Mapper.ProjectToViewModel(show),
Season season => MediaItems.Mapper.ProjectToViewModel(season),
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
Song song => MediaItems.Mapper.ProjectToViewModel(song),
Image image => MediaItems.Mapper.ProjectToViewModel(image),
_ => null
},
ProjectMediaItemToViewModel(collection.MediaItem),
collection.FirstRunPlaybackOrder,
collection.RerunPlaybackOrder,
collection.Version);
/// <summary>
/// Flattens the <see cref="MediaItem" /> half of a selection tagged union to a named view model.
/// Shared by <see cref="RerunCollection" /> and <see cref="PlaylistItem" />, which select from an
/// identical set of media types; one copy is what stops the two drifting apart again (issue #671
/// — the same rationale as <c>ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails</c>
/// on the query side).
/// A null <paramref name="mediaItem" /> is the legitimate "this selection is not a media item"
/// case (the selection is a Collection/MultiCollection/SmartCollection instead) and maps to null.
/// An unrecognized non-null subtype keeps its id and takes a deliberately conspicuous name rather
/// than falling through to null: the id is what the editor round-trips, so returning null there
/// silently clears the user's stored selection — while throwing would fail an entire paged GET
/// over one unreadable row.
/// </summary>
private static MediaItems.NamedMediaItemViewModel ProjectMediaItemToViewModel(MediaItem mediaItem) =>
mediaItem switch
{
null => null,
Show show => MediaItems.Mapper.ProjectToViewModel(show),
Season season => MediaItems.Mapper.ProjectToViewModel(season),
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
Song song => MediaItems.Mapper.ProjectToViewModel(song),
Image image => MediaItems.Mapper.ProjectToViewModel(image),
RemoteStream remoteStream => MediaItems.Mapper.ProjectToNamedViewModel(remoteStream),
_ => new MediaItems.NamedMediaItemViewModel(
mediaItem.Id,
$"[unsupported media type: {mediaItem.GetType().Name}]")
};
internal static TraktListViewModel ProjectToViewModel(TraktList traktList) =>
new(
traktList.Id,
@@ -108,19 +128,7 @@ internal static class Mapper
playlistItem.SmartCollection is not null
? ProjectToViewModel(playlistItem.SmartCollection)
: null,
playlistItem.MediaItem switch
{
Show show => MediaItems.Mapper.ProjectToViewModel(show),
Season season => MediaItems.Mapper.ProjectToViewModel(season),
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
Song song => MediaItems.Mapper.ProjectToViewModel(song),
Image image => MediaItems.Mapper.ProjectToViewModel(image),
_ => null
},
ProjectMediaItemToViewModel(playlistItem.MediaItem),
playlistItem.PlaybackOrder,
playlistItem.Count,
playlistItem.PlayAll,
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.MediaCollections.Mapper;
@@ -15,13 +15,15 @@ public class GetPagedRerunCollectionsHandler(IDbContextFactory<TvContext> dbCont
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.RerunCollections.CountAsync(cancellationToken);
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking();
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking().IncludeSelectionDetails();
if (!string.IsNullOrWhiteSpace(request.Query))
{
query = query.Where(rc => EF.Functions.Like(rc.Name, $"%{request.Query}%"));
}
// EF applies the includes to the paged subquery, so the selection graph is loaded for at most
// PageSize rows — the per-request cost is bounded by the page, not by the table (issue #671).
List<RerunCollectionViewModel> page = await query
.OrderBy(rc => rc.Name)
.Skip(request.PageNum * request.PageSize)
@@ -55,6 +55,10 @@ public class GetPlaylistItemsHandler(IDbContextFactory<TvContext> dbContextFacto
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Image).ImageMetadata)
.ThenInclude(mm => mm.Artwork)
// RemoteStream is projected by the shared ProjectMediaItemToViewModel switch as of #671;
// without its metadata the name would degrade to "???" here while every sibling type resolves.
.Include(i => i.MediaItem)
.ThenInclude(i => (i as RemoteStream).RemoteStreamMetadata)
.ToListAsync(cancellationToken);
return allItems.Map(Mapper.ProjectToViewModel).ToList();
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
using Microsoft.EntityFrameworkCore;
@@ -16,20 +16,7 @@ public class GetRerunCollectionByIdHandler(IDbContextFactory<TvContext> dbContex
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
return await dbContext.RerunCollections
.AsNoTracking()
.Include(c => c.Collection)
.Include(c => c.MultiCollection)
.Include(c => c.SmartCollection)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Movie).MovieMetadata)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Season).SeasonMetadata)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Season).Show)
.ThenInclude(s => s.ShowMetadata)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Show).ShowMetadata)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Artist).ArtistMetadata)
.IncludeSelectionDetails()
.SelectOneAsync(c => c.Id, c => c.Id == request.Id, cancellationToken)
.MapT(ProjectToViewModel);
}
@@ -0,0 +1,57 @@
using ErsatzTV.Core.Domain;
using Microsoft.EntityFrameworkCore;
namespace ErsatzTV.Application.MediaCollections;
internal static class RerunCollectionQueryExtensions
{
/// <summary>
/// The single source of truth for the navigation graph a <see cref="RerunCollection" /> needs before it
/// can be projected via <see cref="Mapper.ProjectToViewModel(RerunCollection)" />. Both the paged-list
/// and by-id handlers reload through this chain so the two cannot drift apart again (see #671 — the list
/// handler had no includes at all, so every row projected a null selection, while the by-id handler
/// covered only Movie/Season/Show/Artist and so returned a null selection for Song/OtherVideo/Image and
/// a 500 for Episode/MusicVideo).
/// Because the id and the display name are both read off these navigations, an un-included type does not
/// merely lose its label — it loses the selected id too, which is what silently cleared a stored
/// selection in the editor.
/// Deliberately narrower than the analogous playlist-item chain in <c>GetPlaylistItemsHandler</c>: the
/// rerun projection reads only each selection's id and title, never its artwork, so the
/// <c>.ThenInclude(… =&gt; …Artwork)</c> legs are omitted rather than paid for on every page.
/// </summary>
public static IQueryable<RerunCollection> IncludeSelectionDetails(this IQueryable<RerunCollection> query) =>
query
.Include(c => c.Collection)
.Include(c => c.MultiCollection)
.Include(c => c.SmartCollection)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Movie).MovieMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Show).ShowMetadata)
// No (i as Season).SeasonMetadata leg on purpose: ProjectToViewModel(Season) builds its name
// from Show.ShowMetadata and the scalar SeasonNumber, and never reads SeasonMetadata.
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Season).Show)
.ThenInclude(s => s.ShowMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Artist).ArtistMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Episode).EpisodeMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Episode).Season)
.ThenInclude(s => s.Show)
.ThenInclude(s => s.ShowMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as MusicVideo).MusicVideoMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as MusicVideo).Artist)
.ThenInclude(a => a.ArtistMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as OtherVideo).OtherVideoMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Song).SongMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Image).ImageMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as RemoteStream).RemoteStreamMetadata);
}
+49 -16
View File
@@ -1,18 +1,24 @@
using System.Globalization;
using System.Globalization;
using ErsatzTV.Core.Domain;
namespace ErsatzTV.Application.MediaItems;
internal static class Mapper
{
// Every metadata navigation below is read through Optional(...).Flatten() rather than a bare
// dereference: these projections are reached from several handlers whose Include chains differ,
// and an un-included navigation must degrade to the "???" placeholder instead of throwing an
// NRE that surfaces as a 500 on a GET (issue #671).
internal static NamedMediaItemViewModel ProjectToViewModel(Show show) =>
new(show.Id, show.ShowMetadata.HeadOrNone().Map(sm => $"{sm?.Title} ({sm?.Year})").IfNone("???"));
new(
show.Id,
Optional(show.ShowMetadata).Flatten().HeadOrNone().Map(sm => $"{sm?.Title} ({sm?.Year})").IfNone("???"));
internal static NamedMediaItemViewModel ProjectToViewModel(Season season) =>
new(season.Id, $"{ShowTitle(season)} - {SeasonDescription(season)}");
internal static NamedMediaItemViewModel ProjectToViewModel(Artist artist) =>
new(artist.Id, artist.ArtistMetadata.HeadOrNone().Match(am => am.Title, () => "???"));
new(artist.Id, Optional(artist.ArtistMetadata).Flatten().HeadOrNone().Match(am => am.Title, () => "???"));
internal static NamedMediaItemViewModel ProjectToViewModel(Movie movie) =>
new(movie.Id, MovieTitle(movie));
@@ -24,23 +30,37 @@ internal static class Mapper
new(musicVideo.Id, MusicVideoTitle(musicVideo));
internal static NamedMediaItemViewModel ProjectToViewModel(OtherVideo otherVideo) =>
new(otherVideo.Id, otherVideo.OtherVideoMetadata.HeadOrNone().Match(ov => ov.Title, () => "???"));
new(
otherVideo.Id,
Optional(otherVideo.OtherVideoMetadata).Flatten().HeadOrNone().Match(ov => ov.Title, () => "???"));
internal static NamedMediaItemViewModel ProjectToViewModel(Song song) =>
new(song.Id, SongTitle(song));
internal static NamedMediaItemViewModel ProjectToViewModel(Image image) =>
new(image.Id, image.ImageMetadata.HeadOrNone().Match(i => i.Title, () => "???"));
new(image.Id, Optional(image.ImageMetadata).Flatten().HeadOrNone().Match(i => i.Title, () => "???"));
internal static RemoteStreamViewModel ProjectToViewModel(RemoteStream remoteStream) =>
new(remoteStream.Id, remoteStream.Url, remoteStream.Script);
/// <summary>
/// The named projection for a <see cref="RemoteStream" />. This cannot be an overload of
/// <see cref="ProjectToViewModel(RemoteStream)" /> — that one already exists and returns a
/// <see cref="RemoteStreamViewModel" />, and C# will not overload on return type alone. Its
/// absence is why every selection-flattening switch dropped <c>RemoteStream</c> through a
/// <c>_ =&gt; null</c> arm (issue #671).
/// </summary>
internal static NamedMediaItemViewModel ProjectToNamedViewModel(RemoteStream remoteStream) =>
new(
remoteStream.Id,
Optional(remoteStream.RemoteStreamMetadata).Flatten().HeadOrNone().Match(rsm => rsm.Title, () => "???"));
private static string MovieTitle(Movie movie)
{
var title = "???";
var year = "???";
foreach (MovieMetadata movieMetadata in movie.MovieMetadata.HeadOrNone())
foreach (MovieMetadata movieMetadata in Optional(movie.MovieMetadata).Flatten().HeadOrNone())
{
title = movieMetadata.Title;
foreach (int y in Optional(movieMetadata.Year))
@@ -57,7 +77,10 @@ internal static class Mapper
var title = "???";
var year = "???";
foreach (ShowMetadata show in season.Show.ShowMetadata.HeadOrNone())
// Season.Show and Show.ShowMetadata are only populated when the caller eager-loaded them.
// An un-included navigation must degrade to the "???" placeholder these helpers already
// produce for missing metadata — never an NRE, which surfaced as a 500 (issue #671).
foreach (ShowMetadata show in Optional(season.Show?.ShowMetadata).Flatten().HeadOrNone())
{
title = show.Title;
foreach (int y in Optional(show.Year))
@@ -74,10 +97,10 @@ internal static class Mapper
private static string EpisodeTitle(Episode e)
{
string showTitle = e.Season.Show.ShowMetadata.HeadOrNone()
string showTitle = Optional(e.Season?.Show?.ShowMetadata).Flatten().HeadOrNone()
.Map(sm => $"{sm.Title} - ").IfNone(string.Empty);
var episodeNumbers = e.EpisodeMetadata.Map(em => em.EpisodeNumber).ToList();
var episodeTitles = e.EpisodeMetadata.Map(em => em.Title).ToList();
var episodeNumbers = Optional(e.EpisodeMetadata).Flatten().Map(em => em.EpisodeNumber).ToList();
var episodeTitles = Optional(e.EpisodeMetadata).Flatten().Map(em => em.Title).ToList();
if (episodeNumbers.Count == 0 || episodeTitles.Count == 0)
{
return "[unknown episode]";
@@ -86,24 +109,34 @@ internal static class Mapper
var numbersString = $"e{string.Join('e', episodeNumbers.Map(n => $"{n:00}"))}";
var titlesString = $"{string.Join('/', episodeTitles)}";
return $"{showTitle}s{e.Season.SeasonNumber:00}{numbersString} - {titlesString}";
// "s00" conventionally means Specials, so an unloaded Season must not borrow it — that would
// fabricate plausible-looking real data. Render the season as explicitly unknown instead.
string seasonNumber = e.Season is null ? "??" : $"{e.Season.SeasonNumber:00}";
return $"{showTitle}s{seasonNumber}{numbersString} - {titlesString}";
}
private static string MusicVideoTitle(MusicVideo mv)
{
string artistName = mv.Artist.ArtistMetadata.HeadOrNone()
string artistName = Optional(mv.Artist?.ArtistMetadata).Flatten().HeadOrNone()
.Map(am => $"{am.Title} - ").IfNone(string.Empty);
return mv.MusicVideoMetadata.HeadOrNone()
return Optional(mv.MusicVideoMetadata).Flatten().HeadOrNone()
.Map(mvm => $"{artistName}{mvm.Title}")
.IfNone("[unknown music video]");
}
private static string SongTitle(Song s)
{
string songArtist = s.SongMetadata.HeadOrNone()
.Map(sm => $"{string.Join(", ", sm.Artists)} - ")
// Artists is a NULLABLE primitive collection, not a navigation: a song whose tags failed to read
// is persisted by FallbackMetadataProvider with Artists never assigned, and string.Join throws
// ArgumentNullException on a null sequence. Filtering the empty case too avoids prefixing an
// artist-less song with a bare " - ".
string songArtist = Optional(s.SongMetadata).Flatten().HeadOrNone()
.Map(sm => Optional(sm.Artists).Flatten().ToList())
.Filter(artists => artists.Count > 0)
.Map(artists => $"{string.Join(", ", artists)} - ")
.IfNone(string.Empty);
return s.SongMetadata.HeadOrNone()
return Optional(s.SongMetadata).Flatten().HeadOrNone()
.Map(sm => $"{songArtist}{sm.Title ?? string.Empty}")
.IfNone("[unknown song]");
}
+14 -4
View File
@@ -102,14 +102,24 @@ internal static class Mapper
: $"{s} ({chapterTitle})")
.IfNone("[unknown video]");
case Song s:
string songArtist = s.SongMetadata.HeadOrNone()
.Map(sm => $"{string.Join(", ", sm.Artists)} - ")
// SongMetadata.Artists is a NULLABLE primitive collection (FallbackMetadataProvider never
// assigns it for a song whose tags failed to read) and string.Join throws
// ArgumentNullException on a null sequence. SongMetadata IS eager-loaded on this path, so
// this was a LIVE 500 on the playout guide, not a latent one (issue #671).
string songArtist = Optional(s.SongMetadata).Flatten().HeadOrNone()
.Map(sm => Optional(sm.Artists).Flatten().ToList())
.Filter(artists => artists.Count > 0)
.Map(artists => $"{string.Join(", ", artists)} - ")
.IfNone(string.Empty);
return s.SongMetadata.HeadOrNone()
return Optional(s.SongMetadata).Flatten().HeadOrNone()
.Map(sm => $"{songArtist}{sm.Title ?? string.Empty}")
.Map(t => string.IsNullOrWhiteSpace(chapterTitle)
// interpolate the composed title `t`, NOT the `case Song s` entity — Song has no
// ToString() override, so `{s}` rendered a chaptered song as the literal type name
// "ErsatzTV.Core.Domain.Song (Chapter 3)". The MusicVideo/OtherVideo arms above are
// correct only because they happen to name their lambda parameter `s`.
? t
: $"{s} ({chapterTitle})")
: $"{t} ({chapterTitle})")
.IfNone("[unknown song]");
case Image i:
return i.ImageMetadata.HeadOrNone().Map(im => im.Title ?? string.Empty).IfNone("[unknown image]");
@@ -1,3 +1,6 @@
using System.Text;
using System.Text.Json;
using Dapper;
using ErsatzTV.Core.Api.Search;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
@@ -11,6 +14,62 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
private const int DefaultLimit = 50;
private const int MaxLimit = 50;
/// <summary>
/// Rows read per round trip when walking the list-valued (JSON-array) columns on
/// <c>SongMetadata</c>, and the ceiling on rows read per request.
/// <para>
/// These count ACTUAL ROWS, and arriving at that took four tries — each earlier attempt bounded a
/// quantity that sounded like rows and was not. A fixed <c>LIMIT</c> budget bounded the RESULT, and
/// the pre-filter (allowed to over-match) starved it with rows that could not match. Keyset paging
/// with a <c>LIMIT</c> bounded CANDIDATES RETURNED — but a query matching nothing must evaluate
/// every eligible row before it can return an empty page, so rows inspected stayed unbounded. A
/// closed <c>Id</c> range bounded KEYSPACE WIDTH — but keyspace is not rows: delete 20,000
/// historical rows, put one song at <c>Id</c> 20001, and the walk burns its whole allowance on empty
/// ranges and inspects nothing.
/// </para>
/// <para>
/// What makes this one hold is that <b>the query has no RESIDUAL predicate</b> — nothing that can
/// discard a row the engine already produced. The only condition is the cursor
/// <c>Id &gt; @AfterId</c>, which is a seek on the <c>ORDER BY</c> key itself, not a filter. So the
/// page returns exactly <see cref="ListValuedBatchRows" /> rows whenever that many logical rows
/// remain, independent of how sparse the matches are or where the <c>Id</c> gaps fall.
/// </para>
/// <para>
/// <b>Be precise about what is bounded: LOGICAL ROWS RETURNED AND MATERIALIZED, and the number of
/// round trips. Not physical work, and not bytes.</b> Two things break the stronger reading, and an
/// earlier version of this comment asserted it anyway:
/// <list type="bullet">
/// <item>
/// MySQL purge lag. Deleted clustered-index records survive until purge runs, and a range
/// scan still traverses them, so returning 2,000 VISIBLE rows can touch far more index
/// records. Deletion history therefore still affects physical work — the very thing the
/// keyspace attempt was trying to make irrelevant.
/// </item>
/// <item>
/// Row width is unbounded. These columns are <c>TEXT</c>/<c>longtext</c>, which both SQLite
/// and InnoDB spill to overflow pages, so a row count implies neither a byte count nor a
/// page-read count.
/// </item>
/// </list>
/// The logical-row bound is still worth having — it is what makes the walk terminate and what caps
/// the number of rows and round trips — but do not restate it as bounded I/O, and do not restate it
/// as bounded MEMORY either: payload width is unrestricted and a single JSON array can hold
/// arbitrarily many strings, every one of which may enter the in-memory set.
/// </para>
/// <para>
/// The trade is real and deliberate: no server-side narrowing, so a query with few matches transfers
/// rows it will discard, up to <see cref="ListValuedMaxRowsRead" />. A query with enough matches
/// stops as soon as it has <c>limit</c> distinct ones, so the dense cases — including an empty
/// <c>q</c> — finish on the first page. See <c>api.search-field-values-sources</c> for the measured
/// cost and for why reintroducing a <c>LIKE</c> is not an option.
/// </para>
/// </summary>
internal const int ListValuedBatchRows = 2000;
/// <inheritdoc cref="ListValuedBatchRows" />
internal const int ListValuedMaxRowsRead = 20000;
public async Task<Option<SearchFieldValuesResponseModel>> Handle(
GetSearchFieldValues request,
CancellationToken cancellationToken)
@@ -24,17 +83,22 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
}
int limit = request.Limit <= 0 ? DefaultLimit : Math.Clamp(request.Limit, 1, MaxLimit);
string qLower = (request.Query ?? string.Empty).ToLower();
string query = request.Query ?? string.Empty;
// Invariant, not current-culture: UseRequestLocalization honours Accept-Language, so a caller can select
// tr-TR and turn `q=I` into `ı` — which then matches nothing a Turkish-dotless-i-free library contains.
// This feeds the EF-translated filter, which has no StringComparison overload EF can translate.
string qLower = query.ToLowerInvariant();
// in-memory special cases (no DB query needed)
switch (request.Name)
{
case "state":
return new SearchFieldValuesResponseModel(
FilterSortTake(Enum.GetNames<MediaItemState>(), qLower, limit));
FilterSortTake(Enum.GetNames<MediaItemState>(), query, limit));
case "video_dynamic_range":
return new SearchFieldValuesResponseModel(
FilterSortTake(["hdr", "sdr"], qLower, limit));
FilterSortTake(["hdr", "sdr"], query, limit));
}
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
@@ -42,34 +106,75 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
if (request.Name == "content_rating")
{
return new SearchFieldValuesResponseModel(
await GetContentRatingValues(dbContext, qLower, limit, cancellationToken));
await GetContentRatingValues(dbContext, query, limit, cancellationToken));
}
IQueryable<string> source = GetSource(dbContext, request.Name);
if (source is null)
string listColumn = GetSongListValuedColumn(request.Name);
if (source is null && listColumn is null)
{
return Option<SearchFieldValuesResponseModel>.None;
}
List<string> values = await source
.Where(v => v != null && v.ToLower().StartsWith(qLower))
.Distinct()
.OrderBy(v => v)
.Take(limit)
.ToListAsync(cancellationToken);
var values = new List<string>();
return new SearchFieldValuesResponseModel(values);
if (source is not null)
{
values.AddRange(
await source
.Where(v => v != null && v.ToLower().StartsWith(qLower))
.Distinct()
.OrderBy(v => v)
.Take(limit)
.ToListAsync(cancellationToken));
}
// ersatztv#668. The query above prefix-matches through SQL LOWER(), and SQLite's LOWER() folds ASCII
// ONLY -- lower('Édith') is 'Édith' unchanged -- so it cannot reach a stored value whose prefix
// carries an uppercase non-ASCII character, from ANY query. It UNDER-matches, and an under-match is
// unrecoverable downstream: no later stage can reintroduce a row SQL never returned. So for the only
// queries that can be affected (those containing a non-ASCII character) run a second, Unicode-correct
// pass and merge it in. This is ADDITIVE on purpose -- the SQL pass above still contributes, so a
// value already reachable today cannot stop being reachable.
//
// MySQL needs none of this: its LOWER() is Unicode-aware, so LOWER('Édith') really is 'édith' and the
// existing predicate reaches the row unaided. Measured on 8.4 -- and note the executed path does NOT
// over-match, even though the column collation (utf8mb4_0900_ai_ci) is accent-insensitive: the driver
// binds the LIKE pattern with a BINARY collation, so the comparison is accent-sensitive in practice.
// A hand-typed probe using a LITERAL pattern DOES over-match; that is a different query from the one
// this code runs, and mistaking the two is how an earlier revision of the decision record got it wrong.
if (source is not null && ContainsNonAscii(query) && IsSqlite(dbContext))
{
values.AddRange(
await GetUnicodeFoldedValues(dbContext, request.Name, query, limit, cancellationToken));
}
if (listColumn is not null)
{
values.AddRange(await GetSongListValuedValues(dbContext, listColumn, query, limit, cancellationToken));
}
// ORDERING IS BEST-EFFORT, NOT EXACT. Each source truncates using its own ordering — the EF source by the
// database collation (SQLite's NOCASE/BINARY is ASCII-only), the list source by primary key — and neither
// is the ordinal ordering applied here. So when a source actually truncates, a value it dropped may have
// outranked one that survived: with "Zulu" and "apple" and limit=1 the database keeps "apple" (its
// ordering is case-insensitive) while ordinal ranks "Zulu" first, so the merge never sees "Zulu".
// Below the truncation points (the normal typeahead case) the result is exact.
return new SearchFieldValuesResponseModel(FilterSortTake(values.Distinct(StringComparer.Ordinal), query, limit));
}
private static IQueryable<string> GetSource(TvContext dbContext, string name) => name switch
internal static IQueryable<string> GetSource(TvContext dbContext, string name) => name switch
{
"genre" or "show_genre" => dbContext.Set<Genre>().Select(g => g.Name),
"studio" => dbContext.Set<Studio>().Select(s => s.Name),
"director" => dbContext.Set<Director>().Select(d => d.Name),
"writer" => dbContext.Set<Writer>().Select(w => w.Name),
"actor" => dbContext.Actors.Select(a => a.Name),
// entity artists only; free-text music-video/song artist credits are not included (known limitation)
"artist" => dbContext.ArtistMetadata.Select(m => m.Title),
// Mirrors what LuceneSearchIndex writes to the `artist` field: the music video's linked artist entity
// (ArtistMetadata.Title) plus its free-text credits (MusicVideoArtist rows). The third contributor —
// SongMetadata.Artists — is a JSON-array column and is handled by GetSongListValuedValues instead.
"artist" => dbContext.ArtistMetadata.Select(m => m.Title)
.Concat(dbContext.Set<MusicVideoArtist>().Select(a => a.Name)),
"tag" => dbContext.Set<Tag>()
.Where(t => t.ExternalTypeId != Tag.NfoCountryTypeId && t.ExternalTypeId != Tag.PlexNetworkTypeId)
.Select(t => t.Name),
@@ -87,9 +192,309 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
_ => null
};
/// <summary>
/// SQL name of the invariant-uppercase fold registered by <c>SqliteUnicodeFunctions</c>. Duplicated
/// rather than referenced because Application must not depend on a provider assembly; a test asserts
/// the two constants are equal so they cannot drift.
/// </summary>
internal const string UpperFunction = "etv_upper";
/// <summary>
/// True when the value contains any character outside US-ASCII, which is exactly when SQLite's
/// ASCII-only <c>LOWER()</c> can under-match. Evaluated on the RAW query, never the lowercased copy:
/// the trigger must not be coupled to the fold.
/// </summary>
internal static bool ContainsNonAscii(string value)
{
foreach (char c in value)
{
if (c > 0x7F)
{
return true;
}
}
return false;
}
// Derived per-context rather than read from the TvContext.IsSqlite static on purpose. Nothing MECHANICALLY
// stops that read -- ProviderStaticsWiringTests only parses the two composition roots for ASSIGNMENTS, not
// readers -- but that test's scanner exemption for IsSqlite is justified in prose as "read only by
// DbInitializer + DatabaseMigratorService, both host-only", and reading it here would make that reason
// false while the test stayed green. Do not "simplify" this to IsSqlite.
private static bool IsSqlite(TvContext dbContext) =>
(dbContext.Database.ProviderName ?? string.Empty).Contains("Sqlite", StringComparison.OrdinalIgnoreCase);
/// <summary>
/// Escapes the LIKE metacharacters in a user-supplied prefix and appends the trailing wildcard. The
/// backslash MUST be escaped first, or the escapes added for <c>%</c>/<c>_</c> would themselves be
/// re-escaped. Paired with an explicit <c>ESCAPE '\'</c> in <see cref="UnicodeFoldSql" />, since raw
/// SQL gets none of the escaping EF does for <c>StartsWith</c>.
/// </summary>
internal static string EscapeLikePrefix(string value) =>
value
.Replace("\\", "\\\\", StringComparison.Ordinal)
.Replace("%", "\\%", StringComparison.Ordinal)
.Replace("_", "\\_", StringComparison.Ordinal) + "%";
/// <summary>
/// One bounded, exact prefix query using the Unicode-correct fold. Unlike the list-valued walk this
/// KEEPS its selectivity in SQL — it is a normal indexed-or-not <c>LIMIT</c>ed query exactly like the
/// EF one it supplements, not a paged walk, so there is no row budget to blow and no reason to strip
/// the discriminator predicates out of it.
/// </summary>
internal static string UnicodeFoldSql(string table, string column, string predicate)
{
var match = $"{UpperFunction}({column}) LIKE @Pattern ESCAPE '\\'";
string where = predicate is null ? match : $"({predicate}) AND {match}";
return $"SELECT DISTINCT {column} AS Value FROM {table} WHERE {where} ORDER BY {column} LIMIT @Limit";
}
/// <summary>
/// The tables/columns behind each EF-sourced field, mirroring <see cref="GetSource" /> 1:1.
/// <para>
/// The discriminator predicates must mirror EF's NULL semantics, not C#'s reading of the source.
/// EF compiles <c>t.ExternalTypeId != Tag.NfoCountryTypeId</c> with null semantics, so a row whose
/// <c>ExternalTypeId</c> is NULL IS included; plain SQL <c>&lt;&gt;</c> against NULL yields NULL and
/// would silently drop it. Hence the explicit <c>IS NULL</c> arm.
/// </para>
/// </summary>
private static IReadOnlyList<UnicodeFoldSource> GetUnicodeFoldSources(string name) => name switch
{
"genre" or "show_genre" => [new UnicodeFoldSource("Genre", "Name")],
"studio" => [new UnicodeFoldSource("Studio", "Name")],
"director" => [new UnicodeFoldSource("Director", "Name")],
"writer" => [new UnicodeFoldSource("Writer", "Name")],
"actor" => [new UnicodeFoldSource("Actor", "Name")],
"artist" =>
[
new UnicodeFoldSource("ArtistMetadata", "Title"),
new UnicodeFoldSource("MusicVideoArtist", "Name")
],
"tag" =>
[
new UnicodeFoldSource(
"Tag",
"Name",
"ExternalTypeId IS NULL OR (ExternalTypeId <> @NfoCountryTypeId AND ExternalTypeId <> @PlexNetworkTypeId)",
new Dictionary<string, object>
{
["NfoCountryTypeId"] = Tag.NfoCountryTypeId,
["PlexNetworkTypeId"] = Tag.PlexNetworkTypeId
})
],
"network" =>
[
new UnicodeFoldSource(
"Tag",
"Name",
"ExternalTypeId = @PlexNetworkTypeId",
new Dictionary<string, object> { ["PlexNetworkTypeId"] = Tag.PlexNetworkTypeId })
],
"collection" => [new UnicodeFoldSource("Collection", "Name")],
"video_codec" =>
[
new UnicodeFoldSource(
"MediaStream",
"Codec",
"MediaStreamKind = @VideoStreamKind AND Codec IS NOT NULL",
new Dictionary<string, object> { ["VideoStreamKind"] = (int)MediaStreamKind.Video })
],
"album" =>
[
new UnicodeFoldSource("MusicVideoMetadata", "Album", "Album IS NOT NULL"),
new UnicodeFoldSource("SongMetadata", "Album", "Album IS NOT NULL")
],
_ => []
};
private static async Task<List<string>> GetUnicodeFoldedValues(
TvContext dbContext,
string name,
string query,
int limit,
CancellationToken cancellationToken)
{
IReadOnlyList<UnicodeFoldSource> sources = GetUnicodeFoldSources(name);
if (sources.Count == 0)
{
return [];
}
// CreateFunction is per-connection, so registration happens here, at the one call site that needs
// the function, rather than through an EF connection interceptor: Dapper opens a closed connection
// itself and a direct ADO open does not raise EF's interceptors, so an interceptor-based seam would
// silently miss exactly this query. Opening first makes the registration order-independent.
await dbContext.Database.OpenConnectionAsync(cancellationToken);
TvContext.RegisterUnicodeCaseFunctions(dbContext.Connection);
string pattern = EscapeLikePrefix(query.ToUpperInvariant());
var values = new List<string>();
foreach (UnicodeFoldSource source in sources)
{
var parameters = new DynamicParameters();
parameters.Add("Pattern", pattern);
parameters.Add("Limit", limit);
if (source.Parameters is not null)
{
foreach ((string key, object value) in source.Parameters)
{
parameters.Add(key, value);
}
}
IEnumerable<string> rows = await dbContext.Connection.QueryAsync<string>(
new CommandDefinition(
UnicodeFoldSql(source.Table, source.Column, source.Predicate),
parameters,
cancellationToken: cancellationToken));
values.AddRange(rows.Where(v => !string.IsNullOrEmpty(v)));
}
return values;
}
private sealed record UnicodeFoldSource(
string Table,
string Column,
string Predicate = null,
IReadOnlyDictionary<string, object> Parameters = null);
/// <summary>
/// Maps a field name onto the <c>SongMetadata</c> column that backs it as an <c>IList&lt;string&gt;</c>.
/// The returned value is a compile-time constant from this switch — never caller input — so it is safe
/// to interpolate into the SQL in <see cref="ListValuedSql" />.
/// </summary>
private static string GetSongListValuedColumn(string name) => name switch
{
"artist" => "Artists",
"album_artist" => "AlbumArtists",
_ => null
};
/// <summary>
/// Reads whole values out of a <c>SongMetadata</c> <c>IList&lt;string&gt;</c> column.
/// <para>
/// EF maps these as primitive collections: one JSON array per row in a single <c>TEXT</c>/
/// <c>longtext</c> column. Neither provider can project the elements server-side — SQLite needs
/// the SQL <c>APPLY</c> operator it doesn't have, and Pomelo MySQL doesn't implement primitive
/// collections at all — so there is no server-side <c>SELECT DISTINCT</c> over the elements.
/// </para>
/// <para>
/// So the rows are walked in primary-key order, keyset-paged by row position, and split +
/// exact-filtered in memory. All selectivity is in memory — the query's only condition is the
/// cursor, a seek on the ordering key that never discards a row, so its <c>LIMIT</c> bounds the
/// LOGICAL ROWS returned. See <see cref="ListValuedBatchRows" /> for the four revisions it took to
/// get that right, and for what that bound does and does not cover.
/// </para>
/// </summary>
private static async Task<List<string>> GetSongListValuedValues(
TvContext dbContext,
string column,
string query,
int limit,
CancellationToken cancellationToken)
{
string sql = ListValuedSql(column);
var distinct = new System.Collections.Generic.HashSet<string>(StringComparer.Ordinal);
var afterId = 0;
var read = 0;
while (read < ListValuedMaxRowsRead && distinct.Count < limit)
{
int batch = Math.Min(ListValuedBatchRows, ListValuedMaxRowsRead - read);
List<ListValuedRow> rows = (await dbContext.Connection.QueryAsync<ListValuedRow>(
new CommandDefinition(
sql,
new { AfterId = afterId, Batch = batch },
cancellationToken: cancellationToken))).AsList();
if (rows.Count == 0)
{
break;
}
read += rows.Count;
afterId = rows[^1].Id;
foreach (ListValuedRow row in rows)
{
foreach (string element in ParseElements(row.Payload))
{
if (element.StartsWith(query, StringComparison.OrdinalIgnoreCase))
{
distinct.Add(element);
}
}
}
if (rows.Count < batch)
{
// With no RESIDUAL predicate -- only the cursor, which selects a range rather than discarding
// rows from it -- a short page can only mean the table is exhausted. It can never mean "this
// stretch happened to match nothing", which is precisely why the residual predicate had to go.
// Advancing from the last returned Id is safe for the same reason: nothing was filtered out
// behind it, so no row can be skipped.
break;
}
}
return distinct.ToList();
}
private static IEnumerable<string> ParseElements(string payload)
{
if (string.IsNullOrWhiteSpace(payload))
{
return [];
}
try
{
return (JsonSerializer.Deserialize<string[]>(payload) ?? []).Where(e => !string.IsNullOrEmpty(e));
}
catch (JsonException)
{
return [];
}
}
/// <summary>
/// One keyset page of rows, by ROW POSITION rather than by <c>Id</c> value.
/// <para>
/// The only condition is the cursor — deliberately <b>no RESIDUAL predicate</b>: no <c>LIKE</c>, no
/// <c>LOWER</c>, not even <c>IS NOT NULL</c>. The distinction that matters is not "no predicate"
/// (the cursor is one); it is that <c>Id &gt; @AfterId</c> is a <i>seekable predicate on the
/// ordering key</i>, which positions the scan and never discards a row, whereas a residual
/// predicate throws away rows the engine already produced. <c>LIMIT</c> only truncates what
/// survives a residual predicate, so with one present it bounds the output rather than the row
/// count — which is how every earlier revision scanned past its own bound. With none, <c>LIMIT n</c>
/// yields <c>n</c> logical rows. Null payloads are dropped in memory by
/// <see cref="ParseElements" />.
/// </para>
/// <para>
/// Note this pins the SQL string only. It cannot pin an execution plan, MVCC visibility work, or
/// payload I/O — and on MySQL, using the index to satisfy <c>ORDER BY</c> is an optimizer choice,
/// not a semantic guarantee.
/// </para>
/// </summary>
internal static string ListValuedSql(string column) =>
$"SELECT Id, {column} AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch";
private sealed class ListValuedRow
{
public int Id { get; init; }
public string Payload { get; init; }
}
private static async Task<List<string>> GetContentRatingValues(
TvContext dbContext,
string qLower,
string query,
int limit,
CancellationToken cancellationToken)
{
@@ -108,13 +513,22 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
.Where(cr => !string.IsNullOrEmpty(cr))
.Distinct();
return FilterSortTake(split, qLower, limit);
return FilterSortTake(split, query, limit);
}
private static List<string> FilterSortTake(IEnumerable<string> values, string qLower, int limit) =>
/// <summary>
/// The one in-memory filter/sort/take every field funnels through. Both the comparison and the ordering
/// are ORDINAL on purpose: <c>UseRequestLocalization</c> honours <c>Accept-Language</c>, so the current
/// culture is caller-controlled, and <c>ToLower()</c> plus the default (linguistic)
/// <c>StartsWith(string)</c> would make the result depend on it — under <c>tr-TR</c>, <c>q=I</c> lowers
/// to <c>ı</c> and stops matching <c>Istanbul</c>. Note this is the LAST stage only: a field sourced by
/// a plain EF query has already been filtered and truncated by the database collation before it gets
/// here, which ordinal semantics downstream cannot undo (ersatztv#668).
/// </summary>
private static List<string> FilterSortTake(IEnumerable<string> values, string query, int limit) =>
values
.Where(v => v.ToLower().StartsWith(qLower))
.OrderBy(v => v)
.Where(v => v.StartsWith(query, StringComparison.OrdinalIgnoreCase))
.OrderBy(v => v, StringComparer.Ordinal)
.Take(limit)
.ToList();
}
@@ -0,0 +1,129 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.Interfaces.FFmpeg;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.FFmpeg.State;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Core.Tests.FFmpeg;
[TestFixture]
public class SongVideoGeneratorTests
{
private ITempFilePool _tempFilePool;
private IImageCache _imageCache;
private IFFmpegProcessService _ffmpegProcessService;
private ILocalFileSystem _localFileSystem;
private SongVideoGenerator _songVideoGenerator;
private string _tempSubtitleFile;
[SetUp]
public void SetUp()
{
_tempSubtitleFile = Path.Combine(Path.GetTempPath(), $"{Guid.NewGuid()}.ass");
_tempFilePool = Substitute.For<ITempFilePool>();
_tempFilePool.GetNextTempFile(Arg.Any<TempFileCategory>()).Returns(_tempSubtitleFile);
_imageCache = Substitute.For<IImageCache>();
_imageCache.GetPathForImage(Arg.Any<string>(), Arg.Any<ArtworkKind>(), Arg.Any<Option<int>>())
.Returns("/fake/watermark.png");
_ffmpegProcessService = Substitute.For<IFFmpegProcessService>();
_ffmpegProcessService.GenerateSongImage(
Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<Option<string>>(),
Arg.Any<Channel>(),
Arg.Any<MediaVersion>(),
Arg.Any<string>(),
Arg.Any<bool>(),
Arg.Any<Option<string>>(),
Arg.Any<WatermarkLocation>(),
Arg.Any<int>(),
Arg.Any<int>(),
Arg.Any<int>(),
Arg.Any<CancellationToken>())
.Returns(Either<BaseError, string>.Right("/fake/song-image.png"));
_localFileSystem = Substitute.For<ILocalFileSystem>();
_localFileSystem.GetCustomOrDefaultFile(Arg.Any<string>(), Arg.Any<string>())
.Returns("/fake/background.png");
_songVideoGenerator = new SongVideoGenerator(
_tempFilePool,
_imageCache,
_ffmpegProcessService,
_localFileSystem);
}
[TearDown]
public void TearDown()
{
if (_tempSubtitleFile is not null && File.Exists(_tempSubtitleFile))
{
File.Delete(_tempSubtitleFile);
}
}
private static Channel BuildChannel()
{
var resolution = new Resolution { Width = 1920, Height = 1080 };
FFmpegProfile ffmpegProfile = FFmpegProfile.New("test", resolution);
return new Channel(Guid.NewGuid())
{
Number = "1",
Name = "Test Channel",
FFmpegProfile = ffmpegProfile,
SongVideoMode = ChannelSongVideoMode.Default
};
}
private static Song BuildUntaggedSong()
{
// an untagged song: FallbackMetadataProvider.GetSongMetadata never assigns
// Artists/AlbumArtists, so they persist (and materialize) as null (ersatztv#691)
var metadata = new SongMetadata
{
MetadataKind = MetadataKind.Fallback,
Title = "Untagged Song",
Artwork = [],
Artists = null,
AlbumArtists = null
};
return new Song
{
SongMetadata = [metadata],
MediaVersions = []
};
}
[Test]
public async Task GenerateSongVideo_should_not_throw_when_artists_and_album_artists_are_null()
{
Song song = BuildUntaggedSong();
Channel channel = BuildChannel();
// SongVideoGenerator randomly picks between two rendering styles (and dereferences
// metadata.Artists/AlbumArtists differently in each); loop enough times that both
// branches -- including the AlbumArtists.Filter(... Artists.Contains ...) branch --
// are exercised with overwhelming probability, so the null guard is proven on both.
for (var i = 0; i < 25; i++)
{
Tuple<string, MediaVersion> result = await _songVideoGenerator.GenerateSongVideo(
song,
channel,
"/usr/bin/ffmpeg",
"/usr/bin/ffprobe",
CancellationToken.None);
result.ShouldNotBeNull();
result.Item1.ShouldBe("/fake/song-image.png");
}
}
}
@@ -1,4 +1,4 @@
namespace ErsatzTV.Core.Domain;
namespace ErsatzTV.Core.Domain;
public class SongMetadata : Metadata
{
+10 -7
View File
@@ -1,4 +1,4 @@
using System.Globalization;
using System.Globalization;
using System.Text;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.FFmpeg;
@@ -85,6 +85,9 @@ public class SongVideoGenerator : ISongVideoGenerator
var sb = new StringBuilder();
List<string> artists = Optional(metadata.Artists).Flatten().ToList();
List<string> albumArtists = Optional(metadata.AlbumArtists).Flatten().ToList();
if (detailsStyle)
{
if (!string.IsNullOrWhiteSpace(metadata.Title))
@@ -92,17 +95,17 @@ public class SongVideoGenerator : ISongVideoGenerator
sb.Append(CultureInfo.InvariantCulture, $"{{\\fs{largeFontSize}}}{metadata.Title}");
}
if (metadata.Artists.Count > 0)
if (artists.Count > 0)
{
var allArtists = string.Join(", ", metadata.Artists);
var allArtists = string.Join(", ", artists);
sb.Append(CultureInfo.InvariantCulture, $"\\N{{\\fs{fontSize}}}{allArtists}");
}
}
else
{
if (metadata.Artists.Count > 0)
if (artists.Count > 0)
{
var allArtists = string.Join(", ", metadata.Artists);
var allArtists = string.Join(", ", artists);
sb.Append(allArtists);
}
@@ -111,11 +114,11 @@ public class SongVideoGenerator : ISongVideoGenerator
sb.Append(CultureInfo.InvariantCulture, $"\\N\"{metadata.Title}\"");
}
if (metadata.AlbumArtists.Count > 0)
if (albumArtists.Count > 0)
{
var allAlbumArtists = string.Join(
", ",
metadata.AlbumArtists.Filter(aa => !metadata.Artists.Contains(aa)));
albumArtists.Filter(aa => !artists.Contains(aa)));
sb.Append(CultureInfo.InvariantCulture, $"\\N{allAlbumArtists}");
}
@@ -0,0 +1,57 @@
using System.Data;
using Microsoft.Data.Sqlite;
namespace ErsatzTV.Infrastructure.Sqlite.Data;
/// <summary>
/// ersatztv#668. SQLite's built-in <c>lower()</c>/<c>upper()</c> fold ASCII ONLY — <c>lower('Édith')</c>
/// returns <c>'Édith'</c> unchanged — so a facet value whose prefix carries an uppercase non-ASCII
/// character can never be matched by the prefix predicate the facet-value endpoint emits. Registering a
/// managed scalar gives that one query a Unicode-correct fold. Wired to
/// <see cref="ErsatzTV.Infrastructure.Data.TvContext.RegisterUnicodeCaseFunctions" /> at startup.
/// </summary>
public static class SqliteUnicodeFunctions
{
/// <summary>
/// SQL name of the invariant-uppercase fold. The facet-value handler interpolates this constant into
/// its SQL, so the two cannot drift apart.
/// </summary>
public const string UpperInvariantFunction = "etv_upper";
/// <summary>
/// Registers <see cref="UpperInvariantFunction" /> on <paramref name="connection" /> when it is a
/// SQLite connection, and does nothing otherwise. Idempotent — a repeat registration replaces the
/// previous delegate with an identical one — so the single call site may call it unconditionally.
/// <para>
/// The property this fold has to satisfy is ONE-SIDED: the SQL stage may over-match freely,
/// because the endpoint applies an exact <see cref="StringComparison.OrdinalIgnoreCase" /> filter
/// in memory afterwards, but it must never UNDER-match — no later stage can reintroduce a row SQL
/// never returned. <see cref="string.ToUpperInvariant" /> satisfies it because
/// <c>OrdinalIgnoreCase</c> equality is a strict SUBSET of invariant-uppercase equality, so
/// folding both sides with it yields a superset of the final filter's matches.
/// </para>
/// <para>
/// Do not restate that as "<c>OrdinalIgnoreCase</c> IS invariant-uppercase-then-ordinal" — it is
/// not, and the difference is measurable: <c>char.ToUpperInvariant('ſ')</c> (U+017F) is <c>'S'</c>,
/// yet <c>"ſweet".StartsWith("S", OrdinalIgnoreCase)</c> is <b>false</b>. That gap is precisely
/// the harmless direction — SQL returns the row, the in-memory filter drops it. The containment,
/// not any identity of the two foldings, is what makes this safe.
/// </para>
/// <para>
/// Registration is per-connection and therefore done at the one call site that uses the function,
/// not through an EF connection interceptor: Dapper opens a closed connection itself, and a direct
/// ADO open does not raise EF's interceptors — so an interceptor-based seam would silently miss
/// exactly the query that needs it.
/// </para>
/// </summary>
public static void Register(IDbConnection connection)
{
if (connection is SqliteConnection sqlite)
{
sqlite.CreateFunction(
UpperInvariantFunction,
(string? value) => value?.ToUpperInvariant(),
isDeterministic: true);
}
}
}
@@ -1144,7 +1144,7 @@ public class MediaCollectionRepository : IMediaCollectionRepository
var allArtists = items.OfType<Song>()
.SelectMany(s => s.SongMetadata)
.Map(sm => sm.AlbumArtists.HeadOrNone().Match(aa => aa, string.Empty))
.Map(sm => Optional(sm.AlbumArtists).Flatten().HeadOrNone().Match(aa => aa, string.Empty))
.Distinct()
.ToList();
@@ -1157,7 +1157,7 @@ public class MediaCollectionRepository : IMediaCollectionRepository
foreach (Song song in items.OfType<Song>())
{
string firstArtist = song.SongMetadata
.SelectMany(sm => sm.AlbumArtists)
.SelectMany(sm => Optional(sm.AlbumArtists).Flatten())
.HeadOrNone()
.Match(aa => aa, string.Empty);
+12
View File
@@ -36,6 +36,18 @@ public class TvContext : DbContext
/// </summary>
public static Func<DbUpdateException, bool> IsUniqueConstraintViolation { get; set; } = static _ => false;
/// <summary>
/// Registers provider-specific SQL scalar functions on a connection, called immediately before a raw
/// query that needs them. Set at startup by the active provider's wiring, mirroring
/// <see cref="IsUniqueConstraintViolation" />: SQLite points this at
/// <c>SqliteUnicodeFunctions.Register</c>, MySQL leaves it a no-op because its own <c>LOWER()</c> is
/// already Unicode-aware and needs no help. Defaults to a no-op, which is safe because the sole
/// caller invokes it only on the SQLite branch that requires it, and an unwired provider then fails
/// LOUDLY ("no such function: etv_upper") rather than returning silently wrong results. See
/// ersatztv#668.
/// </summary>
public static Action<IDbConnection> RegisterUnicodeCaseFunctions { get; set; } = static _ => { };
public IDbConnection Connection => Database.GetDbConnection();
public DbSet<ConfigElement> ConfigElements { get; set; }
+5
View File
@@ -162,6 +162,7 @@ public class Program
TvContext.LastInsertedRowId = "last_insert_rowid()";
TvContext.CaseInsensitiveCollation = "NOCASE";
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
TvContext.RegisterUnicodeCaseFunctions = SqliteUnicodeFunctions.Register;
SqlMapper.AddTypeHandler(new DateTimeOffsetHandler());
SqlMapper.AddTypeHandler(new GuidHandler());
@@ -173,6 +174,10 @@ public class Program
TvContext.LastInsertedRowId = "last_insert_id()";
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
// MySQL's LOWER() is already Unicode-aware; assigned explicitly for the same reason as
// the host — a provider switch must not inherit SQLite's registration.
TvContext.RegisterUnicodeCaseFunctions = static _ => { };
}
services.AddHttpClient();
@@ -0,0 +1,89 @@
using ErsatzTV.Application.MediaCollections;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Application.MediaCollections;
/// <summary>
/// The second consumer of the shared <c>ProjectMediaItemToViewModel</c> switch (issue #671).
/// <c>GetPlaylistItemsHandler</c> had no handler-level test — the controller tests stub the mediator
/// and never execute the query — so the only symptom of a missing include here was a silent "???"
/// name that nothing in the suite could see. Widening the shared switch with a RemoteStream arm
/// obliged this handler to gain a matching include; proving that by inspection would have repeated
/// the very method that produced #671, so it gets the same full matrix the rerun handlers get.
/// </summary>
[TestFixture]
public class GetPlaylistItemsHandlerTests : MediaCollectionHandlerTestBase
{
private static IEnumerable<CollectionType> SupportedSelectionTypes => SelectionSeedData.SupportedSelectionTypes;
[TestCaseSource(nameof(SupportedSelectionTypes))]
public async Task GetPlaylistItems_Should_Resolve_The_Selection(CollectionType collectionType)
{
await SeedSelection(collectionType);
await SeedPlaylistItem(collectionType);
var handler = new GetPlaylistItemsHandler(Db.Factory);
List<PlaylistItemViewModel> items =
await handler.Handle(new GetPlaylistItems(1), CancellationToken.None);
items.Count.ShouldBe(1);
PlaylistItemViewModel item = items[0];
int? selectedId = item.Collection?.Id
?? item.MultiCollection?.Id
?? item.SmartCollection?.Id
?? item.MediaItem?.MediaItemId;
string selectedName = item.Collection?.Name
?? item.MultiCollection?.Name
?? item.SmartCollection?.Name
?? item.MediaItem?.Name;
selectedId.ShouldBe(SelectionSeedData.SelectedId, $"{collectionType} lost its selected id");
selectedName.ShouldBe(
SelectionSeedData.ExpectedName(collectionType),
$"{collectionType} projected the wrong name");
}
private async Task SeedSelection(CollectionType collectionType)
{
await using TvContext context = Db.CreateContext();
await SelectionSeedData.SeedSelection(context, collectionType);
}
private async Task SeedPlaylistItem(CollectionType collectionType)
{
await using TvContext context = Db.CreateContext();
var item = new PlaylistItem
{
Id = 1,
Index = 0,
PlaylistId = 1,
CollectionType = collectionType,
PlaybackOrder = PlaybackOrder.Chronological
};
SelectionSeedData.ApplySelection(
collectionType,
v => item.CollectionId = v,
v => item.MultiCollectionId = v,
v => item.SmartCollectionId = v,
v => item.MediaItemId = v);
context.Playlists.Add(new Playlist
{
Id = 1,
Name = "Playlist",
Items = [item]
});
await context.SaveChangesAsync();
}
}
@@ -0,0 +1,166 @@
using ErsatzTV.Application.MediaCollections;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Application.MediaCollections;
/// <summary>
/// Read-path coverage for the two rerun-collection query handlers (issue #671). The defect was
/// precisely that nobody enumerated the selection types: the list handler eager-loaded nothing, and
/// the by-id handler loaded metadata for only four of the ten media types. So the matrix is derived
/// from the production predicate (see <see cref="SelectionSeedData" />) rather than hand-listed.
/// </summary>
[TestFixture]
public class RerunCollectionQueryHandlerTests : MediaCollectionHandlerTestBase
{
private static IEnumerable<CollectionType> SupportedSelectionTypes => SelectionSeedData.SupportedSelectionTypes;
/// <summary>
/// Completeness guard. Without it, a change that narrowed <c>IsSupportedSelectionType</c> would
/// shrink the matrix silently and every remaining case would still pass — the "filters on the
/// property it asserts" failure mode. Set equality, so it fails on widening too.
/// </summary>
[Test]
public void Supported_Selection_Types_Should_Be_The_Full_Documented_Set()
{
SupportedSelectionTypes.ShouldBe(
[
CollectionType.Collection,
CollectionType.TelevisionShow,
CollectionType.TelevisionSeason,
CollectionType.Artist,
CollectionType.MultiCollection,
CollectionType.SmartCollection,
CollectionType.Movie,
CollectionType.Episode,
CollectionType.MusicVideo,
CollectionType.OtherVideo,
CollectionType.Song,
CollectionType.Image,
CollectionType.RemoteStream
],
ignoreOrder: true);
}
[TestCaseSource(nameof(SupportedSelectionTypes))]
public async Task GetById_Should_Resolve_The_Selection(CollectionType collectionType)
{
await SeedSelection(collectionType);
await SeedRerunCollection(1, collectionType);
var handler = new GetRerunCollectionByIdHandler(Db.Factory);
Option<RerunCollectionViewModel> result =
await handler.Handle(new GetRerunCollectionById(1), CancellationToken.None);
RerunCollectionViewModel vm = result.IfNone(() => throw new AssertionException("Expected a result"));
AssertSelectionResolved(vm, collectionType);
}
[TestCaseSource(nameof(SupportedSelectionTypes))]
public async Task GetPaged_Should_Resolve_The_Selection(CollectionType collectionType)
{
await SeedSelection(collectionType);
await SeedRerunCollection(1, collectionType);
var handler = new GetPagedRerunCollectionsHandler(Db.Factory);
PagedRerunCollectionsViewModel result = await handler.Handle(
new GetPagedRerunCollections(string.Empty, 0, 10),
CancellationToken.None);
result.Page.Count.ShouldBe(1);
AssertSelectionResolved(result.Page[0], collectionType);
}
/// <summary>
/// <c>SongMetadata.Artists</c> is a NULLABLE primitive collection, and a song whose tags failed to
/// read is persisted with it never assigned. Before #671 the rerun list did not load SongMetadata
/// at all, so this was unreachable there; eager-loading it made a latent `string.Join` throw into a
/// live 500 that would take down the whole page.
/// </summary>
[TestCase(null, "Selected song", TestName = "GetById_Song_With_Null_Artists_Should_Not_Throw")]
[TestCase(new string[] { }, "Selected song", TestName = "GetById_Song_With_No_Artists_Should_Not_Prefix")]
public async Task GetById_Should_Tolerate_Song_Artists(string[] artists, string expectedName)
{
await using (TvContext context = Db.CreateContext())
{
context.Songs.Add(new Song
{
Id = SelectionSeedData.SelectedId,
SongMetadata = [new SongMetadata { Title = "Selected song", Artists = artists?.ToList() }]
});
await context.SaveChangesAsync();
}
await SeedRerunCollection(1, CollectionType.Song);
var handler = new GetRerunCollectionByIdHandler(Db.Factory);
Option<RerunCollectionViewModel> result =
await handler.Handle(new GetRerunCollectionById(1), CancellationToken.None);
RerunCollectionViewModel vm = result.IfNone(() => throw new AssertionException("Expected a result"));
vm.MediaItem.ShouldNotBeNull();
vm.MediaItem.MediaItemId.ShouldBe(SelectionSeedData.SelectedId);
vm.MediaItem.Name.ShouldBe(expectedName);
}
/// <summary>
/// Mirrors <c>RerunCollectionController.ProjectToResponseModel</c>, which flattens the tagged
/// union to the single <c>selectedId</c> / <c>selectedName</c> pair the SPA consumes. The id is
/// the load-bearing half: the editor round-trips it, so a null there silently clears the user's
/// stored selection.
/// </summary>
private static void AssertSelectionResolved(RerunCollectionViewModel vm, CollectionType collectionType)
{
int? selectedId = vm.Collection?.Id
?? vm.MultiCollection?.Id
?? vm.SmartCollection?.Id
?? vm.MediaItem?.MediaItemId;
string selectedName = vm.Collection?.Name
?? vm.MultiCollection?.Name
?? vm.SmartCollection?.Name
?? vm.MediaItem?.Name;
selectedId.ShouldBe(SelectionSeedData.SelectedId, $"{collectionType} lost its selected id");
selectedName.ShouldBe(
SelectionSeedData.ExpectedName(collectionType),
$"{collectionType} projected the wrong name");
}
private async Task SeedSelection(CollectionType collectionType)
{
await using TvContext context = Db.CreateContext();
await SelectionSeedData.SeedSelection(context, collectionType);
}
private async Task SeedRerunCollection(int id, CollectionType collectionType)
{
await using TvContext context = Db.CreateContext();
var rerunCollection = new RerunCollection
{
Id = id,
Name = "Rerun",
CollectionType = collectionType,
FirstRunPlaybackOrder = PlaybackOrder.Chronological,
RerunPlaybackOrder = PlaybackOrder.Chronological
};
SelectionSeedData.ApplySelection(
collectionType,
v => rerunCollection.CollectionId = v,
v => rerunCollection.MultiCollectionId = v,
v => rerunCollection.SmartCollectionId = v,
v => rerunCollection.MediaItemId = v);
context.RerunCollections.Add(rerunCollection);
await context.SaveChangesAsync();
}
}
@@ -0,0 +1,96 @@
using ErsatzTV.Core.Domain;
using LanguageExt;
using NUnit.Framework;
using Shouldly;
using Mapper = ErsatzTV.Application.Playouts.Mapper;
namespace ErsatzTV.Tests.Application.Playouts;
/// <summary>
/// <c>SongMetadata.Artists</c> is a nullable EF primitive collection that
/// <c>FallbackMetadataProvider</c> leaves unassigned for a song whose tags failed to read, and
/// <c>string.Join</c> throws <see cref="ArgumentNullException" /> on a null sequence. Because
/// <c>SongMetadata</c> IS eager-loaded on the playout paths, this was a LIVE 500 rather than a
/// latent one — and <c>GetDisplayTitle</c> feeds the playout guide, troubleshooting, media-item
/// info and channel states alike (issue #671).
/// </summary>
[TestFixture]
public class PlayoutMapperDisplayTitleTests
{
[Test]
public void GetDisplayTitle_Should_Not_Throw_When_Song_Artists_Is_Null()
{
var song = new Song
{
Id = 1,
SongMetadata = [new SongMetadata { Title = "Untagged", Artists = null }]
};
string title = Mapper.GetDisplayTitle(song, Option<string>.None);
title.ShouldBe("Untagged");
}
[Test]
public void GetDisplayTitle_Should_Not_Prefix_When_Song_Has_No_Artists()
{
var song = new Song
{
Id = 1,
SongMetadata = [new SongMetadata { Title = "Untagged", Artists = [] }]
};
string title = Mapper.GetDisplayTitle(song, Option<string>.None);
title.ShouldBe("Untagged");
}
[Test]
public void GetDisplayTitle_Should_Prefix_The_Artists_When_Present()
{
var song = new Song
{
Id = 1,
SongMetadata = [new SongMetadata { Title = "Tagged", Artists = ["A", "B"] }]
};
string title = Mapper.GetDisplayTitle(song, Option<string>.None);
title.ShouldBe("A, B - Tagged");
}
/// <summary>
/// The chapter branch interpolated the `case Song s` ENTITY rather than the composed title, and
/// <see cref="Song" /> has no <c>ToString()</c> override — so a chaptered song rendered as the
/// literal "ErsatzTV.Core.Domain.Song (Chapter 1)". Pre-existing; the sibling MusicVideo and
/// OtherVideo arms are correct only because they name their lambda parameter `s` too.
/// </summary>
[Test]
public void GetDisplayTitle_Should_Compose_The_Title_Not_The_Entity_When_Chaptered()
{
var song = new Song
{
Id = 1,
SongMetadata = [new SongMetadata { Title = "Tagged", Artists = ["A"] }]
};
string title = Mapper.GetDisplayTitle(song, Option<string>.Some("Chapter 1"));
title.ShouldBe("A - Tagged (Chapter 1)");
title.ShouldNotContain("ErsatzTV.Core.Domain");
}
[Test]
public void GetDisplayTitle_Should_Not_Throw_When_Chaptered_Song_Has_Null_Artists()
{
var song = new Song
{
Id = 1,
SongMetadata = [new SongMetadata { Title = "Untagged", Artists = null }]
};
string title = Mapper.GetDisplayTitle(song, Option<string>.Some("Chapter 2"));
title.ShouldBe("Untagged (Chapter 2)");
}
}
@@ -1,9 +1,11 @@
using System.Globalization;
using ErsatzTV.Application.Search.Queries;
using ErsatzTV.Core.Api.Search;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
using Microsoft.EntityFrameworkCore;
using NUnit.Framework;
using Shouldly;
@@ -176,6 +178,461 @@ public class GetSearchFieldValuesHandlerTests
networkResult.IfSome(r => r.Values.ShouldBe(new List<string> { "HBO" }));
}
[Test]
public async Task Artist_Merges_Entity_Artists_Music_Video_Credits_And_Song_Credits()
{
await using (TvContext context = _db.CreateContext())
{
context.ArtistMetadata.Add(Artist("Alpha Entity"));
// negative control: an entity artist that must NOT match the "al" prefix
context.ArtistMetadata.Add(Artist("Zeta Entity"));
context.MusicVideoMetadata.AddRange(
MusicVideo("MV One", "Alpha Credit", "Alpha Shared"),
// "Alpha Shared" appears in two rows, so DISTINCT has something to collapse
MusicVideo("MV Two", "Alpha Shared"),
MusicVideo("MV Three", "Zeta Credit"));
context.SongMetadata.AddRange(
Song("Song One", ["Alpha Song", "Zeta Song"]),
Song("Song Two", ["Alpha Song"]),
Song("Song Three", ["Zeta Only"]));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("artist", "al", 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(
new List<string> { "Alpha Credit", "Alpha Entity", "Alpha Shared", "Alpha Song" }));
}
[Test]
public async Task Artist_Returns_Every_Source_For_Empty_Query()
{
await using (TvContext context = _db.CreateContext())
{
context.ArtistMetadata.Add(Artist("Entity"));
context.MusicVideoMetadata.Add(MusicVideo("MV", "Credit"));
context.SongMetadata.Add(Song("Song", ["SongArtist"]));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("artist", string.Empty, 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Credit", "Entity", "SongArtist" }));
}
[Test]
public async Task Album_Artist_Returns_Song_Album_Artists_Instead_Of_NotFound()
{
await using (TvContext context = _db.CreateContext())
{
context.SongMetadata.AddRange(
Song("One", ["Performer"], ["Alpha Album Artist", "Beta Album Artist"]),
// repeated across rows so DISTINCT is exercised
Song("Two", ["Performer"], ["Alpha Album Artist"]),
// negative control: a row whose album artists are absent entirely
Song("Three", ["Performer"], null));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("album_artist", string.Empty, 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Alpha Album Artist", "Beta Album Artist" }));
// the performers on the same rows must not leak into album_artist
result.IfSome(r => r.Values.ShouldNotContain("Performer"));
}
[Test]
public async Task List_Valued_Fields_Match_Whole_Elements_Not_Substrings_And_Ignore_Neighbours()
{
await using (TvContext context = _db.CreateContext())
{
context.SongMetadata.AddRange(
// "Neighbour" arrives on the same row as "Radiohead" -- rows are read whole -- and must be
// dropped by the in-memory exact prefix filter.
Song("One", ["Radiohead", "Neighbour"]),
// "The Radio Dept." contains "radio" but does not start with it
Song("Two", ["The Radio Dept."]),
Song("Three", ["Radio Birdman"]));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("artist", "radio", 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Radio Birdman", "Radiohead" }));
}
[Test]
public async Task List_Valued_Fields_Match_Literally_Including_Json_Escaped_And_Wildcard_Characters()
{
await using (TvContext context = _db.CreateContext())
{
context.SongMetadata.AddRange(
// non-ASCII: stored on disk JSON-escaped as \u00E9, and must survive the round trip
Song("One", ["Beyoncé"]),
// an embedded quote is stored as \u0022
Song("Two", ["\"Weird Al\" Yankovic"]),
// SQL wildcards must be ordinary characters here, matched literally
Song("Three", ["50% Off"]),
Song("Four", ["50 Cent"]));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
(await handler.Handle(new GetSearchFieldValues("artist", "beyoncé", 50), CancellationToken.None))
.IfSome(r => r.Values.ShouldBe(new List<string> { "Beyoncé" }));
(await handler.Handle(new GetSearchFieldValues("artist", "\"weird", 50), CancellationToken.None))
.IfSome(r => r.Values.ShouldBe(new List<string> { "\"Weird Al\" Yankovic" }));
// "50%" must not behave as the wildcard "50<anything>" — "50 Cent" must not come back
(await handler.Handle(new GetSearchFieldValues("artist", "50%", 50), CancellationToken.None))
.IfSome(r => r.Values.ShouldBe(new List<string> { "50% Off" }));
}
/// <summary>
/// Seeds <paramref name="fillerRows" /> non-matching songs through raw SQL — 20k rows via the change
/// tracker is minutes, this is milliseconds.
/// </summary>
private static Task SeedFiller(TvContext context, int fillerRows) =>
context.Database.ExecuteSqlRawAsync(
$"""
WITH RECURSIVE seq(n) AS (SELECT 1 UNION ALL SELECT n + 1 FROM seq WHERE n < {fillerRows})
INSERT INTO SongMetadata (SongId, MetadataKind, Title, Artists, DateAdded, DateUpdated)
SELECT 0, 0, 'Filler ' || n, '["zzz-filler"]', '2026-01-01', '2026-01-01' FROM seq
""");
[Test]
public async Task List_Valued_Walk_Reads_At_Most_20000_Rows()
{
// Pinned in both directions so the ceiling itself is nailed down: a match in row 20000 is read, the same
// match in row 20001 is not. The query has no RESIDUAL predicate -- only the cursor -- so "rows read" is
// what LIMIT returns. That bounds LOGICAL rows, not physical work: the engine may still traverse more
// index records than it returns (MySQL purge lag), and row width is unbounded.
const string needle = "\u00E9clair-the-needle";
await using (TvContext context = _db.CreateContext())
{
await SeedFiller(context, 19999);
context.SongMetadata.Add(Song("Needle", [needle]));
await context.SaveChangesAsync();
(await context.SongMetadata.CountAsync()).ShouldBe(20000);
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
(await handler.Handle(new GetSearchFieldValues("artist", "\u00E9", 50), CancellationToken.None))
.IfSome(r => r.Values.ShouldBe(new List<string> { needle }, "row 20000 is inside the ceiling"));
await using (TvContext context = _db.CreateContext())
{
SongMetadata existing = await context.SongMetadata.SingleAsync(m => m.Title == "Needle");
context.SongMetadata.Remove(existing);
await SeedFiller(context, 1);
await context.SaveChangesAsync();
context.SongMetadata.Add(Song("Needle", [needle]));
await context.SaveChangesAsync();
(await context.SongMetadata.CountAsync()).ShouldBe(20001);
}
(await handler.Handle(new GetSearchFieldValues("artist", "\u00E9", 50), CancellationToken.None))
.IfSome(
r => r.Values.ShouldBeEmpty(
"row 20001 is past the ceiling; this false negative is the documented bounded-best-effort "
+ "contract, deliberately pinned rather than papered over"));
}
[Test]
public async Task List_Valued_Walk_Reads_Live_Rows_Regardless_Of_Id_Density()
{
// THE round-4 killer. That revision bounded the Id KEYSPACE, and keyspace is not rows: with 20,000
// historical rows deleted and one live song at Id 20001, the walk spent its whole allowance on empty
// ranges and returned [] for a table containing exactly one row. Capacity degraded linearly with
// deletion ratio, and no ratio was safe -- one placed gap hid the next match.
//
// Paging by row position rather than Id value makes density irrelevant: LIMIT @Batch returns @Batch
// ROWS, wherever they sit in the keyspace.
await using (TvContext context = _db.CreateContext())
{
await SeedFiller(context, 20000);
await context.Database.ExecuteSqlRawAsync("DELETE FROM SongMetadata");
context.SongMetadata.Add(Song("Survivor", ["Queen"]));
await context.SaveChangesAsync();
// one live row, sitting past the old keyspace allowance
(await context.SongMetadata.CountAsync()).ShouldBe(1);
(await context.SongMetadata.Select(m => m.Id).SingleAsync()).ShouldBeGreaterThan(20000);
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
(await handler.Handle(new GetSearchFieldValues("artist", "que", 50), CancellationToken.None))
.IfSome(
r => r.Values.ShouldBe(
new List<string> { "Queen" },
"a one-row table must be fully readable no matter where its Id sits"));
// and a leading gap must not hide a later match either
await using (TvContext context = _db.CreateContext())
{
context.SongMetadata.Add(Song("Second", ["Queens of the Stone Age"]));
await context.SaveChangesAsync();
}
(await handler.Handle(new GetSearchFieldValues("artist", "que", 50), CancellationToken.None))
.IfSome(r => r.Values.ShouldBe(new List<string> { "Queen", "Queens of the Stone Age" }));
}
[Test]
[TestCase("é", "\u00C9dith Piaf")]
[TestCase("\u00C9", "\u00C9dith Piaf")]
[TestCase("\u00E9dith", "\u00C9dith Piaf")]
[TestCase("bj", "Bj\u00F6rk")]
[TestCase("bj\u00F6", "Bj\u00F6rk")]
[TestCase("BJ\u00D6RK", "Bj\u00F6rk")]
[TestCase("beyonc\u00E9", "Beyonc\u00E9")]
[TestCase("sigur r", "Sigur R\u00F3s")]
[TestCase("\u00D6", "\u00D6zdemir")]
public async Task Matches_NonAscii_Values_In_Any_Casing(string query, string stored)
{
// Accented artists are the common case in a music library, so non-ASCII matching is pinned end to
// end, in both casings of the query.
//
// Historical note, because it is why this suite exists: revision 1b78dc9e narrowed rows in SQL
// with a LIKE built by JSON-encoding the query, which cannot work -- non-ASCII is stored escaped
// (\u00C9) and SQL LOWER() folds the escape TEXT, not the codepoint it denotes. THREE of these nine
// cases fail against that revision (the ones where query and stored casing differ, so \u00e9 and
// \u00C9 diverge); the other six pass it, because when the casings agree the escape texts line up.
// The SQL now has no residual predicate at all -- matching happens in memory, where a string is just
// a string -- so these cases pin current behaviour rather than guard that revision.
await using (TvContext context = _db.CreateContext())
{
context.SongMetadata.AddRange(
Song("Hit", [stored]),
// negative control: a row that must never come back for any of these queries
Song("Other", ["Nothing Relevant"]));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("artist", query, 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { stored }));
}
[Test]
public async Task Results_Do_Not_Depend_On_The_Request_Culture()
{
// UseRequestLocalization honours Accept-Language, so CurrentCulture is caller-controlled. Under tr-TR
// the old `q.ToLower()` turned "I" into "\u0131" and the default linguistic StartsWith(string) compounded
// it, so the same library answered differently per caller. The contract is ordinal: "I" matches
// "Istanbul" and does NOT match "\u0131pek", in every culture.
await using (TvContext context = _db.CreateContext())
{
context.SongMetadata.Add(Song("One", ["Istanbul Orkestrasi", "\u0131pek"]));
context.ArtistMetadata.Add(Artist("Idil Biret"));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
var expected = new List<string> { "Idil Biret", "Istanbul Orkestrasi" };
CultureInfo original = CultureInfo.CurrentCulture;
try
{
foreach (string culture in new[] { "en-US", "tr-TR", "az-AZ", "lt-LT" })
{
CultureInfo.CurrentCulture = new CultureInfo(culture);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("artist", "I", 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(expected, $"culture {culture} changed the result"));
}
}
finally
{
CultureInfo.CurrentCulture = original;
}
}
[Test]
public async Task Ordering_Is_Ordinal_And_Culture_Independent()
{
// The merge sorts ordinally rather than by culture, so the response order does not depend on the caller
// either. Ordinal puts all ASCII uppercase before ASCII lowercase, and non-ASCII last.
await using (TvContext context = _db.CreateContext())
{
context.SongMetadata.Add(Song("One", ["Zulu", "apple", "\u00C9clair", "Apple"]));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
var expected = new List<string> { "Apple", "Zulu", "apple", "\u00C9clair" };
CultureInfo original = CultureInfo.CurrentCulture;
try
{
foreach (string culture in new[] { "en-US", "sv-SE" })
{
CultureInfo.CurrentCulture = new CultureInfo(culture);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("artist", string.Empty, 50),
CancellationToken.None);
result.IfSome(r => r.Values.ShouldBe(expected, $"culture {culture} changed the order"));
}
}
finally
{
CultureInfo.CurrentCulture = original;
}
}
[Test]
public async Task Ordering_Is_Best_Effort_When_A_Source_Truncates()
{
// Documents the acknowledged imprecision rather than claiming exactness the code does not have. The EF
// source truncates by the DATABASE collation, which is NOT the ordinal ordering the merge then applies —
// so a value the database ranked outside its first `limit` never reaches the merge, even if the merge
// would have ranked it first.
//
// "Zulu" vs "apple" is the pair that actually diverges: ordinal puts every ASCII uppercase letter before
// every lowercase one, so ordinal ranks "Zulu" first, while a case-insensitive database ordering ranks
// "apple" first. (An earlier version used "Zulu"/"Éclair", where BOTH orderings pick "Zulu" — it could
// not have told the two apart, and the divergence it claimed to show did not exist.)
await using (TvContext context = _db.CreateContext())
{
context.ArtistMetadata.Add(Artist("Zulu"));
context.ArtistMetadata.Add(Artist("apple"));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
// with room for both, the ordinal merge ranks "Zulu" first
(await handler.Handle(new GetSearchFieldValues("artist", string.Empty, 50), CancellationToken.None))
.IfSome(r => r.Values.ShouldBe(new List<string> { "Zulu", "apple" }));
// with limit=1 the database picks the survivor by ITS ordering, and the merge only ever sees that one
(await handler.Handle(new GetSearchFieldValues("artist", string.Empty, 1), CancellationToken.None))
.IfSome(r => r.Values.ShouldBe(new List<string> { "apple" }));
}
[Test]
public async Task A_Match_Behind_Many_NonMatching_Rows_Is_Still_Found()
{
// Fails a883e5f0, which capped rows at a fixed 1000 AFTER a deliberately over-matching SQL pre-filter:
// the 1001st row -- the only exact match -- was discarded before the in-memory filter ever saw it and
// the endpoint returned []. The pre-filter is gone, and the property it broke now holds for any match
// within the read ceiling: preceding non-matching rows do not hide it. Past the ceiling it is still
// lost by design -- see List_Valued_Walk_Reads_At_Most_20000_Rows, which pins that boundary.
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < 1000; i++)
{
context.SongMetadata.Add(Song($"Filler {i}", ["zzz-filler"], ["zzz-filler-album"]));
}
context.SongMetadata.Add(Song("Needle", ["\u00E9clair"], ["\u00E9clair"]));
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> albumArtist = await handler.Handle(
new GetSearchFieldValues("album_artist", "\u00E9", 50),
CancellationToken.None);
albumArtist.IsSome.ShouldBeTrue();
albumArtist.IfSome(r => r.Values.ShouldBe(new List<string> { "\u00E9clair" }));
// same starvation shape on the merged `artist` field
Option<SearchFieldValuesResponseModel> artist = await handler.Handle(
new GetSearchFieldValues("artist", "\u00E9", 50),
CancellationToken.None);
artist.IsSome.ShouldBeTrue();
artist.IfSome(r => r.Values.ShouldBe(new List<string> { "\u00E9clair" }));
// ... and for a prefix beginning with a character that JSON escapes on disk. That used to collapse the
// SQL pattern to the bare anchor; there is no prefix predicate at all now, so it is simply an ordinary
// prefix -- kept because it is the input shape that broke the old scheme.
await using (TvContext context = _db.CreateContext())
{
context.SongMetadata.Add(Song("Ampersand", ["&Me"]));
await context.SaveChangesAsync();
}
Option<SearchFieldValuesResponseModel> escapedPrefix = await handler.Handle(
new GetSearchFieldValues("artist", "&M", 50),
CancellationToken.None);
escapedPrefix.IsSome.ShouldBeTrue();
escapedPrefix.IfSome(r => r.Values.ShouldBe(new List<string> { "&Me" }));
}
private static ArtistMetadata Artist(string title) => new()
{
MetadataKind = MetadataKind.External,
DateAdded = DateTime.UtcNow,
DateUpdated = DateTime.UtcNow,
Title = title
};
private static MusicVideoMetadata MusicVideo(string title, params string[] artists) => new()
{
MetadataKind = MetadataKind.External,
DateAdded = DateTime.UtcNow,
DateUpdated = DateTime.UtcNow,
Title = title,
Artists = artists.Map(a => new MusicVideoArtist { Name = a }).ToList()
};
private static SongMetadata Song(string title, IList<string> artists, IList<string> albumArtists = null) => new()
{
MetadataKind = MetadataKind.External,
DateAdded = DateTime.UtcNow,
DateUpdated = DateTime.UtcNow,
Title = title,
Artists = artists,
AlbumArtists = albumArtists
};
[Test]
public async Task Dedupes_Repeated_Values()
{
@@ -196,4 +653,226 @@ public class GetSearchFieldValuesHandlerTests
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Action" }));
}
/// <summary>
/// ersatztv#668. The EF-sourced fields prefix-match through SQL <c>LOWER()</c>, which on SQLite folds
/// ASCII only: <c>lower('Édith')</c> returns <c>'Édith'</c> unchanged, so a stored value whose
/// prefix carries an uppercase non-ASCII character is unreachable from any query long enough to reach it.
/// The stored-LOWERCASE case already worked (the handler lowercases the query before it reaches SQL, so
/// both casings of the query fold to the same pattern) and is pinned alongside it, because the fix must
/// SUPPLEMENT that path rather than replace it.
/// </summary>
[TestCase("genre", "é")]
[TestCase("genre", "É")]
public async Task Ef_Sourced_Stored_Uppercase_Accent_Is_Reachable(string field, string query)
{
await using (TvContext context = _db.CreateContext())
{
context.Set<Genre>().AddRange(
new Genre { Name = "Édith" },
new Genre { Name = "Zulu" });
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues(field, query, 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Édith" }));
}
/// <inheritdoc cref="Ef_Sourced_Stored_Uppercase_Accent_Is_Reachable" />
[TestCase("genre", "é")]
[TestCase("genre", "É")]
public async Task Ef_Sourced_Stored_Lowercase_Accent_Stays_Reachable(string field, string query)
{
await using (TvContext context = _db.CreateContext())
{
context.Set<Genre>().AddRange(
new Genre { Name = "édith" },
new Genre { Name = "Zulu" });
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues(field, query, 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { "édith" }));
}
/// <summary>
/// ersatztv#668. The Unicode fold added for the non-ASCII branch may OVER-match — the in-memory
/// <see cref="StringComparison.OrdinalIgnoreCase" /> filter runs afterwards and drops the extras —
/// but it must never UNDER-match. Each case pins the endpoint's answer against what that filter
/// alone would say, so a fold that starts dropping rows fails here. It does NOT catch removal of the
/// in-memory filter — every case here is either a positive that SQL alone returns, or an ASCII-query
/// negative that SQL alone rejects. That direction is
/// <see cref="Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter" />'s job.
/// <para>
/// The negative cases here have ASCII queries, so they exercise the FAST PATH (the fold is
/// skipped entirely) and pin that it is exact: <c>"ſweet".StartsWith("S", OrdinalIgnoreCase)</c>
/// is false even though <c>char.ToUpperInvariant('ſ')</c> IS <c>'S'</c>. The over-match the fold
/// itself produces is a different path and is covered by
/// <see cref="Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter" />.
/// </para>
/// </summary>
[TestCase("Édith", "é", true, TestName = "Fold_UppercaseAccent_LowercaseQuery")]
[TestCase("Édith", "É", true, TestName = "Fold_UppercaseAccent_UppercaseQuery")]
[TestCase("Özdemir", "ö", true, TestName = "Fold_Umlaut")]
[TestCase("Sigur Rós", "sigur", true, TestName = "Fold_AsciiPrefix_NonAsciiLater")]
[TestCase("Straße", "stra", true, TestName = "Fold_Eszett_AsciiQuery")]
// explicit escapes: these three are visually indistinguishable from their ASCII lookalikes in a diff,
// and an ASCII 'K' here would silently turn the KELVIN SIGN case into a trivially-true one
[TestCase("\u017Fweet", "S", false, TestName = "Fold_LongS_IsNotOrdinalEqualToS")]
[TestCase("\u212Aelvin", "k", false, TestName = "Fold_KelvinSign_IsNotOrdinalEqualToK")]
[TestCase("\u0130stanbul", "i", false, TestName = "Fold_DottedCapitalI_IsNotOrdinalEqualToI")]
public async Task Unicode_Fold_Agrees_With_The_Ordinal_Filter(string stored, string query, bool expected)
{
await using (TvContext context = _db.CreateContext())
{
context.Set<Genre>().Add(new Genre { Name = stored });
await context.SaveChangesAsync();
}
// the oracle: what the endpoint's own final filter says, computed independently of the database
stored.StartsWith(query, StringComparison.OrdinalIgnoreCase).ShouldBe(expected);
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("genre", query, 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(expected ? new List<string> { stored } : []));
}
/// <summary>
/// ersatztv#668. Drives a row THROUGH the fold that the ordinal filter must then discard — the
/// harmless over-match direction the whole design rests on, which the ASCII-query negative cases
/// above cannot reach. q="ſ" is non-ASCII so the fold runs; <c>ToUpperInvariant('ſ')</c> is 'S', so
/// the SQL pattern is <c>S%</c> and SQLite genuinely returns "Sword" — and the response must still
/// be empty, because <c>"Sword".StartsWith("ſ", OrdinalIgnoreCase)</c> is false.
/// </summary>
[Test]
public async Task Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter()
{
await using (TvContext context = _db.CreateContext())
{
context.Set<Genre>().Add(new Genre { Name = "Sword" });
await context.SaveChangesAsync();
}
// Premises, asserted because the expectation is an EMPTY list and would otherwise pass for the
// wrong reason -- e.g. if the branch stopped running, or a hand-rolled fold stopped mapping ſ to S,
// SQL would return nothing and this test would still be green.
GetSearchFieldValuesHandler.ContainsNonAscii("\u017F").ShouldBeTrue();
char.ToUpperInvariant('\u017F').ShouldBe('S');
"Sword".StartsWith("\u017F", StringComparison.OrdinalIgnoreCase).ShouldBeFalse();
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("genre", "\u017F", 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBeEmpty());
}
/// <summary>
/// ersatztv#668. The escaping's load-bearing role is NOT filtering — the in-memory ordinal filter
/// already drops an over-match, which is why a plain count assertion stays green even with the
/// escaping removed. It is preventing LIMIT CROWDING: an unescaped <c>_</c> also matches the space,
/// binary ORDER BY ranks "100 Édith" first, LIMIT 1 returns only that, the filter discards it, and
/// the genuine "100_Édith" is never returned at all. This case fails if the escaping is removed.
/// </summary>
[Test]
public async Task Unicode_Fold_Escaping_Prevents_Limit_Crowding()
{
await using (TvContext context = _db.CreateContext())
{
context.Set<Genre>().AddRange(
new Genre { Name = "100 \u00C9dith" },
new Genre { Name = "100_\u00C9dith" });
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("genre", "100_\u00C9", 1),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { "100_\u00C9dith" }));
}
/// <summary>
/// ersatztv#668. The non-ASCII branch is raw SQL, so it gets none of the LIKE-wildcard escaping EF
/// does for <c>StartsWith</c>. An unescaped <c>%</c> or <c>_</c> in the query would match anything.
/// </summary>
[TestCase("100%É", 1, TestName = "Escapes_Percent")]
[TestCase("100_É", 0, TestName = "Escapes_Underscore")]
public async Task Unicode_Fold_Escapes_Like_Wildcards(string query, int expectedCount)
{
await using (TvContext context = _db.CreateContext())
{
context.Set<Genre>().AddRange(
new Genre { Name = "100%Édith" },
new Genre { Name = "100XÉdith" });
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("genre", query, 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.Count.ShouldBe(expectedCount));
}
/// <summary>
/// ersatztv#668. The non-ASCII branch duplicates each field's discriminator predicate in raw SQL, so
/// it must reproduce EF's NULL semantics: EF compiles <c>ExternalTypeId != NfoCountryTypeId</c> with
/// null semantics, which INCLUDES a NULL-typed row. Plain SQL <c>&lt;&gt;</c> would silently drop it.
/// </summary>
[Test]
public async Task Unicode_Fold_Tag_Discriminator_Matches_Ef_Null_Semantics()
{
await using (TvContext context = _db.CreateContext())
{
context.Set<Tag>().AddRange(
new Tag { Name = "Édith", ExternalTypeId = null },
new Tag { Name = "Éclair", ExternalTypeId = Tag.PlexNetworkTypeId },
new Tag { Name = "Ézra", ExternalTypeId = Tag.NfoCountryTypeId });
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(_db.Factory);
Option<SearchFieldValuesResponseModel> tags = await handler.Handle(
new GetSearchFieldValues("tag", "é", 50),
CancellationToken.None);
// the NULL-typed row is a tag; the network- and country-typed rows are excluded
tags.IsSome.ShouldBeTrue();
tags.IfSome(r => r.Values.ShouldBe(new List<string> { "Édith" }));
Option<SearchFieldValuesResponseModel> networks = await handler.Handle(
new GetSearchFieldValues("network", "é", 50),
CancellationToken.None);
networks.IsSome.ShouldBeTrue();
networks.IfSome(r => r.Values.ShouldBe(new List<string> { "Éclair" }));
}
}
@@ -0,0 +1,174 @@
using ErsatzTV.Application.Search.Queries;
using ErsatzTV.Infrastructure;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Sqlite.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Application.Search;
/// <summary>
/// Provider-shape guards for the <c>artist</c> / <c>album_artist</c> facet-value sources (#578).
/// <para>
/// <see cref="GetSearchFieldValuesHandlerTests" /> runs against in-memory SQLite, so it structurally
/// cannot see a MySQL translation or collation difference. These tests build the same LINQ against the
/// Pomelo MySQL provider and assert the generated SQL — <c>ToQueryString</c> compiles the query without
/// touching a server, so no MySQL instance is needed.
/// </para>
/// </summary>
[TestFixture]
[NonParallelizable]
public class SearchFieldValuesQueryShapeTests
{
private bool _wasSqlite;
[SetUp]
public void SetUp() => _wasSqlite = TvContext.IsSqlite;
[TearDown]
public void TearDown() => TvContext.IsSqlite = _wasSqlite;
[Test]
public void Artist_Entity_Union_Translates_On_Both_Providers_With_Lower_And_A_Row_Limit()
{
foreach ((string provider, Func<TvContext> create) in Providers())
{
using TvContext context = create();
// calls the handler's own source builder (internal, via InternalsVisibleTo) rather than rebuilding
// the LINQ here — a copy would keep passing after the handler's query changed underneath it
string sql = GetSearchFieldValuesHandler.GetSource(context, "artist")
.Where(v => v != null && v.ToLower().StartsWith("a"))
.Distinct()
.OrderBy(v => v)
.Take(50)
.ToQueryString();
// case-insensitivity comes from LOWER() on the column, not from the provider's LIKE collation
sql.ShouldContain("LOWER(", Case.Insensitive, $"{provider}: {sql}");
sql.ShouldContain("LIKE", Case.Insensitive, $"{provider}: {sql}");
sql.ShouldContain("MusicVideoArtist", Case.Insensitive, $"{provider}: {sql}");
// the whole thing is one bounded server-side query, never a client-side scan
sql.ShouldContain("LIMIT", Case.Insensitive, $"{provider}: {sql}");
}
}
[Test]
public void Regression_Pin_Song_List_Columns_Cannot_Be_Projected_Server_Side_On_Either_Provider()
{
// REGRESSION PIN, not coverage of #578: this asserts pre-existing EF/provider behaviour and passes
// against the code before this change.
//
// Documents WHY the handler drops to raw SQL for SongMetadata.Artists / .AlbumArtists rather than
// SelectMany-ing them: EF maps them as JSON primitive collections and neither provider can translate
// the projection (SQLite needs APPLY; Pomelo has no primitive-collection support). If a provider
// upgrade ever makes this translate, this test fails and the raw-SQL path can be retired.
foreach ((string provider, Func<TvContext> create) in Providers())
{
using TvContext context = create();
Should.Throw<InvalidOperationException>(
() => context.SongMetadata.SelectMany(m => m.Artists).Distinct().Take(50).ToQueryString(),
$"{provider} unexpectedly translated a primitive-collection projection");
Should.Throw<InvalidOperationException>(
() => context.SongMetadata.SelectMany(m => m.AlbumArtists).Distinct().Take(50).ToQueryString(),
$"{provider} unexpectedly translated a primitive-collection projection");
}
}
[Test]
public void List_Valued_Page_Query_Has_No_Predicate_Beyond_The_Keyset_Cursor()
{
// This is the whole basis of the row bound, so it is asserted rather than assumed. LIMIT truncates what
// survives a RESIDUAL predicate — one that discards rows the engine already produced — so with such a
// predicate present it bounds the output rather than the row count, and the engine may produce and
// discard arbitrarily many rows first. That is how four successive revisions scanned past their own
// bound. The cursor `Id > @AfterId` is NOT such a predicate: it is a seek on the ordering key, which
// positions the scan without discarding anything, so LIMIT n yields n logical rows.
//
// What this test can and cannot do: it pins the SQL STRING. It cannot pin an execution plan, MVCC
// visibility work or payload I/O -- physical work is NOT bounded (see the record: MySQL traverses
// deleted-but-unpurged index records, and TEXT payloads spill to overflow pages).
string sql = GetSearchFieldValuesHandler.ListValuedSql("Artists");
sql.ShouldBe(
"SELECT Id, Artists AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch");
// named explicitly so a future "optimization" that reintroduces server-side selectivity fails here
sql.ShouldNotContain("LIKE");
sql.ShouldNotContain("LOWER");
sql.ShouldNotContain("IS NOT NULL");
}
/// <summary>
/// ersatztv#668. The SQL function name is duplicated — the handler lives in Application, which must
/// not reference a provider assembly, so it cannot use the constant the registration side defines. A
/// rename on one side alone would compile cleanly and fail only at runtime, only on SQLite, only for
/// non-ASCII queries; this pins the two together instead.
/// </summary>
[Test]
public void Unicode_Fold_Function_Name_Matches_The_Registration() =>
GetSearchFieldValuesHandler.UpperFunction.ShouldBe(SqliteUnicodeFunctions.UpperInvariantFunction);
/// <summary>
/// ersatztv#668. Unlike the list-valued walk, this query KEEPS its selectivity in SQL — it is a
/// bounded <c>LIMIT</c>ed prefix query exactly like the EF one it supplements, so a <c>LIKE</c> here
/// is correct rather than the trap the walk's shape test guards against. What must hold is that the
/// fold is the registered Unicode-correct one and NOT the provider's ASCII-only builtin, and that the
/// wildcard escape is declared.
/// </summary>
[Test]
public void Unicode_Fold_Query_Uses_The_Registered_Fold_And_Declares_Its_Escape()
{
string sql = GetSearchFieldValuesHandler.UnicodeFoldSql("Genre", "Name", null);
sql.ShouldBe(
"SELECT DISTINCT Name AS Value FROM Genre "
+ "WHERE etv_upper(Name) LIKE @Pattern ESCAPE '\\' ORDER BY Name LIMIT @Limit");
// The point of the whole change: SQLite's BUILTIN lower()/upper() fold ASCII only, so quietly falling
// back to one reinstates #668. Checked by removing the qualified call first — Shouldly's string
// assertions are case-INSENSITIVE by default, so a bare ShouldNotContain("UPPER(") matches inside
// "etv_upper(" and fails against correct SQL.
sql.ShouldNotContain("LOWER(");
sql.Replace($"{GetSearchFieldValuesHandler.UpperFunction}(", "", StringComparison.Ordinal)
.ShouldNotContain("UPPER(");
// a discriminator predicate is parenthesised and ANDed, so an OR inside it cannot swallow the match
GetSearchFieldValuesHandler.UnicodeFoldSql("Tag", "Name", "ExternalTypeId IS NULL OR X")
.ShouldContain("WHERE (ExternalTypeId IS NULL OR X) AND etv_upper(Name) LIKE @Pattern");
}
private static IEnumerable<(string Provider, Func<TvContext> Create)> Providers() =>
[
("sqlite", Sqlite),
("mysql", MySql)
];
private static TvContext Sqlite()
{
TvContext.IsSqlite = true;
var builder = new DbContextOptionsBuilder<TvContext>();
builder.UseSqlite("Data Source=:memory:");
return Create(builder.Options);
}
private static TvContext MySql()
{
TvContext.IsSqlite = false;
var builder = new DbContextOptionsBuilder<TvContext>();
builder.UseMySql(
"Server=localhost;Database=ersatztv_query_shape;User=root;Password=ersatztv;",
new MySqlServerVersion(new Version(8, 0, 36)));
return Create(builder.Options);
}
private static TvContext Create(DbContextOptions<TvContext> options) =>
new(
options,
NullLoggerFactory.Instance,
new SlowQueryInterceptor(NullLogger<SlowQueryInterceptor>.Instance));
}
@@ -0,0 +1,206 @@
using ErsatzTV.Application.Search.Queries;
using ErsatzTV.Core.Api.Search;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.MySql.Data;
using ErsatzTV.Infrastructure.Sqlite.Data;
using LanguageExt;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using MySqlConnector;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
/// <summary>
/// ersatztv#668, EXECUTED on both providers. The bug was a collation/fold difference, so it lives exactly
/// where a single-provider test cannot see it: SQLite's <c>LOWER()</c> folds ASCII only and UNDER-matched
/// a stored <c>Édith</c>, while MySQL's is Unicode-aware and reaches it unaided. (Its column collation
/// is accent-INsensitive, but the executed comparison is not — see the method docstring below.)
/// <para>
/// <see cref="ErsatzTV.Tests.Application.Search.GetSearchFieldValuesHandlerTests" /> covers the
/// SQLite semantics in depth against in-memory SQLite, and
/// <c>SearchFieldValuesQueryShapeTests</c> pins the generated SQL for both providers without a
/// server. Neither can show that a REAL MySQL server returns the accented value — the fix's central
/// claim is "on both providers", and on MySQL that rests on the server's Unicode-aware
/// <c>LOWER()</c> rather than on any code this repo owns — explicitly NOT on its collation, which
/// the executed comparison bypasses. That is precisely the kind of assumption worth executing.
/// </para>
/// <para>
/// MySQL needs a live server via <c>ETV_TEST_MYSQL_CONNECTION</c>. Without it the MySQL fixture
/// <b>ignores</b> — a visible skip, never a silent pass. Setting <c>ETV_REQUIRE_MYSQL_TESTS=1</c>
/// turns that skip into a hard failure, so an ARMED lane cannot degrade into "connected to nothing
/// and passed".
/// </para>
/// <para>
/// <b>CI does not currently arm it</b>, so in CI this half SKIPS. Running MySQL fixtures against the
/// live service was implemented and then removed as non-deterministic — see the note in
/// <c>.gitea/workflows/docker-build.yml</c>; re-arming is tracked by ersatztv#627. Do not read the
/// REQUIRE variable above as a guarantee that something enforces this today: nothing does. This
/// mirrors <see cref="LibraryFolderDedupeMigrationTests" /> deliberately; the two fixtures share the
/// contract, not code, because their setup needs differ.
/// </para>
/// </summary>
[TestFixture(TestProvider.Sqlite)]
[TestFixture(TestProvider.MySql)]
[NonParallelizable]
public class SearchFieldValuesProviderTests(TestProvider provider)
{
private const string MySqlConnectionVariable = "ETV_TEST_MYSQL_CONNECTION";
private const string MySqlRequiredVariable = "ETV_REQUIRE_MYSQL_TESTS";
private string _databasePath = null!;
private string? _mySqlConnectionString;
private DbContextOptions<TvContext> _options = null!;
[SetUp]
public async Task SetUp()
{
if (provider is TestProvider.Sqlite)
{
TvContext.IsSqlite = true;
TvContext.LastInsertedRowId = "last_insert_rowid()";
TvContext.CaseInsensitiveCollation = "NOCASE";
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
TvContext.RegisterUnicodeCaseFunctions = SqliteUnicodeFunctions.Register;
_databasePath = Path.Combine(Path.GetTempPath(), $"etv668-{Guid.NewGuid():N}.sqlite3");
_options = new DbContextOptionsBuilder<TvContext>()
.UseSqlite($"Data Source={_databasePath}")
.Options;
}
else
{
string? baseConnectionString = Environment.GetEnvironmentVariable(MySqlConnectionVariable);
if (string.IsNullOrWhiteSpace(baseConnectionString))
{
string message =
$"{MySqlConnectionVariable} is not set, so the MySql half of the #668 facet-value fixture "
+ "cannot run. This endpoint's correctness is collation-dependent and therefore "
+ "provider-specific, so the coverage is not optional in CI.";
if (IsTrue(Environment.GetEnvironmentVariable(MySqlRequiredVariable)))
{
Assert.Fail($"{message} {MySqlRequiredVariable} is set, so this is a failure, not a skip.");
}
Assert.Ignore($"{message} Set it to run this locally.");
}
// A database of our own with a name that has never been used, so isolation does not depend on a
// wipe succeeding. Dropped and its pool cleared in TearDown.
_mySqlConnectionString =
new MySqlConnectionStringBuilder(baseConnectionString) { Database = $"etv668_{Guid.NewGuid():N}" }
.ConnectionString;
TvContext.IsSqlite = false;
TvContext.LastInsertedRowId = "last_insert_id()";
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
// Explicitly the no-op: MySQL's own LOWER() is Unicode-aware, so the handler must reach the
// accented value WITHOUT any custom fold. Wiring SQLite's here would mask that.
TvContext.RegisterUnicodeCaseFunctions = static _ => { };
_options = new DbContextOptionsBuilder<TvContext>()
.UseMySql(_mySqlConnectionString, ServerVersion.AutoDetect(_mySqlConnectionString))
.Options;
}
// Schema creation deliberately does NOT happen here: NUnit skips [TearDown] when [SetUp] throws, so
// a failure part-way through EnsureCreatedAsync would strand the created database (and its pooled
// connection) with nothing to drop it. The test body creates it instead, matching the sibling
// fixture, whose SetUp likewise cannot strand one.
}
[TearDown]
public async Task TearDown()
{
if (provider is TestProvider.Sqlite)
{
Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools();
foreach (string path in new[] { _databasePath, $"{_databasePath}-wal", $"{_databasePath}-shm" })
{
if (File.Exists(path))
{
File.Delete(path);
}
}
return;
}
if (_mySqlConnectionString is not null)
{
await using (TvContext context = Create(_options))
{
await context.Database.EnsureDeletedAsync();
}
// MySqlConnector keys pools by connection string; a fresh database name means a fresh pool, and
// leaving it uncleared leaks a server thread per test until max_connections is exhausted.
await using var probe = new MySqlConnection(_mySqlConnectionString);
await MySqlConnection.ClearPoolAsync(probe);
_mySqlConnectionString = null;
}
}
/// <summary>
/// The #668 headline, executed: a stored value whose prefix carries an UPPERCASE non-ASCII character
/// is reachable from both casings of the query, on whichever provider this fixture is running.
/// <para>
/// Negative controls: "Zulu" (trivially unrelated) and "Edith" (unaccented, the near miss).
/// <b>Be precise about what "Edith" does and does not prove.</b> It was added expecting MySQL to
/// OVER-match it — the column collation is <c>utf8mb4_0900_ai_ci</c>, so <c>é</c> equals <c>e</c>
/// — which would have made the in-memory ordinal filter load-bearing here. Measured against a
/// live 8.4 server, it does not: deleting that filter leaves this test green, because the driver
/// binds the LIKE pattern with a BINARY collation and the executed comparison is therefore
/// accent-SENSITIVE. (A literal pattern typed by hand DOES over-match — a different query from
/// the one the handler runs.) So the row pins the accent-sensitive result on both providers and
/// documents the near miss; it does NOT exercise an over-match correction, because with the
/// CURRENT driver there is nothing to correct. That is a driver-contingent fact, not a law: a
/// driver or protocol change that made the pattern ci-collated would restore the over-match, and
/// the ordinal filter — which stays regardless — would then be doing real work here.
/// </para>
/// </summary>
[TestCase("é", TestName = "Uppercase_Accent_Reachable_From_Lowercase_Query")]
[TestCase("É", TestName = "Uppercase_Accent_Reachable_From_Uppercase_Query")]
public async Task Stored_Uppercase_Accent_Is_Reachable(string query)
{
await using (TvContext context = Create(_options))
{
await context.Database.EnsureCreatedAsync();
context.Set<Genre>().AddRange(
new Genre { Name = "Édith" },
new Genre { Name = "Edith" },
new Genre { Name = "Zulu" });
await context.SaveChangesAsync();
}
var handler = new GetSearchFieldValuesHandler(new TestDbContextFactory(_options));
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
new GetSearchFieldValues("genre", query, 50),
CancellationToken.None);
result.IsSome.ShouldBeTrue();
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Édith" }));
}
private static bool IsTrue(string? value) =>
value is "1" || string.Equals(value, "true", StringComparison.OrdinalIgnoreCase);
private static TvContext Create(DbContextOptions<TvContext> options) =>
new(
options,
NullLoggerFactory.Instance,
new SlowQueryInterceptor(NullLogger<SlowQueryInterceptor>.Instance));
private sealed class TestDbContextFactory(DbContextOptions<TvContext> options) : IDbContextFactory<TvContext>
{
public TvContext CreateDbContext() => Create(options);
}
}
@@ -31,6 +31,7 @@ public sealed class InMemoryTvContext : IAsyncDisposable
{
TvContext.IsSqlite = true;
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
TvContext.RegisterUnicodeCaseFunctions = SqliteUnicodeFunctions.Register;
var connection = new SqliteConnection("Data Source=:memory:;Foreign Keys=False");
await connection.OpenAsync();
+212
View File
@@ -0,0 +1,212 @@
using ErsatzTV.Controllers.Api.Requests;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using NUnit.Framework;
namespace ErsatzTV.Tests.Support;
/// <summary>
/// One selection-type matrix, shared by every fixture that exercises a tagged-union selection
/// (rerun collections and playlist items). Both consumers of
/// <c>MediaCollections.Mapper.ProjectMediaItemToViewModel</c> are proved against the SAME data, so
/// widening the shared switch cannot be discharged for the second consumer by inspection alone —
/// which is the method that produced #671 in the first place.
/// </summary>
internal static class SelectionSeedData
{
public const int SelectedId = 42;
/// <summary>
/// Derived from production rather than hand-listed, so a newly-supported type joins the matrix
/// automatically and trips the <c>default:</c> arms below until someone teaches them about it.
/// Note this is the RERUN-COLLECTION predicate, used for playlist items as a deliberate
/// SUPERSET: <c>ReplacePlaylistItemsHandler.CollectionTypeMustBeValid</c> has no
/// <c>RemoteStream</c> case, so a RemoteStream playlist item cannot be created through the write
/// API today and the playlist fixture seeds that row directly. Covering it is forward-looking,
/// not a claim that the two sets are equivalent — split this if they ever legitimately diverge.
/// </summary>
public static IEnumerable<CollectionType> SupportedSelectionTypes =>
Enum.GetValues<CollectionType>().Where(RerunCollectionRequestMapping.IsSupportedSelectionType);
/// <summary>
/// The exact projected name per type. Pinning the whole string — rather than merely asserting
/// "not a placeholder" — is what makes a missing NESTED include leg visible: dropping
/// Episode → Season → Show still yields the placeholder-free "s??e04 - Selected episode", and
/// dropping MusicVideo → Artist still yields "Selected music video". Both would sail past a
/// looser assertion while having lost real data.
/// </summary>
public static string ExpectedName(CollectionType collectionType) =>
collectionType switch
{
CollectionType.Collection => "Selected collection",
CollectionType.MultiCollection => "Selected multi collection",
CollectionType.SmartCollection => "Selected smart collection",
CollectionType.TelevisionShow => "Selected show (2020)",
CollectionType.TelevisionSeason => "Parent show (2020) - Season 3",
CollectionType.Artist => "Selected artist",
CollectionType.Movie => "Selected movie (2019)",
CollectionType.Episode => "Episode's show - s02e04 - Selected episode",
CollectionType.MusicVideo => "Video's artist - Selected music video",
CollectionType.OtherVideo => "Selected other video",
CollectionType.Song => "Song artist - Selected song",
CollectionType.Image => "Selected image",
CollectionType.RemoteStream => "Selected remote stream",
_ => throw new AssertionException($"No expected name pinned for {collectionType}")
};
public static async Task SeedSelection(TvContext context, CollectionType collectionType)
{
switch (collectionType)
{
case CollectionType.Collection:
context.Collections.Add(new Collection
{
Id = SelectedId,
Name = "Selected collection",
MediaItems = []
});
break;
case CollectionType.MultiCollection:
context.MultiCollections.Add(new MultiCollection
{
Id = SelectedId,
Name = "Selected multi collection"
});
break;
case CollectionType.SmartCollection:
context.SmartCollections.Add(new SmartCollection
{
Id = SelectedId,
Name = "Selected smart collection",
Query = "tag:family"
});
break;
case CollectionType.TelevisionShow:
context.Shows.Add(new Show
{
Id = SelectedId,
ShowMetadata = [new ShowMetadata { Title = "Selected show", Year = 2020 }]
});
break;
case CollectionType.TelevisionSeason:
context.Seasons.Add(new Season
{
Id = SelectedId,
SeasonNumber = 3,
Show = new Show
{
Id = 900,
ShowMetadata = [new ShowMetadata { Title = "Parent show", Year = 2020 }]
}
});
break;
case CollectionType.Artist:
context.Artists.Add(new Artist
{
Id = SelectedId,
ArtistMetadata = [new ArtistMetadata { Title = "Selected artist" }]
});
break;
case CollectionType.Movie:
context.Movies.Add(new Movie
{
Id = SelectedId,
MovieMetadata = [new MovieMetadata { Title = "Selected movie", Year = 2019 }]
});
break;
case CollectionType.Episode:
context.Episodes.Add(new Episode
{
Id = SelectedId,
EpisodeMetadata = [new EpisodeMetadata { Title = "Selected episode", EpisodeNumber = 4 }],
Season = new Season
{
Id = 901,
SeasonNumber = 2,
Show = new Show
{
Id = 902,
ShowMetadata = [new ShowMetadata { Title = "Episode's show", Year = 2018 }]
}
}
});
break;
case CollectionType.MusicVideo:
context.MusicVideos.Add(new MusicVideo
{
Id = SelectedId,
MusicVideoMetadata = [new MusicVideoMetadata { Title = "Selected music video" }],
Artist = new Artist
{
Id = 903,
ArtistMetadata = [new ArtistMetadata { Title = "Video's artist" }]
}
});
break;
case CollectionType.OtherVideo:
context.OtherVideos.Add(new OtherVideo
{
Id = SelectedId,
OtherVideoMetadata = [new OtherVideoMetadata { Title = "Selected other video" }]
});
break;
case CollectionType.Song:
context.Songs.Add(new Song
{
Id = SelectedId,
SongMetadata =
[new SongMetadata { Title = "Selected song", Artists = ["Song artist"] }]
});
break;
case CollectionType.Image:
context.Images.Add(new Image
{
Id = SelectedId,
ImageMetadata = [new ImageMetadata { Title = "Selected image" }]
});
break;
case CollectionType.RemoteStream:
context.RemoteStreams.Add(new RemoteStream
{
Id = SelectedId,
Url = "http://example.invalid/stream",
RemoteStreamMetadata = [new RemoteStreamMetadata { Title = "Selected remote stream" }]
});
break;
default:
throw new AssertionException(
$"{collectionType} is a supported selection type but this suite does not know how " +
"to seed it — teach SeedSelection about it rather than narrowing the matrix.");
}
await context.SaveChangesAsync();
}
/// <summary>
/// Assigns the one foreign key the tagged union uses for this type. Shared so the rerun and
/// playlist fixtures cannot disagree about which slot a type occupies.
/// </summary>
public static void ApplySelection(
CollectionType collectionType,
Action<int> setCollectionId,
Action<int> setMultiCollectionId,
Action<int> setSmartCollectionId,
Action<int> setMediaItemId)
{
switch (collectionType)
{
case CollectionType.Collection:
setCollectionId(SelectedId);
break;
case CollectionType.MultiCollection:
setMultiCollectionId(SelectedId);
break;
case CollectionType.SmartCollection:
setSmartCollectionId(SelectedId);
break;
default:
setMediaItemId(SelectedId);
break;
}
}
}
+5 -1
View File
@@ -205,7 +205,11 @@ public class SearchController(IMediator mediator) : ControllerBase
"Returns distinct whole values from the database for the given text field, filtered by an " +
"optional case-insensitive prefix. Powers the visual rule builder's facet-value typeahead. " +
"404 when the field is unknown, is not a text field, or is a text field with no distinct-value " +
"source.")]
"source. The final filter, dedup and ordering applied to the response are ordinal and not " +
"culture-dependent; note that fields sourced by a plain database query are additionally " +
"pre-filtered by the database collation first, which on SQLite is ASCII-only. The list-valued " +
"music fields (artist, album_artist) are bounded best-effort: the server reads a bounded number " +
"of song rows per request, so a library larger than that bound may yield a subset of the matches.")]
[EndpointGroupName("general")]
[ProducesResponseType(typeof(SearchFieldValuesResponseModel), StatusCodes.Status200OK)]
[ProducesResponseType(typeof(ProblemDetails), StatusCodes.Status404NotFound)]
+5
View File
@@ -649,6 +649,7 @@ public class Startup
TvContext.LastInsertedRowId = "last_insert_rowid()";
TvContext.CaseInsensitiveCollation = "NOCASE";
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
TvContext.RegisterUnicodeCaseFunctions = SqliteUnicodeFunctions.Register;
SqlMapper.AddTypeHandler(new DateTimeOffsetHandler());
SqlMapper.AddTypeHandler(new GuidHandler());
@@ -660,6 +661,10 @@ public class Startup
TvContext.LastInsertedRowId = "last_insert_id()";
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
// MySQL's LOWER() is already Unicode-aware, so the facet-value handler never takes the
// custom-fold branch here; assigned explicitly so a provider switch cannot inherit SQLite's.
TvContext.RegisterUnicodeCaseFunctions = static _ => { };
}
Log.Logger.Information("Transcode folder is {Folder}", FileSystemLayout.TranscodeFolder);
+1 -1
View File
@@ -18063,7 +18063,7 @@
"Search"
],
"summary": "List distinct database values for a text search field",
"description": "Returns distinct whole values from the database for the given text field, filtered by an optional case-insensitive prefix. Powers the visual rule builder's facet-value typeahead. 404 when the field is unknown, is not a text field, or is a text field with no distinct-value source.",
"description": "Returns distinct whole values from the database for the given text field, filtered by an optional case-insensitive prefix. Powers the visual rule builder's facet-value typeahead. 404 when the field is unknown, is not a text field, or is a text field with no distinct-value source. The final filter, dedup and ordering applied to the response are ordinal and not culture-dependent; note that fields sourced by a plain database query are additionally pre-filtered by the database collation first, which on SQLite is ASCII-only. The list-valued music fields (artist, album_artist) are bounded best-effort: the server reads a bounded number of song rows per request, so a library larger than that bound may yield a subset of the matches.",
"operationId": "GetSearchFieldValues",
"parameters": [
{
+78 -1
View File
@@ -146,6 +146,41 @@ Exemplars:
`Brief`. `Remediation.Kind` is a mapped **string** ("ExternalDoc"/"AppRoute"), not a wire enum —
same pattern as `Status`. See `decisions.md` 2026-07-17 (#164).
### 2a. Flattening a tagged-union selection (read path)
Several DTOs flatten a "exactly one of these navigations is populated" tagged union to a single
`selectedId` + `selectedName` pair (`RerunCollectionResponseModel`, and the playlist-item shape).
Two rules, both learned from #671, where the list endpoint returned a null selection for **every**
row and the detail GET 500'd for two of its media types:
- **One include chain per projected aggregate, shared by every handler that projects it.** Put it in
a `<Aggregate>QueryExtensions` extension method and call it from the list handler *and* the by-id
handler. Exemplars: `RerunCollectionQueryExtensions.IncludeSelectionDetails()`,
`ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails()`. Two hand-maintained chains
drift, and the one that drifts is usually the paged list, whose rows are individually less
obviously wrong. Applying it before `Skip`/`Take` is fine — EF applies the includes to the paged
subquery, so the cost is bounded by `PageSize`, not by the table.
- **The id and the name must not share a single point of failure.** When both are read off the same
eager-loaded navigation, the id is only ever as available as the name — so an un-included type
doesn't merely render an unlabelled badge, it drops the selected id, and an editor that
round-trips that id silently clears the user's stored selection. Accordingly a media-item
flattening switch never ends in `_ => null`: an unrecognized subtype keeps its id and takes a
conspicuous `[unsupported media type: X]` name. Throwing is the wrong lever — it would fail an
entire paged GET over one unreadable row. The shared switch is
`MediaCollections.Mapper.ProjectMediaItemToViewModel`.
Corollary for the mappers themselves: `MediaItems.Mapper`'s projections are reached from handlers
whose include chains differ, so every metadata navigation is read through `Optional(...).Flatten()`
and degrades to the `"???"` placeholder rather than throwing. A bare `x.Season.Show.ShowMetadata`
inside a projection is a latent 500 on some other caller's GET.
**And it is not only navigations.** `SongMetadata.Artists` is a nullable EF *primitive collection*
(a JSON array in one column), which `FallbackMetadataProvider` leaves unassigned for a song whose
tags failed to read — and `string.Join` throws `ArgumentNullException` on a null sequence, not a
`NullReferenceException`. Adding an include is therefore not automatically safe: it can promote a
latent throw on a previously-unloaded member into a live 500 that fails the whole page. When you
widen an include chain, audit what the newly-reachable projection dereferences.
## 3. Error mapping
Central helper: `ErsatzTV/Extensions/ApiResults.cs`. Use these extension methods instead of
@@ -444,7 +479,7 @@ standard credential (catalog-read tier — no `[RequiresAuthentication]`):
Query params: `q` (optional prefix filter, case-insensitive, default empty) and `limit` (optional,
clamped `1..50`, default 50). `{name}` is allow-listed to `SearchFieldCatalog` fields with
`type: "text"` AND a distinct-value source in the database — an unknown field, a non-text field (e.g.
an enum), or a text field without a source (`title`, `show_title`, `album_artist`) 404s rather than
an enum), or a text field without a source (`title`, `show_title`) 404s rather than
returning an empty list, since enum fields already ship their values inline on
`GET /api/v1/search/fields` and never need this endpoint. Returns `SearchFieldValuesResponseModel`
(`{ values: string[] }`), sourced from a per-field distinct-values DB query (`IDbContextFactory<TvContext>`),
@@ -452,6 +487,48 @@ not the Lucene term dictionary — analyzed text fields store lowercased word to
No server-side caching. Powers the visual rule builder's value-input combobox for text fields; see
`docs/decisions.md` 2026-07-23 (#434) and `spa-conventions.md` §12.
**Bounded best-effort for list-valued fields (#578)**: `artist` and `album_artist` are backed (wholly
or partly) by `SongMetadata.Artists`/`AlbumArtists`, which EF maps as **primitive collections** — one
JSON array per row in a single column, with no server-side projection on either provider. `album_artist`
therefore no longer 404s, and `artist` now also covers free-text music-video (`MusicVideoArtist`) and
song credits, not only entity artists. Those rows are read by a keyset page whose only condition is the
**cursor** — no residual predicate that could discard a row — and filtered in memory, bounded at 20,000
logical rows per request; so on a larger library the
response may be a bounded subset of the matches — bounded in LOGICAL ROWS, which is not the same as
bounded work or bytes. Say so in the `[EndpointDescription]` of any endpoint that adopts this shape.
Three rules generalize beyond this endpoint.
1. **`LIMIT` bounds the OUTPUT, not the row count, whenever a RESIDUAL predicate is present.** The
distinction is not "predicate vs none" — a keyset cursor is a predicate. It is that a *seekable
predicate on the ordering key* positions the scan and never discards a row, while a *residual*
predicate (`LIKE`, `LOWER`, `IS NOT NULL`) throws away rows the engine already produced, so `LIMIT`
truncates the survivors and says nothing about how many were produced — a query matching nothing
must examine every eligible row before it can return an empty page. To bound rows, drop the residual
predicate, page by row position over the primary key, and filter in memory. This endpoint got it
wrong four times: bounding the result, then candidates returned, then `Id` keyspace width (keyspace
is not rows — one live row at `Id` 20001 behind 20,000 deleted ones reads nothing), before arriving
at "cursor only".
**And scope the resulting claim to LOGICAL ROWS.** It is not bounded physical work: MySQL still
traverses deleted-but-unpurged index records, so deletion history keeps affecting cost, and an
unrestricted `TEXT` column spills to overflow pages so a row count implies no byte or page-read
count. A SQL-string assertion pins none of that — not a plan, not visibility work, not I/O.
2. **A SQL pre-filter under an in-memory exact filter may over-match but must never under-match — and
that licence is void the moment the candidate set is truncated.** Widening the predicate then starves
the budget with rows that cannot match. If you find yourself proving a superset property to keep a
pre-filter honest, consider deleting the pre-filter instead: here it removed a JSON-escaping bug
class, an exhaustive Unicode sweep and an `ESCAPE` portability workaround along with it.
3. **Prefix matching, dedup and ordering must be ordinal, not current-culture** (`OrdinalIgnoreCase`,
`StringComparer.Ordinal`): `UseRequestLocalization` honours `Accept-Language`, so `ToLower()` and the
default linguistic `StartsWith(string)` let a caller change the result by changing a header. **Scope
the claim to the stage that actually holds it** — a value set that a database `LOWER`/`DISTINCT`/
`ORDER BY`/`LIMIT` already filtered and truncated is not ordinal no matter what runs after it, and
saying otherwise in an `[EndpointDescription]` publishes a false contract (ersatztv#668).
Full rationale, the measured transfer cost, the four-attempts table and the rejected
normalized-side-table alternative (ersatztv#669): `api.search-field-values-sources` (supersedes
`api.search-field-values`).
**Param + DTO expansion (#293, cap `search/all-items`)**: no new endpoint — `GET /api/v1/search/all-items`
gained two **optional** query params (`pageSize` default 500, clamped 11000 via the §1 Logs `Math.Clamp`
precedent; `pageNum` 0-based, clamped `0..2_000_000` so `pageNum * pageSize` can't overflow `int` to a 500)
+157 -7
View File
@@ -655,11 +655,53 @@ which is itself a small demonstration of why the suite needed to run in CI at al
`small`-lane Python jobs it adds `actions/setup-python@v5` first. Checkout is at default depth: every `git` call in the suite runs
against a temp repo it creates itself, never this repository's history.
A **preflight step** asserts `jq` and `git` are on PATH before running the suite. Those two tests
exec the real shell scripts, which shell out to `jq` ~26 times; the tests shim `curl` on PATH but
not `jq`, so a runner image without it would surface as ~20 opaque assertion failures instead of one
diagnosis. It deliberately **checks** rather than installs — ersatztv#390 removed run-time
`apt-get` from CI; the fix for a genuine miss is to bake the tool into the runner image.
Two **preflight steps** run before the suite. The first asserts `git` is on PATH; the second runs
`scripts/jq-preflight.sh --expect 1.6`, which checks jq's **version**, not merely its presence (see
"The jq contract" below). Those two tests exec the real shell scripts, which shell out to `jq` ~26
times; the tests shim `curl` on PATH but not `jq`, so a runner image without it would surface as ~20
opaque assertion failures instead of one diagnosis. Both deliberately **check** rather than install —
ersatztv#390 removed run-time `apt-get` from CI; the fix for a genuine miss is to bake the tool into
the runner image.
### The jq contract (ersatztv#648)
> Full rationale: `docs/decisions/records/ci/jq-version-contract.md`.
Every shell gate in this repo — `decisions-guard`, `script-tests`'s own harness,
`pretooluse-merge-consent.sh`, `review-verdict.yml`, `scripts/pr-changed-files.sh` — is authored and
tested on a developer Mac shipping **jq 1.8.x**. The CI runner ships **jq 1.6**. Author to the
1.6-compatible subset; three concrete constructs diverge between the two and each one produced a real
bug when it hit CI for the first time:
- **`jq -e` over EMPTY input.** Exits 4 on jq >= 1.7, but **0** on jq 1.6. A guard that infers
"transport failure" from that exit status silently passes an empty/failed page on 1.6.
- **`` contains("\u0000") `` (or any NUL literal).** The NUL escape truncates to `""` on jq 1.6, so
the containment test is vacuously true for **every** string, not just ones containing a NUL. Use
`explode | index(0)` instead — it is version-stable.
- **Parse-error exit code.** `jq empty` exits 5 on jq >= 1.7 but **4** on jq 1.6 — the same code 1.6
uses for "no output produced". Reading that exit code as a specific failure mode conflates garbage
input with an empty-but-valid response.
`scripts/jq-preflight.sh` makes the running version **observable** in every gate job's log (it prints
the parsed version and asserts a floor of 1.6) so a future divergence can be diagnosed from the log
alone instead of guessing at the runner image.
**Pin vs floor is deliberately asymmetric.** `scripts/jq-preflight.sh --expect 1.6` additionally pins
the version and fails loudly if it drifts, but that mode is used **only** by `script-tests`
(`.gitea/workflows/pr-checks.yml`) — advisory, not a required check. `review-verdict.yml` runs the
no-args floor-only mode and never pins, because that workflow writes `review-verdict/h10`, the
branch-protection-**required** status check on `main`: a hard pin there would mean the day the
runner's jq version changes (a base-image bump, a host reimage — nothing this repo controls), every
PR on `main` stops merging until someone notices and re-pins. A required merge gate cannot fail
because an upstream package manager did its job. The narrower pin on `script-tests` exists precisely
because that job is the suite's only 1.6 coverage — if the runner's jq silently changed, that coverage
would evaporate with no signal, so failing loudly there forces a human decision instead.
Baking a pinned jq into `docker/ci/Dockerfile` was considered and rejected: `review-verdict.yml` is
`runs-on: small` with no toolchain-image pin, and per `ci.small-lane-git-only` the small lane is
git-only, so it gets the **host's** jq regardless of what the toolchain image contains — a pin in the
image provably cannot reach the gate that broke. This was checked against the running binary, not
assumed.
## PR gates workflow
@@ -736,7 +778,15 @@ hook's condition (c)) and the `review-verdict/h10` status on the same sha. `BLOC
a commit landed mid-flight it writes **no** status and exits non-zero rather than retargeting your
verdict at a commit you never read.
**Exemptions** are handled by `review-verdict.yml` on every `pull_request` event, which posts the
The status description also records the base branch — `Review-verdict: MERGEABLE @ abc1234 (base:
main)` — and the merge-consent hook denies when that no longer matches the PR's live `base.ref`
(ersatztv#632). Retargeting a PR changes the effective diff without moving the head sha, so the
per-sha binding alone cannot see it. This is **detection on the hook path only**: a commit status
carries no base of its own, so a merge driven through the Gitea UI or API is unaffected. The
comparator is the base *branch*, never its tip sha — a base that merely advances is ordinary churn,
and comparing tips would invalidate every open verdict on every unrelated merge to `main`.
**Exemptions** are handled by `review-verdict.yml` on every `pull_request_target` event, which posts the
status as `success` for **Renovate-authored** PRs (it uses `platformAutomerge: true`, so a required
verdict with no exemption would stall every dependency bump) and for **docs-only** PRs, and as
`pending` for everything else so the block has a visible reason. Both exemptions are **void when the
@@ -744,12 +794,112 @@ PR touches `.claude/`, `.gitea/`, `.husky/`, `scripts/` or `docker/ci/`** — a
gate must not be able to exempt itself from the gate. That includes Renovate's `docker/ci` base
bumps, which already need the manual publish-then-pin two-step anyway.
The Renovate exemption additionally requires **every** changed path to be a dependency manifest —
`Directory.Packages.props` or `.config/dotnet-tools.json`, and only those (ersatztv#698). The npm
manifests are deliberately excluded: `renovate.json` enables only `nuget`/`github-actions`/`dockerfile`,
so npm is unmanaged here, while `package.json` `scripts` are executed by CI (`npm ci`, `npm run build`)
— exempting it would put a code-execution path inside the allow-list for no benefit. An author match alone is not enough, because `pull_request.user.login` is the PR's
*immutable creator* while its head is not: pushing application code onto an open Renovate branch
leaves the PR still "authored by renovate" and, previously, still exempt. A Renovate PR touching
anything else — a `.csproj`, a source file — is not blocked, it just needs a real verdict. **If a
dependency PR is unexpectedly asking for a verdict, this is why**; the status description says so.
The two exemptions are evaluated as **independent predicates**, never as an `elif` chain: a Renovate
PR touching only `docs/` still gets the docs-only exemption on its own merits.
An existing `review-verdict/h10` on the head is **only** left alone when it is positively identifiable
as a human verdict — a non-null `.creator.login` **and** a `Review-verdict:` description, which is what
`post-review-verdict.sh` writes. Anything else, including any shape the workflow does not recognise, is
**re-derived** rather than inherited. (Measured: a status POSTed with a user credential carries a
creator; one POSTed by an Actions job carries `"creator": null`.) Without this, an exemption obtained
once was accepted unchanged on every later run. This is a *provenance* check, not an authentication
one — someone who can POST statuses directly can still impersonate a verdict, which is ersatztv#697.
Deciding either exemption requires the PR's **complete** changed-file list, which the workflow does
not compute itself: it calls `scripts/pr-changed-files.sh`, the single shared implementation also
used by the advisory hook `.claude/hooks/pretooluse-merge-consent.sh` (ersatztv#649). The workflow
reads that script's **exit status** — a non-zero exit means "could not tell" and withholds the
exemption; its stdout is meaningless on any failure path and is never consumed.
**Never write a classification guard as `producer | grep -q…` here.** Under `set -o pipefail`, `grep -q`
exits at its first match, the producer takes SIGPIPE (141), and a MATCH is reported as a failed
pipeline — inverting the guard for any PR whose path list exceeds the pipe buffer. That let a large PR
be classified docs-only, and let one editing `.gitea/` skip the protected-path check entirely. A
here-string is **also** wrong (bash spills a large one to temp storage, which fails the same way when
temp is full). **Count** instead — `grep -c` drains stdin over an ordinary pipe — evaluate the counts
once at top level rather than inline in an `if`, and fail closed on a non-numeric result. Full detail:
`ci.grep-q-pipefail-inversion`.
That script takes the expected base branch as a **required 5th argument** and refuses to enumerate when
the PR's live base does not match it, checked both before and after paging (ersatztv#698).
`/pulls/{n}/files` diffs against the PR's *live* base, so retargeting changes the answer without moving
the head sha — a PR opened into `main` and retargeted mid-run was granted a docs-only exemption while
its diff against `main` carried a C# file. The workflow passes the base from the `pull_request_target`
payload, which a retarget cannot rewrite, and `edited` is in `types:` so a retarget reclassifies.
`edited` gives **detection, not atomicity**: runs are not serialized, so a stale run can still post
`success` after the reclassifying run posts `pending`. The `main → scratch → main` ABA transition is
narrowed and observable, not closed — see the residual in `ci.exemption-provenance`.
Three properties of this workflow are security-relevant and are **structurally** asserted by tests in
`scripts/tests/test_pr_changed_files.py` — those tests pin the workflow's shape, which is not the same
as establishing that the gate cannot be forged (see the residual below, and ersatztv#697/#698):
- **The trigger is `pull_request_target`, scoped to `branches: [main]`** — never plain
`pull_request` (ersatztv#672). Gitea resolves a `pull_request` workflow *definition* from the PR's
own head, so under that trigger a PR editing `review-verdict.yml` ran its own rewritten copy and
could post `review-verdict/h10=success` for itself. The base-ref checkout below binds the scripts
this job runs; only the trigger binds the definition. The `branches` filter is half the fix, not a
refinement of it: base resolution means the *base branch* supplies the gate, so an unfiltered
trigger merely moves the rewrite to an attacker-pushed base — and a status forged there is
inherited by any later PR carrying the same head sha (ersatztv#663). `pull_request_target` is safe
here **only** because this job never checks out or executes head-supplied code. Verified on this
instance with four scratch PRs rather than inferred from GitHub; full rationale in
`docs/decisions/records/ci/gate-trigger-base-resolved.md`. **This closes the rewrite route through
this workflow, not the class:** `docker-build.yml` is also head-resolved and its `ETV_STATUS_AUTH`
credentials can write statuses, so it can still forge `review-verdict/h10` — it must stay on
`pull_request` because it builds the PR's code, so it needs a read-only status identity instead
(ersatztv#697) — and the inventory is every workflow, not that one, because Gitea injects a
write-capable `GITEA_TOKEN` into every job and branch protection binds the *context*, not its
issuer. The exemption path has separate defects of its own (ersatztv#698). One operational
consequence of the trigger change: a PR whose base is not `main` now gets **no**
`review-verdict/h10` at all. That is fail-closed. `edited` **is** now among the trigger's `types`
(ersatztv#698), so a PR retargeted onto `main` reclassifies instead of staying statusless until its
next push — but note that only gives *detection*: runs are not serialized, so a stale run can still
post `success` after the reclassifying run posts `pending` (ersatztv#706).
- **The checkout takes the PR's BASE ref**, `ref: ${{ github.event.pull_request.base.sha }}` with
`persist-credentials: false` — never the head. This job judges the PR, so the PR must not supply
the code that judges it; a head checkout would let a PR edit the enumeration to return an empty
list and exempt itself.
- **`scripts/jq-preflight.sh` runs in floor-only mode**, never `--expect`. This job writes a
branch-protection-**required** status, so an exact version pin would turn any jq upgrade on the
runner into a repo-wide merge deadlock.
A PR whose base predates ersatztv#658 has no such script on its base ref; that case posts `pending`
with the reason rather than dying with no status at all.
⚠️ **Changing `review-verdict.yml` itself: it is not exercised by its own PR.** Base resolution cuts
both ways — the PR editing this workflow runs the version already on `main`, so an edit goes live
**only on merge**, repo-wide, having never run. A broken edit merges green and then breaks the gate
for every subsequent PR, and the PR that would repair it is gated by the same broken workflow. Do not
trust the editing PR's own checks. Verify the way ersatztv#672 did:
1. Push a scratch **base** branch carrying the candidate workflow.
2. Open a throwaway PR from a scratch head *into that base*, so the candidate is the definition that
runs. Have it post a **probe-named** context (e.g. `review-verdict/h10-PROBE`), never the real
`review-verdict/h10` — a probe must not be able to forge the gate it is testing.
3. Read the resulting commit statuses to see which definition actually ran, then delete both
branches.
The same shape is what makes a `branches:`/`types:` change verifiable at all, since neither can be
observed from the editing PR.
It lives in its **own workflow file** on purpose: `pr-checks.yml` sets `cancel-in-progress: true`,
and a cancelled run there would leave an exempt PR with no status and no further push to
re-trigger it. Its own job context (`Review verdict / Set review-verdict status`) is **not** the
required check — a workflow must not satisfy the gate merely by running successfully.
Full rationale: `docs/decisions/records/release/verdict-status-check.md`.
Full rationale: `docs/decisions/records/release/verdict-status-check.md` and
`docs/decisions/records/ci/shared-pr-file-enumeration.md`.
## CI toolchain image (`docker/ci/Dockerfile`, `.gitea/workflows/ci-image.yml`)
+2 -1
View File
@@ -206,7 +206,7 @@ another doc or an old issue comment should land here and then follow the link.
- 2026-07-22 — per-schedule clock-boundary padding is a synthetic content-less Pad over the existing per-episode machinery (#392) — [`sched.clock-padding-schedule-toggle`](decisions/records/sched/clock-padding-schedule-toggle.md)
- 2026-07-23 — Channel health = a server-derived `health` object on the channel DTOs, built-timeline detection (#415) — [`api.channel-health-object`](decisions/records/api/channel-health-object.md)
- 2026-07-23 — Channel origin is immutable creation-provenance, stamped at insert, not a health signal (#414) — [`channel.origin-marker`](decisions/records/channel/origin-marker.md)
- 2026-07-23 — Facet-value typeahead is a new endpoint, allow-listed to text fields, no caching (#434) — [`api.search-field-values`](decisions/records/api/search-field-values.md)
- 2026-07-23 — Facet-value typeahead is a new endpoint, allow-listed to text fields, no caching (#434) — [`api.search-field-values`](decisions/archive/api/search-field-values.md) (superseded by `api.search-field-values-sources`)
- 2026-07-23 — Relative-date rule builder operators are a frontend-only mapping onto existing Lucene macros (#435) — [`rulebuilder.relative-date-macros`](decisions/records/rulebuilder/relative-date-macros.md)
- 2026-07-25 — A media-server sweep also refuses when the api client silently dropped items whose projection threw; the ratio threshold is rejected (#484) — [`scan.projection-failure-sweep-guard`](decisions/records/scan/projection-failure-sweep-guard.md)
- 2026-07-25 — LibraryFolder identity is enforced by a unique index on `(LibraryPathId, PathHash)`, not an in-process lock (#491) — [`scan.libraryfolder-unique-identity`](decisions/records/scan/libraryfolder-unique-identity.md)
@@ -215,3 +215,4 @@ another doc or an old issue comment should land here and then follow the link.
- 2026-07-25 — Rule-builder group nesting is bounded-arbitrary depth (`MAX_GROUP_DEPTH`), not one level (#436) — [`spa.rulebuilder-nesting`](decisions/records/spa/rulebuilder-nesting.md)
- 2026-07-25 — The rationale-edit marker is a git trailer, not a substring anywhere in the commit range (#609) — [`ci.decisions-edit-trailer`](decisions/records/ci/decisions-edit-trailer.md)
- 2026-07-25 — UI-E2E: headless Playwright flows in the existing `functional-e2e` job, browser baked into the CI image (#445) — [`ci.ui-e2e-harness`](decisions/records/ci/ui-e2e-harness.md)
- 2026-07-26 — Facet-value typeahead restated: every artist source covered; the JSON-column source is paged by row position with no residual SQL predicate (#578) — [`api.search-field-values-sources`](decisions/records/api/search-field-values-sources.md)
+12 -4
View File
@@ -28,8 +28,10 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `api.schedule-item-flat-dto` | Schedule-item GET/POST/PUT use a flat, non-polymorphic `ScheduleItemResponseModel` (every subtype field promoted to a nullable top-level member) instead of the polymorphic Application VM hierarchy, with mutation field names matching `ScheduleItemRequest` 1:1 for a lossless round-trip. | 2026-07-10 | [link](records/api/schedule-item-flat-dto.md) |
| `api.scheduling-hardening` | Create/Replace handlers guard against null/whitespace `name` (`IsNullOrWhiteSpace`, not just `Length`) to prevent NRE-500s, template-item overlap validation compares by index (not record value-equality) to catch exact-duplicate items, and unreachable 404 `ProducesResponseType` attributes on create-only actions are trimmed. | 2026-07-13 | [link](records/api/scheduling-hardening.md) |
| `api.search-allitems-paging` | `GET /api/v1/search/all-items` is paginated (capped page size, `Totals` field) to bound DoS exposure; the SPA add-all flow pages to completeness instead of relying on an unbounded response. | 2026-07-18 | [link](records/api/search-allitems-paging.md) |
| `api.search-field-values` | `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for one of a narrow allow-list of catalog fields (not the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a typeahead suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source; case-insensitive prefix-filtered on `q`, `limit` clamped to `[1, 50]` (default 50). | 2026-07-23 | [link](records/api/search-field-values.md) |
| `api.search-field-values-sources` | `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite's `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404'd, as bounded best-effort, and its rows are read by a keyset page carrying **NO RESIDUAL predicate**`SELECT Id, <col> AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL`. The cursor is itself a predicate, but a SEEKABLE one on the ordering key: it positions the scan and never discards a row. A RESIDUAL predicate discards rows the engine already produced, and `LIMIT` truncates only the survivors — so with one present it bounds the OUTPUT rather than the row count. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling. | 2026-07-26 | [link](records/api/search-field-values-sources.md) |
| `api.search-field-values-unicode-fold` | The EF-sourced facet fields (`genre`, `show_genre`, `studio`, `director`, `writer`, `actor`, `tag`, `network`, `collection`, `video_codec`, `album`, and `artist`'s entity half) reach stored values whose prefix carries an uppercase non-ASCII character, on BOTH providers, with no row budget and no accepted loss. The defect was SQLite-only and ONE-SIDED: SQLite's `LOWER()` folds ASCII only (`lower('Édith')` is `'Édith'` unchanged), so the predicate UNDER-matched, which no later stage can repair. MySQL was already correct — its `LOWER()` is Unicode-aware, so `LOWER('Édith')` really is `'édith'` and the existing predicate reaches the row unaided. The fix is a SECOND, ADDITIVE query taken only when `isSqlite && q contains a non-ASCII character`: raw Dapper SQL `SELECT DISTINCT <col> AS Value FROM <table> WHERE [<discriminator> AND] etv_upper(<col>) LIKE @Pattern ESCAPE '\' ORDER BY <col> LIMIT @Limit`, where `etv_upper` is a `SqliteConnection.CreateFunction` scalar implementing `ToUpperInvariant`. Every other case — all-ASCII `q`, and MySQL for all `q` — runs today's EF query BYTE-IDENTICALLY. Keeping selectivity in SQL here is NOT the refuted family from `api.search-field-values-sources`: those four attempts bounded a walk around a predicate that could not be made correct over JSON escape text, whereas this is a correct fold on a plain column in an ordinary `LIMIT`ed query. It narrows that record's "Known limitation inherited, not introduced" clause; everything else it settles still holds. | 2026-07-27 | [link](records/api/search-field-values-unicode-fold.md) |
| `api.search-paging-cap` | Search stays capped at 100 items per media kind; an overflowing kind's "See all" reuses library-browse paging instead of adding new API surface. | 2026-07-11 | [link](records/api/search-paging-cap.md) |
| `api.selection-projection-include-chain` | Every handler that projects an aggregate carrying a tagged-union selection loads it through ONE shared `<Aggregate>QueryExtensions` include chain — `RerunCollectionQueryExtensions.IncludeSelectionDetails()`, joining the existing `ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails()` — called by the paged-list handler and the by-id handler alike, so the two cannot drift. The media-item flattening switch is likewise ONE shared helper, `MediaCollections.Mapper.ProjectMediaItemToViewModel`, covering all ten selectable media types including `RemoteStream`, whose named projection is `MediaItems.Mapper.ProjectToNamedViewModel` (it cannot be an overload of `ProjectToViewModel(RemoteStream)`, which already exists returning the unrelated `RemoteStreamViewModel`; C# will not overload on return type). That switch NEVER ends in `_ => null`: a null MediaItem is the legitimate not-a-media-item case, while an unrecognized non-null subtype keeps its id and takes a conspicuous `[unsupported media type: X]` name. Fail-soft is deliberate — throwing would fail an entire paged GET over one unreadable row. Finally, every metadata navigation inside `MediaItems.Mapper` is read through `Optional(...).Flatten()` and degrades to the `"???"` placeholder, because those projections are reached from handlers whose include chains differ and a bare `x.Season.Show.ShowMetadata` is a latent 500 on some other caller GET. | 2026-07-28 | [link](records/api/selection-projection-include-chain.md) |
| `api.versioning-v1` | The entire `/api` surface is versioned to `/api/v1` uniformly (no unversioned corner); legacy unversioned callers are rewritten in-pipeline (not redirected) with Deprecation/Link/Sunset headers, and post-freeze `/api/v1` is additive-only — a breaking change requires `/api/v2`. | 2026-07-13 | [link](records/api/versioning-v1.md) |
| `blazor.rollback-tag` | The commit immediately preceding the Blazor-removal merge is tagged `blazor-final` (not a `v*` tag, so it doesn't trigger a prod release build) as the documented rollback/restore path. | 2026-07-11 | [link](records/blazor/rollback-tag.md) |
| `blazor.ui-removed` | The legacy Blazor Server UI (`Pages/`, `Shared/`, `ViewModels/`, `Validators/`, MudBlazor + 8 other packages, Blazor Startup wiring) is fully deleted now that the SPA has parity; the legacy `MapWhen` branch is kept only for controllers/docs/OpenAPI/`LegacyUiRedirects`, and the catch-all fallback 302s any unmatched non-api/artwork/docs/openapi path to `/app`. | 2026-07-11 | [link](records/blazor/ui-removed.md) |
@@ -41,10 +43,14 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `ci.decisions-lifecycle-flake` | When `decisions lifecycle` is the **only** red job, do not investigate and do not create a new run to clear it — no rebase, no `--amend`, no no-op push; the operator reruns that single job from the Gitea UI. | 2026-07-21 | [link](records/ci/decisions-lifecycle-flake.md) |
| `ci.docs-only-detect-shallow-safe` | The docs-only detect script must diff against `FETCH_HEAD` (always resolves after `git fetch`, even shallow) using a two-dot tree diff — not `origin/<base>` with three-dot — because a `fetch-depth: 1` shallow clone has no remote-tracking ref and no merge-base, which silently fails the original detect into `docs_only=false` (full matrix, no functional error). A CI-behavior change must be verified by measuring the effect (job durations), not just a green check. | 2026-07-17 | [link](records/ci/docs-only-detect-shallow-safe.md) |
| `ci.docs-only-skip-steps` | A docs-only change must still run every required job (`test`, `migrations`) so their commit-status contexts always report; each heavy job runs `scripts/ci-detect-docs-only.sh` first and gates its real STEPS on `if: steps.detect.outputs.docs_only != 'true'`, never `if:`-skips the whole job (an `if:`-skipped job reports `skipped`, not `success`, which branch protection may never unblock on). Detection biases toward running more on any doubt. | 2026-07-17 | [link](records/ci/docs-only-skip-steps.md) |
| `ci.exemption-provenance` | The three inputs the exemption decision rests on must each be bound to something the judged PR cannot mutate. (1) BASE — `scripts/pr-changed-files.sh` takes the expected base BRANCH as a REQUIRED 5th argument and re-reads it before and after paging, because `/pulls/{n}/files` diffs against the PR's live base and retargeting moves the answer without moving the head sha; the workflow passes `github.event.pull_request.base.ref` from the `pull_request_target` payload, which a retarget cannot rewrite. (2) BOT EXEMPTION — an author match is necessary but never sufficient: `pull_request.user.login` is the PR's immutable CREATOR while its head is not, so the exemption additionally requires EVERY changed path to be a dependency manifest (`Directory.Packages.props` or `.config/dotnet-tools.json`, and ONLY those — the npm manifests are excluded because `package.json` `scripts` are executed by CI). (3) INHERITED SUCCESS — the never-overwrite short-circuit fires only for a status POSITIVELY identified as a human verdict for THIS base, meaning a non-null `.creator.login` AND a `Review-verdict:` description AND, when that description records a base (`(base: …)`, `release.verdict-status-check`), a base matching the PR's — tested by requiring the description to END with the exact literal `(base: <base>)` and to contain exactly ONE such marker, never by extracting a value (see below); a present-but-different base is rejected, an absent one is not, since verdicts predating that convention carry none; every other shape, including any unrecognised one, is re-derived rather than trusted. The bot and docs-only exemptions are evaluated as INDEPENDENT predicates and the decision made afterwards, never as an `elif` chain. `edited` is in the workflow's `types:` so a retarget reclassifies — which gives DETECTION, not atomicity: status writes are not serialized, so a stale run can still post over a fresher one (residual, #706). Path predicates are evaluated by COUNTING with `grep -c`, never `\| grep -q` (SIGPIPE inversion) and never a here-string (temp-space failure) — see `ci.grep-q-pipefail-inversion`. | 2026-07-29 | [link](records/ci/exemption-provenance.md) |
| `ci.format-gate-folder-mode` | The blocking `format` CI job (and matching pre-commit hook) runs `dotnet format whitespace . --folder --include <files>` instead of loading the full MSBuild/Roslyn solution, cutting the gate from ~480s to ~0.5s with unchanged whitespace/charset coverage. | 2026-07-19 | [link](records/ci/format-gate-folder-mode.md) |
| `ci.functional-e2e-harness` | The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. Originally curl-only; since #445 the same job carries a second, headless-browser step for the contracts curl cannot express — see `ci.ui-e2e-harness`. | 2026-07-16 | [link](records/ci/functional-e2e-harness.md) |
| `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class: Gitea injects a write-capable `GITEA_TOKEN` into EVERY job, so any ref-resolved workflow — and a collaborator's own API token, since branch protection binds the context and not its issuer — can still forge `review-verdict/h10`. Tracked in #697; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) |
| `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=<name>` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) |
| `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) |
| `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) |
| `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) |
| `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) |
| `ci.monitor-armed-at-pr-open` | Arm a CI monitor on the PR head sha the moment the PR opens, polling the commit-status endpoint — not at the end of the work. | 2026-07-21 | [link](records/ci/monitor-armed-at-pr-open.md) |
| `ci.no-host-health-gating` | Push when your work is validated — never SSH to bumblebee to sample load/RAM first, and never hand-schedule around other sessions' runs. | 2026-07-21 | [link](records/ci/no-host-health-gating.md) |
@@ -52,6 +58,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `ci.root-screenshot-guard` | The Husky `pre-commit` hook refuses a staged root-level `*.png` (belt-and-suspenders with the `.gitignore` rule); nested `*.png` real assets are unaffected. | 2026-07-12 | [link](records/ci/root-screenshot-guard.md) |
| `ci.runner-placement` | No persistent Roslyn compiler server survives a CI build (`UseSharedCompilation=false` etc., runner env + Dockerfile `ENV`); every `services:` container gets its own explicit `--memory`/`--memory-swap`/`--cpus` cap (it does not inherit the job container's). | 2026-07-17 | [link](records/ci/runner-placement.md) |
| `ci.script-tests-job` | The `scripts/tests/` pytest suite runs on every PR as a dedicated `script-tests` job in `pr-checks.yml` (`runs-on: small`, `setup-python` + `pip install pytest`, `PYTHONPATH=. python3 -m pytest scripts/tests -q`), unconditionally rather than behind a `scripts/**` path filter, and **never as a step inside `decisions-guard`** — a job whose reds a standing rule instructs sessions to ignore must never host a gate whose reds are real. Any new CI gate must be reachable by a failure that is unambiguously attributable to it. | 2026-07-26 | [link](records/ci/script-tests-job.md) |
| `ci.shared-pr-file-enumeration` | A PR's complete set of changed file paths is computed by exactly one implementation, `scripts/pr-changed-files.sh`, called by both `.claude/hooks/pretooluse-merge-consent.sh` (advisory — a failure falls through to a human prompt) and `.gitea/workflows/review-verdict.yml` (enforced — a failure must fail closed, because a match here posts the branch-protection-required `review-verdict/h10` status with nobody in the loop). The script owns exhaustiveness (pagination, rename/path validation, head-sha binding, and base-ref binding — see `ci.exemption-provenance`) and returns exit 0 only for a verified-complete list; it does NOT classify paths — each caller keeps its own docs-only allow-list, and the two allow-lists differ on purpose and stay separate. | 2026-07-26 | [link](records/ci/shared-pr-file-enumeration.md) |
| `ci.small-lane-git-only` | `runs-on: small` is defined by what a job does (git-only), not its usual runtime; the two `docker build` jobs (docker-build.yml, ci-image.yml) move to `ubuntu-latest` because their worst-case memory, not median runtime, was pinning the small lane's per-slot cap. | 2026-07-20 | [link](records/ci/small-lane-git-only.md) |
| `ci.ui-e2e-harness` | The UI-interactive E2E flows run as headless Playwright specs (`web/e2e/*.spec.ts`, driven by `scripts/e2e-ui.sh`) in a **second step of the existing advisory `functional-e2e` job**, never their own job; the browser is `chromium-headless-shell` **baked into the CI toolchain image** (`docker/ci/Dockerfile`, `PLAYWRIGHT_VERSION` kept equal to `web/package.json`'s EXACT `@playwright/test` pin), never installed per run; specs are `serial` with `retries: 0` and assert only contracts the curl harness structurally cannot reach. | 2026-07-25 | [link](records/ci/ui-e2e-harness.md) |
| `ci.verify-locally-ci-confirms` | Treat the local build/verify/review pass as the decision point and CI as confirmation — don't idle waiting on a run you have no reason to doubt. | 2026-07-21 | [link](records/ci/verify-locally-ci-confirms.md) |
@@ -102,7 +109,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `process.local-gate-before-push` | Run the local build/test gate and a cold-context, scoped "review only" adversarial review over the diff, fold the fixes, and only then push or open the PR. | 2026-07-21 | [link](records/process/local-gate-before-push.md) |
| `process.lock-ownership-enumerate-producers` | Before trusting any "single owner / no double release / no cross-release" claim, grep the whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer. | 2026-07-21 | [link](records/process/lock-ownership-enumerate-producers.md) |
| `process.one-worktree-one-committing-agent` | Never run two committing agents concurrently on one worktree — give each parallel slice its own worktree branched off the feature branch and merge back. | 2026-07-21 | [link](records/process/one-worktree-one-committing-agent.md) |
| `process.parallel-session-claim` | Apply the `in-progress` label before starting an issue, and still read its dependency notes before touching shared surfaces — a claim prevents duplicate pickup, not overlapping code changes. | 2026-07-21 | [link](records/process/parallel-session-claim.md) |
| `process.parallel-session-claim` | Before starting an issue, check for an existing claim four ways — open PRs referencing it, remote branches naming it, recent comments (a claim can precede the label), and a fresh `git fetch origin main` — then claim with the `in-progress` label plus a comment. A claim prevents duplicate PICKUP, not duplicate WORK. Re-fetch `origin/main` before every push, not only at branch time. | 2026-07-21 | [link](records/process/parallel-session-claim.md) |
| `process.per-agent-model-routing` | State the model tier (and effort, where the client exposes it) in the dispatch itself for every delegated agent — bounded recon → cheapest fast tier at `low`; mechanical slice against a documented contract → mid tier; judgment-heavy work → orchestrator tier; independent review → a different model family than the implementer. | 2026-07-25 | [link](records/process/per-agent-model-routing.md) |
| `process.pr-routine-sequence` | Worktree off origin/main → implement → regenerate API artifacts → full local tests + cold review + live-E2E ALL before the push → push, open PR, arm the CI monitor at open → fixes after the push are follow-up commits, never amend/force-push. | 2026-07-21 | [link](records/process/pr-routine-sequence.md) |
| `process.review-disagreement-frontier-judge` | When independent reviews disagree on a gate PR, escalate to the frontier judge, and put the proposed fix approach in front of it — not just the disputed finding. | 2026-07-21 | [link](records/process/review-disagreement-frontier-judge.md) |
@@ -117,7 +124,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `release.prepush-clean-worktree-guard` | A fail-open pre-push hook blocks a push when any file in the branch's diff vs `origin/main` also has uncommitted working-tree or index changes, since a stale-index commit (e.g. `git reset --soft` + `git add` over an edited-but-unstaged fix) can silently push, CI-test, and get reviewed a different tree than the one on disk. Scope is precise to pushed-diff files; escape hatch `ETV_ALLOW_DIRTY_PUSH=1`. | 2026-07-17 | [link](records/release/prepush-clean-worktree-guard.md) |
| `release.promotion-floating-prod` | Prod tracks the floating `:prod` image reference; a tag build's immutable `:<version>` image is scanned first, then promotion happens via a separate manual `DeployStack`, with daily auto-update only as a fallback — tag with enough runway before 03:00 to avoid an unscanned promotion. | 2026-07-13 | [link](records/release/promotion-floating-prod.md) |
| `release.review-verdict-gate` | A PR may not merge until a `Review-verdict: <MERGEABLE\|APPROVED\|BLOCKED\|NOT-MERGEABLE> @ <head-sha>` comment references the PR's current head sha (short-sha prefix match against the verdict's OWN `@ <sha>` field, marker at COLUMN 0 (no indent, so indented code blocks cannot self-approve), whole-word verdict token, fenced code blocks stripped with markdown fence-length semantics, negative wins over positive on the same head); folds into the H6 merge-consent hook as condition (c). The grammar lives in ONE tested place, `scripts/check-review-verdict.sh`#629 found three false-opens that survived because it was implemented inline and untested while this record described stricter behaviour than the code had. | 2026-07-12 | [link](records/release/review-verdict-gate.md) |
| `release.verdict-status-check` | The H10 review verdict is written as a `review-verdict/h10` Gitea **commit status** on the exact reviewed sha by `scripts/post-review-verdict.sh`, and that context is a REQUIRED status check on `main`. Because a status belongs to one sha, a later commit cannot inherit it, so Gitea's own `merge_when_checks_succeed` refuses to merge a head no one reviewed. The PreToolUse hook additionally refuses to SCHEDULE an auto-merge unless that status is already green on head. A `pull_request` workflow auto-passes the two exempt classes (Renovate-authored, docs-only) unless the PR touches a protected path (`.claude/`, `.gitea/`, `.husky/`, `scripts/`, `docker/ci/`). This extends — does not supersede — `release.review-verdict-gate` (#303 H10), whose comment convention remains the human-readable artifact and the hook's condition (c). | 2026-07-25 | [link](records/release/verdict-status-check.md) |
| `release.verdict-status-check` | The H10 review verdict is written as a `review-verdict/h10` Gitea **commit status** on the exact reviewed sha by `scripts/post-review-verdict.sh`, and that context is a REQUIRED status check on `main`. Because a status belongs to one sha, a later commit cannot inherit it, so Gitea's own `merge_when_checks_succeed` refuses to merge a head no one reviewed. The PreToolUse hook additionally refuses to SCHEDULE an auto-merge unless that status is already green on head. A `pull_request_target` workflow auto-passes the two exempt classes (Renovate-authored, docs-only) unless the PR touches a protected path (`.claude/`, `.gitea/`, `.husky/`, `scripts/`, `docker/ci/`). This extends — does not supersede — `release.review-verdict-gate` (#303 H10), whose comment convention remains the human-readable artifact and the hook's condition (c). | 2026-07-25 | [link](records/release/verdict-status-check.md) |
| `rulebuilder.relative-date-macros` | The visual rule builder's `inLast`/`notInLast` date operators compile to/parse from the pre-existing `CustomMultiFieldQueryParser` macros `released_inthelast`/`released_notinthelast` and `added_inthelast`/`added_notinthelast`, value form `"<n> day\|week\|month\|year"`; there is no backend change. | 2026-07-23 | [link](records/rulebuilder/relative-date-macros.md) |
| `scan.collections-scan-status` | `GET /api/v1/media-sources/collections-scan-status` reports a family-global (not per-source), boolean-only active-scan set read from `IEntityLocker`; the SPA reconciles authoritatively against it (with a grace-tick helper) instead of a fixed client-side timeout. | 2026-07-12 | [link](records/scan/collections-scan-status.md) |
| `scan.getoraddfolder-db-lookup` | `ILibraryRepository.GetOrAddFolder` resolves the existing folder via a DB query on `(LibraryPathId, Path)`, not the caller's `LibraryPath.LibraryFolders` in-memory navigation, since that navigation is only eager-loaded on the local scan path and is null on remote (Jellyfin) callers. | 2026-07-20 | [link](records/scan/getoraddfolder-db-lookup.md) |
@@ -162,7 +169,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `spa.deco-templates-table` | The deco-templates editor also renders its day/deco assignment as a table, extending (not replacing) the templates-editor-table convention. | 2026-07-09 | [link](records/spa/deco-templates-table.md) |
| `spa.download-sample-gate` | The SPA disables both Download Media Sample and Download Results while a troubleshooting session is starting/running (Blazor only gated Download Results). | 2026-07-09 | [link](records/spa/download-sample-gate.md) |
| `spa.legacy-redirect-matcher` | `LegacyUiRedirects.TryGetRedirect` is a two-tier matcher — an exact `OrdinalIgnoreCase` `Map` (Tier 1) then an ordered segment-template pattern list (Tier 2, first-match-wins) — collision-free by construction, with a guard invariant that no rule may prefix-match `/api`, `/artwork`, `/docs`, `/openapi`, `/iptv`, `/app`, or `/media/sources`. | 2026-07-11 | [link](records/spa/legacy-redirect-matcher.md) |
| `spa.list-completeness-vs-bounded-pickers` | The shared `loadAllPages` helper (`web/src/api/paging.ts`) pages a `/api/v1` list to completeness against `totalCount` and is used ONLY for lists that are bounded by construction (rerun collections, multi-collections, playlists — admin-created, hundreds of rows at most). A `getLibraryBrowseItems` picker over a media-library table (Episode/Song/Image/Movie/MusicVideo, tens of thousands of rows possible) must NOT page to completeness — it fetches ONE bounded page (the server cap) and surfaces the truncation (a `ctv-field-help` hint wired to the real `totalCount`) instead of silently dropping the rest. | 2026-07-26 | [link](records/spa/list-completeness-vs-bounded-pickers.md) |
| `spa.library-pickers-resolve-by-search` | A picker over a media-library table (Episode/Song/Image/Movie/MusicVideo/TelevisionShow/TelevisionSeason/Artist/OtherVideo/RemoteStream) resolves its options by SEARCH — a debounced `SearchPicker` calling `searchLibraryPickerOptions`, which issues at most ONE `getLibraryBrowseItems` request per settled query, bounded to `LIBRARY_PICKER_RESULTS` (25) rows — CLAMPED inside the helper, not merely defaulted — and gated on `LIBRARY_PICKER_MIN_QUERY` (2) characters. It list-loads NOTHING on mount or on a type switch, so there is no truncation to surface and no truncation hint. The typed text is COMPILED (`titleContainsQuery``title:*<escaped>*`), never forwarded raw. The current selection renders from the OWNING RECORD, not from the result set (`selectedName` on a rerun collection / playlist item; a single by-id detail read — `getShow`/`getSeason`/`getArtist` — for a filler preset, which stores only the id), and an edit draft is INITIALIZED ONCE from the detail read — never seeded from the list row, never reconciled against a late response — with the form withheld until it lands, the editor failing CLOSED when the response carries no USABLE concurrency token — absent, empty and whitespace-only ETags are ONE case, normalized in one place, so a PUT without `If-Match` is unreachable, and a deadline plus a route back so a hung request cannot strand it. An id NEVER travels without its namespace: search results are cached against `(source, query)` and list-backed options carry the type they were loaded for, so no id from one type can be offered under another; and every id entering editor state — search result, list-backed option, or a selection restored from a detail read — passes ONE shared `isSelectionId` (int32) predicate at that boundary, an unbindable id being treated as ABSENT rather than coerced. Conflicts are detected at SAVE time via `If-Match` -> 412 -> Reload, and Reload simply drops the draft back to null and re-runs the same initialize-once load, so the form is unmounted while the replacement is in flight; an asynchronously-resolved name is keyed to the id it was resolved for and never overwrites a label naming a different id. The typeahead implements the full ARIA combobox keyboard contract, because it replaces a natively keyboard-operable `<select>`. The other half of the superseded record is UNCHANGED: bounded-by-construction admin lists (collections, multi-collections, smart collections, playlists) still page to completeness via `loadAllPages` and still report `complete`/`hint: incomplete`. Server-side caps are not raised — this is a web-only change. | 2026-07-26 | [link](records/spa/library-pickers-resolve-by-search.md) |
| `spa.logs-page-size-local` | The Logs page rows-per-page preference is stored in `window.localStorage` (`ctv-logs-page-size`), not a server `ConfigElement`. | 2026-07-11 | [link](records/spa/logs-page-size-local.md) |
| `spa.playback-troubleshoot-poll` | The playback-troubleshooting screen reports FFmpeg completion by polling `GET /api/troubleshoot/playback/status` (~2s) rather than a server push channel. | 2026-07-09 | [link](records/spa/playback-troubleshoot-poll.md) |
| `spa.playout-reset-button` | The SPA keeps a single Reset action (server picks the default build mode) and drops Blazor's separate "Schedule reset" button since its capability already exists via the playout's Edit-details flow. | 2026-07-09 | [link](records/spa/playout-reset-button.md) |
@@ -177,6 +184,7 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
| `startup.parallel-orientation` | A fresh session runs two concurrent tracks at startup — Orientation (`AGENTS.md`/`CLAUDE.md``docs/README.md` task-signal map → the active decisions catalog `docs/decisions/README.md`) and, only when no issue is named, Selection (`scripts/select-queue.sh N`, deterministic live-Gitea ranking). A named issue skips Selection entirely. ersatztv#237, the closed pickup tracker this replaces, is reduced to a single archival breadcrumb and MUST NOT be read for live state. | 2026-07-21 | [link](records/startup/parallel-orientation.md) |
| `testing.e2e-cleanup-scope-by-pid` | An E2E harness or agent may only kill processes whose PIDs it captured at launch — capture the PID; whoever owns the lifecycle releases it from a `trap ... EXIT INT TERM`. Never `pkill -f "dotnet ErsatzTV.dll"` (or any pattern that can match a process this run did not start). A foreign listener is reported, not reaped. | 2026-07-25 | [link](records/testing/e2e-cleanup-scope-by-pid.md) |
| `testing.e2e-local-fresh-config-dir` | Always point `scripts/e2e-local.sh` at a fresh config dir — leftover channels/schedules/DB rows bleed state between runs and corrupt assertions. (The *readiness-probe hang* this record was originally written about was fixed in #533; the fresh-dir rule stands on state-bleed grounds alone.) | 2026-07-21 | [link](records/testing/e2e-local-fresh-config-dir.md) |
| `testing.enumerating-guard-identity-not-position` | A guard that cross-checks a hand-reviewed registry against call sites discovered across the whole repo must key each entry on properties INTRINSIC to the site — file, kind, and the value source text — and never on its absolute line or column. A registry keyed on position is a function of every other file in the repo, so a branch that never touches the guard can invalidate it; and because each PR is green against its own base, that failure is structurally invisible pre-merge and lands on `main` after review and after the merge gate. Dropping the position keeps every mutation the guard exists for — a NEW site, a REMOVED site and a CHANGED value each still fail, since each changes the identity multiset — and costs exactly ONE case, which must be stated rather than implied: a SAME-IDENTITY SUBSTITUTION within one file (delete a registered site, add a different unreviewed one with the same kind and value token, net-zero count) now passes. A REPORTED failure still prints the discovered line:column, because identity and diagnostics need not share a format. Comparison stays a MULTISET count rather than set membership, so two sites in one file sharing an identity must be discovered exactly that many times and a third occurrence still fails. A SCANNER test that asserts real AST positions against FIXED inline fixtures is the opposite case and keeps its line/column identity — it has no churn, because its input does not move. | 2026-07-27 | [link](records/testing/enumerating-guard-identity-not-position.md) |
| `testing.live-e2e-prepush-timing` | Run live-E2E via `scripts/e2e-local.sh` before pushing a write-path or UI change, and exercise download endpoints with curl, never a browser tab. | 2026-07-21 | [link](records/testing/live-e2e-prepush-timing.md) |
| `testing.playwright-mcp-download-and-recovery` | In Playwright-MCP E2E, fetch file-download endpoints with curl — never a browser tab or `window.open` — and if browser tools stall repeatedly, `pkill -f ms-playwright-mcp` and drive a fresh session. | 2026-07-21 | [link](records/testing/playwright-mcp-download-and-recovery.md) |
| `testing.scripted-playout-golden-deferred` | The `PlayoutBuildGoldenTests` in-memory golden net covers Sequential (YAML) as of #381. Scripted's *end-to-end pipeline* is excluded — `ScriptedPlayoutBuilder` runs a user-authored external program that drives the engine over HTTP loopback, which the in-memory harness can't pin — so that full-pipeline (integration) harness is deferred to #563. But the scheduling *behavior* those scripts drive lives entirely in the in-process `SchedulingEngine` (the `ScriptedScheduleController` is a 1:1 pass-through to it), which IS directly unit/golden-testable; the earlier "Scripted is un-golden-able by construction" framing overstated the constraint by conflating transport with engine. #395 extracts that shared switch to `ContentEnumeratorBuilder` and adds a direct regression net (`ContentEnumeratorBuilderTests`) over it. | 2026-07-22 | [link](records/testing/scripted-playout-golden-deferred.md) |
@@ -1,13 +1,13 @@
---
key: api.search-field-values
title: 2026-07-23 — Facet-value typeahead is a new endpoint, allow-listed to text fields, no caching (#434)
status: active
status: superseded
since: '2026-07-23'
supersedes: none
superseded-by: none
rule: '`GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for one of a narrow allow-list of catalog fields (not the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a typeahead suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source; case-insensitive prefix-filtered on `q`, `limit` clamped to `[1, 50]` (default 50).'
superseded-by: api.search-field-values-sources@2026-07-26
rule: '(superseded) `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for one of a narrow allow-list of catalog fields (not the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a typeahead suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source; case-insensitive prefix-filtered on `q`, `limit` clamped to `[1, 50]` (default 50).'
signals: 'facet-value typeahead, rule builder value combobox, distinct field values, GetSearchFieldValues, text field allow-list, DB-sourced distinct values, content_rating split · paths: `ErsatzTV/Controllers/Api/SearchController.cs`, `ErsatzTV.Application/Search/Queries/GetSearchFieldValues.cs`, `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `web/src/api/search.ts` · issues: #434, #176'
mechanics: '`SearchController.GetSearchFieldValues`; `GetSearchFieldValuesHandler`; api-conventions.md; spa-conventions.md §12'
mechanics: superseded by `api.search-field-values-sources` (ersatztv#578), which keeps this endpoint contract and reverses the "no distinct-value source" call for the list-valued music fields
---
Enum fields (e.g. `type`, `content_rating` group) already ship their allowed values inline on
@@ -1,13 +1,13 @@
---
key: spa.list-completeness-vs-bounded-pickers
title: '2026-07-26 — `loadAllPages` is for bounded-by-construction lists only; media-library pickers stay bounded and show truncation (#644 follow-up)'
status: active
status: superseded
since: '2026-07-26'
supersedes: none
superseded-by: none
rule: 'The shared `loadAllPages` helper (`web/src/api/paging.ts`) pages a `/api/v1` list to completeness against `totalCount` and is used ONLY for lists that are bounded by construction (rerun collections, multi-collections, playlists — admin-created, hundreds of rows at most). A `getLibraryBrowseItems` picker over a media-library table (Episode/Song/Image/Movie/MusicVideo, tens of thousands of rows possible) must NOT page to completeness — it fetches ONE bounded page (the server cap) and surfaces the truncation (a `ctv-field-help` hint wired to the real `totalCount`) instead of silently dropping the rest.'
superseded-by: spa.library-pickers-resolve-by-search@2026-07-26
rule: '(superseded) The shared `loadAllPages` helper (`web/src/api/paging.ts`) pages a `/api/v1` list to completeness against `totalCount` and is used ONLY for lists that are bounded by construction (rerun collections, multi-collections, playlists — admin-created, hundreds of rows at most). A `getLibraryBrowseItems` picker over a media-library table (Episode/Song/Image/Movie/MusicVideo, tens of thousands of rows possible) must NOT page to completeness — it fetches ONE bounded page (the server cap) and surfaces the truncation (a `ctv-field-help` hint wired to the real `totalCount`) instead of silently dropping the rest.'
signals: '`loadAllPages`, Class A vs Class B picker, LuceneSearchIndex.Search hitsLimit, picker truncation hint, ctv-field-help, PagedResult, `complete` flag · paths: `web/src/api/paging.ts`, `web/src/screens/RerunCollectionsScreen.tsx`, `web/src/screens/PlaylistsScreen.tsx`, `web/src/screens/FillerPresetsScreen.tsx`, `web/src/screens/MultiCollectionsScreen.tsx`, `docs/spa-conventions.md` §3b · issues: #644'
mechanics: '`docs/spa-conventions.md` §3b'
mechanics: 'superseded by `spa.library-pickers-resolve-by-search` (ersatztv#651) — Class A (`loadAllPages` for bounded-by-construction lists) survives there unchanged; only the Class B rule is reversed. See `docs/spa-conventions.md` §3b'
---
`fe342a6a` (#644) extracted the `loadAllPages` client-side paging helper and applied it at every
@@ -0,0 +1,210 @@
---
key: api.search-field-values-sources
title: '2026-07-26 — Facet-value typeahead, restated: every artist-bearing source is covered, and the JSON-column source is paged by ROW POSITION with no RESIDUAL SQL predicate (#578)'
status: active
since: '2026-07-26'
supersedes: api.search-field-values@2026-07-23
superseded-by: none
rule: '`GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite''s `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404''d, as bounded best-effort, and its rows are read by a keyset page carrying **NO RESIDUAL predicate**`SELECT Id, <col> AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL`. The cursor is itself a predicate, but a SEEKABLE one on the ordering key: it positions the scan and never discards a row. A RESIDUAL predicate discards rows the engine already produced, and `LIMIT` truncates only the survivors — so with one present it bounds the OUTPUT rather than the row count. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling.'
signals: 'artist typeahead free-text credits, album_artist 404, artist suggestions missing music videos, SongMetadata.Artists, SongMetadata.AlbumArtists, MusicVideoArtist, EF primitive collection, PrimitiveCollection JSON column, SelectMany requires APPLY on SQLite, Pomelo primitive collections not enabled, LIMIT bounds output not work, seekable cursor vs residual predicate, cursor is a predicate too, MySQL purge lag traverses deleted index records, TEXT overflow pages, logical rows not physical work, keyspace is not rows, page by row position, density-independent paging, deleted rows leave Id gaps, ListValuedBatchRows, ListValuedMaxRowsRead, bounded best-effort facet values, OrdinalIgnoreCase vs InvariantCultureIgnoreCase folding, Accept-Language tr-TR dotless i, content_rating split, text field allow-list · paths: `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `ErsatzTV/Controllers/Api/SearchController.cs`, `ErsatzTV.Tests/Application/Search/GetSearchFieldValuesHandlerTests.cs`, `ErsatzTV.Tests/Application/Search/SearchFieldValuesQueryShapeTests.cs`, `web/src/api/search.ts` · issues: #578, #434, #176, #668, #669'
mechanics: '`GetSearchFieldValuesHandler` (`GetSource`, `GetSongListValuedColumn`, `GetSongListValuedValues`, `ListValuedSql`, `ParseElements`, `FilterSortTake`); `SearchFieldValuesQueryShapeTests.List_Valued_Page_Query_Has_No_Predicate_Beyond_The_Keyset_Cursor`; api-conventions.md; spa-conventions.md §12'
---
Supersedes `api.search-field-values` (#434). That record did not merely carry a stale implementation
detail — it recorded a **call**: free-text music-video/song artist credits were "a known,
intentionally-uncovered gap" and `album_artist` was unsupported. #578 reverses that call, so this is
a supersession, not a line-edit. Everything #434 settled that still holds is restated here rather
than left in the archive: enum fields ship their values inline on `GET /api/v1/search/fields` and
need no lookup; text fields need a live one; the source is the database and never the search index;
`content_rating` splits its compound `"PG-13/TV-14"` strings in memory; there is no result cache.
## The three `artist` sources are three different problems
`LuceneSearchIndex` writes the `artist` field from three places, and only two are ordinary columns:
- `ArtistMetadata.Title` — a plain column. Already worked.
- `MusicVideoArtist.Name` — also a real entity table (`MusicVideoMetadata.HasMany(m => m.Artists)`),
so the free-text music-video credits are directly `SELECT DISTINCT`-able. It just joins the
existing server-side pipeline as a `Concat`, emitted as one bounded `UNION ALL` +
`LOWER(...) LIKE ... LIMIT` on both providers.
- `SongMetadata.Artists` (and, for `album_artist`, `AlbumArtists`) — an `IList<string>` EF 9 maps as
a **primitive collection**: no `HasConversion` anywhere, one JSON array per row in a single
`TEXT`/`longtext` column, with no server-side projection at all. Verified against both providers:
SQLite reports *"Translating this query requires the SQL APPLY operation, which is not supported on
SQLite"*, Pomelo MySQL 9.0.0 reports *"Primitive collections support has not been enabled"*. Both
failures are pinned by a test, so a provider upgrade that fixes them surfaces as a red rather than
leaving a workaround in place forever.
## The SQL predicate is gone, and that is the point
Three revisions tried to narrow the rows in SQL before filtering them in memory. All three were
wrong, in three different ways, and the fourth was wrong too — the history is worth more than the
code, so it is written out below under "four wrong quantities". The conclusion is short: **there is
no `WHERE` clause beyond the keyset cursor.** No `LIKE`, no `LOWER`, not even `IS NOT NULL`.
That deletes an entire family of bugs along with the predicate. Gone with it: the JSON-escape
reasoning (`Édith` is stored `\u00C9dith`, and SQL `LOWER()` folds the escape *text* rather than the
codepoint it denotes, so a `q=é` pattern of `\u00e9` never matched `\u00C9`); the "narrow only on the
leading verbatim-ASCII run" rule and the exhaustive Unicode sweep that proved it sound; the
`ESCAPE '/'` portability workaround; and the whole may-over-match-never-under-match invariant, which
turned out to be conditional on something that was not true. In memory a string is just a string:
`element.StartsWith(query, StringComparison.OrdinalIgnoreCase)`.
Worth keeping one number from that history, because it is the reason the first bug survived review:
the JSON-encoded pattern failed on **three of nine** pinned cases, not all nine — those where the
query's casing differed from the stored casing, so the two escape texts diverged. When the casings
agreed it worked. A bug that fires on some inputs and not others reads as "works" during a spot
check.
## Ordinal everywhere, because the culture is caller-controlled
`UseRequestLocalization` honours `Accept-Language`, so a caller can select `tr-TR` and turn `q=I`
into `ı`. The old chain used `ToLower()` plus the default *linguistic* `StartsWith(string)`, making
the same library answer differently per caller. Comparison is now `OrdinalIgnoreCase` and ordering
`StringComparer.Ordinal` throughout the in-memory stages, including the shared `FilterSortTake` that
`state`, `video_dynamic_range` and `content_rating` also use. That is a deliberate change to shared
behaviour, and **not a cosmetic one**: ordering happens before `Take(limit)`, so changing the
comparer can change *which* values survive, not merely their order. With `"Zulu"` and `"apple"`, an
empty `q` and `limit=1`, linguistic ordering yields `"apple"` and ordinal yields `"Zulu"`. An earlier
version of this record claimed the response sets were unchanged; that was false.
**Scope this claim carefully — it is not endpoint-wide.** A field sourced by a plain EF query runs
the database's `LOWER`, `DISTINCT`, `ORDER BY` and `LIMIT` *before* any ordinal code executes, so the
database has already decided which values survive. Store a genre `"Éclair"` on SQLite and ask for
`genre?q=é`: SQLite's ASCII-only `LOWER()` drops it before the ordinal in-memory filter ever runs, and
a case-insensitive collation's `DISTINCT` can likewise collapse values ordinal dedup would have kept.
The endpoint description and this record's `rule:` therefore say "the final filter, dedup and
ordering", not "matching is ordinal". That gap is now CLOSED by `api.search-field-values-unicode-fold`
(**ersatztv#668**) — not by the client-side filtering guessed at here, but by a registered Unicode-correct
SQL fold on a second, additive query taken only for non-ASCII queries on SQLite. The scoped wording above
still stands as written: it describes what the EF stage itself does, which is unchanged.
## Ordering is best-effort, and the code says so
Merging sources does **not** yield the exact first `limit` of the union. Each source truncates using
its own ordering — the EF source by the database collation, the list source by primary key — and
neither is the ordinal ordering the merge applies. The pair that actually demonstrates it is `"Zulu"`
and `"apple"`: ordinal puts every ASCII uppercase letter before every lowercase one, so the merge
ranks `"Zulu"` first, while the case-insensitive database ordering ranks `"apple"` first — at
`limit=1` the response is `["apple"]`, not the ordinally-first `"Zulu"`. Below the truncation points
— the normal typeahead case — the result is exact. An earlier comment claimed exactness the code does
not have; do not restore it. (An earlier version of this record used `"Zulu"`/`"Éclair"` as the
example, where both orderings pick `"Zulu"` — it demonstrated nothing.)
## What the bound bounds — four attempts, four wrong quantities
Read this before "optimizing" the query. Every one of these looked obviously correct when written,
and each was caught only by someone constructing the adversarial case rather than reading the code.
| # | Bounded | Why it wasn't a bound |
|---|---|---|
| 12 | the **result** — fixed `LIMIT 1000` on pre-filtered rows | the pre-filter was deliberately allowed to over-match, so a widened pattern (any non-ASCII or JSON-escaped prefix collapses it to `%"%`) filled the budget with rows that could not match. 1,000 `"zzz"` songs, `"éclair"` at row 1,001, `q=é``[]` |
| 3 | **candidates returned** — keyset paging + `LIMIT` | a query matching nothing must evaluate every eligible row before it can return an empty page, so the first empty page ended the walk having counted **zero** against the ceiling. Rows returned bounded, rows inspected unbounded |
| 4 | **keyspace width** — closed `Id` range per page | keyspace is not rows. Delete 20,000 historical rows, put one song at `Id` 20001, `q=que``[]`. **One row in the table, zero rows inspected.** Capacity fell linearly with deletion ratio and no ratio was safe: one placed gap hides the next match |
| 5 | **logical rows returned** — keyset page by row position, cursor only, **no residual predicate** | — (physical work still unbounded; see below) |
The through-line: **`LIMIT` truncates what survives a RESIDUAL predicate.** The distinction is not
"predicate vs no predicate" — attempt 5's query still has `Id > @AfterId`. It is:
- a **seekable predicate on the ordering key** (the cursor) positions the scan and never discards a
row, so `LIMIT n` yields `n` rows;
- a **residual predicate** (`LIKE`, `LOWER`, `IS NOT NULL`) throws away rows the engine already
produced, so `LIMIT` bounds the survivors and says nothing about how many were produced.
Attempts 3 and 4 both kept selectivity in SQL and tried to add accounting around it. Attempt 5 drops
the residual predicate and keeps only the cursor, so the accounting becomes trivial:
```sql
SELECT Id, Artists AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch
```
`ListValuedBatchRows = 2000`, `ListValuedMaxRowsRead = 20000`. The walk stops on the first of: enough
distinct matches for `limit`, a short page (with no residual predicate that can only mean exhaustion
— it can never mean "this stretch matched nothing", which is exactly why the residual predicate had
to go), or the ceiling. Round trips: **at most 10** for `album_artist`, **at most 11** for `artist`,
which also runs one EF query for its entity/music-video half.
`SearchFieldValuesQueryShapeTests` pins the SQL string exactly and asserts the absence of `LIKE`,
`LOWER` and `IS NOT NULL`, so a reviewer reintroducing "just a cheap filter" fails a test instead of
silently unbounding the walk.
### Exactly what is bounded — and what is NOT
State this precisely, because an earlier version of this record claimed more and the overclaim is
more dangerous than the code ever was. What holds:
- **at most `ListValuedMaxRowsRead` logical rows returned and materialized per request**, and
- **at most 10 (or 11) round trips.**
That is the whole guarantee. It is what makes the walk terminate and what caps the number of rows and
round trips. **Explicitly retracted**, having been asserted here in earlier revisions:
- ~~"`LIMIT n` reads exactly `n` index entries"~~**false on MySQL.** Deleted clustered-index
records survive until purge runs, and a range scan still traverses them. Hold an old InnoDB
snapshot open, delete a million early `SongMetadata` rows, and query from a newer snapshot with
purge blocked: returning 2,000 *visible* rows can touch far more index records. **Deletion history
therefore still affects physical work** — the very thing attempt 4's failure was supposed to have
made irrelevant. Attempt 5 fixes the *logical* dependence on `Id` distribution; it does not make
physical work independent of deletion history.
- ~~bounded physical work / bounded I/O~~ — row width is unbounded. `Artists`/`AlbumArtists` are
unrestricted `TEXT`/`longtext`, and both SQLite and InnoDB spill large payloads to overflow pages,
so a row count implies neither a byte count nor a page-read count.
- ~~"caps what this process holds in memory"~~ — the same overclaim one level down, and it survived
the first retraction. A row count bounds neither bytes buffered nor set size: payload width is
unrestricted, and one JSON array can contain arbitrarily many strings, every one of which may enter
the in-memory distinct set.
Nor can the query-shape test carry more than it does: it pins the SQL **string**. It cannot pin an
execution plan, MVCC visibility work, or payload I/O — and on MySQL, using the index to satisfy
`ORDER BY` is an optimizer choice, not a SQL semantic.
### The cost, measured
No server-side narrowing means rows are transferred that will be discarded. **This is ONE data point
on ONE library, not a general figure** — see the row-width caveat above: these numbers hold for a
library whose artist credits average ~20 bytes of JSON, and a library with long credit lists would
transfer proportionally more for the same row count. Measured on a seeded 20,000-song library
(in-memory SQLite, so the wall times are a floor, not a production figure):
| case | rows read | round trips | payload | wall |
|---|---|---|---|---|
| worst case — no match, full walk | 20,000 | 10 | **391.9 KiB** (avg 20.1 B/row) | 119 ms SQL / ~40 ms warm end-to-end |
| empty `q` (fills `limit` on page 1) | 2,000 | 1 | ~39 KiB | ~60 ms |
| dense prefix (`rad`) | 2,000 | 1 | ~39 KiB | ~38 ms |
| non-ASCII prefix (`beyoncé`) | 2,000 | 1 | ~39 KiB | ~39 ms |
Judged acceptable **for this shape of library**: the worst case is a debounced typeahead keystroke
that matches nothing, at ~392 KiB and tens of milliseconds against a local SQLite file. Dense queries
— including the empty `q` the combobox opens with — stop on the first page. Re-measure rather than
extrapolate if credit lists are long or the provider is MySQL over a network. **If it ever becomes
unacceptable, do not reintroduce selectivity;** that is the trap this record exists to document. Go
to #669.
### Accepted losses
A match past row 20,000 is not found — 20,000 filler rows then `"éclair"` at 20,001 returns `[]`, and
a test pins exactly that rather than pretending otherwise. That is the documented bounded-best-effort
contract, and unlike attempts 14 it now depends only on row count, not on prefix shape, deletion
history or `Id` distribution.
**Follow-up: a normalized `SongArtist` join table** (the shape `MusicVideoArtist` already has) makes
the predicate seekable, so there is nothing left to bound and nothing to transfer. Cost: a
dual-provider schema migration plus data backfill, changes to every scanner write path populating
`SongMetadata.Artists`, changes to the Lucene indexer, and two representations of the same fact free
to drift. Tracked as **ersatztv#669**; rejected for #578 on blast radius, not on merit.
## `album_artist` 404 → 200 is additive
Nothing consumes the 404 as a signal: the SPA's `getSearchFieldValues` (`web/src/api/search.ts`)
treats any non-200 as "no suggestions, fall back to a free-text input", which it will now do less
often. Per `api.versioning-v1`, widening which fields return values adds capability without removing
any, so no `/api/v2`.
## Known limitation inherited, not introduced
**RESOLVED — see `api.search-field-values-unicode-fold` (ersatztv#668, 2026-07-27).** As written for
#578 this said: the **EF-sourced** fields (`genre`, `studio`, `artist`'s entity half, …) still
prefix-match through SQL `LOWER()`, which on SQLite is ASCII-only, so a stored `Édith` was unreachable
for those fields. That predated #578 and was unchanged by it. It is now fixed — and NOT by the
client-side filtering this section anticipated, which would have reintroduced the very scan #578 bounded.
The surrounding scoped-ordinal wording is still load-bearing and must not be "tidied" into a broader
claim: the EF stage's own behaviour is unchanged, and the defect was SQLite-only and one-sided.
@@ -0,0 +1,72 @@
---
key: api.search-field-values-unicode-fold
title: '2026-07-27 — Facet-value typeahead reaches accented values: a registered Unicode fold on the SQLite non-ASCII branch, not a bounded walk (#668)'
status: active
since: '2026-07-27'
supersedes: none
superseded-by: none
rule: 'The EF-sourced facet fields (`genre`, `show_genre`, `studio`, `director`, `writer`, `actor`, `tag`, `network`, `collection`, `video_codec`, `album`, and `artist`''s entity half) reach stored values whose prefix carries an uppercase non-ASCII character, on BOTH providers, with no row budget and no accepted loss. The defect was SQLite-only and ONE-SIDED: SQLite''s `LOWER()` folds ASCII only (`lower(''Édith'')` is `''Édith''` unchanged), so the predicate UNDER-matched, which no later stage can repair. MySQL was already correct — its `LOWER()` is Unicode-aware, so `LOWER(''Édith'')` really is `''édith''` and the existing predicate reaches the row unaided. The fix is a SECOND, ADDITIVE query taken only when `isSqlite && q contains a non-ASCII character`: raw Dapper SQL `SELECT DISTINCT <col> AS Value FROM <table> WHERE [<discriminator> AND] etv_upper(<col>) LIKE @Pattern ESCAPE ''\'' ORDER BY <col> LIMIT @Limit`, where `etv_upper` is a `SqliteConnection.CreateFunction` scalar implementing `ToUpperInvariant`. Every other case — all-ASCII `q`, and MySQL for all `q` — runs today''s EF query BYTE-IDENTICALLY. Keeping selectivity in SQL here is NOT the refuted family from `api.search-field-values-sources`: those four attempts bounded a walk around a predicate that could not be made correct over JSON escape text, whereas this is a correct fold on a plain column in an ordinary `LIMIT`ed query. It narrows that record''s "Known limitation inherited, not introduced" clause; everything else it settles still holds.'
signals: 'accented facet values missing, Édith not suggested, SQLite LOWER is ASCII only, etv_upper, CreateFunction custom scalar, ToUpperInvariant fold, OrdinalIgnoreCase is not invariant-upper, U+017F long s upper-folds to S, U+212A Kelvin sign, utf8mb4_0900_ai_ci accent insensitive, MySQL LOWER is unicode aware, over-match harmless under-match not, ESCAPE clause raw SQL LIKE wildcards, EF null semantics ExternalTypeId, RegisterUnicodeCaseFunctions provider static, non-sargable LOWER LIKE full table scan · paths: `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `ErsatzTV.Infrastructure.Sqlite/Data/SqliteUnicodeFunctions.cs`, `ErsatzTV.Infrastructure/Data/TvContext.cs`, `ErsatzTV/Startup.cs`, `ErsatzTV.Scanner/Program.cs` · issues: #668, #578, #434, #669'
mechanics: '`GetSearchFieldValuesHandler` (`ContainsNonAscii`, `IsSqlite`, `EscapeLikePrefix`, `UnicodeFoldSql`, `GetUnicodeFoldSources`, `GetUnicodeFoldedValues`, `UpperFunction`); `SqliteUnicodeFunctions.Register`; `TvContext.RegisterUnicodeCaseFunctions`; `GetSearchFieldValuesHandlerTests.Unicode_Fold_Agrees_With_The_Ordinal_Filter`; `SearchFieldValuesQueryShapeTests.Unicode_Fold_Function_Name_Matches_The_Registration`; `ProviderStaticsWiringTests`'
---
Narrows `api.search-field-values-sources` (#578), which deferred this gap; the rest of #578 stands.
## The defect was one-sided, and the issue described it wrongly
The handler lowercases `q` with `ToLowerInvariant` **before** SQL, so both casings produce one pattern.
A stored **lowercase** accented value was therefore always reachable from either casing; only one whose
prefix carries an **uppercase** non-ASCII character was lost. ersatztv#668's body claimed `q=É` failed
against a stored `édith`; false, and a test pins the passing case beside the fixed one.
## MySQL was never broken, for a reason worth recording
Verified on a live MySQL 8.4: `LOWER('Édith')` is `édith`, so the existing predicate reaches the row.
**Measure the query the CODE runs, not one you type.** With a LITERAL pattern `LOWER(name) LIKE 'é%'`
also matches `Edith` (the column is accent-insensitive `utf8mb4_0900_ai_ci`), and an earlier revision of
this record concluded from exactly that probe that MySQL over-matches and the ordinal filter corrects it.
It does not: through EF the driver binds the pattern with a BINARY collation, so the executed comparison
is accent-SENSITIVE and returns `Édith` alone — a driver-contingent fact, not a law. MySQL's correctness
rests on Unicode-aware `LOWER()`, not on the collation.
## Why a fold, and not the #578 walk
Reusing #578's shape — drop SQL selectivity, keyset-walk, filter in memory — answers the wrong question.
That walk is best-effort at 20,000 rows; `Genre` and `Actor` carry one row per media item, so a large
library exceeds the budget and `Édith` stays unreachable — the bug restated. #578 accepts that contract
for `SongMetadata.Artists` because server-side projection is **impossible** there; these are plain
columns, where it is merely inconvenient.
The cost objection to a managed per-row fold is weak: `LOWER(v) LIKE` is non-sargable and **no index on
any of these `Name` columns exists** (every index is on the foreign key), so this swaps a native per-row
call for a managed one on a scan that already happens — and only on the non-ASCII branch.
## The correctness property is containment, not equality
The SQL stage may over-match freely; it must never under-match. `ToUpperInvariant` satisfies that
because **`OrdinalIgnoreCase` equality is a strict subset of invariant-uppercase equality**.
Do not restate this as "`OrdinalIgnoreCase` IS invariant-uppercase-then-ordinal". It is not, and the gap
is measurable: `char.ToUpperInvariant('ſ')` (U+017F) is `'S'`, yet
`"ſweet".StartsWith("S", OrdinalIgnoreCase)` is **false**. The fold returns that row and the filter drops
it — the harmless direction. An earlier draft justified the fold by claiming the opposite;
`Fold_LongS_IsNotOrdinalEqualToS` pins the truth.
That same fact makes the all-ASCII fast path sound: no non-ASCII codepoint is `OrdinalIgnoreCase`-equal to printable ASCII (#578's sweep found 0), so an ASCII query only ever ordinal-matches an ASCII prefix.
## Three traps, each guarded by a test and explained at its call site
Raw SQL gets none of EF's LIKE escaping (`EscapeLikePrefix`, backslash first, explicit `ESCAPE`).
Discriminators must mirror EF's NULL semantics — `t.ExternalTypeId != X` INCLUDES a NULL-typed row,
where plain SQL `<>` drops it. Registration is per-connection and lives at the call site, not in a
`DbConnectionInterceptor`: Dapper opens a closed connection itself and a direct ADO open raises no EF
interceptor, so that seam would miss exactly this query.
## Residuals, stated rather than glossed
**Crowding**: a SQL `LIMIT` can fill with rows the ordinal filter then discards, under-DELIVERING the
count (never a wrong value). Not reachable on MySQL under the CURRENT driver behaviour above (a ci-collated
pattern would restore it); the SQLite fold has it when limit-many values are upper-equal but ordinal-unequal (
`ſ`/`K`/`İ` class), so "no accepted loss" means no unreachable VALUE, not a guaranteed count. An
over-fetch was rejected (it perturbs the pinned `"apple"`/`"Zulu"` examples). **Ordering stays
best-effort** per #578.
@@ -0,0 +1,71 @@
---
key: api.selection-projection-include-chain
title: '2026-07-28 — A tagged-union selection is projected through one shared include chain, and its flattening switch never falls through to null (#671)'
status: active
since: '2026-07-28'
supersedes: none
superseded-by: none
rule: 'Every handler that projects an aggregate carrying a tagged-union selection loads it through ONE shared `<Aggregate>QueryExtensions` include chain — `RerunCollectionQueryExtensions.IncludeSelectionDetails()`, joining the existing `ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails()` — called by the paged-list handler and the by-id handler alike, so the two cannot drift. The media-item flattening switch is likewise ONE shared helper, `MediaCollections.Mapper.ProjectMediaItemToViewModel`, covering all ten selectable media types including `RemoteStream`, whose named projection is `MediaItems.Mapper.ProjectToNamedViewModel` (it cannot be an overload of `ProjectToViewModel(RemoteStream)`, which already exists returning the unrelated `RemoteStreamViewModel`; C# will not overload on return type). That switch NEVER ends in `_ => null`: a null MediaItem is the legitimate not-a-media-item case, while an unrecognized non-null subtype keeps its id and takes a conspicuous `[unsupported media type: X]` name. Fail-soft is deliberate — throwing would fail an entire paged GET over one unreadable row. Finally, every metadata navigation inside `MediaItems.Mapper` is read through `Optional(...).Flatten()` and degrades to the `"???"` placeholder, because those projections are reached from handlers whose include chains differ and a bare `x.Season.Show.ShowMetadata` is a latent 500 on some other caller GET.'
signals: 'rerun collection null selection, selectedId null for every row, list badge renders Collection with no name, detail GET 500 on Episode, detail GET 500 on MusicVideo, RemoteStream dropped by the mapper, underscore arrow null fallthrough, AsNoTracking suppresses navigation fixup, eager load missing on paged list, Include after Skip Take, EpisodeTitle NullReferenceException, MusicVideoTitle bare Artist deref, ShowTitle bare Show deref, id only as available as the name, editor silently clears stored selection, ArgumentNullException value cannot be null parameter values, string.Join on null sequence, SongMetadata Artists is null, nullable primitive collection not a navigation, untagged song fallback metadata, playout guide 500 on a song, song artist prefix bare dash, chaptered song renders ErsatzTV.Core.Domain.Song, GetDisplayTitle interpolates the entity not the title · paths: `ErsatzTV.Application/MediaCollections/RerunCollectionQueryExtensions.cs`, `ErsatzTV.Application/MediaCollections/Mapper.cs`, `ErsatzTV.Application/MediaItems/Mapper.cs`, `ErsatzTV.Application/MediaCollections/Queries/GetPagedRerunCollectionsHandler.cs`, `ErsatzTV.Application/MediaCollections/Queries/GetRerunCollectionByIdHandler.cs`, `docs/api-conventions.md` §2a · issues: #671, #651, #229'
mechanics: '`RerunCollectionQueryExtensions.IncludeSelectionDetails`; `Mapper.ProjectMediaItemToViewModel`; `MediaItems.Mapper.ProjectToNamedViewModel`; `SelectionSeedData` (`SupportedSelectionTypes`, `ExpectedName`, `SeedSelection`, `ApplySelection`); `RerunCollectionQueryHandlerTests` (`GetById_Should_Resolve_The_Selection`, `GetPaged_Should_Resolve_The_Selection`, `Supported_Selection_Types_Should_Be_The_Full_Documented_Set`, `GetById_Should_Tolerate_Song_Artists`); `GetPlaylistItemsHandlerTests`; `Playouts.Mapper.GetDisplayTitle` + `PlayoutMapperDisplayTitleTests`; `RerunCollectionRequestMapping.IsSupportedSelectionType`'
---
Applies the `#229` shared-include-chain remedy to the READ path — that record framed it as a
write-path concern; this is its mirror image, where the GET itself under-loaded.
## The coupling that hid the bug
The id and the display name are read off the SAME navigation, so the id is only ever as available as
the name — the API never knows WHICH item is selected but not what it is called. Hence the symptom
looked like a naming problem (an unlabelled badge) when the real harm is one level down: the selected
id is null too, and an editor that round-trips it clears the stored selection. #651's client-side
merge-instead-of-replace guard made this survivable and stays, but patched a server defect from the
client. The rule: never let the id and the name share a single point of failure — hence the
`_ => null` ban, where an unrecognized subtype surrenders its NAME, never its ID. Fail-soft, not a
throw, which would fail a whole paged GET over one bad row. (`ProgramSchedules.Mapper`'s switch does
throw, correctly — it dispatches on the ITEM type, an internal closed set.)
## Scope deliberately not widened
Nine further media-item switches (`ProgramSchedules.Mapper` ×4, `Scheduling.Mapper` ×5) handle only
Show/Season/Artist — not the same oversight, since those call sites genuinely restrict selection to
those three and load a matching chain. Only RerunCollection and PlaylistItem span the full set, so
exactly those two were merged. A THIRD consumer, `ReplacePlaylistItemsHandler`, projects items whose
navigations are never loaded — inert only because the controller discards the result and re-queries.
**Widening a shared switch incurs a debt in every caller loading for it**, discharged by a TEST, not
by inspection — inspection is the method that produced this bug. `GetPlaylistItemsHandler` had no
handler-level test at all (its controller tests stub the mediator), so it gained the same 13-type
matrix via the shared `SelectionSeedData`.
## `Artists` is a nullable PRIMITIVE COLLECTION, and the sweep must follow the field
`SongMetadata.Artists` is a nullable EF primitive collection — a JSON array in one column, **not a
navigation** — left unassigned by `FallbackMetadataProvider` when a song's tags fail to read, and
`string.Join` throws `ArgumentNullException`, not `NullReferenceException`. So a "null navigation"
audit misses it and so does a grep for `NullReferenceException`. The guard is
`Optional(sm.Artists).Flatten()`, empty filtered too so an artist-less song loses its bare `" - "`.
Two corrections, because a wrong explanation outlives a wrong line. It was **not** introduced here:
`GetPlaylistItemsHandler` already included `SongMetadata` on `origin/main` and already routed `Song`,
so `GET /api/v1/playlists/{id}/items` was ALREADY a live 500 — this branch only made the same throw
reachable on a second path. And fixing the rerun site alone left the mirror standing: `Playouts/Mapper`
had the identical unguarded join on a path that also eager-loads `SongMetadata`, likewise live, swept
here. `LibraryBrowseItemMapper` already wrote `Artists ?? []`, so the codebase knew. Filed separately:
`SongVideoGenerator` dereferences `Artists.Count`/`.Contains` on the playback path. Sweep by FIELD.
Adjacent, same review, fixed here: that Song arm interpolated the `case Song s` ENTITY into its
chapter branch, rendering a chaptered song as the literal `ErsatzTV.Core.Domain.Song (Chapter 3)`.
## Verification worth repeating
Every mechanism was removed in turn and quoted red before restoring it: stripping the list include
chain failed all 13 types on "lost its selected id"; the original four-type by-id chain failed exactly
the six the issue named; reverting the bare dereferences reproduced `NullReferenceException` for
Episode and MusicVideo; reverting either `Artists` guard reproduced `ArgumentNullException`; and
reverting the chapter fix rendered the type name. A green test proves little until shown to fail.
The per-type assertion pins the WHOLE expected string, not merely "is not a placeholder", because the
looser form cannot see a missing NESTED leg: drop Episode → Season → Show and the projection still
reads `s00e04 - Selected episode`, placeholder-free, and passes. Relatedly an absent Season renders
`s??`, never `s00`, which means Specials and would fabricate plausible-looking real data.
@@ -0,0 +1,82 @@
---
key: ci.exemption-provenance
title: '2026-07-29 — the `review-verdict/h10` exemption path binds the base ref, constrains the bot exemption by CONTENT, and re-derives any success it cannot attribute to a human (#698)'
status: active
since: '2026-07-29'
supersedes: none
superseded-by: none
rule: 'The three inputs the exemption decision rests on must each be bound to something the judged PR cannot mutate. (1) BASE — `scripts/pr-changed-files.sh` takes the expected base BRANCH as a REQUIRED 5th argument and re-reads it before and after paging, because `/pulls/{n}/files` diffs against the PR''s live base and retargeting moves the answer without moving the head sha; the workflow passes `github.event.pull_request.base.ref` from the `pull_request_target` payload, which a retarget cannot rewrite. (2) BOT EXEMPTION — an author match is necessary but never sufficient: `pull_request.user.login` is the PR''s immutable CREATOR while its head is not, so the exemption additionally requires EVERY changed path to be a dependency manifest (`Directory.Packages.props` or `.config/dotnet-tools.json`, and ONLY those — the npm manifests are excluded because `package.json` `scripts` are executed by CI). (3) INHERITED SUCCESS — the never-overwrite short-circuit fires only for a status POSITIVELY identified as a human verdict for THIS base, meaning a non-null `.creator.login` AND a `Review-verdict:` description AND, when that description records a base (`(base: …)`, `release.verdict-status-check`), a base matching the PR''s — tested by requiring the description to END with the exact literal `(base: <base>)` and to contain exactly ONE such marker, never by extracting a value (see below); a present-but-different base is rejected, an absent one is not, since verdicts predating that convention carry none; every other shape, including any unrecognised one, is re-derived rather than trusted. The bot and docs-only exemptions are evaluated as INDEPENDENT predicates and the decision made afterwards, never as an `elif` chain. `edited` is in the workflow''s `types:` so a retarget reclassifies — which gives DETECTION, not atomicity: status writes are not serialized, so a stale run can still post over a fresher one (residual, #706). Path predicates are evaluated by COUNTING with `grep -c`, never `| grep -q` (SIGPIPE inversion) and never a here-string (temp-space failure) — see `ci.grep-q-pipefail-inversion`.'
signals: 'forged review-verdict exemption, retarget race against the docs-only classifier, PR base changed mid-run, hijacked Renovate branch, bot exemption on a code change, machine-written success inherited as a verdict, status creator null vs user, never overwrite a human verdict, exemption chain skips docs-only for bots, why is my Renovate PR asking for a verdict, base ref binding on pr-changed-files.sh · paths: `.gitea/workflows/review-verdict.yml`, `scripts/pr-changed-files.sh`, `.claude/hooks/pretooluse-merge-consent.sh`, `scripts/tests/test_pr_changed_files.py` · issues: #698, #697, #672, #663, #649, #632'
mechanics: '`scripts/pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha> <expected-base-ref>` (5 args; a 4-arg call exits 2); workflow env `BASE_REF: ${{ github.event.pull_request.base.ref }}`; `BOT_MANIFESTS` anchored allow-list; short-circuit requires `.creator.login` non-null AND description matching `^Review-verdict:`; `types: [opened, reopened, synchronize, ready_for_review, edited]`'
---
`ci.gate-trigger-base-resolved` stopped a PR supplying the gate's own *definition*. This closes the
layer below: the exemption path still **decided from mutable or unattributed PR state**, and a
machine-written `success` was never revalidated. Three routes, one root cause, one fix.
**Route 1 was reproduced, not theorised** (probe PR #703). A head `H` and scratch base `S` chosen so
`H` vs `S` is docs-only; opened `H → main` so the trusted base definition ran; retargeted to `S`
mid-flight. The job enumerated against the moved base, read docs-only, and posted
`review-verdict/h10=success — "Exempt: docs-only change"`. Retargeted back to `main`: **nothing
reclassified** (`created_at == updated_at`), leaving a PR into `main` whose diff carried a C# file
behind a green required check. Closed unmerged, branches deleted, no forged `h10` left anywhere.
**Why a base BINDING and not a pinned diff.** Diffing two immutable shas would close it outright;
Gitea 1.25.4 cannot serve that — measured: `compare/{base}...{head}` returns no `files`, and a
`--depth=1` fetch of the two shas has no merge base, so three-dot is impossible and two-dot
over-reports everything `main` gained since the branch point. So the base is read before the first page
and after the last, and **the gap is stated plainly**: a retarget opening *and* closing strictly
between the files call and the re-read stays invisible from inside the enumeration.
**An earlier draft claimed `edited` made that residual non-durable. It does not, and cross-family
review was right to call it a Blocker.** `edited` gives DETECTION, not atomicity or ordering: runs are
not serialized, so the stale run can post `success` AFTER the reclassifying run posts `pending`, and an
already-scheduled merge can fire in the green window between them. The `main → scratch → main` ABA
transition is therefore NARROWED and observable, not closed. Tracked as an explicit residual rather
than described as fixed. `edited` and re-derivation remain one fix — `edited` alone re-runs and exits
on the existing `success`; re-derivation alone never gets a second run — but together they are
mitigation, not a guarantee.
**Route 2 — a bot ACCOUNT does not attribute the CODE.** `pull_request.user.login` is the PR's
immutable *creator*; its head is not. Push application code onto an open Renovate branch and the PR is
still "authored by renovate", touches no protected path, and was exempted. Checking the *pusher* fixes
nothing — a git author is self-asserted text. So the exemption is constrained by what a bump can
legitimately *be*: across all 11 Renovate PRs this repo has had, the paths touched were
`Directory.Packages.props` (10) and `.config/dotnet-tools.json` (1) — and ONLY those. An earlier draft
also exempted `web/package.json`/`web/package-lock.json` "so a first SPA bump cannot deadlock"; review
called that a Blocker and was right. `renovate.json` enables only nuget/github-actions/dockerfile, so
npm is unmanaged here and the entry bought nothing, while `package.json` `scripts` are EXECUTED by CI
(`npm ci`, `npm run build`) — widening an exemption onto a code-execution path for no benefit. `*.csproj` is excluded — under Central Package Management
versions live in `Directory.Packages.props`, so a Renovate `.csproj` edit is anomalous by
construction. Cost stated: such a PR is not blocked, it needs a real verdict. The two exemptions are
evaluated as INDEPENDENT predicates: written as an `elif` chain, a Renovate PR touching only `docs/`
entered the bot branch, failed the manifest test, and never reached the docs-only branch.
**A counterfactual, not an incident.** Renovate PR #20 touched a `.csproj` and two C# files but has no
`h10` status: it merged 2026-06-27, the gate landed 2026-07-25. The point is what identity-only *would*
have done. An earlier draft claimed it HAD been exempted — wrong, and the correction is kept because
"was silently exempted" and "would have been" are different claims.
**Route 3 — provenance, and the direction of the test.** The short-circuit exited on any `success`, so
an exemption this job wrote was indistinguishable from a human verdict; obtained once, a forgery was
accepted on every later run. It could not simply be deleted — it exists so `pending` cannot un-approve
a reviewed head. Measured on the **combined** endpoint: a status POSTed with a user
credential carries `.creator.login`, one POSTed by an Actions job carries `"creator": null`. The test is
written in the **positive** direction — short-circuit only on something identified as human — because
spelled the other way ("skip if it looks machine-written") any unrecognised shape falls through to
*trusted*. Both halves are required, so if Gitea later populates `creator` for Actions the description
test still fails: the guard degrades toward re-deriving, never toward trusting.
**What this does NOT close.** Anyone who can POST statuses directly can write both a creator and a
`Review-verdict:` description and impersonate a verdict; branch protection binds the *context*, not its
issuer. A provenance check, not an authentication one — that is `#697`, left open because its durable
fix is credential scoping, partly server-management territory. Severity as `#672`: requires push
access, so the threat model is a compromised contributor.
**Verification honesty.** Route 1 was reproduced live; the "and now it fails" half cannot be shown from
a PR, because `pull_request_target` resolves this definition from `main` — the self-test gap
`ci.gate-trigger-base-resolved` records. Pre-merge evidence is that reproduction plus the
executed-behaviour tests in `scripts/tests/test_pr_changed_files.py`, each verified by mutation; the
live re-check happens on `main` right after merge.
**Separate defect found reviewing this change:** `ci.grep-q-pipefail-inversion` — a pre-existing
SIGPIPE inversion that let a large PR skip the `PROTECTED` guard entirely. Fixed in the same PR.
@@ -0,0 +1,71 @@
---
key: ci.gate-trigger-base-resolved
title: '2026-07-28 — `review-verdict.yml` triggers on `pull_request_target` scoped to `branches: [main]`, so the PR under judgment cannot supply the gate''s own definition (#672)'
status: active
since: '2026-07-28'
supersedes: none
superseded-by: none
rule: 'The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR''s own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch''s gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree''s scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class: Gitea injects a write-capable `GITEA_TOKEN` into EVERY job, so any ref-resolved workflow — and a collaborator''s own API token, since branch protection binds the context and not its issuer — can still forge `review-verdict/h10`. Tracked in #697; the exemption path has its own separate defects in #698.'
signals: 'workflow definition resolved from head, PR rewrites the gate that judges it, self-approve a required status check, pull_request_target vs pull_request, gate trigger branches filter, attacker-supplied base branch, how to test a change to review-verdict.yml, workflow not exercised by its own PR, gate edit goes live only on merge, required_approvals 0 does not bind an author, forged commit status inherited by sha · paths: `.gitea/workflows/review-verdict.yml`, `scripts/tests/test_pr_changed_files.py` · issues: #672, #663, #649, #622'
mechanics: '`on: pull_request_target: {branches: [main], types: [opened, reopened, synchronize, ready_for_review, edited]}` (`edited` added by `ci.exemption-provenance` so a retarget reclassifies); asserted by `test_the_workflow_trigger_is_pull_request_TARGET_scoped_to_main` in `scripts/tests/test_pr_changed_files.py`; the job''s own context is renamed to `... (pull_request_target)` and must stay OUT of branch protection''s required list'
---
`ci.shared-pr-file-enumeration` had this job check out the PR's **base** ref so the PR cannot supply
the *scripts* that judge it — real but partial, as that record said: it does not bind the job
**definition**. This closes that half.
**What was actually wrong.** Gitea, like GitHub, resolves a `pull_request` workflow definition from
the PR's own head, so a PR editing `review-verdict.yml` ran its own rewritten copy — which could
delete the base checkout or skip straight to posting `review-verdict/h10=success` for its head sha.
Two things that look preventive were not: `PROTECTED` is defined by the same rewritten file, and
branch protection requires the *context*, not an author, while carrying `required_approvals: 0`.
**Measured, not inferred.** The premise is a claim about someone else's software, so it was settled
on this instance (Gitea 1.25.4) with **four** scratch PRs, not by analogy to GitHub: `pull_request` ran
the head's rewrite and never wrote the real `h10`; `pull_request_target` ignored the identical rewrite
and the base definition posted `h10=pending` on `opened` and `synchronize` alike, secrets available;
`branches: [main]` produced no run at all from a non-`main` base; and the fourth — the negative one
establishing the residual below — is counted because omitting it turns an honest partial into an
overclaim. Probes posted only probe-named contexts, never a forged `h10`. Full results in #699.
**Why `branches: [main]` is load-bearing rather than tidy.** The *base branch* supplies the
definition, and anyone who can push a branch can make it a base — so dropping the filter trades a
head-supplied gate for a base-supplied one and closes nothing. Worse than lateral: a commit status is
repo-global per sha (`#663`), so a `success` forged against a scratch base is **inherited** by a later
genuine PR into `main` with the same head.
**Why `pull_request_target` is not the footgun it usually is.** Its standard danger is executing
untrusted head code with a privileged token; this job executes none, checking out `base.sha` with
`persist-credentials: false` and running only that tree's scripts. Trigger and checkout are one
decision — under this trigger a head checkout would be strictly worse than #672 was.
**Options not taken.** `required_approvals: 1`, the cheapest mechanical fix, is unusable here: Gitea
forbids approving your own PR and this is effectively a single-maintainer repo, so it deadlocks every
PR instead of gating the dangerous ones. Verifying the status *author* needs an actor the PR cannot
control, and the tampered workflow holds the same `GITEA_TOKEN`.
**Severity, stated plainly.** Never remotely exploitable — pushing a branch requires write access, so
the threat model is a compromised contributor, who has other paths. Fixed because a gate whose
authority the judged thing can assert is not a gate, not because an attack was expected.
**The class is NOT closed, and this record must not be read as claiming otherwise.** This fixed one
instance of "a ref-resolved workflow can obtain credentials that POST a commit status", and that
inventory is not a short list: Gitea injects `GITEA_TOKEN` into **every** job, defaulting to
read/**write**, so head-resolved, `push`-triggered and `workflow_dispatch` workflows alike are routes
(1.24+ loads a dispatched definition from the selected branch). A collaborator's own API token is a
route with no workflow at all — branch protection binds the *context*, not its issuer. Full inventory
in `#697`; the exemption path's own defects are `#698`. No in-repository test can establish
status-authority isolation: the sibling guard added here catches only plain-text naming of the
context.
**The gate is no longer exercised by its own PR** — base resolution cuts both ways, so an edit here
goes live only on merge, repo-wide, untested. Verify one safely per `docs/ci-cd.md` → Review-verdict gate.
**Residual.** The job's own context is renamed to `... (pull_request_target)`, safe only because it
was never one of branch protection's required contexts (the two `docker-build.yml` contexts plus
`review-verdict/h10`); adding it would let the workflow satisfy the gate by merely running. **A trap
for #697:** those two carry the literal `(pull_request)` suffix, so giving `docker-build.yml` the same
treatment renames them and deadlocks merges unless branch protection is edited in the same operation.
A non-`main` base now yields no status where it previously got one — fail-closed, removing a `#663`
hazard. The `edited` gap this section once recorded as a mere inconvenience ("statusless until its next
`synchronize`") was the persistence half of a live forgery; RESOLVED in `ci.exemption-provenance` (#698).
@@ -0,0 +1,59 @@
---
key: ci.grep-q-pipefail-inversion
title: '2026-07-29 — never feed `grep -q` from a pipe under `set -o pipefail`: SIGPIPE turns a MATCH into a failed pipeline and inverts the guard (#698)'
status: active
since: '2026-07-29'
supersedes: none
superseded-by: none
rule: 'In any script running under `set -o pipefail`, a security or classification predicate of the form `producer | grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf ''%s\n'' "$data" | grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep''s status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`.'
signals: 'grep -q pipefail, exit 141, SIGPIPE in a shell guard, large PR classified docs-only, protected path guard skipped, printf pipe grep -q, classification inverts on big input, pipe buffer 64K shell predicate · paths: `.gitea/workflows/review-verdict.yml`, `.claude/hooks/pretooluse-merge-consent.sh`, `scripts/tests/test_pr_changed_files.py` · issues: #698, #649'
mechanics: '`count_matching` / `count_not_matching` helpers in `review-verdict.yml`, DEFINED BEFORE FIRST USE, results precomputed into `n_protected`/`n_not_manifest`/`n_not_docs` at top level and validated numeric; regression tests `test_a_LARGE_pr_*` build 1900+ paths (~171KB) to cross the pipe buffer, `test_the_classify_step_runs_without_SHELL_ERRORS` catches a dead guard, `test_a_protected_path_is_rejected_BY_THE_PROTECTED_BRANCH` asserts the branch rather than the outcome'
---
Found by cross-family adversarial review of the #698 PR, then reproduced directly. It is not a defect
of that change — the construct shipped with the ORIGINAL docs-only check and was live on `main`.
**The mechanism.** `grep -q` stops reading at its first match. If the writer still has data queued, it
gets SIGPIPE and exits 141. `pipefail` makes the pipeline's status that of the failing member, so a
successful MATCH is reported as a FAILED pipeline. Every guard written as `if ! producer | grep -q…`
or `if producer | grep -q…` therefore flips, and only for large inputs — which is why every existing
test missed it: they all used a handful of short paths, far below the buffer.
**Both directions were reproduced**, at 1901 paths / ~171KB, inside the enumerator's 2000-file cap:
- `A.cs` followed by 1900 docs paths → `docs_only=yes`, pipeline status 141. A C# PR classified
docs-only and auto-granted `review-verdict/h10=success`.
- `.gitea/workflows/review-verdict.yml` followed by 1900 docs paths → the `PROTECTED` guard MISSES, so
a PR editing the gate's own definition skips the protected-path check and falls through to the
exemptions.
The second is the serious one: it reaches a green required status on a PR that rewrites the gate, with
**no retarget, no bot account and no forged status** — nothing but a large PR. It is a more direct hole
than any of the three routes #698 was filed for.
**The here-string was the FIRST fix, and it was wrong.** `grep -q… <<< "$data"` does remove the
SIGPIPE, and it shipped for one round. But bash materialises a large here-string through temporary
storage, so it fails when temp space is full or unwritable — and because these predicates sit inside
`if`/`!`, that failure flips them exactly as SIGPIPE did. It did not reproduce on macOS bash 3.2 and
did on Linux bash 5.x, which is the environment CI runs; the disagreement is itself the argument for a
construct that cannot fail either way. Counting with `grep -c` uses an ordinary pipe and drains stdin,
so neither failure mode exists.
**Two follow-on traps, both found only by re-review.** First, the helpers were defined AFTER the
classification chain that called them, so `count_matching` was `command not found` on every run and the
`PROTECTED` branch never fired — while three "protected path" tests stayed green, because a protected
path is also not a manifest and not docs-only, so the job reached `pending` down another route. Second,
`exit 1` inside those helpers only left the command-substitution SUBSHELL, and since the substitution
sat in a conditional, `set -e` never fired either. Hence the rule: define before use, evaluate once at
top level, validate the result is numeric, and fail closed when it is not.
**Two testing lessons.** When several branches produce the SAME outcome, asserting the outcome cannot
tell you which branch ran — assert the discriminator (here the `Decision:` reason line). And a cheap
stderr sweep for `command not found` / `integer expression expected` / `unbound variable` catches a
whole family of silently-skipped guards, because each of those makes an `if` condition merely false
while the job exits 0 and posts a plausible status.
**The input-size lesson.** The whole class was invisible because every test used small inputs. A guard whose behaviour depends on a BUFFER THRESHOLD needs a test that crosses
the threshold; otherwise the suite is measuring the wrong regime entirely and full coverage of the
small regime proves nothing. The regression tests pair each large-input negative with a large-input
POSITIVE control, so "large lists now fail closed" (a merge deadlock) cannot masquerade as a fix.
@@ -0,0 +1,103 @@
---
key: ci.jq-version-contract
title: '2026-07-26 — jq 1.6 is the FLOOR every shell gate must run on; `scripts/jq-preflight.sh` makes the version observable, and only `script-tests` pins it (#648)'
status: active
since: '2026-07-26'
supersedes: none
superseded-by: none
rule: 'Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job''s log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock.'
signals: 'jq version divergence, jq 1.6 vs 1.8, jq -e exit code on empty input, contains NUL false positive, jq parse-error exit code collision, jq-preflight, script-tests --expect, review-verdict jq floor, merge deadlock from a pinned dependency · paths: `scripts/jq-preflight.sh`, `.gitea/workflows/review-verdict.yml`, `.gitea/workflows/pr-checks.yml`, `docs/ci-cd.md` · issues: #643, #647, #648, #649'
mechanics: '`scripts/jq-preflight.sh` (no args) -> floor+observability in every gate job; `scripts/jq-preflight.sh --expect 1.6` -> tripwire, `script-tests` job only; `docs/ci-cd.md` -> "The jq contract"'
---
Three independent jq-version divergences hit inside a single day (#643, #647), all in gates written
and tested on a developer Mac (jq 1.8.x) but running on the CI runner (jq 1.6):
- `jq -e` over EMPTY input exits 4 on jq >= 1.7, but **0** on jq 1.6 — the docs-only pagination guard
inferred "transport failure" from that exit status, so on 1.6 a failed page silently passed and the
loop walked past unread pages while still reporting `files_complete=yes`.
- `contains("\u0000")` — the NUL escape truncates to `""` on jq 1.6, so the containment test is
vacuously **true for every string**, not just ones actually containing a NUL. The H10
review-verdict classifier that relied on this was entirely inert on the runner.
- Parse-error exit code: `jq empty` exits 5 on jq >= 1.7 but **4** on jq 1.6 — the same code jq 1.6
uses for "no output produced". A garbage API response and an empty-but-valid one were
indistinguishable, and the garbage case was read as "no comments."
None of these are exotic jq usage — they are constructs anyone would reach for first, and each one
was discovered only because a real gate broke, not because anyone thought to test jq 1.6. That is the
argument for a *contract*, not three point fixes: the failures share one shape (a shell gate's
behavior is a function of an interpreter version nobody was treating as a variable), so patching each
construct as it's found does not converge — it just narrows the next surprise.
**Why 1.6 is the floor and not 1.8.** The runner is the binding constraint, not the author's machine.
Baking a pinned jq into `docker/ci/Dockerfile` was the obvious first idea and was rejected because it
provably cannot cover the gate that actually broke: `review-verdict.yml` is `runs-on: small` with no
toolchain-image pin, and per `ci.small-lane-git-only` the small lane is git-only — it gets the host's
jq 1.6 no matter what the toolchain image contains. That was checked against the running binary, not
assumed. So the fix has to hold at 1.6, in every gate, regardless of which lane it runs in.
**Why the pin is asymmetric.** `scripts/jq-preflight.sh` has two modes on purpose:
- No arguments — print the parsed version and fail only below the 1.6 floor. This is pure
observability: the jq version CI actually used is now in the job log, so a future divergence can be
diagnosed from the log alone instead of by guessing at the runner image. `review-verdict.yml` runs
this mode. It cannot run the pinned mode, because that job's output is the required
`review-verdict/h10` status check on `main` — a hard version pin there means the day the runner's jq
is upgraded (a base-image bump, a host reimage, anything outside this repo's control), every PR
on `main` stops merging until someone notices and re-pins. A required merge gate cannot have a
failure mode that is "an upstream package manager did its job."
- `--expect 1.6` — pin and fail loudly. Used only by `script-tests`. `scripts/tests/` currently
exercises the 1.6 code path only because the runner happens to ship 1.6; if that silently changed,
the 1.6 coverage this whole contract depends on would evaporate with no signal. The tripwire forces
a human decision — re-pin after re-reading this record, or add a real 1.6 matrix leg — instead of
letting the coverage quietly disappear.
**Be honest about the cost: firing this tripwire DOES block merges.** An earlier draft of this
record claimed the pin was safe because `script-tests` is "advisory, not one of the required
checks". That reasoning is wrong, and the correction is worth recording because it is easy to make
twice. `.claude/hooks/pretooluse-merge-consent.sh` reads the **combined** commit status and denies
on anything that is not `success`/`skipped` — see `ci.advisory-red-blocks-the-merge-gate` (#598).
`script-tests` is a Gitea Actions job, so its red is a context folded into that combined state.
A jq bump therefore reddens `script-tests` and blocks non-docs-only merges until someone re-pins.
One qualification, so this does not over-correct in the other direction: that combined-status read
is guarded by `if [ "$mwcs" != "true" ]`. On the `merge_when_checks_succeed` path the hook does not
read the combined status at all and defers to Gitea, which gates on *required* checks only — and
`script-tests` is not one. So the blast radius is the hook-mediated merge path, not literally every
merge.
The pin is kept anyway, deliberately: the fix is a one-line edit to the `--expect` value in
`pr-checks.yml`, the failure message spells that out, and the alternative — silently losing the
only coverage of the version axis that produced three bugs in one day — is worse than a visible
stop. What is NOT acceptable is believing it is free. The difference from `review-verdict.yml` is
therefore one of *degree and recoverability*, not of "blocks merges vs doesn't": there the check is
required per-sha and a jq bump would deadlock merges with no in-repo remedy at all, whereas here a
human can unblock the repo in one commit.
**The parse is strictly fail-closed, and that has an operational edge once it gates merges.**
`scripts/jq-preflight.sh` accepts only a FIRST line of the form `jq-<X>.<Y>` or `jq version <X>.<Y>`;
anything else — a leading blank line, a wrapper that prints a warning first, a version reported only
on stderr — exits 1 rather than guess. That is the right default for a guard whose whole purpose is
refusing to certify a version it did not parse, and it was arrived at over four revisions in which
every *permissive* variant turned out to be fail-OPEN.
But when the follow-up wires the floor-only mode into `review-verdict.yml`, that strictness sits in
the branch-protection-**required** check. A jq wrapper that starts printing a banner line would then
deadlock merges repo-wide — the very failure the pin/floor asymmetry exists to avoid, arriving
through the parser instead of the pin. If that ever happens the fix is to widen the accepted forms in
`jq-preflight.sh`, **not** to relax the fail-closed behaviour: an unparsed version must never be
treated as satisfying the floor.
**The three constructs to avoid, and their version-stable replacements:**
- Never infer "empty input" from a jq exit status — check the string in shell before invoking jq.
- Never use `contains("\u0000")` (or any raw NUL literal) for a control-character test — use
`explode | index(0)`, which does not depend on jq's NUL-escape handling.
- Never infer "parse error" from `jq`'s exit code on ambiguous input — `jq empty` is the portable
parse-only test, but its exit code collides between "no output" (1.6) and "parse error" (1.6, same
code as 1.8's "no output"). Validate the response shape explicitly rather than reading one exit
code as a specific failure mode.
Full narrative of how these were found (inside the #631 `script-tests` rollout) is in
`docs/decisions/records/ci/script-tests-job.md`; this record is the durable contract that came out of
it, rather than the incident log.
+6 -11
View File
@@ -59,17 +59,12 @@ both reds were real:
The second one is the argument for this record in miniature. It sat in the gate that decides whether
a PR skips the Done-when checks, it was covered by an existing test, and that test could not catch it
on a developer Mac (jq 1.8) — only in CI, where the suite had never run. **Standing rules it leaves behind, all three the same shape — never infer a
CONDITION from a jq exit status or a version-dependent builtin:**
- never infer "empty input" from a jq exit status; check the string (`#643`);
- never infer "parse error" from a jq exit status — `jq empty` is the portable test, because
jq >= 1.7 exits 5 where 1.6 exits 4, and 4 is also "no output" (`#647`);
- never use `contains("\u0000")` — on jq 1.6 the escape truncates to `""` and it matches every
string (`#647`). `explode | index(0)` is version-stable.
The runner ships **jq 1.6**; a developer Mac ships 1.8.x. Three divergences were found in one day,
so the durable fix is to pin or preflight the version rather than keep patching constructs —
tracked on `#647`.
on a developer Mac (jq 1.8) — only in CI, where the suite had never run. Two further divergences of
the same shape (a `contains("\u0000")` false positive and a colliding parse-error exit code) turned
up the same day; the durable contract that came out of all three — the exact constructs to avoid, and
why `jq-preflight.sh` pins in `script-tests` but only floors the version in `review-verdict.yml` — is
recorded once, in `ci.jq-version-contract` (`docs/decisions/records/ci/jq-version-contract.md`), and
is not restated here.
An independent cross-family review of that fix then found **two further fail-opens in the same
enumeration, both reachable with no transport error at all** (#643):
@@ -0,0 +1,126 @@
---
key: ci.shared-pr-file-enumeration
title: '2026-07-26 — `scripts/pr-changed-files.sh` is the ONE enumeration of a PR''s changed files; the advisory hook and the enforced gate share mechanism, never policy (#649)'
status: active
since: '2026-07-26'
supersedes: none
superseded-by: none
rule: 'A PR''s complete set of changed file paths is computed by exactly one implementation, `scripts/pr-changed-files.sh`, called by both `.claude/hooks/pretooluse-merge-consent.sh` (advisory — a failure falls through to a human prompt) and `.gitea/workflows/review-verdict.yml` (enforced — a failure must fail closed, because a match here posts the branch-protection-required `review-verdict/h10` status with nobody in the loop). The script owns exhaustiveness (pagination, rename/path validation, head-sha binding, and base-ref binding — see `ci.exemption-provenance`) and returns exit 0 only for a verified-complete list; it does NOT classify paths — each caller keeps its own docs-only allow-list, and the two allow-lists differ on purpose and stay separate.'
signals: 'duplicated PR file enumeration, enforced gate weaker than advisory hook, docs-only allow-list drift, shared mechanism not shared policy, pr-changed-files.sh, checkout base ref not PR head, gate judging its own PR, exhaustiveness bug in a security predicate · paths: `scripts/pr-changed-files.sh`, `.claude/hooks/pretooluse-merge-consent.sh`, `.gitea/workflows/review-verdict.yml` · issues: #643, #648, #649'
mechanics: '`scripts/pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha> <expected-base-ref>` -> stdout newline-delimited paths, exit 0 only if complete and bound to BOTH the given sha and the given base branch; the 5th argument is REQUIRED and a 4-arg call exits 2 (`ci.exemption-provenance`); callers: `.claude/hooks/pretooluse-merge-consent.sh`, `.gitea/workflows/review-verdict.yml`'
---
Before #649, the PR changed-file enumeration existed as two independent implementations. That would
be an ordinary duplication smell anywhere else; here it was actively dangerous, because the two
copies had unequal *consequence*. The advisory hook's failure mode is a human permission prompt — a
missed guard there just means a person gets asked instead of an automatic decision. The enforced
workflow's failure mode is a `success` write to `review-verdict/h10`, the one status branch
protection actually requires — a missed guard there merges an unreviewed PR with nobody asked at all.
**The drift that motivated this.** Four rounds of #643 hardening landed entirely on the copy with the
*lower* stakes. The hook accumulated CR/LF rejection, `..` rejection, a closed `.status` allow-list,
`previous_filename` validation on every row (not just `renamed`), termination only on a validated
empty page, and head-sha binding — while the enforced workflow kept the original, weaker logic. Its
fail-closed behavior on a garbage API response was incidental (an empty `n` erroring a bash
conditional to false), not a designed property. The gate with real authority was strictly weaker than
the gate with none, which is the wrong way around by construction, not by anyone's mistake in a
single review — nothing in the original layout forced the two to move together.
**Why the fix is "one script, two callers" rather than "copy the hardening across."** Copying keeps
the two-implementation shape; the next hardening round would only need to happen twice again, and
there is no mechanism that would surface a second drift before it mattered. Extracting
`scripts/pr-changed-files.sh` makes the enumeration a single artifact with a single test suite
(`scripts/tests/test_pr_changed_files.py`), so a future guard is added once and both callers get it
atomically.
**Mechanism, not policy — the two allow-lists stay separate on purpose.** The extracted script
answers exactly one question: "what is the complete set of paths this PR touches, at one head, or can
we not tell?" It does not decide whether that set makes the PR docs-only. Each caller keeps its own
classification:
- The hook's docs-only pattern also lets `.claude/`, `.gitea/`, `.husky/` through, which is safe there
*only* because a non-match falls through to a human prompt rather than an auto-grant.
- The workflow's is narrower, because there a match posts a green status with nobody in the loop, and
both docs-only and Renovate exemptions are void when the PR touches `.claude/`, `.gitea/`,
`.husky/`, `scripts/` or `docker/ci/` — the gate must not be able to exempt itself from review by
editing itself.
Merging the two allow-lists would have quietly widened the enforced exemption to match the advisory
one, turning a difference that exists for a reason into an accident of refactoring. Sharing the
enumeration closes the drift that actually caused harm without touching the part that was correctly
different.
**What the shared script owns.** Six guards, all now exercised by one test suite instead of a subset
in each caller:
- CR/LF rejection and `..` rejection on every path.
- A closed `.status` allow-list — `added`/`deleted`/`changed`/`modified`/`renamed`/`copied`, not an
open denylist. Note `changed` and `deleted` are the values live Gitea 1.25.4 actually emits;
`modified` is accepted alongside `changed` because a closed list built from the wrong vocabulary
would gate every genuine docs-only PR. GitHub's `removed` is deliberately **not** in the list — an
earlier draft of this record said it was, which would have sent a maintainer looking for a value
the code rejects.
- `previous_filename` validated on **every** row the extraction consumes, not only rows whose
`.status` is `renamed` — a `modified`/`copied` row can still carry it, and an earlier fix that
validated only the `renamed` case was found incomplete for exactly this reason (see
`ci.script-tests-job` for the review trail).
- Termination only on a validated **empty** page — Gitea's paging can return fewer rows than
requested well before the real end of the list, so "short page" is not a valid termination signal.
- Head-sha binding: the head is re-read after enumeration, and the caller must refuse to trust the
list if it moved mid-enumeration, since paging is several round-trips and a force-push between them
would otherwise yield a list belonging to no single commit. **This detects ONE-WAY movement only.**
An A→B→A force-push round trip restores the expected sha, so the binding holds while the pages came
from two different states — see #664. Closing that needs a commit-pinned files endpoint (Gitea has
none) or a local diff, not a tighter check here; the guarantee is stated narrowly rather than left
to read as complete.
**Base-ref checkout — binds the SCRIPTS to the base, not the workflow itself.** `review-verdict.yml`
checks out the PR's BASE ref (`ref: ${{ github.event.pull_request.base.sha }}`,
`persist-credentials: false`), never the head, so the *scripts the job executes* — above all
`scripts/pr-changed-files.sh` — come from the already-reviewed base rather than from the PR under
judgment. The checkout and its `ref` are the security-relevant parts: a bare `run:` calling the
script would not have been sufficient, since the script would then have come from wherever the
runner happened to be.
**It does NOT mean a PR cannot rewrite the gate that judges it (#672).** Gitea resolves a
`pull_request` workflow *definition* from the PR's own head, so a PR editing `review-verdict.yml`
runs its own rewritten copy — which can delete this checkout, or simply post
`review-verdict/h10=success` for its head sha and stop. Branch protection does not close that: it
requires the *context*, not an author, and carries `required_approvals: 0`. An earlier revision of
this paragraph said the workflow "cannot be rewritten by that same PR to weaken its own judgment",
which is true of the scripts and false of the workflow — and stated in the one sentence a reader
resolving this record from the catalog is most likely to stop at.
**That half is now closed, elsewhere — see `ci.gate-trigger-base-resolved` (#672).** The workflow
triggers on `pull_request_target` scoped to `branches: [main]`, so Gitea resolves its definition from
the base rather than the head. The paragraph above is kept in the past tense rather than deleted
because it names the distinction this record turns on: the base-ref checkout binds the *scripts*, and
only the trigger binds the *definition*. Note the dependency runs the other way too — that checkout is
what makes `pull_request_target` safe to use at all here, since this job never executes head-supplied
code.
An earlier revision of this record stated the requirement in the future tense, because the wiring
was staged over two PRs: the workflow runs the BASE version of the gate, and until the shared script
existed on `main` a wired workflow would have exited 127 on its own PR and blocked the merge gate
through the combined status. That staging is complete. Both halves are asserted by
`scripts/tests/test_pr_changed_files.py`, which parses the workflow YAML rather than substring-
matching it — `head.sha` for `base.sha` is a nine-character diff, and a text-level check would still
pass if a second checkout step took the head afterwards and won.
**What is deliberately NOT claimed.** The `PROTECTED`
path list remains the guard that stops a bot-authored PR from editing the gate and exempting itself,
and mutation testing was what established that `PROTECTED` is load-bearing only on the BOT path —
it and `DOCS_ONLY` are disjoint patterns, so on the docs-only path that clause can never fire. A
test written against a docs-only-plus-protected file list passed with the clause deleted.
**A commit status is repo-global, which this record does not fix either.** `review-verdict/h10` is
attached to a sha in the repository, not to a pull request, so a success earned on one PR is
inherited by any other PR with the same head — including one opened against a different base after
the first is closed (#663). That is the same property that makes #622's per-sha binding work, read
from the other end. Out of scope here; noted so the enumeration's guarantees are not mistaken for a
guarantee about *which PR* a verdict belongs to.
**Severity, stated honestly.** Every enumeration defect found in this area (#643) downgraded a
mechanical deny/ask to a human prompt on the hook side; none produced a silent self-merge on their
own. It is still a real weakening worth fixing — the whole point of #649 is that the same class of
bug on the *enforced* copy would not have been merely a downgrade.
@@ -5,8 +5,8 @@ status: active
since: '2026-07-21'
supersedes: none
superseded-by: none
rule: Apply the `in-progress` label before starting an issue, and still read its dependency notes before touching shared surfaces — a claim prevents duplicate pickup, not overlapping code changes.
signals: 'parallel sessions · in-progress label · claim race · dependency notes · shared surfaces · lore pruning · paths: `docs/handoffs/chicorytv-issue-queue.md` · issues: #542'
rule: 'Before starting an issue, check for an existing claim four ways — open PRs referencing it, remote branches naming it, recent comments (a claim can precede the label), and a fresh `git fetch origin main` — then claim with the `in-progress` label plus a comment. A claim prevents duplicate PICKUP, not duplicate WORK. Re-fetch `origin/main` before every push, not only at branch time.'
signals: 'parallel sessions · in-progress label · claim race · duplicate implementation · stale base · branch reverts merged work · dependency notes · shared surfaces · lore pruning · paths: `docs/handoffs/chicorytv-issue-queue.md` · issues: #542, #649, #666'
mechanics: '`in-progress` label on the Gitea issue. The tiny read→claim race window is accepted; the later claimant backs off. Runner topology: two runners (ci-runner VM 127 + bumblebee-runner), 4 slots total.'
---
@@ -17,3 +17,32 @@ after #231", "coordinate with #215").
When editing the standing lore/handoff doc, prune covered and stale bullets rather than appending — it
is not append-only, and git keeps the history. `git pull --rebase` before committing it, since it is
the single most contended file across parallel sessions.
## The label is not the check (ersatztv#649, 2026-07-26)
#649 was implemented **twice, in parallel, to completion**. One session had labelled it `in-progress`
and was three commits and four review rounds deep when a reviewer noticed `origin/main` had moved ten
commits: the other session had already merged the same work as PR #666. The duplicate branch was
discarded — pushing it would have reverted #666 *and* #667, showing the merged work as deletions
because its diff was computed against a stale base.
Two distinct failures, both now covered by the kickoff's step 3:
1. **The claim was made, and was insufficient.** The other session was presumably already underway
when the label went on. A label answers "has anyone announced this?", not "is anyone doing this?"
The cheap proxies for the second question are an open PR whose body says `fixes #N`, a remote
branch with the number in it, and a claiming *comment* that predates the label — which is exactly
the `CLAIM?` flag `scripts/select-queue.sh` already raises and deliberately does not resolve.
2. **The base went stale and nothing re-checked it.** `origin/main` was read once, at branch time,
and not again across many hours. The tell is a `git diff origin/main` that shows deletions you did
not make. Re-fetch before every push; rebase (never merge main in) when it has moved.
Neither session did anything wrong at the moment of claiming. The lesson is that the *duration* of a
session is the risk: the longer a branch lives, the more the "I checked at the start" evidence decays.
Worth noting what worked: the duplicate effort was not wasted. The merged implementation was better in
one respect (it exports `ETV_GITEA_URL` as well as `GITEA_BASE_URL`, because `pr-changed-files.sh`
reads the former at higher precedence), and the discarded branch's test coverage was salvaged onto the
merged code as an additive tests-only PR. When you discover a collision, diff the two implementations
before throwing yours away — the loser usually contains something the winner lacks.
@@ -5,7 +5,7 @@ status: active
since: '2026-07-25'
supersedes: none
superseded-by: none
rule: 'The H10 review verdict is written as a `review-verdict/h10` Gitea **commit status** on the exact reviewed sha by `scripts/post-review-verdict.sh`, and that context is a REQUIRED status check on `main`. Because a status belongs to one sha, a later commit cannot inherit it, so Gitea''s own `merge_when_checks_succeed` refuses to merge a head no one reviewed. The PreToolUse hook additionally refuses to SCHEDULE an auto-merge unless that status is already green on head. A `pull_request` workflow auto-passes the two exempt classes (Renovate-authored, docs-only) unless the PR touches a protected path (`.claude/`, `.gitea/`, `.husky/`, `scripts/`, `docker/ci/`). This extends — does not supersede — `release.review-verdict-gate` (#303 H10), whose comment convention remains the human-readable artifact and the hook''s condition (c).'
rule: 'The H10 review verdict is written as a `review-verdict/h10` Gitea **commit status** on the exact reviewed sha by `scripts/post-review-verdict.sh`, and that context is a REQUIRED status check on `main`. Because a status belongs to one sha, a later commit cannot inherit it, so Gitea''s own `merge_when_checks_succeed` refuses to merge a head no one reviewed. The PreToolUse hook additionally refuses to SCHEDULE an auto-merge unless that status is already green on head. A `pull_request_target` workflow auto-passes the two exempt classes (Renovate-authored, docs-only) unless the PR touches a protected path (`.claude/`, `.gitea/`, `.husky/`, `scripts/`, `docker/ci/`). This extends — does not supersede — `release.review-verdict-gate` (#303 H10), whose comment convention remains the human-readable artifact and the hook''s condition (c).'
signals: 'merge_when_checks_succeed freezes consent, auto-merge merges an unreviewed head, verdict bound to sha, review-verdict/h10 required check, post-review-verdict.sh, Renovate platformAutomerge exemption · paths: `scripts/post-review-verdict.sh`, `.gitea/workflows/review-verdict.yml`, `.claude/hooks/pretooluse-merge-consent.sh` · issues: #622, #303 (H6/H10), #242, #619'
mechanics: '`scripts/tests/test_post_review_verdict.py` (incl. a TOCTOU head-moved case and cross-checks against the hook''s own condition-(c) regexes); branch protection `status_check_contexts` on `main`'
---
@@ -142,18 +142,52 @@ described as one:
`GET /commits/{sha}/status`, which returns latest-per-context, and the workflow refuses to post
anything at all when that read fails or is unparseable, rather than treating it as "no verdict yet".
3. **Changing a PR's base does not change its head sha**, so a verdict status keeps applying to a diff
that has materially changed. Not currently handled; low exposure here because base changes are rare
and manual.
4. **A PR that edits `review-verdict.yml` is judged by its own edited copy.** Gitea runs
`pull_request` workflows from the PR **head**, not the base — confirmed on the very PR that
introduced this workflow (#630): `review-verdict.yml` does not exist on `main`, yet its job ran
and posted a status. So `PROTECTED` is a guardrail against *accidental* self-exemption, **not** a
tamper-proof control: a PR that rewrote the workflow would be classified by the rewritten rules.
Acceptable for a two-account repo (`timothy`, `renovate`) where the threat is a careless change
rather than a hostile one; it would not be for an untrusted-contributor repo, which would need
the classification moved somewhere the PR cannot edit (a base-branch-pinned workflow, or
server-side policy).
that has materially changed. **Detected, not prevented** (#632): `post-review-verdict.sh` records
the base branch in the status description as a trailing `(base: <ref>)`, and the merge-consent hook
reads it back and denies when it no longer matches the PR's live `base.ref`. That covers the hook
path only — a commit status carries no base of its own, so the server-side required check cannot
see this, and a merge driven through the Gitea UI or API is unaffected. Accepted: base changes are
rare, manual, and this is a two-account repo.
The same head-execution behaviour is what makes the rollout self-hosting in the good case: #630's
Two details are load-bearing and each was chosen against a plausible alternative:
- **The comparator is `base.ref`, not `base.sha`.** `base.sha` tracks the base branch's *tip*,
which moves whenever anything merges to `main`; comparing it would invalidate every open verdict
on every unrelated merge — a rare-event guard turned into a permanent merge deadlock. A base
branch that merely *advances* is deliberately out of scope: rebasing onto it moves the head sha,
which the per-sha binding already covers.
- **The field goes in the status description, not the verdict comment.** The comment body is parsed
by `scripts/check-review-verdict.sh`, whose grammar had three false-opens in its history (#629);
nothing parses the description, so this adds a field without reopening that surface.
Verdicts posted before #632 carry no `(base: …)` and get **no opinion** rather than a deny — the
alternative would block every in-flight PR the day it lands, and the window closes on its own since
verdicts are per-head and short-lived. **"Could not check" is a third outcome**, deliberately not
folded into that one: an unreadable status response or a PR with no resolvable `.base.ref` falls
through to a human `ask`. The first draft collapsed them, so a transient Gitea hiccup skipped the
comparison in silence and a later successful read could still emit "merge gate: satisfied" for a
check that never ran.
**Docs-only PRs exit before this check**, because the docs-only carve-out short-circuits the whole
gate earlier in the hook. That carve-out does not auto-grant — it passes through to an ordinary
permission prompt — so the exposure is a missing warning on a merge a human is already confirming,
not a silent merge. Worth knowing before reading "the hook denies on a retarget" as unconditional.
4. **A PR that edits `review-verdict.yml` WAS judged by its own edited copy — closed in #672, see
`ci.gate-trigger-base-resolved`.** Gitea runs `pull_request` workflows from the PR **head**, not
the base — confirmed on the very PR that introduced this workflow (#630): `review-verdict.yml`
does not exist on `main`, yet its job ran and posted a status. `PROTECTED` was therefore a
guardrail against *accidental* self-exemption, **not** a tamper-proof control: a PR that rewrote
the workflow would be classified by the rewritten rules. The workflow now triggers on
`pull_request_target` scoped to `branches: [main]`, so its definition is taken from the base.
**Only this file's instance is closed, not the class.** Any head-resolved workflow holding
credentials that can POST a commit status can still forge `review-verdict/h10`;
`docker-build.yml` demonstrably can, and must stay head-resolved because it builds the PR's own
code. Tracked in #697. So the "careless change rather than a hostile one" posture below still
describes the repo accurately — it is simply no longer *this* workflow that is the weakest link.
An untrusted-contributor repo would still need the classification moved somewhere no PR can
reach (server-side policy), not merely a base-pinned definition.
The same head-execution behaviour was what made the rollout self-hosting in the good case: #630's
own run correctly identified it as touching `.claude/` and `scripts/`, refused both exemptions,
and posted `review-verdict/h10=pending` with an actionable description.
@@ -0,0 +1,198 @@
---
key: spa.library-pickers-resolve-by-search
title: '2026-07-26 — a media-library picker resolves by SEARCH, never by a window over the type; `loadAllPages` stays for bounded-by-construction lists (#651)'
status: active
since: '2026-07-26'
supersedes: spa.list-completeness-vs-bounded-pickers@2026-07-26
superseded-by: none
rule: 'A picker over a media-library table (Episode/Song/Image/Movie/MusicVideo/TelevisionShow/TelevisionSeason/Artist/OtherVideo/RemoteStream) resolves its options by SEARCH — a debounced `SearchPicker` calling `searchLibraryPickerOptions`, which issues at most ONE `getLibraryBrowseItems` request per settled query, bounded to `LIBRARY_PICKER_RESULTS` (25) rows — CLAMPED inside the helper, not merely defaulted — and gated on `LIBRARY_PICKER_MIN_QUERY` (2) characters. It list-loads NOTHING on mount or on a type switch, so there is no truncation to surface and no truncation hint. The typed text is COMPILED (`titleContainsQuery``title:*<escaped>*`), never forwarded raw. The current selection renders from the OWNING RECORD, not from the result set (`selectedName` on a rerun collection / playlist item; a single by-id detail read — `getShow`/`getSeason`/`getArtist` — for a filler preset, which stores only the id), and an edit draft is INITIALIZED ONCE from the detail read — never seeded from the list row, never reconciled against a late response — with the form withheld until it lands, the editor failing CLOSED when the response carries no USABLE concurrency token — absent, empty and whitespace-only ETags are ONE case, normalized in one place, so a PUT without `If-Match` is unreachable, and a deadline plus a route back so a hung request cannot strand it. An id NEVER travels without its namespace: search results are cached against `(source, query)` and list-backed options carry the type they were loaded for, so no id from one type can be offered under another; and every id entering editor state — search result, list-backed option, or a selection restored from a detail read — passes ONE shared `isSelectionId` (int32) predicate at that boundary, an unbindable id being treated as ABSENT rather than coerced. Conflicts are detected at SAVE time via `If-Match` -> 412 -> Reload, and Reload simply drops the draft back to null and re-runs the same initialize-once load, so the form is unmounted while the replacement is in flight; an asynchronously-resolved name is keyed to the id it was resolved for and never overwrites a label naming a different id. The typeahead implements the full ARIA combobox keyboard contract, because it replaces a natively keyboard-operable `<select>`. The other half of the superseded record is UNCHANGED: bounded-by-construction admin lists (collections, multi-collections, smart collections, playlists) still page to completeness via `loadAllPages` and still report `complete`/`hint: incomplete`. Server-side caps are not raised — this is a web-only change.'
signals: 'library picker typeahead, SearchPicker, searchLibraryPickerOptions, titleContainsQuery, LIBRARY_PICKER_RESULTS, LIBRARY_PICKER_MIN_QUERY, LIBRARY_PICKER_LUCENE_SPECIALS, compile typed text not raw Lucene, Lucene && || escaping, picker truncation hint removed, loadAllPages Class A, LuceneSearchIndex.Search hitsLimit, useIsMountedRef, aria-activedescendant combobox keyboard, initialize-once draft not hydrate-merge, no list-row seeding, fail closed on a missing or blank ETag, usable concurrency token, cross-type id, id never travels without its namespace, results keyed on (source query), failed search not cached as empty, selection id int32 boundary predicate, isSelectionId, npm run typecheck not tsc --noEmit, stale result set not committable by keyboard OR pointer · paths: `web/src/api/libraryBrowse.ts`, `web/src/schedules/pickers.tsx`, `web/src/hooks.ts`, `web/src/screens/RerunCollectionsScreen.tsx`, `web/src/screens/PlaylistsScreen.tsx`, `web/src/screens/FillerPresetsScreen.tsx`, `web/src/api/paging.ts`, `docs/spa-conventions.md` §3b · issues: #651, #644, #578, #440'
mechanics: '`docs/spa-conventions.md` §3b'
---
#644 fixed a silent truncation: three `getLibraryBrowseItems` pickers asked for an over-cap
`pageSize` and got the server's `MaxPageSize` back with no indication. Its follow-up review
(`spa.list-completeness-vs-bounded-pickers`) correctly refused to "fix" that by paging to
completeness — ~200 serial requests against a 20,000-row table, each more expensive than the last
(`LuceneSearchIndex.Search` computes `hitsLimit = skip + limit`), ending in a `<select>` with
20,000 `<option>` nodes — and settled on one bounded page plus a visible `Showing the first 100 of
5000 — use search to narrow.` hint. That removed the *silence*. It did not remove the
*unusability*: a 100-row window over Episode or Song is not a picker, it is an arbitrary alphabetical
prefix, and the hint pointed at a search box that did not exist. The record said so itself, deferring
"a full typeahead/search-driven picker over the media library" to a follow-up issue. This is that
issue.
**The fix is to stop windowing and start searching.** `getLibraryBrowseItems` already took a `query`
param (`CollectionsScreen`/`SmartCollectionDialog` were already using it). The three pickers —
`RerunCollectionsScreen`, `PlaylistsScreen`, `FillerPresetsScreen` — now render the existing
`SearchPicker` for their media-library types instead of a `<Select>`, backed by one shared
`searchLibraryPickerOptions(mediaType, text)`. Selecting a media-library type now issues **zero**
requests; a settled query issues **one**, for at most 25 rows. Both bounds are properties of the
helper, not of a caller's discipline, and are pinned by request-count assertions against a
20,000-row fixture rather than by inspection.
**Typed text is compiled, never forwarded.** The rule from #440's Auto-Tune add-source typeahead now
binds here too, and its helper is shared rather than re-implemented: `titleContainsQuery` moved out
of `AutoTuneScreen` into `web/src/api/libraryBrowse.ts`. The search index's default field does not
match bare title words — `Alpha` finds nothing for "Show Alpha" — so forwarding the literal text the
way an explicit query box does would look broken in a *name* picker. Every Lucene special (and
whitespace) is escaped so the boundary stars are the only live wildcards, the same shape
`builder/rules/compile.ts` emits for `contains`.
**The already-selected item is preserved by rendering it from the record, not the result set.** This
is the failure mode that would make a search picker *worse* than the windowed one: a user opening an
existing record must see what it points at, before typing anything and after any search that doesn't
happen to include it. `SearchPicker` already renders `selectedName` independently of `results`, and
rerun collections and playlist items already carry that name on their own DTOs. `FillerPresetFullResponseModel`
does **not** — it stores only the id — so the edit path resolves the name through a single by-id
detail read (`/api/v1/shows|seasons|artists/{id}`), which is *stricter* than the behaviour it
replaces: the old picker could only name a selection that happened to fall inside the first 100
browse rows, and rendered a bare `#9999` otherwise. A failed resolution degrades to `#id`; it never
clears the id.
A cold cross-family review found that "renders from the record" was not by itself enough, because
the *record* can arrive without its selection. `RerunCollectionsController.ProjectToResponseModel`
derives both `selectedId` and `selectedName` from the same eager-loaded navigation, and
`GetRerunCollectionByIdHandler` loads media metadata only for Show/Season/Artist/Movie while
`MediaCollections/Mapper` maps RemoteStream through `_ => null` — so opening a RemoteStream rerun
collection returned HTTP 200 with a null selection and the edit-load refresh *cleared a stored id*,
leaving Save permanently disabled. The rule is therefore stated as a prohibition on the client:
**no code path may clear a stored id it merely failed to name.** Every affected type (RemoteStream,
Episode, MusicVideo, Song, OtherVideo, Image) is covered by its own test. The underlying read-model
gaps are server-side, tracked as **#671**; this branch is web-only and the client guard stays after
that lands.
**The obvious form of that fix is worse than the bug, and this is the part worth remembering.** The
first attempt coalesced the two fields independently — `refreshed.selectedId ?? current.selectedId`
and `refreshed.selectedName || current.selectedName`. But an id and its display name are ONE value:
against a `Song` response (id resolves, name does not), a user selecting a different song while the
refresh was in flight got the *new* name paired with the *stored* id. The chip read "New Song" and
Save wrote 42 — the user's choice discarded with no error and no visual cue, where the original
defect at least cleared the field visibly. A second cold review caught it. The trade is: a visible
failure is strictly better than a silent one, so a "smarter coalesce" is the wrong shape of fix.
**Round 5 deleted all of it.** What follows is kept because the reasoning is the point, but the
mechanism it describes no longer exists: rounds 2-4 built and rebuilt a layer that reconciled a late
detail response against a draft the user was already editing, and that layer produced a HIGH finding
every single round — three of them cross-user lost updates. The final one was unfixable in kind: the
merge had no immutable baseline, so it could not distinguish "the user changed this" from "the server
changed this", giving both a missed conflict and a false one (the false one leaving `etagRef` null,
turning the next save into a silent force-write). The fix was to **remove the race rather than
referee it**: initialize the draft exactly once from the detail GET, withhold the form until it
lands, and detect conflicts at save time through the `If-Match` -> 412 -> Reload path that already
existed. `touchedRef`, `hydrateDraft`, `hydrateIdentity`, `identityConflicts`, `replaceDraft` and
`replacePending` are all gone.
Two facts made that safe rather than lossy. First, the list row could never have helped: its handler
applies **zero** `.Include()`s where the detail handler applies **fourteen**, and both project
through the same mapper, so the list response is a strict subset — the id it was being seeded with
is null in production for every row (#671). Every round-1 "preserve the id from the list" guarantee
was therefore protecting a value that only existed in test fixtures. Second, the sibling screens
(`FillerPresetsScreen`, `PlaylistsScreen`) already worked this way; `RerunCollectionsScreen` was the
outlier, which is why nearly every finding in rounds 3-5 traced to it.
The historical reasoning, retained because the *classes* still bind anywhere a draft is reconciled:
What replaced it is atomicity plus a race rule — and a third review round showed the first attempt
at *that* had made the same mistake one level up: it enumerated the instance (id/name) instead of
covering the class. **A picker selection is one value spread across three fields**: `collectionType`
says which table an id indexes, `selectedId` picks the row, `selectedName` labels it. Splitting type
from id is the identical bug to splitting id from name — the editor displayed and would have saved
a Collection id as a RemoteStream id, when the record's type changed server-side mid-load. So the
whole `{collectionType, selectedId, selectedName}` unit resolves together: either half touched by
the user pins all of it; a differing type takes the response's unit whole (null selection included,
since an id from the old type's space cannot be carried across); and only once both sides agree on
the type does the id/name rule apply.
Hydration also **loses every race against the user**: a `touchedRef` records which fields have been
edited, through a single `edit()` funnel so "touched" cannot drift from "changed".
**Refresh and replace are different policies and must be different functions.** The same review
found a *cross-user lost update*, the worst defect in the series: the conflict "Reload" — which
exists to discard local edits — ran through the refresh path with a touched-set reset. Because the
reloaded record reports `selectedId: null` under the #671 gap, the keep-ours fallback restored the
user's **dirty** selection, the fresh ETag was installed, and the next Save silently overwrote the
collaborator's change with edits the user had explicitly asked to throw away. `replaceDraft` was made a separate function with the mode carried on the load — machinery that
round 5 then deleted outright along with the rest of the reconciliation layer. Every interleaving is tested by holding the detail response open, acting as the user, then
releasing it.
Symmetrically, a name resolved asynchronously is **keyed to the id it was resolved for** and refuses
to overwrite a label that already names a different id — otherwise a slow edit-load read landing
after the user picked something else labels the new selection with the old item's title while the
saved id says otherwise. Keying the render alone stops the mislabelling but still throws away the
newer, correct label, so both halves are needed.
**Scope: Lucene-backed types only.** `GetLibraryBrowseItemsHandler` applies `query` two different
ways — as a Lucene clause for media items, and as a plain SQL `LIKE` on `Name` for the
collection-family types (Collection / SmartCollection / MultiCollection / RerunCollection /
Playlist). A compiled `title:*x*` sent at the latter would be LIKE-matched literally and match
nothing. So `FillerPresetsScreen` marks only its media-item types `searchable`; its
collection-family types keep the bounded single-page load and the truncation hint, and the Class A
`loadAllPages` paths in the other two screens are untouched. The `api.search-allitems-paging`
precedent holds: the client bounds itself, the server cap is not raised.
**The generalisation that took four rounds: an id never travels without its namespace.** Rounds 2
and 3 made *hydration* treat `{collectionType, selectedId, selectedName}` as one value. Round 4 found
the same defect in three more places, because the fix had been applied to the one structure that was
named rather than to every structure that carries an id. A typeahead cached its results against the
query TEXT, so switching the search source with the same text made the re-query guard *suppress* the
new request and leave the previous namespace's hit clickable under the new label. List-backed
`<select>` options were normalised to `{id, name}`, dropping the type, so on a slow connection the
previous type's rows stayed selectable while the replacement loaded — on both screens. The rule that
covers all of them: **every result, option and cached result set carries its source, and identity is
compared as `(type, id)`.** `SearchPicker` now takes a required `source` prop (required, not
defaulted — a default would silently opt every caller out), and `pickerFor` tags list-backed options
with the type that produced them.
**Two cross-user lost updates make a category, not two incidents.** Round 3's was conflict-Reload
running through the refresh policy. Round 4's was subtler: a touched identity pinned against a
server-side type change is *correct*, but adopting the response's newest ETag alongside it authorized
a Save that silently overwrote the collaborator's change with no 412. The category is **never install
a save-authorizing ETag over a local edit the server contradicts** — such a collision is a conflict to
surface, not a state to reconcile. Relatedly, the Reload path now renders the editor inert while the
replacement is in flight, since the dialog closes immediately and an edit typed in that window was
silently erased.
**Cache provenance must distinguish failure from emptiness — without licensing a retry storm.** The
round-3 re-query guard cached a failed search as an authoritative empty result, so a transient 500
became a permanent "No matches" that no amount of reopening could retry. Recording `ok` fixed that
but created the opposite defect: declining the cached failure re-ran the effect and scheduled a
fresh request every debounce. The two concerns are now separate — `ok` says whether the held answer
is authoritative, and an `attemptRef` suppresses automatic retries until an explicit user action
re-arms one. The picker also races `search` against a deadline (a caller-supplied promise carries no
abort signal) and treats a non-array resolution as a failure, since `client.ts` turns a malformed
2xx body into `undefined` rather than rejecting.
**Replacing a native control means owing its keyboard behaviour.** A `<select>` is fully
keyboard-operable, so an input-plus-listbox that only responds to Tab and click is a regression
introduced by this change rather than a pre-existing gap. `SearchPicker` implements the ARIA
combobox pattern: `role="combobox"` with `aria-expanded`/`aria-controls`/`aria-autocomplete`,
Arrow/Home/End moving a virtual cursor exposed through `aria-activedescendant`, Enter committing,
Escape dismissing, and options as non-tab-stops. Two defects specific to an *asynchronous* combobox
also had to be closed: a stale result set was committable (highlight Alpha for "Al", retype "Be",
press Enter before the debounce — Enter selected Alpha), so the highlight now drops on input change
and the guard lives in the single `choose()` sink rather than on each call site (gating Enter while
leaving `onClick` open was the same defect in another modality, found a round later); and Escape
closed the popup while focus stayed in the input, where `onFocus` can never re-arm it, so the picker
was dead until the user blurred and refocused — typing and ArrowDown now both reopen it, without
re-querying results that are already current, since the duplicate response would reset the cursor
and leave Enter doing nothing.
Folded in from #578 (same components): the rule-builder facet typeahead arms on **focus** rather
than on mount, so an N-row rule tree no longer fires N unrequested `search/fields/*/values`
requests; and both it and `SearchPicker` now pair their `seqRef` stale-response guard with a shared
`useIsMountedRef` (`web/src/hooks.ts`) so a fetch resolving after unmount is dropped. Proving that
guard needs two tests, because React 19 no longer warns on a setState-after-unmount and an unmounted
tree renders nothing either way: a unit test of the hook (including a StrictMode double-invoke for
the re-arm) plus an integration test that mocks the hook module and asserts `SearchPicker` actually
read `current` and saw `false`. #578's remaining item — extending the `artist` facet source beyond
entity artists — is a `GetSearchFieldValuesHandler` change, out of scope for a web-only fix, and is
being done on its own branch.
*(Over the 60-line prose ceiling: checked for redundancy against
`spa.list-completeness-vs-bounded-pickers` in `archive/` and declined to cut. The length is six
distinct findings — the search bound, the compile rule, selection preservation, the
clear-what-you-cannot-name prohibition, the async-name keying, and the keyboard contract — most of
which came from review rounds, and each of which names a specific way the obvious implementation is
wrong. The recurring error across four review rounds was always the same: patching the named
instance instead of covering its class — which is why the record states the rules as classes. Summarising any of them back out would lose the counter-example that makes it actionable.)*
@@ -0,0 +1,59 @@
---
key: testing.enumerating-guard-identity-not-position
title: '2026-07-27 — an enumerating allow-list guard keys its registry on IDENTITY, never on a source position (#650, #651)'
status: active
since: '2026-07-27'
supersedes: none
superseded-by: none
rule: 'A guard that cross-checks a hand-reviewed registry against call sites discovered across the whole repo must key each entry on properties INTRINSIC to the site — file, kind, and the value source text — and never on its absolute line or column. A registry keyed on position is a function of every other file in the repo, so a branch that never touches the guard can invalidate it; and because each PR is green against its own base, that failure is structurally invisible pre-merge and lands on `main` after review and after the merge gate. Dropping the position keeps every mutation the guard exists for — a NEW site, a REMOVED site and a CHANGED value each still fail, since each changes the identity multiset — and costs exactly ONE case, which must be stated rather than implied: a SAME-IDENTITY SUBSTITUTION within one file (delete a registered site, add a different unreviewed one with the same kind and value token, net-zero count) now passes. A REPORTED failure still prints the discovered line:column, because identity and diagnostics need not share a format. Comparison stays a MULTISET count rather than set membership, so two sites in one file sharing an identity must be discovered exactly that many times and a third occurrence still fails. A SCANNER test that asserts real AST positions against FIXED inline fixtures is the opposite case and keeps its line/column identity — it has no churn, because its input does not move.'
signals: 'pageSize call-site guard · enumerating allow-list · registry went stale · UNREGISTERED and STALE report · line churn · line drift · semantic merge conflict · guard born red · registry stale on arrival · cancelled run hid a red · registry keyed on line:column · same-identity substitution residual gap · deviation classification · a registry must not launder a defect into a compliant label · multiset count not set membership · `pageSizeSiteId` vs `registryId` · scanner positions vs registry identity · paths: `web/src/api/pageSizeCallSites.guard.test.ts`, `web/src/api/pageSizeScan.ts`, `web/src/api/pageSizeScan.test.ts` · issues: #684, #650, #651, #676, #644'
mechanics: 'Registry identity is `${file}:${kind}:${value}` (`registryId` in the guard); the scanner keeps `pageSizeSiteId` (`${line}:${column}:${kind}:${value}`) for `pageSizeScan.test.ts`.'
---
The #650 guard enumerates every `pageSize` call site in the SPA and cross-checks it against a
hand-reviewed registry in both directions. That part worked. Its follow-up review (F5/M-6) then made
each entry's identity the site's absolute `line:column`, to distinguish two `pageSize` properties on
one line. Sound about disambiguation, wrong about the cost.
**The guard was born red, and the sequence is the argument.** #651 moved `AutoTuneScreen.tsx` up ten
lines and `FillerPresetsScreen.tsx` down seventy-two, and merged *before* the guard's own PR (#675).
The registry, authored against a pre-#651 base, was stale the instant it landed. **A cancelled CI run
is what let it through**: the guard's own merge run was cancelled, so nothing reported the red, which
first surfaced on the next push (#676's merge — which touches no `web/src` file and is not the
cause). `ci.cancelled-is-not-a-verdict`, paying out.
That is ONE ordering accident, not a recurring pattern; the honest count, because the argument needs
no inflation. **The exposure is general anyway, because CI cannot see it coming.** Each PR is green
against its own base, so the breakage exists only in the merge result and surfaces on `main` after
review and after the merge gate.
**Identity should be what makes the site the thing being guarded.** A site's file, `kind` and value
source text determine whether it is reviewed; where it sits in the file does not. The multiset
comparison is what preserves what F5/M-6 was actually protecting and must not relax to set
membership: `TrashScreen.tsx`'s two `PAGE_SIZE` requests must be discovered exactly twice, so a third
occurrence still fails. The converse case is `pageSizeScan.test.ts`, which correctly KEEPS positions
— verifying real AST positions is its subject and its fixtures cannot drift — which is why this
introduced a separate `registryId` rather than changing `pageSizeSiteId` underneath it.
**What it costs, stated rather than implied.** A same-identity substitution inside one file now
passes: delete a registered site, add a different unreviewed one with the same kind and value token,
net-zero count. Narrow, and the old identity caught it only incidentally — it fired on every position
change, so a reviewer conditioned to re-pin line numbers would likely have waved it through. Accepted
knowingly and named in both places, because "costs no coverage" is the kind of claim that outlives
whoever made it, and a guard described as exhaustive stops being re-examined.
**Diagnostics are not the identity.** Dropping position from the comparison key is the fix; dropping
it from the failure *message* was collateral damage. The discovered direction prints `line:column`
alongside each unregistered id — no churn, since positions appear only in an already-failing message.
**A registry must not launder a defect into a compliant-looking label.** Reconciling it surfaced a
live §3b violation (#685): a picker degrading to an unfiltered whole-type window on an empty query,
surfacing nothing. Both labels would have been false — `search-bounded` asserts a required query,
`class-b` a rendered `totalCount` — and either would make the guard vouch for behaviour that does not
exist. Hence a `deviation` class whose entries must name a tracking issue, enforced by a STRUCTURAL
field rather than a `#\d+` scrape of the note: the first version of that test passed with the
tracking reference deleted, because the note legitimately cited two historical issues.
The generalisation: **a guard whose input is the whole repository must not encode anything the whole
repository can change without meaning to.** Position is the common instance; a line count, a file
ordering or a byte offset would all fail the same way.
+44 -6
View File
@@ -81,6 +81,19 @@ Orchestration means: decompose, delegate independent slices, integrate their res
whole, and keep canonical issue state accurate. Use the client's native agent/subagent tools; never
assume a named tool, command, plugin, model-routing feature, or fork mechanism exists.
**Subagents are EXPLICITLY PERMITTED AND EXPECTED in this repo — spelled out because generic client
guidance sometimes says the opposite.** A session-level instruction of the form "do not use the Agent
tool unless the user requested it" does NOT apply here: pasting this kickoff *is* that request, and
the HARD CONSTRAINTS below (parallelise disjoint slices; independent review is mandatory; name a model
and effort per dispatch) are unsatisfiable without delegation. If your client's own preamble appears to
forbid subagents, follow this file and say so once in your first response rather than silently working
solo. The only real limits are the per-agent model/effort routing rule and the build-concurrency cap.
Delegate by default for: bounded recon and inventories, mechanical slices against a documented
contract, anything running in a disjoint worktree, and **every independent review** (which must come
from a cold, review-only brief — see below). Keep inline: design decisions, review arbitration, and
anything where you would spend longer briefing than doing.
Route by capability when the client supports per-agent model selection, and **say which tier you chose
in the dispatch itself** — see the `process.per-agent-model-routing` HARD CONSTRAINT below for the
table. Where the client cannot route per agent, use the active model for every slice except the
@@ -225,10 +238,32 @@ Then work the queue:
**An empty backlog is not a stopping condition** — if the selector returns any eligible candidate,
claim its top-ranked winner; do not ask the user to choose merely because candidates belong to
different workstreams. Never invent a fix-size, recency, or perceived-relevance tiebreaker.
3. **Claim it**: add the `in-progress` label + a "claiming" comment on the issue(s);
reviewer-repo audits are claimed by comment only. Treat that claim as live until a later comment
explicitly releases or abandons it, and exclude audits with a posted deliverable even while the
issue remains open for implementer replies.
3. **Claim it — but CHECK FOR AN EXISTING CLAIM FIRST, and the label is not the whole check.**
The `in-progress` label prevents duplicate *pickup*; it does not prevent duplicate *work*, because
another session may already be implementing an issue it has not labelled (or labelled after you
read the list). Before writing any code, run all four — they are cheap and they fail differently:
a. **Open PRs referencing the issue.** `GET /repos/{owner}/{repo}/pulls?state=open` and look for
`fixes #N` / `refs #N` in the body, or the number in the branch name. This is the check that
would have caught ersatztv#649 being implemented twice.
b. **Remote branches naming the issue.** `git ls-remote --heads origin '*<N>*'` — a branch usually
exists before the PR does.
c. **Recent comments on the issue**, not just its labels — a "claiming" comment from another
session may predate the label, which is exactly what `CLAIM?` from `select-queue.sh` flags.
d. **`git fetch origin main`**, so you are reading current state rather than your session's
opening snapshot.
If any of those hit, do not start: report it to the user and take the next candidate. If none do,
claim with the `in-progress` label **and** a "claiming" comment (reviewer-repo audits are claimed
by comment only). Treat a claim as live until a later comment explicitly releases or abandons it,
and exclude audits with a posted deliverable even while the issue remains open for implementer
replies.
**Re-fetch `origin/main` before every push, not only at branch time.** A long session can run for
hours across several review rounds; `main` moves underneath it. A branch cut from a stale base
whose diff is computed against that stale base will silently show *other people's merged work as
deletions*, and pushing it reverts them. Rebase (never merge main in) and re-run the local gate
whenever the fetch shows movement. → `process.parallel-session-claim`
4. **Scan for bundle-able siblings** (always, right after claiming — not optional). Check all three
bundle axes from "Bundles" above: the claimed issue's **milestone**, its **cross-references /
backlinks**, and its **shared label(s)** (list the other open issues under each of its labels).
@@ -322,8 +357,11 @@ HARD CONSTRAINTS:
task-specific delta. → `docs.convention-docs-session-start`
- Run `scripts/select-queue.sh` for queue selection; trust its deps/tiering/ordering and resolve only its
`CLAIM?`/`UMBRELLA?` flags. → `startup.parallel-orientation`
- Claim with `in-progress` before working — but a claim prevents duplicate *pickup*, not overlapping code
changes. → `process.parallel-session-claim`
- Claim with `in-progress` before working — but **check for an existing claim first** (open PRs
referencing the issue, remote branches naming it, comments predating the label, a fresh
`git fetch`), because a label prevents duplicate *pickup*, not duplicate *work*: #649 was
implemented twice to completion. And re-fetch `origin/main` before every push — a branch on a stale
base reverts whatever merged meanwhile. → `process.parallel-session-claim`
**Building and reviewing**
- The PR routine is a fixed sequence; for API changes build the app project FIRST, then
+1 -1
View File
@@ -76,7 +76,7 @@ Only after Phase 1 sign-off. Per slice (start with #2a Channels):
- **One branch = one PR.** PR runs `test` + `migrations` (both **required** to merge). Merge to `main` runs `test`+`migrations`+`build`+smoke/E2E. Verify green before closing each sub-issue.
- Migrations only if the model changes — `scripts/add-migration.sh <Name>` does **both** providers.
- Adversarial self-review of the diff before closing (see memory: adversarial-self-review-at-milestones). Then Task Completion Protocol / `/done <sub-issue>`.
- CI poll: `curl -u timothy:ded89Lm4 …/api/v1/repos/timothy/ersatztv/actions/tasks` (jobs by name), or the runs API.
- CI poll: `curl -u "$ETV_GITEA_BASICAUTH" …/api/v1/repos/timothy/ersatztv/actions/tasks` (jobs by name), or the runs API.
## Repo state at handoff
- `main` is green; #1 closed (config/topology, not code — see `docs/m3u-xmltv.md`). #5 triaged as Jellyfin/infra (→ server-management).
+206 -37
View File
@@ -121,7 +121,7 @@ Convention — when a screen keeps stale results visible during a refetch:
current (compare against a ref that always holds the committed value — `SearchScreen` reuses
`lastQueryRef`) and **discard** otherwise. Checking only `activeRef` (mounted) is insufficient.
## 3b. Paged list endpoints clamp server-side — page to completeness ONLY for bounded lists, never a media-library picker
## 3b. Paged list endpoints clamp server-side — page to completeness ONLY for bounded lists; a media-library picker searches instead
Every paged `/api/v1` list endpoint (rerun-collections, multi-collections, library/browse, search,
trakt-lists, …) clamps `pageSize` to its own controller's `MaxPageSize` (100, as of #644) regardless
@@ -131,14 +131,14 @@ UI to notice the gap. This was issue #644 (following on from #634, which fixed t
`SchedulesScreen`'s rerun-collections picker load).
**Two classes of call site, treated differently** (decision record:
`docs/decisions/records/spa/list-completeness-vs-bounded-pickers.md`, `spa.list-completeness-vs-bounded-pickers`
— a #644 follow-up review found the original blanket "use `loadAllPages` for any picker" guidance
below was itself the defect for one class of caller):
`docs/decisions/records/spa/library-pickers-resolve-by-search.md`,
`spa.library-pickers-resolve-by-search`, superseding `spa.list-completeness-vs-bounded-pickers`
the #644 follow-up got Class A right and Class B only half right):
- **Bounded-by-construction lists** (rerun collections, multi-collections, playlists — admin-created,
hundreds of rows at most): genuinely need the complete list, and completeness is cheap. Use the
shared `loadAllPages` helper (`web/src/api/paging.ts`, re-exported via `web/src/api/index.ts`)
instead of an inflated `pageSize`:
- **Class A — bounded-by-construction lists** (collections, multi-collections, smart collections,
playlists — admin-created, hundreds of rows at most): genuinely need the complete list, and
completeness is cheap. Use the shared `loadAllPages` helper (`web/src/api/paging.ts`, re-exported
via `web/src/api/index.ts`) instead of an inflated `pageSize`:
```ts
const { items, complete } = await loadAllPages(getMultiCollections); // pages against totalCount, cap defaults to 100
@@ -150,41 +150,206 @@ below was itself the defect for one class of caller):
a caller that needs the full list must not treat a resolved promise as proof the list is whole (a
partial result is otherwise silently indistinguishable from a complete one — the same defect class
as #644 itself, since `GetLibraryBrowseItemsHandler.HydrateMediaItems` can legitimately drop stale
Lucene hits and produce a short/empty page in normal operation). Pass an `AbortSignal` (4th arg)
from the caller's effect cleanup so a superseded load stops issuing further page requests instead
of hammering the server for a result nobody will see. **Do not raise the server-side cap to work
around this** — the `api.search-allitems-paging` precedent is that the client pages and the server
stays bounded; that's a backend decision, out of scope for a screen fix.
Lucene hits and produce a short/empty page in normal operation). Render `complete: false` as its own
copy ("List may be incomplete — retry to reload"), never through search-narrowing text — a
`loadPickerOptions` result that can come from either class carries a `hint: 'incomplete' | 'none'`
discriminator, not a boolean shared with an unrelated condition (#644 follow-up round-3 review F1).
Pass an `AbortSignal` (4th arg) from the caller's effect cleanup so a superseded load stops issuing
further page requests, and gate any `console.warn` on `!signal?.aborted` — a superseded or
user-aborted load returns `complete: false` too, and that's expected, not a defect. **Do not raise
the server-side cap to work around any of this** — the `api.search-allitems-paging` precedent is
that the client bounds itself and the server stays bounded.
- **Media-library pickers** (`getLibraryBrowseItems` backing a `<select>` for Episode / Song / Image /
Movie / MusicVideo / etc. — the largest tables in an install, tens of thousands of rows possible):
must **NOT** use `loadAllPages`. Paging to completeness here means on the order of 200 serial
requests for a 20k-row library each more expensive than the last, since
`LuceneSearchIndex.Search` computes `hitsLimit = skip + limit` — to populate a native `<select>`
with thousands of `<option>` nodes. That is worse than the truncation bug it would "fix". Instead,
fetch **one bounded page** directly (`pageSize` at the cap) and make the truncation **visible**
rather than silent — e.g. a `ctv-field-help` hint next to the picker: `Showing the first 100 of
5000 — use search to narrow.` (wire the response's real `totalCount`). See
`RerunCollectionsScreen.tsx`/`PlaylistsScreen.tsx`/`FillerPresetsScreen.tsx`'s `loadPickerOptions`
for the pattern. A full typeahead/search-driven picker is a separate, larger feature — out of scope
for this fix.
- **Class B — media-library pickers** (Episode / Song / Image / Movie / MusicVideo / TelevisionShow /
TelevisionSeason / Artist / OtherVideo / RemoteStream — the largest tables in an install, tens of
thousands of rows possible): **resolve by search, do not window the type at all** (#651). Neither
`loadAllPages` (~200 serial requests for a 20k-row library, each more expensive than the last since
`LuceneSearchIndex.Search` computes `hitsLimit = skip + limit`) nor a single bounded page (an
arbitrary alphabetical prefix, unusable as a picker even once the truncation is made visible) is
acceptable. Render the shared `SearchPicker` (`web/src/schedules/pickers.tsx`) over
`searchLibraryPickerOptions` (`web/src/api/libraryBrowse.ts`):
**A Class B truncation and a Class A `complete: false` are different conditions — don't collapse
them into one boolean** (#644 follow-up round-3 review F1): a `loadPickerOptions` result that can
come from either a Class A (`loadAllPages`) or Class B (single bounded page) source should carry a
`hint: 'incomplete' | 'none' | 'truncated'` discriminator, not a `truncated: boolean` reused for
both. `'truncated'` (Class B, an expected cap) keeps the "Showing the first N of M — use search to
narrow" copy; `'incomplete'` (Class A, `loadAllPages`'s `complete: false`) renders different copy
("List may be incomplete — retry to reload") — rendering both through the search-narrowing text
produces a self-contradictory "Showing the first 47 of 47" when a Class A load doesn't converge.
Also gate any `console.warn` on a Class A `complete: false` with `!signal?.aborted` — a superseded
or user-aborted load returns `complete: false` too, and that's expected, not a defect.
```ts
const searchLibrary = useCallback( // memoize: SearchPicker lists `search` in its effect deps
(q: string) => searchLibraryPickerOptions('Episode', q),
[]
);
```
The helper owns both bounds: at most ONE `getLibraryBrowseItems` request per settled query, at most
`LIBRARY_PICKER_RESULTS` (25) rows, and no request at all below `LIBRARY_PICKER_MIN_QUERY` (2)
characters. Selecting a media-library type must issue **zero** requests. There is no truncation, so
there is no truncation hint — the old `Showing the first 100 of 5000 — use search to narrow.` copy
is gone from these pickers along with the window it described. Prove the bound with a
**request-count assertion against a large (20k-row) fixture**, not by inspection.
- **Compile typed text; never forward raw Lucene.** Send `titleContainsQuery(text)`
`title:*<escaped>*`. The index's default field does not match bare title words (`Alpha` finds
nothing for "Show Alpha" — see `e2e-local.md`), so a raw forward looks broken in a *name* picker.
Reuse the helper; do not re-implement the escaping (same rule as the #440 Auto-Tune typeahead,
same shape `builder/rules/compile.ts` emits for `contains`). The escaped set includes `&` and
`|`, because Lucene's boolean operators are `&&`/`||` and a title like `Rock & Roll` otherwise
compiles to something Lucene parses as syntax. **Drive the escaping test from the exported
character set** (`LIBRARY_PICKER_LUCENE_SPECIALS`), one character per case — a test carrying its
own hand-copied "every special" sample cannot see what is missing from that sample.
- **The bound belongs to the helper, not the caller.** `searchLibraryPickerOptions` *clamps*
`pageSize` to `LIBRARY_PICKER_RESULTS`; a documented bound a caller can exceed by passing a
bigger number is not a bound.
- **Render the current selection from the owning record, not from the result set.** An item already
selected but outside the current results must still display — losing it on edit is data loss, not
a cosmetic defect. Rerun collections and playlist items carry `selectedName` on their own DTOs;
`FillerPresetFullResponseModel` stores only an id, so its edit path resolves the name with a
single by-id detail read (`getShow`/`getSeason`/`getArtist`) and degrades to `#id` on failure —
never to a cleared field.
- **A read model that derives an id and its name from the same eager-loaded navigation reports
*no selection at all* when that navigation isn't loaded** — a successful 200 indistinguishable
from "the user cleared it". (`RerunCollectionsController.ProjectToResponseModel` does exactly
this, and `MediaCollections/Mapper` maps RemoteStream through `_ => null`; tracked as **#671**.)
An earlier revision of this section required a client-side guard that preserved the id across
such a response. **That guard is gone and must not be rebuilt** — it only ever preserved a value
seeded from the list row, which is itself null in production for every row, and the reconciliation
it required is what the initialize-once rule below replaced. The correct handling is to show the
server's answer honestly: no selection, Save disabled, and the validation badge saying why.
- **An id NEVER travels without its namespace — in results, in options, in cached result sets.**
A media/collection id only means anything inside the type that produced it, so any structure
holding ids must hold the type too, and identity is compared as `(type, id)`. Three places this
bites, all the same bug:
1. A typeahead's cached results must be keyed on `(source, query)`, not the query text. Same
query, different source ⇒ the results are not *stale*, they are *wrong*: hide them and
re-query. Keying on text alone lets a re-query guard **suppress** the new source's request
and leave the old namespace's hit clickable under the new label.
2. List-backed `<select>` options must carry the type they were loaded for and be dropped the
moment the active type differs — otherwise the previous type's rows stay selectable during
the replacement load on a slow connection.
3. A local edit whose type contradicts the server's is a **conflict**, not something to
reconcile (below).
- **Initialize an edit draft ONCE, from the detail read — never reconcile a late response against
an open form.** This supersedes an earlier prescription here for merging a refresh into a draft
field-by-field/atomically with touched-field tracking. That reconciliation layer produced a HIGH
finding in three consecutive review rounds of #651, including three cross-user lost updates, and
the last of them (a merge with no immutable baseline, so it could not tell a local edit from a
server change) is unfixable without adding a third-way baseline — more machinery on the surface
that was generating the bugs. Instead:
- `draft` starts as `null` for an existing record and the form does not render until the detail
GET lands. There is then no draft for a late response to reconcile against, and no window in
which the user can edit something about to be replaced.
- **Do not seed from the list row.** It is not authoritative: for rerun collections the list
handler applies zero `.Include()`s while the detail handler applies fourteen, and both project
through the same mapper, so the list response is a strict SUBSET of the detail one (#671). A
seed can only add a race, never information. Verify that claim for your endpoint before
relying on it.
- **Fail CLOSED on a missing concurrency token.** Writing the ETag in the same callback that
sets the draft is *not* the same as "a draft implies an ETag" — the response can simply omit
the header, and then the PUT carries no `If-Match` and silently force-writes. No token ⇒ no
editable draft (error + Retry/Back). Note this makes your test mocks load-bearing: a detail
mock that omits `ETag` was previously exercising the force-write path without saying so, so
give every single-record GET mock a real ETag and test the absent case explicitly.
- **Bound the load and always offer a way out.** A caller-supplied fetch with no abort signal can
hang forever; race it against a deadline, and give the loading view a Back control so a hung
request is never a dead end.
- **Detect conflicts at save time** via the existing `If-Match` → 412 → Reload path. Reload sets
the draft back to `null` and re-runs the same initialize-once load, so "replace" needs no
separate policy and the form is unmounted while the replacement is in flight.
`FillerPresetsScreen` and `PlaylistsScreen` already worked this way; `RerunCollectionsScreen` was
the outlier that seeded from its list row, which is where every one of these defects lived.
- **A name resolved asynchronously must be keyed to the id it was resolved FOR**, and must refuse
to overwrite a label that already names a different id. A slow by-id read landing after the user
has picked something else would otherwise label the new selection with the old item's title
while the id — and therefore what gets saved — says otherwise. Keep the guard at the *writer*,
where it is reachable and testable; a second render-time id comparison is unreachable once
every writer sets the label and the id together, and an unreachable guard is an untested one.
- **Only Lucene-backed types.** `GetLibraryBrowseItemsHandler` applies `query` as a Lucene clause
for media items but as a plain SQL `LIKE` on `Name` for the collection-family types (Collection /
SmartCollection / MultiCollection / RerunCollection / Playlist). A compiled `title:*x*` sent at
those matches nothing literally. Keep the collection-family pickers on their Class A / bounded
single-page loads — `FillerPresetsScreen`'s `COLLECTION_TYPES` marks the search-driven entries
with `searchable: true` for exactly this reason.
**If a screen shows a bounded preview or has real paging UI** (a "load more" button, a page-size
selector, a fixed-size typeahead result list), a `pageSize` at or below the cap is correct as-is —
`loadAllPages` is only for "I need literally everything, and the list is small by construction"
call sites.
**Debounced typeaheads: arm on focus, and guard on mounted as well as on sequence.** A typeahead that
fetches on *mount* multiplies by the number of rows on screen (an N-rule tree fired N unrequested
facet lookups before #578); arm the effect on the input's `onFocus` instead. And pair the monotonic
`seqRef` stale-response guard with the shared `useIsMountedRef()` (`web/src/hooks.ts`) in every async
callback — `seqRef` drops an *older* response, but says nothing about whether the component still
exists.
**A custom picker replacing a native control owes you its keyboard behaviour.** A `<select>` is
fully keyboard-operable; swapping in a listbox-and-input is an accessibility *regression* unless it
implements the ARIA combobox pattern — `role="combobox"` + `aria-expanded`/`aria-controls`/
`aria-autocomplete` on the input, ArrowUp/ArrowDown to move a virtual cursor exposed via
`aria-activedescendant`, Enter to commit, Escape to dismiss, options as non-tab-stops
(`tabIndex={-1}`) marked with `aria-selected`. Note this changes what `getAllByRole('combobox')`
matches in tests: count `<select>` elements when that is what you mean. Two failure modes that only
appear once the widget is asynchronous:
- **Freshness is `(source, query)`, and cached failures are not answers — but they are not licences
to retry either.** A `SearchPicker`-style cache must record which source produced the results and
whether the attempt *succeeded*. Caching a failure as an authoritative empty result turns a
transient 500 into a permanent "No matches" that reopening can never clear. But simply declining
the cached failure re-runs the effect and schedules another request every debounce — a **request
storm** on a persistent outage. Keep the two apart: a `resultsFor.ok` flag says whether the held
answer is authoritative, and a separate *attempted* key (a ref, so writing it doesn't re-render)
suppresses automatic retries until an explicit user action — reopen, focus, or edit — re-arms it.
- **Put a validity predicate at the BOUNDARY the class crosses, not at the site the bug was found.**
An entity-reference id (`selectedId`, `collectionId`, `mediaItemId`, …) is bound by the API as a
32-bit integer, so `1.5` or `2147483648` renders and commits fine and then fails on write. Such
ids enter editor state through *several* doors — search results, list-backed `<select>` options,
and the selection restored from a detail read — so a check added to whichever one surfaced the
defect leaves the others open (this is how #651 produced the same finding in two consecutive
rounds). Share one predicate (`isSelectionId` / `selectionIdOrNull` in `web/src/api/selectionId.ts`)
and apply it on every path — including the ones that don't look like pickers, such as a
`playlistGroupId` seeded from the wire into a create dialog. **Treat an unbindable id as ABSENT,
never coerce it** — rounding `1.5` to `1` would submit a *different* record — and **clear its
label with it**: a row still reading "Blade Runner" over a null id makes two contradictory
statements about the same item. Drop rather than render an option that cannot be selected safely.
"Surfaces as no selection" is only true if that screen's Save gate actually checks for one — on
`PlaylistsScreen` it did not, so this claim was false there for a full round after being written
here. **Verify an invariant on every screen it names before writing it down.** Prove it per
ingress by asserting zero writes are reachable *after attempting the write*: a write-count
assertion on a path that never attempts one is trivially true. And unit-test the predicate's
INCLUSIVE endpoints directly — once it is the single point of failure for every ingress, a `>`
for `>=` slip passes an entire screen suite.
- **A caller-supplied promise needs a deadline, and a 2xx body is not a contract.** `client.ts`
turns malformed JSON into `undefined` rather than rejecting, so `setResults(undefined)` throws on
the next render. Validate the **elements, not just the container**: `Array.isArray` accepts
`[null]`, which then throws on `option.id` during render, and an element with a wrong-typed `id`
commits an invalid value through `onSelect`. Treat any malformed payload as a failed attempt (so
it stays retryable), not as an empty answer. And a `search` prop carries no abort signal, so race
it against a timeout — otherwise a never-settling request leaves the picker spinning with no way
back.
- **A stale result set must not be committable — by ANY modality.** Between a keystroke and its
response, `results` still describe the *previous* query, so highlighting an option, retyping, and
pressing Enter commits the old option while the box reads the new text. Drop the highlight on
**input change** (not when the next response arrives) and put the guard in the single `choose()`
sink rather than on each call site — gating Enter and leaving `onClick` open is the same defect in
another modality, and the next path added would be ungated too. Keep the stale list *visible*
(hiding it flickers on every keystroke) but genuinely inert: `aria-disabled` plus a dimmed style,
not merely a handler that silently no-ops on a normal-looking button.
- **Escape must not strand the user.** Closing the popup while focus stays in the input means
`onFocus` can never re-arm it, so typing does nothing and the user has to blur and refocus to
recover. Typing and ArrowDown must both reopen it — and reopening onto results that are already
current must **not** re-query: the duplicate response lands later and resets the cursor the user
has since moved, so Enter silently does nothing. Reopening also places the cursor (ARIA APG)
rather than swallowing the keypress.
**The web typecheck gate is `npm run typecheck`, never `npx tsc --noEmit`.** `web/tsconfig.json` is
solution-style (`"files": []` + `references`), so a bare `tsc --noEmit` resolves to zero input files
and exits 0 **without checking anything** — a green that means "I looked at nothing". CI runs
`npm run typecheck` (`tsc -b --pretty false`), which builds the referenced projects and includes the
test files. Verified by planting a deliberate type error: `--noEmit` stayed green, `-b` caught it.
**Testing an is-mounted guard: React 19 does not warn on a setState-after-unmount, and an unmounted
tree renders nothing either way** — so no DOM assertion can distinguish "the guard stopped it" from
"React discarded it". Prove the *mechanism* (a `useIsMountedRef` unit test, with a StrictMode
double-invoke for the re-arm) **and** the *integration* (mock the hook module and assert the
component actually read `current` — and saw `false` — when the late response landed). Verify each by
removing the mechanism and confirming the test fails.
## 4. API client modules
One file per domain in `web/src/api/`, e.g. `logs.ts`, `blocks.ts`, `playouts.ts`. Pattern (see
@@ -619,12 +784,16 @@ not wired here.
for these two fields — the compiled query is the existing `released_inthelast:"7 day"`-style
`CustomMultiFieldQueryParser` macro, so nothing downstream changes. `validation.ts`'s `ruleError`
requires the value to parse as a positive integer before it's compiled.
- **Facet-value typeahead** (#434, `api.search-field-values`) — the value input for a `text` field
- **Facet-value typeahead** (#434/#578, `api.search-field-values-sources`) — the value input for a `text` field
(not enum) is a combobox backed by `getSearchFieldValues` (`web/src/api/search.ts`
`GET /api/v1/search/fields/{name}/values?q=&limit=`), debounced on keystroke, prefix-matching the
in-progress value against distinct terms already in the index. It always allows free-text entry as a
fallback — a 404 (non-text field) or an empty result list (e.g. ElasticSearch backend) degrades to a
plain text input rather than blocking the rule.
plain text input rather than blocking the rule. Since #578 (`api.search-field-values-sources`)
`album_artist` returns values instead of 404ing, and `artist` covers free-text music-video/song credits
as well as entity artists; for those two the server's list is **bounded best-effort** on a very large
library, so the free-text fallback stays load-bearing — never treat an absent suggestion as an invalid
value.
- **Single-child-group normalization** (#438) — `normalizeGroup` (`validation.ts`) coerces a group's
`match` to `all` whenever it has fewer than two children, recursively. A one-child `any` group is
semantically identical to `all` but doesn't round-trip through `compile``parse` (the compiled Lucene
+173
View File
@@ -0,0 +1,173 @@
#!/usr/bin/env bash
# Make the jq version a job's shell gates run under OBSERVABLE, and any drift LOUD.
#
# ersatztv#648. Every shell gate in this repo is authored and tested on a developer Mac shipping
# jq 1.8.x. The CI runner ships jq 1.6. Nothing pinned or checked that, and until ersatztv#631 the one
# thing that could have noticed (scripts/tests/) never ran on the runner. Three independent divergences
# surfaced in a single day:
#
# ersatztv#643 `jq -e` over EMPTY input -> exit 4 on 1.8, exit 0 on 1.6 (a transport failure
# passed the docs-only pagination guard)
# ersatztv#647 contains("<NUL>") -> false on 1.8, TRUE for every string on 1.6
# (the H10 verdict classifier was entirely inert)
# ersatztv#647 parse-error exit code -> 5 on 1.8, 4 on 1.6 — same as "no output"
# (garbage API response read as "no comments")
#
# All three are fixed with version-stable constructs, but patching constructs one at a time does not
# scale: the failures share one shape — a shell gate's behaviour is a function of its interpreter's
# version, and that version was an UNTESTED AXIS. This script makes the axis explicit.
#
# WHY A FLOOR AND NOT A PIN EVERYWHERE. The obvious fix — bake a pinned jq into the CI toolchain image
# (docker/ci/Dockerfile) — provably does NOT cover the gate that actually broke. `.gitea/workflows/
# review-verdict.yml` is `runs-on: small`, carries no toolchain-image pin, and per `ci.small-lane-git-only`
# the small lane is git-only. It therefore gets the HOST's jq 1.6 no matter what the image contains.
# That was checked, not assumed (ersatztv#648's first Done-when box).
#
# So the contract is the other way round: 1.6 is the FLOOR every gate must work on, and it is the
# runner's own jq that provides the 1.6 coverage `scripts/tests/` runs under.
#
# TWO MODES, deliberately asymmetric:
#
# (no --expect) Print the version and assert it is >= MIN_VERSION. Used by jobs on the merge
# path, including review-verdict.yml. There is NO upper bound here on purpose:
# review-verdict.yml writes `review-verdict/h10`, a REQUIRED status check on
# `main`, so a hard pin there would turn any jq upgrade on the runner into a
# repo-wide merge deadlock. Observability without a deadlock risk.
#
# --expect X.Y Additionally assert the version is exactly X.Y, and FAIL if not. Used by the
# `script-tests` job. This is the tripwire: `scripts/tests/` currently exercises
# the 1.6 path only because the runner happens to ship 1.6. If the runner were
# upgraded, that coverage would vanish SILENTLY and the whole class of bug above
# would go untested again. Going red forces a human to decide — re-pin, or add a
# real 1.6 matrix leg — rather than letting the coverage evaporate unnoticed.
#
# Usage: jq-preflight.sh [--expect <major.minor>]
set -euo pipefail
# The lowest jq every shell gate in this repo must run correctly on. Do not raise this without
# confirming the CI runner has actually been upgraded first — the runner, not the dev Mac, is the
# binding constraint.
MIN_VERSION="1.6"
expect=""
while [ "$#" -gt 0 ]; do
case "$1" in
--expect)
# `shift 2` with a missing value fails under `set -e` and exits 1 with NOTHING on either
# stream — a CI step dying with an empty log is exactly the diagnostic hole this script exists
# to remove. Check explicitly instead.
if [ "$#" -lt 2 ] || [ -z "${2:-}" ]; then
echo "jq-preflight: --expect requires a <major.minor> value" >&2
exit 2
fi
expect="$2"; shift 2 ;;
*) echo "jq-preflight: unknown argument '$1'" >&2; exit 2 ;;
esac
done
if ! command -v jq >/dev/null 2>&1; then
echo "jq-preflight: jq is not on PATH. The shell gates in scripts/ and .gitea/workflows/ shell out to jq; without it they fail as a pile of opaque assertion errors instead of one clear message." >&2
exit 1
fi
# Take jq's EXIT STATUS seriously, and keep stderr OUT of the parse input.
#
# This was `raw=$(jq --version 2>&1 || true)`, which did neither — and that combination turned the
# guard fail-OPEN on the case it most needs to catch. A jq that cannot start (the canonical one is a
# glibc mismatch after a base-image change) exits 127 and writes something like
# `jq: /lib/x86_64-linux-gnu/libc.so.6: version 'GLIBC_2.34' not found` to stderr. Folded into `raw`,
# that string contains `2.34`, which the version pattern happily matched — so the preflight printed
# "parsed 2.34", certified the floor, and exited 0 on a jq that cannot run at all. The strip-based
# parse this replaced failed CLOSED there, so it was a regression introduced by the fix.
# `$?` inside an `if ! cmd; then` block is the NEGATED status (0), not jq's, so capture it explicitly.
set +e
raw=$(jq --version 2>/dev/null)
jq_rc=$?
set -e
if [ "$jq_rc" -ne 0 ]; then
echo "jq-preflight: 'jq --version' failed (exit ${jq_rc}). jq is on PATH but cannot run — a broken build or a missing shared library. Failing closed rather than certifying a version it did not report." >&2
exit 1
fi
# `jq --version` prints e.g. `jq-1.6`, `jq-1.7.1`, or on some builds `jq-1.8.2-dirty`.
# Parse with an explicit regex rather than by stripping around the first `-` and `.`.
#
# The strip approach had a hole that defeated the whole point of this script. It assumed the format
# is exactly `jq-X.Y`, so a build printing anything else — `jq version 1.6` (a distro wrapper),
# `JQ-1.6`, `jq-1.-6` — left ONE of major/minor empty. The old sanity check was
# `case "$major$minor" in *[!a-9]*|"")`, and on `jq version 1.6` that concatenation is "6": non-empty
# and all-digits, so the guard PASSED. The floor comparison then ran `[ "" -lt 1 ]`, which exits 2
# with "integer expression expected" — and `set -e` exempts a failing command in an `if` condition,
# so the whole conditional read false and the script exited 0 having asserted NOTHING, after printing
# a plausible-looking "parsed" line.
#
# That is the silently-untested-axis failure this script was written to eliminate, reproduced inside
# the script itself. Require a real `<digits>.<digits>` match, and fail closed when there isn't one.
# ANCHORED to the leading `jq` token, not "first digits.digits anywhere in the string".
#
# An unanchored match takes whatever number comes first, wherever it is. That accepted a leading
# warning line or a date prefix as the version — `2026.07.26 jq-1.6` parsed as 2026.07, which sails
# over the floor. Anchoring keeps every legitimate form (`jq-1.6`, `jq version 1.6`, `jq-1.7.1`,
# `jq-1.6-dirty`, `jq-1.6 (Debian 1.6-2.1)`) and rejects the rest, which then fails closed below.
# FIRST LINE ONLY, and bounded everywhere. Both bounds are load-bearing; this is the third round on
# this one predicate and each previous version failed for a variant of the same reason.
#
# * First line only. `[[:space:]]` matches NEWLINES, so an "anchored" pattern still scanned the
# whole output: `jq\n2.34: cannot load` matched `jq`, crossed the newline as separator, and
# parsed 2.34 — fail-open, the round-2 bug narrowed but not closed. `[[:blank:]]` (space/tab
# only) plus a first-line slice confines the match to the line that can actually carry a version.
# * Bounded digit runs. This is the round-1 mechanism resurrected. The regex guaranteed the
# operands were digits but not that they fit in `test`'s integer range, so a 23-digit major made
# `[ "$major" -lt "$min_major" ]` error with "integer expression expected" — and `set -e` exempts
# a failing command in an `if` condition, so the conditional read false and THE FLOOR WAS NEVER
# ASSERTED, exit 0. Exactly what the empty-string case did in round 1. `{1,9}` keeps every
# operand inside a 32-bit integer, so the comparison can no longer error.
# * Bounded separator runs, so the pattern cannot be walked across arbitrary filler.
first=${raw%%$'\n'*}
first=${first%$'\r'}
# The separator is one of the two forms real jq actually emits — `jq-1.6` or `jq version 1.6` — not
# "any run of dashes and blanks". A permissive class let the pattern be walked across filler:
# `jq -- 2.34 (real jq-1.6)` parsed as 2.34, and `jq<TAB><TAB>9.9` as 9.9. A blank separator now
# REQUIRES the literal word `version`, which is the only context a real build puts one in.
#
# The trailing `([^0-9]|$)` is what actually bounds the digit runs. `{1,9}` alone does not: the regex
# is unanchored at the end, so `jq-1.99999999999999999999999` simply matched the first 9 digits of
# the minor and compared THAT — a mis-parse that passes the floor. Requiring a non-digit (or
# end-of-string) after the minor makes an over-long run fail to match at all, so it fails closed.
if [[ "$first" =~ ^[[:blank:]]*[Jj][Qq](-v?|[[:blank:]]+version[[:blank:]]+v?)([0-9]{1,9})\.([0-9]{1,9})([^0-9]|$) ]]; then
major="${BASH_REMATCH[2]}"
minor="${BASH_REMATCH[3]}"
else
echo "jq-preflight: could not parse a major.minor version out of '${first}'. Refusing to assert a floor against an unparsed version — that would silently pass." >&2
exit 1
fi
# THIS LINE IS THE POINT of the no-arg mode: the jq version CI actually used is in the job log, so a
# future divergence can be diagnosed from the log alone rather than by guessing at the runner image.
# `$first`, not `$raw`: a multi-line `--version` would split this across lines, breaking the single
# grep-able log line that is the entire point of the no-arg mode.
echo "jq-preflight: jq version in use = ${first} (parsed ${major}.${minor}; floor ${MIN_VERSION})"
min_major=${MIN_VERSION%%.*}
min_minor=${MIN_VERSION#*.}
if [ "$major" -lt "$min_major" ] || { [ "$major" -eq "$min_major" ] && [ "$minor" -lt "$min_minor" ]; }; then
echo "jq-preflight: jq ${major}.${minor} is BELOW the supported floor ${MIN_VERSION}. The gates in scripts/ and .gitea/workflows/ are written against ${MIN_VERSION}+ semantics and will misbehave silently on older builds." >&2
exit 1
fi
if [ -n "$expect" ]; then
if [ "${major}.${minor}" != "$expect" ]; then
echo "jq-preflight: expected jq ${expect}, found ${major}.${minor}." >&2
echo "" >&2
echo "This is a TRIPWIRE, not a defect in your change (ersatztv#648). scripts/tests/ was pinned to" >&2
echo "jq ${expect} because that is what this runner shipped; it now reports ${major}.${minor}. The ${expect}" >&2
echo "coverage the suite assumed has therefore just disappeared, silently — and jq 1.7 altered NUL" >&2
echo "handling, exit codes, @base64d and number precision, every one of which a gate here depends on." >&2
echo "" >&2
echo "Decide explicitly, then update the --expect value in .gitea/workflows/pr-checks.yml:" >&2
echo " * re-pin to the new version after re-reading docs/ci-cd.md -> 'The jq contract', or" >&2
echo " * add a real matrix leg that runs the suite under ${MIN_VERSION} as well." >&2
exit 1
fi
echo "jq-preflight: version matches the expected pin (${expect})."
fi
+37 -2
View File
@@ -92,6 +92,22 @@ pr_url=$(printf '%s' "$prjson" | jq -r '.html_url // ""')
[ "$pr_state" = "open" ] || die "PR #$pr is '$pr_state', not open — refusing to post a verdict"
short=${sha:0:7}
# --- Record the BASE BRANCH the verdict was formed against (ersatztv#632). ----------------------
# The sha binding closes "the head moved under a fixed verdict". It does not close the mirror case:
# RETARGETING a PR's base changes neither the head sha nor the status, yet changes the effective
# diff — so a verdict written while the PR targeted `main` still reads green after it is pointed at
# a branch with a very different merge-base. Consent outliving what it was granted for, reached from
# the other direction.
#
# The comparator is `base.ref` (the BRANCH NAME), deliberately NOT `base.sha`. `base.sha` tracks the
# base branch's tip, which moves every time anything merges to `main` — comparing it would invalidate
# every open verdict on every unrelated merge, i.e. a self-inflicted merge deadlock. `base.ref`
# changes exactly when someone retargets the PR, which is the event being guarded. A base branch that
# merely ADVANCES is out of scope by design: that is ordinary churn, and rebasing onto it changes the
# head sha, which the existing per-sha binding already catches.
base_ref=$(printf '%s' "$prjson" | jq -r '.base.ref // ""')
[ -n "$base_ref" ] || die "PR #$pr has no resolvable base branch (.base.ref) — refusing to post a verdict that cannot record what it was formed against"
# --- The comment (human-readable artifact + the hook's condition-(c) input). --------------------
# The verdict line MUST start the line: the hook anchors its parser to line-start precisely so a
# comment that merely QUOTES the template mid-sentence cannot self-approve a merge.
@@ -107,14 +123,33 @@ printf 'posted comment: Review-verdict: %s @ %s\n' "$verdict" "$short"
# a verdict written for its parent — reintroducing ersatztv#622 at a smaller time scale. We do NOT
# retry against the new head: the new commit is genuinely unreviewed, and silently re-targeting the
# verdict at it is exactly the failure this script exists to prevent.
sha_now=$(api_get "repos/$owner/$repo/pulls/$pr" | jq -r '.head.sha // ""')
# Fail CLOSED if the re-read itself fails. This used to be `sha_now=$(api_get ... | jq ...)`, where
# `set -e` + `pipefail` aborted the script on a failed GET — implicitly, but before any status was
# written. Folding the two reads into one variable with `|| true` would have swallowed that: both
# `sha_now` and `base_now` come back empty, both `[ -n … ]` guards become no-ops, and the status is
# written having confirmed NOTHING about the head or the base. That is a fail-open regression
# introduced by the refactor, so the refusal is now explicit rather than a side effect of `set -e`.
prjson_now=$(api_get "repos/$owner/$repo/pulls/$pr") \
|| die "could not re-read PR #$pr to confirm the head and base had not moved while posting — no status was written. Re-run once Gitea is reachable."
sha_now=$(printf '%s' "$prjson_now" | jq -r '.head.sha // ""')
if [ -n "$sha_now" ] && [ "$sha_now" != "$sha" ]; then
die "head moved from $short to ${sha_now:0:7} while posting — that commit is UNREVIEWED, so no status was written. Re-review the new head and run this again."
fi
# The same TOCTOU window applies to the base (ersatztv#632): a retarget between the read above and
# the status write below would bind the verdict to a base that is no longer the PR's, and the head
# sha check would not notice because retargeting does not move the head.
base_now=$(printf '%s' "$prjson_now" | jq -r '.base.ref // ""')
if [ -n "$base_now" ] && [ "$base_now" != "$base_ref" ]; then
die "base branch changed from '$base_ref' to '$base_now' while posting — the diff you reviewed is not the diff this PR now merges, so no status was written. Re-review against the new base and run this again."
fi
# The base branch goes in the status DESCRIPTION, not in the comment. The comment body is parsed by
# `scripts/check-review-verdict.sh`, whose grammar had three false-opens in its history; nothing
# parses the description today, so this adds a field without reopening that surface. The hook reads
# it back and compares (ersatztv#632).
status_payload=$(jq -n \
--arg s "$state" --arg c "$STATUS_CONTEXT" --arg u "$pr_url" \
--arg d "Review-verdict: $verdict @ $short" \
--arg d "Review-verdict: $verdict @ $short (base: $base_ref)" \
'{state:$s, context:$c, description:$d, target_url:$u}')
api_post "repos/$owner/$repo/statuses/$sha" "$status_payload" >/dev/null \
|| die "failed to post the '$STATUS_CONTEXT' commit status on $short"
+244
View File
@@ -0,0 +1,244 @@
#!/usr/bin/env bash
# Exhaustively enumerate a PR's changed file paths, or fail closed.
#
# ersatztv#649. This is the ONE implementation of the security-critical half of the merge gate.
# It exists because the same logic was written twice — once in `.claude/hooks/pretooluse-merge-consent.sh`
# (advisory: a failure produces a human prompt) and once in `.gitea/workflows/review-verdict.yml`
# (ENFORCED: it writes the branch-protection-required `review-verdict/h10` status). The advisory copy
# accumulated four rounds of hardening (ersatztv#643) that the enforced copy never received, leaving the
# copy with real authority strictly weaker than the copy without. Two copies of a security predicate
# drift; one cannot.
#
# SCOPE — mechanism, not policy. This script answers exactly one question: "what is the complete set of
# paths this PR touches, at one head, or can we not tell?" It deliberately does NOT classify the PR.
# The two callers' allow-lists differ ON PURPOSE and must stay separate:
# * the hook's docs-only pattern also lets .claude/ .gitea/ .husky/ through, which is safe there only
# because it falls through to a HUMAN PROMPT;
# * the workflow's is narrower, because there a match posts a green status with nobody in the loop.
# Sharing the enumeration fixes the drift; sharing the classification would erase an intended difference.
#
# CONTRACT
# Usage: pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha> <expected-base-ref>
# stdout: newline-delimited paths, BOTH sides of every rename, no blank lines. May be empty.
# exit 0 the enumeration is COMPLETE and bound to <expected-head-sha> AND <expected-base-ref>.
# stdout is authoritative.
# exit 1 the enumeration could NOT be completed or verified. stdout is meaningless — the caller
# MUST fail closed (withhold any exemption). A diagnostic goes to stderr.
# exit 2 usage error.
# Callers must treat any non-zero exit as "no exemption". Never read stdout without checking the status.
#
# WHY THE BASE REF IS AN ARGUMENT, AND WHY IT IS NOT OPTIONAL (ersatztv#698 route 1).
# `/pulls/{n}/files` computes the diff against the PR's **live** base, which is mutable. Retargeting a
# PR changes the enumerated file set without moving the head sha, so head-binding alone does not bind
# the ANSWER — only the commit it is nominally about. Reproduced live on this instance: a PR opened
# into `main` and retargeted mid-run to a scratch base enumerated as docs-only and was granted
# `review-verdict/h10=success`, while its diff against `main` carried a C# file (probe PR #703).
#
# REQUIRED rather than optional on purpose. An optional binding on a shared security primitive is an
# opt-out, and the caller that forgets it is precisely the caller that needed it — silently. Five
# arguments or exit 2.
#
# This NARROWS the window, it does not erase it. The base is re-read after the paging round trips
# alongside the head, so a retarget that is still in effect at that point fails closed; a retarget
# that opens and closes strictly between the files call and the re-read is not observable from here.
# Pinning the diff to two shas would close it, and Gitea 1.25.4 cannot serve that: `compare/{base}...
# {head}` returns `total_commits`/`commits` and NO `files`, and a `--depth=1` fetch of the two shas
# has no merge base, so a three-dot diff is impossible while a two-dot one over-reports every commit
# `main` gained since the branch point (both measured, #698). The remainder is covered one level up
# instead, by the workflow reclassifying on `edited` rather than trusting a machine-written success.
#
# AUTH/TRANSPORT is caller-supplied via env, because the two callers authenticate differently:
# ETV_GITEA_TOKEN | GITEA_TOKEN -> `Authorization: token`
# ETV_GITEA_BASICAUTH -> curl -u user:pass
# ETV_GITEA_URL | GITEA_BASE_URL -> API base; defaults to the homelab Gitea. A value ending in
# /api/v1 is used as-is, otherwise /api/v1 is appended.
#
# jq COMPATIBILITY (ersatztv#648). This runs on the CI runner, which ships **jq 1.6**, while it is
# authored on Macs shipping 1.8.x. It is therefore written to the 1.6-compatible subset:
# * never rely on `jq -e`'s exit status over EMPTY input — 1.6 exits 0 where >=1.7 exits 4, which is
# precisely the fail-open that ersatztv#647 found live in the enforced gate. Emptiness is always
# checked explicitly in shell FIRST.
# * never use `contains()` for substring tests — on 1.6 `contains("<NUL>")` is true for every string.
# * never distinguish a parse error from "no output" by exit code — 1.6 returns 4 for both.
# See docs/ci-cd.md -> "The jq contract".
set -euo pipefail
if [ "$#" -ne 5 ]; then
echo "usage: pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha> <expected-base-ref>" >&2
exit 2
fi
owner=$1
repo=$2
pr=$3
expected_sha=$4
expected_base=$5
if [ -z "$owner" ] || [ -z "$repo" ] || [ -z "$pr" ] || [ -z "$expected_sha" ] || [ -z "$expected_base" ]; then
echo "pr-changed-files: empty owner/repo/pr/sha/base argument" >&2
exit 2
fi
base_url="${ETV_GITEA_URL:-${GITEA_BASE_URL:-http://192.168.1.95:3000}}"
case "$base_url" in
*/api/v1) : ;;
*/) base_url="${base_url}api/v1" ;;
*) base_url="${base_url}/api/v1" ;;
esac
# Empty output on ANY failure, so every caller path treats a transport error the same way. The
# emptiness is then rejected explicitly below — never inferred from a jq exit code.
gq() {
local path="$1"
if [ -n "${ETV_GITEA_TOKEN:-}" ]; then
curl -sf -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/$path" 2>/dev/null || true
elif [ -n "${GITEA_TOKEN:-}" ]; then
curl -sf -H "Authorization: token $GITEA_TOKEN" "$base_url/$path" 2>/dev/null || true
elif [ -n "${ETV_GITEA_BASICAUTH:-}" ]; then
curl -sf -u "$ETV_GITEA_BASICAUTH" "$base_url/$path" 2>/dev/null || true
else
printf ''
fi
}
if [ -z "${ETV_GITEA_TOKEN:-}" ] && [ -z "${GITEA_TOKEN:-}" ] && [ -z "${ETV_GITEA_BASICAUTH:-}" ]; then
echo "pr-changed-files: no Gitea credentials in env — cannot enumerate, failing closed" >&2
exit 1
fi
# Bind the BASE before the first page is requested (ersatztv#698 route 1). Checking only afterwards
# would leave the common case — a PR retargeted before the enumeration even starts — indistinguishable
# from an honest one, because every page would agree with every other page while all of them described
# a diff against the wrong base. Both ends are checked; neither alone is sufficient.
prjson_before=$(gq "repos/$owner/$repo/pulls/$pr")
if [ -z "${prjson_before//[[:space:]]/}" ]; then
echo "pr-changed-files: could not read PR #$pr to bind the base ref before enumerating — failing closed" >&2
exit 1
fi
base_before=$(printf '%s' "$prjson_before" | jq -r '.base.ref // ""' 2>/dev/null || true)
if [ -z "$base_before" ] || [ "$base_before" != "$expected_base" ]; then
echo "pr-changed-files: PR #$pr targets '${base_before:-<unreadable>}', not the expected '$expected_base' — the diff would be computed against a different base, failing closed" >&2
exit 1
fi
PAGE_SIZE=50
MAX_PAGES=40 # 2000 files; beyond this we refuse rather than guess
files=""
page=1
complete=no
while [ "$page" -le "$MAX_PAGES" ]; do
raw=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=${PAGE_SIZE}&page=${page}")
# An EMPTY body is rejected in SHELL, before jq sees it. `jq -e` over empty input exits 4 on
# jq >= 1.7 but 0 on jq 1.6, and the runner ships 1.6 — leaving this to jq's exit status is the
# exact fail-open ersatztv#647 found in the enforced copy. A transport failure must never
# masquerade as a legitimate short final page.
if [ -z "${raw//[[:space:]]/}" ]; then
echo "pr-changed-files: empty/unreadable response for page ${page}" >&2
complete=no; break
fi
# VALIDATE EVERY FIELD THE EXTRACTION BELOW CONSUMES, on EVERY row.
#
# * Top-level type alone is not enough: `[{}]` is a well-formed array whose rows carry no
# `filename`, so it contributes no paths, looks like a short page, and would complete the
# enumeration from a PARTIAL list — the same failure one level down. It also rejects arrays of
# scalars, which would otherwise make the extraction fail under `set -e`.
# * CR/LF in a path is rejected outright. `chunk` flattens paths into newline-delimited text, so a
# filename containing a newline splits into TWO lines matched against the allow-list separately:
# "safe.md\ndocs/Program.cs" yields `safe.md` and `docs/Program.cs`, both of which pass, while the
# real single path ends in `.cs`. Git permits newlines in filenames, so this is reachable and was
# reproduced against the hook.
# * `previous_filename` is validated on EVERY row, not only `renamed` ones, because `chunk` emits it
# for every row regardless of `.status`. Validating it only where it is semantically "supposed to"
# appear left a hole one predicate wide: a `status: "modified"` row carrying a newline in
# `previous_filename` was reproducibly exempted. The validation domain must match the CONSUMPTION
# domain.
# * `..` is rejected because the callers' allow-lists anchor `^docs/`, so `docs/../ErsatzTV/Program.cs`
# matches one. Git will not produce such a path; this guard's job is to fail closed on unexpected
# 2xx shapes rather than assume a well-behaved peer.
# * `.status` is checked against a CLOSED set. Be precise about what this does and does not do:
# the extraction below emits `(.previous_filename // empty)` UNCONDITIONALLY, so a present
# `previous_filename` is never dropped on account of `.status`. What the closed set actually buys
# is rejecting rows whose vocabulary we do not recognise — where a source path may be absent, or
# carried in some other field we are not reading. Without it, `"Renamed"` with a capital R, or an
# absent status, silently takes the `else true` branch of the clause below and skips the
# "renamed rows MUST carry previous_filename" requirement entirely. (An earlier version of this
# comment claimed the source path would be "dropped", which is not the mechanism; a maintainer
# who tested that claim would find it false and might conclude the check is redundant.)
# `modified` is accepted alongside `changed` deliberately: live Gitea 1.25.4 emits `changed`, but a
# closed allow-list built from the wrong vocabulary is a worse failure than the hole it closes — it
# would gate every genuine docs-only PR on any version that spells it differently. The property is
# "reject values we do not recognise", not "enumerate one version exactly".
if ! printf '%s' "$raw" \
| jq -e 'def ok: type == "string" and length > 0
and (test("[\\r\\n]") | not)
and (split("/") | index("..") | not);
type == "array" and all(.[];
(.filename | ok)
and (.previous_filename == null or (.previous_filename | ok))
and ((.status // "") as $s | ($s | type) == "string"
and (["added","deleted","changed","modified","renamed","copied"] | index($s)) != null)
and (if .status == "renamed"
then (.previous_filename | type == "string" and length > 0)
else true end))' \
>/dev/null 2>&1; then
echo "pr-changed-files: page ${page} failed row validation" >&2
complete=no; break
fi
# BOTH sides of a rename: Gitea reports a `git mv` as ONE row whose `filename` is the DESTINATION,
# with the source only in `previous_filename`. Reading `filename` alone lets a PR move a protected
# file INTO docs/ and pass as docs-only (verified live: `.gitea/workflows/renovate.yml` ->
# `docs/innocuous-note.md` showed no protected path). One renamed row is therefore ONE row but TWO
# paths, which is why the two counts below are computed differently.
n=$(printf '%s' "$raw" | jq -r 'length')
chunk=$(printf '%s' "$raw" | jq -r '.[] | (.filename // empty), (.previous_filename // empty)')
[ -n "$chunk" ] && files=$(printf '%s\n%s' "$files" "$chunk")
# Terminate ONLY on an explicitly validated EMPTY page — never on a merely SHORT one.
# "Fewer than 50 rows means last page" assumes the server's page size is the 50 we asked for, but
# Gitea caps `limit` at the server-wide MAX_RESPONSE_ITEMS (default 50, configurable) and is free to
# return fewer. A 30-row page followed by a page of code would complete the enumeration over a
# PARTIAL list — the same fail-open, reached without any transport error. Costs one extra request;
# the MAX_PAGES cap still fails closed.
if [ "$n" -eq 0 ]; then complete=yes; break; fi
page=$((page + 1))
done
if [ "$complete" != yes ]; then
echo "pr-changed-files: enumeration incomplete (stopped at page ${page}) — failing closed" >&2
exit 1
fi
# Bind the enumeration to ONE head. Paging is several round-trips; a force-push between them means
# page 1 came from head A and page 2 from head B, so the assembled list belongs to no single commit —
# B's code page can be skipped entirely while B's docs page reads as a clean short tail. Re-read the
# head and refuse if it moved.
prjson=$(gq "repos/$owner/$repo/pulls/$pr")
if [ -z "${prjson//[[:space:]]/}" ]; then
echo "pr-changed-files: could not re-read PR head to bind the enumeration — failing closed" >&2
exit 1
fi
sha_after=$(printf '%s' "$prjson" | jq -r '.head.sha // ""' 2>/dev/null || true)
if [ -z "$sha_after" ] || [ "$sha_after" != "$expected_sha" ]; then
echo "pr-changed-files: head moved during enumeration (${expected_sha:0:7} -> ${sha_after:0:7}) — failing closed" >&2
exit 1
fi
# The same round-trip window applies to the BASE, and the head check cannot see it: retargeting moves
# the diff without moving the head sha (ersatztv#698 route 1). Comparing `.base.ref` — the branch NAME,
# never its tip — is deliberate and matches `post-review-verdict.sh` (ersatztv#632): a base that merely
# ADVANCES is ordinary churn, while comparing tips would fail every enumeration on every unrelated
# merge to `main`.
base_after=$(printf '%s' "$prjson" | jq -r '.base.ref // ""' 2>/dev/null || true)
if [ -z "$base_after" ] || [ "$base_after" != "$expected_base" ]; then
echo "pr-changed-files: base moved during enumeration ('$expected_base' -> '${base_after:-<unreadable>}') — the enumerated diff is against a base this PR no longer targets, failing closed" >&2
exit 1
fi
printf '%s\n' "$files" | grep -v '^$' || true
exit 0
+349
View File
@@ -0,0 +1,349 @@
"""Tests for `scripts/jq-preflight.sh` — the jq version contract (ersatztv#648).
The axis this guards. Every shell gate in this repo is authored on a Mac shipping jq 1.8.x; the CI
runner ships jq 1.6. Nothing pinned or checked that, and three independent divergences surfaced in a
single day `jq -e` over empty input (exit 4 vs 0), `contains("<NUL>")` (false vs true for every
string), and the parse-error exit code (5 vs 4, colliding with "no output"). Each was patched with a
version-stable construct, but patching constructs one at a time leaves the AXIS untested.
These tests shim `jq` on PATH with a fake reporting an arbitrary version, so the preflight's own
behaviour is verified by MEASUREMENT rather than by observing a green CI tick ersatztv#648's third
Done-when box. Doing it here rather than by pushing a deliberately-red commit also keeps the proof
reproducible: it re-runs on every PR instead of living in one CI run's history.
"""
from __future__ import annotations
import os
import shutil
import subprocess
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[2]
SCRIPT = REPO_ROOT / "scripts" / "jq-preflight.sh"
WORKFLOWS = REPO_ROOT / ".gitea" / "workflows"
# Resolved BEFORE PATH is narrowed to the shim dir — the tests strip PATH down to just that
# directory, so `bash` could not be found by name from inside them.
BASH = shutil.which("bash") or "/bin/bash"
def _shq(s):
"""Single-quote a string for /bin/sh."""
return "'" + s.replace("'", "'\\''") + "'"
@pytest.fixture
def preflight(tmp_path):
bindir = tmp_path / "bin"
bindir.mkdir()
class Handle:
def with_jq(self, version_line, stderr="", exit_code=0):
"""Install a fake `jq` reporting `version_line` for --version.
`stderr` and `exit_code` exist because an earlier version of this shim ALWAYS exited 0
and never wrote to stderr so it structurally could not observe the worst failure this
script has: a jq that cannot start. The preflight was folding stderr into the parse via
`2>&1` and discarding the exit status, so a glibc-mismatch message containing `2.34`
parsed as version 2.34 and PASSED the floor. Every case the shim could express was clean,
so every test passed.
"""
shim = bindir / "jq"
body = "#!/bin/sh\nif [ \"$1\" = \"--version\" ]; then\n"
if version_line:
body += ' printf "%%s\\n" %s\n' % _shq(version_line)
if stderr:
body += ' printf "%%s\\n" %s >&2\n' % _shq(stderr)
body += " exit %d\nfi\nexit 0\n" % exit_code
shim.write_text(body)
shim.chmod(0o755)
def without_jq(self):
shim = bindir / "jq"
if shim.exists():
shim.unlink()
def run(self, *args):
env = dict(os.environ)
# PATH contains ONLY the shim dir. An earlier draft appended /usr/bin:/bin "for the
# basics" and the missing-jq test passed vacuously against the developer machine's real
# /usr/bin/jq — the negative case was never negative. The script needs nothing from PATH
# but jq itself (`command -v` is a builtin, and bash is invoked by absolute path), so
# there is nothing to keep.
env["PATH"] = str(bindir)
return subprocess.run([BASH, str(SCRIPT), *args],
env=env, capture_output=True, text=True)
def run_bytes(self, *args):
"""Same, but WITHOUT text mode.
`text=True` enables universal-newlines translation, which rewrites `\\r` to `\\n` in the
captured output so any assertion about a stray carriage return is unfalsifiable through
`run()`. That is not hypothetical: the CR test passed identically with the strip removed
until this was noticed, while the mutant demonstrably emits
`... = jq-1.6\\r (parsed 1.6; ...)` at the byte level.
"""
env = dict(os.environ)
env["PATH"] = str(bindir)
return subprocess.run([BASH, str(SCRIPT), *args],
env=env, capture_output=True)
return Handle()
def test_the_version_is_printed_so_the_job_log_shows_it(preflight):
"""ersatztv#648's second Done-when box: the jq version CI actually uses must be OBSERVABLE."""
preflight.with_jq("jq-1.6")
r = preflight.run()
assert r.returncode == 0, r.stderr
assert "jq-1.6" in r.stdout
def test_floor_mode_accepts_the_runner_version(preflight):
preflight.with_jq("jq-1.6")
assert preflight.run().returncode == 0
def test_floor_mode_accepts_a_newer_jq(preflight):
"""No upper bound in floor mode — review-verdict.yml writes the REQUIRED merge check, so a jq
bump must never be able to deadlock every merge in the repo."""
preflight.with_jq("jq-1.8.2")
assert preflight.run().returncode == 0
def test_below_the_floor_is_LOUD(preflight):
preflight.with_jq("jq-1.5")
r = preflight.run()
assert r.returncode == 1
assert "below the supported floor" in r.stderr.lower()
def test_missing_jq_is_loud(preflight):
preflight.without_jq()
r = preflight.run()
assert r.returncode == 1
assert "not on PATH" in r.stderr
@pytest.mark.parametrize("version_line", ["jq-1.6-dirty", "jq-1.6", "jq-1.6.0"])
def test_build_suffixes_still_parse_as_1_6(preflight, version_line):
"""A packaging suffix must not fail a perfectly ordinary jq closed — that would be a tripwire
firing on noise, which is how tripwires get disabled."""
preflight.with_jq(version_line)
assert preflight.run("--expect", "1.6").returncode == 0, version_line
def test_expect_mismatch_is_LOUD(preflight):
"""THE TRIPWIRE. scripts/tests exercises the jq 1.6 path only because the runner ships 1.6. If
the runner were upgraded that coverage would vanish silently, so the pin must go red instead."""
preflight.with_jq("jq-1.7.1")
r = preflight.run("--expect", "1.6")
assert r.returncode == 1
assert "expected jq 1.6, found 1.7" in r.stderr
def test_expect_match_passes(preflight):
preflight.with_jq("jq-1.6")
assert preflight.run("--expect", "1.6").returncode == 0
def test_unknown_argument_is_a_usage_error(preflight):
preflight.with_jq("jq-1.6")
assert preflight.run("--pin", "1.6").returncode == 2
def test_expect_without_a_value_is_a_usage_error_WITH_output(preflight):
"""`shift 2` on a missing value exits 1 under `set -e` with NOTHING on either stream. A CI step
that dies with an empty log is the diagnostic hole this script exists to remove."""
preflight.with_jq("jq-1.6")
r = preflight.run("--expect")
assert r.returncode == 2
assert "requires a <major.minor> value" in r.stderr
# --- Version parsing: the guard must never assert a floor against an unparsed version ----------
#
# The original strip-based parse assumed the format is exactly `jq-X.Y`. Anything else left major or
# minor EMPTY, and the sanity check concatenated them — so `jq version 1.6` produced "6", which is
# non-empty and all-digits, so the check PASSED. The floor comparison then ran `[ "" -lt 1 ]`, which
# errors; `set -e` exempts a failing command in an `if` condition, so the conditional read false and
# the script exited 0 having asserted NOTHING. That is this script's own stated failure mode,
# reproduced inside itself, which is why these cases are pinned rather than left to inspection.
@pytest.mark.parametrize("version_line", [
"jq version 1.6", # some distro wrappers print this form
"JQ-1.6",
"jq-1.6-dirty",
])
def test_unusual_but_parseable_version_forms_are_accepted(preflight, version_line):
preflight.with_jq(version_line)
r = preflight.run()
assert r.returncode == 0, f"{version_line!r}: {r.stderr}"
assert "parsed 1.6" in r.stdout
@pytest.mark.parametrize("version_line", ["jq-1.-6", "jq-.6", "not-a-version", ""])
def test_unparseable_version_fails_CLOSED_rather_than_asserting_nothing(preflight, version_line):
preflight.with_jq(version_line)
r = preflight.run()
assert r.returncode == 1, (
f"{version_line!r} exited {r.returncode}: an unparsed version must never reach — or "
"silently skip — the floor assertion")
assert "could not parse" in r.stderr
def test_a_jq_that_cannot_START_fails_closed(preflight):
"""THE case the previous shim could not express, and the guard therefore got wrong.
A jq broken by a glibc mismatch (the canonical post-base-image-bump failure) exits 127 and writes
`... version 'GLIBC_2.34' not found` to STDERR. The preflight was reading `jq --version 2>&1` and
discarding the exit status, so that message became the parse input, `2.34` matched, and the floor
was certified green on a jq that cannot run at all.
"""
preflight.with_jq(
"", stderr="jq: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found",
exit_code=127)
r = preflight.run()
assert r.returncode == 1
assert "cannot run" in r.stderr
assert "parsed 2.34" not in r.stdout, "stderr must never be parsed as a version"
@pytest.mark.parametrize("version_line", [
"warning: something 3.14", # a noise line carrying a plausible number
"2026.07.26 jq-1.6", # a date prefix, which outranks the real version if unanchored
"jq-master-v0.0.0-1.6",
])
def test_a_number_that_is_not_the_VERSION_is_not_accepted_as_one(preflight, version_line):
"""Matching the first `<digits>.<digits>` ANYWHERE let a prefix win over the real version.
`2026.07.26 jq-1.6` parsed as 2026.07 and sailed over the floor. The pattern is anchored to the
leading `jq` token, so these fail closed instead."""
preflight.with_jq(version_line)
r = preflight.run()
assert r.returncode == 1, f"{version_line!r} was accepted as a version"
assert "could not parse" in r.stderr
@pytest.mark.parametrize("version_line", [
"jq-99999999999999999999999.0",
"jq-1.99999999999999999999999",
])
def test_an_OUT_OF_RANGE_digit_run_fails_closed(preflight, version_line):
"""The round-1 fail-open mechanism, resurrected via an over-long number.
A regex that guarantees *digits* does not guarantee they fit `test`'s integer range. With a
23-digit major, `[ "$major" -lt "$min_major" ]` errors with "integer expression expected" and
`set -e` exempts a failing command in an `if` condition, so the conditional read false and THE
FLOOR WAS NEVER ASSERTED, exit 0. Identical in shape to the empty-string case that started this.
Bounding the run with `{1,9}` alone was NOT enough either: the pattern is unanchored at the end,
so an over-long minor just matched its first 9 digits and compared that instead a mis-parse
that passes. The trailing non-digit requirement is what actually closes it.
"""
preflight.with_jq(version_line)
r = preflight.run()
assert r.returncode == 1, f"{version_line!r} exited 0 — the floor was not asserted"
@pytest.mark.parametrize("version_line", [
# Killed by the SEPARATOR restriction (a blank separator must be followed by `version`).
"jq\n2.34: cannot load shared library",
"jq\n\n\n99.9",
"jq -- 2.34 (real jq-1.6)",
"jq\t\t9.9",
# Killed ONLY by the first-line slice + `[[:blank:]]`. These carry the literal word `version`,
# so the separator restriction is satisfied and cannot save us — the newline must be excluded
# from the separator class AND the parse confined to line one.
#
# Without these, a round-5 mutation check found that reverting BOTH of those changes together
# (`[[:blank:]]`→`[[:space:]]` and parsing `$raw` instead of `$first`) left the whole suite
# GREEN: the four cases above are all killed by the separator alone, so they attributed the fix
# to the wrong layer. A test that passes for the wrong reason is how the previous three rounds
# each shipped a defect.
"jq\nversion\n9.9",
"jq\nversion 9.9",
"jq \n version \n 9.9",
])
def test_a_number_AFTER_the_jq_token_is_not_reachable_across_filler(preflight, version_line):
"""Two independent layers keep a stray number from being read as the version, and both are
pinned here: the separator must be one of the forms real jq emits (`jq-1.6` / `jq version 1.6`),
AND the match is confined to the first line with `[[:blank:]]` (which, unlike `[[:space:]]`,
does not match a newline). Round 3's 'anchor' had neither and parsed `jq\\n2.34: cannot load`
as 2.34."""
preflight.with_jq(version_line)
r = preflight.run()
assert r.returncode == 1, f"{version_line!r} was accepted as a version"
def test_a_CRLF_version_line_parses_and_logs_without_the_carriage_return(preflight):
"""The trailing `\\r` strip was unpinned — the commit claimed CRLF was verified, but nothing in
the suite contained one. Harmless today (a `\\r` satisfies the trailing non-digit boundary, so
the version still parses) but the log line would carry a stray CR."""
preflight.with_jq("jq-1.6\r")
r = preflight.run_bytes()
assert r.returncode == 0, r.stderr
assert b"parsed 1.6" in r.stdout
# Two separate traps had to be cleared for this assertion to mean anything:
# 1. `str.splitlines()` also splits on `\r`, so inspecting the "version in use" line would drop
# the stray CR before the assertion could see it;
# 2. `subprocess.run(text=True)` translates `\r` to `\n` outright, so even raw-string checks on
# `r.stdout` were unfalsifiable.
# Both made the test pass identically with the strip removed. Hence `run_bytes()` and a bytes
# comparison — verified by mutation, not by reading the code.
assert b"\r" not in r.stdout, "the carriage return leaked into the log line"
def test_the_observability_line_stays_on_ONE_line(preflight):
"""The no-arg mode exists to put a single grep-able version line in the job log; interpolating a
multi-line `--version` would split it."""
preflight.with_jq("jq-1.6\ntrailing noise")
r = preflight.run()
assert r.returncode == 0, r.stderr
version_lines = [ln for ln in r.stdout.splitlines() if "version in use" in ln]
assert len(version_lines) == 1
assert "trailing noise" not in r.stdout
@pytest.mark.parametrize("version_line,expected", [
("jq-1.6 (Debian 1.6-2.1)", "1.6"), # distro packaging suffix
("jq-1.10", "1.10"), # two-digit minor: must compare numerically, not lexically
("jq-1.7.1", "1.7"),
("jq-1.6.0", "1.6"),
("jq-v1.6", "1.6"),
("JQ-1.6", "1.6"),
])
def test_legitimate_forms_still_parse_to_the_right_version(preflight, version_line, expected):
preflight.with_jq(version_line)
r = preflight.run()
assert r.returncode == 0, f"{version_line!r}: {r.stderr}"
assert f"parsed {expected}" in r.stdout
# --- Wiring guards: the preflight is worthless if a caller silently stops running it ------------
def test_script_tests_pins_the_jq_version():
"""The pin is the tripwire, so its presence is asserted rather than merely commented.
SCOPE NOTE the symmetric assertion about `review-verdict.yml` (that it runs the FLOOR-only
mode and must never pin, because it writes the branch-protection-required `review-verdict/h10`
status and a pin would deadlock every merge on a jq bump) lands with the follow-up PR that
wires that workflow. It cannot land here: that workflow checks out the BASE ref, and the base
is `main`, which does not yet contain `scripts/jq-preflight.sh`.
"""
pr_checks = (WORKFLOWS / "pr-checks.yml").read_text()
assert "jq-preflight.sh --expect" in pr_checks, \
"script-tests must pin the jq version — that pin is the tripwire"
def test_review_verdict_never_pins_a_jq_version():
"""Whatever else changes, the REQUIRED merge check must never carry a hard version pin.
Asserted now, before the workflow is wired, so the constraint is already enforced when the
follow-up PR adds the floor-only call rather than being a comment someone can miss.
"""
review_verdict = (WORKFLOWS / "review-verdict.yml").read_text()
assert "jq-preflight.sh --expect" not in review_verdict, \
("review-verdict.yml must NOT pin a jq version: it writes the required review-verdict/h10 "
"status, so a pin would deadlock every merge on a jq bump (ersatztv#648)")
@@ -0,0 +1,206 @@
"""Tests for the base-change detection in `.claude/hooks/pretooluse-merge-consent.sh` (#632).
`review-verdict/h10` is a per-sha commit status, which makes "a new commit inherits an old verdict"
impossible by construction (#622). Retargeting a PR's base reaches the same end by the opposite
route: the head sha does not move, so the status stays green, while the merge-base and therefore
the effective diff the verdict was formed against changes underneath it.
What is asserted here is DETECTION on the hook path only, and the tests are written to keep that
claim narrow:
* a status carries no base field of its own, so the server-side required check cannot see this at
all; a merge driven through the Gitea UI or API is unaffected. No test here implies otherwise.
* a verdict posted before #632 has no `(base: …)` in its description and must get NO opinion,
rather than denying every in-flight PR the day this lands.
Observable contract: the hook exits 0 with EMPTY stdout when it has no opinion (passthrough to
normal permissioning), and emits a JSON `permissionDecision` otherwise.
"""
from __future__ import annotations
import json
import os
import subprocess
from pathlib import Path
import pytest
REPO_ROOT = Path(__file__).resolve().parents[2]
HOOK = REPO_ROOT / ".claude" / "hooks" / "pretooluse-merge-consent.sh"
SHA = "a9e3e23abf337980ca4c05854f5b1e210099d08b"
# The PR is deliberately NOT docs-only: the docs-only exemption short-circuits the whole gate, so a
# docs PR would never reach the base check and the tests would pass without exercising it.
CURL_SHIM = r'''#!/usr/bin/env python3
import json, os, sys, pathlib, urllib.parse
state = pathlib.Path(os.environ["STUB_DIR"])
args = sys.argv[1:]
url = [a for a in args if a.startswith("http")][-1]
if "/pulls/" in url and "/files" in url:
q = urllib.parse.parse_qs(urllib.parse.urlparse(url).query)
page = int(q.get("page", ["1"])[0])
if page == 1:
print(json.dumps([{"filename": "ErsatzTV/Program.cs", "status": "modified"}]))
else:
print("[]")
sys.exit(0)
if "/status" in url:
desc = (state / "verdict_desc").read_text()
if desc == "TRANSPORT-ERROR":
sys.exit(22)
if desc == "GARBAGE":
print('{"message":"internal error"}'); sys.exit(0)
if desc == "SCALAR-ROW":
print('{"state":"success","statuses":[1]}'); sys.exit(0)
if desc == "NONSTRING-DESC":
print(json.dumps({"state": "success", "statuses": [
{"context": "review-verdict/h10", "status": "success", "description": {"x": 1}}]}))
sys.exit(0)
rows = [] if desc == "NONE" else [
{"context": "review-verdict/h10", "status": "success", "description": desc}]
print(json.dumps({"state": "success", "statuses": rows}))
sys.exit(0)
if "/pulls/" in url:
body = {"head": {"sha": os.environ["STUB_SHA"]}, "body": "fixes #1"}
live = (state / "live_base").read_text().strip()
if live != "MISSING":
body["base"] = {"ref": live}
print(json.dumps(body))
sys.exit(0)
print("{}")
'''
@pytest.fixture
def hook(tmp_path):
bindir = tmp_path / "bin"; bindir.mkdir()
curl = bindir / "curl"; curl.write_text(CURL_SHIM); curl.chmod(0o755)
state = tmp_path / "state"; state.mkdir()
(state / "live_base").write_text("main")
(state / "verdict_desc").write_text("Review-verdict: MERGEABLE @ a9e3e23 (base: main)")
env = dict(os.environ)
env["PATH"] = f"{bindir}{os.pathsep}{env['PATH']}"
env["STUB_DIR"] = str(state)
env["STUB_SHA"] = SHA
env["ETV_GITEA_TOKEN"] = "stub"
env["ETV_GITEA_URL"] = "http://gitea.example"
env.pop("ETV_GITEA_BASICAUTH", None)
class Handle:
def set_live_base(self, ref):
(state / "live_base").write_text(ref)
def set_verdict_description(self, desc):
"""'NONE' serves a head with no review-verdict/h10 status at all."""
(state / "verdict_desc").write_text(desc)
def decision(self):
payload = {"tool_input": {"method": "merge", "owner": "timothy",
"repo": "ersatztv", "pull_number": 42}}
r = subprocess.run(["bash", str(HOOK)], input=json.dumps(payload),
env=env, capture_output=True, text=True)
assert r.returncode == 0, r.stderr
if not r.stdout.strip():
return None
return json.loads(r.stdout)
def reason(self):
d = self.decision()
return "" if d is None else json.dumps(d)
return Handle()
def test_a_retargeted_base_denies_a_verdict_formed_against_the_old_one(hook):
hook.set_live_base("release/26.4")
reason = hook.reason()
assert "deny" in reason, "a verdict formed against a different base was allowed to stand"
assert "release/26.4" in reason and "main" in reason, (
"the deny must name both bases; a reader cannot act on 'the base changed'")
def test_positive_control_an_unchanged_base_does_not_trigger_the_base_deny(hook):
"""Without this, the test above could pass because the hook denies on every path — which it
very nearly does, since this PR is non-docs and the rest of the gate is unstubbed."""
reason = hook.reason()
assert "ersatztv#632" not in reason, (
"the base check fired on a PR whose base never moved")
@pytest.mark.parametrize("desc", [
"Review-verdict: MERGEABLE @ a9e3e23", # posted before #632
"NONE", # no verdict status on this head at all
])
def test_a_verdict_with_no_recorded_base_gets_no_opinion(hook, desc):
"""Graceful adoption. Denying here would block every in-flight PR the day this lands, and the
window closes on its own: verdicts are per-head and short-lived, so every verdict posted after
#632 carries the field.
Asserting on the word "base" rather than on the issue tag, per cold review: the tag-only check
would have passed for a base-specific ask or deny whose wording happened to omit it, which is
the failure mode most likely to appear when someone edits these messages.
"""
hook.set_live_base("release/26.4")
hook.set_verdict_description(desc)
assert "base" not in hook.reason(), (
"a pre-#632 verdict drew a base-related decision for a field it could not have carried")
@pytest.mark.parametrize("failure", ["SCALAR-ROW", "NONSTRING-DESC"])
def test_a_malformed_status_MEMBER_asks_too(hook, failure):
"""One level below the previous fix, and it survived it.
Validating only that `.statuses` is an array left `{"statuses":[1]}` passing the guard, after
which `.context` on a number errors and a `|| true` on the extraction turned that error into an
empty description straight back onto the graceful-adoption path, which is precisely the
outcome the guard exists to distinguish from. Same swallow-the-error shape as the bug one level
up, which is why the validation domain must match the CONSUMPTION domain rather than stopping at
the top-level type.
"""
hook.set_live_base("release/26.4")
hook.set_verdict_description(failure)
reason = hook.reason()
assert "ask" in reason and "base" in reason
@pytest.mark.parametrize("failure", ["TRANSPORT-ERROR", "GARBAGE"])
def test_an_UNREADABLE_status_response_asks_rather_than_skipping_the_check(hook, failure):
""""Could not check" is a third outcome, not a quiet synonym for "no base recorded".
The first draft collapsed the two: an unreadable status response produced an empty
`recorded_base`, took the graceful-adoption path, and skipped validation in silence after
which a later successful status read could still auto-grant, emitting "merge gate: satisfied"
for a comparison that never happened. A transient Gitea hiccup is not evidence that the base is
unchanged.
"""
hook.set_live_base("release/26.4")
hook.set_verdict_description(failure)
reason = hook.reason()
assert "ask" in reason, "an unreadable status response silently skipped the base check"
assert "base" in reason, "the ask must name what could not be checked"
def test_a_pr_with_no_resolvable_base_asks(hook):
"""A null/absent `.base.ref` is also 'could not check', not 'nothing to check'."""
hook.set_live_base("MISSING")
reason = hook.reason()
assert "ask" in reason and "base" in reason
def test_the_comparator_is_the_base_REF_not_its_tip_sha():
"""The design decision this test exists to freeze. `base.sha` tracks the base branch's TIP,
which moves every time anything merges to `main` comparing that would invalidate every open
verdict on every unrelated merge, turning a rare-event guard into a permanent merge deadlock.
A base branch that merely ADVANCES must be silent here; rebasing onto it moves the head sha,
which the per-sha binding already covers."""
assert ".base.ref" in HOOK.read_text(), "the hook must compare the base BRANCH, not its tip sha"
assert ".base.sha" not in HOOK.read_text(), (
"comparing base.sha deadlocks every open PR whenever main advances")
+42 -1
View File
@@ -64,7 +64,15 @@ if "/pulls/" in url:
ctr.write_text(str(nread + 1))
if alt.exists() and nread >= 1:
shas = [alt.read_text().strip()]
print(json.dumps({"head": {"sha": shas[0]}, "body": "no linked issue here"}))
# `.base.ref` is served because the hook now reads it and threads it to the enumeration as the
# required 5th argument (ersatztv#698 route 1). Without it the hook passes an empty base, the
# script exits 2, and EVERY docs-only exemption silently stops being granted — which is exactly
# how this stub failed when the argument was added: the eight failures were all positive cases.
# Fail-closed, so not dangerous, but it would have made the advisory hook prompt on every
# docs-only PR.
print(json.dumps({"head": {"sha": shas[0]},
"base": {"ref": os.environ.get("STUB_BASE", "main")},
"body": "no linked issue here"}))
sys.exit(0)
print("{}")
@@ -495,3 +503,36 @@ def test_array_valued_status_does_not_dodge_the_allow_list(hook):
def test_object_valued_status_is_also_rejected(hook):
hook.set_pages([{"filename": "docs/a.md", "status": {"x": "renamed"}}], [])
assert hook.exempted() is False
# --- The `grep -q` / pipefail inversion, on the ADVISORY side (ersatztv#698) --------------------
#
# Round-2 cross-family review noted the enforced gate gained large-input regression tests while the
# hook — which carries the SAME predicate — did not. The hook's blast radius is smaller (a missing
# prompt, not a green required check), but `ci.shared-pr-file-enumeration` exists precisely because
# the copy with LESS authority is the one that quietly keeps a bug. So test both.
#
# `grep -q` exits at its first match; the producer then takes SIGPIPE (141) once the path list exceeds
# the pipe buffer, and under `set -o pipefail` a MATCH is reported as a FAILED pipeline — inverting the
# negated docs-only test. ~171KB is needed to cross the threshold; every other test in this file uses a
# handful of short paths, which is exactly why the class was invisible here.
def _many_docs(n=1900):
return [f"docs/{'d' * 40}-{i:040d}.md" for i in range(n)]
def test_a_LARGE_pr_containing_a_code_file_is_NOT_exempt(hook):
"""The code file goes FIRST so the guard matches immediately and the producer is left with the
bulk of ~171KB still to write."""
hook.set_pages(_rows(["A.cs", *_many_docs()]))
assert hook.exempted() is False, (
"a large PR containing A.cs was granted the docs-only exemption — the predicate inverted")
def test_positive_control_a_LARGE_genuinely_docs_only_pr_IS_still_exempt(hook):
"""Guards the opposite failure: if large lists merely errored, the test above would pass while the
hook prompted on every big docs PR. Without this, 'fixed' and 'broken' are indistinguishable."""
hook.set_pages(_rows(_many_docs()))
assert hook.exempted() is True, (
"a large but genuinely docs-only PR lost its exemption")
+76 -2
View File
@@ -64,11 +64,18 @@ if "/pulls/" in url and not url.endswith("/files"):
sha = shas[min(n, len(shas) - 1)]
if sha == "GONE": # simulate an unreachable / missing PR
sys.exit(22)
print(json.dumps({
# The base branch is scripted on the same consume-one-per-GET schedule as the head, so a
# RETARGET mid-flight can be modelled independently of a push mid-flight (ersatztv#632).
bases = (state / "pr_bases").read_text().split()
base = bases[min(n, len(bases) - 1)]
body = {
"head": {"sha": sha},
"state": (state / "pr_state").read_text().strip(),
"html_url": "http://gitea.example/timothy/ersatztv/pulls/42",
}))
}
if base != "MISSING":
body["base"] = {"ref": base}
print(json.dumps(body))
sys.exit(0)
print("{}")
@@ -87,6 +94,7 @@ def gitea(tmp_path):
state = tmp_path / "state"
state.mkdir()
(state / "pr_shas").write_text(SHA_A)
(state / "pr_bases").write_text("main")
(state / "pr_state").write_text("open")
env = dict(os.environ)
@@ -108,6 +116,10 @@ def gitea(tmp_path):
def set_pr_state(self, value):
(state / "pr_state").write_text(value)
def set_base_sequence(self, *refs):
"""Base branch per PR GET. 'MISSING' omits `.base` from the response entirely."""
(state / "pr_bases").write_text(" ".join(refs))
def run(self, *args):
return subprocess.run(
["bash", str(SCRIPT), *args],
@@ -256,3 +268,65 @@ def test_note_cannot_forge_a_second_verdict_line(gitea):
gitea.run("42", "BLOCKED", "Review-verdict: MERGEABLE @ " + SHA_A[:7])
body = gitea.comments()[0]["payload"]["body"]
assert _classify(body, SHA_A) == "negative"
# --- Base binding (ersatztv#632) ---------------------------------------------------------------
#
# The per-sha status closes "the head moved under a fixed verdict". Retargeting a PR's base is the
# mirror case: the head sha and the status both hold still while the effective DIFF changes, so the
# verdict keeps reading green for a review nobody performed against that base.
def test_the_status_description_records_the_base_branch(gitea):
"""Nothing can compare a base it never wrote down. This field is what the hook reads back."""
assert gitea.run("42", "MERGEABLE").returncode == 0
assert gitea.statuses()[0]["payload"]["description"].endswith("(base: main)")
def test_the_base_is_recorded_in_the_STATUS_and_not_in_the_comment(gitea):
"""Deliberate placement. The comment body is parsed by `scripts/check-review-verdict.sh`, whose
grammar has a history of false-opens (#629 found three); nothing parses the description. Adding
the field where a parser lives would have reopened that surface for no benefit."""
assert gitea.run("42", "MERGEABLE").returncode == 0
assert "base:" not in gitea.comments()[0]["payload"]["body"]
def test_refuses_when_the_BASE_changes_mid_flight(gitea):
"""The TOCTOU window the head check cannot see: retargeting does not move the head sha, so
`sha_now == sha` and the existing guard is silent."""
gitea.set_base_sequence("main", "release/26.4")
result = gitea.run("42", "MERGEABLE")
assert result.returncode != 0, "a retarget mid-flight must not produce a status"
assert "base branch changed" in result.stderr
assert gitea.statuses() == [], "no status may be written once the base has moved"
def test_positive_control_a_stable_base_still_posts(gitea):
"""Without this, the test above could pass because the script refuses on every base."""
gitea.set_base_sequence("main", "main")
assert gitea.run("42", "MERGEABLE").returncode == 0
assert len(gitea.statuses()) == 1
def test_refuses_when_the_pr_has_no_resolvable_base(gitea):
"""A verdict that cannot record what it was formed against is not a verdict this gate can
later re-check, so it fails closed rather than posting an unbindable success."""
gitea.set_base_sequence("MISSING")
result = gitea.run("42", "MERGEABLE")
assert result.returncode != 0
assert gitea.statuses() == []
def test_a_failed_HEAD_RECHECK_writes_no_status(gitea):
"""Fail-closed on the re-read itself, not just on a moved head.
This guard was previously implicit: `sha_now=$(api_get ... | jq ...)` aborted under `set -e` +
`pipefail` when the GET failed. Nothing asserted it, so folding the head and base re-reads into
one `$(... || true)` variable silently converted it to fail-OPEN both guards see an empty
string, both no-op, and the status is written having confirmed nothing. Asserted now so the
behaviour is a contract rather than a side effect of a shell option.
"""
gitea.set_head_sequence(SHA_A, "GONE")
result = gitea.run("42", "MERGEABLE")
assert result.returncode != 0
assert gitea.statuses() == [], (
"a status was written even though the head/base re-read failed — nothing was confirmed")
File diff suppressed because it is too large Load Diff
+9
View File
@@ -11,6 +11,15 @@ REPO_ROOT="$(pwd)"
# dotnet-getdocument against ErsatzTV.dll + ErsatzTV.deps.json, which don't exist in a clean
# tree (e.g. the CI api-docs job, which only restores). Without the build the target fails with
# "The specified deps.json … does not exist" (exit 129). Build first, then generate.
#
# LOCAL-DEV SHARP EDGE: if the project is ALREADY built and nothing changed, MSBuild skips the
# document-generation work but still runs RenameOpenApiFiles (AfterTargets), whose Move then fails
# with MSB3680 "ErsatzTV.json does not exist" — because nothing produced it. The script correctly
# exits non-zero, but a caller that pipes this (`./scripts/update-openapi.sh | tail`) sees the
# PIPELINE's status, i.e. tail's 0, and reads a no-op as success — leaving stale artifacts to fail
# the blocking api-docs CI job. Before verifying artifacts are current, `touch` a file the project
# compiles (or check this script's own exit status, unpiped). CI is unaffected: it restores into a
# clean tree, so the generation never skips.
(cd ErsatzTV && dotnet build && dotnet build -t:GenerateOpenApiDocuments) || exit
cd "$REPO_ROOT" || exit
+1
View File
@@ -17,6 +17,7 @@ export * from './imageFolders';
export * from './languages';
export * from './libraries';
export * from './libraryBrowse';
export * from './selectionId';
export * from './logs';
export * from './maintenance';
export * from './mediaDetail';
+95 -1
View File
@@ -1,5 +1,11 @@
import { afterEach, describe, expect, it, vi } from 'vitest';
import { getLibraryBrowseItems } from './libraryBrowse';
import {
getLibraryBrowseItems,
searchLibraryPickerOptions,
titleContainsQuery,
LIBRARY_PICKER_LUCENE_SPECIALS,
LIBRARY_PICKER_RESULTS
} from './libraryBrowse';
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
@@ -56,3 +62,91 @@ describe('getLibraryBrowseItems', () => {
expect(browseUrl(fetchMock).searchParams.has('parentId')).toBe(false);
});
});
describe('titleContainsQuery (#651 — compile typed text, never forward raw Lucene)', () => {
it('wraps the escaped text in boundary wildcards on the title field', () => {
expect(titleContainsQuery('Show Alpha')).toBe('title:*Show\\ Alpha*');
});
// The previous version of this test hand-copied a sample string and claimed to cover "every
// Lucene special" — it silently omitted `&` and `|`, and a completeness test that carries its own
// list of what to check cannot see what is missing from that list (#651 F2). Drive the assertion
// from the exported character set instead, one character at a time, so adding a character to the
// set without escaping it fails here.
it.each(LIBRARY_PICKER_LUCENE_SPECIALS.split(''))('escapes the Lucene special %j', (char) => {
expect(titleContainsQuery(`a${char}b`)).toBe(`title:*a\\${char}b*`);
});
it.each([' ', '\t', '\n'])('escapes whitespace %j so it cannot split the term', (char) => {
expect(titleContainsQuery(`a${char}b`)).toBe(`title:*a\\${char}b*`);
});
it('leaves every character that is NOT special untouched', () => {
const plain = 'abcXYZ019_,.\'@#$%';
for (const char of plain) {
expect(LIBRARY_PICKER_LUCENE_SPECIALS).not.toContain(char);
}
expect(titleContainsQuery(plain)).toBe(`title:*${plain}*`);
});
it('neutralises the && and || BOOLEAN operators, not just single characters (#651 F2)', () => {
// The regression: `Rock && Roll` used to compile with `&&` live, so Lucene parsed it as boolean
// syntax (or rejected the query) and an exactly-matching title returned nothing.
expect(titleContainsQuery('Rock && Roll')).toBe('title:*Rock\\ \\&\\&\\ Roll*');
expect(titleContainsQuery('A || B')).toBe('title:*A\\ \\|\\|\\ B*');
expect(titleContainsQuery('Rock & Roll')).toBe('title:*Rock\\ \\&\\ Roll*');
});
it('leaves a plain single word alone apart from the boundary stars', () => {
expect(titleContainsQuery('Alpha')).toBe('title:*Alpha*');
});
});
describe('searchLibraryPickerOptions (#651)', () => {
afterEach(() => {
vi.restoreAllMocks();
});
it('issues ONE bounded request with the compiled query and maps to {id, name}', async () => {
const fetchMock = vi.spyOn(window, 'fetch').mockResolvedValue(
jsonResponse({
page: [
{ id: 1, mediaItemId: 7, mediaType: 'Movie', title: 'Show Alpha' },
{ id: 2, mediaItemId: null, mediaType: 'Movie', title: null }
],
totalCount: 20000
})
);
const options = await searchLibraryPickerOptions('Movie', ' Show Alpha ');
expect(fetchMock).toHaveBeenCalledTimes(1);
const url = browseUrl(fetchMock);
expect(url.searchParams.get('query')).toBe('title:*Show\\ Alpha*');
expect(url.searchParams.get('mediaType')).toBe('Movie');
expect(url.searchParams.get('pageNum')).toBe('0');
expect(url.searchParams.get('pageSize')).toBe(String(LIBRARY_PICKER_RESULTS));
// `mediaItemId` wins when present; `id` is the fallback, and a missing title degrades to `#id`.
expect(options).toEqual([
{ id: 7, name: 'Show Alpha' },
{ id: 2, name: '#2' }
]);
});
it('#651 F4: CLAMPS an oversized pageSize rather than forwarding it', async () => {
const fetchMock = vi.spyOn(window, 'fetch').mockResolvedValue(jsonResponse({ page: [], totalCount: 20000 }));
await searchLibraryPickerOptions('Episode', 'Alpha', 5000);
// The 25-row bound is a property of the helper, not of caller discipline.
expect(browseUrl(fetchMock).searchParams.get('pageSize')).toBe(String(LIBRARY_PICKER_RESULTS));
});
it('issues NO request for a query below the minimum length', async () => {
const fetchMock = vi.spyOn(window, 'fetch').mockResolvedValue(jsonResponse({ page: [], totalCount: 0 }));
expect(await searchLibraryPickerOptions('Episode', 'a')).toEqual([]);
expect(await searchLibraryPickerOptions('Episode', ' ')).toEqual([]);
expect(fetchMock).not.toHaveBeenCalled();
});
});
+64
View File
@@ -47,6 +47,70 @@ export function getLibraryBrowseItems(params: GetLibraryBrowseItemsParams = {}):
return request<PagedLibraryBrowseItems>(`/api/v1/library/browse${queryString ? `?${queryString}` : ''}`);
}
// A library picker compiles typed text; it never forwards raw Lucene (#440, #651). The search
// index's default field does NOT match bare title words (`Alpha` finds nothing for "Show Alpha" —
// docs/e2e-local.md), so forwarding the user's literal text the way the explicit query box does
// would look broken in a *name* picker. Escape every Lucene special (and whitespace) so the
// boundary stars are the only live wildcards — the same shape `builder/rules/compile.ts` emits for
// its `contains` operator.
//
// The exhaustive set of characters Lucene's QueryParser treats as syntax. `&` and `|` are in it
// because the boolean operators are `&&`/`||`: escaping each character individually neutralises the
// pair. Leaving them live (as this helper's original AutoTuneScreen-local version did) meant a
// title like `Rock && Roll` compiled to a query Lucene parsed as boolean syntax — or rejected — so
// an exactly-matching title returned nothing (#651 F2). `LIBRARY_PICKER_LUCENE_SPECIALS` is
// exported so the test asserts against the character list itself rather than a hand-copied sample
// that cannot see its own omissions.
export const LIBRARY_PICKER_LUCENE_SPECIALS = '+-&|!(){}[]^"~*?:\\/';
const LUCENE_WILD_SPECIAL = /([\s+\-&|!(){}[\]^"~*?:\\/])/g;
export function titleContainsQuery(text: string): string {
return `title:*${text.replace(LUCENE_WILD_SPECIAL, '\\$1')}*`;
}
export interface LibraryPickerOption {
id: number;
name: string;
}
// How many matches a search-driven library picker offers, and the shortest query worth issuing.
// Both are hard bounds: such a picker NEVER loads more than one page of this size, whatever the
// media type's row count (#651 — decision key `spa.library-pickers-resolve-by-search`).
export const LIBRARY_PICKER_RESULTS = 25;
export const LIBRARY_PICKER_MIN_QUERY = 2;
// Resolve picker options for one media-library type by SEARCH rather than by loading a window of
// the whole type. Exactly one bounded request per (debounced) query; a too-short query issues none
// at all.
//
// `pageSize` is CLAMPED to `LIBRARY_PICKER_RESULTS`, not merely defaulted to it (#651 F4): the
// bound is documented as a property of this helper, so it must not be defeatable by a caller
// passing a larger number.
export function searchLibraryPickerOptions(
mediaType: LibraryBrowseMediaType,
text: string,
pageSize: number = LIBRARY_PICKER_RESULTS
): Promise<LibraryPickerOption[]> {
const trimmed = text.trim();
if (trimmed.length < LIBRARY_PICKER_MIN_QUERY) {
return Promise.resolve([]);
}
const boundedPageSize = Math.max(1, Math.min(Math.floor(pageSize), LIBRARY_PICKER_RESULTS));
return getLibraryBrowseItems({
mediaType,
pageNum: 0,
pageSize: boundedPageSize,
query: titleContainsQuery(trimmed)
}).then((result) =>
(result.page ?? []).map((item) => {
const id = item.mediaItemId ?? item.id;
return { id, name: item.title ?? `#${id}` };
})
);
}
export function messageFromLibraryBrowseError(error: unknown, fallback = 'Unable to load library items'): string {
if (error instanceof ApiError) {
return error.detail ?? error.message;
+576
View File
@@ -0,0 +1,576 @@
import { describe, expect, it } from 'vitest';
import { scanPageSizeSites } from './pageSizeScan';
/**
* #650 guard: an ENUMERATING allow-list over every `pageSize` call site in the SPA.
*
* #644 fixed every call site that requested an OVER-cap `pageSize` (e.g. `pageSize: 1000`) to
* "get everything in one call" a pattern that silently truncates to the server's `MaxPageSize`
* (100 today) with no error and no truncation indicator. #644's own completeness check (box 4)
* was a manual grep for an inflated `pageSize`, which is why it could not see #650: two call
* sites requesting EXACTLY the cap (100) truncate exactly as much as an over-cap request, they
* just don't match a "pageSize above the cap" pattern.
*
* So this guard does NOT pattern-match on the pageSize VALUE (that repeats the #644 mistake for
* the next magic number). It enumerates every `pageSize` property inside a real object-literal
* expression via `scanPageSizeSites` (the TypeScript compiler API see `pageSizeScan.ts`'s doc
* comment for why a hand-rolled text/regex scan was replaced) and cross-checks the discovered set
* against a hand-reviewed registry below, in BOTH directions:
* - a NEWLY discovered, unregistered site fails (a new call site was added without a documented
* classification the exact way #650 could recur invisibly);
* - a REGISTERED site no longer discovered fails (the registry has gone stale e.g. a site was
* removed or refactored to no longer pass a `pageSize` property, and the registry should
* shrink to match, not silently claim coverage of code that no longer exists).
* Both directions are computed and reported in a SINGLE combined failure message (not two
* sequential `expect` calls) an early throw would otherwise hide the second direction's result
* in the same run, understating what actually needs fixing.
*
* **Identity is `(file, kind, value)` deliberately NOT line/column.** The original guard keyed
* each site on its absolute `line:column` (#650 follow-up F5/M-6). That made the registry a
* function of every OTHER file's line count, so a branch that never touches this guard can still
* invalidate it. This guard was BORN RED, and the sequence is the whole argument (#684): #651
* moved `AutoTuneScreen.tsx` up ten lines and `FillerPresetsScreen.tsx` down seventy-two, and
* merged to `main` BEFORE this guard's own PR (#675) did so the registry, authored against a
* pre-#651 base, was stale the instant it landed. Its own merge run was CANCELLED, so nothing
* reported it, and the red first surfaced on the NEXT push (#676's merge, which touches no
* `web/src` file at all and is in no way the cause).
*
* That is one ordering accident, not a recurring two-merge pattern but the exposure is the
* general case, because it is structurally invisible pre-merge: every PR is green against its own
* base, so the breakage exists only in the merge result and lands after review and after the merge
* gate.
*
* A NEW call site, a REMOVED one, and a CHANGED `pageSize` value each still fail, because each
* changes the `(file, kind, value)` multiset. What no longer fails is MOVING an unchanged site
* within its own file no truncation risk, and exactly the churn being removed.
*
* **The one real coverage case this costs, stated rather than implied** (#684 review M2): a
* SAME-IDENTITY SUBSTITUTION inside one file delete a registered site and add a different,
* unreviewed one with the same `kind` and the same value TOKEN, keeping the count equal. Verified
* to pass: deleting `TrashScreen.tsx`'s load-more `pageSize: PAGE_SIZE` and adding a whole-library
* `getLibraryBrowseItems({ mediaType: 'Movie', pageSize: PAGE_SIZE })` is green. It is narrow (same
* file, same kind, same token, net-zero count), and the old identity caught it only incidentally
* it fired on every position change, so a reviewer conditioned to re-pin line numbers would likely
* have waved it through anyway. Accepted knowingly; do not describe this guard as exhaustive.
*
* **Comparison stays a MULTISET count, not set membership** (#650 follow-up M-6, preserved): two
* sites in one file sharing an identifier (`TrashScreen.tsx`'s two `PAGE_SIZE` requests,
* `paging.ts`'s two `loadAllPages` fetches) register as two entries and must be discovered twice.
* So adding a third occurrence, or an accidental duplicate registry entry, is still caught rather
* than one occurrence silently covering the others.
*
* The SCANNER's own positional identity (`pageSizeSiteId`, line:column) is unchanged and still
* asserted by `pageSizeScan.test.ts` verifying the compiler-API scan reports real AST positions
* is that test's actual subject, and it runs against fixed inline fixtures, so it has no churn.
*
* `scanPageSizeSites` itself is verified against inline fixture source strings covering every
* input class a text-level scanner previously got wrong (comment-in-string, template
* interpolation, ternary, `??`, JSX container, same-line duplicates, parameter/nested
* destructuring, a type literal, a string containing the text `pageSize: 100`) in
* `pageSizeScan.test.ts` that test does not depend on the real repo, so it protects the SCANNER
* itself, not just today's snapshot of call sites.
*
* Each registry entry classifies the site per `docs/spa-conventions.md` §3b /
* `spa.library-pickers-resolve-by-search` (#651). NOTE the record path: that key SUPERSEDED
* `spa.list-completeness-vs-bounded-pickers`, whose record has since moved to
* `docs/decisions/archive/spa/` resolve it through `docs/decisions/README.md` by key, never by
* the path a comment happens to name (the breadcrumb rule).
* - 'class-a' bounded-by-construction list, paged to completeness via `loadAllPages`
* (or, for the two sites INSIDE `loadAllPages` itself, its
* implementation), with a `complete`/`incomplete` flag surfaced (never
* silently partial).
* - 'search-bounded' resolves by SEARCH and windows nothing: the typed query is the narrowing
* mechanism, and the row bound is a property of the CODE rather than of a
* caller's discipline (a clamp inside the shared helper for #651's
* library picker; a fixed small constant at the inline preview sites).
* Nothing is list-loaded, so there is no truncation to surface and the
* absence of a truncation hint is correct which is why such a site
* cannot be filed under 'class-b', whose defining evidence IS a surfaced
* `totalCount`. Applies only where the query is genuinely required: a
* site that degrades to an unfiltered browse when the query is empty is
* NOT search-bounded (see 'deviation').
* - 'class-b' one bounded page at (or under) the cap, with the real truncation
* (`totalCount` vs items shown) surfaced to the user. Post-#651 this no
* longer covers media-library pickers (those are 'search-bounded').
* **The RENDER is the entry requirement, not the intent** that is the
* operative rule, and the only one to apply to a new site. Today's
* entries happen to take four shapes: a list bounded by its PARENT
* (`ChannelBuilder`, seasons of one show); the collection-family types
* §3b excludes from search (`FillerPresetsScreen`), which keep the
* bounded page and its hint; a preview over an already-bounded set
* (`AutoTuneScreen`'s channel members); and a preview over an UNBOUNDED
* user-authored query that surfaces its match count
* (`SmartCollectionDialog`). That list is illustrative and NOT
* exhaustive: a site qualifies by rendering a real `totalCount`-backed
* hint, not by resembling one of these four. (#684 review: an earlier
* revision of this comment called it "the whole list" while the registry
* below already held a fourth the same false-exhaustiveness defect this
* PR exists to remove.)
* - 'paged-ui' real paging UI (a page/"load more" control, or a user-adjustable
* page-size selector, keyed to a genuine `totalCount`), so a `pageSize`
* at or below the cap is correct as-is.
* - 'deviation' a KNOWN, TRACKED violation of §3b that this registry refuses to launder
* into a compliant-looking label. A registry exists to state what is
* true; recording a defect as 'class-b' or 'search-bounded' would make
* the guard assert a hint or a query gate that demonstrably does not
* exist, and the next reader would trust it. Every such entry MUST carry
* its tracking issue in the structural `issue` field enforced below,
* and deliberately NOT a `#\d+` scrape of the note, which passed with the
* reference deleted because notes legitimately cite historical issues
* and flips to a real class only when the behaviour is fixed.
*
* **Known residual gap:** object SPREAD (`getFoo({ ...opts })` where `opts` was built elsewhere
* with an at-cap `pageSize`) and a `pageSize` passed as a bare POSITIONAL argument rather than an
* object-literal property (`api/search.ts`'s `getAllSearchItemIds(query, pageNum, pageSize)`, the
* api.search-allitems-paging precedent) are NOT resolvable by this scan there is no `pageSize`
* token inside an object-literal expression to find. A third, pre-existing gap (#684 review L2): a
* `pageSize` whose value is a FORWARDED EXPRESSION rather than a literal or shorthand e.g.
* `api/collections.ts`'s `pageSize: String(pageSize)` is a real object-literal property that
* `scanPageSizeSites` still drops. Written down here, not silently absent: a call site introduced
* through any of the three paths needs a human re-grep if that shape becomes common.
*/
interface RegistryEntry {
/** Path relative to `src/`, e.g. `api/paging.ts`. */
file: string;
kind: 'literal' | 'shorthand';
/** The `pageSize` value's source text — an identifier (`PAGE_SIZE`) or a numeric literal. */
value: string;
classification: 'class-a' | 'search-bounded' | 'class-b' | 'paged-ui' | 'deviation';
/**
* The Gitea issue tracking a 'deviation' REQUIRED for that class and meaningless otherwise.
* A dedicated field rather than a `#\d+` scrape of `note` (#684): notes legitimately cite
* historical issues, so the regex passed even with the tracking reference deleted a test
* satisfiable by text that has nothing to do with what it claims to check.
*/
issue?: number;
note: string;
}
// Keep in file order, then in the order the sites appear within the file, so a diff against the
// discovered set is easy to read. Two entries sharing a `(file, kind, value)` identity are
// deliberate and load-bearing: the multiset comparison requires that site to be discovered exactly
// twice (see the identity note above).
const REGISTRY: RegistryEntry[] = [
{
file: 'api/libraryBrowse.ts',
kind: 'literal',
value: 'boundedPageSize',
classification: 'search-bounded',
note:
"searchLibraryPickerOptions — the #651 shared media-library picker that REPLACED the bounded " +
'windows previously registered for PlaylistsScreen and RerunCollectionsScreen (both now ' +
'correctly absent). The bound is a clamp, not a default: Math.min(pageSize, ' +
'LIBRARY_PICKER_RESULTS) inside the helper, so a caller cannot widen it.'
},
{
file: 'api/paging.ts',
kind: 'shorthand',
value: 'pageSize',
classification: 'class-a',
note:
"loadAllPages's own first-page fetch. This IS the Class A completeness helper every other " +
'bounded list uses — not a defect, the fix itself.'
},
{
file: 'api/paging.ts',
kind: 'shorthand',
value: 'pageSize',
classification: 'class-a',
note: "loadAllPages's subsequent-page fetch inside the completeness loop; same helper as the entry above."
},
{
file: 'builder/ChannelBuilder.tsx',
kind: 'literal',
value: '100',
classification: 'class-b',
note:
'SeasonsDialog: TelevisionSeason browse scoped to one show (parentId), so it is bounded by ' +
'its PARENT rather than being a picker over the whole type — which is why #651 left it as a ' +
"single bounded page. #650 found the response's totalCount went unread; it is now surfaced " +
"as a 'Showing the first N of M seasons' hint if a show somehow exceeds the cap."
},
{
file: 'builder/libraryBrowse.ts',
kind: 'shorthand',
value: 'pageSize',
classification: 'paged-ui',
note:
"loadCollections's per-kind fan-out (#650 fix): forwards a real pageNum/pageSize from the " +
"caller and sums each kind's real totalCount, so the builder's Load more button (canLoadMore) " +
'is meaningful — this is the paged-ui replacement for the original truncating implementation.'
},
{
file: 'builder/libraryBrowse.ts',
kind: 'shorthand',
value: 'pageSize',
classification: 'paged-ui',
note: "loadLibraryItems's per-kind fan-out — same real pageNum/pageSize/totalCount pattern as loadCollections above."
},
{
file: 'builder/SmartCollectionDialog.tsx',
kind: 'literal',
value: '24',
classification: 'class-b',
note:
'Inline smart-query preview while authoring a query. It DOES surface the real truncation — ' +
"the response's totalCount is rendered as a `{count} matches` badge above a 12-row slice of " +
'the 24 fetched — which is precisely what class-b requires, so it is not search-bounded ' +
'despite being query-driven (#684 review M1: it was the counter-example to a claim that no ' +
'such site renders a hint).'
},
{
file: 'screens/AutoTuneScreen.tsx',
kind: 'literal',
value: 'MEMBER_PREVIEW_SIZE',
classification: 'class-b',
note:
"Channel-member preview: a real bounded window over the members, which is why it DOES render " +
"'showing first N' once totalCount exceeds the preview size."
},
{
file: 'screens/AutoTuneScreen.tsx',
kind: 'literal',
value: 'ADD_SOURCE_RESULTS',
classification: 'search-bounded',
note:
'Tiny (8-row) debounced add-source search typeahead — the #440 picker whose compile-the-typed-' +
'text rule #651 generalised. Nothing is windowed: a query narrows, and no hint is owed.'
},
{
file: 'screens/BlockPlayoutTroubleshootingScreen.tsx',
kind: 'shorthand',
value: 'pageSize',
classification: 'paged-ui',
note:
'Playout block history: forwards a user-adjustable `pageSize` state (persisted, backed by a ' +
'page-size <Select>) to a real pager keyed off the response totalCount.'
},
{
file: 'screens/CollectionsScreen.tsx',
kind: 'literal',
value: '50',
classification: 'deviation',
issue: 685,
note:
'TRACKED §3b VIOLATION — #685. AddItemsDialog.runSearch is reachable with an EMPTY query ' +
'(blank form submit, and a kind-chip click, which calls it immediately), and ' +
'getLibraryBrowseItems omits a falsy query — so it degrades to an unfiltered 50-row window ' +
'over the whole media-library type, per kind. Nothing surfaces it: totalCount is never read ' +
'here and no hint renders, and merged.slice(0, 50) drops up to 100 of 150 fetched rows even ' +
'for a real query. Under the cap, so #644 and #650 both missed it. NOT search-bounded (the ' +
'query is not required) and NOT class-b (no hint) — labelling it either would make this ' +
'registry vouch for behaviour that does not exist.'
},
{
file: 'screens/FillerPresetsScreen.tsx',
kind: 'literal',
value: 'LIBRARY_BROWSE_PAGE_CAP',
classification: 'class-b',
note:
'The COLLECTION-FAMILY fallback (Collection / SmartCollection / MultiCollection / ' +
'RerunCollection / Playlist), which spa-conventions §3b explicitly excludes from search ' +
'because GetLibraryBrowseItemsHandler LIKE-matches `query` for those types and would match a ' +
"compiled `title:*x*` literally. Keeps the bounded page AND its truncation hint; this " +
"screen's media-item types went to searchLibraryPickerOptions in #651."
},
{
file: 'screens/LogsScreen.tsx',
kind: 'shorthand',
value: 'pageSize',
classification: 'paged-ui',
note:
'Log listing: forwards a user-adjustable `pageSize` state (persisted, backed by a page-size ' +
'<Select>) to a real pager keyed off the response totalCount.'
},
{
file: 'screens/MediaBrowseScreen.tsx',
kind: 'literal',
value: 'PAGE_SIZE',
classification: 'paged-ui',
note: 'Library browse grid has a real page-number pager driven off the real totalCount.'
},
{
file: 'screens/MediaDetailScreen.tsx',
kind: 'literal',
value: 'CHILD_PAGE_SIZE',
classification: 'paged-ui',
note: 'Season/episode child list has a real page-number pager driven off the real totalCount.'
},
{
file: 'screens/SearchScreen.tsx',
kind: 'literal',
value: 'PAGE_SIZE',
classification: 'paged-ui',
note: 'Per-group search results; hasMore gated on totalCount > items.length with a load-more.'
},
{
file: 'screens/TrashScreen.tsx',
kind: 'literal',
value: 'PAGE_SIZE',
classification: 'paged-ui',
note: 'Per-group trash listing; "See all N" load-more gated on totalCount > items.length.'
},
{
file: 'screens/TrashScreen.tsx',
kind: 'literal',
value: 'PAGE_SIZE',
classification: 'paged-ui',
note:
'The load-more request handler for the SAME per-group trash listing as the entry above — a ' +
'deliberate second occurrence of one identity, which the multiset comparison requires to be ' +
'discovered exactly twice.'
}
];
// Enumerates every source file under `src/` via Vite's `import.meta.glob` — eagerly, as raw text
// (`query: '?raw', import: 'default'`) — INSTEAD OF Node's `fs`/`path`/`url` (#650 follow-up).
// This is the only file under `src` that ever needed real filesystem access, and `@types/node`
// isn't wired into `tsconfig.app.json`'s project (deliberately: it covers production browser code
// too, and a file-local `/// <reference types="node" />` was tried and reverted — under `tsc -b`'s
// single-program compilation it leaked Node's ambient `setTimeout` into the whole app project,
// breaking three unrelated `window.setTimeout` mocks that expect the DOM signature). `import.meta
// .glob` needs neither `node:fs` nor a tsconfig change: it's resolved by Vite at transform time,
// natively available in the browser/app project, and is the idiomatic Vite/vitest way to enumerate
// source files. Keys are POSIX paths from the project root, e.g. `/src/api/pageSizeScan.ts`.
const rawSourceModules = import.meta.glob('/src/**/*.{ts,tsx,mts,cts}', {
query: '?raw',
import: 'default',
eager: true
}) as Record<string, string>;
function basename(path: string): string {
const idx = path.lastIndexOf('/');
return idx === -1 ? path : path.slice(idx + 1);
}
// Extracted from `listSourceFiles`'s inline condition so it's independently testable (#650
// follow-up round 4): a plant that adds a real `.mts` FILE and observes the guard notice it
// proves the behavior exists today, but pins nothing — revert the glob back to `.ts`/`.tsx` and
// both the real-source guard AND `pageSizeScan.test.ts`'s `.mts`/`.cts` PARSING tests stay green,
// because this repo has no committed `.mts`/`.cts` source and `scanPageSizeSites` parses any
// non-`.tsx` filename as plain TS regardless of extension. Testing this predicate directly, by
// filename, is what actually regression-pins the file-discovery fix rather than depending on the
// repo happening to contain (or not contain) a matching file. This predicate is still what the
// glob's results are filtered THROUGH below (`listSourceFiles`) — the extension SET moved into the
// glob literal, but discovery still runs every matched file through this same named, tested
// function, not a second copy of the logic.
export function isScannableSourceFileName(name: string): boolean {
// `.mts`/`.cts` are legal TS extensions `tsconfig.app.json`'s `include` covers alongside
// `.ts`/`.tsx` — none exist in this repo today, but the glob must not silently skip one if it
// ever does (#650 follow-up round 3 MEDIUM finding).
return (
/\.(ts|tsx|mts|cts)$/.test(name) && !/\.test\.(tsx?|mts|cts)$/.test(name) && !name.endsWith('.guard.test.ts')
);
}
interface ScannableSource {
/** Path relative to `src/`, e.g. `api/pageSizeScan.ts` — matches the REGISTRY's `file` field. */
file: string;
text: string;
}
function listSourceFiles(): ScannableSource[] {
const out: ScannableSource[] = [];
for (const [key, text] of Object.entries(rawSourceModules)) {
if (key.includes('/generated/')) {
continue;
}
if (!isScannableSourceFileName(basename(key))) {
continue;
}
out.push({ file: key.replace(/^\/src\//, ''), text });
}
return out;
}
interface DiscoveredSite {
file: string;
line: number;
column: number;
kind: 'literal' | 'shorthand';
value: string;
}
function discoverPageSizeCallSites(): DiscoveredSite[] {
const sites: DiscoveredSite[] = [];
for (const { file, text } of listSourceFiles()) {
for (const site of scanPageSizeSites(text, file)) {
sites.push({ file, ...site });
}
}
return sites;
}
// The REGISTRY's identity: `(file, kind, value)`, with no source position — see the identity note
// in this file's header for why the line/column were dropped. This is deliberately NOT
// `pageSizeSiteId` (which keys on line:column and remains the SCANNER's identity, asserted over
// fixed fixtures in `pageSizeScan.test.ts`); the two answer different questions, so they are
// allowed to differ, and a registry entry has no position to supply anyway.
function registryId(site: { file: string; kind: string; value: string }): string {
return `${site.file}:${site.kind}:${site.value}`;
}
// Identity and REPORT deliberately have different formats (#684 review M3). The comparison key
// carries no position — that is the whole fix — but a bare `TrashScreen.tsx:literal:PAGE_SIZE` is
// useless to whoever has to go find it in a file holding two such sites. So the UNREGISTERED
// direction, which describes DISCOVERED sites and therefore does have real positions, prints them.
// This reintroduces no churn: positions appear only in a failure message, never in a comparison.
function describeDiscovered(sites: DiscoveredSite[], ids: string[]): string[] {
const positions = new Map<string, string[]>();
for (const site of sites) {
const id = registryId(site);
const at = positions.get(id) ?? [];
at.push(`${site.line}:${site.column}`);
positions.set(id, at);
}
return ids.map((id) => {
const at = positions.get(id);
// Every position sharing this identity, not just the excess one: a positionless key genuinely
// cannot tell which occurrence is new, so the candidate set IS the honest answer. Labelled so a
// reader does not take all of them as unregistered (#684 review L-a).
return at && at.length > 0 ? `${id} (identity seen at: ${at.join(', ')})` : id;
});
}
// Multiset (count per identity) comparison, not plain array `.includes` membership (#650
// follow-up M-6) — so a registry that accidentally lists the same identity twice, or a future
// scanner change that could (in principle) emit a duplicate, is still caught rather than one
// occurrence silently covering both.
function toCounts(ids: string[]): Map<string, number> {
const counts = new Map<string, number>();
for (const id of ids) {
counts.set(id, (counts.get(id) ?? 0) + 1);
}
return counts;
}
// Returns entries present in `left` more times than in `right`, expanded per the excess count —
// e.g. a `left` id appearing 3 times against 1 in `right` yields that id listed twice.
function multisetExcess(left: Map<string, number>, right: Map<string, number>): string[] {
const excess: string[] = [];
for (const [id, count] of left) {
const remaining = count - (right.get(id) ?? 0);
for (let i = 0; i < remaining; i++) {
excess.push(id);
}
}
return excess.sort();
}
// These 4 tests are BASELINE assertions about the guard's steady-state behavior against the
// current repo snapshot — they all pass equally on the clean `b90f8a3b` commit (before this
// round's scanner rewrite), so none of them individually PROVE this round's fixes. What actually
// regression-pins the scanner's fixes is `pageSizeScan.test.ts` (synthetic fixtures per input
// class, verified against the prior scanner where the review asked for it) — these 4 just confirm
// the guard, wired to whichever scanner it currently uses, still holds over real source.
describe('pageSize call-site guard (#650)', () => {
it.each([
['screens/TraktListsScreen.ts', true],
['builder/ChannelBuilder.tsx', true],
['builder/libraryBrowse.mts', true],
['api/pageSizeScan.cts', true],
['screens/TraktListsScreen.test.ts', false],
['builder/ChannelBuilder.test.tsx', false],
['builder/libraryBrowse.test.mts', false],
['api/pageSizeScan.test.cts', false],
['api/pageSizeCallSites.guard.test.ts', false],
['api/generated/v1.ts', true], // the predicate itself is filename-only; the 'generated' DIRECTORY exclusion lives in listSourceFiles, tested separately below.
['components.js', false],
['data.json', false],
['README.md', false],
['noextension', false]
])(
'isScannableSourceFileName(%s) === %s — the file-discovery predicate itself, independent of ' +
'whether the repo happens to contain a matching file (#650 follow-up round 4)',
(name, expected) => {
// A prior verification planted a REAL .mts file and observed the guard notice it — that
// proved the .mts/.cts fix works today, but pinned nothing: reverting the glob back to
// `.ts`/`.tsx` leaves both the real-source guard AND pageSizeScan.test.ts's .mts/.cts
// PARSING tests green, since this repo has no committed .mts/.cts source and the scanner
// parses any non-.tsx filename as plain TS regardless of extension. Asserting on the
// predicate BY FILENAME, with no filesystem involved, is what actually regression-pins it.
expect(isScannableSourceFileName(name)).toBe(expected);
}
);
it('scans a healthy number of source files (anti-vacuity: a broken glob must not pass on zero input)', () => {
const files = listSourceFiles();
expect(files.length).toBeGreaterThan(50);
});
it('discovers a healthy number of pageSize call sites (anti-vacuity: a broken scan must not pass on zero matches)', () => {
const sites = discoverPageSizeCallSites();
expect(sites.length).toBeGreaterThan(10);
});
it('matches the discovered pageSize call sites EXACTLY against the reviewed registry (not a non-empty check)', () => {
const discovered = discoverPageSizeCallSites();
const discoveredCounts = toCounts(discovered.map(registryId));
const registeredCounts = toCounts(REGISTRY.map(registryId));
const unregistered = multisetExcess(discoveredCounts, registeredCounts);
const stale = multisetExcess(registeredCounts, discoveredCounts);
// Both directions are folded into ONE assertion so a failure always shows the complete
// picture in a single run (#650 follow-up, line-churn concern) — two sequential `expect`
// calls would throw on the first failing direction and never evaluate/report the second.
if (unregistered.length > 0 || stale.length > 0) {
const report = [
`UNREGISTERED (${unregistered.length}) — discovered pageSize call site(s) missing from the REGISTRY above:`,
...describeDiscovered(discovered, unregistered).map((line) => ` + ${line}`),
`STALE (${stale.length}) — REGISTRY entries no longer found as a real pageSize call site:`,
...stale.map((id) => ` - ${id}`)
].join('\n');
throw new Error(report);
}
});
// A 'deviation' entry is the registry admitting a live defect rather than laundering it into a
// compliant-looking label (#684 review H2). That is only honest if the defect is TRACKED — an
// untracked deviation is just a defect with better manners — so the issue reference is enforced
// here rather than left to a reviewer noticing its absence.
it("every 'deviation' entry names the issue tracking it", () => {
const deviations = REGISTRY.filter((entry) => entry.classification === 'deviation');
// Anti-vacuity: if the deviations are ever all fixed, this must be deleted deliberately, not
// silently pass over an empty list while claiming to enforce something.
expect(deviations.length).toBeGreaterThan(0);
for (const entry of deviations) {
expect(entry.issue, `${entry.file}:${entry.value} is a deviation but names no tracking issue`).toEqual(
expect.any(Number)
);
expect(entry.issue!).toBeGreaterThan(0);
}
// The converse, so the field cannot drift into decoration: only a deviation carries one.
for (const entry of REGISTRY.filter((e) => e.classification !== 'deviation')) {
expect(entry.issue, `${entry.file}:${entry.value} is not a deviation but carries an issue`).toBeUndefined();
}
});
// Pins the report format, not the comparison key (#684 review M3): dropping the position from
// IDENTITY is the fix, dropping it from the failure MESSAGE was collateral damage — it left
// `TrashScreen.tsx:literal:PAGE_SIZE` pointing at a file with two such sites.
it('reports the discovered line:column for an unregistered site, while comparing without it', () => {
const sites = discoverPageSizeCallSites();
const target = sites.find((site) => site.file === 'screens/TrashScreen.tsx');
expect(target).toBeDefined();
const described = describeDiscovered(sites, [registryId(target!)]);
expect(described[0]).toContain(`${target!.line}:${target!.column}`);
// ...and the key it was looked up by still carries no position.
expect(registryId(target!)).not.toContain(String(target!.line));
});
it('every registry entry documents its class per docs/spa-conventions.md §3b', () => {
for (const entry of REGISTRY) {
expect(['class-a', 'search-bounded', 'class-b', 'paged-ui', 'deviation']).toContain(entry.classification);
expect(entry.note.length).toBeGreaterThan(20);
}
});
});
+247
View File
@@ -0,0 +1,247 @@
import { describe, expect, it } from 'vitest';
import { pageSizeSiteId, scanPageSizeSites, type PageSizeSite } from './pageSizeScan';
/**
* Fixture test for `scanPageSizeSites` itself NOT a scan of the real repo (that's
* `pageSizeCallSites.guard.test.ts`). This is what actually protects the SCANNER going forward:
* a prior hand-rolled regex/bracket-tracking version passed the guard test against unmodified
* source at both #650 commits while still being defeated by every case below, because the guard
* only ever exercised today's snapshot of real call sites it never proved the scanner handles
* the INPUT CLASSES that expose a text-level scanner's blind spots. Pinning the exact discovered
* set against synthetic source strings closes that gap.
*
* Not every fixture here is a REGRESSION pin against the prior (round-1, `b90f8a3b`) bracket-
* tracking scanner a round-3 review found that round 1's simple `pageSize:\s*value` regex
* already handled a bare URL-string or a bare `??` context correctly on its own (a `//` inside a
* string, or the token immediately before `{`, only mattered to round 1's OWN heuristics, not to
* a plain regex match). Those two are labelled CONTRACT fixtures below they pin the documented
* behavior going forward, not a fix. The fixtures that genuinely fail against round 1 (verified)
* are: the string CONTAINING the literal text `pageSize: 100`, the template-literal
* interpolation, the same-line ternary identity/multiplicity, the JSX shorthand container,
* parameter destructuring, nested destructuring, and the type-literal declaration plus the
* combined multi-case fixture, which fails round 1 for several of those reasons at once.
*/
function ids(sites: PageSizeSite[]): string[] {
return sites.map(pageSizeSiteId);
}
describe('scanPageSizeSites', () => {
it('finds a literal pageSize: property in a plain object-literal call argument', () => {
const source = `getFoo({ pageSize: 100, query });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:literal:100']);
});
it('finds the ES6 shorthand pageSize property in a plain object-literal call argument', () => {
const source = `getFoo({ pageSize, query });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:shorthand:pageSize']);
});
it('is NOT fooled by a "//" inside a string literal — CONTRACT fixture, not a round-1 regression pin (M-3)', () => {
// NOTE: round 1's unconditional literal regex (`pageSize:\s*(\d+|identifier)`) already
// matched this exact input correctly on its own — a `//` inside a string never confused THAT
// narrower pattern. This pins the AST scanner's documented contract going forward; it is the
// COMBINED multi-case fixture below (and the M-3-shaped case buried inside it — a literal
// `//` immediately preceding a real call site on the SAME conceptual scan) that actually
// fails against round 1's comment-stripping step, not this input in isolation.
const source = [`const endpoint = 'https://example.test';`, `getFoo({ pageSize: 100 });`, ''].join('\n');
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['2:10:literal:100']);
});
it('does NOT match a string literal that merely CONTAINS the text "pageSize: 100" (L-7)', () => {
const source = `const label = "pageSize: 100";\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('finds an object literal passed inside a template-literal interpolation (M-5)', () => {
const source = 'const url = `${await getFoo({ pageSize })}`;\n';
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:31:shorthand:pageSize']);
});
it('finds an object literal in each branch of a ternary, even on the SAME line (M-4, M-6)', () => {
const source = `return ok ? getA({ pageSize }) : getB({ pageSize });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
// Two distinct occurrences on one line get distinct identities (different columns) — a
// single registry entry cannot silently cover both.
expect(ids(sites)).toEqual(['1:20:shorthand:pageSize', '1:41:shorthand:pageSize']);
expect(sites[0].column).not.toBe(sites[1].column);
});
it('finds an object literal on the right-hand side of ?? — CONTRACT fixture, not a round-1 regression pin (M-4)', () => {
// NOTE: like the URL fixture above, round 1's literal-form regex already matched this exact
// `pageSize: 50` text correctly on its own — `??` doesn't change what characters precede the
// match on the line. This pins the documented contract, not a round-1 regression.
const source = `getFoo(options ?? { pageSize: 50 });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:21:literal:50']);
});
it('finds an object literal inside a JSX expression container attribute (M-4)', () => {
const source = `const el = <Component options={{ pageSize }} />;\n`;
const sites = scanPageSizeSites(source, 'fixture.tsx');
expect(ids(sites)).toEqual(['1:34:shorthand:pageSize']);
});
it('does NOT match a parameter destructuring pattern (L-7)', () => {
const source = `function f({ pageSize }: { pageSize: number }) {}\n`;
// The destructured PARAMETER `{ pageSize }` is an ObjectBindingPattern, not an
// ObjectLiteralExpression — excluded by node kind. Its TYPE annotation `{ pageSize: number }`
// is a TypeLiteral (PropertySignature), also excluded by node kind — never an object literal.
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('does NOT match a nested destructuring pattern (L-7)', () => {
const source = `const { nested: { pageSize } } = input;\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('does NOT match a type-literal declaration (L-7)', () => {
const source = `type P = { pageSize: 100 };\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('does NOT match an interface property declaration', () => {
const source = `interface Params {\n pageSize?: number;\n}\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('does NOT match a forwarded call expression (a dynamic passthrough, not a fixed value)', () => {
const source = `getFoo({ pageSize: String(pageSize) });\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('is not confused by a pageSize reference inside a comment', () => {
const source = [`// pageSize: 999 — this is just prose, not code`, `getFoo({ query });`, ''].join('\n');
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('is not confused by a pageSize reference inside a block/JSDoc comment', () => {
const source = ['/**', ' * Uses `pageSize` under the hood — see also `{ pageSize: 100 }`.', ' */', 'getFoo({ query });', ''].join(
'\n'
);
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('does NOT match a React dependency array containing pageSize', () => {
const source = `useCallback(load, [pageNum, pageSize, sortField]);\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('finds a const-identifier literal value (not just a numeric literal)', () => {
const source = `getFoo({ pageSize: LIBRARY_BROWSE_PAGE_CAP });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:literal:LIBRARY_BROWSE_PAGE_CAP']);
});
it('covers every case above together in one multi-line fixture and pins the exact discovered set', () => {
const source = [
`const endpoint = 'https://example.test';`, // M-3: not a comment
`const label = "pageSize: 100";`, // L-7: string contents, not code
`// pageSize: 999 in a line comment`, // not code
`/** block comment mentioning \`pageSize\` */`, // not code
`type P = { pageSize: 100 };`, // L-7: type literal, not a value
`interface Q { pageSize?: number; }`, // not a value
`function f({ pageSize }: { pageSize: number }) {}`, // L-7: destructuring + its type
`const { nested: { pageSize } } = input;`, // L-7: nested destructuring
`useCallback(load, [pageNum, pageSize]);`, // dependency array, not an object literal
`getFoo({ pageSize: String(pageSize) });`, // forwarded call, not a fixed value
`getFoo({ pageSize: 100 });`, // REAL: literal
`getBar({ pageSize });`, // REAL: shorthand
`getBaz(options ?? { pageSize: 50 });`, // REAL: ?? context (M-4)
`const url = \`\${await getQux({ pageSize })}\`;`, // REAL: template interpolation (M-5)
`return ok ? getA({ pageSize }) : getB({ pageSize });` // REAL x2: ternary, same line (M-4/M-6)
].join('\n');
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual([
'11:10:literal:100',
'12:10:shorthand:pageSize',
'13:21:literal:50',
'14:31:shorthand:pageSize',
'15:20:shorthand:pageSize',
'15:41:shorthand:pageSize'
]);
// Anti-vacuity: the fixture packs in 10 non-matching traps ahead of the 6 real sites — a
// scanner that matched everything (or nothing) would fail this count, not just the ids above.
expect(sites.length).toBe(6);
});
it('scans .tsx source using the TSX script kind (JSX does not parse under plain .ts rules)', () => {
const source = `export function C() {\n return <div data={{ pageSize: 10 }} />;\n}\n`;
const sites = scanPageSizeSites(source, 'fixture.tsx');
expect(ids(sites)).toEqual(['2:23:literal:10']);
});
// ---- round-3 MEDIUM finding: transparent TS wrappers around the initializer -----------------
it('finds a literal wrapped in "as const" (transparent to the runtime value)', () => {
const source = `getFoo({ pageSize: 100 as const });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:literal:100']);
});
it('finds a literal wrapped in "satisfies number" (transparent to the runtime value)', () => {
const source = `getFoo({ pageSize: 100 satisfies number });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:literal:100']);
});
it('finds a parenthesized literal', () => {
const source = `getFoo({ pageSize: (100) });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:literal:100']);
});
it('finds a const identifier through a chain of "as"/"satisfies"/parens wrappers', () => {
const source = `getFoo({ pageSize: ((LIBRARY_BROWSE_PAGE_CAP as number) satisfies number) });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:literal:LIBRARY_BROWSE_PAGE_CAP']);
});
it('still rejects a forwarded call expression even when wrapped in "as"', () => {
const source = `getFoo({ pageSize: String(pageSize) as string });\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
// ---- round-3 MEDIUM finding: non-Identifier property names -----------------------------------
it('finds a quoted string property key ("pageSize": 100)', () => {
const source = `getFoo({ 'pageSize': 100 });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:literal:100']);
});
it('finds a statically-resolvable computed property key (["pageSize"]: 100)', () => {
const source = `getFoo({ ['pageSize']: 100 });\n`;
const sites = scanPageSizeSites(source, 'fixture.ts');
expect(ids(sites)).toEqual(['1:10:literal:100']);
});
it('does NOT match a computed property key that cannot be resolved statically', () => {
const source = `const key = getKey();\ngetFoo({ [key]: 100 });\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
it('does NOT match a quoted string key for a DIFFERENT property name', () => {
const source = `getFoo({ 'pageSizeLimit': 100 });\n`;
expect(scanPageSizeSites(source, 'fixture.ts')).toEqual([]);
});
// ---- round-3 MEDIUM finding: .mts/.cts are never silently skipped -----------------------------
it('scans .mts source (parses as plain TS, no JSX grammar)', () => {
const source = `export function loadPage() {\n return getFoo({ pageSize: 100 });\n}\n`;
const sites = scanPageSizeSites(source, 'fixture.mts');
expect(ids(sites)).toEqual(['2:19:literal:100']);
});
it('scans .cts source (parses as plain TS, no JSX grammar)', () => {
const source = `getFoo({ pageSize });\n`;
const sites = scanPageSizeSites(source, 'fixture.cts');
expect(ids(sites)).toEqual(['1:10:shorthand:pageSize']);
});
});
+131
View File
@@ -0,0 +1,131 @@
import * as ts from 'typescript';
/**
* #650 follow-up: an AST-based scanner for every `pageSize` property that appears inside a real
* object LITERAL expression. Extracted into its own module so both the enumerating guard
* (`pageSizeCallSites.guard.test.ts`, which scans the real repo) and a fixture test
* (`pageSizeScan.test.ts`, which scans synthetic source strings and does NOT touch the repo) can
* exercise the exact same scanning logic.
*
* A prior hand-rolled regex/bracket-tracking version of this scan was replaced after a review
* found it defeated by comments-in-strings, template-literal interpolations, ternary/`??`
* contexts, JSX containers, and same-line duplicates each a DIFFERENT input class a text-level
* lexer has to special-case one at a time. The TypeScript compiler API sidesteps the whole
* category: comments and string/template CONTENTS are trivia/literal text the parser never
* revisits as code, and a real object-literal expression (`ObjectLiteralExpression`) is a
* structurally different AST node from a type literal (`type X = { pageSize: number }`,
* `PropertySignature` inside a `TypeLiteralNode`/`InterfaceDeclaration`) or a destructuring
* pattern (`ObjectBindingPattern`, e.g. `function f({ pageSize }) {}` or
* `const { pageSize } = x`) so those are excluded by NODE KIND, not by a preceding-character
* heuristic that can be fooled by an unrelated `{`/`(`/`,`.
*
* A round-3 review found the AST version still had its own smaller, but real false
* negatives: an initializer wrapped in a transparent TS construct (`pageSize: 100 as const`,
* `pageSize: 100 satisfies number`, `pageSize: (100)`) was rejected outright because only a bare
* `NumericLiteral`/`Identifier` was checked; a property written as a quoted string key
* (`'pageSize': 100`) or a statically-resolvable computed key (`['pageSize']: 100`) was missed
* because only an `Identifier` name was checked. `unwrapTransparentExpression` and
* `isPageSizePropertyName` close both see their doc comments below. Genuinely UNRESOLVABLE
* cases remain out of reach on purpose and are documented as a residual gap where this scanner is
* actually used (`pageSizeCallSites.guard.test.ts`'s module doc comment): object SPREAD
* (`getFoo({ ...opts })` built elsewhere) and a `pageSize` passed as a bare POSITIONAL argument
* rather than an object-literal property at all.
*/
export interface PageSizeSite {
/** 1-based source line of the `pageSize` property (name), matching editor line numbers. */
line: number;
/** 1-based source column of the `pageSize` property (name). */
column: number;
kind: 'literal' | 'shorthand';
/**
* For `kind: 'literal'`: the numeric-literal text or the referenced const identifier's name.
* For `kind: 'shorthand'`: always the literal string `'pageSize'` (the shorthand form only ever
* forwards whatever `pageSize` binding is in scope there is no separate "value" to name).
*/
value: string;
}
function scriptKindFor(fileName: string): ts.ScriptKind {
// `.mts`/`.cts` parse as plain TS (no JSX support), same as `.ts` — only `.tsx` needs the JSX
// grammar. `tsconfig.app.json`'s `include` covers all of `src`, and `.mts`/`.cts` are legal
// TS extensions the guard's file-discovery glob must not silently skip even though none exist
// in this repo today (#650 follow-up round 3 MEDIUM finding).
return fileName.endsWith('.tsx') ? ts.ScriptKind.TSX : ts.ScriptKind.TS;
}
// Unwraps TS constructs that are transparent to the runtime VALUE but would otherwise hide a
// numeric literal / identifier from a naive node-kind check: `expr as T`, `expr satisfies T`,
// and `(expr)`. `pageSize: 100 as const` and `pageSize: 100 satisfies number` are both real
// fixed-100 call sites; only the TS type-checking wrapper differs (#650 follow-up round 3 MEDIUM).
function unwrapTransparentExpression(node: ts.Expression): ts.Expression {
let current = node;
for (;;) {
if (ts.isParenthesizedExpression(current)) {
current = current.expression;
} else if (ts.isAsExpression(current)) {
current = current.expression;
} else if (ts.isSatisfiesExpression(current)) {
current = current.expression;
} else {
return current;
}
}
}
// A property name is `pageSize` whether written as a plain identifier (`pageSize: 100`), a
// quoted string key (`'pageSize': 100`), or a computed key that's STATICALLY a `'pageSize'`
// string literal (`['pageSize']: 100`) — all three compile to the identical property, so all
// three are real call sites (#650 follow-up round 3 MEDIUM). A computed key that ISN'T a literal
// (e.g. `[dynamicKeyVar]: 100`) can't be resolved statically and is correctly left unmatched.
function isPageSizePropertyName(name: ts.PropertyName): boolean {
if (ts.isIdentifier(name) || ts.isStringLiteral(name)) {
return name.text === 'pageSize';
}
if (ts.isComputedPropertyName(name)) {
const expr = unwrapTransparentExpression(name.expression);
return ts.isStringLiteral(expr) && expr.text === 'pageSize';
}
return false;
}
export function scanPageSizeSites(sourceText: string, fileName: string): PageSizeSite[] {
const sourceFile = ts.createSourceFile(fileName, sourceText, ts.ScriptTarget.Latest, true, scriptKindFor(fileName));
const sites: PageSizeSite[] = [];
function positionOf(node: ts.Node): { line: number; column: number } {
const { line, character } = sourceFile.getLineAndCharacterOfPosition(node.getStart(sourceFile));
return { line: line + 1, column: character + 1 };
}
function visit(node: ts.Node): void {
if (ts.isObjectLiteralExpression(node)) {
for (const property of node.properties) {
if (ts.isPropertyAssignment(property) && isPageSizePropertyName(property.name)) {
const initializer = unwrapTransparentExpression(property.initializer);
// Only a numeric literal or a bare identifier (a const/variable reference) counts as a
// fixed value baked into THIS call site. A forwarded expression — `String(pageSize)`, a
// ternary, a template, a function call — is a dynamic passthrough of whatever the
// caller supplied, not a literal this site chose; it is deliberately not recorded here
// (see the module doc comment on `loadAllPages`/positional-argument residual gaps).
if (ts.isNumericLiteral(initializer) || ts.isIdentifier(initializer)) {
const { line, column } = positionOf(property.name);
sites.push({ line, column, kind: 'literal', value: initializer.text });
}
} else if (ts.isShorthandPropertyAssignment(property) && property.name.text === 'pageSize') {
const { line, column } = positionOf(property.name);
sites.push({ line, column, kind: 'shorthand', value: 'pageSize' });
}
}
}
ts.forEachChild(node, visit);
}
visit(sourceFile);
return sites.sort((a, b) => (a.line === b.line ? a.column - b.column : a.line - b.line));
}
export function pageSizeSiteId(site: { line: number; column: number; kind: string; value: string }): string {
return `${site.line}:${site.column}:${site.kind}:${site.value}`;
}
+3 -3
View File
@@ -16,9 +16,9 @@
* **This is only for lists that are bounded by construction** (admin-created collections/rerun
* entries/playlists hundreds of rows at most). It must NOT be used as a picker/typeahead data
* source over a media library table (Episode/Song/Image/Movie/MusicVideo can run into the tens of
* thousands) see `docs/decisions/records/spa/list-completeness-vs-bounded-pickers.md` (#644
* follow-up review). Those call sites fetch a single bounded page directly and surface the
* truncation instead.
* thousands) see `docs/decisions/records/spa/library-pickers-resolve-by-search.md` (#651). Those
* call sites resolve by SEARCH instead: one bounded `searchLibraryPickerOptions` request per settled
* query, and no list load at all.
*/
export interface PagedResult<T> {
+5 -2
View File
@@ -37,8 +37,11 @@ export function getRerunCollection(id: number): Promise<RerunCollection> {
}
/** Load a single rerun collection together with its concurrency ETag (issue #253). */
export function getRerunCollectionWithMeta(id: number): Promise<ResponseWithMeta<RerunCollection>> {
return requestWithMeta<RerunCollection>(`/api/v1/rerun-collections/${id}`);
export function getRerunCollectionWithMeta(
id: number,
signal?: AbortSignal
): Promise<ResponseWithMeta<RerunCollection>> {
return requestWithMeta<RerunCollection>(`/api/v1/rerun-collections/${id}`, { signal });
}
export function createRerunCollection(body: CreateRerunCollectionRequest): Promise<RerunCollection> {
+55
View File
@@ -0,0 +1,55 @@
import { describe, expect, it } from 'vitest';
import { INT32_MAX, INT32_MIN, isSelectionId, selectionIdOrNull } from './selectionId';
// This predicate is the single point of failure for every id that reaches editor state across three
// screens (#651 round 8), and until now it was only exercised indirectly through screen tests. The
// endpoints matter most: swapping either `>=`/`<=` for a strict comparison is the classic mutation
// on exactly this code, and nothing else in the suite would notice.
describe('isSelectionId', () => {
it('accepts the INCLUSIVE int32 endpoints', () => {
expect(INT32_MAX).toBe(2_147_483_647);
expect(INT32_MIN).toBe(-2_147_483_648);
// A `>` for `>=` slip in either comparison fails here and nowhere else.
expect(isSelectionId(INT32_MAX)).toBe(true);
expect(isSelectionId(INT32_MIN)).toBe(true);
});
it('rejects values one step outside the range', () => {
expect(isSelectionId(INT32_MAX + 1)).toBe(false);
expect(isSelectionId(INT32_MIN - 1)).toBe(false);
});
it('accepts ordinary ids, including 0 and negatives', () => {
// Bindability, not existence — see the note in selectionId.ts. `id > 0` would break this.
expect(isSelectionId(1)).toBe(true);
expect(isSelectionId(0)).toBe(true);
expect(isSelectionId(-1)).toBe(true);
});
it('rejects non-integers and non-numbers', () => {
for (const value of [1.5, -0.5, Number.NaN, Number.POSITIVE_INFINITY, Number.NEGATIVE_INFINITY]) {
expect(isSelectionId(value)).toBe(false);
}
for (const value of ['1', null, undefined, {}, [], true, 1n]) {
expect(isSelectionId(value)).toBe(false);
}
});
});
describe('selectionIdOrNull', () => {
it('passes a bindable id through unchanged', () => {
expect(selectionIdOrNull(42)).toBe(42);
expect(selectionIdOrNull(INT32_MAX)).toBe(INT32_MAX);
expect(selectionIdOrNull(0)).toBe(0);
});
it('maps anything unbindable to null — never a coerced value', () => {
// Rounding 1.5 to 1 would submit a DIFFERENT record; absence is the only safe answer.
expect(selectionIdOrNull(1.5)).toBeNull();
expect(selectionIdOrNull(INT32_MAX + 1)).toBeNull();
expect(selectionIdOrNull('7')).toBeNull();
expect(selectionIdOrNull(null)).toBeNull();
expect(selectionIdOrNull(undefined)).toBeNull();
});
});
+30
View File
@@ -0,0 +1,30 @@
// A *selection id* is any id an editor stores and later submits as an entity reference —
// `selectedId` on a rerun collection, `collectionId`/`mediaItemId`/… on a playlist item or filler
// preset. The API binds every one of them as a 32-bit integer, so a value outside that domain is
// not merely odd: it renders and commits happily and then fails on write.
//
// This lives in ONE place on purpose. #651 round 7 added the check inside the search picker's
// option validator — the site where the defect was found — leaving list-backed options and the
// selection restored from a detail read unguarded, so the identical malformed value entered editor
// state through a different door (round 8). The predicate belongs at the BOUNDARY the class
// crosses: every path by which an id from the wire becomes editor state.
// DELIBERATELY not `id > 0`. Most id checks in this codebase (`routing.ts` and eight screens) use
// `Number.isInteger(id) && id > 0` because they are asking "could this id EXIST?". This predicate
// asks a different question — "can the API BIND this value as its `int` parameter?" — so `0` and
// negatives are accepted: they are perfectly bindable, and rejecting them here would silently
// convert a server-side 404/422 (a clear answer) into a client-side "no selection" (a confusing
// one). Do not "fix" the inconsistency; the two predicates are answering different questions.
export const INT32_MIN = -2_147_483_648;
export const INT32_MAX = 2_147_483_647;
export function isSelectionId(value: unknown): value is number {
return typeof value === 'number' && Number.isInteger(value) && value >= INT32_MIN && value <= INT32_MAX;
}
// Normalize an id arriving from the wire into editor state. Anything the API cannot bind is treated
// as ABSENT rather than carried: the editor then shows "no selection" with Save disabled — visible
// and honest — instead of a value that looks selected and fails on submit. Never silently coerce
// (rounding 1.5 to 1 would submit a *different* record).
export function selectionIdOrNull(value: unknown): null | number {
return isSelectionId(value) ? value : null;
}
+506 -3
View File
@@ -1,7 +1,8 @@
import { cleanup, fireEvent, render, screen, waitFor, within } from '@testing-library/react';
import { act, cleanup, fireEvent, render, renderHook, screen, waitFor, within } from '@testing-library/react';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { applyDesignSystemTheme } from '../designSystem';
import { ChannelBuilderScreen } from './ChannelBuilder';
import { useLibraryBrowse } from './libraryBrowse';
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
@@ -98,6 +99,7 @@ function requestBodyFor(path: string): Record<string, unknown> {
function mockBuilderApi({
artworkUploadFailure = null,
browseErrorOn = null,
browseHandler = null,
browseItems = [],
channels = [],
@@ -109,7 +111,15 @@ function mockBuilderApi({
fromLineupResponse = { channelId: 1, playlistId: null, playoutId: 3, programScheduleId: 2 }
}: {
artworkUploadFailure?: { status: number } | null;
browseHandler?: ((search: URLSearchParams) => { page: unknown[]; totalCount: number }) | null;
// Predicate over a /api/v1/library/browse request's query params: when it returns true, that
// ONE request answers with a 500 instead of a page (#650 follow-up F4 — simulates one per-kind
// fan-out request rejecting mid-Promise.all).
browseErrorOn?: ((search: URLSearchParams) => boolean) | null;
// May return the page synchronously OR a Promise of one (#650 follow-up F3 — lets a test hold a
// specific request open to simulate a superseded in-flight append).
browseHandler?:
| ((search: URLSearchParams) => { page: unknown[]; totalCount: number } | Promise<{ page: unknown[]; totalCount: number }>)
| null;
browseItems?: unknown[];
channels?: unknown[];
channelTemplates?: unknown[];
@@ -138,8 +148,13 @@ function mockBuilderApi({
if (path.startsWith('/api/v1/library/browse')) {
const search = new URL(path, window.location.origin).searchParams;
if (browseErrorOn?.(search)) {
return Promise.resolve(new Response(null, { status: 500 }));
}
if (browseHandler) {
return Promise.resolve(jsonResponse(browseHandler(search)));
return Promise.resolve(browseHandler(search)).then((body) => jsonResponse(body));
}
const mediaType = search.get('mediaType');
@@ -708,6 +723,314 @@ describe('Channel Builder (#89)', () => {
expect(screen.queryByRole('button', { name: 'Load more' })).not.toBeInTheDocument();
});
it(
'preserves already-loaded rows when a "Load more" append request fails, and lets it be retried ' +
'(#650 follow-up F4)',
async () => {
const page0 = [browseItem({ id: 1, mediaItemId: 1, title: 'Item A' })];
// Loaded once via `Load more`, then again on retry after the first attempt fails.
const page1 = [browseItem({ id: 2, mediaItemId: 2, title: 'Item B' })];
let page1Attempts = 0;
mockBuilderApi({
...builderDefaults(),
browseHandler: (search) => {
if (search.get('mediaType') !== 'TelevisionShow') {
return { page: [], totalCount: 0 };
}
const pageNum = Number(search.get('pageNum') ?? '0');
if (pageNum === 0) {
return { page: page0, totalCount: 3 };
}
page1Attempts += 1;
if (page1Attempts === 1) {
return Promise.reject(new Error('network blip'));
}
return { page: page1, totalCount: 3 };
}
});
await renderBuilder();
expect(await screen.findByText('Item A')).toBeInTheDocument();
const loadMoreButton = screen.getByRole('button', { name: 'Load more' });
fireEvent.click(loadMoreButton);
// The rejected append must NOT wipe Item A — only report the failure inline.
expect(await screen.findByText('network blip')).toBeInTheDocument();
expect(screen.getByText('Item A')).toBeInTheDocument();
expect(screen.queryByText('No titles match.')).not.toBeInTheDocument();
// The button survives the failure so the SAME page can be retried (the page cursor rolls
// back rather than skipping ahead to a page that was never fetched).
const retryButton = screen.getByRole('button', { name: 'Load more' });
fireEvent.click(retryButton);
expect(await screen.findByText('Item B')).toBeInTheDocument();
expect(screen.getByText('Item A')).toBeInTheDocument();
expect(page1Attempts).toBe(2);
}
);
it(
'clears the "Load more" spinner instead of stranding it when the query changes while an ' +
'append request is still in flight (#650 follow-up F3)',
async () => {
const page0ForA = [browseItem({ id: 1, mediaItemId: 1, title: 'Item A' })];
const page0ForZ = [browseItem({ id: 9, mediaItemId: 9, title: 'Item Z' })];
// No-op initializer (not `null`) matching the established gate-helper pattern elsewhere in
// this repo (see `api/libraries.test.ts`'s `releasePost`) — avoids a `tsc -b` project-mode
// control-flow quirk where a `let x: (() => void) | null = null` reassigned only inside a
// Promise executor narrows a later `x?.()` call to `never` (not caught by a bare
// `tsc --noEmit`, only by the real `npm run typecheck` gate).
let releaseStrandedAppend = () => {};
const strandedAppendGate = new Promise<void>((resolve) => {
releaseStrandedAppend = resolve;
});
mockBuilderApi({
...builderDefaults(),
browseHandler: async (search) => {
if (search.get('mediaType') !== 'TelevisionShow') {
return { page: [], totalCount: 0 };
}
const query = search.get('query') ?? '';
const pageNum = Number(search.get('pageNum') ?? '0');
if (pageNum === 0) {
return { page: query === 'zzz' ? page0ForZ : page0ForA, totalCount: 3 };
}
// The one append (page 1) request never resolves until the test releases it — long
// after the query change below has superseded it.
await strandedAppendGate;
return { page: [browseItem({ id: 2, mediaItemId: 2, title: 'Item B' })], totalCount: 3 };
}
});
await renderBuilder();
expect(await screen.findByText('Item A')).toBeInTheDocument();
const loadMoreButton = screen.getByRole('button', { name: 'Load more' });
fireEvent.click(loadMoreButton);
expect((loadMoreButton as HTMLButtonElement).disabled).toBe(true);
// Change the query while the append is still in flight — this supersedes it with a fresh
// (non-append) page-0 fetch.
fireEvent.change(screen.getByPlaceholderText('Search shows & movies…'), { target: { value: 'zzz' } });
// Without the F3 fix, the superseded append's `finally` never fires its `setLoadingMore(false)`
// (its own request id no longer matches) AND the new fetch's `finally` only clears it when
// `append` is true — so the button would stay disabled/loading forever.
expect(await screen.findByText('Item Z')).toBeInTheDocument();
await waitFor(() => {
expect((screen.getByRole('button', { name: 'Load more' }) as HTMLButtonElement).disabled).toBe(false);
});
// Let the stranded append resolve too, after the fact — it must not resurrect stale rows.
releaseStrandedAppend();
await waitFor(() => expect(screen.getByText('Item Z')).toBeInTheDocument());
expect(screen.queryByText('Item A')).not.toBeInTheDocument();
}
);
it(
'single-flight: a "Load more" click while a page-0 refresh is in flight is IGNORED (not ' +
'queued), so a naive inverse settlement order (page 1 resolving before page 0) is no ' +
'longer reachable — page 1 is only ever requested AFTER page 0 settles, and both end up ' +
'present with the cursor correctly at page 2 for the next click (#650 follow-up round 3)',
async () => {
// Round 1 fixed "page 0 settles first, wrongly clears the append spinner". Round 3's
// review found the MIRROR image reachable through the exact same click sequence: if the
// click were still allowed to start B/page-1 while B/page-0 was in flight, page 1 could
// resolve FIRST, append onto stale query-A rows, and then page 0's later resolution would
// silently replace/erase it without resetting the cursor — permanently losing page 1. Round
// 3's fix removes the click's ability to start a second fetch AT ALL while any fetch (page-0
// refresh or append) is outstanding, so this ordering can't arise irrespective of which
// network response happens to land first.
const pageA0 = [browseItem({ id: 1, mediaItemId: 1, title: 'Item A1' })];
const pageB0 = [browseItem({ id: 2, mediaItemId: 2, title: 'Item B1' })];
const pageB1 = [browseItem({ id: 3, mediaItemId: 3, title: 'Item B2' })];
const pageB2 = [browseItem({ id: 4, mediaItemId: 4, title: 'Item B3' })];
// No-op initializer (not `null`) — see the note on `releaseStrandedAppend` above for why.
let releaseB0 = () => {};
const gateB0 = new Promise<void>((resolve) => {
releaseB0 = resolve;
});
const televisionShowPageNumsForB: number[] = [];
const fetchMock = mockBuilderApi({
...builderDefaults(),
browseHandler: async (search) => {
if (search.get('mediaType') !== 'TelevisionShow') {
return { page: [], totalCount: 0 };
}
const query = search.get('query') ?? '';
const pageNum = Number(search.get('pageNum') ?? '0');
if (query === '') {
// Query A: resolves immediately, canLoadMore true (1 of 4).
return { page: pageA0, totalCount: 4 };
}
televisionShowPageNumsForB.push(pageNum);
if (pageNum === 0) {
await gateB0;
return { page: pageB0, totalCount: 4 };
}
if (pageNum === 1) {
return { page: pageB1, totalCount: 4 };
}
return { page: pageB2, totalCount: 4 };
}
});
await renderBuilder();
expect(await screen.findByText('Item A1')).toBeInTheDocument();
const loadMoreButton = screen.getByRole('button', { name: 'Load more' });
expect((loadMoreButton as HTMLButtonElement).disabled).toBe(false);
// Change the query — the search box debounces (280ms) before `query` state (and the reqId
// bump) actually happens, so wait for B/page-0's request to genuinely be ISSUED before
// clicking — otherwise the click below would still target query A's generation.
fireEvent.change(screen.getByPlaceholderText('Search shows & movies…'), { target: { value: 'B' } });
await waitFor(() => {
const b0Calls = fetchMock.mock.calls.filter(([input]) => {
const url = new URL(input.toString(), 'http://localhost');
return (
url.pathname === '/api/v1/library/browse' &&
url.searchParams.get('mediaType') === 'TelevisionShow' &&
url.searchParams.get('query') === 'B' &&
(url.searchParams.get('pageNum') ?? '0') === '0'
);
});
expect(b0Calls.length).toBeGreaterThan(0);
});
// Click "Load more" WHILE B/page-0 is still in flight. Single-flight must IGNORE this —
// no B/page-1 request may be issued yet.
fireEvent.click(loadMoreButton);
expect(televisionShowPageNumsForB).toEqual([0]);
// Release B/page-0 — items replace the stale A rows.
releaseB0();
await screen.findByText('Item B1');
await waitFor(() => {
expect((screen.getByRole('button', { name: 'Load more' }) as HTMLButtonElement).disabled).toBe(false);
});
// NOW a click is genuinely free to fetch page 1 — click again (the earlier click while busy
// was a no-op, not a queued request).
fireEvent.click(screen.getByRole('button', { name: 'Load more' }));
await screen.findByText('Item B2');
expect(screen.getByText('Item B1')).toBeInTheDocument();
expect(televisionShowPageNumsForB).toEqual([0, 1]);
// The cursor is correctly at page 2 for the next click — not skipped, not repeated.
fireEvent.click(screen.getByRole('button', { name: 'Load more' }));
await screen.findByText('Item B3');
expect(televisionShowPageNumsForB).toEqual([0, 1, 2]);
}
);
it(
'fires "Load more" in Collections mode once merged items reach the 100-row server cap, using ' +
'the REAL summed totalCount rather than the truncated page length (#650)',
async () => {
// Boundary: 100 vs 101 Collection-kind rows. #650's bug reported `merged.length` as
// `totalCount`, so `items.length < totalCount` was always `100 < 100` (false) — this pins
// that canLoadMore now reads the real totalCount (101) and DOES fire past the cap.
const page0 = Array.from({ length: 100 }, (_, i) =>
browseItem({
id: i + 1,
mediaItemId: undefined,
collectionId: i + 1,
collectionType: 'Collection',
mediaType: 'Collection',
title: `Coll ${String(i + 1).padStart(3, '0')}`
})
);
const page1 = [
browseItem({
id: 101,
mediaItemId: undefined,
collectionId: 101,
collectionType: 'Collection',
mediaType: 'Collection',
title: 'Coll 101'
})
];
mockBuilderApi({
...builderDefaults(),
browseHandler: (search) => {
if (search.get('mediaType') !== 'Collection') {
return { page: [], totalCount: 0 };
}
const pageNum = Number(search.get('pageNum') ?? '0');
return { page: pageNum === 0 ? page0 : page1, totalCount: 101 };
}
});
await renderBuilder();
fireEvent.click(screen.getByRole('button', { name: 'Collections' }));
expect(await screen.findByText('Coll 001')).toBeInTheDocument();
expect(screen.getByText('Coll 100')).toBeInTheDocument();
expect(screen.queryByText('Coll 101')).not.toBeInTheDocument();
const loadMoreButton = screen.getByRole('button', { name: 'Load more' });
fireEvent.click(loadMoreButton);
expect(await screen.findByText('Coll 101')).toBeInTheDocument();
expect(screen.getByText('Coll 001')).toBeInTheDocument();
expect(screen.queryByRole('button', { name: 'Load more' })).not.toBeInTheDocument();
}
);
it('surfaces a truncation hint in the Seasons drill-in when a show has more than 100 seasons (#650)', async () => {
// TelevisionSeason browse is scoped to one show (parentId) and requests a single page at the
// cap (Class B, spa-conventions §3b) — no real show has 100+ seasons, but the response's
// totalCount must still be read and surfaced rather than silently truncating if it somehow did.
const seasons = Array.from({ length: 100 }, (_, i) =>
browseItem({
id: 200 + i,
mediaItemId: 200 + i,
mediaType: 'TelevisionSeason',
title: `Season ${i + 1}`
})
);
mockBuilderApi({
...builderDefaults(),
browseItems: [browseItem()],
browseHandler: (search) => {
if (search.get('mediaType') === 'TelevisionSeason') {
return { page: seasons, totalCount: 140 };
}
if (search.get('mediaType') === 'TelevisionShow') {
return { page: [browseItem()], totalCount: 1 };
}
return { page: [], totalCount: 0 };
}
});
await renderBuilder();
expect(await screen.findByText('Looney Tunes')).toBeInTheDocument();
fireEvent.click(screen.getByRole('button', { name: 'Seasons' }));
expect(await screen.findByText('Season 1')).toBeInTheDocument();
expect(screen.getByText('Showing the first 100 of 140 seasons.')).toBeInTheDocument();
});
it('saves the current settings as a new template and selects it', async () => {
const created = channelTemplate({
id: 55,
@@ -891,4 +1214,184 @@ describe('Channel Builder (#89)', () => {
expect(document.documentElement).toHaveAttribute('data-theme', 'dual');
expect(screen.getByPlaceholderText('Search shows & movies…')).toBeInTheDocument();
});
it(
'single-flight: calling loadMore() twice back-to-back issues only ONE fetch — the second ' +
'call is ignored, not queued as a second overlapping request (#650 follow-up round 3)',
async () => {
// Drives the hook directly via `renderHook` (bypassing the DOM/disabled-button layer
// entirely) to prove the guard lives in the HOOK, not just the presentation: even a
// synchronous double-call cannot start two fetches.
const page0 = [browseItem({ id: 1, mediaItemId: 1, mediaType: 'TelevisionShow', title: 'Item A' })];
const page1Items = [browseItem({ id: 2, mediaItemId: 2, mediaType: 'TelevisionShow', title: 'Item B' })];
const televisionShowPageNumsRequested: number[] = [];
vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL) => {
const url = new URL(input.toString(), 'http://localhost');
if (url.pathname !== '/api/v1/library/browse') {
return Promise.resolve(new Response(null, { status: 204 }));
}
const mediaType = url.searchParams.get('mediaType');
if (mediaType !== 'TelevisionShow') {
return Promise.resolve(jsonResponse({ page: [], totalCount: 0 }));
}
const pageNum = Number(url.searchParams.get('pageNum') ?? '0');
televisionShowPageNumsRequested.push(pageNum);
if (pageNum === 0) {
return Promise.resolve(jsonResponse({ page: page0, totalCount: 5 }));
}
return Promise.resolve(jsonResponse({ page: page1Items, totalCount: 5 }));
});
const { result } = renderHook(() => useLibraryBrowse('library', '', null));
await waitFor(() => expect(result.current.state.status).toBe('success'));
act(() => {
result.current.loadMore();
result.current.loadMore();
});
await waitFor(() => expect(result.current.state.items.map((item) => item.title)).toContain('Item B'));
// Only ONE page-1 request was ever issued — the second synchronous call was a no-op.
expect(televisionShowPageNumsRequested).toEqual([0, 1]);
expect(result.current.state.items.map((item) => item.title)).toEqual(['Item A', 'Item B']);
}
);
it(
'single-flight: calling loadMore() directly while a page-0 refresh is still in flight is ' +
'IGNORED — no page-1 request is ever issued (#650 follow-up round 4)',
async () => {
// ChannelBuilder.test.tsx's component-level single-flight test asserts on the DISABLED
// BUTTON — during a page-0 refresh, `loadingMore` disables the native button, so
// `fireEvent.click` never even reaches `loadMore`. That test would stay green if the
// generation-change effect's `busyRef.current = true` (libraryBrowse.ts) were deleted,
// because the presentation layer's own disabling would still mask the missing guard. This
// test drives `loadMore()` directly via `renderHook`, bypassing the button entirely, so it
// exercises the HOOK's own guard rather than something adjacent to it.
const pageA0 = [browseItem({ id: 1, mediaItemId: 1, mediaType: 'TelevisionShow', title: 'Item A' })];
const pageB0 = [browseItem({ id: 2, mediaItemId: 2, mediaType: 'TelevisionShow', title: 'Item B0' })];
const pageB1 = [browseItem({ id: 3, mediaItemId: 3, mediaType: 'TelevisionShow', title: 'Item B1' })];
// No-op initializer (not `null`) — see the note on `releaseStrandedAppend` above for why.
let releaseB0 = () => {};
const gateB0 = new Promise<void>((resolve) => {
releaseB0 = resolve;
});
const televisionShowPageNumsForB: number[] = [];
vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL) => {
const url = new URL(input.toString(), 'http://localhost');
if (url.pathname !== '/api/v1/library/browse') {
return Promise.resolve(new Response(null, { status: 204 }));
}
const mediaType = url.searchParams.get('mediaType');
if (mediaType !== 'TelevisionShow') {
return Promise.resolve(jsonResponse({ page: [], totalCount: 0 }));
}
const query = url.searchParams.get('query') ?? '';
const pageNum = Number(url.searchParams.get('pageNum') ?? '0');
if (query === '') {
return Promise.resolve(jsonResponse({ page: pageA0, totalCount: 4 }));
}
televisionShowPageNumsForB.push(pageNum);
if (pageNum === 0) {
return gateB0.then(() => jsonResponse({ page: pageB0, totalCount: 4 }));
}
return Promise.resolve(jsonResponse({ page: pageB1, totalCount: 4 }));
});
const { result, rerender } = renderHook(({ query }) => useLibraryBrowse('library', query, null), {
initialProps: { query: '' }
});
await waitFor(() => expect(result.current.state.status).toBe('success'));
expect(result.current.state.items.map((item) => item.title)).toEqual(['Item A']);
// Change the query — the generation-change effect fires immediately (no debounce at the
// hook level; that's a ChannelBuilderScreen-only concern) and issues B/page-0, held open by
// gateB0.
rerender({ query: 'B' });
await waitFor(() => expect(televisionShowPageNumsForB).toEqual([0]));
// Call loadMore() DIRECTLY while B/page-0 is still in flight — single-flight must ignore
// this outright, not queue a page-1 request behind it.
act(() => {
result.current.loadMore();
});
expect(televisionShowPageNumsForB).toEqual([0]);
// Let B/page-0 resolve; still no page-1 request should have appeared (the ignored call
// above must not have been queued for later, either).
releaseB0();
await waitFor(() => expect(result.current.state.items.map((item) => item.title)).toContain('Item B0'));
expect(televisionShowPageNumsForB).toEqual([0]);
// Now that the single-flight slot is free, a genuine call correctly fetches page 1.
act(() => {
result.current.loadMore();
});
await waitFor(() => expect(result.current.state.items.map((item) => item.title)).toContain('Item B1'));
expect(televisionShowPageNumsForB).toEqual([0, 1]);
}
);
it(
'a failed "Load more" page is retried as the SAME page number, not skipped or duplicated ' +
'(#650 follow-up round 3)',
async () => {
const page0 = [browseItem({ id: 1, mediaItemId: 1, mediaType: 'TelevisionShow', title: 'Item A' })];
const page1Items = [browseItem({ id: 2, mediaItemId: 2, mediaType: 'TelevisionShow', title: 'Item B' })];
const televisionShowPageNumsRequested: number[] = [];
let page1Attempts = 0;
vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL) => {
const url = new URL(input.toString(), 'http://localhost');
if (url.pathname !== '/api/v1/library/browse') {
return Promise.resolve(new Response(null, { status: 204 }));
}
const mediaType = url.searchParams.get('mediaType');
if (mediaType !== 'TelevisionShow') {
return Promise.resolve(jsonResponse({ page: [], totalCount: 0 }));
}
const pageNum = Number(url.searchParams.get('pageNum') ?? '0');
televisionShowPageNumsRequested.push(pageNum);
if (pageNum === 0) {
return Promise.resolve(jsonResponse({ page: page0, totalCount: 5 }));
}
if (pageNum === 1) {
page1Attempts += 1;
if (page1Attempts === 1) {
return Promise.resolve(new Response(null, { status: 500 }));
}
return Promise.resolve(jsonResponse({ page: page1Items, totalCount: 5 }));
}
return Promise.resolve(jsonResponse({ page: [], totalCount: 0 }));
});
const { result } = renderHook(() => useLibraryBrowse('library', '', null));
await waitFor(() => expect(result.current.state.status).toBe('success'));
act(() => {
result.current.loadMore();
});
await waitFor(() => expect(result.current.loadMoreError).not.toBeNull());
// The failed page must not be silently skipped: items stay at just page 0's row, and the
// NEXT "Load more" call must re-request page 1 — never page 2 — or that row is permanently
// lost. (An earlier compare-and-set rollback attempt made this assertion fail: it left the
// cursor at page 1 only when nothing else had raced ahead, but a genuinely single-flight
// hook has no "else" case left to distinguish — the unconditional rollback below is what
// that guarantee simplifies down to.)
expect(result.current.state.items.map((item) => item.title)).toEqual(['Item A']);
expect(televisionShowPageNumsRequested).toEqual([0, 1]);
act(() => {
result.current.loadMore();
});
await waitFor(() => expect(result.current.state.items.map((item) => item.title)).toContain('Item B'));
expect(televisionShowPageNumsRequested).toEqual([0, 1, 1]);
expect(result.current.loadMoreError).toBeNull();
}
);
});
+38 -146
View File
@@ -59,7 +59,6 @@ import {
type FFmpegProfile,
type FillerPreset,
type LibraryBrowseItem,
type LibraryBrowseMediaType,
type MediaSource,
type Watermark
} from '../api';
@@ -107,6 +106,7 @@ import {
type SubtitleMode,
type TranscodeMode
} from './advancedOptions';
import { useLibraryBrowse } from './libraryBrowse';
import { SmartCollectionDialog } from './SmartCollectionDialog';
// Advanced-options model (enum unions, INHERIT/omit semantics, the override hook)
@@ -115,26 +115,6 @@ import { SmartCollectionDialog } from './SmartCollectionDialog';
// TYPE_ICON / TYPE_LABEL / hueOf are shared with the media browse/search/trash screens
// (see ../media/mediaKinds) so the per-kind icon+label map has a single source of truth.
const COLLECTION_MEDIA_TYPES: LibraryBrowseMediaType[] = [
'Collection',
'SmartCollection',
'MultiCollection',
'RerunCollection',
'Playlist'
];
// Browsing a library shows the "pickable" top-level kinds only, not every
// episode/song/etc. nested underneath them. Kept explicit here since
// `GET /api/v1/library/browse` now spans all 10 media kinds when `mediaType` is
// omitted. TelevisionSeason is intentionally excluded so a multi-season show
// renders as a single tile instead of flooding the grid with per-season tiles
// (issue #180); seasons are reachable via the show tile's Seasons drill-in.
const LIBRARY_MEDIA_TYPES: LibraryBrowseMediaType[] = [
'Movie',
'TelevisionShow',
'Artist'
];
// ---- Small pure helpers ----------------------------------------------------
const mono: CSSProperties = { fontFamily: 'var(--font-mono)', fontVariantNumeric: 'tabular-nums' };
@@ -433,15 +413,22 @@ function SeasonsDialog({
onClose: () => void;
}) {
const [seasons, setSeasons] = useState<LibraryBrowseItem[]>([]);
const [totalCount, setTotalCount] = useState(0);
const [status, setStatus] = useState<'loading' | 'error' | 'success'>('loading');
const [error, setError] = useState<string | null>(null);
useEffect(() => {
let active = true;
// Class B (media-library-scoped, spa-conventions §3b): a single bounded page at the
// server cap is intentional here — no real show has anywhere near 100 seasons, so this
// isn't paged to completeness — but #650 found the response's real `totalCount` went
// unread, so a show that somehow did exceed the cap would truncate silently. Read it and
// surface it below instead, same as the other Class B pickers.
getLibraryBrowseItems({ mediaType: 'TelevisionSeason', parentId: show.id, pageSize: 100 })
.then((result) => {
if (active) {
setSeasons(result.page ?? []);
setTotalCount(result.totalCount ?? (result.page ?? []).length);
setStatus('success');
}
})
@@ -479,136 +466,36 @@ function SeasonsDialog({
) : seasons.length === 0 ? (
<div className="ctv-builder-empty">This show has no seasons.</div>
) : (
<div className="ctv-builder-seasons-list">
{seasons.map((season) => {
const added = addedKeys.has(lineupKey(season));
return (
<div key={lineupKey(season)} className="ctv-builder-seasons-row">
<Poster item={season} width={40} height={54} mini />
<div className="ctv-builder-seasons-row-title">{season.title}</div>
<IconButton size="sm" title={added ? 'Added' : 'Add to lineup'} onClick={() => onAdd(season)}>
{added ? (
<Check size={15} color="var(--action-primary)" aria-hidden="true" />
) : (
<Plus size={15} aria-hidden="true" />
)}
</IconButton>
</div>
);
})}
</div>
<>
{seasons.length < totalCount && (
<span className="ctv-field-help">
Showing the first {seasons.length} of {totalCount} seasons.
</span>
)}
<div className="ctv-builder-seasons-list">
{seasons.map((season) => {
const added = addedKeys.has(lineupKey(season));
return (
<div key={lineupKey(season)} className="ctv-builder-seasons-row">
<Poster item={season} width={40} height={54} mini />
<div className="ctv-builder-seasons-row-title">{season.title}</div>
<IconButton size="sm" title={added ? 'Added' : 'Add to lineup'} onClick={() => onAdd(season)}>
{added ? (
<Check size={15} color="var(--action-primary)" aria-hidden="true" />
) : (
<Plus size={15} aria-hidden="true" />
)}
</IconButton>
</div>
);
})}
</div>
</>
)}
</Dialog>
);
}
// ---- Library browse data hook ---------------------------------------------
interface BrowseState {
status: 'loading' | 'error' | 'success';
items: LibraryBrowseItem[];
totalCount: number;
error: string | null;
}
async function loadCollections(query: string): Promise<{ page: LibraryBrowseItem[]; totalCount: number }> {
const results = await Promise.all(
COLLECTION_MEDIA_TYPES.map((mediaType) => getLibraryBrowseItems({ query, mediaType, pageSize: 100 }))
);
const merged = results
.flatMap((result) => result.page)
.sort((left, right) => left.title.localeCompare(right.title, undefined, { sensitivity: 'base' }));
return { page: merged, totalCount: merged.length };
}
// Fan out across the pickable top-level library kinds (movies/shows/artists)
// instead of the unscoped browse, which now also returns episodes/songs/etc.
// Mirrors loadCollections's per-kind Promise.all pattern, but preserves real
// paging (each kind keeps its own pageNum/totalCount, summed across kinds) so
// canLoadMore stays meaningful for large libraries.
async function loadLibraryItems(
query: string,
libraryId: number | null,
pageNum: number,
pageSize: number
): Promise<{ page: LibraryBrowseItem[]; totalCount: number }> {
const results = await Promise.all(
LIBRARY_MEDIA_TYPES.map((mediaType) =>
getLibraryBrowseItems({
query: query || undefined,
libraryId: libraryId ?? undefined,
mediaType,
pageNum,
pageSize
})
)
);
const merged = results
.flatMap((result) => result.page)
.sort((left, right) => left.title.localeCompare(right.title, undefined, { sensitivity: 'base' }));
const totalCount = results.reduce((sum, result) => sum + result.totalCount, 0);
return { page: merged, totalCount };
}
function useLibraryBrowse(source: 'library' | 'collections', query: string, libraryId: number | null) {
const [state, setState] = useState<BrowseState>({ status: 'loading', items: [], totalCount: 0, error: null });
const [loadingMore, setLoadingMore] = useState(false);
const reqRef = useRef(0);
const pageRef = useRef(0);
const runFetch = useCallback(
(pageNum: number, reqId: number, append: boolean) => {
const promise =
source === 'collections' ? loadCollections(query) : loadLibraryItems(query, libraryId, pageNum, 100);
promise
.then((result) => {
if (reqRef.current !== reqId) {
return;
}
setState((prev) => ({
status: 'success',
error: null,
items: append ? [...prev.items, ...result.page] : result.page,
totalCount: result.totalCount
}));
})
.catch((error: unknown) => {
if (reqRef.current !== reqId) {
return;
}
setState({ status: 'error', items: [], totalCount: 0, error: messageFromError(error) });
})
.finally(() => {
if (reqRef.current === reqId && append) {
setLoadingMore(false);
}
});
},
[source, query, libraryId]
);
// Refetch when inputs change. Following the guide-hook pattern, setState only
// runs inside runFetch's async callbacks - the previous results stay visible
// until the new page resolves (initial 'loading' shows the first-load spinner).
useEffect(() => {
const reqId = reqRef.current + 1;
reqRef.current = reqId;
pageRef.current = 0;
runFetch(0, reqId, false);
}, [runFetch]);
const loadMore = useCallback(() => {
const nextPage = pageRef.current + 1;
pageRef.current = nextPage;
setLoadingMore(true);
runFetch(nextPage, reqRef.current, true);
}, [runFetch]);
const canLoadMore = source === 'library' && state.status === 'success' && state.items.length < state.totalCount;
return { state, loadingMore, loadMore, canLoadMore };
}
// ---- Builder support data (templates, pickers, channels) -------------------
interface BuilderData {
channels: ChannelSummary[];
@@ -1276,6 +1163,11 @@ function ChannelBuilder({
<Button variant="secondary" size="sm" loading={browse.loadingMore} onClick={browse.loadMore}>
Load more
</Button>
{browse.loadMoreError && (
<span className="ctv-field-error" role="alert">
{browse.loadMoreError}
</span>
)}
</div>
)}
</>
+204
View File
@@ -0,0 +1,204 @@
import { useCallback, useEffect, useRef, useState } from 'react';
import { getLibraryBrowseItems, messageFromError, type LibraryBrowseItem, type LibraryBrowseMediaType } from '../api';
// Extracted from ChannelBuilder.tsx into its own non-JSX module (#650 follow-up) so
// `useLibraryBrowse` — a plain hook, no component — can be exported without tripping
// `react-refresh/only-export-components` (that lint rule wants a file that exports React
// Fast-Refresh components to export ONLY components). It also lets `ChannelBuilder.test.tsx`
// unit-test the hook's single-flight guard directly via `renderHook`.
export const COLLECTION_MEDIA_TYPES: LibraryBrowseMediaType[] = [
'Collection',
'SmartCollection',
'MultiCollection',
'RerunCollection',
'Playlist'
];
// Browsing a library shows the "pickable" top-level kinds only, not every
// episode/song/etc. nested underneath them. Kept explicit here since
// `GET /api/v1/library/browse` now spans all 10 media kinds when `mediaType` is
// omitted. TelevisionSeason is intentionally excluded so a multi-season show
// renders as a single tile instead of flooding the grid with per-season tiles
// (issue #180); seasons are reachable via the show tile's Seasons drill-in.
export const LIBRARY_MEDIA_TYPES: LibraryBrowseMediaType[] = ['Movie', 'TelevisionShow', 'Artist'];
// ---- Library browse data hook ---------------------------------------------
export interface BrowseState {
status: 'loading' | 'error' | 'success';
items: LibraryBrowseItem[];
totalCount: number;
error: string | null;
}
// Fan out across the "pickable" collection-ish kinds (Collection, SmartCollection,
// MultiCollection, RerunCollection, Playlist). Reports the REAL summed per-kind `totalCount`
// (not `merged.length`, i.e. the truncated page) — #650: returning the page length as the total
// made `canLoadMore`'s `items.length < totalCount` comparison permanently `merged.length >=
// merged.length`, so "load more" could never fire once any one kind hit the 100-row server cap.
// Mirrors loadLibraryItems's per-kind pageNum/totalCount pattern below.
async function loadCollections(
query: string,
pageNum: number,
pageSize: number
): Promise<{ page: LibraryBrowseItem[]; totalCount: number }> {
const results = await Promise.all(
COLLECTION_MEDIA_TYPES.map((mediaType) => getLibraryBrowseItems({ query, mediaType, pageNum, pageSize }))
);
const merged = results
.flatMap((result) => result.page)
.sort((left, right) => left.title.localeCompare(right.title, undefined, { sensitivity: 'base' }));
const totalCount = results.reduce((sum, result) => sum + result.totalCount, 0);
return { page: merged, totalCount };
}
// Fan out across the pickable top-level library kinds (movies/shows/artists)
// instead of the unscoped browse, which now also returns episodes/songs/etc.
// Mirrors loadCollections's per-kind Promise.all pattern, but preserves real
// paging (each kind keeps its own pageNum/totalCount, summed across kinds) so
// canLoadMore stays meaningful for large libraries.
async function loadLibraryItems(
query: string,
libraryId: number | null,
pageNum: number,
pageSize: number
): Promise<{ page: LibraryBrowseItem[]; totalCount: number }> {
const results = await Promise.all(
LIBRARY_MEDIA_TYPES.map((mediaType) =>
getLibraryBrowseItems({
query: query || undefined,
libraryId: libraryId ?? undefined,
mediaType,
pageNum,
pageSize
})
)
);
const merged = results
.flatMap((result) => result.page)
.sort((left, right) => left.title.localeCompare(right.title, undefined, { sensitivity: 'base' }));
const totalCount = results.reduce((sum, result) => sum + result.totalCount, 0);
return { page: merged, totalCount };
}
/**
* #650 follow-up (review round 3): rounds 1-2 kept fixing one overlapping page-0-refresh/append
* interleaving only to have the reviewer find its mirror image (page-0-settles-first, then
* page-1-settles-first; a compare-and-set rollback that traded duplication for a permanently
* skipped page). The hook still PERMITTED a page-0 refresh and an append to be in flight
* together, so every ordering of "which one wins" was a distinct bug to enumerate.
*
* This version enforces SINGLE-FLIGHT instead: at most one fetch (a page-0 refresh OR an append)
* is ever outstanding for the current query generation. `busyRef` is the guard `loadMore` reads
* it SYNCHRONOUSLY and, if a fetch is already in flight, the call is IGNORED outright (not
* queued, not superseded simply a no-op), so a second click can never start a second
* request. The generation-change effect also claims the single-flight slot for its own page-0
* fetch, so `loadMore` is a no-op while a query/library change is still resolving too not only
* while a previous append is pending. With overlapping fetches structurally impossible, the
* append-failure rollback no longer needs a compare-and-set: single-flight guarantees nothing
* else could have moved `pageRef` since this fetch started, so it always retries the exact page
* that failed.
*/
export function useLibraryBrowse(source: 'library' | 'collections', query: string, libraryId: number | null) {
const [state, setState] = useState<BrowseState>({ status: 'loading', items: [], totalCount: 0, error: null });
const [loadingMore, setLoadingMore] = useState(false);
// Append-only error, kept separate from `state.error` (#650 follow-up F4): a rejected "load
// more" must NOT flip `state.status` to 'error' — that branch renders instead of the item grid
// and would destroy every already-loaded row with no way back. This renders inline next to the
// (still-present) Load more button so the fetch can simply be retried.
const [loadMoreError, setLoadMoreError] = useState<string | null>(null);
// Query GENERATION (bumped on query/library/source change) — invalidates every fetch from an
// earlier generation, whether it was a page-0 refresh or an append.
const reqRef = useRef(0);
const pageRef = useRef(0);
// Single-flight guard: true from the moment ANY fetch (page-0 refresh or append) for the
// CURRENT generation is issued, until it settles. `loadMore` bails out synchronously while this
// is true. A plain ref (not state) is deliberate — it must be readable/writable synchronously
// inside the generation-change effect without tripping `react-hooks/set-state-in-effect`
// (setState calls belong in async callbacks only, per that rule; a ref mutation isn't one).
const busyRef = useRef(false);
const runFetch = useCallback(
(pageNum: number, reqId: number, append: boolean) => {
const promise =
source === 'collections'
? loadCollections(query, pageNum, 100)
: loadLibraryItems(query, libraryId, pageNum, 100);
promise
.then((result) => {
if (reqRef.current !== reqId) {
return;
}
setLoadMoreError(null);
setState((prev) => ({
status: 'success',
error: null,
items: append ? [...prev.items, ...result.page] : result.page,
totalCount: result.totalCount
}));
})
.catch((error: unknown) => {
if (reqRef.current !== reqId) {
return;
}
if (append) {
// Single-flight (see the hook's doc comment above) guarantees NOTHING else could
// have advanced `pageRef` since this exact fetch started — no compare-and-set needed,
// just always roll back to retry the SAME page that failed.
pageRef.current = pageNum - 1;
setLoadMoreError(messageFromError(error));
} else {
setState({ status: 'error', items: [], totalCount: 0, error: messageFromError(error) });
}
})
.finally(() => {
if (reqRef.current !== reqId) {
// An abandoned generation's fetch settling late: a newer generation's own fetch
// already owns (or has already released) the single-flight slot — do not touch it.
return;
}
busyRef.current = false;
setLoadingMore(false);
});
},
[source, query, libraryId]
);
// Refetch when inputs change. Following the guide-hook pattern, setState only
// runs inside runFetch's async callbacks - the previous results stay visible
// until the new page resolves (initial 'loading' shows the first-load spinner).
useEffect(() => {
const reqId = reqRef.current + 1;
reqRef.current = reqId;
pageRef.current = 0;
// Claims the single-flight slot for this generation's own page-0 fetch — `loadMore` is a
// no-op until it settles, closing the exact window earlier rounds kept losing: a "Load more"
// click that lands while a fresh query's page-0 refresh is still in flight.
busyRef.current = true;
queueMicrotask(() => setLoadingMore(true));
runFetch(0, reqId, false);
}, [runFetch]);
const loadMore = useCallback(() => {
if (busyRef.current) {
// Single-flight: ignored outright rather than queued. Queuing (or racing) a second fetch
// against whichever one is already in flight is exactly what produced every
// page-ordering defect earlier rounds kept chasing one interleaving at a time.
return;
}
busyRef.current = true;
const nextPage = pageRef.current + 1;
pageRef.current = nextPage;
setLoadingMore(true);
setLoadMoreError(null);
runFetch(nextPage, reqRef.current, true);
}, [runFetch]);
// #650: `totalCount` is now the real summed per-kind total for both sources (loadCollections
// no longer reports the truncated page length as the total), so this comparison is meaningful
// for 'collections' too — no need to gate it to 'library' only.
const canLoadMore = state.status === 'success' && state.items.length < state.totalCount;
return { state, loadingMore, loadMore, canLoadMore, loadMoreError };
}
@@ -156,6 +156,15 @@ describe('RuleBuilder', () => {
const { container } = setupControlled({ match: 'all', children: [{ field: 'genre', operator: 'is', value: '' }] });
const input = screen.getByLabelText('Value') as HTMLInputElement;
// #578: nothing is fetched until the row is focused — mounting N text rules must not fire N
// requests nobody asked for.
await act(async () => {
await vi.advanceTimersByTimeAsync(400);
});
expect(getSearchFieldValues).not.toHaveBeenCalled();
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Ac' } });
expect(input.value).toBe('Ac');
+18 -4
View File
@@ -2,6 +2,7 @@ import { useEffect, useId, useRef, useState } from 'react';
import { Plus, Trash2 } from 'lucide-react';
import { Badge, Button, IconButton } from '../../components';
import { getSearchFieldValues } from '../../api/search';
import { useIsMountedRef } from '../../hooks';
import type { RuleField } from './fieldCatalog';
import {
isGroup,
@@ -48,29 +49,41 @@ function typeOf(fields: RuleField[], name: string): FieldType {
// Facet-value typeahead for text-field rules (#434): backed by a `<datalist>` so free-text is always
// accepted alongside the server-suggested values (the query preview count is the safety net for a
// value that doesn't match anything). Debounces the lookup ~200ms after each keystroke and drops any
// response that arrives after a newer request was issued or the field changed underneath it.
// response that arrives after a newer request was issued, the field changed underneath it, or the
// row unmounted.
//
// #578: the lookup is armed on FOCUS, not on mount. Fetching on mount meant a rule tree with N text
// rows issued N requests just to render — each one bounded server-side (`limit=50`) but none of them
// asked for, matching the `SearchPicker` precedent of not querying a picker nobody has touched. Once
// focused, an empty value still fetches: that top-50 list IS the datalist's browse affordance.
function TextValueInput({ field, value, onChange }: { field: string; value: string; onChange: (v: string) => void }) {
const [suggestions, setSuggestions] = useState<string[]>([]);
const [armed, setArmed] = useState(false);
const seqRef = useRef(0);
const mountedRef = useIsMountedRef();
const listId = useId();
useEffect(() => {
if (!armed) {
return;
}
const seq = ++seqRef.current;
const handle = window.setTimeout(() => {
getSearchFieldValues(field, value.trim())
.then((values) => {
if (seqRef.current === seq) {
if (mountedRef.current && seqRef.current === seq) {
setSuggestions(values);
}
})
.catch(() => {
if (seqRef.current === seq) {
if (mountedRef.current && seqRef.current === seq) {
setSuggestions([]);
}
});
}, 200);
return () => window.clearTimeout(handle);
}, [field, value]);
}, [armed, field, value, mountedRef]);
return (
<>
@@ -79,6 +92,7 @@ function TextValueInput({ field, value, onChange }: { field: string; value: stri
className="ctv-input"
value={value}
onChange={(e) => onChange(e.target.value)}
onFocus={() => setArmed(true)}
type="text"
list={listId}
autoComplete="off"
+53 -2
View File
@@ -20,12 +20,16 @@ const OPS: Record<FieldType, Operator[]> = {
date: ['before', 'after', 'between']
};
// Deterministic LCG so a failing case is reproducible.
// Deterministic LCG so a failing case is reproducible. Divide by 2^32, NOT by 0xffffffff (2^32-1):
// the latter returns exactly 1.0 for the maximum state, and `arr[Math.floor(1.0 * len)]` indexes one
// past the end. Deterministically unreachable on today's seeds, but a latent flake on any new one
// (#578). The generator's own contract is pinned by the "LCG generator" describe below, which
// drives `pick` at the boundary directly rather than waiting for a seed change to expose it.
function lcg(seed: number) {
let s = seed >>> 0;
return () => {
s = (1664525 * s + 1013904223) >>> 0;
return s / 0xffffffff;
return s / 0x100000000;
};
}
const pick = <T,>(rng: () => number, arr: T[]): T => arr[Math.floor(rng() * arr.length)];
@@ -98,3 +102,50 @@ describe('round-trip: parse(compile(tree)) === tree', () => {
expect(deepest).toBe(MAX_GROUP_DEPTH);
});
});
// #578 / #651 review round 2: drive the REAL `lcg` into its maximum state instead of recomputing
// the divisor in the test. The generator is `s -> (1664525*s + 1013904223) mod 2^32`, which is a
// bijection (the multiplier is odd), so the seed whose FIRST step lands on 0xffffffff can be solved
// exactly: seed = (0xffffffff - 1013904223) * 1664525^-1 mod 2^32 = 653637408. On the old
// `/ 0xffffffff` divisor that first value is exactly 1.0 and `pick` indexes one past the end; a
// test that recomputes the formula would pass either way and prove nothing.
const MAX_STATE_SEED = 653637408;
describe('LCG generator', () => {
it('never returns 1.0 — not even on the step that reaches the maximum 32-bit state', () => {
const value = lcg(MAX_STATE_SEED)();
// Guards the seed itself: if the recurrence ever changes, this stops silently testing a
// mid-range state that would pass under either divisor.
expect(value).toBeGreaterThan(0.9999999);
expect(value).toBeLessThan(1);
});
it('pick() stays in bounds on the maximum-state step', () => {
const arr = ['a', 'b', 'c'];
// `arr[Math.floor(1.0 * 3)]` is `arr[3]` — undefined — on the old divisor.
expect(pick(lcg(MAX_STATE_SEED), arr)).toBe('c');
expect(pick(lcg(MAX_STATE_SEED), ['only'])).toBe('only');
});
it('pick() stays in range across the whole output of a real generator', () => {
const arr = ['a', 'b', 'c', 'd'];
for (let seed = 0; seed < 50; seed += 1) {
const rng = lcg(seed);
for (let i = 0; i < 500; i += 1) {
expect(arr).toContain(pick(rng, arr));
}
}
});
it('stays within [0, 1) for every reachable state', () => {
// Every 32-bit state IS reachable from any seed (full-period LCG), so walking a long stream
// from the maximum-state seed covers the boundary rather than hoping to stumble on it.
const rng = lcg(MAX_STATE_SEED);
for (let i = 0; i < 20000; i += 1) {
const value = rng();
expect(value).toBeGreaterThanOrEqual(0);
expect(value).toBeLessThan(1);
}
});
});
+58
View File
@@ -0,0 +1,58 @@
import { cleanup, render } from '@testing-library/react';
import { StrictMode, useEffect } from 'react';
import { afterEach, describe, expect, it } from 'vitest';
import { useIsMountedRef } from './hooks';
afterEach(cleanup);
describe('useIsMountedRef (#578)', () => {
it('reads true while mounted and false after unmount', () => {
let captured: { readonly current: boolean } | null = null;
function Probe() {
captured = useIsMountedRef();
return null;
}
const { unmount } = render(<Probe />);
// The guard every async callback reads: true for as long as the component is on screen...
expect(captured).not.toBeNull();
expect((captured as unknown as { current: boolean }).current).toBe(true);
unmount();
// ...and false afterwards, so a fetch resolving late can be dropped instead of calling setState
// on a component that no longer exists. Without the effect's cleanup this stays true, which is
// exactly the defect the hook exists to prevent.
expect((captured as unknown as { current: boolean }).current).toBe(false);
});
it('is re-armed by a StrictMode double-invoke (mount → cleanup → mount)', () => {
let captured: { readonly current: boolean } | null = null;
let effectRuns = 0;
function Probe() {
captured = useIsMountedRef();
useEffect(() => {
effectRuns += 1;
});
return null;
}
render(
<StrictMode>
<Probe />
</StrictMode>
);
// StrictMode in a dev build mounts, tears down, and remounts every effect. If that did not
// happen, this test would not be exercising re-arming at all, so assert it explicitly rather
// than assume it.
expect(effectRuns).toBeGreaterThan(1);
// The first cleanup set the flag false; the remount must set it back to true. Remove
// `ref.current = true` from the effect body and this reads false.
expect((captured as unknown as { current: boolean }).current).toBe(true);
});
});
+21
View File
@@ -0,0 +1,21 @@
import { useEffect, useRef } from 'react';
// Shared is-mounted guard for async callbacks (#578). A monotonic `seqRef` drops a STALE response
// (an older request resolving after a newer one) but says nothing about whether the component is
// still there: a fetch that resolves after unmount still matches the latest seq and still calls
// setState. Read `ref.current` in every async callback alongside the seq check.
//
// The flag is (re-)armed inside the effect rather than only at `useRef` init so a StrictMode
// double-invoke — mount, unmount, remount on the same instance — leaves it true.
export function useIsMountedRef(): { readonly current: boolean } {
const ref = useRef(true);
useEffect(() => {
ref.current = true;
return () => {
ref.current = false;
};
}, []);
return ref;
}
@@ -435,6 +435,11 @@ function SourcePicker({
selectedId={selectedId ?? null}
selectedName={selectedName ?? null}
search={search}
// The picker is NOT remounted when the item's collection type changes, so results must be
// bound to the type that produced them — otherwise a Collection hit stays clickable under a
// SmartCollection label and stores a Collection id in the SmartCollection field (#651 round 4
// HIGH-1). `type` names both the search function and the id namespace, so it IS the source.
source={type}
onSelect={(id, name) => setSelection(id, name)}
onClear={() => setSelection(null, null)}
/>
+568
View File
@@ -0,0 +1,568 @@
import { act, cleanup, fireEvent, render, screen, waitFor } from '@testing-library/react';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { SearchPicker, type SearchPickerOption } from './pickers';
// Every value `SearchPicker` read out of the shared is-mounted guard, in order. This is what makes
// the unmount test non-vacuous (#651 F7): an unmounted tree renders nothing either way, so
// asserting on the DOM cannot distinguish "the guard stopped the update" from "React discarded it".
// Recording the reads proves the component actually CONSULTS the guard — and specifically that it
// consults it once the late response lands, seeing `false`.
const mountedReads: boolean[] = [];
vi.mock('../hooks', async (importOriginal) => {
const actual = await importOriginal<typeof import('../hooks')>();
const { useRef } = await import('react');
return {
...actual,
useIsMountedRef: () => {
const inner = actual.useIsMountedRef();
// Memoized so the returned object identity is stable across renders — it is an effect
// dependency in SearchPicker, and a fresh object each render would restart the debounce.
const proxyRef = useRef<{ readonly current: boolean } | null>(null);
proxyRef.current ??= {
get current() {
mountedReads.push(inner.current);
return inner.current;
}
};
return proxyRef.current;
}
};
});
beforeEach(() => {
mountedReads.length = 0;
});
afterEach(() => {
cleanup();
vi.restoreAllMocks();
});
function renderPicker(overrides: Partial<Parameters<typeof SearchPicker>[0]> = {}) {
const props = {
label: 'Movie',
onClear: vi.fn(),
onSelect: vi.fn(),
search: vi.fn<(query: string) => Promise<SearchPickerOption[]>>().mockResolvedValue([]),
selectedId: null,
selectedName: null,
source: 'Movie',
...overrides
};
return { props, ...render(<SearchPicker {...props} />) };
}
describe('SearchPicker', () => {
it('#578: a search resolving AFTER unmount consults the is-mounted guard and is dropped', async () => {
// Holder object, not a bare `let`: TS control-flow analysis cannot see that the Promise
// executor already ran, so a plain variable narrows to `null` and `release?.()` fails to
// typecheck as `never`. (The repo already uses this shape in RerunCollectionsScreen.test.)
const release: { resolve: ((items: SearchPickerOption[]) => void) | null } = { resolve: null };
const search = vi.fn(
() =>
new Promise<SearchPickerOption[]>((resolve) => {
release.resolve = resolve;
})
);
const { unmount } = renderPicker({ search });
fireEvent.focus(screen.getByLabelText('Movie search'));
fireEvent.change(screen.getByLabelText('Movie search'), { target: { value: 'Alpha' } });
await waitFor(() => expect(search).toHaveBeenCalledWith('Alpha'));
// While mounted, no guard read can be `false` (the debounce may not have read it yet at all).
expect(mountedReads).not.toContain(false);
unmount();
release.resolve?.([{ id: 1, name: 'Alpha Movie' }]);
// Let the resolved promise's .then run against an unmounted tree.
await new Promise((resolve) => setTimeout(resolve, 0));
// The response handler ran and asked "am I still mounted?", getting `false` — so it skipped the
// setState. Drop the `mountedRef.current &&` from the .then and this read never happens.
expect(mountedReads).toContain(false);
expect(screen.queryByText('Alpha Movie')).not.toBeInTheDocument();
});
it('issues no request below minQueryLength, then exactly one for the settled query', async () => {
const search = vi.fn<(query: string) => Promise<SearchPickerOption[]>>().mockResolvedValue([]);
renderPicker({ minQueryLength: 2, search });
const input = screen.getByLabelText('Movie search');
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'A' } });
await new Promise((resolve) => setTimeout(resolve, 400));
expect(search).not.toHaveBeenCalled();
fireEvent.change(input, { target: { value: 'Alpha' } });
await waitFor(() => expect(search).toHaveBeenCalledTimes(1));
expect(search).toHaveBeenCalledWith('Alpha');
});
it('renders the current selection from the props, independent of any result set', async () => {
const search = vi.fn<(query: string) => Promise<SearchPickerOption[]>>().mockResolvedValue([]);
renderPicker({ search, selectedId: 9999, selectedName: 'Stored Show' });
// Nothing was searched, and 9999 is in no result set — it must still display by name.
expect(screen.getByText('Stored Show')).toBeInTheDocument();
expect(search).not.toHaveBeenCalled();
});
it('falls back to #id when the selection has no name', () => {
renderPicker({ selectedId: 42, selectedName: null });
expect(screen.getByText('#42')).toBeInTheDocument();
});
it('reports no matches once a query has settled empty', async () => {
const search = vi.fn<(query: string) => Promise<SearchPickerOption[]>>().mockResolvedValue([]);
renderPicker({ search });
const input = screen.getByLabelText('Movie search');
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Nope' } });
expect(await screen.findByText(/No matches/)).toBeInTheDocument();
});
// ---- #651 F6: keyboard operability ----
// A native <select> was fully keyboard-operable; replacing it with a custom widget must not
// regress that. Standard ARIA combobox: focus stays on the input, Arrow keys move a virtual
// cursor exposed via aria-activedescendant, Enter commits, Escape dismisses.
describe('keyboard navigation', () => {
const OPTIONS: SearchPickerOption[] = [
{ id: 1, name: 'Alpha' },
{ id: 2, name: 'Beta' },
{ id: 3, name: 'Gamma' }
];
async function openWithResults(onSelect = vi.fn()) {
const search = vi.fn<(query: string) => Promise<SearchPickerOption[]>>().mockResolvedValue(OPTIONS);
renderPicker({ onSelect, search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'a' } });
await screen.findByRole('listbox', { name: 'Movie results' });
return { input, onSelect };
}
const activeOptionName = (input: HTMLInputElement) => {
const id = input.getAttribute('aria-activedescendant');
return id === null ? null : document.getElementById(id)?.textContent ?? null;
};
it('exposes the combobox contract on the input', async () => {
const { input } = await openWithResults();
expect(input).toHaveAttribute('role', 'combobox');
expect(input).toHaveAttribute('aria-expanded', 'true');
expect(input).toHaveAttribute('aria-autocomplete', 'list');
expect(input.getAttribute('aria-controls')).toBe(
screen.getByRole('listbox', { name: 'Movie results' }).getAttribute('id')
);
// Nothing highlighted until the user navigates.
expect(input).not.toHaveAttribute('aria-activedescendant');
});
it('ArrowDown/ArrowUp move the active option and wrap at both ends', async () => {
const { input } = await openWithResults();
fireEvent.keyDown(input, { key: 'ArrowDown' });
expect(activeOptionName(input)).toBe('Alpha');
fireEvent.keyDown(input, { key: 'ArrowDown' });
expect(activeOptionName(input)).toBe('Beta');
fireEvent.keyDown(input, { key: 'ArrowUp' });
expect(activeOptionName(input)).toBe('Alpha');
// Wrap backwards past the first option...
fireEvent.keyDown(input, { key: 'ArrowUp' });
expect(activeOptionName(input)).toBe('Gamma');
// ...and forwards past the last.
fireEvent.keyDown(input, { key: 'ArrowDown' });
expect(activeOptionName(input)).toBe('Alpha');
});
it('ArrowUp from nothing highlighted starts at the LAST option', async () => {
const { input } = await openWithResults();
fireEvent.keyDown(input, { key: 'ArrowUp' });
expect(activeOptionName(input)).toBe('Gamma');
});
it('Home/End jump to the first and last option', async () => {
const { input } = await openWithResults();
fireEvent.keyDown(input, { key: 'End' });
expect(activeOptionName(input)).toBe('Gamma');
fireEvent.keyDown(input, { key: 'Home' });
expect(activeOptionName(input)).toBe('Alpha');
});
it('marks the active option with aria-selected', async () => {
const { input } = await openWithResults();
fireEvent.keyDown(input, { key: 'ArrowDown' });
fireEvent.keyDown(input, { key: 'ArrowDown' });
const options = screen.getAllByRole('option');
expect(options.map((option) => option.getAttribute('aria-selected'))).toEqual(['false', 'true', 'false']);
});
it('Enter commits the highlighted option and closes the list', async () => {
const { input, onSelect } = await openWithResults();
fireEvent.keyDown(input, { key: 'ArrowDown' });
fireEvent.keyDown(input, { key: 'ArrowDown' });
fireEvent.keyDown(input, { key: 'Enter' });
expect(onSelect).toHaveBeenCalledWith(2, 'Beta');
await waitFor(() => expect(screen.queryByRole('listbox', { name: 'Movie results' })).not.toBeInTheDocument());
});
it('Enter with nothing highlighted selects nothing (a form submit is not hijacked)', async () => {
const { input, onSelect } = await openWithResults();
fireEvent.keyDown(input, { key: 'Enter' });
expect(onSelect).not.toHaveBeenCalled();
expect(screen.getByRole('listbox', { name: 'Movie results' })).toBeInTheDocument();
});
it('Escape dismisses the list but keeps the typed text', async () => {
const { input, onSelect } = await openWithResults();
fireEvent.keyDown(input, { key: 'ArrowDown' });
fireEvent.keyDown(input, { key: 'Escape' });
await waitFor(() => expect(screen.queryByRole('listbox', { name: 'Movie results' })).not.toBeInTheDocument());
expect(input.value).toBe('a');
expect(onSelect).not.toHaveBeenCalled();
});
it('drops the highlight when a NEW result set arrives', async () => {
const search = vi
.fn<(query: string) => Promise<SearchPickerOption[]>>()
.mockResolvedValueOnce(OPTIONS)
.mockResolvedValue([{ id: 9, name: 'Zeta' }]);
renderPicker({ search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'a' } });
await screen.findByRole('listbox', { name: 'Movie results' });
fireEvent.keyDown(input, { key: 'ArrowDown' });
expect(activeOptionName(input)).toBe('Alpha');
// Index 0 of the OLD list is not index 0 of the new one — the cursor must reset, not carry
// over onto whatever now happens to sit at that position.
fireEvent.change(input, { target: { value: 'ab' } });
await waitFor(() => expect(screen.getByRole('option', { name: 'Zeta' })).toBeInTheDocument());
expect(input).not.toHaveAttribute('aria-activedescendant');
});
it('BLOCKER 2: Enter cannot commit a result from the PREVIOUS query', async () => {
const onSelect = vi.fn();
const release: { resolve: ((items: SearchPickerOption[]) => void) | null } = { resolve: null };
const search = vi
.fn<(query: string) => Promise<SearchPickerOption[]>>()
.mockResolvedValueOnce(OPTIONS)
.mockImplementation(() => new Promise((resolve) => { release.resolve = resolve; }));
renderPicker({ onSelect, search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Al' } });
await screen.findByRole('listbox', { name: 'Movie results' });
fireEvent.keyDown(input, { key: 'ArrowDown' });
expect(activeOptionName(input)).toBe('Alpha');
// Retype: the results still describe "Al" until the next response lands.
fireEvent.change(input, { target: { value: 'Be' } });
// The highlight is dropped IMMEDIATELY, not when the response happens to arrive.
expect(input).not.toHaveAttribute('aria-activedescendant');
// Neither re-highlighting nor Enter may commit a stale option while the box reads "Be".
fireEvent.keyDown(input, { key: 'ArrowDown' });
fireEvent.keyDown(input, { key: 'Enter' });
expect(onSelect).not.toHaveBeenCalled();
// Once the response for "Be" lands, the keyboard is live again.
await waitFor(() => expect(search).toHaveBeenCalledTimes(2));
release.resolve?.([{ id: 7, name: 'Beta Two' }]);
await waitFor(() => expect(screen.getByRole('option', { name: 'Beta Two' })).toBeInTheDocument());
fireEvent.keyDown(input, { key: 'ArrowDown' });
fireEvent.keyDown(input, { key: 'Enter' });
expect(onSelect).toHaveBeenCalledWith(7, 'Beta Two');
});
it('BLOCKER 3: the picker recovers from Escape by typing, without a blur', async () => {
const search = vi.fn<(query: string) => Promise<SearchPickerOption[]>>().mockResolvedValue(OPTIONS);
renderPicker({ search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'a' } });
await screen.findByRole('listbox', { name: 'Movie results' });
fireEvent.keyDown(input, { key: 'Escape' });
await waitFor(() => expect(screen.queryByRole('listbox', { name: 'Movie results' })).not.toBeInTheDocument());
// Focus never left the input, so onFocus cannot re-arm it. Typing must.
fireEvent.change(input, { target: { value: 'al' } });
expect(await screen.findByRole('listbox', { name: 'Movie results' })).toBeInTheDocument();
});
it('BLOCKER 3 / round 3: reopening onto CURRENT results re-queries nothing and stays navigable', async () => {
const onSelect = vi.fn();
const search = vi.fn<(query: string) => Promise<SearchPickerOption[]>>().mockResolvedValue(OPTIONS);
renderPicker({ onSelect, search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'a' } });
await screen.findByRole('listbox', { name: 'Movie results' });
expect(search).toHaveBeenCalledTimes(1);
fireEvent.keyDown(input, { key: 'Escape' });
await waitFor(() => expect(screen.queryByRole('listbox', { name: 'Movie results' })).not.toBeInTheDocument());
fireEvent.keyDown(input, { key: 'ArrowDown' });
expect(await screen.findByRole('listbox', { name: 'Movie results' })).toBeInTheDocument();
// The cached results are already current, so reopening must NOT re-search: a duplicate
// response would land later and reset the highlight the user has since moved, leaving Enter
// silently doing nothing. Give the debounce more than its 250ms to prove no request follows.
fireEvent.keyDown(input, { key: 'ArrowDown' });
expect(activeOptionName(input)).toBe('Beta');
await new Promise((resolve) => setTimeout(resolve, 400));
expect(search).toHaveBeenCalledTimes(1);
// The highlight survived, and Enter still commits it.
expect(activeOptionName(input)).toBe('Beta');
fireEvent.keyDown(input, { key: 'Enter' });
expect(onSelect).toHaveBeenCalledWith(2, 'Beta');
});
it('round 3 HIGH: a stale result cannot be committed by POINTER either', async () => {
const onSelect = vi.fn();
const release: { resolve: ((items: SearchPickerOption[]) => void) | null } = { resolve: null };
const search = vi
.fn<(query: string) => Promise<SearchPickerOption[]>>()
.mockResolvedValueOnce(OPTIONS)
.mockImplementation(() => new Promise((resolve) => { release.resolve = resolve; }));
renderPicker({ onSelect, search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Al' } });
await screen.findByRole('listbox', { name: 'Movie results' });
// Retype; Alpha is still on screen because hiding the list would flicker every keystroke.
fireEvent.change(input, { target: { value: 'Be' } });
const staleOption = screen.getByRole('option', { name: 'Alpha' });
expect(staleOption).toBeInTheDocument();
// Visible, but marked inert rather than merely looking normal.
expect(staleOption).toHaveAttribute('aria-disabled', 'true');
// Clicking it must not commit a result from the superseded query.
fireEvent.click(staleOption);
expect(onSelect).not.toHaveBeenCalled();
// Once "Be" resolves, pointer commits work again.
await waitFor(() => expect(search).toHaveBeenCalledTimes(2));
release.resolve?.([{ id: 7, name: 'Beta Two' }]);
const fresh = await screen.findByRole('option', { name: 'Beta Two' });
expect(fresh).toHaveAttribute('aria-disabled', 'false');
fireEvent.click(fresh);
expect(onSelect).toHaveBeenCalledWith(7, 'Beta Two');
});
});
// ---- #651 review round 4: results carry their SOURCE, not just their query text ----
describe('result provenance', () => {
const COLLECTION_HITS: SearchPickerOption[] = [{ id: 5, name: 'News Collection' }];
const SMART_HITS: SearchPickerOption[] = [{ id: 8, name: 'News Smart' }];
const OPTIONS: SearchPickerOption[] = [
{ id: 1, name: 'Alpha' },
{ id: 2, name: 'Beta' }
];
it('HIGH-1: changing the SOURCE re-queries and retires the old namespace\'s results', async () => {
const onSelect = vi.fn();
const search = vi
.fn<(query: string) => Promise<SearchPickerOption[]>>()
.mockResolvedValueOnce(COLLECTION_HITS)
.mockResolvedValue(SMART_HITS);
const { rerender } = renderPicker({ onSelect, search, source: 'Collection' });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'news' } });
expect(await screen.findByRole('option', { name: 'News Collection' })).toBeInTheDocument();
expect(search).toHaveBeenCalledTimes(1);
// Same component instance (no `key`), same query text — only the id namespace changed. The
// round-3 re-query guard keyed on query alone, which SUPPRESSED this request and left the
// Collection hit clickable under the new source.
rerender(
<SearchPicker
label="Movie"
onClear={vi.fn()}
onSelect={onSelect}
search={search}
selectedId={null}
selectedName={null}
source="SmartCollection"
/>
);
// The other namespace's result is gone immediately — not merely dimmed. It is not stale, it
// is wrong: a Collection id offered under a SmartCollection label.
expect(screen.queryByRole('option', { name: 'News Collection' })).not.toBeInTheDocument();
await waitFor(() => expect(search).toHaveBeenCalledTimes(2));
expect(await screen.findByRole('option', { name: 'News Smart' })).toBeInTheDocument();
fireEvent.click(screen.getByRole('option', { name: 'News Smart' }));
expect(onSelect).toHaveBeenCalledWith(8, 'News Smart');
expect(onSelect).not.toHaveBeenCalledWith(5, 'News Collection');
});
it('MEDIUM-5: a FAILED search is retried on reopen, not cached as "no matches" forever', async () => {
const search = vi
.fn<(query: string) => Promise<SearchPickerOption[]>>()
.mockRejectedValueOnce(new Error('boom'))
.mockResolvedValue(OPTIONS);
renderPicker({ search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Alien' } });
// The transient failure is reported AS a failure, not as an authoritative empty result...
expect(await screen.findByRole('alert')).toHaveTextContent(/Search failed/i);
expect(search).toHaveBeenCalledTimes(1);
// ...and must NOT retry on its own initiative. The original version of this test pressed
// Escape immediately, cancelling the very timer that would have exposed the storm (#651 round
// 5): a fresh `{ok:false}` object re-ran the effect, the success guard declined it, and
// another request was scheduled 250ms later, forever.
await new Promise((resolve) => setTimeout(resolve, 900));
expect(search).toHaveBeenCalledTimes(1);
// ...but must NOT be cached as an authoritative one either: reopening has to retry.
fireEvent.keyDown(input, { key: 'Escape' });
await waitFor(() => expect(screen.queryByRole('alert')).not.toBeInTheDocument());
fireEvent.keyDown(input, { key: 'ArrowDown' });
await waitFor(() => expect(search).toHaveBeenCalledTimes(2));
expect(await screen.findByRole('option', { name: 'Alpha' })).toBeInTheDocument();
});
it('a SUCCESSFUL empty result is still cached — the retry is for failures only', async () => {
const search = vi.fn<(query: string) => Promise<SearchPickerOption[]>>().mockResolvedValue([]);
renderPicker({ search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Nope' } });
expect(await screen.findByText(/No matches/)).toBeInTheDocument();
fireEvent.keyDown(input, { key: 'Escape' });
fireEvent.keyDown(input, { key: 'ArrowDown' });
await new Promise((resolve) => setTimeout(resolve, 400));
// "There genuinely are no matches" is an answer; re-asking would be the round-3 defect.
expect(search).toHaveBeenCalledTimes(1);
});
it('MEDIUM-3: a malformed 2xx body does not crash the picker, and is retryable', async () => {
// `client.ts` turns malformed JSON into `undefined` rather than rejecting, so a naive
// `setResults(undefined)` throws on the next render reading `results.length`.
const search = vi
.fn<(query: string) => Promise<SearchPickerOption[]>>()
.mockResolvedValueOnce(undefined as unknown as SearchPickerOption[])
.mockResolvedValue(OPTIONS);
renderPicker({ search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Alpha' } });
// Renders rather than throwing, and reports the malformed body as a FAILED attempt.
expect(await screen.findByRole('alert')).toHaveTextContent(/Search failed/i);
await new Promise((resolve) => setTimeout(resolve, 900));
expect(search).toHaveBeenCalledTimes(1);
fireEvent.keyDown(input, { key: 'Escape' });
fireEvent.keyDown(input, { key: 'ArrowDown' });
await waitFor(() => expect(search).toHaveBeenCalledTimes(2));
expect(await screen.findByRole('option', { name: 'Alpha' })).toBeInTheDocument();
});
it.each([
{ label: 'a null element', payload: [null] },
{ label: 'an element with no id', payload: [{ name: 'Nameless' }] },
{ label: 'an element with a string id', payload: [{ id: '7', name: 'Stringly' }] },
{ label: 'an element with no name', payload: [{ id: 7 }] },
{ label: 'an element with a non-string name', payload: [{ id: 7, name: 42 }] },
{ label: 'a NaN id', payload: [{ id: Number.NaN, name: 'Not a number' }] },
{ label: 'one bad element among good ones', payload: [{ id: 1, name: 'Alpha' }, null] },
// The API binds `selectedId` as a 32-bit integer, so these render and commit happily and
// then fail server-side — another adjacent shape (#651 round 7).
{ label: 'a fractional id', payload: [{ id: 1.5, name: 'Fractional' }] },
{ label: 'an id above int32', payload: [{ id: 2_147_483_648, name: 'Too big' }] },
{ label: 'an id below int32', payload: [{ id: -2_147_483_649, name: 'Too small' }] },
{ label: 'an Infinity id', payload: [{ id: Number.POSITIVE_INFINITY, name: 'Infinite' }] }
])('MEDIUM-3 (round 6): rejects $label rather than rendering or committing it', async ({ payload }) => {
// `Array.isArray` checks the CONTAINER, not the CONTENTS: `[null]` passes it, reaches
// `setResults`, and throws on `option.id` during render. A wrong-typed id would commit an
// invalid value through `onSelect`.
const onSelect = vi.fn();
const search = vi
.fn<(query: string) => Promise<SearchPickerOption[]>>()
.mockResolvedValueOnce(payload as unknown as SearchPickerOption[])
.mockResolvedValue(OPTIONS);
renderPicker({ onSelect, search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Alpha' } });
// Renders (does not throw) and offers nothing — a malformed list is not a partial answer.
// And it must READ as a failure: "No matches" would tell the user the library genuinely has
// nothing, with no hint that anything went wrong or that reopening retries (#651 round 7).
expect(await screen.findByRole('alert')).toHaveTextContent(/Search failed/i);
expect(screen.queryByText(/No matches/)).not.toBeInTheDocument();
expect(screen.queryAllByRole('option')).toHaveLength(0);
expect(onSelect).not.toHaveBeenCalled();
// ...and it is a FAILED attempt, so it stays retryable rather than being cached.
fireEvent.keyDown(input, { key: 'Escape' });
fireEvent.keyDown(input, { key: 'ArrowDown' });
await waitFor(() => expect(search).toHaveBeenCalledTimes(2));
expect(await screen.findByRole('option', { name: 'Alpha' })).toBeInTheDocument();
});
it('MEDIUM-3: a never-settling search stops loading instead of spinning forever', async () => {
vi.useFakeTimers();
try {
const search = vi.fn<(query: string) => Promise<SearchPickerOption[]>>(() => new Promise(() => {}));
renderPicker({ search });
const input = screen.getByLabelText('Movie search') as HTMLInputElement;
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'Alpha' } });
await act(async () => {
await vi.advanceTimersByTimeAsync(300);
});
expect(search).toHaveBeenCalledTimes(1);
// Past the picker's own deadline the wait is abandoned and the failure surfaces AS a
// failure — not as "no matches", which would misreport a timeout as an empty library.
await act(async () => {
await vi.advanceTimersByTimeAsync(11_000);
});
expect(screen.getByRole('alert')).toHaveTextContent(/Search failed/i);
} finally {
vi.useRealTimers();
}
});
});
});
+273 -27
View File
@@ -1,37 +1,106 @@
import { useEffect, useRef, useState } from 'react';
import { useEffect, useId, useRef, useState, type KeyboardEvent as ReactKeyboardEvent } from 'react';
import { Check, Search, X } from 'lucide-react';
import { Spinner } from '../components';
import type { SchedulingPickerOption } from '../api';
import { useIsMountedRef } from '../hooks';
import { isSelectionId } from '../api';
// The shape every search-backed picker resolves to. Deliberately structural rather than the
// generated `SchedulingPickerOption` DTO: the scheduling pickers pass that DTO straight through,
// while the media-library pickers (#651) map `LibraryBrowseItem` onto the same two fields.
export interface SearchPickerOption {
id: number;
name: string;
}
// A debounced async autocomplete for the search-backed source pickers (shows/seasons/artists and
// collection/multi/smart searches). Renders the current selection (label) with a clear button, an
// input that queries `search(query)` after a 250ms debounce, and a results list.
// collection/multi/smart searches, plus the media-library pickers from #651). Renders the current
// selection (label) with a clear button, an input that queries `search(query)` after a 250ms
// debounce, and a results list.
export interface SearchPickerProps {
label: string;
selectedId: null | number;
selectedName: null | string;
search: (query: string) => Promise<SchedulingPickerOption[]>;
search: (query: string) => Promise<SearchPickerOption[]>;
// Identifies WHAT `search` queries — the id NAMESPACE its results live in. Results are cached and
// validated against `(source, query)`, never the query alone: an id is only meaningful inside its
// own namespace, so a source change invalidates a result set exactly as a query change does.
// Required, deliberately: a defaulted source would silently opt every caller out of the check.
source: string;
onSelect: (id: number, name: string) => void;
onClear: () => void;
ariaDescribedBy?: string;
disabled?: boolean;
// Hide the visible field label (the surrounding row already renders one) without dropping the
// aria labels that name the input and the results list.
labelHidden?: boolean;
// Shortest query worth issuing a request for. Below it the picker fetches nothing at all.
minQueryLength?: number;
placeholder?: string;
}
// Upper bound on how long a picker will wait for a caller-supplied search promise.
const SEARCH_TIMEOUT_MS = 10_000;
// `Array.isArray` checks the CONTAINER, not the CONTENTS: `[null]` passes it, reaches `setResults`,
// and throws on `option.id` during render; an element with a missing or wrong-typed `id`/`name`
// yields a broken option or commits an invalid value (#651 round 6). Nothing between the network
// and `onSelect` re-checks these, so validate every element before storing it.
// The id must be what the API can actually BIND — see `isSelectionId`. This is one of several
// ingresses for the same class; the predicate is shared so they cannot drift apart (#651 round 8).
function isSearchPickerOption(value: unknown): value is SearchPickerOption {
if (typeof value !== 'object' || value === null) {
return false;
}
const candidate = value as { id?: unknown; name?: unknown };
return isSelectionId(candidate.id) && typeof candidate.name === 'string';
}
export function SearchPicker({
label,
selectedId,
selectedName,
search,
source,
onSelect,
onClear,
ariaDescribedBy,
disabled = false,
labelHidden = false,
minQueryLength = 1,
placeholder = 'Search…'
}: SearchPickerProps) {
const [query, setQuery] = useState('');
const [open, setOpen] = useState(false);
const [results, setResults] = useState<SchedulingPickerOption[]>([]);
const [results, setResults] = useState<SearchPickerOption[]>([]);
// The PROVENANCE of `results`: which source and query produced them, and whether that attempt
// succeeded. All three matter.
// - source: results from another namespace are not stale, they are wrong (#651 round 4 HIGH-1);
// - query: distinguishes an empty result set from "hasn't searched yet", so the no-matches
// copy doesn't flash between a keystroke and its debounce;
// - ok: a FAILED search must not be cached as a successful empty result, or the re-query
// guard below turns a transient 500 into a permanent "No matches" that reopening,
// blurring and refocusing can never retry (#651 round 4 MEDIUM-5).
const [resultsFor, setResultsFor] = useState<null | { ok: boolean; query: string; source: string }>(null);
const [loading, setLoading] = useState(false);
// -1 = no option highlighted (the ARIA combobox "virtual cursor" is parked).
const [activeIndex, setActiveIndex] = useState(-1);
const listboxId = useId();
const seqRef = useRef(0);
// The last (source, query) an actual REQUEST was issued for — success or failure. A failed
// attempt writes a fresh `resultsFor` object, which re-runs the effect; without this the success
// guard would decline the cached failure and schedule another request 250ms later, forever
// (#651 round 5 MEDIUM-2: a persistent outage became a request storm alternating "No matches"
// and a spinner). Reopening or editing the query is what re-arms a retry — both explicit user
// actions — so this is cleared there rather than on a timer.
const attemptRef = useRef<null | string>(null);
// #578: the seq guard drops a stale response but not a post-unmount one. Both are needed.
const mountedRef = useIsMountedRef();
// Re-arm a retry: the same failed query becomes requestable again after an explicit user action.
const rearm = () => {
attemptRef.current = null;
};
// All state transitions happen inside the debounce callback (never synchronously in the effect
// body — see spa-conventions.md §3).
@@ -40,37 +109,178 @@ export function SearchPicker({
return;
}
const trimmed = query.trim();
// Already holding SUCCESSFUL results for exactly this source and query — e.g. reopening after
// Escape. Re-running the search would issue a duplicate request whose response resets the
// highlight the user has since moved, leaving Enter doing nothing (#651 round 3, MEDIUM).
if (resultsFor !== null && resultsFor.ok && resultsFor.source === source && resultsFor.query === trimmed) {
return;
}
// Already TRIED this exact (source, query) and it failed. Do not retry on our own initiative —
// `rearm()` on reopen/edit is what allows another attempt (#651 round 5 MEDIUM-2).
const attempt = `${source}\u0000${trimmed}`;
if (attemptRef.current === attempt) {
return;
}
const seq = ++seqRef.current;
const tooShort = trimmed.length < minQueryLength;
const handle = window.setTimeout(() => {
if (trimmed.length === 0) {
if (seqRef.current === seq) {
if (tooShort) {
if (mountedRef.current && seqRef.current === seq) {
setResults([]);
setResultsFor(null);
setActiveIndex(-1);
setLoading(false);
}
return;
}
setLoading(true);
search(trimmed)
attemptRef.current = attempt;
// Bound the wait. `search` is a caller-supplied promise with no abort signal, so a request
// that never settles would otherwise leave the picker spinning forever with no way back
// (#651 round 5 MEDIUM-3). The underlying request is not cancelled — the seq guard already
// discards a late resolution — but the UI stops waiting on it.
Promise.race([
search(trimmed),
new Promise<never>((_resolve, reject) =>
window.setTimeout(() => reject(new Error('search timed out')), SEARCH_TIMEOUT_MS)
)
])
.then((items) => {
if (seqRef.current === seq) {
if (mountedRef.current && seqRef.current === seq) {
// A malformed 2xx body resolves as `undefined` rather than rejecting (`client.ts`
// `readJsonResponse` swallows a SyntaxError), and `setResults(undefined)` then throws
// on the next render reading `results.length`. Treat anything that is not a well-formed
// list of options as a FAILED attempt — not an authoritative empty answer — so it is
// retryable rather than cached (#651 round 5 MEDIUM-3, round 6).
if (!Array.isArray(items) || !items.every(isSearchPickerOption)) {
setResults([]);
setResultsFor({ ok: false, query: trimmed, source });
setActiveIndex(-1);
setLoading(false);
return;
}
setResults(items);
setResultsFor({ ok: true, query: trimmed, source });
// A new result set invalidates the highlight — index 2 of the old list is not index 2
// of the new one.
setActiveIndex(-1);
setLoading(false);
}
})
.catch(() => {
if (seqRef.current === seq) {
if (mountedRef.current && seqRef.current === seq) {
setResults([]);
// ok: false — a failure, not an authoritative empty result. Marked so the guard above
// retries instead of caching the outage forever.
setResultsFor({ ok: false, query: trimmed, source });
setActiveIndex(-1);
setLoading(false);
}
});
}, trimmed.length === 0 ? 0 : 250);
}, tooShort ? 0 : 250);
return () => window.clearTimeout(handle);
}, [query, open, search]);
}, [query, open, search, source, minQueryLength, mountedRef, resultsFor]);
const trimmedQuery = query.trim();
// Results from a DIFFERENT source aren't stale, they're wrong — a Collection id offered under a
// SmartCollection label — so they are not shown at all. Results from the same source but an older
// query stay VISIBLE while the next response loads (hiding them flickers on every keystroke) but
// become inert: typing "Al", highlighting Alpha, retyping "Be" and pressing Enter must not select
// Alpha (#651 round 2 BLOCKER 2, round 4 HIGH-1).
const sameSource = resultsFor !== null && resultsFor.source === source;
const resultsCurrent = sameSource && resultsFor.query === trimmedQuery;
// A failed attempt (network error, timeout, or a malformed payload rejected wholesale) must NOT
// read as an authoritative "no matches": the user would have no signal that anything went wrong,
// nor that reopening retries (#651 round 7).
const searchFailed = open && !loading && resultsCurrent && !resultsFor.ok;
const noMatches = open && !loading && results.length === 0 && resultsCurrent && resultsFor.ok;
const listboxOpen = open && sameSource && results.length > 0;
const keyboardArmed = listboxOpen && resultsCurrent;
// Keyboard navigation (#651 F6). Replacing a native <select> with a custom widget removed
// keyboard operability, which is a regression, not a pre-existing gap: without this the only way
// to reach a result is to Tab through to its button. This is the standard ARIA combobox pattern —
// focus stays on the input and `aria-activedescendant` points at the visually-highlighted option.
const choose = (option: SearchPickerOption) => {
// ONE gate for every commit path — keyboard, pointer, and anything added later. Gating the two
// call sites individually is what let a stale option stay clickable after Enter was fixed
// (#651 review round 3, HIGH): the class is "committing a result from a superseded query", not
// "pressing Enter".
if (!resultsCurrent) {
return;
}
onSelect(option.id, option.name);
setQuery('');
setResults([]);
setResultsFor(null);
setActiveIndex(-1);
setOpen(false);
};
const onKeyDown = (event: ReactKeyboardEvent<HTMLInputElement>) => {
if (event.key === 'Escape') {
// Dismiss the list but keep what was typed — Escape closes the popup, it doesn't undo input.
setActiveIndex(-1);
setOpen(false);
return;
}
// Escape (or a selection) closes the popup while focus stays in the input, so ArrowDown has to
// be able to REOPEN it. Without this the picker is stuck until the user blurs and refocuses,
// because `onFocus` never fires again (#651 review round 2, BLOCKER 3).
if (!open && (event.key === 'ArrowDown' || event.key === 'ArrowUp')) {
event.preventDefault();
setOpen(true);
rearm();
// Per the ARIA combobox pattern the same keypress also places the cursor, so reopening
// doesn't silently swallow it — but only onto results that are current, the same rule every
// other commit/navigation path obeys.
if (resultsCurrent && results.length > 0) {
setActiveIndex(event.key === 'ArrowDown' ? 0 : results.length - 1);
}
return;
}
if (!keyboardArmed) {
return;
}
if (event.key === 'ArrowDown' || event.key === 'ArrowUp') {
event.preventDefault();
const delta = event.key === 'ArrowDown' ? 1 : -1;
setActiveIndex((current) => {
// From "nothing active", ArrowDown lands on the first option and ArrowUp on the last.
const next = current < 0 ? (delta === 1 ? 0 : results.length - 1) : current + delta;
return (next + results.length) % results.length;
});
return;
}
if (event.key === 'Home' || event.key === 'End') {
event.preventDefault();
setActiveIndex(event.key === 'Home' ? 0 : results.length - 1);
return;
}
if (event.key === 'Enter' && activeIndex >= 0 && activeIndex < results.length) {
// Only swallow Enter when it actually commits a highlighted option, so a form's default
// submit behaviour is untouched otherwise.
event.preventDefault();
choose(results[activeIndex]);
}
};
const optionId = (index: number) => `${listboxId}-option-${index}`;
return (
<div className="ctv-field ctv-field-full">
<span className="ctv-field-label">{label}</span>
{!labelHidden && <span className="ctv-field-label">{label}</span>}
{selectedId != null && !open ? (
<div className="ctv-picker-selected">
<span>{selectedName ?? `#${selectedId}`}</span>
@@ -95,27 +305,63 @@ export function SearchPicker({
value={query}
disabled={disabled}
placeholder={placeholder}
role="combobox"
aria-label={`${label} search`}
onChange={(event) => setQuery(event.target.value)}
onFocus={() => setOpen(true)}
aria-describedby={ariaDescribedBy}
aria-expanded={listboxOpen}
aria-controls={listboxId}
aria-autocomplete="list"
aria-activedescendant={
keyboardArmed && activeIndex >= 0 && activeIndex < results.length ? optionId(activeIndex) : undefined
}
onChange={(event) => {
setQuery(event.target.value);
// The highlight described the OLD result set; drop it now rather than when the next
// response happens to arrive (BLOCKER 2). Typing also reopens a popup Escape closed,
// so the picker recovers without a blur (BLOCKER 3).
setActiveIndex(-1);
setOpen(true);
rearm();
}}
onFocus={() => {
setOpen(true);
rearm();
}}
onKeyDown={onKeyDown}
/>
{loading && <span className="ctv-field-trailing"><Spinner size={13} tone="muted" /></span>}
</span>
{open && results.length > 0 && (
<ul className="ctv-picker-results" role="listbox" aria-label={`${label} results`}>
{results.map((option) => (
{noMatches && <div className="ctv-picker-empty">No matches try a different title.</div>}
{searchFailed && (
<div className="ctv-picker-empty ctv-picker-error" role="alert">
Search failed press Escape and reopen, or edit your search, to try again.
</div>
)}
{listboxOpen && (
<ul
className={`ctv-picker-results${resultsCurrent ? '' : ' ctv-picker-results-stale'}`}
role="listbox"
id={listboxId}
aria-label={`${label} results`}
>
{results.map((option, index) => (
<li key={option.id}>
<button
type="button"
role="option"
aria-selected={option.id === selectedId}
className="ctv-picker-result"
onClick={() => {
onSelect(option.id, option.name);
setQuery('');
setResults([]);
setOpen(false);
}}
id={optionId(index)}
// The option is never a tab stop: the ARIA combobox pattern keeps focus on the
// input and moves a virtual cursor via aria-activedescendant. It stays a
// <button> so pointer users get the same click target as before.
tabIndex={-1}
// Superseded results are inert until their replacement lands. `aria-disabled`
// (not `disabled`) keeps them announced and hoverable while `choose` refuses
// them, so the list doesn't flicker away on every keystroke.
aria-disabled={!resultsCurrent}
aria-selected={keyboardArmed && index === activeIndex}
className={`ctv-picker-result${keyboardArmed && index === activeIndex ? ' ctv-picker-result-active' : ''}`}
onMouseEnter={() => setActiveIndex(index)}
onClick={() => choose(option)}
>
{option.name}
</button>
+1 -11
View File
@@ -54,6 +54,7 @@ import {
messageFromAutoTuneError,
messageFromChannelTemplateError,
messageFromSearchError,
titleContainsQuery,
uploadArtwork,
previewAutoTune,
type AutoTuneAxis,
@@ -1465,17 +1466,6 @@ function DetailPanel({
const ADD_SOURCE_RESULTS = 8;
const ADD_SOURCE_MIN_QUERY = 2;
// The picker takes plain typed text, not Lucene: the index's default field does NOT match bare title
// words (`Alpha` finds nothing for "Show Alpha" — docs/e2e-local.md), so a raw forward like the search
// box does would look broken here. Compile a substring title match instead, escaping every Lucene
// special (and whitespace) so the boundary stars are the only live wildcards — the same shape the rule
// builder's `contains` operator emits (builder/rules/compile.ts).
const LUCENE_WILD_SPECIAL = /([\s+\-!(){}[\]^"~*?:\\/])/g;
function titleContainsQuery(text: string): string {
return `title:*${text.replace(LUCENE_WILD_SPECIAL, '\\$1')}*`;
}
type AddSearchState =
| { status: 'idle' }
| { status: 'error'; query: string; message: string }
+140 -10
View File
@@ -1,4 +1,4 @@
import { cleanup, render, screen, waitFor, within } from '@testing-library/react';
import { cleanup, fireEvent, render, screen, waitFor, within } from '@testing-library/react';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { FillerPresetsScreen } from './FillerPresetsScreen';
@@ -87,6 +87,11 @@ function mockApi(options: MockOptions = {}) {
return Promise.resolve(jsonResponse(browsePage(browseCount, browseTotal)));
}
// The by-id detail read the search-driven picker uses to name an already-stored media item.
if (pathname === '/api/v1/shows/9999' && method === 'GET') {
return Promise.resolve(jsonResponse({ id: 9999, libraryId: 1, mediaSourceKind: 'Local', title: 'Stored Show' }));
}
return Promise.resolve(new Response(null, { status: 204 }));
});
}
@@ -141,19 +146,144 @@ describe('FillerPresetsScreen', () => {
expect(screen.queryByText(/Showing the first/)).not.toBeInTheDocument();
});
it('F2: injects the out-of-list current selection so an id outside the loaded page still renders as selected', async () => {
it('#651: an edit-loaded media-item selection is NAMED by one by-id read, with no library load', async () => {
window.history.pushState({}, '', '/app/filler-presets/1');
mockApi();
const fetchMock = mockApi();
render(<FillerPresetsScreen />);
await waitFor(() => expect(screen.getByDisplayValue('Bumper')).toBeInTheDocument());
// Select order: Kind, Mode, Pad to nearest minute, Collection type, then the activeConfig
// ("Movie") picker last — it shows the out-of-list #9999 option, selected, never falling back
// to "(none)" even though 9999 isn't in the 3-item loaded page.
const comboboxes = await screen.findAllByRole('combobox');
const pickerSelect = comboboxes[comboboxes.length - 1];
expect((pickerSelect as HTMLSelectElement).value).toBe('9999');
expect(within(pickerSelect).getByText('#9999')).toBeInTheDocument();
// The stored media item (id 9999) is displayed by NAME, resolved through a single by-id detail
// read — never by scanning a browse window it may well not be in.
expect(await screen.findByText('Stored Show')).toBeInTheDocument();
expect(screen.getByLabelText('Clear Television Show')).toBeInTheDocument();
const calls = (pathname: string) =>
fetchMock.mock.calls.filter(([u]) => new URL(u.toString(), 'http://localhost').pathname === pathname);
expect(calls('/api/v1/shows/9999')).toHaveLength(1);
// A searchable (media-library) type list-loads NOTHING.
expect(calls('/api/v1/library/browse')).toHaveLength(0);
});
it('#651: the media-library picker compiles typed text and stays bounded against a huge library', async () => {
window.history.pushState({}, '', '/app/filler-presets/add');
const TOTAL = 20000;
const fetchMock = mockApi({
onRequest: (url) => {
if (!url.startsWith('/api/v1/library/browse')) {
return null;
}
const parsed = new URL(url, 'http://localhost');
const pageSize = Number(parsed.searchParams.get('pageSize') ?? '100');
return jsonResponse({
page: Array.from({ length: pageSize }, (_, i) => ({
id: i + 1,
mediaItemId: i + 1,
mediaType: 'TelevisionShow' as const,
title: `Show ${i + 1}`
})),
totalCount: TOTAL
});
}
});
render(<FillerPresetsScreen />);
// Switch the collection type to the media-library-backed Television Show.
const typeSelect = (await screen.findAllByRole('combobox')).find((el) =>
within(el).queryByText('Television Show')
) as HTMLSelectElement;
fireEvent.change(typeSelect, { target: { value: 'TelevisionShow' } });
// Scoped to the searchable type: the default 'Collection' draft does one bounded (non-Lucene,
// LIKE-filtered) load of its own, which #651 deliberately leaves alone.
const browseCalls = () =>
fetchMock.mock.calls.filter(([u]) => {
const parsed = new URL(u.toString(), 'http://localhost');
return parsed.pathname === '/api/v1/library/browse' && parsed.searchParams.get('mediaType') === 'TelevisionShow';
});
const searchInput = await screen.findByLabelText('Television Show search');
expect(browseCalls()).toHaveLength(0);
fireEvent.focus(searchInput);
fireEvent.change(searchInput, { target: { value: 'Show Alpha' } });
await waitFor(() => expect(browseCalls().length).toBe(1));
// Give any (nonexistent) paging loop room to fire.
await new Promise((resolve) => setTimeout(resolve, 400));
expect(browseCalls()).toHaveLength(1);
const url = new URL(browseCalls()[0][0].toString(), 'http://localhost');
expect(url.searchParams.get('query')).toBe('title:*Show\\ Alpha*');
expect(url.searchParams.get('mediaType')).toBe('TelevisionShow');
expect(Number(url.searchParams.get('pageSize'))).toBe(25);
const results = screen.getByRole('listbox', { name: 'Television Show results' });
expect(within(results).getAllByRole('option')).toHaveLength(25);
});
it('#651 F3: a SLOW edit-load name resolution never relabels a newer selection', async () => {
window.history.pushState({}, '', '/app/filler-presets/1');
// Hold the by-id read for the STORED show (9999) open, so it resolves only after the user has
// already picked a different show. One bespoke mock — layering a second spy over mockApi's
// would re-enter itself.
// Holder object rather than a bare `let` — TS narrows a closure-assigned variable to `null`.
const releaseStoredName: { resolve: ((response: Response) => void) | null } = { resolve: null };
vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const pathname = new URL(input.toString(), 'http://localhost').pathname;
const method = (init?.method ?? 'GET').toUpperCase();
if (pathname === '/api/v1/shows/9999') {
return new Promise<Response>((resolve) => {
releaseStoredName.resolve = resolve;
});
}
if (pathname === '/api/v1/filler-presets' && method === 'GET') {
return Promise.resolve(jsonResponse(presetList));
}
if (pathname === '/api/v1/filler-presets/1' && method === 'GET') {
return Promise.resolve(jsonResponse(editPreset));
}
if (pathname === '/api/v1/library/browse' && method === 'GET') {
return Promise.resolve(
jsonResponse({
page: [{ id: 123, mediaItemId: 123, mediaType: 'TelevisionShow', title: 'New Show' }],
totalCount: 1
})
);
}
return Promise.resolve(new Response(null, { status: 204 }));
});
render(<FillerPresetsScreen />);
await waitFor(() => expect(screen.getByDisplayValue('Bumper')).toBeInTheDocument());
// No name yet — the read is still in flight, so the picker shows the id it definitely has.
expect(await screen.findByText('#9999')).toBeInTheDocument();
// The user picks a different show while the stored name is still resolving.
fireEvent.click(screen.getByRole('button', { name: 'Change' }));
const searchInput = await screen.findByLabelText('Television Show search');
fireEvent.focus(searchInput);
fireEvent.change(searchInput, { target: { value: 'New Show' } });
fireEvent.click(await screen.findByRole('option', { name: 'New Show' }));
const chip = () => screen.getByLabelText('Clear Television Show').closest('.ctv-picker-selected');
await waitFor(() => expect(chip()?.textContent).toContain('New Show'));
// NOW the stale read lands. It must not relabel the newer selection: the draft holds 123, so a
// title resolved for 9999 is simply not about the current selection.
releaseStoredName.resolve?.(
new Response(JSON.stringify({ id: 9999, libraryId: 1, mediaSourceKind: 'Local', title: 'Stored Show' }), {
headers: { 'Content-Type': 'application/json' },
status: 200
})
);
await new Promise((resolve) => setTimeout(resolve, 0));
expect(chip()?.textContent).toContain('New Show');
expect(chip()?.textContent).not.toContain('Stored Show');
});
});
+155 -35
View File
@@ -6,25 +6,32 @@ import { Badge, Button, Card, Checkbox, ConfirmDialog, IconButton, Input, Select
import {
createFillerPreset,
deleteFillerPreset,
getArtist,
getFillerPreset,
getFillerPresets,
getLibraryBrowseItems,
getSeason,
getShow,
messageFromFillerPresetError,
searchLibraryPickerOptions,
selectionIdOrNull,
updateFillerPreset,
isSelectionId,
LIBRARY_PICKER_MIN_QUERY,
type CreateFillerPresetRequest,
type FillerPreset,
type LibraryBrowseItem,
type LibraryBrowseMediaType
} from '../api';
import { SearchPicker } from '../schedules/pickers';
const BASE_PATH = '/app/filler-presets';
// Class B picker (#644 follow-up): the filler-preset collection-type picker browses the largest
// media-library tables (Episode, Song, Image, Movie, MusicVideo, ...), which can run into the tens
// of thousands of rows. Paging to completeness would mean ~200 serial requests — each more
// expensive than the last (LuceneSearchIndex.Search computes hitsLimit = skip + limit) — to
// populate a native <select> with thousands of <option> nodes. Load ONE bounded page instead and
// surface the truncation (see docs/decisions/records/spa/list-completeness-vs-bounded-pickers.md).
// The bounded page size for the picker types that are NOT media-library tables (Collection /
// MultiCollection / SmartCollection / Playlist — admin-created, bounded by construction, and
// filtered server-side by a plain SQL LIKE rather than Lucene). The media-library types
// (TelevisionShow / TelevisionSeason / Artist) do not list-load at all: they resolve by search
// (#651, decision key `spa.library-pickers-resolve-by-search`).
const LIBRARY_BROWSE_PAGE_CAP = 100;
type Draft = CreateFillerPresetRequest;
@@ -50,23 +57,49 @@ const MODE_OPTIONS = [
const PAD_OPTIONS = [5, 10, 15, 30, 60].map((minutes) => ({ label: String(minutes), value: String(minutes) }));
// CollectionType -> the library-browse media type + which request id field it fills.
// `searchable` marks the media-library types: those resolve by search (#651) and are never
// list-loaded. The rest are admin-created lists filtered server-side by a plain SQL LIKE — which
// a compiled Lucene query would NOT match — so they keep the bounded single-page load.
const COLLECTION_TYPES: Array<{
browse: LibraryBrowseMediaType;
field: 'collectionId' | 'mediaItemId' | 'multiCollectionId' | 'playlistId' | 'smartCollectionId';
itemId: (item: LibraryBrowseItem) => null | number | undefined;
label: string;
playlistOnly?: boolean;
searchable?: boolean;
value: CollectionType;
}> = [
{ browse: 'Collection', field: 'collectionId', itemId: (item) => item.collectionId, label: 'Collection', value: 'Collection' },
{ browse: 'TelevisionShow', field: 'mediaItemId', itemId: (item) => item.mediaItemId ?? item.id, label: 'Television Show', value: 'TelevisionShow' },
{ browse: 'TelevisionSeason', field: 'mediaItemId', itemId: (item) => item.mediaItemId ?? item.id, label: 'Television Season', value: 'TelevisionSeason' },
{ browse: 'Artist', field: 'mediaItemId', itemId: (item) => item.mediaItemId ?? item.id, label: 'Artist', value: 'Artist' },
{ browse: 'TelevisionShow', field: 'mediaItemId', itemId: (item) => item.mediaItemId ?? item.id, label: 'Television Show', searchable: true, value: 'TelevisionShow' },
{ browse: 'TelevisionSeason', field: 'mediaItemId', itemId: (item) => item.mediaItemId ?? item.id, label: 'Television Season', searchable: true, value: 'TelevisionSeason' },
{ browse: 'Artist', field: 'mediaItemId', itemId: (item) => item.mediaItemId ?? item.id, label: 'Artist', searchable: true, value: 'Artist' },
{ browse: 'MultiCollection', field: 'multiCollectionId', itemId: (item) => item.multiCollectionId, label: 'Multi Collection', value: 'MultiCollection' },
{ browse: 'SmartCollection', field: 'smartCollectionId', itemId: (item) => item.smartCollectionId, label: 'Smart Collection', value: 'SmartCollection' },
{ browse: 'Playlist', field: 'playlistId', itemId: (item) => item.playlistId, label: 'Playlist', playlistOnly: true, value: 'Playlist' }
];
// Unlike a rerun collection or a playlist item, a filler preset stores ONLY the media item's id —
// there is no name on `FillerPresetFullResponseModel`. So the edit path resolves the stored id's
// title through the by-id detail endpoint: exactly ONE bounded request, and the picker can show the
// real selection instead of a bare `#id` whether or not it would have been in any browse window.
function resolveMediaItemName(type: CollectionType, id: number): Promise<null | string> {
switch (type) {
case 'TelevisionShow':
return getShow(id).then((show) => show.title || null);
case 'TelevisionSeason':
// Mirror the browse label for a season ("Show - Season 3"): the detail DTO splits it into the
// show title and the season name.
return getSeason(id).then((season) => {
const parts = [season.title, season.name].filter((part) => part != null && part.trim().length > 0);
return parts.length > 0 ? parts.join(' - ') : null;
});
case 'Artist':
return getArtist(id).then((artist) => artist.name || null);
default:
return Promise.resolve(null);
}
}
const COLLECTION_ID_FIELDS = ['collectionId', 'mediaItemId', 'multiCollectionId', 'smartCollectionId', 'playlistId'] as const;
/* ---------- TimeSpan (constant "c" format) helpers ---------- */
@@ -184,19 +217,22 @@ function defaultDraft(): Draft {
function draftFromPreset(preset: FillerPreset & Draft): Draft {
return {
allowWatermarks: preset.allowWatermarks,
collectionId: preset.collectionId,
// Every entity-reference id is normalized on the way in: a value the API cannot bind is treated
// as ABSENT so it surfaces as "no selection" (Save disabled) rather than as an apparently-valid
// selection that fails on submit (#651 round 8).
collectionId: selectionIdOrNull(preset.collectionId),
collectionType: preset.collectionType,
count: preset.count,
duration: preset.duration,
expression: preset.expression,
fillerKind: preset.fillerKind,
fillerMode: preset.fillerMode,
mediaItemId: preset.mediaItemId,
multiCollectionId: preset.multiCollectionId,
mediaItemId: selectionIdOrNull(preset.mediaItemId),
multiCollectionId: selectionIdOrNull(preset.multiCollectionId),
name: preset.name,
padToNearestMinute: preset.padToNearestMinute,
playlistId: preset.playlistId,
smartCollectionId: preset.smartCollectionId,
playlistId: selectionIdOrNull(preset.playlistId),
smartCollectionId: selectionIdOrNull(preset.smartCollectionId),
useChaptersAsMediaItems: preset.useChaptersAsMediaItems
};
}
@@ -454,6 +490,16 @@ function FillerEditor({ mode }: { mode: { id: number; kind: 'edit' } | { kind: '
const [pickerError, setPickerError] = useState<null | string>(null);
const [pickerTruncated, setPickerTruncated] = useState(false);
const [pickerTotalCount, setPickerTotalCount] = useState<null | number>(null);
// The searchable (media-library) picker's current selection LABEL. The draft holds only the id,
// so this is what keeps an already-selected item displayable while it sits outside — or ahead
// of — any search result set.
//
// It is stored WITH the id it names (#651 F3) rather than as a bare string. A slow edit-load name
// resolution for the stored id can otherwise land after the user has already picked something
// else, labelling the new id with the old item's title while `mediaItemId` — and therefore what
// gets saved — says otherwise. The id travels with the name so the resolution callback can tell
// whether it is still describing the current selection.
const [selectedLabel, setSelectedLabel] = useState<null | { id: number; name: string }>(null);
const isEdit = mode.kind === 'edit';
@@ -469,8 +515,34 @@ function FillerEditor({ mode }: { mode: { id: number; kind: 'edit' } | { kind: '
getFillerPreset(loadEditId)
.then((preset) => {
if (active) {
setDraft(draftFromPreset(preset as FillerPreset & Draft));
if (!active) {
return;
}
const loaded = draftFromPreset(preset as FillerPreset & Draft);
setDraft(loaded);
// Resolve the stored media item's name for the search-driven picker (one bounded request).
// A failure here is cosmetic — the picker falls back to `#id` and the id is still saved —
// so it must never surface as a load error.
const config = COLLECTION_TYPES.find((entry) => entry.value === loaded.collectionType);
const requestedId = loaded.mediaItemId;
if (config?.searchable && requestedId != null) {
resolveMediaItemName(loaded.collectionType, requestedId)
.then((name) => {
if (active && name) {
// Tagged with the id it was resolved FOR — see `selectedLabel` above — and refused
// outright if the label we already hold names a DIFFERENT id, i.e. the user picked
// something else while this was in flight. The render-time id check alone would
// stop the mislabelling but still discard the newer, correct label.
setSelectedLabel((current) =>
current !== null && current.id !== requestedId ? current : { id: requestedId, name }
);
}
})
.catch(() => {
/* cosmetic only — keep the `#id` fallback */
});
}
})
.catch((error: unknown) => {
@@ -487,17 +559,17 @@ function FillerEditor({ mode }: { mode: { id: number; kind: 'edit' } | { kind: '
const collectionType = draft?.collectionType;
// Load the browse items for the active collection type so the picker can show names. All
// resets happen in the async callbacks (never synchronously in the effect body).
// resets happen in the async callbacks (never synchronously in the effect body). Searchable
// (media-library) types load NOTHING here — they resolve by search (#651).
useEffect(() => {
const config = COLLECTION_TYPES.find((entry) => entry.value === collectionType);
if (!config) {
if (!config || config.searchable) {
return;
}
let active = true;
// Class B (#644 follow-up) — load ONE bounded page rather than paging to completeness; see the
// module-level comment on LIBRARY_BROWSE_PAGE_CAP.
// Bounded single page — see the module-level comment on LIBRARY_BROWSE_PAGE_CAP.
getLibraryBrowseItems({ mediaType: config.browse, pageNum: 0, pageSize: LIBRARY_BROWSE_PAGE_CAP })
.then((result) => {
if (active) {
@@ -523,6 +595,16 @@ function FillerEditor({ mode }: { mode: { id: number; kind: 'edit' } | { kind: '
};
}, [collectionType]);
const searchConfig = COLLECTION_TYPES.find((entry) => entry.value === collectionType && entry.searchable);
const searchBrowseType = searchConfig?.browse;
// Stable across renders: `SearchPicker`'s debounce effect lists `search` in its deps, so a fresh
// closure every render would restart the debounce on each keystroke's re-render.
const searchLibrary = useCallback(
(query: string) => (searchBrowseType ? searchLibraryPickerOptions(searchBrowseType, query) : Promise.resolve([])),
[searchBrowseType]
);
if (loadError) {
return (
<div className="ctv-settings-error" role="alert">
@@ -549,6 +631,8 @@ function FillerEditor({ mode }: { mode: { id: number; kind: 'edit' } | { kind: '
const setKind = (fillerKind: FillerKind) => setDraft((current) => (current ? applyKind(current, fillerKind) : current));
const setCollectionType = (type: CollectionType) => {
// The id refs are cleared below, so the search picker's label must go with them.
setSelectedLabel(null);
setDraft((current) => {
if (!current) {
return current;
@@ -581,10 +665,9 @@ function FillerEditor({ mode }: { mode: { id: number; kind: 'edit' } | { kind: '
const pickerSelectedInList =
pickerSelectedId != null &&
pickerItems.some((item) => activeConfig?.itemId(item) === pickerSelectedId);
// Out-of-list injection (round-3 review F2): the Class B picker loads only ONE bounded page
// (LIBRARY_BROWSE_PAGE_CAP), so a preset whose stored id sits outside that page would otherwise
// render as "(none)" while the draft still holds the id — mirrors RerunCollectionsScreen's and
// PlaylistsScreen's `selectedInList` prepend.
// Out-of-list injection (round-3 review F2): the non-searchable picker loads only ONE bounded
// page (LIBRARY_BROWSE_PAGE_CAP), so a preset whose stored id sits outside that page would
// otherwise render as "(none)" while the draft still holds the id.
const pickerOptions = [
{ label: '(none)', value: '' },
...(pickerSelectedId != null && !pickerSelectedInList
@@ -592,15 +675,18 @@ function FillerEditor({ mode }: { mode: { id: number; kind: 'edit' } | { kind: '
: []),
...pickerItems.flatMap((item) => {
const id = activeConfig?.itemId(item);
return id == null ? [] : [{ label: item.title ?? `#${id}`, value: String(id) }];
// Drop, don't offer: an option whose id the API cannot bind is unselectable by construction.
return isSelectionId(id) ? [{ label: item.title ?? `#${id}`, value: String(id) }] : [];
})
];
const pickerHelp = pickerError
? pickerError
: pickerTruncated
? `Showing the first ${pickerItems.length} of ${pickerTotalCount} — use search to narrow.`
: undefined;
: searchBrowseType
? `Type at least ${LIBRARY_PICKER_MIN_QUERY} characters to search the library.`
: pickerTruncated
? `Showing the first ${pickerItems.length} of ${pickerTotalCount} — use search to narrow.`
: undefined;
const collectionTypeOptions = COLLECTION_TYPES.filter(
(entry) => !entry.playlistOnly || (draft.fillerKind !== 'Fallback' && draft.fillerKind !== 'Tail')
@@ -739,13 +825,47 @@ function FillerEditor({ mode }: { mode: { id: number; kind: 'edit' } | { kind: '
</Row>
{activeConfig && (
<Row control={360} help={pickerHelp} label={activeConfig.label}>
<Select
onChange={(event) =>
set({ [activeConfig.field]: event.target.value === '' ? null : Number(event.target.value) })
}
options={pickerOptions}
value={pickerValue}
/>
{searchBrowseType ? (
// #651: the media library is resolved by search, never list-loaded. The current
// selection renders from `selectedLabel` (resolved once by id on edit-load), not from
// the result set, so an already-selected item survives every search — and even a
// failed name resolution degrades to `#id` rather than losing the id.
<SearchPicker
key={activeConfig.value}
label={activeConfig.label}
labelHidden
minQueryLength={LIBRARY_PICKER_MIN_QUERY}
onClear={() => {
setSelectedLabel(null);
set({ [activeConfig.field]: null });
}}
onSelect={(id, name) => {
setSelectedLabel({ id, name });
set({ [activeConfig.field]: id });
}}
placeholder={`Search ${activeConfig.label.toLowerCase()}`}
search={searchLibrary}
source={activeConfig.value}
selectedId={pickerSelectedId}
// No id comparison here: there is no reachable path TODAY on which `selectedLabel`
// names an id other than the draft's while a selection is shown. (Not "every writer
// sets both" — the initial load writes the id alone, and a stale resolver can
// repopulate the label after a clear because `current === null`; in both cases the
// draft id is null, so the picker renders its search input and the label is unused.)
// Adding a render-time check back would be defensive, not a fix, and an unreachable
// guard is an untested one — so if a path ever appears, add the guard WITH the test
// that reaches it. #651 review round 3.
selectedName={selectedLabel?.name ?? null}
/>
) : (
<Select
onChange={(event) =>
set({ [activeConfig.field]: event.target.value === '' ? null : Number(event.target.value) })
}
options={pickerOptions}
value={pickerValue}
/>
)}
</Row>
)}
<Row
+189 -2
View File
@@ -253,10 +253,197 @@ describe('PlaylistsScreen', () => {
// Select the Movie item row to open the detail form.
fireEvent.click(await screen.findByText('Cool Movie'));
// Detail-form selects: [0] Collection Type, [1] Selection, [2] Playback Order.
// Movie is a media-library type, so its Selection picker is a search typeahead (#651), not a
// combobox: [0] Collection Type, [1] Playback Order.
const selects = await screen.findAllByRole('combobox');
const orderSelect = selects[2];
const orderSelect = selects[1];
expect(orderSelect).toBeDisabled();
expect(within(orderSelect).getAllByRole('option').map((o) => o.textContent)).toEqual(['None']);
});
it('media-library picker: keeps the stored selection and resolves by compiled search (#651)', async () => {
const fetchMock = mockApi({
// A 20,000-row library: an unbounded loader would show up both as extra requests and as a
// huge option list.
onRequest: (url) => {
if (!url.startsWith('/api/v1/library/browse')) {
return null;
}
const pageSize = Number(new URL(url, 'http://localhost').searchParams.get('pageSize') ?? '100');
return jsonResponse({
page: [
{ id: 42, mediaItemId: 42, mediaType: 'Movie', title: 'Another Movie' },
...Array.from({ length: pageSize - 1 }, (_, i) => ({
id: i + 100,
mediaItemId: i + 100,
mediaType: 'Movie' as const,
title: `Filler Movie ${i + 1}`
}))
],
totalCount: 20000
});
}
});
render(<PlaylistsScreen />);
fireEvent.click(await screen.findByText('Bumps'));
fireEvent.click(await screen.findByText('Cool Movie'));
const browseCalls = () =>
fetchMock.mock.calls.filter(
([u]) => new URL(u.toString(), 'http://localhost').pathname === '/api/v1/library/browse'
);
// The already-selected media item (id 7, outside any result set) renders from the draft item's
// own `selectedName` — with NO library load at all.
expect(await screen.findByLabelText('Clear Movie')).toBeInTheDocument();
expect(browseCalls()).toHaveLength(0);
// "Change" reveals the typeahead; typed text is compiled, never forwarded raw.
fireEvent.click(screen.getByRole('button', { name: 'Change' }));
const searchInput = await screen.findByLabelText('Movie search');
fireEvent.focus(searchInput);
fireEvent.change(searchInput, { target: { value: 'Show Alpha' } });
await waitFor(() => expect(browseCalls().length).toBe(1));
const url = new URL(browseCalls()[0][0].toString(), 'http://localhost');
expect(url.searchParams.get('query')).toBe('title:*Show\\ Alpha*');
expect(url.searchParams.get('mediaType')).toBe('Movie');
// Picking a result replaces the selection (and its label) without any further load.
// One bounded page of options — not 20,000 nodes, and not a paging loop.
const results = screen.getByRole('listbox', { name: 'Movie results' });
expect(within(results).getAllByRole('option')).toHaveLength(25);
fireEvent.click(await screen.findByRole('option', { name: 'Another Movie' }));
const chip = (await screen.findByLabelText('Clear Movie')).closest('.ctv-picker-selected');
expect(chip?.textContent).toContain('Another Movie');
expect(browseCalls()).toHaveLength(1);
});
it('HIGH-2: options from the PREVIOUS item type are not selectable while the new type loads', async () => {
// Same defect as RerunCollectionsScreen, second screen: switching a playlist item's type on a
// slow connection must not leave the old namespace's options on offer under the new label
// (#651 round 4 HIGH-2).
const smartRelease: { resolve: (() => void) | null } = { resolve: null };
const fetchMock = mockApi();
const inner = fetchMock.getMockImplementation() as (i: RequestInfo | URL, r?: RequestInit) => Promise<Response>;
fetchMock.mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const url = input.toString();
if (url === '/api/v1/smart-collections' && (init?.method ?? 'GET').toUpperCase() === 'GET') {
return new Promise<Response>((resolve) => {
smartRelease.resolve = () => resolve(jsonResponse([{ id: 77, name: 'Smart Pick' }]));
});
}
return inner(input, init);
});
render(<PlaylistsScreen />);
fireEvent.click(await screen.findByText('Bumps'));
fireEvent.click(await screen.findByText('Favorites'));
// The Collection item's picker is offering its own options.
const picker = () => screen.getAllByRole('combobox')[1];
await within(picker()).findByRole('option', { name: 'Favorites' });
const typeSelect = screen.getAllByRole('combobox')[0];
fireEvent.change(typeSelect, { target: { value: 'SmartCollection' } });
// Mid-load: no options from the previous namespace.
expect(within(picker()).queryByRole('option', { name: 'Favorites' })).not.toBeInTheDocument();
smartRelease.resolve?.();
expect(await within(picker()).findByRole('option', { name: 'Smart Pick' })).toBeInTheDocument();
expect(within(picker()).queryByRole('option', { name: 'Favorites' })).not.toBeInTheDocument();
});
// ---- #651 round 9: making the stated invariant actually true on this screen ----
it.each([
{ id: 1.5, label: 'a fractional mediaItemId' },
{ id: 2_147_483_648, label: 'a mediaItemId above int32' }
])('round 9: $label drops its NAME too, blocks Save, and cannot reach a PUT', async ({ id }) => {
// The invariant claimed in spa-conventions — "surfaces as no selection, Save disabled, zero
// writes reachable" — was false here in all three respects: the name survived the dropped id
// (so the row read "Cool Movie" while the draft held null), Save had no selection check, and
// clicking it issued the PUT with `mediaItemId: null`.
const fetchMock = mockApi({
onRequest: (url, method) => {
if (url === '/api/v1/playlists/10/items' && method === 'GET') {
return jsonResponse([
{
...(playlistItems[10] as Array<Record<string, unknown>>)[1],
mediaItemId: id,
mediaItemName: 'Cool Movie'
}
]);
}
return null;
}
});
render(<PlaylistsScreen />);
fireEvent.click(await screen.findByText('Bumps'));
// LOAD-BEARING 1: the label goes with the id — no row claiming a selection the draft lacks.
expect(await screen.findByText(/no movie selected/i)).toBeInTheDocument();
expect(screen.queryByText('Cool Movie')).not.toBeInTheDocument();
// LOAD-BEARING 2: Save is gated on it, with a visible reason. Asserted as the EXACT string —
// a loose /need a selection/i matched both "1 item need" and "1 items needs", so it could not
// see the grammar it appeared to cover, and singular is the common case.
expect(screen.getByText('1 item needs a selection')).toBeInTheDocument();
expect(screen.getByRole('button', { name: 'Save playlist' })).toBeDisabled();
// LOAD-BEARING 3: and the write is genuinely unreachable. Clicking a disabled button is usually
// a no-op restatement of `toBeDisabled()` (see RerunCollectionsScreen.test.tsx, where it is
// deliberately omitted) — it earns its place HERE because on the parent this button was
// ENABLED, so the click actually issued the PUT and this assertion is what discriminates.
fireEvent.click(screen.getByRole('button', { name: 'Save playlist' }));
await new Promise((resolve) => setTimeout(resolve, 0));
expect(
fetchMock.mock.calls.filter(([u, i]) => u === '/api/v1/playlists/10/items' && (i?.method ?? '').toUpperCase() === 'PUT')
).toHaveLength(0);
});
it('round 9: an unbindable playlist GROUP id is not offered and cannot be posted', async () => {
// `playlistGroupId` is seeded from the wire and submitted as an entity reference — the same
// class, on the same screen, which the round-8 sweep missed.
const fetchMock = mockApi({
onRequest: (url, method) =>
url === '/api/v1/playlists/groups' && method === 'GET'
? jsonResponse([
{ id: 2_147_483_648, isSystem: false, name: 'Bogus Group', playlistCount: 0 },
{ id: 1, isSystem: false, name: 'Idents', playlistCount: 1 }
])
: null
});
render(<PlaylistsScreen />);
fireEvent.click(await screen.findByRole('button', { name: 'Add playlist' }));
const dialog = await screen.findByRole('dialog');
const groupSelect = within(dialog).getByRole('combobox');
// LOAD-BEARING: the unbindable group is not offered, so it cannot be chosen or submitted.
expect(within(groupSelect).getByRole('option', { name: 'Idents' })).toBeInTheDocument();
expect(within(groupSelect).queryByRole('option', { name: 'Bogus Group' })).not.toBeInTheDocument();
fireEvent.change(groupSelect, { target: { value: '2147483648' } });
fireEvent.change(within(dialog).getByPlaceholderText('Playlist name'), { target: { value: 'New' } });
fireEvent.click(within(dialog).getByRole('button', { name: 'Create' }));
await new Promise((resolve) => setTimeout(resolve, 0));
const posts = fetchMock.mock.calls.filter(
([u, i]) => u === '/api/v1/playlists' && (i?.method ?? '').toUpperCase() === 'POST'
);
// Unconditional FIRST: `posts` is empty on the fixed build (the unmatched value leaves the
// select at '', so Create is disabled and jsdom won't dispatch its onClick), which means the
// loop below runs ZERO assertions on its own. Without this line a future change that re-enabled
// Create and POSTed `playlistGroupId: null` would still pass.
expect(posts).toHaveLength(0);
for (const post of posts) {
expect(JSON.parse(String(post[1]?.body)).playlistGroupId).not.toBe(2_147_483_648);
}
});
});
+114 -60
View File
@@ -24,7 +24,6 @@ import {
deletePlaylist,
deletePlaylistGroup,
getCollections,
getLibraryBrowseItems,
getMultiCollections,
getPlaylistById,
getPlaylistGroups,
@@ -34,8 +33,12 @@ import {
loadAllPages,
messageFromPlaylistError,
previewPlaylist,
searchLibraryPickerOptions,
selectionIdOrNull,
isSelectionId,
updatePlaylist,
updatePlaylistGroup,
LIBRARY_PICKER_MIN_QUERY,
type LibraryBrowseMediaType,
type Playlist,
type PlaylistGroup,
@@ -43,6 +46,7 @@ import {
type PlaylistItemRequest,
type PlaylistPreviewItem
} from '../api';
import { SearchPicker } from '../schedules/pickers';
type CollectionType = PlaylistItemRequest['collectionType'];
type PlaybackOrder = PlaylistItemRequest['playbackOrder'];
@@ -52,22 +56,17 @@ interface PickerOption {
name: string;
}
// #644 follow-up: `browse` (media-library) picker sources are Class B — bounded to one page, with
// `hint: 'truncated'`/`totalCount` telling the caller there's more than fits (a real, expected
// cap — narrow via search). `collection`/`multi`/`smart` sources stay Class A (page to
// completeness) since they're inherently small, admin-created lists; `hint: 'incomplete'` there
// instead reflects `loadAllPages`'s `complete` flag, i.e. a defensive load that did not converge,
// not a cap. These two are NOT the same condition and must render different copy (round-3 review
// F1): "showing the first N of M" is arithmetically vacuous — and points at a search box that
// doesn't exist for this picker — when N === M on an incomplete Class A load.
// #651: `browse` (media-library) picker sources are no longer list-loaded at all — they resolve by
// SEARCH through a `SearchPicker`, so `loadPickerOptions` never returns items for them and the
// truncation hint they used to carry is gone with the truncation. `collection`/`multi`/`smart`
// sources stay Class A (page to completeness) since they're inherently small, admin-created lists;
// `hint: 'incomplete'` reflects `loadAllPages`'s `complete` flag, i.e. a defensive load that did
// not converge.
interface PickerLoadResult {
items: PickerOption[];
totalCount: number | null;
hint: 'incomplete' | 'none' | 'truncated';
hint: 'incomplete' | 'none';
}
const LIBRARY_BROWSE_PAGE_CAP = 100;
// The 12 playlist item types (mirrors PlaylistEditor.razor's Collection Type select).
// Playlist / RemoteStream and the rerun-only types are intentionally excluded here.
// Each entry knows how to load its picker options; every media-item type maps to a
@@ -156,48 +155,45 @@ function orderOptionsWithCurrent(type: CollectionType, current: PlaybackOrder):
return options.some((option) => option.value === current) ? options : [...options, orderOption(current)];
}
// See RerunCollectionsScreen: one ingress for list-backed options, dropping ids the API cannot bind.
function toPickerOptions(list: Array<{ id: unknown; name?: null | string }>): PickerOption[] {
return list.flatMap((entry) =>
isSelectionId(entry.id) ? [{ id: entry.id, name: entry.name ?? `#${entry.id}` }] : []
);
}
function loadPickerOptions(type: CollectionType, signal?: AbortSignal): Promise<PickerLoadResult> {
const config = configFor(type);
if (!config) {
return Promise.resolve({ hint: 'none', items: [], totalCount: 0 });
return Promise.resolve({ hint: 'none', items: [] });
}
switch (config.source) {
case 'collection':
return getCollections().then((list) => {
const items = list.map((entry) => ({ id: entry.id, name: entry.name ?? `#${entry.id}` }));
return { hint: 'none' as const, items, totalCount: items.length };
const items = toPickerOptions(list);
return { hint: 'none' as const, items };
});
case 'multi':
return loadAllPages(getMultiCollections, undefined, undefined, signal).then(({ complete, items: list }) => {
const items = list.map((entry) => ({ id: entry.id, name: entry.name ?? `#${entry.id}` }));
const items = toPickerOptions(list);
if (!complete && !signal?.aborted) {
// #644 follow-up F3: a superseded/aborted load (a type switch mid-load) also returns
// `complete: false` — that's expected, not a defect, so don't warn on it.
console.warn('PlaylistsScreen: multi-collection picker load did not complete; some items may be missing');
}
return { hint: complete ? ('none' as const) : ('incomplete' as const), items, totalCount: items.length };
return { hint: complete ? ('none' as const) : ('incomplete' as const), items };
});
case 'smart':
return getSmartCollections().then((list) => {
const items = list.map((entry) => ({ id: entry.id, name: entry.name ?? `#${entry.id}` }));
return { hint: 'none' as const, items, totalCount: items.length };
const items = toPickerOptions(list);
return { hint: 'none' as const, items };
});
default:
// Class B (#644 follow-up): a media-library picker over the largest tables (Episode, Song,
// Image, Movie, MusicVideo, ...), which can run into the tens of thousands of rows. Paging to
// completeness here would mean ~200 serial requests — each more expensive than the last
// (LuceneSearchIndex.Search computes hitsLimit = skip + limit) — to populate a native <select>
// with thousands of <option> nodes. Load ONE bounded page instead and surface the truncation
// (see docs/decisions/records/spa/list-completeness-vs-bounded-pickers.md).
return getLibraryBrowseItems({ mediaType: config.browse, pageNum: 0, pageSize: LIBRARY_BROWSE_PAGE_CAP }).then(
(result) => {
const page = result.page ?? [];
const items = page.map((item) => ({ id: item.mediaItemId ?? item.id, name: item.title ?? `#${item.id}` }));
const totalCount = result.totalCount ?? items.length;
return { hint: totalCount > items.length ? ('truncated' as const) : ('none' as const), items, totalCount };
}
);
// A media-library picker over the largest tables (Episode, Song, Image, Movie, MusicVideo,
// ...) loads NOTHING up front (#651): it resolves by search through `SearchPicker` below, so
// no path here can load an unbounded — or even a 100-row — window of the media library.
return Promise.resolve({ hint: 'none', items: [] });
}
}
@@ -237,25 +233,34 @@ function draftFromItem(item: PlaylistItem): DraftItem {
const source = configFor(item.collectionType)?.source ?? 'browse';
let selectedId: number | null;
let selectedName: string;
// NOTE the name is cleared with the id below whenever `selectionIdOrNull` rejects one: keeping a
// label for an id we refused to store makes the row claim "Blade Runner" while the draft holds
// null — two contradictory statements about the same item, and no "no selection" signal
// (#651 round 9).
switch (source) {
case 'collection':
selectedId = item.collectionId ?? null;
selectedId = selectionIdOrNull(item.collectionId);
selectedName = item.collectionName ?? '';
break;
case 'multi':
selectedId = item.multiCollectionId ?? null;
selectedId = selectionIdOrNull(item.multiCollectionId);
selectedName = item.multiCollectionName ?? '';
break;
case 'smart':
selectedId = item.smartCollectionId ?? null;
selectedId = selectionIdOrNull(item.smartCollectionId);
selectedName = item.smartCollectionName ?? '';
break;
default:
selectedId = item.mediaItemId ?? null;
selectedId = selectionIdOrNull(item.mediaItemId);
selectedName = item.mediaItemName ?? '';
}
if (selectedId === null) {
// An id we declined to store cannot have a valid label.
selectedName = '';
}
return {
collectionType: item.collectionType,
count: item.count != null ? String(item.count) : '',
@@ -356,10 +361,14 @@ function AddPlaylistDialog({
onSubmit: (values: { name: string; playlistGroupId: number }) => void;
open: boolean;
}) {
const [groupId, setGroupId] = useState(() => (groups[0] ? String(groups[0].id) : ''));
// The group id is seeded from the wire and submitted as an entity reference, so it is a selection
// id by `selectionId.ts`'s own definition and gets the same boundary treatment — otherwise "every
// path by which an id from the wire becomes editor state" is not literally true (#651 round 9).
const bindableGroups = groups.filter((group) => isSelectionId(group.id));
const [groupId, setGroupId] = useState(() => (bindableGroups[0] ? String(bindableGroups[0].id) : ''));
const [name, setName] = useState('');
const trimmed = name.trim();
const numericGroupId = groupId === '' ? null : Number(groupId);
const numericGroupId = selectionIdOrNull(groupId === '' ? null : Number(groupId));
return (
<Dialog
@@ -386,7 +395,7 @@ function AddPlaylistDialog({
<Select
label="Playlist group"
onChange={(event) => setGroupId(event.target.value)}
options={groups.map((group) => ({ label: group.name, value: String(group.id) }))}
options={bindableGroups.map((group) => ({ label: group.name, value: String(group.id) }))}
value={groupId}
/>
<div style={{ marginTop: 12 }}>
@@ -413,10 +422,11 @@ function PlaylistEditor({ playlistId, onBack, onSaved }: { playlistId: number; o
const [name, setName] = useState('');
const [items, setItems] = useState<DraftItem[]>([]);
const [selectedKey, setSelectedKey] = useState<string | null>(null);
const [pickerItems, setPickerItems] = useState<PickerOption[]>([]);
// Options carry the TYPE that produced them — the previous type's options must not stay
// selectable under the new type's label while its load is in flight (#651 round 4 HIGH-2).
const [pickerFor, setPickerFor] = useState<{ items: PickerOption[]; type: CollectionType } | null>(null);
const [pickerError, setPickerError] = useState<string | null>(null);
const [pickerHint, setPickerHint] = useState<PickerLoadResult['hint']>('none');
const [pickerTotalCount, setPickerTotalCount] = useState<number | null>(null);
const pickerHelpId = useId();
const [saving, setSaving] = useState(false);
const [saveError, setSaveError] = useState<string | null>(null);
@@ -480,20 +490,18 @@ function PlaylistEditor({ playlistId, onBack, onSaved }: { playlistId: number; o
const controller = new AbortController();
loadPickerOptions(selectedType, controller.signal)
.then(({ hint, items, totalCount }) => {
.then(({ hint, items }) => {
if (!controller.signal.aborted) {
setPickerItems(items);
setPickerFor({ items, type: selectedType });
setPickerError(null);
setPickerHint(hint);
setPickerTotalCount(totalCount);
}
})
.catch((error: unknown) => {
if (!controller.signal.aborted) {
setPickerItems([]);
setPickerFor({ items: [], type: selectedType });
setPickerError(messageFromPlaylistError(error, 'Unable to load picker items'));
setPickerHint('none');
setPickerTotalCount(null);
}
});
@@ -502,6 +510,19 @@ function PlaylistEditor({ playlistId, onBack, onSaved }: { playlistId: number; o
};
}, [selectedType]);
// Only options loaded FOR the active type may be offered; anything else belongs to another id
// namespace and must not be selectable, however briefly.
const pickerItems = pickerFor !== null && pickerFor.type === selectedType ? pickerFor.items : [];
const browseMediaType = selectedType !== undefined ? configFor(selectedType)?.browse : undefined;
// Stable across renders: `SearchPicker`'s debounce effect lists `search` in its deps, so a fresh
// closure every render would restart the debounce on each keystroke's re-render.
const searchLibrary = useCallback(
(query: string) => (browseMediaType ? searchLibraryPickerOptions(browseMediaType, query) : Promise.resolve([])),
[browseMediaType]
);
const updateItem = (key: string, patch: Partial<DraftItem>) => {
setItems((current) => current.map((item) => (item.key === key ? { ...item, ...patch } : item)));
};
@@ -563,8 +584,20 @@ function PlaylistEditor({ playlistId, onBack, onSaved }: { playlistId: number; o
const buildRequest = () => ({ items: items.map(toItemRequest), name: name.trim() });
// Every item must carry a selection the API can bind. Without this the screen happily PUT an item
// with a null id — the server 422s it (`ReplacePlaylistItemsHandler.CollectionTypeMustBeValid`),
// but the DB would persist it (`PlaylistItemConfiguration` marks all four FKs `IsRequired(false)`),
// so that handler check is the only thing standing there. Fail closed on the client too.
const itemsMissingSelection = items.filter((item) => item.selectedId == null).length;
const validationError =
name.trim().length === 0
? 'Name is required'
: itemsMissingSelection > 0
? `${itemsMissingSelection} item${itemsMissingSelection === 1 ? '' : 's'} need${itemsMissingSelection === 1 ? 's' : ''} a selection`
: null;
const save = async () => {
if (isSystem || saving || name.trim().length === 0) {
if (isSystem || saving || validationError !== null) {
return;
}
@@ -676,8 +709,9 @@ function PlaylistEditor({ playlistId, onBack, onSaved }: { playlistId: number; o
>
Add item
</Button>
{!isSystem && validationError && <Badge tone="neutral">{validationError}</Badge>}
<Button
disabled={isSystem || saving || name.trim().length === 0}
disabled={isSystem || saving || validationError !== null}
loading={saving}
onClick={() => void save()}
size="sm"
@@ -779,24 +813,44 @@ function PlaylistEditor({ playlistId, onBack, onSaved }: { playlistId: number; o
/>
<div style={{ marginTop: 12 }}>
<Select
ariaDescribedBy={pickerError || pickerHint !== 'none' ? pickerHelpId : undefined}
label={activeConfig?.label ?? 'Selection'}
onChange={(event) => setItemSelection(selectedItem.key, event.target.value)}
options={pickerOptions}
value={selectedItem.selectedId == null ? '' : String(selectedItem.selectedId)}
/>
{browseMediaType ? (
// #651: the media library is resolved by search, never list-loaded. The current
// selection renders from the DRAFT ITEM (`selectedName`, loaded with the playlist),
// not from the result set — so editing an existing item shows its real selection
// before, during and after any search, and never silently loses it.
<SearchPicker
ariaDescribedBy={pickerHelpId}
key={`${selectedItem.key}:${selectedItem.collectionType}`}
label={activeConfig?.label ?? 'Selection'}
minQueryLength={LIBRARY_PICKER_MIN_QUERY}
onClear={() => updateItem(selectedItem.key, { selectedId: null, selectedName: '' })}
onSelect={(id, name) => updateItem(selectedItem.key, { selectedId: id, selectedName: name })}
placeholder={`Search ${(activeConfig?.label ?? 'items').toLowerCase()}`}
search={searchLibrary}
source={selectedItem.collectionType}
selectedId={selectedItem.selectedId}
selectedName={selectedItem.selectedName || null}
/>
) : (
<Select
ariaDescribedBy={pickerError || pickerHint !== 'none' ? pickerHelpId : undefined}
label={activeConfig?.label ?? 'Selection'}
onChange={(event) => setItemSelection(selectedItem.key, event.target.value)}
options={pickerOptions}
value={selectedItem.selectedId == null ? '' : String(selectedItem.selectedId)}
/>
)}
{pickerError && (
<span className="ctv-field-error" id={pickerHelpId} role="alert">
{pickerError}
</span>
)}
{!pickerError && pickerHint === 'truncated' && (
{!pickerError && browseMediaType && (
<span className="ctv-field-help" id={pickerHelpId}>
Showing the first {pickerItems.length} of {pickerTotalCount} use search to narrow.
Type at least {LIBRARY_PICKER_MIN_QUERY} characters to search the library.
</span>
)}
{!pickerError && pickerHint === 'incomplete' && (
{!pickerError && !browseMediaType && pickerHint === 'incomplete' && (
<span className="ctv-field-help" id={pickerHelpId}>
List may be incomplete retry to reload.
</span>
+693 -42
View File
@@ -1,7 +1,17 @@
import { cleanup, fireEvent, render, screen, waitFor, within } from '@testing-library/react';
import { act, cleanup, fireEvent, render, screen, waitFor, within } from '@testing-library/react';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { RerunCollectionsScreen } from './RerunCollectionsScreen';
// A single-record GET response. It ALWAYS carries an ETag, because a real server does and because
// the editor now refuses to open without one (#651 round 6): mocks that omitted the header were
// silently exercising a force-write path that must not exist. Use this for every detail GET.
function detailResponse(body: unknown, etag = '"v1"'): Response {
return new Response(JSON.stringify(body), {
headers: { 'Content-Type': 'application/json', ETag: etag },
status: 200
});
}
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
headers: { 'Content-Type': 'application/json' },
@@ -27,13 +37,27 @@ const manualCollections = [
const multiCollections = [{ id: 8, items: [], name: 'Bundle' }];
// A stored rerun collection whose selection the DETAIL endpoint may fail to name (#651 F1).
const storedSelection = {
collectionType: 'RemoteStream',
firstRunPlaybackOrder: 'Chronological',
id: 9,
name: 'Stored Rerun',
rerunPlaybackOrder: 'Chronological',
selectedId: 42,
selectedName: 'Stored Item'
};
interface MockOptions {
browsePage?: unknown;
list?: unknown[];
onRequest?: (url: string, method: string, body: unknown) => Response | null;
}
function mockApi(options: MockOptions = {}) {
const list = options.list ?? rerunCollections;
const browsePage = options.browsePage ?? { page: [], totalCount: 0 };
return vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const url = input.toString();
@@ -52,7 +76,7 @@ function mockApi(options: MockOptions = {}) {
const rerunById = url.match(/^\/api\/v1\/rerun-collections\/(\d+)$/);
if (rerunById && method === 'GET') {
const found = (list as Array<{ id: number }>).find((r) => String(r.id) === rerunById[1]) ?? list[0];
return Promise.resolve(jsonResponse(found));
return Promise.resolve(detailResponse(found));
}
if (url.startsWith('/api/v1/rerun-collections') && method === 'GET') {
@@ -72,7 +96,7 @@ function mockApi(options: MockOptions = {}) {
}
if (url.startsWith('/api/v1/library/browse')) {
return Promise.resolve(jsonResponse({ page: [], totalCount: 0 }));
return Promise.resolve(jsonResponse(browsePage));
}
return Promise.resolve(new Response(null, { status: 204 }));
@@ -162,37 +186,19 @@ describe('RerunCollectionsScreen', () => {
expect(await within(refreshed[1]).findByText('Bundle')).toBeInTheDocument();
});
it('Class B picker: a media-library type issues exactly ONE /library/browse request and surfaces the truncation hint (#644 follow-up)', async () => {
const total = 5000;
const cap = 100;
const page = Array.from({ length: cap }, (_, i) => ({
id: i + 1,
mediaItemId: i + 1,
mediaType: 'Movie' as const,
title: `Movie ${i + 1}`
}));
const fetchMock = mockApi({ list: [] });
fetchMock.mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const url = new URL(input.toString(), 'http://localhost');
const method = (init?.method ?? 'GET').toUpperCase();
if (url.pathname === '/api/v1/library/browse' && method === 'GET') {
return Promise.resolve(jsonResponse({ page, totalCount: total }));
}
if (url.pathname === '/api/v1/rerun-collections' && method === 'GET') {
return Promise.resolve(jsonResponse({ page: [], totalCount: 0 }));
}
if (url.pathname === '/api/v1/collections' && method === 'GET') {
return Promise.resolve(jsonResponse(manualCollections));
}
if (url.pathname === '/api/v1/multi-collections' && method === 'GET') {
return Promise.resolve(jsonResponse({ page: multiCollections, totalCount: multiCollections.length }));
}
if (url.pathname === '/api/v1/smart-collections' && method === 'GET') {
return Promise.resolve(jsonResponse([]));
}
return Promise.resolve(new Response(null, { status: 204 }));
it('media-library picker: resolves by SEARCH — zero browse requests until the user types (#651)', async () => {
const fetchMock = mockApi({
list: [],
onRequest: (url) =>
url.startsWith('/api/v1/library/browse')
? jsonResponse({
page: [
{ id: 1, mediaItemId: 1, mediaType: 'Movie', title: 'Movie 1' },
{ id: 2, mediaItemId: 2, mediaType: 'Movie', title: 'Movie 2' }
],
totalCount: 2
})
: null
});
render(<RerunCollectionsScreen />);
@@ -201,16 +207,96 @@ describe('RerunCollectionsScreen', () => {
const typeSelect = (await screen.findAllByRole('combobox'))[0];
fireEvent.change(typeSelect, { target: { value: 'Movie' } });
// The bounded page renders (100 options) and the truncation hint appears.
expect(await screen.findByText(/Showing the first 100 of 5000/)).toBeInTheDocument();
const picker = (await screen.findAllByRole('combobox'))[1];
expect(await within(picker).findByText('Movie 100')).toBeInTheDocument();
// The picker is a typeahead, not a <select>: only the three type/order selects remain as
// native <select>s (the typeahead input carries role="combobox" per the ARIA pattern, so count
// elements, not roles).
const searchInput = await screen.findByLabelText('Movie search');
expect(document.querySelectorAll('select')).toHaveLength(3);
expect(screen.getByText(/Type at least 2 characters to search the library/)).toBeInTheDocument();
// Selecting the type loads NOTHING from the media library.
const browseCalls = () =>
fetchMock.mock.calls.filter(
([u]) => new URL(u.toString(), 'http://localhost').pathname === '/api/v1/library/browse'
);
expect(browseCalls()).toHaveLength(0);
// A single character is below the minimum — still no request.
fireEvent.focus(searchInput);
fireEvent.change(searchInput, { target: { value: 'S' } });
await new Promise((resolve) => setTimeout(resolve, 400));
expect(browseCalls()).toHaveLength(0);
// Typed text is COMPILED to a bounded, escaped title-contains query — never forwarded raw.
fireEvent.change(searchInput, { target: { value: 'Show Alpha' } });
await waitFor(() => expect(browseCalls().length).toBeGreaterThan(0));
const url = new URL(browseCalls()[0][0].toString(), 'http://localhost');
expect(url.searchParams.get('query')).toBe('title:*Show\\ Alpha*');
expect(url.searchParams.get('mediaType')).toBe('Movie');
expect(url.searchParams.get('pageNum')).toBe('0');
expect(Number(url.searchParams.get('pageSize'))).toBeLessThanOrEqual(25);
expect(await screen.findByRole('option', { name: 'Movie 1' })).toBeInTheDocument();
});
it('media-library picker: a HUGE library stays bounded — one request per settled query, one page each (#651)', async () => {
// 20,000 rows: the exact shape that made paging-to-completeness untenable. Whatever pageSize is
// asked for, the fixture returns at most that many rows, so an unbounded loader would show up
// BOTH as extra requests and as a huge option list.
const TOTAL = 20000;
const fetchMock = mockApi({ list: [] });
fetchMock.mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const url = new URL(input.toString(), 'http://localhost');
const method = (init?.method ?? 'GET').toUpperCase();
if (url.pathname === '/api/v1/library/browse' && method === 'GET') {
const pageSize = Number(url.searchParams.get('pageSize') ?? '100');
const pageNum = Number(url.searchParams.get('pageNum') ?? '0');
return Promise.resolve(
jsonResponse({
page: Array.from({ length: Math.min(pageSize, TOTAL - pageNum * pageSize) }, (_, i) => ({
id: pageNum * pageSize + i + 1,
mediaItemId: pageNum * pageSize + i + 1,
mediaType: 'Episode' as const,
title: `Episode ${pageNum * pageSize + i + 1}`
})),
totalCount: TOTAL
})
);
}
if (url.pathname === '/api/v1/rerun-collections' && method === 'GET') {
return Promise.resolve(jsonResponse({ page: [], totalCount: 0 }));
}
if (url.pathname === '/api/v1/collections' && method === 'GET') {
return Promise.resolve(jsonResponse(manualCollections));
}
return Promise.resolve(new Response(null, { status: 204 }));
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByRole('button', { name: 'New rerun collection' }));
const typeSelect = (await screen.findAllByRole('combobox'))[0];
fireEvent.change(typeSelect, { target: { value: 'Episode' } });
const searchInput = await screen.findByLabelText('Episode search');
fireEvent.focus(searchInput);
fireEvent.change(searchInput, { target: { value: 'Episode' } });
const results = () => screen.queryByRole('listbox', { name: 'Episode results' });
await waitFor(() => expect(results()).not.toBeNull());
// Let any (nonexistent) follow-up paging loop have time to fire.
await new Promise((resolve) => setTimeout(resolve, 500));
// Exactly ONE /library/browse request — no paging-to-completeness loop over the media library.
const browseCalls = fetchMock.mock.calls.filter(
([u]) => new URL(u.toString(), 'http://localhost').pathname === '/api/v1/library/browse'
);
// ONE request for the settled query — not ~200 pages against a 20k-row table.
expect(browseCalls).toHaveLength(1);
expect(Number(new URL(browseCalls[0][0].toString(), 'http://localhost').searchParams.get('pageSize'))).toBe(25);
// ...and at most one bounded page of options rendered.
expect(within(results() as HTMLElement).getAllByRole('option').length).toBeLessThanOrEqual(25);
});
it('preserves an out-of-set stored playback order on edit-load and re-saves it unchanged', async () => {
@@ -236,10 +322,20 @@ describe('RerunCollectionsScreen', () => {
fireEvent.click(await screen.findByText('Odd Order Reruns'));
// combobox order: [0] Collection Type, [1] selection picker, [2] First Run, [3] Rerun.
// Artist is a media-library type, so its selection picker is a search typeahead, not a
// combobox: [0] Collection Type, [1] First Run, [2] Rerun.
const comboboxes = await screen.findAllByRole('combobox');
const firstRunSelect = comboboxes[2];
const rerunSelect = comboboxes[3];
const firstRunSelect = comboboxes[1];
const rerunSelect = comboboxes[2];
// #651: the stored selection is shown from the RECORD's own selectedName, with no library load
// at all — an edit must never lose (or fail to name) the item it already points at.
expect(await screen.findByText('Some Artist')).toBeInTheDocument();
expect(
fetchMock.mock.calls.filter(
([u]) => new URL(u.toString(), 'http://localhost').pathname === '/api/v1/library/browse'
)
).toHaveLength(0);
// Artist's normal order set (Chronological / Random / Shuffle) does not include
// SeasonEpisode, but the stored value must still be present and selected.
@@ -378,4 +474,559 @@ describe('RerunCollectionsScreen', () => {
).toBe(true);
});
});
it('a server-side TYPE change is adopted whole — no cross-type id survives', async () => {
// The list opened a Collection (id 5, "Favorites"); by the time the detail GET lands the record
// has been changed to RemoteStream, whose response carries the #671 null selection. The draft
// comes wholly from that response, so the Collection id cannot leak into the RemoteStream
// namespace — the defect this originally caught, now prevented structurally rather than by a
// merge rule (#651 round 3 HIGH, round 5 redesign).
let detailCalls = 0;
mockApi({
list: [{ ...storedSelection, collectionType: 'Collection', selectedId: 5, selectedName: 'Favorites' }],
onRequest: (url, method) => {
if (url === '/api/v1/rerun-collections/9' && method === 'GET') {
detailCalls += 1;
return detailResponse({ ...storedSelection, collectionType: 'RemoteStream', selectedId: null, selectedName: null });
}
return null;
}
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
await waitFor(() => expect(detailCalls).toBe(1));
await new Promise((resolve) => setTimeout(resolve, 0));
const typeSelect = (await screen.findAllByRole('combobox'))[0] as HTMLSelectElement;
expect(typeSelect.value).toBe('RemoteStream');
// The Collection id must NOT have been carried across into the RemoteStream id space.
expect(screen.queryByText('Favorites')).not.toBeInTheDocument();
expect(screen.queryByText('#5')).not.toBeInTheDocument();
expect(screen.getByLabelText('Remote Stream search')).toBeInTheDocument();
});
it('HIGH-2: options from the PREVIOUS type are not selectable while the new type loads', async () => {
// Slow connection: switching Collection -> SmartCollection clears the selection immediately but
// the smart-collections request is still in flight. The old Collection options must not remain
// on offer under the SmartCollection label — picking one would write a Collection id into the
// SmartCollection namespace (#651 round 4 HIGH-2).
const smartRelease: { resolve: (() => void) | null } = { resolve: null };
const fetchMock = mockApi({ list: [] });
fetchMock.mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const url = new URL(input.toString(), 'http://localhost');
const method = (init?.method ?? 'GET').toUpperCase();
if (url.pathname === '/api/v1/smart-collections' && method === 'GET') {
return new Promise<Response>((resolve) => {
smartRelease.resolve = () => resolve(jsonResponse([{ id: 77, name: 'Smart Pick' }]));
});
}
if (url.pathname === '/api/v1/collections' && method === 'GET') {
return Promise.resolve(jsonResponse(manualCollections));
}
if (url.pathname === '/api/v1/rerun-collections' && method === 'GET') {
return Promise.resolve(jsonResponse({ page: [], totalCount: 0 }));
}
return Promise.resolve(new Response(null, { status: 204 }));
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByRole('button', { name: 'New rerun collection' }));
const picker = () => (screen.getAllByRole('combobox'))[1];
await within(picker()).findByText('Favorites');
const typeSelect = screen.getAllByRole('combobox')[0];
fireEvent.change(typeSelect, { target: { value: 'SmartCollection' } });
// Mid-load: the picker offers nothing rather than the wrong namespace's rows.
expect(within(picker()).queryByText('Favorites')).not.toBeInTheDocument();
expect(within(picker()).queryByRole('option', { name: 'Favorites' })).not.toBeInTheDocument();
smartRelease.resolve?.();
expect(await within(picker()).findByText('Smart Pick')).toBeInTheDocument();
expect(within(picker()).queryByText('Favorites')).not.toBeInTheDocument();
});
// ---- #651 review round 5: the editor initializes ONCE, from the detail GET ----
//
// Every reconciliation test above was deleted with the machinery it exercised. Rounds 2-4 each
// produced a HIGH finding in the layer that merged a late detail response into a draft the user
// was already editing; round 5 removed the race instead of refereeing it. What is asserted now is
// the ABSENCE of that surface: there is no editable draft until the record lands, so there is
// nothing to reconcile, nothing to pin, and no way to pair one record's id with another's ETag.
it('renders no editable form until the detail GET lands — there is no draft to reconcile', async () => {
const detailRelease: { resolve: (() => void) | null } = { resolve: null };
const fetchMock = mockApi({ list: [storedSelection] });
const inner = fetchMock.getMockImplementation() as (i: RequestInfo | URL, r?: RequestInit) => Promise<Response>;
fetchMock.mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const url = input.toString();
if (url === '/api/v1/rerun-collections/9' && (init?.method ?? 'GET').toUpperCase() === 'GET') {
return new Promise<Response>((resolve) => {
detailRelease.resolve = () => resolve(detailResponse(storedSelection));
});
}
return inner(input, init);
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
// Loading, not an editable form seeded from the list row.
expect(await screen.findByText('Loading rerun collection…')).toBeInTheDocument();
expect(screen.queryByPlaceholderText('Rerun collection name')).not.toBeInTheDocument();
expect(screen.queryByRole('button', { name: 'Save rerun collection' })).not.toBeInTheDocument();
detailRelease.resolve?.();
expect(await screen.findByPlaceholderText('Rerun collection name')).toBeInTheDocument();
});
it('takes the draft SOLELY from the detail GET — the list row is never a source', async () => {
// The list row is stale in every field. None of it may reach the editor: in production it is
// strictly less informative than the detail read (#671 — the list handler applies no Includes).
mockApi({
list: [{ ...storedSelection, collectionType: 'Collection', name: 'Stale List Name', selectedId: 5, selectedName: 'Stale List Selection' }],
onRequest: (url, method) =>
url === '/api/v1/rerun-collections/9' && method === 'GET'
? detailResponse({ ...storedSelection, collectionType: 'Collection', name: 'Server Name', selectedId: 5, selectedName: 'Favorites' })
: null
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stale List Name'));
const nameInput = (await screen.findByPlaceholderText('Rerun collection name')) as HTMLInputElement;
expect(nameInput.value).toBe('Server Name');
expect(screen.queryByText('Stale List Selection')).not.toBeInTheDocument();
});
it('a failed detail GET shows an error instead of an editable form', async () => {
mockApi({
list: [storedSelection],
onRequest: (url, method) =>
url === '/api/v1/rerun-collections/9' && method === 'GET' ? new Response(null, { status: 500 }) : null
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
expect(await screen.findByRole('alert')).toBeInTheDocument();
// Editing a draft that was never loaded is what produced every reconciliation defect.
expect(screen.queryByPlaceholderText('Rerun collection name')).not.toBeInTheDocument();
expect(screen.queryByRole('button', { name: 'Save rerun collection' })).not.toBeInTheDocument();
});
it('Save always carries If-Match — a draft cannot exist without its ETag', async () => {
// The round-4 false-conflict hole: a path that produced a draft but left `etagRef` null made the
// next PUT a FORCE-WRITE. Writing the ETag in the same callback that sets the draft makes that
// unreachable by construction; this pins it.
const fetchMock = mockApi({
list: [storedSelection],
onRequest: (url, method) => {
if (url === '/api/v1/rerun-collections/9' && method === 'GET') {
return new Response(JSON.stringify(storedSelection), {
headers: { 'Content-Type': 'application/json', ETag: '"v7"' },
status: 200
});
}
return url === '/api/v1/rerun-collections/9' && method === 'PUT' ? jsonResponse({ id: 9 }, 200) : null;
}
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
await screen.findByPlaceholderText('Rerun collection name');
fireEvent.click(screen.getByRole('button', { name: 'Save rerun collection' }));
await waitFor(() => {
const put = fetchMock.mock.calls.find(
([u, init]) => u === '/api/v1/rerun-collections/9' && (init?.method ?? '').toUpperCase() === 'PUT'
);
expect(put).toBeDefined();
expect((put?.[1]?.headers as Record<string, string>)['If-Match']).toBe('"v7"');
});
});
it('412 -> Reload: the form is ABSENT while pending, and a dirty selection is discarded even when the server sends none', async () => {
// Replaces the deleted "conflict Reload with a null server selection" regression. Both halves
// matter and the previous version had neither: its second GET resolved immediately (so it never
// observed a pending Reload) and returned a non-null selection (so removing `setDraft(null)`
// could leave it green). Here the reload is HELD OPEN and returns `selectedId: null` — the #671
// shape that round 3 showed could resurrect the user's dirty id over a collaborator's change.
const reloadRelease: { resolve: (() => void) | null } = { resolve: null };
let detailCalls = 0;
const fetchMock = mockApi({
browsePage: { page: [{ id: 99, mediaItemId: 99, mediaType: 'RemoteStream', title: 'New Stream' }], totalCount: 1 },
list: [{ ...storedSelection, collectionType: 'RemoteStream' }],
onRequest: (url, method) =>
url === '/api/v1/rerun-collections/9' && method === 'PUT' ? new Response(null, { status: 412 }) : null
});
const inner = fetchMock.getMockImplementation() as (i: RequestInfo | URL, r?: RequestInit) => Promise<Response>;
fetchMock.mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const url = input.toString();
if (url === '/api/v1/rerun-collections/9' && (init?.method ?? 'GET').toUpperCase() === 'GET') {
detailCalls += 1;
if (detailCalls === 1) {
return Promise.resolve(
detailResponse({ ...storedSelection, collectionType: 'RemoteStream', selectedId: 42, selectedName: 'Stored Stream' })
);
}
return new Promise<Response>((resolve) => {
reloadRelease.resolve = () =>
resolve(
detailResponse(
{ ...storedSelection, collectionType: 'RemoteStream', selectedId: null, selectedName: null },
'"v2"'
)
);
});
}
return inner(input, init);
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
await screen.findByPlaceholderText('Rerun collection name');
// Dirty the selection, then hit the conflict and choose Reload.
fireEvent.click(await screen.findByRole('button', { name: 'Change' }));
const input = await screen.findByLabelText('Remote Stream search');
fireEvent.focus(input);
fireEvent.change(input, { target: { value: 'New Stream' } });
fireEvent.click(await screen.findByRole('option', { name: 'New Stream' }));
expect(screen.getByLabelText(/^Clear /)).toBeInTheDocument();
fireEvent.click(screen.getByRole('button', { name: 'Save rerun collection' }));
const dialog = await screen.findByRole('dialog');
fireEvent.click(within(dialog).getByRole('button', { name: 'Reload' }));
await waitFor(() => expect(detailCalls).toBe(2));
// (1) While the reload is PENDING the form does not exist — nothing to edit, nothing to preserve.
expect(await screen.findByText('Loading rerun collection…')).toBeInTheDocument();
expect(screen.queryByPlaceholderText('Rerun collection name')).not.toBeInTheDocument();
expect(screen.queryByRole('button', { name: 'Save rerun collection' })).not.toBeInTheDocument();
reloadRelease.resolve?.();
await screen.findByPlaceholderText('Rerun collection name');
// (2) The server sent NO selection, and the user's dirty id 99 is gone rather than resurrected.
expect(screen.queryByText('New Stream')).not.toBeInTheDocument();
expect(screen.queryByLabelText(/^Clear /)).not.toBeInTheDocument();
expect(screen.getByRole('button', { name: 'Save rerun collection' })).toBeDisabled();
expect(
fetchMock.mock.calls.filter(([u, i]) => u === '/api/v1/rerun-collections/9' && (i?.method ?? '').toUpperCase() === 'PUT')
).toHaveLength(1);
});
it('a #671 unnamed selection disables Save with a visible reason — it never silently saves a guess', async () => {
// Replaces the F1 family. Under initialize-once there is no list-supplied id to preserve (there
// never was one in production), so the honest outcome is: no selection, Save disabled, and the
// badge says why. The defect to avoid is a SILENT one — saving something the server didn't say.
mockApi({
list: [{ ...storedSelection, collectionType: 'RemoteStream' }],
onRequest: (url, method) =>
url === '/api/v1/rerun-collections/9' && method === 'GET'
? detailResponse({ ...storedSelection, collectionType: 'RemoteStream', selectedId: null, selectedName: null })
: null
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
await screen.findByPlaceholderText('Rerun collection name');
expect(screen.getByText('A selection is required')).toBeInTheDocument();
expect(screen.getByRole('button', { name: 'Save rerun collection' })).toBeDisabled();
});
// Absent, empty and whitespace-only are not three cases — they are one: "no usable concurrency
// token". Validating only `null` let an empty ETag through the gate into an editable draft, where
// `updateRerunCollection`'s `ifMatch ? … : undefined` dropped it as falsy and force-wrote over a
// collaborator (#651 round 7 HIGH). Each shape must be unable to produce a draft at all.
it.each([
{ etag: undefined, label: 'no ETag header at all' },
{ etag: '', label: 'an empty ETag' },
{ etag: ' ', label: 'a whitespace-only ETag' },
{ etag: '\t\n', label: 'a tab/newline ETag' }
])('HIGH: $label yields an error, never an editable draft and never a PUT', async ({ etag }) => {
const fetchMock = mockApi({
list: [storedSelection],
onRequest: (url, method) =>
url === '/api/v1/rerun-collections/9' && method === 'GET'
? new Response(JSON.stringify(storedSelection), {
headers:
etag === undefined
? { 'Content-Type': 'application/json' }
: { 'Content-Type': 'application/json', ETag: etag },
status: 200
})
: null
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
expect(await screen.findByRole('alert')).toBeInTheDocument();
expect(screen.getByText(/without a version tag/i)).toBeInTheDocument();
expect(screen.queryByPlaceholderText('Rerun collection name')).not.toBeInTheDocument();
expect(screen.queryByRole('button', { name: 'Save rerun collection' })).not.toBeInTheDocument();
// The decisive assertion: no unconditional write is reachable at all.
expect(
fetchMock.mock.calls.filter(([u, i]) => u === '/api/v1/rerun-collections/9' && (i?.method ?? '').toUpperCase() === 'PUT')
).toHaveLength(0);
});
it('a padded ETag is TRIMMED and sent as If-Match, not dropped', async () => {
// The other half of "one class": a usable token surrounded by whitespace must still be USED.
// NOTE this is a regression guard, not a demonstration of a defect — `Headers` strips outer
// HTTP whitespace before the app sees it, so this passes pre-#651-round-7 too. It exists so a
// future `usableEtag` that rejects (rather than trims) padding cannot land silently.
const fetchMock = mockApi({
list: [storedSelection],
onRequest: (url, method) => {
if (url === '/api/v1/rerun-collections/9' && method === 'GET') {
return new Response(JSON.stringify(storedSelection), {
headers: { 'Content-Type': 'application/json', ETag: ' "v3" ' },
status: 200
});
}
return url === '/api/v1/rerun-collections/9' && method === 'PUT' ? jsonResponse({ id: 9 }, 200) : null;
}
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
await screen.findByPlaceholderText('Rerun collection name');
fireEvent.click(screen.getByRole('button', { name: 'Save rerun collection' }));
await waitFor(() => {
const put = fetchMock.mock.calls.find(
([u, i]) => u === '/api/v1/rerun-collections/9' && (i?.method ?? '').toUpperCase() === 'PUT'
);
expect(put).toBeDefined();
expect((put?.[1]?.headers as Record<string, string>)['If-Match']).toBe('"v3"');
});
});
// A fetch that honours its AbortSignal, as the real one does. A stub that ignores the signal
// would let an abandoned request live forever and hide exactly what these tests pin.
function abortableDetailFetch() {
const state = { aborts: 0, inFlight: 0, starts: 0 };
vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL, init?: RequestInit) => {
const url = input.toString();
if (url === '/api/v1/rerun-collections/9') {
state.starts += 1;
state.inFlight += 1;
return new Promise<Response>((_resolve, reject) => {
init?.signal?.addEventListener('abort', () => {
state.aborts += 1;
state.inFlight -= 1;
reject(new DOMException('Aborted', 'AbortError'));
});
});
}
if (url.startsWith('/api/v1/rerun-collections')) {
return Promise.resolve(jsonResponse({ page: [storedSelection], totalCount: 1 }));
}
return Promise.resolve(new Response(null, { status: 204 }));
});
return state;
}
it('a never-settling detail GET is ABORTED at the deadline, with a way back and a working Retry', async () => {
vi.useFakeTimers();
try {
const state = abortableDetailFetch();
render(<RerunCollectionsScreen />);
await act(async () => {
await vi.advanceTimersByTimeAsync(10);
});
fireEvent.click(screen.getByText('Stored Rerun'));
await act(async () => {
await vi.advanceTimersByTimeAsync(10);
});
// A route out exists while waiting — a hung request is never a dead end.
expect(screen.getByRole('button', { name: 'All rerun collections' })).toBeInTheDocument();
expect(state.inFlight).toBe(1);
await act(async () => {
await vi.advanceTimersByTimeAsync(16_000);
});
// The deadline CANCELS the work rather than merely abandoning the wait: otherwise each Retry
// would stack another live connection (#651 round 7).
expect(state.aborts).toBe(1);
expect(state.inFlight).toBe(0);
expect(screen.getByRole('alert')).toBeInTheDocument();
expect(screen.getByText(/timed out/i)).toBeInTheDocument();
// Retry reissues exactly one new request — not one per previous attempt.
fireEvent.click(screen.getByRole('button', { name: 'Retry' }));
await act(async () => {
await vi.advanceTimersByTimeAsync(10);
});
expect(state.starts).toBe(2);
expect(state.inFlight).toBe(1);
expect(screen.getByText('Loading rerun collection…')).toBeInTheDocument();
} finally {
vi.useRealTimers();
}
});
it('unmounting aborts the in-flight detail read and clears its deadline', async () => {
vi.useFakeTimers();
try {
const state = abortableDetailFetch();
const { unmount } = render(<RerunCollectionsScreen />);
await act(async () => {
await vi.advanceTimersByTimeAsync(10);
});
fireEvent.click(screen.getByText('Stored Rerun'));
await act(async () => {
await vi.advanceTimersByTimeAsync(10);
});
expect(state.inFlight).toBe(1);
unmount();
expect(state.aborts).toBe(1);
expect(state.inFlight).toBe(0);
// The deadline timer is cleared too, so it cannot fire against an unmounted tree.
await act(async () => {
await vi.advanceTimersByTimeAsync(30_000);
});
expect(state.aborts).toBe(1);
} finally {
vi.useRealTimers();
}
});
// Guards the ABORT/RACE COMPOSITION, not a round-6 defect: round 6's race guard already covered
// the ordinary case. What this fails is an abort-ONLY implementation whose fetch ignores its
// signal — which is exactly the mutation it was written for, and why the stub below deliberately
// ignores `init.signal`.
it('a detail read that settles LATE, after the deadline, cannot revive the editor', async () => {
vi.useFakeTimers();
try {
const late: { resolve: ((r: Response) => void) | null } = { resolve: null };
vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL) => {
const url = input.toString();
if (url === '/api/v1/rerun-collections/9') {
// Deliberately IGNORES the abort signal — the worst case, to prove the `active`/error
// state does not depend on cancellation actually working.
return new Promise<Response>((resolve) => {
late.resolve = resolve;
});
}
if (url.startsWith('/api/v1/rerun-collections')) {
return Promise.resolve(jsonResponse({ page: [storedSelection], totalCount: 1 }));
}
return Promise.resolve(new Response(null, { status: 204 }));
});
render(<RerunCollectionsScreen />);
await act(async () => {
await vi.advanceTimersByTimeAsync(10);
});
fireEvent.click(screen.getByText('Stored Rerun'));
await act(async () => {
await vi.advanceTimersByTimeAsync(16_000);
});
expect(screen.getByRole('alert')).toBeInTheDocument();
// The abandoned request finally answers. It must not resurrect a form the user was told failed.
await act(async () => {
late.resolve?.(
new Response(JSON.stringify(storedSelection), {
headers: { 'Content-Type': 'application/json', ETag: '"late"' },
status: 200
})
);
await vi.advanceTimersByTimeAsync(10);
});
expect(screen.getByRole('alert')).toBeInTheDocument();
expect(screen.queryByPlaceholderText('Rerun collection name')).not.toBeInTheDocument();
} finally {
vi.useRealTimers();
}
});
// ---- #651 round 8: the int32 boundary, on EVERY ingress ----
//
// Round 7 put the predicate inside the search picker's option validator — the site the defect was
// found at — leaving the two other doors into `draft.selectedId` open. These pin them.
it.each([
{ id: 1.5, label: 'a fractional selectedId' },
{ id: 2_147_483_648, label: 'a selectedId above int32' },
{ id: -2_147_483_649, label: 'a selectedId below int32' }
])('round 8: $label in the DETAIL response cannot enable Save or reach a PUT', async ({ id }) => {
mockApi({
list: [storedSelection],
onRequest: (url, method) =>
url === '/api/v1/rerun-collections/9' && method === 'GET'
? detailResponse({ ...storedSelection, collectionType: 'Collection', selectedId: id, selectedName: 'Bogus' })
: null
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByText('Stored Rerun'));
await screen.findByPlaceholderText('Rerun collection name');
// Treated as ABSENT — visibly, with the reason — rather than as a selection that fails on write.
// These two ARE the load-bearing assertions. Clicking the disabled Save is deliberately NOT done
// here: this button is disabled on the parent too, so the click would only restate
// `toBeDisabled()`. (PlaylistsScreen.test.tsx does click it — there the parent left the button
// enabled, so the click genuinely discriminates. The difference is what the parent did, not a
// difference of policy.)
expect(screen.getByText('A selection is required')).toBeInTheDocument();
expect(screen.getByRole('button', { name: 'Save rerun collection' })).toBeDisabled();
});
it.each([
{ id: 1.5, label: 'a fractional id' },
{ id: 2_147_483_648, label: 'an id above int32' }
])('round 8: a LIST-BACKED option with $label is not offered, so it cannot be selected or saved', async ({ id }) => {
const fetchMock = mockApi({
list: [],
onRequest: (url, method) =>
url === '/api/v1/collections' && method === 'GET'
? jsonResponse([
{ collectionType: 'Collection', id, name: 'Bogus Collection', state: 'Normal', useCustomPlaybackOrder: false },
{ collectionType: 'Collection', id: 5, name: 'Favorites', state: 'Normal', useCustomPlaybackOrder: false }
])
: null
});
render(<RerunCollectionsScreen />);
fireEvent.click(await screen.findByRole('button', { name: 'New rerun collection' }));
const picker = (await screen.findAllByRole('combobox'))[1];
// LOAD-BEARING: the bindable option is offered, the malformed one is not rendered at all.
expect(await within(picker).findByText('Favorites')).toBeInTheDocument();
expect(within(picker).queryByText('Bogus Collection')).not.toBeInTheDocument();
// Now actually try to submit it. Selecting a value the <select> never offered leaves the draft
// without a selection, so the save is refused — asserting a POST count without attempting one
// would be trivially true and would pass against the unguarded parent too.
fireEvent.change(picker, { target: { value: String(id) } });
fireEvent.change(screen.getByPlaceholderText('Rerun collection name'), { target: { value: 'Attempted' } });
fireEvent.click(screen.getByRole('button', { name: 'Add rerun collection' }));
await new Promise((resolve) => setTimeout(resolve, 0));
expect(
fetchMock.mock.calls.filter(([u, i]) => u === '/api/v1/rerun-collections' && (i?.method ?? '').toUpperCase() === 'POST')
).toHaveLength(0);
});
});
+252 -82
View File
@@ -8,18 +8,22 @@ import {
createRerunCollection,
deleteRerunCollection,
getCollections,
getLibraryBrowseItems,
getMultiCollections,
getRerunCollectionWithMeta,
getRerunCollections,
getSmartCollections,
loadAllPages,
messageFromRerunCollectionError,
searchLibraryPickerOptions,
selectionIdOrNull,
updateRerunCollection,
isSelectionId,
LIBRARY_PICKER_MIN_QUERY,
type CreateRerunCollectionRequest,
type LibraryBrowseMediaType,
type RerunCollection
} from '../api';
import { SearchPicker } from '../schedules/pickers';
type RerunCollectionType = CreateRerunCollectionRequest['collectionType'];
type PlaybackOrder = CreateRerunCollectionRequest['firstRunPlaybackOrder'];
@@ -29,22 +33,17 @@ interface PickerOption {
name: string;
}
// #644 follow-up: `browse` (media-library) picker sources are Class B — bounded to one page, with
// `hint: 'truncated'`/`totalCount` telling the caller there's more than fits (a real, expected
// cap — narrow via search). `collection`/`multi`/`smart` sources stay Class A (page to
// completeness) since they're inherently small, admin-created lists; `hint: 'incomplete'` there
// instead reflects `loadAllPages`'s `complete` flag, i.e. a defensive load that did not converge,
// not a cap. These two are NOT the same condition and must render different copy (round-3 review
// F1): "showing the first N of M" is arithmetically vacuous — and points at a search box that
// doesn't exist for this picker — when N === M on an incomplete Class A load.
// #651: `browse` (media-library) picker sources are no longer list-loaded at all — they resolve by
// SEARCH through a `SearchPicker`, so `loadPickerOptions` never returns items for them and the
// truncation hint they used to carry is gone with the truncation. `collection`/`multi`/`smart`
// sources stay Class A (page to completeness) since they're inherently small, admin-created lists;
// `hint: 'incomplete'` reflects `loadAllPages`'s `complete` flag, i.e. a defensive load that did
// not converge.
interface PickerLoadResult {
items: PickerOption[];
totalCount: number | null;
hint: 'incomplete' | 'none' | 'truncated';
hint: 'incomplete' | 'none';
}
const LIBRARY_BROWSE_PAGE_CAP = 100;
// The REST-supported selection types (RerunCollectionRequestMapping.IsSupportedSelectionType).
// Playlist / RerunFirstRun / RerunRerun / SearchQuery / Fake* are intentionally excluded.
// Each entry knows how to load its picker options; every media-item type maps to a
@@ -109,48 +108,47 @@ function orderOptionsWithCurrent(type: RerunCollectionType, current: PlaybackOrd
return options.some((option) => option.value === current) ? options : [...options, orderOption(current)];
}
// One ingress for list-backed options. An id the API cannot bind is DROPPED rather than offered:
// an unselectable option is better than one that fails on save (#651 round 8 — the boundary, not
// the site).
function toPickerOptions(list: Array<{ id: unknown; name?: null | string }>): PickerOption[] {
return list.flatMap((entry) =>
isSelectionId(entry.id) ? [{ id: entry.id, name: entry.name ?? `#${entry.id}` }] : []
);
}
function loadPickerOptions(type: RerunCollectionType, signal?: AbortSignal): Promise<PickerLoadResult> {
const config = COLLECTION_TYPES.find((entry) => entry.value === type);
if (!config) {
return Promise.resolve({ hint: 'none', items: [], totalCount: 0 });
return Promise.resolve({ hint: 'none', items: [] });
}
switch (config.source) {
case 'collection':
return getCollections().then((list) => {
const items = list.map((entry) => ({ id: entry.id, name: entry.name ?? `#${entry.id}` }));
return { hint: 'none' as const, items, totalCount: items.length };
const items = toPickerOptions(list);
return { hint: 'none' as const, items };
});
case 'multi':
return loadAllPages(getMultiCollections, undefined, undefined, signal).then(({ complete, items: list }) => {
const items = list.map((entry) => ({ id: entry.id, name: entry.name ?? `#${entry.id}` }));
const items = toPickerOptions(list);
if (!complete && !signal?.aborted) {
// #644 follow-up F3: a superseded/aborted load (Retry, or a type switch mid-load) also
// returns `complete: false` — that's expected, not a defect, so don't warn on it.
console.warn('RerunCollectionsScreen: multi-collection picker load did not complete; some items may be missing');
}
return { hint: complete ? ('none' as const) : ('incomplete' as const), items, totalCount: items.length };
return { hint: complete ? ('none' as const) : ('incomplete' as const), items };
});
case 'smart':
return getSmartCollections().then((list) => {
const items = list.map((entry) => ({ id: entry.id, name: entry.name ?? `#${entry.id}` }));
return { hint: 'none' as const, items, totalCount: items.length };
const items = toPickerOptions(list);
return { hint: 'none' as const, items };
});
default:
// Class B (#644 follow-up): a media-library picker over the largest tables (Episode, Song,
// Image, Movie, MusicVideo, ...), which can run into the tens of thousands of rows. Paging to
// completeness here would mean ~200 serial requests — each more expensive than the last
// (LuceneSearchIndex.Search computes hitsLimit = skip + limit) — to populate a native <select>
// with thousands of <option> nodes. Load ONE bounded page instead and surface the truncation
// (see docs/decisions/records/spa/list-completeness-vs-bounded-pickers.md).
return getLibraryBrowseItems({ mediaType: config.browse, pageNum: 0, pageSize: LIBRARY_BROWSE_PAGE_CAP }).then(
(result) => {
const page = result.page ?? [];
const items = page.map((item) => ({ id: item.mediaItemId ?? item.id, name: item.title ?? `#${item.id}` }));
const totalCount = result.totalCount ?? items.length;
return { hint: totalCount > items.length ? ('truncated' as const) : ('none' as const), items, totalCount };
}
);
// A media-library picker over the largest tables (Episode, Song, Image, Movie, MusicVideo,
// ...) loads NOTHING up front (#651): it resolves by search through `SearchPicker` below, so
// no path here can load an unbounded — or even a 100-row — window of the media library.
return Promise.resolve({ hint: 'none', items: [] });
}
}
@@ -231,17 +229,49 @@ interface Draft {
selectedName: string;
}
// Upper bound on the detail read. A never-settling request must not strand the editor.
const LOAD_TIMEOUT_MS = 15_000;
// The ONE place that decides whether a concurrency token is usable. `null`, `''` and a
// whitespace-only header are not three cases — they are one: "no usable token". Treating only
// `null` as absent let an empty ETag through the fail-closed gate and into an editable draft, where
// `updateRerunCollection`'s `ifMatch ? … : undefined` then dropped it as falsy and force-wrote over
// a collaborator (#651 round 7 HIGH). Returning the TRIMMED token means `etagRef` can only ever
// hold something that will actually be sent.
function usableEtag(etag: string | null | undefined): string | null {
const trimmed = etag?.trim() ?? '';
return trimmed.length > 0 ? trimmed : null;
}
function draftFromRerun(rerun: RerunCollection): Draft {
return {
collectionType: rerun.collectionType,
firstRunPlaybackOrder: rerun.firstRunPlaybackOrder,
name: rerun.name ?? '',
rerunPlaybackOrder: rerun.rerunPlaybackOrder,
selectedId: rerun.selectedId ?? null,
// The other ingress: a malformed id in an otherwise-successful detail response must not become
// an apparently-valid selection. Treated as absent, so Save is disabled and says why.
selectedId: selectionIdOrNull(rerun.selectedId),
selectedName: rerun.selectedName ?? ''
};
}
// NOTE (#651 review round 5): there is deliberately NO draft-reconciliation machinery here — no
// touched-field tracking, no field-wise hydrate, no conflict predicate. Every one of those existed
// to merge a late detail response into a draft the user was already editing, and that reconciliation
// surface produced a HIGH finding in three consecutive review rounds, including three cross-user
// lost updates. The race is removed rather than refereed: the draft is initialized EXACTLY ONCE,
// from the detail GET, and the form does not exist until it lands.
//
// Seeding from the list row (which is what created the race) could never have helped anyway:
// `GetPagedRerunCollectionsHandler` applies ZERO `.Include()`s while `GetRerunCollectionByIdHandler`
// applies fourteen, and both project through the same `ProjectToResponseModel`, which derives
// `selectedId`/`selectedName` from those navigations. The list response is therefore a strict SUBSET
// of the detail one — it can never supply an id the detail lacks (see #671).
//
// Conflicts are detected where they belong: at save time, by `If-Match` -> 412 -> Reload, which
// already existed and already works. Reload re-runs this same initialize-once path.
function RerunCollectionEditor({
initial,
onBack,
@@ -251,9 +281,13 @@ function RerunCollectionEditor({
onBack: () => void;
onSaved: () => void;
}) {
const [draft, setDraft] = useState<Draft>(() =>
// ONE source of truth, set ONCE. `null` means "not loaded yet" — the form does not render, so
// there is no draft for a late response to reconcile against and no window in which the user can
// edit something that is about to be replaced. A NEW record starts from defaults immediately
// because it has nothing to load.
const [draft, setDraft] = useState<Draft | null>(() =>
initial
? draftFromRerun(initial)
? null
: {
collectionType: 'Collection',
firstRunPlaybackOrder: 'Chronological',
@@ -263,71 +297,127 @@ function RerunCollectionEditor({
selectedName: ''
}
);
const [pickerItems, setPickerItems] = useState<PickerOption[]>([]);
// Options carry the TYPE that produced them. Keeping a bare array meant the previous type's
// options stayed on screen under the new type's label while its load was in flight, so a slow
// connection let the user store e.g. a Collection id in the SmartCollection namespace (#651
// round 4 HIGH-2). Same rule as the search picker's `source`: an id is only meaningful inside
// the namespace it came from.
const [pickerFor, setPickerFor] = useState<{ items: PickerOption[]; type: RerunCollectionType } | null>(null);
const [pickerError, setPickerError] = useState<string | null>(null);
const [pickerHint, setPickerHint] = useState<PickerLoadResult['hint']>('none');
const [pickerTotalCount, setPickerTotalCount] = useState<number | null>(null);
const pickerHelpId = useId();
const [loadError, setLoadError] = useState<string | null>(null);
const [saveError, setSaveError] = useState<string | null>(null);
const [saving, setSaving] = useState(false);
const [conflictOpen, setConflictOpen] = useState(false);
const [reloadKey, setReloadKey] = useState(0);
// Bumped by the conflict Reload to re-run the initialize-once load. Reload needs no separate
// "replace" policy: dropping the draft back to `null` IS the replace.
const [loadKey, setLoadKey] = useState(0);
// Concurrency ETag (issue #253): captured from the single-record GET when editing an existing
// rerun collection, sent as If-Match on save. There's no rotation-on-save because the editor
// always returns to the list after a successful save (see onSaved below).
// rerun collection, sent as If-Match on save. It is written in the same callback that sets the
// draft, so a draft can never exist without the ETag that authorizes saving it — which is what
// makes an accidental force-write (a PUT with no If-Match) unreachable.
const etagRef = useRef<string | null>(null);
const { collectionType } = draft;
const collectionType = draft?.collectionType;
// Re-fetch the record's ETag (and current data, on a conflict reload) when editing an existing
// rerun collection. Runs once on mount and again after reloadAfterConflict bumps reloadKey.
// Initialize the draft from the detail GET — the ONLY place it is created for an existing record.
// Re-runs on `loadKey` (the conflict Reload), which drops the draft back to `null` first, so the
// form is unmounted while the replacement is in flight and there is nothing to edit or preserve.
useEffect(() => {
if (!initial) {
return;
}
let active = true;
let timeoutId = 0;
// The deadline ABORTS the request rather than just abandoning the wait: an abandoned GET stays
// in flight, so each Retry would stack another outstanding connection and eventually delay the
// very retry meant to recover (#651 round 7). Cleared on settlement and on unmount.
const controller = new AbortController();
let timedOutFlag = false;
// Two jobs, deliberately BOTH: `abort` cancels the work so retries cannot stack connections,
// and the rejected race stops the UI waiting even if the abort never propagates (a fetch that
// ignores its signal, or one already past the point of cancellation, would otherwise leave the
// editor on a spinner forever — cancellation and giving-up are not the same guarantee).
const timeout = new Promise<never>((_resolve, reject) => {
timeoutId = window.setTimeout(() => {
timedOutFlag = true;
controller.abort(new Error('timeout'));
reject(new Error('Timed out loading rerun collection'));
}, LOAD_TIMEOUT_MS);
});
const timedOut = () => timedOutFlag;
getRerunCollectionWithMeta(initial.id)
Promise.race([getRerunCollectionWithMeta(initial.id, controller.signal), timeout])
.then((meta) => {
if (active) {
etagRef.current = meta.etag;
setDraft(draftFromRerun(meta.data));
if (!active) {
return;
}
// FAIL CLOSED on the absence of a USABLE concurrency token. "The draft is only created
// here, alongside the ETag" is not the same as "a draft implies a token": the header can be
// absent, empty, or whitespace, and each of those ends with `updateRerunCollection` sending
// no `If-Match` at all — a silent force-write. An editor that cannot save safely must not
// exist (#651 rounds 6 and 7).
const token = usableEtag(meta.etag);
if (token === null) {
setLoadError(
'This rerun collection was served without a version tag, so it cannot be edited safely. Reload the page or try again.'
);
return;
}
etagRef.current = token;
setDraft(draftFromRerun(meta.data));
})
.catch((error: unknown) => {
if (active) {
setSaveError(messageFromRerunCollectionError(error, 'Unable to load rerun collection'));
if (!active) {
return;
}
setLoadError(
timedOut()
? 'Timed out loading this rerun collection.'
: messageFromRerunCollectionError(error, 'Unable to load rerun collection')
);
})
.finally(() => {
window.clearTimeout(timeoutId);
});
return () => {
active = false;
window.clearTimeout(timeoutId);
// Cancel the in-flight read on unmount/reload so a superseded request cannot outlive it.
controller.abort(new Error('superseded'));
};
}, [initial, reloadKey]);
}, [initial, loadKey]);
// Load the picker list for the active type. Resets only in the async callbacks (never
// synchronously in the effect body) per spa-conventions §3. Uses an AbortController so a type
// switch mid-load (a Class A `loadAllPages` loop, e.g. multi-collections) stops issuing further
// requests rather than just discarding the eventual result (#644 follow-up F2).
useEffect(() => {
if (collectionType === undefined) {
return;
}
const controller = new AbortController();
loadPickerOptions(collectionType, controller.signal)
.then(({ hint, items, totalCount }) => {
.then(({ hint, items }) => {
if (!controller.signal.aborted) {
setPickerItems(items);
setPickerFor({ items, type: collectionType });
setPickerError(null);
setPickerHint(hint);
setPickerTotalCount(totalCount);
}
})
.catch((error: unknown) => {
if (!controller.signal.aborted) {
setPickerItems([]);
setPickerFor({ items: [], type: collectionType });
setPickerError(messageFromRerunCollectionError(error, 'Unable to load picker items'));
setPickerHint('none');
setPickerTotalCount(null);
}
});
@@ -336,24 +426,84 @@ function RerunCollectionEditor({
};
}, [collectionType]);
// Computed before the early returns below: `useCallback` is a hook and must run on every render.
const browseMediaType = COLLECTION_TYPES.find(
(entry) => entry.value === collectionType && entry.source === 'browse'
)?.browse;
// Stable across renders: `SearchPicker`'s debounce effect lists `search` in its deps, so a fresh
// closure every render would restart the debounce on each keystroke's re-render.
const searchLibrary = useCallback(
(query: string) => (browseMediaType ? searchLibraryPickerOptions(browseMediaType, query) : Promise.resolve([])),
[browseMediaType]
);
// Nothing to edit until the record has loaded. This early return is what removes the whole
// reconciliation problem: below this line `draft` is non-null and immutable-by-anyone-but-the-user.
if (loadError) {
return (
<div className="ctv-settings-error" role="alert">
<TriangleAlert aria-hidden="true" size={16} />
<span>{loadError}</span>
<Button
onClick={() => {
setLoadError(null);
setLoadKey((key) => key + 1);
}}
size="sm"
variant="secondary"
>
Retry
</Button>
<Button onClick={onBack} size="sm" variant="ghost">
All rerun collections
</Button>
</div>
);
}
if (!draft) {
return (
<div className="ctv-collections">
<div className="ctv-channels-actionbar">
{/* A way out while loading: a hung request must never be a dead end (#651 round 6). */}
<Button onClick={onBack} size="sm" startIcon={<ArrowLeft aria-hidden="true" size={14} />} variant="ghost">
All rerun collections
</Button>
</div>
<div className="ctv-collections-loading" role="status">
<Spinner size={18} />
<span>Loading rerun collection</span>
</div>
</div>
);
}
const patch = (delta: Partial<Draft>) => setDraft((current) => (current ? { ...current, ...delta } : current));
const setCollectionType = (type: RerunCollectionType) => {
setDraft((current) => ({
...current,
patch({
collectionType: type,
firstRunPlaybackOrder: defaultOrderFor(type),
rerunPlaybackOrder: defaultOrderFor(type),
selectedId: null,
selectedName: ''
}));
});
};
const setSelected = (value: string) => {
const id = value === '' ? null : Number(value);
const match = pickerItems.find((item) => item.id === id);
setDraft((current) => ({ ...current, selectedId: id, selectedName: match?.name ?? current.selectedName }));
setDraft((current) =>
current ? { ...current, selectedId: id, selectedName: match?.name ?? current.selectedName } : current
);
};
const activeConfig = COLLECTION_TYPES.find((entry) => entry.value === collectionType);
// Only options loaded FOR the active type may be offered; anything else belongs to another id
// namespace and must not be selectable, however briefly.
const pickerItems = pickerFor !== null && pickerFor.type === draft.collectionType ? pickerFor.items : [];
const activeConfig = COLLECTION_TYPES.find((entry) => entry.value === draft.collectionType);
const trimmedName = draft.name.trim();
const validationError =
trimmedName.length === 0 ? 'Name is required' : draft.selectedId == null ? 'A selection is required' : null;
@@ -397,7 +547,11 @@ function RerunCollectionEditor({
const reloadAfterConflict = () => {
setConflictOpen(false);
setSaveError(null);
setReloadKey((key) => key + 1);
// Dropping the draft IS the replace: the form unmounts, so there is no window in which the user
// can edit something the incoming record is about to overwrite (#651 round 4 MEDIUM-4, now
// structural rather than guarded).
setDraft(null);
setLoadKey((key) => key + 1);
};
// Ensure the currently-selected item is always an option even if it isn't in the fetched
@@ -411,8 +565,8 @@ function RerunCollectionEditor({
...pickerItems.map((item) => ({ label: item.name, value: String(item.id) }))
];
const firstRunOrderOptions = orderOptionsWithCurrent(collectionType, draft.firstRunPlaybackOrder);
const rerunOrderOptions = orderOptionsWithCurrent(collectionType, draft.rerunPlaybackOrder);
const firstRunOrderOptions = orderOptionsWithCurrent(draft.collectionType, draft.firstRunPlaybackOrder);
const rerunOrderOptions = orderOptionsWithCurrent(draft.collectionType, draft.rerunPlaybackOrder);
return (
<div className="ctv-collections">
@@ -440,7 +594,7 @@ function RerunCollectionEditor({
<Card title="Rerun collection">
<Input
label="Name"
onChange={(event) => setDraft((current) => ({ ...current, name: event.target.value }))}
onChange={(event) => patch({ name: event.target.value })}
placeholder="Rerun collection name"
value={draft.name}
/>
@@ -455,24 +609,44 @@ function RerunCollectionEditor({
</div>
<div style={{ marginTop: 12 }}>
<Select
ariaDescribedBy={pickerError || pickerHint !== 'none' ? pickerHelpId : undefined}
label={activeConfig?.label ?? 'Selection'}
onChange={(event) => setSelected(event.target.value)}
options={pickerOptions}
value={draft.selectedId == null ? '' : String(draft.selectedId)}
/>
{browseMediaType ? (
// #651: the media library is resolved by search, never list-loaded. The current
// selection renders from the DRAFT (`selectedName`, loaded with the record), not from
// the result set — so editing an existing rerun collection shows its real selection
// before, during and after any search, and never silently loses it.
<SearchPicker
ariaDescribedBy={pickerHelpId}
key={draft.collectionType}
label={activeConfig?.label ?? 'Selection'}
minQueryLength={LIBRARY_PICKER_MIN_QUERY}
onClear={() => patch({ selectedId: null, selectedName: '' })}
onSelect={(id, name) => patch({ selectedId: id, selectedName: name })}
placeholder={`Search ${(activeConfig?.label ?? 'items').toLowerCase()}`}
search={searchLibrary}
source={draft.collectionType}
selectedId={draft.selectedId}
selectedName={draft.selectedName || null}
/>
) : (
<Select
ariaDescribedBy={pickerError || pickerHint !== 'none' ? pickerHelpId : undefined}
label={activeConfig?.label ?? 'Selection'}
onChange={(event) => setSelected(event.target.value)}
options={pickerOptions}
value={draft.selectedId == null ? '' : String(draft.selectedId)}
/>
)}
{pickerError && (
<span className="ctv-field-error" id={pickerHelpId} role="alert">
{pickerError}
</span>
)}
{!pickerError && pickerHint === 'truncated' && (
{!pickerError && browseMediaType && (
<span className="ctv-field-help" id={pickerHelpId}>
Showing the first {pickerItems.length} of {pickerTotalCount} use search to narrow.
Type at least {LIBRARY_PICKER_MIN_QUERY} characters to search the library.
</span>
)}
{!pickerError && pickerHint === 'incomplete' && (
{!pickerError && !browseMediaType && pickerHint === 'incomplete' && (
<span className="ctv-field-help" id={pickerHelpId}>
List may be incomplete retry to reload.
</span>
@@ -482,9 +656,7 @@ function RerunCollectionEditor({
<div style={{ marginTop: 12 }}>
<Select
label="First Run Playback Order"
onChange={(event) =>
setDraft((current) => ({ ...current, firstRunPlaybackOrder: event.target.value as PlaybackOrder }))
}
onChange={(event) => patch({ firstRunPlaybackOrder: event.target.value as PlaybackOrder })}
options={firstRunOrderOptions}
value={draft.firstRunPlaybackOrder}
/>
@@ -493,9 +665,7 @@ function RerunCollectionEditor({
<div style={{ marginTop: 12 }}>
<Select
label="Rerun Playback Order"
onChange={(event) =>
setDraft((current) => ({ ...current, rerunPlaybackOrder: event.target.value as PlaybackOrder }))
}
onChange={(event) => patch({ rerunPlaybackOrder: event.target.value as PlaybackOrder })}
options={rerunOrderOptions}
value={draft.rerunPlaybackOrder}
/>
+136
View File
@@ -0,0 +1,136 @@
import { cleanup, render, screen } from '@testing-library/react';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { TraktListsScreen } from './TraktListsScreen';
function jsonResponse(body: unknown, status = 200): Response {
return new Response(JSON.stringify(body), {
headers: { 'Content-Type': 'application/json' },
status
});
}
describe('TraktListsScreen', () => {
afterEach(() => {
cleanup();
vi.restoreAllMocks();
});
it(
'pages to completeness: a 101-item list (one over the 100 server cap) issues a SECOND request and ' +
'renders every row, with the footer count agreeing with the row count (#650)',
async () => {
// Trakt lists are bounded-by-construction (admin-added), so this pins the exact boundary named
// in #650: a request for EXACTLY the cap (100) truncates just as much as an over-cap request,
// so 101 rows (one over the cap) must still page to completeness rather than showing "101
// lists" over a 100-row table (the self-contradictory symptom #650 described).
const total = 101;
const cap = 100;
const all = Array.from({ length: total }, (_, i) => ({
autoRefresh: false,
generatePlaylist: false,
id: i + 1,
itemCount: 10,
matchCount: 5,
name: `List ${i + 1}`,
slug: `list-${i + 1}`,
traktId: 1000 + i
}));
const fetchMock = vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL) => {
const url = new URL(input.toString(), 'http://localhost');
if (url.pathname === '/api/v1/trakt/lists') {
const pageNum = Number(url.searchParams.get('pageNum') ?? '0');
const pageSize = Number(url.searchParams.get('pageSize') ?? String(cap));
const start = pageNum * pageSize;
return Promise.resolve(jsonResponse({ page: all.slice(start, start + pageSize), totalCount: total }));
}
if (url.pathname === '/api/v1/trakt/status') {
return Promise.resolve(jsonResponse({ busy: false }));
}
return Promise.resolve(new Response(null, { status: 204 }));
});
render(<TraktListsScreen />);
expect(await screen.findByText('List 101')).toBeInTheDocument();
expect(screen.getByText('List 1')).toBeInTheDocument();
// Row count and footer count must AGREE at the boundary — the #650 defect was rendering
// "101 lists" above a table truncated to 100 rows.
expect(screen.getAllByRole('row')).toHaveLength(total + 1); // +1 header row
expect(screen.getByText(`${total} lists`)).toBeInTheDocument();
expect(screen.queryByText('List may be incomplete — retry to reload')).not.toBeInTheDocument();
const listCalls = fetchMock.mock.calls.filter(
([u]) => new URL(u.toString(), 'http://localhost').pathname === '/api/v1/trakt/lists'
);
expect(listCalls).toHaveLength(2);
}
);
it('surfaces an incomplete-load warning when a page comes back short of totalCount (#650)', async () => {
// Defensive break in loadAllPages: an empty (or short) page before totalCount converges must
// render the same "may be incomplete" state used by the other Class A screens, not silently
// show a partial list as if it were the whole one.
vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL) => {
const url = new URL(input.toString(), 'http://localhost');
if (url.pathname === '/api/v1/trakt/lists') {
const pageNum = Number(url.searchParams.get('pageNum') ?? '0');
// Page 0 returns one row; every subsequent page comes back empty even though
// totalCount (50) never converges — the defensive break loadAllPages relies on.
const page =
pageNum === 0
? [
{
autoRefresh: false,
generatePlaylist: false,
id: 1,
itemCount: 1,
matchCount: 1,
name: 'Only List',
slug: 'only-list',
traktId: 1
}
]
: [];
return Promise.resolve(jsonResponse({ page, totalCount: 50 }));
}
if (url.pathname === '/api/v1/trakt/status') {
return Promise.resolve(jsonResponse({ busy: false }));
}
return Promise.resolve(new Response(null, { status: 204 }));
});
render(<TraktListsScreen />);
expect(await screen.findByText('Only List')).toBeInTheDocument();
expect(screen.getByText('List may be incomplete — retry to reload')).toBeInTheDocument();
expect(screen.getByText('1 list')).toBeInTheDocument();
});
it(
'does NOT claim "No Trakt lists yet" when zero rows accumulated from an incomplete load ' +
'(#650 follow-up F2)',
async () => {
// Every page comes back empty even though totalCount (50) never converges, so
// loadAllPages resolves { items: [], complete: false } — zero rows AND incomplete at the
// same time. The empty-state text is an unsupported positive claim ("there are zero lists")
// when the load never actually finished; only the incomplete badge/message should show.
vi.spyOn(window, 'fetch').mockImplementation((input: RequestInfo | URL) => {
const url = new URL(input.toString(), 'http://localhost');
if (url.pathname === '/api/v1/trakt/lists') {
return Promise.resolve(jsonResponse({ page: [], totalCount: 50 }));
}
if (url.pathname === '/api/v1/trakt/status') {
return Promise.resolve(jsonResponse({ busy: false }));
}
return Promise.resolve(new Response(null, { status: 204 }));
});
render(<TraktListsScreen />);
expect(await screen.findByText('List may be incomplete — retry to reload')).toBeInTheDocument();
expect(screen.queryByText('No Trakt lists yet.')).not.toBeInTheDocument();
}
);
});
+38 -9
View File
@@ -9,6 +9,7 @@ import {
getTraktListById,
getTraktLists,
getTraktStatus,
loadAllPages,
matchTraktList,
messageFromTraktError,
updateTraktList,
@@ -265,7 +266,7 @@ export function TraktListsScreen() {
const editingId = traktListIdFromPathname(window.location.pathname);
const [lists, setLists] = useState<TraktList[]>([]);
const [totalCount, setTotalCount] = useState(0);
const [incomplete, setIncomplete] = useState(false);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const [addOpen, setAddOpen] = useState(false);
@@ -276,6 +277,12 @@ export function TraktListsScreen() {
const [deleteError, setDeleteError] = useState<string | null>(null);
const [rowError, setRowError] = useState<string | null>(null);
const activeRef = useRef(true);
// Monotonic request id (spa-conventions §3): `refresh()`/the busy->idle transition can
// re-trigger `load()`, and now that a load is a multi-request `loadAllPages` loop, an older
// loop can resolve after a newer one — guarding on `activeRef` (still mounted) alone isn't
// enough (#644 follow-up F3).
const seqRef = useRef(0);
const abortRef = useRef<AbortController | null>(null);
// TopBar "Add Trakt List" primary action. On the list, open the add dialog. On the
// detail sub-route (/app/trakt-lists/{id}) the add dialog isn't mounted and this component
@@ -294,21 +301,36 @@ export function TraktListsScreen() {
// shows the spinner; later quiet reloads (e.g. after a busy -> idle transition) never
// touch `loading`, so the table stays visible instead of flashing back to a spinner.
const load = useCallback(() => {
getTraktLists({ pageSize: 100 })
.then((paged) => {
if (activeRef.current) {
setLists(paged.page ?? []);
setTotalCount(paged.totalCount ?? 0);
abortRef.current?.abort();
const controller = new AbortController();
abortRef.current = controller;
const seq = (seqRef.current += 1);
// Trakt lists are admin-created (bounded by construction), so this pages to completeness
// rather than requesting a single at-cap page (#650 — the same defect class as #644/#634:
// treating one page as the whole list, just triggered by an exact-cap request instead of
// an over-cap one).
loadAllPages(getTraktLists, undefined, undefined, controller.signal)
.then(({ complete, items }) => {
if (activeRef.current && seqRef.current === seq) {
if (!complete && !controller.signal.aborted) {
// #644 follow-up F4: a partial result is otherwise indistinguishable from a complete
// one. Surfaced via `incomplete` below; also logged so it shows up outside the UI.
console.warn('TraktListsScreen: trakt-lists load did not complete; some lists may be missing');
}
setLists(items);
setIncomplete(!complete);
setError(null);
}
})
.catch((loadError: unknown) => {
if (activeRef.current) {
if (activeRef.current && seqRef.current === seq) {
setError(messageFromTraktError(loadError));
}
})
.finally(() => {
if (activeRef.current) {
if (activeRef.current && seqRef.current === seq) {
setLoading(false);
}
});
@@ -325,6 +347,7 @@ export function TraktListsScreen() {
return () => {
activeRef.current = false;
abortRef.current?.abort();
};
}, [load]);
@@ -394,6 +417,7 @@ export function TraktListsScreen() {
<Spinner size={12} /> Busy
</Badge>
)}
{incomplete && <Badge tone="warn">List may be incomplete retry to reload</Badge>}
<span className="ctv-channels-spacer" />
<Button
disabled={busy}
@@ -432,6 +456,11 @@ export function TraktListsScreen() {
<Spinner size={18} />
<span>Loading Trakt lists</span>
</div>
) : lists.length === 0 && incomplete ? (
// #650 follow-up F2: an incomplete load with zero accumulated rows must not also claim
// "No Trakt lists yet." — that's an unsupported positive claim when the load never
// finished. The "may be incomplete" badge above already carries the real state.
<div className="ctv-collections-empty">Unable to load Trakt lists retry to reload.</div>
) : lists.length === 0 ? (
<div className="ctv-collections-empty">No Trakt lists yet.</div>
) : (
@@ -497,7 +526,7 @@ export function TraktListsScreen() {
</div>
<div className="ctv-channels-footer">
<span>
{totalCount} list{totalCount === 1 ? '' : 's'}
{lists.length} list{lists.length === 1 ? '' : 's'}
</span>
</div>
</div>
+20 -1
View File
@@ -1612,11 +1612,30 @@ body {
font-size: 13px;
}
.ctv-picker-result:hover,
/* Results for a superseded query: still visible (hiding them flickers on every keystroke) but
plainly not actionable until their replacement lands. `choose` refuses them regardless this is
the affordance, not the guard. */
.ctv-picker-results-stale {
opacity: 0.5;
}
.ctv-picker-results-stale .ctv-picker-result {
cursor: default;
}
/* `-active` is the keyboard virtual cursor (aria-activedescendant); it deliberately shares the
hover treatment so pointer and keyboard users get the same affordance. */
.ctv-picker-results:not(.ctv-picker-results-stale) .ctv-picker-result:hover,
.ctv-picker-result-active,
.ctv-picker-result[aria-selected='true'] {
background: var(--surface-3, rgba(255, 255, 255, 0.08));
}
.ctv-picker-error {
color: var(--status-error, #f87171);
border-color: currentColor;
}
.ctv-picker-empty {
padding: 8px 10px;
font-size: 12px;