Compare commits
6
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 timothy
|
1ef581403c | ||
|
timothy
|
990d32f31a | ||
|
timothy
|
851dca2596 | ||
|
timothy
|
aa4a8fb849 | ||
|
timothy
|
243bec708d | ||
|
timothy
|
4596603020 |
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# ersatztv#521 — the line-level append-only mechanic is retired. Decision integrity is now enforced by
|
||||
# the lifecycle validator. A `Decisions-Edit: yes` trailer survives ONLY for rationale-prose edits (validator
|
||||
# the lifecycle validator. `[decisions-edit]` survives ONLY for rationale-prose edits (validator
|
||||
# body-diff, CI). This shim runs the structural validator over the working tree; the body-diff/no-
|
||||
# vanish checks run in CI where a base/head is available. Fail-open on any tooling trouble.
|
||||
set -uo pipefail
|
||||
|
||||
@@ -7,14 +7,6 @@
|
||||
# (c) a review-verdict comment on the PR references the CURRENT head sha (H10) — proving the
|
||||
# LATEST commit was reviewed, not a stale earlier diff (the ersatztv#242 failure mode:
|
||||
# "re-review the fix commit, not just the initial PR diff").
|
||||
#
|
||||
# EVERY ONE OF THOSE IS A SNAPSHOT, taken when the merge tool is called. That is sound for an
|
||||
# immediate merge and UNSOUND for a scheduled one: with merge_when_checks_succeed, Gitea merges
|
||||
# later, against whatever head is green then (ersatztv#622). So the sha-bound half of H10 is
|
||||
# enforced by the SERVER, not here — `review-verdict/h10` is a required status check on `main`,
|
||||
# written per-sha by scripts/post-review-verdict.sh, and a new commit cannot inherit it. This hook
|
||||
# additionally refuses to SCHEDULE an auto-merge unless that status is already green on head, so the
|
||||
# two mechanisms agree at the only moment they can both observe the same commit.
|
||||
# The "## Done-when" issue-body checklist is the convention (docs/decisions.md, CLAUDE.md Task
|
||||
# Completion Protocol). One box is "adversarial review passed"; the others are per-issue.
|
||||
# The H10 review-verdict convention: after reviewing a PR (or its latest fix commit), post a PR
|
||||
@@ -85,58 +77,8 @@ sha=$(printf '%s' "$prjson" | jq -r '.head.sha // ""' 2>/dev/null || true)
|
||||
body=$(printf '%s' "$prjson" | jq -r '.body // ""' 2>/dev/null || true)
|
||||
|
||||
# --- Docs-only exemption: if every changed file is docs/process, skip the gate. ---
|
||||
# The file list must be enumerated EXHAUSTIVELY, validated row by row, and bound to ONE head, or the
|
||||
# exemption is unsafe. ALL of that now lives in scripts/pr-changed-files.sh — the single shared
|
||||
# implementation, also called by .gitea/workflows/review-verdict.yml (ersatztv#649).
|
||||
#
|
||||
# Why it moved: this logic was written twice. This copy is ADVISORY (a failure produces a human
|
||||
# prompt); the workflow's copy is ENFORCED (it writes the branch-protection-required
|
||||
# `review-verdict/h10` status). Four rounds of ersatztv#643 hardening landed here and never reached
|
||||
# there, leaving the copy with real authority strictly weaker than the copy without — and its safe
|
||||
# behaviour resting on a bash arithmetic error rather than an intentional guard. Two copies of a
|
||||
# security predicate drift; one cannot.
|
||||
#
|
||||
# What is NOT shared, deliberately: the docs-only allow-list below. This one also lets .claude/,
|
||||
# .gitea/ and .husky/ through, which is safe HERE only because a match falls through to a human
|
||||
# prompt rather than auto-granting. The workflow's list is narrower for exactly that reason. Sharing
|
||||
# the enumeration fixes the drift; sharing the classification would erase an intended difference.
|
||||
#
|
||||
# A non-zero exit means "could not tell" and MUST withhold the exemption — never read stdout without
|
||||
# checking the status. An empty `$sha` (unparseable PR JSON) reaches the script as an empty argument
|
||||
# and is rejected there, so that path also fails closed.
|
||||
#
|
||||
# The 5th argument binds the enumeration to a base branch (ersatztv#698 route 1), because
|
||||
# `/pulls/{n}/files` diffs against the PR's LIVE base and retargeting moves that without moving the
|
||||
# head. Be precise about what it buys HERE, which is less than what it buys in the workflow: the
|
||||
# workflow passes the base from a `pull_request_target` event payload, fixed at event time and beyond
|
||||
# a retarget's reach, so it detects a retarget outright. This hook has no such trusted snapshot — it
|
||||
# passes the base it just read from the live PR, so what it asserts is that the base did not move
|
||||
# between that read and the enumeration. Narrower, and still worth having: without it the hook cannot
|
||||
# tell a mid-flight retarget from an honest read at all. An empty/unparseable `.base.ref` reaches the
|
||||
# script as an empty argument and is rejected there, so that path fails closed too.
|
||||
base_ref=$(printf '%s' "$prjson" | jq -r '.base.ref // ""' 2>/dev/null || true)
|
||||
repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
|
||||
files=""; files_complete=no
|
||||
if files=$("$repo_root/scripts/pr-changed-files.sh" "$owner" "$repo" "$pr" "$sha" "$base_ref" 2>/dev/null); then
|
||||
files_complete=yes
|
||||
fi
|
||||
|
||||
# HOW THIS PREDICATE IS EVALUATED, matching the enforced gate (ersatztv#698,
|
||||
# `ci.grep-q-pipefail-inversion`). `printf … | grep -q` INVERTS under `set -o pipefail`: grep -q exits
|
||||
# at its first match, printf then takes SIGPIPE (141), and a MATCH is reported as a failed pipeline —
|
||||
# so this negated test would grant a spurious docs-only exemption for any PR whose path list exceeds
|
||||
# the pipe buffer. A here-string fixes that but is materialised via temporary storage for large inputs,
|
||||
# so it can fail when temp space is full or unwritable and flip the predicate the same way. Counting
|
||||
# with `grep -c` drains stdin (no SIGPIPE) over an ordinary pipe (no temp file); `grep -c` exits 1 for
|
||||
# a zero count, which is a legitimate answer, so only a status >1 is a real error and is treated as
|
||||
# "cannot tell" -> no exemption.
|
||||
# Advisory here, so the blast radius is a missing prompt rather than a green required check; the
|
||||
# construct is identical on purpose, because the two copies drifting is what ersatztv#649 was about.
|
||||
docs_nonmatching=$(printf '%s\n' "$files" | grep -cvE '^(docs/|\.claude/|\.husky/|\.gitea/|.*\.md$)') || docs_grep_status=$?
|
||||
if [ "${docs_grep_status:-0}" -gt 1 ]; then
|
||||
docs_nonmatching=1 # grep itself failed: cannot tell, so withhold the exemption
|
||||
fi
|
||||
if [ "$files_complete" = yes ] && [ -n "$files" ] && [ "${docs_nonmatching:-1}" -eq 0 ]; then
|
||||
files=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=100" | jq -r '.[].filename // empty' 2>/dev/null || true)
|
||||
if [ -n "$files" ] && ! printf '%s\n' "$files" | grep -qvE '^(docs/|\.claude/|\.husky/|\.gitea/|.*\.md$)'; then
|
||||
# Docs/process-only PR: the Done-when + review-verdict gate doesn't apply — but this exemption is a
|
||||
# file-TYPE bypass, NOT the a+b+c "provably reviewed & ready" proof, so it does NOT auto-grant. It
|
||||
# passes through to normal permissioning (one prompt). This deliberately keeps a human in the loop for
|
||||
@@ -146,76 +88,6 @@ if [ "$files_complete" = yes ] && [ -n "$files" ] && [ "${docs_nonmatching:-1}"
|
||||
decide allow "" # passthrough (exit 0 → normal prompt), NOT grant
|
||||
fi
|
||||
|
||||
# --- Base-change detection: a verdict is bound to a head AND to a base (ersatztv#632). ---
|
||||
# `review-verdict/h10` is per-sha, which makes "the head moved under a fixed verdict" impossible by
|
||||
# construction. Retargeting a PR's base is the mirror case and slips through: it changes neither the
|
||||
# head sha nor the status, so a verdict formed while the PR targeted `main` still reads green after
|
||||
# the PR is pointed at a branch with a very different merge-base. The diff moves while the verdict
|
||||
# and the head both hold still.
|
||||
#
|
||||
# DETECTION, NOT PREVENTION, and only on this path. A commit status carries no base, so the
|
||||
# server-side required check cannot see this; a merge driven through the Gitea UI or API is
|
||||
# unaffected. That is the accepted exposure — base changes are rare, manual, and this is a
|
||||
# two-account repo — but it is now recorded in a place that fails LOUD rather than only in a doc.
|
||||
#
|
||||
# GRACEFUL ADOPTION, mirroring (b) and (c): a description with no `(base: …)` field is a verdict
|
||||
# posted before ersatztv#632 and gets NO opinion, rather than denying every in-flight PR the day
|
||||
# this lands. The window closes on its own — verdicts are per-head and short-lived, so every verdict
|
||||
# posted after this carries the field.
|
||||
# "Could not check" is a THIRD outcome, distinct from both "matches" and "no base recorded". Cold
|
||||
# review found the first draft collapsing it into the latter: an unreadable status response yielded
|
||||
# an empty `recorded_base`, which took the graceful-adoption path and skipped validation silently —
|
||||
# after which a later, successful status read could still auto-grant. A transient failure would then
|
||||
# have produced a "merge gate: satisfied" message for a comparison that never happened. Every
|
||||
# unreadable input here therefore falls through to a human (`ask`), never to silence.
|
||||
live_base=$(printf '%s' "$prjson" | jq -r '.base.ref // ""' 2>/dev/null || true)
|
||||
if [ -z "$live_base" ]; then
|
||||
decide ask "H10 merge gate: PR #$pr reports no base branch (.base.ref), so the verdict cannot be checked against the branch it was formed for (ersatztv#632). Confirm the PR still targets the branch it was reviewed against before merging."
|
||||
fi
|
||||
if [ -n "$sha" ]; then
|
||||
# This is the THIRD read of this endpoint in a worst-case hook run (the ordinary-CI branch and the
|
||||
# scheduled-auto-merge branch each do their own). Sharing one snapshot would close a narrow
|
||||
# same-run window where two reads disagree, but the later branches derive different decisions from
|
||||
# a failed read than this one does, so threading a shared response through them is a change to
|
||||
# pre-existing logic rather than to ersatztv#632's. Left deliberately, noted so it is not
|
||||
# rediscovered as an oversight: every `decide` exits immediately, so the reads cannot produce a
|
||||
# single self-contradictory message — only a later decision made on a fresher snapshot.
|
||||
vjson_base=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100")
|
||||
# Same jq-1.6 rule as everywhere else in this file: check emptiness in SHELL first, never via
|
||||
# `jq -e`'s exit status over empty input.
|
||||
# VALIDATE EVERY FIELD THE EXTRACTION CONSUMES, on EVERY row — the same rule the file-enumeration
|
||||
# guard learned the hard way. Checking only that `.statuses` is an array left a hole one level
|
||||
# down: `{"statuses":[1]}` passes a top-level type check, then `.context` on a number errors, and
|
||||
# a `|| true` on the extraction turned that error into an empty `vdesc` — i.e. straight back onto
|
||||
# the graceful-adoption path this block exists to distinguish from. That is the identical
|
||||
# swallow-the-error shape fixed a few lines up, surviving one level deeper.
|
||||
if [ -z "${vjson_base//[[:space:]]/}" ] \
|
||||
|| ! printf '%s' "$vjson_base" \
|
||||
| jq -e '.statuses | type == "array"
|
||||
and all(.[]; type == "object"
|
||||
and (.context | type == "string")
|
||||
and (.description == null or (.description | type == "string")))' \
|
||||
>/dev/null 2>&1; then
|
||||
decide ask "H10 merge gate: could not read the commit statuses for PR #$pr head ${sha:0:7}, so the verdict could not be checked against the PR's base branch (ersatztv#632). Confirm the review covered the branch this PR currently targets ('$live_base') before merging."
|
||||
fi
|
||||
# No `|| true` here. The validation above makes an error unreachable, but a swallowed error would
|
||||
# be indistinguishable from "no base recorded" — the exact confusion this block removes — so the
|
||||
# failure is handled explicitly rather than left to a fallback that reads as a benign result.
|
||||
if ! vdesc=$(printf '%s' "$vjson_base" \
|
||||
| jq -r '[.statuses[] | select(.context == "review-verdict/h10")] | first | .description // ""' \
|
||||
2>/dev/null); then
|
||||
decide ask "H10 merge gate: the commit statuses for PR #$pr head ${sha:0:7} could not be parsed to find the review verdict, so it could not be checked against the PR's base branch (ersatztv#632). Confirm the review covered the branch this PR currently targets ('$live_base') before merging."
|
||||
fi
|
||||
# The field is written by scripts/post-review-verdict.sh as a trailing `(base: <ref>)`. Its
|
||||
# ABSENCE is the one benign case: a verdict posted before ersatztv#632 could not have carried it,
|
||||
# and denying those would block every in-flight PR the day this lands. The window closes on its
|
||||
# own, since verdicts are per-head and short-lived.
|
||||
recorded_base=$(printf '%s' "$vdesc" | sed -n 's/.*(base: \(.*\))$/\1/p')
|
||||
if [ -n "$recorded_base" ] && [ "$recorded_base" != "$live_base" ]; then
|
||||
decide deny "H10 merge gate: BLOCKED — the review verdict on head ${sha:0:7} was formed while PR #$pr targeted '$recorded_base', but it now targets '$live_base'. Retargeting a base does not move the head sha, so the per-sha verdict status still reads green even though the effective diff has changed (ersatztv#632). Re-review against the new base and run: scripts/post-review-verdict.sh $pr MERGEABLE"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Linked issue: Gitea auto-close keywords in the PR body. ---
|
||||
issues=$(printf '%s' "$body" | grep -ioE '(close[sd]?|fix(e[sd])?|resolve[sd]?) +#[0-9]+' | grep -oE '[0-9]+' | sort -u || true)
|
||||
[ -n "$issues" ] || decide ask "H6 merge gate: PR #$pr has no linked issue (no 'fixes #N' / 'closes #N' in its body), so there is no Done-when checklist to derive consent from. Confirm the work is complete + reviewed, then approve."
|
||||
@@ -241,75 +113,11 @@ done
|
||||
# --- (a) CI combined status must be green (unless deferring to Gitea's own check-gate). ---
|
||||
if [ "$mwcs" != "true" ]; then
|
||||
[ -n "$sha" ] || decide ask "H6 merge gate: could not resolve PR #$pr head sha to check CI. Verify CI is green before merging."
|
||||
cistatus=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100")
|
||||
state=$(printf '%s' "$cistatus" | jq -r '.state // ""' 2>/dev/null || true)
|
||||
state=$(gq "repos/$owner/$repo/commits/$sha/status" | jq -r '.state // ""' 2>/dev/null || true)
|
||||
case "$state" in
|
||||
success) : ;;
|
||||
"") decide ask "H6 merge gate: could not read CI status for PR #$pr ($sha). Verify CI is green before merging." ;;
|
||||
*)
|
||||
# `review-verdict/h10` is itself one of the contexts folded into the COMBINED state, so a PR
|
||||
# awaiting its verdict reports combined 'pending' and would otherwise be reported as a CI
|
||||
# problem — sending the reader to build logs when the missing thing is the review. Name the
|
||||
# real blocker when the verdict is the only thing outstanding.
|
||||
#
|
||||
# "Not green" is anything that is not `success`, NOT just pending/failure: Gitea also has
|
||||
# `error` (and `warning`), and omitting those would let an errored build hide behind the
|
||||
# verdict and produce the flatly false claim "every CI check is green". `skipped` IS treated
|
||||
# as green — the image-push job skips on every PR (ersatztv#593: a skipped context is not red).
|
||||
nongreen=$(printf '%s' "$cistatus" \
|
||||
| jq -r '[.statuses[]? | select(.status != "success" and .status != "skipped")]
|
||||
| map("\(.context)=\(.status)") | join(", ")' 2>/dev/null || true)
|
||||
# The verdict's OWN state decides the wording: absent/pending means nobody has reviewed this
|
||||
# head, while failure/error means someone reviewed it and said no. Telling a reviewer to "post
|
||||
# a verdict" when they already posted a BLOCKED one would be actively misleading.
|
||||
vonly=$(printf '%s' "$cistatus" \
|
||||
| jq -r '[.statuses[]? | select(.status != "success" and .status != "skipped")]
|
||||
| if (length == 1 and .[0].context == "review-verdict/h10") then .[0].status else "" end' 2>/dev/null || true)
|
||||
case "$vonly" in
|
||||
pending)
|
||||
decide deny "H6/H10 merge gate: BLOCKED — every CI check on PR #$pr is green; the only outstanding context is 'review-verdict/h10' on head ${sha:0:7}, i.e. this head has no review verdict yet. Review it and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
|
||||
failure|error)
|
||||
decide deny "H6/H10 merge gate: BLOCKED — every CI check on PR #$pr is green, but 'review-verdict/h10' is '$vonly' on head ${sha:0:7}: this head was reviewed and REJECTED. Resolve the findings, then run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
|
||||
esac
|
||||
decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success' (not green: ${nongreen:-unknown}). Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging."
|
||||
;;
|
||||
esac
|
||||
else
|
||||
# --- SCHEDULED auto-merge: everything this hook proves is a SNAPSHOT (ersatztv#622). ----------
|
||||
# With merge_when_checks_succeed, Gitea performs the merge later, against whatever head is green
|
||||
# at THAT moment — but (b) and (c) below are evaluated against the head that exists right now.
|
||||
# Any commit pushed in between would merge with no verdict covering it. Demonstrated as a
|
||||
# controlled A/B (#622): with a slow CI check pending so Gitea waits, an unreviewed commit pushed
|
||||
# after scheduling MERGED without the required verdict context and was REFUSED with it.
|
||||
#
|
||||
# The durable fix is server-side and lives outside this hook: `review-verdict/h10` is a REQUIRED
|
||||
# status check on `main`, and a commit status belongs to exactly ONE sha, so a later commit cannot
|
||||
# inherit it and Gitea's own gate refuses to merge until that head is re-reviewed.
|
||||
#
|
||||
# What we add HERE is the matching precondition at SCHEDULING time: refuse to arm an auto-merge
|
||||
# unless the sha-bound status already exists on this head. Checking the comment alone (condition
|
||||
# (c) below) is not enough for this path — the comment is what a human reads, the status is what
|
||||
# the server enforces, and only the latter survives a new push. Deny rather than ask: the remedy
|
||||
# is a single documented command, so there is nothing here for a human to adjudicate.
|
||||
[ -n "$sha" ] || decide ask "H6 merge gate: could not resolve PR #$pr head sha to check the review-verdict status. Verify the review covered the latest commit before scheduling an auto-merge."
|
||||
# Read the COMBINED endpoint, not `/statuses/{sha}`: the latter returns one row per status POST
|
||||
# rather than per context and pages at 50, so a head with a few CI reruns can push the verdict off
|
||||
# the first page and read as absent — a confusing false deny. The combined endpoint returns
|
||||
# latest-per-context, which is exactly the question being asked.
|
||||
vjson=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100")
|
||||
# Same portability point as the file-pagination guard above: do not let jq's empty-input exit
|
||||
# status decide this. Here the fallthrough happens to land on `vstate=""` -> deny (fail-CLOSED,
|
||||
# so this was never a hole), but it would have surfaced the wrong message — a "BLOCKED, no
|
||||
# verdict" deny instead of the "could not read the status" ask this branch exists to give.
|
||||
if [ -z "${vjson//[[:space:]]/}" ] || ! printf '%s' "$vjson" | jq -e '.statuses | type == "array"' >/dev/null 2>&1; then
|
||||
decide ask "H6/H10 merge gate: could not read the 'review-verdict/h10' status for PR #$pr head ${sha:0:7} (Gitea unreachable or an unexpected response). Confirm the current head is reviewed before scheduling an auto-merge."
|
||||
fi
|
||||
vstate=$(printf '%s' "$vjson" | jq -r '[.statuses[] | select(.context == "review-verdict/h10")] | first | .status // ""')
|
||||
case "$vstate" in
|
||||
success) : ;;
|
||||
"") decide deny "H6/H10 merge gate: BLOCKED — PR #$pr has no 'review-verdict/h10' commit status on head ${sha:0:7}, so scheduling an auto-merge would freeze consent at a head Gitea may not be the one to merge (ersatztv#622). Review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
|
||||
pending) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is still pending on PR #$pr head ${sha:0:7} (no verdict posted for this commit yet). Review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
|
||||
*) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is '$vstate' on PR #$pr head ${sha:0:7}. Resolve the findings, then run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
|
||||
*) decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success'. Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging." ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
@@ -324,53 +132,52 @@ comments=$(gq "repos/$owner/$repo/issues/$pr/comments?limit=100")
|
||||
if [ -z "$comments" ]; then
|
||||
decide ask "H10 merge gate: could not fetch PR #$pr comments to verify a head-referencing review verdict ($short). Confirm the adversarial/Codex review covered the latest commit before merging."
|
||||
fi
|
||||
# Classification is delegated to `scripts/check-review-verdict.sh` — the single source of truth for
|
||||
# the H10 grammar, extracted in #629 so it could be TESTED. While it lived here it had none, and three
|
||||
# false-opens survived in it: a prefix-matched token (`MERGEABLE-LATER` graded positive), a verdict
|
||||
# inside a fenced code block (documentation showing the convention counted as a real verdict), and a
|
||||
# sha taken from the first `@<hex>` anywhere on the line (a markdown link could supply it). Every
|
||||
# decision the classifier makes is documented there; this file only maps a class onto a hook decision.
|
||||
verdict_script="${CLAUDE_PROJECT_DIR:-.}/scripts/check-review-verdict.sh"
|
||||
if [ ! -x "$verdict_script" ]; then
|
||||
decide ask "H10 merge gate: verdict classifier not found at $verdict_script, so the review state can't be derived. Confirm the review covered the latest commit before merging."
|
||||
# Verdict lines across all comment bodies: a real verdict line STARTS with the marker (after optional
|
||||
# leading whitespace). Anchoring to line-start is deliberate — it rejects a comment that merely QUOTES
|
||||
# the positive template mid-sentence (an instruction "please post: Review-verdict: MERGEABLE @ <sha>",
|
||||
# or the gate's own suggestion text echoed back), which would otherwise self-approve the merge.
|
||||
verdicts=$(printf '%s' "$comments" | jq -r '.[].body // empty' 2>/dev/null | grep -iE '^[[:space:]]*review-verdict:' || true)
|
||||
if [ -z "$verdicts" ]; then
|
||||
decide ask "H10 merge gate: no 'Review-verdict:' comment found on PR #$pr referencing head $short. Post the adversarial/Codex verdict (e.g. 'Review-verdict: MERGEABLE @ $short'), or confirm the review covered the latest commit and approve."
|
||||
fi
|
||||
# An input error (exit 2) is NOT a classification — fall through to a human rather than guessing.
|
||||
if ! class=$(printf '%s' "$comments" | "$verdict_script" --head "$sha" 2>/dev/null); then
|
||||
decide ask "H10 merge gate: could not classify the review verdicts on PR #$pr (malformed comments payload or unreadable head). Confirm the review covered the latest commit ($short) before merging."
|
||||
# Classify each verdict line by the sha it references (its "@ <sha>" field) and its verdict word.
|
||||
# A line references the CURRENT head iff head BEGINS WITH that sha token AND the token is >=7 chars
|
||||
# (git short-sha prefix semantics) — NOT a loose substring test: an older sha that merely contains
|
||||
# the head prefix, or the head prefix appearing in an unrelated URL on the line, must NOT count
|
||||
# (adversarial false-opens). The verdict token must sit right after the marker on the same line.
|
||||
head_pos=0; head_neg=0; stale=0
|
||||
while IFS= read -r line; do
|
||||
[ -n "$line" ] || continue
|
||||
# The sha the line references: the hex token in its "@ <sha>" field (>=7 chars), lowercased.
|
||||
ref=$(printf '%s' "$line" | grep -ioE '@[[:space:]]*[0-9a-f]{7,40}' | head -1 \
|
||||
| grep -oiE '[0-9a-f]{7,40}' | tr 'A-F' 'a-f' || true)
|
||||
is_pos=0
|
||||
# Positive iff the line's OWN leading verdict word (right after the line-start marker) is positive —
|
||||
# anchored so a second, later `review-verdict: mergeable` substring on a BLOCKED line can't flip it.
|
||||
if printf '%s' "$line" | grep -iqE '^[[:space:]]*review-verdict:[[:space:]]*(mergeable|approved|lgtm)'; then is_pos=1; fi
|
||||
[ -z "$ref" ] && continue # marker present but no @<sha> -> falls through to the final ask
|
||||
case "$sha" in
|
||||
"$ref"*) if [ "$is_pos" = 1 ]; then head_pos=1; else head_neg=1; fi ;;
|
||||
*) stale=1 ;;
|
||||
esac
|
||||
done <<VERDICTS
|
||||
$verdicts
|
||||
VERDICTS
|
||||
|
||||
# A negative verdict on head wins over a positive one (a later BLOCKED retracts an earlier MERGEABLE
|
||||
# on the SAME head; and if the head were fixed the sha would change, so this can't wrongly block).
|
||||
if [ "$head_neg" = 1 ]; then
|
||||
decide deny "H10 merge gate: BLOCKED — a review verdict for the current head ($short) is negative (BLOCKED/NOT-MERGEABLE). Resolve the findings and post a fresh 'Review-verdict: MERGEABLE @ $short' before merging PR #$pr."
|
||||
fi
|
||||
|
||||
case "$class" in
|
||||
negative)
|
||||
# A negative verdict on head wins over a positive one (a later BLOCKED retracts an earlier
|
||||
# MERGEABLE on the SAME head; if the head were fixed the sha would change, so this can't
|
||||
# wrongly block).
|
||||
decide deny "H10 merge gate: BLOCKED — a review verdict for the current head ($short) is negative (BLOCKED/NOT-MERGEABLE). Resolve the findings and post a fresh 'Review-verdict: MERGEABLE @ $short' before merging PR #$pr." ;;
|
||||
stale)
|
||||
decide deny "H10 merge gate: BLOCKED — a review-verdict comment references an older commit, not the current head ($short). The latest commit(s) are unreviewed (ersatztv#242: re-review the fix commit, not just the initial diff). Re-review the head and post 'Review-verdict: MERGEABLE @ $short'." ;;
|
||||
unknown)
|
||||
decide ask "H10 merge gate: a 'Review-verdict:' comment on PR #$pr uses an unrecognized verdict token (not MERGEABLE/APPROVED/LGTM/BLOCKED/NOT-MERGEABLE). It is deliberately NOT read as approval. Post a verdict using the documented vocabulary — e.g. 'Review-verdict: MERGEABLE @ $short'." ;;
|
||||
no-sha)
|
||||
# Marker(s) exist but reference no sha at all -> ask (don't mislabel as a stale older-commit review).
|
||||
decide ask "H10 merge gate: a 'Review-verdict:' comment on PR #$pr references no commit sha in its own '@ <sha>' field. Post one referencing the current head ($short) — e.g. 'Review-verdict: MERGEABLE @ $short' — or confirm the review covered the latest commit and approve." ;;
|
||||
absent)
|
||||
decide ask "H10 merge gate: no 'Review-verdict:' comment found on PR #$pr referencing head $short. Post the adversarial/Codex verdict (e.g. 'Review-verdict: MERGEABLE @ $short'), or confirm the review covered the latest commit and approve." ;;
|
||||
positive) : ;;
|
||||
*)
|
||||
decide ask "H10 merge gate: unrecognized verdict classification '$class' for PR #$pr. Confirm the review covered the latest commit ($short) before merging." ;;
|
||||
esac
|
||||
|
||||
if [ "$class" = "positive" ]; then
|
||||
# (a) CI + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant.
|
||||
# The reason string must not claim more than was actually checked: on the merge_when_checks_succeed
|
||||
# path this hook never read the CI status at all (it is delegated to Gitea), so saying "CI green"
|
||||
# there was a plain falsehood in the one message a human reads to decide whether to trust the gate.
|
||||
if [ "$mwcs" = "true" ]; then
|
||||
decide grant "H6/H10 merge gate: satisfied — all Done-when boxes ticked, and both a positive Review-verdict comment and the 'review-verdict/h10' status cover the current head ($short). CI is gated by Gitea (merge_when_checks_succeed), and because the verdict status is bound to this sha, a commit pushed before Gitea merges will clear it and block the merge (ersatztv#622). Auto-granted."
|
||||
fi
|
||||
if [ "$head_pos" = 1 ]; then
|
||||
# (a) CI green + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant.
|
||||
decide grant "H6/H10 merge gate: satisfied — CI green, all Done-when boxes ticked, and a positive Review-verdict references the current head ($short). Auto-granted (no separate confirmation needed)."
|
||||
fi
|
||||
if [ "$stale" = 1 ]; then
|
||||
decide deny "H10 merge gate: BLOCKED — a review-verdict comment references an older commit, not the current head ($short). The latest commit(s) are unreviewed (ersatztv#242: re-review the fix commit, not just the initial diff). Re-review the head and post 'Review-verdict: MERGEABLE @ $short'."
|
||||
fi
|
||||
# Marker(s) exist but reference no sha at all -> ask (don't mislabel as a stale older-commit review).
|
||||
decide ask "H10 merge gate: a 'Review-verdict:' comment on PR #$pr references no commit sha. Post one referencing the current head ($short) — e.g. 'Review-verdict: MERGEABLE @ $short' — or confirm the review covered the latest commit and approve."
|
||||
|
||||
# Unreachable: the `case` above exits on every class, and `positive` exits in the block above. Kept as
|
||||
# a fail-safe so a future class added to the classifier without a branch here cannot fall off the end
|
||||
# of the script (which would exit 0 = silent passthrough, the one outcome a gate must never produce).
|
||||
decide ask "H10 merge gate: verdict classification for PR #$pr produced no decision. Confirm the review covered the latest commit ($short) before merging."
|
||||
# All derivable and satisfied -> auto-grant (defensive: the head_pos branch above already exits here).
|
||||
decide grant "H6/H10 merge gate: satisfied — auto-granted."
|
||||
|
||||
@@ -1,125 +1,37 @@
|
||||
---
|
||||
name: ersatztv
|
||||
description: "ErsatzTV custom IPTV channel management — REST API, SQLite DB, Jellyfin integration, FFmpeg profiles. Use when creating or modifying IPTV channels, managing collections and schedules, building playouts, adding channel logos, scanning media libraries, troubleshooting channel issues, or resetting playouts. Also use for any questions about the ErsatzTV database schema (Channel, Collection, ProgramSchedule, Playout tables), M3U/XMLTV feeds, custom TV channel setup, or the channel creation checklist. IMPORTANT: the fork has a full versioned REST API at /api/v1 including write paths — prefer it over SQLite scripting, which is a recovery fallback only."
|
||||
description: ErsatzTV custom IPTV channel management — REST API, SQLite DB, Jellyfin integration, FFmpeg profiles. Use when managing custom TV channels.
|
||||
---
|
||||
|
||||
> **Canonical copy: `~/ersatztv/.claude/skills/ersatztv/SKILL.md`** (ersatztv owns this skill per that
|
||||
> repo's `CLAUDE.md` → Project Boundaries). `~/server-management/.claude/skills/ersatztv` is a symlink
|
||||
> to it. Edit it in the ersatztv repo; never fork a second copy (ersatztv#617).
|
||||
|
||||
# ErsatzTV Channel Management
|
||||
|
||||
Container: `ersatztv` | Port: `8409`
|
||||
Web UI: `https://ersatztv.tblindustries.be` (via bumblebee's `external-proxy` → `192.168.1.29:8409`) or `http://localhost:8409` on the host
|
||||
Host: **jazz** (`192.168.1.29`) since 2026-07-20 (#633) — moved off bumblebee together with Jellyfin. `dispatcharr` and `plex` stayed on bumblebee, so Dispatcharr now reaches ErsatzTV **by IP** (`http://192.168.1.29:8409`), not by Docker DNS name.
|
||||
Compose env: `ForwardedHeaders__KnownNetworks=192.168.1.99/32` (proxied traffic arrives SNAT'd from bumblebee's LAN address; wrong value breaks Authelia OIDC login only, plain HTTP still works)
|
||||
Host: **jazz (192.168.1.29)**. Prod container `ersatztv` port **8409**; test `ersatztv-test` port
|
||||
**8410** (tracks `:latest` via Komodo auto-update, daily 03:00 — a same-day validation needs the
|
||||
manual pull below).
|
||||
SQLite DB: `~/downloadswarm/ersatztv/ersatztv.sqlite3` (owned by root — use `sudo sqlite3`)
|
||||
Image: `192.168.1.95:3000/timothy/ersatztv:prod` (our fork; **floating** release tag — check `git tag -l 'v*' --sort=-v:refname | head -1` in `~/ersatztv` for the current release rather than trusting a version written here). Upstream `ghcr.io/ersatztv/ersatztv` was archived at v26.3.0 and is **not** what runs here.
|
||||
Release tags are `vYY.<release-seq>.<patch>` — year · sequential release-within-year · patch — **not** year.month.
|
||||
|
||||
## Test/Prod topology — fork CI images (#481)
|
||||
|
||||
We maintain an **ErsatzTV fork** (`~/ersatztv`); its Gitea Actions pipeline builds and pushes images to the
|
||||
private Gitea registry `192.168.1.95:3000/timothy/ersatztv` on every push to `main` (`:latest` + `:<short-sha>`)
|
||||
and, on a `v*` tag, additionally `:prod` + `:<version>`. jazz is `docker login`'d to that registry and has `192.168.1.95:3000` in `insecure-registries`.
|
||||
|
||||
| | Prod | Test |
|
||||
|---|---|---|
|
||||
| Container | `ersatztv` | `ersatztv-test` |
|
||||
| Host port | 8409 | 8410 |
|
||||
| Stack | Komodo **`jazz-media`**; source `docker/jazz/stacks/media-servers/compose.yaml` (stack name ≠ directory — `media-servers` is bumblebee's; Komodo stack names are globally unique) | Komodo `ersatztv`; source `docker/jazz/stacks/ersatztv/compose.yaml` |
|
||||
| Image | `192.168.1.95:3000/timothy/ersatztv:prod` (floating release tag) | `192.168.1.95:3000/timothy/ersatztv:latest` (fork CI) |
|
||||
| Config (host) | `~/downloadswarm/ersatztv/` → `/config` | `~/downloadswarm/ersatztv-test/` → `/config` (one-time prod snapshot, refresh on demand) |
|
||||
| Jellyfin/Dispatcharr tuner | connected (live lineup) | **NOT** wired downstream (avoids ghost channels) |
|
||||
| Media mounts | RO | same mounts, RO |
|
||||
| `/dev/dri` | yes (**VAAPI on Intel iHD**, jazz — see hw note) | yes (`/dev/dri` + `group_add: '992'`) |
|
||||
| Auto-update | **None** (`auto_update: false`) — promotion is a manual `DeployStack jazz-media`, with no 03:00 fallback | Komodo auto-update, daily 03:00 (tracks `:latest`) |
|
||||
| Env | `TZ`, restricted forwarded-header network, empty-by-default local-admin seed hook | `TZ`, `ETV_CONFIG_FOLDER=/config`, `ETV_TRANSCODE_FOLDER=/transcode`, `ETV_DISABLE_VULKAN=1` |
|
||||
|
||||
**Watchtower is retired.** Test auto-updates via Komodo; **prod does not** — `auto_update: false`, so
|
||||
promoting a release is always a manual `DeployStack jazz-media`. Prod's stack has a
|
||||
fail-closed pre-deploy hook: a changed compose block or `:prod` digest triggers a PBS-backed snapshot and then a
|
||||
migration rehearsal against a throwaway copy of that snapshot before container recreation (#585/#589).
|
||||
|
||||
**Refresh test snapshot from prod** (zero prod downtime — WAL online backup):
|
||||
```bash
|
||||
ssh timothy@192.168.1.29
|
||||
docker stop ersatztv-test
|
||||
sudo sqlite3 ~/downloadswarm/ersatztv/ersatztv.sqlite3 ".backup '/home/timothy/downloadswarm/ersatztv-test/ersatztv.sqlite3'"
|
||||
sudo rsync -a --exclude='ersatztv.sqlite3*' --exclude='logs/' ~/downloadswarm/ersatztv/ ~/downloadswarm/ersatztv-test/
|
||||
docker start ersatztv-test
|
||||
```
|
||||
|
||||
**Prod cutover to the fork** — ✅ DONE 2026-06-27 (#481). Prod runs `…/timothy/ersatztv:prod` (v26.3.1);
|
||||
validated `:prod` on test first, then `etv-prod-deploy.sh` backed up + cut over (43 channels, healthy,
|
||||
clean migrations). Downstream (Dispatcharr M3U acct 3 + EPG src 9) is name-based, so the container IP
|
||||
change was transparent. Prod stays a **manual** gate (no Watchtower label) and still lives in the
|
||||
`media-servers` stack (the optional move into the `ersatztv` stack was not done).
|
||||
|
||||
**Future prod releases** (push `v*` tag in `~/ersatztv` → CI builds `:prod`/`:<version>`): scan the immutable
|
||||
`:<version>` image on jazz first, then execute Komodo `DeployStack` for `jazz-media`. The pre-deploy hook
|
||||
backs up and runs the migration-on-prod-copy smoke before recreation. **There is no auto-update fallback for
|
||||
prod** — if you don't `DeployStack`, nothing ships. Note the stack is named **`jazz-media`** even though the
|
||||
compose *project* is still `media-servers`; a dead `media-servers` stack lingers on bumblebee and deploying it
|
||||
fails silently. Roll back with the immutable prior image plus the pre-deploy DB snapshot; migrations are
|
||||
forward-only. See the `komodo` skill and `docs/Docker/ErsatzTV.md` for the current procedure.
|
||||
|
||||
## Backup & deploy safety (#482)
|
||||
|
||||
Every prod deploy runs forward-only EF Core migrations against the live 285 MB SQLite DB — a bad one
|
||||
can't be undone by re-deploying the old image, so the **only** rollback is restoring a pre-deploy DB
|
||||
snapshot. Three scripts in `~/scripts/` (source of truth: `scripts/` in this repo) handle
|
||||
them. **⚠️ These were installed on bumblebee, where ErsatzTV no longer runs (#633) — verify they exist on
|
||||
jazz and that the Komodo `pre_deploy` hook is set on the `jazz-media` stack before relying on
|
||||
"no backup, no deploy". Until confirmed, take a manual `etv-backup.sh` snapshot before every prod deploy.**
|
||||
this. **Run as root** (DB + PBS creds are root-owned) except the deploy wrapper (run as `timothy`).
|
||||
|
||||
| Script | Run as | What it does |
|
||||
|---|---|---|
|
||||
| `etv-backup.sh [--target prod\|test] [--no-offbox]` | root (sudo) | Online `sqlite3 .backup` (zero-downtime) + `integrity_check`, provenance `manifest.txt` (image ref/digest + last `__EFMigrationsHistory` id), bundles `data-protection/` + `*-secrets.json`. Local **keep-last-5** under `~/downloadswarm/ersatztv-backups/<UTC-ts>/`; prod also pushes off-box to PBS. Prints the snapshot dir on stdout. |
|
||||
| `etv-prod-deploy.sh` | **timothy** (needs private-registry creds; sudo's for the backup) | Backup (abort deploy if it fails) → `compose pull` + `up -d ersatztv` → health + M3U gate → prints a copy-paste rollback block on trouble. |
|
||||
| `etv-restore.sh --target prod\|test --from <snapshot-dir>` | root (sudo) | Verifies snapshot → stop → saves current DB aside (`*.pre-restore-<ts>`) → swaps DB, drops stale `-wal/-shm`, restores `data-protection` → start → health/channel check. |
|
||||
|
||||
- **Off-box:** prod backups go to PBS `data-local` (.68) as backup-id **`ersatztv-predeploy`** (own
|
||||
group, dedups against the nightly host backup), via the existing `/root/.proxmox-backup-client.env`.
|
||||
- **Retention:** local keep-last-5 (instant rollback); PBS via the datastore-wide `data-local-prune`
|
||||
job (7 daily / 4 weekly / 6 monthly), no separate prune job needed.
|
||||
- **Restore from PBS** instead of a local dir:
|
||||
```bash
|
||||
source /root/.proxmox-backup-client.env
|
||||
proxmox-backup-client restore ersatztv-predeploy/<snapshot> etv.pxar <outdir>
|
||||
sudo ~/scripts/etv-restore.sh --target prod --from <outdir>
|
||||
```
|
||||
- `docker exec` always curls the container-internal port **8409** (even for test, whose host port is
|
||||
8410). `etv-restore.sh` leaves a `*.pre-restore-<ts>` safety copy in `/config` — delete once happy.
|
||||
- Validated 2026-06-27: first prod backup → PBS group created; full restore round-trip on `ersatztv-test`
|
||||
returned 43 channels. Design: `plans/2026-06-27-ersatztv-backup-before-deploy-design.md`.
|
||||
Image: **our fork**, `192.168.1.95:3000/timothy/ersatztv` (`:prod` / `:latest`). Upstream
|
||||
`ghcr.io/ersatztv/ersatztv` was archived at v26.3.0 and is NOT what runs here.
|
||||
|
||||
## Architecture
|
||||
|
||||
**ErsatzTV is for channel creation only.** Consumers (Jellyfin, Kodi) never connect to ErsatzTV directly — everything goes through Dispatcharr as the single aggregation point. Pipeline: ErsatzTV → Dispatcharr → Jellyfin/Kodi.
|
||||
**This section described upstream v26.3.0 and was wrong for the fork — corrected 2026-07-21.**
|
||||
|
||||
ErsatzTV uses **MediatR + the ChicoryTV React SPA**. The legacy Blazor UI was removed in v26.7.0 (#91
|
||||
phase b) — the SPA at `/app` is the **only** UI, and legacy routes 302 there. The versioned `/api/v1`
|
||||
surface provides full CRUD — channels, collections, schedules, playouts and media sources; browser calls
|
||||
use a local-admin/OIDC session cookie plus `X-CSRF` on mutations, and machine clients use `X-Api-Key`.
|
||||
**Do not hand-edit SQLite for something the API can do** — direct SQLite writes are a recovery fallback,
|
||||
not the normal management path, and the DB recipes below survive only for gaps with no endpoint.
|
||||
- The **Blazor UI is gone** (#91 phase b). The only UI is the ChicoryTV React SPA at `/app`; legacy
|
||||
routes 302 there.
|
||||
- There **is** a full versioned REST API under **`/api/v1`**, write paths included — channels,
|
||||
collections, schedules, playouts and media sources have CRUD. **Do not hand-edit SQLite for
|
||||
something the API can do.** The DB-scripting recipes below survive only for gaps with no endpoint.
|
||||
- Controllers stay thin and delegate to MediatR handlers; the SPA talks to `/api/v1` only.
|
||||
- Authoritative endpoint list: `docs/endpoint-index.md` (generated) + `docs/api-conventions.md`.
|
||||
Prefer those over any list in this file — a hand-maintained copy drifts.
|
||||
|
||||
Controllers stay thin and delegate to MediatR handlers. **Authoritative endpoint list:
|
||||
`docs/endpoint-index.md` (generated) + `docs/api-conventions.md` in the ersatztv repo — prefer those
|
||||
over any list in this file**, which is hand-maintained and drifts.
|
||||
## REST API access (auth-gated — read before curling)
|
||||
|
||||
## REST API
|
||||
Calls need **`X-Api-Key`** (machine clients) or a browser session. An unauthenticated call returns a
|
||||
401 JSON body that is easy to mistake for real data — see the silent-401 trap in Gotchas.
|
||||
|
||||
```bash
|
||||
# Via docker exec (api.key is readable inside the container)
|
||||
docker exec ersatztv curl -s -H "X-Api-Key: $(docker exec ersatztv cat /config/api.key)" \
|
||||
http://localhost:8409/api/v1/ENDPOINT
|
||||
```
|
||||
|
||||
From the **host**, the key file is root-owned `0600`, so an unsudo'd `cat` fails *silently* and sends an
|
||||
empty header. Read it with `sudo`, inline, so the value is never printed:
|
||||
The key file is **root-owned `0600`**, so `cat` as `timothy` fails *silently* and yields an empty
|
||||
header. Read it with `sudo`, inline, so the value is never printed:
|
||||
|
||||
```bash
|
||||
# prod (8409); test is identical with .../ersatztv-test/api.key and port 8410
|
||||
@@ -127,23 +39,12 @@ ssh timothy@192.168.1.29 'K=$(sudo -n cat /home/timothy/downloadswarm/ersatztv/a
|
||||
curl -s -H "X-Api-Key: $K" http://localhost:8409/api/v1/channels'
|
||||
```
|
||||
|
||||
### Paging — 0-based (ersatztv#616, `api.paging-zero-based`)
|
||||
|
||||
- **`pageNum` is 0-based** across the whole `/api/v1` surface and every wrapper of it (MCP tools, SPA
|
||||
hooks, docs). Starting at 1 silently skips a page and returns a short set **with no error**.
|
||||
- **`pageSize` is clamped per-endpoint** — 100 typical, 200 auto-tune members, 1000 search/all-items —
|
||||
and the offset derives from the *effective* (clamped) size, not the requested one. Page to
|
||||
completeness against `totalCount`; never conclude "that's all of them" from a single page.
|
||||
- **`POST /api/v1/channels/{id}/playout/reset` takes a CHANNEL id, not the playout id.** The id spaces
|
||||
overlap numerically, so passing a playout row's `Id` returns a plausible 202 against a *different*
|
||||
channel. Playout rows carry `channelId` — use that.
|
||||
|
||||
Settings live under `/api/v1/settings/*` — `settings/ffmpeg` (`workAheadSegmenterLimit`,
|
||||
`qsvExtraHardwareFrames`) and `settings/logging` (`streamingMinimumLogLevel`). Note the order: it is
|
||||
`settings/ffmpeg`, **not** `ffmpeg/settings`.
|
||||
|
||||
Refresh test to the newest `:latest` without waiting for the 03:00 auto-update — scope it to the
|
||||
service, since a bare `up -d` would recreate everything else in the compose project:
|
||||
Refresh test to the newest `:latest` without waiting for 03:00 — scope it to the service, since a
|
||||
bare `up -d` would recreate everything else in the compose project:
|
||||
|
||||
```bash
|
||||
D=/etc/komodo/stacks/ersatztv/docker/jazz/stacks/ersatztv
|
||||
@@ -151,18 +52,26 @@ docker compose -f $D/compose.yaml pull ersatztv-test
|
||||
docker compose -f $D/compose.yaml up -d --no-deps ersatztv-test
|
||||
```
|
||||
|
||||
The unversioned `/api/*` endpoints below predate the `/api/v1` surface — verify one against
|
||||
`docs/endpoint-index.md` before relying on it.
|
||||
|
||||
```bash
|
||||
# Via docker exec
|
||||
docker exec ersatztv curl -s http://localhost:8409/api/ENDPOINT
|
||||
```
|
||||
|
||||
### Read Endpoints (GET)
|
||||
```
|
||||
/api/v1/channels # List channels
|
||||
/api/v1/collections # List collections
|
||||
/api/v1/schedules # List schedules
|
||||
/api/v1/playouts # List playouts
|
||||
/api/v1/media-items # List media items
|
||||
/api/v1/search # Search items
|
||||
/api/v1/ffmpeg/profiles # FFmpeg profiles
|
||||
/api/v1/settings/ffmpeg # Global FFmpeg settings — workAheadSegmenterLimit,
|
||||
# initialSegmentCount, hlsSegmenterIdleTimeout
|
||||
/api/v1/watermarks # Watermarks
|
||||
/api/channels # List channels
|
||||
/api/collections # List collections
|
||||
/api/schedules # List schedules
|
||||
/api/playouts # List playouts
|
||||
/api/shows # List shows
|
||||
/api/movies # List movies
|
||||
/api/artists # List artists
|
||||
/api/search # Search items
|
||||
/api/ffmpeg/profiles # FFmpeg profiles
|
||||
/api/watermarks # Watermarks
|
||||
/iptv/channels.m3u # M3U playlist (for Jellyfin)
|
||||
/iptv/xmltv.xml # XMLTV guide data
|
||||
```
|
||||
@@ -170,14 +79,14 @@ docker compose -f $D/compose.yaml up -d --no-deps ersatztv-test
|
||||
### Mutation Endpoints (POST)
|
||||
```bash
|
||||
# Library scan
|
||||
POST /api/v1/libraries/{id}/scan
|
||||
POST /api/libraries/{id}/scan
|
||||
|
||||
# Scan single show
|
||||
POST /api/v1/libraries/{id}/scan-show \
|
||||
POST /api/libraries/{id}/scan-show \
|
||||
-H "Content-Type: application/json" -d '{"ShowTitle":"Name","DeepScan":false}'
|
||||
|
||||
# Reset channel playout (rebuilds schedule)
|
||||
POST /api/v1/channels/{channelId}/playout/reset
|
||||
POST /api/channels/{channelNumber}/playout/reset
|
||||
```
|
||||
|
||||
## SQLite DB Operations
|
||||
@@ -197,56 +106,25 @@ docker start ersatztv
|
||||
-- List channels
|
||||
SELECT Id, Number, Name FROM Channel ORDER BY CAST(Number AS INTEGER);
|
||||
|
||||
-- List collections with item counts (CollectionItem has no Id column — use rowid)
|
||||
SELECT c.Id, c.Name, COUNT(ci.rowid) as items
|
||||
FROM Collection c LEFT JOIN CollectionItem ci ON ci.CollectionId = c.Id GROUP BY c.Id;
|
||||
-- List collections with item counts
|
||||
SELECT c.Id, c.Name, COUNT(ci.Id) as items FROM Collection c LEFT JOIN CollectionItem ci ON ci.CollectionId = c.Id GROUP BY c.Id;
|
||||
|
||||
-- List schedules
|
||||
SELECT Id, Name FROM ProgramSchedule;
|
||||
|
||||
-- Playout with item count (check if playout is actually built)
|
||||
SELECT p.Id, c.Number, c.Name, ps.Name as Schedule, p.ScheduleKind, COUNT(pi.Id) as items
|
||||
FROM Playout p JOIN Channel c ON p.ChannelId = c.Id
|
||||
LEFT JOIN ProgramSchedule ps ON p.ProgramScheduleId = ps.Id
|
||||
LEFT JOIN PlayoutItem pi ON pi.PlayoutId = p.Id
|
||||
GROUP BY p.Id ORDER BY CAST(c.Number AS INTEGER);
|
||||
-- Playout (channel-schedule links)
|
||||
SELECT p.Id, c.Number, c.Name, ps.Name as Schedule FROM Playout p JOIN Channel c ON p.ChannelId = c.Id LEFT JOIN ProgramSchedule ps ON p.ProgramScheduleId = ps.Id;
|
||||
|
||||
-- Media counts
|
||||
SELECT 'Shows' as type, COUNT(*) FROM Show UNION ALL SELECT 'Movies', COUNT(*) FROM Movie UNION ALL SELECT 'Episodes', COUNT(*) FROM Episode UNION ALL SELECT 'MusicVideos', COUNT(*) FROM MusicVideo;
|
||||
|
||||
-- Collection content (via file paths — Movie table has only Id, metadata is via MediaVersion→MediaFile)
|
||||
SELECT ci.MediaItemId, mf.Path
|
||||
FROM CollectionItem ci
|
||||
JOIN MediaVersion mv ON mv.MovieId = ci.MediaItemId
|
||||
JOIN MediaFile mf ON mf.MediaVersionId = mv.Id
|
||||
WHERE ci.CollectionId = <id>
|
||||
ORDER BY mf.Path;
|
||||
|
||||
-- Jellyfin source
|
||||
SELECT jms.Id, jc.Address, jms.ServerName FROM JellyfinMediaSource jms JOIN JellyfinConnection jc ON jc.JellyfinMediaSourceId = jms.Id;
|
||||
|
||||
-- Library sync status
|
||||
SELECT l.Id, l.Name, l.MediaKind, jl.ShouldSyncItems FROM Library l JOIN JellyfinLibrary jl ON jl.Id = l.Id;
|
||||
|
||||
-- Music library folder breakdown
|
||||
SELECT DISTINCT substr(mf.Path, 1, instr(substr(mf.Path, 13), '/') + 12) as folder, COUNT(*) as items
|
||||
FROM MediaFile mf WHERE mf.Path LIKE '/data/music/%' GROUP BY folder ORDER BY folder;
|
||||
```
|
||||
|
||||
### Table Schema Notes
|
||||
|
||||
**CollectionItem**: Has `CollectionId` + `MediaItemId` columns only (no `Id` column — use `rowid` for counting).
|
||||
|
||||
**MediaVersion**: Links to content via `MovieId`, `EpisodeId`, `MusicVideoId` columns (NOT a generic `MediaItemId`). Use `mv.MovieId = ci.MediaItemId` for movie/music video collections.
|
||||
|
||||
**Movie / Show / Episode / MusicVideo**: Inheritance from `MediaItem`. These tables have only an `Id` column (PK = MediaItem.Id). Titles and metadata are in separate `*Metadata` tables.
|
||||
|
||||
**Artwork**: Channel logos use `ArtworkKind=2` with `ChannelId` set. `Path` column is SHA256 hash (uppercase) of the image file. Files stored at `/config/cache/artwork/logos/{Path[0:2]}/{Path}`.
|
||||
|
||||
**ChannelWatermark**: Global watermark config (Id=1, "Channel Bug"). All channels share this via `Channel.WatermarkId=1`. This is the burn-in watermark overlay, NOT the channel logo.
|
||||
|
||||
**ProgramScheduleItem subtype tables**: `ProgramScheduleOneItem`, `ProgramScheduleDurationItem`, `ProgramScheduleFloodItem`, `ProgramScheduleMultipleItem`. MUST insert into the matching subtype table (usually `ProgramScheduleOneItem`).
|
||||
|
||||
### Channel Setup Workflow (DB)
|
||||
|
||||
**Show-specific channel** (single TV show, shuffled):
|
||||
@@ -258,65 +136,26 @@ VALUES (<id>, 0, 0, '<name>', 1, 0, 1);
|
||||
INSERT INTO ProgramScheduleItem (Id, CollectionType, FillWithGroupMode, GuideMode, "Index", MarathonGroupBy, MarathonShuffleGroups, MarathonShuffleItems, MediaItemId, PlaybackOrder, ProgramScheduleId)
|
||||
VALUES (<id>, 1, 0, 0, 0, 0, 0, 0, <show_id>, 3, <schedule_id>);
|
||||
INSERT INTO ProgramScheduleOneItem (Id) VALUES (<item_id>);
|
||||
-- 3. Channel (StreamingMode=4 = HLS Segmenter — ETV default; works fine through Dispatcharr. See Gotchas → Streaming mode.)
|
||||
-- 3. Channel
|
||||
INSERT INTO Channel (Id, Categories, FFmpegProfileId, FallbackFillerId, "Group", IdleBehavior, IsEnabled, MirrorSourceChannelId, MusicVideoCreditsMode, MusicVideoCreditsTemplate, Name, Number, PlayoutMode, PlayoutOffset, PlayoutSource, PreferredAudioLanguageCode, PreferredAudioTitle, PreferredSubtitleLanguageCode, ShowInEpg, SongVideoMode, SortNumber, StreamSelector, StreamSelectorMode, StreamingMode, SubtitleMode, TranscodeMode, UniqueId, WatermarkId)
|
||||
VALUES (<id>, '', 1, NULL, '<category>', 0, 1, NULL, 0, NULL, '<name>', '<number>', 0, NULL, 0, NULL, NULL, 'eng', 1, 0, <number>.0, NULL, 0, 4, 2, 0, lower(hex(randomblob(4)))||'-'||lower(hex(randomblob(2)))||'-4'||substr(lower(hex(randomblob(2))),2)||'-'||lower(hex(randomblob(2)))||'-'||lower(hex(randomblob(6))), 1);
|
||||
-- 4. Playout (ScheduleKind=1 required — 0 is broken)
|
||||
-- 4. Playout
|
||||
INSERT INTO Playout (Id, ChannelId, ProgramScheduleId, ScheduleKind, Seed)
|
||||
VALUES (<id>, <channel_id>, <schedule_id>, 1, abs(random()) % 1000000);
|
||||
VALUES (<id>, <channel_id>, <schedule_id>, 0, abs(random()) % 1000000);
|
||||
```
|
||||
|
||||
**Collection-based channel** (multiple movies/videos, shuffled):
|
||||
**Collection-based channel** (multiple shows, shuffled):
|
||||
```sql
|
||||
-- 1. Collection + items (MediaItemId = Movie.Id from MediaVersion→MediaFile lookup)
|
||||
-- 1. Collection + items (MediaItemId = Show.Id)
|
||||
INSERT INTO Collection (Id, Name, UseCustomPlaybackOrder) VALUES (<id>, '<name>', 0);
|
||||
INSERT INTO CollectionItem (CollectionId, MediaItemId) VALUES (<coll_id>, <movie_id>);
|
||||
-- To bulk-add items from a folder:
|
||||
INSERT INTO CollectionItem (CollectionId, MediaItemId)
|
||||
SELECT <coll_id>, mv.MovieId FROM MediaFile mf
|
||||
JOIN MediaVersion mv ON mf.MediaVersionId = mv.Id
|
||||
WHERE mf.Path LIKE '/data/music/<folder>/%'
|
||||
AND mv.MovieId NOT IN (SELECT MediaItemId FROM CollectionItem WHERE CollectionId = <coll_id>);
|
||||
|
||||
-- 2. Schedule + item (CollectionType=0, PlaybackOrder=3)
|
||||
INSERT INTO ProgramSchedule (Id, FixedStartTimeBehavior, KeepMultiPartEpisodesTogether, Name, RandomStartPoint, ShuffleScheduleItems, TreatCollectionsAsShows)
|
||||
VALUES (<id>, 0, 0, '<name>', 1, 1, 0);
|
||||
INSERT INTO ProgramScheduleItem (Id, CollectionId, CollectionType, FillWithGroupMode, GuideMode, "Index", MarathonGroupBy, MarathonShuffleGroups, MarathonShuffleItems, PlaybackOrder, ProgramScheduleId)
|
||||
VALUES (<id>, <coll_id>, 0, 0, 0, 0, 0, 0, 0, 3, <schedule_id>);
|
||||
INSERT INTO ProgramScheduleOneItem (Id) VALUES (<item_id>);
|
||||
-- 3-4. Channel + Playout same as show-specific (ScheduleKind=1)
|
||||
INSERT INTO CollectionItem (CollectionId, MediaItemId) VALUES (<coll_id>, <show_id>);
|
||||
-- 2. Schedule (same as above but CollectionType=0, CollectionId set instead of MediaItemId)
|
||||
INSERT INTO ProgramScheduleItem (Id, CollectionId, CollectionType, ..., PlaybackOrder, ProgramScheduleId)
|
||||
VALUES (<id>, <coll_id>, 0, ..., 3, <schedule_id>);
|
||||
-- 3-4. Channel + Playout same as show-specific
|
||||
```
|
||||
|
||||
After creating: `POST /api/v1/channels/{id}/playout/reset`
|
||||
|
||||
### Channel Logo Workflow
|
||||
|
||||
Logos are stored as `Artwork` rows (ArtworkKind=2) with images in the cache directory.
|
||||
|
||||
```bash
|
||||
# 1. Create logo PNG (transparent background, white text)
|
||||
magick -size 512x180 xc:transparent -font "DejaVu-Sans-Bold" -pointsize 48 \
|
||||
-fill white -stroke black -strokewidth 2 -gravity center \
|
||||
-annotate +0+0 "CHANNEL NAME" PNG32:/tmp/logo.png
|
||||
|
||||
# 2. Calculate SHA256 and place in ErsatzTV cache
|
||||
HASH=$(sha256sum /tmp/logo.png | cut -d' ' -f1 | tr 'a-f' 'A-F')
|
||||
LOGO_DIR=~/downloadswarm/ersatztv/cache/artwork/logos
|
||||
sudo mkdir -p "$LOGO_DIR/${HASH:0:2}"
|
||||
sudo cp /tmp/logo.png "$LOGO_DIR/${HASH:0:2}/$HASH"
|
||||
|
||||
# 3. Insert Artwork row (stop container first for writes)
|
||||
docker stop ersatztv
|
||||
sudo sqlite3 ~/downloadswarm/ersatztv/ersatztv.sqlite3 "
|
||||
INSERT INTO Artwork (ArtworkKind, ChannelId, DateAdded, DateUpdated, Path)
|
||||
VALUES (2, <channel_db_id>, datetime('now'), datetime('now'), '$HASH');
|
||||
"
|
||||
docker start ersatztv
|
||||
|
||||
# 4. After ETV restarts, push logos to Jellyfin (see docs/Docker/ErsatzTV.md for fix_logos.py)
|
||||
```
|
||||
|
||||
**Important**: Channel DB Id (from Channel table) is NOT the channel number. E.g., channel #407 might have DB Id 43.
|
||||
After creating: `POST /api/channels/{number}/playout/reset`
|
||||
|
||||
## Volume Mounts (matches Jellyfin)
|
||||
|
||||
@@ -331,133 +170,46 @@ docker start ersatztv
|
||||
|
||||
## FFmpeg & Hardware
|
||||
|
||||
- **QSV encode + VA-API decode on Intel (iHD)** — ErsatzTV runs on **jazz** (i7-10700K, Intel iGPU) since #633. The single `FFmpegProfile` row (`Id = 1`, referenced by all 43 channels) has `HardwareAcceleration = 1` (**Qsv**), `QsvPreferNativeDecoder = 1` (ON), `QsvExtraHardwareFrames = 64`, `VaapiDevice = /dev/dri/renderD128`. Verified live 2026-07-26. The profile is still *named* "1080p VAAPI h264 aac" — cosmetic, ignore the name.
|
||||
- **The old "do NOT set QSV" rule is RETIRED — #498 fixed the blocker it was based on.** The 2026-07-20 regression was real (QSV's *decoder* is far stricter than VAAPI about malformed NAL units and failed 3 of 6 cold-starts: `Error splitting the input into NAL units`), and the stated cause was that one `HardwareAcceleration` column governed both decode and encode. **#498 added `QsvPreferNativeDecoder` (default ON, Linux-only)**, which splits them exactly like Jellyfin: decode with the tolerant VA-API decoder, encode with QSV. That is what prod runs now. Do not "fix" prod back to `3` (Vaapi) on the strength of the old note.
|
||||
- **Two QSV traps already paid for, both fixed in code — don't re-derive them:**
|
||||
- `QsvExtraHardwareFrames` must never be `0`: the software→QSV `hwupload` bridge has no headroom and the transcode writes **zero segments** on any unthrottled read (#523/#529). Code now floors it at 64 (`ffmpeg.qsv-extra-hw-frames-floor`).
|
||||
- **HDR tonemapping never uses `vpp_qsv=tonemap`** — on this Gen9.5 iGPU that filter is a *silent no-op* (byte-identical output, exit 0, no warning), so it looked like GPU tonemapping while doing nothing. ErsatzTV now tonemaps via VA-API→OpenCL (#505, `ffmpeg.qsv-hdr-tonemap-opencl`). Same trap applies to Jellyfin's `EnableVppTonemapping` on this host — keep it off.
|
||||
- Fallback if VAAPI also misbehaves (see #631, VAAPI `hwupload -22` on 10-bit): `HardwareAcceleration = 0` (software). jazz has 16 threads at load ~2, so it is affordable and maximally tolerant of imperfect sources.
|
||||
- QSV (Intel Quick Sync) hardware acceleration
|
||||
- Resolution: 1920x1080, H264, AAC stereo
|
||||
- Device: `/dev/dri` passed through (`renderD128`)
|
||||
- HardwareAccelerationKind: 0=None, 1=Qsv, 2=Nvenc, 3=Vaapi, 4=VideoToolbox, 5=Amf — **jazz uses 1 (Qsv)** with `QsvPreferNativeDecoder` ON (see above)
|
||||
- jazz's iGPU is shared with Jellyfin only (Frigate stayed on bumblebee); render GID is 992 on both hosts, so `group_add: '992'` carried over unchanged
|
||||
- Device: `/dev/dri` passed through
|
||||
- HardwareAccelerationKind: 0=None, 1=Qsv, 2=Nvenc, 3=Vaapi, 4=VideoToolbox, 5=Amf
|
||||
|
||||
## Jellyfin Integration
|
||||
|
||||
- Secrets: `/config/jellyfin-secrets.json` (`{"Address":"http://jellyfin:8096","ApiKey":"978033be716d46678a5d3c54ae0e0ff9"}`)
|
||||
- **ErsatzTV** library ids (verified 2026-07-26): Jellyfin source → Movies **10**, TV Shows **11**,
|
||||
Music Videos **16**; Local source → Standup **14**. These are *ErsatzTV* ids and are **not** the same
|
||||
as Jellyfin's own library ids — don't reuse one for the other. Re-derive with
|
||||
`GET /api/v1/media-sources` rather than trusting this list.
|
||||
- Scan a library with `POST /api/v1/libraries/{id}/scan` (there is no `PUT …/sync`).
|
||||
- Libraries: Movies(10), TV Shows(11), Music Videos(8), Standup(9)
|
||||
- `JellyfinLibrary.ShouldSyncItems` must be `1` for scans to work
|
||||
|
||||
## Gotchas
|
||||
|
||||
### Post-move to jazz (#633)
|
||||
- **Any rsync from bumblebee's `~/downloadswarm/ersatztv/` re-reverts the QSV setting** — it overwrites `ersatztv.sqlite3`, restoring bumblebee's AMD-era values. Apply config changes **after** the final sync, then re-verify. (Same trap for Jellyfin's `encoding.xml` and `livetv.xml`.)
|
||||
- **The config dir has root-owned files** (`ersatztv.sqlite3`, `cache/channel-guide/*`), so rsync needs sudo at **both** ends:
|
||||
```bash
|
||||
sudo rsync -a --delete -e "ssh -i /home/timothy/.ssh/id_rsa" --rsync-path="sudo rsync" \
|
||||
timothy@192.168.1.99:/home/timothy/downloadswarm/ersatztv/ /home/timothy/downloadswarm/ersatztv/
|
||||
```
|
||||
- **Dispatcharr caches ErsatzTV's XMLTV.** Repointing its DB rows is not enough — it keeps serving a stale EPG full of dead `ersatztv:8409` artwork URLs (breaks Kodi artwork). Force a refresh (EPG source 9):
|
||||
```bash
|
||||
ssh timothy@192.168.1.99 'docker exec dispatcharr python manage.py shell -c \
|
||||
"from apps.epg.tasks import refresh_epg_data; refresh_epg_data(9)"'
|
||||
```
|
||||
- **`/api/health` returns 401** (needs an API key). The Telegraf probe has no `response_string_match`, so ErsatzTV reads as **unhealthy in Grafana** — a false alarm, and **pre-existing**, not caused by the move. The container healthcheck uses the unauthenticated internal `/health` and is unaffected.
|
||||
- **A Komodo deploy alone may not apply bind-mounted config changes** — containers kept serving the pre-checkout inode despite a current `deployed_hash`. `docker restart` explicitly and verify inside the container.
|
||||
|
||||
### Common Mistakes (check every time)
|
||||
- **Playout not building**: Three things must all be correct: (1) `ProgramScheduleOneItem` row exists for the schedule item, (2) `PlaybackOrder=3` (Shuffle), (3) `ScheduleKind=1` on Playout. Missing any one results in 0 playout items — this is the most common issue.
|
||||
- **Collection queries fail**: `CollectionItem` has no `Id` column — use `rowid` for counting. Content lookup goes through `MediaVersion.MovieId` → `MediaFile.Path` (not a generic MediaItemId join).
|
||||
- **Channel logos forgotten**: After creating a channel, add an Artwork row (ArtworkKind=2) + logo file, then run `fix_logos.py` to push to Jellyfin. Without this, the channel shows no logo in the EPG.
|
||||
- **Playout reset required**: After any schedule/collection change, run `POST /api/v1/channels/{id}/playout/reset`. Wait 5-10s for the playout to build before verifying item count.
|
||||
|
||||
### Streaming mode + the Dispatcharr reliability fix — #500
|
||||
Consumers reach ETV **only through Dispatcharr** (`ErsatzTV → Dispatcharr → Jellyfin/Kodi`), which proxies every channel with `ffmpeg -i <etv-url> -c copy -f mpegts`. **Both HLS Segmenter (`StreamingMode=4`) and MPEG-TS (`StreamingMode=1`, `ts-legacy`) work** — Dispatcharr remuxes either to mpegts, and ETV's HLS segments are themselves mpegts with in-band SPS/PPS, so `-c copy` carries codec init either way. We run **42 channels on HLS** (ETV default; ts-legacy showed more visual glitching) + Jungle(407) on TS.
|
||||
- **What the ~6 s cold-start actually was — ersatztv#350 (fixed 2026-07-20).** `-readrate 1.05` paces input at wall clock so the channel behaves like live TV, and it applies from the **first** read; with 4 s HLS segments a throttled session could not serve the playlist sooner than ~3.8 s. Only `workAheadSegmenterLimit` sessions (prod: **1**, see `/api/v1/settings/ffmpeg`) start unthrottled, so **concurrent tune-ins are the slow ones** — measured 866 ms for the slot winner vs 3845/6357 ms for two simultaneous tunes. Subtitle burn-in, source GOP length and NFS were investigated and **ruled out** (accurate-seek costs 30–100 ms). Fixed with `-readrate_initial_burst` (5369 → 648 ms at the ffmpeg level); end-to-end verification tracked in `timothy/ersatztv#519`, so until that lands treat it as expected rather than confirmed. Diagnose with `docker logs ersatztv | grep "HLS cold-start"` — the line splits `setup / startup (prep + ffmpegInit + firstGop) / fill`.
|
||||
- **The reliability bug was NOT the streaming mode — it was a Dispatcharr teardown race.** Any tune spins up a fresh ETV transcode (historically ~6 s cold-start, same for HLS and TS — see above). With Dispatcharr's default `channel_shutdown_delay=0`, the instant a client's open-timeout drops it the channel tears down, and the retry hits a 503 → ETV cold-starts again → death-spiral (Dispatcharr#503/#851). **Fix lives in Dispatcharr: `channel_shutdown_delay=15`** (see dispatcharr skill → Gotchas). Verified by reverting all channels to HLS while keeping the delay → reliable starts + correct audio sync (2026-06-28).
|
||||
- **Corrected theory:** the first #500 pass blamed HLS for `Invalid avcC`/codec-init and switched everything to MPEG-TS. **That was wrong** — `-c copy` of mpegts HLS segments carries SPS/PPS fine; the `avcC` log line was transient/info-level and appeared on TS too. The isolation test (HLS + the delay) proved `channel_shutdown_delay` was the actual fix, and we reverted to HLS for better quality.
|
||||
- Flip a channel's mode live (no restart — ETV reads it per M3U request): `UPDATE Channel SET StreamingMode=4 WHERE …;` then sync Dispatcharr's stored stream URL for that channel (`.m3u8?mode=segmenter` ↔ `.ts?mode=ts-legacy`).
|
||||
- **Open / in progress:** through Dispatcharr's `-c copy` proxy, HLS showed a one-time skip-back shortly after start (Dispatcharr's `new_client_behind_seconds` repositioning the client behind live — set to 0 to test) and TS showed more glitching. Artifact tuning continues — see the dispatcharr skill and the #500 follow-up.
|
||||
|
||||
### Measuring what is actually deployed / what actually happened
|
||||
- **The api.key file is root-owned, and an unsudo'd read fails SILENTLY.** `cat` returns nothing, the
|
||||
header goes out empty, and the 401 body parses as a dict — so a naive script reports "0 channels"
|
||||
rather than an auth error. If a query returns a suspiciously empty result, **check auth before
|
||||
believing it.** (Cost a wrong reading on 2026-07-21.)
|
||||
- **A container's OCI labels lie about what is running** — they are inherited from the base image (they
|
||||
claimed `2026-06-27` on an image built minutes earlier). Tags and `StartedAt` lie too. To prove which
|
||||
build is live, compare `docker inspect <c> --format '{{.Image}}'` (the manifest digest on jazz) to the
|
||||
registry's `Docker-Content-Digest` header for that tag — not `.config.digest`. (ersatztv#350)
|
||||
- **Container log lines carry a LOCAL-time bracket (`[18:48:13 DBG]`) while `docker logs -t` emits
|
||||
UTC**, so `--since` windows silently mis-slice. For before/after measurements capture by **line
|
||||
offset** instead (`wc -l` before, `tail -n +N` after).
|
||||
|
||||
### DB & Architecture
|
||||
- DB owned by root — always use `sudo sqlite3`
|
||||
- **The api.key file is root-owned too, and an unsudo'd read fails SILENTLY.** `cat` returns nothing,
|
||||
the header goes out empty, and the 401 body parses as a dict — so a naive script reports "0
|
||||
channels" rather than an auth error. If a query returns a suspiciously empty result, check auth
|
||||
before believing it. (Cost a wrong reading on 2026-07-21.)
|
||||
- WAL mode: reads OK while running, stop container for writes
|
||||
- Full REST CRUD is available under `/api/v1`; prefer it over direct DB writes
|
||||
- ~~No REST API for channel/collection/schedule CRUD~~ — **false since the fork's `/api/v1`**; use the
|
||||
API, not DB scripting, wherever an endpoint exists
|
||||
- **A container's OCI labels lie about what is running** — they are inherited from the linuxserver
|
||||
base image (they claimed `2026-06-27` on an image built minutes earlier). To prove which build is
|
||||
live, compare `docker inspect <c> --format '{{.Image}}'` to the registry's `Docker-Content-Digest`
|
||||
for that tag
|
||||
- **Container log lines carry a LOCAL-time bracket (`[18:48:13 DBG]`) while `docker logs -t` emits
|
||||
UTC**, so `--since` windows silently mis-slice. For before/after measurements capture by line
|
||||
offset instead (`wc -l` before, `tail -n +N` after)
|
||||
- Secrets file uses PascalCase JSON (`Address`, `ApiKey`)
|
||||
- Scanner is separate binary (`ErsatzTV.Scanner`) — check with `docker top ersatztv | grep Scanner`
|
||||
- EF TPT inheritance: `ProgramScheduleItem` has subtype tables (`ProgramScheduleOneItem`, etc.) — inserting into the subtype table is required or EF Core won't recognize the row
|
||||
- `/health` is the unauthenticated container-health gate; use an authenticated `/api/v1` read to verify the API
|
||||
|
||||
### Enums
|
||||
- PlaybackOrder: 2=Chronological (broken for collections — produces empty playouts), 3=Shuffle, 6=SeasonEpisode — use 3 for reliable results
|
||||
- CollectionType: 0=Collection, 1=Show (direct show reference via MediaItemId)
|
||||
- EF TPT inheritance: `ProgramScheduleItem` has subtype tables (`ProgramScheduleOneItem`, etc.) — MUST insert into subtype table
|
||||
- External URL logos work for M3U but NOT for watermark burn-in (code checks `File.Exists()`)
|
||||
- `/api/health` predates the Blazor removal; verify the API with an authenticated `/api/v1/channels` instead
|
||||
- PlaybackOrder enum: 3=Shuffle, 6=SeasonEpisode (use 3 for all channels)
|
||||
- CollectionType enum: 0=Collection, 1=Show (direct show reference via MediaItemId)
|
||||
- SubtitleMode: 0=None, 2=Burn-in. Set to 2 with PreferredSubtitleLanguageCode='eng' for non-music channels
|
||||
- MediaItem.State: 0=Normal, 1=FileNotFound — clean up state=1 items by deleting cascading deps
|
||||
- ScheduleKind: 0=None (broken — playout never builds), 1=Fixed — use 1
|
||||
- StreamingMode: 4=HLS Segmenter (`…/channel/N.m3u8?mode=segmenter`) — **ETV default, what we run** (42 channels); 1=MPEG-TS (`…/channel/N.ts?mode=ts-legacy`, Jungle/407 only). Both work through Dispatcharr (it remuxes either to mpegts via `-c copy`). Read live per M3U request → flipping needs **no container restart**. The #500 reliability fix was a Dispatcharr setting (`channel_shutdown_delay`), NOT the mode — see "Streaming mode" gotcha.
|
||||
|
||||
### Channel Creation Checklist
|
||||
1. Collection + CollectionItems (for collection-based) OR MediaItemId (for show-specific)
|
||||
2. ProgramSchedule (all NOT NULL columns: FixedStartTimeBehavior, KeepMultiPartEpisodesTogether, RandomStartPoint, ShuffleScheduleItems, TreatCollectionsAsShows)
|
||||
3. ProgramScheduleItem (PlaybackOrder=3) + ProgramScheduleOneItem subtype row
|
||||
4. Channel (SongVideoMode=0, WatermarkId=1, all required columns)
|
||||
5. Playout (ScheduleKind=1)
|
||||
6. Artwork (ArtworkKind=2) + logo file in cache
|
||||
7. `POST /api/v1/channels/{id}/playout/reset`
|
||||
8. Run `fix_logos.py` to push logo to Jellyfin
|
||||
|
||||
### Logo System
|
||||
- **External-URL logos now work for the on-screen bug too** — fixed in ersatztv#502 (2026-07-20,
|
||||
`ffmpeg.external-logo-graphics-engine`). The old claim that they work for M3U but not watermark
|
||||
burn-in described a `WatermarkSelector` `File.Exists()` gate that is gone; an external logo is
|
||||
fetched, decode-budget-validated and stored in the image cache at **save** time
|
||||
(`graphics.channel-logo-caching`), so the render path never fetches over HTTP and a bad URL fails
|
||||
the save with a 422.
|
||||
- **M3U/XMLTV absolute URLs are no longer stuck on the request-derived host.** They used to bake in
|
||||
whatever host fetched the feed (the historical `http://localhost:8409` symptom, Gitea #1/#171),
|
||||
which Jellyfin can't resolve from inside its container. Set the optional advertised base URL —
|
||||
`GET`/`PUT /api/v1/settings/iptv` (`iptv.base_url`, ersatztv#340, `iptv.base-url`) — to pin them to
|
||||
a fixed public origin; unset falls back byte-identical to the old behavior. The base64-upload
|
||||
workaround in `docs/Docker/ErsatzTV.md` is only needed if that setting is left unset.
|
||||
- **No usable logo ⇒ no on-screen bug, from every attachment point** (ersatztv#510, 2026-07-26,
|
||||
`ffmpeg.watermark-resolution-unified`). A `ChannelLogo` watermark resolves through one shared
|
||||
`WatermarkSelector.ResolveWatermark` whether it came from a playout item, the channel, the global
|
||||
setting, **or a deco**. A missing cached file, an un-migrated external URL, and a channel with no logo
|
||||
artwork each render *without* a bug and log a warning. So when debugging "this channel has a watermark
|
||||
configured but no bug appears", grep the log for `has no logo artwork` / `no longer exists` before
|
||||
suspecting the ffmpeg pipeline.
|
||||
- Before #510 the **deco** path alone was unchecked and returned the generated-initials nameplate
|
||||
(`/iptv/logos/gen`) for a logoless channel — it genuinely rendered. That fallback is now off
|
||||
everywhere; reviving it via the image cache is ersatztv#652.
|
||||
- **Not covered:** the song-progress overlay is built as a `WatermarkOptions` directly by the
|
||||
streaming/troubleshooting handlers, bypassing the resolver, and is still unchecked — ersatztv#653.
|
||||
- **`/iptv/logos/gen` is unauthenticated**, unlike the rest of `/iptv`: `ConditionalIptvAuthorizeFilter`
|
||||
is a class-level attribute on `IptvController` only, and that route lives on `ArtworkController`.
|
||||
Handy for probing, and the reason a container-internal self-fetch of a generated logo succeeds.
|
||||
- **Seeding a deco watermark for testing is fully API-driven** (no SQLite needed): `POST /api/v1/watermarks`
|
||||
(needs the full required field set — check `v1.json`), `POST /api/v1/decos/groups`, `POST /api/v1/decos`,
|
||||
`PUT /api/v1/decos/{id}` (set `watermarkMode` + `watermarkIds`), then `PUT /api/v1/playouts/{id}/deco`.
|
||||
Use `watermarkMode: "Override"` to make the deco watermark the only one selected. Note branding is
|
||||
**not** testable through the troubleshooting-playback API (`testing.troubleshoot-path-cannot-test-branding`)
|
||||
— drive a real channel playout and capture a frame.
|
||||
- `logo_XX.png` files in the logos root dir are HTML garbage (broken downloads), not actual logos — ignore them
|
||||
|
||||
### Other
|
||||
- Upstream was archived in Feb 2026; `timothy/ersatztv` is the maintained fork and release source
|
||||
- ProgramSchedule required NOT NULL columns: FixedStartTimeBehavior, KeepMultiPartEpisodesTogether, RandomStartPoint, ShuffleScheduleItems, TreatCollectionsAsShows
|
||||
- Channel required NOT NULL columns: SongVideoMode (set 0), plus all standard columns (see Channel table schema)
|
||||
- After schedule changes, rebuild playout: `POST /api/channels/{number}/playout/reset`
|
||||
- Playout `ScheduleKind` must be `1` (not `0`/None) — `0` causes "Cannot build playout type None" error
|
||||
- M3U `tvg-logo` URLs hardcode `http://localhost:8409` — Jellyfin can't fetch these from inside its container. Fix by downloading logos from ETV and base64-uploading to Jellyfin (see `docs/Docker/ErsatzTV.md` for script). Tracked in issue #171
|
||||
- Repo archived Feb 2026, v26.3.0 is final stable version. Maintainer welcomes forks
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
../../../server-management/.claude/skills/jellyfin
|
||||
@@ -0,0 +1,120 @@
|
||||
---
|
||||
name: jellyfin
|
||||
description: Jellyfin media server management — API for libraries, items, streaming, users. Use when managing media library or checking Jellyfin status.
|
||||
---
|
||||
|
||||
# Jellyfin Management
|
||||
|
||||
Container: `jellyfin` | Port: `8096` | IP: `172.16.238.20` (may change on restart)
|
||||
API Token: `978033be716d46678a5d3c54ae0e0ff9`
|
||||
Web UI: `https://jellyfin.tblindustries.be` (NO Authelia — native login, password: `coup1802`)
|
||||
Config: `/home/timothy/downloadswarm/jellyfin/` on jazz
|
||||
|
||||
## Access Pattern
|
||||
|
||||
```bash
|
||||
docker exec jellyfin curl -s 'http://localhost:8096/ENDPOINT' \
|
||||
-H 'X-Emby-Token: 978033be716d46678a5d3c54ae0e0ff9'
|
||||
```
|
||||
|
||||
## Volume Mounts
|
||||
|
||||
| Host Path | Container Path | Content |
|
||||
|-----------|---------------|---------|
|
||||
| `/mnt/teramind/episodes` | `/data/tvshows` | TV shows |
|
||||
| `/mnt/episodes` | `/data/episodes` | More episodes |
|
||||
| `/mnt/media/movies` | `/data/movies` | Movies |
|
||||
| `/mnt/media/standup` | `/data/standup` | Standup |
|
||||
| `/mnt/media/music_videos` | `/data/music` | Music videos |
|
||||
| `/mnt/media/audio/music` | `/data/audio` | Music audio (ro) |
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### System
|
||||
```
|
||||
GET /System/Info # Server info, version
|
||||
GET /System/Info/Public # Public info (no auth needed)
|
||||
POST /System/Restart # Restart server
|
||||
```
|
||||
|
||||
### Items (Search & Browse)
|
||||
```bash
|
||||
# Search items
|
||||
GET /Items?includeItemTypes=Movie,Episode,Series&recursive=true&searchTerm=QUERY&fields=Path&limit=20
|
||||
|
||||
# Get item details
|
||||
GET /Items?ids=ITEM_ID&fields=Path,MediaStreams,Overview
|
||||
|
||||
# Get all movies
|
||||
GET /Items?includeItemTypes=Movie&recursive=true&fields=Path&limit=1000
|
||||
|
||||
# Get series
|
||||
GET /Items?includeItemTypes=Series&recursive=true&fields=Path
|
||||
|
||||
# Get episodes for a series
|
||||
GET /Shows/{seriesId}/Episodes?fields=Path,MediaStreams
|
||||
|
||||
# Filter by library (parentId)
|
||||
GET /Items?parentId=LIBRARY_ID&recursive=true&fields=Path
|
||||
```
|
||||
|
||||
### Libraries
|
||||
```
|
||||
GET /Library/VirtualFolders # List all libraries
|
||||
POST /Library/Refresh # Trigger full library scan
|
||||
POST /Items/{id}/Refresh # Refresh single item metadata
|
||||
```
|
||||
|
||||
### Streaming
|
||||
```bash
|
||||
# Test stream URL
|
||||
GET /Videos/{itemId}/stream?static=true
|
||||
|
||||
# Get playback info
|
||||
GET /Items/{itemId}/PlaybackInfo
|
||||
```
|
||||
|
||||
### Users
|
||||
```
|
||||
GET /Users # List users
|
||||
GET /Users/{userId} # User details
|
||||
```
|
||||
|
||||
## Library IDs
|
||||
|
||||
Check with: `curl -s -H "X-Emby-Token: TOKEN" http://localhost:8096/Library/VirtualFolders`
|
||||
|
||||
## Live TV
|
||||
|
||||
- **ErsatzTV** (channels <1000): M3U `http://ersatztv:8409/iptv/channels.m3u`, XMLTV `http://ersatztv:8409/iptv/xmltv.xml`
|
||||
- **Dispatcharr** (channels 1000+): IPTV stream manager on port 9191, separate tuner
|
||||
- Configured in Jellyfin Admin > Live TV
|
||||
- Guide refresh task ID: `bea9b218c97bbf98c5dc1303bdb9a0ca` — trigger via `POST /ScheduledTasks/Running/{id}`
|
||||
- **Logo fix after guide refresh**: ErsatzTV logos break (aspect ratio=0) because M3U uses `localhost:8409`. Fix script in `docs/Docker/ErsatzTV.md` downloads from ETV and base64-uploads to `POST /Items/{id}/Images/Primary` (body = base64, Content-Type = image/png)
|
||||
- **Image upload format**: Jellyfin expects base64-encoded body (NOT raw binary) for `POST /Items/{id}/Images/Primary`
|
||||
|
||||
## Gotchas
|
||||
|
||||
- **Passwords**: `coup1802` (NOT `ded89Lm4`) — Jellyfin has native auth, no Authelia
|
||||
- Auth header is `X-Emby-Token` (Jellyfin is an Emby fork)
|
||||
- **Music videos are typed `MusicVideo`, NOT `Movie`** (corrected 2026-07-21, ersatztv#177). The old
|
||||
"typed as Movie" note described a deliberate DB reclassification workaround that existed only because
|
||||
ErsatzTV could not consume `MusicVideo` items — ersatztv#42 shipped that sync, so the workaround's
|
||||
premise is gone. Verified live: the `Music Videos` library (`/data/music`, collection type
|
||||
`musicvideos`) holds 1437 items typed `MusicVideo` and **zero** typed `Movie`. Query with
|
||||
`includeItemTypes=MusicVideo`. (Reclassification to `Movie` may still apply to concert/standup content
|
||||
in the `movies`/`mixed` libraries — that is a different set; see the server-management jellyfin skill.)
|
||||
- **`Album` is not an `ItemFields` value.** It is a plain `BaseItemDto` property serialized whenever set,
|
||||
so it comes back regardless of the `fields=` query param — do NOT add it to `fields` (verified: 111 of
|
||||
1437 music videos returned `Album` with `fields=Path` alone). Contrast `Genres`/`People`/`Chapters`,
|
||||
which ARE `ItemFields` and must be requested. Check the enum before extending `fields`.
|
||||
- **`IndexNumber` is the track number; `ParentIndexNumber` is the disc/season axis.** Frequency misleads
|
||||
here — on the live music video library `ParentIndexNumber` is populated on 66 items vs 4 for
|
||||
`IndexNumber`, but where both exist `ParentIndexNumber` is `1` while `IndexNumber` holds the real
|
||||
ordinal, and where only `ParentIndexNumber` exists it is a collection grouping tracking the album
|
||||
(`Glastonbury: 2022` -> 230). `AlbumId` is always null on these items.
|
||||
- Music library at `/data/music` maps to `/mnt/media/music_videos` on host (not actual music)
|
||||
- Items return 404 on stream if source volume is unmounted
|
||||
- Jellyfin preserves item IDs across restarts unless files are renamed
|
||||
- Full library scan can take a long time — prefer targeted `/Items/{id}/Refresh`
|
||||
- `ffprobe` available in container for checking media streams: `docker exec jellyfin ffprobe -v quiet -print_format json -show_streams FILE`
|
||||
@@ -3,7 +3,7 @@
|
||||
"isRoot": true,
|
||||
"tools": {
|
||||
"jetbrains.resharper.globaltools": {
|
||||
"version": "2025.3.5",
|
||||
"version": "2025.3.4.1",
|
||||
"commands": [
|
||||
"jb"
|
||||
],
|
||||
|
||||
@@ -408,16 +408,6 @@ jobs:
|
||||
done
|
||||
echo "::endgroup::"
|
||||
|
||||
# NOTE (ersatztv#491 -> #627): running the LibraryFolder dedupe fixture against the live `mysql`
|
||||
# service was implemented here and then REMOVED. The coverage gap it closes is real — the two
|
||||
# checks above only ever apply migrations to a fresh EMPTY database, so they execute no rows of any
|
||||
# data-migration logic, and two MySql-only collation defects escaped exactly this gate. But the
|
||||
# fixture proved non-deterministic in CI across three attempts (stale pooled session after a drop,
|
||||
# then lost isolation from a shared database name, then a connect-before-create), and an
|
||||
# intermittently-red gate is worse than none: it trains everyone to re-run instead of read, which is
|
||||
# how the original defects escaped. The fixture itself is retained and is opt-in via
|
||||
# ETV_TEST_MYSQL_CONNECTION (skipped, visibly, without it). Re-arming it here is tracked by #627.
|
||||
|
||||
functional-e2e:
|
||||
name: Functional E2E (curl + UI contracts)
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -3,9 +3,8 @@ name: PR Gates
|
||||
# Fast, git-only PR gates split out of docker-build.yml into a dedicated `on: pull_request`
|
||||
# workflow (ersatztv#535) so they are NEVER created on a tag/main push.
|
||||
#
|
||||
# WHY THIS FILE EXISTS. These checks are cheap `checkout + git diff` gates (or, for `script-tests`,
|
||||
# checkout + pytest): they carry no `container:`, run on the `small` lane (git-only, 1 GiB;
|
||||
# server-management#639), and are PR-only.
|
||||
# WHY THIS FILE EXISTS. These three checks are pure `checkout + git diff` gates: they carry no
|
||||
# `container:`, run on the `small` lane (git-only, 1 GiB; server-management#639), and are PR-only.
|
||||
# While they lived in docker-build.yml — which also triggers on push to main and on `v*` tags —
|
||||
# Gitea still DISPATCHED them as runner tasks on every such push to evaluate the `if:` skip, because
|
||||
# **Gitea dispatches a job as a runner task even when its `if` skips it** (docs/ci-cd.md -> the
|
||||
@@ -23,10 +22,10 @@ name: PR Gates
|
||||
#
|
||||
# These stay on `runs-on: small` and carry NO CI toolchain image pin, so `ci-image-pin`'s grep of
|
||||
# docker-build.yml still validates the five pin-bearing jobs (test/migrations/functional-e2e/
|
||||
# api-docs/format) that remain there. None of these jobs are required checks — branch protection
|
||||
# requires only `Build & test (.NET)`, `EF migration integrity` and `review-verdict/h10` — so
|
||||
# relocating them (which changes their status-context prefix from "Build ErsatzTV Image / …" to
|
||||
# "PR Gates / …") does not affect merges. See docs/ci-cd.md -> "PR gates workflow".
|
||||
# api-docs/format) that remain there. None of these three are required checks — branch protection
|
||||
# requires only `Build & test (.NET)` and `EF migration integrity` — so relocating them (which
|
||||
# changes their status-context prefix from "Build ErsatzTV Image / …" to "PR Gates / …") does not
|
||||
# affect merges. See docs/ci-cd.md -> "PR gates workflow".
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
@@ -161,8 +160,7 @@ jobs:
|
||||
|
||||
# BLOCKING (ersatztv#521, supersedes the ersatztv#303 H9 append-only mechanic): validates decision-
|
||||
# record lifecycle invariants (metadata schema, one active record per key, reciprocal
|
||||
# supersedes/superseded-by links, no rationale-prose rewrite without a Decisions-Edit: yes git
|
||||
# trailer (ersatztv#609 — never a bare substring, which prose about the marker could arm), no record
|
||||
# supersedes/superseded-by links, no rationale-prose rewrite without [decisions-edit], no record
|
||||
# vanishing from the active set without an archive copy) and that the generated active catalog
|
||||
# (docs/decisions/README.md) is in sync. Same validator the Husky pre-commit hook shim calls, so
|
||||
# local and CI enforcement can't drift. Seconds-long git diff + parse -> keep it off the build runners.
|
||||
@@ -188,75 +186,3 @@ jobs:
|
||||
run: PYTHONPATH=. python3 scripts/build_decisions_catalog.py --check
|
||||
- name: Kickoff guard
|
||||
run: bash scripts/check-kickoff-guard.sh
|
||||
|
||||
# FAILS THE RUN on a red (ersatztv#631) — like its sibling gates here it is not (yet) a required
|
||||
# status check, so it reddens the PR without hard-blocking the merge button; see the header.
|
||||
# Runs scripts/tests/ — the pytest suite covering the decision-corpus
|
||||
# parser/validator/catalog builder, the #610 migration-equivalence harness, the merge-consent
|
||||
# exemption logic and the #622 review-verdict poster. Until #631 NOTHING executed these: no
|
||||
# workflow and no Husky hook invoked pytest, so the suite guarding our merge-gating machinery was
|
||||
# local-only and a regression in it was caught only by luck. `decisions-guard` above runs that
|
||||
# code, but never its tests.
|
||||
#
|
||||
# WHY ITS OWN JOB rather than a step inside decisions-guard (which the issue proposed as the
|
||||
# cheapest home): `ci.decisions-lifecycle-flake` is a STANDING instruction that a lone
|
||||
# `decisions lifecycle` red is a known infra flake to be ignored — "do not investigate". Folding
|
||||
# the suite into that job would make a genuine pytest regression present as exactly the red every
|
||||
# session is told to wave through, which is the same silently-green failure mode #631 exists to
|
||||
# close. A distinct job name keeps a real failure unambiguous.
|
||||
#
|
||||
# Runs UNCONDITIONALLY on every PR rather than behind a `scripts/**` path filter. The suite's
|
||||
# corpus tests are fixture/tmp-repo based, but test_post_review_verdict.py and
|
||||
# test_merge_consent_exemption.py execute the REAL `scripts/post-review-verdict.sh` and
|
||||
# `.claude/hooks/pretooluse-merge-consent.sh`, so its true input set spans at least two top-level
|
||||
# directories. A `scripts/**` filter would silently miss a `.claude/hooks/**` edit — and at ~10s a
|
||||
# filter buys nothing but drift.
|
||||
script-tests:
|
||||
name: Script tests (pytest)
|
||||
runs-on: small
|
||||
if: github.event_name == 'pull_request'
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.x'
|
||||
# pytest + PyYAML. PyYAML is NOT a contradiction of the dependency-free decisions READ path:
|
||||
# `decisions_lib._read_frontmatter` is hand-written precisely so validation runs where nothing
|
||||
# is installed, but the one-shot WRITE path `migrate_decisions_split.py` uses PyYAML by
|
||||
# design — and `test_migration_equivalence.py` imports that module, so the suite needs it.
|
||||
# `pytest` and `yaml` are the complete third-party set, established by an AST import scan over
|
||||
# all of scripts/ rather than by reading the files that seemed relevant: the first cut of this
|
||||
# job claimed "pure stdlib", passed locally on a machine that happened to have PyYAML, and
|
||||
# went red in CI on a collection error.
|
||||
- name: Install test dependencies
|
||||
run: python3 -m pip install --disable-pip-version-check --quiet pytest pyyaml
|
||||
# Preflight, not an install (ersatztv#390 removed run-time `apt-get` from CI on purpose).
|
||||
# test_post_review_verdict.py and test_merge_consent_exemption.py exec the REAL
|
||||
# post-review-verdict.sh / pretooluse-merge-consent.sh, which shell out to `jq` ~26 times.
|
||||
# `curl` those tests shim on PATH; `jq` they do NOT. If it were missing, the suite would fail
|
||||
# as ~20 opaque assertion errors — this turns that into one actionable line.
|
||||
- name: Preflight external tools
|
||||
run: |
|
||||
if ! command -v git >/dev/null 2>&1; then
|
||||
echo "::error::script-tests needs git on PATH but it is absent. The suite execs real" \
|
||||
"shell scripts that use it. Bake it into the runner image rather than apt-get" \
|
||||
"installing here (see ersatztv#390)."
|
||||
exit 1
|
||||
fi
|
||||
echo "Preflight OK: $(git --version)"
|
||||
# jq gets its OWN step because its VERSION, not merely its presence, is load-bearing
|
||||
# (ersatztv#648). `--expect` makes this a TRIPWIRE: scripts/tests exercises the jq 1.6 code path
|
||||
# only because this runner ships 1.6, so an upgrade would silently delete that coverage — and
|
||||
# the three divergences found in ersatztv#643/#647 all lived exactly there. Going red forces an
|
||||
# explicit human decision instead of letting the coverage evaporate.
|
||||
#
|
||||
# The pin lives HERE and deliberately NOT in review-verdict.yml: that workflow writes the
|
||||
# branch-protection-required `review-verdict/h10` status, so pinning a version there would turn
|
||||
# any jq bump on the runner into a repo-wide merge deadlock. It gets the floor-only mode.
|
||||
# See docs/ci-cd.md -> "The jq contract".
|
||||
- name: Preflight jq version
|
||||
run: ./scripts/jq-preflight.sh --expect 1.6
|
||||
- name: Run scripts/tests
|
||||
run: PYTHONPATH=. python3 -m pytest scripts/tests -q
|
||||
|
||||
@@ -1,853 +0,0 @@
|
||||
name: Review verdict
|
||||
|
||||
# ersatztv#622 — server-side H10 enforcement.
|
||||
#
|
||||
# THE INVARIANT. `review-verdict/h10` is a REQUIRED status check on `main`, and a Gitea commit
|
||||
# status belongs to exactly ONE sha. So a commit that did not exist when a verdict was written can
|
||||
# never inherit that verdict: push a new head and the required context is simply absent, which
|
||||
# Gitea's merge-requirement check treats as not-passing. `merge_when_checks_succeed` therefore
|
||||
# refuses to fire until someone re-reviews THAT head. This is what closes the ersatztv#622 hole,
|
||||
# where the PreToolUse hook proved conditions (b) and (c) against the head at SCHEDULING time and
|
||||
# Gitea then merged whatever head happened to be green minutes later.
|
||||
#
|
||||
# WHAT THIS WORKFLOW DOES — and, importantly, does NOT do. It does NOT decide whether code was
|
||||
# reviewed; only a human/agent review does that, via `scripts/post-review-verdict.sh`, which writes
|
||||
# the `review-verdict/h10` status directly. This workflow only handles the two EXEMPT classes that
|
||||
# would otherwise deadlock, and marks everything else `pending` so the PR shows an explicit,
|
||||
# actionable blocking reason instead of a silently-missing check:
|
||||
#
|
||||
# 1. Bot-authored PRs (Renovate). Renovate uses `platformAutomerge: true` — i.e. Gitea's OWN
|
||||
# auto-merge — to land patch bumps unattended. A required verdict context with no exemption
|
||||
# would stall every dependency PR forever waiting on a human verdict.
|
||||
# 2. Docs-only PRs, mirroring the merge-consent hook's existing docs-only carve-out.
|
||||
#
|
||||
# BOTH exemptions are void when the PR touches a PROTECTED path (see PROTECTED below): the gate,
|
||||
# the CI definition, the git hooks, the scripts they call, and the CI toolchain image. A PR that
|
||||
# weakens the merge gate must never be able to exempt itself from the merge gate — that is the one
|
||||
# self-referential failure worth spending an explicit rule on. Note this also (deliberately) means
|
||||
# Renovate's `docker/ci/Dockerfile` base bumps need a real verdict; those already require the
|
||||
# manual publish-then-pin two-step (docs/ci-cd.md -> "CI toolchain image"), so unattended merge was
|
||||
# never correct for them anyway.
|
||||
#
|
||||
# WHY ITS OWN FILE, not a job in pr-checks.yml: that workflow sets `cancel-in-progress: true`, so a
|
||||
# superseding push cancels its runs. A cancelled run here would leave an EXEMPT PR with no success
|
||||
# status and no further pushes to re-trigger it — Renovate would stall silently. This workflow
|
||||
# therefore takes no cancelling concurrency group.
|
||||
#
|
||||
# This job's OWN status context ("Review verdict / Set review-verdict status (pull_request_target)")
|
||||
# is NOT the required check and is not what gates merges — `review-verdict/h10`, the status it
|
||||
# POSTS, is. Keeping them distinct is deliberate: a workflow cannot be allowed to satisfy the gate
|
||||
# merely by running successfully. The context string carries the trigger name, so the #672 switch
|
||||
# renamed it; that is safe only because it was never in branch protection's required list (which is
|
||||
# the two `docker-build.yml` job contexts plus `review-verdict/h10`). Adding it there later would
|
||||
# undo the distinction this paragraph exists to protect.
|
||||
#
|
||||
# THE CHANGED-FILE ENUMERATION IS NOT INLINE HERE (ersatztv#649). It lives in
|
||||
# `scripts/pr-changed-files.sh`, the single implementation this job and the advisory hook
|
||||
# `.claude/hooks/pretooluse-merge-consent.sh` both call. It used to be written twice, and drifted in
|
||||
# the dangerous direction: four rounds of ersatztv#643 hardening landed on the ADVISORY copy (whose
|
||||
# failure mode is a human prompt) and never reached THIS one (whose failure mode is a `success`
|
||||
# write to a required status with nobody in the loop). See
|
||||
# `docs/decisions/records/ci/shared-pr-file-enumeration.md`.
|
||||
#
|
||||
# WHY THE CHECKOUT TAKES THE PR'S **BASE** REF, NEVER THE HEAD. This job judges the PR, so the PR
|
||||
# must not be able to supply the code that judges it. Checking out `head.sha` would let a PR edit
|
||||
# `scripts/pr-changed-files.sh` to return an empty list and exempt itself — the `PROTECTED` list
|
||||
# below would flag the edit, but only if the enumeration that feeds it were trustworthy, and it
|
||||
# would be the PR's own. `base.sha` is the commit the PR merges INTO: already on `main`, already
|
||||
# reviewed. `persist-credentials: false` because nothing here pushes, and a checkout that leaves a
|
||||
# token in `.git/config` hands it to every script the job runs.
|
||||
#
|
||||
# WHY THE TRIGGER IS `pull_request_target`, NOT `pull_request` (ersatztv#672). The base-ref checkout
|
||||
# above binds the SCRIPTS this job runs to the base. It does not bind the job DEFINITION. Gitea
|
||||
# resolves a `pull_request` workflow definition from the PR's own head commit, so a PR editing THIS
|
||||
# FILE ran its own rewritten copy — which could delete the checkout above, or skip straight to
|
||||
# posting `review-verdict/h10=success` for its head sha. `PROTECTED` did not help (the rewrite
|
||||
# defines `PROTECTED` too) and neither did branch protection, which requires the *context* and
|
||||
# carries `required_approvals: 0`, so a self-posted success satisfied it outright.
|
||||
#
|
||||
# Measured on this instance (Gitea 1.25.4) rather than inferred from GitHub, because the whole point
|
||||
# is that the gate's authority is derived, not asserted. A scratch PR rewriting this file to post a
|
||||
# distinct probe context posted exactly that context, and the real `review-verdict/h10` was never
|
||||
# written at all — the base's definition never ran. Under `pull_request_target` the same rewrite was
|
||||
# ignored: the BASE definition ran and posted `h10=pending`, on both `opened` and `synchronize`,
|
||||
# with `secrets` still available.
|
||||
#
|
||||
# `pull_request_target` is normally the DANGEROUS trigger, and it is worth being explicit about why
|
||||
# that reputation does not transfer here. Its footgun is running untrusted HEAD code with a
|
||||
# privileged token. This job never checks out the head and never executes anything the PR supplies:
|
||||
# it checks out `base.sha` and runs only scripts from that tree. The base-ref checkout is what makes
|
||||
# this trigger safe, so the two must be read as one decision — reintroducing a head checkout under
|
||||
# this trigger would be far worse than the bug being fixed here.
|
||||
#
|
||||
# `branches: [main]` IS LOAD-BEARING, not cosmetic. Base resolution means the BASE branch supplies
|
||||
# the definition, so without this filter a PR opened into an attacker-pushed base branch would run
|
||||
# THAT branch's rewritten gate — trading a head-supplied definition for a base-supplied one and
|
||||
# closing nothing. It matters more than it looks because a commit status is repo-global per sha
|
||||
# (#663): a `success` forged on a head sha under a scratch base is inherited by a later, real PR
|
||||
# into `main` carrying the same head. With the filter, a PR whose base is not `main` produces no run
|
||||
# and no status at all (verified the same way).
|
||||
|
||||
# `edited` IS LOAD-BEARING (ersatztv#698 route 1), not completeness for its own sake. Gitea fires it
|
||||
# when a PR's base is retargeted, and a retarget changes the effective diff WITHOUT moving the head
|
||||
# sha — so none of the other four types fire and the per-sha status stays exactly as it was. That is
|
||||
# what made route 1 persist rather than merely exist: a PR was opened into `main`, retargeted to a
|
||||
# scratch base while this job was in flight so the enumeration read docs-only and posted an exemption
|
||||
# `success`, then retargeted BACK to `main`, where the forged success sat unchallenged on a head whose
|
||||
# diff against `main` carried a C# file (reproduced as probe PR #703; `created_at == updated_at`
|
||||
# afterwards proves nothing reclassified). With `edited`, the retarget back re-runs this job — and the
|
||||
# short-circuit below now re-derives machine-written successes instead of inheriting them, which is
|
||||
# the half that makes the re-run actually change the answer. The two are one fix; `edited` alone would
|
||||
# re-run and then bail out on the existing `success`.
|
||||
#
|
||||
# BE PRECISE ABOUT WHAT THIS BUYS: detection, not atomicity or ordering. Runs are NOT serialized, so
|
||||
# the stale run can post `success` AFTER the reclassifying run posts `pending` — restoring the forged
|
||||
# state with no further event left to correct it — and an already-scheduled auto-merge can fire in the
|
||||
# green window between them. The `main -> scratch -> main` ABA transition is therefore NARROWED and
|
||||
# observable, not closed. Tracked as ersatztv#706; do not read this block as claiming otherwise.
|
||||
on:
|
||||
pull_request_target:
|
||||
branches: [main]
|
||||
types: [opened, reopened, synchronize, ready_for_review, edited]
|
||||
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
|
||||
jobs:
|
||||
set-verdict-status:
|
||||
name: Set review-verdict status
|
||||
runs-on: small # a few API calls; keep it off the build runners
|
||||
steps:
|
||||
# BASE, not head — see the header. `fetch-depth: 1` is enough: nothing here reads history,
|
||||
# only the working tree's `scripts/`.
|
||||
- name: Checkout the PR's BASE ref
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.base.sha }}
|
||||
fetch-depth: 1
|
||||
persist-credentials: false
|
||||
|
||||
# FLOOR ONLY — never `--expect` in this workflow. `--expect` pins an exact version and fails
|
||||
# when it drifts, which is right for `script-tests` (advisory) and catastrophic here: this job
|
||||
# writes `review-verdict/h10`, a REQUIRED check on `main`, so a pin would turn any jq bump on
|
||||
# the runner into a repo-wide merge deadlock. Asserting the 1.6 floor is what the gates below
|
||||
# are written against; see docs/ci-cd.md -> "The jq contract".
|
||||
#
|
||||
# A hard failure here is correct and fails CLOSED: the job dies, no `review-verdict/h10` is
|
||||
# posted, and an absent required check blocks the merge. Guarded on presence because a PR
|
||||
# whose BASE predates ersatztv#658 has no such script, and "the base is old" is not a jq
|
||||
# problem — that case is handled as an enumeration failure below, with an actionable status.
|
||||
- name: jq preflight (floor only)
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -x ./scripts/jq-preflight.sh ]; then
|
||||
./scripts/jq-preflight.sh
|
||||
else
|
||||
echo "::warning::The PR's base ref has no scripts/jq-preflight.sh; skipping the version assertion. The enumeration step below will fail closed on its own."
|
||||
fi
|
||||
|
||||
- name: Classify the PR and post the review-verdict status
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
BASE_URL: ${{ github.server_url }}/api/v1
|
||||
# `scripts/pr-changed-files.sh` reads GITEA_BASE_URL (not BASE_URL) and takes owner/repo as
|
||||
# two SEPARATE arguments (not one `owner/repo` string). Getting either wrong is silent, not
|
||||
# loud: the script would fall back to its hardcoded LAN default and enumerate the wrong
|
||||
# repo, or a wrong host that answers, rather than erroring. A value already ending in
|
||||
# /api/v1 is used as-is by the script.
|
||||
GITEA_BASE_URL: ${{ github.server_url }}/api/v1
|
||||
# BOTH names, same value, on purpose. The script's precedence is
|
||||
# `ETV_GITEA_URL` > `GITEA_BASE_URL` > a hardcoded LAN default (and `ETV_GITEA_TOKEN` >
|
||||
# `GITEA_TOKEN`), because its other caller is a developer Mac using the ETV_* convention.
|
||||
# Setting only the GITEA_* names would leave this job's explicit configuration NON-
|
||||
# authoritative: a runner that happened to export a stale ETV_GITEA_URL would silently
|
||||
# enumerate a different Gitea instance and post the verdict here from a diff read there.
|
||||
# Cheap to make deterministic; leave both set even though only one is read.
|
||||
ETV_GITEA_URL: ${{ github.server_url }}/api/v1
|
||||
ETV_GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
REPO: ${{ github.repository }}
|
||||
PR: ${{ github.event.pull_request.number }}
|
||||
SHA: ${{ github.event.pull_request.head.sha }}
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
# The base BRANCH the event was raised for, passed down to the enumeration so the diff it
|
||||
# reads cannot silently be one against a different base (ersatztv#698 route 1). This comes
|
||||
# from the `pull_request_target` event payload, which is fixed at event time and is exactly
|
||||
# what a mid-run retarget cannot rewrite — the live PR object can, which is the whole bug.
|
||||
# `branches: [main]` means this is always `main` today; it is threaded through as a value
|
||||
# rather than hardcoded so the two stay consistent if the filter ever widens.
|
||||
BASE_REF: ${{ github.event.pull_request.base.ref }}
|
||||
AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
PR_URL: ${{ github.event.pull_request.html_url }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
CONTEXT="review-verdict/h10"
|
||||
# The description this job writes when it repairs its own raced exemption (#706 race 2).
|
||||
# It is a SENTINEL, not just a message: `read_existing_verdict` recognises it, and the
|
||||
# classification below refuses to post `success` over it. Without that, the repair lasted
|
||||
# exactly one event — the next run saw a machine-written `pending`, re-derived it, and
|
||||
# posted `success` again, with its own freshly-taken high-water mark now ABOVE the human
|
||||
# row, so the post-write check stayed silent and the rejection went green a second time.
|
||||
# Found by cold review. Refusing here can only ever withhold an exemption, never grant one.
|
||||
REPAIR_DESC="Human verdict raced this exemption write — re-post the verdict"
|
||||
# Accounts whose PRs may merge without a human verdict. Renovate only — keep this list
|
||||
# minimal and explicit; every entry is an account that can land code unreviewed.
|
||||
BOTS="renovate"
|
||||
# The bot exemption is additionally constrained by CONTENT (ersatztv#698 route 2), because
|
||||
# identity alone is not attributable to whoever wrote the code. `AUTHOR` is
|
||||
# `pull_request.user.login` — the PR's CREATOR, which is immutable — while the head a PR
|
||||
# points at is not: force-push application code onto an open Renovate branch and the PR is
|
||||
# still authored by `renovate`, still touches no protected path, and was exempted. Nothing
|
||||
# in the identity check attributes the CODE to the bot.
|
||||
#
|
||||
# Checking the pusher instead would not fix it — a git author/committer is self-asserted
|
||||
# text and forgeable. So the exemption is gated on what a dependency bump can legitimately
|
||||
# BE: an unattended merge is justified only for the manifests Renovate actually edits.
|
||||
#
|
||||
# The set is measured, not guessed: across all 11 Renovate PRs this repo has ever had, the
|
||||
# paths touched were `Directory.Packages.props` (10 of them) and `.config/dotnet-tools.json`
|
||||
# (1). The npm manifests are deliberately NOT included — see the BOT_MANIFESTS note below.
|
||||
#
|
||||
# Deliberately EXCLUDED, with the cost stated: `*.csproj` and any source file. The one
|
||||
# historical Renovate PR outside the set above is #20, which touched a `.csproj` AND two C#
|
||||
# files — and received an unattended bot exemption for a source change. Under Central
|
||||
# Package Management versions live in `Directory.Packages.props`, so a `.csproj` edit
|
||||
# attributed to Renovate is anomalous by construction. Such a PR is not blocked, it simply
|
||||
# needs a real verdict, which is the correct handling for a PR carrying source changes.
|
||||
# NOTE the npm manifests are deliberately ABSENT. An earlier draft included
|
||||
# `web/package.json` / `web/package-lock.json` "so a first SPA bump cannot deadlock". That was
|
||||
# a self-inflicted code-execution vector for zero benefit: `renovate.json` sets
|
||||
# `enabledManagers: ["nuget", "github-actions", "dockerfile"]`, so Renovate does not manage npm
|
||||
# in this repo at all, while `package.json` carries `scripts` that CI EXECUTES (`npm ci`,
|
||||
# `npm run build` in docker-build.yml). Exempting it would let a hijacked bot branch run
|
||||
# arbitrary shell in CI while every path still "looked like a manifest". If npm is ever added
|
||||
# to enabledManagers, the lockfile may be exemptible but `package.json` is not.
|
||||
BOT_MANIFESTS='^(Directory\.Packages\.props|\.config/dotnet-tools\.json)$'
|
||||
# Paths where NEITHER exemption applies, because a change here can alter the gate itself,
|
||||
# what CI runs, or what the hooks enforce.
|
||||
#
|
||||
# `.codex/` is listed alongside `.claude/` (ersatztv#711). `.codex/hooks/` is a
|
||||
# byte-identical mirror of `.claude/hooks/` — including `pretooluse-merge-consent.sh` —
|
||||
# generated as the Codex-side port of the same enforcement hooks. Without it the rule "a PR
|
||||
# that can weaken the gate must not exempt itself from the gate" was expressed as a path
|
||||
# list that had gone incomplete: editing `.claude/hooks/pretooluse-merge-consent.sh`
|
||||
# correctly voided the exemption while editing its `.codex/` twin did not. Today that is
|
||||
# LATENT rather than live — `.codex/` is untracked and gitignored, and a PR cannot touch a
|
||||
# path that is not in the repo — but it becomes live the moment anyone tracks it, which is
|
||||
# the natural instinct given `.claude/` is tracked. Listed now because the cost is one
|
||||
# alternation and the failure mode is silent.
|
||||
#
|
||||
# The list stays ENUMERATIVE rather than derived (e.g. "any dotted top-level directory
|
||||
# containing executable hooks"). A derived rule has to be evaluated against the PR's own
|
||||
# file list, which is the very thing being classified — more moving parts inside a security
|
||||
# predicate, to remove a maintenance burden that is one line per new tooling directory.
|
||||
PROTECTED='^(\.claude/|\.codex/|\.gitea/|\.husky/|scripts/|docker/ci/)'
|
||||
# Docs-only: prose and decision records. Deliberately narrower than the hook's pattern,
|
||||
# which also lets .claude/.gitea/.husky through — that carve-out is safe there only
|
||||
# because it falls through to a HUMAN PROMPT, whereas here it would post a green status
|
||||
# with nobody in the loop.
|
||||
DOCS_ONLY='^(docs/|[^/]*\.md$)'
|
||||
|
||||
if [ -z "${GITEA_TOKEN:-}" ]; then
|
||||
echo "::error::No GITEA_TOKEN available, so the ${CONTEXT} status cannot be written. An exempt (bot/docs-only) PR will stall until this is fixed; a normal PR is unaffected — post its verdict with scripts/post-review-verdict.sh."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
gh() { curl -sf -H "Authorization: token $GITEA_TOKEN" "$@"; }
|
||||
|
||||
# DEFINED HERE, BEFORE ANY USE. An earlier round defined these AFTER the classification
|
||||
# chain that calls them, so `count_matching` was `command not found` on every run, the
|
||||
# PROTECTED branch silently never fired, and three "protected path" tests still passed —
|
||||
# they reached `pending` by another route, so the guard being dead was invisible.
|
||||
#
|
||||
# HOW THE PATH PREDICATES ARE EVALUATED, and why neither obvious spelling is used.
|
||||
#
|
||||
# `producer | grep -q…` is FORBIDDEN here: `grep -q` exits at its first match, the producer
|
||||
# then takes SIGPIPE and exits 141 once the list exceeds the pipe buffer, and under
|
||||
# `set -o pipefail` the pipeline is a FAILURE even though grep MATCHED — inverting the guard
|
||||
# for exactly the large PRs that matter. Reproduced with `A.cs` + 1900 docs paths (171KB,
|
||||
# inside the enumerator's 2000-file cap): `docs_only=yes`, status 141; and a `.gitea/` path
|
||||
# made `PROTECTED` MISS. That construct predates #698 and was live on `main`.
|
||||
#
|
||||
# A here-string (`grep -q… <<< "$files"`) fixes the SIGPIPE but bash materialises a large
|
||||
# here-string via TEMPORARY STORAGE, so it can fail when the runner's temp space is full or
|
||||
# unwritable — and because these run inside `if`/`!`, that failure would flip the predicate
|
||||
# the same way. Trading a buffer bug for an environmental one is not a fix.
|
||||
#
|
||||
# So: count with `grep -c`, which DRAINS stdin (no early exit, no SIGPIPE) over an ordinary
|
||||
# pipe (no temp file), and treat grep's own exit status honestly — `grep -c` exits 1 when the
|
||||
# count is zero, which is a legitimate answer, while anything >1 is a real error and must FAIL
|
||||
# THE JOB rather than silently read as "no match". `set -e` would not catch these on its own
|
||||
# because they sit inside command substitution in a conditional.
|
||||
count_matching() { # how many lines of $2 match $1
|
||||
local out st=0
|
||||
out=$(printf '%s\n' "$2" | grep -cE "$1") || st=$?
|
||||
# NOT `exit 1`: these run inside `$( )`, so an exit leaves only the SUBSHELL and, because
|
||||
# the substitution sits in a conditional, `set -e` does not fire either — the job would sail
|
||||
# on with the predicate silently reading as "no match". Emit a NON-NUMERIC sentinel instead
|
||||
# and let the caller, at top level, refuse to classify.
|
||||
if [ "$st" -gt 1 ]; then
|
||||
echo "::error::grep failed (status ${st}) evaluating a path predicate." >&2
|
||||
printf 'ERR'
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "${out:-0}"
|
||||
}
|
||||
count_not_matching() { # how many lines of $2 do NOT match $1
|
||||
local out st=0
|
||||
out=$(printf '%s\n' "$2" | grep -cvE "$1") || st=$?
|
||||
if [ "$st" -gt 1 ]; then
|
||||
echo "::error::grep failed (status ${st}) evaluating a path predicate." >&2
|
||||
printf 'ERR'
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "${out:-0}"
|
||||
}
|
||||
|
||||
|
||||
# --- Is there already a verdict for THIS sha? ----------------------------------------
|
||||
# NOTE the heading no longer says "never overwrite". It cannot promise that: the read below
|
||||
# and the POST at the end of this job are not atomic, so a human verdict posted in between is
|
||||
# still overwritten. The re-read immediately before the POST narrows that window; it does not
|
||||
# close it. Tracked as ersatztv#706 rather than claimed as solved.
|
||||
# Reads the CONTEXT row for $SHA and sets ex_state / ex_creator / ex_desc / ex_human.
|
||||
# Factored into a function because it is now called TWICE — once here, and once immediately
|
||||
# before the POST (see below). An unreadable/unparseable response must NOT be read as "no
|
||||
# verdict exists": the job dies WITHOUT posting, so a transient API error can never overwrite
|
||||
# a verdict.
|
||||
#
|
||||
# The empty case is checked EXPLICITLY, not left to jq's exit status: `jq -e` over empty input
|
||||
# exits 4 on jq >= 1.7 but 0 on jq 1.6, and THE RUNNER SHIPS 1.6 (ersatztv#647) — so on a
|
||||
# transient error this guard passed, the row came back "", and the job posted over a
|
||||
# possibly-existing human verdict.
|
||||
#
|
||||
# The COMBINED endpoint is read, not `/statuses/{sha}`: the latter returns one row per POST
|
||||
# (not per context) and pages at 50, so a head with a few CI reruns can push an earlier verdict
|
||||
# off the first page.
|
||||
read_existing_verdict() {
|
||||
local json row
|
||||
json=$(gh "$BASE_URL/repos/$REPO/commits/$SHA/status?limit=100") || json=""
|
||||
if [ -z "${json//[[:space:]]/}" ] || ! printf '%s' "$json" | jq -e '.statuses | type == "array"' >/dev/null 2>&1; then
|
||||
echo "::error::Could not read existing commit statuses for ${SHA:0:7}. Refusing to post anything rather than risk overwriting an existing verdict."
|
||||
exit 1
|
||||
fi
|
||||
row=$(printf '%s' "$json" | jq -r --arg c "$CONTEXT" '[.statuses[] | select(.context == $c)] | first // {}')
|
||||
ex_state=$(printf '%s' "$row" | jq -r '.status // ""')
|
||||
ex_creator=$(printf '%s' "$row" | jq -r '.creator.login // ""')
|
||||
ex_desc=$(printf '%s' "$row" | jq -r '.description // ""')
|
||||
# A `case` prefix test rather than grep: the description is a single short string, and this
|
||||
# removes one more pipeline from a security predicate entirely. The PATTERN is a literal, so
|
||||
# there is no glob-injection concern from $ex_desc.
|
||||
# A human verdict also has to have been formed against THIS base (ersatztv#698, found in
|
||||
# round-4 review). `post-review-verdict.sh` records the base it reviewed in the status
|
||||
# description — `Review-verdict: MERGEABLE @ abc1234 (base: main)` — precisely because
|
||||
# retargeting changes the effective diff without moving the head sha (ersatztv#632).
|
||||
# Without this check the sha-binding is escapable through the HUMAN path rather than the
|
||||
# exemption path: get a genuine `success` on head H while it targets a scratch base S with
|
||||
# a benign diff, then retarget H onto `main`, where its diff contains unreviewed code. The
|
||||
# status is real, its creator is real, and it was silently inherited. The merge-consent
|
||||
# hook compares the base and would object, but that is advisory and covers only its own
|
||||
# path — a merge through the Gitea UI or API just sees a green required check.
|
||||
#
|
||||
# An ABSENT base is deliberately NOT treated as a mismatch: verdicts predating #632 carry
|
||||
# no `(base: …)`, and re-deriving over one would un-approve a genuinely reviewed head. Only
|
||||
# a base that is PRESENT and DIFFERENT is rejected, which is exactly the escape above.
|
||||
ex_human=no
|
||||
ex_repair=no
|
||||
case "$ex_desc" in
|
||||
"$REPAIR_DESC"*) ex_repair=yes ;;
|
||||
esac
|
||||
case "$ex_desc" in
|
||||
"Review-verdict:"*)
|
||||
if [ -n "$ex_creator" ]; then ex_human=yes; fi
|
||||
;;
|
||||
esac
|
||||
if [ "$ex_human" = yes ]; then
|
||||
# COMPARE, NEVER PARSE. Two earlier attempts both extracted the base out of the
|
||||
# description and both were defeated, the second in a way that looked like a fix for the
|
||||
# first:
|
||||
# * `${ex_desc##*"(base: "}` (LAST occurrence) let an APPENDED `(base: main)` override a
|
||||
# genuine `(base: probe/scratch)`;
|
||||
# * `${ex_desc#*"(base: "}` (FIRST occurrence) fixed that, but `${...%%)*}` still
|
||||
# truncates at the first `)`. `main)evil` IS A VALID GIT BRANCH NAME
|
||||
# (`git check-ref-format --branch 'main)evil'` succeeds), so a verdict earned while
|
||||
# targeting it reads `(base: main)evil)`, truncates to exactly `main`, and is
|
||||
# INHERITED after retargeting onto `main`. No forged description, no #697 needed.
|
||||
# The comment here previously asserted a `)` in a branch name "mismatches — safe
|
||||
# direction"; that was generalised from `feat/foo)bar` and is FALSE for any branch
|
||||
# whose name starts with the target base.
|
||||
#
|
||||
# So extract nothing. `post-review-verdict.sh` writes the marker LAST, so require the
|
||||
# description to END with the exact literal `(base: <this PR's base>)` and to contain
|
||||
# exactly ONE marker — which kills the append trick without having to decide which
|
||||
# occurrence is authoritative. Pure shell; no truncation exists to abuse.
|
||||
#
|
||||
# `${#}` arithmetic rather than a `grep -o | wc -l` pipeline; 7 is the length of
|
||||
# "(base: ". An ABSENT marker is still not a mismatch (verdicts predate #632).
|
||||
ex_stripped=${ex_desc//"(base: "/}
|
||||
ex_markers=$(( (${#ex_desc} - ${#ex_stripped}) / 7 ))
|
||||
if [ "$ex_markers" -ne 0 ]; then
|
||||
ex_base_ok=no
|
||||
if [ "$ex_markers" -eq 1 ]; then
|
||||
case "$ex_desc" in
|
||||
*"(base: $BASE_REF)") ex_base_ok=yes ;;
|
||||
esac
|
||||
fi
|
||||
if [ "$ex_base_ok" != yes ]; then
|
||||
ex_human=no
|
||||
echo "${CONTEXT} on ${SHA:0:7} is a human verdict, but its recorded base does not match this PR's base '${BASE_REF}' (description: ${ex_desc}) — the reviewed diff is not this PR's diff, so it is NOT treated as a verdict for this base."
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# --- The retarget fence (ersatztv#706 race 1) ----------------------------------------
|
||||
# THE PROBLEM THIS SOLVES. Two `pull_request_target` runs for one PR overlap, and the OLDER
|
||||
# one can finish LAST — so a run that classified against a base the PR no longer targets can
|
||||
# post its stale answer over a fresher run's correct one, permanently. Measured on this
|
||||
# instance rather than assumed: probe PR #722 run 7520 (`opened`) ran to completion 20s AFTER
|
||||
# run 7521 (`synchronize`) had started.
|
||||
#
|
||||
# WHY NOT A CONCURRENCY GROUP, which is the obvious answer and what #706 proposed. It does
|
||||
# not work here, also measured: with `concurrency: {group: …-${{ pr number }},
|
||||
# cancel-in-progress: false}` active on an identical probe, runs 7528 and 7529 still ran
|
||||
# CONCURRENTLY and 7528 ended 36s after 7529 began. Gitea 1.25.4 does auto-cancel superseded
|
||||
# `push` runs on a branch — a negative control with no `concurrency:` key at all showed that —
|
||||
# but that behaviour does NOT extend to `pull_request_target`. `cancel-in-progress: true` is
|
||||
# deliberately untried: cancellation is the one thing this workflow's own header refuses,
|
||||
# because a cancelled run leaves an EXEMPT PR statusless with nothing left to re-trigger it.
|
||||
#
|
||||
# WHY A COUNTER AND NOT THE BRANCH NAME. The attack is an ABA: `main → S → main`. Every
|
||||
# name-based check reads `main` at both ends and passes, which is exactly how route 1 got a
|
||||
# forged exemption. Gitea's issue timeline records each retarget as a `change_target_branch`
|
||||
# event with `old_ref`/`new_ref`; the COUNT of those events is monotonic and cannot alias.
|
||||
# Verified on the real route-1 reproduction (PR #703: two events, `main → probe698/base-S` at
|
||||
# 18:17:29 and back at 18:18:31) with a negative control (PR #717, never retargeted: zero).
|
||||
#
|
||||
# WHY ABSTAINING IS NOT A STALL — the property the whole design rests on. A retarget always
|
||||
# fires `edited`, which is in this workflow's `types:` (see the header). So the very event
|
||||
# that makes this run abstain has already queued a successor whose window opens after it.
|
||||
# Abstention hands off; it does not drop the PR. The induction terminates when retargeting
|
||||
# stops, and the last run has a clean window and writes the final answer. This is why the
|
||||
# fence does not reintroduce the statusless-exempt-PR failure that rules out cancellation:
|
||||
# it never stops a run from RUNNING, only from WRITING state it knows is stale.
|
||||
#
|
||||
# `updated_at` was considered as the key and rejected: it moves for comments and labels,
|
||||
# which fire none of this workflow's `types:`, so a run could abstain with no successor
|
||||
# coming — a real stall. The retarget count moves only for the mutation that actually
|
||||
# invalidates a classification, and that mutation always brings its own re-run.
|
||||
#
|
||||
# Completeness is a guard, not an assumption (`ci.paged-endpoint-completeness`): the count is
|
||||
# trusted ONLY when paging reached a validated EMPTY page. A short page, a non-array body, a
|
||||
# non-numeric length or the page cap all leave `rt_ok=no`, and an untrusted count is treated
|
||||
# below as "cannot tell" rather than as zero.
|
||||
count_retargets() {
|
||||
rt_count=0
|
||||
rt_ok=no
|
||||
local page=1 raw n m total=0
|
||||
while [ "$page" -le 20 ]; do
|
||||
raw=$(gh "$BASE_URL/repos/$REPO/issues/$PR/timeline?limit=50&page=${page}") || return 0
|
||||
if [ -z "${raw//[[:space:]]/}" ] || ! printf '%s' "$raw" | jq -e 'type == "array"' >/dev/null 2>&1; then
|
||||
return 0
|
||||
fi
|
||||
n=$(printf '%s' "$raw" | jq -r 'length')
|
||||
case "$n" in ''|*[!0-9]*) return 0 ;; esac
|
||||
if [ "$n" -eq 0 ]; then rt_ok=yes; rt_count=$total; return 0; fi
|
||||
m=$(printf '%s' "$raw" | jq -r '[.[] | select(.type == "change_target_branch")] | length')
|
||||
case "$m" in ''|*[!0-9]*) return 0 ;; esac
|
||||
total=$(( total + m ))
|
||||
page=$(( page + 1 ))
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
ex_repair=no
|
||||
count_retargets
|
||||
retargets_before=$rt_count
|
||||
retargets_before_ok=$rt_ok
|
||||
echo "Retarget fence: ${retargets_before} retarget event(s) observed before classifying (trusted=${retargets_before_ok})."
|
||||
|
||||
# --- Whose verdict is it? (ersatztv#698 route 3) -------------------------------------
|
||||
# This short-circuit used to exit on ANY existing `success`, which made an exemption this job
|
||||
# wrote indistinguishable from a verdict a human wrote. That is what let a forged exemption
|
||||
# survive: obtained once — via the route-1 retarget race, a sibling workflow holding
|
||||
# status-write credentials (#697), a direct API call, or inheritance across PRs by sha (#663)
|
||||
# — it was thereafter accepted unchanged on every run, because the guard exited before it
|
||||
# looked at the PR, the base, the author or the files.
|
||||
#
|
||||
# The guard still exists for its original reason: re-posting `pending` over a real human
|
||||
# verdict would un-approve a reviewed head and stall the PR. So it discriminates by PROVENANCE.
|
||||
#
|
||||
# MEASURED on this instance (Gitea 1.25.4), on the COMBINED endpoint this job reads: a status
|
||||
# POSTed with a USER credential — how `scripts/post-review-verdict.sh` writes a verdict —
|
||||
# carries `.creator.login`, while one POSTed by an Actions job with the built-in `GITEA_TOKEN`
|
||||
# carries `"creator": null`. A real verdict read back `creator=timothy`; this job's own
|
||||
# exemption read back `creator=null`.
|
||||
#
|
||||
# BOTH conditions are required, and the DIRECTION of the test is the point: we short-circuit
|
||||
# only on something POSITIVELY identified as a human verdict. Anything else, including anything
|
||||
# we do not recognise, is RE-DERIVED. Written the other way round ("skip if it looks
|
||||
# machine-written") an unrecognised shape would be trusted — the fail-open this issue is about.
|
||||
#
|
||||
# What this does NOT claim: the test asks "was this POSTed by a user credential", NOT "by a
|
||||
# reviewer". `ETV_STATUS_AUTH` is basic auth, so head-controlled code can POST a success with a
|
||||
# non-null creator AND an attacker-chosen `Review-verdict:` description, which this guard then
|
||||
# preserves. That is #697 — provenance, not authentication.
|
||||
read_existing_verdict
|
||||
if [ "$ex_human" = yes ] && { [ "$ex_state" = "success" ] || [ "$ex_state" = "failure" ]; }; then
|
||||
echo "${CONTEXT} is already '${ex_state}' on ${SHA:0:7}, written by '${ex_creator}' as a human verdict — leaving it alone."
|
||||
exit 0
|
||||
fi
|
||||
if [ -n "$ex_state" ]; then
|
||||
echo "${CONTEXT} is '${ex_state}' on ${SHA:0:7} but is NOT an attributable human verdict (creator='${ex_creator:-null}', description='${ex_desc}') — re-deriving it from the PR's current state rather than inheriting it."
|
||||
fi
|
||||
|
||||
# --- Changed files: the SHARED enumeration, or no exemption. -------------------------
|
||||
# `scripts/pr-changed-files.sh` (from the BASE checkout) owns every guard this job used to
|
||||
# carry inline and six it did not: CR/LF rejection, `..` rejection, a closed `.status`
|
||||
# allow-list, `previous_filename` validated on EVERY row rather than only `renamed` ones,
|
||||
# termination only on a validated EMPTY page rather than a merely short one, and head-sha
|
||||
# binding across the paging round-trips. ersatztv#649.
|
||||
#
|
||||
# READ THE EXIT STATUS, NEVER THE STDOUT OF A FAILED RUN. exit 0 means "complete and bound
|
||||
# to $SHA"; anything else means "could not tell" and stdout is meaningless. That the
|
||||
# script happens to print nothing on its failure paths is redundancy, not contract —
|
||||
# `files` is therefore cleared explicitly rather than trusted to be empty. stderr is left
|
||||
# attached to the job log on purpose: its diagnostic is the only thing that distinguishes
|
||||
# a force-push mid-enumeration from a dead API.
|
||||
ENUM=./scripts/pr-changed-files.sh
|
||||
files=""
|
||||
complete=no
|
||||
enum_error=""
|
||||
if [ ! -x "$ENUM" ]; then
|
||||
# Only reachable for a PR whose BASE predates ersatztv#658. Fail closed with a readable
|
||||
# status rather than an absent one, so the PR shows why instead of stalling silently.
|
||||
enum_error="the PR's base ref (${BASE_SHA:0:7}) has no executable ${ENUM}"
|
||||
elif files=$("$ENUM" "${REPO%%/*}" "${REPO#*/}" "$PR" "$SHA" "$BASE_REF"); then
|
||||
complete=yes
|
||||
else
|
||||
files=""
|
||||
enum_error="scripts/pr-changed-files.sh could not enumerate PR #${PR} at ${SHA:0:7} exhaustively (see the step log)"
|
||||
fi
|
||||
|
||||
files=$(printf '%s\n' "$files" | grep -v '^$' || true)
|
||||
count=$(printf '%s\n' "$files" | grep -c . || true)
|
||||
echo "Changed files (${count}, complete=${complete}):"
|
||||
printf '%s\n' "$files" | sed 's/^/ /'
|
||||
|
||||
# Evaluated ONCE, at TOP LEVEL, so a failure can actually stop the job. Evaluating them
|
||||
# inline inside the `if`/`elif` chain is what hid the two defects above: a bad status or a
|
||||
# missing function turned into an empty string, `[ "" -gt 0 ]` errored, and the branch was
|
||||
# simply skipped. A non-numeric result here is fatal and posts nothing — an absent required
|
||||
# check blocks the merge, which is the correct direction.
|
||||
n_protected=$(count_matching "$PROTECTED" "$files")
|
||||
n_not_manifest=$(count_not_matching "$BOT_MANIFESTS" "$files")
|
||||
n_not_docs=$(count_not_matching "$DOCS_ONLY" "$files")
|
||||
for v in "$n_protected" "$n_not_manifest" "$n_not_docs"; do
|
||||
case "$v" in
|
||||
''|*[!0-9]*)
|
||||
echo "::error::A path predicate returned '${v}' instead of a count — the classifier is not operating, so no ${CONTEXT} status will be written for ${SHA:0:7}."
|
||||
exit 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
exempt=no
|
||||
reason=""
|
||||
if [ "$complete" != yes ]; then
|
||||
reason="${enum_error} — no exemption"
|
||||
elif [ "${count:-0}" -eq 0 ]; then
|
||||
reason="no changed files could be read from the API — no exemption"
|
||||
elif [ "$n_protected" -gt 0 ]; then
|
||||
reason="touches a protected path (gate/CI/hooks/scripts/ci-image) — exemptions do not apply"
|
||||
else
|
||||
# The two exemptions are evaluated as INDEPENDENT predicates rather than as a chain.
|
||||
# An `elif` chain was wrong once the bot exemption gained a second condition
|
||||
# (ersatztv#698 route 2): a Renovate PR that changes only `docs/` would enter the bot
|
||||
# branch, fail the manifest test, and never reach the docs-only branch at all — silently
|
||||
# withdrawing an exemption that the docs-only rule grants on its own merits, for any
|
||||
# author. Composing the predicates and deciding afterwards keeps each rule's meaning
|
||||
# independent of the order they happen to be written in.
|
||||
#
|
||||
# `grep -qv` asks "is there any line NOT in this allow-list", so an unrecognised path
|
||||
# withholds the exemption instead of being ignored — the same closed-set direction the
|
||||
# enumeration itself uses. Both are safe against an empty `$files` because `count -eq 0`
|
||||
# is handled above.
|
||||
# Written as `if`/`then`, never as `cmd && var=yes`: under `set -e` a bare `A && B`
|
||||
# statement whose `A` fails takes the failure as the statement's own exit status and
|
||||
# kills the job. That would fail closed here (no status posted, absent required check
|
||||
# blocks the merge) but it would do so on the ORDINARY path — every non-bot PR — so the
|
||||
# gate would look broken rather than strict. `cmd || var=yes` is safe for the same
|
||||
# reason it is confusing; both are spelled out instead.
|
||||
# BOTS is a short fixed literal, so it cannot reach the pipe buffer; it is still written
|
||||
# with an explicit status capture so a grep error cannot read as "not a bot" by accident.
|
||||
is_bot=no
|
||||
bot_hits=$(printf '%s\n' "$BOTS" | tr ' ' '\n' | grep -cxF "$AUTHOR") || bot_hits=0
|
||||
if [ "${bot_hits:-0}" -gt 0 ]; then is_bot=yes; fi
|
||||
manifests_only=no
|
||||
if [ "$n_not_manifest" -eq 0 ]; then manifests_only=yes; fi
|
||||
docs_only=no
|
||||
if [ "$n_not_docs" -eq 0 ]; then docs_only=yes; fi
|
||||
|
||||
if [ "$is_bot" = yes ] && [ "$manifests_only" = yes ]; then
|
||||
exempt=yes
|
||||
reason="authored by the '$AUTHOR' bot account, touches no protected path, and changes only dependency manifests"
|
||||
elif [ "$docs_only" = yes ]; then
|
||||
exempt=yes
|
||||
reason="docs-only change (no code, no protected path)"
|
||||
elif [ "$is_bot" = yes ]; then
|
||||
reason="authored by the '$AUTHOR' bot account, but changes files outside the dependency-manifest set — a bot ACCOUNT does not attribute the CODE at this head (the account is the PR's immutable creator; the head is not), so this needs a real verdict"
|
||||
else
|
||||
reason="awaiting an H10 review verdict for head ${SHA:0:7}"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$ex_repair" = yes ]; then
|
||||
# A previous run of this job already repaired a raced exemption on this sha, which means a
|
||||
# human verdict was written for it and then lost. Re-granting the exemption would bury that
|
||||
# rejection again. The PR needs a real verdict; only a human can clear this.
|
||||
exempt=no
|
||||
reason="a human verdict raced a previous exemption write on this head and was overwritten — this head needs a re-posted verdict, not another exemption"
|
||||
fi
|
||||
if [ "$exempt" = yes ]; then
|
||||
state=success
|
||||
desc="Exempt: $reason"
|
||||
elif [ "$ex_repair" = yes ]; then
|
||||
# CARRY THE SENTINEL FORWARD. This branch exists because the first version of it did not,
|
||||
# and cold review reproduced the consequence: refusing the exemption but posting the
|
||||
# GENERIC pending description overwrote the very sentinel the refusal depends on, so the
|
||||
# next run saw an ordinary machine `pending`, re-derived it, and posted `success` — burying
|
||||
# the human rejection two events after the repair instead of one. The block has to be a
|
||||
# FIXED POINT: what this branch writes must be what re-triggers this branch.
|
||||
#
|
||||
# It is keyed on `ex_repair` alone rather than on the exempt path, so the marker also
|
||||
# survives runs where the PR was not exemptible anyway — the fact being recorded is "a
|
||||
# human verdict was lost on this sha", which is a property of the sha, not of this run's
|
||||
# classification.
|
||||
state=pending
|
||||
desc="$REPAIR_DESC"
|
||||
else
|
||||
state=pending
|
||||
desc="Awaiting review verdict for ${SHA:0:7}"
|
||||
fi
|
||||
echo "Decision: state=${state} — ${reason}"
|
||||
|
||||
# HIGH-WATER MARK for the post-write verification (ersatztv#706 race 2). Taken FIRST — before
|
||||
# the re-read below, before the fence, before the POST — and the ORDER IS THE POINT.
|
||||
#
|
||||
# An earlier version captured it just before the POST, "as late as possible". Cold review
|
||||
# caught that as a High: everything between the re-read and a late mark is a blind gap. A
|
||||
# human verdict landing there is invisible to the re-read (which already happened) AND
|
||||
# excluded from the post-write check (its id is BELOW a mark taken afterwards), so it is
|
||||
# silently overwritten with no repair. That gap spans the entire retarget re-count — up to 20
|
||||
# timeline round-trips — so it was far wider than the one-round-trip residual being claimed.
|
||||
#
|
||||
# Taking the mark first closes the read side completely: any row newer than the mark is caught
|
||||
# either by the re-read (abstain, post nothing) or by the post-write check (repair). There is
|
||||
# no false-fire cost to being early, because the test is `id > mark` — rows already present
|
||||
# when the mark is taken are below it and stay invisible either way.
|
||||
#
|
||||
# Presence alone would be the wrong test, and wrong in the direction that breaks the gate: the
|
||||
# short-circuit deliberately does NOT stop for a human verdict whose recorded base does not
|
||||
# match this PR's (`ex_human` is reset to `no` — see `read_existing_verdict`). Such a row stays
|
||||
# in the history forever, so a presence test would fire on EVERY later run of that PR,
|
||||
# downgrade every exemption to `pending`, and deadlock it permanently.
|
||||
#
|
||||
# `max` over an empty array is `null`, hence `// 0`. `.id? // 0` rather than `.id`: a bare
|
||||
# `.[].id` hard-errors under `set -e` if the array ever holds a non-object, which would kill
|
||||
# the job before it posts and strand an ordinary PR with no status at all.
|
||||
max_id_before=-1
|
||||
hist_before=$(gh "$BASE_URL/repos/$REPO/statuses/$SHA?limit=100") || hist_before=""
|
||||
if [ -n "${hist_before//[[:space:]]/}" ] && printf '%s' "$hist_before" | jq -e 'type == "array"' >/dev/null 2>&1; then
|
||||
mark=$(printf '%s' "$hist_before" | jq -r '[.[] | .id? // 0] | max // 0' 2>/dev/null || true)
|
||||
case "$mark" in
|
||||
''|*[!0-9]*)
|
||||
# SKIP the check rather than treat everything as raced. A mark of 0 would make every
|
||||
# pre-existing human row look newer than the mark and repair every exemption away.
|
||||
echo "::warning::Status high-water mark for ${SHA:0:7} was not numeric ('${mark}'); the post-write race check will be skipped."
|
||||
max_id_before=-1 ;;
|
||||
*) max_id_before=$mark ;;
|
||||
esac
|
||||
else
|
||||
# Not fatal: the POST below is still correct, only the after-the-fact verification is
|
||||
# weakened. Recorded so a silent degradation is visible in the log.
|
||||
echo "::warning::Could not establish a status high-water mark for ${SHA:0:7}; the post-write race check will be skipped."
|
||||
max_id_before=-1
|
||||
fi
|
||||
|
||||
# LAST-MOMENT RE-READ (ersatztv#706). Classification takes several API round-trips, and a
|
||||
# reviewer can post a verdict during them — most dangerously a `failure`, which this job would
|
||||
# then overwrite with an exemption `success`, turning an explicit human rejection green. The
|
||||
# first read cannot see that; this one can. It NARROWS the window, it does not close it: there
|
||||
# is no compare-and-set on Gitea's status API, so a verdict landing between this read and the
|
||||
# POST below is still lost — which is what the post-write repair below is for.
|
||||
read_existing_verdict
|
||||
if [ "$ex_human" = yes ]; then
|
||||
echo "::notice::A human verdict ('${ex_state}' by '${ex_creator}') landed on ${SHA:0:7} while this job was classifying — leaving it alone and posting nothing."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# A SENTINEL THAT APPEARED MID-RUN (ersatztv#706, round-3 review). The re-read above recomputes
|
||||
# `ex_repair`, and until this guard existed nothing downstream read it: the POST writes the
|
||||
# `$state` frozen at classification time, so a STALE OVERLAPPING RUN would post its `success`
|
||||
# straight over a sentinel another run had just written — burying a human rejection, with no
|
||||
# repair (the human row is below this run's mark) and no log. That fails toward SUCCESS, so it
|
||||
# was not covered by the "repair fails toward pending" residual; it is the exact outcome this
|
||||
# whole change exists to prevent, reached through the run overlap this branch itself measured.
|
||||
#
|
||||
# THE RULE IS "NEVER REPLACE A SENTINEL WITH A NON-SENTINEL", not "never overwrite it with a
|
||||
# success". A first draft of this guard tested `state = success`, which is one branch too
|
||||
# narrow: a run can reach the POST on `state=pending` carrying the GENERIC description — most
|
||||
# realistically after a transient enumeration failure (`complete != yes`) — and that run
|
||||
# passes a success-only guard, passes the fence, and overwrites the sentinel with ordinary
|
||||
# text. The next run then sees no sentinel, re-derives, and posts `success`: the same buried
|
||||
# human rejection as before, reached in two steps instead of one.
|
||||
#
|
||||
# Comparing the DESCRIPTION rather than the state is exactly as precise and strictly more
|
||||
# general. A sentinel present at the FIRST read forces `desc="$REPAIR_DESC"` (the carry-forward
|
||||
# branch in the decision above), so this guard cannot fire on the ordinary repaired-head path
|
||||
# and the fixed point is intact. Any other description alongside `ex_repair=yes` means the
|
||||
# sentinel arrived DURING this run, whatever this run concluded.
|
||||
#
|
||||
# Abstaining is strictly correct here and, unlike the retarget fence, needs no successor run:
|
||||
# the sentinel row is already `pending` and already carries the re-post instruction.
|
||||
if [ "$ex_repair" = yes ] && [ "$desc" != "$REPAIR_DESC" ]; then
|
||||
echo "::notice::A repair sentinel was written on ${SHA:0:7} while this job was classifying, meaning a human verdict was raced and repaired by another run. This run's exemption is stale — posting NOTHING and leaving the sentinel standing."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# THE FENCE ITSELF (ersatztv#706 race 1). Re-count the retargets as late as possible and
|
||||
# refuse to write anything if the PR was retargeted since this run began. See the long note
|
||||
# at `count_retargets` for why this is a handoff rather than a stall, and why the count is
|
||||
# the only key that survives an ABA.
|
||||
#
|
||||
# The refusal covers `pending` as well as `success`, not just the dangerous write. A stale
|
||||
# `pending` over a fresh `success` is only a stall rather than a forged green, so gating it
|
||||
# is not strictly required — but the successor run is guaranteed either way, so there is
|
||||
# nothing to buy by writing a value this run already knows was computed against a base the
|
||||
# PR no longer targets. One rule, one direction, nothing to reason about per state.
|
||||
#
|
||||
# An UNTRUSTED count on either side (`rt_ok=no`: paging never reached a validated empty
|
||||
# page, a page was unreadable, the cap was hit) is NOT treated as "no retarget". It blocks
|
||||
# the exemption `success` only, and lets `pending` through: `pending` cannot turn a rejection
|
||||
# or an unreviewed head green, so withholding it would strand PRs for no safety gain, while
|
||||
# a `success` written on a count we could not verify is exactly the forged-green outcome
|
||||
# this fence exists to prevent.
|
||||
count_retargets
|
||||
if [ "$retargets_before_ok" = yes ] && [ "$rt_ok" = yes ] && [ "$rt_count" -ne "$retargets_before" ]; then
|
||||
echo "::notice::PR #${PR} was retargeted while this job was classifying (${retargets_before} -> ${rt_count} retarget events). This run's classification was computed against a base the PR may no longer target, so it posts NOTHING. The retarget fired an 'edited' event, so a successor run is already queued and will write the authoritative status for ${SHA:0:7}."
|
||||
exit 0
|
||||
fi
|
||||
if { [ "$retargets_before_ok" != yes ] || [ "$rt_ok" != yes ]; } && [ "$state" = "success" ]; then
|
||||
echo "::error::Could not establish a trusted retarget count for PR #${PR} (before=${retargets_before_ok}, after=${rt_ok}), so an exemption 'success' cannot be shown to have been computed against the PR's current base. Posting nothing; ${CONTEXT} stays absent, which blocks the merge. NOTE a later run only helps if the cause was transient — a PR whose timeline exceeds the page cap will fail this way on every run, and needs a human verdict."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
payload=$(jq -n --arg s "$state" --arg c "$CONTEXT" --arg d "$desc" --arg u "$PR_URL" \
|
||||
'{state:$s, context:$c, description:$d, target_url:$u}')
|
||||
gh -X POST -H 'Content-Type: application/json' -d "$payload" \
|
||||
"$BASE_URL/repos/$REPO/statuses/$SHA" >/dev/null
|
||||
echo "Posted ${CONTEXT}=${state} on ${SHA:0:7}."
|
||||
|
||||
# --- POST-POST VERIFICATION (ersatztv#706 race 2) ------------------------------------
|
||||
# The last-moment re-read above narrows the window between reading and writing; it cannot
|
||||
# close it, because Gitea's status API has no conditional write (no ETag, no If-Match, no
|
||||
# expected-previous-state), so there is no compare-and-set to make the read and the POST one
|
||||
# operation. A human `failure` landing in that remaining gap is overwritten by the POST above
|
||||
# — turning an explicit human REJECTION green, which is the worst outcome this gate can
|
||||
# produce and strictly worse than any stall.
|
||||
#
|
||||
# So verify AFTERWARDS and repair in the safe direction. This runs ONLY on the `success`
|
||||
# path, and that restriction is the point rather than an optimisation: `pending` cannot
|
||||
# turn a rejection green — it already blocks the merge — so the only write that can cause
|
||||
# the damage is the exemption `success`.
|
||||
#
|
||||
# WHY A DIFFERENT ENDPOINT. Everywhere else this job reads the COMBINED endpoint
|
||||
# (`/commits/{sha}/status`), which returns the LATEST status per context — and that is now
|
||||
# OUR success, with the human's row buried underneath it. The combined view is therefore
|
||||
# structurally incapable of showing the thing being looked for. `/statuses/{sha}` returns one
|
||||
# row per POST instead. Measured on this instance, the two really do differ in shape as well
|
||||
# as content: the combined endpoint returns an OBJECT with a `.statuses` array (12 rows on a
|
||||
# live head), `/statuses/{sha}` a BARE ARRAY (24 rows on the same head) — hence the different
|
||||
# `type == "array"` guard here.
|
||||
#
|
||||
# No claim is made about the order rows come back in, because the check does not depend on
|
||||
# it: it selects by id against the high-water mark rather than inspecting the top of the
|
||||
# list. A verdict older than the mark is invisible to it no matter where it sits.
|
||||
#
|
||||
# The repair is `pending`, NEVER a copy of the human's state. Re-posting their `failure`
|
||||
# would attribute a human verdict to this job — the exact provenance confusion the
|
||||
# `creator`-based short-circuit above exists to prevent, and it would be written with the
|
||||
# machine credential, so it would read as machine-derived to every later run. `pending`
|
||||
# asserts nothing about the review: it blocks the merge and asks for a real verdict, which
|
||||
# is true and safe regardless of which way the human ruled. The reviewer is told to re-post.
|
||||
#
|
||||
# A read failure here does NOT fail the job: the status is already posted, so `exit 1` would
|
||||
# change nothing about the gate's state while turning a routine API hiccup into a red run.
|
||||
# It is reported loudly and left alone — the residual is the read/POST gap either way.
|
||||
if [ "$state" = "success" ] && [ "$max_id_before" -ge 0 ]; then
|
||||
post_hist=$(gh "$BASE_URL/repos/$REPO/statuses/$SHA?limit=100") || post_hist=""
|
||||
if [ -z "${post_hist//[[:space:]]/}" ] || ! printf '%s' "$post_hist" | jq -e 'type == "array"' >/dev/null 2>&1; then
|
||||
echo "::warning::Could not re-read the status history for ${SHA:0:7} after posting, so a human verdict landing during the write window would not be detected. The exemption ${CONTEXT}=success stands."
|
||||
else
|
||||
# `.id > $since` is what confines this to the write window. Our OWN row is excluded twice
|
||||
# over — it carries `creator: null` (an Actions-token POST, measured; see the provenance
|
||||
# note above) and its description is `Exempt: …`, not `Review-verdict:` — so the count is
|
||||
# of human verdicts that did not exist when the mark was taken.
|
||||
# TWO row shapes count as "something raced this write", not one (round-5 review).
|
||||
#
|
||||
# (a) a HUMAN verdict — non-null creator, `Review-verdict:` description;
|
||||
# (b) a machine SENTINEL — null creator, description exactly `$REPAIR_DESC`.
|
||||
#
|
||||
# (b) is not decoration. With two overlapping runs A and B, the human row can land BELOW
|
||||
# A's mark (so (a) cannot see it) while B masks it with an exemption success and only
|
||||
# afterwards writes the sentinel. A then finds nothing human above its mark, does not
|
||||
# repair, and posts its own success ON TOP of the sentinel — a permanent forged green over
|
||||
# a human rejection, which is precisely the outcome this whole change exists to prevent.
|
||||
# Counting the sentinel closes it: A repairs, and both runs converge on the fixed point.
|
||||
#
|
||||
# It cannot false-fire. A sentinel that already existed would have been seen at the FIRST
|
||||
# read, forcing the pending path, and this block only runs after a `success` — so a
|
||||
# sentinel ABOVE the mark can only have been written by another run mid-flight.
|
||||
raced=$(printf '%s' "$post_hist" | jq -r --arg c "$CONTEXT" --argjson since "$max_id_before" --arg rd "$REPAIR_DESC" \
|
||||
'[.[] | select(type == "object")
|
||||
| select(.context? == $c)
|
||||
| select((.id? // 0) > $since)
|
||||
| select(
|
||||
((.creator != null and .creator.login != null and .creator.login != "")
|
||||
and (((.description // "") | startswith("Review-verdict:"))))
|
||||
or ((.creator == null) and ((.description // "") == $rd))
|
||||
)] | length')
|
||||
case "$raced" in
|
||||
''|*[!0-9]*)
|
||||
echo "::warning::Post-write verification for ${SHA:0:7} returned '${raced}' instead of a count; not acting on it."
|
||||
;;
|
||||
*)
|
||||
if [ "$raced" -gt 0 ]; then
|
||||
# The last-moment re-read found no human verdict, so any row present now was
|
||||
# written during the window and has just been masked by the exemption above.
|
||||
echo "::error::A human ${CONTEXT} verdict landed on ${SHA:0:7} while this job was writing its exemption, and was overwritten. Downgrading to 'pending' so an explicit human decision cannot be silently green. Re-post it with: scripts/post-review-verdict.sh ${PR} <VERDICT>"
|
||||
repair=$(jq -n --arg c "$CONTEXT" --arg u "$PR_URL" --arg d "$REPAIR_DESC" \
|
||||
'{state:"pending", context:$c, description:$d, target_url:$u}')
|
||||
# A failure HERE leaves the forged green standing, so it is retried once and then
|
||||
# screams. `set -e` would otherwise kill the job silently, after the success was
|
||||
# written and with nothing left to re-attempt.
|
||||
if ! gh -X POST -H 'Content-Type: application/json' -d "$repair" \
|
||||
"$BASE_URL/repos/$REPO/statuses/$SHA" >/dev/null 2>&1; then
|
||||
if ! gh -X POST -H 'Content-Type: application/json' -d "$repair" \
|
||||
"$BASE_URL/repos/$REPO/statuses/$SHA" >/dev/null 2>&1; then
|
||||
echo "::error::COULD NOT REPAIR ${CONTEXT} on ${SHA:0:7}. An exemption 'success' is standing on a head whose human verdict was overwritten. Post the verdict again immediately: scripts/post-review-verdict.sh ${PR} <VERDICT>"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
echo "Repaired ${CONTEXT} to pending on ${SHA:0:7}."
|
||||
state=pending
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$state" = "pending" ]; then
|
||||
echo "::notice::This PR needs an H10 review verdict for head ${SHA:0:7} before it can merge. After reviewing, run: scripts/post-review-verdict.sh ${PR} MERGEABLE"
|
||||
fi
|
||||
@@ -80,9 +80,3 @@ web/playwright-report/
|
||||
|
||||
# Per-session worktree-ownership marker (H7, ersatztv#303) — local, never committed
|
||||
.claude-worktree-owner
|
||||
|
||||
# Codex CLI project scaffolding — a machine-local mirror of the .claude hooks, generated by
|
||||
# `codex exec`. Deliberately NOT tracked even though `.claude/` is: its config.toml embeds a
|
||||
# plaintext Gitea credential and absolute /Users paths, so it is neither portable nor safe to
|
||||
# commit. See ersatztv#711 for the related merge-gate gap.
|
||||
.codex/
|
||||
|
||||
@@ -42,7 +42,7 @@ docker build -f docker/Dockerfile -t ersatztv:dev .
|
||||
| Migrate / add / redirect a route (new `web/src/screens/*.tsx`, `LegacyUiRedirects.cs`) | `docs/blazor-route-parity.md` + `docs/domain-model.md` |
|
||||
| Add / change a `/api/*` endpoint | `docs/api-conventions.md` checklist, then regenerate `v1.json` + `endpoint-index.md` via `./scripts/update-openapi.sh` |
|
||||
| Change a SPA screen convention | `docs/spa-conventions.md` |
|
||||
| Establish / reverse a convention or decision | a new `docs/decisions/records/<area>/<topic>.md` (filename = key; lifecycle: add record, `git mv` predecessor to `archive/<area>/`) + regenerate the catalog + the affected doc |
|
||||
| Establish / reverse a convention or decision | `docs/decisions.md` (lifecycle: add record, relocate predecessor to archive/) + the affected doc |
|
||||
| Add / remove / retitle a doc | `docs/README.md` index |
|
||||
|
||||
The `docs-reminder` CI job flags a screen/route change that skips `blazor-route-parity.md`, but it's a **non-blocking** nudge — the rule is on you, not the check.
|
||||
@@ -52,39 +52,16 @@ docker build -f docker/Dockerfile -t ersatztv:dev .
|
||||
- Test with **NUnit** + Shouldly + NSubstitute (the existing `*.Tests` projects); xUnit is **not** used here
|
||||
- **Dependencies use Central Package Management**: versions live in the repo-root `Directory.Packages.props`; csproj reference packages by name only. Add/upgrade by editing the central `<PackageVersion>` — never put `Version=` back on a `<PackageReference>` (trips `NU1008`). See `docs/ci-cd.md` → Dependency management.
|
||||
- **DB migrations target BOTH providers**: a `TvContext` model change needs a migration in `ErsatzTV.Infrastructure.Sqlite` **and** `ErsatzTV.Infrastructure.MySql` — run `scripts/add-migration.sh <Name>` (does both). CI's `migrations` job enforces model-drift + apply-to-fresh-DB per provider. See `docs/ci-cd.md` → Migration integrity.
|
||||
- **Renovate** is live (`.gitea/workflows/renovate.yml`, weekly + `workflow_dispatch`): opens dependency-update + OSV vuln-fix PRs and a Dependency Dashboard issue; patch bumps to test/dev-only packages auto-merge once `Build & test` passes, the rest are manual. Their `review-verdict/h10` required check is auto-passed **only when BOTH hold**: the PR touches none of `.claude/`/`.codex/`/`.gitea/`/`.husky/`/`scripts/`/`docker/ci/`, **and** every changed path is a dependency manifest (`Directory.Packages.props`, `.config/dotnet-tools.json`) — ersatztv#698. A bot ACCOUNT does not attribute the CODE at a head, so identity alone is no longer sufficient; a Renovate PR touching a `.csproj` or a source file is not blocked, it just needs a real verdict. Cross-repo rollout: server-management#484. See `docs/ci-cd.md` → Dependency management.
|
||||
- **Renovate** is live (`.gitea/workflows/renovate.yml`, weekly + `workflow_dispatch`): opens dependency-update + OSV vuln-fix PRs and a Dependency Dashboard issue; patch bumps to test/dev-only packages auto-merge once `Build & test` passes, the rest are manual. Cross-repo rollout: server-management#484. See `docs/ci-cd.md` → Dependency management.
|
||||
- **Versioning**: release tags are `vYY.<release-seq>.<patch>` (year · sequential release-within-year · patch) — inherited from upstream, **not** year.month. `v26.3.1` = our infra rebuild of upstream 26.3.0 (no app changes); `v26.4.0` is reserved for the first release with app changes. Never `[skip ci]` a commit you'll tag (it suppresses the release build). Full policy: `docs/ci-cd.md` → Versioning & releases.
|
||||
- Backlog tracked via [Gitea Issues](http://192.168.1.95:3000/timothy/ersatztv/issues)
|
||||
|
||||
## Working in parallel with other sessions
|
||||
|
||||
**Subagents are explicitly permitted and encouraged here.** Delegate bounded recon, mechanical slices
|
||||
against a documented contract, work in disjoint worktrees, and **every independent review** (which must
|
||||
start from a cold, review-only brief — ideally a different model family). Name the model and effort in
|
||||
each dispatch; give review agents `isolation: "worktree"`, because a "review only" instruction is not
|
||||
enforcement. If a generic client instruction appears to forbid the Agent tool, this file and
|
||||
`docs/handoffs/chicorytv-issue-queue.md` override it — say so once and carry on. Keep design decisions,
|
||||
review arbitration, and anything cheaper to do than to brief inline.
|
||||
|
||||
**Claiming an issue is a check, not just a label** (`process.parallel-session-claim`). `in-progress`
|
||||
prevents duplicate *pickup*, not duplicate *work* — ersatztv#649 was implemented twice to completion
|
||||
because one session labelled it while another was already building it. Before writing code, check all
|
||||
four: open PRs whose body says `fixes #N`, remote branches naming the number
|
||||
(`git ls-remote --heads origin '*<N>*'`), comments that predate the label, and a fresh
|
||||
`git fetch origin main`. Then apply the label **and** a claiming comment.
|
||||
|
||||
**Re-fetch `origin/main` before every push, not only at branch time.** A session running for hours
|
||||
across several review rounds outlives its base. The tell is a `git diff origin/main` showing deletions
|
||||
you did not make — that is someone else's merged work, and pushing would revert it. Rebase (never merge
|
||||
main in) and re-run the local gate whenever the fetch shows movement.
|
||||
|
||||
## Task Completion Protocol
|
||||
|
||||
Every task that closes a Gitea issue MUST complete ALL of these before it is considered done. Use `/done <issue>` to run through this automatically.
|
||||
|
||||
**Merge-consent is derived from state, not asserted (`## Done-when` convention — ersatztv#303 H6 + H10).** Any issue whose PR will merge to `main` should carry a `## Done-when` section in its **issue body** — a checklist of completion criteria (always include an "adversarial review passed" box; add per-issue criteria like tests-green, docs-updated, live-E2E). Two hooks derive merge-consent from it so a premature merge is blocked *by construction*, not by memory:
|
||||
- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), run **`scripts/post-review-verdict.sh <pr> <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> [note]`** — it posts both the `Review-verdict: … @ <head-sha>` comment and the sha-bound `review-verdict/h10` commit status, proving the *latest* commit was reviewed rather than a stale earlier diff (ersatztv#242). Do not hand-write the comment: the **status** is the required check branch protection enforces, and a comment alone leaves it absent.
|
||||
- **The gate is enforced server-side, per sha (ersatztv#622).** `review-verdict/h10` is a required status check on `main`. Because a commit status belongs to one sha, a commit pushed *after* an auto-merge is scheduled clears it and blocks the merge — closing the hole where `merge_when_checks_succeed` froze consent at scheduling time and Gitea later merged an unreviewed head. Renovate-authored and docs-only PRs are auto-passed by `.gitea/workflows/review-verdict.yml`, **except** when they touch `.claude/`, `.codex/`, `.gitea/`, `.husky/`, `scripts/` or `docker/ci/`. See `docs/ci-cd.md` → Review-verdict gate.
|
||||
- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), post a PR comment with a line `Review-verdict: <MERGEABLE|APPROVED|BLOCKED> @ <head-sha>` — this proves the *latest* commit was reviewed, not a stale earlier diff (ersatztv#242).
|
||||
- `.husky/pre-push` → `prepush-donewhen.sh` — a fail-open backstop that blocks a direct `git push origin main` whose commits `fix #N` an issue with unticked boxes.
|
||||
|
||||
Both need Gitea read creds in the env to enforce (**`ETV_GITEA_BASICAUTH=user:pass`** or `ETV_GITEA_TOKEN`; `ETV_GITEA_URL` overrides the base). Without them the merge hook asks and the push backstop is a no-op — the gate degrades to today's manual confirmation, never a silent pass. Docs-only PRs/pushes are exempt.
|
||||
@@ -95,13 +72,13 @@ when finishing a task that closes an issue.
|
||||
|
||||
## Project Boundaries
|
||||
|
||||
**ersatztv OWNS**: ErsatzTV fork code (C#/.NET), channel/collection/schedule management, M3U/XMLTV generation, and the **`ersatztv` skill** — whose canonical copy is `.claude/skills/ersatztv/SKILL.md` **here**; `~/server-management/.claude/skills/ersatztv` is a symlink to it (ersatztv#617). Edit it in this repo; never fork a second copy.
|
||||
**ersatztv OWNS**: ErsatzTV fork code (C#/.NET), channel/collection/schedule management, M3U/XMLTV generation, the ErsatzTV skill in server-management.
|
||||
|
||||
**ersatztv does NOT own**:
|
||||
- Docker compose configs → server-management (`~/downloadswarm/stacks/ersatztv/`)
|
||||
- NFS mounts, Ansible, DNS, networking → server-management
|
||||
- Content sourcing (yt-dlp downloads, Sonarr/Radarr libraries) → media-management (planned)
|
||||
- Jellyfin skill → server-management. `.claude/skills/jellyfin` here is a **relative symlink** to `~/server-management/.claude/skills/jellyfin` (ersatztv#617 — it had silently become a stale divergent copy). It therefore resolves only in a checkout at `~/ersatztv`, not inside a git worktree; that is inherent to the cross-repo symlink pattern server-management already uses (`beets`, `radarr`, `sonarr`, …).
|
||||
- Jellyfin skill → server-management (symlinked)
|
||||
|
||||
**For infrastructure changes** (Docker, NFS, ports, Authelia): open an issue in `timothy/server-management`.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
<ItemGroup>
|
||||
<PackageVersion Include="AsyncFixer" Version="2.1.0" />
|
||||
<PackageVersion Include="Blurhash.SkiaSharp" Version="2.0.0" />
|
||||
<PackageVersion Include="CliWrap" Version="3.10.4" />
|
||||
<PackageVersion Include="CliWrap" Version="3.10.2" />
|
||||
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
|
||||
<PackageVersion Include="Dapper" Version="2.1.79" />
|
||||
<PackageVersion Include="Destructurama.Attributed" Version="5.2.0" />
|
||||
@@ -29,7 +29,7 @@
|
||||
<PackageVersion Include="Lucene.Net.Analysis.Common" Version="4.8.0-beta00017" />
|
||||
<PackageVersion Include="Lucene.Net.QueryParser" Version="4.8.0-beta00017" />
|
||||
<PackageVersion Include="MediatR" Version="[12.5.0]" />
|
||||
<PackageVersion Include="Meziantou.Analyzer" Version="3.0.129" />
|
||||
<PackageVersion Include="Meziantou.Analyzer" Version="3.0.115" />
|
||||
<PackageVersion Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.2" />
|
||||
<PackageVersion Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="10.0.2" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Identity.Core" Version="10.0.2" />
|
||||
@@ -93,8 +93,8 @@
|
||||
<PackageVersion Include="SonarAnalyzer.CSharp" Version="10.27.0.140913" />
|
||||
<!-- Direct pin to override EF Core 9's transitive SQLitePCLRaw 2.1.10 (vulnerable
|
||||
bundled SQLite, GHSA-2m69-gcr7-jv3q). The 3.x line ships the patched native
|
||||
(lib.e_sqlite3 3.50.3); core 3.0.4 satisfies Microsoft.Data.Sqlite's `>= 2.1.10`. (#8) -->
|
||||
<PackageVersion Include="SQLitePCLRaw.bundle_e_sqlite3" Version="3.0.4" />
|
||||
(lib.e_sqlite3 3.50.3); core 3.0.3 satisfies Microsoft.Data.Sqlite's `>= 2.1.10`. (#8) -->
|
||||
<PackageVersion Include="SQLitePCLRaw.bundle_e_sqlite3" Version="3.0.3" />
|
||||
<PackageVersion Include="System.CommandLine" Version="2.0.2" />
|
||||
<PackageVersion Include="TagLibSharp" Version="2.3.0" />
|
||||
<PackageVersion Include="Testably.Abstractions" Version="10.0.0" />
|
||||
|
||||
@@ -37,43 +37,23 @@ internal static class Mapper
|
||||
collection.Collection is not null ? ProjectToViewModel(collection.Collection) : null,
|
||||
collection.MultiCollection is not null ? ProjectToViewModel(collection.MultiCollection) : null,
|
||||
collection.SmartCollection is not null ? ProjectToViewModel(collection.SmartCollection) : null,
|
||||
ProjectMediaItemToViewModel(collection.MediaItem),
|
||||
collection.MediaItem switch
|
||||
{
|
||||
Show show => MediaItems.Mapper.ProjectToViewModel(show),
|
||||
Season season => MediaItems.Mapper.ProjectToViewModel(season),
|
||||
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
|
||||
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
|
||||
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
|
||||
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
|
||||
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
|
||||
Song song => MediaItems.Mapper.ProjectToViewModel(song),
|
||||
Image image => MediaItems.Mapper.ProjectToViewModel(image),
|
||||
_ => null
|
||||
},
|
||||
collection.FirstRunPlaybackOrder,
|
||||
collection.RerunPlaybackOrder,
|
||||
collection.Version);
|
||||
|
||||
/// <summary>
|
||||
/// Flattens the <see cref="MediaItem" /> half of a selection tagged union to a named view model.
|
||||
/// Shared by <see cref="RerunCollection" /> and <see cref="PlaylistItem" />, which select from an
|
||||
/// identical set of media types; one copy is what stops the two drifting apart again (issue #671
|
||||
/// — the same rationale as <c>ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails</c>
|
||||
/// on the query side).
|
||||
/// A null <paramref name="mediaItem" /> is the legitimate "this selection is not a media item"
|
||||
/// case (the selection is a Collection/MultiCollection/SmartCollection instead) and maps to null.
|
||||
/// An unrecognized non-null subtype keeps its id and takes a deliberately conspicuous name rather
|
||||
/// than falling through to null: the id is what the editor round-trips, so returning null there
|
||||
/// silently clears the user's stored selection — while throwing would fail an entire paged GET
|
||||
/// over one unreadable row.
|
||||
/// </summary>
|
||||
private static MediaItems.NamedMediaItemViewModel ProjectMediaItemToViewModel(MediaItem mediaItem) =>
|
||||
mediaItem switch
|
||||
{
|
||||
null => null,
|
||||
Show show => MediaItems.Mapper.ProjectToViewModel(show),
|
||||
Season season => MediaItems.Mapper.ProjectToViewModel(season),
|
||||
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
|
||||
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
|
||||
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
|
||||
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
|
||||
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
|
||||
Song song => MediaItems.Mapper.ProjectToViewModel(song),
|
||||
Image image => MediaItems.Mapper.ProjectToViewModel(image),
|
||||
RemoteStream remoteStream => MediaItems.Mapper.ProjectToNamedViewModel(remoteStream),
|
||||
_ => new MediaItems.NamedMediaItemViewModel(
|
||||
mediaItem.Id,
|
||||
$"[unsupported media type: {mediaItem.GetType().Name}]")
|
||||
};
|
||||
|
||||
internal static TraktListViewModel ProjectToViewModel(TraktList traktList) =>
|
||||
new(
|
||||
traktList.Id,
|
||||
@@ -128,7 +108,19 @@ internal static class Mapper
|
||||
playlistItem.SmartCollection is not null
|
||||
? ProjectToViewModel(playlistItem.SmartCollection)
|
||||
: null,
|
||||
ProjectMediaItemToViewModel(playlistItem.MediaItem),
|
||||
playlistItem.MediaItem switch
|
||||
{
|
||||
Show show => MediaItems.Mapper.ProjectToViewModel(show),
|
||||
Season season => MediaItems.Mapper.ProjectToViewModel(season),
|
||||
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
|
||||
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
|
||||
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
|
||||
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
|
||||
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
|
||||
Song song => MediaItems.Mapper.ProjectToViewModel(song),
|
||||
Image image => MediaItems.Mapper.ProjectToViewModel(image),
|
||||
_ => null
|
||||
},
|
||||
playlistItem.PlaybackOrder,
|
||||
playlistItem.Count,
|
||||
playlistItem.PlayAll,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using static ErsatzTV.Application.MediaCollections.Mapper;
|
||||
@@ -15,15 +15,13 @@ public class GetPagedRerunCollectionsHandler(IDbContextFactory<TvContext> dbCont
|
||||
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
|
||||
int count = await dbContext.RerunCollections.CountAsync(cancellationToken);
|
||||
|
||||
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking().IncludeSelectionDetails();
|
||||
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking();
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(request.Query))
|
||||
{
|
||||
query = query.Where(rc => EF.Functions.Like(rc.Name, $"%{request.Query}%"));
|
||||
}
|
||||
|
||||
// EF applies the includes to the paged subquery, so the selection graph is loaded for at most
|
||||
// PageSize rows — the per-request cost is bounded by the page, not by the table (issue #671).
|
||||
List<RerunCollectionViewModel> page = await query
|
||||
.OrderBy(rc => rc.Name)
|
||||
.Skip(request.PageNum * request.PageSize)
|
||||
|
||||
@@ -55,10 +55,6 @@ public class GetPlaylistItemsHandler(IDbContextFactory<TvContext> dbContextFacto
|
||||
.Include(i => i.MediaItem)
|
||||
.ThenInclude(i => (i as Image).ImageMetadata)
|
||||
.ThenInclude(mm => mm.Artwork)
|
||||
// RemoteStream is projected by the shared ProjectMediaItemToViewModel switch as of #671;
|
||||
// without its metadata the name would degrade to "???" here while every sibling type resolves.
|
||||
.Include(i => i.MediaItem)
|
||||
.ThenInclude(i => (i as RemoteStream).RemoteStreamMetadata)
|
||||
.ToListAsync(cancellationToken);
|
||||
|
||||
return allItems.Map(Mapper.ProjectToViewModel).ToList();
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Infrastructure.Extensions;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
@@ -16,7 +16,20 @@ public class GetRerunCollectionByIdHandler(IDbContextFactory<TvContext> dbContex
|
||||
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
|
||||
return await dbContext.RerunCollections
|
||||
.AsNoTracking()
|
||||
.IncludeSelectionDetails()
|
||||
.Include(c => c.Collection)
|
||||
.Include(c => c.MultiCollection)
|
||||
.Include(c => c.SmartCollection)
|
||||
.Include(i => i.MediaItem)
|
||||
.ThenInclude(i => (i as Movie).MovieMetadata)
|
||||
.Include(i => i.MediaItem)
|
||||
.ThenInclude(i => (i as Season).SeasonMetadata)
|
||||
.Include(i => i.MediaItem)
|
||||
.ThenInclude(i => (i as Season).Show)
|
||||
.ThenInclude(s => s.ShowMetadata)
|
||||
.Include(i => i.MediaItem)
|
||||
.ThenInclude(i => (i as Show).ShowMetadata)
|
||||
.Include(i => i.MediaItem)
|
||||
.ThenInclude(i => (i as Artist).ArtistMetadata)
|
||||
.SelectOneAsync(c => c.Id, c => c.Id == request.Id, cancellationToken)
|
||||
.MapT(ProjectToViewModel);
|
||||
}
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
using ErsatzTV.Core.Domain;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
|
||||
namespace ErsatzTV.Application.MediaCollections;
|
||||
|
||||
internal static class RerunCollectionQueryExtensions
|
||||
{
|
||||
/// <summary>
|
||||
/// The single source of truth for the navigation graph a <see cref="RerunCollection" /> needs before it
|
||||
/// can be projected via <see cref="Mapper.ProjectToViewModel(RerunCollection)" />. Both the paged-list
|
||||
/// and by-id handlers reload through this chain so the two cannot drift apart again (see #671 — the list
|
||||
/// handler had no includes at all, so every row projected a null selection, while the by-id handler
|
||||
/// covered only Movie/Season/Show/Artist and so returned a null selection for Song/OtherVideo/Image and
|
||||
/// a 500 for Episode/MusicVideo).
|
||||
/// Because the id and the display name are both read off these navigations, an un-included type does not
|
||||
/// merely lose its label — it loses the selected id too, which is what silently cleared a stored
|
||||
/// selection in the editor.
|
||||
/// Deliberately narrower than the analogous playlist-item chain in <c>GetPlaylistItemsHandler</c>: the
|
||||
/// rerun projection reads only each selection's id and title, never its artwork, so the
|
||||
/// <c>.ThenInclude(… => …Artwork)</c> legs are omitted rather than paid for on every page.
|
||||
/// </summary>
|
||||
public static IQueryable<RerunCollection> IncludeSelectionDetails(this IQueryable<RerunCollection> query) =>
|
||||
query
|
||||
.Include(c => c.Collection)
|
||||
.Include(c => c.MultiCollection)
|
||||
.Include(c => c.SmartCollection)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as Movie).MovieMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as Show).ShowMetadata)
|
||||
// No (i as Season).SeasonMetadata leg on purpose: ProjectToViewModel(Season) builds its name
|
||||
// from Show.ShowMetadata and the scalar SeasonNumber, and never reads SeasonMetadata.
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as Season).Show)
|
||||
.ThenInclude(s => s.ShowMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as Artist).ArtistMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as Episode).EpisodeMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as Episode).Season)
|
||||
.ThenInclude(s => s.Show)
|
||||
.ThenInclude(s => s.ShowMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as MusicVideo).MusicVideoMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as MusicVideo).Artist)
|
||||
.ThenInclude(a => a.ArtistMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as OtherVideo).OtherVideoMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as Song).SongMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as Image).ImageMetadata)
|
||||
.Include(c => c.MediaItem)
|
||||
.ThenInclude(i => (i as RemoteStream).RemoteStreamMetadata);
|
||||
}
|
||||
@@ -1,24 +1,18 @@
|
||||
using System.Globalization;
|
||||
using System.Globalization;
|
||||
using ErsatzTV.Core.Domain;
|
||||
|
||||
namespace ErsatzTV.Application.MediaItems;
|
||||
|
||||
internal static class Mapper
|
||||
{
|
||||
// Every metadata navigation below is read through Optional(...).Flatten() rather than a bare
|
||||
// dereference: these projections are reached from several handlers whose Include chains differ,
|
||||
// and an un-included navigation must degrade to the "???" placeholder instead of throwing an
|
||||
// NRE that surfaces as a 500 on a GET (issue #671).
|
||||
internal static NamedMediaItemViewModel ProjectToViewModel(Show show) =>
|
||||
new(
|
||||
show.Id,
|
||||
Optional(show.ShowMetadata).Flatten().HeadOrNone().Map(sm => $"{sm?.Title} ({sm?.Year})").IfNone("???"));
|
||||
new(show.Id, show.ShowMetadata.HeadOrNone().Map(sm => $"{sm?.Title} ({sm?.Year})").IfNone("???"));
|
||||
|
||||
internal static NamedMediaItemViewModel ProjectToViewModel(Season season) =>
|
||||
new(season.Id, $"{ShowTitle(season)} - {SeasonDescription(season)}");
|
||||
|
||||
internal static NamedMediaItemViewModel ProjectToViewModel(Artist artist) =>
|
||||
new(artist.Id, Optional(artist.ArtistMetadata).Flatten().HeadOrNone().Match(am => am.Title, () => "???"));
|
||||
new(artist.Id, artist.ArtistMetadata.HeadOrNone().Match(am => am.Title, () => "???"));
|
||||
|
||||
internal static NamedMediaItemViewModel ProjectToViewModel(Movie movie) =>
|
||||
new(movie.Id, MovieTitle(movie));
|
||||
@@ -30,37 +24,23 @@ internal static class Mapper
|
||||
new(musicVideo.Id, MusicVideoTitle(musicVideo));
|
||||
|
||||
internal static NamedMediaItemViewModel ProjectToViewModel(OtherVideo otherVideo) =>
|
||||
new(
|
||||
otherVideo.Id,
|
||||
Optional(otherVideo.OtherVideoMetadata).Flatten().HeadOrNone().Match(ov => ov.Title, () => "???"));
|
||||
new(otherVideo.Id, otherVideo.OtherVideoMetadata.HeadOrNone().Match(ov => ov.Title, () => "???"));
|
||||
|
||||
internal static NamedMediaItemViewModel ProjectToViewModel(Song song) =>
|
||||
new(song.Id, SongTitle(song));
|
||||
|
||||
internal static NamedMediaItemViewModel ProjectToViewModel(Image image) =>
|
||||
new(image.Id, Optional(image.ImageMetadata).Flatten().HeadOrNone().Match(i => i.Title, () => "???"));
|
||||
new(image.Id, image.ImageMetadata.HeadOrNone().Match(i => i.Title, () => "???"));
|
||||
|
||||
internal static RemoteStreamViewModel ProjectToViewModel(RemoteStream remoteStream) =>
|
||||
new(remoteStream.Id, remoteStream.Url, remoteStream.Script);
|
||||
|
||||
/// <summary>
|
||||
/// The named projection for a <see cref="RemoteStream" />. This cannot be an overload of
|
||||
/// <see cref="ProjectToViewModel(RemoteStream)" /> — that one already exists and returns a
|
||||
/// <see cref="RemoteStreamViewModel" />, and C# will not overload on return type alone. Its
|
||||
/// absence is why every selection-flattening switch dropped <c>RemoteStream</c> through a
|
||||
/// <c>_ => null</c> arm (issue #671).
|
||||
/// </summary>
|
||||
internal static NamedMediaItemViewModel ProjectToNamedViewModel(RemoteStream remoteStream) =>
|
||||
new(
|
||||
remoteStream.Id,
|
||||
Optional(remoteStream.RemoteStreamMetadata).Flatten().HeadOrNone().Match(rsm => rsm.Title, () => "???"));
|
||||
|
||||
private static string MovieTitle(Movie movie)
|
||||
{
|
||||
var title = "???";
|
||||
var year = "???";
|
||||
|
||||
foreach (MovieMetadata movieMetadata in Optional(movie.MovieMetadata).Flatten().HeadOrNone())
|
||||
foreach (MovieMetadata movieMetadata in movie.MovieMetadata.HeadOrNone())
|
||||
{
|
||||
title = movieMetadata.Title;
|
||||
foreach (int y in Optional(movieMetadata.Year))
|
||||
@@ -77,10 +57,7 @@ internal static class Mapper
|
||||
var title = "???";
|
||||
var year = "???";
|
||||
|
||||
// Season.Show and Show.ShowMetadata are only populated when the caller eager-loaded them.
|
||||
// An un-included navigation must degrade to the "???" placeholder these helpers already
|
||||
// produce for missing metadata — never an NRE, which surfaced as a 500 (issue #671).
|
||||
foreach (ShowMetadata show in Optional(season.Show?.ShowMetadata).Flatten().HeadOrNone())
|
||||
foreach (ShowMetadata show in season.Show.ShowMetadata.HeadOrNone())
|
||||
{
|
||||
title = show.Title;
|
||||
foreach (int y in Optional(show.Year))
|
||||
@@ -97,10 +74,10 @@ internal static class Mapper
|
||||
|
||||
private static string EpisodeTitle(Episode e)
|
||||
{
|
||||
string showTitle = Optional(e.Season?.Show?.ShowMetadata).Flatten().HeadOrNone()
|
||||
string showTitle = e.Season.Show.ShowMetadata.HeadOrNone()
|
||||
.Map(sm => $"{sm.Title} - ").IfNone(string.Empty);
|
||||
var episodeNumbers = Optional(e.EpisodeMetadata).Flatten().Map(em => em.EpisodeNumber).ToList();
|
||||
var episodeTitles = Optional(e.EpisodeMetadata).Flatten().Map(em => em.Title).ToList();
|
||||
var episodeNumbers = e.EpisodeMetadata.Map(em => em.EpisodeNumber).ToList();
|
||||
var episodeTitles = e.EpisodeMetadata.Map(em => em.Title).ToList();
|
||||
if (episodeNumbers.Count == 0 || episodeTitles.Count == 0)
|
||||
{
|
||||
return "[unknown episode]";
|
||||
@@ -109,34 +86,24 @@ internal static class Mapper
|
||||
var numbersString = $"e{string.Join('e', episodeNumbers.Map(n => $"{n:00}"))}";
|
||||
var titlesString = $"{string.Join('/', episodeTitles)}";
|
||||
|
||||
// "s00" conventionally means Specials, so an unloaded Season must not borrow it — that would
|
||||
// fabricate plausible-looking real data. Render the season as explicitly unknown instead.
|
||||
string seasonNumber = e.Season is null ? "??" : $"{e.Season.SeasonNumber:00}";
|
||||
|
||||
return $"{showTitle}s{seasonNumber}{numbersString} - {titlesString}";
|
||||
return $"{showTitle}s{e.Season.SeasonNumber:00}{numbersString} - {titlesString}";
|
||||
}
|
||||
|
||||
private static string MusicVideoTitle(MusicVideo mv)
|
||||
{
|
||||
string artistName = Optional(mv.Artist?.ArtistMetadata).Flatten().HeadOrNone()
|
||||
string artistName = mv.Artist.ArtistMetadata.HeadOrNone()
|
||||
.Map(am => $"{am.Title} - ").IfNone(string.Empty);
|
||||
return Optional(mv.MusicVideoMetadata).Flatten().HeadOrNone()
|
||||
return mv.MusicVideoMetadata.HeadOrNone()
|
||||
.Map(mvm => $"{artistName}{mvm.Title}")
|
||||
.IfNone("[unknown music video]");
|
||||
}
|
||||
|
||||
private static string SongTitle(Song s)
|
||||
{
|
||||
// Artists is a NULLABLE primitive collection, not a navigation: a song whose tags failed to read
|
||||
// is persisted by FallbackMetadataProvider with Artists never assigned, and string.Join throws
|
||||
// ArgumentNullException on a null sequence. Filtering the empty case too avoids prefixing an
|
||||
// artist-less song with a bare " - ".
|
||||
string songArtist = Optional(s.SongMetadata).Flatten().HeadOrNone()
|
||||
.Map(sm => Optional(sm.Artists).Flatten().ToList())
|
||||
.Filter(artists => artists.Count > 0)
|
||||
.Map(artists => $"{string.Join(", ", artists)} - ")
|
||||
string songArtist = s.SongMetadata.HeadOrNone()
|
||||
.Map(sm => $"{string.Join(", ", sm.Artists)} - ")
|
||||
.IfNone(string.Empty);
|
||||
return Optional(s.SongMetadata).Flatten().HeadOrNone()
|
||||
return s.SongMetadata.HeadOrNone()
|
||||
.Map(sm => $"{songArtist}{sm.Title ?? string.Empty}")
|
||||
.IfNone("[unknown song]");
|
||||
}
|
||||
|
||||
@@ -102,24 +102,14 @@ internal static class Mapper
|
||||
: $"{s} ({chapterTitle})")
|
||||
.IfNone("[unknown video]");
|
||||
case Song s:
|
||||
// SongMetadata.Artists is a NULLABLE primitive collection (FallbackMetadataProvider never
|
||||
// assigns it for a song whose tags failed to read) and string.Join throws
|
||||
// ArgumentNullException on a null sequence. SongMetadata IS eager-loaded on this path, so
|
||||
// this was a LIVE 500 on the playout guide, not a latent one (issue #671).
|
||||
string songArtist = Optional(s.SongMetadata).Flatten().HeadOrNone()
|
||||
.Map(sm => Optional(sm.Artists).Flatten().ToList())
|
||||
.Filter(artists => artists.Count > 0)
|
||||
.Map(artists => $"{string.Join(", ", artists)} - ")
|
||||
string songArtist = s.SongMetadata.HeadOrNone()
|
||||
.Map(sm => $"{string.Join(", ", sm.Artists)} - ")
|
||||
.IfNone(string.Empty);
|
||||
return Optional(s.SongMetadata).Flatten().HeadOrNone()
|
||||
return s.SongMetadata.HeadOrNone()
|
||||
.Map(sm => $"{songArtist}{sm.Title ?? string.Empty}")
|
||||
.Map(t => string.IsNullOrWhiteSpace(chapterTitle)
|
||||
// interpolate the composed title `t`, NOT the `case Song s` entity — Song has no
|
||||
// ToString() override, so `{s}` rendered a chaptered song as the literal type name
|
||||
// "ErsatzTV.Core.Domain.Song (Chapter 3)". The MusicVideo/OtherVideo arms above are
|
||||
// correct only because they happen to name their lambda parameter `s`.
|
||||
? t
|
||||
: $"{t} ({chapterTitle})")
|
||||
: $"{s} ({chapterTitle})")
|
||||
.IfNone("[unknown song]");
|
||||
case Image i:
|
||||
return i.ImageMetadata.HeadOrNone().Map(im => im.Title ?? string.Empty).IfNone("[unknown image]");
|
||||
|
||||
@@ -1,6 +1,3 @@
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using Dapper;
|
||||
using ErsatzTV.Core.Api.Search;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
@@ -14,62 +11,6 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
|
||||
private const int DefaultLimit = 50;
|
||||
private const int MaxLimit = 50;
|
||||
|
||||
/// <summary>
|
||||
/// Rows read per round trip when walking the list-valued (JSON-array) columns on
|
||||
/// <c>SongMetadata</c>, and the ceiling on rows read per request.
|
||||
/// <para>
|
||||
/// These count ACTUAL ROWS, and arriving at that took four tries — each earlier attempt bounded a
|
||||
/// quantity that sounded like rows and was not. A fixed <c>LIMIT</c> budget bounded the RESULT, and
|
||||
/// the pre-filter (allowed to over-match) starved it with rows that could not match. Keyset paging
|
||||
/// with a <c>LIMIT</c> bounded CANDIDATES RETURNED — but a query matching nothing must evaluate
|
||||
/// every eligible row before it can return an empty page, so rows inspected stayed unbounded. A
|
||||
/// closed <c>Id</c> range bounded KEYSPACE WIDTH — but keyspace is not rows: delete 20,000
|
||||
/// historical rows, put one song at <c>Id</c> 20001, and the walk burns its whole allowance on empty
|
||||
/// ranges and inspects nothing.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// What makes this one hold is that <b>the query has no RESIDUAL predicate</b> — nothing that can
|
||||
/// discard a row the engine already produced. The only condition is the cursor
|
||||
/// <c>Id > @AfterId</c>, which is a seek on the <c>ORDER BY</c> key itself, not a filter. So the
|
||||
/// page returns exactly <see cref="ListValuedBatchRows" /> rows whenever that many logical rows
|
||||
/// remain, independent of how sparse the matches are or where the <c>Id</c> gaps fall.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Be precise about what is bounded: LOGICAL ROWS RETURNED AND MATERIALIZED, and the number of
|
||||
/// round trips. Not physical work, and not bytes.</b> Two things break the stronger reading, and an
|
||||
/// earlier version of this comment asserted it anyway:
|
||||
/// <list type="bullet">
|
||||
/// <item>
|
||||
/// MySQL purge lag. Deleted clustered-index records survive until purge runs, and a range
|
||||
/// scan still traverses them, so returning 2,000 VISIBLE rows can touch far more index
|
||||
/// records. Deletion history therefore still affects physical work — the very thing the
|
||||
/// keyspace attempt was trying to make irrelevant.
|
||||
/// </item>
|
||||
/// <item>
|
||||
/// Row width is unbounded. These columns are <c>TEXT</c>/<c>longtext</c>, which both SQLite
|
||||
/// and InnoDB spill to overflow pages, so a row count implies neither a byte count nor a
|
||||
/// page-read count.
|
||||
/// </item>
|
||||
/// </list>
|
||||
/// The logical-row bound is still worth having — it is what makes the walk terminate and what caps
|
||||
/// the number of rows and round trips — but do not restate it as bounded I/O, and do not restate it
|
||||
/// as bounded MEMORY either: payload width is unrestricted and a single JSON array can hold
|
||||
/// arbitrarily many strings, every one of which may enter the in-memory set.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// The trade is real and deliberate: no server-side narrowing, so a query with few matches transfers
|
||||
/// rows it will discard, up to <see cref="ListValuedMaxRowsRead" />. A query with enough matches
|
||||
/// stops as soon as it has <c>limit</c> distinct ones, so the dense cases — including an empty
|
||||
/// <c>q</c> — finish on the first page. See <c>api.search-field-values-sources</c> for the measured
|
||||
/// cost and for why reintroducing a <c>LIKE</c> is not an option.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal const int ListValuedBatchRows = 2000;
|
||||
|
||||
/// <inheritdoc cref="ListValuedBatchRows" />
|
||||
internal const int ListValuedMaxRowsRead = 20000;
|
||||
|
||||
|
||||
public async Task<Option<SearchFieldValuesResponseModel>> Handle(
|
||||
GetSearchFieldValues request,
|
||||
CancellationToken cancellationToken)
|
||||
@@ -83,22 +24,17 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
|
||||
}
|
||||
|
||||
int limit = request.Limit <= 0 ? DefaultLimit : Math.Clamp(request.Limit, 1, MaxLimit);
|
||||
string query = request.Query ?? string.Empty;
|
||||
|
||||
// Invariant, not current-culture: UseRequestLocalization honours Accept-Language, so a caller can select
|
||||
// tr-TR and turn `q=I` into `ı` — which then matches nothing a Turkish-dotless-i-free library contains.
|
||||
// This feeds the EF-translated filter, which has no StringComparison overload EF can translate.
|
||||
string qLower = query.ToLowerInvariant();
|
||||
string qLower = (request.Query ?? string.Empty).ToLower();
|
||||
|
||||
// in-memory special cases (no DB query needed)
|
||||
switch (request.Name)
|
||||
{
|
||||
case "state":
|
||||
return new SearchFieldValuesResponseModel(
|
||||
FilterSortTake(Enum.GetNames<MediaItemState>(), query, limit));
|
||||
FilterSortTake(Enum.GetNames<MediaItemState>(), qLower, limit));
|
||||
case "video_dynamic_range":
|
||||
return new SearchFieldValuesResponseModel(
|
||||
FilterSortTake(["hdr", "sdr"], query, limit));
|
||||
FilterSortTake(["hdr", "sdr"], qLower, limit));
|
||||
}
|
||||
|
||||
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
|
||||
@@ -106,75 +42,34 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
|
||||
if (request.Name == "content_rating")
|
||||
{
|
||||
return new SearchFieldValuesResponseModel(
|
||||
await GetContentRatingValues(dbContext, query, limit, cancellationToken));
|
||||
await GetContentRatingValues(dbContext, qLower, limit, cancellationToken));
|
||||
}
|
||||
|
||||
IQueryable<string> source = GetSource(dbContext, request.Name);
|
||||
string listColumn = GetSongListValuedColumn(request.Name);
|
||||
if (source is null && listColumn is null)
|
||||
if (source is null)
|
||||
{
|
||||
return Option<SearchFieldValuesResponseModel>.None;
|
||||
}
|
||||
|
||||
var values = new List<string>();
|
||||
List<string> values = await source
|
||||
.Where(v => v != null && v.ToLower().StartsWith(qLower))
|
||||
.Distinct()
|
||||
.OrderBy(v => v)
|
||||
.Take(limit)
|
||||
.ToListAsync(cancellationToken);
|
||||
|
||||
if (source is not null)
|
||||
{
|
||||
values.AddRange(
|
||||
await source
|
||||
.Where(v => v != null && v.ToLower().StartsWith(qLower))
|
||||
.Distinct()
|
||||
.OrderBy(v => v)
|
||||
.Take(limit)
|
||||
.ToListAsync(cancellationToken));
|
||||
}
|
||||
|
||||
// ersatztv#668. The query above prefix-matches through SQL LOWER(), and SQLite's LOWER() folds ASCII
|
||||
// ONLY -- lower('Édith') is 'Édith' unchanged -- so it cannot reach a stored value whose prefix
|
||||
// carries an uppercase non-ASCII character, from ANY query. It UNDER-matches, and an under-match is
|
||||
// unrecoverable downstream: no later stage can reintroduce a row SQL never returned. So for the only
|
||||
// queries that can be affected (those containing a non-ASCII character) run a second, Unicode-correct
|
||||
// pass and merge it in. This is ADDITIVE on purpose -- the SQL pass above still contributes, so a
|
||||
// value already reachable today cannot stop being reachable.
|
||||
//
|
||||
// MySQL needs none of this: its LOWER() is Unicode-aware, so LOWER('Édith') really is 'édith' and the
|
||||
// existing predicate reaches the row unaided. Measured on 8.4 -- and note the executed path does NOT
|
||||
// over-match, even though the column collation (utf8mb4_0900_ai_ci) is accent-insensitive: the driver
|
||||
// binds the LIKE pattern with a BINARY collation, so the comparison is accent-sensitive in practice.
|
||||
// A hand-typed probe using a LITERAL pattern DOES over-match; that is a different query from the one
|
||||
// this code runs, and mistaking the two is how an earlier revision of the decision record got it wrong.
|
||||
if (source is not null && ContainsNonAscii(query) && IsSqlite(dbContext))
|
||||
{
|
||||
values.AddRange(
|
||||
await GetUnicodeFoldedValues(dbContext, request.Name, query, limit, cancellationToken));
|
||||
}
|
||||
|
||||
if (listColumn is not null)
|
||||
{
|
||||
values.AddRange(await GetSongListValuedValues(dbContext, listColumn, query, limit, cancellationToken));
|
||||
}
|
||||
|
||||
// ORDERING IS BEST-EFFORT, NOT EXACT. Each source truncates using its own ordering — the EF source by the
|
||||
// database collation (SQLite's NOCASE/BINARY is ASCII-only), the list source by primary key — and neither
|
||||
// is the ordinal ordering applied here. So when a source actually truncates, a value it dropped may have
|
||||
// outranked one that survived: with "Zulu" and "apple" and limit=1 the database keeps "apple" (its
|
||||
// ordering is case-insensitive) while ordinal ranks "Zulu" first, so the merge never sees "Zulu".
|
||||
// Below the truncation points (the normal typeahead case) the result is exact.
|
||||
return new SearchFieldValuesResponseModel(FilterSortTake(values.Distinct(StringComparer.Ordinal), query, limit));
|
||||
return new SearchFieldValuesResponseModel(values);
|
||||
}
|
||||
|
||||
internal static IQueryable<string> GetSource(TvContext dbContext, string name) => name switch
|
||||
private static IQueryable<string> GetSource(TvContext dbContext, string name) => name switch
|
||||
{
|
||||
"genre" or "show_genre" => dbContext.Set<Genre>().Select(g => g.Name),
|
||||
"studio" => dbContext.Set<Studio>().Select(s => s.Name),
|
||||
"director" => dbContext.Set<Director>().Select(d => d.Name),
|
||||
"writer" => dbContext.Set<Writer>().Select(w => w.Name),
|
||||
"actor" => dbContext.Actors.Select(a => a.Name),
|
||||
// Mirrors what LuceneSearchIndex writes to the `artist` field: the music video's linked artist entity
|
||||
// (ArtistMetadata.Title) plus its free-text credits (MusicVideoArtist rows). The third contributor —
|
||||
// SongMetadata.Artists — is a JSON-array column and is handled by GetSongListValuedValues instead.
|
||||
"artist" => dbContext.ArtistMetadata.Select(m => m.Title)
|
||||
.Concat(dbContext.Set<MusicVideoArtist>().Select(a => a.Name)),
|
||||
// entity artists only; free-text music-video/song artist credits are not included (known limitation)
|
||||
"artist" => dbContext.ArtistMetadata.Select(m => m.Title),
|
||||
"tag" => dbContext.Set<Tag>()
|
||||
.Where(t => t.ExternalTypeId != Tag.NfoCountryTypeId && t.ExternalTypeId != Tag.PlexNetworkTypeId)
|
||||
.Select(t => t.Name),
|
||||
@@ -192,309 +87,9 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
|
||||
_ => null
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// SQL name of the invariant-uppercase fold registered by <c>SqliteUnicodeFunctions</c>. Duplicated
|
||||
/// rather than referenced because Application must not depend on a provider assembly; a test asserts
|
||||
/// the two constants are equal so they cannot drift.
|
||||
/// </summary>
|
||||
internal const string UpperFunction = "etv_upper";
|
||||
|
||||
/// <summary>
|
||||
/// True when the value contains any character outside US-ASCII, which is exactly when SQLite's
|
||||
/// ASCII-only <c>LOWER()</c> can under-match. Evaluated on the RAW query, never the lowercased copy:
|
||||
/// the trigger must not be coupled to the fold.
|
||||
/// </summary>
|
||||
internal static bool ContainsNonAscii(string value)
|
||||
{
|
||||
foreach (char c in value)
|
||||
{
|
||||
if (c > 0x7F)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
// Derived per-context rather than read from the TvContext.IsSqlite static on purpose. Nothing MECHANICALLY
|
||||
// stops that read -- ProviderStaticsWiringTests only parses the two composition roots for ASSIGNMENTS, not
|
||||
// readers -- but that test's scanner exemption for IsSqlite is justified in prose as "read only by
|
||||
// DbInitializer + DatabaseMigratorService, both host-only", and reading it here would make that reason
|
||||
// false while the test stayed green. Do not "simplify" this to IsSqlite.
|
||||
private static bool IsSqlite(TvContext dbContext) =>
|
||||
(dbContext.Database.ProviderName ?? string.Empty).Contains("Sqlite", StringComparison.OrdinalIgnoreCase);
|
||||
|
||||
/// <summary>
|
||||
/// Escapes the LIKE metacharacters in a user-supplied prefix and appends the trailing wildcard. The
|
||||
/// backslash MUST be escaped first, or the escapes added for <c>%</c>/<c>_</c> would themselves be
|
||||
/// re-escaped. Paired with an explicit <c>ESCAPE '\'</c> in <see cref="UnicodeFoldSql" />, since raw
|
||||
/// SQL gets none of the escaping EF does for <c>StartsWith</c>.
|
||||
/// </summary>
|
||||
internal static string EscapeLikePrefix(string value) =>
|
||||
value
|
||||
.Replace("\\", "\\\\", StringComparison.Ordinal)
|
||||
.Replace("%", "\\%", StringComparison.Ordinal)
|
||||
.Replace("_", "\\_", StringComparison.Ordinal) + "%";
|
||||
|
||||
/// <summary>
|
||||
/// One bounded, exact prefix query using the Unicode-correct fold. Unlike the list-valued walk this
|
||||
/// KEEPS its selectivity in SQL — it is a normal indexed-or-not <c>LIMIT</c>ed query exactly like the
|
||||
/// EF one it supplements, not a paged walk, so there is no row budget to blow and no reason to strip
|
||||
/// the discriminator predicates out of it.
|
||||
/// </summary>
|
||||
internal static string UnicodeFoldSql(string table, string column, string predicate)
|
||||
{
|
||||
var match = $"{UpperFunction}({column}) LIKE @Pattern ESCAPE '\\'";
|
||||
string where = predicate is null ? match : $"({predicate}) AND {match}";
|
||||
return $"SELECT DISTINCT {column} AS Value FROM {table} WHERE {where} ORDER BY {column} LIMIT @Limit";
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The tables/columns behind each EF-sourced field, mirroring <see cref="GetSource" /> 1:1.
|
||||
/// <para>
|
||||
/// The discriminator predicates must mirror EF's NULL semantics, not C#'s reading of the source.
|
||||
/// EF compiles <c>t.ExternalTypeId != Tag.NfoCountryTypeId</c> with null semantics, so a row whose
|
||||
/// <c>ExternalTypeId</c> is NULL IS included; plain SQL <c><></c> against NULL yields NULL and
|
||||
/// would silently drop it. Hence the explicit <c>IS NULL</c> arm.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private static IReadOnlyList<UnicodeFoldSource> GetUnicodeFoldSources(string name) => name switch
|
||||
{
|
||||
"genre" or "show_genre" => [new UnicodeFoldSource("Genre", "Name")],
|
||||
"studio" => [new UnicodeFoldSource("Studio", "Name")],
|
||||
"director" => [new UnicodeFoldSource("Director", "Name")],
|
||||
"writer" => [new UnicodeFoldSource("Writer", "Name")],
|
||||
"actor" => [new UnicodeFoldSource("Actor", "Name")],
|
||||
"artist" =>
|
||||
[
|
||||
new UnicodeFoldSource("ArtistMetadata", "Title"),
|
||||
new UnicodeFoldSource("MusicVideoArtist", "Name")
|
||||
],
|
||||
"tag" =>
|
||||
[
|
||||
new UnicodeFoldSource(
|
||||
"Tag",
|
||||
"Name",
|
||||
"ExternalTypeId IS NULL OR (ExternalTypeId <> @NfoCountryTypeId AND ExternalTypeId <> @PlexNetworkTypeId)",
|
||||
new Dictionary<string, object>
|
||||
{
|
||||
["NfoCountryTypeId"] = Tag.NfoCountryTypeId,
|
||||
["PlexNetworkTypeId"] = Tag.PlexNetworkTypeId
|
||||
})
|
||||
],
|
||||
"network" =>
|
||||
[
|
||||
new UnicodeFoldSource(
|
||||
"Tag",
|
||||
"Name",
|
||||
"ExternalTypeId = @PlexNetworkTypeId",
|
||||
new Dictionary<string, object> { ["PlexNetworkTypeId"] = Tag.PlexNetworkTypeId })
|
||||
],
|
||||
"collection" => [new UnicodeFoldSource("Collection", "Name")],
|
||||
"video_codec" =>
|
||||
[
|
||||
new UnicodeFoldSource(
|
||||
"MediaStream",
|
||||
"Codec",
|
||||
"MediaStreamKind = @VideoStreamKind AND Codec IS NOT NULL",
|
||||
new Dictionary<string, object> { ["VideoStreamKind"] = (int)MediaStreamKind.Video })
|
||||
],
|
||||
"album" =>
|
||||
[
|
||||
new UnicodeFoldSource("MusicVideoMetadata", "Album", "Album IS NOT NULL"),
|
||||
new UnicodeFoldSource("SongMetadata", "Album", "Album IS NOT NULL")
|
||||
],
|
||||
_ => []
|
||||
};
|
||||
|
||||
private static async Task<List<string>> GetUnicodeFoldedValues(
|
||||
TvContext dbContext,
|
||||
string name,
|
||||
string query,
|
||||
int limit,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
IReadOnlyList<UnicodeFoldSource> sources = GetUnicodeFoldSources(name);
|
||||
if (sources.Count == 0)
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
// CreateFunction is per-connection, so registration happens here, at the one call site that needs
|
||||
// the function, rather than through an EF connection interceptor: Dapper opens a closed connection
|
||||
// itself and a direct ADO open does not raise EF's interceptors, so an interceptor-based seam would
|
||||
// silently miss exactly this query. Opening first makes the registration order-independent.
|
||||
await dbContext.Database.OpenConnectionAsync(cancellationToken);
|
||||
TvContext.RegisterUnicodeCaseFunctions(dbContext.Connection);
|
||||
|
||||
string pattern = EscapeLikePrefix(query.ToUpperInvariant());
|
||||
var values = new List<string>();
|
||||
|
||||
foreach (UnicodeFoldSource source in sources)
|
||||
{
|
||||
var parameters = new DynamicParameters();
|
||||
parameters.Add("Pattern", pattern);
|
||||
parameters.Add("Limit", limit);
|
||||
if (source.Parameters is not null)
|
||||
{
|
||||
foreach ((string key, object value) in source.Parameters)
|
||||
{
|
||||
parameters.Add(key, value);
|
||||
}
|
||||
}
|
||||
|
||||
IEnumerable<string> rows = await dbContext.Connection.QueryAsync<string>(
|
||||
new CommandDefinition(
|
||||
UnicodeFoldSql(source.Table, source.Column, source.Predicate),
|
||||
parameters,
|
||||
cancellationToken: cancellationToken));
|
||||
|
||||
values.AddRange(rows.Where(v => !string.IsNullOrEmpty(v)));
|
||||
}
|
||||
|
||||
return values;
|
||||
}
|
||||
|
||||
private sealed record UnicodeFoldSource(
|
||||
string Table,
|
||||
string Column,
|
||||
string Predicate = null,
|
||||
IReadOnlyDictionary<string, object> Parameters = null);
|
||||
|
||||
/// <summary>
|
||||
/// Maps a field name onto the <c>SongMetadata</c> column that backs it as an <c>IList<string></c>.
|
||||
/// The returned value is a compile-time constant from this switch — never caller input — so it is safe
|
||||
/// to interpolate into the SQL in <see cref="ListValuedSql" />.
|
||||
/// </summary>
|
||||
private static string GetSongListValuedColumn(string name) => name switch
|
||||
{
|
||||
"artist" => "Artists",
|
||||
"album_artist" => "AlbumArtists",
|
||||
_ => null
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// Reads whole values out of a <c>SongMetadata</c> <c>IList<string></c> column.
|
||||
/// <para>
|
||||
/// EF maps these as primitive collections: one JSON array per row in a single <c>TEXT</c>/
|
||||
/// <c>longtext</c> column. Neither provider can project the elements server-side — SQLite needs
|
||||
/// the SQL <c>APPLY</c> operator it doesn't have, and Pomelo MySQL doesn't implement primitive
|
||||
/// collections at all — so there is no server-side <c>SELECT DISTINCT</c> over the elements.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// So the rows are walked in primary-key order, keyset-paged by row position, and split +
|
||||
/// exact-filtered in memory. All selectivity is in memory — the query's only condition is the
|
||||
/// cursor, a seek on the ordering key that never discards a row, so its <c>LIMIT</c> bounds the
|
||||
/// LOGICAL ROWS returned. See <see cref="ListValuedBatchRows" /> for the four revisions it took to
|
||||
/// get that right, and for what that bound does and does not cover.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private static async Task<List<string>> GetSongListValuedValues(
|
||||
TvContext dbContext,
|
||||
string column,
|
||||
string query,
|
||||
int limit,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
string sql = ListValuedSql(column);
|
||||
|
||||
var distinct = new System.Collections.Generic.HashSet<string>(StringComparer.Ordinal);
|
||||
var afterId = 0;
|
||||
var read = 0;
|
||||
|
||||
while (read < ListValuedMaxRowsRead && distinct.Count < limit)
|
||||
{
|
||||
int batch = Math.Min(ListValuedBatchRows, ListValuedMaxRowsRead - read);
|
||||
|
||||
List<ListValuedRow> rows = (await dbContext.Connection.QueryAsync<ListValuedRow>(
|
||||
new CommandDefinition(
|
||||
sql,
|
||||
new { AfterId = afterId, Batch = batch },
|
||||
cancellationToken: cancellationToken))).AsList();
|
||||
|
||||
if (rows.Count == 0)
|
||||
{
|
||||
break;
|
||||
}
|
||||
|
||||
read += rows.Count;
|
||||
afterId = rows[^1].Id;
|
||||
|
||||
foreach (ListValuedRow row in rows)
|
||||
{
|
||||
foreach (string element in ParseElements(row.Payload))
|
||||
{
|
||||
if (element.StartsWith(query, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
distinct.Add(element);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (rows.Count < batch)
|
||||
{
|
||||
// With no RESIDUAL predicate -- only the cursor, which selects a range rather than discarding
|
||||
// rows from it -- a short page can only mean the table is exhausted. It can never mean "this
|
||||
// stretch happened to match nothing", which is precisely why the residual predicate had to go.
|
||||
// Advancing from the last returned Id is safe for the same reason: nothing was filtered out
|
||||
// behind it, so no row can be skipped.
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return distinct.ToList();
|
||||
}
|
||||
|
||||
private static IEnumerable<string> ParseElements(string payload)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(payload))
|
||||
{
|
||||
return [];
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
return (JsonSerializer.Deserialize<string[]>(payload) ?? []).Where(e => !string.IsNullOrEmpty(e));
|
||||
}
|
||||
catch (JsonException)
|
||||
{
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// One keyset page of rows, by ROW POSITION rather than by <c>Id</c> value.
|
||||
/// <para>
|
||||
/// The only condition is the cursor — deliberately <b>no RESIDUAL predicate</b>: no <c>LIKE</c>, no
|
||||
/// <c>LOWER</c>, not even <c>IS NOT NULL</c>. The distinction that matters is not "no predicate"
|
||||
/// (the cursor is one); it is that <c>Id > @AfterId</c> is a <i>seekable predicate on the
|
||||
/// ordering key</i>, which positions the scan and never discards a row, whereas a residual
|
||||
/// predicate throws away rows the engine already produced. <c>LIMIT</c> only truncates what
|
||||
/// survives a residual predicate, so with one present it bounds the output rather than the row
|
||||
/// count — which is how every earlier revision scanned past its own bound. With none, <c>LIMIT n</c>
|
||||
/// yields <c>n</c> logical rows. Null payloads are dropped in memory by
|
||||
/// <see cref="ParseElements" />.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Note this pins the SQL string only. It cannot pin an execution plan, MVCC visibility work, or
|
||||
/// payload I/O — and on MySQL, using the index to satisfy <c>ORDER BY</c> is an optimizer choice,
|
||||
/// not a semantic guarantee.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
internal static string ListValuedSql(string column) =>
|
||||
$"SELECT Id, {column} AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch";
|
||||
|
||||
private sealed class ListValuedRow
|
||||
{
|
||||
public int Id { get; init; }
|
||||
|
||||
public string Payload { get; init; }
|
||||
}
|
||||
|
||||
private static async Task<List<string>> GetContentRatingValues(
|
||||
TvContext dbContext,
|
||||
string query,
|
||||
string qLower,
|
||||
int limit,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
@@ -513,22 +108,13 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
|
||||
.Where(cr => !string.IsNullOrEmpty(cr))
|
||||
.Distinct();
|
||||
|
||||
return FilterSortTake(split, query, limit);
|
||||
return FilterSortTake(split, qLower, limit);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The one in-memory filter/sort/take every field funnels through. Both the comparison and the ordering
|
||||
/// are ORDINAL on purpose: <c>UseRequestLocalization</c> honours <c>Accept-Language</c>, so the current
|
||||
/// culture is caller-controlled, and <c>ToLower()</c> plus the default (linguistic)
|
||||
/// <c>StartsWith(string)</c> would make the result depend on it — under <c>tr-TR</c>, <c>q=I</c> lowers
|
||||
/// to <c>ı</c> and stops matching <c>Istanbul</c>. Note this is the LAST stage only: a field sourced by
|
||||
/// a plain EF query has already been filtered and truncated by the database collation before it gets
|
||||
/// here, which ordinal semantics downstream cannot undo (ersatztv#668).
|
||||
/// </summary>
|
||||
private static List<string> FilterSortTake(IEnumerable<string> values, string query, int limit) =>
|
||||
private static List<string> FilterSortTake(IEnumerable<string> values, string qLower, int limit) =>
|
||||
values
|
||||
.Where(v => v.StartsWith(query, StringComparison.OrdinalIgnoreCase))
|
||||
.OrderBy(v => v, StringComparer.Ordinal)
|
||||
.Where(v => v.ToLower().StartsWith(qLower))
|
||||
.OrderBy(v => v)
|
||||
.Take(limit)
|
||||
.ToList();
|
||||
}
|
||||
|
||||
@@ -1,104 +0,0 @@
|
||||
using System.Text.RegularExpressions;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
namespace ErsatzTV.Architecture.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#491: <c>TvContext</c> carries settable provider statics (<c>LastInsertedRowId</c>,
|
||||
/// <c>CaseInsensitiveCollation</c>, <c>IsUniqueConstraintViolation</c>, …) that Infrastructure code
|
||||
/// reads at runtime. There are TWO composition roots that execute that Infrastructure code —
|
||||
/// <c>ErsatzTV/Startup.cs</c> (the host) and <c>ErsatzTV.Scanner/Program.cs</c> (a separate
|
||||
/// executable launched per scan by <c>CallLibraryScannerHandler</c>) — and each wires the statics in
|
||||
/// its own copy of the provider branch.
|
||||
/// <para>
|
||||
/// The failure mode this guards is "a static nobody assigned": #491 added
|
||||
/// <c>IsUniqueConstraintViolation</c> to <c>Startup</c> only, so every production caller of
|
||||
/// <c>GetOrAddFolder</c> (all of which live in the scanner) silently kept the conservative
|
||||
/// <c>_ => false</c> default and the new catch was inert. Nothing about that is visible in a
|
||||
/// unit test, because every test harness wires the classifier itself.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Source-level rather than reflective on purpose: the wiring lives inside a host-builder
|
||||
/// lambda that cannot be invoked without standing up a real application, and the thing being
|
||||
/// asserted is precisely that a line of code exists in both files.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
[TestFixture]
|
||||
public class ProviderStaticsWiringTests
|
||||
{
|
||||
/// <summary>
|
||||
/// Statics the host wires that the scanner deliberately does not. Add to this only with a reason:
|
||||
/// the default must be provably harmless in the scanner process.
|
||||
/// </summary>
|
||||
private static readonly Dictionary<string, string> ScannerExemptions = new()
|
||||
{
|
||||
// Only read by DbInitializer / DatabaseMigratorService, which run in the host exclusively; no
|
||||
// Infrastructure code on a scan path reads it. Pre-dates #491.
|
||||
["IsSqlite"] = "read only by DbInitializer + DatabaseMigratorService, both host-only"
|
||||
};
|
||||
|
||||
private static string HostSource => ReadRepoFile(Path.Combine("ErsatzTV", "Startup.cs"));
|
||||
|
||||
private static string ScannerSource => ReadRepoFile(Path.Combine("ErsatzTV.Scanner", "Program.cs"));
|
||||
|
||||
[Test]
|
||||
public void Scanner_should_wire_every_TvContext_provider_static_the_host_wires()
|
||||
{
|
||||
HashSet<string> host = AssignedStatics(HostSource);
|
||||
HashSet<string> scanner = AssignedStatics(ScannerSource);
|
||||
|
||||
// sanity: the parser found the wiring at all, so a rename can't turn this test into a no-op
|
||||
host.ShouldContain("LastInsertedRowId");
|
||||
host.ShouldContain("IsUniqueConstraintViolation");
|
||||
scanner.ShouldContain("LastInsertedRowId");
|
||||
|
||||
List<string> missing = host
|
||||
.Except(scanner)
|
||||
.Except(ScannerExemptions.Keys)
|
||||
.OrderBy(name => name, StringComparer.Ordinal)
|
||||
.ToList();
|
||||
|
||||
missing.ShouldBeEmpty(
|
||||
"ErsatzTV.Scanner/Program.cs does not assign TvContext static(s) that ErsatzTV/Startup.cs "
|
||||
+ $"assigns: {string.Join(", ", missing)}. The scanner is a separate process, so an unassigned "
|
||||
+ "static keeps its default in every library scan. Wire it in BOTH provider branches, or add "
|
||||
+ "it to ScannerExemptions with a reason if the default is provably harmless there.");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Both_hosts_should_wire_the_unique_constraint_classifier_for_both_providers()
|
||||
{
|
||||
// The specific #491 regression, asserted directly rather than via set arithmetic: the classifier
|
||||
// must be pointed at a real provider implementation on BOTH branches of BOTH composition roots.
|
||||
foreach ((string name, string source) in new[] { ("host", HostSource), ("scanner", ScannerSource) })
|
||||
{
|
||||
source.ShouldContain(
|
||||
"TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation",
|
||||
customMessage: $"{name} does not wire the Sqlite unique-constraint classifier");
|
||||
source.ShouldContain(
|
||||
"TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation",
|
||||
customMessage: $"{name} does not wire the MySql unique-constraint classifier");
|
||||
}
|
||||
}
|
||||
|
||||
private static HashSet<string> AssignedStatics(string source) =>
|
||||
Regex.Matches(source, @"\bTvContext\.(?<name>[A-Za-z_][A-Za-z0-9_]*)\s*=[^=]")
|
||||
.Select(m => m.Groups["name"].Value)
|
||||
.ToHashSet(StringComparer.Ordinal);
|
||||
|
||||
private static string ReadRepoFile(string relativePath)
|
||||
{
|
||||
var directory = new DirectoryInfo(AppContext.BaseDirectory);
|
||||
while (directory is not null && !File.Exists(Path.Combine(directory.FullName, "ErsatzTV.sln")))
|
||||
{
|
||||
directory = directory.Parent;
|
||||
}
|
||||
|
||||
directory.ShouldNotBeNull("could not locate the repository root (no ErsatzTV.sln above the test binary)");
|
||||
|
||||
string path = Path.Combine(directory!.FullName, relativePath);
|
||||
File.Exists(path).ShouldBeTrue($"expected source file not found: {path}");
|
||||
return File.ReadAllText(path);
|
||||
}
|
||||
}
|
||||
@@ -1,129 +0,0 @@
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Core.FFmpeg;
|
||||
using ErsatzTV.Core.Interfaces.FFmpeg;
|
||||
using ErsatzTV.Core.Interfaces.Images;
|
||||
using ErsatzTV.Core.Interfaces.Metadata;
|
||||
using ErsatzTV.FFmpeg.State;
|
||||
using NSubstitute;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
namespace ErsatzTV.Core.Tests.FFmpeg;
|
||||
|
||||
[TestFixture]
|
||||
public class SongVideoGeneratorTests
|
||||
{
|
||||
private ITempFilePool _tempFilePool;
|
||||
private IImageCache _imageCache;
|
||||
private IFFmpegProcessService _ffmpegProcessService;
|
||||
private ILocalFileSystem _localFileSystem;
|
||||
private SongVideoGenerator _songVideoGenerator;
|
||||
private string _tempSubtitleFile;
|
||||
|
||||
[SetUp]
|
||||
public void SetUp()
|
||||
{
|
||||
_tempSubtitleFile = Path.Combine(Path.GetTempPath(), $"{Guid.NewGuid()}.ass");
|
||||
|
||||
_tempFilePool = Substitute.For<ITempFilePool>();
|
||||
_tempFilePool.GetNextTempFile(Arg.Any<TempFileCategory>()).Returns(_tempSubtitleFile);
|
||||
|
||||
_imageCache = Substitute.For<IImageCache>();
|
||||
_imageCache.GetPathForImage(Arg.Any<string>(), Arg.Any<ArtworkKind>(), Arg.Any<Option<int>>())
|
||||
.Returns("/fake/watermark.png");
|
||||
|
||||
_ffmpegProcessService = Substitute.For<IFFmpegProcessService>();
|
||||
_ffmpegProcessService.GenerateSongImage(
|
||||
Arg.Any<string>(),
|
||||
Arg.Any<string>(),
|
||||
Arg.Any<Option<string>>(),
|
||||
Arg.Any<Channel>(),
|
||||
Arg.Any<MediaVersion>(),
|
||||
Arg.Any<string>(),
|
||||
Arg.Any<bool>(),
|
||||
Arg.Any<Option<string>>(),
|
||||
Arg.Any<WatermarkLocation>(),
|
||||
Arg.Any<int>(),
|
||||
Arg.Any<int>(),
|
||||
Arg.Any<int>(),
|
||||
Arg.Any<CancellationToken>())
|
||||
.Returns(Either<BaseError, string>.Right("/fake/song-image.png"));
|
||||
|
||||
_localFileSystem = Substitute.For<ILocalFileSystem>();
|
||||
_localFileSystem.GetCustomOrDefaultFile(Arg.Any<string>(), Arg.Any<string>())
|
||||
.Returns("/fake/background.png");
|
||||
|
||||
_songVideoGenerator = new SongVideoGenerator(
|
||||
_tempFilePool,
|
||||
_imageCache,
|
||||
_ffmpegProcessService,
|
||||
_localFileSystem);
|
||||
}
|
||||
|
||||
[TearDown]
|
||||
public void TearDown()
|
||||
{
|
||||
if (_tempSubtitleFile is not null && File.Exists(_tempSubtitleFile))
|
||||
{
|
||||
File.Delete(_tempSubtitleFile);
|
||||
}
|
||||
}
|
||||
|
||||
private static Channel BuildChannel()
|
||||
{
|
||||
var resolution = new Resolution { Width = 1920, Height = 1080 };
|
||||
FFmpegProfile ffmpegProfile = FFmpegProfile.New("test", resolution);
|
||||
|
||||
return new Channel(Guid.NewGuid())
|
||||
{
|
||||
Number = "1",
|
||||
Name = "Test Channel",
|
||||
FFmpegProfile = ffmpegProfile,
|
||||
SongVideoMode = ChannelSongVideoMode.Default
|
||||
};
|
||||
}
|
||||
|
||||
private static Song BuildUntaggedSong()
|
||||
{
|
||||
// an untagged song: FallbackMetadataProvider.GetSongMetadata never assigns
|
||||
// Artists/AlbumArtists, so they persist (and materialize) as null (ersatztv#691)
|
||||
var metadata = new SongMetadata
|
||||
{
|
||||
MetadataKind = MetadataKind.Fallback,
|
||||
Title = "Untagged Song",
|
||||
Artwork = [],
|
||||
Artists = null,
|
||||
AlbumArtists = null
|
||||
};
|
||||
|
||||
return new Song
|
||||
{
|
||||
SongMetadata = [metadata],
|
||||
MediaVersions = []
|
||||
};
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task GenerateSongVideo_should_not_throw_when_artists_and_album_artists_are_null()
|
||||
{
|
||||
Song song = BuildUntaggedSong();
|
||||
Channel channel = BuildChannel();
|
||||
|
||||
// SongVideoGenerator randomly picks between two rendering styles (and dereferences
|
||||
// metadata.Artists/AlbumArtists differently in each); loop enough times that both
|
||||
// branches -- including the AlbumArtists.Filter(... Artists.Contains ...) branch --
|
||||
// are exercised with overwhelming probability, so the null guard is proven on both.
|
||||
for (var i = 0; i < 25; i++)
|
||||
{
|
||||
Tuple<string, MediaVersion> result = await _songVideoGenerator.GenerateSongVideo(
|
||||
song,
|
||||
channel,
|
||||
"/usr/bin/ffmpeg",
|
||||
"/usr/bin/ffprobe",
|
||||
CancellationToken.None);
|
||||
|
||||
result.ShouldNotBeNull();
|
||||
result.Item1.ShouldBe("/fake/song-image.png");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,589 +0,0 @@
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Core.Domain.Filler;
|
||||
using ErsatzTV.Core.Domain.Scheduling;
|
||||
using ErsatzTV.Core.FFmpeg;
|
||||
using ErsatzTV.Core.Images;
|
||||
using ErsatzTV.Core.Interfaces.FFmpeg;
|
||||
using ErsatzTV.Core.Interfaces.Images;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using NSubstitute;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
using Testably.Abstractions.Testing;
|
||||
|
||||
namespace ErsatzTV.Core.Tests.FFmpeg;
|
||||
|
||||
/// <summary>
|
||||
/// Pins ersatztv#510: a watermark attached through a DECO resolves by exactly the same policy as the three
|
||||
/// precedence levels (playout item, channel, global).
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Before #510 the deco path had its own copy of the image-source switch that resolved paths *unchecked*, so
|
||||
/// one channel could disagree with itself about whether a bug rendered purely by how the watermark was
|
||||
/// attached. The divergence covered all three <see cref="ChannelWatermarkImageSource" /> values, not just
|
||||
/// <c>ChannelLogo</c>:
|
||||
/// <list type="bullet">
|
||||
/// <item>a missing local file was handed downstream as a dead path (and a dead LOCAL path can reach
|
||||
/// ffmpeg as a bare <c>-i</c> argument via <c>CanUseFFmpegNativeWatermark</c>, so it is worse than a
|
||||
/// skipped overlay);</item>
|
||||
/// <item>an un-migrated external-URL logo was handed down as a renderable URL, which
|
||||
/// <c>graphics.channel-logo-caching</c> (#525) forbids the render path from fetching;</item>
|
||||
/// <item>a channel with no logo artwork got the generated-initials localhost URL, which a live-E2E on a
|
||||
/// real transcoded frame confirmed DID render — the deco path only. #510 resolved that split in favour
|
||||
/// of "no on-screen bug" everywhere.</item>
|
||||
/// </list>
|
||||
/// The <c>Deco_And_Channel_Level_Should_Resolve_Identically</c> cases are the structural guard: they assert
|
||||
/// the two callers agree, so re-introducing a per-caller policy fails here rather than silently in prod.
|
||||
/// </remarks>
|
||||
[TestFixture]
|
||||
public class WatermarkSelectorDecoResolutionTests
|
||||
{
|
||||
private const string ExternalLogoUrl = "https://cdn.example.com/logos/channel.png";
|
||||
private const string LogoStoredPath = "abc123.png";
|
||||
private const string LogoCachePath = "/cache/logos/ab/abc123.png";
|
||||
private const string CustomStoredPath = "def456.png";
|
||||
private const string CustomCachePath = "/cache/watermarks/de/def456.png";
|
||||
private const string ResourceImage = "song-progress.png";
|
||||
|
||||
private static string ResourcePath => Path.Combine(FileSystemLayout.ResourcesCacheFolder, ResourceImage);
|
||||
|
||||
/// <summary>Builds a selector whose mock filesystem contains exactly <paramref name="existingFiles" />.</summary>
|
||||
private static WatermarkSelector Selector(Deco playoutDeco, params string[] existingFiles)
|
||||
{
|
||||
// one Initialize() call, chained -- calling it per file would leave "does a second Initialize()
|
||||
// preserve the first file?" untested, and a silently under-seeded filesystem makes a
|
||||
// "resolves to nothing" assertion pass for the wrong reason
|
||||
var mockFileSystem = new MockFileSystem();
|
||||
if (existingFiles.Length > 0)
|
||||
{
|
||||
var initialized = mockFileSystem.Initialize().WithFile(existingFiles[0]);
|
||||
foreach (string file in existingFiles.Skip(1))
|
||||
{
|
||||
initialized = initialized.WithFile(file);
|
||||
}
|
||||
}
|
||||
|
||||
var fakeImageCache = Substitute.For<IImageCache>();
|
||||
fakeImageCache.GetPathForImage(Arg.Any<string>(), Arg.Is(ArtworkKind.Logo), Arg.Any<Option<int>>())
|
||||
.Returns(_ => LogoCachePath);
|
||||
fakeImageCache.GetPathForImage(Arg.Any<string>(), Arg.Is(ArtworkKind.Watermark), Arg.Any<Option<int>>())
|
||||
.Returns(_ => CustomCachePath);
|
||||
|
||||
// Faithful to the real ImageCache.GetPathForImage, which does fileName[..2] and therefore THROWS on a
|
||||
// blank/null name. Modelling that is what makes the blank-image guard tests mutation-sensitive: before
|
||||
// #510 the channel and global arms had no guard and this threw out of stream startup.
|
||||
fakeImageCache
|
||||
.GetPathForImage(
|
||||
Arg.Is<string>(s => string.IsNullOrWhiteSpace(s)),
|
||||
Arg.Any<ArtworkKind>(),
|
||||
Arg.Any<Option<int>>())
|
||||
.Returns<string>(_ => throw new ArgumentOutOfRangeException(nameof(IImageCache.GetPathForImage)));
|
||||
|
||||
var decoSelector = Substitute.For<IDecoSelector>();
|
||||
decoSelector.GetDecoEntries(Arg.Any<Playout>(), Arg.Any<DateTimeOffset>())
|
||||
.Returns(new DecoEntries(Option<Deco>.None, Optional(playoutDeco)));
|
||||
|
||||
return new WatermarkSelector(
|
||||
mockFileSystem,
|
||||
fakeImageCache,
|
||||
decoSelector,
|
||||
NullLogger<WatermarkSelector>.Instance);
|
||||
}
|
||||
|
||||
private static ChannelWatermark Watermark(ChannelWatermarkImageSource source, string image = "") =>
|
||||
new()
|
||||
{
|
||||
Id = 7,
|
||||
Name = "Deco Bug",
|
||||
ImageSource = source,
|
||||
Image = image,
|
||||
Mode = ChannelWatermarkMode.Permanent
|
||||
};
|
||||
|
||||
private static Deco DecoWith(ChannelWatermark watermark) =>
|
||||
new()
|
||||
{
|
||||
Id = 1,
|
||||
Name = "Test Deco",
|
||||
WatermarkMode = DecoMode.Override,
|
||||
UseWatermarkDuringFiller = true,
|
||||
DecoWatermarks = [new DecoWatermark { WatermarkId = watermark.Id, Watermark = watermark }],
|
||||
Watermarks = []
|
||||
};
|
||||
|
||||
private static Channel ChannelWith(string logoPath, ChannelWatermark channelWatermark = null)
|
||||
{
|
||||
var channel = new Channel(Guid.Empty)
|
||||
{
|
||||
Id = 1,
|
||||
Number = "1",
|
||||
Name = "Test",
|
||||
StreamingMode = StreamingMode.TransportStream,
|
||||
Artwork = [],
|
||||
Watermark = channelWatermark,
|
||||
WatermarkId = channelWatermark?.Id
|
||||
};
|
||||
|
||||
if (logoPath is not null)
|
||||
{
|
||||
channel.Artwork.Add(new Artwork { ArtworkKind = ArtworkKind.Logo, Path = logoPath });
|
||||
}
|
||||
|
||||
return channel;
|
||||
}
|
||||
|
||||
private static PlayoutItem PlayoutItem() =>
|
||||
new()
|
||||
{
|
||||
FillerKind = FillerKind.None,
|
||||
DisableWatermarks = false,
|
||||
Watermarks = [],
|
||||
Playout = new Playout()
|
||||
};
|
||||
|
||||
private static List<WatermarkOptions> SelectViaDeco(
|
||||
ChannelWatermark watermark,
|
||||
Channel channel,
|
||||
params string[] existingFiles)
|
||||
{
|
||||
WatermarkSelector selector = Selector(DecoWith(watermark), existingFiles);
|
||||
return selector.SelectWatermarks(
|
||||
Option<ChannelWatermark>.None,
|
||||
channel,
|
||||
PlayoutItem(),
|
||||
DateTimeOffset.Now);
|
||||
}
|
||||
|
||||
// ---- positive control: the arrangement CAN produce a watermark ------------------------------
|
||||
//
|
||||
// Without this, every "resolves to nothing" assertion below could pass vacuously (a broken deco
|
||||
// arrangement that never reaches the resolver at all looks identical to a correct refusal).
|
||||
|
||||
[Test]
|
||||
public void Deco_ChannelLogo_Should_Use_Cached_Path_When_Local_Logo_Exists()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel, LogoCachePath);
|
||||
|
||||
result.Count.ShouldBe(1);
|
||||
result[0].ImagePath.ShouldBe(LogoCachePath);
|
||||
}
|
||||
|
||||
// ---- ChannelLogo: the three cases #510 was filed for ----------------------------------------
|
||||
|
||||
[Test]
|
||||
public void Deco_ChannelLogo_Should_Be_Ignored_When_Logo_Is_An_External_Url()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
Channel channel = ChannelWith(ExternalLogoUrl);
|
||||
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
|
||||
|
||||
result.ShouldBeEmpty();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Deco_ChannelLogo_Should_Be_Ignored_When_Local_Logo_File_Is_Missing()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
// nothing on disk
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
|
||||
|
||||
result.ShouldBeEmpty();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The #510 policy decision: with no logo artwork the generated-initials fallback is NOT used. It
|
||||
/// genuinely rendered here before (confirmed by live-E2E on a real frame), so this is a deliberate,
|
||||
/// recorded behavior change — not a no-op cleanup.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void Deco_ChannelLogo_Should_Be_Ignored_When_Channel_Has_No_Logo_Artwork()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
Channel channel = ChannelWith(null);
|
||||
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
|
||||
|
||||
result.ShouldBeEmpty();
|
||||
|
||||
// Folded in from a separate test that asserted only this. On its own it was vacuous — an empty list
|
||||
// trivially contains no URL — so it is a second assertion here rather than a test implying independent
|
||||
// coverage. It earns its place by naming the value if this ever starts returning options again (#652).
|
||||
result.Select(o => o.ImagePath)
|
||||
.ShouldNotContain(ChannelLogoGenerator.GenerateChannelLogoUrl(channel));
|
||||
}
|
||||
|
||||
// ---- Custom and Resource: the two arms #510 did not mention but that diverged too -----------
|
||||
|
||||
[Test]
|
||||
public void Deco_Custom_Should_Use_Cached_Path_When_File_Exists()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel, CustomCachePath);
|
||||
|
||||
result.Count.ShouldBe(1);
|
||||
result[0].ImagePath.ShouldBe(CustomCachePath);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Deco_Custom_Should_Be_Ignored_When_File_Is_Missing()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
|
||||
|
||||
result.ShouldBeEmpty();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Deco_Custom_Should_Be_Ignored_When_Image_Is_Blank()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Custom, " ");
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel, CustomCachePath);
|
||||
|
||||
result.ShouldBeEmpty();
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Deco_Resource_Should_Use_Resource_Path_When_File_Exists()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Resource, ResourceImage);
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel, ResourcePath);
|
||||
|
||||
result.Count.ShouldBe(1);
|
||||
result[0].ImagePath.ShouldBe(ResourcePath);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Deco_Resource_Should_Be_Ignored_When_File_Is_Missing()
|
||||
{
|
||||
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Resource, ResourceImage);
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
|
||||
|
||||
result.ShouldBeEmpty();
|
||||
}
|
||||
|
||||
// ---- non-deco consequences of the SAME unification -------------------------------------------
|
||||
//
|
||||
// These pin precedence-level behavior rather than deco behavior, but they exist because of the #510
|
||||
// unification: one is the single piece of per-caller policy deliberately kept, the others are arms that
|
||||
// used to throw. Without them a future refactor can delete the survivor, or re-introduce the crash, with
|
||||
// a fully green suite.
|
||||
|
||||
/// <summary>
|
||||
/// The one surviving per-caller policy: a playout-item `Custom` watermark with a blank image falls
|
||||
/// THROUGH to the channel/global watermark rather than resolving to "no watermark". Unifying
|
||||
/// resolution must not change which watermark WINS.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void Blank_Custom_Playout_Item_Watermark_Should_Fall_Through_To_Channel_Watermark()
|
||||
{
|
||||
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.Custom, " ");
|
||||
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
|
||||
channelWatermark.Id = 8;
|
||||
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
|
||||
|
||||
Option<WatermarkOptions> result = Selector(null, CustomCachePath)
|
||||
.GetWatermarkOptions(channel, playoutItemWatermark, Option<ChannelWatermark>.None);
|
||||
|
||||
// the CHANNEL watermark wins -- not None, and not the blank playout-item one
|
||||
result.IsSome.ShouldBeTrue();
|
||||
WatermarkOptions options = result.IfNone(() => throw new InvalidOperationException());
|
||||
options.ImagePath.ShouldBe(CustomCachePath);
|
||||
options.Watermark.Id.ShouldBe(8);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Before #510 the channel and global arms had no blank-image guard, so they reached
|
||||
/// <c>ImageCache.GetPathForImage</c> whose <c>fileName[..2]</c> threw out of stream startup. Now a
|
||||
/// warning plus no watermark.
|
||||
/// </summary>
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void Channel_Level_Blank_Custom_Watermark_Should_Resolve_To_None_Not_Throw(string image)
|
||||
{
|
||||
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.Custom, image);
|
||||
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
|
||||
|
||||
Option<WatermarkOptions> result = Should.NotThrow(
|
||||
() => Selector(null).GetWatermarkOptions(
|
||||
channel,
|
||||
Option<ChannelWatermark>.None,
|
||||
Option<ChannelWatermark>.None));
|
||||
|
||||
result.IsNone.ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Same fall-through, but landing on the GLOBAL watermark — the channel-level variant above cannot
|
||||
/// distinguish "fell through correctly" from "stopped at the channel by accident".
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void Blank_Custom_Playout_Item_Watermark_Should_Fall_Through_To_Global_Watermark()
|
||||
{
|
||||
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.Custom, " ");
|
||||
ChannelWatermark globalWatermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
|
||||
globalWatermark.Id = 9;
|
||||
|
||||
// no channel-level watermark, so the only remaining candidate is the global one
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
Option<WatermarkOptions> result = Selector(null, CustomCachePath)
|
||||
.GetWatermarkOptions(channel, playoutItemWatermark, globalWatermark);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
WatermarkOptions options = result.IfNone(() => throw new InvalidOperationException());
|
||||
options.ImagePath.ShouldBe(CustomCachePath);
|
||||
options.Watermark.Id.ShouldBe(9);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The complement of the fall-through cases: a NON-blank custom image whose file is merely missing must
|
||||
/// NOT fall through — it resolves to "no watermark" and the channel watermark never gets a turn.
|
||||
/// Without this, widening the blank-image guard to "any unresolvable custom" would pass unnoticed.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// The channel-level fallback is deliberately an INDEPENDENTLY RESOLVABLE `ChannelLogo` watermark whose
|
||||
/// cached file exists. An earlier version of this test gave the fallback the same missing custom path as
|
||||
/// the playout-item watermark, which made it unfalsifiable: a wrongly-widened guard would have fallen
|
||||
/// through to a fallback that also resolved to None, so the assertion held either way.
|
||||
/// </remarks>
|
||||
[Test]
|
||||
public void Missing_But_Named_Custom_Playout_Item_Watermark_Should_Not_Fall_Through()
|
||||
{
|
||||
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
|
||||
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
channelWatermark.Id = 8;
|
||||
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
|
||||
|
||||
// the channel logo's cached file EXISTS, so a fall-through would return it and fail this test;
|
||||
// the custom watermark's file does not, so the playout-item watermark is unresolvable
|
||||
Option<WatermarkOptions> result = Selector(null, LogoCachePath)
|
||||
.GetWatermarkOptions(channel, playoutItemWatermark, Option<ChannelWatermark>.None);
|
||||
|
||||
result.IsNone.ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Positive control for the test above: the same arrangement, but with the playout-item watermark BLANK
|
||||
/// rather than missing, must fall through and return the resolvable channel logo. Together the pair
|
||||
/// shows the guard distinguishes blank from unresolvable, rather than both landing on None.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Parameterized over all three blank forms because the guard is <c>IsNullOrWhiteSpace</c>: testing only
|
||||
/// <c>" "</c> would let a mutation to <c>image == " "</c> pass while silently breaking fall-through
|
||||
/// for <c>null</c> and <c>""</c> — and <c>null</c> is the form the API actually persists.
|
||||
/// </remarks>
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
[TestCase(" ")]
|
||||
public void Blank_Custom_Playout_Item_Watermark_Should_Fall_Through_To_A_Resolvable_Channel_Logo(string image)
|
||||
{
|
||||
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.Custom, image);
|
||||
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
channelWatermark.Id = 8;
|
||||
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
|
||||
|
||||
Option<WatermarkOptions> result = Selector(null, LogoCachePath)
|
||||
.GetWatermarkOptions(channel, playoutItemWatermark, Option<ChannelWatermark>.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfNone(() => throw new InvalidOperationException()).ImagePath.ShouldBe(LogoCachePath);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Pins the <c>ImageSource is Custom</c> half of the blank-image guard, which nothing else covers.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// A <c>ChannelLogo</c> watermark's <c>Image</c> is NORMALLY blank — the API persists `Image = null` for
|
||||
/// every non-`Custom` source — so if the guard's `is Custom` discriminator were dropped, leaving only
|
||||
/// `IsNullOrWhiteSpace(Image)`, every playout-item `ChannelLogo` watermark would fall through to
|
||||
/// channel/global instead of resolving the channel's own logo. This test fails on that mutation: the
|
||||
/// playout-item watermark carries a distinguishing Id, so falling through is observable even though both
|
||||
/// levels would resolve to the same cached path.
|
||||
/// </remarks>
|
||||
[Test]
|
||||
public void Blank_Image_ChannelLogo_Playout_Item_Watermark_Should_Win_And_Not_Fall_Through()
|
||||
{
|
||||
// Image is left blank, exactly as the API stores a ChannelLogo watermark
|
||||
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
playoutItemWatermark.Id = 42;
|
||||
|
||||
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
channelWatermark.Id = 8;
|
||||
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
|
||||
|
||||
Option<WatermarkOptions> result = Selector(null, LogoCachePath)
|
||||
.GetWatermarkOptions(channel, playoutItemWatermark, Option<ChannelWatermark>.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
WatermarkOptions options = result.IfNone(() => throw new InvalidOperationException());
|
||||
options.ImagePath.ShouldBe(LogoCachePath);
|
||||
|
||||
// the PLAYOUT-ITEM watermark won; a fall-through would have returned the channel's (Id 8)
|
||||
options.Watermark.Id.ShouldBe(42);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// `CreateWatermarkHandler`/`UpdateWatermarkHandler` write `Image = null` for every non-`Custom`
|
||||
/// watermark, so an API-created `Resource` watermark hits `Path.Combine(folder, null)` — an
|
||||
/// `ArgumentNullException` out of stream startup. Uses the persisted shape (null), not a hand-made
|
||||
/// filename, which is what the rest of the fixture would otherwise assume.
|
||||
/// </summary>
|
||||
[TestCase(null)]
|
||||
[TestCase("")]
|
||||
public void Resource_Watermark_With_No_Image_Name_Should_Resolve_To_None_Not_Throw(string image)
|
||||
{
|
||||
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.Resource, image);
|
||||
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
|
||||
|
||||
Option<WatermarkOptions> result = Should.NotThrow(
|
||||
() => Selector(null).GetWatermarkOptions(
|
||||
channel,
|
||||
Option<ChannelWatermark>.None,
|
||||
Option<ChannelWatermark>.None));
|
||||
|
||||
result.IsNone.ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Dropping an unresolvable watermark shortens the list handed to
|
||||
/// <c>CanUseFFmpegNativeWatermark</c>, whose predicate includes `Count == 1`. So this is also the pin on
|
||||
/// the observable routing change: two attached permanent watermarks, one missing, now yield ONE option
|
||||
/// (ffmpeg-native) where they previously yielded two (graphics engine).
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void Deco_With_One_Valid_And_One_Missing_Watermark_Should_Return_Only_The_Valid_One()
|
||||
{
|
||||
ChannelWatermark valid = Watermark(ChannelWatermarkImageSource.ChannelLogo);
|
||||
ChannelWatermark missing = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
|
||||
missing.Id = 8;
|
||||
|
||||
var deco = new Deco
|
||||
{
|
||||
Id = 1,
|
||||
Name = "Test Deco",
|
||||
WatermarkMode = DecoMode.Override,
|
||||
UseWatermarkDuringFiller = true,
|
||||
DecoWatermarks =
|
||||
[
|
||||
new DecoWatermark { WatermarkId = valid.Id, Watermark = valid },
|
||||
new DecoWatermark { WatermarkId = missing.Id, Watermark = missing }
|
||||
],
|
||||
Watermarks = []
|
||||
};
|
||||
|
||||
// only the channel logo's cached file exists; the custom watermark's does not
|
||||
List<WatermarkOptions> result = Selector(deco, LogoCachePath).SelectWatermarks(
|
||||
Option<ChannelWatermark>.None,
|
||||
ChannelWith(LogoStoredPath),
|
||||
PlayoutItem(),
|
||||
DateTimeOffset.Now);
|
||||
|
||||
result.Count.ShouldBe(1);
|
||||
result[0].ImagePath.ShouldBe(LogoCachePath);
|
||||
|
||||
// The routing claim itself, not just the filtering: call the real predicate. Asserting Count == 1 alone
|
||||
// would leave the decision record's "now routes ffmpeg-native" statement unpinned, since the decision
|
||||
// lives in FFmpegLibraryProcessService rather than in the selector.
|
||||
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark(0, result).ShouldBeTrue();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Before #510 the global arm had no <c>Resource</c> case and hit <c>default: throw</c>.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void Global_Level_Resource_Watermark_Should_Resolve_Instead_Of_Throwing()
|
||||
{
|
||||
ChannelWatermark globalWatermark = Watermark(ChannelWatermarkImageSource.Resource, ResourceImage);
|
||||
Channel channel = ChannelWith(LogoStoredPath);
|
||||
|
||||
Option<WatermarkOptions> result = Should.NotThrow(
|
||||
() => Selector(null, ResourcePath).GetWatermarkOptions(
|
||||
channel,
|
||||
Option<ChannelWatermark>.None,
|
||||
globalWatermark));
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfNone(() => throw new InvalidOperationException()).ImagePath.ShouldBe(ResourcePath);
|
||||
}
|
||||
|
||||
// ---- the structural guard: deco and channel-level must agree, case for case ------------------
|
||||
|
||||
private static IEnumerable<TestCaseData> ParityCases()
|
||||
{
|
||||
// (image source, watermark.Image, channel logo path, files that exist)
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.ChannelLogo, "", LogoStoredPath, new[] { LogoCachePath })
|
||||
.SetName("ChannelLogo, local file present");
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.ChannelLogo, "", LogoStoredPath, Array.Empty<string>())
|
||||
.SetName("ChannelLogo, local file missing");
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.ChannelLogo, "", ExternalLogoUrl, Array.Empty<string>())
|
||||
.SetName("ChannelLogo, external URL");
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.ChannelLogo, "", null, Array.Empty<string>())
|
||||
.SetName("ChannelLogo, no logo artwork");
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.Custom, CustomStoredPath, LogoStoredPath, new[] { CustomCachePath })
|
||||
.SetName("Custom, file present");
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.Custom, CustomStoredPath, LogoStoredPath, Array.Empty<string>())
|
||||
.SetName("Custom, file missing");
|
||||
// Both sides agree here by construction (each returns nothing), which is the point: it documents that
|
||||
// the blank-image fall-through asymmetry lives ONLY at the playout-item level -- covered by
|
||||
// Blank_Custom_Playout_Item_Watermark_Should_Fall_Through_To_Channel_Watermark -- rather than leaving
|
||||
// the omission looking like an evasion.
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.Custom, " ", LogoStoredPath, Array.Empty<string>())
|
||||
.SetName("Custom, blank image");
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.Resource, ResourceImage, LogoStoredPath, new[] { ResourcePath })
|
||||
.SetName("Resource, file present");
|
||||
yield return new TestCaseData(
|
||||
ChannelWatermarkImageSource.Resource, ResourceImage, LogoStoredPath, Array.Empty<string>())
|
||||
.SetName("Resource, file missing");
|
||||
}
|
||||
|
||||
[TestCaseSource(nameof(ParityCases))]
|
||||
public void Deco_And_Channel_Level_Should_Resolve_Identically(
|
||||
ChannelWatermarkImageSource source,
|
||||
string image,
|
||||
string logoPath,
|
||||
string[] existingFiles)
|
||||
{
|
||||
// deco path
|
||||
ChannelWatermark decoWatermark = Watermark(source, image);
|
||||
List<WatermarkOptions> viaDeco = SelectViaDeco(decoWatermark, ChannelWith(logoPath), existingFiles);
|
||||
|
||||
// channel precedence level, same watermark definition and same channel
|
||||
ChannelWatermark channelWatermark = Watermark(source, image);
|
||||
Channel channel = ChannelWith(logoPath, channelWatermark);
|
||||
Option<WatermarkOptions> viaChannel = Selector(null, existingFiles)
|
||||
.GetWatermarkOptions(channel, Option<ChannelWatermark>.None, Option<ChannelWatermark>.None);
|
||||
|
||||
List<string> decoPaths = viaDeco.Select(o => o.ImagePath).ToList();
|
||||
|
||||
// built explicitly rather than via Option.ToList(), which yields a LanguageExt Lst<string>
|
||||
var channelPaths = new List<string>();
|
||||
viaChannel.IfSome(o => channelPaths.Add(o.ImagePath));
|
||||
|
||||
decoPaths.ShouldBe(channelPaths);
|
||||
}
|
||||
}
|
||||
@@ -8,7 +8,6 @@ public record PlayoutResponseModel(
|
||||
PlayoutScheduleKind ScheduleKind,
|
||||
string ChannelName,
|
||||
string ChannelNumber,
|
||||
int ChannelId,
|
||||
ChannelPlayoutMode PlayoutMode,
|
||||
string ScheduleName,
|
||||
string? ScheduleFile,
|
||||
@@ -24,7 +23,6 @@ public record PlayoutResponseModel(
|
||||
PlayoutScheduleKind scheduleKind,
|
||||
string channelName,
|
||||
string channelNumber,
|
||||
int channelId,
|
||||
ChannelPlayoutMode playoutMode,
|
||||
string scheduleName,
|
||||
string? scheduleFile,
|
||||
@@ -39,7 +37,6 @@ public record PlayoutResponseModel(
|
||||
scheduleKind,
|
||||
channelName,
|
||||
channelNumber,
|
||||
channelId,
|
||||
playoutMode,
|
||||
scheduleName,
|
||||
scheduleFile,
|
||||
|
||||
@@ -1,19 +1,9 @@
|
||||
namespace ErsatzTV.Core.Domain;
|
||||
namespace ErsatzTV.Core.Domain;
|
||||
|
||||
public class LibraryFolder
|
||||
{
|
||||
public int Id { get; set; }
|
||||
public string Path { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// SHA-256 hex of <see cref="Path" /> (<see cref="ErsatzTV.Core.PathUtils.GetPathHash" />), the
|
||||
/// indexable stand-in for the unbounded <see cref="Path" /> column that backs the unique
|
||||
/// <c>(LibraryPathId, PathHash)</c> constraint — the same shape as <c>MediaFile.PathHash</c>.
|
||||
/// Nullable: rows created before ersatztv#491 carry <c>null</c> until a scan heals them, and a
|
||||
/// unique index treats nulls as distinct so those legacy rows never collide.
|
||||
/// </summary>
|
||||
public string PathHash { get; set; }
|
||||
|
||||
public int LibraryPathId { get; set; }
|
||||
public LibraryPath LibraryPath { get; set; }
|
||||
public int? ParentId { get; set; }
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
namespace ErsatzTV.Core.Domain;
|
||||
namespace ErsatzTV.Core.Domain;
|
||||
|
||||
public class SongMetadata : Metadata
|
||||
{
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using System.Globalization;
|
||||
using System.Globalization;
|
||||
using System.Text;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Core.Interfaces.FFmpeg;
|
||||
@@ -85,9 +85,6 @@ public class SongVideoGenerator : ISongVideoGenerator
|
||||
|
||||
var sb = new StringBuilder();
|
||||
|
||||
List<string> artists = Optional(metadata.Artists).Flatten().ToList();
|
||||
List<string> albumArtists = Optional(metadata.AlbumArtists).Flatten().ToList();
|
||||
|
||||
if (detailsStyle)
|
||||
{
|
||||
if (!string.IsNullOrWhiteSpace(metadata.Title))
|
||||
@@ -95,17 +92,17 @@ public class SongVideoGenerator : ISongVideoGenerator
|
||||
sb.Append(CultureInfo.InvariantCulture, $"{{\\fs{largeFontSize}}}{metadata.Title}");
|
||||
}
|
||||
|
||||
if (artists.Count > 0)
|
||||
if (metadata.Artists.Count > 0)
|
||||
{
|
||||
var allArtists = string.Join(", ", artists);
|
||||
var allArtists = string.Join(", ", metadata.Artists);
|
||||
sb.Append(CultureInfo.InvariantCulture, $"\\N{{\\fs{fontSize}}}{allArtists}");
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
if (artists.Count > 0)
|
||||
if (metadata.Artists.Count > 0)
|
||||
{
|
||||
var allArtists = string.Join(", ", artists);
|
||||
var allArtists = string.Join(", ", metadata.Artists);
|
||||
sb.Append(allArtists);
|
||||
}
|
||||
|
||||
@@ -114,11 +111,11 @@ public class SongVideoGenerator : ISongVideoGenerator
|
||||
sb.Append(CultureInfo.InvariantCulture, $"\\N\"{metadata.Title}\"");
|
||||
}
|
||||
|
||||
if (albumArtists.Count > 0)
|
||||
if (metadata.AlbumArtists.Count > 0)
|
||||
{
|
||||
var allAlbumArtists = string.Join(
|
||||
", ",
|
||||
albumArtists.Filter(aa => !artists.Contains(aa)));
|
||||
metadata.AlbumArtists.Filter(aa => !metadata.Artists.Contains(aa)));
|
||||
sb.Append(CultureInfo.InvariantCulture, $"\\N{allAlbumArtists}");
|
||||
}
|
||||
|
||||
|
||||
@@ -171,136 +171,125 @@ public class WatermarkSelector(
|
||||
// check for playout item watermark
|
||||
foreach (ChannelWatermark watermark in playoutItemWatermark)
|
||||
{
|
||||
// A custom watermark with no image at all is a bad-form-validation artifact, and it has always
|
||||
// fallen THROUGH to the channel/global watermark rather than resolving to "no watermark". That
|
||||
// stays true: unifying *resolution* (#510) must not change which watermark WINS.
|
||||
if (watermark.ImageSource is ChannelWatermarkImageSource.Custom
|
||||
&& string.IsNullOrWhiteSpace(watermark.Image))
|
||||
switch (watermark.ImageSource)
|
||||
{
|
||||
logger.LogWarning(
|
||||
"Watermark {Name} has custom image configured with no image; ignoring",
|
||||
watermark.Name);
|
||||
break;
|
||||
}
|
||||
// used for song progress overlay
|
||||
case ChannelWatermarkImageSource.Resource:
|
||||
string resourcePath = fileSystem.Path.Combine(
|
||||
FileSystemLayout.ResourcesCacheFolder,
|
||||
watermark.Image);
|
||||
if (fileSystem.File.Exists(resourcePath))
|
||||
{
|
||||
return new WatermarkOptions(watermark, resourcePath, Option<int>.None);
|
||||
}
|
||||
|
||||
logger.LogDebug("Watermark will come from playout item ({ImageSource})", watermark.ImageSource);
|
||||
return ResolveWatermark(channel, watermark);
|
||||
logger.LogWarning(
|
||||
"Watermark resource no longer exists at {Path} and will be ignored",
|
||||
resourcePath);
|
||||
return None;
|
||||
case ChannelWatermarkImageSource.Custom:
|
||||
// bad form validation makes this possible
|
||||
if (string.IsNullOrWhiteSpace(watermark.Image))
|
||||
{
|
||||
logger.LogWarning(
|
||||
"Watermark {Name} has custom image configured with no image; ignoring",
|
||||
watermark.Name);
|
||||
break;
|
||||
}
|
||||
|
||||
logger.LogDebug("Watermark will come from playout item (custom)");
|
||||
|
||||
string customPath = imageCache.GetPathForImage(
|
||||
watermark.Image,
|
||||
ArtworkKind.Watermark,
|
||||
Option<int>.None);
|
||||
|
||||
if (fileSystem.File.Exists(customPath))
|
||||
{
|
||||
return new WatermarkOptions(watermark, customPath, None);
|
||||
}
|
||||
|
||||
logger.LogWarning(
|
||||
"Custom watermark no longer exists at {Path} and will be ignored",
|
||||
customPath);
|
||||
return None;
|
||||
case ChannelWatermarkImageSource.ChannelLogo:
|
||||
logger.LogDebug("Watermark will come from playout item (channel logo)");
|
||||
|
||||
return ChannelLogoWatermarkOptions(channel, watermark);
|
||||
default:
|
||||
throw new NotSupportedException("Unsupported watermark image source");
|
||||
}
|
||||
}
|
||||
|
||||
// check for channel watermark
|
||||
if (channel.Watermark != null)
|
||||
{
|
||||
logger.LogDebug("Watermark will come from channel ({ImageSource})", channel.Watermark.ImageSource);
|
||||
return ResolveWatermark(channel, channel.Watermark);
|
||||
switch (channel.Watermark.ImageSource)
|
||||
{
|
||||
case ChannelWatermarkImageSource.Custom:
|
||||
logger.LogDebug("Watermark will come from channel (custom)");
|
||||
|
||||
string customPath = imageCache.GetPathForImage(
|
||||
channel.Watermark.Image,
|
||||
ArtworkKind.Watermark,
|
||||
Option<int>.None);
|
||||
|
||||
if (fileSystem.File.Exists(customPath))
|
||||
{
|
||||
return new WatermarkOptions(channel.Watermark, customPath, None);
|
||||
}
|
||||
|
||||
logger.LogWarning(
|
||||
"Custom watermark no longer exists at {Path} and will be ignored",
|
||||
customPath);
|
||||
return None;
|
||||
case ChannelWatermarkImageSource.ChannelLogo:
|
||||
logger.LogDebug("Watermark will come from channel (channel logo)");
|
||||
|
||||
return ChannelLogoWatermarkOptions(channel, channel.Watermark);
|
||||
default:
|
||||
throw new NotSupportedException("Unsupported watermark image source");
|
||||
}
|
||||
}
|
||||
|
||||
// check for global watermark
|
||||
foreach (ChannelWatermark watermark in globalWatermark)
|
||||
{
|
||||
logger.LogDebug("Watermark will come from global ({ImageSource})", watermark.ImageSource);
|
||||
return ResolveWatermark(channel, watermark);
|
||||
switch (watermark.ImageSource)
|
||||
{
|
||||
case ChannelWatermarkImageSource.Custom:
|
||||
logger.LogDebug("Watermark will come from global (custom)");
|
||||
|
||||
string customPath = imageCache.GetPathForImage(
|
||||
watermark.Image,
|
||||
ArtworkKind.Watermark,
|
||||
Option<int>.None);
|
||||
|
||||
if (fileSystem.File.Exists(customPath))
|
||||
{
|
||||
return new WatermarkOptions(watermark, customPath, None);
|
||||
}
|
||||
|
||||
logger.LogWarning(
|
||||
"Custom watermark no longer exists at {Path} and will be ignored",
|
||||
customPath);
|
||||
return None;
|
||||
case ChannelWatermarkImageSource.ChannelLogo:
|
||||
logger.LogDebug("Watermark will come from global (channel logo)");
|
||||
|
||||
return ChannelLogoWatermarkOptions(channel, watermark);
|
||||
default:
|
||||
throw new NotSupportedException("Unsupported watermark image source");
|
||||
}
|
||||
}
|
||||
|
||||
return Option<WatermarkOptions>.None;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The single place a <see cref="ChannelWatermark" /> becomes a renderable image path, shared by every
|
||||
/// watermark source: the three precedence levels (playout item, channel, global) AND the deco path.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Before #510 the deco path had its own copy of this switch that resolved paths *unchecked* — it handed
|
||||
/// down a nonexistent file, an un-migrated external URL, and the generated-initials localhost URL. The
|
||||
/// playout-item level checked all three sources; the channel and global levels checked
|
||||
/// <c>Custom</c>/<c>ChannelLogo</c> and *threw* for <c>Resource</c> (no arm, so `default:`). So the same
|
||||
/// channel could disagree with itself about whether a bug rendered, purely by how the watermark was
|
||||
/// attached. Duplication is what let that drift happen (it existed in triplicate before #502), so there is
|
||||
/// now one resolver. Exactly one piece of per-caller policy survives, and it lives in the CALLER rather
|
||||
/// than here: a playout-item <c>Custom</c> watermark with a blank image falls through to channel/global
|
||||
/// (see <see cref="GetWatermarkOptions" />). An unresolvable watermark resolves to "no on-screen bug",
|
||||
/// never a dead path passed downstream: a dead LOCAL path could reach ffmpeg as a bare <c>-i</c> argument
|
||||
/// via <c>CanUseFFmpegNativeWatermark</c>, which is materially worse than a skipped overlay.
|
||||
/// <para>
|
||||
/// Watermarks built OUTSIDE this selector are not covered — the song-progress overlay is constructed as a
|
||||
/// <c>WatermarkOptions</c> directly by the streaming and troubleshooting handlers and is still unchecked
|
||||
/// (#653).
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
private Option<WatermarkOptions> ResolveWatermark(Channel channel, ChannelWatermark watermark)
|
||||
{
|
||||
switch (watermark.ImageSource)
|
||||
{
|
||||
// NOT dead code and NOT only hand-edited rows: CreateWatermarkHandler/UpdateWatermarkHandler
|
||||
// persist whatever ImageSource the request names, so a Resource watermark is creatable through
|
||||
// the API -- always with Image = null, which is why the guard below is essential.
|
||||
// Separately, the real song-progress overlay does NOT come through here: it is built directly as a
|
||||
// WatermarkOptions by the streaming/troubleshooting handlers, which bypass this resolver and are
|
||||
// still unchecked (#653).
|
||||
case ChannelWatermarkImageSource.Resource:
|
||||
// Image is NULL for every non-Custom watermark the API writes (CreateWatermarkHandler /
|
||||
// UpdateWatermarkHandler both set `Image = null` unless ImageSource is Custom), so this guard is
|
||||
// load-bearing, not defensive: Path.Combine(folder, null) throws ArgumentNullException, which
|
||||
// would surface as a failed stream start rather than a missing overlay.
|
||||
if (string.IsNullOrWhiteSpace(watermark.Image))
|
||||
{
|
||||
logger.LogWarning(
|
||||
"Watermark {Name} uses a resource image but has no image name; ignoring",
|
||||
watermark.Name);
|
||||
return None;
|
||||
}
|
||||
|
||||
string resourcePath = fileSystem.Path.Combine(
|
||||
FileSystemLayout.ResourcesCacheFolder,
|
||||
watermark.Image);
|
||||
if (fileSystem.File.Exists(resourcePath))
|
||||
{
|
||||
return new WatermarkOptions(watermark, resourcePath, Option<int>.None);
|
||||
}
|
||||
|
||||
logger.LogWarning(
|
||||
"Watermark resource no longer exists at {Path} and will be ignored",
|
||||
resourcePath);
|
||||
return None;
|
||||
|
||||
case ChannelWatermarkImageSource.Custom:
|
||||
// bad form validation makes this possible
|
||||
if (string.IsNullOrWhiteSpace(watermark.Image))
|
||||
{
|
||||
logger.LogWarning(
|
||||
"Watermark {Name} has custom image configured with no image; ignoring",
|
||||
watermark.Name);
|
||||
return None;
|
||||
}
|
||||
|
||||
string customPath = imageCache.GetPathForImage(
|
||||
watermark.Image,
|
||||
ArtworkKind.Watermark,
|
||||
Option<int>.None);
|
||||
|
||||
if (fileSystem.File.Exists(customPath))
|
||||
{
|
||||
return new WatermarkOptions(watermark, customPath, None);
|
||||
}
|
||||
|
||||
logger.LogWarning(
|
||||
"Custom watermark no longer exists at {Path} and will be ignored",
|
||||
customPath);
|
||||
return None;
|
||||
|
||||
case ChannelWatermarkImageSource.ChannelLogo:
|
||||
return ChannelLogoWatermarkOptions(channel, watermark);
|
||||
|
||||
// deliberately loud: a newly-added image source must fail visibly rather than silently
|
||||
// resolve to some neighbouring source's behavior
|
||||
default:
|
||||
throw new NotSupportedException("Unsupported watermark image source");
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Resolves a <see cref="ChannelWatermarkImageSource.ChannelLogo" /> watermark to a renderable path.
|
||||
/// Since #510 this is reached from <see cref="ResolveWatermark" />, so all FOUR sources — the playout-item,
|
||||
/// channel and global precedence levels AND the deco path — agree.
|
||||
/// Resolves a <see cref="ChannelWatermarkImageSource.ChannelLogo" /> watermark to a renderable path,
|
||||
/// shared by the playout-item, channel and global precedence levels so all three agree.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// As of #525 an external-URL logo is downloaded and cached at save time, so a URL path here can only
|
||||
@@ -335,52 +324,79 @@ public class WatermarkSelector(
|
||||
return None;
|
||||
}
|
||||
|
||||
// With no logo artwork at all the only candidate is the generated-initials image, served over HTTP from
|
||||
// ChannelLogoGenerator.GenerateChannelLogoUrl -- a URL that hardcodes localhost (issue #1, closed as a
|
||||
// topology problem without removing the hardcode).
|
||||
//
|
||||
// Until #510 that URL WAS returned by the deco path, and it genuinely rendered: a live-E2E on a real
|
||||
// transcoded frame confirmed the nameplate compositing through the graphics engine (the /iptv/logos/gen
|
||||
// route sits on ArtworkController, which carries no auth filter, so the container-internal self-fetch
|
||||
// succeeded). It never rendered at the three precedence levels. #510 resolved that split in favour of
|
||||
// "no bug", because a render-time HTTP fetch inside stream startup is exactly what `graphics.channel-logo-caching`
|
||||
// (#525) eliminated for logos -- so the fallback is now off everywhere rather than on for one caller.
|
||||
// Reviving it properly means generating the image into the image cache so it resolves to a LOCAL path;
|
||||
// that is deliberately out of scope here and tracked separately.
|
||||
// with no logo artwork the only candidate is the generated-initials image, whose URL hardcodes
|
||||
// localhost (ChannelLogoGenerator.GenerateChannelLogoUrl, issue #1). It has never rendered here and
|
||||
// reviving it is deliberately deferred in docs/decisions.md, so it stays ignored.
|
||||
logger.LogWarning(
|
||||
"Channel {Channel} has no logo artwork; rendering without an on-screen bug. The generated-initials "
|
||||
+ "fallback ({Url}) is deliberately not used by the render path",
|
||||
channel.Number,
|
||||
"Channel logo no longer exists at {Path} and will be ignored",
|
||||
ChannelLogoGenerator.GenerateChannelLogoUrl(channel));
|
||||
return None;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Resolves the watermarks attached to a deco. Since #510 this shares <see cref="ResolveWatermark" />
|
||||
/// with the three precedence levels rather than carrying its own unchecked copy of the same switch.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Resolution is now identical to the precedence levels; what stays deco-specific is only WHICH
|
||||
/// watermarks apply and whether they merge with or override the rest (handled in
|
||||
/// <see cref="SelectWatermarks" />).
|
||||
/// <para>
|
||||
/// The routing PREDICATE is unchanged — <c>CanUseFFmpegNativeWatermark</c> still keys off the resolved
|
||||
/// path alone and sends any URL to the graphics engine regardless of provenance. Its INPUT can change,
|
||||
/// though: dropping an unresolvable watermark shortens this list, so a deco carrying one valid and one
|
||||
/// missing permanent watermark now yields count 1 (ffmpeg-native) where it previously yielded count 2
|
||||
/// (graphics engine). That is intended — the surviving watermark is a single valid permanent local image,
|
||||
/// exactly what the native path is for — but it IS an observable routing change, not a no-op.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
private List<WatermarkOptions> OptionsForWatermarks(Channel channel, IEnumerable<ChannelWatermark> watermarks)
|
||||
{
|
||||
var result = new List<WatermarkOptions>();
|
||||
|
||||
foreach (var watermark in watermarks)
|
||||
{
|
||||
result.AddRange(ResolveWatermark(channel, watermark));
|
||||
result.AddRange(GetWatermarkOptions(channel, watermark));
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private Option<WatermarkOptions> GetWatermarkOptions(Channel channel, ChannelWatermark watermark)
|
||||
{
|
||||
switch (watermark.ImageSource)
|
||||
{
|
||||
// used for song progress overlay
|
||||
case ChannelWatermarkImageSource.Resource:
|
||||
return new WatermarkOptions(
|
||||
watermark,
|
||||
Path.Combine(FileSystemLayout.ResourcesCacheFolder, watermark.Image),
|
||||
Option<int>.None);
|
||||
case ChannelWatermarkImageSource.Custom:
|
||||
// bad form validation makes this possible
|
||||
if (string.IsNullOrWhiteSpace(watermark.Image))
|
||||
{
|
||||
logger.LogWarning(
|
||||
"Watermark {Name} has custom image configured with no image; ignoring",
|
||||
watermark.Name);
|
||||
break;
|
||||
}
|
||||
|
||||
string customPath = imageCache.GetPathForImage(
|
||||
watermark.Image,
|
||||
ArtworkKind.Watermark,
|
||||
Option<int>.None);
|
||||
return new WatermarkOptions(
|
||||
watermark,
|
||||
customPath,
|
||||
None);
|
||||
case ChannelWatermarkImageSource.ChannelLogo:
|
||||
// deliberately NOT ChannelLogoWatermarkOptions: the deco path has always passed its resolved
|
||||
// path through unchecked, so #502's File.Exists defect never reached it and its *resolution*
|
||||
// is unchanged here. Aligning its missing-file / no-artwork policy with the three precedence
|
||||
// levels above is a behavior change beyond this fix — tracked in #510.
|
||||
// Note this only scopes resolution: the ffmpeg-native-vs-graphics-engine routing in
|
||||
// FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark keys off the resolved path alone, so a
|
||||
// deco watermark resolving to a URL (an external logo, or the generated-initials URL below) is
|
||||
// rerouted to the graphics engine like any other. That is intended: it is the URL-aware path.
|
||||
string channelPath = ChannelLogoGenerator.GenerateChannelLogoUrl(channel);
|
||||
Option<Artwork> maybeLogoArtwork =
|
||||
Optional(channel.Artwork.Find(a => a.ArtworkKind == ArtworkKind.Logo));
|
||||
foreach (var logoArtwork in maybeLogoArtwork)
|
||||
{
|
||||
channelPath = Artwork.IsExternalUrl(logoArtwork.Path)
|
||||
? logoArtwork.Path
|
||||
: imageCache.GetPathForImage(logoArtwork.Path, ArtworkKind.Logo, Option<int>.None);
|
||||
}
|
||||
|
||||
return new WatermarkOptions(watermark, channelPath, None);
|
||||
default:
|
||||
throw new NotSupportedException("Unsupported watermark image source");
|
||||
}
|
||||
|
||||
return Option<WatermarkOptions>.None;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,7 +2,6 @@ using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Text.RegularExpressions;
|
||||
using ErsatzTV.FFmpeg.Capabilities;
|
||||
using ErsatzTV.FFmpeg.Filter;
|
||||
using ErsatzTV.FFmpeg.Format;
|
||||
using ErsatzTV.FFmpeg.OutputFormat;
|
||||
using ErsatzTV.FFmpeg.Pipeline;
|
||||
@@ -21,7 +20,6 @@ namespace ErsatzTV.FFmpeg.Tests.Pipeline;
|
||||
public class QsvPipelineBuilderTests
|
||||
{
|
||||
private readonly ILogger _logger = Substitute.For<ILogger>();
|
||||
private Option<SubtitleInputFile> _lastSubtitleInputFile;
|
||||
|
||||
[Test]
|
||||
public void Qsv_PreferNativeDecoder_Should_Decode_Via_Vaapi_To_Software_Then_Qsv_Encode()
|
||||
@@ -124,142 +122,6 @@ public class QsvPipelineBuilderTests
|
||||
command.ShouldContain("hwupload=extra_hw_frames=128");
|
||||
}
|
||||
|
||||
// ersatztv#505. Measured on the deployed FFmpeg 8.1.2 / iHD 25.1.4 / UHD 630: a graph ending in
|
||||
// "vpp_qsv=tonemap=1" returns a frame that is BYTE-IDENTICAL (same md5) to the same graph with
|
||||
// no tonemap step at all — QSV VPP tonemapping needs Gen11+, and pre-Gen11 iHD ignores it with
|
||||
// no warning. So the assertion that matters is not "GPU tonemap is used" but "the silent no-op
|
||||
// is never emitted", which is why every case below asserts its absence.
|
||||
[TestCase(true, true)]
|
||||
[TestCase(true, false)]
|
||||
[TestCase(false, true)]
|
||||
[TestCase(false, false)]
|
||||
public void Qsv_Hdr_Should_Never_Emit_The_Silently_No_Op_Vpp_Qsv_Tonemap(
|
||||
bool preferNativeDecoder,
|
||||
bool deinterlace)
|
||||
{
|
||||
string command = BuildHdrAndPrint(preferNativeDecoder, deinterlace);
|
||||
|
||||
// assert against the vpp_qsv OPTION, not the bare substring: "tonemap=1" alone could match
|
||||
// an unrelated filter, and would miss an equivalent spelling
|
||||
command.ShouldNotContain("vpp_qsv=tonemap");
|
||||
Regex.IsMatch(command, @"vpp_qsv=[^,\s]*tonemap")
|
||||
.ShouldBeFalse(command);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Qsv_Hdr_NativeDecode_Should_Tonemap_On_The_Gpu_Via_OpenCL()
|
||||
{
|
||||
string command = BuildHdrAndPrint(preferNativeDecoder: true);
|
||||
|
||||
// upload to VA-API explicitly: "-filter_hw_device hw" points at the QSV device, so a bare
|
||||
// hwupload here would land on a QSV surface, which cannot be mapped to OpenCL
|
||||
command.ShouldContain("hwupload=derive_device=vaapi");
|
||||
|
||||
// scale BEFORE tonemap, on the VA-API device (tonemapping full-size costs ~50% more wall
|
||||
// clock than the software tonemap this replaces)
|
||||
int scaleAt = command.IndexOf("scale_vaapi", StringComparison.Ordinal);
|
||||
int tonemapAt = command.IndexOf("tonemap_opencl", StringComparison.Ordinal);
|
||||
scaleAt.ShouldBeGreaterThan(-1, command);
|
||||
tonemapAt.ShouldBeGreaterThan(-1, command);
|
||||
scaleAt.ShouldBeLessThan(tonemapAt, command);
|
||||
|
||||
// no vpp_qsv scale on this path — a QSV surface could not reach OpenCL afterwards
|
||||
command.ShouldNotContain("vpp_qsv");
|
||||
|
||||
// and no CPU tonemap, which is the cost ersatztv#505 was filed about
|
||||
command.ShouldNotContain("zscale");
|
||||
|
||||
// the hardware filters strip color info, so the output has to be re-tagged bt709 — without
|
||||
// this the picture is tonemapped but still ANNOUNCES bt2020 primaries, and the player
|
||||
// converts it a second time (verified against ffprobe on the Intel host)
|
||||
command.ShouldContain("all=bt709");
|
||||
|
||||
command.ShouldContain("h264_qsv");
|
||||
|
||||
// pin the exact graph measured on the Intel host, in order — the assertions above would
|
||||
// all still pass with setFormat off, hwdownload dropped, or the wrong tonemap output
|
||||
// format, any of which breaks the validated command
|
||||
command.ShouldContain(
|
||||
"format=nv12|p010le|vaapi,hwupload=derive_device=vaapi," +
|
||||
"scale_vaapi=1280:720:force_divisible_by=2:format=p010,setsar=1," +
|
||||
"hwmap=derive_device=opencl,tonemap_opencl=tonemap=linear:format=nv12," +
|
||||
"hwdownload,format=nv12");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Qsv_Hdr_Should_Retag_Bt709_Even_When_Color_Normalization_Is_Disabled()
|
||||
{
|
||||
// a tonemap converts the PIXELS to SDR, so the stream must stop announcing bt2020 whether
|
||||
// or not the profile asks for color normalization — otherwise the player converts twice
|
||||
string command = BuildAndPrint(preferNativeDecoder: true, hdr: true, normalizeColors: false);
|
||||
|
||||
command.ShouldContain("tonemap_opencl");
|
||||
command.ShouldContain("all=bt709");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Qsv_Hdr_Anamorphic_Should_Fall_Back_To_Software_Tonemap()
|
||||
{
|
||||
// ScaleVaapiFilter multiplies by ffmpeg's runtime `sar` instead of the SAR VideoStream
|
||||
// calculates, so anamorphic sources keep the software tonemap they already had
|
||||
string command = BuildAndPrint(preferNativeDecoder: true, hdr: true, anamorphic: true);
|
||||
|
||||
command.ShouldContain("zscale");
|
||||
command.ShouldNotContain("tonemap_opencl");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Qsv_Hdr_With_Image_Subtitle_Should_Scale_The_Subtitle_To_Match_The_Video()
|
||||
{
|
||||
// the video is scaled by ScaleVaapiFilter on this path; if the subtitle-scaling predicate
|
||||
// does not recognize it, the burned-in subtitle canvas stays at source resolution
|
||||
string command = BuildAndPrint(preferNativeDecoder: true, hdr: true, imageSubtitle: true);
|
||||
|
||||
command.ShouldContain("scale_vaapi");
|
||||
|
||||
var subtitleSteps = new List<IPipelineFilterStep>();
|
||||
foreach (SubtitleInputFile subtitle in _lastSubtitleInputFile)
|
||||
{
|
||||
subtitleSteps.AddRange(subtitle.FilterSteps);
|
||||
}
|
||||
|
||||
subtitleSteps.ShouldContain(s => s is ScaleImageFilter, "subtitle canvas was never resized");
|
||||
}
|
||||
|
||||
[TestCase(true, true, TestName = "Qsv_Hdr_Interlaced_Falls_Back_To_Software_Tonemap")]
|
||||
[TestCase(false, false, TestName = "Qsv_Hdr_QsvDecode_Falls_Back_To_Software_Tonemap")]
|
||||
public void Qsv_Hdr_Should_Fall_Back_To_Software_Tonemap_When_Frames_Cannot_Reach_OpenCL(
|
||||
bool preferNativeDecoder,
|
||||
bool deinterlace)
|
||||
{
|
||||
// both cases put frames on a QSV surface before the tonemap would run (deinterlace_qsv, or
|
||||
// the QSV decoder itself), and a QSV surface maps to neither OpenCL nor VA-API. Slower on
|
||||
// the CPU, but correct — unlike the vpp_qsv no-op this replaces.
|
||||
string command = BuildHdrAndPrint(preferNativeDecoder, deinterlace);
|
||||
|
||||
command.ShouldContain("zscale");
|
||||
command.ShouldNotContain("tonemap_opencl");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Qsv_Hdr_Should_Fall_Back_To_Software_Tonemap_Without_The_OpenCL_Filter()
|
||||
{
|
||||
// an ffmpeg build with no tonemap_opencl must not silently skip tonemapping
|
||||
string command = BuildAndPrint(preferNativeDecoder: true, hdr: true, hasOpenClTonemap: false);
|
||||
|
||||
command.ShouldContain("zscale");
|
||||
command.ShouldNotContain("tonemap_opencl");
|
||||
command.ShouldNotContain("tonemap=1");
|
||||
}
|
||||
|
||||
private string BuildHdrAndPrint(bool preferNativeDecoder, bool deinterlace = false) =>
|
||||
BuildAndPrint(
|
||||
preferNativeDecoder,
|
||||
maybeExtraHardwareFrames: default,
|
||||
deinterlace ? ScanKind.Interlaced : ScanKind.Progressive,
|
||||
deinterlace,
|
||||
hdr: true);
|
||||
|
||||
private string BuildInterlacedAndPrint(Option<int> maybeExtraHardwareFrames = default) =>
|
||||
BuildAndPrint(
|
||||
preferNativeDecoder: true,
|
||||
@@ -271,40 +133,21 @@ public class QsvPipelineBuilderTests
|
||||
bool preferNativeDecoder,
|
||||
Option<int> maybeExtraHardwareFrames = default,
|
||||
ScanKind scanKind = ScanKind.Progressive,
|
||||
bool deinterlace = false,
|
||||
bool hdr = false,
|
||||
bool hasOpenClTonemap = true,
|
||||
bool normalizeColors = true,
|
||||
bool anamorphic = false,
|
||||
bool imageSubtitle = false)
|
||||
bool deinterlace = false)
|
||||
{
|
||||
(VideoInputFile videoInputFile, AudioInputFile audioInputFile, FFmpegState ffmpegState, FrameState desiredState) =
|
||||
BuildQsvH264Pipeline(preferNativeDecoder, scanKind, deinterlace, hdr, anamorphic);
|
||||
BuildQsvH264Pipeline(preferNativeDecoder, scanKind, deinterlace);
|
||||
|
||||
ffmpegState = ffmpegState with { MaybeQsvExtraHardwareFrames = maybeExtraHardwareFrames };
|
||||
|
||||
if (!normalizeColors)
|
||||
{
|
||||
desiredState = desiredState with { ColorsAreBt709 = false };
|
||||
}
|
||||
|
||||
Option<SubtitleInputFile> subtitleInputFile = imageSubtitle
|
||||
? new SubtitleInputFile(
|
||||
"/tmp/whatever.mkv",
|
||||
new List<MediaStream> { new(2, "hdmv_pgs_subtitle", StreamKind.Subtitle) },
|
||||
SubtitleMethod.Burn)
|
||||
: Option<SubtitleInputFile>.None;
|
||||
|
||||
var builder = new QsvPipelineBuilder(
|
||||
hasOpenClTonemap
|
||||
? new DefaultFFmpegCapabilities(FFmpegKnownFilter.TonemapOpenCL.Name)
|
||||
: new DefaultFFmpegCapabilities(),
|
||||
new DefaultFFmpegCapabilities(),
|
||||
new DefaultHardwareCapabilities(),
|
||||
HardwareAccelerationMode.Qsv,
|
||||
videoInputFile,
|
||||
audioInputFile,
|
||||
None,
|
||||
subtitleInputFile,
|
||||
None,
|
||||
None,
|
||||
Option<GraphicsEngineInput>.None,
|
||||
"",
|
||||
@@ -313,34 +156,26 @@ public class QsvPipelineBuilderTests
|
||||
|
||||
FFmpegPipeline result = builder.Build(ffmpegState, desiredState);
|
||||
|
||||
// the subtitle input's filter steps never reach CommandGenerator, so expose them for the
|
||||
// subtitle-scaling assertion
|
||||
_lastSubtitleInputFile = subtitleInputFile;
|
||||
|
||||
return PrintCommand(videoInputFile, audioInputFile, None, None, None, result);
|
||||
}
|
||||
|
||||
private static (VideoInputFile, AudioInputFile, FFmpegState, FrameState) BuildQsvH264Pipeline(
|
||||
bool preferNativeDecoder,
|
||||
ScanKind scanKind,
|
||||
bool deinterlace,
|
||||
bool hdr = false,
|
||||
bool anamorphic = false)
|
||||
bool deinterlace)
|
||||
{
|
||||
// the real trigger: HEVC Main10, BT.2020 primaries, smpte2084 (PQ) transfer — matching the
|
||||
// prod sources this was validated against on jazz
|
||||
var videoInputFile = new VideoInputFile(
|
||||
"/tmp/whatever.mkv",
|
||||
new List<VideoStream>
|
||||
{
|
||||
new(
|
||||
0,
|
||||
hdr ? VideoFormat.Hevc : VideoFormat.H264,
|
||||
VideoFormat.H264,
|
||||
VideoProfile.Main,
|
||||
hdr ? new PixelFormatYuv420P10Le() : new PixelFormatYuv420P(),
|
||||
hdr ? new ColorParams("tv", "bt2020nc", "smpte2084", "bt2020") : ColorParams.Default,
|
||||
hdr ? new FrameSize(3840, 1608) : new FrameSize(1920, 1080),
|
||||
anamorphic ? "4:3" : "1:1",
|
||||
new PixelFormatYuv420P(),
|
||||
ColorParams.Default,
|
||||
new FrameSize(1920, 1080),
|
||||
"1:1",
|
||||
"16:9",
|
||||
FrameRate.DefaultFrameRate,
|
||||
false,
|
||||
@@ -377,9 +212,7 @@ public class QsvPipelineBuilderTests
|
||||
2000,
|
||||
4000,
|
||||
90_000,
|
||||
// HDR output is normalized to bt709, which is what makes the colorspace filter
|
||||
// reachable at all; leaving this false would hide the output-tagging assertions
|
||||
hdr,
|
||||
false,
|
||||
deinterlace);
|
||||
|
||||
var ffmpegState = new FFmpegState(
|
||||
@@ -437,11 +270,11 @@ public class QsvPipelineBuilderTests
|
||||
return command;
|
||||
}
|
||||
|
||||
public class DefaultFFmpegCapabilities(params string[] filters) : FFmpegCapabilities(
|
||||
public class DefaultFFmpegCapabilities() : FFmpegCapabilities(
|
||||
string.Empty,
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>(filters),
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>());
|
||||
|
||||
@@ -593,97 +593,7 @@ public class PipelineBuilderBaseTests
|
||||
command.ShouldNotContain("-readrate_initial_burst");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Realtime_Input_Should_Catch_Up_When_Option_Is_Supported()
|
||||
{
|
||||
string command = BuildRealtimeCommand(new CatchupCapableFFmpegCapabilities());
|
||||
|
||||
// -readrate paces an input off its furthest-behind stream, so a sparse stream sharing the
|
||||
// input pins throughput below realtime; catchup lets it recover (ersatztv#726). anchor on
|
||||
// the input path so this can't be satisfied by some other input carrying the option
|
||||
// this overlaps Bitmap_Subtitle_Burn_In_... by design: that one pins the #726 MECHANISM on a
|
||||
// bitmap pipeline, this one pins the plain no-subtitle shape plus the uniqueness guard below
|
||||
command.ShouldContain("-readrate 1.05 -readrate_initial_burst 8 -readrate_catchup 6.0 -i /tmp/whatever.mkv");
|
||||
Regex.Matches(command, Regex.Escape("-readrate_catchup 6.0")).Count.ShouldBe(1);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Realtime_Input_Should_Not_Catch_Up_A_Still_Image()
|
||||
{
|
||||
// mirrors the burst's still-image exclusion (ersatztv#350): the video input takes no readrate
|
||||
// at all, so catchup would only reach the separate audio input and run it ahead of a graph
|
||||
// that the realtime filter is already pacing. pinned so the divergence can't reappear silently
|
||||
string command = BuildRealtimeCommand(new CatchupCapableFFmpegCapabilities(), stillImage: true);
|
||||
|
||||
// the positive anchor keeps this from passing vacuously if the helper ever stops
|
||||
// producing a realtime audio input at all
|
||||
command.ShouldContain("-readrate 1.05");
|
||||
command.ShouldNotContain("-readrate_catchup");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Realtime_Input_Should_Not_Catch_Up_When_Option_Is_Unsupported()
|
||||
{
|
||||
// an older binary silently keeps today's behavior rather than failing to start
|
||||
string command = BuildRealtimeCommand(new BurstCapableFFmpegCapabilities());
|
||||
|
||||
// the positive anchor keeps this from passing vacuously if the helper ever stops
|
||||
// producing a realtime input at all
|
||||
command.ShouldContain("-readrate 1.05");
|
||||
command.ShouldNotContain("-readrate_catchup");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Concat_Should_Never_Catch_Up()
|
||||
{
|
||||
// concat reads already-written segments from the running segmenter at a flat 1.0; it has no
|
||||
// sparse stream to lag on, and letting it catch up would gallop through the segments
|
||||
var concatInputFile = new ConcatInputFile("http://localhost:8080/ffmpeg/concat/1", new FrameSize(1920, 1080));
|
||||
|
||||
var builder = new SoftwarePipelineBuilder(
|
||||
new CatchupCapableFFmpegCapabilities(),
|
||||
HardwareAccelerationMode.None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
concatInputFile,
|
||||
Option<GraphicsEngineInput>.None,
|
||||
"",
|
||||
"",
|
||||
_logger);
|
||||
|
||||
FFmpegPipeline result = builder.Concat(concatInputFile, FFmpegState.Concat(false, "Some Channel"));
|
||||
|
||||
string command = PrintCommand(None, None, None, concatInputFile, None, result);
|
||||
|
||||
command.ShouldContain("-readrate 1.0");
|
||||
command.ShouldNotContain("-readrate_catchup");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Bitmap_Subtitle_Burn_In_Should_Catch_Up_On_The_Shared_Video_Input()
|
||||
{
|
||||
// THE #726 regression test. an embedded bitmap subtitle is read through the SAME -i as the
|
||||
// video (SubtitleInputFile carries the video's path and resolves to a stream specifier on
|
||||
// that input), and being sparse it drags that input's pacing down to ~0.53x realtime.
|
||||
// this must be built on a BITMAP subtitle: a text subtitle is fetched by the libass filter
|
||||
// outside the demuxer, so the same assertions would pass vacuously while the bug is present.
|
||||
string command = BuildRealtimeCommand(new CatchupCapableFFmpegCapabilities(), imageSubtitle: true);
|
||||
|
||||
// the mechanism itself: subtitle stream 2 resolves onto input 0 -- the VIDEO's input -- so it
|
||||
// is read through the throttled demuxer that catchup is being applied to. if the subtitle
|
||||
// ever moves to an input of its own this label changes and the test fails, which is the point
|
||||
command.ShouldContain("[0:0][0:2]overlay");
|
||||
|
||||
// ...so the catchup has to be on that input
|
||||
command.ShouldContain("-readrate 1.05 -readrate_initial_burst 8 -readrate_catchup 6.0 -i /tmp/whatever.mkv");
|
||||
}
|
||||
|
||||
private string BuildRealtimeCommand(
|
||||
IFFmpegCapabilities capabilities,
|
||||
bool stillImage = false,
|
||||
bool imageSubtitle = false)
|
||||
private string BuildRealtimeCommand(IFFmpegCapabilities capabilities, bool stillImage = false)
|
||||
{
|
||||
var videoInputFile = new VideoInputFile(
|
||||
"/tmp/whatever.mkv",
|
||||
@@ -766,22 +676,13 @@ public class PipelineBuilderBaseTests
|
||||
AudioFilter.None,
|
||||
Option<double>.None));
|
||||
|
||||
// an embedded bitmap subtitle carries the VIDEO's path, which is how it ends up sharing the
|
||||
// video's single throttled -i rather than getting one of its own (ersatztv#726)
|
||||
Option<SubtitleInputFile> subtitleInputFile = imageSubtitle
|
||||
? new SubtitleInputFile(
|
||||
"/tmp/whatever.mkv",
|
||||
new List<MediaStream> { new(2, "dvdsub", StreamKind.Subtitle) },
|
||||
SubtitleMethod.Burn)
|
||||
: Option<SubtitleInputFile>.None;
|
||||
|
||||
var builder = new SoftwarePipelineBuilder(
|
||||
capabilities,
|
||||
HardwareAccelerationMode.None,
|
||||
videoInputFile,
|
||||
audioInputFile,
|
||||
None,
|
||||
subtitleInputFile,
|
||||
None,
|
||||
None,
|
||||
Option<GraphicsEngineInput>.None,
|
||||
"",
|
||||
@@ -834,19 +735,4 @@ public class PipelineBuilderBaseTests
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string> { FFmpegKnownOption.ReadrateInitialBurst.Name },
|
||||
new System.Collections.Generic.HashSet<string>());
|
||||
|
||||
// a binary new enough for -readrate_catchup also has -readrate_initial_burst, so this models a
|
||||
// real ffmpeg rather than an impossible catchup-without-burst one
|
||||
public class CatchupCapableFFmpegCapabilities() : FFmpegCapabilities(
|
||||
string.Empty,
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>(),
|
||||
new System.Collections.Generic.HashSet<string>
|
||||
{
|
||||
FFmpegKnownOption.ReadrateInitialBurst.Name,
|
||||
FFmpegKnownOption.ReadrateCatchup.Name
|
||||
},
|
||||
new System.Collections.Generic.HashSet<string>());
|
||||
}
|
||||
|
||||
@@ -13,15 +13,8 @@ public record FFmpegKnownOption
|
||||
// ffmpeg 6.1+; lets a readrate-throttled input read flat out for an initial window
|
||||
public static FFmpegKnownOption ReadrateInitialBurst => new("readrate_initial_burst");
|
||||
|
||||
// ffmpeg 8.0+ (added 2025-02-15 in 6232f416b, first released in 8.0); lets a readrate-throttled
|
||||
// input read faster than its readrate *while it is behind*, so a sparse stream sharing that
|
||||
// input cannot pin throughput below realtime (ersatztv#726). verified present in 8.1.2, the
|
||||
// pinned base image — note this is NEWER than 7.1, so it is detected at runtime, never assumed
|
||||
public static FFmpegKnownOption ReadrateCatchup => new("readrate_catchup");
|
||||
|
||||
public static IList<string> AllOptions =>
|
||||
[
|
||||
ReadrateInitialBurst.Name,
|
||||
ReadrateCatchup.Name
|
||||
ReadrateInitialBurst.Name
|
||||
];
|
||||
}
|
||||
|
||||
@@ -1,30 +0,0 @@
|
||||
using ErsatzTV.FFmpeg.Format;
|
||||
|
||||
namespace ErsatzTV.FFmpeg.Filter.Qsv;
|
||||
|
||||
// vpp_qsv=tonemap=1 is a SILENT no-op on pre-Gen11 Intel graphics (ersatztv#505): the frame comes
|
||||
// back untouched, byte for byte, with no warning and no error, so HDR content ships untonemapped.
|
||||
// The QSV pipeline therefore tonemaps through OpenCL, the same route VaapiPipelineBuilder takes.
|
||||
//
|
||||
// This filter always runs on VA-API frames and hands SOFTWARE frames back: QSV surfaces cannot be
|
||||
// mapped to OpenCL ("Media sharing must be enabled on context creation") and cannot be mapped to
|
||||
// VA-API either (hwmap returns -38, function not implemented), so the only route from a QSV-encode
|
||||
// profile into tonemap_opencl is to stay on the VA-API device the QSV device was derived from.
|
||||
public class TonemapOpenClQsvFilter(FFmpegState ffmpegState, IPixelFormat desiredPixelFormat) : BaseFilter
|
||||
{
|
||||
public override string Filter =>
|
||||
$"hwmap=derive_device=opencl,tonemap_opencl=tonemap={ffmpegState.TonemapAlgorithm}:format={OutputFormat}," +
|
||||
$"hwdownload,format={OutputFormat}";
|
||||
|
||||
private string OutputFormat =>
|
||||
desiredPixelFormat.BitDepth == 10 ? FFmpegFormat.P010LE : FFmpegFormat.NV12;
|
||||
|
||||
public override FrameState NextState(FrameState currentState) =>
|
||||
currentState with
|
||||
{
|
||||
FrameDataLocation = FrameDataLocation.Software,
|
||||
PixelFormat = desiredPixelFormat.BitDepth == 10
|
||||
? new PixelFormatP010()
|
||||
: new PixelFormatNv12(desiredPixelFormat.Name)
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
namespace ErsatzTV.FFmpeg.Filter.Qsv;
|
||||
|
||||
public class TonemapQsvFilter : BaseFilter
|
||||
{
|
||||
public override string Filter => "vpp_qsv=tonemap=1";
|
||||
|
||||
public override FrameState NextState(FrameState currentState) =>
|
||||
currentState with
|
||||
{
|
||||
FrameDataLocation = FrameDataLocation.Hardware
|
||||
};
|
||||
}
|
||||
@@ -1,28 +1,16 @@
|
||||
namespace ErsatzTV.FFmpeg.Filter.Vaapi;
|
||||
namespace ErsatzTV.FFmpeg.Filter.Vaapi;
|
||||
|
||||
public class HardwareUploadVaapiFilter : BaseFilter
|
||||
{
|
||||
private readonly bool _deriveDevice;
|
||||
private readonly bool _setFormat;
|
||||
|
||||
// deriveDevice matters only where the graph's default filter device is NOT the VA-API one: the
|
||||
// QSV pipeline sets "-filter_hw_device hw" (the QSV device), so a bare hwupload there would
|
||||
// upload to QSV instead of VA-API. It defaults to false so the VA-API pipeline, whose default
|
||||
// filter device already is VA-API, keeps emitting exactly what it emitted before.
|
||||
public HardwareUploadVaapiFilter(bool setFormat, bool deriveDevice = false)
|
||||
{
|
||||
_setFormat = setFormat;
|
||||
_deriveDevice = deriveDevice;
|
||||
}
|
||||
public HardwareUploadVaapiFilter(bool setFormat) => _setFormat = setFormat;
|
||||
|
||||
public override string Filter
|
||||
public override string Filter => _setFormat switch
|
||||
{
|
||||
get
|
||||
{
|
||||
string hwupload = _deriveDevice ? "hwupload=derive_device=vaapi" : "hwupload";
|
||||
return _setFormat ? $"format=nv12|p010le|vaapi,{hwupload}" : hwupload;
|
||||
}
|
||||
}
|
||||
false => "hwupload",
|
||||
true => "format=nv12|p010le|vaapi,hwupload"
|
||||
};
|
||||
|
||||
public override FrameState NextState(FrameState currentState) =>
|
||||
currentState with { FrameDataLocation = FrameDataLocation.Hardware };
|
||||
|
||||
@@ -3,11 +3,10 @@ using ErsatzTV.FFmpeg.Environment;
|
||||
|
||||
namespace ErsatzTV.FFmpeg.InputOption;
|
||||
|
||||
public class ReadrateInputOption(double readRate, Option<int> initialBurstSeconds, Option<double> catchupReadRate)
|
||||
: IInputOption
|
||||
public class ReadrateInputOption(double readRate, Option<int> initialBurstSeconds) : IInputOption
|
||||
{
|
||||
public ReadrateInputOption(double readRate)
|
||||
: this(readRate, Option<int>.None, Option<double>.None)
|
||||
: this(readRate, Option<int>.None)
|
||||
{
|
||||
}
|
||||
|
||||
@@ -31,17 +30,6 @@ public class ReadrateInputOption(double readRate, Option<int> initialBurstSecond
|
||||
result.Add(burst.ToString(CultureInfo.InvariantCulture));
|
||||
}
|
||||
|
||||
// -readrate paces the WHOLE input off its furthest-behind stream, so one sparse stream
|
||||
// (an embedded PGS/DVD bitmap subtitle feeding the overlay) drags the video down with it
|
||||
// and output collapses to ~0.53x realtime. catchup lets a lagging input read faster until
|
||||
// it is level again; it is a ceiling that only applies WHILE behind, never a target, so
|
||||
// caught-up input still paces at readRate and cannot race ahead (ersatztv#726)
|
||||
foreach (double catchup in catchupReadRate)
|
||||
{
|
||||
result.Add("-readrate_catchup");
|
||||
result.Add(catchup.ToString("0.0####", CultureInfo.InvariantCulture));
|
||||
}
|
||||
|
||||
return result.ToArray();
|
||||
}
|
||||
|
||||
|
||||
@@ -22,14 +22,6 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
|
||||
// an operator who raises that setting above 2 gets less of the benefit (ersatztv#350)
|
||||
private const int InitialBurstSeconds = OutputFormatHls.SegmentSeconds * 2;
|
||||
|
||||
// how fast a LAGGING realtime input may read until it is level again. measured on the #726
|
||||
// repro (embedded dvd_subtitle -> overlay, QSV encode): 1.05 alone sustains 0.53x, catchup 2.0
|
||||
// reaches 0.711x, and 6.0 restores the full 1.067x that the same pipeline achieves with no
|
||||
// subtitle at all. 20.0 also measures 1.067x — i.e. the value is not a throughput dial above
|
||||
// the point where the input catches up, so 6.0 is chosen as the smallest measured-sufficient
|
||||
// ceiling rather than the largest that works (ersatztv#726)
|
||||
private const double CatchupReadRate = 6.0;
|
||||
|
||||
private readonly Option<AudioInputFile> _audioInputFile;
|
||||
private readonly Option<ConcatInputFile> _concatInputFile;
|
||||
private readonly IFFmpegCapabilities _ffmpegCapabilities;
|
||||
@@ -879,26 +871,8 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
|
||||
? InitialBurstSeconds
|
||||
: Option<int>.None;
|
||||
|
||||
// -readrate paces an input off its furthest-behind stream. an embedded bitmap subtitle is
|
||||
// read through the SAME -i as the video (its SubtitleInputFile carries the video's path and
|
||||
// resolves to a stream specifier on that input), and being sparse it falls further behind
|
||||
// every second, dragging video throughput to ~0.53x — well under the 1.0x a live client
|
||||
// consumes at. catchup lets the lagging input recover instead of pinning the whole process.
|
||||
// applied to every realtime input, not just subtitle pipelines: it is inert unless an input
|
||||
// is actually behind, and any sparse stream can cause this (ersatztv#726).
|
||||
//
|
||||
// a still image is excluded for the SAME reason the burst above excludes it: its video input
|
||||
// takes no readrate at all, so this would reach only the separate audio input and let it run
|
||||
// ahead of the video, which is exactly what #350 declined. for a non-still-image item both
|
||||
// inputs carry identical options, so the symmetry is preserved there. and an image-based
|
||||
// subtitle always rides the video path, so this shape cannot suffer the starvation anyway
|
||||
Option<double> catchupReadRate =
|
||||
!isStillImage && _ffmpegCapabilities.HasOption(FFmpegKnownOption.ReadrateCatchup)
|
||||
? CatchupReadRate
|
||||
: Option<double>.None;
|
||||
|
||||
_audioInputFile.Iter(a => a.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds, catchupReadRate)));
|
||||
videoInputFile.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds, catchupReadRate));
|
||||
_audioInputFile.Iter(a => a.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds)));
|
||||
videoInputFile.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds));
|
||||
}
|
||||
|
||||
protected static void SetStillImageLoop(
|
||||
|
||||
@@ -6,7 +6,6 @@ using ErsatzTV.FFmpeg.Encoder.Qsv;
|
||||
using ErsatzTV.FFmpeg.Environment;
|
||||
using ErsatzTV.FFmpeg.Filter;
|
||||
using ErsatzTV.FFmpeg.Filter.Qsv;
|
||||
using ErsatzTV.FFmpeg.Filter.Vaapi;
|
||||
using ErsatzTV.FFmpeg.Format;
|
||||
using ErsatzTV.FFmpeg.GlobalOption.HardwareAcceleration;
|
||||
using ErsatzTV.FFmpeg.InputOption;
|
||||
@@ -19,7 +18,6 @@ namespace ErsatzTV.FFmpeg.Pipeline;
|
||||
|
||||
public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
{
|
||||
private readonly IFFmpegCapabilities _ffmpegCapabilities;
|
||||
private readonly IHardwareCapabilities _hardwareCapabilities;
|
||||
private readonly ILogger _logger;
|
||||
|
||||
@@ -48,7 +46,6 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
fontsFolder,
|
||||
logger)
|
||||
{
|
||||
_ffmpegCapabilities = ffmpegCapabilities;
|
||||
_hardwareCapabilities = hardwareCapabilities;
|
||||
_logger = logger;
|
||||
}
|
||||
@@ -218,31 +215,12 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
};
|
||||
}
|
||||
|
||||
// HDR has to be tonemapped through OpenCL on the VA-API device (ersatztv#505); when that is
|
||||
// the plan the downscale has to happen in scale_vaapi rather than vpp_qsv, because a QSV
|
||||
// surface can be mapped neither to OpenCL nor back to VA-API. Decided once, up front, so
|
||||
// the scale and tonemap steps cannot disagree about which device the frames are on.
|
||||
bool useOpenClTonemap = UseOpenClTonemap(videoStream, context, ffmpegState, currentState);
|
||||
|
||||
// _logger.LogDebug("After decode: {PixelFormat}", currentState.PixelFormat);
|
||||
currentState = SetDeinterlace(videoInputFile, context, ffmpegState, currentState);
|
||||
// _logger.LogDebug("After deinterlace: {PixelFormat}", currentState.PixelFormat);
|
||||
currentState = SetScale(
|
||||
videoInputFile,
|
||||
videoStream,
|
||||
context,
|
||||
ffmpegState,
|
||||
desiredState,
|
||||
currentState,
|
||||
useOpenClTonemap);
|
||||
currentState = SetScale(videoInputFile, videoStream, context, ffmpegState, desiredState, currentState);
|
||||
// _logger.LogDebug("After scale: {PixelFormat}", currentState.PixelFormat);
|
||||
currentState = SetTonemap(
|
||||
videoInputFile,
|
||||
videoStream,
|
||||
ffmpegState,
|
||||
desiredState,
|
||||
currentState,
|
||||
useOpenClTonemap);
|
||||
currentState = SetTonemap(videoInputFile, videoStream, ffmpegState, desiredState, currentState);
|
||||
currentState = SetPad(videoInputFile, videoStream, desiredState, currentState);
|
||||
// _logger.LogDebug("After pad: {PixelFormat}", currentState.PixelFormat);
|
||||
currentState = SetCrop(videoInputFile, desiredState, currentState);
|
||||
@@ -357,15 +335,9 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
|
||||
IPixelFormat formatForDownload = pixelFormat;
|
||||
|
||||
// "did a hardware filter run", not "was it a QSV one": these all strip or rewrite the
|
||||
// frame's color info, so the colorspace filter below has to re-assert it explicitly.
|
||||
// The VA-API/OpenCL tonemap route (ersatztv#505) belongs here too — leaving it out
|
||||
// shipped a correctly-tonemapped picture still TAGGED bt2020 primaries, which invites
|
||||
// the player to convert it a second time.
|
||||
bool usesVppQsv =
|
||||
videoInputFile.FilterSteps.Any(f =>
|
||||
f is QsvFormatFilter or ScaleQsvFilter or DeinterlaceQsvFilter
|
||||
or ScaleVaapiFilter or TonemapOpenClQsvFilter);
|
||||
f is QsvFormatFilter or ScaleQsvFilter or DeinterlaceQsvFilter or TonemapQsvFilter);
|
||||
|
||||
// if we have no filters, check whether we need to convert pixel format
|
||||
// since qsv doesn't seem to like doing that at the encoder
|
||||
@@ -414,15 +386,7 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
}
|
||||
}
|
||||
|
||||
// A tonemap converted the PIXELS to SDR, so the stream must stop announcing HDR — that
|
||||
// is a correctness requirement, not a normalization preference, and it holds even when
|
||||
// the profile has NormalizeColors off. Without this an operator with NormalizeColors
|
||||
// disabled gets tonemapped pixels still tagged bt2020 and the player converts them a
|
||||
// second time. Deliberately NOT done by hoisting usesVppQsv out of the guard: a
|
||||
// scale-only hardware chain on non-HDR content should still respect the preference.
|
||||
bool tonemapped = videoInputFile.FilterSteps.Any(f => f is TonemapOpenClQsvFilter or TonemapFilter);
|
||||
|
||||
if (tonemapped || (desiredState.ColorsAreBt709 && (!videoStream.ColorParams.IsBt709 || usesVppQsv)))
|
||||
if (desiredState.ColorsAreBt709 && (!videoStream.ColorParams.IsBt709 || usesVppQsv))
|
||||
{
|
||||
// _logger.LogDebug("Adding colorspace filter");
|
||||
|
||||
@@ -615,12 +579,8 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
}
|
||||
}
|
||||
|
||||
// only scale if scaling or padding was used for main video stream.
|
||||
// ScaleVaapiFilter belongs here too: the HDR/OpenCL tonemap path (ersatztv#505)
|
||||
// scales the video with it, and leaving it out left the subtitle canvas at
|
||||
// source resolution while the video shrank. VaapiPipelineBuilder already lists it.
|
||||
if (videoInputFile.FilterSteps.Any(s =>
|
||||
s is ScaleFilter or ScaleQsvFilter or ScaleVaapiFilter or PadFilter))
|
||||
// only scale if scaling or padding was used for main video stream
|
||||
if (videoInputFile.FilterSteps.Any(s => s is ScaleFilter or ScaleQsvFilter or PadFilter))
|
||||
{
|
||||
var scaleFilter = new ScaleImageFilter(desiredState.PaddedSize);
|
||||
subtitle.FilterSteps.Add(scaleFilter);
|
||||
@@ -680,84 +640,14 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
return currentState;
|
||||
}
|
||||
|
||||
// The QSV pipeline can only reach tonemap_opencl through the VA-API device that its own QSV
|
||||
// device is derived from, so every condition here is about that device existing and being
|
||||
// reachable with software frames in hand.
|
||||
private bool UseOpenClTonemap(
|
||||
VideoStream videoStream,
|
||||
PipelineContext context,
|
||||
FFmpegState ffmpegState,
|
||||
FrameState currentState)
|
||||
{
|
||||
if (!videoStream.ColorParams.IsHdr)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// The backstop for every case below, and for any future filter that lands ahead of the
|
||||
// tonemap: the route starts with hwupload, so the frames have to actually be in software.
|
||||
// Checked against the state rather than inferred from the enumeration, so a later change
|
||||
// that puts frames on a surface earlier degrades to the software tonemap instead of
|
||||
// emitting a second upload on top of an existing one.
|
||||
if (currentState.FrameDataLocation == FrameDataLocation.Hardware)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// ffmpeg has no vaapi on Windows, so there is no device to derive OpenCL from
|
||||
if (OperatingSystem.IsWindows())
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// with no configured device QsvHardwareAccelerationOption emits a bare "-init_hw_device
|
||||
// qsv=hw" and never initializes a VA-API device at all
|
||||
if (ffmpegState.VaapiDevice.Filter(d => !string.IsNullOrWhiteSpace(d)).IsNone)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// frames from the QSV decoder are ALREADY on a QSV surface (-hwaccel_output_format qsv),
|
||||
// and a QSV surface maps to neither OpenCL nor VA-API, so there is no route to the GPU
|
||||
// tonemap from here. Software tonemap is slower but it is the only one that is correct.
|
||||
if (ffmpegState.DecoderHardwareAccelerationMode == HardwareAccelerationMode.Qsv)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// deinterlace_qsv runs before the scale and leaves frames on a QSV surface, same problem
|
||||
if (context.ShouldDeinterlace)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
// ScaleQsvFilter is handed the SAR that VideoStream CALCULATES (it has a fallback for a
|
||||
// missing or 0:0 SAR); ScaleVaapiFilter instead multiplies by ffmpeg's runtime `sar`, which
|
||||
// is not the same value when the decoded frame leaves SAR unspecified. Rather than ship an
|
||||
// anamorphic HDR graph nobody has run, keep anamorphic sources on the software tonemap —
|
||||
// which is exactly what they got before this change, so it costs nothing they had.
|
||||
if (videoStream.IsAnamorphic)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
return _ffmpegCapabilities.HasFilter(FFmpegKnownFilter.TonemapOpenCL);
|
||||
}
|
||||
|
||||
private static FrameState SetScale(
|
||||
VideoInputFile videoInputFile,
|
||||
VideoStream videoStream,
|
||||
PipelineContext context,
|
||||
FFmpegState ffmpegState,
|
||||
FrameState desiredState,
|
||||
FrameState currentState,
|
||||
bool useOpenClTonemap)
|
||||
FrameState currentState)
|
||||
{
|
||||
if (useOpenClTonemap)
|
||||
{
|
||||
return SetScaleVaapiForTonemap(videoInputFile, desiredState, currentState);
|
||||
}
|
||||
|
||||
IPipelineFilterStep scaleStep;
|
||||
|
||||
bool useSoftwareFilter = ffmpegState is
|
||||
@@ -811,40 +701,6 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
return currentState;
|
||||
}
|
||||
|
||||
// HDR frames arrive from the VA-API decoder in system memory (the QSV pipeline deliberately
|
||||
// omits -hwaccel_output_format), so upload them to the VA-API device and scale THERE. Scaling
|
||||
// first matters: tonemapping the full-size frame instead costs ~50% more wall clock than the
|
||||
// software tonemap it replaces, which is the difference between above and below realtime.
|
||||
private static FrameState SetScaleVaapiForTonemap(
|
||||
VideoInputFile videoInputFile,
|
||||
FrameState desiredState,
|
||||
FrameState currentState)
|
||||
{
|
||||
// the decoder's yuv420p10le is not a VA-API surface format; p010/nv12 are
|
||||
IPixelFormat uploadFormat = currentState.PixelFormat.Map(pf => pf.BitDepth).IfNone(10) == 10
|
||||
? new PixelFormatP010()
|
||||
: new PixelFormatNv12(currentState.PixelFormat.Map(pf => pf.Name).IfNone(FFmpegFormat.NV12));
|
||||
|
||||
var upload = new HardwareUploadVaapiFilter(setFormat: true, deriveDevice: true);
|
||||
currentState = upload.NextState(currentState) with { PixelFormat = Some(uploadFormat) };
|
||||
videoInputFile.FilterSteps.Add(upload);
|
||||
|
||||
var scaleStep = new ScaleVaapiFilter(
|
||||
currentState,
|
||||
desiredState.ScaledSize,
|
||||
desiredState.PaddedSize,
|
||||
desiredState.CroppedSize,
|
||||
VideoStream.IsAnamorphicEdgeCase);
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(scaleStep.Filter))
|
||||
{
|
||||
currentState = scaleStep.NextState(currentState);
|
||||
videoInputFile.FilterSteps.Add(scaleStep);
|
||||
}
|
||||
|
||||
return currentState;
|
||||
}
|
||||
|
||||
private static FrameState SetDeinterlace(
|
||||
VideoInputFile videoInputFile,
|
||||
PipelineContext context,
|
||||
@@ -866,24 +722,26 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
|
||||
VideoStream videoStream,
|
||||
FFmpegState ffmpegState,
|
||||
FrameState desiredState,
|
||||
FrameState currentState,
|
||||
bool useOpenClTonemap)
|
||||
FrameState currentState)
|
||||
{
|
||||
if (videoStream.ColorParams.IsHdr)
|
||||
{
|
||||
foreach (IPixelFormat pixelFormat in desiredState.PixelFormat)
|
||||
{
|
||||
// NOTE: vpp_qsv=tonemap=1 is deliberately NOT an option here. On pre-Gen11 Intel
|
||||
// graphics it returns the frame untouched with no warning, so it does not tonemap,
|
||||
// it only LOOKS like it did (ersatztv#505). Either OpenCL tonemaps on the GPU or
|
||||
// the software filter does it on the CPU; there is no silently-wrong third branch.
|
||||
IPipelineFilterStep filter = useOpenClTonemap
|
||||
? new TonemapOpenClQsvFilter(ffmpegState, pixelFormat)
|
||||
: new TonemapFilter(ffmpegState, currentState, pixelFormat);
|
||||
|
||||
currentState = filter.NextState(currentState);
|
||||
videoStream.ResetColorParams(ColorParams.Default);
|
||||
videoInputFile.FilterSteps.Add(filter);
|
||||
if (ffmpegState.DecoderHardwareAccelerationMode == HardwareAccelerationMode.Qsv)
|
||||
{
|
||||
var filter = new TonemapQsvFilter();
|
||||
currentState = filter.NextState(currentState);
|
||||
videoStream.ResetColorParams(ColorParams.Default);
|
||||
videoInputFile.FilterSteps.Add(filter);
|
||||
}
|
||||
else
|
||||
{
|
||||
var filter = new TonemapFilter(ffmpegState, currentState, pixelFormat);
|
||||
currentState = filter.NextState(currentState);
|
||||
videoStream.ResetColorParams(ColorParams.Default);
|
||||
videoInputFile.FilterSteps.Add(filter);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
-7314
File diff suppressed because it is too large
Load Diff
-172
@@ -1,172 +0,0 @@
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace ErsatzTV.Infrastructure.MySql.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class Add_LibraryFolder_PathHash_UniqueIndex : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
// ersatztv#491 — audit/clean pre-existing duplicate LibraryFolder rows before the unique index.
|
||||
// Mirrors the Sqlite migration; see it for the full rationale. The one provider difference is
|
||||
// that every Path comparison here is forced BYTE-EXACT with CONVERT(... USING binary), because
|
||||
// MySql's string comparison differs from Sqlite's on two independent axes and the dedupe
|
||||
// deletes rows irreversibly:
|
||||
// * case — the server default (utf8mb4_general_ci) is case-INsensitive, so grouping under it
|
||||
// would collapse sibling folders differing only in case, legal on a case-sensitive fs;
|
||||
// * trailing spaces — utf8mb4_bin, the obvious fix for the case half, is a PAD SPACE
|
||||
// collation (verified on 8.4: '/media/Foo' = '/media/Foo ' is TRUE under it), so it would
|
||||
// still collapse "/media/Foo" and "/media/Foo ", two distinct legal directories.
|
||||
// Binary comparison is NO PAD and byte-exact, which is exactly what PathUtils.GetPathHash does
|
||||
// — so the dedupe now destroys only rows the unique index would actually have rejected, and
|
||||
// the two providers' migrations are semantically equivalent. (utf8mb4_0900_bin is also NO PAD
|
||||
// but carries a server-version floor; CONVERT USING binary does not.)
|
||||
// DROP TABLE IF EXISTS makes a retry after a partial failure safe (DDL implicitly commits on
|
||||
// MySql, so the migration is not atomic).
|
||||
migrationBuilder.Sql("DROP TABLE IF EXISTS `__LibraryFolderDedupe`");
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
CREATE TABLE `__LibraryFolderDedupe` (
|
||||
LoserId INT NOT NULL PRIMARY KEY,
|
||||
KeeperId INT NOT NULL
|
||||
)
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
INSERT INTO `__LibraryFolderDedupe` (LoserId, KeeperId)
|
||||
SELECT l.Id, k.KeeperId
|
||||
FROM LibraryFolder l
|
||||
INNER JOIN (
|
||||
SELECT LibraryPathId, CONVERT(Path USING binary) AS BinPath, MIN(Id) AS KeeperId
|
||||
FROM LibraryFolder
|
||||
GROUP BY LibraryPathId, CONVERT(Path USING binary)
|
||||
) k ON k.LibraryPathId = l.LibraryPathId AND k.BinPath = CONVERT(l.Path USING binary)
|
||||
WHERE l.Id <> k.KeeperId
|
||||
""");
|
||||
|
||||
// media files recorded against a duplicate folder follow the keeper (MediaFile.LibraryFolderId
|
||||
// is Restrict, so the delete below would fail otherwise)
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
UPDATE MediaFile
|
||||
SET LibraryFolderId = (
|
||||
SELECT KeeperId FROM `__LibraryFolderDedupe` WHERE LoserId = MediaFile.LibraryFolderId)
|
||||
WHERE LibraryFolderId IN (SELECT LoserId FROM `__LibraryFolderDedupe`)
|
||||
""");
|
||||
|
||||
// child folders parented on a duplicate follow the keeper (ParentId is Restrict as well)
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
UPDATE LibraryFolder
|
||||
SET ParentId = (
|
||||
SELECT KeeperId FROM `__LibraryFolderDedupe` WHERE LoserId = LibraryFolder.ParentId)
|
||||
WHERE ParentId IN (SELECT LoserId FROM `__LibraryFolderDedupe`)
|
||||
""");
|
||||
|
||||
// A folder parented on its OWN duplicate would become its own parent above. Unreachable from
|
||||
// any code path today, but this is a tree the scanner walks, so remove the cycle class rather
|
||||
// than reason about it.
|
||||
migrationBuilder.Sql("UPDATE LibraryFolder SET ParentId = NULL WHERE ParentId = Id");
|
||||
|
||||
// Clear the survivor's etag. Which duplicate the scanner was actually writing to was
|
||||
// arbitrary, so MIN(Id)'s etag may describe a stale view of the folder and would suppress the
|
||||
// next rescan. A null etag costs exactly one rescan and cannot be wrong.
|
||||
migrationBuilder.Sql(
|
||||
"UPDATE LibraryFolder SET Etag = NULL WHERE Id IN (SELECT KeeperId FROM `__LibraryFolderDedupe`)");
|
||||
|
||||
// ImageFolderDuration is 1:1 with LibraryFolder (unique index on LibraryFolderId), so the
|
||||
// duplicates' rows cannot all be repointed. Keep the keeper's own setting when it has one;
|
||||
// otherwise promote exactly one loser's (lowest Id) and drop the rest.
|
||||
migrationBuilder.Sql("DROP TABLE IF EXISTS `__LibraryFolderDedupeIfd`");
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
CREATE TABLE `__LibraryFolderDedupeIfd` (
|
||||
KeeperId INT NOT NULL PRIMARY KEY,
|
||||
IfdId INT NOT NULL
|
||||
)
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
INSERT INTO `__LibraryFolderDedupeIfd` (KeeperId, IfdId)
|
||||
SELECT d.KeeperId, MIN(i.Id)
|
||||
FROM `__LibraryFolderDedupe` d
|
||||
INNER JOIN ImageFolderDuration i ON i.LibraryFolderId = d.LoserId
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM ImageFolderDuration ki WHERE ki.LibraryFolderId = d.KeeperId)
|
||||
GROUP BY d.KeeperId
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
DELETE FROM ImageFolderDuration
|
||||
WHERE LibraryFolderId IN (SELECT LoserId FROM `__LibraryFolderDedupe`)
|
||||
AND Id NOT IN (SELECT IfdId FROM `__LibraryFolderDedupeIfd`)
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
UPDATE ImageFolderDuration
|
||||
SET LibraryFolderId = (
|
||||
SELECT KeeperId FROM `__LibraryFolderDedupeIfd` WHERE IfdId = ImageFolderDuration.Id)
|
||||
WHERE Id IN (SELECT IfdId FROM `__LibraryFolderDedupeIfd`)
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"DELETE FROM LibraryFolder WHERE Id IN (SELECT LoserId FROM `__LibraryFolderDedupe`)");
|
||||
|
||||
migrationBuilder.Sql("DROP TABLE `__LibraryFolderDedupeIfd`");
|
||||
migrationBuilder.Sql("DROP TABLE `__LibraryFolderDedupe`");
|
||||
|
||||
migrationBuilder.AddColumn<string>(
|
||||
name: "PathHash",
|
||||
table: "LibraryFolder",
|
||||
type: "varchar(64)",
|
||||
maxLength: 64,
|
||||
nullable: true)
|
||||
.Annotation("MySql:CharSet", "utf8mb4");
|
||||
|
||||
// Existing rows keep a null hash on purpose: a unique index treats nulls as distinct, so the
|
||||
// index applies cleanly to any database, and LibraryRepository.GetOrAddFolder heals each row
|
||||
// (SHA-256 of Path) the first time a scan touches it. Those rows are deduplicated above and are
|
||||
// still found by the Path lookup, so no insert can race them in the meantime.
|
||||
//
|
||||
// Order matters on MySql, and EF scaffolds it the other way round: InnoDB refuses to drop the
|
||||
// FK's only backing index ("Cannot drop index 'IX_LibraryFolder_LibraryPathId': needed in a
|
||||
// foreign key constraint"). Create the composite first — LibraryPathId is its leftmost column,
|
||||
// so it takes over as the FK's backing index — then drop the now-redundant single-column one.
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_LibraryFolder_LibraryPathId_PathHash",
|
||||
table: "LibraryFolder",
|
||||
columns: new[] { "LibraryPathId", "PathHash" },
|
||||
unique: true);
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_LibraryFolder_LibraryPathId",
|
||||
table: "LibraryFolder");
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
// mirror of Up: restore the single-column index before dropping the composite one, so the
|
||||
// foreign key is never left without a backing index
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_LibraryFolder_LibraryPathId",
|
||||
table: "LibraryFolder",
|
||||
column: "LibraryPathId");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_LibraryFolder_LibraryPathId_PathHash",
|
||||
table: "LibraryFolder");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "PathHash",
|
||||
table: "LibraryFolder");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1353,16 +1353,11 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
|
||||
b.Property<string>("Path")
|
||||
.HasColumnType("longtext");
|
||||
|
||||
b.Property<string>("PathHash")
|
||||
.HasMaxLength(64)
|
||||
.HasColumnType("varchar(64)");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("ParentId");
|
||||
b.HasIndex("LibraryPathId");
|
||||
|
||||
b.HasIndex("LibraryPathId", "PathHash")
|
||||
.IsUnique();
|
||||
b.HasIndex("ParentId");
|
||||
|
||||
b.ToTable("LibraryFolder", (string)null);
|
||||
});
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
using System.Data;
|
||||
using Microsoft.Data.Sqlite;
|
||||
|
||||
namespace ErsatzTV.Infrastructure.Sqlite.Data;
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. SQLite's built-in <c>lower()</c>/<c>upper()</c> fold ASCII ONLY — <c>lower('Édith')</c>
|
||||
/// returns <c>'Édith'</c> unchanged — so a facet value whose prefix carries an uppercase non-ASCII
|
||||
/// character can never be matched by the prefix predicate the facet-value endpoint emits. Registering a
|
||||
/// managed scalar gives that one query a Unicode-correct fold. Wired to
|
||||
/// <see cref="ErsatzTV.Infrastructure.Data.TvContext.RegisterUnicodeCaseFunctions" /> at startup.
|
||||
/// </summary>
|
||||
public static class SqliteUnicodeFunctions
|
||||
{
|
||||
/// <summary>
|
||||
/// SQL name of the invariant-uppercase fold. The facet-value handler interpolates this constant into
|
||||
/// its SQL, so the two cannot drift apart.
|
||||
/// </summary>
|
||||
public const string UpperInvariantFunction = "etv_upper";
|
||||
|
||||
/// <summary>
|
||||
/// Registers <see cref="UpperInvariantFunction" /> on <paramref name="connection" /> when it is a
|
||||
/// SQLite connection, and does nothing otherwise. Idempotent — a repeat registration replaces the
|
||||
/// previous delegate with an identical one — so the single call site may call it unconditionally.
|
||||
/// <para>
|
||||
/// The property this fold has to satisfy is ONE-SIDED: the SQL stage may over-match freely,
|
||||
/// because the endpoint applies an exact <see cref="StringComparison.OrdinalIgnoreCase" /> filter
|
||||
/// in memory afterwards, but it must never UNDER-match — no later stage can reintroduce a row SQL
|
||||
/// never returned. <see cref="string.ToUpperInvariant" /> satisfies it because
|
||||
/// <c>OrdinalIgnoreCase</c> equality is a strict SUBSET of invariant-uppercase equality, so
|
||||
/// folding both sides with it yields a superset of the final filter's matches.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Do not restate that as "<c>OrdinalIgnoreCase</c> IS invariant-uppercase-then-ordinal" — it is
|
||||
/// not, and the difference is measurable: <c>char.ToUpperInvariant('ſ')</c> (U+017F) is <c>'S'</c>,
|
||||
/// yet <c>"ſweet".StartsWith("S", OrdinalIgnoreCase)</c> is <b>false</b>. That gap is precisely
|
||||
/// the harmless direction — SQL returns the row, the in-memory filter drops it. The containment,
|
||||
/// not any identity of the two foldings, is what makes this safe.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Registration is per-connection and therefore done at the one call site that uses the function,
|
||||
/// not through an EF connection interceptor: Dapper opens a closed connection itself, and a direct
|
||||
/// ADO open does not raise EF's interceptors — so an interceptor-based seam would silently miss
|
||||
/// exactly the query that needs it.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
public static void Register(IDbConnection connection)
|
||||
{
|
||||
if (connection is SqliteConnection sqlite)
|
||||
{
|
||||
sqlite.CreateFunction(
|
||||
UpperInvariantFunction,
|
||||
(string? value) => value?.ToUpperInvariant(),
|
||||
isDeterministic: true);
|
||||
}
|
||||
}
|
||||
}
|
||||
-7139
File diff suppressed because it is too large
Load Diff
-157
@@ -1,157 +0,0 @@
|
||||
using Microsoft.EntityFrameworkCore.Migrations;
|
||||
|
||||
#nullable disable
|
||||
|
||||
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
|
||||
{
|
||||
/// <inheritdoc />
|
||||
public partial class Add_LibraryFolder_PathHash_UniqueIndex : Migration
|
||||
{
|
||||
/// <inheritdoc />
|
||||
protected override void Up(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
// ersatztv#491 — audit/clean pre-existing duplicate LibraryFolder rows before the unique index.
|
||||
// Duplicates were reachable before #488 (the folder lookup read a scan-start in-memory snapshot,
|
||||
// so a folder created earlier in the SAME scan was invisible and inserted again) and via the
|
||||
// check-then-insert race the index now closes. Keep the lowest Id per (LibraryPathId, Path) and
|
||||
// repoint every dependent row at it before deleting the losers. The helper tables keep the
|
||||
// statements readable; DROP TABLE IF EXISTS makes a retry after a partial failure safe.
|
||||
migrationBuilder.Sql("DROP TABLE IF EXISTS __LibraryFolderDedupe");
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
CREATE TABLE __LibraryFolderDedupe (
|
||||
LoserId INTEGER NOT NULL PRIMARY KEY,
|
||||
KeeperId INTEGER NOT NULL
|
||||
)
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
INSERT INTO __LibraryFolderDedupe (LoserId, KeeperId)
|
||||
SELECT l.Id, k.KeeperId
|
||||
FROM LibraryFolder l
|
||||
INNER JOIN (
|
||||
SELECT LibraryPathId, Path, MIN(Id) AS KeeperId
|
||||
FROM LibraryFolder
|
||||
GROUP BY LibraryPathId, Path
|
||||
) k ON k.LibraryPathId = l.LibraryPathId AND k.Path = l.Path
|
||||
WHERE l.Id <> k.KeeperId
|
||||
""");
|
||||
|
||||
// media files recorded against a duplicate folder follow the keeper (MediaFile.LibraryFolderId
|
||||
// is Restrict, so the delete below would fail otherwise)
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
UPDATE MediaFile
|
||||
SET LibraryFolderId = (
|
||||
SELECT KeeperId FROM __LibraryFolderDedupe WHERE LoserId = MediaFile.LibraryFolderId)
|
||||
WHERE LibraryFolderId IN (SELECT LoserId FROM __LibraryFolderDedupe)
|
||||
""");
|
||||
|
||||
// child folders parented on a duplicate follow the keeper (ParentId is Restrict as well)
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
UPDATE LibraryFolder
|
||||
SET ParentId = (
|
||||
SELECT KeeperId FROM __LibraryFolderDedupe WHERE LoserId = LibraryFolder.ParentId)
|
||||
WHERE ParentId IN (SELECT LoserId FROM __LibraryFolderDedupe)
|
||||
""");
|
||||
|
||||
// A folder parented on its OWN duplicate would become its own parent above. Unreachable from
|
||||
// any code path today, but this is a tree the scanner walks, so remove the cycle class rather
|
||||
// than reason about it.
|
||||
migrationBuilder.Sql("UPDATE LibraryFolder SET ParentId = NULL WHERE ParentId = Id");
|
||||
|
||||
// Clear the survivor's etag. Which duplicate the scanner was actually writing to was
|
||||
// arbitrary, so MIN(Id)'s etag may describe a stale view of the folder and would suppress the
|
||||
// next rescan. A null etag costs exactly one rescan and cannot be wrong.
|
||||
migrationBuilder.Sql(
|
||||
"UPDATE LibraryFolder SET Etag = NULL WHERE Id IN (SELECT KeeperId FROM __LibraryFolderDedupe)");
|
||||
|
||||
// ImageFolderDuration is 1:1 with LibraryFolder (unique index on LibraryFolderId), so the
|
||||
// duplicates' rows cannot all be repointed. Keep the keeper's own setting when it has one;
|
||||
// otherwise promote exactly one loser's (lowest Id) and drop the rest.
|
||||
migrationBuilder.Sql("DROP TABLE IF EXISTS __LibraryFolderDedupeIfd");
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
CREATE TABLE __LibraryFolderDedupeIfd (
|
||||
KeeperId INTEGER NOT NULL PRIMARY KEY,
|
||||
IfdId INTEGER NOT NULL
|
||||
)
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
INSERT INTO __LibraryFolderDedupeIfd (KeeperId, IfdId)
|
||||
SELECT d.KeeperId, MIN(i.Id)
|
||||
FROM __LibraryFolderDedupe d
|
||||
INNER JOIN ImageFolderDuration i ON i.LibraryFolderId = d.LoserId
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM ImageFolderDuration ki WHERE ki.LibraryFolderId = d.KeeperId)
|
||||
GROUP BY d.KeeperId
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
DELETE FROM ImageFolderDuration
|
||||
WHERE LibraryFolderId IN (SELECT LoserId FROM __LibraryFolderDedupe)
|
||||
AND Id NOT IN (SELECT IfdId FROM __LibraryFolderDedupeIfd)
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"""
|
||||
UPDATE ImageFolderDuration
|
||||
SET LibraryFolderId = (
|
||||
SELECT KeeperId FROM __LibraryFolderDedupeIfd WHERE IfdId = ImageFolderDuration.Id)
|
||||
WHERE Id IN (SELECT IfdId FROM __LibraryFolderDedupeIfd)
|
||||
""");
|
||||
|
||||
migrationBuilder.Sql(
|
||||
"DELETE FROM LibraryFolder WHERE Id IN (SELECT LoserId FROM __LibraryFolderDedupe)");
|
||||
|
||||
migrationBuilder.Sql("DROP TABLE __LibraryFolderDedupeIfd");
|
||||
migrationBuilder.Sql("DROP TABLE __LibraryFolderDedupe");
|
||||
|
||||
migrationBuilder.AddColumn<string>(
|
||||
name: "PathHash",
|
||||
table: "LibraryFolder",
|
||||
type: "TEXT",
|
||||
maxLength: 64,
|
||||
nullable: true);
|
||||
|
||||
// Existing rows keep a null hash on purpose: a unique index treats nulls as distinct, so the
|
||||
// index applies cleanly to any database, and LibraryRepository.GetOrAddFolder heals each row
|
||||
// (SHA-256 of Path) the first time a scan touches it. Those rows are deduplicated above and are
|
||||
// still found by the Path lookup, so no insert can race them in the meantime.
|
||||
//
|
||||
// Create-then-drop rather than EF's scaffolded drop-then-create, matching the MySql copy, where
|
||||
// InnoDB refuses to drop the foreign key's only backing index.
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_LibraryFolder_LibraryPathId_PathHash",
|
||||
table: "LibraryFolder",
|
||||
columns: new[] { "LibraryPathId", "PathHash" },
|
||||
unique: true);
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_LibraryFolder_LibraryPathId",
|
||||
table: "LibraryFolder");
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override void Down(MigrationBuilder migrationBuilder)
|
||||
{
|
||||
migrationBuilder.CreateIndex(
|
||||
name: "IX_LibraryFolder_LibraryPathId",
|
||||
table: "LibraryFolder",
|
||||
column: "LibraryPathId");
|
||||
|
||||
migrationBuilder.DropIndex(
|
||||
name: "IX_LibraryFolder_LibraryPathId_PathHash",
|
||||
table: "LibraryFolder");
|
||||
|
||||
migrationBuilder.DropColumn(
|
||||
name: "PathHash",
|
||||
table: "LibraryFolder");
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1298,16 +1298,11 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
|
||||
b.Property<string>("Path")
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.Property<string>("PathHash")
|
||||
.HasMaxLength(64)
|
||||
.HasColumnType("TEXT");
|
||||
|
||||
b.HasKey("Id");
|
||||
|
||||
b.HasIndex("ParentId");
|
||||
b.HasIndex("LibraryPathId");
|
||||
|
||||
b.HasIndex("LibraryPathId", "PathHash")
|
||||
.IsUnique();
|
||||
b.HasIndex("ParentId");
|
||||
|
||||
b.ToTable("LibraryFolder", (string)null);
|
||||
});
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Metadata.Builders;
|
||||
|
||||
@@ -10,19 +10,6 @@ public class LibraryFolderConfiguration : IEntityTypeConfiguration<LibraryFolder
|
||||
{
|
||||
builder.ToTable("LibraryFolder");
|
||||
|
||||
// ersatztv#491: GetOrAddFolder is a check-then-insert, so two callers racing the same folder both
|
||||
// miss the lookup and both insert. The unique index makes the duplicate impossible at the storage
|
||||
// layer (the loser gets a constraint violation the repository catches and re-reads).
|
||||
// Indexed on PathHash rather than Path because Path is unbounded (MySQL longtext, which cannot be
|
||||
// indexed without a prefix length, and whose default collation is case-INsensitive — a prefix
|
||||
// index would also false-collide sibling folders differing only in case on a case-sensitive
|
||||
// filesystem). This mirrors the existing MediaFile.Path/PathHash pair.
|
||||
builder.Property(f => f.PathHash)
|
||||
.HasMaxLength(64);
|
||||
|
||||
builder.HasIndex(f => new { f.LibraryPathId, f.PathHash })
|
||||
.IsUnique();
|
||||
|
||||
builder.HasOne(f => f.Parent)
|
||||
.WithMany(p => p.Children)
|
||||
.HasForeignKey(f => f.ParentId)
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
using System.IO.Abstractions;
|
||||
using Dapper;
|
||||
using ErsatzTV.Core;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Core.Interfaces.Repositories;
|
||||
using ErsatzTV.Infrastructure.Extensions;
|
||||
@@ -111,35 +110,15 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
|
||||
|
||||
if (knownFolder.IsNone)
|
||||
{
|
||||
var newFolder = new LibraryFolder
|
||||
{
|
||||
Path = path,
|
||||
PathHash = PathUtils.GetPathHash(path),
|
||||
Etag = etag,
|
||||
LibraryPathId = libraryPath.Id
|
||||
};
|
||||
|
||||
try
|
||||
{
|
||||
await dbContext.LibraryFolders.AddAsync(newFolder);
|
||||
await dbContext.SaveChangesAsync();
|
||||
}
|
||||
catch (DbUpdateException ex) when (TvContext.IsUniqueConstraintViolation(ex))
|
||||
{
|
||||
// ersatztv#491: a concurrent caller created this folder between the caller's lookup and
|
||||
// this insert. The etag write is the whole point of the call, so apply it to the winner's
|
||||
// row rather than failing the scan.
|
||||
dbContext.Entry(newFolder).State = EntityState.Detached;
|
||||
LibraryFolder winner = await GetFolder(dbContext, libraryPath.Id, path);
|
||||
if (winner is null)
|
||||
await dbContext.LibraryFolders.AddAsync(
|
||||
new LibraryFolder
|
||||
{
|
||||
throw;
|
||||
}
|
||||
Path = path,
|
||||
Etag = etag,
|
||||
LibraryPathId = libraryPath.Id
|
||||
});
|
||||
|
||||
await dbContext.Connection.ExecuteAsync(
|
||||
"UPDATE LibraryFolder SET Etag = @Etag WHERE Id = @Id",
|
||||
new { winner.Id, Etag = etag });
|
||||
}
|
||||
await dbContext.SaveChangesAsync();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -195,74 +174,11 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
|
||||
// local scan path (via GetLibrary) and is null on the remote (Jellyfin) sync path, which used
|
||||
// to NRE every Jellyfin music-video scan here (ersatztv#488). The local scanners already hit
|
||||
// the db once per folder via GetParentFolderId, so this adds no new query pattern.
|
||||
LibraryFolder knownFolder = await GetFolder(dbContext, libraryPath.Id, folder);
|
||||
|
||||
// add new folder to library path
|
||||
if (knownFolder is null)
|
||||
{
|
||||
LibraryFolder newFolder = CreateNewFolder(libraryPath, maybeParentFolder, folder);
|
||||
try
|
||||
{
|
||||
await dbContext.LibraryFolders.AddAsync(newFolder);
|
||||
await dbContext.SaveChangesAsync();
|
||||
knownFolder = newFolder;
|
||||
}
|
||||
catch (DbUpdateException ex) when (TvContext.IsUniqueConstraintViolation(ex))
|
||||
{
|
||||
// ersatztv#491: the lookup above is not atomic with this insert, so a concurrent caller
|
||||
// scanning the same folder can slip its row in between. The unique index on
|
||||
// (LibraryPathId, PathHash) turns that lost race into a constraint violation instead of a
|
||||
// duplicate row; adopt the winner's row rather than failing the scan. Detach first so the
|
||||
// failed insert is not retried by anything reusing this context.
|
||||
dbContext.Entry(newFolder).State = EntityState.Detached;
|
||||
knownFolder = await GetFolder(dbContext, libraryPath.Id, folder);
|
||||
if (knownFolder is null)
|
||||
{
|
||||
// no winner to adopt — the violation came from somewhere else, so don't swallow it
|
||||
throw;
|
||||
}
|
||||
}
|
||||
}
|
||||
else if (string.IsNullOrEmpty(knownFolder.PathHash))
|
||||
{
|
||||
// Heal a row created before the PathHash column existed, so it participates in the unique
|
||||
// index from here on (a null hash is distinct from every other value, so it does not).
|
||||
//
|
||||
// This is opportunistic maintenance on a hot scan path, so it must never be able to abort a
|
||||
// scan. It goes through EF rather than a raw Dapper UPDATE precisely so a collision surfaces
|
||||
// as a classifiable DbUpdateException instead of a bare provider exception, and a lost heal
|
||||
// is simply left for the next scan. Reachable only if some other row already owns
|
||||
// (LibraryPathId, hash) — a legacy duplicate the migration's dedupe could not see (e.g. one
|
||||
// with a NULL Path, which `NULL = NULL` excludes from its grouping).
|
||||
string pathHash = PathUtils.GetPathHash(folder);
|
||||
LibraryFolder tracked = null;
|
||||
try
|
||||
{
|
||||
// the predicate must agree with the IsNullOrEmpty guard above, or a PathHash = '' row would
|
||||
// enter this branch, match nothing, and silently never heal
|
||||
tracked = await dbContext.LibraryFolders
|
||||
.FirstOrDefaultAsync(f => f.Id == knownFolder.Id && (f.PathHash == null || f.PathHash == ""));
|
||||
if (tracked is not null)
|
||||
{
|
||||
tracked.PathHash = pathHash;
|
||||
await dbContext.SaveChangesAsync();
|
||||
knownFolder.PathHash = pathHash;
|
||||
}
|
||||
}
|
||||
catch (DbUpdateException ex) when (
|
||||
TvContext.IsUniqueConstraintViolation(ex) || ex is DbUpdateConcurrencyException)
|
||||
{
|
||||
// Either another row already owns this hash, or the row was deleted out from under us by a
|
||||
// concurrent library edit (DbUpdateConcurrencyException derives from DbUpdateException but
|
||||
// carries no provider exception, so the classifier does NOT recognize it). Both mean "the
|
||||
// heal is moot" — leave the row unhealed rather than fail the scan, per the invariant above.
|
||||
if (tracked is not null)
|
||||
{
|
||||
// drop the failed change so it cannot be replayed by a later save on this context
|
||||
dbContext.Entry(tracked).State = EntityState.Detached;
|
||||
}
|
||||
}
|
||||
}
|
||||
LibraryFolder knownFolder = await dbContext.LibraryFolders
|
||||
.AsNoTracking()
|
||||
.Filter(f => f.LibraryPathId == libraryPath.Id && f.Path == folder)
|
||||
.FirstOrDefaultAsync()
|
||||
?? CreateNewFolder(libraryPath, maybeParentFolder, folder);
|
||||
|
||||
// update parent folder if not present
|
||||
foreach (int parentFolder in maybeParentFolder)
|
||||
@@ -277,6 +193,13 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
|
||||
}
|
||||
}
|
||||
|
||||
// add new folder to library path
|
||||
if (knownFolder.Id < 1)
|
||||
{
|
||||
await dbContext.LibraryFolders.AddAsync(knownFolder);
|
||||
await dbContext.SaveChangesAsync();
|
||||
}
|
||||
|
||||
return knownFolder;
|
||||
}
|
||||
|
||||
@@ -298,71 +221,6 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
|
||||
new { Path = normalizedLibraryPath, libraryPath.Id });
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The in-memory half of <see cref="GetFolder" />, lifted out so the ordinal decision is pinned by a
|
||||
/// test with no database at all: the collation behaviour that makes it necessary is MySQL-only, so a
|
||||
/// SQLite-backed test cannot exercise it (SQLite's <c>=</c> on TEXT is already binary and never
|
||||
/// returns the case-differing candidate). Given the candidates a case-INsensitive server may return,
|
||||
/// pick the one whose path matches ordinally; callers pass them lowest <c>Id</c> first.
|
||||
/// </summary>
|
||||
public static LibraryFolder ResolveExact(IReadOnlyList<LibraryFolder> candidates, string folder)
|
||||
{
|
||||
for (var i = 0; i < candidates.Count; i++)
|
||||
{
|
||||
if (string.Equals(candidates[i].Path, folder, StringComparison.Ordinal))
|
||||
{
|
||||
return candidates[i];
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Resolve a folder by its exact path within a library path.
|
||||
/// <para>
|
||||
/// The SQL equality is only a *narrowing* filter, not the identity test. On MySQL, `Path` is a
|
||||
/// `longtext` whose collation the schema does not pin — only the `utf8mb4` charset — so the
|
||||
/// effective comparison is whatever the server defaults to, and it differs from byte equality:
|
||||
/// <list type="bullet">
|
||||
/// <item>
|
||||
/// always case-INsensitive: both plausible defaults are `_ci` (8.4 verified:
|
||||
/// `utf8mb4_0900_ai_ci`; older servers `utf8mb4_general_ci`), which is why
|
||||
/// <see cref="TvContext.CaseInsensitiveCollation" /> exists at all;
|
||||
/// </item>
|
||||
/// <item>
|
||||
/// possibly PAD SPACE, making trailing spaces insignificant — true of
|
||||
/// `utf8mb4_general_ci`, but NOT of `utf8mb4_0900_ai_ci`, which is NO PAD. So this axis
|
||||
/// is server-dependent rather than guaranteed, and must be tolerated rather than
|
||||
/// assumed either way.
|
||||
/// </item>
|
||||
/// </list>
|
||||
/// `Path = @folder` can therefore also match siblings differing only in case, or (on a PAD
|
||||
/// SPACE server) in trailing whitespace — all legal on a case-sensitive filesystem, and all
|
||||
/// preserved by the #491 migration. Crucially the SQL predicate is a *superset*: every such
|
||||
/// quirk makes it more permissive, never less, so it cannot miss a byte-exact match. Identity
|
||||
/// is then settled in memory by <see cref="ResolveExact" /> with an ORDINAL comparison,
|
||||
/// matching <c>PathUtils.GetPathHash</c>, which hashes the exact bytes. Without this the lookup
|
||||
/// and the hash disagree, and the PathHash heal could stamp one sibling's hash onto the other's
|
||||
/// row.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Ordered by Id so the result is deterministic: an unordered <c>FirstOrDefault</c> may return a
|
||||
/// different candidate run to run as the query plan changes (adding the composite index alone
|
||||
/// can flip it), which would make the heal non-idempotent.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private static async Task<LibraryFolder> GetFolder(TvContext dbContext, int libraryPathId, string folder)
|
||||
{
|
||||
List<LibraryFolder> candidates = await dbContext.LibraryFolders
|
||||
.AsNoTracking()
|
||||
.Filter(f => f.LibraryPathId == libraryPathId && f.Path == folder)
|
||||
.OrderBy(f => f.Id)
|
||||
.ToListAsync();
|
||||
|
||||
return ResolveExact(candidates, folder);
|
||||
}
|
||||
|
||||
private static LibraryFolder CreateNewFolder(LibraryPath libraryPath, Option<int> maybeParentFolder, string folder)
|
||||
{
|
||||
int? parentId = null;
|
||||
@@ -374,7 +232,6 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
|
||||
return new LibraryFolder
|
||||
{
|
||||
Path = folder,
|
||||
PathHash = PathUtils.GetPathHash(folder),
|
||||
Etag = null,
|
||||
LibraryPathId = libraryPath.Id,
|
||||
ParentId = parentId
|
||||
|
||||
@@ -1144,7 +1144,7 @@ public class MediaCollectionRepository : IMediaCollectionRepository
|
||||
|
||||
var allArtists = items.OfType<Song>()
|
||||
.SelectMany(s => s.SongMetadata)
|
||||
.Map(sm => Optional(sm.AlbumArtists).Flatten().HeadOrNone().Match(aa => aa, string.Empty))
|
||||
.Map(sm => sm.AlbumArtists.HeadOrNone().Match(aa => aa, string.Empty))
|
||||
.Distinct()
|
||||
.ToList();
|
||||
|
||||
@@ -1157,7 +1157,7 @@ public class MediaCollectionRepository : IMediaCollectionRepository
|
||||
foreach (Song song in items.OfType<Song>())
|
||||
{
|
||||
string firstArtist = song.SongMetadata
|
||||
.SelectMany(sm => Optional(sm.AlbumArtists).Flatten())
|
||||
.SelectMany(sm => sm.AlbumArtists)
|
||||
.HeadOrNone()
|
||||
.Match(aa => aa, string.Empty);
|
||||
|
||||
|
||||
@@ -36,18 +36,6 @@ public class TvContext : DbContext
|
||||
/// </summary>
|
||||
public static Func<DbUpdateException, bool> IsUniqueConstraintViolation { get; set; } = static _ => false;
|
||||
|
||||
/// <summary>
|
||||
/// Registers provider-specific SQL scalar functions on a connection, called immediately before a raw
|
||||
/// query that needs them. Set at startup by the active provider's wiring, mirroring
|
||||
/// <see cref="IsUniqueConstraintViolation" />: SQLite points this at
|
||||
/// <c>SqliteUnicodeFunctions.Register</c>, MySQL leaves it a no-op because its own <c>LOWER()</c> is
|
||||
/// already Unicode-aware and needs no help. Defaults to a no-op, which is safe because the sole
|
||||
/// caller invokes it only on the SQLite branch that requires it, and an unwired provider then fails
|
||||
/// LOUDLY ("no such function: etv_upper") rather than returning silently wrong results. See
|
||||
/// ersatztv#668.
|
||||
/// </summary>
|
||||
public static Action<IDbConnection> RegisterUnicodeCaseFunctions { get; set; } = static _ => { };
|
||||
|
||||
public IDbConnection Connection => Database.GetDbConnection();
|
||||
|
||||
public DbSet<ConfigElement> ConfigElements { get; set; }
|
||||
|
||||
@@ -202,51 +202,6 @@ public class ToolCatalogTests
|
||||
(tool.QueryParameters ?? new HashSet<string>()).ShouldNotContain("ifMatch");
|
||||
}
|
||||
|
||||
// #616: the catalog described pageNum as "1-based" while every paged REST controller defaults it
|
||||
// to 0 and skips pageNum * pageSize. A caller that trusted the description started at page 1 and
|
||||
// silently lost the first page — no error, just a short set that reads like missing data. The
|
||||
// description is the whole contract an MCP client has, so it is pinned here for EVERY paged tool.
|
||||
[Test]
|
||||
public void Paged_Tools_Should_Document_PageNum_As_Zero_Based()
|
||||
{
|
||||
ToolDefinition[] paged = ToolCatalog.All
|
||||
.Where(t => t.InputSchema.RootElement.GetProperty("properties").TryGetProperty("pageNum", out _))
|
||||
.ToArray();
|
||||
|
||||
// Guard the guard twice over. An emptiness check alone is not enough: this test filters on
|
||||
// tools that ALREADY declare pageNum, so a tool wrapping a paged endpoint while declaring no
|
||||
// paging args escapes the filter entirely and the test still passes. That is not
|
||||
// hypothetical — ersatztv_list_playouts and ersatztv_get_playout_items did exactly that, and
|
||||
// because ToolArgumentValidator rejects undeclared arguments, an MCP caller was hard-capped
|
||||
// at the first 100 rows with no way to ask for more. So the expected set is named here: a
|
||||
// new tool over a paged endpoint must be added to it, and dropping paging from any of these
|
||||
// fails the test rather than silently shrinking its scope.
|
||||
string[] mustDeclarePaging =
|
||||
[
|
||||
"ersatztv_get_collection_items",
|
||||
"ersatztv_get_playout_items",
|
||||
"ersatztv_list_playouts",
|
||||
"ersatztv_search",
|
||||
"ersatztv_search_all_items"
|
||||
];
|
||||
|
||||
paged.Select(t => t.Name).OrderBy(n => n, StringComparer.Ordinal)
|
||||
.ShouldBe(mustDeclarePaging.OrderBy(n => n, StringComparer.Ordinal));
|
||||
|
||||
foreach (ToolDefinition tool in paged)
|
||||
{
|
||||
string description = tool.InputSchema.RootElement
|
||||
.GetProperty("properties")
|
||||
.GetProperty("pageNum")
|
||||
.GetProperty("description")
|
||||
.GetString()
|
||||
.ShouldNotBeNull();
|
||||
|
||||
description.ShouldContain("0-based");
|
||||
description.ShouldNotContain("1-based");
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Scan_Library_Tool_Should_Register_Deep_As_A_Query_Parameter()
|
||||
{
|
||||
|
||||
@@ -29,14 +29,9 @@ public static class ToolCatalog
|
||||
Get("ersatztv_list_schedules", "List schedules.", "/api/v1/schedules"),
|
||||
Get("ersatztv_get_schedule", "Get a schedule by id.", "/api/v1/schedules/{id}", IdPath("Schedule id.")),
|
||||
Get("ersatztv_get_schedule_items", "Get a schedule's items. Emits the schedule ETag.", "/api/v1/schedules/{id}/items", IdPath("Schedule id.")),
|
||||
Get("ersatztv_list_playouts", "List playouts (paged).", "/api/v1/playouts", [], Page()),
|
||||
Get("ersatztv_list_playouts", "List playouts.", "/api/v1/playouts"),
|
||||
Get("ersatztv_get_playout", "Get a playout by id.", "/api/v1/playouts/{id}", IdPath("Playout id.")),
|
||||
Get(
|
||||
"ersatztv_get_playout_items",
|
||||
"Get upcoming items (and unscheduled gaps) for a playout (paged).",
|
||||
"/api/v1/playouts/{id}/items",
|
||||
[IdPath("Playout id.")],
|
||||
Page()),
|
||||
Get("ersatztv_get_playout_items", "Get upcoming items (and unscheduled gaps) for a playout.", "/api/v1/playouts/{id}/items", IdPath("Playout id.")),
|
||||
Get("ersatztv_list_ffmpeg_profiles", "List FFmpeg profiles.", "/api/v1/ffmpeg/profiles"),
|
||||
Get("ersatztv_get_ffmpeg_profile", "Get an FFmpeg profile by id.", "/api/v1/ffmpeg/profiles/{id}", IdPath("FFmpeg profile id.")),
|
||||
Get(
|
||||
@@ -166,12 +161,7 @@ public static class ToolCatalog
|
||||
"ersatztv_reset_channel_playout",
|
||||
"Queue a rebuild of a channel's playout (202 Accepted; 409 if a build is already running).",
|
||||
"/api/v1/channels/{id}/playout/reset",
|
||||
[
|
||||
IdPath(
|
||||
"CHANNEL id — not the playout id. The two id spaces overlap numerically, so "
|
||||
+ "passing a playout id here silently resets a different channel; take this "
|
||||
+ "value from a playout row's channelId field (issue #616).")
|
||||
],
|
||||
[IdPath("Channel id.")],
|
||||
[Str("mode", "Optional playout build mode; omit for the default. GET a playout to see valid values.", arg: In.Query)]),
|
||||
Delete("ersatztv_delete_channel", "Delete a channel.", "/api/v1/channels/{id}", IdPath("Channel id.")),
|
||||
Put(
|
||||
@@ -240,21 +230,10 @@ public static class ToolCatalog
|
||||
private static Arg ObjArray(string name, string description, bool required = false) =>
|
||||
new(name, "array", description, required, In.Body, ItemType: "object");
|
||||
|
||||
// Paging mirrors the REST API it wraps, which is 0-BASED everywhere (issue #616): every paged
|
||||
// controller defaults pageNum to 0 and skips `pageNum * pageSize`. The description said "1-based",
|
||||
// so a caller that trusted it silently skipped the first page and read the result as data loss.
|
||||
// Documented rather than translated: a 1-based MCP over a 0-based API would make the same
|
||||
// parameter name mean two different things depending on which surface you were reading.
|
||||
// pageSize is clamped server-side and the cap is PER-ENDPOINT (100 for most reads, 200 for
|
||||
// auto-tune members, 1000 for search/all-items). The invariant is not a single number: the
|
||||
// offset always derives from the EFFECTIVE page size, never the requested one.
|
||||
private static Arg[] Page() =>
|
||||
[
|
||||
Int("pageNum", "0-based page number; the first page is 0 (optional, default 0).", arg: In.Query),
|
||||
Int(
|
||||
"pageSize",
|
||||
"Page size (optional). Clamped server-side, so pages may be narrower than requested.",
|
||||
arg: In.Query)
|
||||
Int("pageNum", "1-based page number (optional).", arg: In.Query),
|
||||
Int("pageSize", "Page size (optional).", arg: In.Query)
|
||||
];
|
||||
|
||||
// The channel create/update body (CreateChannelRequest / UpdateChannelRequest — the id comes from
|
||||
|
||||
@@ -24,7 +24,6 @@ using ErsatzTV.Infrastructure.Emby;
|
||||
using ErsatzTV.Infrastructure.Images;
|
||||
using ErsatzTV.Infrastructure.Jellyfin;
|
||||
using ErsatzTV.Infrastructure.Metadata;
|
||||
using ErsatzTV.Infrastructure.MySql.Data;
|
||||
using ErsatzTV.Infrastructure.Plex;
|
||||
using ErsatzTV.Infrastructure.Runtime;
|
||||
using ErsatzTV.Infrastructure.Search;
|
||||
@@ -153,16 +152,10 @@ public class Program
|
||||
}
|
||||
});
|
||||
|
||||
// Keep this block in sync with ErsatzTV/Startup.cs — the scanner is a SEPARATE process
|
||||
// (launched by CallLibraryScannerHandler), so any TvContext provider static the host wires
|
||||
// has to be wired here too or it silently keeps its default in every scan.
|
||||
// ProviderStaticsWiringTests enforces that parity.
|
||||
if (databaseProvider == Provider.Sqlite.Name)
|
||||
{
|
||||
TvContext.LastInsertedRowId = "last_insert_rowid()";
|
||||
TvContext.CaseInsensitiveCollation = "NOCASE";
|
||||
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
|
||||
TvContext.RegisterUnicodeCaseFunctions = SqliteUnicodeFunctions.Register;
|
||||
|
||||
SqlMapper.AddTypeHandler(new DateTimeOffsetHandler());
|
||||
SqlMapper.AddTypeHandler(new GuidHandler());
|
||||
@@ -173,11 +166,6 @@ public class Program
|
||||
{
|
||||
TvContext.LastInsertedRowId = "last_insert_id()";
|
||||
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
|
||||
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
|
||||
|
||||
// MySQL's LOWER() is already Unicode-aware; assigned explicitly for the same reason as
|
||||
// the host — a provider switch must not inherit SQLite's registration.
|
||||
TvContext.RegisterUnicodeCaseFunctions = static _ => { };
|
||||
}
|
||||
|
||||
services.AddHttpClient();
|
||||
|
||||
@@ -61,93 +61,6 @@ public class GetCollectionItemsHandlerTests
|
||||
page.Page.Select(i => i.Title).ShouldBe(["Zeta", "Alpha", "Beta"]);
|
||||
}
|
||||
|
||||
// Paging semantics, pinned because #616 reported them as two bugs that measurement did not
|
||||
// support. pageNum is 0-BASED (the trap: the MCP catalog documented it as 1-based, so a caller
|
||||
// starting at 1 silently skipped the first page and read a short set as data loss).
|
||||
[Test]
|
||||
public async Task Handle_Should_Treat_PageNum_As_Zero_Based()
|
||||
{
|
||||
await SeedNumberedCollection(150);
|
||||
var handler = new GetCollectionItemsHandler(_db.Factory);
|
||||
|
||||
Either<BaseError, PagedLibraryBrowseItemsResponseModel> first =
|
||||
await handler.Handle(new GetCollectionItems(10, 0, 10), CancellationToken.None);
|
||||
Either<BaseError, PagedLibraryBrowseItemsResponseModel> second =
|
||||
await handler.Handle(new GetCollectionItems(10, 1, 10), CancellationToken.None);
|
||||
|
||||
// Page 0 is the FIRST page, not a skipped one; page 1 is the second.
|
||||
first.RightToSeq().Single().Page.Select(i => i.Title).First().ShouldBe("Item 001");
|
||||
second.RightToSeq().Single().Page.Select(i => i.Title).First().ShouldBe("Item 011");
|
||||
}
|
||||
|
||||
// The second half of #616's claim was that an over-large pageSize caps the returned page but
|
||||
// leaves the OFFSET computed from the requested value, so page 1 at pageSize=500 would land past
|
||||
// item 500. It does not: the size is clamped first and the offset derives from the clamped value.
|
||||
[Test]
|
||||
public async Task Handle_Should_Derive_Offset_From_The_Clamped_PageSize()
|
||||
{
|
||||
await SeedNumberedCollection(150);
|
||||
var handler = new GetCollectionItemsHandler(_db.Factory);
|
||||
|
||||
// pageSize 500 clamps to 100, so page 1 starts at item 101 and runs to the end (50 items).
|
||||
// If the offset honored the requested 500, this page would start past the collection and be
|
||||
// empty — which is exactly what the mutation of this fix produces.
|
||||
Either<BaseError, PagedLibraryBrowseItemsResponseModel> result =
|
||||
await handler.Handle(new GetCollectionItems(10, 1, 500), CancellationToken.None);
|
||||
|
||||
PagedLibraryBrowseItemsResponseModel page = result.RightToSeq().Single();
|
||||
page.TotalCount.ShouldBe(150);
|
||||
page.Page.Count.ShouldBe(50);
|
||||
page.Page.Select(i => i.Title).First().ShouldBe("Item 101");
|
||||
page.Page.Select(i => i.Title).Last().ShouldBe("Item 150");
|
||||
}
|
||||
|
||||
private async Task SeedNumberedCollection(int count)
|
||||
{
|
||||
await using TvContext context = _db.CreateContext();
|
||||
|
||||
var library = new LocalLibrary
|
||||
{
|
||||
Id = 1,
|
||||
Name = "Library",
|
||||
MediaKind = LibraryMediaKind.Movies,
|
||||
Paths = []
|
||||
};
|
||||
var path = new LibraryPath
|
||||
{
|
||||
Id = 1,
|
||||
Path = "/media",
|
||||
Library = library,
|
||||
LibraryFolders = [],
|
||||
MediaItems = []
|
||||
};
|
||||
library.Paths.Add(path);
|
||||
|
||||
var collection = new Collection
|
||||
{
|
||||
Id = 10,
|
||||
Name = "Manual",
|
||||
UseCustomPlaybackOrder = false,
|
||||
MediaItems = [],
|
||||
CollectionItems = [],
|
||||
MultiCollections = [],
|
||||
MultiCollectionItems = []
|
||||
};
|
||||
|
||||
var movies = new List<Movie>();
|
||||
for (var i = 1; i <= count; i++)
|
||||
{
|
||||
// Zero-padded so the handler's title ordering matches numeric order.
|
||||
movies.Add(MakeMovie(1000 + i, path, $"Item {i:D3}"));
|
||||
collection.CollectionItems.Add(new CollectionItem { MediaItemId = 1000 + i });
|
||||
}
|
||||
|
||||
context.LocalLibraries.Add(library);
|
||||
context.Movies.AddRange(movies);
|
||||
context.Collections.Add(collection);
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
private async Task SeedCollectionGraph(bool useCustomPlaybackOrder)
|
||||
{
|
||||
await using TvContext context = _db.CreateContext();
|
||||
|
||||
@@ -1,89 +0,0 @@
|
||||
using ErsatzTV.Application.MediaCollections;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Tests.Support;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
namespace ErsatzTV.Tests.Application.MediaCollections;
|
||||
|
||||
/// <summary>
|
||||
/// The second consumer of the shared <c>ProjectMediaItemToViewModel</c> switch (issue #671).
|
||||
/// <c>GetPlaylistItemsHandler</c> had no handler-level test — the controller tests stub the mediator
|
||||
/// and never execute the query — so the only symptom of a missing include here was a silent "???"
|
||||
/// name that nothing in the suite could see. Widening the shared switch with a RemoteStream arm
|
||||
/// obliged this handler to gain a matching include; proving that by inspection would have repeated
|
||||
/// the very method that produced #671, so it gets the same full matrix the rerun handlers get.
|
||||
/// </summary>
|
||||
[TestFixture]
|
||||
public class GetPlaylistItemsHandlerTests : MediaCollectionHandlerTestBase
|
||||
{
|
||||
private static IEnumerable<CollectionType> SupportedSelectionTypes => SelectionSeedData.SupportedSelectionTypes;
|
||||
|
||||
[TestCaseSource(nameof(SupportedSelectionTypes))]
|
||||
public async Task GetPlaylistItems_Should_Resolve_The_Selection(CollectionType collectionType)
|
||||
{
|
||||
await SeedSelection(collectionType);
|
||||
await SeedPlaylistItem(collectionType);
|
||||
|
||||
var handler = new GetPlaylistItemsHandler(Db.Factory);
|
||||
|
||||
List<PlaylistItemViewModel> items =
|
||||
await handler.Handle(new GetPlaylistItems(1), CancellationToken.None);
|
||||
|
||||
items.Count.ShouldBe(1);
|
||||
|
||||
PlaylistItemViewModel item = items[0];
|
||||
|
||||
int? selectedId = item.Collection?.Id
|
||||
?? item.MultiCollection?.Id
|
||||
?? item.SmartCollection?.Id
|
||||
?? item.MediaItem?.MediaItemId;
|
||||
|
||||
string selectedName = item.Collection?.Name
|
||||
?? item.MultiCollection?.Name
|
||||
?? item.SmartCollection?.Name
|
||||
?? item.MediaItem?.Name;
|
||||
|
||||
selectedId.ShouldBe(SelectionSeedData.SelectedId, $"{collectionType} lost its selected id");
|
||||
selectedName.ShouldBe(
|
||||
SelectionSeedData.ExpectedName(collectionType),
|
||||
$"{collectionType} projected the wrong name");
|
||||
}
|
||||
|
||||
private async Task SeedSelection(CollectionType collectionType)
|
||||
{
|
||||
await using TvContext context = Db.CreateContext();
|
||||
await SelectionSeedData.SeedSelection(context, collectionType);
|
||||
}
|
||||
|
||||
private async Task SeedPlaylistItem(CollectionType collectionType)
|
||||
{
|
||||
await using TvContext context = Db.CreateContext();
|
||||
|
||||
var item = new PlaylistItem
|
||||
{
|
||||
Id = 1,
|
||||
Index = 0,
|
||||
PlaylistId = 1,
|
||||
CollectionType = collectionType,
|
||||
PlaybackOrder = PlaybackOrder.Chronological
|
||||
};
|
||||
|
||||
SelectionSeedData.ApplySelection(
|
||||
collectionType,
|
||||
v => item.CollectionId = v,
|
||||
v => item.MultiCollectionId = v,
|
||||
v => item.SmartCollectionId = v,
|
||||
v => item.MediaItemId = v);
|
||||
|
||||
context.Playlists.Add(new Playlist
|
||||
{
|
||||
Id = 1,
|
||||
Name = "Playlist",
|
||||
Items = [item]
|
||||
});
|
||||
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
}
|
||||
@@ -1,166 +0,0 @@
|
||||
using ErsatzTV.Application.MediaCollections;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Tests.Support;
|
||||
using LanguageExt;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
namespace ErsatzTV.Tests.Application.MediaCollections;
|
||||
|
||||
/// <summary>
|
||||
/// Read-path coverage for the two rerun-collection query handlers (issue #671). The defect was
|
||||
/// precisely that nobody enumerated the selection types: the list handler eager-loaded nothing, and
|
||||
/// the by-id handler loaded metadata for only four of the ten media types. So the matrix is derived
|
||||
/// from the production predicate (see <see cref="SelectionSeedData" />) rather than hand-listed.
|
||||
/// </summary>
|
||||
[TestFixture]
|
||||
public class RerunCollectionQueryHandlerTests : MediaCollectionHandlerTestBase
|
||||
{
|
||||
private static IEnumerable<CollectionType> SupportedSelectionTypes => SelectionSeedData.SupportedSelectionTypes;
|
||||
|
||||
/// <summary>
|
||||
/// Completeness guard. Without it, a change that narrowed <c>IsSupportedSelectionType</c> would
|
||||
/// shrink the matrix silently and every remaining case would still pass — the "filters on the
|
||||
/// property it asserts" failure mode. Set equality, so it fails on widening too.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void Supported_Selection_Types_Should_Be_The_Full_Documented_Set()
|
||||
{
|
||||
SupportedSelectionTypes.ShouldBe(
|
||||
[
|
||||
CollectionType.Collection,
|
||||
CollectionType.TelevisionShow,
|
||||
CollectionType.TelevisionSeason,
|
||||
CollectionType.Artist,
|
||||
CollectionType.MultiCollection,
|
||||
CollectionType.SmartCollection,
|
||||
CollectionType.Movie,
|
||||
CollectionType.Episode,
|
||||
CollectionType.MusicVideo,
|
||||
CollectionType.OtherVideo,
|
||||
CollectionType.Song,
|
||||
CollectionType.Image,
|
||||
CollectionType.RemoteStream
|
||||
],
|
||||
ignoreOrder: true);
|
||||
}
|
||||
|
||||
[TestCaseSource(nameof(SupportedSelectionTypes))]
|
||||
public async Task GetById_Should_Resolve_The_Selection(CollectionType collectionType)
|
||||
{
|
||||
await SeedSelection(collectionType);
|
||||
await SeedRerunCollection(1, collectionType);
|
||||
|
||||
var handler = new GetRerunCollectionByIdHandler(Db.Factory);
|
||||
|
||||
Option<RerunCollectionViewModel> result =
|
||||
await handler.Handle(new GetRerunCollectionById(1), CancellationToken.None);
|
||||
|
||||
RerunCollectionViewModel vm = result.IfNone(() => throw new AssertionException("Expected a result"));
|
||||
AssertSelectionResolved(vm, collectionType);
|
||||
}
|
||||
|
||||
[TestCaseSource(nameof(SupportedSelectionTypes))]
|
||||
public async Task GetPaged_Should_Resolve_The_Selection(CollectionType collectionType)
|
||||
{
|
||||
await SeedSelection(collectionType);
|
||||
await SeedRerunCollection(1, collectionType);
|
||||
|
||||
var handler = new GetPagedRerunCollectionsHandler(Db.Factory);
|
||||
|
||||
PagedRerunCollectionsViewModel result = await handler.Handle(
|
||||
new GetPagedRerunCollections(string.Empty, 0, 10),
|
||||
CancellationToken.None);
|
||||
|
||||
result.Page.Count.ShouldBe(1);
|
||||
AssertSelectionResolved(result.Page[0], collectionType);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// <c>SongMetadata.Artists</c> is a NULLABLE primitive collection, and a song whose tags failed to
|
||||
/// read is persisted with it never assigned. Before #671 the rerun list did not load SongMetadata
|
||||
/// at all, so this was unreachable there; eager-loading it made a latent `string.Join` throw into a
|
||||
/// live 500 that would take down the whole page.
|
||||
/// </summary>
|
||||
[TestCase(null, "Selected song", TestName = "GetById_Song_With_Null_Artists_Should_Not_Throw")]
|
||||
[TestCase(new string[] { }, "Selected song", TestName = "GetById_Song_With_No_Artists_Should_Not_Prefix")]
|
||||
public async Task GetById_Should_Tolerate_Song_Artists(string[] artists, string expectedName)
|
||||
{
|
||||
await using (TvContext context = Db.CreateContext())
|
||||
{
|
||||
context.Songs.Add(new Song
|
||||
{
|
||||
Id = SelectionSeedData.SelectedId,
|
||||
SongMetadata = [new SongMetadata { Title = "Selected song", Artists = artists?.ToList() }]
|
||||
});
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
await SeedRerunCollection(1, CollectionType.Song);
|
||||
|
||||
var handler = new GetRerunCollectionByIdHandler(Db.Factory);
|
||||
|
||||
Option<RerunCollectionViewModel> result =
|
||||
await handler.Handle(new GetRerunCollectionById(1), CancellationToken.None);
|
||||
|
||||
RerunCollectionViewModel vm = result.IfNone(() => throw new AssertionException("Expected a result"));
|
||||
vm.MediaItem.ShouldNotBeNull();
|
||||
vm.MediaItem.MediaItemId.ShouldBe(SelectionSeedData.SelectedId);
|
||||
vm.MediaItem.Name.ShouldBe(expectedName);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Mirrors <c>RerunCollectionController.ProjectToResponseModel</c>, which flattens the tagged
|
||||
/// union to the single <c>selectedId</c> / <c>selectedName</c> pair the SPA consumes. The id is
|
||||
/// the load-bearing half: the editor round-trips it, so a null there silently clears the user's
|
||||
/// stored selection.
|
||||
/// </summary>
|
||||
private static void AssertSelectionResolved(RerunCollectionViewModel vm, CollectionType collectionType)
|
||||
{
|
||||
int? selectedId = vm.Collection?.Id
|
||||
?? vm.MultiCollection?.Id
|
||||
?? vm.SmartCollection?.Id
|
||||
?? vm.MediaItem?.MediaItemId;
|
||||
|
||||
string selectedName = vm.Collection?.Name
|
||||
?? vm.MultiCollection?.Name
|
||||
?? vm.SmartCollection?.Name
|
||||
?? vm.MediaItem?.Name;
|
||||
|
||||
selectedId.ShouldBe(SelectionSeedData.SelectedId, $"{collectionType} lost its selected id");
|
||||
selectedName.ShouldBe(
|
||||
SelectionSeedData.ExpectedName(collectionType),
|
||||
$"{collectionType} projected the wrong name");
|
||||
}
|
||||
|
||||
private async Task SeedSelection(CollectionType collectionType)
|
||||
{
|
||||
await using TvContext context = Db.CreateContext();
|
||||
await SelectionSeedData.SeedSelection(context, collectionType);
|
||||
}
|
||||
|
||||
private async Task SeedRerunCollection(int id, CollectionType collectionType)
|
||||
{
|
||||
await using TvContext context = Db.CreateContext();
|
||||
|
||||
var rerunCollection = new RerunCollection
|
||||
{
|
||||
Id = id,
|
||||
Name = "Rerun",
|
||||
CollectionType = collectionType,
|
||||
FirstRunPlaybackOrder = PlaybackOrder.Chronological,
|
||||
RerunPlaybackOrder = PlaybackOrder.Chronological
|
||||
};
|
||||
|
||||
SelectionSeedData.ApplySelection(
|
||||
collectionType,
|
||||
v => rerunCollection.CollectionId = v,
|
||||
v => rerunCollection.MultiCollectionId = v,
|
||||
v => rerunCollection.SmartCollectionId = v,
|
||||
v => rerunCollection.MediaItemId = v);
|
||||
|
||||
context.RerunCollections.Add(rerunCollection);
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
}
|
||||
@@ -1,96 +0,0 @@
|
||||
using ErsatzTV.Core.Domain;
|
||||
using LanguageExt;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
using Mapper = ErsatzTV.Application.Playouts.Mapper;
|
||||
|
||||
namespace ErsatzTV.Tests.Application.Playouts;
|
||||
|
||||
/// <summary>
|
||||
/// <c>SongMetadata.Artists</c> is a nullable EF primitive collection that
|
||||
/// <c>FallbackMetadataProvider</c> leaves unassigned for a song whose tags failed to read, and
|
||||
/// <c>string.Join</c> throws <see cref="ArgumentNullException" /> on a null sequence. Because
|
||||
/// <c>SongMetadata</c> IS eager-loaded on the playout paths, this was a LIVE 500 rather than a
|
||||
/// latent one — and <c>GetDisplayTitle</c> feeds the playout guide, troubleshooting, media-item
|
||||
/// info and channel states alike (issue #671).
|
||||
/// </summary>
|
||||
[TestFixture]
|
||||
public class PlayoutMapperDisplayTitleTests
|
||||
{
|
||||
[Test]
|
||||
public void GetDisplayTitle_Should_Not_Throw_When_Song_Artists_Is_Null()
|
||||
{
|
||||
var song = new Song
|
||||
{
|
||||
Id = 1,
|
||||
SongMetadata = [new SongMetadata { Title = "Untagged", Artists = null }]
|
||||
};
|
||||
|
||||
string title = Mapper.GetDisplayTitle(song, Option<string>.None);
|
||||
|
||||
title.ShouldBe("Untagged");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void GetDisplayTitle_Should_Not_Prefix_When_Song_Has_No_Artists()
|
||||
{
|
||||
var song = new Song
|
||||
{
|
||||
Id = 1,
|
||||
SongMetadata = [new SongMetadata { Title = "Untagged", Artists = [] }]
|
||||
};
|
||||
|
||||
string title = Mapper.GetDisplayTitle(song, Option<string>.None);
|
||||
|
||||
title.ShouldBe("Untagged");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void GetDisplayTitle_Should_Prefix_The_Artists_When_Present()
|
||||
{
|
||||
var song = new Song
|
||||
{
|
||||
Id = 1,
|
||||
SongMetadata = [new SongMetadata { Title = "Tagged", Artists = ["A", "B"] }]
|
||||
};
|
||||
|
||||
string title = Mapper.GetDisplayTitle(song, Option<string>.None);
|
||||
|
||||
title.ShouldBe("A, B - Tagged");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The chapter branch interpolated the `case Song s` ENTITY rather than the composed title, and
|
||||
/// <see cref="Song" /> has no <c>ToString()</c> override — so a chaptered song rendered as the
|
||||
/// literal "ErsatzTV.Core.Domain.Song (Chapter 1)". Pre-existing; the sibling MusicVideo and
|
||||
/// OtherVideo arms are correct only because they name their lambda parameter `s` too.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void GetDisplayTitle_Should_Compose_The_Title_Not_The_Entity_When_Chaptered()
|
||||
{
|
||||
var song = new Song
|
||||
{
|
||||
Id = 1,
|
||||
SongMetadata = [new SongMetadata { Title = "Tagged", Artists = ["A"] }]
|
||||
};
|
||||
|
||||
string title = Mapper.GetDisplayTitle(song, Option<string>.Some("Chapter 1"));
|
||||
|
||||
title.ShouldBe("A - Tagged (Chapter 1)");
|
||||
title.ShouldNotContain("ErsatzTV.Core.Domain");
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void GetDisplayTitle_Should_Not_Throw_When_Chaptered_Song_Has_Null_Artists()
|
||||
{
|
||||
var song = new Song
|
||||
{
|
||||
Id = 1,
|
||||
SongMetadata = [new SongMetadata { Title = "Untagged", Artists = null }]
|
||||
};
|
||||
|
||||
string title = Mapper.GetDisplayTitle(song, Option<string>.Some("Chapter 2"));
|
||||
|
||||
title.ShouldBe("Untagged (Chapter 2)");
|
||||
}
|
||||
}
|
||||
@@ -1,11 +1,9 @@
|
||||
using System.Globalization;
|
||||
using ErsatzTV.Application.Search.Queries;
|
||||
using ErsatzTV.Core.Api.Search;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Tests.Support;
|
||||
using LanguageExt;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
@@ -178,461 +176,6 @@ public class GetSearchFieldValuesHandlerTests
|
||||
networkResult.IfSome(r => r.Values.ShouldBe(new List<string> { "HBO" }));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Artist_Merges_Entity_Artists_Music_Video_Credits_And_Song_Credits()
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.ArtistMetadata.Add(Artist("Alpha Entity"));
|
||||
|
||||
// negative control: an entity artist that must NOT match the "al" prefix
|
||||
context.ArtistMetadata.Add(Artist("Zeta Entity"));
|
||||
|
||||
context.MusicVideoMetadata.AddRange(
|
||||
MusicVideo("MV One", "Alpha Credit", "Alpha Shared"),
|
||||
// "Alpha Shared" appears in two rows, so DISTINCT has something to collapse
|
||||
MusicVideo("MV Two", "Alpha Shared"),
|
||||
MusicVideo("MV Three", "Zeta Credit"));
|
||||
|
||||
context.SongMetadata.AddRange(
|
||||
Song("Song One", ["Alpha Song", "Zeta Song"]),
|
||||
Song("Song Two", ["Alpha Song"]),
|
||||
Song("Song Three", ["Zeta Only"]));
|
||||
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("artist", "al", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(
|
||||
new List<string> { "Alpha Credit", "Alpha Entity", "Alpha Shared", "Alpha Song" }));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Artist_Returns_Every_Source_For_Empty_Query()
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.ArtistMetadata.Add(Artist("Entity"));
|
||||
context.MusicVideoMetadata.Add(MusicVideo("MV", "Credit"));
|
||||
context.SongMetadata.Add(Song("Song", ["SongArtist"]));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("artist", string.Empty, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Credit", "Entity", "SongArtist" }));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Album_Artist_Returns_Song_Album_Artists_Instead_Of_NotFound()
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.AddRange(
|
||||
Song("One", ["Performer"], ["Alpha Album Artist", "Beta Album Artist"]),
|
||||
// repeated across rows so DISTINCT is exercised
|
||||
Song("Two", ["Performer"], ["Alpha Album Artist"]),
|
||||
// negative control: a row whose album artists are absent entirely
|
||||
Song("Three", ["Performer"], null));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("album_artist", string.Empty, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Alpha Album Artist", "Beta Album Artist" }));
|
||||
|
||||
// the performers on the same rows must not leak into album_artist
|
||||
result.IfSome(r => r.Values.ShouldNotContain("Performer"));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task List_Valued_Fields_Match_Whole_Elements_Not_Substrings_And_Ignore_Neighbours()
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.AddRange(
|
||||
// "Neighbour" arrives on the same row as "Radiohead" -- rows are read whole -- and must be
|
||||
// dropped by the in-memory exact prefix filter.
|
||||
Song("One", ["Radiohead", "Neighbour"]),
|
||||
// "The Radio Dept." contains "radio" but does not start with it
|
||||
Song("Two", ["The Radio Dept."]),
|
||||
Song("Three", ["Radio Birdman"]));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("artist", "radio", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Radio Birdman", "Radiohead" }));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task List_Valued_Fields_Match_Literally_Including_Json_Escaped_And_Wildcard_Characters()
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.AddRange(
|
||||
// non-ASCII: stored on disk JSON-escaped as \u00E9, and must survive the round trip
|
||||
Song("One", ["Beyoncé"]),
|
||||
// an embedded quote is stored as \u0022
|
||||
Song("Two", ["\"Weird Al\" Yankovic"]),
|
||||
// SQL wildcards must be ordinary characters here, matched literally
|
||||
Song("Three", ["50% Off"]),
|
||||
Song("Four", ["50 Cent"]));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", "beyoncé", 50), CancellationToken.None))
|
||||
.IfSome(r => r.Values.ShouldBe(new List<string> { "Beyoncé" }));
|
||||
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", "\"weird", 50), CancellationToken.None))
|
||||
.IfSome(r => r.Values.ShouldBe(new List<string> { "\"Weird Al\" Yankovic" }));
|
||||
|
||||
// "50%" must not behave as the wildcard "50<anything>" — "50 Cent" must not come back
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", "50%", 50), CancellationToken.None))
|
||||
.IfSome(r => r.Values.ShouldBe(new List<string> { "50% Off" }));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Seeds <paramref name="fillerRows" /> non-matching songs through raw SQL — 20k rows via the change
|
||||
/// tracker is minutes, this is milliseconds.
|
||||
/// </summary>
|
||||
private static Task SeedFiller(TvContext context, int fillerRows) =>
|
||||
context.Database.ExecuteSqlRawAsync(
|
||||
$"""
|
||||
WITH RECURSIVE seq(n) AS (SELECT 1 UNION ALL SELECT n + 1 FROM seq WHERE n < {fillerRows})
|
||||
INSERT INTO SongMetadata (SongId, MetadataKind, Title, Artists, DateAdded, DateUpdated)
|
||||
SELECT 0, 0, 'Filler ' || n, '["zzz-filler"]', '2026-01-01', '2026-01-01' FROM seq
|
||||
""");
|
||||
|
||||
[Test]
|
||||
public async Task List_Valued_Walk_Reads_At_Most_20000_Rows()
|
||||
{
|
||||
// Pinned in both directions so the ceiling itself is nailed down: a match in row 20000 is read, the same
|
||||
// match in row 20001 is not. The query has no RESIDUAL predicate -- only the cursor -- so "rows read" is
|
||||
// what LIMIT returns. That bounds LOGICAL rows, not physical work: the engine may still traverse more
|
||||
// index records than it returns (MySQL purge lag), and row width is unbounded.
|
||||
const string needle = "\u00E9clair-the-needle";
|
||||
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
await SeedFiller(context, 19999);
|
||||
context.SongMetadata.Add(Song("Needle", [needle]));
|
||||
await context.SaveChangesAsync();
|
||||
|
||||
(await context.SongMetadata.CountAsync()).ShouldBe(20000);
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", "\u00E9", 50), CancellationToken.None))
|
||||
.IfSome(r => r.Values.ShouldBe(new List<string> { needle }, "row 20000 is inside the ceiling"));
|
||||
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
SongMetadata existing = await context.SongMetadata.SingleAsync(m => m.Title == "Needle");
|
||||
context.SongMetadata.Remove(existing);
|
||||
await SeedFiller(context, 1);
|
||||
await context.SaveChangesAsync();
|
||||
context.SongMetadata.Add(Song("Needle", [needle]));
|
||||
await context.SaveChangesAsync();
|
||||
|
||||
(await context.SongMetadata.CountAsync()).ShouldBe(20001);
|
||||
}
|
||||
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", "\u00E9", 50), CancellationToken.None))
|
||||
.IfSome(
|
||||
r => r.Values.ShouldBeEmpty(
|
||||
"row 20001 is past the ceiling; this false negative is the documented bounded-best-effort "
|
||||
+ "contract, deliberately pinned rather than papered over"));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task List_Valued_Walk_Reads_Live_Rows_Regardless_Of_Id_Density()
|
||||
{
|
||||
// THE round-4 killer. That revision bounded the Id KEYSPACE, and keyspace is not rows: with 20,000
|
||||
// historical rows deleted and one live song at Id 20001, the walk spent its whole allowance on empty
|
||||
// ranges and returned [] for a table containing exactly one row. Capacity degraded linearly with
|
||||
// deletion ratio, and no ratio was safe -- one placed gap hid the next match.
|
||||
//
|
||||
// Paging by row position rather than Id value makes density irrelevant: LIMIT @Batch returns @Batch
|
||||
// ROWS, wherever they sit in the keyspace.
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
await SeedFiller(context, 20000);
|
||||
await context.Database.ExecuteSqlRawAsync("DELETE FROM SongMetadata");
|
||||
context.SongMetadata.Add(Song("Survivor", ["Queen"]));
|
||||
await context.SaveChangesAsync();
|
||||
|
||||
// one live row, sitting past the old keyspace allowance
|
||||
(await context.SongMetadata.CountAsync()).ShouldBe(1);
|
||||
(await context.SongMetadata.Select(m => m.Id).SingleAsync()).ShouldBeGreaterThan(20000);
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", "que", 50), CancellationToken.None))
|
||||
.IfSome(
|
||||
r => r.Values.ShouldBe(
|
||||
new List<string> { "Queen" },
|
||||
"a one-row table must be fully readable no matter where its Id sits"));
|
||||
|
||||
// and a leading gap must not hide a later match either
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.Add(Song("Second", ["Queens of the Stone Age"]));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", "que", 50), CancellationToken.None))
|
||||
.IfSome(r => r.Values.ShouldBe(new List<string> { "Queen", "Queens of the Stone Age" }));
|
||||
}
|
||||
|
||||
[Test]
|
||||
[TestCase("é", "\u00C9dith Piaf")]
|
||||
[TestCase("\u00C9", "\u00C9dith Piaf")]
|
||||
[TestCase("\u00E9dith", "\u00C9dith Piaf")]
|
||||
[TestCase("bj", "Bj\u00F6rk")]
|
||||
[TestCase("bj\u00F6", "Bj\u00F6rk")]
|
||||
[TestCase("BJ\u00D6RK", "Bj\u00F6rk")]
|
||||
[TestCase("beyonc\u00E9", "Beyonc\u00E9")]
|
||||
[TestCase("sigur r", "Sigur R\u00F3s")]
|
||||
[TestCase("\u00D6", "\u00D6zdemir")]
|
||||
public async Task Matches_NonAscii_Values_In_Any_Casing(string query, string stored)
|
||||
{
|
||||
// Accented artists are the common case in a music library, so non-ASCII matching is pinned end to
|
||||
// end, in both casings of the query.
|
||||
//
|
||||
// Historical note, because it is why this suite exists: revision 1b78dc9e narrowed rows in SQL
|
||||
// with a LIKE built by JSON-encoding the query, which cannot work -- non-ASCII is stored escaped
|
||||
// (\u00C9) and SQL LOWER() folds the escape TEXT, not the codepoint it denotes. THREE of these nine
|
||||
// cases fail against that revision (the ones where query and stored casing differ, so \u00e9 and
|
||||
// \u00C9 diverge); the other six pass it, because when the casings agree the escape texts line up.
|
||||
// The SQL now has no residual predicate at all -- matching happens in memory, where a string is just
|
||||
// a string -- so these cases pin current behaviour rather than guard that revision.
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.AddRange(
|
||||
Song("Hit", [stored]),
|
||||
// negative control: a row that must never come back for any of these queries
|
||||
Song("Other", ["Nothing Relevant"]));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("artist", query, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { stored }));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Results_Do_Not_Depend_On_The_Request_Culture()
|
||||
{
|
||||
// UseRequestLocalization honours Accept-Language, so CurrentCulture is caller-controlled. Under tr-TR
|
||||
// the old `q.ToLower()` turned "I" into "\u0131" and the default linguistic StartsWith(string) compounded
|
||||
// it, so the same library answered differently per caller. The contract is ordinal: "I" matches
|
||||
// "Istanbul" and does NOT match "\u0131pek", in every culture.
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.Add(Song("One", ["Istanbul Orkestrasi", "\u0131pek"]));
|
||||
context.ArtistMetadata.Add(Artist("Idil Biret"));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
var expected = new List<string> { "Idil Biret", "Istanbul Orkestrasi" };
|
||||
|
||||
CultureInfo original = CultureInfo.CurrentCulture;
|
||||
try
|
||||
{
|
||||
foreach (string culture in new[] { "en-US", "tr-TR", "az-AZ", "lt-LT" })
|
||||
{
|
||||
CultureInfo.CurrentCulture = new CultureInfo(culture);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("artist", "I", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(expected, $"culture {culture} changed the result"));
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
CultureInfo.CurrentCulture = original;
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Ordering_Is_Ordinal_And_Culture_Independent()
|
||||
{
|
||||
// The merge sorts ordinally rather than by culture, so the response order does not depend on the caller
|
||||
// either. Ordinal puts all ASCII uppercase before ASCII lowercase, and non-ASCII last.
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.Add(Song("One", ["Zulu", "apple", "\u00C9clair", "Apple"]));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
var expected = new List<string> { "Apple", "Zulu", "apple", "\u00C9clair" };
|
||||
|
||||
CultureInfo original = CultureInfo.CurrentCulture;
|
||||
try
|
||||
{
|
||||
foreach (string culture in new[] { "en-US", "sv-SE" })
|
||||
{
|
||||
CultureInfo.CurrentCulture = new CultureInfo(culture);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("artist", string.Empty, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IfSome(r => r.Values.ShouldBe(expected, $"culture {culture} changed the order"));
|
||||
}
|
||||
}
|
||||
finally
|
||||
{
|
||||
CultureInfo.CurrentCulture = original;
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Ordering_Is_Best_Effort_When_A_Source_Truncates()
|
||||
{
|
||||
// Documents the acknowledged imprecision rather than claiming exactness the code does not have. The EF
|
||||
// source truncates by the DATABASE collation, which is NOT the ordinal ordering the merge then applies —
|
||||
// so a value the database ranked outside its first `limit` never reaches the merge, even if the merge
|
||||
// would have ranked it first.
|
||||
//
|
||||
// "Zulu" vs "apple" is the pair that actually diverges: ordinal puts every ASCII uppercase letter before
|
||||
// every lowercase one, so ordinal ranks "Zulu" first, while a case-insensitive database ordering ranks
|
||||
// "apple" first. (An earlier version used "Zulu"/"Éclair", where BOTH orderings pick "Zulu" — it could
|
||||
// not have told the two apart, and the divergence it claimed to show did not exist.)
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.ArtistMetadata.Add(Artist("Zulu"));
|
||||
context.ArtistMetadata.Add(Artist("apple"));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
// with room for both, the ordinal merge ranks "Zulu" first
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", string.Empty, 50), CancellationToken.None))
|
||||
.IfSome(r => r.Values.ShouldBe(new List<string> { "Zulu", "apple" }));
|
||||
|
||||
// with limit=1 the database picks the survivor by ITS ordering, and the merge only ever sees that one
|
||||
(await handler.Handle(new GetSearchFieldValues("artist", string.Empty, 1), CancellationToken.None))
|
||||
.IfSome(r => r.Values.ShouldBe(new List<string> { "apple" }));
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task A_Match_Behind_Many_NonMatching_Rows_Is_Still_Found()
|
||||
{
|
||||
// Fails a883e5f0, which capped rows at a fixed 1000 AFTER a deliberately over-matching SQL pre-filter:
|
||||
// the 1001st row -- the only exact match -- was discarded before the in-memory filter ever saw it and
|
||||
// the endpoint returned []. The pre-filter is gone, and the property it broke now holds for any match
|
||||
// within the read ceiling: preceding non-matching rows do not hide it. Past the ceiling it is still
|
||||
// lost by design -- see List_Valued_Walk_Reads_At_Most_20000_Rows, which pins that boundary.
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
for (var i = 0; i < 1000; i++)
|
||||
{
|
||||
context.SongMetadata.Add(Song($"Filler {i}", ["zzz-filler"], ["zzz-filler-album"]));
|
||||
}
|
||||
|
||||
context.SongMetadata.Add(Song("Needle", ["\u00E9clair"], ["\u00E9clair"]));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> albumArtist = await handler.Handle(
|
||||
new GetSearchFieldValues("album_artist", "\u00E9", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
albumArtist.IsSome.ShouldBeTrue();
|
||||
albumArtist.IfSome(r => r.Values.ShouldBe(new List<string> { "\u00E9clair" }));
|
||||
|
||||
// same starvation shape on the merged `artist` field
|
||||
Option<SearchFieldValuesResponseModel> artist = await handler.Handle(
|
||||
new GetSearchFieldValues("artist", "\u00E9", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
artist.IsSome.ShouldBeTrue();
|
||||
artist.IfSome(r => r.Values.ShouldBe(new List<string> { "\u00E9clair" }));
|
||||
|
||||
// ... and for a prefix beginning with a character that JSON escapes on disk. That used to collapse the
|
||||
// SQL pattern to the bare anchor; there is no prefix predicate at all now, so it is simply an ordinary
|
||||
// prefix -- kept because it is the input shape that broke the old scheme.
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.SongMetadata.Add(Song("Ampersand", ["&Me"]));
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
Option<SearchFieldValuesResponseModel> escapedPrefix = await handler.Handle(
|
||||
new GetSearchFieldValues("artist", "&M", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
escapedPrefix.IsSome.ShouldBeTrue();
|
||||
escapedPrefix.IfSome(r => r.Values.ShouldBe(new List<string> { "&Me" }));
|
||||
}
|
||||
|
||||
private static ArtistMetadata Artist(string title) => new()
|
||||
{
|
||||
MetadataKind = MetadataKind.External,
|
||||
DateAdded = DateTime.UtcNow,
|
||||
DateUpdated = DateTime.UtcNow,
|
||||
Title = title
|
||||
};
|
||||
|
||||
private static MusicVideoMetadata MusicVideo(string title, params string[] artists) => new()
|
||||
{
|
||||
MetadataKind = MetadataKind.External,
|
||||
DateAdded = DateTime.UtcNow,
|
||||
DateUpdated = DateTime.UtcNow,
|
||||
Title = title,
|
||||
Artists = artists.Map(a => new MusicVideoArtist { Name = a }).ToList()
|
||||
};
|
||||
|
||||
private static SongMetadata Song(string title, IList<string> artists, IList<string> albumArtists = null) => new()
|
||||
{
|
||||
MetadataKind = MetadataKind.External,
|
||||
DateAdded = DateTime.UtcNow,
|
||||
DateUpdated = DateTime.UtcNow,
|
||||
Title = title,
|
||||
Artists = artists,
|
||||
AlbumArtists = albumArtists
|
||||
};
|
||||
|
||||
[Test]
|
||||
public async Task Dedupes_Repeated_Values()
|
||||
{
|
||||
@@ -653,226 +196,4 @@ public class GetSearchFieldValuesHandlerTests
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Action" }));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. The EF-sourced fields prefix-match through SQL <c>LOWER()</c>, which on SQLite folds
|
||||
/// ASCII only: <c>lower('Édith')</c> returns <c>'Édith'</c> unchanged, so a stored value whose
|
||||
/// prefix carries an uppercase non-ASCII character is unreachable from any query long enough to reach it.
|
||||
/// The stored-LOWERCASE case already worked (the handler lowercases the query before it reaches SQL, so
|
||||
/// both casings of the query fold to the same pattern) and is pinned alongside it, because the fix must
|
||||
/// SUPPLEMENT that path rather than replace it.
|
||||
/// </summary>
|
||||
[TestCase("genre", "é")]
|
||||
[TestCase("genre", "É")]
|
||||
public async Task Ef_Sourced_Stored_Uppercase_Accent_Is_Reachable(string field, string query)
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.Set<Genre>().AddRange(
|
||||
new Genre { Name = "Édith" },
|
||||
new Genre { Name = "Zulu" });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues(field, query, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Édith" }));
|
||||
}
|
||||
|
||||
/// <inheritdoc cref="Ef_Sourced_Stored_Uppercase_Accent_Is_Reachable" />
|
||||
[TestCase("genre", "é")]
|
||||
[TestCase("genre", "É")]
|
||||
public async Task Ef_Sourced_Stored_Lowercase_Accent_Stays_Reachable(string field, string query)
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.Set<Genre>().AddRange(
|
||||
new Genre { Name = "édith" },
|
||||
new Genre { Name = "Zulu" });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues(field, query, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { "édith" }));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. The Unicode fold added for the non-ASCII branch may OVER-match — the in-memory
|
||||
/// <see cref="StringComparison.OrdinalIgnoreCase" /> filter runs afterwards and drops the extras —
|
||||
/// but it must never UNDER-match. Each case pins the endpoint's answer against what that filter
|
||||
/// alone would say, so a fold that starts dropping rows fails here. It does NOT catch removal of the
|
||||
/// in-memory filter — every case here is either a positive that SQL alone returns, or an ASCII-query
|
||||
/// negative that SQL alone rejects. That direction is
|
||||
/// <see cref="Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter" />'s job.
|
||||
/// <para>
|
||||
/// The negative cases here have ASCII queries, so they exercise the FAST PATH (the fold is
|
||||
/// skipped entirely) and pin that it is exact: <c>"ſweet".StartsWith("S", OrdinalIgnoreCase)</c>
|
||||
/// is false even though <c>char.ToUpperInvariant('ſ')</c> IS <c>'S'</c>. The over-match the fold
|
||||
/// itself produces is a different path and is covered by
|
||||
/// <see cref="Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter" />.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
[TestCase("Édith", "é", true, TestName = "Fold_UppercaseAccent_LowercaseQuery")]
|
||||
[TestCase("Édith", "É", true, TestName = "Fold_UppercaseAccent_UppercaseQuery")]
|
||||
[TestCase("Özdemir", "ö", true, TestName = "Fold_Umlaut")]
|
||||
[TestCase("Sigur Rós", "sigur", true, TestName = "Fold_AsciiPrefix_NonAsciiLater")]
|
||||
[TestCase("Straße", "stra", true, TestName = "Fold_Eszett_AsciiQuery")]
|
||||
// explicit escapes: these three are visually indistinguishable from their ASCII lookalikes in a diff,
|
||||
// and an ASCII 'K' here would silently turn the KELVIN SIGN case into a trivially-true one
|
||||
[TestCase("\u017Fweet", "S", false, TestName = "Fold_LongS_IsNotOrdinalEqualToS")]
|
||||
[TestCase("\u212Aelvin", "k", false, TestName = "Fold_KelvinSign_IsNotOrdinalEqualToK")]
|
||||
[TestCase("\u0130stanbul", "i", false, TestName = "Fold_DottedCapitalI_IsNotOrdinalEqualToI")]
|
||||
public async Task Unicode_Fold_Agrees_With_The_Ordinal_Filter(string stored, string query, bool expected)
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.Set<Genre>().Add(new Genre { Name = stored });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
// the oracle: what the endpoint's own final filter says, computed independently of the database
|
||||
stored.StartsWith(query, StringComparison.OrdinalIgnoreCase).ShouldBe(expected);
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("genre", query, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(expected ? new List<string> { stored } : []));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. Drives a row THROUGH the fold that the ordinal filter must then discard — the
|
||||
/// harmless over-match direction the whole design rests on, which the ASCII-query negative cases
|
||||
/// above cannot reach. q="ſ" is non-ASCII so the fold runs; <c>ToUpperInvariant('ſ')</c> is 'S', so
|
||||
/// the SQL pattern is <c>S%</c> and SQLite genuinely returns "Sword" — and the response must still
|
||||
/// be empty, because <c>"Sword".StartsWith("ſ", OrdinalIgnoreCase)</c> is false.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public async Task Unicode_Fold_Over_Match_Is_Discarded_By_The_Ordinal_Filter()
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.Set<Genre>().Add(new Genre { Name = "Sword" });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
// Premises, asserted because the expectation is an EMPTY list and would otherwise pass for the
|
||||
// wrong reason -- e.g. if the branch stopped running, or a hand-rolled fold stopped mapping ſ to S,
|
||||
// SQL would return nothing and this test would still be green.
|
||||
GetSearchFieldValuesHandler.ContainsNonAscii("\u017F").ShouldBeTrue();
|
||||
char.ToUpperInvariant('\u017F').ShouldBe('S');
|
||||
"Sword".StartsWith("\u017F", StringComparison.OrdinalIgnoreCase).ShouldBeFalse();
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("genre", "\u017F", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBeEmpty());
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. The escaping's load-bearing role is NOT filtering — the in-memory ordinal filter
|
||||
/// already drops an over-match, which is why a plain count assertion stays green even with the
|
||||
/// escaping removed. It is preventing LIMIT CROWDING: an unescaped <c>_</c> also matches the space,
|
||||
/// binary ORDER BY ranks "100 Édith" first, LIMIT 1 returns only that, the filter discards it, and
|
||||
/// the genuine "100_Édith" is never returned at all. This case fails if the escaping is removed.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public async Task Unicode_Fold_Escaping_Prevents_Limit_Crowding()
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.Set<Genre>().AddRange(
|
||||
new Genre { Name = "100 \u00C9dith" },
|
||||
new Genre { Name = "100_\u00C9dith" });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("genre", "100_\u00C9", 1),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { "100_\u00C9dith" }));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. The non-ASCII branch is raw SQL, so it gets none of the LIKE-wildcard escaping EF
|
||||
/// does for <c>StartsWith</c>. An unescaped <c>%</c> or <c>_</c> in the query would match anything.
|
||||
/// </summary>
|
||||
[TestCase("100%É", 1, TestName = "Escapes_Percent")]
|
||||
[TestCase("100_É", 0, TestName = "Escapes_Underscore")]
|
||||
public async Task Unicode_Fold_Escapes_Like_Wildcards(string query, int expectedCount)
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.Set<Genre>().AddRange(
|
||||
new Genre { Name = "100%Édith" },
|
||||
new Genre { Name = "100XÉdith" });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("genre", query, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.Count.ShouldBe(expectedCount));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. The non-ASCII branch duplicates each field's discriminator predicate in raw SQL, so
|
||||
/// it must reproduce EF's NULL semantics: EF compiles <c>ExternalTypeId != NfoCountryTypeId</c> with
|
||||
/// null semantics, which INCLUDES a NULL-typed row. Plain SQL <c><></c> would silently drop it.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public async Task Unicode_Fold_Tag_Discriminator_Matches_Ef_Null_Semantics()
|
||||
{
|
||||
await using (TvContext context = _db.CreateContext())
|
||||
{
|
||||
context.Set<Tag>().AddRange(
|
||||
new Tag { Name = "Édith", ExternalTypeId = null },
|
||||
new Tag { Name = "Éclair", ExternalTypeId = Tag.PlexNetworkTypeId },
|
||||
new Tag { Name = "Ézra", ExternalTypeId = Tag.NfoCountryTypeId });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(_db.Factory);
|
||||
|
||||
Option<SearchFieldValuesResponseModel> tags = await handler.Handle(
|
||||
new GetSearchFieldValues("tag", "é", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
// the NULL-typed row is a tag; the network- and country-typed rows are excluded
|
||||
tags.IsSome.ShouldBeTrue();
|
||||
tags.IfSome(r => r.Values.ShouldBe(new List<string> { "Édith" }));
|
||||
|
||||
Option<SearchFieldValuesResponseModel> networks = await handler.Handle(
|
||||
new GetSearchFieldValues("network", "é", 50),
|
||||
CancellationToken.None);
|
||||
|
||||
networks.IsSome.ShouldBeTrue();
|
||||
networks.IfSome(r => r.Values.ShouldBe(new List<string> { "Éclair" }));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,174 +0,0 @@
|
||||
using ErsatzTV.Application.Search.Queries;
|
||||
using ErsatzTV.Infrastructure;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Infrastructure.Sqlite.Data;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
namespace ErsatzTV.Tests.Application.Search;
|
||||
|
||||
/// <summary>
|
||||
/// Provider-shape guards for the <c>artist</c> / <c>album_artist</c> facet-value sources (#578).
|
||||
/// <para>
|
||||
/// <see cref="GetSearchFieldValuesHandlerTests" /> runs against in-memory SQLite, so it structurally
|
||||
/// cannot see a MySQL translation or collation difference. These tests build the same LINQ against the
|
||||
/// Pomelo MySQL provider and assert the generated SQL — <c>ToQueryString</c> compiles the query without
|
||||
/// touching a server, so no MySQL instance is needed.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
[TestFixture]
|
||||
[NonParallelizable]
|
||||
public class SearchFieldValuesQueryShapeTests
|
||||
{
|
||||
private bool _wasSqlite;
|
||||
|
||||
[SetUp]
|
||||
public void SetUp() => _wasSqlite = TvContext.IsSqlite;
|
||||
|
||||
[TearDown]
|
||||
public void TearDown() => TvContext.IsSqlite = _wasSqlite;
|
||||
|
||||
[Test]
|
||||
public void Artist_Entity_Union_Translates_On_Both_Providers_With_Lower_And_A_Row_Limit()
|
||||
{
|
||||
foreach ((string provider, Func<TvContext> create) in Providers())
|
||||
{
|
||||
using TvContext context = create();
|
||||
|
||||
// calls the handler's own source builder (internal, via InternalsVisibleTo) rather than rebuilding
|
||||
// the LINQ here — a copy would keep passing after the handler's query changed underneath it
|
||||
string sql = GetSearchFieldValuesHandler.GetSource(context, "artist")
|
||||
.Where(v => v != null && v.ToLower().StartsWith("a"))
|
||||
.Distinct()
|
||||
.OrderBy(v => v)
|
||||
.Take(50)
|
||||
.ToQueryString();
|
||||
|
||||
// case-insensitivity comes from LOWER() on the column, not from the provider's LIKE collation
|
||||
sql.ShouldContain("LOWER(", Case.Insensitive, $"{provider}: {sql}");
|
||||
sql.ShouldContain("LIKE", Case.Insensitive, $"{provider}: {sql}");
|
||||
sql.ShouldContain("MusicVideoArtist", Case.Insensitive, $"{provider}: {sql}");
|
||||
// the whole thing is one bounded server-side query, never a client-side scan
|
||||
sql.ShouldContain("LIMIT", Case.Insensitive, $"{provider}: {sql}");
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Regression_Pin_Song_List_Columns_Cannot_Be_Projected_Server_Side_On_Either_Provider()
|
||||
{
|
||||
// REGRESSION PIN, not coverage of #578: this asserts pre-existing EF/provider behaviour and passes
|
||||
// against the code before this change.
|
||||
//
|
||||
// Documents WHY the handler drops to raw SQL for SongMetadata.Artists / .AlbumArtists rather than
|
||||
// SelectMany-ing them: EF maps them as JSON primitive collections and neither provider can translate
|
||||
// the projection (SQLite needs APPLY; Pomelo has no primitive-collection support). If a provider
|
||||
// upgrade ever makes this translate, this test fails and the raw-SQL path can be retired.
|
||||
foreach ((string provider, Func<TvContext> create) in Providers())
|
||||
{
|
||||
using TvContext context = create();
|
||||
|
||||
Should.Throw<InvalidOperationException>(
|
||||
() => context.SongMetadata.SelectMany(m => m.Artists).Distinct().Take(50).ToQueryString(),
|
||||
$"{provider} unexpectedly translated a primitive-collection projection");
|
||||
|
||||
Should.Throw<InvalidOperationException>(
|
||||
() => context.SongMetadata.SelectMany(m => m.AlbumArtists).Distinct().Take(50).ToQueryString(),
|
||||
$"{provider} unexpectedly translated a primitive-collection projection");
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void List_Valued_Page_Query_Has_No_Predicate_Beyond_The_Keyset_Cursor()
|
||||
{
|
||||
// This is the whole basis of the row bound, so it is asserted rather than assumed. LIMIT truncates what
|
||||
// survives a RESIDUAL predicate — one that discards rows the engine already produced — so with such a
|
||||
// predicate present it bounds the output rather than the row count, and the engine may produce and
|
||||
// discard arbitrarily many rows first. That is how four successive revisions scanned past their own
|
||||
// bound. The cursor `Id > @AfterId` is NOT such a predicate: it is a seek on the ordering key, which
|
||||
// positions the scan without discarding anything, so LIMIT n yields n logical rows.
|
||||
//
|
||||
// What this test can and cannot do: it pins the SQL STRING. It cannot pin an execution plan, MVCC
|
||||
// visibility work or payload I/O -- physical work is NOT bounded (see the record: MySQL traverses
|
||||
// deleted-but-unpurged index records, and TEXT payloads spill to overflow pages).
|
||||
string sql = GetSearchFieldValuesHandler.ListValuedSql("Artists");
|
||||
|
||||
sql.ShouldBe(
|
||||
"SELECT Id, Artists AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch");
|
||||
|
||||
// named explicitly so a future "optimization" that reintroduces server-side selectivity fails here
|
||||
sql.ShouldNotContain("LIKE");
|
||||
sql.ShouldNotContain("LOWER");
|
||||
sql.ShouldNotContain("IS NOT NULL");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. The SQL function name is duplicated — the handler lives in Application, which must
|
||||
/// not reference a provider assembly, so it cannot use the constant the registration side defines. A
|
||||
/// rename on one side alone would compile cleanly and fail only at runtime, only on SQLite, only for
|
||||
/// non-ASCII queries; this pins the two together instead.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void Unicode_Fold_Function_Name_Matches_The_Registration() =>
|
||||
GetSearchFieldValuesHandler.UpperFunction.ShouldBe(SqliteUnicodeFunctions.UpperInvariantFunction);
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668. Unlike the list-valued walk, this query KEEPS its selectivity in SQL — it is a
|
||||
/// bounded <c>LIMIT</c>ed prefix query exactly like the EF one it supplements, so a <c>LIKE</c> here
|
||||
/// is correct rather than the trap the walk's shape test guards against. What must hold is that the
|
||||
/// fold is the registered Unicode-correct one and NOT the provider's ASCII-only builtin, and that the
|
||||
/// wildcard escape is declared.
|
||||
/// </summary>
|
||||
[Test]
|
||||
public void Unicode_Fold_Query_Uses_The_Registered_Fold_And_Declares_Its_Escape()
|
||||
{
|
||||
string sql = GetSearchFieldValuesHandler.UnicodeFoldSql("Genre", "Name", null);
|
||||
|
||||
sql.ShouldBe(
|
||||
"SELECT DISTINCT Name AS Value FROM Genre "
|
||||
+ "WHERE etv_upper(Name) LIKE @Pattern ESCAPE '\\' ORDER BY Name LIMIT @Limit");
|
||||
|
||||
// The point of the whole change: SQLite's BUILTIN lower()/upper() fold ASCII only, so quietly falling
|
||||
// back to one reinstates #668. Checked by removing the qualified call first — Shouldly's string
|
||||
// assertions are case-INSENSITIVE by default, so a bare ShouldNotContain("UPPER(") matches inside
|
||||
// "etv_upper(" and fails against correct SQL.
|
||||
sql.ShouldNotContain("LOWER(");
|
||||
sql.Replace($"{GetSearchFieldValuesHandler.UpperFunction}(", "", StringComparison.Ordinal)
|
||||
.ShouldNotContain("UPPER(");
|
||||
|
||||
// a discriminator predicate is parenthesised and ANDed, so an OR inside it cannot swallow the match
|
||||
GetSearchFieldValuesHandler.UnicodeFoldSql("Tag", "Name", "ExternalTypeId IS NULL OR X")
|
||||
.ShouldContain("WHERE (ExternalTypeId IS NULL OR X) AND etv_upper(Name) LIKE @Pattern");
|
||||
}
|
||||
|
||||
private static IEnumerable<(string Provider, Func<TvContext> Create)> Providers() =>
|
||||
[
|
||||
("sqlite", Sqlite),
|
||||
("mysql", MySql)
|
||||
];
|
||||
|
||||
private static TvContext Sqlite()
|
||||
{
|
||||
TvContext.IsSqlite = true;
|
||||
var builder = new DbContextOptionsBuilder<TvContext>();
|
||||
builder.UseSqlite("Data Source=:memory:");
|
||||
return Create(builder.Options);
|
||||
}
|
||||
|
||||
private static TvContext MySql()
|
||||
{
|
||||
TvContext.IsSqlite = false;
|
||||
var builder = new DbContextOptionsBuilder<TvContext>();
|
||||
builder.UseMySql(
|
||||
"Server=localhost;Database=ersatztv_query_shape;User=root;Password=ersatztv;",
|
||||
new MySqlServerVersion(new Version(8, 0, 36)));
|
||||
return Create(builder.Options);
|
||||
}
|
||||
|
||||
private static TvContext Create(DbContextOptions<TvContext> options) =>
|
||||
new(
|
||||
options,
|
||||
NullLoggerFactory.Instance,
|
||||
new SlowQueryInterceptor(NullLogger<SlowQueryInterceptor>.Instance));
|
||||
}
|
||||
@@ -1,183 +0,0 @@
|
||||
using System.Text.RegularExpressions;
|
||||
using ErsatzTV.Tests.Support;
|
||||
using Microsoft.OpenApi;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
namespace ErsatzTV.Tests.Controllers;
|
||||
|
||||
/// <summary>
|
||||
/// Pins the <c>api.paging-zero-based</c> contract onto the generated OpenAPI document (ersatztv#633).
|
||||
/// The spec is the contract REST consumers read — and what generated clients surface to their users —
|
||||
/// so a paging parameter that documents nothing forces every consumer to infer the base from
|
||||
/// <c>default: 0</c>. That is exactly the inference that cost ersatztv#487 a verification pass on the
|
||||
/// MCP side, where the description was present but wrong. The MCP wrapper is pinned the same way in
|
||||
/// <c>ErsatzTV.Mcp.Tests.ToolCatalogTests</c>; this is the API-side half.
|
||||
/// </summary>
|
||||
[TestFixture]
|
||||
public class OpenApiPagingContractTests
|
||||
{
|
||||
/// <summary>
|
||||
/// Every operation that pages. Named explicitly rather than discovered, because a test that only
|
||||
/// FILTERS on "declares pageNum" cannot see the endpoint that should page and does not — the
|
||||
/// defect escapes the filter and the test still passes green over a shrinking scope. That is not
|
||||
/// hypothetical: ersatztv#616 found two MCP tools doing precisely that. So the expected set is
|
||||
/// pinned here, and <see cref="Paged_Operations_Should_Be_Exactly_The_Pinned_Set" /> asserts the
|
||||
/// discovered set equals it in BOTH directions — a new paged endpoint fails until it is added
|
||||
/// (with descriptions), and an endpoint that silently drops paging fails too.
|
||||
/// </summary>
|
||||
private static readonly string[] PagedOperations =
|
||||
[
|
||||
"GET /api/v1/channels/auto-tune/members",
|
||||
"GET /api/v1/collections/{id}/items",
|
||||
"GET /api/v1/library/browse",
|
||||
"GET /api/v1/logs",
|
||||
"GET /api/v1/multi-collections",
|
||||
"GET /api/v1/playouts",
|
||||
"GET /api/v1/playouts/{id}/blocks/{blockId}/history",
|
||||
"GET /api/v1/playouts/{id}/items",
|
||||
"GET /api/v1/rerun-collections",
|
||||
"GET /api/v1/search",
|
||||
"GET /api/v1/search/all-items",
|
||||
"GET /api/v1/trakt/lists"
|
||||
];
|
||||
|
||||
private static OpenApiDocument _document = null!;
|
||||
|
||||
[OneTimeSetUp]
|
||||
public async Task BuildDocument() => _document = await GeneratedOpenApiDocument.BuildV1Async();
|
||||
|
||||
[Test]
|
||||
public void Paged_Operations_Should_Be_Exactly_The_Pinned_Set()
|
||||
{
|
||||
List<string> discovered = EnumerateOperations()
|
||||
.Where(op => ParameterNames(op.Operation).Overlaps(new[] { "pageNum", "pageSize" }))
|
||||
.Select(op => $"{op.Method} {op.Path}")
|
||||
.OrderBy(s => s, StringComparer.Ordinal)
|
||||
.ToList();
|
||||
|
||||
discovered.ShouldBe(PagedOperations.OrderBy(s => s, StringComparer.Ordinal).ToList());
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Every_Paged_Operation_Should_Declare_Both_Paging_Parameters()
|
||||
{
|
||||
foreach (string key in PagedOperations)
|
||||
{
|
||||
HashSet<string> names = ParameterNames(Find(key));
|
||||
|
||||
names.ShouldContain("pageNum", $"{key} should declare pageNum");
|
||||
names.ShouldContain("pageSize", $"{key} should declare pageSize");
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Every_PageNum_Parameter_Should_Document_The_ZeroBased_Contract()
|
||||
{
|
||||
foreach (string key in PagedOperations)
|
||||
{
|
||||
string description = Description(key, "pageNum");
|
||||
|
||||
// The whole point of the record: a consumer must not have to infer the base from `default: 0`.
|
||||
description.ShouldContain("0-based", Case.Insensitive, $"{key} pageNum should say it is 0-based");
|
||||
description.ShouldNotContain("1-based", Case.Insensitive, $"{key} pageNum must not claim 1-based");
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void Every_PageSize_Parameter_Should_Document_The_Cap_And_The_Effective_Offset()
|
||||
{
|
||||
foreach (string key in PagedOperations)
|
||||
{
|
||||
string description = Description(key, "pageSize");
|
||||
|
||||
// `api.paging-zero-based` is explicit that the cap is PER-ENDPOINT and must not be documented
|
||||
// as one number, and that the offset derives from the effective (capped) size — so an
|
||||
// over-large pageSize narrows the page without widening the offset.
|
||||
description.ShouldContain("capped at", Case.Insensitive, $"{key} pageSize should state its cap");
|
||||
description.ShouldContain("this endpoint", Case.Insensitive, $"{key} pageSize cap should be scoped to the endpoint");
|
||||
description.ShouldContain("effective", Case.Insensitive, $"{key} pageSize should explain the effective-size offset");
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public void PageSize_Caps_Should_Match_The_Values_The_Controllers_Actually_Clamp_To()
|
||||
{
|
||||
// The caps genuinely differ per endpoint, which is why the record forbids documenting one number.
|
||||
// A description naming the wrong cap is worse than none — a wrong justification outlives a wrong
|
||||
// line — so pin each against the value its controller clamps to.
|
||||
var expectedCaps = new Dictionary<string, int>(StringComparer.Ordinal)
|
||||
{
|
||||
["GET /api/v1/channels/auto-tune/members"] = 200,
|
||||
["GET /api/v1/collections/{id}/items"] = 100,
|
||||
["GET /api/v1/library/browse"] = 100,
|
||||
["GET /api/v1/logs"] = 100,
|
||||
["GET /api/v1/multi-collections"] = 100,
|
||||
["GET /api/v1/playouts"] = 100,
|
||||
["GET /api/v1/playouts/{id}/blocks/{blockId}/history"] = 100,
|
||||
["GET /api/v1/playouts/{id}/items"] = 100,
|
||||
["GET /api/v1/rerun-collections"] = 100,
|
||||
["GET /api/v1/search"] = 100,
|
||||
["GET /api/v1/search/all-items"] = 1000,
|
||||
["GET /api/v1/trakt/lists"] = 100
|
||||
};
|
||||
|
||||
// Guard the guard: every pinned operation must carry an expected cap, so adding one above
|
||||
// without its cap here cannot quietly skip this assertion.
|
||||
expectedCaps.Keys.OrderBy(k => k, StringComparer.Ordinal)
|
||||
.ShouldBe(PagedOperations.OrderBy(k => k, StringComparer.Ordinal));
|
||||
|
||||
foreach ((string key, int cap) in expectedCaps)
|
||||
{
|
||||
// Enumerate EVERY cap claim in the description and require the set to be exactly one
|
||||
// number, the right one. Two weaker forms were rejected on the way here:
|
||||
// - ShouldContain("capped at 100") is satisfied by the string "capped at 1000", so a
|
||||
// cap-100 endpoint claiming 1000 passed — the very defect this test exists to catch.
|
||||
// - Matching one occurrence as a whole token ("capped at 100(?!\d)") fixes that, but
|
||||
// still passes a description that names a wrong cap somewhere ELSE in the sentence
|
||||
// and the right one later. Presence of a true claim is not absence of a false one.
|
||||
List<int> claimedCaps = Regex
|
||||
.Matches(Description(key, "pageSize"), @"capped at (\d+)", RegexOptions.IgnoreCase)
|
||||
.Select(match => int.Parse(match.Groups[1].Value))
|
||||
.ToList();
|
||||
|
||||
claimedCaps.ShouldBe([cap], $"{key} pageSize should make exactly one cap claim, of {cap}");
|
||||
}
|
||||
}
|
||||
|
||||
private static string Description(string key, string parameterName)
|
||||
{
|
||||
// Not `First(...)`: a missing parameter would throw "Sequence contains no matching element",
|
||||
// which names neither the endpoint nor the parameter and reads as a broken test rather than
|
||||
// the contract violation it is.
|
||||
IOpenApiParameter parameter = (Find(key).Parameters ?? [])
|
||||
.FirstOrDefault(p => string.Equals(p.Name, parameterName, StringComparison.Ordinal))
|
||||
.ShouldNotBeNull($"{key} should declare a {parameterName} parameter");
|
||||
|
||||
string? description = parameter.Description;
|
||||
description.ShouldNotBeNullOrWhiteSpace($"{key} {parameterName} should carry a description");
|
||||
|
||||
return description!;
|
||||
}
|
||||
|
||||
private static OpenApiOperation Find(string key) =>
|
||||
EnumerateOperations()
|
||||
.Where(op => string.Equals($"{op.Method} {op.Path}", key, StringComparison.Ordinal))
|
||||
.Select(op => op.Operation)
|
||||
.FirstOrDefault()
|
||||
.ShouldNotBeNull($"{key} should exist in the generated document");
|
||||
|
||||
private static HashSet<string> ParameterNames(OpenApiOperation operation) =>
|
||||
(operation.Parameters ?? []).Select(p => p.Name ?? string.Empty).ToHashSet(StringComparer.Ordinal);
|
||||
|
||||
private static IEnumerable<(string Method, string Path, OpenApiOperation Operation)> EnumerateOperations()
|
||||
{
|
||||
foreach ((string path, IOpenApiPathItem item) in _document.Paths)
|
||||
{
|
||||
foreach ((HttpMethod method, OpenApiOperation operation) in item.Operations!)
|
||||
{
|
||||
yield return (method.Method.ToUpperInvariant(), path, operation);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -166,25 +166,6 @@ public class PlayoutControllerTests
|
||||
.Seed.ShouldBe(4242);
|
||||
}
|
||||
|
||||
// #616: the playout DETAIL response carried channelName/channelNumber but no channelId, while
|
||||
// reset_channel_playout takes a CHANNEL id. The two id spaces overlap numerically, so a caller
|
||||
// that reached for the row's `id` reset a different channel and got a plausible 202 back. The
|
||||
// list rows gained channelId in #297; this pins the same field on the detail response, and the
|
||||
// distinct ids below prove it is the channel's, not the playout's.
|
||||
[Test]
|
||||
public async Task GetById_Should_Surface_ChannelId_Distinct_From_PlayoutId()
|
||||
{
|
||||
_mediator.Send(Arg.Any<GetPlayoutById>(), Arg.Any<CancellationToken>())
|
||||
.Returns(Option<PlayoutNameViewModel>.Some(MakePlayout(9) with { ChannelId = 400 }));
|
||||
|
||||
IActionResult result = await _controller.GetById(9, CancellationToken.None);
|
||||
|
||||
PlayoutResponseModel body = result.ShouldBeOfType<OkObjectResult>()
|
||||
.Value.ShouldBeOfType<PlayoutResponseModel>();
|
||||
body.Id.ShouldBe(9);
|
||||
body.ChannelId.ShouldBe(400);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task GetAll_Should_Surface_Seed()
|
||||
{
|
||||
@@ -1451,7 +1432,6 @@ public class PlayoutControllerTests
|
||||
vm.ScheduleKind,
|
||||
vm.ChannelName,
|
||||
vm.ChannelNumber,
|
||||
vm.ChannelId,
|
||||
vm.PlayoutMode,
|
||||
vm.ScheduleName,
|
||||
vm.ScheduleFile,
|
||||
|
||||
@@ -1,302 +0,0 @@
|
||||
using ErsatzTV.Core;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Infrastructure.Data.Repositories;
|
||||
using ErsatzTV.Infrastructure.Sqlite.Data;
|
||||
using ErsatzTV.Tests.Support;
|
||||
using LanguageExt;
|
||||
using Microsoft.Data.Sqlite;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.EntityFrameworkCore.Diagnostics;
|
||||
using NSubstitute;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
using IFileSystem = System.IO.Abstractions.IFileSystem;
|
||||
|
||||
namespace ErsatzTV.Tests.Integration;
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#491: <c>ILibraryRepository.GetOrAddFolder</c> is a check-then-insert, so two callers
|
||||
/// racing the same <c>(LibraryPathId, Path)</c> both miss the lookup and both insert. The fix is a
|
||||
/// unique index on <c>(LibraryPathId, PathHash)</c> plus a catch-and-re-read in the repository, so
|
||||
/// the loser adopts the winner's row instead of creating a duplicate.
|
||||
/// </summary>
|
||||
[TestFixture]
|
||||
public class LibraryFolderConcurrencyTests
|
||||
{
|
||||
private const string LibraryPathValue = "/data/music";
|
||||
private const string FolderPath = "/data/music/artist1";
|
||||
|
||||
private static LibraryRepository Repository(IDbContextFactory<TvContext> factory) =>
|
||||
new(Substitute.For<IFileSystem>(), factory);
|
||||
|
||||
private static async Task<int> SeedLibraryPath(Func<TvContext> createContext)
|
||||
{
|
||||
await using TvContext context = createContext();
|
||||
var libraryPath = new LibraryPath { Path = LibraryPathValue };
|
||||
await context.LibraryPaths.AddAsync(libraryPath);
|
||||
await context.SaveChangesAsync();
|
||||
return libraryPath.Id;
|
||||
}
|
||||
|
||||
private static async Task<int> FolderCount(Func<TvContext> createContext, int libraryPathId, string path)
|
||||
{
|
||||
await using TvContext context = createContext();
|
||||
return await context.LibraryFolders.CountAsync(f => f.LibraryPathId == libraryPathId && f.Path == path);
|
||||
}
|
||||
|
||||
private static async Task InsertFolderRaw(
|
||||
SharedCacheTvContext db,
|
||||
int libraryPathId,
|
||||
string path,
|
||||
string pathHash,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
await using SqliteConnection connection = db.OpenConnection();
|
||||
await using SqliteCommand command = connection.CreateCommand();
|
||||
command.CommandText =
|
||||
"INSERT INTO \"LibraryFolder\" (\"LibraryPathId\", \"Path\", \"PathHash\", \"Etag\", \"ParentId\") " +
|
||||
"VALUES ($libraryPathId, $path, $pathHash, NULL, NULL)";
|
||||
command.Parameters.AddWithValue("$libraryPathId", libraryPathId);
|
||||
command.Parameters.AddWithValue("$path", path);
|
||||
command.Parameters.AddWithValue("$pathHash", (object?)pathHash ?? DBNull.Value);
|
||||
await command.ExecuteNonQueryAsync(cancellationToken);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Simulates the concurrent "winner": exactly once, on a SEPARATE connection, insert the same
|
||||
/// folder and commit — AFTER the intercepted context read the (stale) absent lookup but BEFORE its
|
||||
/// own INSERT runs. This interposes the race deterministically instead of hoping for a timing
|
||||
/// window. <see cref="Fired" /> proves the race actually happened (non-vacuity).
|
||||
/// </summary>
|
||||
private sealed class InsertConflictingFolderOnce(SharedCacheTvContext db, int libraryPathId, string path)
|
||||
: SaveChangesInterceptor
|
||||
{
|
||||
private int _fired;
|
||||
|
||||
public int Fired => _fired;
|
||||
|
||||
public override async ValueTask<InterceptionResult<int>> SavingChangesAsync(
|
||||
DbContextEventData eventData,
|
||||
InterceptionResult<int> result,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (Interlocked.Exchange(ref _fired, 1) == 0)
|
||||
{
|
||||
await InsertFolderRaw(db, libraryPathId, path, PathUtils.GetPathHash(path), cancellationToken);
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>Counts insert attempts so the multi-threaded test can prove it really raced.</summary>
|
||||
private sealed class CountSaveAttempts : SaveChangesInterceptor
|
||||
{
|
||||
private int _attempts;
|
||||
|
||||
public int Attempts => _attempts;
|
||||
|
||||
public override ValueTask<InterceptionResult<int>> SavingChangesAsync(
|
||||
DbContextEventData eventData,
|
||||
InterceptionResult<int> result,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
Interlocked.Increment(ref _attempts);
|
||||
return ValueTask.FromResult(result);
|
||||
}
|
||||
}
|
||||
|
||||
// ----- Negative control #1: the index itself. Without the new unique index this test fails, because
|
||||
// the second insert simply succeeds and there is no violation to classify. -----
|
||||
|
||||
[Test]
|
||||
public async Task Duplicate_LibraryFolder_Insert_Throws_A_Classified_UniqueViolation()
|
||||
{
|
||||
await using SharedCacheTvContext db = await SharedCacheTvContext.CreateAsync("etv491-index");
|
||||
int libraryPathId = await SeedLibraryPath(db.CreateContext);
|
||||
string hash = PathUtils.GetPathHash(FolderPath);
|
||||
|
||||
await InsertFolderRaw(db, libraryPathId, FolderPath, hash);
|
||||
|
||||
await using TvContext context = db.CreateContext();
|
||||
await context.LibraryFolders.AddAsync(
|
||||
new LibraryFolder { LibraryPathId = libraryPathId, Path = FolderPath, PathHash = hash });
|
||||
|
||||
DbUpdateException ex = await Should.ThrowAsync<DbUpdateException>(() => context.SaveChangesAsync());
|
||||
SqliteErrorClassifier.IsUniqueConstraintViolation(ex).ShouldBeTrue();
|
||||
|
||||
// and the classifier is not a blanket "true"
|
||||
SqliteErrorClassifier.IsUniqueConstraintViolation(
|
||||
new DbUpdateException("nope", new InvalidOperationException())).ShouldBeFalse();
|
||||
}
|
||||
|
||||
// The migration leaves pre-#491 rows with a null hash; a unique index treats nulls as distinct, so
|
||||
// applying the index to an existing database can never fail on them. Documents that premise.
|
||||
[Test]
|
||||
public async Task Legacy_Null_PathHash_Rows_Do_Not_Collide()
|
||||
{
|
||||
await using SharedCacheTvContext db = await SharedCacheTvContext.CreateAsync("etv491-nulls");
|
||||
int libraryPathId = await SeedLibraryPath(db.CreateContext);
|
||||
|
||||
await InsertFolderRaw(db, libraryPathId, "/data/music/a", null);
|
||||
await InsertFolderRaw(db, libraryPathId, "/data/music/b", null);
|
||||
|
||||
await using TvContext context = db.CreateContext();
|
||||
(await context.LibraryFolders.CountAsync(f => f.LibraryPathId == libraryPathId)).ShouldBe(2);
|
||||
}
|
||||
|
||||
// ----- The deterministic cross-connection race through the real repository -----
|
||||
|
||||
[Test]
|
||||
public async Task GetOrAddFolder_Losing_The_Race_Adopts_The_Winner_Instead_Of_Duplicating()
|
||||
{
|
||||
await using SharedCacheTvContext db = await SharedCacheTvContext.CreateAsync("etv491-race");
|
||||
int libraryPathId = await SeedLibraryPath(db.CreateContext);
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = LibraryPathValue, LibraryFolders = null };
|
||||
|
||||
var racer = new InsertConflictingFolderOnce(db, libraryPathId, FolderPath);
|
||||
LibraryRepository repository = Repository(db.Factory(racer));
|
||||
|
||||
LibraryFolder result = await repository.GetOrAddFolder(libraryPath, Option<int>.None, FolderPath);
|
||||
|
||||
racer.Fired.ShouldBe(1); // the race genuinely occurred — this assertion is the vacuity guard
|
||||
result.ShouldNotBeNull();
|
||||
result.Id.ShouldBeGreaterThan(0);
|
||||
result.Path.ShouldBe(FolderPath);
|
||||
(await FolderCount(db.CreateContext, libraryPathId, FolderPath)).ShouldBe(1);
|
||||
|
||||
// the returned row is the winner's persisted row, not a phantom
|
||||
await using TvContext context = db.CreateContext();
|
||||
LibraryFolder persisted = await context.LibraryFolders.SingleAsync(f => f.LibraryPathId == libraryPathId);
|
||||
persisted.Id.ShouldBe(result.Id);
|
||||
persisted.PathHash.ShouldBe(PathUtils.GetPathHash(FolderPath));
|
||||
}
|
||||
|
||||
// The loser must still apply the parent id it was asked to set — to the WINNER's row.
|
||||
[Test]
|
||||
public async Task GetOrAddFolder_Losing_The_Race_Still_Persists_The_ParentId()
|
||||
{
|
||||
await using SharedCacheTvContext db = await SharedCacheTvContext.CreateAsync("etv491-race-parent");
|
||||
int libraryPathId = await SeedLibraryPath(db.CreateContext);
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = LibraryPathValue, LibraryFolders = null };
|
||||
|
||||
LibraryFolder parent = await Repository(db.Factory())
|
||||
.GetOrAddFolder(libraryPath, Option<int>.None, LibraryPathValue);
|
||||
|
||||
var racer = new InsertConflictingFolderOnce(db, libraryPathId, FolderPath);
|
||||
LibraryFolder result = await Repository(db.Factory(racer))
|
||||
.GetOrAddFolder(libraryPath, Option<int>.Some(parent.Id), FolderPath);
|
||||
|
||||
racer.Fired.ShouldBe(1);
|
||||
result.ParentId.ShouldBe(parent.Id);
|
||||
|
||||
await using TvContext context = db.CreateContext();
|
||||
LibraryFolder persisted = await context.LibraryFolders.SingleAsync(f => f.Path == FolderPath);
|
||||
persisted.Id.ShouldBe(result.Id);
|
||||
persisted.ParentId.ShouldBe(parent.Id);
|
||||
}
|
||||
|
||||
// ----- Negative control #2: invert the real condition (the provider classifier) and the SAME race
|
||||
// must blow up, proving the catch in GetOrAddFolder is load-bearing rather than decorative. -----
|
||||
|
||||
[Test]
|
||||
public async Task GetOrAddFolder_Rethrows_When_The_Provider_Does_Not_Classify_The_Violation()
|
||||
{
|
||||
Func<DbUpdateException, bool> original = TvContext.IsUniqueConstraintViolation;
|
||||
try
|
||||
{
|
||||
await using SharedCacheTvContext db = await SharedCacheTvContext.CreateAsync("etv491-negctl");
|
||||
int libraryPathId = await SeedLibraryPath(db.CreateContext);
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = LibraryPathValue, LibraryFolders = null };
|
||||
|
||||
var racer = new InsertConflictingFolderOnce(db, libraryPathId, FolderPath);
|
||||
LibraryRepository repository = Repository(db.Factory(racer));
|
||||
|
||||
TvContext.IsUniqueConstraintViolation = _ => false;
|
||||
await Should.ThrowAsync<DbUpdateException>(
|
||||
() => repository.GetOrAddFolder(libraryPath, Option<int>.None, FolderPath));
|
||||
|
||||
racer.Fired.ShouldBe(1);
|
||||
}
|
||||
finally
|
||||
{
|
||||
TvContext.IsUniqueConstraintViolation = original;
|
||||
}
|
||||
}
|
||||
|
||||
// ----- N threads x rounds over a single (LibraryPathId, Path) -----
|
||||
|
||||
[Test]
|
||||
public async Task Concurrent_GetOrAddFolder_Never_Produces_Duplicate_Rows()
|
||||
{
|
||||
const int threads = 8;
|
||||
const int rounds = 10;
|
||||
|
||||
await using SharedCacheTvContext db = await SharedCacheTvContext.CreateAsync("etv491-threads");
|
||||
int libraryPathId = await SeedLibraryPath(db.CreateContext);
|
||||
|
||||
var counter = new CountSaveAttempts();
|
||||
IDbContextFactory<TvContext> factory = db.Factory(counter);
|
||||
|
||||
for (var round = 0; round < rounds; round++)
|
||||
{
|
||||
string path = $"{LibraryPathValue}/round{round}";
|
||||
using var gate = new Barrier(threads);
|
||||
|
||||
var tasks = new Task<LibraryFolder>[threads];
|
||||
for (var thread = 0; thread < threads; thread++)
|
||||
{
|
||||
tasks[thread] = Task.Run(async () =>
|
||||
{
|
||||
// every thread carries its own detached LibraryPath, as the scanners do
|
||||
var libraryPath = new LibraryPath
|
||||
{
|
||||
Id = libraryPathId, Path = LibraryPathValue, LibraryFolders = null
|
||||
};
|
||||
|
||||
gate.SignalAndWait();
|
||||
return await Repository(factory).GetOrAddFolder(libraryPath, Option<int>.None, path);
|
||||
});
|
||||
}
|
||||
|
||||
LibraryFolder[] results = await Task.WhenAll(tasks);
|
||||
|
||||
// every caller got the same single row back...
|
||||
results.Select(f => f.Id).Distinct().Count().ShouldBe(1);
|
||||
results[0].Id.ShouldBeGreaterThan(0);
|
||||
|
||||
// ...and exactly one row exists for it
|
||||
(await FolderCount(db.CreateContext, libraryPathId, path)).ShouldBe(1);
|
||||
}
|
||||
|
||||
await using TvContext context = db.CreateContext();
|
||||
(await context.LibraryFolders.CountAsync(f => f.LibraryPathId == libraryPathId)).ShouldBe(rounds);
|
||||
|
||||
// Vacuity guard: one insert attempt per round would mean the threads never actually collided and
|
||||
// the test proved nothing. More attempts than rounds means at least one caller lost the race and
|
||||
// was rescued by the index + catch.
|
||||
counter.Attempts.ShouldBeGreaterThan(rounds);
|
||||
}
|
||||
|
||||
// ----- SetEtag is the repository's other check-then-insert on LibraryFolder -----
|
||||
|
||||
[Test]
|
||||
public async Task SetEtag_Losing_The_Race_Updates_The_Winner_Instead_Of_Duplicating()
|
||||
{
|
||||
await using SharedCacheTvContext db = await SharedCacheTvContext.CreateAsync("etv491-setetag");
|
||||
int libraryPathId = await SeedLibraryPath(db.CreateContext);
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = LibraryPathValue, LibraryFolders = null };
|
||||
|
||||
var racer = new InsertConflictingFolderOnce(db, libraryPathId, FolderPath);
|
||||
LibraryRepository repository = Repository(db.Factory(racer));
|
||||
|
||||
await repository.SetEtag(libraryPath, Option<LibraryFolder>.None, FolderPath, "etag-1");
|
||||
|
||||
racer.Fired.ShouldBe(1);
|
||||
|
||||
await using TvContext context = db.CreateContext();
|
||||
LibraryFolder persisted = await context.LibraryFolders.SingleAsync(f => f.Path == FolderPath);
|
||||
persisted.Etag.ShouldBe("etag-1");
|
||||
}
|
||||
}
|
||||
@@ -1,445 +0,0 @@
|
||||
using Dapper;
|
||||
using ErsatzTV.Infrastructure;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Infrastructure.MySql.Data;
|
||||
using ErsatzTV.Infrastructure.Sqlite.Data;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using MySqlConnector;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
namespace ErsatzTV.Tests.Integration;
|
||||
|
||||
public enum TestProvider
|
||||
{
|
||||
Sqlite,
|
||||
MySql
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#491: the unique index on <c>LibraryFolder(LibraryPathId, PathHash)</c> ships with an
|
||||
/// audit/cleanup of pre-existing duplicate rows (reachable before #488, when the folder lookup read a
|
||||
/// scan-start in-memory snapshot and could not see a folder created earlier in the same scan). This
|
||||
/// drives the REAL migration against a database seeded at the previous migration, so the cleanup SQL
|
||||
/// is exercised rather than restated.
|
||||
/// <para>
|
||||
/// Runs against BOTH providers from ONE fixture body. The dedupe deletes rows irreversibly and its
|
||||
/// correctness turns on string-comparison semantics that differ per provider — two MySQL-only
|
||||
/// collation defects escaped review in the #491 session (a case-insensitive grouping, then a PAD
|
||||
/// SPACE one), and neither was reachable from a SQLite-only test, nor from CI's MySQL job, which
|
||||
/// only applies migrations to a fresh EMPTY database and so executes no dedupe rows at all.
|
||||
/// Parameterizing one fixture is what makes "the two providers agree" a checked property rather
|
||||
/// than an assumption; a separate MySQL-only copy would drift and recreate the gap.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// MySQL needs a live server, supplied via <c>ETV_TEST_MYSQL_CONNECTION</c>. Without it the MySQL
|
||||
/// fixture <b>ignores</b> — a visible skip, never a silent pass — so local runs need no MySQL. CI
|
||||
/// sets <c>ETV_REQUIRE_MYSQL_TESTS=1</c>, which turns that skip into a hard failure, so the gate
|
||||
/// cannot quietly degrade into "connected to nothing and passed".
|
||||
/// </para>
|
||||
/// </summary>
|
||||
[TestFixture(TestProvider.Sqlite)]
|
||||
[TestFixture(TestProvider.MySql)]
|
||||
[NonParallelizable]
|
||||
public class LibraryFolderDedupeMigrationTests(TestProvider provider)
|
||||
{
|
||||
// the migration immediately preceding Add_LibraryFolder_PathHash_UniqueIndex
|
||||
private const string PreviousMigration = "Add_Channel_Origin";
|
||||
|
||||
private const string MySqlConnectionVariable = "ETV_TEST_MYSQL_CONNECTION";
|
||||
private const string MySqlRequiredVariable = "ETV_REQUIRE_MYSQL_TESTS";
|
||||
|
||||
private string _databasePath = null!;
|
||||
private string? _mySqlConnectionString;
|
||||
|
||||
// Seeding writes deliberately partial object graphs (a LibraryPath with no Library, a MediaFile with
|
||||
// no MediaVersion), so it runs with foreign keys OFF.
|
||||
private DbContextOptions<TvContext> _seedOptions = null!;
|
||||
|
||||
// The migration itself runs with foreign keys ON, matching production. This matters: the single most
|
||||
// dangerous statement in the #491 migration is DELETE FROM LibraryFolder against two Restrict foreign
|
||||
// keys (MediaFile.LibraryFolderId, LibraryFolder.ParentId). With enforcement off, a wrong repoint
|
||||
// order would still pass; with it on, the delete fails loudly.
|
||||
private DbContextOptions<TvContext> _migrateOptions = null!;
|
||||
|
||||
[SetUp]
|
||||
public async Task SetUp()
|
||||
{
|
||||
if (provider is TestProvider.Sqlite)
|
||||
{
|
||||
TvContext.IsSqlite = true;
|
||||
TvContext.LastInsertedRowId = "last_insert_rowid()";
|
||||
TvContext.CaseInsensitiveCollation = "NOCASE";
|
||||
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
|
||||
|
||||
_databasePath = Path.Combine(Path.GetTempPath(), $"etv491-{Guid.NewGuid():N}.sqlite3");
|
||||
_seedOptions = SqliteOptions(foreignKeys: false);
|
||||
_migrateOptions = SqliteOptions(foreignKeys: true);
|
||||
return;
|
||||
}
|
||||
|
||||
string? baseConnectionString = Environment.GetEnvironmentVariable(MySqlConnectionVariable);
|
||||
if (string.IsNullOrWhiteSpace(baseConnectionString))
|
||||
{
|
||||
string message =
|
||||
$"{MySqlConnectionVariable} is not set, so the MySql half of the #491 dedupe fixture cannot "
|
||||
+ "run. The dedupe deletes rows irreversibly and its correctness is provider-specific, so "
|
||||
+ "this coverage is not optional in CI.";
|
||||
|
||||
// A skip is fine locally; in CI it is the very failure mode this fixture exists to prevent.
|
||||
if (IsTrue(Environment.GetEnvironmentVariable(MySqlRequiredVariable)))
|
||||
{
|
||||
Assert.Fail($"{message} {MySqlRequiredVariable} is set, so this is a failure, not a skip.");
|
||||
}
|
||||
|
||||
Assert.Ignore($"{message} Set it to run this locally.");
|
||||
}
|
||||
|
||||
// A database of our own, NOT the one the surrounding CI step migrates, and a FRESH one per test:
|
||||
// isolation by construction. A name that has never been used cannot contain another test's rows,
|
||||
// so no wipe has to succeed for the fixture to be correct. It is not created here — the test's own
|
||||
// MigrateAsync(PreviousMigration) creates it, which keeps EF the single owner of the schema.
|
||||
_mySqlConnectionString =
|
||||
new MySqlConnectionStringBuilder(baseConnectionString) { Database = $"etv491_{Guid.NewGuid():N}" }
|
||||
.ConnectionString;
|
||||
|
||||
TvContext.IsSqlite = false;
|
||||
TvContext.LastInsertedRowId = "last_insert_id()";
|
||||
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
|
||||
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
|
||||
|
||||
// AutoDetect opens its own connection, so resolve the version once and share it between the two
|
||||
// option sets instead of connecting twice per test.
|
||||
ServerVersion serverVersion = ServerVersion.AutoDetect(_mySqlConnectionString);
|
||||
_seedOptions = MySqlOptions(serverVersion);
|
||||
_migrateOptions = MySqlOptions(serverVersion);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Drop this test's database and clear the connection pool that was keyed to it.
|
||||
/// <para>
|
||||
/// Both halves are required, and earlier revisions of this fixture each got one wrong. Measured
|
||||
/// against a real 8.4 server:
|
||||
/// </para>
|
||||
/// <list type="number">
|
||||
/// <item>
|
||||
/// <b>A pooled session outlives <c>DROP DATABASE</c>.</b> Reopening the dropped database's
|
||||
/// connection string succeeds — MySqlConnector hands back the still-alive session whose
|
||||
/// default schema is gone — so whether a later caller sees success or <c>Unknown database</c>
|
||||
/// depends on whether the pool reuses that session or opens a fresh one (a fresh handshake
|
||||
/// names the dropped schema and fails 1049). <c>ClearPool</c> after the drop removes it.
|
||||
/// Note this hazard needs the connection string to be REUSED after the drop, which a
|
||||
/// never-repeated database name already makes impossible; clearing the pool is the belt to
|
||||
/// that brace, and closes the leak below.
|
||||
/// </item>
|
||||
/// <item>
|
||||
/// <b>An uncleared pool leaks a server connection per test.</b> MySqlConnector keys pools by
|
||||
/// connection string, so a fresh database name means a fresh pool; left uncleared it leaked
|
||||
/// ~1 server thread per iteration and eventually exhausted <c>max_connections</c>.
|
||||
/// <c>ClearPoolAsync</c> on that exact connection string fixes it completely — measured at
|
||||
/// 0 leaked threads over 30 iterations — so per-test isolation costs nothing. A previous
|
||||
/// revision instead collapsed to one shared database to stop the leak; that traded isolation
|
||||
/// for a wipe that has to succeed, and when it silently did not, the second test seeded on
|
||||
/// top of the first's rows and failed with a duplicate primary key.
|
||||
/// </item>
|
||||
/// </list>
|
||||
/// <para>
|
||||
/// EF owns the drop: <c>EnsureDeletedAsync</c> is guarded (a no-op when the database is absent,
|
||||
/// unlike a raw <c>DROP DATABASE</c>) and uses the same connection string EF migrated with.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
private async Task DropMySqlDatabase()
|
||||
{
|
||||
await using (TvContext context = MigrateContext())
|
||||
{
|
||||
await context.Database.EnsureDeletedAsync();
|
||||
}
|
||||
|
||||
await using var probe = new MySqlConnection(_mySqlConnectionString);
|
||||
await MySqlConnection.ClearPoolAsync(probe);
|
||||
}
|
||||
|
||||
[TearDown]
|
||||
public async Task TearDown()
|
||||
{
|
||||
if (provider is TestProvider.Sqlite)
|
||||
{
|
||||
Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools();
|
||||
foreach (string path in new[] { _databasePath, $"{_databasePath}-wal", $"{_databasePath}-shm" })
|
||||
{
|
||||
if (File.Exists(path))
|
||||
{
|
||||
File.Delete(path);
|
||||
}
|
||||
}
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
if (_mySqlConnectionString is not null)
|
||||
{
|
||||
await DropMySqlDatabase();
|
||||
_mySqlConnectionString = null;
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Migration_Collapses_Duplicate_Folders_And_Repoints_Their_Dependents()
|
||||
{
|
||||
await using (TvContext context = SeedContext())
|
||||
{
|
||||
await context.Database.MigrateAsync(PreviousMigration);
|
||||
await using SeedSession seed = await SeedSession.OpenAsync(context, provider);
|
||||
|
||||
// one library path with the SAME folder recorded three times (ids 1, 2, 3), an unrelated
|
||||
// folder (4), a child parented on one of the duplicates (5), a case-differing sibling (6),
|
||||
// a folder parented on its own duplicate (7/8 — the cycle the ParentId null-out guards), and
|
||||
// a sibling differing only by a TRAILING SPACE (9/10). The last two pairs are distinct legal
|
||||
// directories on Linux that hash differently, so the unique index accepts both and the dedupe
|
||||
// must not collapse them. On MySql a case-insensitive grouping deletes 6, and a PAD SPACE one
|
||||
// deletes 10 — utf8mb4_bin, the obvious fix for the first, is itself PAD SPACE, which is why
|
||||
// the migration groups on CONVERT(Path USING binary) instead.
|
||||
await seed.ExecuteAsync(
|
||||
"INSERT INTO LibraryPath (Id, LibraryId, Path) VALUES (1, 1, '/data/music')");
|
||||
await seed.ExecuteAsync(
|
||||
"""
|
||||
INSERT INTO LibraryFolder (Id, LibraryPathId, Path, ParentId, Etag) VALUES
|
||||
(1, 1, '/data/music/artist1', NULL, 'etag-keeper'),
|
||||
(2, 1, '/data/music/artist1', NULL, 'etag-dupe-a'),
|
||||
(3, 1, '/data/music/artist1', NULL, 'etag-dupe-b'),
|
||||
(4, 1, '/data/music/artist2', NULL, NULL),
|
||||
(5, 1, '/data/music/artist1/album', 3, NULL),
|
||||
(6, 1, '/data/music/ARTIST2', NULL, NULL),
|
||||
(7, 1, '/data/music/artist3', 8, NULL),
|
||||
(8, 1, '/data/music/artist3', NULL, NULL),
|
||||
(9, 1, '/data/music/pad', NULL, NULL),
|
||||
(10, 1, '/data/music/pad ', NULL, NULL)
|
||||
""");
|
||||
|
||||
// a media file on each duplicate and on each half of the trailing-space pair, plus an
|
||||
// image-folder-duration on duplicates only
|
||||
await seed.ExecuteAsync(
|
||||
"""
|
||||
INSERT INTO MediaFile (Id, Path, PathHash, MediaVersionId, LibraryFolderId) VALUES
|
||||
(1, '/data/music/artist1/a.mkv', 'hash-a', 1, 1),
|
||||
(2, '/data/music/artist1/b.mkv', 'hash-b', 2, 2),
|
||||
(3, '/data/music/artist1/c.mkv', 'hash-c', 3, 3),
|
||||
(4, '/data/music/pad/d.mkv', 'hash-d', 4, 9),
|
||||
(5, '/data/music/pad /e.mkv', 'hash-e', 5, 10)
|
||||
""");
|
||||
await seed.ExecuteAsync(
|
||||
"""
|
||||
INSERT INTO ImageFolderDuration (Id, LibraryFolderId, DurationSeconds) VALUES
|
||||
(1, 2, 30.0),
|
||||
(2, 3, 45.0)
|
||||
""");
|
||||
}
|
||||
|
||||
await using (TvContext context = MigrateContext())
|
||||
{
|
||||
await AssertForeignKeysEnforced(context);
|
||||
await context.Database.MigrateAsync();
|
||||
}
|
||||
|
||||
await using (TvContext context = MigrateContext())
|
||||
{
|
||||
// Read the surviving rows once and assert in memory. Deliberately NOT `WHERE Path = '...'`:
|
||||
// that predicate is itself collation-dependent (on MySQL it would also match the
|
||||
// case-differing and trailing-space siblings), so an assertion written that way would quietly
|
||||
// mean something different on each provider — the exact class of bug this fixture guards.
|
||||
List<FolderRow> folders = (await context.Connection.QueryAsync<FolderRow>(
|
||||
"SELECT Id, Path, ParentId, Etag FROM LibraryFolder ORDER BY Id")).ToList();
|
||||
|
||||
// survivors: keeper 1, artist2 (4), the child (5), the case-differing sibling (6), artist3's
|
||||
// keeper (7), and BOTH halves of the trailing-space pair (9, 10). Losers 2, 3 and 8 are gone.
|
||||
folders.Select(f => f.Id).ToList().ShouldBe([1, 4, 5, 6, 7, 9, 10]);
|
||||
|
||||
// and their paths survive BYTE-exactly (ordinal comparison here, matching the hash)
|
||||
folders.Select(f => f.Path).ToList().ShouldBe(
|
||||
[
|
||||
"/data/music/artist1",
|
||||
"/data/music/artist2",
|
||||
"/data/music/artist1/album",
|
||||
"/data/music/ARTIST2",
|
||||
"/data/music/artist3",
|
||||
"/data/music/pad",
|
||||
"/data/music/pad "
|
||||
]);
|
||||
|
||||
// the keeper's etag is cleared: which duplicate the scanner was writing to was arbitrary, so
|
||||
// MIN(Id)'s etag could suppress the rescan that repairs the collapsed folder
|
||||
folders.Single(f => f.Id == 1).Etag.ShouldBeNull();
|
||||
|
||||
// the child folder is reparented off the deleted duplicate onto the keeper
|
||||
folders.Single(f => f.Id == 5).ParentId.ShouldBe(1);
|
||||
|
||||
// the folder parented on its own duplicate did not become its own parent
|
||||
folders.Single(f => f.Id == 7).ParentId.ShouldBeNull();
|
||||
|
||||
// every media file follows the keeper — nothing orphaned, nothing deleted — while the
|
||||
// trailing-space pair's dependents stay attached to their OWN folder
|
||||
List<int> mediaFolderIds = (await context.Connection.QueryAsync<int>(
|
||||
"SELECT LibraryFolderId FROM MediaFile ORDER BY Id")).ToList();
|
||||
mediaFolderIds.ShouldBe([1, 1, 1, 9, 10]);
|
||||
|
||||
// the keeper had no ImageFolderDuration, so exactly one duplicate's setting is promoted to it
|
||||
// (the lowest id) and the rest are dropped — the 1:1 unique index cannot hold both
|
||||
List<int> durationFolderIds = (await context.Connection.QueryAsync<int>(
|
||||
"SELECT LibraryFolderId FROM ImageFolderDuration ORDER BY Id")).ToList();
|
||||
durationFolderIds.ShouldBe([1]);
|
||||
|
||||
await AssertHelperTablesDropped(context);
|
||||
await AssertUniqueIndexExists(context);
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task Migration_Leaves_A_Database_Without_Duplicates_Alone()
|
||||
{
|
||||
// positive control: the cleanup must not touch rows that were already unique
|
||||
await using (TvContext context = SeedContext())
|
||||
{
|
||||
await context.Database.MigrateAsync(PreviousMigration);
|
||||
await using SeedSession seed = await SeedSession.OpenAsync(context, provider);
|
||||
|
||||
await seed.ExecuteAsync(
|
||||
"INSERT INTO LibraryPath (Id, LibraryId, Path) VALUES (1, 1, '/data/music')");
|
||||
await seed.ExecuteAsync(
|
||||
"""
|
||||
INSERT INTO LibraryFolder (Id, LibraryPathId, Path, ParentId, Etag) VALUES
|
||||
(1, 1, '/data/music/artist1', NULL, 'etag-1'),
|
||||
(2, 1, '/data/music/artist2', 1, 'etag-2')
|
||||
""");
|
||||
await seed.ExecuteAsync(
|
||||
"INSERT INTO ImageFolderDuration (Id, LibraryFolderId, DurationSeconds) VALUES (1, 2, 30.0)");
|
||||
}
|
||||
|
||||
await using (TvContext context = MigrateContext())
|
||||
{
|
||||
await AssertForeignKeysEnforced(context);
|
||||
await context.Database.MigrateAsync();
|
||||
}
|
||||
|
||||
await using (TvContext context = MigrateContext())
|
||||
{
|
||||
List<FolderRow> folders = (await context.Connection.QueryAsync<FolderRow>(
|
||||
"SELECT Id, Path, ParentId, Etag FROM LibraryFolder ORDER BY Id")).ToList();
|
||||
|
||||
folders.Select(f => f.Id).ToList().ShouldBe([1, 2]);
|
||||
folders.Single(f => f.Id == 1).Etag.ShouldBe("etag-1");
|
||||
folders.Single(f => f.Id == 2).Etag.ShouldBe("etag-2");
|
||||
folders.Single(f => f.Id == 2).ParentId.ShouldBe(1);
|
||||
|
||||
(await context.Connection.ExecuteScalarAsync<int>(
|
||||
"SELECT LibraryFolderId FROM ImageFolderDuration WHERE Id = 1")).ShouldBe(2);
|
||||
|
||||
// existing rows keep a null hash — the index applies because nulls are distinct
|
||||
(await context.Connection.ExecuteScalarAsync<int>(
|
||||
"SELECT COUNT(*) FROM LibraryFolder WHERE PathHash IS NULL")).ShouldBe(2);
|
||||
}
|
||||
}
|
||||
|
||||
// Foreign-key enforcement is the whole point of splitting the seed and migrate contexts, so prove it
|
||||
// took effect rather than trusting a connection-string keyword: a typo, a pragma reset or a leaked
|
||||
// SET FOREIGN_KEY_CHECKS=0 would silently revert this to its weaker form and still pass everything.
|
||||
private async Task AssertForeignKeysEnforced(TvContext context)
|
||||
{
|
||||
string sql = provider is TestProvider.Sqlite
|
||||
? "PRAGMA foreign_keys"
|
||||
: "SELECT @@SESSION.foreign_key_checks";
|
||||
|
||||
(await context.Connection.ExecuteScalarAsync<long>(sql)).ShouldBe(
|
||||
1,
|
||||
"the migration must run with foreign keys ENFORCED — otherwise DELETE FROM LibraryFolder is "
|
||||
+ "not actually tested against the Restrict constraints it must not violate");
|
||||
}
|
||||
|
||||
private async Task AssertHelperTablesDropped(TvContext context)
|
||||
{
|
||||
string sql = provider is TestProvider.Sqlite
|
||||
? "SELECT COUNT(*) FROM sqlite_master WHERE type = 'table' AND name LIKE '__LibraryFolderDedupe%'"
|
||||
: "SELECT COUNT(*) FROM information_schema.TABLES WHERE TABLE_SCHEMA = DATABASE() "
|
||||
+ "AND TABLE_NAME LIKE '\\_\\_LibraryFolderDedupe%'";
|
||||
|
||||
(await context.Connection.ExecuteScalarAsync<int>(sql)).ShouldBe(0, "helper tables were left behind");
|
||||
}
|
||||
|
||||
private async Task AssertUniqueIndexExists(TvContext context)
|
||||
{
|
||||
string sql = provider is TestProvider.Sqlite
|
||||
? "SELECT COUNT(*) FROM sqlite_master WHERE type = 'index' "
|
||||
+ "AND name = 'IX_LibraryFolder_LibraryPathId_PathHash'"
|
||||
: "SELECT COUNT(DISTINCT INDEX_NAME) FROM information_schema.STATISTICS "
|
||||
+ "WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'LibraryFolder' "
|
||||
+ "AND INDEX_NAME = 'IX_LibraryFolder_LibraryPathId_PathHash'";
|
||||
|
||||
(await context.Connection.ExecuteScalarAsync<int>(sql)).ShouldBe(1, "the unique index was not created");
|
||||
}
|
||||
|
||||
private static bool IsTrue(string? value) =>
|
||||
value is "1" || string.Equals(value, "true", StringComparison.OrdinalIgnoreCase);
|
||||
|
||||
private DbContextOptions<TvContext> SqliteOptions(bool foreignKeys) =>
|
||||
new DbContextOptionsBuilder<TvContext>()
|
||||
.UseSqlite(
|
||||
$"Data Source={_databasePath};Foreign Keys={foreignKeys}",
|
||||
o => o.MigrationsAssembly("ErsatzTV.Infrastructure.Sqlite"))
|
||||
.Options;
|
||||
|
||||
private DbContextOptions<TvContext> MySqlOptions(ServerVersion serverVersion) =>
|
||||
new DbContextOptionsBuilder<TvContext>()
|
||||
.UseMySql(
|
||||
_mySqlConnectionString,
|
||||
serverVersion,
|
||||
o => o.MigrationsAssembly("ErsatzTV.Infrastructure.MySql"))
|
||||
.Options;
|
||||
|
||||
private TvContext SeedContext() => Create(_seedOptions);
|
||||
|
||||
private TvContext MigrateContext() => Create(_migrateOptions);
|
||||
|
||||
private static TvContext Create(DbContextOptions<TvContext> options) =>
|
||||
new(
|
||||
options,
|
||||
NullLoggerFactory.Instance,
|
||||
new SlowQueryInterceptor(NullLogger<SlowQueryInterceptor>.Instance));
|
||||
|
||||
// Settable properties rather than a positional record: SQLite hands back INTEGER as Int64 while MySQL
|
||||
// hands back INT as Int32, and Dapper only narrows for property setters, not constructor matching.
|
||||
private sealed class FolderRow
|
||||
{
|
||||
public int Id { get; set; }
|
||||
public string Path { get; set; } = null!;
|
||||
public int? ParentId { get; set; }
|
||||
public string Etag { get; set; } = null!;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Holds one connection open for the whole seeding block. SQLite disables foreign keys through a
|
||||
/// connection-string keyword, but MySQL's <c>foreign_key_checks</c> is a SESSION variable — and
|
||||
/// Dapper closes a connection it had to open itself, which would reset it between statements.
|
||||
/// Opening explicitly keeps the session, and therefore the setting, alive across every insert.
|
||||
/// </summary>
|
||||
private sealed class SeedSession : IAsyncDisposable
|
||||
{
|
||||
private readonly TvContext _context;
|
||||
|
||||
private SeedSession(TvContext context) => _context = context;
|
||||
|
||||
public static async Task<SeedSession> OpenAsync(TvContext context, TestProvider provider)
|
||||
{
|
||||
await context.Database.OpenConnectionAsync();
|
||||
if (provider is TestProvider.MySql)
|
||||
{
|
||||
await context.Connection.ExecuteAsync("SET SESSION foreign_key_checks = 0");
|
||||
}
|
||||
|
||||
return new SeedSession(context);
|
||||
}
|
||||
|
||||
public Task ExecuteAsync(string sql) => _context.Connection.ExecuteAsync(sql);
|
||||
|
||||
public async ValueTask DisposeAsync() => await _context.Database.CloseConnectionAsync();
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,3 @@
|
||||
using ErsatzTV.Core;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Infrastructure.Data.Repositories;
|
||||
@@ -106,225 +105,6 @@ public class LibraryRepositoryTests
|
||||
count.ShouldBe(1);
|
||||
}
|
||||
|
||||
// ersatztv#491: the unique index is on (LibraryPathId, PathHash) because Path is unbounded, so every
|
||||
// new row must carry the hash or the constraint is unenforceable for it.
|
||||
[Test]
|
||||
public async Task GetOrAddFolder_Should_Populate_PathHash_On_New_Folder()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
|
||||
|
||||
LibraryFolder result = await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
|
||||
|
||||
result.PathHash.ShouldBe(PathUtils.GetPathHash("/data/music/artist1"));
|
||||
|
||||
await using TvContext context = _db.CreateContext();
|
||||
LibraryFolder persisted = await context.LibraryFolders.SingleAsync(f => f.Id == result.Id);
|
||||
persisted.PathHash.ShouldBe(PathUtils.GetPathHash("/data/music/artist1"));
|
||||
}
|
||||
|
||||
// ersatztv#491: rows that predate the PathHash column are left null by the migration (nulls are
|
||||
// distinct in a unique index, so the index applies cleanly); the first scan that touches one heals it.
|
||||
[Test]
|
||||
public async Task GetOrAddFolder_Should_Heal_A_Legacy_Null_PathHash()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
|
||||
|
||||
int legacyId;
|
||||
await using (TvContext seed = _db.CreateContext())
|
||||
{
|
||||
var legacy = new LibraryFolder
|
||||
{
|
||||
LibraryPathId = libraryPathId, Path = "/data/music/artist1", PathHash = null
|
||||
};
|
||||
await seed.LibraryFolders.AddAsync(legacy);
|
||||
await seed.SaveChangesAsync();
|
||||
legacyId = legacy.Id;
|
||||
}
|
||||
|
||||
LibraryFolder result = await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
|
||||
|
||||
result.Id.ShouldBe(legacyId);
|
||||
result.PathHash.ShouldBe(PathUtils.GetPathHash("/data/music/artist1"));
|
||||
|
||||
await using TvContext context = _db.CreateContext();
|
||||
LibraryFolder persisted = await context.LibraryFolders.SingleAsync(f => f.Id == legacyId);
|
||||
persisted.PathHash.ShouldBe(PathUtils.GetPathHash("/data/music/artist1"));
|
||||
(await context.LibraryFolders.CountAsync(f => f.LibraryPathId == libraryPathId)).ShouldBe(1);
|
||||
}
|
||||
|
||||
// ersatztv#491 / H1: when the lookup matches more than one row (legacy duplicates that predate the
|
||||
// unique index, which is exactly the state the migration cleans up), it must resolve deterministically
|
||||
// to the lowest Id. An unordered FirstOrDefault can return a different row as the plan changes — and
|
||||
// adding the composite index alone can flip it — which would make the PathHash heal non-idempotent:
|
||||
// each scan would heal a different row and the second would collide on (LibraryPathId, PathHash).
|
||||
[Test]
|
||||
public async Task GetOrAddFolder_Should_Resolve_Legacy_Duplicates_Deterministically()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
|
||||
|
||||
int firstId;
|
||||
await using (TvContext seed = _db.CreateContext())
|
||||
{
|
||||
var a = new LibraryFolder { LibraryPathId = libraryPathId, Path = "/data/music/artist1" };
|
||||
var b = new LibraryFolder { LibraryPathId = libraryPathId, Path = "/data/music/artist1" };
|
||||
await seed.LibraryFolders.AddRangeAsync(a, b);
|
||||
await seed.SaveChangesAsync();
|
||||
firstId = Math.Min(a.Id, b.Id);
|
||||
}
|
||||
|
||||
// repeated calls must agree, and must agree with MIN(Id) — the same row the migration keeps
|
||||
LibraryFolder first = await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
|
||||
LibraryFolder second = await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
|
||||
|
||||
first.Id.ShouldBe(firstId);
|
||||
second.Id.ShouldBe(firstId);
|
||||
|
||||
await using TvContext context = _db.CreateContext();
|
||||
|
||||
// exactly one of the two got the hash — the heal is idempotent, not alternating
|
||||
(await context.LibraryFolders.CountAsync(
|
||||
f => f.LibraryPathId == libraryPathId && f.PathHash != null)).ShouldBe(1);
|
||||
LibraryFolder healed = await context.LibraryFolders.SingleAsync(f => f.Id == firstId);
|
||||
healed.PathHash.ShouldBe(PathUtils.GetPathHash("/data/music/artist1"));
|
||||
}
|
||||
|
||||
// ersatztv#491 / H1: on MySQL the lookup's SQL equality is case-INsensitive (longtext under the
|
||||
// default collation), so it hands back BOTH "/x/Foo" and "/x/foo" for a "/x/foo" scan — verified on a
|
||||
// real MySQL 8.4 server, where an unordered LIMIT 1 returns "/x/Foo". PathHash is a case-SENSITIVE
|
||||
// hash, so identity has to be settled ordinally or the heal stamps the wrong path's hash onto a row.
|
||||
//
|
||||
// This pins that decision with no database at all, because no SQLite-backed test can: SQLite's `=` on
|
||||
// TEXT is binary, so the case-differing candidate never reaches the in-memory step. Feeding the
|
||||
// candidate list directly is the only way to exercise it in CI.
|
||||
[Test]
|
||||
public void ResolveExact_Should_Pick_The_Ordinal_Match_Not_A_Case_Variant()
|
||||
{
|
||||
var candidates = new List<LibraryFolder>
|
||||
{
|
||||
new() { Id = 10, Path = "/x/Foo" },
|
||||
new() { Id = 11, Path = "/x/foo" }
|
||||
};
|
||||
|
||||
LibraryRepository.ResolveExact(candidates, "/x/foo").Id.ShouldBe(11);
|
||||
LibraryRepository.ResolveExact(candidates, "/x/Foo").Id.ShouldBe(10);
|
||||
|
||||
// a spelling that matches nothing ordinally is absent, not "close enough"
|
||||
LibraryRepository.ResolveExact(candidates, "/x/FOO").ShouldBeNull();
|
||||
LibraryRepository.ResolveExact([], "/x/foo").ShouldBeNull();
|
||||
}
|
||||
|
||||
// MySQL's comparison is always case-insensitive and, on a PAD SPACE collation, also ignores trailing
|
||||
// spaces ('/x/foo' = '/x/foo ' is TRUE under utf8mb4_general_ci and utf8mb4_bin; 8.4's default
|
||||
// utf8mb4_0900_ai_ci is NO PAD, so this axis is server-dependent). Where it applies, the SQL narrowing
|
||||
// hands back trailing-space siblings too. Those are distinct directories on Linux and hash
|
||||
// differently, so the ordinal settle has to keep them apart — same defect class as the case variant,
|
||||
// second axis. Ordinal compares length first, so this holds; pin it.
|
||||
[Test]
|
||||
public void ResolveExact_Should_Distinguish_Paths_Differing_Only_In_Trailing_Space()
|
||||
{
|
||||
var candidates = new List<LibraryFolder>
|
||||
{
|
||||
new() { Id = 20, Path = "/x/foo" },
|
||||
new() { Id = 21, Path = "/x/foo " }
|
||||
};
|
||||
|
||||
LibraryRepository.ResolveExact(candidates, "/x/foo").Id.ShouldBe(20);
|
||||
LibraryRepository.ResolveExact(candidates, "/x/foo ").Id.ShouldBe(21);
|
||||
LibraryRepository.ResolveExact(candidates, "/x/foo ").ShouldBeNull();
|
||||
|
||||
// the two spellings must also hash differently, or the unique index would reject one of them and
|
||||
// the migration's decision to keep both would be wrong
|
||||
PathUtils.GetPathHash("/x/foo").ShouldNotBe(PathUtils.GetPathHash("/x/foo "));
|
||||
}
|
||||
|
||||
// Candidates arrive ordered by Id, and the first ordinal match wins — so true duplicates resolve to
|
||||
// MIN(Id), the same row the #491 migration keeps.
|
||||
[Test]
|
||||
public void ResolveExact_Should_Prefer_The_First_Candidate_On_A_True_Duplicate()
|
||||
{
|
||||
var candidates = new List<LibraryFolder>
|
||||
{
|
||||
new() { Id = 3, Path = "/x/Foo" },
|
||||
new() { Id = 7, Path = "/x/foo" },
|
||||
new() { Id = 9, Path = "/x/foo" }
|
||||
};
|
||||
|
||||
LibraryRepository.ResolveExact(candidates, "/x/foo").Id.ShouldBe(7);
|
||||
}
|
||||
|
||||
// End-to-end companion to the ResolveExact tests above. On SQLite this passes on pre-fix code too (the
|
||||
// SQL equality already excludes the case variant); its value is guarding the in-memory step against
|
||||
// later being relaxed to OrdinalIgnoreCase, and covering the heal/no-duplicate behaviour around it.
|
||||
[Test]
|
||||
public async Task GetOrAddFolder_Should_Not_Resolve_A_Folder_Differing_Only_In_Case()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
|
||||
|
||||
int upperId;
|
||||
await using (TvContext seed = _db.CreateContext())
|
||||
{
|
||||
var upper = new LibraryFolder { LibraryPathId = libraryPathId, Path = "/data/music/Foo" };
|
||||
var lower = new LibraryFolder { LibraryPathId = libraryPathId, Path = "/data/music/foo" };
|
||||
await seed.LibraryFolders.AddRangeAsync(upper, lower);
|
||||
await seed.SaveChangesAsync();
|
||||
upperId = upper.Id;
|
||||
}
|
||||
|
||||
LibraryFolder result = await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/foo");
|
||||
|
||||
result.Path.ShouldBe("/data/music/foo");
|
||||
result.Id.ShouldNotBe(upperId);
|
||||
result.PathHash.ShouldBe(PathUtils.GetPathHash("/data/music/foo"));
|
||||
|
||||
await using TvContext context = _db.CreateContext();
|
||||
|
||||
// the case-differing sibling must be untouched — no foreign hash stamped onto it
|
||||
LibraryFolder upperPersisted = await context.LibraryFolders.SingleAsync(f => f.Id == upperId);
|
||||
upperPersisted.Path.ShouldBe("/data/music/Foo");
|
||||
upperPersisted.PathHash.ShouldBeNull();
|
||||
|
||||
// and no duplicate was inserted for either spelling
|
||||
(await context.LibraryFolders.CountAsync(f => f.LibraryPathId == libraryPathId)).ShouldBe(2);
|
||||
}
|
||||
|
||||
// The heal is opportunistic maintenance on a hot scan path: if some other row already owns
|
||||
// (LibraryPathId, hash) — a legacy duplicate the migration's grouping could not see — it must leave
|
||||
// the row unhealed rather than abort the scan.
|
||||
[Test]
|
||||
public async Task GetOrAddFolder_Should_Not_Fail_The_Scan_When_The_PathHash_Heal_Collides()
|
||||
{
|
||||
int libraryPathId = await SeedLibraryPath("/data/music");
|
||||
var libraryPath = new LibraryPath { Id = libraryPathId, Path = "/data/music", LibraryFolders = null };
|
||||
string hash = PathUtils.GetPathHash("/data/music/artist1");
|
||||
|
||||
int legacyId;
|
||||
await using (TvContext seed = _db.CreateContext())
|
||||
{
|
||||
// a legacy row with a null hash, plus a squatter that already owns the hash it would heal to
|
||||
var legacy = new LibraryFolder { LibraryPathId = libraryPathId, Path = "/data/music/artist1" };
|
||||
var squatter = new LibraryFolder
|
||||
{
|
||||
LibraryPathId = libraryPathId, Path = "/data/music/squatter", PathHash = hash
|
||||
};
|
||||
await seed.LibraryFolders.AddRangeAsync(legacy, squatter);
|
||||
await seed.SaveChangesAsync();
|
||||
legacyId = legacy.Id;
|
||||
}
|
||||
|
||||
// must not throw — the scan continues and simply returns the folder
|
||||
LibraryFolder result = await _repository.GetOrAddFolder(libraryPath, Option<int>.None, "/data/music/artist1");
|
||||
|
||||
result.Id.ShouldBe(legacyId);
|
||||
|
||||
await using TvContext context = _db.CreateContext();
|
||||
LibraryFolder persisted = await context.LibraryFolders.SingleAsync(f => f.Id == legacyId);
|
||||
persisted.PathHash.ShouldBeNull(); // heal declined, not half-applied
|
||||
}
|
||||
|
||||
private async Task<int> SeedLibraryPath(string path)
|
||||
{
|
||||
await using TvContext context = _db.CreateContext();
|
||||
|
||||
@@ -1,206 +0,0 @@
|
||||
using ErsatzTV.Application.Search.Queries;
|
||||
using ErsatzTV.Core.Api.Search;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using ErsatzTV.Infrastructure.MySql.Data;
|
||||
using ErsatzTV.Infrastructure.Sqlite.Data;
|
||||
using LanguageExt;
|
||||
using Microsoft.EntityFrameworkCore;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using MySqlConnector;
|
||||
using NUnit.Framework;
|
||||
using Shouldly;
|
||||
|
||||
namespace ErsatzTV.Tests.Integration;
|
||||
|
||||
/// <summary>
|
||||
/// ersatztv#668, EXECUTED on both providers. The bug was a collation/fold difference, so it lives exactly
|
||||
/// where a single-provider test cannot see it: SQLite's <c>LOWER()</c> folds ASCII only and UNDER-matched
|
||||
/// a stored <c>Édith</c>, while MySQL's is Unicode-aware and reaches it unaided. (Its column collation
|
||||
/// is accent-INsensitive, but the executed comparison is not — see the method docstring below.)
|
||||
/// <para>
|
||||
/// <see cref="ErsatzTV.Tests.Application.Search.GetSearchFieldValuesHandlerTests" /> covers the
|
||||
/// SQLite semantics in depth against in-memory SQLite, and
|
||||
/// <c>SearchFieldValuesQueryShapeTests</c> pins the generated SQL for both providers without a
|
||||
/// server. Neither can show that a REAL MySQL server returns the accented value — the fix's central
|
||||
/// claim is "on both providers", and on MySQL that rests on the server's Unicode-aware
|
||||
/// <c>LOWER()</c> rather than on any code this repo owns — explicitly NOT on its collation, which
|
||||
/// the executed comparison bypasses. That is precisely the kind of assumption worth executing.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// MySQL needs a live server via <c>ETV_TEST_MYSQL_CONNECTION</c>. Without it the MySQL fixture
|
||||
/// <b>ignores</b> — a visible skip, never a silent pass. Setting <c>ETV_REQUIRE_MYSQL_TESTS=1</c>
|
||||
/// turns that skip into a hard failure, so an ARMED lane cannot degrade into "connected to nothing
|
||||
/// and passed".
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>CI does not currently arm it</b>, so in CI this half SKIPS. Running MySQL fixtures against the
|
||||
/// live service was implemented and then removed as non-deterministic — see the note in
|
||||
/// <c>.gitea/workflows/docker-build.yml</c>; re-arming is tracked by ersatztv#627. Do not read the
|
||||
/// REQUIRE variable above as a guarantee that something enforces this today: nothing does. This
|
||||
/// mirrors <see cref="LibraryFolderDedupeMigrationTests" /> deliberately; the two fixtures share the
|
||||
/// contract, not code, because their setup needs differ.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
[TestFixture(TestProvider.Sqlite)]
|
||||
[TestFixture(TestProvider.MySql)]
|
||||
[NonParallelizable]
|
||||
public class SearchFieldValuesProviderTests(TestProvider provider)
|
||||
{
|
||||
private const string MySqlConnectionVariable = "ETV_TEST_MYSQL_CONNECTION";
|
||||
private const string MySqlRequiredVariable = "ETV_REQUIRE_MYSQL_TESTS";
|
||||
|
||||
private string _databasePath = null!;
|
||||
private string? _mySqlConnectionString;
|
||||
private DbContextOptions<TvContext> _options = null!;
|
||||
|
||||
[SetUp]
|
||||
public async Task SetUp()
|
||||
{
|
||||
if (provider is TestProvider.Sqlite)
|
||||
{
|
||||
TvContext.IsSqlite = true;
|
||||
TvContext.LastInsertedRowId = "last_insert_rowid()";
|
||||
TvContext.CaseInsensitiveCollation = "NOCASE";
|
||||
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
|
||||
TvContext.RegisterUnicodeCaseFunctions = SqliteUnicodeFunctions.Register;
|
||||
|
||||
_databasePath = Path.Combine(Path.GetTempPath(), $"etv668-{Guid.NewGuid():N}.sqlite3");
|
||||
_options = new DbContextOptionsBuilder<TvContext>()
|
||||
.UseSqlite($"Data Source={_databasePath}")
|
||||
.Options;
|
||||
}
|
||||
else
|
||||
{
|
||||
string? baseConnectionString = Environment.GetEnvironmentVariable(MySqlConnectionVariable);
|
||||
if (string.IsNullOrWhiteSpace(baseConnectionString))
|
||||
{
|
||||
string message =
|
||||
$"{MySqlConnectionVariable} is not set, so the MySql half of the #668 facet-value fixture "
|
||||
+ "cannot run. This endpoint's correctness is collation-dependent and therefore "
|
||||
+ "provider-specific, so the coverage is not optional in CI.";
|
||||
|
||||
if (IsTrue(Environment.GetEnvironmentVariable(MySqlRequiredVariable)))
|
||||
{
|
||||
Assert.Fail($"{message} {MySqlRequiredVariable} is set, so this is a failure, not a skip.");
|
||||
}
|
||||
|
||||
Assert.Ignore($"{message} Set it to run this locally.");
|
||||
}
|
||||
|
||||
// A database of our own with a name that has never been used, so isolation does not depend on a
|
||||
// wipe succeeding. Dropped and its pool cleared in TearDown.
|
||||
_mySqlConnectionString =
|
||||
new MySqlConnectionStringBuilder(baseConnectionString) { Database = $"etv668_{Guid.NewGuid():N}" }
|
||||
.ConnectionString;
|
||||
|
||||
TvContext.IsSqlite = false;
|
||||
TvContext.LastInsertedRowId = "last_insert_id()";
|
||||
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
|
||||
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
|
||||
|
||||
// Explicitly the no-op: MySQL's own LOWER() is Unicode-aware, so the handler must reach the
|
||||
// accented value WITHOUT any custom fold. Wiring SQLite's here would mask that.
|
||||
TvContext.RegisterUnicodeCaseFunctions = static _ => { };
|
||||
|
||||
_options = new DbContextOptionsBuilder<TvContext>()
|
||||
.UseMySql(_mySqlConnectionString, ServerVersion.AutoDetect(_mySqlConnectionString))
|
||||
.Options;
|
||||
}
|
||||
|
||||
// Schema creation deliberately does NOT happen here: NUnit skips [TearDown] when [SetUp] throws, so
|
||||
// a failure part-way through EnsureCreatedAsync would strand the created database (and its pooled
|
||||
// connection) with nothing to drop it. The test body creates it instead, matching the sibling
|
||||
// fixture, whose SetUp likewise cannot strand one.
|
||||
}
|
||||
|
||||
[TearDown]
|
||||
public async Task TearDown()
|
||||
{
|
||||
if (provider is TestProvider.Sqlite)
|
||||
{
|
||||
Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools();
|
||||
foreach (string path in new[] { _databasePath, $"{_databasePath}-wal", $"{_databasePath}-shm" })
|
||||
{
|
||||
if (File.Exists(path))
|
||||
{
|
||||
File.Delete(path);
|
||||
}
|
||||
}
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
if (_mySqlConnectionString is not null)
|
||||
{
|
||||
await using (TvContext context = Create(_options))
|
||||
{
|
||||
await context.Database.EnsureDeletedAsync();
|
||||
}
|
||||
|
||||
// MySqlConnector keys pools by connection string; a fresh database name means a fresh pool, and
|
||||
// leaving it uncleared leaks a server thread per test until max_connections is exhausted.
|
||||
await using var probe = new MySqlConnection(_mySqlConnectionString);
|
||||
await MySqlConnection.ClearPoolAsync(probe);
|
||||
|
||||
_mySqlConnectionString = null;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// The #668 headline, executed: a stored value whose prefix carries an UPPERCASE non-ASCII character
|
||||
/// is reachable from both casings of the query, on whichever provider this fixture is running.
|
||||
/// <para>
|
||||
/// Negative controls: "Zulu" (trivially unrelated) and "Edith" (unaccented, the near miss).
|
||||
/// <b>Be precise about what "Edith" does and does not prove.</b> It was added expecting MySQL to
|
||||
/// OVER-match it — the column collation is <c>utf8mb4_0900_ai_ci</c>, so <c>é</c> equals <c>e</c>
|
||||
/// — which would have made the in-memory ordinal filter load-bearing here. Measured against a
|
||||
/// live 8.4 server, it does not: deleting that filter leaves this test green, because the driver
|
||||
/// binds the LIKE pattern with a BINARY collation and the executed comparison is therefore
|
||||
/// accent-SENSITIVE. (A literal pattern typed by hand DOES over-match — a different query from
|
||||
/// the one the handler runs.) So the row pins the accent-sensitive result on both providers and
|
||||
/// documents the near miss; it does NOT exercise an over-match correction, because with the
|
||||
/// CURRENT driver there is nothing to correct. That is a driver-contingent fact, not a law: a
|
||||
/// driver or protocol change that made the pattern ci-collated would restore the over-match, and
|
||||
/// the ordinal filter — which stays regardless — would then be doing real work here.
|
||||
/// </para>
|
||||
/// </summary>
|
||||
[TestCase("é", TestName = "Uppercase_Accent_Reachable_From_Lowercase_Query")]
|
||||
[TestCase("É", TestName = "Uppercase_Accent_Reachable_From_Uppercase_Query")]
|
||||
public async Task Stored_Uppercase_Accent_Is_Reachable(string query)
|
||||
{
|
||||
await using (TvContext context = Create(_options))
|
||||
{
|
||||
await context.Database.EnsureCreatedAsync();
|
||||
context.Set<Genre>().AddRange(
|
||||
new Genre { Name = "Édith" },
|
||||
new Genre { Name = "Edith" },
|
||||
new Genre { Name = "Zulu" });
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
var handler = new GetSearchFieldValuesHandler(new TestDbContextFactory(_options));
|
||||
|
||||
Option<SearchFieldValuesResponseModel> result = await handler.Handle(
|
||||
new GetSearchFieldValues("genre", query, 50),
|
||||
CancellationToken.None);
|
||||
|
||||
result.IsSome.ShouldBeTrue();
|
||||
result.IfSome(r => r.Values.ShouldBe(new List<string> { "Édith" }));
|
||||
}
|
||||
|
||||
private static bool IsTrue(string? value) =>
|
||||
value is "1" || string.Equals(value, "true", StringComparison.OrdinalIgnoreCase);
|
||||
|
||||
private static TvContext Create(DbContextOptions<TvContext> options) =>
|
||||
new(
|
||||
options,
|
||||
NullLoggerFactory.Instance,
|
||||
new SlowQueryInterceptor(NullLogger<SlowQueryInterceptor>.Instance));
|
||||
|
||||
private sealed class TestDbContextFactory(DbContextOptions<TvContext> options) : IDbContextFactory<TvContext>
|
||||
{
|
||||
public TvContext CreateDbContext() => Create(options);
|
||||
}
|
||||
}
|
||||
@@ -31,7 +31,6 @@ public sealed class InMemoryTvContext : IAsyncDisposable
|
||||
{
|
||||
TvContext.IsSqlite = true;
|
||||
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
|
||||
TvContext.RegisterUnicodeCaseFunctions = SqliteUnicodeFunctions.Register;
|
||||
|
||||
var connection = new SqliteConnection("Data Source=:memory:;Foreign Keys=False");
|
||||
await connection.OpenAsync();
|
||||
|
||||
@@ -1,212 +0,0 @@
|
||||
using ErsatzTV.Controllers.Api.Requests;
|
||||
using ErsatzTV.Core.Domain;
|
||||
using ErsatzTV.Infrastructure.Data;
|
||||
using NUnit.Framework;
|
||||
|
||||
namespace ErsatzTV.Tests.Support;
|
||||
|
||||
/// <summary>
|
||||
/// One selection-type matrix, shared by every fixture that exercises a tagged-union selection
|
||||
/// (rerun collections and playlist items). Both consumers of
|
||||
/// <c>MediaCollections.Mapper.ProjectMediaItemToViewModel</c> are proved against the SAME data, so
|
||||
/// widening the shared switch cannot be discharged for the second consumer by inspection alone —
|
||||
/// which is the method that produced #671 in the first place.
|
||||
/// </summary>
|
||||
internal static class SelectionSeedData
|
||||
{
|
||||
public const int SelectedId = 42;
|
||||
|
||||
/// <summary>
|
||||
/// Derived from production rather than hand-listed, so a newly-supported type joins the matrix
|
||||
/// automatically and trips the <c>default:</c> arms below until someone teaches them about it.
|
||||
/// Note this is the RERUN-COLLECTION predicate, used for playlist items as a deliberate
|
||||
/// SUPERSET: <c>ReplacePlaylistItemsHandler.CollectionTypeMustBeValid</c> has no
|
||||
/// <c>RemoteStream</c> case, so a RemoteStream playlist item cannot be created through the write
|
||||
/// API today and the playlist fixture seeds that row directly. Covering it is forward-looking,
|
||||
/// not a claim that the two sets are equivalent — split this if they ever legitimately diverge.
|
||||
/// </summary>
|
||||
public static IEnumerable<CollectionType> SupportedSelectionTypes =>
|
||||
Enum.GetValues<CollectionType>().Where(RerunCollectionRequestMapping.IsSupportedSelectionType);
|
||||
|
||||
/// <summary>
|
||||
/// The exact projected name per type. Pinning the whole string — rather than merely asserting
|
||||
/// "not a placeholder" — is what makes a missing NESTED include leg visible: dropping
|
||||
/// Episode → Season → Show still yields the placeholder-free "s??e04 - Selected episode", and
|
||||
/// dropping MusicVideo → Artist still yields "Selected music video". Both would sail past a
|
||||
/// looser assertion while having lost real data.
|
||||
/// </summary>
|
||||
public static string ExpectedName(CollectionType collectionType) =>
|
||||
collectionType switch
|
||||
{
|
||||
CollectionType.Collection => "Selected collection",
|
||||
CollectionType.MultiCollection => "Selected multi collection",
|
||||
CollectionType.SmartCollection => "Selected smart collection",
|
||||
CollectionType.TelevisionShow => "Selected show (2020)",
|
||||
CollectionType.TelevisionSeason => "Parent show (2020) - Season 3",
|
||||
CollectionType.Artist => "Selected artist",
|
||||
CollectionType.Movie => "Selected movie (2019)",
|
||||
CollectionType.Episode => "Episode's show - s02e04 - Selected episode",
|
||||
CollectionType.MusicVideo => "Video's artist - Selected music video",
|
||||
CollectionType.OtherVideo => "Selected other video",
|
||||
CollectionType.Song => "Song artist - Selected song",
|
||||
CollectionType.Image => "Selected image",
|
||||
CollectionType.RemoteStream => "Selected remote stream",
|
||||
_ => throw new AssertionException($"No expected name pinned for {collectionType}")
|
||||
};
|
||||
|
||||
public static async Task SeedSelection(TvContext context, CollectionType collectionType)
|
||||
{
|
||||
switch (collectionType)
|
||||
{
|
||||
case CollectionType.Collection:
|
||||
context.Collections.Add(new Collection
|
||||
{
|
||||
Id = SelectedId,
|
||||
Name = "Selected collection",
|
||||
MediaItems = []
|
||||
});
|
||||
break;
|
||||
case CollectionType.MultiCollection:
|
||||
context.MultiCollections.Add(new MultiCollection
|
||||
{
|
||||
Id = SelectedId,
|
||||
Name = "Selected multi collection"
|
||||
});
|
||||
break;
|
||||
case CollectionType.SmartCollection:
|
||||
context.SmartCollections.Add(new SmartCollection
|
||||
{
|
||||
Id = SelectedId,
|
||||
Name = "Selected smart collection",
|
||||
Query = "tag:family"
|
||||
});
|
||||
break;
|
||||
case CollectionType.TelevisionShow:
|
||||
context.Shows.Add(new Show
|
||||
{
|
||||
Id = SelectedId,
|
||||
ShowMetadata = [new ShowMetadata { Title = "Selected show", Year = 2020 }]
|
||||
});
|
||||
break;
|
||||
case CollectionType.TelevisionSeason:
|
||||
context.Seasons.Add(new Season
|
||||
{
|
||||
Id = SelectedId,
|
||||
SeasonNumber = 3,
|
||||
Show = new Show
|
||||
{
|
||||
Id = 900,
|
||||
ShowMetadata = [new ShowMetadata { Title = "Parent show", Year = 2020 }]
|
||||
}
|
||||
});
|
||||
break;
|
||||
case CollectionType.Artist:
|
||||
context.Artists.Add(new Artist
|
||||
{
|
||||
Id = SelectedId,
|
||||
ArtistMetadata = [new ArtistMetadata { Title = "Selected artist" }]
|
||||
});
|
||||
break;
|
||||
case CollectionType.Movie:
|
||||
context.Movies.Add(new Movie
|
||||
{
|
||||
Id = SelectedId,
|
||||
MovieMetadata = [new MovieMetadata { Title = "Selected movie", Year = 2019 }]
|
||||
});
|
||||
break;
|
||||
case CollectionType.Episode:
|
||||
context.Episodes.Add(new Episode
|
||||
{
|
||||
Id = SelectedId,
|
||||
EpisodeMetadata = [new EpisodeMetadata { Title = "Selected episode", EpisodeNumber = 4 }],
|
||||
Season = new Season
|
||||
{
|
||||
Id = 901,
|
||||
SeasonNumber = 2,
|
||||
Show = new Show
|
||||
{
|
||||
Id = 902,
|
||||
ShowMetadata = [new ShowMetadata { Title = "Episode's show", Year = 2018 }]
|
||||
}
|
||||
}
|
||||
});
|
||||
break;
|
||||
case CollectionType.MusicVideo:
|
||||
context.MusicVideos.Add(new MusicVideo
|
||||
{
|
||||
Id = SelectedId,
|
||||
MusicVideoMetadata = [new MusicVideoMetadata { Title = "Selected music video" }],
|
||||
Artist = new Artist
|
||||
{
|
||||
Id = 903,
|
||||
ArtistMetadata = [new ArtistMetadata { Title = "Video's artist" }]
|
||||
}
|
||||
});
|
||||
break;
|
||||
case CollectionType.OtherVideo:
|
||||
context.OtherVideos.Add(new OtherVideo
|
||||
{
|
||||
Id = SelectedId,
|
||||
OtherVideoMetadata = [new OtherVideoMetadata { Title = "Selected other video" }]
|
||||
});
|
||||
break;
|
||||
case CollectionType.Song:
|
||||
context.Songs.Add(new Song
|
||||
{
|
||||
Id = SelectedId,
|
||||
SongMetadata =
|
||||
[new SongMetadata { Title = "Selected song", Artists = ["Song artist"] }]
|
||||
});
|
||||
break;
|
||||
case CollectionType.Image:
|
||||
context.Images.Add(new Image
|
||||
{
|
||||
Id = SelectedId,
|
||||
ImageMetadata = [new ImageMetadata { Title = "Selected image" }]
|
||||
});
|
||||
break;
|
||||
case CollectionType.RemoteStream:
|
||||
context.RemoteStreams.Add(new RemoteStream
|
||||
{
|
||||
Id = SelectedId,
|
||||
Url = "http://example.invalid/stream",
|
||||
RemoteStreamMetadata = [new RemoteStreamMetadata { Title = "Selected remote stream" }]
|
||||
});
|
||||
break;
|
||||
default:
|
||||
throw new AssertionException(
|
||||
$"{collectionType} is a supported selection type but this suite does not know how " +
|
||||
"to seed it — teach SeedSelection about it rather than narrowing the matrix.");
|
||||
}
|
||||
|
||||
await context.SaveChangesAsync();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Assigns the one foreign key the tagged union uses for this type. Shared so the rerun and
|
||||
/// playlist fixtures cannot disagree about which slot a type occupies.
|
||||
/// </summary>
|
||||
public static void ApplySelection(
|
||||
CollectionType collectionType,
|
||||
Action<int> setCollectionId,
|
||||
Action<int> setMultiCollectionId,
|
||||
Action<int> setSmartCollectionId,
|
||||
Action<int> setMediaItemId)
|
||||
{
|
||||
switch (collectionType)
|
||||
{
|
||||
case CollectionType.Collection:
|
||||
setCollectionId(SelectedId);
|
||||
break;
|
||||
case CollectionType.MultiCollection:
|
||||
setMultiCollectionId(SelectedId);
|
||||
break;
|
||||
case CollectionType.SmartCollection:
|
||||
setSmartCollectionId(SelectedId);
|
||||
break;
|
||||
default:
|
||||
setMediaItemId(SelectedId);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,3 @@
|
||||
using System.ComponentModel;
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
using System.Threading.Channels;
|
||||
using ErsatzTV.Application;
|
||||
@@ -265,12 +264,8 @@ public class ChannelController(
|
||||
public async Task<PagedLibraryBrowseItemsResponseModel> GetAutoTuneChannelMembers(
|
||||
[FromQuery] AutoTuneAxis axis,
|
||||
[FromQuery] string value,
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum,
|
||||
[FromQuery]
|
||||
[Description("Rows per page; capped at 200 for this endpoint. A value of 0 or less falls back to 100 rather than being clamped to 1. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize,
|
||||
[FromQuery] int pageNum,
|
||||
[FromQuery] int pageSize,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
pageNum = Math.Max(0, pageNum);
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
using System.ComponentModel;
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
using ErsatzTV.Application.MediaCollections;
|
||||
using ErsatzTV.Controllers.Api.Requests;
|
||||
@@ -47,12 +46,8 @@ public class CollectionController(IMediator mediator) : ControllerBase
|
||||
[ProducesResponseType(typeof(ProblemDetails), StatusCodes.Status404NotFound)]
|
||||
public async Task<IActionResult> GetItems(
|
||||
int id,
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
int clampedPageNum = Math.Max(0, pageNum);
|
||||
|
||||
@@ -21,12 +21,8 @@ public class LibraryBrowseController(IMediator mediator) : ControllerBase
|
||||
[FromQuery] string query = "",
|
||||
[FromQuery] int? libraryId = null,
|
||||
[FromQuery] LibraryBrowseMediaType? mediaType = null,
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
[FromQuery]
|
||||
[Description("Parent id for a drill-in listing; only used with mediaType=TelevisionSeason (that show's seasons), mediaType=Episode (that season's episodes) or mediaType=MusicVideo (that artist's music videos), ignored otherwise")]
|
||||
int? parentId = null,
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
using System.ComponentModel;
|
||||
using System.Linq.Expressions;
|
||||
using ErsatzTV.Application.Logs;
|
||||
using ErsatzTV.Core.Api.Logs;
|
||||
@@ -30,12 +29,8 @@ public class LogsController(IMediator mediator) : ControllerBase
|
||||
[EndpointGroupName("general")]
|
||||
[ProducesResponseType(typeof(PagedLogEntriesResponseModel), StatusCodes.Status200OK)]
|
||||
public async Task<PagedLogEntriesResponseModel> GetLogs(
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
[FromQuery] string filter = "",
|
||||
[FromQuery] string sortField = "timestamp",
|
||||
[FromQuery] string sortDirection = "desc",
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
using System.ComponentModel;
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
using ErsatzTV.Application.MediaCollections;
|
||||
using ErsatzTV.Controllers.Api.Requests;
|
||||
@@ -23,12 +22,8 @@ public class MultiCollectionController(IMediator mediator) : ControllerBase
|
||||
[ProducesResponseType(typeof(PagedMultiCollectionsResponseModel), StatusCodes.Status200OK)]
|
||||
public async Task<PagedMultiCollectionsResponseModel> GetAll(
|
||||
[FromQuery] string query = "",
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
int clampedPageNum = Math.Max(0, pageNum);
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
using System.ComponentModel;
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
using ErsatzTV.Application.Playouts;
|
||||
using ErsatzTV.Application.ProgramSchedules;
|
||||
@@ -42,12 +41,8 @@ public class PlayoutController(IMediator mediator, IEntityLocker entityLocker) :
|
||||
[ProducesResponseType(typeof(PagedPlayoutsResponseModel), StatusCodes.Status200OK)]
|
||||
public async Task<PagedPlayoutsResponseModel> GetAll(
|
||||
[FromQuery] string query = "",
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
pageNum = Math.Max(0, pageNum);
|
||||
@@ -88,12 +83,8 @@ public class PlayoutController(IMediator mediator, IEntityLocker entityLocker) :
|
||||
public async Task<IActionResult> GetItems(
|
||||
int id,
|
||||
[FromQuery] bool showFiller = false,
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
Option<PlayoutNameViewModel> maybePlayout = await mediator.Send(new GetPlayoutById(id), cancellationToken);
|
||||
@@ -552,12 +543,8 @@ public class PlayoutController(IMediator mediator, IEntityLocker entityLocker) :
|
||||
public async Task<IActionResult> GetBlockHistory(
|
||||
int id,
|
||||
int blockId,
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
Option<PlayoutNameViewModel> maybePlayout = await mediator.Send(new GetPlayoutById(id), cancellationToken);
|
||||
@@ -831,7 +818,6 @@ public class PlayoutController(IMediator mediator, IEntityLocker entityLocker) :
|
||||
vm.ScheduleKind,
|
||||
vm.ChannelName,
|
||||
vm.ChannelNumber,
|
||||
vm.ChannelId,
|
||||
vm.PlayoutMode,
|
||||
vm.ScheduleName,
|
||||
vm.ScheduleFile,
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
using System.ComponentModel;
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
using ErsatzTV.Application.MediaCollections;
|
||||
using ErsatzTV.Controllers.Api.Requests;
|
||||
@@ -23,12 +22,8 @@ public class RerunCollectionController(IMediator mediator) : ControllerBase
|
||||
[ProducesResponseType(typeof(PagedRerunCollectionsResponseModel), StatusCodes.Status200OK)]
|
||||
public async Task<PagedRerunCollectionsResponseModel> GetAll(
|
||||
[FromQuery] string query = "",
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
int clampedPageNum = Math.Max(0, pageNum);
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
using System.ComponentModel;
|
||||
using ErsatzTV.Application.MediaCollections;
|
||||
using ErsatzTV.Application.MediaItems;
|
||||
using ErsatzTV.Application.Search;
|
||||
@@ -36,12 +35,8 @@ public class SearchController(IMediator mediator) : ControllerBase
|
||||
[ProducesResponseType(typeof(ProblemDetails), StatusCodes.Status422UnprocessableEntity)]
|
||||
public async Task<IActionResult> Search(
|
||||
[FromQuery] string query = "",
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 50); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 50,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 50,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(query))
|
||||
@@ -70,12 +65,8 @@ public class SearchController(IMediator mediator) : ControllerBase
|
||||
[ProducesResponseType(typeof(ProblemDetails), StatusCodes.Status422UnprocessableEntity)]
|
||||
public async Task<IActionResult> SearchAllItems(
|
||||
[FromQuery] string query = "",
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0. Unlike the other paged endpoints this one is also bounded ABOVE, at 2000000, so that pageNum * pageSize cannot overflow; a larger value is clamped down to that maximum rather than rejected.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 500); capped at 1000 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = DefaultAllItemsPageSize,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = DefaultAllItemsPageSize,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(query))
|
||||
@@ -205,11 +196,7 @@ public class SearchController(IMediator mediator) : ControllerBase
|
||||
"Returns distinct whole values from the database for the given text field, filtered by an " +
|
||||
"optional case-insensitive prefix. Powers the visual rule builder's facet-value typeahead. " +
|
||||
"404 when the field is unknown, is not a text field, or is a text field with no distinct-value " +
|
||||
"source. The final filter, dedup and ordering applied to the response are ordinal and not " +
|
||||
"culture-dependent; note that fields sourced by a plain database query are additionally " +
|
||||
"pre-filtered by the database collation first, which on SQLite is ASCII-only. The list-valued " +
|
||||
"music fields (artist, album_artist) are bounded best-effort: the server reads a bounded number " +
|
||||
"of song rows per request, so a library larger than that bound may yield a subset of the matches.")]
|
||||
"source.")]
|
||||
[EndpointGroupName("general")]
|
||||
[ProducesResponseType(typeof(SearchFieldValuesResponseModel), StatusCodes.Status200OK)]
|
||||
[ProducesResponseType(typeof(ProblemDetails), StatusCodes.Status404NotFound)]
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
using System.ComponentModel;
|
||||
using System.ComponentModel.DataAnnotations;
|
||||
using System.Text.RegularExpressions;
|
||||
using System.Threading.Channels;
|
||||
@@ -29,12 +28,8 @@ public partial class TraktController(
|
||||
[EndpointGroupName("general")]
|
||||
[ProducesResponseType(typeof(PagedTraktListsResponseModel), StatusCodes.Status200OK)]
|
||||
public async Task<PagedTraktListsResponseModel> GetAll(
|
||||
[FromQuery]
|
||||
[Description("0-based page index: the first page is 0, not 1. A negative value is clamped to 0.")]
|
||||
int pageNum = 0,
|
||||
[FromQuery]
|
||||
[Description("Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.")]
|
||||
int pageSize = 100,
|
||||
[FromQuery] int pageNum = 0,
|
||||
[FromQuery] int pageSize = 100,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
int clampedPageNum = Math.Max(0, pageNum);
|
||||
|
||||
@@ -649,7 +649,6 @@ public class Startup
|
||||
TvContext.LastInsertedRowId = "last_insert_rowid()";
|
||||
TvContext.CaseInsensitiveCollation = "NOCASE";
|
||||
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
|
||||
TvContext.RegisterUnicodeCaseFunctions = SqliteUnicodeFunctions.Register;
|
||||
|
||||
SqlMapper.AddTypeHandler(new DateTimeOffsetHandler());
|
||||
SqlMapper.AddTypeHandler(new GuidHandler());
|
||||
@@ -661,10 +660,6 @@ public class Startup
|
||||
TvContext.LastInsertedRowId = "last_insert_id()";
|
||||
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
|
||||
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
|
||||
|
||||
// MySQL's LOWER() is already Unicode-aware, so the facet-value handler never takes the
|
||||
// custom-fold branch here; assigned explicitly so a provider switch cannot inherit SQLite's.
|
||||
TvContext.RegisterUnicodeCaseFunctions = static _ => { };
|
||||
}
|
||||
|
||||
Log.Logger.Information("Transcode folder is {Folder}", FileSystemLayout.TranscodeFolder);
|
||||
|
||||
@@ -2578,7 +2578,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32"
|
||||
@@ -2587,7 +2586,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page; capped at 200 for this endpoint. A value of 0 or less falls back to 100 rather than being clamped to 1. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32"
|
||||
@@ -3865,7 +3863,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -3875,7 +3872,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -9389,7 +9385,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -9399,7 +9394,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -10160,7 +10154,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -10170,7 +10163,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -10768,7 +10760,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -10778,7 +10769,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -12489,7 +12479,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -12499,7 +12488,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -13087,7 +13075,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -13097,7 +13084,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -14054,7 +14040,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -14064,7 +14049,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -15549,7 +15533,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -15559,7 +15542,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -17369,7 +17351,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -17379,7 +17360,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 50); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -17474,7 +17454,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0. Unlike the other paged endpoints this one is also bounded ABOVE, at 2000000, so that pageNum * pageSize cannot overflow; a larger value is clamped down to that maximum rather than rejected.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -17484,7 +17463,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 500); capped at 1000 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -18063,7 +18041,7 @@
|
||||
"Search"
|
||||
],
|
||||
"summary": "List distinct database values for a text search field",
|
||||
"description": "Returns distinct whole values from the database for the given text field, filtered by an optional case-insensitive prefix. Powers the visual rule builder's facet-value typeahead. 404 when the field is unknown, is not a text field, or is a text field with no distinct-value source. The final filter, dedup and ordering applied to the response are ordinal and not culture-dependent; note that fields sourced by a plain database query are additionally pre-filtered by the database collation first, which on SQLite is ASCII-only. The list-valued music fields (artist, album_artist) are bounded best-effort: the server reads a bounded number of song rows per request, so a library larger than that bound may yield a subset of the matches.",
|
||||
"description": "Returns distinct whole values from the database for the given text field, filtered by an optional case-insensitive prefix. Powers the visual rule builder's facet-value typeahead. 404 when the field is unknown, is not a text field, or is a text field with no distinct-value source.",
|
||||
"operationId": "GetSearchFieldValues",
|
||||
"parameters": [
|
||||
{
|
||||
@@ -21064,7 +21042,6 @@
|
||||
{
|
||||
"name": "pageNum",
|
||||
"in": "query",
|
||||
"description": "0-based page index: the first page is 0, not 1. A negative value is clamped to 0.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -21074,7 +21051,6 @@
|
||||
{
|
||||
"name": "pageSize",
|
||||
"in": "query",
|
||||
"description": "Rows per page (default 100); capped at 100 for this endpoint. The page offset is derived from the effective (capped) size, so a larger value narrows the page instead of widening the offset.",
|
||||
"schema": {
|
||||
"type": "integer",
|
||||
"format": "int32",
|
||||
@@ -28978,7 +28954,6 @@
|
||||
"scheduleKind",
|
||||
"channelName",
|
||||
"channelNumber",
|
||||
"channelId",
|
||||
"playoutMode",
|
||||
"scheduleName",
|
||||
"scheduleFile",
|
||||
@@ -29004,10 +28979,6 @@
|
||||
"channelNumber": {
|
||||
"type": "string"
|
||||
},
|
||||
"channelId": {
|
||||
"type": "integer",
|
||||
"format": "int32"
|
||||
},
|
||||
"playoutMode": {
|
||||
"$ref": "#/components/schemas/ChannelPlayoutMode"
|
||||
},
|
||||
|
||||
+4
-8
@@ -28,7 +28,7 @@ doc below, or that changes which sections a task signal points to.**
|
||||
| Adding/changing a UI-E2E browser flow | `docs/e2e-local.md` → "UI-E2E harness" + `scripts/e2e-ui.sh` |
|
||||
| What does a test suite cover | `docs/testing.md` |
|
||||
| Legacy Blazor route lookup | `docs/blazor-route-parity.md` (historical #91 phase (b) inventory) |
|
||||
| "Why do we do X this way" / challenging a convention | **Catalog-first**: `docs/decisions/README.md` (active rows) → follow the row's link to `docs/decisions/records/<area>/<topic>.md` for full rationale. `docs/decisions/archive/<area>/` only for "what did the rule used to be." |
|
||||
| "Why do we do X this way" / challenging a convention | **Catalog-first**: `docs/decisions/README.md` (active rows) → `docs/decisions.md` + `docs/decisions/*.md` for full rationale. `docs/decisions/archive/` only for "what did the rule used to be." |
|
||||
|
||||
## Knowledge retrieval (MemPalace + catalog + Gitea)
|
||||
|
||||
@@ -72,13 +72,9 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K
|
||||
- **`docs/blazor-route-parity.md`** — historical record of the completed #91 phase (b) cutover:
|
||||
the Blazor Server UI is removed and every legacy route now 302-redirects to its SPA equivalent
|
||||
(or falls through to the catch-all → `/app`). Read it for the full legacy→SPA route inventory.
|
||||
- **`docs/decisions/records/<area>/<topic>.md`** — one active decision record per file, YAML
|
||||
frontmatter (`key`/`title`/`status`/`since`/`supersedes`/`superseded-by`, plus optional
|
||||
`stale-after`/`sources` — ersatztv#603), rationale prose in the body. The **filename is the key**,
|
||||
so one-active-record-per-key is a filesystem property (ersatztv#610). `docs/decisions.md` and the
|
||||
topic files remain as the lifecycle-schema narrative plus a "Records formerly in this file" index,
|
||||
which is what keeps older date-based pointers resolvable. **Generated active view**:
|
||||
`docs/decisions/README.md`
|
||||
- **`docs/decisions.md`** + **`docs/decisions/*.md`** — active decision records (lifecycle schema:
|
||||
key/status/since/supersedes/superseded-by, plus optional `stale-after`/`Sources:` —
|
||||
ersatztv#603). **Generated active view**: `docs/decisions/README.md`
|
||||
(catalog / task router) — start there. Superseded/retired records live in
|
||||
`docs/decisions/archive/` and are read only for history, never for "what is the current rule."
|
||||
- **`docs/ci-cd.md`** — build/test/release pipeline, versioning, dependency management.
|
||||
|
||||
+2
-91
@@ -41,19 +41,7 @@ Exemplars:
|
||||
- **Paged GET with clamped params**: `ErsatzTV/Controllers/Api/LogsController.cs` —
|
||||
`pageNum` clamped via `Math.Max(0, pageNum)`, `pageSize` via `Math.Clamp(pageSize, 1, MaxPageSize)`
|
||||
(`MaxPageSize = 100`). Any new paged endpoint should clamp the same way — don't trust client
|
||||
input for page math. **`pageNum` is 0-based** across the whole surface (the first page is `0`) and
|
||||
the offset is always derived from the *clamped* `pageSize`, so an over-large `pageSize` yields
|
||||
narrower pages — it never widens the offset. Say "0-based" in the description of any paging
|
||||
parameter you expose, including on wrapper surfaces like the MCP tool catalog: describing it as
|
||||
1-based makes a caller skip the first page silently, which reads as data loss rather than as an
|
||||
off-by-one (ersatztv#616). Put that description on the parameter itself with
|
||||
`[Description("...")]` (`System.ComponentModel`, on the `[FromQuery]` parameter) so it reaches the
|
||||
generated OpenAPI document — an attribute-free paging parameter is emitted with no description at
|
||||
all, leaving a REST consumer to infer the base from `default: 0` (ersatztv#633). State the
|
||||
endpoint's **own** cap, never one global number: the caps differ (100 typical, 200 auto-tune
|
||||
members, 1000 `search/all-items`). `OpenApiPagingContractTests` pins this and names the expected
|
||||
set of paged operations, so a new paged endpoint fails until it is added there **with**
|
||||
descriptions. See `api.paging-zero-based`.
|
||||
input for page math.
|
||||
- **Sortable GET with allow-listed sort params**: same file — `sortField`/`sortDirection` are
|
||||
normalized against a fixed allow-list (`AllowedSortFields`) rather than trusted or rejected with
|
||||
a 422: an unrecognized `sortField` silently falls back to the default field, an unrecognized
|
||||
@@ -146,41 +134,6 @@ Exemplars:
|
||||
`Brief`. `Remediation.Kind` is a mapped **string** ("ExternalDoc"/"AppRoute"), not a wire enum —
|
||||
same pattern as `Status`. See `decisions.md` 2026-07-17 (#164).
|
||||
|
||||
### 2a. Flattening a tagged-union selection (read path)
|
||||
|
||||
Several DTOs flatten a "exactly one of these navigations is populated" tagged union to a single
|
||||
`selectedId` + `selectedName` pair (`RerunCollectionResponseModel`, and the playlist-item shape).
|
||||
Two rules, both learned from #671, where the list endpoint returned a null selection for **every**
|
||||
row and the detail GET 500'd for two of its media types:
|
||||
|
||||
- **One include chain per projected aggregate, shared by every handler that projects it.** Put it in
|
||||
a `<Aggregate>QueryExtensions` extension method and call it from the list handler *and* the by-id
|
||||
handler. Exemplars: `RerunCollectionQueryExtensions.IncludeSelectionDetails()`,
|
||||
`ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails()`. Two hand-maintained chains
|
||||
drift, and the one that drifts is usually the paged list, whose rows are individually less
|
||||
obviously wrong. Applying it before `Skip`/`Take` is fine — EF applies the includes to the paged
|
||||
subquery, so the cost is bounded by `PageSize`, not by the table.
|
||||
- **The id and the name must not share a single point of failure.** When both are read off the same
|
||||
eager-loaded navigation, the id is only ever as available as the name — so an un-included type
|
||||
doesn't merely render an unlabelled badge, it drops the selected id, and an editor that
|
||||
round-trips that id silently clears the user's stored selection. Accordingly a media-item
|
||||
flattening switch never ends in `_ => null`: an unrecognized subtype keeps its id and takes a
|
||||
conspicuous `[unsupported media type: X]` name. Throwing is the wrong lever — it would fail an
|
||||
entire paged GET over one unreadable row. The shared switch is
|
||||
`MediaCollections.Mapper.ProjectMediaItemToViewModel`.
|
||||
|
||||
Corollary for the mappers themselves: `MediaItems.Mapper`'s projections are reached from handlers
|
||||
whose include chains differ, so every metadata navigation is read through `Optional(...).Flatten()`
|
||||
and degrades to the `"???"` placeholder rather than throwing. A bare `x.Season.Show.ShowMetadata`
|
||||
inside a projection is a latent 500 on some other caller's GET.
|
||||
|
||||
**And it is not only navigations.** `SongMetadata.Artists` is a nullable EF *primitive collection*
|
||||
(a JSON array in one column), which `FallbackMetadataProvider` leaves unassigned for a song whose
|
||||
tags failed to read — and `string.Join` throws `ArgumentNullException` on a null sequence, not a
|
||||
`NullReferenceException`. Adding an include is therefore not automatically safe: it can promote a
|
||||
latent throw on a previously-unloaded member into a live 500 that fails the whole page. When you
|
||||
widen an include chain, audit what the newly-reachable projection dereferences.
|
||||
|
||||
## 3. Error mapping
|
||||
|
||||
Central helper: `ErsatzTV/Extensions/ApiResults.cs`. Use these extension methods instead of
|
||||
@@ -479,7 +432,7 @@ standard credential (catalog-read tier — no `[RequiresAuthentication]`):
|
||||
Query params: `q` (optional prefix filter, case-insensitive, default empty) and `limit` (optional,
|
||||
clamped `1..50`, default 50). `{name}` is allow-listed to `SearchFieldCatalog` fields with
|
||||
`type: "text"` AND a distinct-value source in the database — an unknown field, a non-text field (e.g.
|
||||
an enum), or a text field without a source (`title`, `show_title`) 404s rather than
|
||||
an enum), or a text field without a source (`title`, `show_title`, `album_artist`) 404s rather than
|
||||
returning an empty list, since enum fields already ship their values inline on
|
||||
`GET /api/v1/search/fields` and never need this endpoint. Returns `SearchFieldValuesResponseModel`
|
||||
(`{ values: string[] }`), sourced from a per-field distinct-values DB query (`IDbContextFactory<TvContext>`),
|
||||
@@ -487,48 +440,6 @@ not the Lucene term dictionary — analyzed text fields store lowercased word to
|
||||
No server-side caching. Powers the visual rule builder's value-input combobox for text fields; see
|
||||
`docs/decisions.md` 2026-07-23 (#434) and `spa-conventions.md` §12.
|
||||
|
||||
**Bounded best-effort for list-valued fields (#578)**: `artist` and `album_artist` are backed (wholly
|
||||
or partly) by `SongMetadata.Artists`/`AlbumArtists`, which EF maps as **primitive collections** — one
|
||||
JSON array per row in a single column, with no server-side projection on either provider. `album_artist`
|
||||
therefore no longer 404s, and `artist` now also covers free-text music-video (`MusicVideoArtist`) and
|
||||
song credits, not only entity artists. Those rows are read by a keyset page whose only condition is the
|
||||
**cursor** — no residual predicate that could discard a row — and filtered in memory, bounded at 20,000
|
||||
logical rows per request; so on a larger library the
|
||||
response may be a bounded subset of the matches — bounded in LOGICAL ROWS, which is not the same as
|
||||
bounded work or bytes. Say so in the `[EndpointDescription]` of any endpoint that adopts this shape.
|
||||
|
||||
Three rules generalize beyond this endpoint.
|
||||
|
||||
1. **`LIMIT` bounds the OUTPUT, not the row count, whenever a RESIDUAL predicate is present.** The
|
||||
distinction is not "predicate vs none" — a keyset cursor is a predicate. It is that a *seekable
|
||||
predicate on the ordering key* positions the scan and never discards a row, while a *residual*
|
||||
predicate (`LIKE`, `LOWER`, `IS NOT NULL`) throws away rows the engine already produced, so `LIMIT`
|
||||
truncates the survivors and says nothing about how many were produced — a query matching nothing
|
||||
must examine every eligible row before it can return an empty page. To bound rows, drop the residual
|
||||
predicate, page by row position over the primary key, and filter in memory. This endpoint got it
|
||||
wrong four times: bounding the result, then candidates returned, then `Id` keyspace width (keyspace
|
||||
is not rows — one live row at `Id` 20001 behind 20,000 deleted ones reads nothing), before arriving
|
||||
at "cursor only".
|
||||
**And scope the resulting claim to LOGICAL ROWS.** It is not bounded physical work: MySQL still
|
||||
traverses deleted-but-unpurged index records, so deletion history keeps affecting cost, and an
|
||||
unrestricted `TEXT` column spills to overflow pages so a row count implies no byte or page-read
|
||||
count. A SQL-string assertion pins none of that — not a plan, not visibility work, not I/O.
|
||||
2. **A SQL pre-filter under an in-memory exact filter may over-match but must never under-match — and
|
||||
that licence is void the moment the candidate set is truncated.** Widening the predicate then starves
|
||||
the budget with rows that cannot match. If you find yourself proving a superset property to keep a
|
||||
pre-filter honest, consider deleting the pre-filter instead: here it removed a JSON-escaping bug
|
||||
class, an exhaustive Unicode sweep and an `ESCAPE` portability workaround along with it.
|
||||
3. **Prefix matching, dedup and ordering must be ordinal, not current-culture** (`OrdinalIgnoreCase`,
|
||||
`StringComparer.Ordinal`): `UseRequestLocalization` honours `Accept-Language`, so `ToLower()` and the
|
||||
default linguistic `StartsWith(string)` let a caller change the result by changing a header. **Scope
|
||||
the claim to the stage that actually holds it** — a value set that a database `LOWER`/`DISTINCT`/
|
||||
`ORDER BY`/`LIMIT` already filtered and truncated is not ordinal no matter what runs after it, and
|
||||
saying otherwise in an `[EndpointDescription]` publishes a false contract (ersatztv#668).
|
||||
|
||||
Full rationale, the measured transfer cost, the four-attempts table and the rejected
|
||||
normalized-side-table alternative (ersatztv#669): `api.search-field-values-sources` (supersedes
|
||||
`api.search-field-values`).
|
||||
|
||||
**Param + DTO expansion (#293, cap `search/all-items`)**: no new endpoint — `GET /api/v1/search/all-items`
|
||||
gained two **optional** query params (`pageSize` default 500, clamped 1–1000 via the §1 Logs `Math.Clamp`
|
||||
precedent; `pageNum` 0-based, clamped `0..2_000_000` so `pageNum * pageSize` can't overflow `int` to a 500)
|
||||
|
||||
+2
-21
@@ -201,27 +201,8 @@ with no bug until you re-save it. Historical context (the old `File.Exists`-on-a
|
||||
bounded render-time fetch that preceded caching) is in `docs/decisions.md` under
|
||||
`graphics.channel-logo-caching`, #502 and #511.
|
||||
|
||||
**No usable logo means no on-screen bug — from every attachment point (#510).** A `ChannelLogo`
|
||||
watermark resolves through one shared resolver (`WatermarkSelector.ResolveWatermark`) whether it is
|
||||
attached via a playout item, the channel, the global setting, **or a deco**. All four agree: an
|
||||
un-migrated external URL, a missing cached file, and a channel with no logo artwork each render
|
||||
*without* a bug and log a warning. The selector never hands a dead path or a URL downstream — a dead
|
||||
local path could otherwise reach ffmpeg as a bare `-i` argument and break the stream, which is worse
|
||||
than a skipped overlay. (One watermark is built *outside* the selector and is still unchecked: the
|
||||
song-progress overlay — see #653.)
|
||||
|
||||
Before #510 the deco path had its own unchecked copy of that resolution, so the same channel could
|
||||
disagree with itself about whether a bug rendered based only on how the watermark was attached. The
|
||||
divergence covered `Custom` and `Resource` image sources too, not just `ChannelLogo`.
|
||||
|
||||
That change switched off one thing that *did* work: a channel with **no** logo artwork used to get a
|
||||
generated-initials nameplate (`/iptv/logos/gen`, drawn by `ChannelLogoGenerator`) when — and only
|
||||
when — the watermark came from a deco. It is now off everywhere, because serving it means an HTTP
|
||||
fetch inside stream startup, exactly what `graphics.channel-logo-caching` (#525) removed for logos,
|
||||
and because `ChannelLogoGenerator.GenerateChannelLogoUrl()` hardcodes `localhost` (issue #1, closed
|
||||
as a topology problem without removing the hardcode). Reviving it properly means generating the image
|
||||
into the image cache so it resolves to a local path — tracked as **#652**; the rationale is in
|
||||
`ffmpeg.watermark-resolution-unified`.
|
||||
Note: `ChannelLogoGenerator.GenerateChannelLogoUrl()` hardcodes `localhost` for watermark logo
|
||||
fetching — see issue #1 for details.
|
||||
|
||||
## On Now / Next overlay (#74)
|
||||
|
||||
|
||||
+22
-345
@@ -39,8 +39,6 @@ Upstream's final release was **`v26.3.0`** (archived). Our line continues from t
|
||||
| `v26.10.0` | Auto-Tune channel workflow (#69) + weighted content distribution (#70); scheduling refactors, health-check remediation UX (#164), HLS cold-start instrumentation (#350), security hardening (#293/#376/#308). |
|
||||
| `v26.11.0` | **QSV profiles decode via VA-API** — `QsvPreferNativeDecoder`, default **on**, fixes ~50% channel cold-start failures on Intel (#498); unified logo/on-screen bug via a shared watermark preset (#67). Media-scanner resilience: Jellyfin mixed-content libraries (#489), music-video scan correctness (#488/#494/#497), remote-stream probing before ffmpeg (#473/#480); weighted-distribution SPA (#404). **First release deployed to `jazz`** (server-management#633). |
|
||||
| `v26.12.0` | **`ErsatzTV.Mcp` MCP server** — read + cautious-write over `/api/v1`, `ERSATZTV_ALLOW_WRITES`-gated (#58). **External channel-logo URLs download + cache at save time** (#525), with the on-screen bug now rendered for external-URL logos (#502). HLS cold-start hardening: burst-read the first segments so start isn't `-readrate`-bound (#350) and floor QSV extra hardware frames so an unthrottled read can't exhaust the pool (#529); remote graphics-engine image fetches bounded — timeout, size cap, decode cap, redirects, pooling (#511). Decision-lifecycle tooling + parallel-orientation startup rewrite (#520/#521); CI `docker build` lane rebalance (#508). |
|
||||
| `v26.13.0` | **RuleBuilder maturation** — arbitrary-depth group nesting (#436), inline smart-query authoring in Channel Builder (#437), DB-sourced facet typeahead + relative-date operators + validation (#434/#435/#438), and an artist typeahead covering music-video/song credits with `album_artist` no longer 404ing (#578). **Per-channel On Now/Next transient overlay** (#74/#570) and **per-schedule clock-boundary padding** (#392); in-browser channel preview (#60); Auto-Tune per-source weight steppers + exclude/add-untagged (#440). Library-browse pickers now resolve by search instead of a 100-row window, closing several silent at-cap truncations (#644/#650/#651/#634). Correctness: one watermark resolver for all four attachment points, incl. `MiddleCenter` (#503/#510); QSV HDR tonemaps through OpenCL because `vpp_qsv=tonemap` is a silent no-op (#505); `LibraryFolder` unique index + concurrent-insert tolerance (#491); per-library music-video identity with soft trash (#496); Jellyfin Album/Track music-video projection (#177); metadata-collection dedup (#500); accented facet values via a registered Unicode fold on SQLite (#668); `WorkAheadSlots` atomic slot claim, never a negative count (#536/#539); on-demand guide rebuild on thaw (#68). Process/CI: the H10 review-verdict gate became a sha-bound **required** commit status and was hardened through its false-open chain (#622/#629/#632/#648/#649/#672/#698), the decision corpus split to one YAML-frontmatter record per file (#610/#620), and headless Playwright UI-E2E flows landed (#445/#533). Five dual-provider migrations. |
|
||||
| `v26.14.0` | **Live TV no longer starves on embedded bitmap subtitles** — `-readrate` paces an input off its *furthest-behind* stream, and a PGS/DVD subtitle read through the video's own `-i` is sparse enough to drag the whole process to **0.53x realtime** against the 1.0x a client consumes, draining the buffer until the channel stalls. Fixed with a capability-gated `-readrate_catchup` (ffmpeg 8.0+) on realtime inputs, keeping `-readrate` on the frame-producing path so the `ffmpeg.qsv-extra-hw-frames-floor` bound is untouched; measured 0.533x → 1.067x on QSV and software, with a 240s QSV soak clean of allocation errors (#726). Affects items carrying an embedded bitmap subtitle matching the channel's subtitle mode — 3,182 of 24,646 media versions on prod, and a property of the *item*, not the channel, which is why the stall presented as random. Process/CI: the H10 review-verdict gate's repair sentinel became a fixed point and its write is now fenced on the timeline retarget count, closing a raced-sentinel false-open (#706/#707/#711). **The decisions validator now cross-checks its dependency-free frontmatter parse against PyYAML** and reports both the truncating unquoted `` #`` and the scalar-closing bare apostrophe as errors, so a record whose `rule:` silently halves under PyYAML fails the local gate instead of CI (#674/#688) — the ceiling-calibration claim was also split so the suite pins what the derivation MEANS rather than live-corpus order statistics. Dependencies: CliWrap 3.10.4, JetBrains.ReSharper.GlobalTools 2025.3.5. |
|
||||
|
||||
**Before cutting a release — sweep `docs/decisions.md` + `docs/decisions/`** (ersatztv#521, supersedes
|
||||
the ersatztv#303 H9 append-only ritual). Supersession/retirement is now a same-PR act (add the new
|
||||
@@ -48,53 +46,22 @@ active record, relocate the predecessor to `docs/decisions/archive/` with recipr
|
||||
`supersedes`/`superseded-by` links), not a release-boundary batch job — most of the old "consolidate"
|
||||
step is now continuous. The release boundary is instead where you:
|
||||
1. Run `PYTHONPATH=. python3 scripts/decisions_validate.py` — confirms lifecycle metadata is
|
||||
well-formed and every `supersedes`/`superseded-by` link resolves both ways. Since **ersatztv#674**
|
||||
it also cross-checks its dependency-free frontmatter parse against **PyYAML when PyYAML is
|
||||
importable**, failing on any file PyYAML rejects (a bare apostrophe in a single-quoted value) or
|
||||
reads differently (an unquoted ` #`, which YAML truncates as a comment). Where PyYAML is absent —
|
||||
the `decisions-guard` job, the Husky hooks — the cross-check is **skipped with a `::notice::`**
|
||||
and every other check still runs; the read path stays dependency-free.
|
||||
well-formed and every `supersedes`/`superseded-by` link resolves both ways.
|
||||
2. Confirm every record already classified `superseded`/`retired` actually lives under
|
||||
`docs/decisions/archive/` (the validator fails this, but eyeball it at the boundary too).
|
||||
3. Regenerate the active catalog: `PYTHONPATH=. python3 scripts/build_decisions_catalog.py` and
|
||||
commit any drift.
|
||||
4. Read the corpus size signals. Since **ersatztv#620** these are two separate things:
|
||||
- a **per-record prose ceiling** (`decisions_validate.py --record-ceiling <n>`, default **60**)
|
||||
— a **non-blocking `::warning::`** naming every record over it. This is the actionable signal:
|
||||
it points at a file. The 60 is derived from the distribution, not picked as a round number.
|
||||
Its **calibration is guarded in two pieces of different robustness** (ersatztv#688), because
|
||||
four earlier single-assertion versions all failed — the first two by being vacuous or
|
||||
accepting an absurd ceiling, the last two by ratcheting:
|
||||
- **blocking** (`script-tests`) — only the coarse property that the ceiling flags a
|
||||
**meaningful minority** of records (`0.02 <= fraction_over <= 0.25`). One record moves a
|
||||
fraction by at most 1/N, so no SINGLE ordinary addition can cross it. This is measured
|
||||
headroom, not immunity: from today's 18/183 it takes 38 consecutive over-ceiling additions to
|
||||
breach the cap, 718 short ones to dilute below the floor, or — the tightest arm —
|
||||
consolidating 15 of the 18 offenders away. The floor is
|
||||
a fraction rather than "at least one record", which would accept any ceiling up to 229 on the
|
||||
live corpus; as a fraction the accepted range is 43..180.
|
||||
- **reported, never asserted against the LIVE corpus** — the fine claim that the ceiling sits
|
||||
between the **90th and 95th percentile**, i.e. at the tail boundary. `main()` prints a
|
||||
`::notice::` when it drifts; the tests assert it only on distributions they own.
|
||||
It is an order statistic over a sparse distribution, so a single new record could move p90 by
|
||||
21 lines and red the blocking job for whoever wrote it; a ceiling going out of date is
|
||||
the passage of corpus growth, not a defect in the commit under test, so it is treated like
|
||||
`stale-after`. Re-derive the constant when the notice says so.
|
||||
- the **aggregate prose total**, printed every run as an unthresholded `::notice::` **trend**.
|
||||
It has no pass/fail. A total over a monotonically growing corpus can only ratchet: the old
|
||||
4800→5600 budget went quiet at 5228 after #610 changed the metric and was back over at 5658
|
||||
**three and a half hours later the same evening**, with nobody consolidating anything — the
|
||||
"permanently red = no signal" failure, not in slow motion at all. It reports record prose and
|
||||
non-record scaffolding separately, because they are not the same unit. The generated catalog is no longer counted at all — it
|
||||
gains one row per record and cannot be consolidated away.
|
||||
|
||||
**Being listed by the ceiling is an invitation to check for redundancy, not an instruction to
|
||||
cut.** A long record that is entirely distinct findings is a legitimate decline — say so in the
|
||||
record and move on. (`--budget` is still accepted and ignored, so old invocations keep working.)
|
||||
4. Check the aggregate active-corpus budget (`decisions_validate.py --budget <n>`, default **4800**
|
||||
lines across `docs/decisions.md` + topic files + the catalog — replaces the old single-file
|
||||
1800-line floor). **Re-baselined 2026-07-21 (#520)**: the corpus is now fully migrated at
|
||||
~4366 lines; 4800 gives headroom so the warning fires on real future growth, not on the expected
|
||||
post-migration size. Going over budget is a **non-blocking warning** (`::warning::` to stderr,
|
||||
not a validator error) — a ratchet/reminder to extract a new topic file or archive more history,
|
||||
not a release gate.
|
||||
5. Report the remaining `legacy-unmigrated` count (the validator prints it as a `::notice::`) so the
|
||||
backlog is visible, even though it isn't required to hit zero before a release.
|
||||
A genuine rationale-prose rewrite still needs a `Decisions-Edit: yes` git trailer on a **non-merge**
|
||||
commit in the range (see the `decisions.md` header) — routine lifecycle metadata writes above do not.
|
||||
A genuine rationale-prose rewrite still needs `[decisions-edit]` in the commit message (see the
|
||||
`decisions.md` header) — routine lifecycle metadata writes above do not.
|
||||
|
||||
**Cutting a release:** keep build and promotion as two explicit phases (#335):
|
||||
|
||||
@@ -182,12 +149,8 @@ rather than in `docker-build.yml` — see that section (ersatztv#535).
|
||||
|
||||
**`small` is git-only, and that is load-bearing (server-management#639).** Everything in
|
||||
the lane is a checkout plus a `git diff`: `decisions-guard`, `ci-image-pin`,
|
||||
`docs-reminder` — plus `script-tests`, which is a checkout plus a `pytest` run needing only
|
||||
`pytest` and `pyyaml` (ersatztv#631; it is NOT stdlib-only — that assumption is what turned the
|
||||
job red on its first CI run, see below). Nothing there runs a compiler or a `docker build`, which is why the lane
|
||||
can be capped at 1 GiB per job. The lightweight-Python jobs are the deliberate edge of the
|
||||
"git-only" rule, not an exception to it: `setup-python` + `pip install pytest` + a suite whose
|
||||
heaviest allocation is a handful of temp-dir git repos stays far under the cap. Route a heavy job here and it will OOM — give it
|
||||
`docs-reminder`. Nothing there runs a compiler or a `docker build`, which is why the lane
|
||||
can be capped at 1 GiB per job. Route a heavy job here and it will OOM — give it
|
||||
`ubuntu-latest`, or its own label on `ci-runner`, the only host with no prod workload.
|
||||
|
||||
**Lane assignment (ersatztv#390).** *Slot counts below are as-of 2026-07-17; the table above is
|
||||
@@ -544,8 +507,8 @@ expensive 787-migration replay is skipped; the service is capped and idle for se
|
||||
|
||||
`api-docs` and `format` already short-circuit on docs-only changes via their own path detection (no
|
||||
API path / no `.cs` changed → they pass in ~5s), so they needed no change. `docs-reminder`,
|
||||
`decisions-guard`, `ci-image-pin` and `script-tests` keep running on docs-only changes — the first
|
||||
two are *about* docs and must, and `script-tests` is unconditional by design (ersatztv#631).
|
||||
`decisions-guard` and `ci-image-pin` keep running on docs-only changes — the first two are *about*
|
||||
docs and must.
|
||||
|
||||
Not in scope: the within-run triple `dotnet build` (ersatztv#398; measured and rejected as
|
||||
build-once — see `docs/decisions.md`). The separate redundancy of running the **whole matrix on a
|
||||
@@ -615,15 +578,7 @@ blocking `api-docs` job, and `docs/decisions.md` by the blocking `decisions-guar
|
||||
Enforces decision-record lifecycle invariants (ersatztv#521, supersedes the ersatztv#303 H9
|
||||
append-only mechanic): well-formed 5-field metadata, exactly one `active` record per `key`,
|
||||
reciprocal `supersedes`/`superseded-by` links, no record vanishing from the active set without an
|
||||
archive copy, no rationale-prose rewrite without a `Decisions-Edit: yes` trailer on a non-merge commit
|
||||
in the range (ersatztv#609), a structural per-path check that every `*.md` under
|
||||
`docs/decisions/records/**` and `docs/decisions/archive/**` parses to **exactly one keyed
|
||||
record** (ersatztv#621 — without it, a file the dependency-free frontmatter reader cannot parse,
|
||||
such as one using a YAML block scalar, yields `[]` and vanishes from the corpus with every check
|
||||
still reporting green; a file directly in `archive/` is exempt only when it really is a stripped index — one keyless
|
||||
record with a known generated heading — never merely by its location; the single further exemption,
|
||||
`archive/README.md`, is by exact relative path, never by basename, which would otherwise exempt the
|
||||
same filename in the active wing),
|
||||
archive copy, no rationale-prose rewrite without the `[decisions-edit]` token in the commit range,
|
||||
and the generated active catalog (`docs/decisions/README.md`) in sync with source. Two steps:
|
||||
`scripts/decisions_validate.py --base origin/<base> --head HEAD` (the merge-base diff checks, which
|
||||
need a base/head range — CI-only) and `scripts/build_decisions_catalog.py --check` (catalog drift).
|
||||
@@ -635,102 +590,12 @@ compiler/docker build), so it doesn't violate the "small is git-only" lane rule.
|
||||
`docs-reminder`, otherwise a seconds-long `git diff` + parse with no dotnet/node setup
|
||||
(`runs-on: small`).
|
||||
|
||||
### `script-tests` job (`Script tests (pytest)`, PR-only — in `pr-checks.yml`)
|
||||
|
||||
> Reddens the run on failure, but like the other `pr-checks.yml` gates it is **not** one of the
|
||||
> three required status checks on `main` (`Build & test (.NET)`, `EF migration integrity`,
|
||||
> `review-verdict/h10`). Promoting it to required is a branch-protection change, tracked separately.
|
||||
|
||||
Runs the repository's Python test suite: `PYTHONPATH=. python3 -m pytest scripts/tests -q`
|
||||
(~190 tests at time of writing, ~10s; the suite grows, so treat the figure as indicative). It covers the decision-corpus parser/validator/catalog builder, the ersatztv#610
|
||||
migration-equivalence harness, the merge-consent exemption logic and the ersatztv#622 review-verdict
|
||||
poster.
|
||||
|
||||
**Until ersatztv#631, nothing ran these tests.** No workflow and no Husky hook invoked `pytest`.
|
||||
`decisions-guard` executes `decisions_validate.py` and `build_decisions_catalog.py` directly — it
|
||||
exercises that *code* but never its *tests* — and the `test` job is `dotnet test` only. The suite
|
||||
guarding our merge-gating machinery was therefore local-only, and a test added "for CI enforcement"
|
||||
was decorative.
|
||||
|
||||
**Why it is its own job, not a step inside `decisions-guard`.** `decisions-guard` is covered by the
|
||||
standing `ci.decisions-lifecycle-flake` rule: a lone `decisions lifecycle` red is a known infra
|
||||
flake and sessions are instructed *not to investigate it*. Adding the suite there would make a
|
||||
genuine pytest regression surface as precisely the red everyone is told to wave through — the same
|
||||
"reports success while doing nothing" failure mode ersatztv#631 exists to close. A distinct job
|
||||
name keeps a real failure unambiguous.
|
||||
|
||||
**Why it runs unconditionally** rather than behind a `scripts/**` path filter: the suite's true
|
||||
input set spans more than one directory — `test_post_review_verdict.py` and
|
||||
`test_merge_consent_exemption.py` execute the real `scripts/post-review-verdict.sh` and
|
||||
`.claude/hooks/pretooluse-merge-consent.sh` — so a `scripts/**` filter would silently miss a
|
||||
`.claude/hooks/**` edit. At ~10s, a filter buys nothing but drift.
|
||||
|
||||
**Dependencies: `pytest` and `pyyaml`** — the complete third-party set across `scripts/`, established
|
||||
by an AST import scan rather than by reading the files that looked relevant. PyYAML does **not**
|
||||
contradict the dependency-free decisions *read* path: `decisions_lib._read_frontmatter` is
|
||||
hand-written exactly so validation runs where nothing is installed, but the one-shot *write* path
|
||||
`migrate_decisions_split.py` uses PyYAML by design, and `test_migration_equivalence.py` imports that
|
||||
module. (The first cut of this job claimed "pure stdlib + pytest", passed locally on a machine that
|
||||
happened to have PyYAML installed, and went red in CI on a `ModuleNotFoundError` at collection —
|
||||
which is itself a small demonstration of why the suite needed to run in CI at all.) Like the other
|
||||
`small`-lane Python jobs it adds `actions/setup-python@v5` first. Checkout is at default depth: every `git` call in the suite runs
|
||||
against a temp repo it creates itself, never this repository's history.
|
||||
|
||||
Two **preflight steps** run before the suite. The first asserts `git` is on PATH; the second runs
|
||||
`scripts/jq-preflight.sh --expect 1.6`, which checks jq's **version**, not merely its presence (see
|
||||
"The jq contract" below). Those two tests exec the real shell scripts, which shell out to `jq` ~26
|
||||
times; the tests shim `curl` on PATH but not `jq`, so a runner image without it would surface as ~20
|
||||
opaque assertion failures instead of one diagnosis. Both deliberately **check** rather than install —
|
||||
ersatztv#390 removed run-time `apt-get` from CI; the fix for a genuine miss is to bake the tool into
|
||||
the runner image.
|
||||
|
||||
### The jq contract (ersatztv#648)
|
||||
|
||||
> Full rationale: `docs/decisions/records/ci/jq-version-contract.md`.
|
||||
|
||||
Every shell gate in this repo — `decisions-guard`, `script-tests`'s own harness,
|
||||
`pretooluse-merge-consent.sh`, `review-verdict.yml`, `scripts/pr-changed-files.sh` — is authored and
|
||||
tested on a developer Mac shipping **jq 1.8.x**. The CI runner ships **jq 1.6**. Author to the
|
||||
1.6-compatible subset; three concrete constructs diverge between the two and each one produced a real
|
||||
bug when it hit CI for the first time:
|
||||
|
||||
- **`jq -e` over EMPTY input.** Exits 4 on jq >= 1.7, but **0** on jq 1.6. A guard that infers
|
||||
"transport failure" from that exit status silently passes an empty/failed page on 1.6.
|
||||
- **`` contains("\u0000") `` (or any NUL literal).** The NUL escape truncates to `""` on jq 1.6, so
|
||||
the containment test is vacuously true for **every** string, not just ones containing a NUL. Use
|
||||
`explode | index(0)` instead — it is version-stable.
|
||||
- **Parse-error exit code.** `jq empty` exits 5 on jq >= 1.7 but **4** on jq 1.6 — the same code 1.6
|
||||
uses for "no output produced". Reading that exit code as a specific failure mode conflates garbage
|
||||
input with an empty-but-valid response.
|
||||
|
||||
`scripts/jq-preflight.sh` makes the running version **observable** in every gate job's log (it prints
|
||||
the parsed version and asserts a floor of 1.6) so a future divergence can be diagnosed from the log
|
||||
alone instead of guessing at the runner image.
|
||||
|
||||
**Pin vs floor is deliberately asymmetric.** `scripts/jq-preflight.sh --expect 1.6` additionally pins
|
||||
the version and fails loudly if it drifts, but that mode is used **only** by `script-tests`
|
||||
(`.gitea/workflows/pr-checks.yml`) — advisory, not a required check. `review-verdict.yml` runs the
|
||||
no-args floor-only mode and never pins, because that workflow writes `review-verdict/h10`, the
|
||||
branch-protection-**required** status check on `main`: a hard pin there would mean the day the
|
||||
runner's jq version changes (a base-image bump, a host reimage — nothing this repo controls), every
|
||||
PR on `main` stops merging until someone notices and re-pins. A required merge gate cannot fail
|
||||
because an upstream package manager did its job. The narrower pin on `script-tests` exists precisely
|
||||
because that job is the suite's only 1.6 coverage — if the runner's jq silently changed, that coverage
|
||||
would evaporate with no signal, so failing loudly there forces a human decision instead.
|
||||
|
||||
Baking a pinned jq into `docker/ci/Dockerfile` was considered and rejected: `review-verdict.yml` is
|
||||
`runs-on: small` with no toolchain-image pin, and per `ci.small-lane-git-only` the small lane is
|
||||
git-only, so it gets the **host's** jq regardless of what the toolchain image contains — a pin in the
|
||||
image provably cannot reach the gate that broke. This was checked against the running binary, not
|
||||
assumed.
|
||||
|
||||
## PR gates workflow
|
||||
|
||||
**File:** `.gitea/workflows/pr-checks.yml` — `on: pull_request` only.
|
||||
|
||||
The four git-only PR gates — `ci-image-pin`, `docs-reminder`, `decisions-guard`, `script-tests`
|
||||
(all described above) — live here, **not** in `docker-build.yml`, and that separation is the fix
|
||||
for **ersatztv#535**.
|
||||
The three git-only PR gates — `ci-image-pin`, `docs-reminder`, `decisions-guard` (described above)
|
||||
— live here, **not** in `docker-build.yml`, and that separation is the fix for **ersatztv#535**.
|
||||
|
||||
**Why they are split out.** All three are pure `checkout + git diff` gates on the `small` lane
|
||||
(no `container:`) and are PR-only (`if: github.event_name == 'pull_request'`). While they lived in
|
||||
@@ -754,198 +619,10 @@ grep of `docker-build.yml` still validates the five pin-bearing jobs
|
||||
(`test`/`migrations`/`functional-e2e`/`api-docs`/`format`) that remain there. `api-docs` and
|
||||
`format` stay in `docker-build.yml` because they carry the shared-image `container:` + pin and run
|
||||
on the healthy `ubuntu-latest` lane (where they skipped correctly). None of the three moved jobs is
|
||||
a **required** check — branch protection requires `Build & test (.NET)`, `EF migration integrity`
|
||||
and `review-verdict/h10` (next section) — so relocating them (their status-context prefix changes
|
||||
from `Build ErsatzTV Image / …` to `PR Gates / …`) does not affect merges. The file declares
|
||||
`defaults: run: shell: bash` because `ci-image-pin` uses `mapfile`/`set -o pipefail`.
|
||||
|
||||
## Review-verdict gate (`review-verdict/h10`, required — `.gitea/workflows/review-verdict.yml`)
|
||||
|
||||
**A required status check named `review-verdict/h10`, written per-sha, is what actually stops an
|
||||
unreviewed commit from merging** (ersatztv#622). It is not produced by a job's success/failure; it
|
||||
is a commit status that `scripts/post-review-verdict.sh` POSTs onto one specific sha.
|
||||
|
||||
**The hole it closes.** `pretooluse-merge-consent.sh` proves its three consent conditions at the
|
||||
moment the merge tool is called. Pass `merge_when_checks_succeed=true` and Gitea performs the merge
|
||||
*later*, against whatever head is green then — while the Done-when and review-verdict checks were
|
||||
proven against the head at **scheduling** time. Every commit pushed in between merges unreviewed.
|
||||
|
||||
This was demonstrated as a controlled A/B rather than inferred (`ci/fake` stands in for a slow CI
|
||||
check so Gitea waits, as it really does): review head A → post its verdict → schedule auto-merge →
|
||||
push an unreviewed commit B → CI greens on B. **Without** the required verdict context, B merged.
|
||||
**With** it, the same sequence was refused, and merged only once B itself was reviewed.
|
||||
|
||||
Note the motivating anecdote in ersatztv#622 — "PR #619 merged 263 insertions with no verdict" —
|
||||
is **wrong**: #619 does carry `Review-verdict: MERGEABLE @ 02c82b35`, posted six seconds before the
|
||||
merge, explicitly re-reviewing the follow-up commits. It was filed from an API read that lagged.
|
||||
The gap is real anyway, and structural: nothing *forced* that re-review inside the 45-minute window
|
||||
where Gitea would have merged whatever went green. This turns diligence into construction.
|
||||
|
||||
**Why a commit status fixes it and a smarter hook cannot.** A status belongs to exactly one sha, so
|
||||
a new commit *cannot inherit it*: the required context is simply absent on the new head, Gitea's
|
||||
merge-requirement check reads that as not-passing, and the scheduled auto-merge refuses to fire.
|
||||
The invariant self-invalidates — nothing has to notice the push. It also covers merge paths the
|
||||
hook never sees (Gitea UI, raw API, another agent's session).
|
||||
|
||||
**Posting a verdict.** After reviewing a PR's *current* head:
|
||||
|
||||
```bash
|
||||
ETV_GITEA_BASICAUTH=user:pass scripts/post-review-verdict.sh <pr> MERGEABLE [note...]
|
||||
```
|
||||
|
||||
That posts both the `Review-verdict: … @ <sha>` comment (the human-readable artifact, and the
|
||||
hook's condition (c)) and the `review-verdict/h10` status on the same sha. `BLOCKED` /
|
||||
`NOT-MERGEABLE` post a `failure` status instead. The script re-reads the head after commenting: if
|
||||
a commit landed mid-flight it writes **no** status and exits non-zero rather than retargeting your
|
||||
verdict at a commit you never read.
|
||||
|
||||
The status description also records the base branch — `Review-verdict: MERGEABLE @ abc1234 (base:
|
||||
main)` — and the merge-consent hook denies when that no longer matches the PR's live `base.ref`
|
||||
(ersatztv#632). Retargeting a PR changes the effective diff without moving the head sha, so the
|
||||
per-sha binding alone cannot see it. This is **detection on the hook path only**: a commit status
|
||||
carries no base of its own, so a merge driven through the Gitea UI or API is unaffected. The
|
||||
comparator is the base *branch*, never its tip sha — a base that merely advances is ordinary churn,
|
||||
and comparing tips would invalidate every open verdict on every unrelated merge to `main`.
|
||||
|
||||
**Exemptions** are handled by `review-verdict.yml` on every `pull_request_target` event, which posts the
|
||||
status as `success` for **Renovate-authored** PRs (it uses `platformAutomerge: true`, so a required
|
||||
verdict with no exemption would stall every dependency bump) and for **docs-only** PRs, and as
|
||||
`pending` for everything else so the block has a visible reason. Both exemptions are **void when the
|
||||
PR touches `.claude/`, `.codex/`, `.gitea/`, `.husky/`, `scripts/` or `docker/ci/`** — a PR that can weaken the
|
||||
gate must not be able to exempt itself from the gate. That includes Renovate's `docker/ci` base
|
||||
bumps, which already need the manual publish-then-pin two-step anyway.
|
||||
|
||||
The Renovate exemption additionally requires **every** changed path to be a dependency manifest —
|
||||
`Directory.Packages.props` or `.config/dotnet-tools.json`, and only those (ersatztv#698). The npm
|
||||
manifests are deliberately excluded: `renovate.json` enables only `nuget`/`github-actions`/`dockerfile`,
|
||||
so npm is unmanaged here, while `package.json` `scripts` are executed by CI (`npm ci`, `npm run build`)
|
||||
— exempting it would put a code-execution path inside the allow-list for no benefit. An author match alone is not enough, because `pull_request.user.login` is the PR's
|
||||
*immutable creator* while its head is not: pushing application code onto an open Renovate branch
|
||||
leaves the PR still "authored by renovate" and, previously, still exempt. A Renovate PR touching
|
||||
anything else — a `.csproj`, a source file — is not blocked, it just needs a real verdict. **If a
|
||||
dependency PR is unexpectedly asking for a verdict, this is why**; the status description says so.
|
||||
|
||||
The two exemptions are evaluated as **independent predicates**, never as an `elif` chain: a Renovate
|
||||
PR touching only `docs/` still gets the docs-only exemption on its own merits.
|
||||
|
||||
An existing `review-verdict/h10` on the head is **only** left alone when it is positively identifiable
|
||||
as a human verdict — a non-null `.creator.login` **and** a `Review-verdict:` description, which is what
|
||||
`post-review-verdict.sh` writes. Anything else, including any shape the workflow does not recognise, is
|
||||
**re-derived** rather than inherited. (Measured: a status POSTed with a user credential carries a
|
||||
creator; one POSTed by an Actions job carries `"creator": null`.) Without this, an exemption obtained
|
||||
once was accepted unchanged on every later run. This is a *provenance* check, not an authentication
|
||||
one — someone who can POST statuses directly can still impersonate a verdict, which is ersatztv#697.
|
||||
|
||||
Deciding either exemption requires the PR's **complete** changed-file list, which the workflow does
|
||||
not compute itself: it calls `scripts/pr-changed-files.sh`, the single shared implementation also
|
||||
used by the advisory hook `.claude/hooks/pretooluse-merge-consent.sh` (ersatztv#649). The workflow
|
||||
reads that script's **exit status** — a non-zero exit means "could not tell" and withholds the
|
||||
exemption; its stdout is meaningless on any failure path and is never consumed.
|
||||
|
||||
**Never write a classification guard as `producer | grep -q…` here.** Under `set -o pipefail`, `grep -q`
|
||||
exits at its first match, the producer takes SIGPIPE (141), and a MATCH is reported as a failed
|
||||
pipeline — inverting the guard for any PR whose path list exceeds the pipe buffer. That let a large PR
|
||||
be classified docs-only, and let one editing `.gitea/` skip the protected-path check entirely. A
|
||||
here-string is **also** wrong (bash spills a large one to temp storage, which fails the same way when
|
||||
temp is full). **Count** instead — `grep -c` drains stdin over an ordinary pipe — evaluate the counts
|
||||
once at top level rather than inline in an `if`, and fail closed on a non-numeric result. Full detail:
|
||||
`ci.grep-q-pipefail-inversion`.
|
||||
|
||||
That script takes the expected base branch as a **required 5th argument** and refuses to enumerate when
|
||||
the PR's live base does not match it, checked both before and after paging (ersatztv#698).
|
||||
`/pulls/{n}/files` diffs against the PR's *live* base, so retargeting changes the answer without moving
|
||||
the head sha — a PR opened into `main` and retargeted mid-run was granted a docs-only exemption while
|
||||
its diff against `main` carried a C# file. The workflow passes the base from the `pull_request_target`
|
||||
payload, which a retarget cannot rewrite, and `edited` is in `types:` so a retarget reclassifies.
|
||||
`edited` gives **detection, not atomicity**: runs are not serialized, so a stale run could still post
|
||||
`success` after the reclassifying run posted `pending`.
|
||||
|
||||
**That residual is now fenced (ersatztv#706).** Runs are still not serialized — instead a run that was
|
||||
overtaken *declines to write*. The job counts `change_target_branch` events on the PR's issue timeline
|
||||
at start and again immediately before its POST, and posts **nothing** if the count moved. The count is
|
||||
the key precisely because the branch *name* is ABA-vulnerable: `main → scratch → main` reads `main` at
|
||||
both ends, which is how the forged exemption was obtained in the first place. Abstaining never strands
|
||||
a PR, because every retarget fires `edited` — the event that makes one run abstain has already queued
|
||||
its successor.
|
||||
|
||||
If the count can't be established (unreadable timeline, paging that never reached a validated empty
|
||||
page), only the exemption `success` is withheld; `pending` still posts, since `pending` cannot turn an
|
||||
unreviewed head green and withholding it would strand ordinary PRs for nothing. **If an exempt PR is
|
||||
unexpectedly missing its status after a retarget, this is why** — the job log names the counts.
|
||||
|
||||
Worth knowing before reaching for the obvious alternative: **a concurrency group does not work here**,
|
||||
measured rather than assumed. Gitea 1.25.4 auto-cancels superseded `push` runs on a branch, but *not*
|
||||
`pull_request_target` runs — two runs for one PR genuinely overlap, and adding
|
||||
`concurrency: {…, cancel-in-progress: false}` changed nothing (probe runs still overlapped by 36s).
|
||||
`cancel-in-progress: true` is deliberately untried, because a cancelled run leaves an exempt PR
|
||||
statusless with nothing left to re-trigger it. Full measurements and the two surviving residuals:
|
||||
`ci.verdict-write-retarget-fence`.
|
||||
|
||||
Separately, after posting an exemption `success` the job re-reads the per-POST status history and, if
|
||||
a human `Review-verdict:` row appeared during the write window, overwrites its own status with
|
||||
`pending` and logs an error — so a human `BLOCKED` can never be silently turned green. The repair is
|
||||
`pending`, never a copy of the human's verdict, which would attribute a human decision to the job.
|
||||
|
||||
Three properties of this workflow are security-relevant and are **structurally** asserted by tests in
|
||||
`scripts/tests/test_pr_changed_files.py` — those tests pin the workflow's shape, which is not the same
|
||||
as establishing that the gate cannot be forged (see the residual below, and ersatztv#697/#698):
|
||||
|
||||
- **The trigger is `pull_request_target`, scoped to `branches: [main]`** — never plain
|
||||
`pull_request` (ersatztv#672). Gitea resolves a `pull_request` workflow *definition* from the PR's
|
||||
own head, so under that trigger a PR editing `review-verdict.yml` ran its own rewritten copy and
|
||||
could post `review-verdict/h10=success` for itself. The base-ref checkout below binds the scripts
|
||||
this job runs; only the trigger binds the definition. The `branches` filter is half the fix, not a
|
||||
refinement of it: base resolution means the *base branch* supplies the gate, so an unfiltered
|
||||
trigger merely moves the rewrite to an attacker-pushed base — and a status forged there is
|
||||
inherited by any later PR carrying the same head sha (ersatztv#663). `pull_request_target` is safe
|
||||
here **only** because this job never checks out or executes head-supplied code. Verified on this
|
||||
instance with four scratch PRs rather than inferred from GitHub; full rationale in
|
||||
`docs/decisions/records/ci/gate-trigger-base-resolved.md`. **This closes the rewrite route through
|
||||
this workflow, not the class:** `docker-build.yml` is also head-resolved and its `ETV_STATUS_AUTH`
|
||||
credentials can write statuses, so it can still forge `review-verdict/h10` — it must stay on
|
||||
`pull_request` because it builds the PR's code, so it needs a read-only status identity instead
|
||||
(ersatztv#697) — and the inventory is every workflow, not that one, because Gitea injects a
|
||||
write-capable `GITEA_TOKEN` into every job and branch protection binds the *context*, not its
|
||||
issuer. The exemption path has separate defects of its own (ersatztv#698). One operational
|
||||
consequence of the trigger change: a PR whose base is not `main` now gets **no**
|
||||
`review-verdict/h10` at all. That is fail-closed. `edited` **is** now among the trigger's `types`
|
||||
(ersatztv#698), so a PR retargeted onto `main` reclassifies instead of staying statusless until its
|
||||
next push — but note that only gives *detection*: runs are not serialized, so a stale run can still
|
||||
post `success` after the reclassifying run posts `pending` (ersatztv#706).
|
||||
- **The checkout takes the PR's BASE ref**, `ref: ${{ github.event.pull_request.base.sha }}` with
|
||||
`persist-credentials: false` — never the head. This job judges the PR, so the PR must not supply
|
||||
the code that judges it; a head checkout would let a PR edit the enumeration to return an empty
|
||||
list and exempt itself.
|
||||
- **`scripts/jq-preflight.sh` runs in floor-only mode**, never `--expect`. This job writes a
|
||||
branch-protection-**required** status, so an exact version pin would turn any jq upgrade on the
|
||||
runner into a repo-wide merge deadlock.
|
||||
|
||||
A PR whose base predates ersatztv#658 has no such script on its base ref; that case posts `pending`
|
||||
with the reason rather than dying with no status at all.
|
||||
|
||||
⚠️ **Changing `review-verdict.yml` itself: it is not exercised by its own PR.** Base resolution cuts
|
||||
both ways — the PR editing this workflow runs the version already on `main`, so an edit goes live
|
||||
**only on merge**, repo-wide, having never run. A broken edit merges green and then breaks the gate
|
||||
for every subsequent PR, and the PR that would repair it is gated by the same broken workflow. Do not
|
||||
trust the editing PR's own checks. Verify the way ersatztv#672 did:
|
||||
|
||||
1. Push a scratch **base** branch carrying the candidate workflow.
|
||||
2. Open a throwaway PR from a scratch head *into that base*, so the candidate is the definition that
|
||||
runs. Have it post a **probe-named** context (e.g. `review-verdict/h10-PROBE`), never the real
|
||||
`review-verdict/h10` — a probe must not be able to forge the gate it is testing.
|
||||
3. Read the resulting commit statuses to see which definition actually ran, then delete both
|
||||
branches.
|
||||
|
||||
The same shape is what makes a `branches:`/`types:` change verifiable at all, since neither can be
|
||||
observed from the editing PR.
|
||||
|
||||
It lives in its **own workflow file** on purpose: `pr-checks.yml` sets `cancel-in-progress: true`,
|
||||
and a cancelled run there would leave an exempt PR with no status and no further push to
|
||||
re-trigger it. Its own job context (`Review verdict / Set review-verdict status`) is **not** the
|
||||
required check — a workflow must not satisfy the gate merely by running successfully.
|
||||
|
||||
Full rationale: `docs/decisions/records/release/verdict-status-check.md` and
|
||||
`docs/decisions/records/ci/shared-pr-file-enumeration.md`.
|
||||
a **required** check — branch protection requires only `Build & test (.NET)` and `EF migration
|
||||
integrity` — so relocating them (their status-context prefix changes from `Build ErsatzTV Image /
|
||||
…` to `PR Gates / …`) does not affect merges. The file declares `defaults: run: shell: bash`
|
||||
because `ci-image-pin` uses `mapfile`/`set -o pipefail`.
|
||||
|
||||
## CI toolchain image (`docker/ci/Dockerfile`, `.gitea/workflows/ci-image.yml`)
|
||||
|
||||
@@ -1385,7 +1062,7 @@ entirely in `web/`), the wiring is:
|
||||
3. **`commit-msg`** — enforces the CLAUDE.md protocol: the message must carry a
|
||||
`Co-Authored-By:` trailer, else the commit is rejected (merge commits are exempt, detected
|
||||
via `git rev-parse --verify MERGE_HEAD`). The decision-lifecycle check lives in `pre-commit`
|
||||
(above), not here — the `Decisions-Edit:` trailer is read from the commit message, but only by the CI
|
||||
(above), not here — `[decisions-edit]` is read from the commit message, but only by the CI
|
||||
`decisions lifecycle` job's body-diff step (`range` mode over the PR's merge-base diff), which is
|
||||
the only place a base/head range exists to diff against.
|
||||
|
||||
|
||||
+3867
-114
File diff suppressed because it is too large
Load Diff
@@ -1,10 +0,0 @@
|
||||
# ersatztv#610 — decision records are filed by their key's AREA, so directory names here come
|
||||
# from the key vocabulary, not from us. Several collide with the root .gitignore's build-output
|
||||
# rules: `release/` already did (all 9 records under records/release/ were silently dropped from
|
||||
# the migration commit, caught only by the body-diff guard), and `bin/`, `obj/`, `build/`,
|
||||
# `debug/`, `x64/` would do the same the day someone coins such an area.
|
||||
#
|
||||
# `!*/` un-ignores every directory under docs/decisions/, which fixes the CLASS rather than the
|
||||
# one instance. It must negate the DIRECTORY: git never descends into an excluded directory, so
|
||||
# negating only the files inside would not work.
|
||||
!*/
|
||||
+166
-187
@@ -8,190 +8,169 @@ the link for rationale. Superseded/retired history lives in `archive/`. Regenera
|
||||
|
||||
| Key | Current rule | Since | Record |
|
||||
| --- | --- | --- | --- |
|
||||
| `api.artwork-rooted-urls` | API response DTOs return artwork as rooted, directly-usable URLs (plus passthrough for absolute/proxy URLs), never relative Blazor-convention paths. | 2026-07-07 | [link](records/api/artwork-rooted-urls.md) |
|
||||
| `api.async-op-contract` | Queue-triggering `/api/*` endpoints normalize onto one contract — 202 Accepted (queued), 404 (missing entity), 409 (lock held), 422 (domain precondition) — with Trakt as the reference implementation; playout list/detail GETs also carry an `isLocked` observability flag as the HTTP-observable substitute for a live push channel. | 2026-07-11 | [link](records/api/async-op-contract.md) |
|
||||
| `api.channel-health-object` | `ChannelResponseModel`/`ChannelDetailResponseModel` carry a server-derived `health` object (`ChannelHealthResponseModel { Status, Faults[], PlayoutCount, BrokenSourceItemCount }`) computed **read-time** from the built timeline (`Playout.BuildStatus` + upcoming `PlayoutItem → MediaItem.State`, `Finish >= now`), kind-agnostic across all 5 `PlayoutScheduleKind` values; `Status`/`Faults` are const-string classes (`ChannelHealthStatus`, `ChannelFault`), not C# enums, so the SPA hand-maintains the union (mirrors `ChannelPreviewAvailability`). This supersedes #72's "raw fact only, no derived enum, empty-schedule/broken-source deliberately not computed" stance now that the auto-tune taxonomy churn (#383/#384) it was waiting on has landed (see `channel.origin-marker` sibling record, #414). | 2026-07-23 | [link](records/api/channel-health-object.md) |
|
||||
| `api.channel-preview-capability` | Whether a channel can be previewed in the browser is declared by the server, not derived by the SPA, as an additive `Preview` field (`{Availability, ManifestUrl, UnavailableReason}`) on `ChannelResponseModel`. | 2026-07-21 | [link](records/api/channel-preview-capability.md) |
|
||||
| `api.decode-by-id` | Endpoints that decode/expand opaque stored state accept a database row id and resolve it server-side rather than round-tripping client-supplied serialized state. | 2026-07-07 | [link](records/api/decode-by-id.md) |
|
||||
| `api.from-lineup-clear-to-none` | `POST /api/v1/channels/from-lineup` (and the Auto-Tune per-channel `advanced`, which reuses the same DTO) distinguishes *inherit* from *clear-to-none* with a typed `clear` enum list on `advanced`. A field left null/omitted still inherits the template value (unchanged for every existing client); naming a field in `clear` forces it to none on the new channel even when the template sets one. Sending both a set value and a clear for the same field is a validation error. | 2026-07-21 | [link](records/api/from-lineup-clear-to-none.md) |
|
||||
| `api.healthcheck-remediation-dto` | Health-check remediation is server-declared `{Kind, Target}` metadata on an additive DTO field; the SPA renders/acts on it, it doesn't derive labels itself. | 2026-07-17 | [link](records/api/healthcheck-remediation-dto.md) |
|
||||
| `api.healthcheck-ttl-cache` | Health-check results are held in a 30s TTL cache inside `HealthCheckService`; a non-forced `GET /api/v1/health` returns the cached list, and `?refresh=true` (or a forced internal caller) bypasses it to run fresh. | 2026-07-19 | [link](records/api/healthcheck-ttl-cache.md) |
|
||||
| `api.logs-sort-params` | `GET /api/logs` takes allow-listed `sortField` (`timestamp`\|`level`) and `sortDirection` (`asc`\|`desc`) query params, normalized (not rejected) on an unrecognized value. | 2026-07-11 | [link](records/api/logs-sort-params.md) |
|
||||
| `api.mediatr-passthrough` | The REST API is thin controllers over existing MediatR handlers, with no new service/business-logic layer. | 2026-06 | [link](records/api/mediatr-passthrough.md) |
|
||||
| `api.openapi-mirrors-runtime` | The generated OpenAPI spec is made to match the runtime Newtonsoft wire contract (via `NewtonsoftSchemaNamingTransformer`), not the reverse. | 2026-07-09 | [link](records/api/openapi-mirrors-runtime.md) |
|
||||
| `api.paging-zero-based` | `pageNum` is 0-based across the entire `/api/v1` surface and every wrapper of it (MCP tool catalog, SPA hooks, docs); the page offset is always derived from the EFFECTIVE (bounded) `pageSize`, never the requested one, so a `pageSize` above an endpoint's cap narrows the page without widening the offset. The cap itself is per-endpoint (100 typical, 200 auto-tune members, 1000 search/all-items) and must not be documented as one number. A paging parameter description that omits or contradicts "0-based" is a defect. | 2026-07-25 | [link](records/api/paging-zero-based.md) |
|
||||
| `api.parentid-drillin` | Media drill-in (season/episode/artist/music-video) is served by an optional `parentId` query param on library-browse, not dedicated per-kind child-listing endpoints. | 2026-07-07 | [link](records/api/parentid-drillin.md) |
|
||||
| `api.playout-build-lock-409` | Every id-keyed playout/channel mutation endpoint checks `IEntityLocker.IsPlayoutLocked(id)` and returns 409 Conflict while a build is in-flight, mirroring Blazor's disabled-buttons behavior; reset-all stays 202 and silently skips locked playouts. | 2026-07-10 | [link](records/api/playout-build-lock-409.md) |
|
||||
| `api.postcommit-cancellation-none` | Once a mutation commits, its entire compensating side effect (enqueues, publishes, reindexes, cache refresh, and any post-commit lookup gating one of those) runs on `CancellationToken.None` so a late client disconnect can't half-abort an already-committed change. | 2026-07-11 | [link](records/api/postcommit-cancellation-none.md) |
|
||||
| `api.put-replace-index-order` | PUT-replace-the-whole-list endpoints derive each item's `Index` from its request-array position, never a client-supplied field; alternate-schedule/playout-template rows are evaluated in `Index` order with the least-conditional row placed last as the catch-all default. | 2026-07 | [link](records/api/put-replace-index-order.md) |
|
||||
| `api.response-dtos` | New REST response DTOs live in `ErsatzTV.Core/Api/<Domain>/*ResponseModel.cs` with a file-scoped `#nullable enable` pragma; controllers never expose Application VM types directly. | 2026-07 | [link](records/api/response-dtos.md) |
|
||||
| `api.schedule-item-flat-dto` | Schedule-item GET/POST/PUT use a flat, non-polymorphic `ScheduleItemResponseModel` (every subtype field promoted to a nullable top-level member) instead of the polymorphic Application VM hierarchy, with mutation field names matching `ScheduleItemRequest` 1:1 for a lossless round-trip. | 2026-07-10 | [link](records/api/schedule-item-flat-dto.md) |
|
||||
| `api.scheduling-hardening` | Create/Replace handlers guard against null/whitespace `name` (`IsNullOrWhiteSpace`, not just `Length`) to prevent NRE-500s, template-item overlap validation compares by index (not record value-equality) to catch exact-duplicate items, and unreachable 404 `ProducesResponseType` attributes on create-only actions are trimmed. | 2026-07-13 | [link](records/api/scheduling-hardening.md) |
|
||||
| `api.search-allitems-paging` | `GET /api/v1/search/all-items` is paginated (capped page size, `Totals` field) to bound DoS exposure; the SPA add-all flow pages to completeness instead of relying on an unbounded response. | 2026-07-18 | [link](records/api/search-allitems-paging.md) |
|
||||
| `api.search-field-values-sources` | `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for a narrow allow-list of catalog fields (never the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source (`title`, `show_title` only); `limit` clamped to `[1, 50]` (default 50). The FINAL filter, dedup and ordering applied to the response are ORDINAL (`OrdinalIgnoreCase` / `StringComparer.Ordinal`), never current-culture, because `UseRequestLocalization` makes the culture caller-controlled — scoped to the in-memory stages on purpose: a field sourced by a plain EF query is filtered and truncated by the DATABASE collation first (SQLite's `LOWER()` is ASCII-only), which ordinal semantics downstream cannot undo (ersatztv#668). A field whose values live in an EF **primitive collection** (one JSON array per row in a single column: `SongMetadata.Artists`, `SongMetadata.AlbumArtists`) is served, not 404'd, as bounded best-effort, and its rows are read by a keyset page carrying **NO RESIDUAL predicate** — `SELECT Id, <col> AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch`, no `LIKE`, no `LOWER`, not even `IS NOT NULL`. The cursor is itself a predicate, but a SEEKABLE one on the ordering key: it positions the scan and never discards a row. A RESIDUAL predicate discards rows the engine already produced, and `LIMIT` truncates only the survivors — so with one present it bounds the OUTPUT rather than the row count. All selectivity is in memory. The guarantee is scoped: **at most 20,000 LOGICAL rows returned/materialized and at most 10 round trips (11 for `artist`)** — NOT bounded physical work and NOT bounded bytes, because MySQL traverses deleted-but-unpurged index records and the `TEXT`/`longtext` payload width is unrestricted. The walk pages 2,000 rows at a time, stopping on the first of enough distinct matches, a short page, or the ceiling. | 2026-07-26 | [link](records/api/search-field-values-sources.md) |
|
||||
| `api.search-field-values-unicode-fold` | The EF-sourced facet fields (`genre`, `show_genre`, `studio`, `director`, `writer`, `actor`, `tag`, `network`, `collection`, `video_codec`, `album`, and `artist`'s entity half) reach stored values whose prefix carries an uppercase non-ASCII character, on BOTH providers, with no row budget and no accepted loss. The defect was SQLite-only and ONE-SIDED: SQLite's `LOWER()` folds ASCII only (`lower('Édith')` is `'Édith'` unchanged), so the predicate UNDER-matched, which no later stage can repair. MySQL was already correct — its `LOWER()` is Unicode-aware, so `LOWER('Édith')` really is `'édith'` and the existing predicate reaches the row unaided. The fix is a SECOND, ADDITIVE query taken only when `isSqlite && q contains a non-ASCII character`: raw Dapper SQL `SELECT DISTINCT <col> AS Value FROM <table> WHERE [<discriminator> AND] etv_upper(<col>) LIKE @Pattern ESCAPE '\' ORDER BY <col> LIMIT @Limit`, where `etv_upper` is a `SqliteConnection.CreateFunction` scalar implementing `ToUpperInvariant`. Every other case — all-ASCII `q`, and MySQL for all `q` — runs today's EF query BYTE-IDENTICALLY. Keeping selectivity in SQL here is NOT the refuted family from `api.search-field-values-sources`: those four attempts bounded a walk around a predicate that could not be made correct over JSON escape text, whereas this is a correct fold on a plain column in an ordinary `LIMIT`ed query. It narrows that record's "Known limitation inherited, not introduced" clause; everything else it settles still holds. | 2026-07-27 | [link](records/api/search-field-values-unicode-fold.md) |
|
||||
| `api.search-paging-cap` | Search stays capped at 100 items per media kind; an overflowing kind's "See all" reuses library-browse paging instead of adding new API surface. | 2026-07-11 | [link](records/api/search-paging-cap.md) |
|
||||
| `api.selection-projection-include-chain` | Every handler that projects an aggregate carrying a tagged-union selection loads it through ONE shared `<Aggregate>QueryExtensions` include chain — `RerunCollectionQueryExtensions.IncludeSelectionDetails()`, joining the existing `ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails()` — called by the paged-list handler and the by-id handler alike, so the two cannot drift. The media-item flattening switch is likewise ONE shared helper, `MediaCollections.Mapper.ProjectMediaItemToViewModel`, covering all ten selectable media types including `RemoteStream`, whose named projection is `MediaItems.Mapper.ProjectToNamedViewModel` (it cannot be an overload of `ProjectToViewModel(RemoteStream)`, which already exists returning the unrelated `RemoteStreamViewModel`; C# will not overload on return type). That switch NEVER ends in `_ => null`: a null MediaItem is the legitimate not-a-media-item case, while an unrecognized non-null subtype keeps its id and takes a conspicuous `[unsupported media type: X]` name. Fail-soft is deliberate — throwing would fail an entire paged GET over one unreadable row. Finally, every metadata navigation inside `MediaItems.Mapper` is read through `Optional(...).Flatten()` and degrades to the `"???"` placeholder, because those projections are reached from handlers whose include chains differ and a bare `x.Season.Show.ShowMetadata` is a latent 500 on some other caller GET. | 2026-07-28 | [link](records/api/selection-projection-include-chain.md) |
|
||||
| `api.versioning-v1` | The entire `/api` surface is versioned to `/api/v1` uniformly (no unversioned corner); legacy unversioned callers are rewritten in-pipeline (not redirected) with Deprecation/Link/Sunset headers, and post-freeze `/api/v1` is additive-only — a breaking change requires `/api/v2`. | 2026-07-13 | [link](records/api/versioning-v1.md) |
|
||||
| `blazor.rollback-tag` | The commit immediately preceding the Blazor-removal merge is tagged `blazor-final` (not a `v*` tag, so it doesn't trigger a prod release build) as the documented rollback/restore path. | 2026-07-11 | [link](records/blazor/rollback-tag.md) |
|
||||
| `blazor.ui-removed` | The legacy Blazor Server UI (`Pages/`, `Shared/`, `ViewModels/`, `Validators/`, MudBlazor + 8 other packages, Blazor Startup wiring) is fully deleted now that the SPA has parity; the legacy `MapWhen` branch is kept only for controllers/docs/OpenAPI/`LegacyUiRedirects`, and the catch-all fallback 302s any unmatched non-api/artwork/docs/openapi path to `/app`. | 2026-07-11 | [link](records/blazor/ui-removed.md) |
|
||||
| `channel.origin-marker` | A new `Channel.Origin` (`ChannelOrigin` enum — `Unknown`/`UserCreated`/`AutoTuned`) records how a channel row was created and is stamped exactly once at insert (`AutoTuned` in `CreateChannelFromLineupHandler`, `UserCreated` in `CreateChannelHandler`), and is never mutated on a later edit. It is surfaced as a raw `origin` field on `ChannelResponseModel`; the SPA badges only `AutoTuned`. Rows predating the column read `Unknown` — provenance is **not** back-filled. | 2026-07-23 | [link](records/channel/origin-marker.md) |
|
||||
| `ci.batch-pushes-no-cancel-route` | Hold review fixes, doc corrections and format fixes locally and push **once** — a superseded run cannot be cancelled from the agent side and holds a runner slot until it finishes. | 2026-07-21 | [link](records/ci/batch-pushes-no-cancel-route.md) |
|
||||
| `ci.build-once-rejected` | CI build-once (a shared compile artifact across jobs) was implemented, measured, and rejected for a 40-85% wall-clock regression; keep the #420 cross-run tree-identity skip instead. | 2026-07-18 | [link](records/ci/build-once-rejected.md) |
|
||||
| `ci.cancelled-is-not-a-verdict` | Treat a `cancelled` conclusion as "no verdict" — never as pass or fail — and report FAILED and CANCELLED counts separately in any CI monitor. | 2026-07-21 | [link](records/ci/cancelled-is-not-a-verdict.md) |
|
||||
| `ci.decisions-edit-trailer` | The body-diff exemption is armed by an affirmative `Decisions-Edit:` **git trailer** (`yes`/`true`/`1`, case-insensitive, read with `unfold`) on some NON-MERGE commit in the PR's merge-base range — never by a substring search over the message text. A non-affirmative value (`no`) does not arm it, the retired `[decisions-edit]` substring arms nothing (the validator emits a `::warning::` nudge when it sees one without a trailer), and a git error leaves the guard ON. | 2026-07-25 | [link](records/ci/decisions-edit-trailer.md) |
|
||||
| `ci.decisions-lifecycle-flake` | When `decisions lifecycle` is the **only** red job, do not investigate and do not create a new run to clear it — no rebase, no `--amend`, no no-op push; the operator reruns that single job from the Gitea UI. | 2026-07-21 | [link](records/ci/decisions-lifecycle-flake.md) |
|
||||
| `ci.docs-only-detect-shallow-safe` | The docs-only detect script must diff against `FETCH_HEAD` (always resolves after `git fetch`, even shallow) using a two-dot tree diff — not `origin/<base>` with three-dot — because a `fetch-depth: 1` shallow clone has no remote-tracking ref and no merge-base, which silently fails the original detect into `docs_only=false` (full matrix, no functional error). A CI-behavior change must be verified by measuring the effect (job durations), not just a green check. | 2026-07-17 | [link](records/ci/docs-only-detect-shallow-safe.md) |
|
||||
| `ci.docs-only-skip-steps` | A docs-only change must still run every required job (`test`, `migrations`) so their commit-status contexts always report; each heavy job runs `scripts/ci-detect-docs-only.sh` first and gates its real STEPS on `if: steps.detect.outputs.docs_only != 'true'`, never `if:`-skips the whole job (an `if:`-skipped job reports `skipped`, not `success`, which branch protection may never unblock on). Detection biases toward running more on any doubt. | 2026-07-17 | [link](records/ci/docs-only-skip-steps.md) |
|
||||
| `ci.exemption-provenance` | The three inputs the exemption decision rests on must each be bound to something the judged PR cannot mutate. (1) BASE — `scripts/pr-changed-files.sh` takes the expected base BRANCH as a REQUIRED 5th argument and re-reads it before and after paging, because `/pulls/{n}/files` diffs against the PR's live base and retargeting moves the answer without moving the head sha; the workflow passes `github.event.pull_request.base.ref` from the `pull_request_target` payload, which a retarget cannot rewrite. (2) BOT EXEMPTION — an author match is necessary but never sufficient: `pull_request.user.login` is the PR's immutable CREATOR while its head is not, so the exemption additionally requires EVERY changed path to be a dependency manifest (`Directory.Packages.props` or `.config/dotnet-tools.json`, and ONLY those — the npm manifests are excluded because `package.json` `scripts` are executed by CI). (3) INHERITED SUCCESS — the never-overwrite short-circuit fires only for a status POSITIVELY identified as a human verdict for THIS base, meaning a non-null `.creator.login` AND a `Review-verdict:` description AND, when that description records a base (`(base: …)`, `release.verdict-status-check`), a base matching the PR's — tested by requiring the description to END with the exact literal `(base: <base>)` and to contain exactly ONE such marker, never by extracting a value (see below); a present-but-different base is rejected, an absent one is not, since verdicts predating that convention carry none; every other shape, including any unrecognised one, is re-derived rather than trusted. The bot and docs-only exemptions are evaluated as INDEPENDENT predicates and the decision made afterwards, never as an `elif` chain. `edited` is in the workflow's `types:` so a retarget reclassifies — which gives DETECTION, not atomicity: status writes are not serialized, so a stale run can still post over a fresher one. That residual is now FENCED rather than merely tracked — the job refuses to write at all if the PR's timeline retarget COUNT moved while it was classifying (`ci.verdict-write-retarget-fence`, #706) — leaving only the sub-round-trip window that no API without compare-and-set can close. The PROTECTED path list additionally covers `.codex/` (#711), which mirrors `.claude/hooks/` byte for byte including the merge-consent hook — latent while that directory is untracked, live the moment it is tracked; the list stays ENUMERATIVE rather than derived, because a derived rule would have to be evaluated against the very file list being classified. Path predicates are evaluated by COUNTING with `grep -c`, never `\| grep -q` (SIGPIPE inversion) and never a here-string (temp-space failure) — see `ci.grep-q-pipefail-inversion`. | 2026-07-29 | [link](records/ci/exemption-provenance.md) |
|
||||
| `ci.format-gate-folder-mode` | The blocking `format` CI job (and matching pre-commit hook) runs `dotnet format whitespace . --folder --include <files>` instead of loading the full MSBuild/Roslyn solution, cutting the gate from ~480s to ~0.5s with unchanged whitespace/charset coverage. | 2026-07-19 | [link](records/ci/format-gate-folder-mode.md) |
|
||||
| `ci.functional-e2e-harness` | The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. Originally curl-only; since #445 the same job carries a second, headless-browser step for the contracts curl cannot express — see `ci.ui-e2e-harness`. | 2026-07-16 | [link](records/ci/functional-e2e-harness.md) |
|
||||
| `ci.gate-trigger-base-resolved` | The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR's own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch's gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree's scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class: Gitea injects a write-capable `GITEA_TOKEN` into EVERY job, so any ref-resolved workflow — and a collaborator's own API token, since branch protection binds the context and not its issuer — can still forge `review-verdict/h10`. Tracked in #697; the exemption path has its own separate defects in #698. | 2026-07-28 | [link](records/ci/gate-trigger-base-resolved.md) |
|
||||
| `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=<name>` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](records/ci/gitea-milestone-filter-noop.md) |
|
||||
| `ci.grep-q-pipefail-inversion` | In any script running under `set -o pipefail`, a security or classification predicate of the form `producer \| grep -q…` is FORBIDDEN: `grep -q` exits at its first match, the producer then takes SIGPIPE and exits 141 once the data exceeds the pipe buffer (~64K), so `pipefail` reports the pipeline as FAILED even though grep MATCHED — inverting the predicate exactly when the input is large. A here-string (`grep -q… <<< "$data"`) is ALSO forbidden: bash materialises a large here-string via temporary storage, so it fails when temp space is full or unwritable, and inside an `if`/`!` that failure flips the predicate the same way. COUNT instead — `n=$(printf '%s\n' "$data" \| grep -cE "$re")` — because `grep -c` drains stdin (no early exit, no SIGPIPE) over an ordinary pipe (no temp file). Read grep's status honestly: exit 1 means a zero count and is a legitimate answer, anything >1 is a real error. Evaluate the counts ONCE at TOP LEVEL, never inline inside an `if`/`elif` condition: inside `$( )` an `exit` leaves only the subshell and `set -e` does not fire, so an error silently reads as "no match". Validate that each result is numeric and fail closed if not. This applies to both the enforced gate `.gitea/workflows/review-verdict.yml` and the advisory hook `.claude/hooks/pretooluse-merge-consent.sh`. | 2026-07-29 | [link](records/ci/grep-q-pipefail-inversion.md) |
|
||||
| `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](records/ci/infra-shaped-red-under-load.md) |
|
||||
| `ci.jq-version-contract` | Every shell gate that shells out to `jq` is authored to the jq 1.6-compatible subset, because the CI runner ships jq 1.6 while every developer Mac ships 1.8.x. `scripts/jq-preflight.sh` (no args) prints the parsed version and asserts a floor of 1.6 in every gate job's log; `scripts/jq-preflight.sh --expect 1.6` additionally pins and fails loudly, but ONLY in the `script-tests` job. `review-verdict.yml` never pins — it writes the branch-protection-required `review-verdict/h10` status, so a hard pin there would turn any jq bump into a repo-wide merge deadlock. | 2026-07-26 | [link](records/ci/jq-version-contract.md) |
|
||||
| `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](records/ci/killed-job-triage.md) |
|
||||
| `ci.monitor-armed-at-pr-open` | Arm a CI monitor on the PR head sha the moment the PR opens, polling the commit-status endpoint — not at the end of the work. | 2026-07-21 | [link](records/ci/monitor-armed-at-pr-open.md) |
|
||||
| `ci.no-host-health-gating` | Push when your work is validated — never SSH to bumblebee to sample load/RAM first, and never hand-schedule around other sessions' runs. | 2026-07-21 | [link](records/ci/no-host-health-gating.md) |
|
||||
| `ci.peak-anon-measurement` | The `test` job's headline memory figure is a sampled high-water mark of cgroup `anon`, produced by `scripts/ci-peak-anon.sh`; `memory.peak` and the end-of-job `anon`/`file` split are kept only as a cache-inflated reference. | 2026-07-19 | [link](records/ci/peak-anon-measurement.md) |
|
||||
| `ci.root-screenshot-guard` | The Husky `pre-commit` hook refuses a staged root-level `*.png` (belt-and-suspenders with the `.gitignore` rule); nested `*.png` real assets are unaffected. | 2026-07-12 | [link](records/ci/root-screenshot-guard.md) |
|
||||
| `ci.runner-placement` | No persistent Roslyn compiler server survives a CI build (`UseSharedCompilation=false` etc., runner env + Dockerfile `ENV`); every `services:` container gets its own explicit `--memory`/`--memory-swap`/`--cpus` cap (it does not inherit the job container's). | 2026-07-17 | [link](records/ci/runner-placement.md) |
|
||||
| `ci.script-tests-job` | The `scripts/tests/` pytest suite runs on every PR as a dedicated `script-tests` job in `pr-checks.yml` (`runs-on: small`, `setup-python` + `pip install pytest`, `PYTHONPATH=. python3 -m pytest scripts/tests -q`), unconditionally rather than behind a `scripts/**` path filter, and **never as a step inside `decisions-guard`** — a job whose reds a standing rule instructs sessions to ignore must never host a gate whose reds are real. Any new CI gate must be reachable by a failure that is unambiguously attributable to it. | 2026-07-26 | [link](records/ci/script-tests-job.md) |
|
||||
| `ci.shared-pr-file-enumeration` | A PR's complete set of changed file paths is computed by exactly one implementation, `scripts/pr-changed-files.sh`, called by both `.claude/hooks/pretooluse-merge-consent.sh` (advisory — a failure falls through to a human prompt) and `.gitea/workflows/review-verdict.yml` (enforced — a failure must fail closed, because a match here posts the branch-protection-required `review-verdict/h10` status with nobody in the loop). The script owns exhaustiveness (pagination, rename/path validation, head-sha binding, base-ref binding — see `ci.exemption-provenance` — and base-TIP binding, #707: the ref answers "did this PR RETARGET", the tip answers "did the base ADVANCE mid-enumeration", and only the second can see `/pulls/{n}/files` recomputing each offset-paged page against a moved base and dropping a path out of an already-consumed range; both ends of the window are bound, and an advance BEFORE the window is deliberately not an error, or ordinary churn on `main` would fail every open PR) and returns exit 0 only for a verified-complete list; it does NOT classify paths — each caller keeps its own docs-only allow-list, and the two allow-lists differ on purpose and stay separate. | 2026-07-26 | [link](records/ci/shared-pr-file-enumeration.md) |
|
||||
| `ci.small-lane-git-only` | `runs-on: small` is defined by what a job does (git-only), not its usual runtime; the two `docker build` jobs (docker-build.yml, ci-image.yml) move to `ubuntu-latest` because their worst-case memory, not median runtime, was pinning the small lane's per-slot cap. | 2026-07-20 | [link](records/ci/small-lane-git-only.md) |
|
||||
| `ci.ui-e2e-harness` | The UI-interactive E2E flows run as headless Playwright specs (`web/e2e/*.spec.ts`, driven by `scripts/e2e-ui.sh`) in a **second step of the existing advisory `functional-e2e` job**, never their own job; the browser is `chromium-headless-shell` **baked into the CI toolchain image** (`docker/ci/Dockerfile`, `PLAYWRIGHT_VERSION` kept equal to `web/package.json`'s EXACT `@playwright/test` pin), never installed per run; specs are `serial` with `retries: 0` and assert only contracts the curl harness structurally cannot reach. | 2026-07-25 | [link](records/ci/ui-e2e-harness.md) |
|
||||
| `ci.verdict-write-retarget-fence` | The `review-verdict/h10` job counts `change_target_branch` events on the PR's issue timeline at run start and again immediately before its POST, and writes NOTHING if the count moved. The COUNT is the key because the branch NAME is ABA-vulnerable — `main -> S -> main` reads `main` at both ends, which is how #698 route 1 obtained a forged exemption — while the event count is monotonic and cannot alias. Abstaining is a handoff, not a stall, and that is the property the design rests on: every retarget fires `edited`, which is in this workflow's `types:`, so the event that makes a run abstain has already queued a successor whose window opens after it; the induction terminates when retargeting stops and the last run writes the final answer. `updated_at` was REJECTED as the key because it also moves for comments and labels, which fire none of this workflow's `types:` — a run could abstain with no successor coming, which is a real stall. The count is trusted only when paging reached a validated EMPTY page; an untrusted count (unreadable page, non-array body, non-numeric length, page cap hit) blocks the exemption `success` ONLY and still lets `pending` through, because `pending` cannot turn an unreviewed head green while withholding it would strand ordinary PRs for no safety gain. SEPARATELY, and for the human-verdict race the fence does nothing about: after posting an exemption `success` the job re-reads `/statuses/{sha}` and, if a human `Review-verdict:` row appeared with an id ABOVE a high-water mark taken just before the POST, overwrites its own status with `pending` and logs an error. The repair is `pending`, NEVER a copy of the human's state, since re-posting their `failure` under the machine credential would attribute a human verdict to the job; its description is a SENTINEL that the classification refuses to grant an exemption over AND re-writes verbatim on every later run, so the block is a FIXED POINT rather than decaying — writing the generic `pending` description there instead erases the marker and the exemption simply returns one event later. The mark is captured BEFORE the last-moment re-read, not merely before the POST — a later mark leaves a multi-round-trip blind gap in which a verdict is neither seen by the re-read nor repaired afterwards. The id comparison is load-bearing: a mere presence test would fire forever on a base-mismatched verdict that `read_existing_verdict` deliberately declines to honour, deadlocking that PR's exemption permanently. Finally, a run whose last-moment re-read finds a sentinel it did not see at its FIRST read ABSTAINS instead of posting: that can only mean an overlapping run repaired a raced verdict mid-flight, and this run's `success` — frozen at classification time, with the human row below its own mark, so neither the fence nor the post-write check would catch it — would otherwise bury the rejection. That is the one path in this design that failed toward SUCCESS rather than `pending`. The post-write check counts TWO row shapes above the mark, not one — a human `Review-verdict:` row AND a machine sentinel — because with two overlapping runs the human row can sit BELOW the second run's mark while the first masks it and only then writes the sentinel, leaving the second to post its own `success` on top; counting the sentinel converges both runs on the fixed point instead. | 2026-08-03 | [link](records/ci/verdict-write-retarget-fence.md) |
|
||||
| `ci.verify-locally-ci-confirms` | Treat the local build/verify/review pass as the decision point and CI as confirmation — don't idle waiting on a run you have no reason to doubt. | 2026-07-21 | [link](records/ci/verify-locally-ci-confirms.md) |
|
||||
| `ci.web-test-per-test-timeouts` | Give heavy-render web tests an explicit per-test vitest timeout (e.g. 15s); never raise the global default to fix one slow test. | 2026-07-21 | [link](records/ci/web-test-per-test-timeouts.md) |
|
||||
| `concurrency.diff-scalar-fanout` | The frozen Block optimistic-concurrency recipe (api-conventions §7a) fans out to Collection/Playout×2/MultiCollection/RerunCollection, keeping a guard-returned `PreconditionFailedError` out of any handler's generic `catch(Exception)`→422 mapping, and preserving each aggregate's existing `SaveChangesAsync() > 0` gate semantics under the new unconditional `Version++`. | 2026-07-11 | [link](records/concurrency/diff-scalar-fanout.md) |
|
||||
| `concurrency.etag-rotation-completion` | Every handler that mutates a versioned root's editor-visible config state must bump `Version` (rotating the ETag) with no per-aggregate carve-outs, short-circuiting on a genuine no-op before the bump so idempotent re-submits don't fire spurious rebuild fan-out; `SaveChangesForcingVersion` rebases the retry (stored + pending delta), never adopts the stored token verbatim. | 2026-07-12 | [link](records/concurrency/etag-rotation-completion.md) |
|
||||
| `concurrency.force-write-non-ifmatch` | Any handler that leaves a versioned root `Modified` or `Deleted` but takes no `If-Match` (deletes, item add/remove bumpers, scalar-config writers) must save through `ConcurrencyExtensions.SaveChangesForcingVersion` — force-write past a concurrent `Version` bump rather than throw an unhandled `DbUpdateConcurrencyException` (500). | 2026-07-12 | [link](records/concurrency/force-write-non-ifmatch.md) |
|
||||
| `concurrency.idempotent-concurrent-add` | A concurrent duplicate `Add*ToCollection` that loses the race on the composite-key unique constraint is treated as an idempotent no-op (skip the reindex/rebuild fan-out), not a 500 — detected via a provider-specific `TvContext.IsUniqueConstraintViolation` delegate defaulting to "no". | 2026-07-18 | [link](records/concurrency/idempotent-concurrent-add.md) |
|
||||
| `concurrency.ifmatch-rfc7232` | `ConcurrencyHeaders.ParseIfMatch` is a real RFC 7232 entity-tag/list parser: a syntactically-valid tag that doesn't strong-match (weak/empty/non-canonical/out-of-range/list) returns 412, and only a genuine grammar violation returns 400. | 2026-07-12 | [link](records/concurrency/ifmatch-rfc7232.md) |
|
||||
| `concurrency.replace-all-contract` | Replace-all aggregate PUTs carry a uniform plain `int Version` concurrency token (EF `.IsConcurrencyToken()`), checked pre-save and enforced by the EF UPDATE guard, returning 412 (not 409) on a stale `If-Match`. | 2026-07-11 | [link](records/concurrency/replace-all-contract.md) |
|
||||
| `concurrency.schedule-item-child-identity` | `PUT /api/schedules/{id}/items` reconciles by an optional round-tripped child `Id` (null/absent/0 ⇒ new item), never by array position, so fill-group/shuffle state follows the logical item across reorders; an unknown or duplicate id is rejected 422 (checked after the §7a `CheckVersion`, so 412 precedes 422). | 2026-07-11 | [link](records/concurrency/schedule-item-child-identity.md) |
|
||||
| `docs.convention-docs-session-start` | Docs-first, not source-first: conventions (api-conventions, spa-conventions, e2e-local, blazor-route-parity, domain-model, decisions, README) are read from docs, not reverse-engineered from code, via `docs/README.md`'s task-signal map — only the sections it points to for the task at hand, not the whole set. Each doc is updated in the same PR that changes what it documents, replacing deferred/follow-up doc updates. | 2026-07-07 | [link](records/docs/convention-docs-session-start.md) |
|
||||
| `docs.corpus-size-signal` | The corpus's size signal is a per-record prose ceiling (`decisions_validate.py --record-ceiling`, default 60, chosen at a natural gap in the distribution), reported as a NON-BLOCKING `::warning::` naming each record over it. The aggregate prose total is still printed every run but carries NO threshold — it is a `::notice::` trend only — because a total over a monotonically growing corpus can only ratchet, and the generated catalog (`docs/decisions/README.md`) is no longer counted at all since it gains one row per record and cannot be consolidated away. Being listed by the ceiling is an invitation to check for REDUNDANCY, never an instruction to cut: a long record that is all distinct findings is a legitimate decline, and should be recorded as one. The ceiling's CALIBRATION is guarded in two pieces of different robustness (#688): the blocking test asserts only the coarse, non-ratcheting property that the ceiling flags a MEANINGFUL MINORITY of records (`0.02 <= fraction_over <= 0.25`), while the fine claim — that it sits between p90 and p95 — is REPORTED by `main()` as a `::notice::` and never asserted against the live corpus. A ceiling drifting out of date is the passage of corpus growth, not a defect in the commit under test, so it gets `stale_records`' treatment rather than a red in the blocking `script-tests` job. | 2026-07-26 | [link](records/docs/corpus-size-signal.md) |
|
||||
| `docs.decision-lifecycle` | every decision `##` record (active or archived) carries a 5-field metadata block (`key`, `status`, `since`, `supersedes`, `superseded-by`) checked by `scripts/decisions_validate.py`; a record is never deleted or line-edited to reverse a call — it is moved to `docs/decisions/archive/` with `status: superseded`/`retired` and a reciprocal `superseded-by`/`supersedes` key pair to its replacement. | 2026-07-21 | [link](records/docs/decision-lifecycle.md) |
|
||||
| `docs.decision-one-file-per-record` | Each decision record is its own file at `docs/decisions/records/<area>/<topic>.md` (archived ones at `docs/decisions/archive/<area>/<topic>.md`) with YAML frontmatter; the filename IS the key, so one-active-record-per-key is a filesystem property rather than a validator check, and supersession is a `git mv`. | 2026-07-25 | [link](records/docs/decision-one-file-per-record.md) |
|
||||
| `docs.decision-optional-provenance` | Decision records gain two OPTIONAL fields — `stale-after: YYYY-MM-DD` on the metadata line and a `**Sources:**` line in the metadata block; the Open Knowledge Format (OKF) itself is NOT adopted as the record format. | 2026-07-25 | [link](records/docs/decision-optional-provenance.md) |
|
||||
| `docs.frontmatter-pyyaml-crosscheck` | `decisions_validate.py` runs `pyyaml_frontmatter_faults()` over every record-wing file: it loads the frontmatter with PyYAML and reports an ERROR when PyYAML rejects the document OR when any key's value differs from what the dependency-free `dl._read_frontmatter` read. PyYAML is the WRITER of these files (`migrate_decisions_split.render_record` emits them with `yaml.safe_dump`), so on any disagreement PyYAML is authoritative and the defect is in the FILE, not in either parser. The check is strictly additive: when PyYAML is not importable it is SKIPPED and `main()` says so with a `::notice::`, never silently — the read path stays dependency-free because `decisions-guard`, the Husky hooks and contributor machines install nothing. The comparison has exactly ONE implementation, called by both the validator and `test_frontmatter_reader_matches_pyyaml_on_every_real_record`, so the suite and the tool cannot drift on what "matches PyYAML" means. | 2026-08-04 | [link](records/docs/frontmatter-pyyaml-crosscheck.md) |
|
||||
| `docs.record-wing-parse-guard` | `decisions_validate.py` asserts, per PATH, that every `*.md` under `docs/decisions/records/**` and `docs/decisions/archive/**` parses to exactly one record carrying a `key` — an ERROR, not a warning, since a file in the record wings that is not a record is a mistake by definition. A file sitting DIRECTLY in `archive/` is exempt only when it actually looks like a #610 stripped index — exactly one keyless record with a known generated heading — never merely by living there. The one other exemption, `archive/README.md`, is by exact RELATIVE PATH; nothing is ever exempt by BASENAME, since that would exempt the same filename in the active wing too. `_read_frontmatter` is deliberately NOT extended to accept YAML block scalars: every record value goes on ONE line, and the structural check is what makes that limitation loud instead of silent. | 2026-07-26 | [link](records/docs/record-wing-parse-guard.md) |
|
||||
| `docs.tracker-comment-retrofit` | When the knowledge exporter flags an over-cap tracker issue and excludes it from ingestion, triage its comments instead of assuming a retrofit is owed — and for each decision-shaped item check the **worked issue first**, because a tracker session comment is by construction a précis of the fuller closing record posted on the issue it narrates. Applied to #237 (111 comments) this yielded **zero** records, so server-management#642's "a fact found only in a #237 comment" retrieval row has no valid subject and its interim target (an already-migrated record) is permanent. | 2026-07-21 | [link](records/docs/tracker-comment-retrofit.md) |
|
||||
| `ffmpeg.external-logo-graphics-engine` | External-URL channel logos pass through to the graphics engine like any other watermark source; `WatermarkSelector` must never gate them on `File.Exists` (always false for a URL) and never route them through the ffmpeg-native overlay shortcut. | 2026-07-20 | [link](records/ffmpeg/external-logo-graphics-engine.md) |
|
||||
| `ffmpeg.hls-cold-start-burst` | HLS cold-start latency is fixed with a bounded `-readrate_initial_burst` (gated on FFmpeg ≥6.1 capability detection), not by raising `work_ahead_limit`, which would remove the concurrency guarantee it exists for. | 2026-07-20 | [link](records/ffmpeg/hls-cold-start-burst.md) |
|
||||
| `ffmpeg.qsv-decode-encode-split` | QSV decode is decoupled from QSV encode via a single `FFmpegProfile.QsvPreferNativeDecoder` bool (default ON, Linux-only), so a QSV encode profile can decode with the more tolerant native VA-API decoder instead of the QSV decoder, mirroring Jellyfin's hybrid decode/encode toggle instead of a general decode-family enum. | 2026-07-20 | [link](records/ffmpeg/qsv-decode-encode-split.md) |
|
||||
| `ffmpeg.qsv-extra-hw-frames-floor` | a QSV upload never emits `extra_hw_frames` below `FFmpegState.MinimumQsvExtraHardwareFrames` (64); a stored `0` or negative value is treated as "no pool configured" rather than honored literally, because with no headroom any unthrottled read exhausts the pool and the transcode writes nothing at all. | 2026-07-21 | [link](records/ffmpeg/qsv-extra-hw-frames-floor.md) |
|
||||
| `ffmpeg.qsv-hdr-tonemap-opencl` | the QSV pipeline never emits `vpp_qsv=tonemap=1`, which is a SILENT no-op on pre-Gen11 Intel graphics; HDR is tonemapped on the GPU via `hwupload=derive_device=vaapi` → `scale_vaapi` → `hwmap=derive_device=opencl` → `tonemap_opencl` when a VA-API device exists, the frames are still in software, and `tonemap_opencl` is available, and by the software `TonemapFilter` otherwise. The scale runs BEFORE the tonemap, and any hardware filter on the path forces the output to be re-tagged bt709. | 2026-07-26 | [link](records/ffmpeg/qsv-hdr-tonemap-opencl.md) |
|
||||
| `ffmpeg.readrate-catchup-sparse-streams` | a realtime video/audio input also gets `-readrate_catchup` (6.0) when the binary supports it — but NOT a still-image input (mirroring the #350 exclusion) and NOT a concat input, which keep at most bare `-readrate` (a still image's video input takes none at all). Reason: `-readrate` paces the whole input off its furthest-behind stream, so a sparse stream sharing that input (an embedded PGS/DVD bitmap subtitle feeding the overlay) otherwise pins output at ~0.53x realtime. Catchup is a ceiling that applies only WHILE an input is behind, never a target, so it does not let a caught-up input race ahead. | 2026-08-04 | [link](records/ffmpeg/readrate-catchup-sparse-streams.md) |
|
||||
| `ffmpeg.remote-image-fetcher-bounded` | remote graphics-engine images are fetched through `IRemoteImageFetcher` with a pooled `HttpClientFactory` client, a body-covering deadline, a wire-transfer size cap, and a decoder-enforced `DecoderOptions.MaxFrames` bound re-verified post-decode — never cached, re-fetched per element init. | 2026-07-20 | [link](records/ffmpeg/remote-image-fetcher-bounded.md) |
|
||||
| `ffmpeg.watermark-resolution-unified` | Every watermark `WatermarkSelector` resolves goes through one shared `ResolveWatermark` — the playout-item, channel and global precedence levels AND the deco path, for all three `ChannelWatermarkImageSource` values. An unresolvable watermark (missing file, un-migrated external URL, or no logo artwork) resolves to no on-screen bug plus a warning, never a dead path or a URL handed downstream; the one deliberate exception is a playout-item `Custom` with a blank image, which still falls THROUGH to channel/global. The generated-initials fallback is therefore off everywhere, including the deco path where it demonstrably rendered. Watermarks built OUTSIDE the selector (the song-progress overlay, #653) are not covered and remain unchecked. | 2026-07-26 | [link](records/ffmpeg/watermark-resolution-unified.md) |
|
||||
| `ffmpeg.work-ahead-slot-atomic` | `workAheadSegmenterLimit` is enforced by a single compare-exchange claim on a shared `WorkAheadSlots` pool taken by the *caller* of `Transcode`, which then passes ownership in and gets the release in `Transcode`'s `finally` — never a `Volatile.Read` compare in one place and an `Interlocked.Increment` in another. | 2026-07-21 | [link](records/ffmpeg/work-ahead-slot-atomic.md) |
|
||||
| `ffmpeg.work-ahead-slot-release-never-negative` | `Release()` reads the count and compare-exchanges `current - 1` only when `current > 0`; a release against an empty pool records an unbalanced release and returns `false` **without ever writing a negative value**. It never decrements first and clamps afterward. The single caller (`HlsSessionWorker.Transcode`'s `finally`) logs a warning on the `false` return. | 2026-07-21 | [link](records/ffmpeg/work-ahead-slot-release-never-negative.md) |
|
||||
| `graphics.channel-level-attachment` | A channel can attach `GraphicsElement`s directly via a new `ChannelGraphicsElement` join table (a base layer under deco/playout-item elements), and a built-in text element (`on-now-next.yml`) is seeded once per database so the On Now/Next overlay works out of the box. | 2026-07-22 | [link](records/graphics/channel-level-attachment.md) |
|
||||
| `graphics.channel-logo-caching` | An external `http(s)` channel-logo URL is fetched, decode-budget-validated, and stored in the image cache under a content-hash name at SAVE time — becoming byte-identical to an uploaded logo — so the render path never fetches a logo over HTTP; a bad URL fails the save with a 422 (BaseError → ValidationProblemDetails). | 2026-07-21 | [link](records/graphics/channel-logo-caching.md) |
|
||||
| `iptv.base-url` | An optional advertised base URL (`iptv.base_url`) is resolved centrally via a pure Core helper (`AdvertisedBaseUrl`) inside the two IPTV generation handlers (M3U + XMLTV); unset/malformed values fall back byte-identical to the request-derived host, and it's a new `iptv` settings group distinct from `ETV_BASE_URL` and out of scope for HDHomeRun. | 2026-07-16 | [link](records/iptv/base-url.md) |
|
||||
| `iptv.logo-drives-bug-preset` | One uploaded channel logo drives both the listing logo and the on-screen bug via a shared, seeded `ChannelLogo`-sourced watermark preset (`Channel Bug`), not new per-channel schema. | 2026-07-20 | [link](records/iptv/logo-drives-bug-preset.md) |
|
||||
| `locking.entitylocker-atomic-flags` | `EntityLocker` uses `Interlocked.CompareExchange`-guarded atomic flags plus a documented single-owner-release discipline (no owner tokens/leases); `Unlock*` on an already-unlocked slot returns `false` and logs a Warning rather than throwing. | 2026-07-11 | [link](records/locking/entitylocker-atomic-flags.md) |
|
||||
| `mcp.server-foundation` | `ErsatzTV.Mcp` is a fresh stdio JSON-RPC server wrapping frozen `/api/v1` with explicit narrow per-endpoint tools, read-only-by-default enforced at runtime (`ERSATZTV_ALLOW_WRITES`), machine-key auth, and opt-in `If-Match`. | 2026-07-20 | [link](records/mcp/server-foundation.md) |
|
||||
| `media.lastscan-null-boundary` | A never-scanned `LastScan` surfaces as `null` at the API/MCP boundary, not the `0001-01-01` MinValue sentinel — enforced by an ongoing read-boundary coercion plus a one-time data migration cleanup. | 2026-07-18 | [link](records/media/lastscan-null-boundary.md) |
|
||||
| `media.remote-stream-probe` | `ValidatePlayoutItemPath` probes the Plex/Jellyfin/Emby remote-stream URL via `IRemoteStreamProber` before returning it; only a redirected 404 fails closed (`PlayoutItemNotAvailableFromMediaServer`), everything else fails open, and there is no toggle. | 2026-07-19 | [link](records/media/remote-stream-probe.md) |
|
||||
| `media.remote-stream-probe-externaljson` | External-JSON playout channels' `StreamRemotely` now probes the remote-stream URL through the same `IRemoteStreamProber` seam as the generated-playout path, closing the #473 scope gap for a channel kind with no DB `PlayoutItem` rows. | 2026-07-20 | [link](records/media/remote-stream-probe-externaljson.md) |
|
||||
| `media.source-mgmt-write-api` | Media-source management (local/Plex/Jellyfin/Emby) is a REST write API + SPA under `/app/libraries/*`, wrapping existing MediatR commands 1:1 with no new commands or DB migration; connection GETs never leak a stored `apiKey`, and each PUT-replace family's identity contract is documented per-family (not assumed uniform). | 2026-07-11 | [link](records/media/source-mgmt-write-api.md) |
|
||||
| `process.bom-format-detection-recipe` | Before any push touching `.cs`, detect BOMs with the `xxd` byte check and verify the format gate with `dotnet format --include` run under `bash -c`, never bare zsh. | 2026-07-21 | [link](records/process/bom-format-detection-recipe.md) |
|
||||
| `process.branch-off-feature-branch` | To fix work on an unmerged feature branch, branch off that branch and land by fast-forward push — and after creating a worktree, drive the first Edit/Read from ITS absolute paths and `git status` it before building. | 2026-07-21 | [link](records/process/branch-off-feature-branch.md) |
|
||||
| `process.build-concurrency-limits` | Run at most 3–4 concurrent dotnet/npm builds on this Mac, gate launches on FREE RAM rather than CPU load, and never set `ETV_UPDATE_GOLDENS` / `ETV_UPDATE_PLAYOUT_GOLDENS`. | 2026-07-21 | [link](records/process/build-concurrency-limits.md) |
|
||||
| `process.codex-cheap-worker-launch` | For bounded tool-bearing selector/recon work, launch a Codex worker with `codex exec -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only`; `spawn_agent` buys parallelism but no cost savings. | 2026-07-21 | [link](records/process/codex-cheap-worker-launch.md) |
|
||||
| `process.consistency-fix-new-code-scrutiny` | Review a "make X consistent with Y" change as new code, not as a mechanical copy — and for any timer or effect involved, ask explicitly "when does this fire?", including on mount. | 2026-07-21 | [link](records/process/consistency-fix-new-code-scrutiny.md) |
|
||||
| `process.enumerate-workaround-behaviors-before-deleting` | When an issue says "delete X", enumerate every behavior X provided before removing it — a workaround often serves a second purpose that outlives the first. | 2026-07-21 | [link](records/process/enumerate-workaround-behaviors-before-deleting.md) |
|
||||
| `process.foreign-worktree-plumbing-merge` | Never commit or merge inside a worktree another session created; land the merge with git plumbing against the branch ref instead. | 2026-07-21 | [link](records/process/foreign-worktree-plumbing-merge.md) |
|
||||
| `process.harden-with-runtime-posture-not-clamp` | When a security fix constrains a capability the roadmap will later want, make the safe state the DEFAULT OF A SWITCH rather than a wall — and read the feature's own issue for its end-state first. | 2026-07-21 | [link](records/process/harden-with-runtime-posture-not-clamp.md) |
|
||||
| `process.independent-review-rubric` | Run an independent review pass — preferably a different model family, otherwise a cold-context review-only agent — on any diff touching locks/concurrency, auth/security, API write-path handlers, or DB migrations, or larger than ~150 changed C# lines; skip only for a pure-SPA/docs leaf with no server-state effect, and state the skip and its reason in the PR or close comment. | 2026-07-21 | [link](records/process/independent-review-rubric.md) |
|
||||
| `process.issue-qualification-audit` | Run `scripts/issue-qualification-audit.sh` at session end and label everything it flags, including issues you filed that session. | 2026-07-21 | [link](records/process/issue-qualification-audit.md) |
|
||||
| `process.local-gate-before-push` | Run the local build/test gate and a cold-context, scoped "review only" adversarial review over the diff, fold the fixes, and only then push or open the PR. | 2026-07-21 | [link](records/process/local-gate-before-push.md) |
|
||||
| `process.lock-ownership-enumerate-producers` | Before trusting any "single owner / no double release / no cross-release" claim, grep the whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer. | 2026-07-21 | [link](records/process/lock-ownership-enumerate-producers.md) |
|
||||
| `process.one-worktree-one-committing-agent` | Never run two committing agents concurrently on one worktree — give each parallel slice its own worktree branched off the feature branch and merge back. | 2026-07-21 | [link](records/process/one-worktree-one-committing-agent.md) |
|
||||
| `process.parallel-session-claim` | Before starting an issue, check for an existing claim four ways — open PRs referencing it, remote branches naming it, recent comments (a claim can precede the label), and a fresh `git fetch origin main` — then claim with the `in-progress` label plus a comment. A claim prevents duplicate PICKUP, not duplicate WORK. Re-fetch `origin/main` before every push, not only at branch time. | 2026-07-21 | [link](records/process/parallel-session-claim.md) |
|
||||
| `process.per-agent-model-routing` | State the model tier (and effort, where the client exposes it) in the dispatch itself for every delegated agent — bounded recon → cheapest fast tier at `low`; mechanical slice against a documented contract → mid tier; judgment-heavy work → orchestrator tier; independent review → a different model family than the implementer. | 2026-07-25 | [link](records/process/per-agent-model-routing.md) |
|
||||
| `process.pr-routine-sequence` | Worktree off origin/main → implement → regenerate API artifacts → full local tests + cold review + live-E2E ALL before the push → push, open PR, arm the CI monitor at open → fixes after the push are follow-up commits, never amend/force-push. | 2026-07-21 | [link](records/process/pr-routine-sequence.md) |
|
||||
| `process.review-disagreement-frontier-judge` | When independent reviews disagree on a gate PR, escalate to the frontier judge, and put the proposed fix approach in front of it — not just the disputed finding. | 2026-07-21 | [link](records/process/review-disagreement-frontier-judge.md) |
|
||||
| `process.shared-tree-readonly` | Never commit in `/Users/timothy/ersatztv` and never read its `git log`/`git status`/HEAD to infer anything about `main` — work in a worktree off `origin/main`, which is the only source of truth. | 2026-07-21 | [link](records/process/shared-tree-readonly.md) |
|
||||
| `process.subagent-drop-resume` | Treat a subagent connection drop as laptop sleep or transient network and re-resume via SendMessage — the work survives. | 2026-07-21 | [link](records/process/subagent-drop-resume.md) |
|
||||
| `release.api-contract-ci-gate` | A PR touching `ErsatzTV/Controllers/Api/**` or `ErsatzTV.Core/Api/**` must ship regenerated OpenAPI artifacts (`v1.json`, `v1.d.ts`, `endpoint-index.md`) in the same diff, enforced by a blocking `api-docs` CI job that regenerates-and-diffs against a fresh build. | 2026-07-12 | [link](records/release/api-contract-ci-gate.md) |
|
||||
| `release.done-when-merge-consent` | A PR may merge only when its linked issue's `## Done-when` checklist is fully ticked and the PR's CI is green, enforced by a PreToolUse hook on the Gitea merge tool (deny/allow/ask) plus a pre-push backstop for direct pushes to main. | 2026-07-12 | [link](records/release/done-when-merge-consent.md) |
|
||||
| `release.format-as-you-touch-rebase` | A blocking `format` CI job runs `dotnet format --verify-no-changes` scoped only to the PR's changed `.cs` files (never the legacy BOM backlog), and a PR branch must be kept current by rebasing on `origin/main` (never merging main in), enforced by `.husky/pre-push` → `prepush-rebase-check.sh`. | 2026-07-12 | [link](records/release/format-as-you-touch-rebase.md) |
|
||||
| `release.live-e2e-required` | A PR that changes an API write-path handler must include a live-E2E pass (driving the real endpoint/screen and confirming the round-trip through a subsequent read), not only unit/characterization tests, and must state whether live-E2E ran or wasn't required. | 2026-07-12 | [link](records/release/live-e2e-required.md) |
|
||||
| `release.merge-consent-autogrant` | When Done-when boxes are ticked, CI is green, and a fresh positive Review-verdict references head, the merge-consent hook emits `permissionDecision: allow` to actually suppress the redundant mechanical prompt — the derived state IS the consent, no separate conversational confirmation on that path. | 2026-07-12 | [link](records/release/merge-consent-autogrant.md) |
|
||||
| `release.migration-rehearsal-prodcopy` | Before promoting a migration-bearing release, rehearse the new image's migrations against a throwaway copy of the latest prod backup (`scripts/migration-smoke.sh`), gating PASS on the migrator's completion log line rather than HTTP readiness alone. | 2026-07-12 | [link](records/release/migration-rehearsal-prodcopy.md) |
|
||||
| `release.prepush-clean-worktree-guard` | A fail-open pre-push hook blocks a push when any file in the branch's diff vs `origin/main` also has uncommitted working-tree or index changes, since a stale-index commit (e.g. `git reset --soft` + `git add` over an edited-but-unstaged fix) can silently push, CI-test, and get reviewed a different tree than the one on disk. Scope is precise to pushed-diff files; escape hatch `ETV_ALLOW_DIRTY_PUSH=1`. | 2026-07-17 | [link](records/release/prepush-clean-worktree-guard.md) |
|
||||
| `release.promotion-floating-prod` | Prod tracks the floating `:prod` image reference; a tag build's immutable `:<version>` image is scanned first, then promotion happens via a separate manual `DeployStack`, with daily auto-update only as a fallback — tag with enough runway before 03:00 to avoid an unscanned promotion. | 2026-07-13 | [link](records/release/promotion-floating-prod.md) |
|
||||
| `release.review-verdict-gate` | A PR may not merge until a `Review-verdict: <MERGEABLE\|APPROVED\|BLOCKED\|NOT-MERGEABLE> @ <head-sha>` comment references the PR's current head sha (short-sha prefix match against the verdict's OWN `@ <sha>` field, marker at COLUMN 0 (no indent, so indented code blocks cannot self-approve), whole-word verdict token, fenced code blocks stripped with markdown fence-length semantics, negative wins over positive on the same head); folds into the H6 merge-consent hook as condition (c). The grammar lives in ONE tested place, `scripts/check-review-verdict.sh` — #629 found three false-opens that survived because it was implemented inline and untested while this record described stricter behaviour than the code had. | 2026-07-12 | [link](records/release/review-verdict-gate.md) |
|
||||
| `release.verdict-status-check` | The H10 review verdict is written as a `review-verdict/h10` Gitea **commit status** on the exact reviewed sha by `scripts/post-review-verdict.sh`, and that context is a REQUIRED status check on `main`. Because a status belongs to one sha, a later commit cannot inherit it, so Gitea's own `merge_when_checks_succeed` refuses to merge a head no one reviewed. The PreToolUse hook additionally refuses to SCHEDULE an auto-merge unless that status is already green on head. A `pull_request_target` workflow auto-passes the two exempt classes (Renovate-authored, docs-only) unless the PR touches a protected path (`.claude/`, `.codex/`, `.gitea/`, `.husky/`, `scripts/`, `docker/ci/`). This extends — does not supersede — `release.review-verdict-gate` (#303 H10), whose comment convention remains the human-readable artifact and the hook's condition (c). | 2026-07-25 | [link](records/release/verdict-status-check.md) |
|
||||
| `rulebuilder.relative-date-macros` | The visual rule builder's `inLast`/`notInLast` date operators compile to/parse from the pre-existing `CustomMultiFieldQueryParser` macros `released_inthelast`/`released_notinthelast` and `added_inthelast`/`added_notinthelast`, value form `"<n> day\|week\|month\|year"`; there is no backend change. | 2026-07-23 | [link](records/rulebuilder/relative-date-macros.md) |
|
||||
| `scan.collections-scan-status` | `GET /api/v1/media-sources/collections-scan-status` reports a family-global (not per-source), boolean-only active-scan set read from `IEntityLocker`; the SPA reconciles authoritatively against it (with a grace-tick helper) instead of a fixed client-side timeout. | 2026-07-12 | [link](records/scan/collections-scan-status.md) |
|
||||
| `scan.getoraddfolder-db-lookup` | `ILibraryRepository.GetOrAddFolder` resolves the existing folder via a DB query on `(LibraryPathId, Path)`, not the caller's `LibraryPath.LibraryFolders` in-memory navigation, since that navigation is only eager-loaded on the local scan path and is null on remote (Jellyfin) callers. | 2026-07-20 | [link](records/scan/getoraddfolder-db-lookup.md) |
|
||||
| `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](records/scan/jellyfin-mixed-content-library.md) |
|
||||
| `scan.libraryfolder-unique-identity` | `LibraryFolder` uniqueness per `(LibraryPathId, Path)` is enforced by a database unique index over a SHA-256 `PathHash` (Path is unbounded and not portably indexable), and `LibraryRepository.GetOrAddFolder`/`SetEtag` tolerate the constraint violation by re-reading and adopting the winner's row. | 2026-07-25 | [link](records/scan/libraryfolder-unique-identity.md) |
|
||||
| `scan.musicvideo-server-identity` | Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added. | 2026-07-25 | [link](records/scan/musicvideo-server-identity.md) |
|
||||
| `scan.projection-failure-sweep-guard` | `MediaServerReconciliationGuard` takes a per-enumeration projection-failure count and refuses the file-not-found sweep (logged loudly) whenever it is non-zero against a non-empty existing set — at all six sweeps, including the nested per-show season and per-season episode ones #477 left unguarded (via `ShouldFlagMissingDescendants`, which applies the failure refusal but not #477's empty-fetch branch). Deliberate guard-clause skips (STRM, virtual, unsupported type) are explicitly **not** failures and never suppress a sweep. The missing-fraction / ratio threshold floated by #477 is **rejected**, not deferred. | 2026-07-25 | [link](records/scan/projection-failure-sweep-guard.md) |
|
||||
| `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](records/scan/zero-item-fetch-guard.md) |
|
||||
| `sched.auto-tune-foundation` | Auto-tune preview enumeration uses EF distinct+count queries for exact counts, while each created channel is persisted as a live SmartCollection; coexistence with existing channels/numbers is additive-only, never mutating. | 2026-07-16 | [link](records/sched/auto-tune-foundation.md) |
|
||||
| `sched.autotune-detailpanel-members` | The Auto-Tune DetailPanel's per-channel content-source list is a live `ISearchIndex.Search` roll-up through the server-owned `AutoTuneAxisMap.GenerateQuery`, not an EF distinct+count query, so the preview matches exactly what the built channel's SmartCollection will contain. | 2026-07-17 | [link](records/sched/autotune-detailpanel-members.md) |
|
||||
| `sched.autotune-per-channel-overrides` | Auto-Tune per-channel overrides reuse the Channel Builder's advanced-options DTO verbatim; per-source weights and bug-colour logo are deferred to #425. | 2026-07-17 | [link](records/sched/autotune-per-channel-overrides.md) |
|
||||
| `sched.autotune-per-source-weights` | Auto-Tune per-source rotation weights and query corrections are supplied at bulk-create time via #70's MultiCollection/SmartCollection machinery, not a post-hoc PUT. | 2026-07-18 | [link](records/sched/autotune-per-source-weights.md) |
|
||||
| `sched.clock-padding-existing` | Clock-boundary padding already exists via `FillerPreset`'s `FillerMode.Pad` (Classic) and `pad_to_next`/`pad_until` (Sequential/YAML); #77 is closed as verified+documented, not built new, with a one-click per-channel toggle deferred behind the #388 design-system epic. | 2026-07-17 | [link](records/sched/clock-padding-existing.md) |
|
||||
| `sched.clock-padding-schedule-toggle` | A `ProgramSchedule.PadToNearestMinute` (nullable int; null = off) makes the Classic builder pad every content item up to the next N-minute clock boundary without a hand-wired Pad `FillerPreset`, by reusing the existing per-content-item Pad path in `PlayoutModeSchedulerBase.AddFiller`. It extends — does not supersede — `sched.clock-padding-existing` (#77/#388). | 2026-07-22 | [link](records/sched/clock-padding-schedule-toggle.md) |
|
||||
| `sched.playbackorder-support-matrix` | Every build-time dispatch site logs a loud (non-fatal) warning on an unsupported `PlaybackOrder`, and a declared `PlaybackOrderSupport` matrix + partition tripwire test makes adding a new order safe by construction. | 2026-07-18 | [link](records/sched/playbackorder-support-matrix.md) |
|
||||
| `sched.reshuffle-scoped-reset` | `POST /api/v1/playouts/{id}/reshuffle` runs `ErasePlayoutHistory` (reseeds `Playout.Seed` + clears anchors/rerun-history) then enqueues a scoped `Reset` build, so reshuffle always reseeds — even for the non-Classic kinds `Reset` alone wouldn't reseed; `Playout.Seed` is surfaced on list/detail DTOs as visible confirmation. | 2026-07-16 | [link](records/sched/reshuffle-scoped-reset.md) |
|
||||
| `sched.seasonal-scheduling-existing` | Seasonal/date-conditional scheduling already ships first-class via `IAlternateScheduleItem` (Classic `ProgramScheduleAlternate`, Block `PlayoutTemplate`) evaluated by `AlternateScheduleSelector.GetScheduleForDate` (first match in `Index` order, catch-all last); #73 is closed as already-implemented with a docs-only "seasonal/holiday" recipe added, not new code. | 2026-07-17 | [link](records/sched/seasonal-scheduling-existing.md) |
|
||||
| `sched.shuffle-source-builder` | Shuffle-source construction moves to a static, DI-free `ShuffleSourceBuilder` (a shared seam, not a service) so Classic and Playlist stop cross-engine reaching into `PlayoutBuilder` statics; a unified Classic+Playlist enumerator factory is explicitly rejected as a god-factory. Block/Scripted/YAML duplication is left alone, deferred to a follow-up gated on #381. | 2026-07-17 | [link](records/sched/shuffle-source-builder.md) |
|
||||
| `sched.weighted-shuffle` | Fair-share/weighted airtime distribution ships as one new `PlaybackOrder.WeightedShuffle = 9` order (equal weights = fair-share), not a retrofit of `ShuffleInOrder` (which only anti-clumps, since its padding spacers emit nothing) and not a separate orthogonal "distribution" setting; weights live on `MultiCollectionItem`/`MultiCollectionSmartItem` (DB default 1, dual-provider migration), bounded at write (1..1000) and clamped again in the enumerator, and the write path rejects `WeightedShuffle` at every dispatch site that doesn't handle it rather than let it silently degrade to unweighted random. | 2026-07-17 | [link](records/sched/weighted-shuffle.md) |
|
||||
| `sched.weightedshuffle-editor` | WeightedShuffle per-source weights are edited on the multi-collection editor (property of the MultiCollection), while the WeightedShuffle order itself is offered only on classic MultiCollection schedule items; fair-share is a "reset weights to 1" action, not a stored mode. | 2026-07-19 | [link](records/sched/weightedshuffle-editor.md) |
|
||||
| `scheduling.ondemand-guide-refresh-on-thaw` | When `PlayoutTimeShifter.TimeShift` slides an on-demand playout's materialized timeline forward on tune-in, it reports the channel numbers whose cached guide is now stale — the shifted channel **plus any channels that mirror it** — and `TimeShiftOnDemandPlayoutHandler` enqueues a `RefreshChannelData` for each, so every affected cached XMLTV fragment is regenerated from the just-shifted `PlayoutItem` rows. The guide and playback both read the same stored `PlayoutItem.Start/Finish`, but the guide is served from a **cached** projection — rewriting the rows without rebuilding the cache would leave the guide advertising a stale timeline. | 2026-07-21 | [link](records/scheduling/ondemand-guide-refresh-on-thaw.md) |
|
||||
| `security.artwork-content-type-sniff` | Artwork content type is always derived from the stored bytes (never the client-declared value or a `?contentType=` query param) at both upload and serve, clamped to an image allow-list, closing the unauthenticated stored-XSS chain; Kestrel `MaxRequestBodySize` bounds upload DoS. | 2026-07-12 | [link](records/security/artwork-content-type-sniff.md) |
|
||||
| `security.baseline-response-headers` | `SecurityHeadersMiddleware`, registered first in the pipeline, sets `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, and `Referrer-Policy: strict-origin-when-cross-origin` on every response (CSP/HSTS deliberately deferred); API-key comparison is constant-time and playout pagination is clamped. | 2026-07-11 | [link](records/security/baseline-response-headers.md) |
|
||||
| `security.blazor-removal-auth-posture` | Removing the Blazor UI's OIDC-challenged surface exposes nothing a user couldn't already reach via the already-open `/app` SPA (open since phase (a)); real SPA/API authentication is deliberately deferred to #197, and the removal PR must preserve `ConditionalIptvAuthorizeFilter`, `ApiKeyAuthorizationFilter`, and `JwtHelper` access_token support. | 2026-07-11 | [link](records/security/blazor-removal-auth-posture.md) |
|
||||
| `security.contract-freeze-honesty` | The OpenAPI doc's declared security/401 scheme is generated from the same `ApiKeyAuthorizationFilter.EndpointRequiresKey` predicate the runtime enforces (so declared auth can't drift from enforced auth), every `/api/*` action returns a ResponseModel (no raw Application VMs), and Channel REST resources are keyed by immutable `Id`, never mutable `Number`. | 2026-07-12 | [link](records/security/contract-freeze-honesty.md) |
|
||||
| `security.corp-same-origin` | `SecurityHeadersMiddleware` sends `Cross-Origin-Resource-Policy: same-origin` on every response including `/docs`/`/openapi`, blocking cross-origin `no-cors` embedding without affecting allowed CORS-mode fetches or server-side Jellyfin `/iptv/*` requests. | 2026-07-13 | [link](records/security/corp-same-origin.md) |
|
||||
| `security.csp-permissions-policy` | `SecurityHeadersMiddleware` sends an enforcing (not report-only) `Content-Security-Policy` (no `unsafe-inline`/`unsafe-eval`; the one inline theme-bootstrap script allow-listed by hash) and a deny-all `Permissions-Policy` on the SPA/`/api`/`/artwork`/`/iptv`; `/docs` and `/openapi` keep only the baseline headers, excluded from CSP because Scalar needs inline bootstrap. | 2026-07-12 | [link](records/security/csp-permissions-policy.md) |
|
||||
| `security.fail-closed-api-auth` | Every mutating `/api` request requires `X-Api-Key` (no open mode); reads are gated by `Api:RequireKeyForReads` (default true) OR `[RequiresApiKey]` on sensitive controllers; CORS is an exact-origin allowlist (`ApiCors`); `ForwardedHeaders` trust stays configurable but defaults to trust-all-with-warning. | 2026-07-12 | [link](records/security/fail-closed-api-auth.md) |
|
||||
| `security.iptv-access-token-transport` | The `/iptv` `?access_token=` value is percent-encoded (`Uri.EscapeDataString`) everywhere it is interpolated into an M3U/HLS/XMLTV URL (XMLTV additionally XML-escapes the encoded value), so a structural character can't malform the manifest or guide; Serilog logs a scrubbed request path (`access_token` → `***` via `IncludeQueryInRequestPath = false` + a `RequestPathScrubbed` enricher), so a 5xx/Debug `/iptv` request never writes the token; and every dynamic token-bearing `/iptv` manifest (`channels.m3u`, `xmltv.xml`, the HLS multi-variant/media playlists) returns `Cache-Control: private, no-store`. | 2026-07-23 | [link](records/security/iptv-access-token-transport.md) |
|
||||
| `security.iptv-browser-token` | Under a JWT-enabled deployment (`JWT:IssuerSigningKey` set), the browser SPA obtains a short-lived, globally-scoped `/iptv/*` access token from an authenticated `GET /api/v1/auth/iptv-token` and appends it as `?access_token=`; the endpoint answers 204 when JWT is disabled (nothing to mint). Lifetime defaults to 60 min, configurable via `JWT:BrowserTokenLifetimeMinutes`. | 2026-07-22 | [link](records/security/iptv-browser-token.md) |
|
||||
| `security.session-auth-dual-credential` | `ApiAuthorizationFilter` accepts a request when a valid `X-Api-Key` matches OR the principal is an authenticated session (cookie `ctv-session`, `HttpOnly`/`SameSite=Lax`); session-authenticated mutations require the presence-only `X-CSRF` header or are rejected 403. This narrows the OIDC-inert sub-claim of `security.blazor-removal-auth-posture` (#206) — the rest of that record's auth-surface enumeration still holds. | 2026-07-12 | [link](records/security/session-auth-dual-credential.md) |
|
||||
| `security.session-cutover-postify` | The browser SPA authenticates cookie-only (no more `X-Api-Key` from `web/`); the machine key is repurposed to external/MCP-only via `GET /api/auth/machine-key`; every side-effecting GET/HEAD under `/api` is converted to POST so the existing CSRF gate covers it (standing rule: never add a side-effecting GET/HEAD under `/api`). | 2026-07-12 | [link](records/security/session-cutover-postify.md) |
|
||||
| `session.shared-checkout-refresh` | Session end runs `scripts/refresh-shared-checkout.sh`, which fast-forwards `/Users/timothy/ersatztv` to `origin/main` (and reinstalls `web/node_modules` when the lockfile moved), refusing to touch anything unless that tree is on a clean, non-ahead `main`. | 2026-07-21 | [link](records/session/shared-checkout-refresh.md) |
|
||||
| `spa.add-to-layer` | All add-to-collection/playlist/schedule affordances share one component layer at `web/src/media/addTo/`; multi-select is an explicit screen-level toggle, and the per-card menu offers schedule only for the server-validated kinds. | 2026-07-10 | [link](records/spa/add-to-layer.md) |
|
||||
| `spa.app-shell-extraction` | `App.tsx` is only the composition root over `web/src/app/routes.tsx` (stable route-object identity), `app/AppShell.tsx` (shell chrome), and `app/ScreenContent.tsx` (exhaustive screen dispatch); primary actions are one explicit `PrimaryActionProvider` registration per screen, replacing the old global `ctv:primary-action` window event. | 2026-07-15 | [link](records/spa/app-shell-extraction.md) |
|
||||
| `spa.autotune-detailpanel-slideover` | The Auto-Tune DetailPanel SPA is a reusable `SlideOver` primitive sharing `useOverlayBehavior` with `Dialog`, plus a shared advanced-options model extracted from ChannelBuilder; decorative panes without backend support are dropped. | 2026-07-18 | [link](records/spa/autotune-detailpanel-slideover.md) |
|
||||
| `spa.channel-editor-create-logo` | Bare-channel create is a "New blank channel" action on the channels list (reusing Blazor's add-mode defaults) that navigates into the full editor, and an external logo URL always wins over an uploaded logo, matching `ChannelEditViewModel` precedence. | 2026-07-11 | [link](records/spa/channel-editor-create-logo.md) |
|
||||
| `spa.channel-renumber-prompt` | Channel renumbering uses a sequential `prompt()`-driven "Renumber" action instead of drag-to-reorder. | 2026-07-09 | [link](records/spa/channel-renumber-prompt.md) |
|
||||
| `spa.channels-screen-extraction` | The Channels domain is a single-file zero-prop screen (`web/src/screens/ChannelsScreen.tsx`) with a colocated test file and no sibling helper directory, since its pure logic is too small (~30 lines) to justify a separate business-rule layer like Schedules' `itemRules.ts`. | 2026-07-11 | [link](records/spa/channels-screen-extraction.md) |
|
||||
| `spa.collection-custom-order-ui` | Collection custom ordering uses per-row Move up/Move down buttons (not drag) and is offered for any manual collection with custom order enabled, not just movies-only. | 2026-07-09 | [link](records/spa/collection-custom-order-ui.md) |
|
||||
| `spa.datetime-local-input` | The channel-mode date/time input uses a native `<input type="datetime-local">` instead of free-text Chronic natural-language parsing. | 2026-07-09 | [link](records/spa/datetime-local-input.md) |
|
||||
| `spa.deco-templates-table` | The deco-templates editor also renders its day/deco assignment as a table, extending (not replacing) the templates-editor-table convention. | 2026-07-09 | [link](records/spa/deco-templates-table.md) |
|
||||
| `spa.download-sample-gate` | The SPA disables both Download Media Sample and Download Results while a troubleshooting session is starting/running (Blazor only gated Download Results). | 2026-07-09 | [link](records/spa/download-sample-gate.md) |
|
||||
| `spa.legacy-redirect-matcher` | `LegacyUiRedirects.TryGetRedirect` is a two-tier matcher — an exact `OrdinalIgnoreCase` `Map` (Tier 1) then an ordered segment-template pattern list (Tier 2, first-match-wins) — collision-free by construction, with a guard invariant that no rule may prefix-match `/api`, `/artwork`, `/docs`, `/openapi`, `/iptv`, `/app`, or `/media/sources`. | 2026-07-11 | [link](records/spa/legacy-redirect-matcher.md) |
|
||||
| `spa.library-pickers-resolve-by-search` | A picker over a media-library table (Episode/Song/Image/Movie/MusicVideo/TelevisionShow/TelevisionSeason/Artist/OtherVideo/RemoteStream) resolves its options by SEARCH — a debounced `SearchPicker` calling `searchLibraryPickerOptions`, which issues at most ONE `getLibraryBrowseItems` request per settled query, bounded to `LIBRARY_PICKER_RESULTS` (25) rows — CLAMPED inside the helper, not merely defaulted — and gated on `LIBRARY_PICKER_MIN_QUERY` (2) characters. It list-loads NOTHING on mount or on a type switch, so there is no truncation to surface and no truncation hint. The typed text is COMPILED (`titleContainsQuery` → `title:*<escaped>*`), never forwarded raw. The current selection renders from the OWNING RECORD, not from the result set (`selectedName` on a rerun collection / playlist item; a single by-id detail read — `getShow`/`getSeason`/`getArtist` — for a filler preset, which stores only the id), and an edit draft is INITIALIZED ONCE from the detail read — never seeded from the list row, never reconciled against a late response — with the form withheld until it lands, the editor failing CLOSED when the response carries no USABLE concurrency token — absent, empty and whitespace-only ETags are ONE case, normalized in one place, so a PUT without `If-Match` is unreachable, and a deadline plus a route back so a hung request cannot strand it. An id NEVER travels without its namespace: search results are cached against `(source, query)` and list-backed options carry the type they were loaded for, so no id from one type can be offered under another; and every id entering editor state — search result, list-backed option, or a selection restored from a detail read — passes ONE shared `isSelectionId` (int32) predicate at that boundary, an unbindable id being treated as ABSENT rather than coerced. Conflicts are detected at SAVE time via `If-Match` -> 412 -> Reload, and Reload simply drops the draft back to null and re-runs the same initialize-once load, so the form is unmounted while the replacement is in flight; an asynchronously-resolved name is keyed to the id it was resolved for and never overwrites a label naming a different id. The typeahead implements the full ARIA combobox keyboard contract, because it replaces a natively keyboard-operable `<select>`. The other half of the superseded record is UNCHANGED: bounded-by-construction admin lists (collections, multi-collections, smart collections, playlists) still page to completeness via `loadAllPages` and still report `complete`/`hint: incomplete`. Server-side caps are not raised — this is a web-only change. | 2026-07-26 | [link](records/spa/library-pickers-resolve-by-search.md) |
|
||||
| `spa.logs-page-size-local` | The Logs page rows-per-page preference is stored in `window.localStorage` (`ctv-logs-page-size`), not a server `ConfigElement`. | 2026-07-11 | [link](records/spa/logs-page-size-local.md) |
|
||||
| `spa.playback-troubleshoot-poll` | The playback-troubleshooting screen reports FFmpeg completion by polling `GET /api/troubleshoot/playback/status` (~2s) rather than a server push channel. | 2026-07-09 | [link](records/spa/playback-troubleshoot-poll.md) |
|
||||
| `spa.playout-reset-button` | The SPA keeps a single Reset action (server picks the default build mode) and drops Blazor's separate "Schedule reset" button since its capability already exists via the playout's Edit-details flow. | 2026-07-09 | [link](records/spa/playout-reset-button.md) |
|
||||
| `spa.playouts-screen-extraction` | The Playouts domain (including its unguarded `PlayoutsRouteScreen` route wrapper with local pathname/popstate state) moved as one unit into `web/src/screens/PlayoutsScreen.tsx`, keeping its screen-specific sub-path route ownership colocated with the base screen; a pure structural move with no API/route/CSS/behavior change. | 2026-07-14 | [link](records/spa/playouts-screen-extraction.md) |
|
||||
| `spa.rulebuilder-nesting` | The visual rule builder's `Group` nests recursively to a single shared cap, `MAX_GROUP_DEPTH` (`types.ts`, currently 5, root group = depth 0) — read by the UI's "Add group" gate, `parse.ts` and the round-trip property-test generator alike; everything else about the builder is unchanged from #176 (compile-only closed Lucene subset over the stored query string, no stored rule AST, field vocabulary from `GET /api/v1/search/fields`). | 2026-07-25 | [link](records/spa/rulebuilder-nesting.md) |
|
||||
| `spa.schedules-editor-draft-save` | The schedules SPA editor mutates a local draft and flushes one explicit Save (`PUT /api/schedules/{id}/items`) instead of instant-persisting each action; Copy deep-copies all source references (fixing a Blazor omission); the shuffled-schedule GET's `EnforceProperties` lossy normalization is preserved and mirrored in the SPA's option lists. | 2026-07-11 | [link](records/spa/schedules-editor-draft-save.md) |
|
||||
| `spa.sidebar-collapsible-accordions` | The shell sidebar's collapse + nav-group-accordion state persists under two hyphenated `ctv-sidebar-*` localStorage keys (matching the repo's `ctv-` convention, not the prototype's dotted names); labeled groups default-collapsed. | 2026-07-18 | [link](records/spa/sidebar-collapsible-accordions.md) |
|
||||
| `spa.spa-rebuild-decision` | The UI is a full React SPA (ChicoryTV) rebuild over the REST API, not a Blazor Server reskin. | 2026-06 | [link](records/spa/spa-rebuild-decision.md) |
|
||||
| `spa.templates-editor-table` | The SPA templates editor renders day/block assignment as a table, not Blazor's drag-and-drop calendar grid — an accepted, deliberate parity deviation. | 2026-07 | [link](records/spa/templates-editor-table.md) |
|
||||
| `spa.topbar-primary-action` | The TopBar's primary-action "+" button renders only when the active route declares a non-empty `primaryAction`, is wired (via a shared `usePrimaryAction` hook) only on single-unambiguous-create-flow list screens, and is dropped everywhere else rather than left as a dead/no-op button. | 2026-07-12 | [link](records/spa/topbar-primary-action.md) |
|
||||
| `spa.yaml-validator-textarea` | The YAML playout validator takes pasted YAML via a `<textarea>`, not a server-side file path, since the SPA has no filesystem access. | 2026-07-09 | [link](records/spa/yaml-validator-textarea.md) |
|
||||
| `startup.parallel-orientation` | A fresh session runs two concurrent tracks at startup — Orientation (`AGENTS.md`/`CLAUDE.md` → `docs/README.md` task-signal map → the active decisions catalog `docs/decisions/README.md`) and, only when no issue is named, Selection (`scripts/select-queue.sh N`, deterministic live-Gitea ranking). A named issue skips Selection entirely. ersatztv#237, the closed pickup tracker this replaces, is reduced to a single archival breadcrumb and MUST NOT be read for live state. | 2026-07-21 | [link](records/startup/parallel-orientation.md) |
|
||||
| `testing.e2e-cleanup-scope-by-pid` | An E2E harness or agent may only kill processes whose PIDs it captured at launch — capture the PID; whoever owns the lifecycle releases it from a `trap ... EXIT INT TERM`. Never `pkill -f "dotnet ErsatzTV.dll"` (or any pattern that can match a process this run did not start). A foreign listener is reported, not reaped. | 2026-07-25 | [link](records/testing/e2e-cleanup-scope-by-pid.md) |
|
||||
| `testing.e2e-local-fresh-config-dir` | Always point `scripts/e2e-local.sh` at a fresh config dir — leftover channels/schedules/DB rows bleed state between runs and corrupt assertions. (The *readiness-probe hang* this record was originally written about was fixed in #533; the fresh-dir rule stands on state-bleed grounds alone.) | 2026-07-21 | [link](records/testing/e2e-local-fresh-config-dir.md) |
|
||||
| `testing.enumerating-guard-identity-not-position` | A guard that cross-checks a hand-reviewed registry against call sites discovered across the whole repo must key each entry on properties INTRINSIC to the site — file, kind, and the value source text — and never on its absolute line or column. A registry keyed on position is a function of every other file in the repo, so a branch that never touches the guard can invalidate it; and because each PR is green against its own base, that failure is structurally invisible pre-merge and lands on `main` after review and after the merge gate. Dropping the position keeps every mutation the guard exists for — a NEW site, a REMOVED site and a CHANGED value each still fail, since each changes the identity multiset — and costs exactly ONE case, which must be stated rather than implied: a SAME-IDENTITY SUBSTITUTION within one file (delete a registered site, add a different unreviewed one with the same kind and value token, net-zero count) now passes. A REPORTED failure still prints the discovered line:column, because identity and diagnostics need not share a format. Comparison stays a MULTISET count rather than set membership, so two sites in one file sharing an identity must be discovered exactly that many times and a third occurrence still fails. A SCANNER test that asserts real AST positions against FIXED inline fixtures is the opposite case and keeps its line/column identity — it has no churn, because its input does not move. | 2026-07-27 | [link](records/testing/enumerating-guard-identity-not-position.md) |
|
||||
| `testing.live-e2e-prepush-timing` | Run live-E2E via `scripts/e2e-local.sh` before pushing a write-path or UI change, and exercise download endpoints with curl, never a browser tab. | 2026-07-21 | [link](records/testing/live-e2e-prepush-timing.md) |
|
||||
| `testing.playwright-mcp-download-and-recovery` | In Playwright-MCP E2E, fetch file-download endpoints with curl — never a browser tab or `window.open` — and if browser tools stall repeatedly, `pkill -f ms-playwright-mcp` and drive a fresh session. | 2026-07-21 | [link](records/testing/playwright-mcp-download-and-recovery.md) |
|
||||
| `testing.scripted-playout-golden-deferred` | The `PlayoutBuildGoldenTests` in-memory golden net covers Sequential (YAML) as of #381. Scripted's *end-to-end pipeline* is excluded — `ScriptedPlayoutBuilder` runs a user-authored external program that drives the engine over HTTP loopback, which the in-memory harness can't pin — so that full-pipeline (integration) harness is deferred to #563. But the scheduling *behavior* those scripts drive lives entirely in the in-process `SchedulingEngine` (the `ScriptedScheduleController` is a 1:1 pass-through to it), which IS directly unit/golden-testable; the earlier "Scripted is un-golden-able by construction" framing overstated the constraint by conflating transport with engine. #395 extracts that shared switch to `ContentEnumeratorBuilder` and adds a direct regression net (`ContentEnumeratorBuilderTests`) over it. | 2026-07-22 | [link](records/testing/scripted-playout-golden-deferred.md) |
|
||||
| `testing.troubleshoot-path-cannot-test-branding` | Verify logo/watermark/bug changes through a real channel playout — a green troubleshoot run proves nothing about branding. | 2026-07-21 | [link](records/testing/troubleshoot-path-cannot-test-branding.md) |
|
||||
| `api.artwork-rooted-urls` | API response DTOs return artwork as rooted, directly-usable URLs (plus passthrough for absolute/proxy URLs), never relative Blazor-convention paths. | 2026-07-07 | [link](../decisions.md#2026-07-07--api-artwork-contract-rooted-urls-produced-server-side) |
|
||||
| `api.async-op-contract` | Queue-triggering `/api/*` endpoints normalize onto one contract — 202 Accepted (queued), 404 (missing entity), 409 (lock held), 422 (domain precondition) — with Trakt as the reference implementation; playout list/detail GETs also carry an `isLocked` observability flag as the HTTP-observable substitute for a live push channel. | 2026-07-11 | [link](../decisions.md#2026-07-11--async-op-api-contract-normalization--playout-build-observability--f9-scan-endpoints-235) |
|
||||
| `api.channel-health-object` | `ChannelResponseModel`/`ChannelDetailResponseModel` carry a server-derived `health` object (`ChannelHealthResponseModel { Status, Faults[], PlayoutCount, BrokenSourceItemCount }`) computed **read-time** from the built timeline (`Playout.BuildStatus` + upcoming `PlayoutItem → MediaItem.State`, `Finish >= now`), kind-agnostic across all 5 `PlayoutScheduleKind` values; `Status`/`Faults` are const-string classes (`ChannelHealthStatus`, `ChannelFault`), not C# enums, so the SPA hand-maintains the union (mirrors `ChannelPreviewAvailability`). This supersedes #72's "raw fact only, no derived enum, empty-schedule/broken-source deliberately not computed" stance now that the auto-tune taxonomy churn (#383/#384) it was waiting on has landed (see `channel.origin-marker` sibling record, #414). | 2026-07-23 | [link](../decisions.md#2026-07-23--channel-health--a-server-derived-health-object-on-the-channel-dtos-built-timeline-detection-415) |
|
||||
| `api.channel-preview-capability` | Whether a channel can be previewed in the browser is declared by the server, not derived by the SPA, as an additive `Preview` field (`{Availability, ManifestUrl, UnavailableReason}`) on `ChannelResponseModel`. | 2026-07-21 | [link](../decisions.md#2026-07-21--browser-channel-preview-is-a-server-declared-per-channel-capability-60) |
|
||||
| `api.decode-by-id` | Endpoints that decode/expand opaque stored state accept a database row id and resolve it server-side rather than round-tripping client-supplied serialized state. | 2026-07-07 | [link](../decisions.md#2026-07-07--decode-style-endpoints-take-a-row-id-and-look-up-server-side) |
|
||||
| `api.from-lineup-clear-to-none` | `POST /api/v1/channels/from-lineup` (and the Auto-Tune per-channel `advanced`, which reuses the same DTO) distinguishes *inherit* from *clear-to-none* with a typed `clear` enum list on `advanced`. A field left null/omitted still inherits the template value (unchanged for every existing client); naming a field in `clear` forces it to none on the new channel even when the template sets one. Sending both a set value and a clear for the same field is a validation error. | 2026-07-21 | [link](../decisions.md#2026-07-21--from-lineup-advanced-overrides-express-clear-to-none-via-a-typed-clear-enum-list-135) |
|
||||
| `api.healthcheck-remediation-dto` | Health-check remediation is server-declared `{Kind, Target}` metadata on an additive DTO field; the SPA renders/acts on it, it doesn't derive labels itself. | 2026-07-17 | [link](../decisions.md#2026-07-17--health-check-remediation-is-server-declared-kind-target-on-an-additive-dto-the-spa-acts-on-it-164) |
|
||||
| `api.healthcheck-ttl-cache` | Health-check results are held in a 30s TTL cache inside `HealthCheckService`; a non-forced `GET /api/v1/health` returns the cached list, and `?refresh=true` (or a forced internal caller) bypasses it to run fresh. | 2026-07-19 | [link](../decisions.md#2026-07-19--health-check-results-are-ttl-cached-refreshtrue-forces-a-fresh-run-431) |
|
||||
| `api.logs-sort-params` | `GET /api/logs` takes allow-listed `sortField` (`timestamp`\|`level`) and `sortDirection` (`asc`\|`desc`) query params, normalized (not rejected) on an unrecognized value. | 2026-07-11 | [link](../decisions.md#2026-07-11--logs-column-sorting-allow-listed-sortfieldsortdirection-on-get-apilogs) |
|
||||
| `api.mediatr-passthrough` | The REST API is thin controllers over existing MediatR handlers, with no new service/business-logic layer. | 2026-06 | [link](../decisions.md#2026-06--rest-api-wraps-existing-mediatr-handlers-11-no-service-layer) |
|
||||
| `api.openapi-mirrors-runtime` | The generated OpenAPI spec is made to match the runtime Newtonsoft wire contract (via `NewtonsoftSchemaNamingTransformer`), not the reverse. | 2026-07-09 | [link](../decisions.md#2026-07-09--openapi-spec-mirrors-the-runtime-newtonsoft-serializer-198) |
|
||||
| `api.parentid-drillin` | Media drill-in (season/episode/artist/music-video) is served by an optional `parentId` query param on library-browse, not dedicated per-kind child-listing endpoints. | 2026-07-07 | [link](../decisions.md#2026-07-07--seasonepisodemusic-video-drill-in-via-parentid-not-new-child-listing-endpoints) |
|
||||
| `api.playout-build-lock-409` | Every id-keyed playout/channel mutation endpoint checks `IEntityLocker.IsPlayoutLocked(id)` and returns 409 Conflict while a build is in-flight, mirroring Blazor's disabled-buttons behavior; reset-all stays 202 and silently skips locked playouts. | 2026-07-10 | [link](../decisions.md#2026-07-10--playout-api-mutations-return-409-while-the-build-lock-is-held-215) |
|
||||
| `api.postcommit-cancellation-none` | Once a mutation commits, its entire compensating side effect (enqueues, publishes, reindexes, cache refresh, and any post-commit lookup gating one of those) runs on `CancellationToken.None` so a late client disconnect can't half-abort an already-committed change. | 2026-07-11 | [link](../decisions.md#2026-07-11--post-commit-side-effects-run-on-cancellationtokennone-generalized-from-251-to-254) |
|
||||
| `api.put-replace-index-order` | PUT-replace-the-whole-list endpoints derive each item's `Index` from its request-array position, never a client-supplied field; alternate-schedule/playout-template rows are evaluated in `Index` order with the least-conditional row placed last as the catch-all default. | 2026-07 | [link](../decisions.md#2026-07--put-replace-list-endpoints-derive-index-from-array-order-alternate-schedules-last-row--catch-all-default) |
|
||||
| `api.response-dtos` | New REST response DTOs live in `ErsatzTV.Core/Api/<Domain>/*ResponseModel.cs` with a file-scoped `#nullable enable` pragma; controllers never expose Application VM types directly. | 2026-07 | [link](../decisions.md#2026-07--response-dtos-live-in-ersatztvcoreapi-file-scoped-nullable-enable) |
|
||||
| `api.schedule-item-flat-dto` | Schedule-item GET/POST/PUT use a flat, non-polymorphic `ScheduleItemResponseModel` (every subtype field promoted to a nullable top-level member) instead of the polymorphic Application VM hierarchy, with mutation field names matching `ScheduleItemRequest` 1:1 for a lossless round-trip. | 2026-07-10 | [link](../decisions.md#2026-07-10--schedule-item-get-returns-a-flat-non-polymorphic-dto-scheduleitemresponsemodel) |
|
||||
| `api.scheduling-hardening` | Create/Replace handlers guard against null/whitespace `name` (`IsNullOrWhiteSpace`, not just `Length`) to prevent NRE-500s, template-item overlap validation compares by index (not record value-equality) to catch exact-duplicate items, and unreachable 404 `ProducesResponseType` attributes on create-only actions are trimmed. | 2026-07-13 | [link](../decisions.md#2026-07-13--scheduling-api-hardening-null-name-500s-duplicate-template-items-unreachable-404-172) |
|
||||
| `api.search-allitems-paging` | `GET /api/v1/search/all-items` is paginated (capped page size, `Totals` field) to bound DoS exposure; the SPA add-all flow pages to completeness instead of relying on an unbounded response. | 2026-07-18 | [link](../decisions.md#2026-07-18--search-all-items-is-paged-to-cap-dos-exposure-spa-add-all-pages-to-completeness-293) |
|
||||
| `api.search-field-values` | `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for one of a narrow allow-list of catalog fields (not the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a typeahead suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source; case-insensitive prefix-filtered on `q`, `limit` clamped to `[1, 50]` (default 50). | 2026-07-23 | [link](../decisions.md#2026-07-23--facet-value-typeahead-is-a-new-endpoint-allow-listed-to-text-fields-no-caching-434) |
|
||||
| `api.search-paging-cap` | Search stays capped at 100 items per media kind; an overflowing kind's "See all" reuses library-browse paging instead of adding new API surface. | 2026-07-11 | [link](../decisions.md#2026-07-11--trash-see-all-reuses-library-browse-paging-search-stays-capped-per-kind-213) |
|
||||
| `api.versioning-v1` | The entire `/api` surface is versioned to `/api/v1` uniformly (no unversioned corner); legacy unversioned callers are rewritten in-pipeline (not redirected) with Deprecation/Link/Sunset headers, and post-freeze `/api/v1` is additive-only — a breaking change requires `/api/v2`. | 2026-07-13 | [link](../decisions.md#2026-07-13--api-versioning-the-whole-api-surface-is-mounted-at-apiv1-additive-only-after-freeze-286) |
|
||||
| `blazor.rollback-tag` | The commit immediately preceding the Blazor-removal merge is tagged `blazor-final` (not a `v*` tag, so it doesn't trigger a prod release build) as the documented rollback/restore path. | 2026-07-11 | [link](../decisions.md#2026-07-11--pre-removal-blazor-rollback-tag-blazor-final-205) |
|
||||
| `blazor.ui-removed` | The legacy Blazor Server UI (`Pages/`, `Shared/`, `ViewModels/`, `Validators/`, MudBlazor + 8 other packages, Blazor Startup wiring) is fully deleted now that the SPA has parity; the legacy `MapWhen` branch is kept only for controllers/docs/OpenAPI/`LegacyUiRedirects`, and the catch-all fallback 302s any unmatched non-api/artwork/docs/openapi path to `/app`. | 2026-07-11 | [link](../decisions.md#2026-07-11--blazor-server-ui-removed-91-phase-b) |
|
||||
| `channel.origin-marker` | A new `Channel.Origin` (`ChannelOrigin` enum — `Unknown`/`UserCreated`/`AutoTuned`) records how a channel row was created and is stamped exactly once at insert (`AutoTuned` in `CreateChannelFromLineupHandler`, `UserCreated` in `CreateChannelHandler`), and is never mutated on a later edit. It is surfaced as a raw `origin` field on `ChannelResponseModel`; the SPA badges only `AutoTuned`. Rows predating the column read `Unknown` — provenance is **not** back-filled. | 2026-07-23 | [link](../decisions.md#2026-07-23--channel-origin-is-immutable-creation-provenance-stamped-at-insert-not-a-health-signal-414) |
|
||||
| `ci.batch-pushes-no-cancel-route` | Hold review fixes, doc corrections and format fixes locally and push **once** — a superseded run cannot be cancelled from the agent side and holds a runner slot until it finishes. | 2026-07-21 | [link](workflow-process.md#2026-07-21--batch-your-pushes-there-is-no-agent-side-cancel-route-on-gitea-1254-542) |
|
||||
| `ci.build-once-rejected` | CI build-once (a shared compile artifact across jobs) was implemented, measured, and rejected for a 40-85% wall-clock regression; keep the #420 cross-run tree-identity skip instead. | 2026-07-18 | [link](../decisions.md#2026-07-18--ci-build-once-was-measured-and-rejected-keep-the-420-tree-skip) |
|
||||
| `ci.cancelled-is-not-a-verdict` | Treat a `cancelled` conclusion as "no verdict" — never as pass or fail — and report FAILED and CANCELLED counts separately in any CI monitor. | 2026-07-21 | [link](workflow-process.md#2026-07-21--cancelled-is-not-failure-a-cancelled-run-is-no-verdict-542) |
|
||||
| `ci.decisions-lifecycle-flake` | When `decisions lifecycle` is the **only** red job, do not investigate and do not create a new run to clear it — no rebase, no `--amend`, no no-op push; the operator reruns that single job from the Gitea UI. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-lone-decisions-lifecycle-red-is-a-known-infra-flake-do-nothing-542) |
|
||||
| `ci.docs-only-detect-shallow-safe` | The docs-only detect script must diff against `FETCH_HEAD` (always resolves after `git fetch`, even shallow) using a two-dot tree diff — not `origin/<base>` with three-dot — because a `fetch-depth: 1` shallow clone has no remote-tracking ref and no merge-base, which silently fails the original detect into `docs_only=false` (full matrix, no functional error). A CI-behavior change must be verified by measuring the effect (job durations), not just a green check. | 2026-07-17 | [link](../decisions.md#2026-07-17--docs-only-detect-must-be-shallow-checkout-safe-fetch_head--two-dot-not-originmain--three-dot-416-follow-up) |
|
||||
| `ci.docs-only-skip-steps` | A docs-only change must still run every required job (`test`, `migrations`) so their commit-status contexts always report; each heavy job runs `scripts/ci-detect-docs-only.sh` first and gates its real STEPS on `if: steps.detect.outputs.docs_only != 'true'`, never `if:`-skips the whole job (an `if:`-skipped job reports `skipped`, not `success`, which branch protection may never unblock on). Detection biases toward running more on any doubt. | 2026-07-17 | [link](../decisions.md#2026-07-17--docs-only-ci-skip-gates-steps-in-always-running-required-jobs-never-if-skips-them-416) |
|
||||
| `ci.format-gate-folder-mode` | The blocking `format` CI job (and matching pre-commit hook) runs `dotnet format whitespace . --folder --include <files>` instead of loading the full MSBuild/Roslyn solution, cutting the gate from ~480s to ~0.5s with unchanged whitespace/charset coverage. | 2026-07-19 | [link](../decisions.md#2026-07-19--the-format-gate-runs-dotnet-format-whitespace----folder-not-the-full-solution-format-469) |
|
||||
| `ci.functional-e2e-harness` | The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. Originally curl-only; since #445 the same job carries a second, headless-browser step for the contracts curl cannot express — see `ci.ui-e2e-harness`. | 2026-07-16 | [link](../decisions.md#2026-07-16--functional-e2e-ci-harness-advisory-curl-contract-job-over-an-app-booted-from-source-299) |
|
||||
| `ci.gitea-milestone-filter-noop` | Never filter issues with the server-side `?milestones=<name>` parameter — fetch all open issues once and filter LOCALLY on each issue's `.milestone.title`. | 2026-07-21 | [link](workflow-process.md#2026-07-21--giteas-milestones-issue-filter-silently-no-ops-on-names-containing--or--542) |
|
||||
| `ci.infra-shaped-red-under-load` | When a job dies inside a setup/cache step before your code compiles, check the runner host's load before diagnosing the diff, and never file a CI bug off one sample under pressure. | 2026-07-21 | [link](workflow-process.md#2026-07-21--an-infra-shaped-red-under-host-load-is-not-a-code-failure-542) |
|
||||
| `ci.killed-job-triage` | Never trust a job's `conclusion` field alone — read the log tail and require an `❌ Failure - Main …` marker before treating a red as a real failure. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-killed-ci-job-reports-conclusion-failure-read-the-log-tail-before-diagnosing-the-diff-542) |
|
||||
| `ci.monitor-armed-at-pr-open` | Arm a CI monitor on the PR head sha the moment the PR opens, polling the commit-status endpoint — not at the end of the work. | 2026-07-21 | [link](workflow-process.md#2026-07-21--arm-the-ci-monitor-at-pr-open-via-the-commit-status-endpoint-542) |
|
||||
| `ci.no-host-health-gating` | Push when your work is validated — never SSH to bumblebee to sample load/RAM first, and never hand-schedule around other sessions' runs. | 2026-07-21 | [link](workflow-process.md#2026-07-21--do-not-gate-or-throttle-pushes-on-host-health-542) |
|
||||
| `ci.peak-anon-measurement` | The `test` job's headline memory figure is a sampled high-water mark of cgroup `anon`, produced by `scripts/ci-peak-anon.sh`; `memory.peak` and the end-of-job `anon`/`file` split are kept only as a cache-inflated reference. | 2026-07-19 | [link](../decisions.md#2026-07-19--ci-test-job-reports-a-sampled-true-peak-anon-not-cache-inflated-memorypeak-412) |
|
||||
| `ci.root-screenshot-guard` | The Husky `pre-commit` hook refuses a staged root-level `*.png` (belt-and-suspenders with the `.gitignore` rule); nested `*.png` real assets are unaffected. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--root-screenshot-guard-pre-commit-refuses-root-level-png-303-h3) |
|
||||
| `ci.runner-placement` | No persistent Roslyn compiler server survives a CI build (`UseSharedCompilation=false` etc., runner env + Dockerfile `ENV`); every `services:` container gets its own explicit `--memory`/`--memory-swap`/`--cpus` cap (it does not inherit the job container's). | 2026-07-17 | [link](../decisions.md#2026-07-17--no-persistent-compiler-servers-in-ci-every-services-container-gets-an-explicit-cap-390s-small-lane-move-reversed-406) |
|
||||
| `ci.small-lane-git-only` | `runs-on: small` is defined by what a job does (git-only), not its usual runtime; the two `docker build` jobs (docker-build.yml, ci-image.yml) move to `ubuntu-latest` because their worst-case memory, not median runtime, was pinning the small lane's per-slot cap. | 2026-07-20 | [link](../decisions.md#2026-07-20--runs-on-small-means-git-only-the-two-docker-build-jobs-move-to-ubuntu-latest-server-management639) |
|
||||
| `ci.ui-e2e-harness` | The UI-interactive E2E flows run as headless Playwright specs (`web/e2e/*.spec.ts`, driven by `scripts/e2e-ui.sh`) in a **second step of the existing advisory `functional-e2e` job**, never their own job; the browser is `chromium-headless-shell` **baked into the CI toolchain image** (`docker/ci/Dockerfile`, `PLAYWRIGHT_VERSION` kept equal to `web/package.json`'s EXACT `@playwright/test` pin), never installed per run; specs are `serial` with `retries: 0` and assert only contracts the curl harness structurally cannot reach. | 2026-07-25 | [link](../decisions.md#2026-07-25--ui-e2e-headless-playwright-flows-in-the-existing-functional-e2e-job-browser-baked-into-the-ci-image-445) |
|
||||
| `ci.verify-locally-ci-confirms` | Treat the local build/verify/review pass as the decision point and CI as confirmation — don't idle waiting on a run you have no reason to doubt. | 2026-07-21 | [link](workflow-process.md#2026-07-21--build-and-verify-locally-then-trust-it-ci-confirms-542) |
|
||||
| `ci.web-test-per-test-timeouts` | Give heavy-render web tests an explicit per-test vitest timeout (e.g. 15s); never raise the global default to fix one slow test. | 2026-07-21 | [link](workflow-process.md#2026-07-21--heavy-render-web-tests-need-explicit-per-test-vitest-timeouts-on-the-ci-vm-542) |
|
||||
| `concurrency.diff-scalar-fanout` | The frozen Block optimistic-concurrency recipe (api-conventions §7a) fans out to Collection/Playout×2/MultiCollection/RerunCollection, keeping a guard-returned `PreconditionFailedError` out of any handler's generic `catch(Exception)`→422 mapping, and preserving each aggregate's existing `SaveChangesAsync() > 0` gate semantics under the new unconditional `Version++`. | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--253-pr3-diff--scalar-concurrency-fan-out-collection--playout2--multicollection--reruncollection) |
|
||||
| `concurrency.etag-rotation-completion` | Every handler that mutates a versioned root's editor-visible config state must bump `Version` (rotating the ETag) with no per-aggregate carve-outs, short-circuiting on a genuine no-op before the bump so idempotent re-submits don't fire spurious rebuild fan-out; `SaveChangesForcingVersion` rebases the retry (stored + pending delta), never adopts the stored token verbatim. | 2026-07-12 | [link](optimistic-concurrency.md#2026-07-12--cross-editor-etag-rotation-completed-for-collectionplayout-config-siblings-269) |
|
||||
| `concurrency.force-write-non-ifmatch` | Any handler that leaves a versioned root `Modified` or `Deleted` but takes no `If-Match` (deletes, item add/remove bumpers, scalar-config writers) must save through `ConcurrencyExtensions.SaveChangesForcingVersion` — force-write past a concurrent `Version` bump rather than throw an unhandled `DbUpdateConcurrencyException` (500). | 2026-07-12 | [link](optimistic-concurrency.md#2026-07-12-269--non-if-match-root-writers-force-write-past-a-concurrent-version-bump) |
|
||||
| `concurrency.idempotent-concurrent-add` | A concurrent duplicate `Add*ToCollection` that loses the race on the composite-key unique constraint is treated as an idempotent no-op (skip the reindex/rebuild fan-out), not a 500 — detected via a provider-specific `TvContext.IsUniqueConstraintViolation` delegate defaulting to "no". | 2026-07-18 | [link](optimistic-concurrency.md#2026-07-18--concurrent-same-item-add-is-idempotent-not-a-500-catch-the-unique-violation-per-provider-308) |
|
||||
| `concurrency.ifmatch-rfc7232` | `ConcurrencyHeaders.ParseIfMatch` is a real RFC 7232 entity-tag/list parser: a syntactically-valid tag that doesn't strong-match (weak/empty/non-canonical/out-of-range/list) returns 412, and only a genuine grammar violation returns 400. | 2026-07-12 | [link](optimistic-concurrency.md#2026-07-12--if-match-evaluates-per-rfc-7232-valid-but-non-matching--412-only-grammar-violations--400-265) |
|
||||
| `concurrency.replace-all-contract` | Replace-all aggregate PUTs carry a uniform plain `int Version` concurrency token (EF `.IsConcurrencyToken()`), checked pre-save and enforced by the EF UPDATE guard, returning 412 (not 409) on a stale `If-Match`. | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--optimistic-concurrency-contract-for-replace-all-puts-253-pr1-infra--block-reference) |
|
||||
| `concurrency.schedule-item-child-identity` | `PUT /api/schedules/{id}/items` reconciles by an optional round-tripped child `Id` (null/absent/0 ⇒ new item), never by array position, so fill-group/shuffle state follows the logical item across reorders; an unknown or duplicate id is rejected 422 (checked after the §7a `CheckVersion`, so 412 precedes 422). | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--stable-child-identity-for-schedule-item-replace-259-split-from-252253) |
|
||||
| `docs.convention-docs-session-start` | Docs-first, not source-first: conventions (api-conventions, spa-conventions, e2e-local, blazor-route-parity, domain-model, decisions, README) are read from docs, not reverse-engineered from code, via `docs/README.md`'s task-signal map — only the sections it points to for the task at hand, not the whole set. Each doc is updated in the same PR that changes what it documents, replacing deferred/follow-up doc updates. | 2026-07-07 | [link](../decisions.md#2026-07-07--convention-docs-read-at-session-start-updated-in-pr) |
|
||||
| `docs.decision-edit-token-scope` | `scripts/decisions_validate.py` arms `[decisions-edit]` only from a commit's **subject line** or a **`Decisions-Edit:` git trailer** — never from anywhere else in the message body, so a commit that merely *describes* the token cannot silently disable the rationale-rewrite guard. | 2026-07-25 | [link](../decisions.md#2026-07-25--the-decisions-edit-token-is-recognized-only-in-the-subject-line-or-a-trailer-never-in-body-prose-609) |
|
||||
| `docs.decision-lifecycle` | every decision `##` record (active or archived) carries a 5-field metadata block (`key`, `status`, `since`, `supersedes`, `superseded-by`) checked by `scripts/decisions_validate.py`; a record is never deleted or line-edited to reverse a call — it is moved to `docs/decisions/archive/` with `status: superseded`/`retired` and a reciprocal `superseded-by`/`supersedes` key pair to its replacement. | 2026-07-21 | [link](../decisions.md#2026-07-21--decision-records-carry-a-lifecycle-schema-validated-by-a-script-append-only-by-diff-is-retired-521) |
|
||||
| `docs.decision-optional-provenance` | Decision records gain two OPTIONAL fields — `stale-after: YYYY-MM-DD` on the metadata line and a `**Sources:**` line in the metadata block; the Open Knowledge Format (OKF) itself is NOT adopted as the record format. | 2026-07-25 | [link](../decisions.md#2026-07-25--okf-evaluated-and-rejected-as-a-replacement-two-of-its-optional-fields-adopted-603) |
|
||||
| `docs.tracker-comment-retrofit` | When the knowledge exporter flags an over-cap tracker issue and excludes it from ingestion, triage its comments instead of assuming a retrofit is owed — and for each decision-shaped item check the **worked issue first**, because a tracker session comment is by construction a précis of the fuller closing record posted on the issue it narrates. Applied to #237 (111 comments) this yielded **zero** records, so server-management#642's "a fact found only in a #237 comment" retrieval row has no valid subject and its interim target (an already-migrated record) is permanent. | 2026-07-21 | [link](../decisions.md#2026-07-21--check-the-worked-issue-before-the-decision-corpus-a-closed-trackers-comments-need-no-retrofit-524) |
|
||||
| `ffmpeg.external-logo-graphics-engine` | External-URL channel logos pass through to the graphics engine like any other watermark source; `WatermarkSelector` must never gate them on `File.Exists` (always false for a URL) and never route them through the ffmpeg-native overlay shortcut. | 2026-07-20 | [link](../decisions.md#2026-07-20--external-url-channel-logos-pass-through-to-the-graphics-engine-never-fileexists-gated-never-ffmpeg-native-502) |
|
||||
| `ffmpeg.hls-cold-start-burst` | HLS cold-start latency is fixed with a bounded `-readrate_initial_burst` (gated on FFmpeg ≥6.1 capability detection), not by raising `work_ahead_limit`, which would remove the concurrency guarantee it exists for. | 2026-07-20 | [link](../decisions.md#2026-07-20--hls-cold-start-is-fixed-with--readrate_initial_burst-not-by-raising-the-work-ahead-limit-350) |
|
||||
| `ffmpeg.qsv-decode-encode-split` | QSV decode is decoupled from QSV encode via a single `FFmpegProfile.QsvPreferNativeDecoder` bool (default ON, Linux-only), so a QSV encode profile can decode with the more tolerant native VA-API decoder instead of the QSV decoder, mirroring Jellyfin's hybrid decode/encode toggle instead of a general decode-family enum. | 2026-07-20 | [link](../decisions.md#2026-07-20-498--qsv-decode-is-split-from-qsv-encode-via-a-single-qsvprefernativedecoder-bool) |
|
||||
| `ffmpeg.qsv-extra-hw-frames-floor` | a QSV upload never emits `extra_hw_frames` below `FFmpegState.MinimumQsvExtraHardwareFrames` (64); a stored `0` or negative value is treated as "no pool configured" rather than honored literally, because with no headroom any unthrottled read exhausts the pool and the transcode writes nothing at all. | 2026-07-21 | [link](../decisions.md#2026-07-21--qsv-hardware-frame-headroom-is-a-floor-not-an-operator-preference-529) |
|
||||
| `ffmpeg.remote-image-fetcher-bounded` | remote graphics-engine images are fetched through `IRemoteImageFetcher` with a pooled `HttpClientFactory` client, a body-covering deadline, a wire-transfer size cap, and a decoder-enforced `DecoderOptions.MaxFrames` bound re-verified post-decode — never cached, re-fetched per element init. | 2026-07-20 | [link](../decisions.md#2026-07-20--remote-graphics-engine-images-are-fetched-through-a-bounded-pooled-iremoteimagefetcher-re-fetched-per-element-init-not-cached-511) |
|
||||
| `ffmpeg.work-ahead-slot-atomic` | `workAheadSegmenterLimit` is enforced by a single compare-exchange claim on a shared `WorkAheadSlots` pool taken by the *caller* of `Transcode`, which then passes ownership in and gets the release in `Transcode`'s `finally` — never a `Volatile.Read` compare in one place and an `Interlocked.Increment` in another. | 2026-07-21 | [link](../decisions.md#2026-07-21--work-ahead-slots-are-claimed-atomically-by-the-caller-released-by-the-transcode-it-hands-them-to-536) |
|
||||
| `ffmpeg.work-ahead-slot-release-never-negative` | `Release()` reads the count and compare-exchanges `current - 1` only when `current > 0`; a release against an empty pool records an unbalanced release and returns `false` **without ever writing a negative value**. It never decrements first and clamps afterward. The single caller (`HlsSessionWorker.Transcode`'s `finally`) logs a warning on the `false` return. | 2026-07-21 | [link](../decisions.md#2026-07-21--workaheadslotsrelease-clamps-before-decrementing-and-reports-unbalance-in-band-539) |
|
||||
| `graphics.channel-level-attachment` | A channel can attach `GraphicsElement`s directly via a new `ChannelGraphicsElement` join table (a base layer under deco/playout-item elements), and a built-in text element (`on-now-next.yml`) is seeded once per database so the On Now/Next overlay works out of the box. | 2026-07-22 | [link](../decisions.md#2026-07-22--channel-level-graphics-element-attachment--seeded-on-nownext-text-element-74) |
|
||||
| `graphics.channel-logo-caching` | An external `http(s)` channel-logo URL is fetched, decode-budget-validated, and stored in the image cache under a content-hash name at SAVE time — becoming byte-identical to an uploaded logo — so the render path never fetches a logo over HTTP; a bad URL fails the save with a 422 (BaseError → ValidationProblemDetails). | 2026-07-21 | [link](../decisions.md#2026-07-21--external-channel-logo-urls-are-downloaded-and-cached-at-save-time-the-render-path-never-fetches-a-logo-525) |
|
||||
| `iptv.base-url` | An optional advertised base URL (`iptv.base_url`) is resolved centrally via a pure Core helper (`AdvertisedBaseUrl`) inside the two IPTV generation handlers (M3U + XMLTV); unset/malformed values fall back byte-identical to the request-derived host, and it's a new `iptv` settings group distinct from `ETV_BASE_URL` and out of scope for HDHomeRun. | 2026-07-16 | [link](../decisions.md#2026-07-16--optional-advertised-iptv-base-url-iptvbase_url-resolved-centrally-in-the-two-generators-340) |
|
||||
| `iptv.logo-drives-bug-preset` | One uploaded channel logo drives both the listing logo and the on-screen bug via a shared, seeded `ChannelLogo`-sourced watermark preset (`Channel Bug`), not new per-channel schema. | 2026-07-20 | [link](../decisions.md#2026-07-20--one-logo-drives-the-bug-via-a-shared-channellogo-preset-not-new-schema-67) |
|
||||
| `locking.entitylocker-atomic-flags` | `EntityLocker` uses `Interlocked.CompareExchange`-guarded atomic flags plus a documented single-owner-release discipline (no owner tokens/leases); `Unlock*` on an already-unlocked slot returns `false` and logs a Warning rather than throwing. | 2026-07-11 | [link](../decisions.md#2026-07-11--entitylocker-atomic-flags--single-owner-release-discipline-no-owner-tokens-231) |
|
||||
| `mcp.server-foundation` | `ErsatzTV.Mcp` is a fresh stdio JSON-RPC server wrapping frozen `/api/v1` with explicit narrow per-endpoint tools, read-only-by-default enforced at runtime (`ERSATZTV_ALLOW_WRITES`), machine-key auth, and opt-in `If-Match`. | 2026-07-20 | [link](../decisions.md#2026-07-20--mcp-server-ersatztvmcp-built-fresh-over-frozen-apiv1-read--cautious-writes-58) |
|
||||
| `media.lastscan-null-boundary` | A never-scanned `LastScan` surfaces as `null` at the API/MCP boundary, not the `0001-01-01` MinValue sentinel — enforced by an ongoing read-boundary coercion plus a one-time data migration cleanup. | 2026-07-18 | [link](../decisions.md#2026-07-18--never-scanned-lastscan-surfaces-as-null-at-the-api-boundary-not-the-0001-01-01-minvalue-sentinel-409) |
|
||||
| `media.remote-stream-probe` | `ValidatePlayoutItemPath` probes the Plex/Jellyfin/Emby remote-stream URL via `IRemoteStreamProber` before returning it; only a redirected 404 fails closed (`PlayoutItemNotAvailableFromMediaServer`), everything else fails open, and there is no toggle. | 2026-07-19 | [link](../decisions.md#2026-07-19--media-server-remote-stream-urls-are-probed-before-use-a-redirected-404-fails-closed-everything-else-fails-open-no-toggle-473) |
|
||||
| `media.remote-stream-probe-externaljson` | External-JSON playout channels' `StreamRemotely` now probes the remote-stream URL through the same `IRemoteStreamProber` seam as the generated-playout path, closing the #473 scope gap for a channel kind with no DB `PlayoutItem` rows. | 2026-07-20 | [link](../decisions.md#2026-07-20--external-json-playout-channels-now-probe-the-remote-stream-url-too-closing-the-473-scope-gap-480) |
|
||||
| `media.source-mgmt-write-api` | Media-source management (local/Plex/Jellyfin/Emby) is a REST write API + SPA under `/app/libraries/*`, wrapping existing MediatR commands 1:1 with no new commands or DB migration; connection GETs never leak a stored `apiKey`, and each PUT-replace family's identity contract is documented per-family (not assumed uniform). | 2026-07-11 | [link](../decisions.md#2026-07-11--media-source-management-rest-write-api--spa-202) |
|
||||
| `process.bom-format-detection-recipe` | Before any push touching `.cs`, detect BOMs with the `xxd` byte check and verify the format gate with `dotnet format --include` run under `bash -c`, never bare zsh. | 2026-07-21 | [link](workflow-process.md#2026-07-21--bomformat-pre-push-detection-use-the-xxd-loop-and-run-dotnet-format---include-under-bash--c-542) |
|
||||
| `process.branch-off-feature-branch` | To fix work on an unmerged feature branch, branch off that branch and land by fast-forward push — and after creating a worktree, drive the first Edit/Read from ITS absolute paths and `git status` it before building. | 2026-07-21 | [link](workflow-process.md#2026-07-21--fix-onto-an-unmerged-feature-branch-by-branching-off-it-and-ff-pushing-542) |
|
||||
| `process.build-concurrency-limits` | Run at most 3–4 concurrent dotnet/npm builds on this Mac, gate launches on FREE RAM rather than CPU load, and never set `ETV_UPDATE_GOLDENS` / `ETV_UPDATE_PLAYOUT_GOLDENS`. | 2026-07-21 | [link](workflow-process.md#2026-07-21--bound-parallel-builds-by-free-ram-never-regenerate-goldens-542) |
|
||||
| `process.codex-cheap-worker-launch` | For bounded tool-bearing selector/recon work, launch a Codex worker with `codex exec -m gpt-5.4-mini -c model_reasoning_effort=low -s read-only`; `spawn_agent` buys parallelism but no cost savings. | 2026-07-21 | [link](workflow-process.md#2026-07-21--launch-codex-cheap-workers-via-codex-exec-not-spawn_agent-542) |
|
||||
| `process.consistency-fix-new-code-scrutiny` | Review a "make X consistent with Y" change as new code, not as a mechanical copy — and for any timer or effect involved, ask explicitly "when does this fire?", including on mount. | 2026-07-21 | [link](workflow-process.md#2026-07-21--make-x-consistent-with-y-review-findings-are-new-code-and-get-new-code-scrutiny-542) |
|
||||
| `process.enumerate-workaround-behaviors-before-deleting` | When an issue says "delete X", enumerate every behavior X provided before removing it — a workaround often serves a second purpose that outlives the first. | 2026-07-21 | [link](workflow-process.md#2026-07-21--before-deleting-a-workaround-enumerate-every-behavior-it-provided-542) |
|
||||
| `process.foreign-worktree-plumbing-merge` | Never commit or merge inside a worktree another session created; land the merge with git plumbing against the branch ref instead. | 2026-07-21 | [link](workflow-process.md#2026-07-21--never-commit-or-merge-inside-a-worktree-you-did-not-create-542) |
|
||||
| `process.harden-with-runtime-posture-not-clamp` | When a security fix constrains a capability the roadmap will later want, make the safe state the DEFAULT OF A SWITCH rather than a wall — and read the feature's own issue for its end-state first. | 2026-07-21 | [link](workflow-process.md#2026-07-21--harden-a-soon-to-grow-feature-with-a-runtime-posture-not-a-hardcoded-clamp-542) |
|
||||
| `process.independent-review-rubric` | Run an independent review pass — preferably a different model family, otherwise a cold-context review-only agent — on any diff touching locks/concurrency, auth/security, API write-path handlers, or DB migrations, or larger than ~150 changed C# lines; skip only for a pure-SPA/docs leaf with no server-state effect, and state the skip and its reason in the PR or close comment. | 2026-07-21 | [link](workflow-process.md#2026-07-21--independent-cross-model-review-is-mandatory-on-risky-diffs-a-skip-is-a-stated-auditable-exemption-542) |
|
||||
| `process.issue-qualification-audit` | Run `scripts/issue-qualification-audit.sh` at session end and label everything it flags, including issues you filed that session. | 2026-07-21 | [link](workflow-process.md#2026-07-21--every-open-issue-carries-a-priority-label--run-the-h12-audit-at-session-end-542) |
|
||||
| `process.local-gate-before-push` | Run the local build/test gate and a cold-context, scoped "review only" adversarial review over the diff, fold the fixes, and only then push or open the PR. | 2026-07-21 | [link](workflow-process.md#2026-07-21--local-buildtest-gate--cold-context-review-run-before-the-push-not-after-542) |
|
||||
| `process.lock-ownership-enumerate-producers` | Before trusting any "single owner / no double release / no cross-release" claim, grep the whole host project for every writer of that channel message (or acquirer of that lock) — the background scheduler/worker is the usual missing producer. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-lockchannel-no-cross-release-verdict-must-enumerate-every-producer-via-grep-542) |
|
||||
| `process.one-worktree-one-committing-agent` | Never run two committing agents concurrently on one worktree — give each parallel slice its own worktree branched off the feature branch and merge back. | 2026-07-21 | [link](workflow-process.md#2026-07-21--one-worktree-one-committing-agent-542) |
|
||||
| `process.parallel-session-claim` | Apply the `in-progress` label before starting an issue, and still read its dependency notes before touching shared surfaces — a claim prevents duplicate pickup, not overlapping code changes. | 2026-07-21 | [link](workflow-process.md#2026-07-21--claim-with-in-progress-before-working-claiming-is-not-collision-safety-542) |
|
||||
| `process.per-agent-model-routing` | State the model tier (and effort, where the client exposes it) in the dispatch itself for every delegated agent — bounded recon → cheapest fast tier at `low`; mechanical slice against a documented contract → mid tier; judgment-heavy work → orchestrator tier; independent review → a different model family than the implementer. | 2026-07-25 | [link](workflow-process.md#2026-07-25--name-the-model-tier-for-every-dispatched-agent-a-pretooluse-gate-makes-the-silent-default-visible-583) |
|
||||
| `process.pr-routine-sequence` | Worktree off origin/main → implement → regenerate API artifacts → full local tests + cold review + live-E2E ALL before the push → push, open PR, arm the CI monitor at open → fixes after the push are follow-up commits, never amend/force-push. | 2026-07-21 | [link](workflow-process.md#2026-07-21--the-pr-routine-is-a-fixed-sequence-validate-locally-then-push-then-only-follow-up-commits-542) |
|
||||
| `process.review-disagreement-frontier-judge` | When independent reviews disagree on a gate PR, escalate to the frontier judge, and put the proposed fix approach in front of it — not just the disputed finding. | 2026-07-21 | [link](workflow-process.md#2026-07-21--review-disagreement-on-a-gate-pr-escalates-to-the-frontier-judge--and-the-proposed-fix-escalates-with-it-542) |
|
||||
| `process.shared-tree-readonly` | Never commit in `/Users/timothy/ersatztv` and never read its `git log`/`git status`/HEAD to infer anything about `main` — work in a worktree off `origin/main`, which is the only source of truth. | 2026-07-21 | [link](workflow-process.md#2026-07-21--the-shared-tree-at-userstimothyersatztv-is-read-only-and-tells-you-nothing-about-main-542) |
|
||||
| `process.subagent-drop-resume` | Treat a subagent connection drop as laptop sleep or transient network and re-resume via SendMessage — the work survives. | 2026-07-21 | [link](workflow-process.md#2026-07-21--a-dropped-subagent-connection-is-transient-resume-dont-restart-542) |
|
||||
| `release.api-contract-ci-gate` | A PR touching `ErsatzTV/Controllers/Api/**` or `ErsatzTV.Core/Api/**` must ship regenerated OpenAPI artifacts (`v1.json`, `v1.d.ts`, `endpoint-index.md`) in the same diff, enforced by a blocking `api-docs` CI job that regenerates-and-diffs against a fresh build. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--blocking-ci-gate-for-api-contract-artifacts-303-h4h5) |
|
||||
| `release.done-when-merge-consent` | A PR may merge only when its linked issue's `## Done-when` checklist is fully ticked and the PR's CI is green, enforced by a PreToolUse hook on the Gitea merge tool (deny/allow/ask) plus a pre-push backstop for direct pushes to main. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--merge-consent-derived-from-state-via-a--done-when-issue-checklist-303-h6) |
|
||||
| `release.format-as-you-touch-rebase` | A blocking `format` CI job runs `dotnet format --verify-no-changes` scoped only to the PR's changed `.cs` files (never the legacy BOM backlog), and a PR branch must be kept current by rebasing on `origin/main` (never merging main in), enforced by `.husky/pre-push` → `prepush-rebase-check.sh`. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--formatting-as-you-touch-enforced-rebase-not-merge-for-pr-branches-311-h11--format-ci) |
|
||||
| `release.live-e2e-required` | A PR that changes an API write-path handler must include a live-E2E pass (driving the real endpoint/screen and confirming the round-trip through a subsequent read), not only unit/characterization tests, and must state whether live-E2E ran or wasn't required. | 2026-07-12 | [link](../decisions.md#2026-07-12--live-e2e-is-a-required-step-for-api-write-path-handler-changes-303) |
|
||||
| `release.merge-consent-autogrant` | When Done-when boxes are ticked, CI is green, and a fresh positive Review-verdict references head, the merge-consent hook emits `permissionDecision: allow` to actually suppress the redundant mechanical prompt — the derived state IS the consent, no separate conversational confirmation on that path. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--merge-consent-gate-auto-grants-when-satisfied-no-redundant-prompt-state-is-the-consent-314) |
|
||||
| `release.migration-rehearsal-prodcopy` | Before promoting a migration-bearing release, rehearse the new image's migrations against a throwaway copy of the latest prod backup (`scripts/migration-smoke.sh`), gating PASS on the migrator's completion log line rather than HTTP readiness alone. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--release-path-rehearses-migrations-on-a-prod-db-copy-before-promoting-315) |
|
||||
| `release.prepush-clean-worktree-guard` | A fail-open pre-push hook blocks a push when any file in the branch's diff vs `origin/main` also has uncommitted working-tree or index changes, since a stale-index commit (e.g. `git reset --soft` + `git add` over an edited-but-unstaged fix) can silently push, CI-test, and get reviewed a different tree than the one on disk. Scope is precise to pushed-diff files; escape hatch `ETV_ALLOW_DIRTY_PUSH=1`. | 2026-07-17 | [link](../decisions.md#2026-07-17--pre-push-guard-dont-push-a-file-whose-working-tree-copy-is-uncommitted-h13-416-session) |
|
||||
| `release.promotion-floating-prod` | Prod tracks the floating `:prod` image reference; a tag build's immutable `:<version>` image is scanned first, then promotion happens via a separate manual `DeployStack`, with daily auto-update only as a fallback — tag with enough runway before 03:00 to avoid an unscanned promotion. | 2026-07-13 | [link](release-ci-governance.md#2026-07-13--release-promotion-floating-prod-exact-image-scan-before-manual-deploy-335) |
|
||||
| `release.review-verdict-gate` | A PR may not merge until a `Review-verdict: <MERGEABLE\|APPROVED\|BLOCKED\|NOT-MERGEABLE> @ <head-sha>` comment references the PR's current head sha (short-sha prefix match, line-start marker only, negative wins over positive on the same head); folds into the H6 merge-consent hook as condition (c). | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--review-verdict-merge-gate-latest-commit-must-be-reviewed-303-h10) |
|
||||
| `rulebuilder.relative-date-macros` | The visual rule builder's `inLast`/`notInLast` date operators compile to/parse from the pre-existing `CustomMultiFieldQueryParser` macros `released_inthelast`/`released_notinthelast` and `added_inthelast`/`added_notinthelast`, value form `"<n> day\|week\|month\|year"`; there is no backend change. | 2026-07-23 | [link](../decisions.md#2026-07-23--relative-date-rule-builder-operators-are-a-frontend-only-mapping-onto-existing-lucene-macros-435) |
|
||||
| `scan.collections-scan-status` | `GET /api/v1/media-sources/collections-scan-status` reports a family-global (not per-source), boolean-only active-scan set read from `IEntityLocker`; the SPA reconciles authoritatively against it (with a grace-tick helper) instead of a fixed client-side timeout. | 2026-07-12 | [link](../decisions.md#2026-07-12--external-collections-scans-get-an-authoritative-status-surface-271-the-spa-timeout-is-retired) |
|
||||
| `scan.getoraddfolder-db-lookup` | `ILibraryRepository.GetOrAddFolder` resolves the existing folder via a DB query on `(LibraryPathId, Path)`, not the caller's `LibraryPath.LibraryFolders` in-memory navigation, since that navigation is only eager-loaded on the local scan path and is null on remote (Jellyfin) callers. | 2026-07-20 | [link](../decisions.md#2026-07-20--ilibraryrepositorygetoraddfolder-resolves-the-folder-from-the-db-not-the-callers-librarypathlibraryfolders-navigation-488) |
|
||||
| `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](../decisions.md#2026-07-20-489--jellyfin-mixed-content-libraries-map-to-one-library-holding-many-kinds) |
|
||||
| `scan.musicvideo-server-identity` | Jellyfin music videos carry a per-library server identity (`JellyfinMusicVideo : MusicVideo` with `ItemId`/`Etag`, TPT table + ItemId index), so `JellyfinMusicVideoLibraryScanner` folds onto a shared `MediaServerMusicVideoLibraryScanner` base that diffs the **server item id** and soft-trashes (`FlagFileNotFound`) instead of diffing local paths and hard-deleting. Rows predating the identity are **adopted in place** — the identity row is inserted against the same `MediaItem` id, scoped to the scanned library's own `LibraryPath` — never deleted and re-added. | 2026-07-25 | [link](../decisions.md#2026-07-25--music-videos-carry-a-per-library-server-identity-reconciliation-is-an-itemid-diff--soft-trash-496) |
|
||||
| `scan.projection-failure-sweep-guard` | `MediaServerReconciliationGuard` takes a per-enumeration projection-failure count and refuses the file-not-found sweep (logged loudly) whenever it is non-zero against a non-empty existing set — at all six sweeps, including the nested per-show season and per-season episode ones #477 left unguarded (via `ShouldFlagMissingDescendants`, which applies the failure refusal but not #477's empty-fetch branch). Deliberate guard-clause skips (STRM, virtual, unsupported type) are explicitly **not** failures and never suppress a sweep. The missing-fraction / ratio threshold floated by #477 is **rejected**, not deferred. | 2026-07-25 | [link](../decisions.md#2026-07-25--a-media-server-sweep-also-refuses-when-the-api-client-silently-dropped-items-whose-projection-threw-the-ratio-threshold-is-rejected-484) |
|
||||
| `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](../decisions.md#2026-07-19--a-media-server-library-sweep-refuses-to-flag-when-a-successful-fetch-returns-zero-items-rather-than-nuking-the-whole-library-477) |
|
||||
| `sched.auto-tune-foundation` | Auto-tune preview enumeration uses EF distinct+count queries for exact counts, while each created channel is persisted as a live SmartCollection; coexistence with existing channels/numbers is additive-only, never mutating. | 2026-07-16 | [link](../decisions.md#2026-07-16--auto-tuning-enumerates-via-ef-persists-via-smartcollection-additive-coexistence-69) |
|
||||
| `sched.autotune-detailpanel-members` | The Auto-Tune DetailPanel's per-channel content-source list is a live `ISearchIndex.Search` roll-up through the server-owned `AutoTuneAxisMap.GenerateQuery`, not an EF distinct+count query, so the preview matches exactly what the built channel's SmartCollection will contain. | 2026-07-17 | [link](../decisions.md#2026-07-17--auto-tune-detailpanel-member-list--live-search-index-roll-up-not-ef-enumeration-384) |
|
||||
| `sched.autotune-per-channel-overrides` | Auto-Tune per-channel overrides reuse the Channel Builder's advanced-options DTO verbatim; per-source weights and bug-colour logo are deferred to #425. | 2026-07-17 | [link](../decisions.md#2026-07-17--auto-tune-per-channel-overrides-reuse-the-channel-builder-advanced-options-dto-weights--bug-colour-logo-split-out-to-425-385) |
|
||||
| `sched.autotune-per-source-weights` | Auto-Tune per-source rotation weights and query corrections are supplied at bulk-create time via #70's MultiCollection/SmartCollection machinery, not a post-hoc PUT. | 2026-07-18 | [link](../decisions.md#2026-07-18--auto-tune-per-source-weights-ride-70s-multicollection-machinery-created-at-tune-time-not-a-post-hoc-put-425) |
|
||||
| `sched.clock-padding-existing` | Clock-boundary padding already exists via `FillerPreset`'s `FillerMode.Pad` (Classic) and `pad_to_next`/`pad_until` (Sequential/YAML); #77 is closed as verified+documented, not built new, with a one-click per-channel toggle deferred behind the #388 design-system epic. | 2026-07-17 | [link](../decisions.md#2026-07-17--clock-boundary-schedule-padding-already-exists-fillermodepad-77-verified-convenience-toggle-deferred) |
|
||||
| `sched.clock-padding-schedule-toggle` | A `ProgramSchedule.PadToNearestMinute` (nullable int; null = off) makes the Classic builder pad every content item up to the next N-minute clock boundary without a hand-wired Pad `FillerPreset`, by reusing the existing per-content-item Pad path in `PlayoutModeSchedulerBase.AddFiller`. It extends — does not supersede — `sched.clock-padding-existing` (#77/#388). | 2026-07-22 | [link](../decisions.md#2026-07-22--per-schedule-clock-boundary-padding-is-a-synthetic-content-less-pad-over-the-existing-per-episode-machinery-392) |
|
||||
| `sched.playbackorder-support-matrix` | Every build-time dispatch site logs a loud (non-fatal) warning on an unsupported `PlaybackOrder`, and a declared `PlaybackOrderSupport` matrix + partition tripwire test makes adding a new order safe by construction. | 2026-07-18 | [link](../decisions.md#2026-07-18--unsupported-playbackorder-is-loud-at-build-time-a-declared-support-matrix-and-tripwire-test-make-new-orders-safe-by-construction-403) |
|
||||
| `sched.reshuffle-scoped-reset` | `POST /api/v1/playouts/{id}/reshuffle` runs `ErasePlayoutHistory` (reseeds `Playout.Seed` + clears anchors/rerun-history) then enqueues a scoped `Reset` build, so reshuffle always reseeds — even for the non-Classic kinds `Reset` alone wouldn't reseed; `Playout.Seed` is surfaced on list/detail DTOs as visible confirmation. | 2026-07-16 | [link](../decisions.md#2026-07-16--per-playout-reshuffle--scoped-reset-build-seed-surfaced-71) |
|
||||
| `sched.seasonal-scheduling-existing` | Seasonal/date-conditional scheduling already ships first-class via `IAlternateScheduleItem` (Classic `ProgramScheduleAlternate`, Block `PlayoutTemplate`) evaluated by `AlternateScheduleSelector.GetScheduleForDate` (first match in `Index` order, catch-all last); #73 is closed as already-implemented with a docs-only "seasonal/holiday" recipe added, not new code. | 2026-07-17 | [link](../decisions.md#2026-07-17--seasonal--date-conditional-scheduling-already-exists-alternate-schedules--playout-templates-73-closed-as-implemented) |
|
||||
| `sched.shuffle-source-builder` | Shuffle-source construction moves to a static, DI-free `ShuffleSourceBuilder` (a shared seam, not a service) so Classic and Playlist stop cross-engine reaching into `PlayoutBuilder` statics; a unified Classic+Playlist enumerator factory is explicitly rejected as a god-factory. Block/Scripted/YAML duplication is left alone, deferred to a follow-up gated on #381. | 2026-07-17 | [link](../decisions.md#2026-07-17--shuffle-source-construction-extracted-to-shufflesourcebuilder-per-family-seam-not-a-god-factory-380) |
|
||||
| `sched.weighted-shuffle` | Fair-share/weighted airtime distribution ships as one new `PlaybackOrder.WeightedShuffle = 9` order (equal weights = fair-share), not a retrofit of `ShuffleInOrder` (which only anti-clumps, since its padding spacers emit nothing) and not a separate orthogonal "distribution" setting; weights live on `MultiCollectionItem`/`MultiCollectionSmartItem` (DB default 1, dual-provider migration), bounded at write (1..1000) and clamped again in the enumerator, and the write path rejects `WeightedShuffle` at every dispatch site that doesn't handle it rather than let it silently degrade to unweighted random. | 2026-07-17 | [link](../decisions.md#2026-07-17--weighted--fair-share-distribution-is-a-new-weightedshuffle-order-shuffleinorder-is-anti-clumping-not-fair-share-70) |
|
||||
| `sched.weightedshuffle-editor` | WeightedShuffle per-source weights are edited on the multi-collection editor (property of the MultiCollection), while the WeightedShuffle order itself is offered only on classic MultiCollection schedule items; fair-share is a "reset weights to 1" action, not a stored mode. | 2026-07-19 | [link](../decisions.md#2026-07-19--weightedshuffle-spa-weights-edited-on-the-multi-collection-order-offered-only-on-classic-multicollection-schedule-items-fair-share-is-a-reset-not-a-mode-404) |
|
||||
| `scheduling.ondemand-guide-refresh-on-thaw` | When `PlayoutTimeShifter.TimeShift` slides an on-demand playout's materialized timeline forward on tune-in, it reports the channel numbers whose cached guide is now stale — the shifted channel **plus any channels that mirror it** — and `TimeShiftOnDemandPlayoutHandler` enqueues a `RefreshChannelData` for each, so every affected cached XMLTV fragment is regenerated from the just-shifted `PlayoutItem` rows. The guide and playback both read the same stored `PlayoutItem.Start/Finish`, but the guide is served from a **cached** projection — rewriting the rows without rebuilding the cache would leave the guide advertising a stale timeline. | 2026-07-21 | [link](../decisions.md#2026-07-21--an-on-demand-time-shift-rebuilds-the-channels-cached-xmltv-so-the-guide-cant-lag-playback-68) |
|
||||
| `security.artwork-content-type-sniff` | Artwork content type is always derived from the stored bytes (never the client-declared value or a `?contentType=` query param) at both upload and serve, clamped to an image allow-list, closing the unauthenticated stored-XSS chain; Kestrel `MaxRequestBodySize` bounds upload DoS. | 2026-07-12 | [link](api-auth-security.md#2026-07-12--artwork-content-type-is-sniffed-never-reflected-283-s4s9-stored-xss) |
|
||||
| `security.baseline-response-headers` | `SecurityHeadersMiddleware`, registered first in the pipeline, sets `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, and `Referrer-Policy: strict-origin-when-cross-origin` on every response (CSP/HSTS deliberately deferred); API-key comparison is constant-time and playout pagination is clamped. | 2026-07-11 | [link](api-auth-security.md#2026-07-11--baseline-security-response-headers--phase-0-api-hardening-197-pr-279) |
|
||||
| `security.blazor-removal-auth-posture` | Removing the Blazor UI's OIDC-challenged surface exposes nothing a user couldn't already reach via the already-open `/app` SPA (open since phase (a)); real SPA/API authentication is deliberately deferred to #197, and the removal PR must preserve `ConditionalIptvAuthorizeFilter`, `ApiKeyAuthorizationFilter`, and `JwtHelper` access_token support. | 2026-07-11 | [link](api-auth-security.md#2026-07-11--blazor-removal-auth-posture-no-new-exposure-beyond-phase-a-real-auth-deferred-to-197-206) |
|
||||
| `security.contract-freeze-honesty` | The OpenAPI doc's declared security/401 scheme is generated from the same `ApiKeyAuthorizationFilter.EndpointRequiresKey` predicate the runtime enforces (so declared auth can't drift from enforced auth), every `/api/*` action returns a ResponseModel (no raw Application VMs), and Channel REST resources are keyed by immutable `Id`, never mutable `Number`. | 2026-07-12 | [link](api-auth-security.md#2026-07-12-197-bundle-c--contract-freeze-honesty) |
|
||||
| `security.corp-same-origin` | `SecurityHeadersMiddleware` sends `Cross-Origin-Resource-Policy: same-origin` on every response including `/docs`/`/openapi`, blocking cross-origin `no-cors` embedding without affecting allowed CORS-mode fetches or server-side Jellyfin `/iptv/*` requests. | 2026-07-13 | [link](api-auth-security.md#2026-07-13--cross-origin-resource-policy-same-origin-on-every-response-330) |
|
||||
| `security.csp-permissions-policy` | `SecurityHeadersMiddleware` sends an enforcing (not report-only) `Content-Security-Policy` (no `unsafe-inline`/`unsafe-eval`; the one inline theme-bootstrap script allow-listed by hash) and a deny-all `Permissions-Policy` on the SPA/`/api`/`/artwork`/`/iptv`; `/docs` and `/openapi` keep only the baseline headers, excluded from CSP because Scalar needs inline bootstrap. | 2026-07-12 | [link](api-auth-security.md#2026-07-12--enforcing-csp--permissions-policy-on-the-host-319-zap-baseline) |
|
||||
| `security.fail-closed-api-auth` | Every mutating `/api` request requires `X-Api-Key` (no open mode); reads are gated by `Api:RequireKeyForReads` (default true) OR `[RequiresApiKey]` on sensitive controllers; CORS is an exact-origin allowlist (`ApiCors`); `ForwardedHeaders` trust stays configurable but defaults to trust-all-with-warning. | 2026-07-12 | [link](api-auth-security.md#2026-07-12--fail-closed-api-auth--sensitive-read-tier--corsforwardedheaders-lockdown-197-bundle-a-pr-292) |
|
||||
| `security.iptv-access-token-transport` | The `/iptv` `?access_token=` value is percent-encoded (`Uri.EscapeDataString`) everywhere it is interpolated into an M3U/HLS/XMLTV URL (XMLTV additionally XML-escapes the encoded value), so a structural character can't malform the manifest or guide; Serilog logs a scrubbed request path (`access_token` → `***` via `IncludeQueryInRequestPath = false` + a `RequestPathScrubbed` enricher), so a 5xx/Debug `/iptv` request never writes the token; and every dynamic token-bearing `/iptv` manifest (`channels.m3u`, `xmltv.xml`, the HLS multi-variant/media playlists) returns `Cache-Control: private, no-store`. | 2026-07-23 | [link](api-auth-security.md#2026-07-23--access_token-transport-hardening-percent-encode-in-m3uhls-redact-from-logs-no-store-on-tokened-manifests-421-559) |
|
||||
| `security.iptv-browser-token` | Under a JWT-enabled deployment (`JWT:IssuerSigningKey` set), the browser SPA obtains a short-lived, globally-scoped `/iptv/*` access token from an authenticated `GET /api/v1/auth/iptv-token` and appends it as `?access_token=`; the endpoint answers 204 when JWT is disabled (nothing to mint). Lifetime defaults to 60 min, configurable via `JWT:BrowserTokenLifetimeMinutes`. | 2026-07-22 | [link](api-auth-security.md#2026-07-22--short-lived-browser-iptv-token-so-the-spa-reaches-iptv-under-jwt-auth-552) |
|
||||
| `security.session-auth-dual-credential` | `ApiAuthorizationFilter` accepts a request when a valid `X-Api-Key` matches OR the principal is an authenticated session (cookie `ctv-session`, `HttpOnly`/`SameSite=Lax`); session-authenticated mutations require the presence-only `X-CSRF` header or are rejected 403. This narrows the OIDC-inert sub-claim of `security.blazor-removal-auth-posture` (#206) — the rest of that record's auth-surface enumeration still holds. | 2026-07-12 | [link](api-auth-security.md#2026-07-12--browser-spa-session-auth-api-accepts-session-or-machine-key-295-pr1-server-only) |
|
||||
| `security.session-cutover-postify` | The browser SPA authenticates cookie-only (no more `X-Api-Key` from `web/`); the machine key is repurposed to external/MCP-only via `GET /api/auth/machine-key`; every side-effecting GET/HEAD under `/api` is converted to POST so the existing CSRF gate covers it (standing rule: never add a side-effecting GET/HEAD under `/api`). | 2026-07-12 | [link](api-auth-security.md#2026-07-12--295-pr2-spa-session-cutover--301-side-effecting-get-post-ification) |
|
||||
| `session.shared-checkout-refresh` | Session end runs `scripts/refresh-shared-checkout.sh`, which fast-forwards `/Users/timothy/ersatztv` to `origin/main` (and reinstalls `web/node_modules` when the lockfile moved), refusing to touch anything unless that tree is on a clean, non-ahead `main`. | 2026-07-21 | [link](../decisions.md#2026-07-21--session-end-fast-forwards-the-shared-checkout-a-stale-tree-serves-stale-files-541) |
|
||||
| `spa.add-to-layer` | All add-to-collection/playlist/schedule affordances share one component layer at `web/src/media/addTo/`; multi-select is an explicit screen-level toggle, and the per-card menu offers schedule only for the server-validated kinds. | 2026-07-10 | [link](../decisions.md#2026-07-10--shared-add-to-layer-lives-in-websrcmediaaddto-select-mode-is-an-explicit-toggle) |
|
||||
| `spa.app-shell-extraction` | `App.tsx` is only the composition root over `web/src/app/routes.tsx` (stable route-object identity), `app/AppShell.tsx` (shell chrome), and `app/ScreenContent.tsx` (exhaustive screen dispatch); primary actions are one explicit `PrimaryActionProvider` registration per screen, replacing the old global `ctv:primary-action` window event. | 2026-07-15 | [link](spa-modularization.md#2026-07-15--app-shellrouting-extraction--explicit-primary-action-ownership-247) |
|
||||
| `spa.autotune-detailpanel-slideover` | The Auto-Tune DetailPanel SPA is a reusable `SlideOver` primitive sharing `useOverlayBehavior` with `Dialog`, plus a shared advanced-options model extracted from ChannelBuilder; decorative panes without backend support are dropped. | 2026-07-18 | [link](../decisions.md#2026-07-18--auto-tune-detailpanel-spa-reusable-slideover--shared-advanced-options-model-decorative-panes-dropped-to-match-the-backend-386) |
|
||||
| `spa.channel-editor-create-logo` | Bare-channel create is a "New blank channel" action on the channels list (reusing Blazor's add-mode defaults) that navigates into the full editor, and an external logo URL always wins over an uploaded logo, matching `ChannelEditViewModel` precedence. | 2026-07-11 | [link](../decisions.md#2026-07-11--channel-editor-bare-create-entry-point--external-logo-mutual-exclusion-212) |
|
||||
| `spa.channel-renumber-prompt` | Channel renumbering uses a sequential `prompt()`-driven "Renumber" action instead of drag-to-reorder. | 2026-07-09 | [link](../decisions.md#2026-07-09--channel-numbers-prompt-driven-sequential-renumber-instead-of-drag-to-reorder) |
|
||||
| `spa.channels-screen-extraction` | The Channels domain is a single-file zero-prop screen (`web/src/screens/ChannelsScreen.tsx`) with a colocated test file and no sibling helper directory, since its pure logic is too small (~30 lines) to justify a separate business-rule layer like Schedules' `itemRules.ts`. | 2026-07-11 | [link](spa-modularization.md#2026-07-11--channels-screen-extraction-244-single-file-screen-no-sibling-helper-dir-epic-243-phase-1) |
|
||||
| `spa.collection-custom-order-ui` | Collection custom ordering uses per-row Move up/Move down buttons (not drag) and is offered for any manual collection with custom order enabled, not just movies-only. | 2026-07-09 | [link](../decisions.md#2026-07-09--collection-custom-order-move-updown-buttons-any-kind-collections) |
|
||||
| `spa.datetime-local-input` | The channel-mode date/time input uses a native `<input type="datetime-local">` instead of free-text Chronic natural-language parsing. | 2026-07-09 | [link](../decisions.md#2026-07-09--datetime-local-instead-of-chronic-natural-language-start-parsing) |
|
||||
| `spa.deco-templates-table` | The deco-templates editor also renders its day/deco assignment as a table, extending (not replacing) the templates-editor-table convention. | 2026-07-09 | [link](../decisions.md#2026-07-09--table-not-calendar-convention-also-covers-the-deco-templates-editor) |
|
||||
| `spa.download-sample-gate` | The SPA disables both Download Media Sample and Download Results while a troubleshooting session is starting/running (Blazor only gated Download Results). | 2026-07-09 | [link](../decisions.md#2026-07-09--spa-gates-download-media-sample-while-a-session-is-active) |
|
||||
| `spa.legacy-redirect-matcher` | `LegacyUiRedirects.TryGetRedirect` is a two-tier matcher — an exact `OrdinalIgnoreCase` `Map` (Tier 1) then an ordered segment-template pattern list (Tier 2, first-match-wins) — collision-free by construction, with a guard invariant that no rule may prefix-match `/api`, `/artwork`, `/docs`, `/openapi`, `/iptv`, `/app`, or `/media/sources`. | 2026-07-11 | [link](../decisions.md#2026-07-11--legacyspa-redirect-matcher-exact-map--ordered-segment-template-patterns-204) |
|
||||
| `spa.logs-page-size-local` | The Logs page rows-per-page preference is stored in `window.localStorage` (`ctv-logs-page-size`), not a server `ConfigElement`. | 2026-07-11 | [link](../decisions.md#2026-07-11--logs-page-size-is-a-client-local-preference-not-a-server-configelement) |
|
||||
| `spa.playback-troubleshoot-poll` | The playback-troubleshooting screen reports FFmpeg completion by polling `GET /api/troubleshoot/playback/status` (~2s) rather than a server push channel. | 2026-07-09 | [link](../decisions.md#2026-07-09--playback-troubleshooting-completion-feedback-poll-status-no-push-channel) |
|
||||
| `spa.playout-reset-button` | The SPA keeps a single Reset action (server picks the default build mode) and drops Blazor's separate "Schedule reset" button since its capability already exists via the playout's Edit-details flow. | 2026-07-09 | [link](../decisions.md#2026-07-09--per-playout-schedule-reset-button-dropped-reset-uses-the-server-default-build-mode) |
|
||||
| `spa.playouts-screen-extraction` | The Playouts domain (including its unguarded `PlayoutsRouteScreen` route wrapper with local pathname/popstate state) moved as one unit into `web/src/screens/PlayoutsScreen.tsx`, keeping its screen-specific sub-path route ownership colocated with the base screen; a pure structural move with no API/route/CSS/behavior change. | 2026-07-14 | [link](spa-modularization.md#2026-07-14--playouts-screen-extraction-245-screen-owned-route-wrapper-epic-243-phase-2) |
|
||||
| `spa.rulebuilder-nesting` | The visual rule builder's `Group` nests recursively to a single shared cap, `MAX_GROUP_DEPTH` (`types.ts`, currently 5, root group = depth 0) — read by the UI's "Add group" gate, `parse.ts` and the round-trip property-test generator alike; everything else about the builder is unchanged from #176 (compile-only closed Lucene subset over the stored query string, no stored rule AST, field vocabulary from `GET /api/v1/search/fields`). | 2026-07-25 | [link](../decisions.md#2026-07-25--rule-builder-group-nesting-is-bounded-arbitrary-depth-max_group_depth-not-one-level-436) |
|
||||
| `spa.schedules-editor-draft-save` | The schedules SPA editor mutates a local draft and flushes one explicit Save (`PUT /api/schedules/{id}/items`) instead of instant-persisting each action; Copy deep-copies all source references (fixing a Blazor omission); the shuffled-schedule GET's `EnforceProperties` lossy normalization is preserved and mirrored in the SPA's option lists. | 2026-07-11 | [link](../decisions.md#2026-07-11--schedules-spa-editor-draftexplicit-save-over-instant-persist-copy-includes-multismartrerun-shuffled-get-normalization-preserved) |
|
||||
| `spa.sidebar-collapsible-accordions` | The shell sidebar's collapse + nav-group-accordion state persists under two hyphenated `ctv-sidebar-*` localStorage keys (matching the repo's `ctv-` convention, not the prototype's dotted names); labeled groups default-collapsed. | 2026-07-18 | [link](../decisions.md#2026-07-18--collapsible-sidebar--nav-group-accordions-two-ctv-sidebar--localstorage-keys-labeled-groups-default-collapsed-396) |
|
||||
| `spa.spa-rebuild-decision` | The UI is a full React SPA (ChicoryTV) rebuild over the REST API, not a Blazor Server reskin. | 2026-06 | [link](../decisions.md#2026-06--ui-rebuild-is-a-react-spa-chicorytv-on-the-rest-api-not-a-blazor-reskin) |
|
||||
| `spa.templates-editor-table` | The SPA templates editor renders day/block assignment as a table, not Blazor's drag-and-drop calendar grid — an accepted, deliberate parity deviation. | 2026-07 | [link](../decisions.md#2026-07--templates-editor-in-the-spa-is-a-table-not-blazors-drag-calendar) |
|
||||
| `spa.topbar-primary-action` | The TopBar's primary-action "+" button renders only when the active route declares a non-empty `primaryAction`, is wired (via a shared `usePrimaryAction` hook) only on single-unambiguous-create-flow list screens, and is dropped everywhere else rather than left as a dead/no-op button. | 2026-07-12 | [link](../decisions.md#2026-07-12--topbar-primary-action-button-wire-creates-drop-the-rest-238) |
|
||||
| `spa.yaml-validator-textarea` | The YAML playout validator takes pasted YAML via a `<textarea>`, not a server-side file path, since the SPA has no filesystem access. | 2026-07-09 | [link](../decisions.md#2026-07-09--yaml-playout-validator-paste-textarea-instead-of-a-server-file-path) |
|
||||
| `startup.parallel-orientation` | A fresh session runs two concurrent tracks at startup — Orientation (`AGENTS.md`/`CLAUDE.md` → `docs/README.md` task-signal map → the active decisions catalog `docs/decisions/README.md`) and, only when no issue is named, Selection (`scripts/select-queue.sh N`, deterministic live-Gitea ranking). A named issue skips Selection entirely. ersatztv#237, the closed pickup tracker this replaces, is reduced to a single archival breadcrumb and MUST NOT be read for live state. | 2026-07-21 | [link](../decisions.md#2026-07-21--parallel-orientation--selection-is-the-startup-protocol-237-retired-520) |
|
||||
| `testing.e2e-cleanup-scope-by-pid` | An E2E harness or agent may only kill processes whose PIDs it captured at launch — capture the PID; whoever owns the lifecycle releases it from a `trap ... EXIT INT TERM`. Never `pkill -f "dotnet ErsatzTV.dll"` (or any pattern that can match a process this run did not start). A foreign listener is reported, not reaped. | 2026-07-25 | [link](workflow-process.md#2026-07-25--e2e-cleanup-kills-only-the-pids-it-started-never-a-pkill--f-pattern-586) |
|
||||
| `testing.e2e-local-fresh-config-dir` | Always point `scripts/e2e-local.sh` at a fresh config dir — leftover channels/schedules/DB rows bleed state between runs and corrupt assertions. (The *readiness-probe hang* this record was originally written about was fixed in #533; the fresh-dir rule stands on state-bleed grounds alone.) | 2026-07-21 | [link](workflow-process.md#2026-07-21--run-scriptse2e-localsh-against-a-fresh-config-dir-a-reused-one-hangs-the-readiness-probe-542) |
|
||||
| `testing.live-e2e-prepush-timing` | Run live-E2E via `scripts/e2e-local.sh` before pushing a write-path or UI change, and exercise download endpoints with curl, never a browser tab. | 2026-07-21 | [link](workflow-process.md#2026-07-21--live-e2e-runs-before-the-push-and-downloads-are-curled-not-browsed-542) |
|
||||
| `testing.playwright-mcp-download-and-recovery` | In Playwright-MCP E2E, fetch file-download endpoints with curl — never a browser tab or `window.open` — and if browser tools stall repeatedly, `pkill -f ms-playwright-mcp` and drive a fresh session. | 2026-07-21 | [link](workflow-process.md#2026-07-21--playwright-mcp-curl-download-endpoints-never-open-a-tab-or-windowopen-542) |
|
||||
| `testing.scripted-playout-golden-deferred` | The `PlayoutBuildGoldenTests` in-memory golden net covers Sequential (YAML) as of #381. Scripted's *end-to-end pipeline* is excluded — `ScriptedPlayoutBuilder` runs a user-authored external program that drives the engine over HTTP loopback, which the in-memory harness can't pin — so that full-pipeline (integration) harness is deferred to #563. But the scheduling *behavior* those scripts drive lives entirely in the in-process `SchedulingEngine` (the `ScriptedScheduleController` is a 1:1 pass-through to it), which IS directly unit/golden-testable; the earlier "Scripted is un-golden-able by construction" framing overstated the constraint by conflating transport with engine. #395 extracts that shared switch to `ContentEnumeratorBuilder` and adds a direct regression net (`ContentEnumeratorBuilderTests`) over it. | 2026-07-22 | [link](../decisions.md#2026-07-22--sequential-yaml-playout-gets-a-golden-scripted-is-excluded-from-the-golden-net-by-construction-381) |
|
||||
| `testing.troubleshoot-path-cannot-test-branding` | Verify logo/watermark/bug changes through a real channel playout — a green troubleshoot run proves nothing about branding. | 2026-07-21 | [link](workflow-process.md#2026-07-21--channel-branding-is-not-testable-through-the-troubleshooting-playback-api-542) |
|
||||
|
||||
## Review due
|
||||
|
||||
@@ -201,6 +180,6 @@ record. Sorted soonest-first.
|
||||
|
||||
| Stale after | Key | Record |
|
||||
| --- | --- | --- |
|
||||
| 2027-01-15 | `ci.runner-placement` | [link](records/ci/runner-placement.md) |
|
||||
| 2027-02-15 | `ci.infra-shaped-red-under-load` | [link](records/ci/infra-shaped-red-under-load.md) |
|
||||
| 2027-03-15 | `ci.peak-anon-measurement` | [link](records/ci/peak-anon-measurement.md) |
|
||||
| 2027-01-15 | `ci.runner-placement` | [link](../decisions.md#2026-07-17--no-persistent-compiler-servers-in-ci-every-services-container-gets-an-explicit-cap-390s-small-lane-move-reversed-406) |
|
||||
| 2027-02-15 | `ci.infra-shaped-red-under-load` | [link](workflow-process.md#2026-07-21--an-infra-shaped-red-under-host-load-is-not-a-code-failure-542) |
|
||||
| 2027-03-15 | `ci.peak-anon-measurement` | [link](../decisions.md#2026-07-19--ci-test-job-reports-a-sampled-true-peak-anon-not-cache-inflated-memorypeak-412) |
|
||||
|
||||
@@ -13,22 +13,515 @@ contract-freeze), #206 (Blazor-removal auth posture), #283 (artwork content-type
|
||||
|
||||
## Contents
|
||||
|
||||
- [2026-07-11 — Blazor removal auth posture: no new exposure beyond phase (a); real auth deferred to #197 (#206)](#2026-07-11--blazor-removal-auth-posture-no-new-exposure-beyond-phase-a-real-auth-deferred-to-197-206)
|
||||
- [2026-07-11 — Baseline security response headers + Phase-0 API hardening (#197, PR #279)](#2026-07-11--baseline-security-response-headers--phase-0-api-hardening-197-pr-279)
|
||||
- [2026-07-12 — Artwork content-type is sniffed, never reflected (#283, S4/S9 stored XSS)](#2026-07-12--artwork-content-type-is-sniffed-never-reflected-283-s4s9-stored-xss)
|
||||
- [2026-07-12 — Fail-closed API auth + sensitive-read tier + CORS/ForwardedHeaders lockdown (#197 Bundle A, PR #292)](#2026-07-12--fail-closed-api-auth--sensitive-read-tier--corsforwardedheaders-lockdown-197-bundle-a-pr-292)
|
||||
- [2026-07-12 (#197 Bundle C — contract-freeze honesty)](#2026-07-12-197-bundle-c--contract-freeze-honesty)
|
||||
- [2026-07-12 — Browser SPA session auth: `/api` accepts session OR machine key (#295 PR1, server-only)](#2026-07-12--browser-spa-session-auth-api-accepts-session-or-machine-key-295-pr1-server-only)
|
||||
- [2026-07-12 — #295 PR2: SPA session cutover + #301 side-effecting-GET POST-ification](#2026-07-12--295-pr2-spa-session-cutover--301-side-effecting-get-post-ification)
|
||||
- [2026-07-12 — Enforcing CSP + Permissions-Policy on the host (#319, ZAP baseline)](#2026-07-12--enforcing-csp--permissions-policy-on-the-host-319-zap-baseline)
|
||||
- [2026-07-13 — Cross-origin resource policy: `same-origin` on every response (#330)](#2026-07-13--cross-origin-resource-policy-same-origin-on-every-response-330)
|
||||
- [2026-07-22 — Short-lived browser IPTV token so the SPA reaches `/iptv/*` under JWT auth (#552)](#2026-07-22--short-lived-browser-iptv-token-so-the-spa-reaches-iptv-under-jwt-auth-552)
|
||||
- [2026-07-23 — access_token transport hardening: percent-encode in M3U/HLS, redact from logs, no-store on tokened manifests (#421, #559)](#2026-07-23--access_token-transport-hardening-percent-encode-in-m3uhls-redact-from-logs-no-store-on-tokened-manifests-421-559)
|
||||
|
||||
---
|
||||
|
||||
## Records formerly in this file
|
||||
## 2026-07-11 — Blazor removal auth posture: no new exposure beyond phase (a); real auth deferred to #197 (#206)
|
||||
`key: security.blazor-removal-auth-posture` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** Removing the Blazor UI's OIDC-challenged surface exposes nothing a user couldn't already reach via the already-open `/app` SPA (open since phase (a)); real SPA/API authentication is deliberately deferred to #197, and the removal PR must preserve `ConditionalIptvAuthorizeFilter`, `ApiKeyAuthorizationFilter`, and `JwtHelper` access_token support.
|
||||
**Signals:** Blazor removal, auth posture sign-off, OIDC attachment point · paths: `ErsatzTV/Startup.cs`, `ErsatzTV/Pages` · issues: #206, #91, #197
|
||||
**Mechanics:** `ErsatzTV/Startup.cs` (Razor Pages/OIDC registration); #91 phase (b) removal PR
|
||||
|
||||
Each record below moved to its own file under `records/` (ersatztv#610); the rationale is
|
||||
unchanged. Resolve by **key** — that is the stable identity. A date-based pointer from
|
||||
another doc or an old issue comment should land here and then follow the link.
|
||||
Sign-off for the #91 phase (b) removal-gate item #206 ("deleting the last challenged Blazor page leaves
|
||||
only the open SPA"). The actual authorization wiring in `ErsatzTV/Startup.cs` + `ErsatzTV/Pages` was
|
||||
enumerated in code (not assumed) before clearing the gate.
|
||||
|
||||
- 2026-07-11 — Baseline security response headers + Phase-0 API hardening (#197, PR #279) — [`security.baseline-response-headers`](records/security/baseline-response-headers.md)
|
||||
- 2026-07-11 — Blazor removal auth posture: no new exposure beyond phase (a); real auth deferred to #197 (#206) — [`security.blazor-removal-auth-posture`](records/security/blazor-removal-auth-posture.md)
|
||||
- 2026-07-12 (#197 Bundle C — contract-freeze honesty) — [`security.contract-freeze-honesty`](records/security/contract-freeze-honesty.md)
|
||||
- 2026-07-12 — #295 PR2: SPA session cutover + #301 side-effecting-GET POST-ification — [`security.session-cutover-postify`](records/security/session-cutover-postify.md)
|
||||
- 2026-07-12 — Artwork content-type is sniffed, never reflected (#283, S4/S9 stored XSS) — [`security.artwork-content-type-sniff`](records/security/artwork-content-type-sniff.md)
|
||||
- 2026-07-12 — Browser SPA session auth: `/api` accepts session OR machine key (#295 PR1, server-only) — [`security.session-auth-dual-credential`](records/security/session-auth-dual-credential.md)
|
||||
- 2026-07-12 — Enforcing CSP + Permissions-Policy on the host (#319, ZAP baseline) — [`security.csp-permissions-policy`](records/security/csp-permissions-policy.md)
|
||||
- 2026-07-12 — Fail-closed API auth + sensitive-read tier + CORS/ForwardedHeaders lockdown (#197 Bundle A, PR #292) — [`security.fail-closed-api-auth`](records/security/fail-closed-api-auth.md)
|
||||
- 2026-07-13 — Cross-origin resource policy: `same-origin` on every response (#330) — [`security.corp-same-origin`](records/security/corp-same-origin.md)
|
||||
- 2026-07-22 — Short-lived browser IPTV token so the SPA reaches `/iptv/*` under JWT auth (#552) — [`security.iptv-browser-token`](records/security/iptv-browser-token.md)
|
||||
- 2026-07-23 — access_token transport hardening: percent-encode in M3U/HLS, redact from logs, no-store on tokened manifests (#421, #559) — [`security.iptv-access-token-transport`](records/security/iptv-access-token-transport.md)
|
||||
**What is gated today**
|
||||
- **OIDC** (`OidcHelper.IsEnabled` — active only when `Authority`/`ClientId`/`ClientSecret` are configured):
|
||||
`AddAuthentication` (cookie default, `oidc` challenge) + `AddAuthorization` `DefaultPolicy =
|
||||
RequireAuthenticatedUser` + **`AddRazorPages(… AuthorizeFolder("/"))`** (Startup.cs:379-385) +
|
||||
`blazor.UseAuthentication()/UseAuthorization()` inside the Blazor `MapWhen` branch (Startup.cs:764-770).
|
||||
`AuthorizeFolder("/")` gates **Razor Pages only**, and the sole user-facing Razor Page is
|
||||
`Pages/_Host.cshtml` — the Blazor Server host (the other `.cshtml`, `Shared/_Favicons.cshtml`, is a
|
||||
cosmetic partial). **So the OIDC challenge protects exactly the Blazor UI and nothing else.**
|
||||
- **`/app` (SPA)** is served by its own `MapWhen(path=/app)` static-file branch (Startup.cs:701-714) with
|
||||
**no authentication/authorization middleware** — open since phase (a) (`/`→`/app`, PR #148).
|
||||
- **`/api/*` controllers** carry no `[Authorize]` (verified: zero attributes in `Controllers/`); the
|
||||
Razor-Pages `AuthorizeFolder`/`DefaultPolicy` never reach them. Their only optional gate is the
|
||||
per-endpoint `ApiKeyAuthorizationFilter` (API-key on mutating JSON endpoints), independent of OIDC/Blazor.
|
||||
- **`/iptv/*`** is gated by `ConditionalIptvAuthorizeFilter` (JWT `JwtOnlyScheme`, active only when
|
||||
`JwtHelper.IsEnabled`) in its own `MapWhen` branch (Startup.cs:797-803) — independent of Blazor.
|
||||
|
||||
**Posture after Blazor removal.** Removing `Pages/_Host.cshtml`, `AddRazorPages`/`AuthorizeFolder("/")`,
|
||||
`blazor.UseAuthentication/UseAuthorization`, `MapBlazorHub`, and `MapFallbackToPage("/_Host")` deletes the
|
||||
OIDC challenge's **only attachment point** — no user-facing surface remains challenged. **No capability is
|
||||
lost:** every Blazor-served capability already has an open SPA equivalent (the #91 parity effort), and the
|
||||
SPA was already the unauthenticated path since phase (a), so removal exposes nothing a user could not already
|
||||
reach via `/app`.
|
||||
|
||||
**The one honest caveat (not a regression introduced by removal):** an OIDC-configured operator's *Blazor*
|
||||
admin UI sits behind a login today; after removal there is no login-gated admin UI at all (the SPA admin UI
|
||||
is open). That exposure delta already happened at **phase (a)** (the open SPA became the default admin
|
||||
surface); removal only deletes the now-redundant challenged duplicate. Designing real SPA/API authentication
|
||||
is deliberately deferred to **#197** (cold API security review — a HARD GATE before any remote exposure).
|
||||
|
||||
**Removal-PR must-not-break (independent gates that survive):** `ConditionalIptvAuthorizeFilter` (`/iptv/*`
|
||||
JWT), `ApiKeyAuthorizationFilter` (mutating `/api/*`), and `JwtHelper` access_token query support. **Leave
|
||||
the OIDC service registrations in place** (conditional on config, inert once no Razor Page consumes them) —
|
||||
ripping OIDC out is a #197 decision, not a removal-PR one. The removal PR removes only the Blazor-attached
|
||||
pieces above; `MapControllers()` + `/docs` (Scalar), currently co-hosted in the Blazor `MapWhen` branch, must
|
||||
survive the surgical reduction.
|
||||
|
||||
## 2026-07-11 — Baseline security response headers + Phase-0 API hardening (#197, PR #279)
|
||||
`key: security.baseline-response-headers` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** `SecurityHeadersMiddleware`, registered first in the pipeline, sets `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, and `Referrer-Policy: strict-origin-when-cross-origin` on every response (CSP/HSTS deliberately deferred); API-key comparison is constant-time and playout pagination is clamped.
|
||||
**Signals:** security headers, nosniff, Phase-0 hardening, constant-time comparison · paths: `ErsatzTV/Middleware/SecurityHeadersMiddleware` · issues: #197, #279, #283
|
||||
**Mechanics:** `ErsatzTV/Middleware/SecurityHeadersMiddleware`
|
||||
|
||||
Phase-0 of the #197 remediation — the posture-**independent** safe subset, shipped ahead of the
|
||||
fail-closed/CORS/versioning posture work tracked in #280–#289.
|
||||
|
||||
- **Baseline security headers on every response.** New `ErsatzTV/Middleware/SecurityHeadersMiddleware`,
|
||||
registered **first** in the pipeline (before the `/iptv` `MapWhen` branch and `UseCors`), so it covers
|
||||
`/api`, `/iptv`, `/artwork`, static, the SPA fallback, and filter-produced 4xx alike — which is why it's
|
||||
middleware, not an MVC filter. It sets `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, and
|
||||
`Referrer-Policy: strict-origin-when-cross-origin`. `nosniff` is the standing backstop for the artwork
|
||||
content-type MIME-sniffing risk (#283). **CSP and HSTS are deliberately NOT included here**: CSP must be
|
||||
validated against the ChicoryTV SPA's inline assets, and HSTS is a proxy/TLS-termination decision — both
|
||||
belong to the #197 posture design (#284/roadmap), not this baseline. Headers are set eagerly (not via
|
||||
`Response.OnStarting`); safe today because the pipeline has no `UseExceptionHandler`/`UseStatusCodePages`
|
||||
that would `Response.Clear()` — switch to `OnStarting` if one is ever added.
|
||||
- **Constant-time API-key comparison.** `ApiKeyAuthorizationFilter` compares `X-Api-Key` with
|
||||
`CryptographicOperations.FixedTimeEquals` (over UTF-8 bytes) instead of ordinal `string.Equals`, removing
|
||||
the response-timing oracle on the write key. Accept/reject behavior is otherwise identical.
|
||||
- **Playout pagination clamped.** `GET /api/playouts` and `GET /api/playouts/{id}/items` now clamp
|
||||
`Math.Clamp(pageSize, 1, 100)` + `Math.Max(0, pageNum)` before the query — applying the api-conventions §1
|
||||
clamp convention the other paged endpoints already follow (these two were passing the raw client value
|
||||
straight to EF `Take()`).
|
||||
|
||||
The larger #197 posture (fail-closed writes, sensitive-read auth tier, CORS lockdown, `/api/v1` versioning,
|
||||
the OpenAPI security scheme) is decomposed into #280–#289 with the phased roadmap on #197; those PRs will
|
||||
append their own decisions here as they land.
|
||||
|
||||
## 2026-07-12 — Artwork content-type is sniffed, never reflected (#283, S4/S9 stored XSS)
|
||||
`key: security.artwork-content-type-sniff` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** Artwork content type is always derived from the stored bytes (never the client-declared value or a `?contentType=` query param) at both upload and serve, clamped to an image allow-list, closing the unauthenticated stored-XSS chain; Kestrel `MaxRequestBodySize` bounds upload DoS.
|
||||
**Signals:** stored XSS, content-type sniffing, artwork upload/serve · paths: `ErsatzTV.Core/Images/ImageContentTypes.DetectContentType`, `GetCachedImagePathHandler` · issues: #283, S4, S9
|
||||
**Mechanics:** `ErsatzTV.Core/Images/ImageContentTypes`
|
||||
|
||||
The artwork upload/serve path trusted client-supplied content types at both ends, giving a stored-XSS
|
||||
chain on **unauthenticated** GET sinks: upload `<script>` bytes declared `image/png` →
|
||||
`GET /iptv/logos/{hash}?contentType=text/html` served them as HTML in the ErsatzTV origin. The #279
|
||||
`nosniff` header is not a fix here — the server was *explicitly declaring* `text/html`, which the browser
|
||||
honors regardless of `nosniff`. The trust was the bug; the fix removes it at both ends.
|
||||
|
||||
- **Upload derives the content type from the bytes, never the declared value.** `UploadArtworkHandler`
|
||||
buffers the (size-bounded) upload and calls `ErsatzTV.Core/Images/ImageContentTypes.DetectContentType`,
|
||||
which uses SkiaSharp's `SKCodec` to identify the format from the image header only — pixels are **not**
|
||||
decoded, so this can't be turned into a decompression-bomb vector. A payload that isn't one of the
|
||||
accepted raster formats (png/jpeg/gif/webp) is rejected 422; the declared `Content-Type` is no longer
|
||||
read at all (the field was dropped from the `UploadArtwork` command).
|
||||
- **Serve sniffs the stored file; the `?contentType=` reflection is gone.** `GetCachedImagePath` no longer
|
||||
carries a `ContentType`, and `GetImage` (`/iptv/logos`) / `GetWatermark` (`/artwork/watermarks`) dropped
|
||||
their `[FromQuery] contentType` binding. `GetCachedImagePathHandler` always derives the MIME type from the
|
||||
file (`MimeTypes.GetMimeTypeFromFile`) and **clamps it to the image allow-list** (`ImageContentTypes.IsAccepted`),
|
||||
serving `application/octet-stream` for anything else — so a file whose bytes are not an accepted image (a
|
||||
legacy cache entry poisoned before the upload sniff landed, or a hypothetical polyglot) is a non-renderable
|
||||
download, never HTML/script. The removal is **structural** — there is no longer any request path that lets a
|
||||
client choose the served `Content-Type`. `ArtworkContentTypeModel.UrlWithContentType` now returns the bare path, and the SPA
|
||||
watermark/logo previews no longer append the query.
|
||||
- **Defense-in-depth on the persisted JSON DTOs.** The `{path, contentType}` bodies (channel logo, watermark)
|
||||
run their content type through `ArtworkContentTypeModel.Sanitized()`, which blanks anything outside the
|
||||
image allow-list before it is stored — so a stale/hostile value can't be reflected by any future code path
|
||||
even though the serve route already ignores it.
|
||||
- **S9 upload-size DoS.** Kestrel `Limits.MaxRequestBodySize` is now set from `ETV_MAXIMUM_UPLOAD_MB`, so an
|
||||
oversized body is rejected as it is read rather than only after the controller's post-binding `file.Length`
|
||||
check (kept as the friendly-error backstop). This is a global bound; the app has no other large inbound
|
||||
body (streaming is outbound GET).
|
||||
|
||||
`ImageContentTypes` is the single source of truth for the accepted image types (the allow-list previously
|
||||
duplicated in `UploadArtworkHandler`). Both serve sinks are `[ApiExplorerSettings(IgnoreApi = true)]`, so
|
||||
none of this changes the OpenAPI document.
|
||||
|
||||
## 2026-07-12 — Fail-closed API auth + sensitive-read tier + CORS/ForwardedHeaders lockdown (#197 Bundle A, PR #292)
|
||||
`key: security.fail-closed-api-auth` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** Every mutating `/api` request requires `X-Api-Key` (no open mode); reads are gated by `Api:RequireKeyForReads` (default true) OR `[RequiresApiKey]` on sensitive controllers; CORS is an exact-origin allowlist (`ApiCors`); `ForwardedHeaders` trust stays configurable but defaults to trust-all-with-warning.
|
||||
**Signals:** fail-closed auth, sensitive-read tier, CORS allowlist, single API key · paths: `IApiKeyProvider`, `ErsatzTV/Services/ApiKeyProvider.cs`, `FileSystemLayout.ApiKeyPath` · issues: #197, #280, #281, #282, #284, #285
|
||||
**Mechanics:** `docs/api-conventions.md` §5; `ApiControllerSecurityTests`
|
||||
|
||||
Phase-1 of the #197 remediation — the auth posture that must land before any remote exposure.
|
||||
Owner decisions (confirmed this session): **single API key** (no read/write split), and
|
||||
**`Api:RequireKeyForReads` defaults `true`** (the whole `/api` surface requires the key). This does
|
||||
**not** affect Jellyfin/streaming: `/iptv/*` (playlist/guide/streams/logos) and `/artwork/*` are outside
|
||||
the filter's `/api` scope and keep their own optional access-token; only the management API the SPA talks
|
||||
to is gated.
|
||||
|
||||
- **Fail-closed writes (#280, S1).** The empty-key "open" branch is deleted; there is no open mode. New
|
||||
`IApiKeyProvider` (`ErsatzTV/Services/ApiKeyProvider.cs`, singleton, resolved once at startup) yields a
|
||||
never-empty key: `Api:WriteKey` if set, else a key persisted at `FileSystemLayout.ApiKeyPath`
|
||||
(`/config/api.key`, `0600`, path logged not value), else a generated 256-bit hex key. Every mutating
|
||||
`/api` request now requires `X-Api-Key`.
|
||||
- **Sensitive-read tier (#282, S3/S5).** Reads are gated by `Api:RequireKeyForReads` (default true) OR a
|
||||
new `[RequiresApiKey]` marker (mirror of `[SkipApiKeyAuthorization]`) applied to
|
||||
`Troubleshoot`/`Logs`/`Settings`/`Maintenance`, so that tier stays gated even if an operator opts reads
|
||||
open. `OPTIONS` preflight is exempt (CORS middleware owns it). `ApiControllerSecurityTests` asserts the
|
||||
tier reflectively.
|
||||
- **Delete dead non-`/api` mutation surfaces (#281, S2).** `SortController`
|
||||
(`POST media/collections/{id}/items`, dead Blazor SortableJS residue — the SPA uses
|
||||
`PUT /api/collections/{id}/custom-order`) and `AccountController` (`POST account/logout`, dead OIDC)
|
||||
bypassed the key because they sat outside `/api`. Removed rather than guarded.
|
||||
- **CORS opt-in (#284, S6).** `AllowAnyOrigin/Method/Header` is replaced by the `ApiCors` policy: an
|
||||
exact-origin allowlist from `Api:CorsAllowedOrigins` (semicolon list) that permits `X-Api-Key`/`If-Match`
|
||||
and exposes `ETag`; with no origins configured there is no cross-origin access (the SPA is same-origin).
|
||||
- **ForwardedHeaders trust + scanner loopback (#285, S7/S10).** `GET /api/maintenance/gc` → `POST`
|
||||
(crawler-triggerable GC; spec regenerated). `ForwardedHeaders` trust is configurable via
|
||||
`ForwardedHeaders:KnownProxies`/`KnownNetworks` — **unconfigured preserves the current trust-all
|
||||
behavior but logs a warning** (flipping the default to loopback-only would break reverse-proxy scheme/host
|
||||
detection and thus M3U/XMLTV absolute URLs — the operator must name their proxy network). `ScannerController`
|
||||
gains `[LocalhostOnly]` (the scanner always calls back over `http://localhost:{UiPort}/api/scan/...`), which
|
||||
is only spoof-resistant once ForwardedHeaders trust is restricted — the two interlock. `search/all-items`
|
||||
DoS-paging is **deferred** (it feeds the SPA "add all" flow and needs coordinated pagination; the unauth
|
||||
exposure is already closed by read-gating).
|
||||
- **SPA (`web/`).** The client sends the stored key (`ctv-api-key`) on **every** method (not just
|
||||
mutations); a new keyless **API Key** screen (`/app/api-key`) lets the user paste the generated key, and a
|
||||
shell-level banner points there on any 401. See spa-conventions §5e. **First-run/upgrade UX:** with reads
|
||||
gated by default, the SPA shows no data until the key (from `/config/api.key`) is entered — an intended
|
||||
consequence of the strict default.
|
||||
|
||||
Phase-2 (contract freeze) — the declarative OpenAPI security scheme, global 401 docs, and `/api/v1`
|
||||
versioning — remains #286/#287/#288. Phase-3 follow-ups: #265, #269, #172 remainder, `search/all-items`
|
||||
paging, per-key rate limiting.
|
||||
|
||||
## 2026-07-12 (#197 Bundle C — contract-freeze honesty)
|
||||
`key: security.contract-freeze-honesty` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** The OpenAPI doc's declared security/401 scheme is generated from the same `ApiKeyAuthorizationFilter.EndpointRequiresKey` predicate the runtime enforces (so declared auth can't drift from enforced auth), every `/api/*` action returns a ResponseModel (no raw Application VMs), and Channel REST resources are keyed by immutable `Id`, never mutable `Number`.
|
||||
**Signals:** OpenAPI contract honesty, ResponseModel wrapping, Id vs Number key · paths: `OpenApiContractHonestyTests`, `ErsatzTV.Core/Api` · issues: #197, #287, #288
|
||||
**Mechanics:** `OpenApiContractHonestyTests`; `docs/api-conventions.md`
|
||||
|
||||
**#287 — OpenAPI contract honesty by construction.** The "v1" document now emits the `ApiKey` security
|
||||
scheme plus per-operation `security`/`401` derived from the *same*
|
||||
`ApiKeyAuthorizationFilter.EndpointRequiresKey` predicate the runtime filter enforces, so declared auth
|
||||
can never drift from enforced auth. Every operation also gets a synthesized stable `operationId` (the
|
||||
framework only assigned one when `Name=` was set — ~90 were missing), and body/param-binding operations
|
||||
get the documented `400 ValidationProblemDetails` they actually return. `DayOfWeek` is now a string enum
|
||||
in the schema (added to `Startup.UseStringEnumSchemas`), removing the SPA's `WithDayNames` wart. Pinned by
|
||||
in-process document generation in tests (`OpenApiContractHonestyTests`) rather than the committed `v1.json`.
|
||||
|
||||
**#288 — Wrap the last raw ViewModels; reverse the §7a "intentional `version` leak."** Minted
|
||||
`MediaCollectionResponseModel`, `ProgramScheduleResponseModel`, and `ChannelDetailResponseModel` (all
|
||||
`#nullable enable`) and routed `CollectionController` / `ScheduleController` / `SmartCollectionController` /
|
||||
`ResolutionController.GetResolutionByName` / the channel detail GET+writes through ResponseModels, so no
|
||||
`/api/*` action returns an Application VM. This reverses the earlier §7a judgment that a ResponseModel
|
||||
"purely to hide one field was disproportionate": `Version` is now header-only (ETag) on every aggregate
|
||||
body — confirmed safe by grepping `web/src` (the SPA reads `version` from the ETag header, never the
|
||||
response body). `ChannelDetailResponseModel` is the *full editable* field set the channel editor needs
|
||||
(distinct from the lean list `ChannelResponseModel`; drops only the derived `webEncodedName`). Also flipped
|
||||
`#nullable enable` onto the remaining 24 lagging `ErsatzTV.Core/Api/` files for schema honesty, and added
|
||||
`pageNum` paging to `GET /api/search`.
|
||||
|
||||
**Channel REST resources are keyed by database `Id`, never by `Number`.** `Channel.Number` is user-mutable
|
||||
(editable on update, bulk-renumbered via `/api/channels/bulk/renumber`, transiently invalid mid-renumber),
|
||||
so the immutable int PK is the canonical key for all `/api/channels/*` single-item routes, sub-resources
|
||||
(including `playout/reset`, re-keyed from `{channelNumber}` to `{id:int}` in Bundle C), and `Location`
|
||||
headers. `Number` remains the identity on broadcast surfaces only (IPTV/M3U/XMLTV), a separate contract. A
|
||||
number-based lookup endpoint may be added additively later; `UniqueId` (Guid) stays out of the REST
|
||||
contract absent a federation requirement.
|
||||
|
||||
## 2026-07-12 — Browser SPA session auth: `/api` accepts session OR machine key (#295 PR1, server-only)
|
||||
`key: security.session-auth-dual-credential` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** `ApiAuthorizationFilter` accepts a request when a valid `X-Api-Key` matches OR the principal is an authenticated session (cookie `ctv-session`, `HttpOnly`/`SameSite=Lax`); session-authenticated mutations require the presence-only `X-CSRF` header or are rejected 403. This narrows the OIDC-inert sub-claim of `security.blazor-removal-auth-posture` (#206) — the rest of that record's auth-surface enumeration still holds.
|
||||
**Signals:** session auth, dual credential, CSRF, OIDC revival, local admin · paths: `ApiAuthorizationFilter`, `ErsatzTV/Startup.cs` · issues: #295, #206, #197
|
||||
**Mechanics:** `docs/spa-conventions.md` §5e (PR2); `RootWriterForceVersionTests`-adjacent auth tests
|
||||
|
||||
Implements the ratified #295 design (Fable [PLAN-MODE] pass, issue comment 9548). Supersedes the #206
|
||||
"OIDC wiring stays inert until #197" note: the retained OIDC service registration is now **revived**, and a
|
||||
cookie session becomes a first-class `/api` credential alongside the machine `X-Api-Key`. **PR1 is
|
||||
server-only and backward compatible** — the SPA keeps sending its stored key; the SPA login flow, the
|
||||
`ApiKeyScreen`→machine-key repurpose, and `spa-conventions §5e` land in **PR2**.
|
||||
|
||||
**One gate, evolved (not `[Authorize]`-per-controller).** `ApiKeyAuthorizationFilter` → `ApiAuthorizationFilter`,
|
||||
same fail-closed-by-omission logic (a forgotten `[Authorize]` fails *open* — the #280 failure mode — so the
|
||||
global filter stays the gate). It now accepts a request when a valid `X-Api-Key` matches **OR** the principal
|
||||
is an authenticated session; the "does this endpoint need auth?" decision is still the single shared
|
||||
`EndpointRequiresKey(...)` predicate (also drives OpenAPI, so the spec can't drift). Attributes renamed to
|
||||
match the widened meaning: `[RequiresApiKey]`→`[RequiresAuthentication]`, `[SkipApiKeyAuthorization]`→
|
||||
`[SkipApiAuthorization]`. `IApiKeyProvider`, the `X-Api-Key` header, and `Api:WriteKey`/`Api:RequireKeyForReads`
|
||||
are unchanged — **machine/key behavior is byte-identical** (verified: no OpenAPI drift, existing filter tests
|
||||
still green).
|
||||
|
||||
**CSRF (session only).** The machine key is CSRF-immune (a browser can't set a custom header cross-origin
|
||||
without a credentialed CORS grant we never issue). A cookie session is not: a session-authenticated **mutation**
|
||||
must carry the `X-CSRF` header (presence-only — a custom header forces a CORS preflight a cross-site page can't
|
||||
satisfy) or is rejected **403**. Reinforced by `SameSite=Lax` + CORS without `AllowCredentials` (cross-origin
|
||||
cookie auth is impossible by design). No antiforgery-token machinery.
|
||||
|
||||
**Cookie `ctv-session`.** Always registered (local login works with no IdP); OIDC handler added only when
|
||||
`OIDC:*` is configured. `HttpOnly`, `SameSite=Lax`, `SecurePolicy=SameAsRequest` (so a plain-HTTP LAN isn't
|
||||
bricked), 14-day sliding. `/api` XHR gets **401/403, not a redirect** (`OnRedirectToLogin`/`AccessDenied`).
|
||||
The `UseAuthentication`/`UseAuthorization` middleware — deleted with Blazor in #91b — is **revived in the
|
||||
`legacy` `MapWhen` branch only** (hosts `/api` + OIDC `/callback` + `/docs`; `/iptv` and `/app` untouched).
|
||||
|
||||
**Local store = `ConfigElement` rows, single admin, NO migration** (owner ruling F2):
|
||||
`AuthLocalAdminUsername`, `AuthLocalAdminPasswordHash` (ASP.NET `PasswordHasher`, PBKDF2, via
|
||||
`Microsoft.Extensions.Identity.Core`), `AuthSecurityStamp`. A password change rotates the stamp; the cookie
|
||||
`OnValidatePrincipal` (`CookieSecurityStampValidator`) compares the claim to the stored stamp and rejects a
|
||||
stale session (revocation). OIDC sessions carry an `etv:auth_method=oidc` claim and skip the stamp check
|
||||
(governed by the IdP).
|
||||
|
||||
**Fail-closed out of the box + recovery.** An unconfigured instance keeps `/api` gated (the key still works);
|
||||
first-run is a **setup-claim** (`POST /api/auth/setup`, first-claim-wins, only valid while unconfigured —
|
||||
owner ruling F1). Recovery without the browser: `Auth:LocalAdmin:Password` env seed (`LocalAdminSeedService`,
|
||||
overwrites + rotates the stamp on startup) or the machine key. Login hardening: per-IP rate limit
|
||||
(`[EnableRateLimiting("auth")]`, 10 / 5 min) on login/setup/password, dummy-hash verify on unknown/unconfigured
|
||||
user (no enumeration).
|
||||
|
||||
**Authelia = app-owned OIDC session; never trust proxy identity headers** (owner ruling F3): the container is
|
||||
LAN-reachable bypassing the proxy, so `Remote-User`/`Remote-Email` header trust is spoofable. OIDC→Authelia
|
||||
gives SSO without a double login. **`ForwardedHeaders` behaviour is kept unchanged from #285** (trust any peer
|
||||
with a warning; restrict via `KnownProxies`/`:KnownNetworks`). A stricter "ignore `X-Forwarded-*` unless a proxy
|
||||
is configured" default was implemented and then **reverted** after review (cold fork M1): the forwarded
|
||||
scheme/host feed `/iptv` M3U/XMLTV/HLS absolute-URL generation (`Request.Scheme` in `GetChannelGuideHandler`/
|
||||
`IptvController`), so ignoring them would regress stream URLs to `http`/internal-host for a proxied deployment
|
||||
that hasn't set `KnownProxies`. **Deployment coordination:** operators behind a proxy should set
|
||||
`ForwardedHeaders:KnownProxies`/`:KnownNetworks` — it gives the login rate limiter an unspoofable client IP and
|
||||
marks the session cookie `Secure` behind TLS. The residual (a direct LAN peer can spoof `X-Forwarded-For` to
|
||||
evade the per-IP login limit when unrestricted) is accepted defense-in-depth loss, mitigated by PBKDF2 +
|
||||
no-enumeration.
|
||||
|
||||
**Review hardening (fork + independent Codex pass, folded into PR1).** Codex caught concurrency defects the
|
||||
fork missed — folded in: (a) **atomic first-claim-wins** — setup writes the three credential rows in one
|
||||
transaction guarded by the unique `ConfigElement.Key` index (a lost race → `DbUpdateException` → 409), so a
|
||||
concurrent claim can't produce a mixed-state credential; (b) **consistent login snapshot** — login reads the
|
||||
hash + stamp in one query and no longer rehashes-on-verify, so a login racing a password change can't capture a
|
||||
newer stamp than the hash it verified (a concurrent change either fails the old password or leaves the issued
|
||||
cookie carrying the pre-change stamp → revoked next request); (c) **env-seed waits on
|
||||
`SystemStartup.WaitForDatabase`** (the migrator is a `BackgroundService`, so registration order alone didn't
|
||||
guarantee the schema existed) — moved to `Services/RunOnce/`. Also: **logout + password require `X-CSRF`**
|
||||
(the `[SkipApiAuthorization]` auth surface isn't covered by the filter's CSRF check → forced-logout CSRF), and
|
||||
input length caps on username/password. **Logout rotates the security stamp** when called from a local session
|
||||
(E2E-caught: `SignOutAsync` alone only clears the *client* cookie, leaving the stateless encrypted ticket
|
||||
replayable server-side) — so signing out actually ends the session server-side; for the single admin this
|
||||
revokes all local sessions ("log out everywhere"). Gated on an authenticated session so an unauthenticated
|
||||
caller can't force-revoke the admin. **Deferred with a tracked gate:** side-effecting `[RequiresAuthentication]`
|
||||
GETs (troubleshoot playback/archive) aren't CSRF-covered — **#301**, gating PR2 (latent in PR1: the SPA still
|
||||
uses the machine key). OIDC-session revocation lever (no local stamp) noted for PR3 operator docs.
|
||||
|
||||
A **fix-commit re-review** (Codex, #242 discipline) then confirmed the above resolved and caught a second round:
|
||||
(a) **HIGH — env-seed vs. setup race**: an attacker could claim admin in the startup window before
|
||||
`LocalAdminSeedService` runs, and the seed's insert would then be swallowed (attacker's credential persists,
|
||||
defeating the env recovery path). Fixed structurally: **the setup-claim endpoint is closed whenever
|
||||
`Auth:LocalAdmin:Password` is configured** — the env seed owns the credential, so there is no claim to race
|
||||
(this also strengthens the setup-claim TOFU posture: an operator on an untrusted network sets the env password
|
||||
and browser setup is disabled). (b) **LOW**: a concurrent setup race-loser now returns **409** (not 422), and
|
||||
`ClaimLocalAdmin`'s `DbUpdateException` catch re-checks existence and **rethrows genuine/transient DB errors**
|
||||
rather than masking them as "already configured". (c) **MEDIUM — accepted**: two *simultaneous* authenticated
|
||||
password changes are a non-serializable lost-update (last-write-wins; the loser's cookie may be immediately
|
||||
revoked). Accepted for a **single-admin** system: it needs two concurrent authenticated sessions both submitting
|
||||
the correct current password at the same instant, and the outcome is self-healing (re-login). Adding EF
|
||||
optimistic concurrency to the credential rows is disproportionate here.
|
||||
|
||||
**OpenAPI = `ApiKey`-only; `/api/auth/*` excluded** (owner ruling F4): the spec's audience is machine/MCP
|
||||
clients, and a browser-interactive cookie login isn't something a generated client drives, so the cookie path
|
||||
is an additional accepted credential the doc needn't express. `AuthController` is `[ApiExplorerSettings(IgnoreApi
|
||||
= true)]`. Verified: no `v1.json`/`v1.d.ts`/`endpoint-index` drift from this PR.
|
||||
|
||||
**Phasing.** PR1 = this (server only, no migration). PR2 = SPA (drop the key header for browser calls + add
|
||||
`X-CSRF`, `AuthContext` + boot gate, login/setup screens, `ApiKeyScreen`→machine-key management, E2E,
|
||||
`spa-conventions §5e`). PR3 = key rotation + operator docs (Authelia client + env reference). Rollout: PR1→PR2
|
||||
same release, then a manual Authelia round-trip checklist before the prod pin bump.
|
||||
|
||||
## 2026-07-12 — #295 PR2: SPA session cutover + #301 side-effecting-GET POST-ification
|
||||
`key: security.session-cutover-postify` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** The browser SPA authenticates cookie-only (no more `X-Api-Key` from `web/`); the machine key is repurposed to external/MCP-only via `GET /api/auth/machine-key`; every side-effecting GET/HEAD under `/api` is converted to POST so the existing CSRF gate covers it (standing rule: never add a side-effecting GET/HEAD under `/api`).
|
||||
**Signals:** SPA cookie-only cutover, CSRF POST-ification, AuthGate boot flow · paths: `web/src/api/client.ts`, `web/src/AuthGate.tsx` · issues: #295, #301, #197
|
||||
**Mechanics:** `docs/spa-conventions.md` §5e; `docs/api-conventions.md` §9; `docs/e2e-local.md`
|
||||
|
||||
PR1 shipped the server side (previous entry): `/api` accepts a session cookie OR the machine `X-Api-Key`, with
|
||||
`X-CSRF` required on session-authenticated mutations. **PR2 is the SPA cutover** — the browser now authenticates
|
||||
with the session only — plus **#301** (a session-cookie CSRF hole in side-effecting GETs).
|
||||
|
||||
**Browser is cookie-only; the machine key is external/MCP-only.** `web/src/api/client.ts` no longer attaches
|
||||
`X-Api-Key`; it relies on the same-origin session cookie and sets `X-Csrf: '1'` on every mutating verb centrally.
|
||||
The former "paste your key" `ApiKeyScreen` is repurposed to **machine-key management**: it reads the server key
|
||||
from the new `GET /api/auth/machine-key` (session-gated; masked with Reveal + Copy) so an operator can hand it to
|
||||
MCP / external REST clients — the browser itself never sends it again. *Why:* one credential per audience (the
|
||||
ratified #295 model); leaving a browser key path alive would keep a CSRF-immune bypass around and defeat the
|
||||
point.
|
||||
|
||||
**Boot gate, not a route** (`web/src/AuthGate.tsx`, wrapping `<App/>` in `main.tsx`): on load it calls the public
|
||||
`GET /api/auth/config` then `GET /api/auth/session` and renders Setup (first-run local-admin claim) / Login
|
||||
(local form + an OIDC "Sign in with SSO" button when `oidcEnabled`) / the app. Login and Setup mint **no URL** —
|
||||
the gate renders them at whatever `/app/*` path was requested, so a deep link survives login for free and **no
|
||||
`blazor-route-parity.md`/`domain-model.md` route rows are added**. It publishes `AuthContext`
|
||||
(`{ username, method, signOut, requireLogin }`); the 401 signal (`notifyUnauthorized`) now drives re-login via a
|
||||
passive shell banner (never yanks a dirty draft — it consults the navigation guard first). Auth flows that expect
|
||||
a 401 inline (login, change-password) pass `suppressUnauthorizedSignal`.
|
||||
|
||||
**#301 — POST-ify, don't gate-the-GET.** A side-effecting GET is a CSRF vector once a `SameSite=Lax` cookie is a
|
||||
normal credential (it rides a cross-site top-level navigation). The three offenders became mutating verbs so the
|
||||
existing filter CSRF gate covers them with zero new machinery: `GET /api/troubleshoot/playback.m3u8` →
|
||||
**`POST /api/troubleshoot/playback/start`** returning `200 { url }` (the open `/iptv` manifest the player then
|
||||
loads — so hls.js/native-HLS needs no header injection, strictly better than X-CSRF-on-GET); the archive and
|
||||
sample GETs → **POST** (SPA downloads them via a fetch-blob helper, never `window.open`). Removing the HEAD
|
||||
variants also fixed a latent bug: a HEAD opened the `DeleteOnClose` stream and destroyed the artifact. Standing
|
||||
rule added to `api-conventions.md §9`: **never add a side-effecting GET/HEAD under `/api`.**
|
||||
|
||||
**Machine-key GET discloses the key to any authenticated session** — deliberate: the session principal is the
|
||||
single admin (local or OIDC), same-origin policy blocks a cross-site page from reading the response body, and it
|
||||
is how the "copy the key for MCP" UX works without a rotation endpoint (rotation is a later PR). **Accepted
|
||||
residual (OIDC logout):** `POST /api/auth/logout` ends the *app* cookie but not the IdP session, so an OIDC user
|
||||
who clicks "Sign out" then "Sign in with SSO" returns without re-entering credentials — a `returnUrl`/RP-initiated
|
||||
logout is a future nicety. Docs: `spa-conventions.md §5e` (SPA seams), `api-conventions.md §9`, `e2e-local.md`
|
||||
(browser setup/login flow). Refs #295 #301 #197.
|
||||
|
||||
## 2026-07-12 — Enforcing CSP + Permissions-Policy on the host (#319, ZAP baseline)
|
||||
`key: security.csp-permissions-policy` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** `SecurityHeadersMiddleware` sends an enforcing (not report-only) `Content-Security-Policy` (no `unsafe-inline`/`unsafe-eval`; the one inline theme-bootstrap script allow-listed by hash) and a deny-all `Permissions-Policy` on the SPA/`/api`/`/artwork`/`/iptv`; `/docs` and `/openapi` keep only the baseline headers, excluded from CSP because Scalar needs inline bootstrap.
|
||||
**Signals:** enforcing CSP, script-src hash allowlist, Permissions-Policy · paths: `ErsatzTV/Middleware/SecurityHeadersMiddleware`, `SecurityHeadersMiddlewareTests` · issues: #319, #314, #197, #279
|
||||
**Mechanics:** `SecurityHeadersMiddlewareTests.Csp_Script_Hash_Should_Match_The_Spa_Index`
|
||||
|
||||
Completes the CSP that the #279 baseline-headers entry deferred ("CSP must be validated against the ChicoryTV
|
||||
SPA's inline assets"). Surfaced by the #314 out-of-ecosystem ZAP baseline (missing CSP/Permissions-Policy WARNs);
|
||||
a **#197 exit item**. `SecurityHeadersMiddleware` now also sets `Permissions-Policy` (deny-all for
|
||||
camera/microphone/geolocation/payment/usb) and an **enforcing** `Content-Security-Policy`.
|
||||
|
||||
- **Enforce, not report-only.** Report-only was the issue's acceptable fallback, but the SPA's asset graph is
|
||||
small and fully knowable, so we ship an enforcing policy (report-only leaves the ZAP WARN and provides no real
|
||||
protection). The policy: `default-src 'self'`; `script-src 'self' '<sha256 of the inline theme-bootstrap
|
||||
script>'` (**no** `'unsafe-inline'`/`'unsafe-eval'` — the real XSS win); `style-src 'self' 'unsafe-inline'
|
||||
https://fonts.googleapis.com`; `img-src 'self' data: blob:`; `font-src 'self' data: https://fonts.gstatic.com`;
|
||||
`connect-src 'self'`; `object-src 'none'`; `base-uri 'self'`; `frame-ancestors 'none'`; `form-action 'self'`.
|
||||
- **Why each relaxation.** The SPA is a static file, so a per-response nonce is impossible → the one inline
|
||||
theme-bootstrap `<script>` is allow-listed **by hash**; `SecurityHeadersMiddlewareTests.Csp_Script_Hash_Should_
|
||||
Match_The_Spa_Index` hashes the built `wwwroot/app/index.html` when present (else the committed `web/index.html`
|
||||
source, since the built artifact is gitignored/absent in CI — Vite copies the inline script verbatim) and fails
|
||||
if it drifts from the middleware constant. `style-src 'unsafe-inline'` covers
|
||||
React's inline `style=""` attributes (no CSS-in-JS lib to hash). The **Google Fonts** hosts are required — the
|
||||
SPA CSS `@import`s the Geist web font (caught by **live-E2E**, which the static grep missed); self-hosting the
|
||||
font to drop the Google dependency is a follow-on hardening, not this issue. `img-src data: blob:` covers
|
||||
favicon/generated-image data URIs and object-URL upload previews.
|
||||
- **Scoped: `/docs` (Scalar) and `/openapi` are excluded.** The Scalar API-reference UI relies on inline bootstrap
|
||||
scripts/styles a strict CSP would break; it keeps the baseline headers (nosniff/frame/referrer) but no CSP.
|
||||
Hardening that admin surface (self-hosted Scalar or a Scalar-tuned CSP) is a #197 follow-up. Everything else —
|
||||
SPA, `/api`, `/artwork`, `/iptv` — gets the CSP (non-HTML responses simply never exercise the script/style
|
||||
directives). Verified by live-E2E (SPA renders clean, zero CSP violations) + curl (CSP present on `/app`/`/api`,
|
||||
absent on `/docs`/`/openapi`). HSTS remains out (proxy/TLS decision). Refs #319 #314 #197.
|
||||
|
||||
## 2026-07-13 — Cross-origin resource policy: `same-origin` on every response (#330)
|
||||
`key: security.corp-same-origin` · `status: active` · `since: 2026-07-13` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** `SecurityHeadersMiddleware` sends `Cross-Origin-Resource-Policy: same-origin` on every response including `/docs`/`/openapi`, blocking cross-origin `no-cors` embedding without affecting allowed CORS-mode fetches or server-side Jellyfin `/iptv/*` requests.
|
||||
**Signals:** CORP, same-origin, cross-origin embedding · paths: `ErsatzTV/Middleware/SecurityHeadersMiddleware` · issues: #330, #319, #314
|
||||
**Mechanics:** `ErsatzTV/Middleware/SecurityHeadersMiddleware`
|
||||
|
||||
The authenticated #314 ZAP scan found that ErsatzTV's baseline response posture omitted
|
||||
`Cross-Origin-Resource-Policy`. `SecurityHeadersMiddleware` now sends
|
||||
`Cross-Origin-Resource-Policy: same-origin` on every response, including `/docs` and `/openapi`.
|
||||
Those two paths remain exempt only from the strict CSP that would break Scalar's inline bootstrap;
|
||||
CORP has no equivalent rendering conflict and belongs with the middleware's path-independent baseline
|
||||
headers.
|
||||
|
||||
`same-origin` requires the browser request and response to share the exact scheme, host, and port. It
|
||||
blocks cross-origin `no-cors` loads, so direct browser embedding of ErsatzTV artwork or media from an
|
||||
alternate origin is deliberately unsupported. It does not reject an allowed CORS-mode API fetch, so the
|
||||
explicit `Api:CorsAllowedOrigins` machine-client path continues to work. It is also not enforced by
|
||||
server-side HTTP clients, so Jellyfin's `/iptv/*` requests are unaffected; same-origin SPA artwork and
|
||||
IPTV requests remain allowed. This is defense in depth for browser embedding and does not replace CORS
|
||||
or authentication. Refs #330 #319 #314.
|
||||
|
||||
## 2026-07-22 — Short-lived browser IPTV token so the SPA reaches `/iptv/*` under JWT auth (#552)
|
||||
`key: security.iptv-browser-token` · `status: active` · `since: 2026-07-22` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** Under a JWT-enabled deployment (`JWT:IssuerSigningKey` set), the browser SPA obtains a short-lived, globally-scoped `/iptv/*` access token from an authenticated `GET /api/v1/auth/iptv-token` and appends it as `?access_token=`; the endpoint answers 204 when JWT is disabled (nothing to mint). Lifetime defaults to 60 min, configurable via `JWT:BrowserTokenLifetimeMinutes`.
|
||||
**Signals:** channel preview / playback-troubleshooting 401ing under JWT; `/iptv/*` not accepting `ctv-session`; minting a JWT for the browser · paths: `ErsatzTV/JwtHelper.cs`, `ErsatzTV/Controllers/Api/AuthController.cs`, `web/src/media/iptvToken.ts` · issues: #552, #60
|
||||
**Mechanics:** `JwtHelper.GenerateBrowserToken()`; `AuthController.IptvToken`; `withIptvToken(url)` (SPA)
|
||||
|
||||
`/iptv/*` is gated by `ConditionalIptvAuthorizeFilter` only when `JWT:IssuerSigningKey` is configured,
|
||||
and the `"jwt"` scheme accepts only a bearer token or `?access_token=` — **not** the SPA's `ctv-session`
|
||||
cookie (a distinct scheme). Nothing minted a JWT for the browser, so under JWT the #60 channel preview
|
||||
was declared `Unavailable` and the pre-existing playback-troubleshooting screen was latently broken. This
|
||||
closes both with one seam.
|
||||
|
||||
- **Endpoint.** `GET /api/v1/auth/iptv-token` on `AuthController` (already `[SkipApiAuthorization]` +
|
||||
self-checks the principal, like `machine-key`). Requires any authenticated session (401 otherwise);
|
||||
returns `{ token, expiresAt }` when `JwtHelper.IsEnabled`, else **204 No Content** — `/iptv/*` is open
|
||||
then, so there is nothing to append and the SPA plays the plain URL. `Cache-Control: no-store`. Excluded
|
||||
from the OpenAPI document (`AuthController` is `[IgnoreApi]`, per the #295 F4 ruling — a
|
||||
browser-interactive credential is not something a generated client drives).
|
||||
- **A GET is correct here** despite the "no side-effecting GET under `/api`" rule (`api-conventions.md §9`):
|
||||
minting a JWT writes **no server state** (stateless token, no DB row, no revocation list), so it is not a
|
||||
CSRF-relevant side effect, and same-origin policy already blocks a cross-site page from reading the
|
||||
credentialed response body — identical reasoning to the `machine-key` GET.
|
||||
- **Scope: global.** The `"jwt"` scheme validates only signature + lifetime (no audience/channel claims),
|
||||
and the token is minted only to the already-authenticated admin who can reach every channel.
|
||||
Channel-scoping would mean adding claim-based auth to `ConditionalIptvAuthorizeFilter` and the streaming
|
||||
path — deferred until a non-admin preview audience exists.
|
||||
- **Lifetime: 60 min default, `JWT:BrowserTokenLifetimeMinutes` override, clamped to 24h.** The token
|
||||
re-validates on every `/iptv/*` request, so lifetime is the max continuous watch before playback stalls;
|
||||
60 min comfortably covers an operator verification session, an expired idle session just needs Retry
|
||||
(mints fresh), and a security-conscious operator can tighten it. A non-positive/unparseable value falls
|
||||
back to the default rather than minting an already-expired token; a value above 24h (a seconds-vs-minutes
|
||||
typo would otherwise mint a multi-year bearer token) is clamped down. **Revocation is by short lifetime
|
||||
only** — a stateless JWT has no per-token revocation; rotating `JWT:IssuerSigningKey` invalidates all
|
||||
tokens (the existing lever). The SPA's `resetIptvTokenCache()` (called on the preview panel's Retry and on
|
||||
each troubleshooting Play) makes a user-initiated retry re-mint, so a stale token or a stale "JWT disabled"
|
||||
latch from a since-reconfigured backend can't wedge a recovery attempt.
|
||||
- **Deferred hardening (broader than #552) — RESOLVED by `security.iptv-access-token-transport` (#421, #559).**
|
||||
The `?access_token=` transport itself had pre-existing weaknesses this feature inherited, then bounded by the
|
||||
short lifetime: Serilog's request log included the full query (so a token could reach logs on an `/iptv` 5xx),
|
||||
and the token-bearing dynamic manifests carried no `Cache-Control: no-store`. Both predate this feature
|
||||
(Jellyfin and the M3U playlist already pass `access_token` in `/iptv` URLs); their fixes were cross-cutting
|
||||
changes to shared request-logging / manifest behavior, tracked as a follow-up and now landed in the record
|
||||
below (which also adds the #421 M3U quote-safety encoding).
|
||||
- **Only the top-level manifest needs the token.** The multi-variant playlist embeds `access_token` into
|
||||
its variant URL (`IptvController.GetMultiVariantPlaylist`), and HLS segments are served by
|
||||
`UseStaticFiles` at `RequestPath=/iptv/session` — **outside** `ConditionalIptvAuthorizeFilter` (which is
|
||||
a `[ServiceFilter]` on `IptvController` only) — so segment GETs are ungated regardless. The SPA's
|
||||
`withIptvToken(url)` appends the token to the one manifest URL (a no-op when JWT is off; caches the token
|
||||
in memory until shortly before expiry) and is used by both the channel-preview panel and the
|
||||
playback-troubleshooting screen.
|
||||
|
||||
## 2026-07-23 — access_token transport hardening: percent-encode in M3U/HLS, redact from logs, no-store on tokened manifests (#421, #559)
|
||||
`key: security.iptv-access-token-transport` · `status: active` · `since: 2026-07-23` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** The `/iptv` `?access_token=` value is percent-encoded (`Uri.EscapeDataString`) everywhere it is interpolated into an M3U/HLS/XMLTV URL (XMLTV additionally XML-escapes the encoded value), so a structural character can't malform the manifest or guide; Serilog logs a scrubbed request path (`access_token` → `***` via `IncludeQueryInRequestPath = false` + a `RequestPathScrubbed` enricher), so a 5xx/Debug `/iptv` request never writes the token; and every dynamic token-bearing `/iptv` manifest (`channels.m3u`, `xmltv.xml`, the HLS multi-variant/media playlists) returns `Cache-Control: private, no-store`.
|
||||
**Signals:** access_token quote-safety, url-tvg breakout, M3U attribute escaping, XMLTV icon src encoding, request-log token redaction, no-store manifest, token replay from logs/cache · paths: `ErsatzTV.Core/Iptv/ChannelPlaylist.cs`, `ErsatzTV/Controllers/IptvController.cs`, `ErsatzTV.Application/Channels/Queries/GetChannelGuideHandler.cs`, `ErsatzTV/Middleware/RequestLogScrubber.cs`, `ErsatzTV/Startup.cs`, `ErsatzTV.Core/FFmpeg/FFmpegLibraryProcessService.cs` · issues: #421, #559, #552, #376
|
||||
**Mechanics:** `ChannelPlaylistAccessTokenTests`; `RequestLogScrubberTests`; `IptvControllerCacheHeaderTests`
|
||||
|
||||
Resolves the "deferred hardening" bullet under `security.iptv-browser-token` (#552). The shared `?access_token=`
|
||||
transport had three pre-existing weaknesses that #552's short-lived browser token only *bounded* (not fixed);
|
||||
they predate #552 because Jellyfin and the M3U playlist already pass `access_token` in `/iptv` URLs.
|
||||
|
||||
- **Percent-encode the token in M3U/HLS URLs (#421).** `ChannelPlaylist.ToM3U` interpolates the token into
|
||||
quoted M3U attributes (`url-tvg="..."`, `tvg-logo="..."`) and bare stream URLs; `IptvController.AccessTokenQuery`
|
||||
and the internal segmenter URL (`FFmpegLibraryProcessService.WrapSegmenter`) do the same. A token containing a
|
||||
`"` could terminate a quoted attribute early, and an `&` could split the query — for strict parsers. The value
|
||||
is a URL query parameter, so the correct fix is percent-encoding (`Uri.EscapeDataString`) at each interpolation
|
||||
site; ASP.NET decodes `Request.Query`, so the round-trip is lossless and validation is unchanged. For a normal
|
||||
base64url JWT this is a **no-op** (every character is RFC 3986 unreserved), so the M3U goldens are byte-identical.
|
||||
This is the M3U analogue of #376's XMLTV `SecurityElement.Escape` fix — #376 correctly did **not** apply to M3U
|
||||
(M3U is not XML), which is why #421 was split out. The **XMLTV twin** (`GetChannelGuideHandler`) is fixed the same
|
||||
way for symmetry: the token there lands in a URL query value *inside* an XML attribute, so it now percent-encodes
|
||||
**then** XML-escapes (`SecurityElement.Escape(Uri.EscapeDataString(token))`) — #376's XML-escape alone guarded XML
|
||||
well-formedness but not URL correctness, so a token with `&` would still truncate the query after the consumer
|
||||
XML-unescapes the attribute.
|
||||
- **Redact the token from request logs (#559).** `UseSerilogRequestLogging` had `IncludeQueryInRequestPath = true`,
|
||||
and the `ex != null` / `StatusCode > 499` branches log at **Error** regardless of path — so a 5xx on a tokened
|
||||
`/iptv` URL wrote the replayable, globally-scoped token to the log under default settings. Fix:
|
||||
`IncludeQueryInRequestPath = false` (built-in `RequestPath` is now path-only) plus a `RequestPathScrubbed`
|
||||
diagnostic-context property — built by `RequestLogScrubber.ScrubbedPath`, which rebuilds the query with the
|
||||
`access_token` value masked to `***` and everything else (e.g. `mode=segmenter`) preserved for debugging — that
|
||||
the message template references in place of `{RequestPath}`. A distinct property name is required because Serilog
|
||||
appends its own `RequestPath` *after* enriched properties, so reusing the name would let the built-in value win.
|
||||
**Load-bearing residual:** this scrubs only `UseSerilogRequestLogging`. ASP.NET's own
|
||||
`Microsoft.AspNetCore.Hosting.Diagnostics` "Request starting" line logs the raw query (token and all) at
|
||||
Information; it is suppressed today solely by the `"Microsoft": "Warning"` override in `appsettings.json`, which
|
||||
is **not** reachable from the runtime logging-settings UI (the `HttpLevelSwitch` overrides only
|
||||
`Serilog.AspNetCore.RequestLoggingMiddleware`). Keep that override — lowering `Microsoft` to Information/Debug for
|
||||
routing/auth debugging re-leaks the token. A category-agnostic scrub filter is the follow-up if that override is
|
||||
ever relaxed.
|
||||
- **`no-store` on token-bearing dynamic manifests (#559).** `channels.m3u`, `xmltv.xml`, and the HLS
|
||||
multi-variant/media playlists embed the caller's token in their body/URLs but set no cache headers, so a browser
|
||||
or intermediary could cache the token-bearing manifest. Each now returns `Cache-Control: private, no-store`. This
|
||||
covers the Jellyfin-facing M3U/XMLTV as well as the #552 HLS preview manifests — all three share the same
|
||||
cache-replay risk — and does not affect Jellyfin, which refetches the playlist/guide on its own schedule
|
||||
regardless of HTTP cache headers. The `.ts` live streams and the ungated HLS segments (served by `UseStaticFiles`
|
||||
at `/iptv/session`, outside `ConditionalIptvAuthorizeFilter`) carry no token and are unchanged.
|
||||
|
||||
@@ -7,10 +7,50 @@ in the active read-path). Active successor for channel health: `api.channel-heal
|
||||
|
||||
---
|
||||
|
||||
## Records formerly in this file
|
||||
## 2026-07-17 — Channel health on the API = the raw `PlayoutCount` fact on the list DTO, not a derived status enum (#72)
|
||||
`key: api.channel-health-signal` · `status: superseded` · `since: 2026-07-17` · `supersedes: none` · `superseded-by: api.channel-health-object@2026-07-23`
|
||||
**Rule:** (superseded) Channel health rides `ChannelResponseModel`/`ChannelListItem` DTOs as a raw `int PlayoutCount` fact (free — `GetAll` already `Include`s `Playouts`), not a new endpoint, not `/channels/state` (runtime-liveness cadence), and not a derived `ChannelHealth` enum (would freeze policy before the #383/#384 auto-tune status taxonomy lands).
|
||||
**Signals:** channel health, PlayoutCount, config-derived vs runtime-liveness cadence, frozen /api/v1 · paths: `ChannelRepository.GetChannel`, `Mapper.GetPlayoutsCount`, `api-conventions.md` §3a · issues: #72, #383, #384, #401
|
||||
**Mechanics:** superseded by `api.channel-health-object` (ersatztv#415); see `docs/decisions.md` → that record for the `health` object that replaces `PlayoutCount`-as-verdict
|
||||
|
||||
Each record below moved to its own file under `records/` (ersatztv#610); the rationale is
|
||||
unchanged. Resolve by **key** — that is the stable identity. A date-based pointer from
|
||||
another doc or an old issue comment should land here and then follow the link.
|
||||
#72 asks for per-channel status in the Channels list, "especially channels that will fail to play".
|
||||
|
||||
- 2026-07-17 — Channel health on the API = the raw `PlayoutCount` fact on the list DTO, not a derived status enum (#72) — [`api.channel-health-signal`](api/channel-health-signal.md)
|
||||
**Where it lives: `ChannelResponseModel` (the lean list DTO), not a new endpoint and not
|
||||
`/channels/state`.** Channel health is *config-derived* — it changes when someone edits a playout, not
|
||||
tick to tick — whereas `/channels/state` is the fast-poll runtime-liveness feed (`OnAir` = someone is
|
||||
streaming *right now*). Folding health into the polled feed would recompute rarely-changing data every
|
||||
tick and mix two cadences in one DTO; a third endpoint is over-engineering for one derived integer on a
|
||||
list whose consumer already reads it. Precedent: `api-conventions.md` §3a stamps the server-derived
|
||||
`IsLocked` onto `PlayoutListItemResponseModel` for exactly this reason. **It is also free**: `GetAll`
|
||||
already `Include`s `Playouts` and `MirrorSourceChannel.Playouts` and was discarding them, so no extra
|
||||
query and no N+1 on a large lineup.
|
||||
|
||||
**A raw fact (`int PlayoutCount`), not a `ChannelHealth` enum.** v1 has exactly one trustworthy negative
|
||||
signal, and the auto-tune arc (#383/#384) is about to churn the status taxonomy (origin, auto-tune
|
||||
outcomes) — freezing a server-side enum now guarantees a breaking rev of a frozen-additive `/api/v1`
|
||||
surface. `PlayoutCount` mirrors the long-standing `ChannelViewModel.PlayoutCount` 1:1, is a fact rather
|
||||
than a policy, and leaves the SPA to derive `0 ⇒ "No playout"` in one predicate.
|
||||
|
||||
**What v1 deliberately does NOT compute** — each was considered and ruled out, so don't "finish" them
|
||||
without reading this:
|
||||
- **Empty schedule behind an existing playout.** `EmptyScheduleHealthCheck` only understands **Classic**
|
||||
`ProgramSchedule` playouts. Block, Sequential, Scripted and ExternalJson channels have no
|
||||
`ProgramSchedule` at all, so a badge driven off that query would be silently absent or wrong for four of
|
||||
the five schedule kinds — the #71 "verify a shared primitive covers ALL variants" trap. Needs a
|
||||
per-kind emptiness notion first.
|
||||
- **Broken / missing source.** `FileNotFound`/`Unavailable` are server-wide media-item counts with no
|
||||
channel attribution; mapping media → collection → schedule → channel is a project, not a field.
|
||||
- **User-defined vs auto-generated origin** (#72 scope item a). No honest signal exists:
|
||||
`Channel` has no origin column, and `ChannelPlayoutSource.Generated` is a *playout-strategy* value that
|
||||
SPA-created blank channels also carry, so it would mislabel them. Requires a new column + a dual-provider
|
||||
migration, and provenance belongs to the auto-tune arc that stamps it at creation. A join through the
|
||||
`"Channel Lineups"` system playlist group was **rejected**: it is a heuristic that breaks the moment a
|
||||
user edits the channel. Deferred to a follow-up blocked on the auto-tune backend.
|
||||
|
||||
Keeping all three out held #72 to a **read-path-only** change: no migration, no write-handler live-E2E.
|
||||
|
||||
**Corrected in passing:** `ChannelRepository.GetChannel` never included `Playouts`, so
|
||||
`GET /api/v1/channels/{id}` reported `playoutCount: 0` for every channel, which silently disabled the
|
||||
channel editor's playout-source guard. Both call sites now share `Mapper.GetPlayoutsCount` (Mirror-aware).
|
||||
The related *silent* server-side coercion of Mirror→Generated (a 200 that discards the caller's intent,
|
||||
against the §3 "surface it, don't silently filter" rule) is filed as **#401**, not fixed here.
|
||||
|
||||
@@ -1,53 +0,0 @@
|
||||
---
|
||||
key: api.channel-health-signal
|
||||
title: 2026-07-17 — Channel health on the API = the raw `PlayoutCount` fact on the list DTO, not a derived status enum (#72)
|
||||
status: superseded
|
||||
since: '2026-07-17'
|
||||
supersedes: none
|
||||
superseded-by: api.channel-health-object@2026-07-23
|
||||
rule: '(superseded) Channel health rides `ChannelResponseModel`/`ChannelListItem` DTOs as a raw `int PlayoutCount` fact (free — `GetAll` already `Include`s `Playouts`), not a new endpoint, not `/channels/state` (runtime-liveness cadence), and not a derived `ChannelHealth` enum (would freeze policy before the #383/#384 auto-tune status taxonomy lands).'
|
||||
signals: 'channel health, PlayoutCount, config-derived vs runtime-liveness cadence, frozen /api/v1 · paths: `ChannelRepository.GetChannel`, `Mapper.GetPlayoutsCount`, `api-conventions.md` §3a · issues: #72, #383, #384, #401'
|
||||
mechanics: superseded by `api.channel-health-object` (ersatztv#415); see `docs/decisions.md` → that record for the `health` object that replaces `PlayoutCount`-as-verdict
|
||||
---
|
||||
|
||||
#72 asks for per-channel status in the Channels list, "especially channels that will fail to play".
|
||||
|
||||
**Where it lives: `ChannelResponseModel` (the lean list DTO), not a new endpoint and not
|
||||
`/channels/state`.** Channel health is *config-derived* — it changes when someone edits a playout, not
|
||||
tick to tick — whereas `/channels/state` is the fast-poll runtime-liveness feed (`OnAir` = someone is
|
||||
streaming *right now*). Folding health into the polled feed would recompute rarely-changing data every
|
||||
tick and mix two cadences in one DTO; a third endpoint is over-engineering for one derived integer on a
|
||||
list whose consumer already reads it. Precedent: `api-conventions.md` §3a stamps the server-derived
|
||||
`IsLocked` onto `PlayoutListItemResponseModel` for exactly this reason. **It is also free**: `GetAll`
|
||||
already `Include`s `Playouts` and `MirrorSourceChannel.Playouts` and was discarding them, so no extra
|
||||
query and no N+1 on a large lineup.
|
||||
|
||||
**A raw fact (`int PlayoutCount`), not a `ChannelHealth` enum.** v1 has exactly one trustworthy negative
|
||||
signal, and the auto-tune arc (#383/#384) is about to churn the status taxonomy (origin, auto-tune
|
||||
outcomes) — freezing a server-side enum now guarantees a breaking rev of a frozen-additive `/api/v1`
|
||||
surface. `PlayoutCount` mirrors the long-standing `ChannelViewModel.PlayoutCount` 1:1, is a fact rather
|
||||
than a policy, and leaves the SPA to derive `0 ⇒ "No playout"` in one predicate.
|
||||
|
||||
**What v1 deliberately does NOT compute** — each was considered and ruled out, so don't "finish" them
|
||||
without reading this:
|
||||
- **Empty schedule behind an existing playout.** `EmptyScheduleHealthCheck` only understands **Classic**
|
||||
`ProgramSchedule` playouts. Block, Sequential, Scripted and ExternalJson channels have no
|
||||
`ProgramSchedule` at all, so a badge driven off that query would be silently absent or wrong for four of
|
||||
the five schedule kinds — the #71 "verify a shared primitive covers ALL variants" trap. Needs a
|
||||
per-kind emptiness notion first.
|
||||
- **Broken / missing source.** `FileNotFound`/`Unavailable` are server-wide media-item counts with no
|
||||
channel attribution; mapping media → collection → schedule → channel is a project, not a field.
|
||||
- **User-defined vs auto-generated origin** (#72 scope item a). No honest signal exists:
|
||||
`Channel` has no origin column, and `ChannelPlayoutSource.Generated` is a *playout-strategy* value that
|
||||
SPA-created blank channels also carry, so it would mislabel them. Requires a new column + a dual-provider
|
||||
migration, and provenance belongs to the auto-tune arc that stamps it at creation. A join through the
|
||||
`"Channel Lineups"` system playlist group was **rejected**: it is a heuristic that breaks the moment a
|
||||
user edits the channel. Deferred to a follow-up blocked on the auto-tune backend.
|
||||
|
||||
Keeping all three out held #72 to a **read-path-only** change: no migration, no write-handler live-E2E.
|
||||
|
||||
**Corrected in passing:** `ChannelRepository.GetChannel` never included `Playouts`, so
|
||||
`GET /api/v1/channels/{id}` reported `playoutCount: 0` for every channel, which silently disabled the
|
||||
channel editor's playout-source guard. Both call sites now share `Mapper.GetPlayoutsCount` (Mirror-aware).
|
||||
The related *silent* server-side coercion of Mirror→Generated (a 200 that discards the caller's intent,
|
||||
against the §3 "surface it, don't silently filter" rule) is filed as **#401**, not fixed here.
|
||||
@@ -1,46 +0,0 @@
|
||||
---
|
||||
key: api.search-field-values
|
||||
title: 2026-07-23 — Facet-value typeahead is a new endpoint, allow-listed to text fields, no caching (#434)
|
||||
status: superseded
|
||||
since: '2026-07-23'
|
||||
supersedes: none
|
||||
superseded-by: api.search-field-values-sources@2026-07-26
|
||||
rule: '(superseded) `GET /api/v1/search/fields/{name}/values?q=&limit=` returns distinct WHOLE values from the database for one of a narrow allow-list of catalog fields (not the Lucene term dictionary — analyzed `TextField`s store lowercased word tokens, e.g. "Science Fiction" → `science`/`fiction`, useless as a typeahead suggestion), 404 for an unknown field, a non-`text` field, or a `text` field with no distinct-value source; case-insensitive prefix-filtered on `q`, `limit` clamped to `[1, 50]` (default 50).'
|
||||
signals: 'facet-value typeahead, rule builder value combobox, distinct field values, GetSearchFieldValues, text field allow-list, DB-sourced distinct values, content_rating split · paths: `ErsatzTV/Controllers/Api/SearchController.cs`, `ErsatzTV.Application/Search/Queries/GetSearchFieldValues.cs`, `ErsatzTV.Application/Search/Queries/GetSearchFieldValuesHandler.cs`, `web/src/api/search.ts` · issues: #434, #176'
|
||||
mechanics: superseded by `api.search-field-values-sources` (ersatztv#578), which keeps this endpoint contract and reverses the "no distinct-value source" call for the list-valued music fields
|
||||
---
|
||||
|
||||
Enum fields (e.g. `type`, `content_rating` group) already ship their allowed values inline on
|
||||
`SearchFieldResponseModel` from the existing `GET /api/v1/search/fields` catalog (`spa.smartcollection-rule-builder`,
|
||||
#176), so they need no endpoint — a client already has the full value set. **Text** fields (title, studio,
|
||||
genre-as-free-text, etc.) don't: their values are whatever strings the library actually contains, so the
|
||||
rule builder's value input for a text field needs a live lookup rather than a fixed list.
|
||||
The handler allow-lists on `field.Type != "text"` (matching the same `SearchFieldCatalog.Fields` the
|
||||
`/fields` endpoint serves) and returns `Option.None` → 404 for anything else, rather than silently returning
|
||||
an empty list for a field that will never have values — a 404 tells a caller "wrong field kind," an empty
|
||||
200 would look like "no matches yet."
|
||||
|
||||
**DB-sourced, not the search index.** The handler injects `IDbContextFactory<TvContext>` and resolves an
|
||||
explicit per-field-name `IQueryable<string>` (or, for a few special cases, an in-memory list) rather than
|
||||
querying `ISearchIndex`: `genre`/`show_genre` → `Set<Genre>()`, `studio` → `Set<Studio>()`, `director` →
|
||||
`Set<Director>()`, `writer` → `Set<Writer>()`, `actor` → `Actors`, `artist` → `ArtistMetadata.Title` (entity
|
||||
artists only — free-text music-video/song artist credits are a known, intentionally-uncovered gap), `tag` →
|
||||
`Set<Tag>()` excluding `Tag.NfoCountryTypeId`/`Tag.PlexNetworkTypeId` (reapplying the indexer's own
|
||||
exclusions so country/network strings don't leak in as tags), `network` → `Set<Tag>()` filtered to
|
||||
`Tag.PlexNetworkTypeId`, `collection` → `Collections`, `video_codec` → `MediaStreams` filtered to
|
||||
`MediaStreamKind.Video`, `album` → `MusicVideoMetadata.Album` concatenated with `SongMetadata.Album`. Every
|
||||
DB-sourced field runs the same pipeline: `.Where(v => v.ToLower().StartsWith(qLower)).Distinct().OrderBy(v =>
|
||||
v).Take(limit)`, translated to SQL by EF for both SQLite and MySQL. Two fields are computed in memory instead
|
||||
of queried: `state` (the fixed 4-value `MediaItemState` enum) and `video_dynamic_range` (the literal
|
||||
`["hdr", "sdr"]`). `content_rating` is special-cased: the DB stores an unsplit `"PG-13/TV-14"` string across
|
||||
`MovieMetadata`/`ShowMetadata`/`OtherVideoMetadata`/`RemoteStreamMetadata`, so the handler pulls the distinct
|
||||
raw strings then `Split('/')`s, trims, and dedupes in memory before the same prefix-filter/sort/take — this
|
||||
matches what search actually matches on, rather than surfacing the compound string as one facet value.
|
||||
**`title`, `show_title`, `album_artist` are explicitly NOT supported** (404, free-text fallback): `title`/
|
||||
`show_title` are near-unique free-text fields spanning ~9 metadata tables where a distinct list of every
|
||||
title isn't a useful facet; `album_artist` backs onto `SongMetadata.AlbumArtists`, a value-converted
|
||||
`IList<string>` column EF can't translate into a server-side distinct query.
|
||||
|
||||
**Why a thin query, not a cache.** No result cache, no debounce on the server side (the SPA combobox
|
||||
debounces the keystroke) — each per-field query is a bounded, indexed `Distinct`/`Take`; adding a cache
|
||||
before there's a measured cost would be premature.
|
||||
@@ -1,33 +0,0 @@
|
||||
---
|
||||
key: docs.append-only-guard
|
||||
title: 2026-07-12 — decisions.md is append-only, enforced; root-screenshot guard (#303 H9/H3)
|
||||
status: superseded
|
||||
since: '2026-07-12'
|
||||
supersedes: none
|
||||
superseded-by: docs.decision-lifecycle@2026-07-21
|
||||
rule: (superseded) `docs/decisions.md` is append-only, enforced by hook + CI.
|
||||
signals: 'decisions-guard hook · paths: `.claude/hooks/decisions-guard.sh`, `.husky/commit-msg` · issues: #303 (H9)'
|
||||
mechanics: superseded by `scripts/decisions_validate.py` (ersatztv#521); see `docs/decisions.md` → `docs.decision-lifecycle`
|
||||
---
|
||||
|
||||
**This log is append-only by construction, not just by convention.** A commit or PR that deletes or
|
||||
modifies an existing line of `docs/decisions.md` is blocked — by the Husky `commit-msg` hook
|
||||
(`.claude/hooks/decisions-guard.sh staged`) locally and the blocking `decisions-guard` CI job (same
|
||||
script, `range` mode) on PRs. Shared detection, deliberately different granularity: the Husky hook
|
||||
gates **each commit** (its own message must carry the token); CI gates the **PR-wide** net diff
|
||||
(token in any commit of the range suffices), so the local hook is the stricter primary gate and CI the
|
||||
push/bypass backstop. Insertions anywhere are always allowed, so a normal new entry (TOC line
|
||||
near the top + a block appended at the bottom, both pure insertions) passes untouched. Detection is
|
||||
`git diff --numstat` deleted-count > 0, which is robust to markdown `-` list markers (a byte-level `-`
|
||||
prefix would false-match). The block is lifted only by the literal **`[decisions-edit]`** token in the
|
||||
commit message, reserved for two cases: fixing a factual error, and superseding a reversed decision
|
||||
(add the new entry, prepend a `> **Superseded …**` banner to the old one, tag its Index line
|
||||
`(superseded)` — keep the old rationale, never silently rewrite). **Consolidation** of superseded
|
||||
entries is a release-checklist step (`docs/ci-cd.md` → Versioning & releases), backstopped by a
|
||||
non-blocking 1800-line **size floor** in the `decisions-guard` job (the read-cost point past which the
|
||||
log no longer fits one default agent Read), so append-only doesn't accrete contradictory *or
|
||||
unreadably-large* history between releases (Timothy's call, 2026-07-12: mark-and-keep on reversal,
|
||||
consolidate at each milestone, size-floor backstop).
|
||||
|
||||
The companion H3 root-screenshot guard was split out during the #521 migration to the active record
|
||||
`ci.root-screenshot-guard` in `docs/decisions/release-ci-governance.md`.
|
||||
@@ -1,37 +0,0 @@
|
||||
---
|
||||
key: docs.queue-state-gitea-tracker
|
||||
title: 2026-07-11 — Queue state lives in the pinned Gitea tracker (#237), not in the handoff file
|
||||
status: superseded
|
||||
since: '2026-07-11'
|
||||
supersedes: none
|
||||
superseded-by: startup.parallel-orientation@2026-07-21
|
||||
rule: '(superseded) Volatile session/queue state lives in pinned Gitea tracker issue #237 (goal + arc in the body, append-only session-comment log, in-progress/review labels), not in a wholesale-rewritten handoff file; the handoff file keeps only the static kickoff prompt and append-only lore.'
|
||||
signals: 'last-writer-wins race, claim/`in-progress` label, triage verdict · paths: `docs/handoffs/chicorytv-issue-queue.md` · issues: #237, #520'
|
||||
mechanics: superseded by `scripts/select-queue.sh` (ersatztv#520); see `docs/decisions.md` → `startup.parallel-orientation`
|
||||
---
|
||||
|
||||
With multiple sessions/agents working the repo in parallel, the old protocol — every session
|
||||
wholesale-rewrites `docs/handoffs/chicorytv-issue-queue.md` on main (session state + queue +
|
||||
next-session prompt) — became a last-writer-wins race. New protocol: **volatile queue state
|
||||
moved to Gitea**, which is concurrency-safe by construction. Pinned tracker issue **#237**
|
||||
holds the goal + ordered arc in its body (edited rarely, only on arc changes, re-read before
|
||||
edit) and an append-only session-comment log (fixed template: Closed / Filed / Triage /
|
||||
Arc change / Recommended next). Milestone `Blazor removal (#91 phase b)` + the `review` and
|
||||
`in-progress` labels are the machine-queryable view. Sessions **claim** an issue before working
|
||||
it (`in-progress` label + claim comment; the tiny read→claim race window is accepted, later
|
||||
claimant backs off; stale claims — no commits/comments ~48h — may be taken over with a comment).
|
||||
Every new issue gets an explicit end-of-session triage verdict — gate-blocker (milestone + arc
|
||||
slot) or backlog (label only) — so review findings adjust the queue only through that step and
|
||||
the arc doesn't drift. The handoff file keeps only the **static kickoff prompt** and the
|
||||
**append-only Lessons lore** (per-session prompts are gone; task context lives in issue bodies).
|
||||
|
||||
**Why superseded (#520, 2026-07-21):** the arc completed and #237 closed (2026-07-13); a closed
|
||||
tracker cannot serve as live queue state, and continuing to read it as such was a live regression
|
||||
risk (an agent skimming an old comment or this very record could re-treat #237's prose as current).
|
||||
`scripts/select-queue.sh` (2026-07-19) already replaced the mechanical parts of this rule with a
|
||||
deterministic, live-Gitea-only query — this record's job was really "queue state lives in Gitea,
|
||||
not in the handoff file," and that half is still true; what's superseded is the *specific store*
|
||||
(#237's body/comments) now that maintenance/backlog mode has no arc to narrate. See
|
||||
`startup.parallel-orientation` for the replacement: two concurrent session-start tracks
|
||||
(orientation via the docs/decisions catalog + `docs/README.md` map, and selection via the script),
|
||||
with #237 reduced to a single archival breadcrumb.
|
||||
@@ -14,10 +14,30 @@ directory is and `docs/decisions/migration-map.md` for the full mapping.
|
||||
|
||||
---
|
||||
|
||||
## Records formerly in this file
|
||||
## 2026-07-12 — decisions.md is append-only, enforced; root-screenshot guard (#303 H9/H3)
|
||||
`key: docs.append-only-guard` · `status: superseded` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: docs.decision-lifecycle@2026-07-21`
|
||||
**Rule:** (superseded) `docs/decisions.md` is append-only, enforced by hook + CI.
|
||||
**Signals:** decisions-guard hook · paths: `.claude/hooks/decisions-guard.sh`, `.husky/commit-msg` · issues: #303 (H9)
|
||||
**Mechanics:** superseded by `scripts/decisions_validate.py` (ersatztv#521); see `docs/decisions.md` → `docs.decision-lifecycle`
|
||||
|
||||
Each record below moved to its own file under `records/` (ersatztv#610); the rationale is
|
||||
unchanged. Resolve by **key** — that is the stable identity. A date-based pointer from
|
||||
another doc or an old issue comment should land here and then follow the link.
|
||||
**This log is append-only by construction, not just by convention.** A commit or PR that deletes or
|
||||
modifies an existing line of `docs/decisions.md` is blocked — by the Husky `commit-msg` hook
|
||||
(`.claude/hooks/decisions-guard.sh staged`) locally and the blocking `decisions-guard` CI job (same
|
||||
script, `range` mode) on PRs. Shared detection, deliberately different granularity: the Husky hook
|
||||
gates **each commit** (its own message must carry the token); CI gates the **PR-wide** net diff
|
||||
(token in any commit of the range suffices), so the local hook is the stricter primary gate and CI the
|
||||
push/bypass backstop. Insertions anywhere are always allowed, so a normal new entry (TOC line
|
||||
near the top + a block appended at the bottom, both pure insertions) passes untouched. Detection is
|
||||
`git diff --numstat` deleted-count > 0, which is robust to markdown `-` list markers (a byte-level `-`
|
||||
prefix would false-match). The block is lifted only by the literal **`[decisions-edit]`** token in the
|
||||
commit message, reserved for two cases: fixing a factual error, and superseding a reversed decision
|
||||
(add the new entry, prepend a `> **Superseded …**` banner to the old one, tag its Index line
|
||||
`(superseded)` — keep the old rationale, never silently rewrite). **Consolidation** of superseded
|
||||
entries is a release-checklist step (`docs/ci-cd.md` → Versioning & releases), backstopped by a
|
||||
non-blocking 1800-line **size floor** in the `decisions-guard` job (the read-cost point past which the
|
||||
log no longer fits one default agent Read), so append-only doesn't accrete contradictory *or
|
||||
unreadably-large* history between releases (Timothy's call, 2026-07-12: mark-and-keep on reversal,
|
||||
consolidate at each milestone, size-floor backstop).
|
||||
|
||||
- 2026-07-12 — decisions.md is append-only, enforced; root-screenshot guard (#303 H9/H3) — [`docs.append-only-guard`](docs/append-only-guard.md)
|
||||
The companion H3 root-screenshot guard was split out during the #521 migration to the active record
|
||||
`ci.root-screenshot-guard` in `docs/decisions/release-ci-governance.md`.
|
||||
|
||||
@@ -7,10 +7,43 @@ verbatim, never in the active read-path). Active successor for music-video recon
|
||||
|
||||
---
|
||||
|
||||
## Records formerly in this file
|
||||
## 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)
|
||||
`key: scan.musicvideo-reconciliation` · `status: superseded` · `since: 2026-07-20` · `supersedes: none` · `superseded-by: scan.musicvideo-server-identity@2026-07-25`
|
||||
**Rule:** (superseded) `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity.
|
||||
**Signals:** music-video trash sweep, path-based identity, cross-kind safety, path-keyed identity, empty-fetch guard reuse, remove-stale+add-new dedup · paths: `JellyfinMusicVideoLibraryScanner.TrashMissingMusicVideos`, `FindMusicVideoPaths`/`DeleteByPath`, `IMusicVideoRepository`, `MediaServerReconciliationGuard` · issues: #494, #477, #488, #496, #500
|
||||
**Mechanics:** `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`; `ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items`; integration tests extending the #488 harness. #500 — when mirroring the remove-stale + add-new idiom, dedup the incoming set on **the same key its add filter compares** (the filter is materialized before the loop mutates `existing`, so duplicates both pass): `Name`, `Guid` for guids, and for Plex `Actors` an artwork-preferring dedup shared with the remove filter (whose key is `(Name, artwork-presence)`). Remaining un-deduped copies of the idiom: #600. Superseded by `scan.musicvideo-server-identity` (ersatztv#496): music videos gained a `JellyfinMusicVideo` ItemId/Etag identity, so the path diff + hard delete became an itemId diff + soft `FileNotFound` trash.
|
||||
|
||||
Each record below moved to its own file under `records/` (ersatztv#610); the rationale is
|
||||
unchanged. Resolve by **key** — that is the stable identity. A date-based pointer from
|
||||
another doc or an old issue comment should land here and then follow the link.
|
||||
The Jellyfin music-video scanner did add/update only — a music video removed on the Jellyfin side lingered in
|
||||
ErsatzTV forever and could still be scheduled. It now runs a trash sweep at the end of `ScanLibrary`
|
||||
(`TrashMissingMusicVideos`), mirroring the `MediaServer{Movie,Television,OtherVideo}LibraryScanner` "gone
|
||||
upstream ⇒ remove" pattern but with a deliberately different identity function, because music videos lack the
|
||||
media-server identity those base scanners rely on.
|
||||
|
||||
- 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494) — [`scan.musicvideo-reconciliation`](scan/musicvideo-reconciliation.md)
|
||||
- **Identity is (LibraryPathId, path), not server itemId.** The base scanners diff `GetExisting*` (keyed by
|
||||
`MediaServerItemId`) against the incoming server item ids, then soft-trash via `FlagFileNotFound`. Music videos
|
||||
have **no `JellyfinMusicVideo` entity and no `ItemId`/`Etag`** — the scanner is a standalone
|
||||
`IJellyfinMusicVideoLibraryScanner` that injects the *local* `IMusicVideoRepository`, which offers no
|
||||
itemId-keyed existing-set or flag seam. So the sweep diffs the **local path** set instead: existing =
|
||||
`FindMusicVideoPaths(libraryPath)` `.Except` the incoming items' replaced local paths, then hard-deletes the
|
||||
remainder with `DeleteByPath` + `IScannerProxy.RemoveMediaItems`, and cleans now-empty artists with
|
||||
`IArtistRepository.DeleteEmptyArtists`. Hard delete (not soft `FileNotFound` trash) because there is no
|
||||
per-item FileNotFound seam on this path and the issue's Done-when is "removed".
|
||||
- **Cross-kind safety is a property of the queries, not the media kind.** `MediaItem` is TPT with `LibraryPathId`
|
||||
on the abstract base, so a Movie, Show and MusicVideo can share one `LibraryPath` (a mixed Jellyfin library).
|
||||
Both `FindMusicVideoPaths` and `DeleteByPath` filter `LibraryPathId` **and** join the concrete `MusicVideo`
|
||||
table, so the sweep can only ever see/delete music videos — a Movie/Show under the same `LibraryPath` is
|
||||
invisible to it. Pinned by `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`.
|
||||
- **Reuses the #477 empty-fetch guard.** The sweep is gated by `MediaServerReconciliationGuard.ShouldFlagMissing`
|
||||
— a successful fetch that returns zero items (server mid-restore / transient) is indistinguishable from a real
|
||||
emptying, so the whole-library wipe is refused and logged. Pinned as a negative control by
|
||||
`ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items` (removing the guard flips it red).
|
||||
- **Known limitation (deferred to per-library identity).** `MusicVideoRepository.GetOrAdd` dedups a path
|
||||
**globally** (no `LibraryPathId` predicate), so a file served by two libraries with overlapping local paths is
|
||||
a single row owned by whichever library scanned it first. If that owner later stops reporting the file while
|
||||
another library still serves it, this sweep removes the shared row. A proper fix needs per-library music-video
|
||||
identity (a `JellyfinMusicVideo` etag entity + migration) — the issue's "option 2 / fold into the base
|
||||
scanner" refactor — tracked as #496.
|
||||
- **Tests.** Integration tests (real `ArtistRepository`/`MusicVideoRepository`/`LibraryRepository` over in-memory
|
||||
SQLite, extending the #488 harness) pin removal, empty-artist cleanup, cross-kind safety, and the empty-fetch
|
||||
guard. Proven non-vacuous: all four fail against the pre-fix scanner except the guard control, which only
|
||||
earns its keep once the sweep exists.
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
---
|
||||
key: scan.musicvideo-reconciliation
|
||||
title: 2026-07-20 — `JellyfinMusicVideoLibraryScanner` reconciles by library-scoped path diff + hard delete, not server itemId soft-trash (#494)
|
||||
status: superseded
|
||||
since: '2026-07-20'
|
||||
supersedes: none
|
||||
superseded-by: scan.musicvideo-server-identity@2026-07-25
|
||||
rule: (superseded) `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity.
|
||||
signals: 'music-video trash sweep, path-based identity, cross-kind safety, path-keyed identity, empty-fetch guard reuse, remove-stale+add-new dedup · paths: `JellyfinMusicVideoLibraryScanner.TrashMissingMusicVideos`, `FindMusicVideoPaths`/`DeleteByPath`, `IMusicVideoRepository`, `MediaServerReconciliationGuard` · issues: #494, #477, #488, #496, #500'
|
||||
mechanics: '`ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`; `ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items`; integration tests extending the #488 harness. #500 — when mirroring the remove-stale + add-new idiom, dedup the incoming set on **the same key its add filter compares** (the filter is materialized before the loop mutates `existing`, so duplicates both pass): `Name`, `Guid` for guids, and for Plex `Actors` an artwork-preferring dedup shared with the remove filter (whose key is `(Name, artwork-presence)`). Remaining un-deduped copies of the idiom: #600. Superseded by `scan.musicvideo-server-identity` (ersatztv#496): music videos gained a `JellyfinMusicVideo` ItemId/Etag identity, so the path diff + hard delete became an itemId diff + soft `FileNotFound` trash.'
|
||||
---
|
||||
|
||||
The Jellyfin music-video scanner did add/update only — a music video removed on the Jellyfin side lingered in
|
||||
ErsatzTV forever and could still be scheduled. It now runs a trash sweep at the end of `ScanLibrary`
|
||||
(`TrashMissingMusicVideos`), mirroring the `MediaServer{Movie,Television,OtherVideo}LibraryScanner` "gone
|
||||
upstream ⇒ remove" pattern but with a deliberately different identity function, because music videos lack the
|
||||
media-server identity those base scanners rely on.
|
||||
|
||||
- **Identity is (LibraryPathId, path), not server itemId.** The base scanners diff `GetExisting*` (keyed by
|
||||
`MediaServerItemId`) against the incoming server item ids, then soft-trash via `FlagFileNotFound`. Music videos
|
||||
have **no `JellyfinMusicVideo` entity and no `ItemId`/`Etag`** — the scanner is a standalone
|
||||
`IJellyfinMusicVideoLibraryScanner` that injects the *local* `IMusicVideoRepository`, which offers no
|
||||
itemId-keyed existing-set or flag seam. So the sweep diffs the **local path** set instead: existing =
|
||||
`FindMusicVideoPaths(libraryPath)` `.Except` the incoming items' replaced local paths, then hard-deletes the
|
||||
remainder with `DeleteByPath` + `IScannerProxy.RemoveMediaItems`, and cleans now-empty artists with
|
||||
`IArtistRepository.DeleteEmptyArtists`. Hard delete (not soft `FileNotFound` trash) because there is no
|
||||
per-item FileNotFound seam on this path and the issue's Done-when is "removed".
|
||||
- **Cross-kind safety is a property of the queries, not the media kind.** `MediaItem` is TPT with `LibraryPathId`
|
||||
on the abstract base, so a Movie, Show and MusicVideo can share one `LibraryPath` (a mixed Jellyfin library).
|
||||
Both `FindMusicVideoPaths` and `DeleteByPath` filter `LibraryPathId` **and** join the concrete `MusicVideo`
|
||||
table, so the sweep can only ever see/delete music videos — a Movie/Show under the same `LibraryPath` is
|
||||
invisible to it. Pinned by `ScanLibrary_Should_Not_CrossDelete_Movie_Or_Show_Sharing_The_LibraryPath`.
|
||||
- **Reuses the #477 empty-fetch guard.** The sweep is gated by `MediaServerReconciliationGuard.ShouldFlagMissing`
|
||||
— a successful fetch that returns zero items (server mid-restore / transient) is indistinguishable from a real
|
||||
emptying, so the whole-library wipe is refused and logged. Pinned as a negative control by
|
||||
`ScanLibrary_Should_Not_Sweep_When_Jellyfin_Returns_Zero_Items` (removing the guard flips it red).
|
||||
- **Known limitation (deferred to per-library identity).** `MusicVideoRepository.GetOrAdd` dedups a path
|
||||
**globally** (no `LibraryPathId` predicate), so a file served by two libraries with overlapping local paths is
|
||||
a single row owned by whichever library scanned it first. If that owner later stops reporting the file while
|
||||
another library still serves it, this sweep removes the shared row. A proper fix needs per-library music-video
|
||||
identity (a `JellyfinMusicVideo` etag entity + migration) — the issue's "option 2 / fold into the base
|
||||
scanner" refactor — tracked as #496.
|
||||
- **Tests.** Integration tests (real `ArtistRepository`/`MusicVideoRepository`/`LibraryRepository` over in-memory
|
||||
SQLite, extending the #488 harness) pin removal, empty-artist cleanup, cross-kind safety, and the empty-fetch
|
||||
guard. Proven non-vacuous: all four fail against the pre-fix scanner except the guard control, which only
|
||||
earns its keep once the sweep exists.
|
||||
@@ -7,10 +7,42 @@ in the active read-path). Active successor for the rule builder: `spa.rulebuilde
|
||||
|
||||
---
|
||||
|
||||
## Records formerly in this file
|
||||
## 2026-07-18 — SmartCollection rule builder: compile-only closed subset, no stored AST, one-level nesting (#176)
|
||||
`key: spa.smartcollection-rule-builder` · `status: superseded` · `since: 2026-07-18` · `supersedes: none` · `superseded-by: spa.rulebuilder-nesting@2026-07-25`
|
||||
**Rule:** (superseded) The SmartCollection visual rule builder compiles to/from a closed subset of the Lucene grammar over the existing stored query string — no new AST, one level of group nesting.
|
||||
**Signals:** SmartCollection, rule builder, Lucene compile/parse · paths: `web/src/builder/rules/`, `compile.ts`, `parse.ts`, `roundtrip.test.ts` · issues: #176, #69
|
||||
**Mechanics:** superseded by `spa.rulebuilder-nesting` (ersatztv#436) — the compile-only closed-subset and field-catalog stances carry forward there; only the one-level nesting cap was reversed. See `docs/decisions.md` → that record; `spa-conventions.md` §12
|
||||
|
||||
Each record below moved to its own file under `records/` (ersatztv#610); the rationale is
|
||||
unchanged. Resolve by **key** — that is the stable identity. A date-based pointer from
|
||||
another doc or an old issue comment should land here and then follow the link.
|
||||
The SmartCollection create/edit dialog gained a visual rule builder (`web/src/builder/rules/`)
|
||||
alongside the existing raw-Lucene textarea. **The SmartCollection still stores a plain Lucene query
|
||||
string — no new stored rule AST, no schema change.** The builder compiles its in-memory rule tree
|
||||
into a **closed subset** of the Lucene grammar (`compile.ts`) and parses exactly that subset back out
|
||||
(`parse.ts`, the exact inverse — returns `null`, not a best-effort guess, for anything outside the
|
||||
subset); escaping is total, so any builder-authored query round-trips losslessly, proven by a 500-tree
|
||||
property test (`roundtrip.test.ts`, including Lucene special characters). Opening an existing
|
||||
SmartCollection tries the parse first and falls back to raw-text mode on `null` (fuzzy queries,
|
||||
boosts, mixed AND/OR at one nesting level, or nesting deeper than one level).
|
||||
|
||||
- 2026-07-18 — SmartCollection rule builder: compile-only closed subset, no stored AST, one-level nesting (#176) — [`spa.smartcollection-rule-builder`](spa/smartcollection-rule-builder.md)
|
||||
**Why compile-only over persisting an authoritative rule AST**: an AST would still need a
|
||||
Lucene→rules parser to open every *pre-existing* free-text query — including every query the
|
||||
Auto-Tune feature (#69) generates — so the AST would buy almost nothing (it still can't represent
|
||||
arbitrary hand-written Lucene) while costing a dual-provider EF migration and a second source of
|
||||
truth to keep in sync with the Lucene grammar. Compile-only keeps the query string as the single
|
||||
source of truth and treats the builder as a structured *editor* over it, not a new storage model.
|
||||
|
||||
**One-level-nesting "Kodi" model.** `types.ts` defines a top `Group` (`match: all|any`) over `Rule`s
|
||||
and/or **one level** of sub-`Group`s — enough to express `type:movie AND (genre:Horror OR
|
||||
genre:Thriller)`, which covers the smart-playlist patterns Kodi-style rule builders are known for.
|
||||
Arbitrary/recursive nesting was scoped out as YAGNI; revisit only if a real query needs it.
|
||||
|
||||
**Field vocabulary comes from a new catalog endpoint, not a hardcoded list.** `GET
|
||||
/api/v1/search/fields` (read-only, MCP-introspectable; see `api-conventions.md`) returns the curated,
|
||||
typed, labeled field set derived from `LuceneSearchIndex` — name/label/type/group/values — and is the
|
||||
single source of truth the builder's field pickers (`fieldCatalog.ts`'s `useSearchFields` hook) and
|
||||
operator/value-input choices are driven from, so the builder's vocabulary can't drift from what the
|
||||
index actually supports.
|
||||
|
||||
**Deferred as separate follow-up issues** (explicitly out of scope for #176): facet-value typeahead
|
||||
for value inputs, relative-date operators, nesting deeper than one level, and inline adoption of
|
||||
`RuleBuilder` by ChannelBuilder / Auto-Tune (it was built reusable for exactly that reuse — see
|
||||
`spa-conventions.md` §12).
|
||||
|
||||
@@ -1,53 +0,0 @@
|
||||
---
|
||||
key: spa.list-completeness-vs-bounded-pickers
|
||||
title: '2026-07-26 — `loadAllPages` is for bounded-by-construction lists only; media-library pickers stay bounded and show truncation (#644 follow-up)'
|
||||
status: superseded
|
||||
since: '2026-07-26'
|
||||
supersedes: none
|
||||
superseded-by: spa.library-pickers-resolve-by-search@2026-07-26
|
||||
rule: '(superseded) The shared `loadAllPages` helper (`web/src/api/paging.ts`) pages a `/api/v1` list to completeness against `totalCount` and is used ONLY for lists that are bounded by construction (rerun collections, multi-collections, playlists — admin-created, hundreds of rows at most). A `getLibraryBrowseItems` picker over a media-library table (Episode/Song/Image/Movie/MusicVideo, tens of thousands of rows possible) must NOT page to completeness — it fetches ONE bounded page (the server cap) and surfaces the truncation (a `ctv-field-help` hint wired to the real `totalCount`) instead of silently dropping the rest.'
|
||||
signals: '`loadAllPages`, Class A vs Class B picker, LuceneSearchIndex.Search hitsLimit, picker truncation hint, ctv-field-help, PagedResult, `complete` flag · paths: `web/src/api/paging.ts`, `web/src/screens/RerunCollectionsScreen.tsx`, `web/src/screens/PlaylistsScreen.tsx`, `web/src/screens/FillerPresetsScreen.tsx`, `web/src/screens/MultiCollectionsScreen.tsx`, `docs/spa-conventions.md` §3b · issues: #644'
|
||||
mechanics: 'superseded by `spa.library-pickers-resolve-by-search` (ersatztv#651) — Class A (`loadAllPages` for bounded-by-construction lists) survives there unchanged; only the Class B rule is reversed. See `docs/spa-conventions.md` §3b'
|
||||
---
|
||||
|
||||
`fe342a6a` (#644) extracted the `loadAllPages` client-side paging helper and applied it at every
|
||||
call site that had been requesting an over-cap `pageSize` to "get everything in one call" — a
|
||||
pattern that silently truncated to the server's `MaxPageSize` (100) with no error and no
|
||||
truncation indicator. A cold adversarial review of that fix found it was correct for the
|
||||
admin-created lists (rerun collections, multi-collections, playlists — bounded by construction,
|
||||
hundreds of rows at most) but dangerous for three call sites: the `getLibraryBrowseItems` pickers
|
||||
in `RerunCollectionsScreen`, `PlaylistsScreen`, and `FillerPresetsScreen`, which populate a native
|
||||
`<select>` whose `mediaType` can be `Episode`, `Song`, `Image`, `Movie`, or `MusicVideo` — the
|
||||
largest tables in an install. Paging one of those to completeness means on the order of 200 serial
|
||||
requests against a 20,000-row library, each **more** expensive than the last (`LuceneSearchIndex
|
||||
.Search` computes `hitsLimit = skip + limit`, so later pages re-scan a growing prefix), ending in a
|
||||
`<select>` with 20,000 `<option>` nodes rendered into the DOM. That is worse than the defect #644
|
||||
set out to fix.
|
||||
|
||||
The fix keeps `loadAllPages` unchanged in behavior for the bounded lists (it now also reports a
|
||||
`complete: boolean` flag and accepts an `AbortSignal`, per the same follow-up review's F4/F2
|
||||
findings) and removes it entirely from the three media-library picker call sites. Those instead
|
||||
call `getLibraryBrowseItems` directly for a single page at the server cap (`pageSize: 100`) and
|
||||
read the response's `totalCount` to detect truncation. The defect named in #644's title is
|
||||
"*silently* truncate" — the silence is the bug, not the bound. So a truncated picker load renders a
|
||||
`ctv-field-help` hint next to the `<select>` (`Showing the first 100 of 5000 — use search to
|
||||
narrow.`) instead of either paging forever or truncating without saying so. A full
|
||||
typeahead/search-driven picker over the media library is a materially larger feature (a `query`
|
||||
param already exists on `getLibraryBrowseItems` for it) and is deliberately out of scope here — a
|
||||
follow-up issue, not this fix.
|
||||
|
||||
**2026-07-26 addendum (round-3 review F1):** `loadPickerOptions`'s `multi` branch (a Class A
|
||||
source — `MultiCollection`) reused the same `truncated: boolean` field as the Class B media-library
|
||||
pickers, but the two conditions are not the same thing: Class B's `truncated` means "there are more
|
||||
rows than fit in one page — narrow via search," while a Class A picker's flag meant "the
|
||||
`loadAllPages` loop didn't converge" (`complete: false`) — a defensive/incomplete load, not a cap.
|
||||
Rendering both through the shared "Showing the first N of M — use search to narrow" copy produced a
|
||||
self-contradictory "Showing the first 47 of 47" on an incomplete Class A load, pointing at a search
|
||||
box that picker doesn't have. `RerunCollectionsScreen.tsx`/`PlaylistsScreen.tsx` now return a
|
||||
`hint: 'incomplete' | 'none' | 'truncated'` discriminator instead of a boolean, and render distinct
|
||||
copy per value — `'truncated'` keeps the existing search-narrowing text, `'incomplete'` renders
|
||||
"List may be incomplete — retry to reload" (matching the wording already used for the Class A
|
||||
list-load warn `Badge`). A picker's `console.warn` on an incomplete load — and the analogous one in
|
||||
`SchedulesScreen.loadAllRerunCollections` — is also gated on `!signal?.aborted`, so a superseded or
|
||||
user-aborted load (Retry, or a type switch mid-load) no longer logs a false "did not complete"
|
||||
warning.
|
||||
@@ -1,45 +0,0 @@
|
||||
---
|
||||
key: spa.smartcollection-rule-builder
|
||||
title: '2026-07-18 — SmartCollection rule builder: compile-only closed subset, no stored AST, one-level nesting (#176)'
|
||||
status: superseded
|
||||
since: '2026-07-18'
|
||||
supersedes: none
|
||||
superseded-by: spa.rulebuilder-nesting@2026-07-25
|
||||
rule: (superseded) The SmartCollection visual rule builder compiles to/from a closed subset of the Lucene grammar over the existing stored query string — no new AST, one level of group nesting.
|
||||
signals: 'SmartCollection, rule builder, Lucene compile/parse · paths: `web/src/builder/rules/`, `compile.ts`, `parse.ts`, `roundtrip.test.ts` · issues: #176, #69'
|
||||
mechanics: superseded by `spa.rulebuilder-nesting` (ersatztv#436) — the compile-only closed-subset and field-catalog stances carry forward there; only the one-level nesting cap was reversed. See `docs/decisions.md` → that record; `spa-conventions.md` §12
|
||||
---
|
||||
|
||||
The SmartCollection create/edit dialog gained a visual rule builder (`web/src/builder/rules/`)
|
||||
alongside the existing raw-Lucene textarea. **The SmartCollection still stores a plain Lucene query
|
||||
string — no new stored rule AST, no schema change.** The builder compiles its in-memory rule tree
|
||||
into a **closed subset** of the Lucene grammar (`compile.ts`) and parses exactly that subset back out
|
||||
(`parse.ts`, the exact inverse — returns `null`, not a best-effort guess, for anything outside the
|
||||
subset); escaping is total, so any builder-authored query round-trips losslessly, proven by a 500-tree
|
||||
property test (`roundtrip.test.ts`, including Lucene special characters). Opening an existing
|
||||
SmartCollection tries the parse first and falls back to raw-text mode on `null` (fuzzy queries,
|
||||
boosts, mixed AND/OR at one nesting level, or nesting deeper than one level).
|
||||
|
||||
**Why compile-only over persisting an authoritative rule AST**: an AST would still need a
|
||||
Lucene→rules parser to open every *pre-existing* free-text query — including every query the
|
||||
Auto-Tune feature (#69) generates — so the AST would buy almost nothing (it still can't represent
|
||||
arbitrary hand-written Lucene) while costing a dual-provider EF migration and a second source of
|
||||
truth to keep in sync with the Lucene grammar. Compile-only keeps the query string as the single
|
||||
source of truth and treats the builder as a structured *editor* over it, not a new storage model.
|
||||
|
||||
**One-level-nesting "Kodi" model.** `types.ts` defines a top `Group` (`match: all|any`) over `Rule`s
|
||||
and/or **one level** of sub-`Group`s — enough to express `type:movie AND (genre:Horror OR
|
||||
genre:Thriller)`, which covers the smart-playlist patterns Kodi-style rule builders are known for.
|
||||
Arbitrary/recursive nesting was scoped out as YAGNI; revisit only if a real query needs it.
|
||||
|
||||
**Field vocabulary comes from a new catalog endpoint, not a hardcoded list.** `GET
|
||||
/api/v1/search/fields` (read-only, MCP-introspectable; see `api-conventions.md`) returns the curated,
|
||||
typed, labeled field set derived from `LuceneSearchIndex` — name/label/type/group/values — and is the
|
||||
single source of truth the builder's field pickers (`fieldCatalog.ts`'s `useSearchFields` hook) and
|
||||
operator/value-input choices are driven from, so the builder's vocabulary can't drift from what the
|
||||
index actually supports.
|
||||
|
||||
**Deferred as separate follow-up issues** (explicitly out of scope for #176): facet-value typeahead
|
||||
for value inputs, relative-date operators, nesting deeper than one level, and inline adoption of
|
||||
`RuleBuilder` by ChannelBuilder / Auto-Tune (it was built reusable for exactly that reuse — see
|
||||
`spa-conventions.md` §12).
|
||||
@@ -7,10 +7,34 @@ never in the active read-path). Active successor: `startup.parallel-orientation`
|
||||
|
||||
---
|
||||
|
||||
## Records formerly in this file
|
||||
## 2026-07-11 — Queue state lives in the pinned Gitea tracker (#237), not in the handoff file
|
||||
`key: docs.queue-state-gitea-tracker` · `status: superseded` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: startup.parallel-orientation@2026-07-21`
|
||||
**Rule:** (superseded) Volatile session/queue state lives in pinned Gitea tracker issue #237 (goal + arc in the body, append-only session-comment log, in-progress/review labels), not in a wholesale-rewritten handoff file; the handoff file keeps only the static kickoff prompt and append-only lore.
|
||||
**Signals:** last-writer-wins race, claim/`in-progress` label, triage verdict · paths: `docs/handoffs/chicorytv-issue-queue.md` · issues: #237, #520
|
||||
**Mechanics:** superseded by `scripts/select-queue.sh` (ersatztv#520); see `docs/decisions.md` → `startup.parallel-orientation`
|
||||
|
||||
Each record below moved to its own file under `records/` (ersatztv#610); the rationale is
|
||||
unchanged. Resolve by **key** — that is the stable identity. A date-based pointer from
|
||||
another doc or an old issue comment should land here and then follow the link.
|
||||
With multiple sessions/agents working the repo in parallel, the old protocol — every session
|
||||
wholesale-rewrites `docs/handoffs/chicorytv-issue-queue.md` on main (session state + queue +
|
||||
next-session prompt) — became a last-writer-wins race. New protocol: **volatile queue state
|
||||
moved to Gitea**, which is concurrency-safe by construction. Pinned tracker issue **#237**
|
||||
holds the goal + ordered arc in its body (edited rarely, only on arc changes, re-read before
|
||||
edit) and an append-only session-comment log (fixed template: Closed / Filed / Triage /
|
||||
Arc change / Recommended next). Milestone `Blazor removal (#91 phase b)` + the `review` and
|
||||
`in-progress` labels are the machine-queryable view. Sessions **claim** an issue before working
|
||||
it (`in-progress` label + claim comment; the tiny read→claim race window is accepted, later
|
||||
claimant backs off; stale claims — no commits/comments ~48h — may be taken over with a comment).
|
||||
Every new issue gets an explicit end-of-session triage verdict — gate-blocker (milestone + arc
|
||||
slot) or backlog (label only) — so review findings adjust the queue only through that step and
|
||||
the arc doesn't drift. The handoff file keeps only the **static kickoff prompt** and the
|
||||
**append-only Lessons lore** (per-session prompts are gone; task context lives in issue bodies).
|
||||
|
||||
- 2026-07-11 — Queue state lives in the pinned Gitea tracker (#237), not in the handoff file — [`docs.queue-state-gitea-tracker`](docs/queue-state-gitea-tracker.md)
|
||||
**Why superseded (#520, 2026-07-21):** the arc completed and #237 closed (2026-07-13); a closed
|
||||
tracker cannot serve as live queue state, and continuing to read it as such was a live regression
|
||||
risk (an agent skimming an old comment or this very record could re-treat #237's prose as current).
|
||||
`scripts/select-queue.sh` (2026-07-19) already replaced the mechanical parts of this rule with a
|
||||
deterministic, live-Gitea-only query — this record's job was really "queue state lives in Gitea,
|
||||
not in the handoff file," and that half is still true; what's superseded is the *specific store*
|
||||
(#237's body/comments) now that maintenance/backlog mode has no arc to narrate. See
|
||||
`startup.parallel-orientation` for the replacement: two concurrent session-start tracks
|
||||
(orientation via the docs/decisions catalog + `docs/README.md` map, and selection via the script),
|
||||
with #237 reduced to a single archival breadcrumb.
|
||||
|
||||
@@ -11,18 +11,308 @@ cross-editor ETag rotation). Refs #197.
|
||||
|
||||
## Contents
|
||||
|
||||
- [2026-07-11 — Optimistic-concurrency contract for replace-all PUTs (#253 PR1: infra + Block reference)](#2026-07-11--optimistic-concurrency-contract-for-replace-all-puts-253-pr1-infra--block-reference)
|
||||
- [2026-07-11 — #253 PR3: Diff + Scalar concurrency fan-out (Collection / Playout×2 / MultiCollection / RerunCollection)](#2026-07-11--253-pr3-diff--scalar-concurrency-fan-out-collection--playout2--multicollection--reruncollection)
|
||||
- [2026-07-11 — Stable child identity for schedule-item replace (#259, split from #252/#253)](#2026-07-11--stable-child-identity-for-schedule-item-replace-259-split-from-252253)
|
||||
- [2026-07-12 (#269 — non-If-Match root writers force-write past a concurrent Version bump)](#2026-07-12-269--non-if-match-root-writers-force-write-past-a-concurrent-version-bump)
|
||||
- [2026-07-12 — Cross-editor ETag rotation completed for Collection/Playout config siblings (#269)](#2026-07-12--cross-editor-etag-rotation-completed-for-collectionplayout-config-siblings-269)
|
||||
- [2026-07-12 — If-Match evaluates per RFC 7232: valid-but-non-matching → 412, only grammar violations → 400 (#265)](#2026-07-12--if-match-evaluates-per-rfc-7232-valid-but-non-matching--412-only-grammar-violations--400-265)
|
||||
- [2026-07-18 — Concurrent same-item add is idempotent, not a 500: catch the unique-violation per provider (#308)](#2026-07-18--concurrent-same-item-add-is-idempotent-not-a-500-catch-the-unique-violation-per-provider-308)
|
||||
|
||||
---
|
||||
|
||||
## Records formerly in this file
|
||||
## 2026-07-11 — Optimistic-concurrency contract for replace-all PUTs (#253 PR1: infra + Block reference)
|
||||
`key: concurrency.replace-all-contract` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** Replace-all aggregate PUTs carry a uniform plain `int Version` concurrency token (EF `.IsConcurrencyToken()`), checked pre-save and enforced by the EF UPDATE guard, returning 412 (not 409) on a stale `If-Match`.
|
||||
**Signals:** ETag, If-Match, Version token, 412 Precondition Failed · paths: `api-conventions.md` §7a, `ConcurrencyHeaders`, `ApiResults.ToErrorResult` · issues: #253, #197, #265, #259
|
||||
**Mechanics:** `docs/api-conventions.md` §7a; `SaveChangesWithConcurrencyGuard`
|
||||
|
||||
Each record below moved to its own file under `records/` (ersatztv#610); the rationale is
|
||||
unchanged. Resolve by **key** — that is the stable identity. A date-based pointer from
|
||||
another doc or an old issue comment should land here and then follow the link.
|
||||
Replace-all aggregate PUTs had **no** optimistic concurrency — a stale second tab silently overwrote a
|
||||
fresher edit (200, no signal) across ~10 aggregate surfaces. PR1 lands the shared contract on the Block
|
||||
reference aggregate; PRs 2–4 fan it out. The full ratified design + independent-review hardening is
|
||||
[#253#issuecomment-8472](http://192.168.1.95:3000/timothy/ersatztv/issues/253#issuecomment-8472);
|
||||
the mechanics live in `api-conventions.md` §7a. Decisions frozen here:
|
||||
|
||||
- 2026-07-11 — #253 PR3: Diff + Scalar concurrency fan-out (Collection / Playout×2 / MultiCollection / RerunCollection) — [`concurrency.diff-scalar-fanout`](records/concurrency/diff-scalar-fanout.md)
|
||||
- 2026-07-11 — Optimistic-concurrency contract for replace-all PUTs (#253 PR1: infra + Block reference) — [`concurrency.replace-all-contract`](records/concurrency/replace-all-contract.md)
|
||||
- 2026-07-11 — Stable child identity for schedule-item replace (#259, split from #252/#253) — [`concurrency.schedule-item-child-identity`](records/concurrency/schedule-item-child-identity.md)
|
||||
- 2026-07-12 (#269 — non-If-Match root writers force-write past a concurrent Version bump) — [`concurrency.force-write-non-ifmatch`](records/concurrency/force-write-non-ifmatch.md)
|
||||
- 2026-07-12 — Cross-editor ETag rotation completed for Collection/Playout config siblings (#269) — [`concurrency.etag-rotation-completion`](records/concurrency/etag-rotation-completion.md)
|
||||
- 2026-07-12 — If-Match evaluates per RFC 7232: valid-but-non-matching → 412, only grammar violations → 400 (#265) — [`concurrency.ifmatch-rfc7232`](records/concurrency/ifmatch-rfc7232.md)
|
||||
- 2026-07-18 — Concurrent same-item add is idempotent, not a 500: catch the unique-violation per provider (#308) — [`concurrency.idempotent-concurrent-add`](records/concurrency/idempotent-concurrent-add.md)
|
||||
- **Token = uniform plain `int Version`** on each root implementing `IVersionedAggregate`, EF-mapped
|
||||
`.IsConcurrencyToken()`, one dual-provider migration (`AddAggregateVersions`, `defaultValue: 0`). **Not**
|
||||
a reused `DateUpdated` (tick-collision, SQLite TEXT precision, couples UI cosmetics to correctness) and
|
||||
**not** a MySQL-native rowversion (portability over provider-native).
|
||||
- **412 Precondition Failed**, not 409 — 409 stays the §3a EntityLocker "build in progress" guard;
|
||||
distinct codes → distinct SPA UX. New `PreconditionFailedError : BaseError` → 412 in `ApiResults.ToErrorResult`.
|
||||
- **Pre-check AND EF token both required.** The handler pre-check (a standalone `Either` introduced AFTER
|
||||
the validation pipeline — never via `Apply`, which `Join()`-flattens the subtype to 422) gives a clean
|
||||
412; the unconditional `root.Version++` + `IsConcurrencyToken` UPDATE-guard + a `SaveChangesWithConcurrencyGuard`
|
||||
backstop closes the residual load→save TOCTOU (`DbUpdateConcurrencyException` → 412).
|
||||
- **Unconditional bump** (not "only when a child changed"): EF writes the root row only when a scalar
|
||||
differs, so a no-op PUT-back must still bump to fire the token and rotate every other client's ETag.
|
||||
- **Config-only aggregate boundary**: every mutating handler of a root's *editor-visible config state*
|
||||
bumps `Version` (incl. bulk `ExecuteUpdate/Delete` writers via `.SetProperty`); regenerated build output
|
||||
(playout items/history) is outside the token — neither bumped nor guarded.
|
||||
- **Header-only ETag**, strong tag of the decimal `Version`; parsed/emitted by `ConcurrencyHeaders`. The
|
||||
successful PUT returns the new ETag (else a same-tab second save 412s against its own write).
|
||||
- **Phasing**: Phase 1 (this arc) = a missing `If-Match` force-writes (zero breakage) while the SPA starts
|
||||
echoing; Phase 2 (a later PR) flips missing → **428** after every editor echoes and one release soaks.
|
||||
`If-Match: *` stays the scripted force-write escape hatch.
|
||||
- **Child stable-identity is OUT of #253** (the "moved fill-group item inherits the wrong slot's state"
|
||||
concern on the positional reconcile) — root-anchored versioning is orthogonal to it; split to **#259**.
|
||||
- **If-Match status semantics** (non-canonical/weak/list → 400) are fail-safe; the stricter RFC 7232
|
||||
"valid-but-non-matching → 412" refinement is deferred to #197 (**#265**).
|
||||
|
||||
## 2026-07-11 — #253 PR3: Diff + Scalar concurrency fan-out (Collection / Playout×2 / MultiCollection / RerunCollection)
|
||||
`key: concurrency.diff-scalar-fanout` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** The frozen Block optimistic-concurrency recipe (api-conventions §7a) fans out to Collection/Playout×2/MultiCollection/RerunCollection, keeping a guard-returned `PreconditionFailedError` out of any handler's generic `catch(Exception)`→422 mapping, and preserving each aggregate's existing `SaveChangesAsync() > 0` gate semantics under the new unconditional `Version++`.
|
||||
**Signals:** concurrency fan-out, PreconditionFailedError, SaveChangesWithConcurrencyGuard · paths: `api-conventions.md` §7a · issues: #253, #269, #232, #197
|
||||
**Mechanics:** `RootWriterForceVersionTests`-adjacent handler tests; `api-conventions.md` §7a
|
||||
|
||||
**Context.** PR3 of the #253 optimistic-concurrency arc fans the frozen Block recipe (api-conventions §7a)
|
||||
across the five Diff/Scalar aggregates. Three judgment calls beyond the mechanical copy:
|
||||
|
||||
**H1 — Playout `catch(Exception)`→422.** The two Playout replace handlers wrap `SaveChangesAsync` in a
|
||||
`catch(Exception)` that maps any exception to a bare `BaseError` (→ 422). Rather than let the guard's
|
||||
concurrency failure be reshaped into a 422, the guarded save (`SaveChangesWithConcurrencyGuard`) returns a
|
||||
`PreconditionFailedError` **Left as a value** and the handler returns it before the post-commit block —
|
||||
so it never reaches the catch. Proven by the pre-check-subtype tests (a `.Apply` flatten would fail
|
||||
`ShouldBeOfType<PreconditionFailedError>`) plus a non-vacuous Playout racing-save test.
|
||||
|
||||
**M2 — the `SaveChangesAsync() > 0` gates.** RerunCollection and Collection-custom-order run their
|
||||
playout-refresh **unconditionally** on a successful save (the unconditional `Version++` makes the old gate
|
||||
always-true; the "nothing changed" branch is dead). MultiCollection is the exception: it saved the name
|
||||
first specifically so a name-only change wouldn't rebuild playouts, so we bump `Version` on that **first**
|
||||
save and leave the **second** (items) save's `> 0` gate intact — a name-only edit still bumps + rotates the
|
||||
ETag but does not rebuild. Enumerating every behavior the gate provided before reworking it (the #232 lesson).
|
||||
|
||||
**Sibling-writer scope (deferred).** §7a's config-only boundary says every writer of an aggregate's
|
||||
editor-visible config bumps `Version`. PR3 ships the five primary endpoints' full contract + the one
|
||||
design-named bulk writer (`UpdateDefaultDecoHandler`, safe via `.SetProperty`). It **defers** the other
|
||||
same-root non-bulk config writers (`UpdateCollectionHandler`, `RemoveItemsFromCollectionHandler`,
|
||||
`UpdatePlayoutHandler`, the `ScheduleFile` handlers) and the repository-mediated `Add*ToCollection` family.
|
||||
Rationale: the primary endpoints' own bump+guard fully cover the two-tab lost-update the issue targets;
|
||||
the deferred writers only affect cross-editor ETag *rotation*, and adding an unconditional bump to a handler
|
||||
that uses plain `SaveChangesAsync` (not the guard) converts a latent lost-update into a **new 500**
|
||||
(`DbUpdateConcurrencyException`) — doing it safely needs a uniform guard+bump+412 pass of its own, better
|
||||
done with the #197 contract work. Tracked as a follow-up issue.
|
||||
|
||||
**VMs.** `Playout.Version` surfaces via `PlayoutNameViewModel` (required arg); the three collection VMs
|
||||
(`MediaCollectionViewModel`, `MultiCollectionViewModel`, `RerunCollectionViewModel`) carry `int Version = 0`
|
||||
(defaulted — 0 for the selection-placeholder constructions, real value from the Mapper projection).
|
||||
Header-only via ETag, never echoed in a response body (the Block precedent).
|
||||
|
||||
**Post-merge addendum (PR3 review, #269).** Activating the `Version` token means EF guards *every* root
|
||||
UPDATE, so non-participating root-scalar writers that use plain `SaveChangesAsync` (playout settings /
|
||||
schedule-file / on-demand-checkpoint, collection name) would 500 on a concurrent bump. The realistic
|
||||
UPDATE writers were fixed in-PR with `ConcurrencyExtensions.SaveChangesForcingVersion` (Phase-1
|
||||
force-write on conflict: adopt the stored token, retry, never revert the concurrent bump). The deferral
|
||||
above is re-scoped to the DELETE handlers + repository `Add*` writers only (→ #269).
|
||||
|
||||
## 2026-07-11 — Stable child identity for schedule-item replace (#259, split from #252/#253)
|
||||
`key: concurrency.schedule-item-child-identity` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** `PUT /api/schedules/{id}/items` reconciles by an optional round-tripped child `Id` (null/absent/0 ⇒ new item), never by array position, so fill-group/shuffle state follows the logical item across reorders; an unknown or duplicate id is rejected 422 (checked after the §7a `CheckVersion`, so 412 precedes 422).
|
||||
**Signals:** stable child identity, schedule-item replace, id-based reconcile · paths: `api-conventions.md` §7c · issues: #259, #252, #253, #197
|
||||
**Mechanics:** `docs/api-conventions.md` §7c
|
||||
|
||||
`PUT /api/schedules/{id}/items` now reconciles by an optional round-tripped child id, not by array
|
||||
position, so an item's persisted fill-group/shuffle state (`PlayoutScheduleItemFillGroupIndex`, FK
|
||||
`OnDelete(Cascade)`) follows the logical item across reorders/inserts instead of being inherited by
|
||||
whatever previously held its new slot. Contract + rules in **api-conventions §7c**. Key decisions:
|
||||
|
||||
- **`ScheduleItemRequest.Id` (`int?`)**: null/absent/`0` ⇒ new item (controller normalizes `0`→null so the
|
||||
handler is two-state). Any id present ⇒ id-based reconcile; a fully id-less payload keeps the verbatim
|
||||
positional fallback (legacy; retires with the §7a Phase-2 `If-Match`→428 flip).
|
||||
- **Unknown or duplicate id ⇒ 422, nothing persisted**; the guards live in the handler **after** §7a
|
||||
`CheckVersion`, so **412 precedes 422** — a client that is both version-stale and id-stale gets the reload
|
||||
signal, not a payload-bug signal. Rationale for reject-not-insert on an unknown id: under Phase-1
|
||||
force-write a stale id is a live lost-update signal, so silently inserting-as-new would duplicate the item
|
||||
and return a different id than the client sent (the exact class §7a exists to surface). This is also the
|
||||
correct #197 posture — never honor an unrecognized identifier.
|
||||
- **Scope = schedule items only.** Blocks/templates/deco-templates/playlists stay positional: their children
|
||||
are stateless config rows (no FK'd state to misattribute; #3/#4 have no GET child id). Child ids are added
|
||||
only where a child row anchors server-side state; the contract can be retrofitted per-endpoint later
|
||||
(field stays optional) — so this is not #197 ossification pressure.
|
||||
- **TPT subtype change at a matched id** stays delete+insert (EF can't retype in place); state resets and a
|
||||
new id is returned, so the SPA must re-seed item state from the PUT response (a stale id on a second save
|
||||
now 422s).
|
||||
|
||||
## 2026-07-12 (#269 — non-If-Match root writers force-write past a concurrent Version bump)
|
||||
`key: concurrency.force-write-non-ifmatch` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** Any handler that leaves a versioned root `Modified` or `Deleted` but takes no `If-Match` (deletes, item add/remove bumpers, scalar-config writers) must save through `ConcurrencyExtensions.SaveChangesForcingVersion` — force-write past a concurrent `Version` bump rather than throw an unhandled `DbUpdateConcurrencyException` (500).
|
||||
**Signals:** force-write, non-If-Match writers, DbUpdateConcurrencyException · paths: `ConcurrencyExtensions.SaveChangesForcingVersion` · issues: #269, #253, #302, #197
|
||||
**Mechanics:** `RootWriterForceVersionTests`; `docs/api-conventions.md` §7a
|
||||
|
||||
**Routing the aggregate delete handlers + `UpdateProgramScheduleHandler` through `SaveChangesForcingVersion`.**
|
||||
Once #253 made each replace-all root's `Version` an `IsConcurrencyToken`, EF started guarding *every*
|
||||
UPDATE **and DELETE** of that row with `WHERE Version=@orig` — so any writer that is not part of the
|
||||
If-Match contract but still saves via plain `SaveChangesAsync` throws an unhandled
|
||||
`DbUpdateConcurrencyException`→**500** if a replace-all editor bumps the row in its narrow load→save window.
|
||||
PR3 already force-wrote the exposed *UPDATE* siblings (Playout settings/`ScheduleFile`/checkpoint,
|
||||
`UpdateCollectionHandler`); a completeness sweep for #269 found the gap was wider than reported —
|
||||
**18 writers** in total, all on plain `SaveChangesAsync`. **The correct exposure filter is "any handler
|
||||
that leaves a versioned root `Modified` or `Deleted`", NOT just `Version`-bumpers + deletes** — an early
|
||||
sweep used the narrower filter and a review of PR #302 caught what it missed (`ErasePlayoutHistory` below):
|
||||
- the **nine versioned-root delete handlers** (`DeletePlayout`/`DeleteCollection`/`DeleteMultiCollection`/
|
||||
`DeleteRerunCollection`/`DeletePlaylist`/`DeleteBlock`/`DeleteTemplate`/`DeleteDecoTemplate`/
|
||||
`DeleteProgramSchedule`) — a DELETE is now token-guarded too;
|
||||
- `UpdateProgramScheduleHandler` (bumps `Version` then saved plainly — the ProgramSchedule case PR3 only
|
||||
*suspected*);
|
||||
- the **seven item add/remove bumpers** that PR2 wired to bump their root but left on plain save —
|
||||
`AddProgramScheduleItem`/`DeleteProgramScheduleItem` and the five
|
||||
`Add{Items,Movie,Show,Season,Episode}ToPlaylist` handlers;
|
||||
- **`ErasePlayoutHistoryHandler`** — modifies Playout root **scalars** (`Seed`/`Anchor`/`OnDemandCheckpoint`)
|
||||
**without** bumping `Version`, inside an explicit transaction with no try/catch → the one the bumper-only
|
||||
filter missed; reachable via `POST /api/playouts/{id}/erase-items-and-history`.
|
||||
|
||||
All now save through `ConcurrencyExtensions.SaveChangesForcingVersion`.
|
||||
|
||||
**Two deliberate boundaries (documented, not gaps):** (1) the background build/time-shift Playout-scalar
|
||||
writers (`BuildPlayoutHandler` via `PlayoutBuilder`'s `Anchor`/`Seed`; `PlayoutTimeShifter`'s
|
||||
`OnDemandCheckpoint`) are token-guarded too but **intentionally left on plain save** — they never surface a
|
||||
request-path 500 (`BuildPlayoutHandler` catches → a build-failure `BaseError`; `PlayoutTimeShifter` runs only
|
||||
via the background worker), and force-writing would be *wrong*: a concurrent config edit that bumped
|
||||
`Version` also enqueues a rebuild, so failing the in-flight build and letting the rebuild redo it with fresh
|
||||
config is correct (force-writing would persist output built from stale config). (2)
|
||||
Item-add force-write can leave a duplicate/gap `Index` (accepted Phase-1 effect): the handler computes the
|
||||
new index from its stale child list, so if a concurrent replace-all grew the list the item lands at a
|
||||
now-colliding index (no unique constraint on `PlaylistItem.Index`/`ProgramScheduleItem.Index`) — non-
|
||||
corrupting, self-correcting on the next edit, still strictly better than the pre-#269 500; a
|
||||
reload-and-recompute-on-conflict refinement is a candidate for #197. Decision:
|
||||
**force-write, not 412** — these endpoints take no `If-Match` (an unconditional DELETE/settings-edit should
|
||||
win over a concurrent editor), matching the Phase-1 force-write posture. A delete has no ETag to rotate, so
|
||||
it needs only the force-write, not a `Version` bump. A genuine row-deletion race (two concurrent deletes)
|
||||
still surfaces as a `DbUpdateConcurrencyException` — accepted (rare, non-corrupting, the resource is already
|
||||
gone). **Still deferred to #197:** *cross-editor ETag rotation* for the non-bumping config siblings and the
|
||||
scanner-shared `Add*ToCollection` family (they don't 500 — they insert children / `ExecuteDelete`, neither
|
||||
of which is token-guarded — they just don't rotate an open editor's ETag). Non-vacuously tested by racing a
|
||||
bump *through the handler* via a pre-tracked context (`RootWriterForceVersionTests`), plus an explicit
|
||||
negative control proving the plain-save path throws.
|
||||
|
||||
## 2026-07-12 — Cross-editor ETag rotation completed for Collection/Playout config siblings (#269)
|
||||
`key: concurrency.etag-rotation-completion` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** Every handler that mutates a versioned root's editor-visible config state must bump `Version` (rotating the ETag) with no per-aggregate carve-outs, short-circuiting on a genuine no-op before the bump so idempotent re-submits don't fire spurious rebuild fan-out; `SaveChangesForcingVersion` rebases the retry (stored + pending delta), never adopts the stored token verbatim.
|
||||
**Signals:** ETag rotation, no-op idempotence, SaveChangesForcingVersion rebase · paths: `CollectionEtagRotationTests`, `PlayoutScheduleFileEtagRotationTests` · issues: #269, #253, #197, #308
|
||||
**Mechanics:** `docs/api-conventions.md` §7a; `CollectionEtagRotationTests`, `PlayoutScheduleFileEtagRotationTests`
|
||||
|
||||
The #253 optimistic-concurrency contract (§7a) had a documented tail: the non-If-Match config-sibling
|
||||
writers of a versioned root mutated editor-visible state **without** bumping `Version`, so editing through
|
||||
them did not rotate an open editor's ETag (a cross-editor invalidation gap — never a lost-update or a 500,
|
||||
which the primary endpoints' bump+guard already cover). #269's first slice (PR #302) removed the 500 exposure
|
||||
by routing those writers through `SaveChangesForcingVersion`; this slice completes the **rotation**.
|
||||
|
||||
Handlers now bumping `Version` (all via `SaveChangesForcingVersion`, since they take no `If-Match` → a
|
||||
concurrent replace-all bump force-writes, never 412/500): the Collection `Add*ToCollection` family (11
|
||||
handlers) and `RemoveItemsFromCollectionHandler` bump `Collection.Version`; `UpdateCollectionHandler`
|
||||
(name/flag), `UpdatePlayoutHandler` (`DailyRebuildTime`), and the three `ScheduleFile` writers
|
||||
(`UpdateSequential`/`UpdateScripted`/`UpdateExternalJsonPlayout`) — which already force-wrote — now also bump.
|
||||
|
||||
Decisions frozen (ratified with Fable before implementation, feeding the #197 contract freeze):
|
||||
|
||||
- **Rotate on every editor-visible config change, no per-aggregate carve-outs.** §7a's config-only boundary
|
||||
("every mutating handler of a root's editor-visible config bumps `Version`") already held for Playlist
|
||||
`Add*`/schedule item writers; the Collection/Playout siblings were an inconsistency, not a judgment call. A
|
||||
membership add rotating an open custom-order editor's ETag (→ 412 → reload) is correct: its list is genuinely
|
||||
stale. Blast radius of the aggressive-but-safe rotation is a reload, never data loss.
|
||||
- **No-op idempotence — the trap Fable caught.** These handlers gate their reindex/`BuildPlayout` fan-out on
|
||||
`SaveChanges() > 0`. An *unconditional* bump makes that gate always-true, so an idempotent re-add / same-value
|
||||
re-submit would fire spurious rebuilds across every playout using the aggregate. Fix: short-circuit a genuine
|
||||
no-op **before** the bump — the Add handlers by an explicit membership check (which also fixes the latent
|
||||
duplicate-`CollectionItem` insert on a *sequential* re-add; two *concurrent* same-item adds can still both
|
||||
pass the check and the loser 500s on the composite-PK unique violation — `SaveChangesForcingVersion` catches
|
||||
only `DbUpdateConcurrencyException`, not `DbUpdateException`. That race is narrow and pre-existing, deferred
|
||||
to #308), the scalar writers (`UpdateCollection`, `UpdatePlayout`, the
|
||||
three `ScheduleFile` writers) by `ChangeTracker.HasChanges()`. A no-op neither bumps nor rebuilds nor rotates
|
||||
the ETag — which is itself correct (nothing changed).
|
||||
- **The `Add*ToCollection` family is not repository-mediated.** #269's original framing ("repository-mediated,
|
||||
shared with the scanner hot path") was wrong: `IMediaCollectionRepository` is read-only; each handler loads
|
||||
the `Collection` into its own `dbContext` and writes directly. So the rotation bump is a pure API-layer
|
||||
concern and the scanner's separate membership-write path is untouched — a background scan does **not** rotate
|
||||
the editor ETag (correct: background indexing is not an editor action).
|
||||
- **Force-write rebases the bump, never adopts the stored token verbatim (Codex review of this PR).**
|
||||
`SaveChangesForcingVersion` originally resolved a conflict by setting current=original=stored — which
|
||||
silently *discarded* a sibling's pending `Version++` when a versioned writer committed in its load→save
|
||||
window (sibling loads 1, bumps to pending 2, concurrent PUT commits 2 → retry wrote 2, so the concurrent
|
||||
writer's ETag "2" stayed valid and the rotation was lost under exactly the race it exists for). Fixed in
|
||||
this PR (it affects all 25 bumpers routed through the helper, including the pre-existing playlist/schedule
|
||||
ones): the retry now rebases — original = stored, current = stored + (pending current − pending original) —
|
||||
so a bumper lands at stored+1 and a non-bumper (delta 0, e.g. `ErasePlayoutHistory`) adopts stored unchanged.
|
||||
The race tests assert the post-race Version (3, not 2) and fail against the verbatim-adopt implementation.
|
||||
- **No new status codes.** These endpoints take no `If-Match` and force-write, so they never 412; no
|
||||
`[ProducesResponseType(...412...)]` and no OpenAPI regen (response types unchanged). Only §7a prose changes.
|
||||
|
||||
Tests: `CollectionEtagRotationTests` (rotation + no-op-without-bump-or-rebuild + force-write-past-concurrent-bump
|
||||
for Add/Remove/Update) and `PlayoutScheduleFileEtagRotationTests` (ScheduleFile rotation + no-op-without-refresh),
|
||||
the no-op guard proven non-vacuous by inverting the membership check. The `#265` RFC-7232 If-Match parser
|
||||
refinement (valid-but-non-matching/weak/list → 412 not 400) is a **separate** PR (disjoint surface: the shared
|
||||
parser + `CheckVersion`, not the handler saves). Refs #253 #269 #197 · `api-conventions.md` §7a.
|
||||
|
||||
## 2026-07-12 — If-Match evaluates per RFC 7232: valid-but-non-matching → 412, only grammar violations → 400 (#265)
|
||||
`key: concurrency.ifmatch-rfc7232` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** `ConcurrencyHeaders.ParseIfMatch` is a real RFC 7232 entity-tag/list parser: a syntactically-valid tag that doesn't strong-match (weak/empty/non-canonical/out-of-range/list) returns 412, and only a genuine grammar violation returns 400.
|
||||
**Signals:** RFC 7232, If-Match parsing, strong-tag matching · paths: `ConcurrencyHeaders`, `IfMatchCondition`, `VersionedAggregateExtensions.CheckVersion` · issues: #265, #253, #197
|
||||
**Mechanics:** `docs/api-conventions.md` §7a
|
||||
|
||||
Closing the last #253 concurrency-contract piece. `ConcurrencyHeaders.ParseIfMatch` previously classified
|
||||
**any** non-canonical/weak/list `If-Match` value as `Malformed → 400` (a deliberate fail-safe: reject rather
|
||||
than risk a stale write, deferred from the reference-aggregate PR). That was RFC-incorrect. Per **RFC 7232
|
||||
§3.1**, a syntactically-valid entity-tag that simply doesn't strong-match must return **412 Precondition
|
||||
Failed**, and **400** is reserved for a genuine grammar violation.
|
||||
|
||||
**What changed.** The parser is now a real RFC 7232 entity-tag/list parser (`If-Match = "*" / 1#entity-tag`).
|
||||
It **scans** the list (it does *not* `Split(',')` — a comma is a valid `etagc`, so it can appear inside a quoted
|
||||
opaque-tag: `"3,5"` is ONE tag, and a comma separates members only outside the quotes), trims only RFC OWS
|
||||
(SP/HTAB — not `string.Trim()`, which would strip NBSP and let `" * "` masquerade as the `*` force-write),
|
||||
validates each member as `[ "W/" ] DQUOTE *etagc DQUOTE`, and collects the versions of the **strong** members
|
||||
whose opaque text is the exact canonical decimal we emit. Outcomes:
|
||||
- **weak** (`W/"3"`), **empty** (`""`), **non-canonical** (`"03"`, `"3.0"`, `"+3"`), **out-of-range**
|
||||
(`"99999999999999999999"`) → valid tags that contribute no version → **412** (a `Version`-kind with an
|
||||
*empty* candidate set is a guaranteed no-match).
|
||||
- **list** (`"3", "5"`) → any strong member that matches proceeds; weak/non-canonical members drop out.
|
||||
- genuine grammar violations (unquoted `3`, SP inside the tag `" 3 "`, unterminated `"3`, `garbage`, a
|
||||
separator-only header) → **400**.
|
||||
|
||||
**Type reshape.** `IfMatchCondition.ExpectedVersion : Option<int>` → `ExpectedVersions : Option<Seq<int>>`
|
||||
(`None` = force-write; `Some(set)` = strong-match against the set, empty ⇒ always 412), and
|
||||
`VersionedAggregateExtensions.CheckVersion(Option<int>)` → `CheckVersion(Option<Seq<int>>)` = set membership.
|
||||
This threads through all 10 replace/update commands + handlers + request mappers + 9 controllers uniformly; no
|
||||
wire-contract change (400 and 412 were already declared on every PUT; the field is header-derived and internal,
|
||||
so no OpenAPI/DTO change).
|
||||
|
||||
*Why now, not #197:* it is the shared parser all replace-all PUTs copy, and the 412-vs-404 ordering the issue
|
||||
worried about was already correct (each handler loads/validates → 404 before `CheckVersion`). *Why safe:* the
|
||||
first-party SPA only ever echoes the single canonical strong tag we emit, so no shipped client changes behavior;
|
||||
the change only makes a hand-written/tooling `If-Match` get the RFC-correct status. Docs: `api-conventions.md`
|
||||
§7a. Refs #265 #253 #197.
|
||||
|
||||
## 2026-07-18 — Concurrent same-item add is idempotent, not a 500: catch the unique-violation per provider (#308)
|
||||
`key: concurrency.idempotent-concurrent-add` · `status: active` · `since: 2026-07-18` · `supersedes: none` · `superseded-by: none`
|
||||
**Rule:** A concurrent duplicate `Add*ToCollection` that loses the race on the composite-key unique constraint is treated as an idempotent no-op (skip the reindex/rebuild fan-out), not a 500 — detected via a provider-specific `TvContext.IsUniqueConstraintViolation` delegate defaulting to "no".
|
||||
**Signals:** idempotent add, unique-constraint violation, provider error classifier · paths: `TvContext.IsUniqueConstraintViolation`, `SqliteErrorClassifier`, `MySqlErrorClassifier` · issues: #308, #269, #253
|
||||
**Mechanics:** `docs/api-conventions.md` §7a ("Idempotent insert under concurrency"); `ConcurrencyExtensions.TrySaveChangesForcingVersion`
|
||||
|
||||
**Decision.** The `Add*ToCollection` family's membership pre-check (#269) is not atomic with the insert, so two
|
||||
*concurrent* adds of the same item both observe it absent and both stage the `CollectionItem` composite key; the
|
||||
loser's `SaveChangesForcingVersion` threw a unique/PK-violation `DbUpdateException` (SQLite error 19 / MySQL 1062)
|
||||
it did not catch → **500**. We now treat that loss as an **idempotent no-op**, not an error: the desired end state
|
||||
(the item is a member) already holds because the racing winner inserted it, rotated the ETag, and fanned out the
|
||||
rebuild.
|
||||
|
||||
**Mechanism.** A `bool`-returning sibling `ConcurrencyExtensions.TrySaveChangesForcingVersion` wraps
|
||||
`SaveChangesForcingVersion` and catches *only* a classified unique/PK violation, returning `false`. The 10
|
||||
single-item handlers return `Unit.Default` on `false` (skip the reindex/rebuild fan-out — the winner did it). The
|
||||
bulk `AddItemsToCollection` handler cannot no-op — that would silently drop the non-colliding items when a batch
|
||||
partially overlaps a concurrent add — so it **retries** on a fresh context against recomputed membership (bounded
|
||||
loop; the common no-collision path runs once).
|
||||
|
||||
**Provider seam.** Detection is provider-specific but the Application layer must not reference the provider
|
||||
packages, so it follows the existing `TvContext` static-provider-config idiom (`IsSqlite`, `LastInsertedRowId`): a
|
||||
settable `TvContext.IsUniqueConstraintViolation` delegate, pointed at `SqliteErrorClassifier` (extended codes 1555
|
||||
PK / 2067 UNIQUE) or `MySqlErrorClassifier` (`Number == 1062`) from `Startup.cs`, defaulting to a conservative
|
||||
"no" so an unwired provider never silently swallows a save failure. Chosen over DI to avoid threading a new
|
||||
service through 11 handlers, and because the provider discriminator already lives as a `TvContext` static.
|
||||
|
||||
**Scope boundary.** `Add*ToPlaylist` is deliberately **untouched**: `PlaylistItem` has its own identity PK and no
|
||||
unique index on `(PlaylistId, MediaItemId)` — a playlist may legitimately contain the same item more than once, so
|
||||
there is no constraint to violate.
|
||||
|
||||
**Tests.** A negative-control anchor proves the race genuinely throws a classified `DbUpdateException`; the fix's
|
||||
end-to-end handler tests reproduce a *real cross-connection* race via a shared-cache SQLite harness + a
|
||||
`SavingChanges` interceptor that inserts the conflicting row on another connection mid-save (the single-connection
|
||||
in-memory fixture cannot). Every fix-dependent test was verified to fail with the catch disabled. Mechanics:
|
||||
`api-conventions.md` §7a ("Idempotent insert under concurrency"). Refs #308 #269 #253.
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
---
|
||||
key: api.artwork-rooted-urls
|
||||
title: '2026-07-07 — API artwork contract: rooted URLs produced server-side'
|
||||
status: active
|
||||
since: '2026-07-07'
|
||||
supersedes: none
|
||||
superseded-by: none
|
||||
rule: API response DTOs return artwork as rooted, directly-usable URLs (plus passthrough for absolute/proxy URLs), never relative Blazor-convention paths.
|
||||
signals: 'no SPA `<base href>`, `ApiArtwork` helper · paths: `ErsatzTV.Core/Api/ApiArtwork.cs`, `ErsatzTV.Application/LibraryBrowse/Queries/GetLibraryBrowseItemsHandler.cs` · issues: none cited (PR #181, PR #183)'
|
||||
mechanics: '`ErsatzTV.Core/Api/ApiArtwork.cs`'
|
||||
---
|
||||
|
||||
API response DTOs return artwork as rooted, directly-usable URLs (`/artwork/posters/...`,
|
||||
`/artwork/thumbnails/...`, `/artwork/fanart/...`), plus passthrough for absolute `http(s)://` URLs
|
||||
and Jellyfin/Emby proxy variants. Established by PR #181
|
||||
(`ErsatzTV.Application/LibraryBrowse/Queries/GetLibraryBrowseItemsHandler.cs`, private `Artwork(...)`
|
||||
helper — comment: *"Returns a rooted, directly-usable artwork URL for the SPA's `<img src>`... the
|
||||
SPA [needs it pre-rooted]"*), then generalized into the reusable `ApiArtwork` helper
|
||||
(`ErsatzTV.Core/Api/ApiArtwork.cs`, PR #183). Root cause: the SPA has no `<base href>`, unlike
|
||||
Blazor, so relative artwork paths that worked for Blazor pages 404 in the SPA. Do **not** reuse the
|
||||
Application-layer Mappers used by Blazor (e.g. `MediaCards`/`Television` mappers) for new API
|
||||
DTOs — those still return old Blazor-convention relative paths; map from the domain/VM directly and
|
||||
root the path via `ApiArtwork`.
|
||||
@@ -1,64 +0,0 @@
|
||||
---
|
||||
key: api.async-op-contract
|
||||
title: 2026-07-11 — Async-op API contract normalization + playout build observability + F9 scan endpoints (#235)
|
||||
status: active
|
||||
since: '2026-07-11'
|
||||
supersedes: none
|
||||
superseded-by: none
|
||||
rule: Queue-triggering `/api/*` endpoints normalize onto one contract — 202 Accepted (queued), 404 (missing entity), 409 (lock held), 422 (domain precondition) — with Trakt as the reference implementation; playout list/detail GETs also carry an `isLocked` observability flag as the HTTP-observable substitute for a live push channel.
|
||||
signals: '`QueueShowScanResult`, `ResetAllPlayoutsResponseModel`, `MaintenanceController.EmptyTrash`/`CleanArtwork` · paths: `LibrariesController.ScanShow`, `ChannelController.ResetPlayout`, `PlayoutController.ResetAll` · issues: #235, adversarial-reviewer#20 F7/F8/F9, #232, #215'
|
||||
mechanics: '`docs/api-conventions.md` §3a/§3b'
|
||||
---
|
||||
|
||||
Reviewer#20 F7/F8/F9. Normalizes the queue-triggering `/api/*` endpoints onto one contract, closes the two
|
||||
F9 `Libraries.razor` parity gaps, and hardens the Trakt batch-lock lifecycle. Much of the F8 surface was
|
||||
**already normalized** by #232 (library scan → `QueueLibraryScanResult` 202/404/409/422) and #215 (per-id
|
||||
playout mutations + reset → 409 lock guard) — this issue finished the remaining outliers.
|
||||
|
||||
**Normalized async-op contract** (queue-triggering endpoints): **202 Accepted** = work queued; **404
|
||||
ProblemDetails** = entity missing (controller pre-check); **409 ProblemDetails** = lock held (the running
|
||||
job, or a mutation racing it — §3a/§3b); **422 ProblemDetails** = domain precondition (sync disabled /
|
||||
unsupported / start failed). Trakt was the reference implementation. Changes made:
|
||||
- `MaintenanceController.EmptyTrash` — error path **500 text/plain → 404/422 ProblemDetails** (`ToErrorResult`).
|
||||
- `MaintenanceController.CleanArtwork` — silent **200 → 202** (fire-and-forget enqueue). No SPA consumer.
|
||||
- `LibrariesController.ScanShow` — conflated **400 `{error}` → 202/404/409/422** via a new
|
||||
`QueueShowScanResult` enum (6 outcomes incl. an honest `ScanFailed`→422, distinct from `Unsupported`).
|
||||
- `ChannelController.ResetPlayout` — **200 → 202** (queue-triggering; 404/409 unchanged).
|
||||
- `PlayoutController.ResetAll` — **202 (no body) → 202 + `ResetAllPlayoutsResponseModel`** reporting
|
||||
`queuedPlayoutIds` / `skippedLocked` / `skippedUnsupported` (replaces the silent skip; still 202, still
|
||||
skips locked/ExternalJson by design per §3a — now it *reports* what it skipped).
|
||||
- `TroubleshootController.TroubleshootPlayback` — bare body-less `NotFound()` → **404/422 ProblemDetails**
|
||||
with distinguishing detail. **Status codes the SPA HLS player depends on were preserved** — verified
|
||||
`HlsPlayer.tsx` never branches on this endpoint's status (playback state comes from the separate
|
||||
`/api/troubleshoot/playback/status` poll); only the error *body* was enriched.
|
||||
|
||||
**Playout build observability**: the list endpoint (`GET /api/playouts`) already stamped `isLocked` +
|
||||
`BuildStatus` on `PlayoutListItemResponseModel` (#215); this issue adds **`isLocked` to the single-playout
|
||||
`GET /api/playouts/{id}`** (`PlayoutResponseModel`), so the detail poll surface carries the §3a lock flag
|
||||
too. No dedicated `GET /api/playouts/{id}/status` push channel was added — the flag on the existing GETs is
|
||||
the HTTP-observable substitute for Blazor's live lock event, matching the `GET /api/trakt/status` precedent.
|
||||
|
||||
**F9 parity endpoints** (the `Libraries.razor` deletion gate — #202 did NOT close these):
|
||||
- **Deep scan**: `POST /api/libraries/{id}/scan` gains `?deep=false`, threaded through
|
||||
`QueueLibraryScanByLibraryId(LibraryId, DeepScan=false)` into `ForceSynchronize{Plex,Jellyfin,Emby}LibraryById(id, deep)`
|
||||
(was hardcoded `false`). Non-breaking: existing callers omit it.
|
||||
- **External-collections scan**: new `POST /api/media-sources/{plex|jellyfin|emby}/{id}/scan-collections?deep=false`
|
||||
on the three #202 media-source controllers, dispatching `Synchronize{X}Collections(id, ForceScan:true, deep)`.
|
||||
Each pre-checks source existence (404), acquires the per-source **collections** lock (`Lock{X}Collections()` —
|
||||
the lock *is* the running scan, so a false = **409**), then enqueues and returns 202; the controller
|
||||
compensating-unlocks in a `catch` if the enqueue throws (§3b), and `ScannerService` releases in its `finally`.
|
||||
Thin SPA clients shipped (`scanLibrary(id, deep)`, `scanCollections`); **the SPA deep-scan / collections
|
||||
buttons are the removal PR's remaining parity work** (parity doc §5).
|
||||
|
||||
**F7 Trakt batch-lock leak fix**: the global Trakt lock was released only when the *terminal* batch message
|
||||
(`Unlock: true`) was processed; a `WorkerService` shutdown/cancellation before that message leaked the lock
|
||||
permanently (subsequent Trakt ops 409 until restart — same class as #231/#233/#234). Fix: `WorkerService`
|
||||
now releases the Trakt lock in a `finally` on read-loop exit if still held. Non-vacuous regression test proven
|
||||
against an inverted-condition control.
|
||||
|
||||
**Accepted-by-design** (per the issue's decision-record ask): the worker's channels are **unbounded** and
|
||||
there is **no shutdown drain** — messages still queued at process exit are dropped. This is acceptable because
|
||||
the entity locks are **in-memory singletons that die with the process**, so a dropped message can't strand a
|
||||
lock across restarts (the F7 `finally` covers the *within-process* shutdown-break leak, which is the only way
|
||||
a lock outlives its batch while the process keeps running). Adding a bounded-channel backpressure / graceful
|
||||
drain is out of scope and would not fix a correctness bug.
|
||||
@@ -1,21 +0,0 @@
|
||||
---
|
||||
key: api.channel-health-object
|
||||
title: 2026-07-23 — Channel health = a server-derived `health` object on the channel DTOs, built-timeline detection (#415)
|
||||
status: active
|
||||
since: '2026-07-23'
|
||||
supersedes: api.channel-health-signal@2026-07-17
|
||||
superseded-by: none
|
||||
rule: '`ChannelResponseModel`/`ChannelDetailResponseModel` carry a server-derived `health` object (`ChannelHealthResponseModel { Status, Faults[], PlayoutCount, BrokenSourceItemCount }`) computed **read-time** from the built timeline (`Playout.BuildStatus` + upcoming `PlayoutItem → MediaItem.State`, `Finish >= now`), kind-agnostic across all 5 `PlayoutScheduleKind` values; `Status`/`Faults` are const-string classes (`ChannelHealthStatus`, `ChannelFault`), not C# enums, so the SPA hand-maintains the union (mirrors `ChannelPreviewAvailability`). This supersedes #72''s "raw fact only, no derived enum, empty-schedule/broken-source deliberately not computed" stance now that the auto-tune taxonomy churn (#383/#384) it was waiting on has landed (see `channel.origin-marker` sibling record, #414).'
|
||||
signals: 'channel health, ChannelHealthResponseModel, ChannelHealthStatus, ChannelFault, Healthy/Problems/Unknown, NoPlayout/NeverBuilt/BuildFailed/EmptyUpcoming/BrokenSource, built-timeline detection, BuildStatus, PlayoutItem MediaItem.State FileNotFound Unavailable, assessable gate, on-demand suppresses absence signals, Problems rollup filter, willNeverPlay hasProblems · paths: `ErsatzTV.Core/Api/Channels/ChannelHealthResponseModel.cs`, `ChannelRepository.GetAll`, `Mapper.GetHealth`, `GetAllChannelsForApiHandler`, `web/src/screens/ChannelsScreen.tsx`, `api-conventions.md`, `domain-model.md`, `spa-conventions.md` · issues: #415, #72, #71, #383, #384, #414'
|
||||
mechanics: '`docs/superpowers/specs/2026-07-23-channel-fault-detection-design.md` (full design); `api-conventions.md` (health object shape); `domain-model.md` (channel-health row); `spa-conventions.md` (Problems filter + badge convention)'
|
||||
---
|
||||
|
||||
#415 was deferred from #72 scope item (b): "empty schedule" and "broken/missing source" faults were real but uncomputed, each explicitly ruled out in the superseded record for a stated reason. This record reverses both rulings now that the blocking condition — the #383/#384 auto-tune status taxonomy churning the DTO shape — has resolved (#414 landed the origin column as a sibling, non-health field).
|
||||
|
||||
**Built-timeline (kind-agnostic) detection, not per-kind config introspection.** Every fault falls out of what the scheduler has already materialized — `Playout.BuildStatus` (`{LastBuild, Success, Message}`) for never-built/build-failed, and `Playout.Items` (the built `PlayoutItem` timeline, each carrying `MediaItemId`/`MediaItem`) for empty-upcoming and broken-source. Because the timeline is the same shape for all five `PlayoutScheduleKind` values (Classic, Block, Sequential, Scripted, ExternalJson), coverage is *by construction* — the #71 "verify a shared primitive covers ALL variants" trap, which the superseded record's own `EmptyScheduleHealthCheck` (Classic-only) fell into, cannot bite here. Scripted, which has no schedule entity to introspect at all, needs no special case. `MediaItem.State` flips on scan (not build), which rules out a build-time snapshot — detection is necessarily read-time, costed as one bounded `GROUP BY PlayoutId` aggregate query (not an N+1) over upcoming `PlayoutItem`s.
|
||||
|
||||
**Five-fault taxonomy, rolled up to one `status`.** `NoPlayout` (0 playouts, the absorbed #72 fact), `NeverBuilt` (assessable playout never built), `BuildFailed` (last build `Success == false`), `EmptyUpcoming` (built OK, 0 upcoming items), `BrokenSource` (≥1 upcoming item pointing at a `FileNotFound`/`Unavailable` `MediaItem`). Rollup: `Problems` if any contributing playout has a fault, `Healthy` if any is assessable-and-clean with none, `Unknown` if nothing is assessable — never a false `Healthy` and never a false `Problems`.
|
||||
|
||||
**The assessable gate distinguishes absence signals from presence signals.** `NeverBuilt`/`EmptyUpcoming` are inferred from *missing* content and are suppressed for `PlayoutMode == OnDemand` (an idle on-demand playout legitimately has no fresh build and drains its timeline between tune-ins — without suppression this is a false-positive storm across every on-demand channel; a suppressed absence signal contributes `Unknown`, not a false `Problems`). `BuildFailed`/`BrokenSource` are proven by content that *is* there and is bad, so they stay live in every `PlayoutMode` — they only fire when the bad thing actually exists and so cannot false-positive on legitimate idleness.
|
||||
|
||||
**Server owns the rollup so SPA and MCP read one verdict.** `health` rides the same `list channels`/`get channel` response both clients already fetch — no second endpoint to correlate by id, and no client re-deriving policy from raw facts (the thing the superseded record explicitly avoided freezing before the taxonomy existed). `PlayoutCount` is retained unchanged on the DTO for backward compatibility (additive-only `/api/v1` freeze); the SPA's "Problems" filter (`web/src/screens/ChannelsScreen.tsx`, `hasProblems`, replacing the old single-fault `willNeverPlay`/"No playout" filter) and per-row badges read `health.status`/`health.faults` instead.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user