Compare commits

..
Author SHA1 Message Date
timothy 1ef581403c fix(609): close round-5 test gaps and a prose misattribution
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 19s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m23s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 29s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m20s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m10s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21m57s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Fable's round-5 review ran the 16 tmp_path tests that no prior round could
execute (16/16 pass) and mutation-tested every fix. Six of seven reverts were
killed; one survived, which is finding 1.

1. The exact-arity refusal in `_token_armed` had ZERO coverage -- deleting it
   passed all 55 tests, because nothing fed malformed git-log output to that
   function. Now pinned by a test that stubs `_run` with 2-field and 4-field
   output and asserts refusal, plus a 3-field control proving the refusal is
   about arity rather than the token. Verified the new test kills the mutation.

2. `test_integration_separator_in_subject_cannot_inject` did not actually pin
   the NUL framing: with `\x1f` framing restored and the arity check kept, it
   still passed, because one or two injected separators break arity and get
   absorbed. Added a case with THREE separators, which restores a multiple-of-3
   arity and would false-arm under that revert -- so it pins the framing itself.

3. The decision record attributed the "old git echoes the trailers atom" case to
   the arity check. Wrong: an echoed atom is one well-formed field, so arity
   cannot catch it -- that case is handled by the `git --version` capability
   probe. Corrected in the record.

Not changed: review also noted subject matching is now case-insensitive, so
`[DECISIONS-EDIT]` arms where the old substring check was case-sensitive.
Deliberate and harmless -- arming still requires typing the token.
2026-07-25 18:25:26 +02:00
timothy 990d32f31a fix(609): anchor and bound the git version probe
Both round-4 findings, both in the version regex introduced in round 3. Both
reproduced against the old code and confirmed closed.

1. The regex was UNANCHORED, so the first dotted number anywhere in the output
   won. `wrapper 2026.1; git version 2.20.1` read as 2026.1 -> True, enabling
   trailer parsing on a git that cannot expand the atom, whose verbatim echo then
   reads as a non-empty trailer and FALSELY ARMS. Now anchored to the canonical
   `git version X.Y` prefix.

2. Digits were unbounded, so a pathological version string raised ValueError
   instead of returning the documented safe False -- Python refuses int()
   conversion of a literal over 4300 digits. Digits are now bounded to 5 each,
   plus a try/except that the bounded regex should make unreachable.

Adds seven probe cases: the wrapper-prefix and multiline-shim strings, Apple git,
an rc suffix, a three-digit major, and the 5000-digit pathological input.
2026-07-25 18:25:26 +02:00
timothy 851dca2596 fix(609): close round-3 review findings
All four LOW; no HIGH remained. The subject_of fix from round 2 was confirmed
correct across every message shape and all 38 historical commits.

1. The old-git compat check was a VALUE sentinel: it blanked any trailer whose
   value happened to equal the atom string, so a legitimate
   `Decisions-Edit: %(trailers:key=Decisions-Edit,valueonly)` was silently
   discarded. Replaced with a capability probe on `git --version` (>= 2.22).
   Detecting by version instead of by sniffing output removes the collision
   class entirely rather than narrowing it. Unknown/unparseable version resolves
   to False -- trailers ignored, subject-only matching -- which is the safe
   direction: a trailer-only token not arming is an annoyance, whereas reading an
   unexpanded atom as a value would falsely arm and disable the guard.

2. The compat test never called `_token_armed`, so it pinned nothing -- deleting
   the guard would have left it green. Replaced with three tests that drive the
   real function through a stubbed `_run`, covering old git (trailers ignored),
   modern git (trailer arms), a tokened subject surviving an unusable trailer,
   and version-string parsing incl. unparseable input. Proven non-vacuous:
   forcing the probe True makes the old-git test fail.

3. `_repo()` still ignored return codes from init/config/base-commit and never
   checked that the base sha resolved, so a rejected base could leave it
   returning ("", <root sha>) and negative range tests would pass vacuously. All
   commands are now checked and the base sha is asserted to be a full 40 chars.

4. docs/decisions.md line 65 still said "append it, as every prior use does".
   37 of 38 append; docs(434) is mid-subject.
2026-07-25 18:25:26 +02:00
timothy aa4a8fb849 fix(609): close round-2 review findings
HIGH -- `subject_of` used `lstrip("\n")`, so it returned the first NON-EMPTY
line. `git commit --cleanup=verbatim` accepts a message that begins with a blank
line and `%B` returns it raw, so body prose on line 2 was promoted to "subject"
and armed the token. Now literally line 1: an empty first line yields "", which
arms nothing -- failing toward the guard running.

LOW -- the old-git compat guard was a PREFIX match (`startswith("%(trailers")`)
that also `continue`d before the subject was evaluated. So a legitimate trailer
value beginning with that text was discarded, and worse, a perfectly good tokened
SUBJECT was thrown away because of its trailer field. Now an exact match against
the full atom, neutralising only the trailer and leaving the subject honoured.

LOW -- docstrings still said every historical use "appends" the token. Of the 38
uses, 37 append and `docs(434)` is mid-subject.

LOW (plausible) -- the `_repo` test helper ignored every git return code, so a
rejected commit would leave HEAD at base and every negative assertion would pass
vacuously. Return codes are now checked and HEAD is asserted to have moved.

Adds a regression test for the verbatim leading-blank-line case and one pinning
the compat guard to an exact atom match.
2026-07-25 18:25:26 +02:00
timothy 243bec708d fix(609): close two false-arm holes found in cross-family review
Codex review of the first attempt found both, and both were in the git plumbing
that my unit tests never touched -- they only exercised the pure predicate.

1. HIGH: git's `%s` is the first PARAGRAPH, not the first line. It joins
   consecutive non-blank lines with spaces, so
     `fix: harmless subject`
     `This explains [decisions-edit] on line two.`
   came back as ONE line containing the token and armed it -- the exact
   false-arm this change exists to prevent. The first line is now taken from
   `%B` via `subject_of()`.

2. HIGH: the in-band `\x1f`/`\x1e` field separators were injectable. A subject
   containing a literal `\x1f` was split at the wrong place and its tail read as
   a trailer, arming the token. Framing is now NUL, which git forbids inside a
   commit message and which therefore cannot be injected, with exact-arity
   parsing (fields must be a multiple of three) that refuses to arm otherwise.

Also from the same review:
- Refuse to arm on a `%(trailers:...)` atom echoed literally by a git older than
  2.22, which would otherwise read as a non-empty trailer (exit 0, so `_run`
  returns it rather than None).
- Record corrected: 38 subject-tokened commits in ancestry, not "twenty"; and it
  no longer claims a blanket fail-safe -- `_token_armed` failing is safe, but the
  surrounding `_diff_findings` fails open earlier on an unresolvable merge-base,
  skipping every check. That predates this change.

Adds 8 integration tests that drive `_token_armed` against a real throwaway git
repo -- the gap that let both defects pass. Verified non-vacuous by
reconstructing the old implementation in memory: it arms on both inputs, the new
one does not.

Note `--format` uses git's `%x00` escape, not a literal NUL: a NUL in argv raises
ValueError from subprocess, which broke every diff-engine test until fixed.
2026-07-25 18:25:26 +02:00
timothy 4596603020 fix(609): scope the decisions edit token to the subject line or a trailer
The token was armed by a bare substring match over every commit message in the
range, so a commit that merely DESCRIBED the mechanism armed it and skipped the
entire `if not token:` block -- all three rationale-rewrite comparisons (active
survivors, active->archive laundering, archive survivors). `removed` and `demoted`
still ran, so the job printed `decisions-validate: OK` while doing nothing. It
bit in PR#605, which had hand-resolved an append-vs-append conflict inside
docs/decisions.md -- precisely the operation the guard exists to police.

Now recognized in exactly two places:
  * the commit SUBJECT line -- the established form. All twenty prior tokened
    commits append it to the subject (or place it mid-subject, as docs(434)
    does); none put it on its own line, so the obvious "own-line only" rule
    would have broken every historical use.
  * a `Decisions-Edit: <reason>` git trailer -- the forward-looking form, which
    can carry a reason the bracketed marker cannot.

Fail-open posture unchanged: unresolvable git means the token reads unarmed, so
the guard still runs.

Verified by measuring the guard rather than reading a green check -- a positive
control over the real corpus across all three placements: no token fires (exit 1),
subject token suppresses (exit 0), body-only mention fires (exit 1). Plus an
end-to-end matcher test against a throwaway git repo covering the established
form, mid-subject placement, the trailer, a merge commit quoting a tokened PR
title, a multi-commit range, and an unresolvable ref.

fixes #609
2026-07-25 18:25:26 +02:00
575 changed files with 8084 additions and 106452 deletions
+1 -8
View File
@@ -1,16 +1,9 @@
#!/usr/bin/env bash
# ersatztv#521 — the line-level append-only mechanic is retired. Decision integrity is now enforced by
# the lifecycle validator. A `Decisions-Edit: yes` trailer survives ONLY for rationale-prose edits (validator
# the lifecycle validator. `[decisions-edit]` survives ONLY for rationale-prose edits (validator
# body-diff, CI). This shim runs the structural validator over the working tree; the body-diff/no-
# vanish checks run in CI where a base/head is available. Fail-open on any tooling trouble.
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# git hook: decides by exit code, and its stdout is live progress text.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin decisions-guard "" stream || true
cd "$(git rev-parse --show-toplevel)" || exit 0
command -v python3 >/dev/null 2>&1 || exit 0 # no python -> fail-open
PYTHONPATH=. python3 scripts/decisions_validate.py
-7
View File
@@ -17,13 +17,6 @@
# This is a reminder, never a hard gate — `start` only injects context; `finish` is a one-shot Stop nudge.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin design-sync-reminder "${1:-}" capture || true
UI_RE='(^|/)web/src/.*\.(tsx|css)$'
TEST_RE='\.test\.(tsx|ts)$'
@@ -4,13 +4,6 @@
# a sibling worktree another session created apart from this session's own.
# Fail-safe: any parse trouble → do nothing (the guard stays fail-open without a marker).
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin posttooluse-worktree-marker "" capture || true
input=$(cat)
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
cwd=$(printf '%s' "$input" | jq -r '.cwd // ""' 2>/dev/null || true)
@@ -15,13 +15,6 @@
# no origin/main, HEAD unresolved -> allow. Deliberate escape: ETV_ALLOW_DIRTY_PUSH=1.
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# git hook: decides by exit code, and its stdout is live progress text.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin prepush-clean-worktree-check "" stream || true
[ "${ETV_ALLOW_DIRTY_PUSH:-}" = "1" ] && exit 0
git rev-parse --git-dir >/dev/null 2>&1 || exit 0
-7
View File
@@ -12,13 +12,6 @@
# Auth (never committed): ETV_GITEA_TOKEN or ETV_GITEA_BASICAUTH; ETV_GITEA_URL overrides the base.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# git hook: decides by exit code, and its stdout is live progress text.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin prepush-donewhen "" stream || true
# git passes "<localref> <localsha> <remoteref> <remotesha>" lines on stdin.
refs=$(cat || true)
printf '%s\n' "$refs" | grep -q 'refs/heads/main' || exit 0 # only gate pushes to main
-39
View File
@@ -9,48 +9,9 @@
# a positively-proven "behind origin/main". Deliberate exception: ETV_SKIP_REBASE_CHECK=1.
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# git hook: decides by exit code, and its stdout is live progress text.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin prepush-rebase-check "" stream || true
[ "${ETV_SKIP_REBASE_CHECK:-}" = "1" ] && exit 0
git rev-parse --git-dir >/dev/null 2>&1 || exit 0
# Tag-only push exemption (ersatztv#719): the release cut tags a commit on main while the local
# branch sits 1 commit behind origin/main, so H11 blocked EVERY release -- and its "rebase first"
# advice did not even apply, since no branch was being pushed. A tag push cannot revert anyone's
# merged work, which is the failure mode H11 exists to prevent, so skip the freshness check when
# EVERY ref being pushed is under refs/tags/. (See #719 for the observed flow.)
#
# Read pushed refs from stdin: git feeds pre-push hooks one line per ref, "<local ref> <local sha>
# <remote ref> <remote sha>" (.husky/pre-push forwards the lines it already captured). Ignore blank
# lines. VACUOUS-TRUTH GUARD: "all refs are tags" is trivially true when there are zero ref lines
# (hook run manually, stdin not forwarded, etc.) -- that would silently disable H11 for every push.
# Require at least one parsed ref line before granting the exemption; with zero lines, fall through
# to the existing branch-freshness check below (current behavior preserved).
#
# `[ -t 0 ] ||` so an interactive run does not hang waiting on a terminal: this script had no stdin
# reader before #719, and its own docs call "run by hand" a supported case. A TTY yields no ref
# lines, which is exactly the zero-line fall-through.
_h11_refs_seen=0
_h11_all_tags=1
[ -t 0 ] || while IFS=' ' read -r _h11_local_ref _h11_local_sha _h11_remote_ref _h11_remote_sha \
|| [ -n "${_h11_local_ref:-}" ]; do # `|| [ -n ... ]` also processes a final line with no trailing newline
[ -z "${_h11_local_ref:-}" ] && continue
_h11_refs_seen=1
case "${_h11_remote_ref:-}" in
refs/tags/*) ;;
*) _h11_all_tags=0 ;;
esac
_h11_local_ref=''
done
if [ "$_h11_refs_seen" = "1" ] && [ "$_h11_all_tags" = "1" ]; then
exit 0
fi
# Best-effort fetch of the latest main; offline / no network -> don't block.
git fetch origin main --quiet 2>/dev/null || exit 0
git rev-parse --verify --quiet origin/main >/dev/null 2>&1 || exit 0
-7
View File
@@ -40,13 +40,6 @@
# Fail-open by design: any parse trouble -> allow (exit 0, no output).
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-agent-model "" capture || true
input=$(cat)
tool=$(printf '%s' "$input" | jq -r '.tool_name // ""' 2>/dev/null || true)
-7
View File
@@ -3,13 +3,6 @@
# The historic 8-9-way crash was RAM starvation, not CPU load; gate on FREE RAM.
# Fail-open: if memory_pressure is unavailable/unparsable → allow.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-agent-ram "" capture || true
free=$(memory_pressure -Q 2>/dev/null | grep -oE 'free percentage: [0-9]+' | grep -oE '[0-9]+' || true)
[ -z "${free:-}" ] && exit 0
-7
View File
@@ -2,13 +2,6 @@
# PreToolUse / Bash — deny commands that violate a HARD RULE.
# Fail-open: any parse trouble → allow (exit 0 with no output).
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-bash-guard "" capture || true
input=$(cat)
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
+1 -12
View File
@@ -18,13 +18,6 @@
# the reason a commit can't happen; CI is still the backstop.
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-bom-guard "" capture || true
input=$(cat)
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
[ -n "$cmd" ] || exit 0
@@ -73,11 +66,7 @@ while IFS= read -r f; do
esac
p="$root/$f"
[ -f "$p" ] || continue
# `od`, NOT `xxd`. `xxd` ships with vim and is absent on plain Linux hosts including this repo's
# CI runner, where the command substitution yielded empty, never equalled `efbbbf`, and this guard
# therefore passed every BOM in silence. It has been fail-open on any host without vim since it
# was written. `od -A n -t x1 -N 3` is POSIX and produces byte-identical output on macOS and Linux.
if [ "$(od -A n -t x1 -N 3 < "$p" 2>/dev/null | tr -d ' \n')" = "efbbbf" ]; then
if [ "$(head -c3 "$p" 2>/dev/null | xxd -p 2>/dev/null)" = "efbbbf" ]; then
bad="${bad} ${f}"$'\n'
fi
done < /tmp/.bom-guard-files.$$
+48 -748
View File
@@ -7,22 +7,10 @@
# (c) a review-verdict comment on the PR references the CURRENT head sha (H10) — proving the
# LATEST commit was reviewed, not a stale earlier diff (the ersatztv#242 failure mode:
# "re-review the fix commit, not just the initial PR diff").
#
# EVERY ONE OF THOSE IS A SNAPSHOT, taken when the merge tool is called. The window is SMALL for an
# immediate merge and UNBOUNDED for a scheduled one. Small is not zero, and this comment used to say
# "sound", which is the overclaim ersatztv#778 removed: this hook returns `allow` and a SEPARATE call
# performs the merge, so a push can still land in between. The merge API accepts an optional
# `head_commit_id` that would make that call a true compare-and-set; a PreToolUse hook cannot add an
# argument, only refuse without one. With merge_when_checks_succeed, Gitea merges
# later, against whatever head is green then (ersatztv#622). So the sha-bound half of H10 is
# enforced by the SERVER, not here — `review-verdict/h10` is a required status check on `main`,
# written per-sha by scripts/post-review-verdict.sh, and a new commit cannot inherit it. This hook
# additionally refuses to SCHEDULE an auto-merge unless that status is already green on head, so the
# two mechanisms agree at the only moment they can both observe the same commit.
# The "## Done-when" issue-body checklist is the convention (docs/decisions.md, CLAUDE.md Task
# Completion Protocol). One box is "adversarial review passed"; the others are per-issue.
# The H10 review-verdict convention: after reviewing a PR (or its latest fix commit), post a PR
# comment carrying a line `Review-verdict: <MERGEABLE|APPROVED|LGTM|BLOCKED|NOT-MERGEABLE> @ <head-sha>`.
# comment carrying a line `Review-verdict: <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> @ <head-sha>`.
#
# Decision policy — a CONSENT gate, so it does NOT fail silently open:
# - state derivable and satisfied -> grant (auto-approve: permissionDecision "allow",
@@ -44,30 +32,6 @@
# Gitea auth from env (never committed): ETV_GITEA_TOKEN (a token) OR ETV_GITEA_BASICAUTH (user:pass).
# ETV_GITEA_URL overrides the base (default: the LAN instance; a LAN address, not a secret).
set -euo pipefail
# THE FIRE-LOG PATH BELOW IS SELF-LOCATED, not `${CLAUDE_PROJECT_DIR:-...}` (ersatztv#858, #891).
# Written here rather than beside the assignment because the instrumentation preamble that follows is
# machine-compared: `test_hook_fire_log.py::test_the_stripper_removes_EXACTLY_the_preamble_and_nothing_else`
# permits only its own recognised lines in that block, so a comment inside it fails the suite.
#
# That line is `. `-SOURCED, so whatever it names runs AS CODE inside this hook, before stdin is read
# and before `decide` exists. It is therefore not "telemetry" in any sense a gate can rely on.
# MEASURED 2026-08-30: with the env-var-first form, a `hook-fire-log.sh` in an env-var-named tree
# that prints an `allow` decision and exits 0 GRANTS THE MERGE outright, having bypassed every check
# below. Self-locating binds it to the tree this hook was loaded from and closes that.
#
# The other twelve tracked hooks still carry the env-var-first form and are deliberately NOT changed
# here; that sweep needs its own population and review (ersatztv#891, where the reachable case is
# measured — husky launches the prepush hooks as `./.claude/hooks/...`, a RELATIVE path independent
# of `$CLAUDE_PROJECT_DIR`, so the two roots genuinely diverge there). This copy is fixed because
# leaving a total gate bypass 500 lines above the gate this PR hardens would make the rest decorative.
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-merge-consent "" capture || true
input=$(cat)
decide() { # $1=grant|allow|deny|ask $2=reason
@@ -113,59 +77,8 @@ sha=$(printf '%s' "$prjson" | jq -r '.head.sha // ""' 2>/dev/null || true)
body=$(printf '%s' "$prjson" | jq -r '.body // ""' 2>/dev/null || true)
# --- Docs-only exemption: if every changed file is docs/process, skip the gate. ---
# The file list must be enumerated EXHAUSTIVELY, validated row by row, and checked for head/base
# movement across the paging round trips, or the exemption is unsafe. (That check detects ONE-WAY
# movement only — this said "bound to ONE head" until 2026-08-28, ersatztv#803.) ALL of that now lives in scripts/pr-changed-files.sh — the single shared
# implementation, also called by .gitea/workflows/review-verdict.yml (ersatztv#649).
#
# Why it moved: this logic was written twice. This copy is ADVISORY (a failure produces a human
# prompt); the workflow's copy is ENFORCED (it writes the branch-protection-required
# `review-verdict/h10` status). Four rounds of ersatztv#643 hardening landed here and never reached
# there, leaving the copy with real authority strictly weaker than the copy without — and its safe
# behaviour resting on a bash arithmetic error rather than an intentional guard. Two copies of a
# security predicate drift; one cannot.
#
# What is NOT shared, deliberately: the docs-only allow-list below. This one also lets .claude/,
# .gitea/ and .husky/ through, which is safe HERE only because a match falls through to a human
# prompt rather than auto-granting. The workflow's list is narrower for exactly that reason. Sharing
# the enumeration fixes the drift; sharing the classification would erase an intended difference.
#
# A non-zero exit means "could not tell" and MUST withhold the exemption — never read stdout without
# checking the status. An empty `$sha` (unparseable PR JSON) reaches the script as an empty argument
# and is rejected there, so that path also fails closed.
#
# The 5th argument binds the enumeration to a base branch (ersatztv#698 route 1), because
# `/pulls/{n}/files` diffs against the PR's LIVE base and retargeting moves that without moving the
# head. Be precise about what it buys HERE, which is less than what it buys in the workflow: the
# workflow passes the base from a `pull_request_target` event payload, fixed at event time and beyond
# a retarget's reach, so it detects a retarget outright. This hook has no such trusted snapshot — it
# passes the base it just read from the live PR, so what it asserts is that the base did not move
# between that read and the enumeration. Narrower, and still worth having: without it the hook cannot
# tell a mid-flight retarget from an honest read at all. An empty/unparseable `.base.ref` reaches the
# script as an empty argument and is rejected there, so that path fails closed too.
base_ref=$(printf '%s' "$prjson" | jq -r '.base.ref // ""' 2>/dev/null || true)
repo_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
files=""; files_complete=no
if files=$("$repo_root/scripts/pr-changed-files.sh" "$owner" "$repo" "$pr" "$sha" "$base_ref" 2>/dev/null); then
files_complete=yes
fi
# HOW THIS PREDICATE IS EVALUATED, matching the enforced gate (ersatztv#698,
# `ci.grep-q-pipefail-inversion`). `printf … | grep -q` INVERTS under `set -o pipefail`: grep -q exits
# at its first match, printf then takes SIGPIPE (141), and a MATCH is reported as a failed pipeline —
# so this negated test would grant a spurious docs-only exemption for any PR whose path list exceeds
# the pipe buffer. A here-string fixes that but is materialised via temporary storage for large inputs,
# so it can fail when temp space is full or unwritable and flip the predicate the same way. Counting
# with `grep -c` drains stdin (no SIGPIPE) over an ordinary pipe (no temp file); `grep -c` exits 1 for
# a zero count, which is a legitimate answer, so only a status >1 is a real error and is treated as
# "cannot tell" -> no exemption.
# Advisory here, so the blast radius is a missing prompt rather than a green required check; the
# construct is identical on purpose, because the two copies drifting is what ersatztv#649 was about.
docs_nonmatching=$(printf '%s\n' "$files" | grep -cvE '^(docs/|\.claude/|\.husky/|\.gitea/|.*\.md$)') || docs_grep_status=$?
if [ "${docs_grep_status:-0}" -gt 1 ]; then
docs_nonmatching=1 # grep itself failed: cannot tell, so withhold the exemption
fi
if [ "$files_complete" = yes ] && [ -n "$files" ] && [ "${docs_nonmatching:-1}" -eq 0 ]; then
files=$(gq "repos/$owner/$repo/pulls/$pr/files?limit=100" | jq -r '.[].filename // empty' 2>/dev/null || true)
if [ -n "$files" ] && ! printf '%s\n' "$files" | grep -qvE '^(docs/|\.claude/|\.husky/|\.gitea/|.*\.md$)'; then
# Docs/process-only PR: the Done-when + review-verdict gate doesn't apply — but this exemption is a
# file-TYPE bypass, NOT the a+b+c "provably reviewed & ready" proof, so it does NOT auto-grant. It
# passes through to normal permissioning (one prompt). This deliberately keeps a human in the loop for
@@ -175,147 +88,6 @@ if [ "$files_complete" = yes ] && [ -n "$files" ] && [ "${docs_nonmatching:-1}"
decide allow "" # passthrough (exit 0 → normal prompt), NOT grant
fi
# --- Base-change detection: a verdict is bound to a head AND to a base (ersatztv#632). ---
# `review-verdict/h10` is per-sha, which makes "the head moved under a fixed verdict" impossible by
# construction. Retargeting a PR's base is the mirror case and slips through: it changes neither the
# head sha nor the status, so a verdict formed while the PR targeted `main` still reads green after
# the PR is pointed at a branch with a very different merge-base. The diff moves while the verdict
# and the head both hold still.
#
# DETECTION, NOT PREVENTION, and only on this path. A commit status carries no base, so the
# server-side required check cannot see this; a merge driven through the Gitea UI or API is
# unaffected. That is the accepted exposure — base changes are rare, manual, and this is a
# two-account repo — but it is now recorded in a place that fails LOUD rather than only in a doc.
#
# GRACEFUL ADOPTION, mirroring (b) and (c): a description with no `(base: …)` field is a verdict
# posted before ersatztv#632 and gets NO opinion, rather than denying every in-flight PR the day
# this lands. The window closes on its own — verdicts are per-head and short-lived, so every verdict
# posted after this carries the field.
# "Could not check" is a THIRD outcome, distinct from both "matches" and "no base recorded". Cold
# review found the first draft collapsing it into the latter: an unreadable status response yielded
# an empty `recorded_base`, which took the graceful-adoption path and skipped validation silently —
# after which a later, successful status read could still auto-grant. A transient failure would then
# have produced a "merge gate: satisfied" message for a comparison that never happened. Every
# unreadable input here therefore falls through to a human (`ask`), never to silence.
# RE-READ THE BASE HERE, ONCE, FOR EVERY PATH BELOW (ersatztv#778).
#
# "Below" is literal, and the one consumer ABOVE is disclosed rather than implied: the docs-only
# enumeration still runs against the snapshot `$base_ref` and can `decide allow` before reaching
# this point. That is bounded and deliberate — a docs-only match is a PASSTHROUGH to the ordinary
# human prompt, never an auto-grant, so a stale base there costs a prompt someone was going to see
# anyway. Every path that can GRANT passes through the check below.
#
# `$base_ref` above comes from the PR snapshot taken at the top of this hook, and the docs-only
# enumeration between there and here is up to forty round trips. A PERSISTENT retarget in that gap
# needs no ABA and no force-push: every base-dependent decision below would be formed against a
# branch the PR no longer targets. Checking a stale identifier is not checking — which is the whole
# of `process.check-and-use-pins-a-version`, so the guard enforcing that rule must not break it.
#
# This re-read first landed inside the scheduled-auto-merge branch only, which fixed the branch-
# protection lookup and left the #632 retarget DETECTION below still reading the stale snapshot. Cold
# review demonstrated the consequence with this repo's own fixture: scheduled+retarget denied, while
# immediate+retarget auto-GRANTED. That is the twin-missed shape — a fix applied to the path where it
# was noticed — so the re-read is hoisted above every consumer rather than duplicated into each.
prjson_now=$(gq "repos/$owner/$repo/pulls/$pr")
if [ -z "${prjson_now//[[:space:]]/}" ] || ! printf '%s' "$prjson_now" | jq -e 'type == "object"' >/dev/null 2>&1; then
decide ask "H10 merge gate: could not re-read PR #$pr to confirm it still targets '$base_ref' before checking the verdict against it. Confirm the target branch, then merge."
fi
base_now=$(printf '%s' "$prjson_now" | jq -r '.base.ref // ""' 2>/dev/null || true)
if [ -z "$base_now" ]; then
decide ask "H10 merge gate: PR #$pr reports no base branch (.base.ref), so the verdict cannot be checked against the branch it was formed for (ersatztv#632). Confirm the PR still targets the branch it was reviewed against before merging."
fi
if [ -n "$base_ref" ] && [ "$base_now" != "$base_ref" ]; then
decide deny "H6/H10 merge gate: BLOCKED — PR #$pr was retargeted from '$base_ref' to '$base_now' while this gate was evaluating. Every check formed against '$base_ref', including the changed-file enumeration and the review verdict, describes a merge that is no longer the one being requested (ersatztv#632). Re-review against '$base_now' and run: scripts/post-review-verdict.sh $pr MERGEABLE"
fi
# From here on both names are the freshly-confirmed base; they are equal by the check above.
base_ref=$base_now
live_base=$base_now
# THE HEAD IS RE-READ AT THE SAME HOIST, FROM THE SAME RESPONSE (ersatztv#803).
#
# `$sha` comes from the PR snapshot at the top of this hook, and until 2026-08-28 every later check
# consumed that captured value: the CI combined status, the `review-verdict/h10` status, and the
# verdict-comment classification were all evaluated against `/commits/$sha/status` and `--head $sha`.
# A push landing in the gap — which includes the docs-only enumeration's up-to-forty round trips —
# was therefore checked against the commit it had just replaced, and the hook would report "a
# positive Review-verdict references the current head" about a head that was no longer current.
#
# This is the SAME defect the base had until #778 hoisted the re-read above, and it is fixed the same
# way rather than a different way. Reading `.head.sha` off `$prjson_now` — the response the base
# check already fetched — costs NO extra round trip, and it keeps the two axes on ONE snapshot, so
# they cannot disagree about which moment they describe. Two separate reads would answer about two
# different instants while reading as one check.
#
# DENY, not ask, and for the same reason the `stale` verdict class denies: a head that moved means
# the verdict this hook is about to accept covers an OLDER commit, which is a state we have
# positively established rather than failed to establish. An UNREADABLE `.head.sha` is the different
# case and asks.
#
# WHAT THIS DOES NOT CLOSE, said here rather than left to be inferred. A push landing after this
# check still passes, exactly as a retarget does — the file's rule against a second re-read applies
# unchanged (see the branch-protection block below), because two reads only move the window rather
# than closing it. That residual is bounded server-side and this hook is not what bounds it: the new
# head has no `review-verdict/h10` status, and that context is REQUIRED on `main`, so Gitea refuses
# the merge (#622). The hook's job here is to stop CLAIMING a head is reviewed when it can see that
# it is not — an advisory gate that states something false is worse than one that asks.
if [ -n "$sha" ]; then
sha_now=$(printf '%s' "$prjson_now" | jq -r '.head.sha // ""' 2>/dev/null || true)
if [ -z "$sha_now" ]; then
decide ask "H10 merge gate: PR #$pr reports no head commit (.head.sha) on re-read, so whether the review verdict still covers the current head could not be confirmed. Check the PR, then merge."
fi
if [ "$sha_now" != "$sha" ]; then
decide deny "H6/H10 merge gate: BLOCKED — PR #$pr's head moved from ${sha:0:7} to ${sha_now:0:7} while this gate was evaluating. Every check formed against ${sha:0:7} — the changed-file enumeration, the CI status and the review verdict — describes a commit that is no longer the one being merged (ersatztv#803). Re-review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE"
fi
# From here on `$sha` is the freshly-confirmed head; the two are equal by the check above. Mirrors
# `base_ref=$base_now` a few lines up, and is written for the same reason that one is: it makes the
# value every later check consumes the one that was just re-read, so a future edit moving a
# consumer above this point fails visibly rather than silently reading the stale capture.
sha=$sha_now
fi
if [ -n "$sha" ]; then
# This is the THIRD read of this endpoint in a worst-case hook run (the ordinary-CI branch and the
# scheduled-auto-merge branch each do their own). Sharing one snapshot would close a narrow
# same-run window where two reads disagree, but the later branches derive different decisions from
# a failed read than this one does, so threading a shared response through them is a change to
# pre-existing logic rather than to ersatztv#632's. Left deliberately, noted so it is not
# rediscovered as an oversight: every `decide` exits immediately, so the reads cannot produce a
# single self-contradictory message — only a later decision made on a fresher snapshot.
vjson_base=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100")
# Same jq-1.6 rule as everywhere else in this file: check emptiness in SHELL first, never via
# `jq -e`'s exit status over empty input.
# VALIDATE EVERY FIELD THE EXTRACTION CONSUMES, on EVERY row — the same rule the file-enumeration
# guard learned the hard way. Checking only that `.statuses` is an array left a hole one level
# down: `{"statuses":[1]}` passes a top-level type check, then `.context` on a number errors, and
# a `|| true` on the extraction turned that error into an empty `vdesc` — i.e. straight back onto
# the graceful-adoption path this block exists to distinguish from. That is the identical
# swallow-the-error shape fixed a few lines up, surviving one level deeper.
if [ -z "${vjson_base//[[:space:]]/}" ] \
|| ! printf '%s' "$vjson_base" \
| jq -e '.statuses | type == "array"
and all(.[]; type == "object"
and (.context | type == "string")
and (.description == null or (.description | type == "string")))' \
>/dev/null 2>&1; then
decide ask "H10 merge gate: could not read the commit statuses for PR #$pr head ${sha:0:7}, so the verdict could not be checked against the PR's base branch (ersatztv#632). Confirm the review covered the branch this PR currently targets ('$live_base') before merging."
fi
# No `|| true` here. The validation above makes an error unreachable, but a swallowed error would
# be indistinguishable from "no base recorded" — the exact confusion this block removes — so the
# failure is handled explicitly rather than left to a fallback that reads as a benign result.
if ! vdesc=$(printf '%s' "$vjson_base" \
| jq -r '[.statuses[] | select(.context == "review-verdict/h10")] | first | .description // ""' \
2>/dev/null); then
decide ask "H10 merge gate: the commit statuses for PR #$pr head ${sha:0:7} could not be parsed to find the review verdict, so it could not be checked against the PR's base branch (ersatztv#632). Confirm the review covered the branch this PR currently targets ('$live_base') before merging."
fi
# The field is written by scripts/post-review-verdict.sh as a trailing `(base: <ref>)`. Its
# ABSENCE is the one benign case: a verdict posted before ersatztv#632 could not have carried it,
# and denying those would block every in-flight PR the day this lands. The window closes on its
# own, since verdicts are per-head and short-lived.
recorded_base=$(printf '%s' "$vdesc" | sed -n 's/.*(base: \(.*\))$/\1/p')
if [ -n "$recorded_base" ] && [ "$recorded_base" != "$live_base" ]; then
decide deny "H10 merge gate: BLOCKED — the review verdict on head ${sha:0:7} was formed while PR #$pr targeted '$recorded_base', but it now targets '$live_base'. Retargeting a base does not move the head sha, so the per-sha verdict status still reads green even though the effective diff has changed (ersatztv#632). Re-review against the new base and run: scripts/post-review-verdict.sh $pr MERGEABLE"
fi
fi
# --- Linked issue: Gitea auto-close keywords in the PR body. ---
issues=$(printf '%s' "$body" | grep -ioE '(close[sd]?|fix(e[sd])?|resolve[sd]?) +#[0-9]+' | grep -oE '[0-9]+' | sort -u || true)
[ -n "$issues" ] || decide ask "H6 merge gate: PR #$pr has no linked issue (no 'fixes #N' / 'closes #N' in its body), so there is no Done-when checklist to derive consent from. Confirm the work is complete + reviewed, then approve."
@@ -338,353 +110,14 @@ for n in $issues; do
fi
done
# ONE branch-protection READ per run (ersatztv#859). Two arms consume this endpoint — the scheduled
# path's `review-verdict/h10` required-check test, and the guard-scope freshness check at the bottom
# — and they used to issue independent GETs, so a scheduled auto-merge hit it twice (measured: the
# test stub recorded 2 URLs).
#
# THE ROUND TRIP IS THE SMALLER HALF. What matters is that branch protection is MUTABLE config: two
# reads can return two different answers, and the gap between them is a gap in which the two arms
# decide about different repo states — one concluding `review-verdict/h10` is required on the base
# while the other classifies a rule list that no longer says so. Neither arm can detect that; both
# would report confidently. Caching makes a single run internally consistent BY CONSTRUCTION, which
# is a property no retry or ordering change can supply.
#
# WHY #787 DID NOT ALREADY SHARE IT, since the obvious question is why two reads existed at all: the
# arms ask genuinely different QUESTIONS — one about `$base_ref` and its required contexts, one about
# `main` and snapshot freshness — so their classifications must stay separate. But they ask those
# questions of the same URL with the same credentials, so the RESPONSE is shareable even though the
# verdicts are not. Cache the bytes; never cache a verdict.
#
# This does NOT pin anything: protection can still change after the read, and the honest ceiling is
# unchanged (`process.check-and-use-pins-a-version`). It removes a second window, it does not remove
# the first.
bp_fetched=no
bp_cache=""
bp_cache_code=""
fetch_branch_protections() {
# Idempotent by design: every caller invokes it unconditionally and the FIRST one pays. A caller
# that had to know whether it was first would be a second place for the two arms to disagree.
if [ "$bp_fetched" = yes ]; then return 0; fi
bp_fetched=yes
local f
# A temp-file failure gets its own sentinel rather than an HTTP-shaped one, so each caller can
# keep the distinct message it had before this was shared. Reporting a mktemp failure as HTTP
# '000 — Gitea unreachable' would state a cause that did not happen, which is the defect class
# this whole file is organised around.
f=$(mktemp) || { bp_cache=""; bp_cache_code=mktemp-failed; return 0; }
if [ -n "${ETV_GITEA_TOKEN:-}" ]; then
bp_cache_code=$(curl -s -o "$f" -w '%{http_code}' -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true)
else
bp_cache_code=$(curl -s -o "$f" -w '%{http_code}' -u "$ETV_GITEA_BASICAUTH" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true)
fi
bp_cache=$(cat "$f" 2>/dev/null || true)
rm -f "$f"
}
# --- (a) CI combined status must be green (unless deferring to Gitea's own check-gate). ---
if [ "$mwcs" != "true" ]; then
[ -n "$sha" ] || decide ask "H6 merge gate: could not resolve PR #$pr head sha to check CI. Verify CI is green before merging."
cistatus=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100")
state=$(printf '%s' "$cistatus" | jq -r '.state // ""' 2>/dev/null || true)
state=$(gq "repos/$owner/$repo/commits/$sha/status" | jq -r '.state // ""' 2>/dev/null || true)
case "$state" in
success) : ;;
"") decide ask "H6 merge gate: could not read CI status for PR #$pr ($sha). Verify CI is green before merging." ;;
*)
# `review-verdict/h10` is itself one of the contexts folded into the COMBINED state, so a PR
# awaiting its verdict reports combined 'pending' and would otherwise be reported as a CI
# problem — sending the reader to build logs when the missing thing is the review. Name the
# real blocker when the verdict is the only thing outstanding.
#
# "Not green" is anything that is not `success`, NOT just pending/failure: Gitea also has
# `error` (and `warning`), and omitting those would let an errored build hide behind the
# verdict and produce the flatly false claim "every CI check is green". `skipped` IS treated
# as green — the image-push job skips on every PR (ersatztv#593: a skipped context is not red).
nongreen=$(printf '%s' "$cistatus" \
| jq -r '[.statuses[]? | select(.status != "success" and .status != "skipped")]
| map("\(.context)=\(.status)") | join(", ")' 2>/dev/null || true)
# The verdict's OWN state decides the wording: absent/pending means nobody has reviewed this
# head, while failure/error means someone reviewed it and said no. Telling a reviewer to "post
# a verdict" when they already posted a BLOCKED one would be actively misleading.
vonly=$(printf '%s' "$cistatus" \
| jq -r '[.statuses[]? | select(.status != "success" and .status != "skipped")]
| if (length == 1 and .[0].context == "review-verdict/h10") then .[0].status else "" end' 2>/dev/null || true)
case "$vonly" in
pending)
decide deny "H6/H10 merge gate: BLOCKED — every CI check on PR #$pr is green; the only outstanding context is 'review-verdict/h10' on head ${sha:0:7}, i.e. this head has no review verdict yet. Review it and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
failure|error)
decide deny "H6/H10 merge gate: BLOCKED — every CI check on PR #$pr is green, but 'review-verdict/h10' is '$vonly' on head ${sha:0:7}: this head was reviewed and REJECTED. Resolve the findings, then run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
esac
decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success' (not green: ${nongreen:-unknown}). Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging."
;;
esac
else
# --- SCHEDULED auto-merge: everything this hook proves is a SNAPSHOT (ersatztv#622). ----------
# With merge_when_checks_succeed, Gitea performs the merge later, against whatever head is green
# at THAT moment — but (b) and (c) below are evaluated against the head that exists right now.
# Any commit pushed in between would merge with no verdict covering it. Demonstrated as a
# controlled A/B (#622): with a slow CI check pending so Gitea waits, an unreviewed commit pushed
# after scheduling MERGED without the required verdict context and was REFUSED with it.
#
# The durable fix is server-side and lives outside this hook: `review-verdict/h10` is a REQUIRED
# status check on `main`, and a commit status belongs to exactly ONE sha, so a later commit cannot
# inherit it and Gitea's own gate refuses to merge until that head is re-reviewed.
#
# What we add HERE is the matching precondition at SCHEDULING time: refuse to arm an auto-merge
# unless the sha-bound status already exists on this head. Checking the comment alone (condition
# (c) below) is not enough for this path — the comment is what a human reads, the status is what
# the server enforces, and only the latter survives a new push. Deny rather than ask: the remedy
# is a single documented command, so there is nothing here for a human to adjudicate.
[ -n "$sha" ] || decide ask "H6 merge gate: could not resolve PR #$pr head sha to check the review-verdict status. Verify the review covered the latest commit before scheduling an auto-merge."
# Read the COMBINED endpoint, not `/statuses/{sha}`: the latter returns one row per status POST
# rather than per context and pages at 50, so a head with a few CI reruns can push the verdict off
# the first page and read as absent — a confusing false deny. The combined endpoint returns
# latest-per-context, which is exactly the question being asked.
vjson=$(gq "repos/$owner/$repo/commits/$sha/status?limit=100")
# Same portability point as the file-pagination guard above: do not let jq's empty-input exit
# status decide this. Here the fallthrough happens to land on `vstate=""` -> deny (fail-CLOSED,
# so this was never a hole), but it would have surfaced the wrong message — a "BLOCKED, no
# verdict" deny instead of the "could not read the status" ask this branch exists to give.
# Validate the MEMBERS, not just the array. `.statuses | type == "array"` passes for
# `{"statuses":[1]}`, and the extraction below then errors with "Cannot index number with string"
# and exits 5 — which, under `set -e`, aborts this hook with NO JSON on stdout at all. A consent
# hook that emits nothing has violated its own contract: it neither grants, denies nor asks. Same
# one-level-down swallow as the #632 base-change guard and the branch-protection shape check
# below; the validation domain must match the CONSUMPTION domain (ersatztv#778).
if [ -z "${vjson//[[:space:]]/}" ] \
|| ! printf '%s' "$vjson" \
| jq -e '(.statuses | type == "array")
and all(.statuses[]; type == "object"
and ((.context | type) == "string")
and ((.status | type) == "string"))' >/dev/null 2>&1; then
decide ask "H6/H10 merge gate: could not read the 'review-verdict/h10' status for PR #$pr head ${sha:0:7} (Gitea unreachable, or a response whose status rows are not the expected shape). Confirm the current head is reviewed before scheduling an auto-merge."
fi
vstate=$(printf '%s' "$vjson" | jq -r '[.statuses[] | select(.context == "review-verdict/h10")] | first | .status // ""')
case "$vstate" in
success) : ;;
"") decide deny "H6/H10 merge gate: BLOCKED — PR #$pr has no 'review-verdict/h10' commit status on head ${sha:0:7}, so scheduling an auto-merge would freeze consent at a head Gitea may not be the one to merge (ersatztv#622). Review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
pending) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is still pending on PR #$pr head ${sha:0:7} (no verdict posted for this commit yet). Review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
*) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is '$vstate' on PR #$pr head ${sha:0:7}. Resolve the findings, then run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
esac
# --- The mitigation this path RESTS on, verified instead of asserted (ersatztv#778). -----------
# Everything above proves a property of the head that exists NOW. What makes that safe under
# merge_when_checks_succeed is stated in the paragraph opening this branch: `review-verdict/h10`
# is a REQUIRED status check on the base, a commit status belongs to exactly ONE sha, so a commit
# pushed after scheduling cannot inherit it and Gitea's own gate refuses the merge.
#
# That guarantee is branch-protection CONFIG. It lives outside this repo, no code here owned it,
# and until #778 nothing compared the two — so the grant reason handed to a human cited a
# protection that could have been switched off with no signal anywhere. The comment above and the
# grant string below are claims about the past; a dated claim is not a check.
#
# This is the hook's OWN defect class (#778 / `process.check-and-use-pins-a-version`): a check
# ("a later push clears the status") authorizes an action ("arm an auto-merge that Gitea completes
# later") over state that can change in between, with nothing pinning it. The read here does not
# pin anything either — branch protection can still be edited after this call — but it converts an
# ASSUMPTION that was never observed into a precondition that is, which is the honest ceiling for
# a config whose API offers no version, ETag or conditional read.
#
# Tri-state, matching this file's idiom throughout: unreadable -> ask (a human adjudicates),
# present -> proceed, ABSENT -> deny. Absence is not a degraded read; it is #622's hole reopened,
# and the whole point of that issue is that the failure is silent from the merge caller's side.
# Belt-and-braces: `$base_ref` was proven non-empty and re-confirmed at the hoisted check above,
# so this cannot fire today. Kept because it is the precondition this block's URL depends on, and
# a future edit that moves either piece should fail loudly here rather than request a URL with an
# empty path segment.
[ -n "$base_ref" ] || decide ask "H6/H10 merge gate: could not resolve PR #$pr's base branch, so the 'review-verdict/h10' required-check protection that makes a scheduled auto-merge safe (ersatztv#622) can't be confirmed. Verify branch protection on the base, or merge immediately instead of scheduling."
# The base was re-read and confirmed unchanged above, for every path — see the hoist comment
# there. It is deliberately NOT re-read a second time here: two reads would create a window
# between them for no gain, and the hoisted check already covers the enumeration gap that made
# this necessary.
# A read failure here is NOT evidence about the branch. The deleted by-name endpoint answered 404
# for "no rule with this name", which was a finding; the LIST endpoint's 404 means the repo was not
# found or is invisible to this credential, which is a read failure. Absence is now established by
# the classifier returning `nomatch` over a list that WAS read, never by an HTTP status.
# ALWAYS enumerate the rule LIST; never look a rule up by name. The by-name endpoint
# (`branch_protections/{name}`) is an exact DB lookup — `GetProtectedBranchRuleByName` — which
# performs no matching and knows nothing about precedence, so a 200 from it means only "a rule
# with this NAME exists and lists this context", never "this context is required on this branch".
#
# It was used first, with the list consulted only on a 404, and cold review found what that left
# behind: the precedence argument below guarded the 404 path while the 200 path — the one this
# repo actually takes — granted without it. Given a rule `main` requiring `review-verdict/h10` and
# a rule `m*` with better Priority that does not, Gitea applies `m*`, and the by-name hit on
# `main` granted anyway. The hardened path was dead code and the unhardened one was live. Deleting
# the twin rather than documenting it is the point: one fetch, one classifier, one argument, and
# no second path to keep in step. The ref no longer reaches a URL segment, so it needs no
# encoding either.
fetch_branch_protections
if [ "$bp_cache_code" = "mktemp-failed" ]; then
decide ask "H6/H10 merge gate: could not allocate a temp file to read branch protection for '$base_ref'. Confirm the 'review-verdict/h10' required check manually before scheduling an auto-merge."
fi
bp_code=$bp_cache_code
bp_list=$bp_cache
bp=""
if [ "$bp_code" = "200" ] && printf '%s' "$bp_list" | jq -e 'type == "array"' >/dev/null 2>&1; then
# DO NOT claim parity with Gitea's matcher — this code cannot have it, and asserting it would
# be the exact defect this PR records (a mitigation outside the code, asserted rather than
# verified). Gitea compiles a rule name with gobwas/glob and a `/` separator, so its `*` does
# NOT cross a slash, `?`/`[…]`/`{a,b}` are wildcards, and a plain name is folded case-
# insensitively. Reimplementing that here would be a second copy of somebody else's parser.
#
# So the classification is deliberately THREE-way, and each arm is safe without knowing the
# dialect:
# exact — no glob rule could apply, AND some rule name has no glob metacharacter and
# equals the base case-insensitively. Only then is a single rule decidable.
#
# UNDECIDABLE IS EVALUATED FIRST, and the order is the point. Gitea picks the
# governing rule with `GetFirstMatched` over a list sorted by Priority, THEN
# by plain-name-ness — so a glob rule with a better Priority outranks an
# exactly-named one. Preferring `exact` would therefore inspect a rule Gitea
# might not be applying: if the exact rule requires `review-verdict/h10` and a
# higher-priority glob rule does not, the gate auto-grants on a base where the
# check is not enforced. Asking whenever ANY glob rule could apply is sound
# without knowing the precedence rules at all, which is the only claim this
# code is entitled to make about somebody else's resolver.
#
# Case folding is ASCII-only here, while Gitea's `EqualFold` is
# Unicode-aware — so a rule `ünstable` and a base `Ünstable` fold equal there
# and not here. ASCII-fold equality implies EqualFold equality, so the gap can
# only MISS a match, never invent one; but a miss lands on `none`, which
# DENIES with the stated cause that no rule can govern the base. The backslash
# paragraph below rejects "nearly unreachable" as a standard for that arm, and
# the same standard has to apply here, so a rule name carrying any non-ASCII
# byte is `undecidable` rather than fold-compared. Two fold-equal plain names
# are undecidable too: this code picks by list order while Gitea picks by
# Priority, and guessing which one is enforced is the defect the arm order
# above exists to avoid.
# undecidable — some glob rule COULD govern this base. Tested with a provable SUPERSET of any
# glob dialect: literal prefix before the first metacharacter, `.*`, literal
# suffix after the last. If even that does not match, no dialect can, because
# every dialect requires the literal head and tail to match literally.
#
# BACKSLASH counts as a metacharacter for that purpose, and it is the one case that breaks the
# superset proof if it does not. gobwas/glob reads `\{` as a LITERAL brace, so a rule `a\{b`
# governs the base `a{b` — while a superset that treated `\` as literal would build `a\.*b`,
# fail to match, and answer `none`, i.e. deny a base that IS protected. Git ref rules make this
# nearly unreachable (a branch name may not contain `*`, `?`, `[` or `\`, though it MAY contain
# `{`), but `none` is the arm that authorises a DENY on the stated grounds "nothing can govern
# this base", so its premise has to hold unconditionally rather than usually.
# none — nothing can possibly govern the base, so it is genuinely unprotected.
#
# `undecidable` asks rather than granting or denying. Over-matching would auto-grant on a base
# whose protection we never established (#622's hole, reached through the block written to
# close it); under-matching would deny with a stated cause that is false, which this block's
# own comment calls the worse outcome. Asking is the only answer that is honest in both
# directions, and it is rare in practice: as of 2026-08-19 this repo's only rule is the plain
# name `main`, which the classifier resolves to `exact` on every run. That is a dated
# observation about mutable remote config, not a property to rely on.
# The classifier is a FILE now (ersatztv#787), so its absence is a new failure mode: `jq -f` on a
# missing program exits 2 with empty stdout, which reaches the `*)` arm below and asks that "this
# repo's branch-protection rules came back in a shape this hook could not parse" — blaming the
# payload for a missing local file. That is precisely the states-a-cause-that-did-not-happen defect
# the two comments beside that arm were written to fix, so it is checked here rather than inherited.
classifier="$repo_root/scripts/lib/branch-rule-classifier.jq"
if [ ! -r "$classifier" ]; then
decide ask "H6/H10 merge gate: the shared branch-protection rule classifier is missing or unreadable at $classifier, so which rule governs '$base_ref' — and therefore whether 'review-verdict/h10' is required on it — could not be derived (ersatztv#787). Restore the file, or confirm the required checks manually."
fi
bp_verdict=$(printf '%s' "$bp_list" | jq --arg b "$base_ref" -c -f "$classifier" 2>/dev/null || true)
case $(printf '%s' "$bp_verdict" | jq -r '.verdict // ""' 2>/dev/null || true) in
exact) bp=$(printf '%s' "$bp_verdict" | jq -c '.rule' 2>/dev/null || true); bp_code=200 ;;
undecidable) decide ask "H6/H10 merge gate: no branch-protection rule on this repo governs '$base_ref' decidably — a GLOB rule could govern it, or two rule names fold-equal, or a name is non-ASCII. This hook deliberately does not reimplement Gitea's glob matcher, so whether 'review-verdict/h10' is required on this base cannot be derived here (ersatztv#778). Confirm it in the repo's branch-protection settings, or merge immediately instead of scheduling." ;;
none) bp_code=nomatch; bp="" ;;
# A DECLARED class of the classifier's contract (ersatztv#859), with its OWN sentinel — not
# merely its own arm. The first draft gave it an arm that set `unreadable-rules`, the same value
# the catch-all sets, and that arm was measured to be a no-op: deleting it left the WHOLE suite
# green, because nothing downstream could tell the two apart. An arm no observation can
# distinguish is not a fix, it is a comment with syntax. (The invariant is "no test reddens",
# not a test count — a count goes stale the next time anyone adds one.)
#
# They are different findings and now say so. `unnamed-rule` means the list was READ and a rule
# in it carries no usable name; `unreadable-rules` means jq died or answered a word this hook
# does not know. Same decision (ask), different cause — and naming the cause accurately is the
# entire subject of this issue, so collapsing them here would have reproduced the defect being
# fixed, one arm over.
unreadable) bp_code=unnamed-rule; bp="" ;;
*) bp_code=unreadable-rules; bp="" ;;
esac
else
# A 200 whose body is NOT an array never reaches the classifier — it is diverted by the array
# gate above — so it needs the same sentinel, or the generic ask below reports
# "HTTP '200' — Gitea unreachable" about a read that plainly succeeded. Same defect as the
# throw-inside-the-classifier arm, one branch earlier; fixing only the arm where it was noticed
# is the twin-missed shape this PR is largely about.
if [ "$bp_code" = "200" ]; then
bp_code=unreadable-rules
else
bp_code=${bp_code:-000} # a real transport/HTTP failure -> the ask arm below
fi
bp=""
fi
# `nomatch` is the CLASSIFIER's verdict, deliberately not an HTTP code. Reusing 404 for it made
# this deny reachable from an HTTP 404 on the list read too — repo not found, or invisible to the
# credential, which Gitea also answers 404 — and then the reason claimed "the full rule list was
# read and none matches" about a read that never happened. A transport failure must reach the ask
# below, not a deny stating a finding.
if [ "$bp_code" = "nomatch" ]; then
decide deny "H6/H10 merge gate: BLOCKED — no branch-protection rule on this repo can govern '$base_ref' (the full rule list was read and none matches), so 'review-verdict/h10' is not a required check on it. A scheduled auto-merge is safe ONLY because that per-sha required check stops a commit pushed after scheduling from merging unreviewed (ersatztv#622). Restore branch protection on '$base_ref', or merge immediately (without merge_when_checks_succeed) once CI is green."
fi
# `unnamed-rule` is the classifier reporting a rule whose NAME it could not use. Two distinct
# shapes, and the reason string must cover both or it states a cause that did not happen: EITHER
# both fields supply no name (absent, null, or empty), OR one of them is present holding a
# non-string, which poisons the rule however good its sibling is. It is deliberately NOT reported as
# "no rule matches": a rule that cannot be read might be the rule Gitea is applying, so a list
# containing one supports no finding about which rule governs the base. That was the #859 defect —
# `""` is a valid name that matches nothing, so an unreadable rule DENIED with a stated cause that
# had not happened.
if [ "$bp_code" = "unnamed-rule" ]; then
decide ask "H6/H10 merge gate: a branch-protection rule on this repo carries no name this hook can use — either both 'branch_name' and 'rule_name' are absent/null/empty, or one of them is present holding something that is not a string. Which rule governs '$base_ref', and whether 'review-verdict/h10' is required on it, therefore could not be derived. A rule that cannot be read might be the one Gitea applies, so this is deliberately NOT reported as 'no rule matches' (ersatztv#859). Inspect the branch-protection rules, or merge immediately instead of scheduling."
fi
# `unreadable-rules` is the CLASSIFIER failing on a 200 this hook could not turn into a verdict —
# jq died, or answered a word this contract does not define. It gets its own sentinel for the same
# reason `nomatch` does: reporting "HTTP '000' — Gitea unreachable" about a successful 200 read
# states a cause that did not happen, which is the defect fixed one arm over for the deny.
#
# A numeric `branch_name` was the worked example here until ersatztv#859 and no longer reaches this
# arm: it is not a usable NAME, so the classifier now classifies it rather than throwing on it, and
# it lands on `unnamed-rule` above with the cause that actually applies. The example is corrected
# rather than dropped, because it is the one shape a reader is likely to reach for when testing.
if [ "$bp_code" = "unreadable-rules" ]; then
decide ask "H6/H10 merge gate: this repo's branch-protection rules came back in a shape this hook could not parse, so whether 'review-verdict/h10' is required on '$base_ref' is unknown. Check the rules manually, or merge immediately instead of scheduling."
fi
if [ "$bp_code" != "200" ] || [ -z "${bp//[[:space:]]/}" ] || ! printf '%s' "$bp" | jq -e 'type == "object"' >/dev/null 2>&1; then
decide ask "H6/H10 merge gate: could not read this repo's branch-protection rules (HTTP '${bp_code:-none}' — Gitea unreachable, or these credentials lack the repo-admin scope that endpoint needs), so whether 'review-verdict/h10' is required on '$base_ref' is unknown. Scheduling an auto-merge is only safe while 'review-verdict/h10' is a REQUIRED check there (ersatztv#622) — confirm that manually, or merge immediately instead of scheduling."
fi
# The membership test is `any(.[]; . == …)` over a value FIRST PROVEN to be an array of strings —
# never `index()`. `index` on a STRING is substring search, so a `status_check_contexts` that
# arrived as the string "prefix-review-verdict/h10-suffix" would answer "yes" and auto-grant a
# merge on a base where no such context is required. That is a FALSE-OPEN in the gate, reachable
# from any payload shape drift, and it is the direction that matters: a false-closed costs a
# prompt, a false-open costs an unreviewed merge.
#
# Validating `$bp` as an object does not make its MEMBERS well-formed, which is the same
# one-level-down swallow that survived the first fix in the #632 base-change guard — the
# validation domain has to match the CONSUMPTION domain, not stop at the top-level type. So the
# shape is checked explicitly and anything else becomes "unknown" rather than a decision.
#
# `null` and `[]` are legitimate (an unprotected-in-practice branch) and answer "no", not
# "unknown": absent IS the finding here, not a read failure. The word is then matched
# exhaustively, because "" is not a third synonym for "no".
# `// []` defaults on FALSE as well as on null, because jq's alternative operator fires for both.
# So `"status_check_contexts": false` — a malformed shape — became `[]` and answered "no", i.e. a
# confident DENY derived from a payload that was never understood. Absent and null are defaulted
# explicitly; every other non-array is "unknown".
# `enable_status_check` is validated as a BOOLEAN before it is trusted, for the same reason the
# contexts list is: `"true"` (the string) is not `true`, and comparing it to `true` yields a
# confident "no" -> deny derived from a payload never understood. Every malformed shape on this
# endpoint has to reach the same "unknown" -> ask arm, or the tri-state is only two states.
guarded=$(printf '%s' "$bp" \
| jq -r 'def ctxs: if (has("status_check_contexts") | not) or .status_check_contexts == null
then [] else .status_check_contexts end;
if (.enable_status_check | type) != "boolean" then "unknown"
elif (ctxs | type) != "array" or any(ctxs[]; type != "string") then "unknown"
elif (.enable_status_check == true) and any(ctxs[]; . == "review-verdict/h10") then "yes"
else "no" end' 2>/dev/null || true)
case "$guarded" in
yes) : ;;
no) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is NOT a required status check on '$base_ref' (branch protection reports enable_status_check/status_check_contexts without it). A scheduled auto-merge is safe ONLY because that per-sha required check stops a commit pushed after scheduling from merging unreviewed (ersatztv#622); without it, arming merge_when_checks_succeed freezes consent at a head Gitea may not be the one to merge. Restore it in branch protection, or merge immediately (without merge_when_checks_succeed) once CI is green." ;;
*) decide ask "H6/H10 merge gate: branch protection for '$base_ref' came back in an unexpected shape, so the 'review-verdict/h10' required check that makes a scheduled auto-merge safe (ersatztv#622) could not be confirmed either way. Check it manually, or merge immediately instead of scheduling." ;;
*) decide deny "H6 merge gate: BLOCKED — PR #$pr CI status is '$state', not 'success'. Wait for a green build (or pass merge_when_checks_succeed to let Gitea gate it) before merging." ;;
esac
fi
@@ -699,185 +132,52 @@ comments=$(gq "repos/$owner/$repo/issues/$pr/comments?limit=100")
if [ -z "$comments" ]; then
decide ask "H10 merge gate: could not fetch PR #$pr comments to verify a head-referencing review verdict ($short). Confirm the adversarial/Codex review covered the latest commit before merging."
fi
# Classification is delegated to `scripts/check-review-verdict.sh` — the single source of truth for
# the H10 grammar, extracted in #629 so it could be TESTED. While it lived here it had none, and three
# false-opens survived in it: a prefix-matched token (`MERGEABLE-LATER` graded positive), a verdict
# inside a fenced code block (documentation showing the convention counted as a real verdict), and a
# sha taken from the first `@<hex>` anywhere on the line (a markdown link could supply it). Every
# decision the classifier makes is documented there; this file only maps a class onto a hook decision.
# RESOLVED FROM `$repo_root`, never `$CLAUDE_PROJECT_DIR` — the rule, the threat model and the
# boundary are in `process.hook-resolves-inputs-from-repo-root` (ersatztv#858). Written once there
# rather than twice here: this file carried two resolutions of the same question, and the guard-scope
# arm below is the other one. Two answers in one file is the state most likely to be "tidied" toward
# the weaker side, so neither site restates the argument now.
#
# Site-specific consequence only: a `$CLAUDE_PROJECT_DIR` naming a sibling worktree — routine here —
# would classify THIS PR's comments with THAT tree's copy of the H10 grammar.
#
# `ETV_HOOK_FIRE_LIB` at the top of this file is bound the same way, and for a STRONGER reason — it
# is sourced, so it is code. See the block above it. The twelve other hooks still resolve it from
# the env var and are ersatztv#891.
verdict_script="$repo_root/scripts/check-review-verdict.sh"
if [ ! -x "$verdict_script" ]; then
decide ask "H10 merge gate: verdict classifier not found at $verdict_script, so the review state can't be derived. Confirm the review covered the latest commit before merging."
fi
# An input error (exit 2) is NOT a classification — fall through to a human rather than guessing.
if ! class=$(printf '%s' "$comments" | "$verdict_script" --head "$sha" 2>/dev/null); then
decide ask "H10 merge gate: could not classify the review verdicts on PR #$pr (malformed comments payload or unreadable head). Confirm the review covered the latest commit ($short) before merging."
# Verdict lines across all comment bodies: a real verdict line STARTS with the marker (after optional
# leading whitespace). Anchoring to line-start is deliberate — it rejects a comment that merely QUOTES
# the positive template mid-sentence (an instruction "please post: Review-verdict: MERGEABLE @ <sha>",
# or the gate's own suggestion text echoed back), which would otherwise self-approve the merge.
verdicts=$(printf '%s' "$comments" | jq -r '.[].body // empty' 2>/dev/null | grep -iE '^[[:space:]]*review-verdict:' || true)
if [ -z "$verdicts" ]; then
decide ask "H10 merge gate: no 'Review-verdict:' comment found on PR #$pr referencing head $short. Post the adversarial/Codex verdict (e.g. 'Review-verdict: MERGEABLE @ $short'), or confirm the review covered the latest commit and approve."
fi
# Classify each verdict line by the sha it references (its "@ <sha>" field) and its verdict word.
# A line references the CURRENT head iff head BEGINS WITH that sha token AND the token is >=7 chars
# (git short-sha prefix semantics) — NOT a loose substring test: an older sha that merely contains
# the head prefix, or the head prefix appearing in an unrelated URL on the line, must NOT count
# (adversarial false-opens). The verdict token must sit right after the marker on the same line.
head_pos=0; head_neg=0; stale=0
while IFS= read -r line; do
[ -n "$line" ] || continue
# The sha the line references: the hex token in its "@ <sha>" field (>=7 chars), lowercased.
ref=$(printf '%s' "$line" | grep -ioE '@[[:space:]]*[0-9a-f]{7,40}' | head -1 \
| grep -oiE '[0-9a-f]{7,40}' | tr 'A-F' 'a-f' || true)
is_pos=0
# Positive iff the line's OWN leading verdict word (right after the line-start marker) is positive —
# anchored so a second, later `review-verdict: mergeable` substring on a BLOCKED line can't flip it.
if printf '%s' "$line" | grep -iqE '^[[:space:]]*review-verdict:[[:space:]]*(mergeable|approved|lgtm)'; then is_pos=1; fi
[ -z "$ref" ] && continue # marker present but no @<sha> -> falls through to the final ask
case "$sha" in
"$ref"*) if [ "$is_pos" = 1 ]; then head_pos=1; else head_neg=1; fi ;;
*) stale=1 ;;
esac
done <<VERDICTS
$verdicts
VERDICTS
case "$class" in
negative)
# A negative verdict on head wins over a positive one (a later BLOCKED retracts an earlier
# MERGEABLE on the SAME head; if the head were fixed the sha would change, so this can't
# wrongly block).
decide deny "H10 merge gate: BLOCKED — a review verdict for the current head ($short) is negative (BLOCKED/NOT-MERGEABLE). Resolve the findings and post a fresh 'Review-verdict: MERGEABLE @ $short' before merging PR #$pr." ;;
stale)
decide deny "H10 merge gate: BLOCKED — a review-verdict comment references an older commit, not the current head ($short). The latest commit(s) are unreviewed (ersatztv#242: re-review the fix commit, not just the initial diff). Re-review the head and post 'Review-verdict: MERGEABLE @ $short'." ;;
unknown)
decide ask "H10 merge gate: a 'Review-verdict:' comment on PR #$pr uses an unrecognized verdict token (not MERGEABLE/APPROVED/LGTM/BLOCKED/NOT-MERGEABLE). It is deliberately NOT read as approval. Post a verdict using the documented vocabulary — e.g. 'Review-verdict: MERGEABLE @ $short'." ;;
no-sha)
# Marker(s) exist but reference no sha at all -> ask (don't mislabel as a stale older-commit review).
decide ask "H10 merge gate: a 'Review-verdict:' comment on PR #$pr references no commit sha in its own '@ <sha>' field. Post one referencing the current head ($short) — e.g. 'Review-verdict: MERGEABLE @ $short' — or confirm the review covered the latest commit and approve." ;;
absent)
decide ask "H10 merge gate: no 'Review-verdict:' comment found on PR #$pr referencing head $short. Post the adversarial/Codex verdict (e.g. 'Review-verdict: MERGEABLE @ $short'), or confirm the review covered the latest commit and approve." ;;
positive) : ;;
*)
decide ask "H10 merge gate: unrecognized verdict classification '$class' for PR #$pr. Confirm the review covered the latest commit ($short) before merging." ;;
esac
# --- (d) Guard-scope freshness (ersatztv#787): the committed mirror of `main`'s required status
# checks must still match the server. ------------------------------------------------------
# ORDERED LAST, and that is a severity argument rather than a stylistic one. Every check above
# can DENY; this one can only ever downgrade an otherwise-satisfied auto-grant to a prompt. Run
# earlier it would preempt those verdicts and report a stale guard scope at a reader whose merge
# is blocked for a completely different and more serious reason, and it would ask on payloads the
# checks above are about to reject anyway. Placed here it is also PAST the point where the two
# merge paths converge, so it covers both without duplicating anything.
# `scripts/tests/test_ci_dropped_step_guard.py` DERIVES which jobs must carry per-step execution
# markers from `.gitea/required-status-contexts.json`, because its CI job checks out with
# `persist-credentials: false` and cannot ask Gitea. That makes the snapshot the single
# hand-maintained input in the chain: a fourth required context added on the server leaves the
# snapshot — and therefore the guard's scope — silently behind, which is the whole of #787.
#
# THIS RUNS ON BOTH MERGE PATHS, deliberately, and it is placed here rather than beside the
# branch-protection read in the scheduled-auto-merge branch for that reason.
#
# WHAT IT DOES NOT COVER, said here rather than left to be discovered: a PR whose changed files are
# all docs/process — `.gitea/` included — exits at the docs-only passthrough far above, so this arm
# never runs for it. A PR that edits ONLY `.gitea/required-status-contexts.json` is docs-only BY
# CONSTRUCTION, and that is exactly the snapshot-NARROWING direction the decision record names as
# this design's residual. Excluding that path from the allow-list would not buy the protection it
# looks like it would: this arm compares the live server against the snapshot in the LOCAL CHECKOUT,
# not against the version the PR proposes, so it cannot see a narrowing that has not landed yet.
# What does hold is that the passthrough is a passthrough — a human prompt, never an auto-grant —
# which is the `.gitea/` treatment ersatztv#317 asked for. That read is inside
# `else` (mwcs = true) and never executes on an immediate merge, which is the common case; hanging
# the freshness check off it would fire it only when an auto-merge is armed. This file already
# records that exact defect one section up — the base re-read "first landed inside the
# scheduled-auto-merge branch only", and cold review found scheduled+retarget denied while
# immediate+retarget auto-GRANTED. Same shape, so it is not repeated here.
#
# It reads `main` (the branch the snapshot names), NOT `$base_ref`. That is a DIFFERENT question
# from the one the scheduled branch asks — "is review-verdict/h10 required on the base I am merging
# into" — so this is not a second copy of that classifier and the two cannot drift into disagreeing:
# they consume different fields of different rules for different decisions.
#
# ASK, NEVER DENY. Drift does not make THIS merge unsafe: Gitea enforces the live required set
# server-side, so a newly required context with no status blocks the merge on its own. What has gone
# stale is a guard's scope — a different artifact, on a different clock. Denying would state
# something false about the change in front of the reader. Every non-`match` class asks, so a
# comparison that could not be made is surfaced rather than skipped (`unknown` is not `fine`).
# ONE base for both the checker and the snapshot, and it is `$repo_root` — see
# `process.hook-resolves-inputs-from-repo-root` for why an env var may not select either
# (ersatztv#787, #858). The reason specific to THIS arm is that both halves of a comparison are
# resolved here: from two different roots the hook would classify one checkout's snapshot with
# another checkout's script — mismatched halves of a comparison whose entire job is to detect a
# mismatch — and answer `match` about a tree nobody asked about.
ctx_base="$repo_root"
ctx_snapshot="$ctx_base/.gitea/required-status-contexts.json"
ctx_script="$ctx_base/scripts/check-required-contexts.sh"
# THIS ARM IS ABOUT ONE REPO, and the merge tool is not. Every other check here reads
# `$owner/$repo` from the tool input and is repo-agnostic; this one compares a HARDCODED branch
# against a snapshot committed in THIS checkout. Merging a PR in another repo from a session opened
# here would otherwise weigh that repo's live contexts against this repo's mirror and report a
# confident, flatly false finding about it — measured: server-management returns `[]`, which
# classifies as `nomatch`. So the snapshot names the repo it describes and the arm runs only for it.
# An unreadable snapshot cannot answer "is this my repo?" either, so it asks rather than skipping.
ctx_repo=$(jq -r 'if (.repo | type) == "string" then .repo else "" end' "$ctx_snapshot" 2>/dev/null || true)
if [ -z "$ctx_repo" ]; then
decide ask "H6 merge gate: $ctx_snapshot is missing, unreadable, or names no \`repo\`, so the dropped-step guard's scope could not be checked against branch protection — nor could it be established whether this snapshot even describes $owner/$repo (ersatztv#787). Restore the file, or check the required checks manually."
# A negative verdict on head wins over a positive one (a later BLOCKED retracts an earlier MERGEABLE
# on the SAME head; and if the head were fixed the sha would change, so this can't wrongly block).
if [ "$head_neg" = 1 ]; then
decide deny "H10 merge gate: BLOCKED — a review verdict for the current head ($short) is negative (BLOCKED/NOT-MERGEABLE). Resolve the findings and post a fresh 'Review-verdict: MERGEABLE @ $short' before merging PR #$pr."
fi
# CASE-FOLDED, because Gitea resolves owner/repo case-insensitively: verified live, both
# `/repos/timothy/ersatztv` and `/repos/TIMOTHY/ErsatzTV` answer 200. A byte-exact compare would let
# any case variant sail through every other arm and SKIP this one, so drift would go unreported with
# no ask — the gate failing open on a spelling. The hook already treats case folding as
# decision-relevant one section up, where `MAIN` vs `main` makes the governing rule undecidable.
ctx_repo_fold=$(printf '%s' "$ctx_repo" | tr '[:upper:]' '[:lower:]')
target_repo_fold=$(printf '%s' "$owner/$repo" | tr '[:upper:]' '[:lower:]')
if [ "$ctx_repo_fold" = "$target_repo_fold" ]; then
if [ ! -x "$ctx_script" ]; then
decide ask "H6 merge gate: the required-contexts checker is missing or not executable at $ctx_script, so whether the dropped-step guard's scope still matches branch protection on 'main' could not be derived (ersatztv#787). Check it manually, or restore the script."
fi
# THE SHARED READ (ersatztv#859). On a scheduled merge the arm above already fetched this; here that
# call is a cache hit, so the endpoint is read once per run instead of twice. On the IMMEDIATE path
# this is the only consumer and it performs the fetch itself, which is why the call sits AFTER the
# `[ ! -x "$ctx_script" ]` check above: a missing checker must ask without having touched the
# network, and a test pins exactly that by asserting no branch-protection URL was recorded.
fetch_branch_protections
if [ "$bp_cache_code" = "mktemp-failed" ]; then
decide ask "H6 merge gate: could not allocate a temp file to read branch protection for the guard-scope freshness check (ersatztv#787)."
fi
ctx_code=$bp_cache_code
# ONE temp file, and it holds the checker's STDERR. Until ersatztv#859 this was `mktemp` for the
# payload plus an unmanaged `$bpf.err` beside it — a second path mktemp never created and therefore
# never made unpredictable. The payload now comes from the shared cache over a pipe, so the only
# thing still needing a file is the diagnostic, and it gets the mktemp'd one.
ctx_err=$(mktemp) || decide ask "H6 merge gate: could not allocate a temp file for the guard-scope freshness check's diagnostics (ersatztv#787)."
if [ "$ctx_code" = "200" ]; then
# stderr is KEPT, not sent to /dev/null. The checker exits 2 with a diagnostic on a usage error —
# an unreadable snapshot, a branch mismatch, a missing classifier — and discarding it made all of
# those arrive at the operator as the catch-all's "returned 'nothing'", which names no cause. That
# is the same states-a-cause-that-did-not-happen shape this arm was careful about elsewhere.
ctx_class=$(printf '%s' "$bp_cache" | "$ctx_script" --branch main --snapshot "$ctx_snapshot" 2>"$ctx_err" || true)
ctx_diag=$(tr '\n' ' ' < "$ctx_err" 2>/dev/null | cut -c1-300 || true)
else
ctx_class=readfail
ctx_diag=""
fi
rm -f "$ctx_err"
case "$ctx_class" in
match) : ;;
drift)
decide ask "H6 merge gate: the required status checks on 'main' no longer match .gitea/required-status-contexts.json (ersatztv#787). scripts/tests/test_ci_dropped_step_guard.py derives its marked-job scope from that snapshot, so until it is reconciled a required context may have NO dropped-step guard — a step the runner drops would conclude success and take that check green having done no work (ersatztv#756). Re-read the live list and update the snapshot in a PR (the guard will then demand markers for any newly required job, or an ACCOUNTED_ELSEWHERE entry naming what covers it). This does not make the merge in front of you unsafe — Gitea enforces the live required set server-side — so approve if you have judged it unrelated." ;;
nomatch)
decide ask "H6 merge gate: no branch-protection rule governs 'main' at all, so the required status checks the dropped-step guard scopes itself to could not be confirmed (ersatztv#787). Branch protection on 'main' is what makes 'review-verdict/h10' load-bearing (ersatztv#743) — check it before merging." ;;
undecidable)
decide ask "H6 merge gate: a glob branch-protection rule could govern 'main', so which rule's required contexts to compare against .gitea/required-status-contexts.json is not derivable without reimplementing Gitea's matcher (ersatztv#787). Confirm the required checks manually." ;;
unreadable)
decide ask "H6 merge gate: branch protection for 'main', or .gitea/required-status-contexts.json itself, came back in a shape the required-contexts checker could not consume, so whether the dropped-step guard's scope is still current is unknown (ersatztv#787). Check the rules and the snapshot manually." ;;
readfail)
decide ask "H6 merge gate: could not read branch protection for the guard-scope freshness check (HTTP '${ctx_code:-none}' — Gitea unreachable, or these credentials lack the repo-admin scope that endpoint needs), so whether .gitea/required-status-contexts.json is still current is unknown (ersatztv#787). Confirm the required checks on 'main' manually." ;;
*)
decide ask "H6 merge gate: the required-contexts checker returned '${ctx_class:-nothing}', which is not a class this hook understands, so the dropped-step guard's scope could not be confirmed against branch protection (ersatztv#787).${ctx_diag:+ It said: ${ctx_diag}}Check scripts/check-required-contexts.sh." ;;
esac
fi # end of the guard-scope freshness arm (opened at `if [ "$ctx_repo_fold" = ... ]` above). The
# body is left unindented to match the rest of this file, which is flat throughout; the marker
# is here because the block is long enough that its extent is otherwise easy to misread.
if [ "$class" = "positive" ]; then
# (a) CI + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant.
# The reason string must not claim more than was actually checked: on the merge_when_checks_succeed
# path this hook never read the CI status at all (it is delegated to Gitea), so saying "CI green"
# there was a plain falsehood in the one message a human reads to decide whether to trust the gate.
if [ "$mwcs" = "true" ]; then
decide grant "H6/H10 merge gate: satisfied — all Done-when boxes ticked, and both a positive Review-verdict comment and the 'review-verdict/h10' status cover the current head ($short). CI is gated by Gitea (merge_when_checks_succeed). A commit pushed before Gitea merges clears the sha-bound verdict status and is blocked by the 'review-verdict/h10' required check (ersatztv#622) — which this hook has just CONFIRMED is still required on '$base_ref' — read from the repo's full rule list and matched with Gitea's own plain-vs-glob split, refusing rather than guessing wherever precedence or folding is not derivable. That guarantee holds while that branch protection stands; if it is weakened after this check, nothing here would see it (ersatztv#778). Auto-granted."
fi
if [ "$head_pos" = 1 ]; then
# (a) CI green + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant.
decide grant "H6/H10 merge gate: satisfied — CI green, all Done-when boxes ticked, and a positive Review-verdict references the current head ($short). Auto-granted (no separate confirmation needed)."
fi
if [ "$stale" = 1 ]; then
decide deny "H10 merge gate: BLOCKED — a review-verdict comment references an older commit, not the current head ($short). The latest commit(s) are unreviewed (ersatztv#242: re-review the fix commit, not just the initial diff). Re-review the head and post 'Review-verdict: MERGEABLE @ $short'."
fi
# Marker(s) exist but reference no sha at all -> ask (don't mislabel as a stale older-commit review).
decide ask "H10 merge gate: a 'Review-verdict:' comment on PR #$pr references no commit sha. Post one referencing the current head ($short) — e.g. 'Review-verdict: MERGEABLE @ $short' — or confirm the review covered the latest commit and approve."
# Unreachable: the `case` above exits on every class, and `positive` exits in the block above. Kept as
# a fail-safe so a future class added to the classifier without a branch here cannot fall off the end
# of the script (which would exit 0 = silent passthrough, the one outcome a gate must never produce).
decide ask "H10 merge gate: verdict classification for PR #$pr produced no decision. Confirm the review covered the latest commit ($short) before merging."
# All derivable and satisfied -> auto-grant (defensive: the head_pos branch above already exits here).
decide grant "H6/H10 merge gate: satisfied — auto-granted."
-7
View File
@@ -2,13 +2,6 @@
# PreToolUse / browser-navigate — deny opening download/stream endpoints in a tab
# (they hang the MCP session; curl them instead). Fail-open on parse trouble.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-nav-guard "" capture || true
input=$(cat)
url=$(printf '%s' "$input" | jq -r '.tool_input.url // ""' 2>/dev/null || true)
@@ -8,13 +8,6 @@
# So the main tree (never marked) and pre-convention worktrees (no marker) are unaffected;
# only a commit/merge into another session's marked worktree is blocked.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-worktree-guard "" capture || true
input=$(cat)
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
cwd=$(printf '%s' "$input" | jq -r '.cwd // ""' 2>/dev/null || true)
+86 -434
View File
@@ -1,132 +1,37 @@
---
name: ersatztv
description: "ErsatzTV custom IPTV channel management — REST API, SQLite DB, Jellyfin integration, FFmpeg profiles. Use when creating or modifying IPTV channels, managing collections and schedules, building playouts, adding channel logos, scanning media libraries, troubleshooting channel issues, or resetting playouts. Also use for any questions about the ErsatzTV database schema (Channel, Collection, ProgramSchedule, Playout tables), M3U/XMLTV feeds, custom TV channel setup, or the channel creation checklist. IMPORTANT: the fork has a full versioned REST API at /api/v1 including write paths — prefer it over SQLite scripting, which is a recovery fallback only."
description: ErsatzTV custom IPTV channel management — REST API, SQLite DB, Jellyfin integration, FFmpeg profiles. Use when managing custom TV channels.
---
> **Canonical copy: `~/ersatztv/.claude/skills/ersatztv/SKILL.md`** (ersatztv owns this skill per that
> repo's `CLAUDE.md` → Project Boundaries and `process.ersatztv-owns-code-not-operations`). Both
> `~/server-management/.claude/skills/ersatztv` **and** `~/media-management/.claude/skills/ersatztv`
> are symlinks to it. Edit it in the ersatztv repo; never fork a second copy (ersatztv#617, #755) —
> media-management's copy had silently become a divergent fork still describing a Blazor UI that no
> longer exists, which is what made this the rule rather than a preference.
>
> **Channel OPERATIONS (create/edit a live channel, lineup, collection, schedule, playout, logo,
> overlay) are `media-management`'s job**; ersatztv owns the fork code, `/api/v1`, CI and releases.
> This skill serves both — it is the operator's reference *and* the developer's map.
# ErsatzTV Channel Management
Container: `ersatztv` | Port: `8409`
Web UI: `https://ersatztv.tblindustries.be` (via bumblebee's `external-proxy``192.168.1.29:8409`) or `http://localhost:8409` on the host
Host: **jazz** (`192.168.1.29`) since 2026-07-20 (#633) — moved off bumblebee together with Jellyfin. `dispatcharr` and `plex` stayed on bumblebee, so Dispatcharr now reaches ErsatzTV **by IP** (`http://192.168.1.29:8409`), not by Docker DNS name.
Compose env: `ForwardedHeaders__KnownNetworks=192.168.1.99/32` (proxied traffic arrives SNAT'd from bumblebee's LAN address; wrong value breaks Authelia OIDC login only, plain HTTP still works)
Host: **jazz (192.168.1.29)**. Prod container `ersatztv` port **8409**; test `ersatztv-test` port
**8410** (tracks `:latest` via Komodo auto-update, daily 03:00 — a same-day validation needs the
manual pull below).
SQLite DB: `~/downloadswarm/ersatztv/ersatztv.sqlite3` (owned by root — use `sudo sqlite3`)
Image: `192.168.1.95:3000/timothy/ersatztv:prod` (our fork; **floating** release tag — check `git tag -l 'v*' --sort=-v:refname | head -1` in `~/ersatztv` for the current release rather than trusting a version written here). Upstream `ghcr.io/ersatztv/ersatztv` was archived at v26.3.0 and is **not** what runs here.
Release tags are `vYY.<release-seq>.<patch>` — year · sequential release-within-year · patch — **not** year.month.
## Test/Prod topology — fork CI images (#481)
We maintain an **ErsatzTV fork** (`~/ersatztv`); its Gitea Actions pipeline builds and pushes images to the
private Gitea registry `192.168.1.95:3000/timothy/ersatztv` on every push to `main` (`:latest` + `:<short-sha>`)
and, on a `v*` tag, additionally `:prod` + `:<version>`. jazz is `docker login`'d to that registry and has `192.168.1.95:3000` in `insecure-registries`.
| | Prod | Test |
|---|---|---|
| Container | `ersatztv` | `ersatztv-test` |
| Host port | 8409 | 8410 |
| Stack | Komodo **`jazz-media`**; source `docker/jazz/stacks/media-servers/compose.yaml` (stack name ≠ directory — `media-servers` is bumblebee's; Komodo stack names are globally unique) | Komodo `ersatztv`; source `docker/jazz/stacks/ersatztv/compose.yaml` |
| Image | `192.168.1.95:3000/timothy/ersatztv:prod` (floating release tag) | `192.168.1.95:3000/timothy/ersatztv:latest` (fork CI) |
| Config (host) | `~/downloadswarm/ersatztv/``/config` | `~/downloadswarm/ersatztv-test/``/config` (one-time prod snapshot, refresh on demand) |
| Jellyfin/Dispatcharr tuner | connected (live lineup) | **NOT** wired downstream (avoids ghost channels) |
| Media mounts | RO | same mounts, RO |
| `/dev/dri` | yes (**VAAPI on Intel iHD**, jazz — see hw note) | yes (`/dev/dri` + `group_add: '992'`) |
| Auto-update | **None** (`auto_update: false`) — promotion is a manual `DeployStack jazz-media`, with no 03:00 fallback | Komodo auto-update, daily 03:00 (tracks `:latest`) |
| Env | `TZ`, restricted forwarded-header network, empty-by-default local-admin seed hook | `TZ`, `ETV_CONFIG_FOLDER=/config`, `ETV_TRANSCODE_FOLDER=/transcode`, `ETV_DISABLE_VULKAN=1` |
**Watchtower is retired.** Test auto-updates via Komodo; **prod does not**`auto_update: false`, so
promoting a release is always a manual `DeployStack jazz-media`. Prod's stack has a
fail-closed pre-deploy hook: a changed compose block or `:prod` digest triggers a PBS-backed snapshot and then a
migration rehearsal against a throwaway copy of that snapshot before container recreation (#585/#589).
**Refresh test snapshot from prod** (zero prod downtime — WAL online backup):
```bash
ssh timothy@192.168.1.29
docker stop ersatztv-test
sudo sqlite3 ~/downloadswarm/ersatztv/ersatztv.sqlite3 ".backup '/home/timothy/downloadswarm/ersatztv-test/ersatztv.sqlite3'"
sudo rsync -a --exclude='ersatztv.sqlite3*' --exclude='logs/' ~/downloadswarm/ersatztv/ ~/downloadswarm/ersatztv-test/
docker start ersatztv-test
```
**Prod cutover to the fork** — ✅ DONE 2026-06-27 (#481). Prod runs `…/timothy/ersatztv:prod` (v26.3.1);
validated `:prod` on test first, then `etv-prod-deploy.sh` backed up + cut over (43 channels, healthy,
clean migrations). Downstream (Dispatcharr M3U acct 3 + EPG src 9) is name-based, so the container IP
change was transparent. Prod stays a **manual** gate (no Watchtower label) and still lives in the
`media-servers` stack (the optional move into the `ersatztv` stack was not done).
**Future prod releases** (push `v*` tag in `~/ersatztv` → CI builds `:prod`/`:<version>`): scan the immutable
`:<version>` image on jazz first, then execute Komodo `DeployStack` for `jazz-media`. The pre-deploy hook
backs up and runs the migration-on-prod-copy smoke before recreation. **There is no auto-update fallback for
prod** — if you don't `DeployStack`, nothing ships. Note the stack is named **`jazz-media`** even though the
compose *project* is still `media-servers`; a dead `media-servers` stack lingers on bumblebee and deploying it
fails silently. Roll back with the immutable prior image plus the pre-deploy DB snapshot; migrations are
forward-only. See the `komodo` skill and `docs/Docker/ErsatzTV.md` for the current procedure.
## Backup & deploy safety (#482)
Every prod deploy runs forward-only EF Core migrations against the live 285 MB SQLite DB — a bad one
can't be undone by re-deploying the old image, so the **only** rollback is restoring a pre-deploy DB
snapshot. Three scripts in `~/scripts/` (source of truth: `scripts/` in this repo) handle
them. **⚠️ These were installed on bumblebee, where ErsatzTV no longer runs (#633) — verify they exist on
jazz and that the Komodo `pre_deploy` hook is set on the `jazz-media` stack before relying on
"no backup, no deploy". Until confirmed, take a manual `etv-backup.sh` snapshot before every prod deploy.**
this. **Run as root** (DB + PBS creds are root-owned) except the deploy wrapper (run as `timothy`).
| Script | Run as | What it does |
|---|---|---|
| `etv-backup.sh [--target prod\|test] [--no-offbox]` | root (sudo) | Online `sqlite3 .backup` (zero-downtime) + `integrity_check`, provenance `manifest.txt` (image ref/digest + last `__EFMigrationsHistory` id), bundles `data-protection/` + `*-secrets.json`. Local **keep-last-5** under `~/downloadswarm/ersatztv-backups/<UTC-ts>/`; prod also pushes off-box to PBS. Prints the snapshot dir on stdout. |
| `etv-prod-deploy.sh` | **timothy** (needs private-registry creds; sudo's for the backup) | Backup (abort deploy if it fails) → `compose pull` + `up -d ersatztv` → health + M3U gate → prints a copy-paste rollback block on trouble. |
| `etv-restore.sh --target prod\|test --from <snapshot-dir>` | root (sudo) | Verifies snapshot → stop → saves current DB aside (`*.pre-restore-<ts>`) → swaps DB, drops stale `-wal/-shm`, restores `data-protection` → start → health/channel check. |
- **Off-box:** prod backups go to PBS `data-local` (.68) as backup-id **`ersatztv-predeploy`** (own
group, dedups against the nightly host backup), via the existing `/root/.proxmox-backup-client.env`.
- **Retention:** local keep-last-5 (instant rollback); PBS via the datastore-wide `data-local-prune`
job (7 daily / 4 weekly / 6 monthly), no separate prune job needed.
- **Restore from PBS** instead of a local dir:
```bash
source /root/.proxmox-backup-client.env
proxmox-backup-client restore ersatztv-predeploy/<snapshot> etv.pxar <outdir>
sudo ~/scripts/etv-restore.sh --target prod --from <outdir>
```
- `docker exec` always curls the container-internal port **8409** (even for test, whose host port is
8410). `etv-restore.sh` leaves a `*.pre-restore-<ts>` safety copy in `/config` — delete once happy.
- Validated 2026-06-27: first prod backup → PBS group created; full restore round-trip on `ersatztv-test`
returned 43 channels. Design: `plans/2026-06-27-ersatztv-backup-before-deploy-design.md`.
Image: **our fork**, `192.168.1.95:3000/timothy/ersatztv` (`:prod` / `:latest`). Upstream
`ghcr.io/ersatztv/ersatztv` was archived at v26.3.0 and is NOT what runs here.
## Architecture
**ErsatzTV is for channel creation only.** Consumers (Jellyfin, Kodi) never connect to ErsatzTV directly — everything goes through Dispatcharr as the single aggregation point. Pipeline: ErsatzTV → Dispatcharr → Jellyfin/Kodi.
**This section described upstream v26.3.0 and was wrong for the fork — corrected 2026-07-21.**
ErsatzTV uses **MediatR + the ChicoryTV React SPA**. The legacy Blazor UI was removed in v26.7.0 (#91
phase b) — the SPA at `/app` is the **only** UI, and legacy routes 302 there. The versioned `/api/v1`
surface provides full CRUD — channels, collections, schedules, playouts and media sources; browser calls
use a local-admin/OIDC session cookie plus `X-CSRF` on mutations, and machine clients use `X-Api-Key`.
**Do not hand-edit SQLite for something the API can do** — direct SQLite writes are a recovery fallback,
not the normal management path, and the DB recipes below survive only for gaps with no endpoint.
- The **Blazor UI is gone** (#91 phase b). The only UI is the ChicoryTV React SPA at `/app`; legacy
routes 302 there.
- There **is** a full versioned REST API under **`/api/v1`**, write paths included — channels,
collections, schedules, playouts and media sources have CRUD. **Do not hand-edit SQLite for
something the API can do.** The DB-scripting recipes below survive only for gaps with no endpoint.
- Controllers stay thin and delegate to MediatR handlers; the SPA talks to `/api/v1` only.
- Authoritative endpoint list: `docs/endpoint-index.md` (generated) + `docs/api-conventions.md`.
Prefer those over any list in this file — a hand-maintained copy drifts.
Controllers stay thin and delegate to MediatR handlers. **Authoritative endpoint list:
`docs/endpoint-index.md` (generated) + `docs/api-conventions.md` in the ersatztv repo — prefer those
over any list in this file**, which is hand-maintained and drifts.
## REST API access (auth-gated — read before curling)
## REST API
Calls need **`X-Api-Key`** (machine clients) or a browser session. An unauthenticated call returns a
401 JSON body that is easy to mistake for real data — see the silent-401 trap in Gotchas.
```bash
# Via docker exec (api.key is readable inside the container)
docker exec ersatztv curl -s -H "X-Api-Key: $(docker exec ersatztv cat /config/api.key)" \
http://localhost:8409/api/v1/ENDPOINT
```
From the **host**, the key file is root-owned `0600`, so an unsudo'd `cat` fails *silently* and sends an
empty header. Read it with `sudo`, inline, so the value is never printed:
The key file is **root-owned `0600`**, so `cat` as `timothy` fails *silently* and yields an empty
header. Read it with `sudo`, inline, so the value is never printed:
```bash
# prod (8409); test is identical with .../ersatztv-test/api.key and port 8410
@@ -134,23 +39,12 @@ ssh timothy@192.168.1.29 'K=$(sudo -n cat /home/timothy/downloadswarm/ersatztv/a
curl -s -H "X-Api-Key: $K" http://localhost:8409/api/v1/channels'
```
### Paging — 0-based (ersatztv#616, `api.paging-zero-based`)
- **`pageNum` is 0-based** across the whole `/api/v1` surface and every wrapper of it (MCP tools, SPA
hooks, docs). Starting at 1 silently skips a page and returns a short set **with no error**.
- **`pageSize` is clamped per-endpoint** — 100 typical, 200 auto-tune members, 1000 search/all-items —
and the offset derives from the *effective* (clamped) size, not the requested one. Page to
completeness against `totalCount`; never conclude "that's all of them" from a single page.
- **`POST /api/v1/channels/{id}/playout/reset` takes a CHANNEL id, not the playout id.** The id spaces
overlap numerically, so passing a playout row's `Id` returns a plausible 202 against a *different*
channel. Playout rows carry `channelId` — use that.
Settings live under `/api/v1/settings/*``settings/ffmpeg` (`workAheadSegmenterLimit`,
`qsvExtraHardwareFrames`) and `settings/logging` (`streamingMinimumLogLevel`). Note the order: it is
`settings/ffmpeg`, **not** `ffmpeg/settings`.
Refresh test to the newest `:latest` without waiting for the 03:00 auto-update — scope it to the
service, since a bare `up -d` would recreate everything else in the compose project:
Refresh test to the newest `:latest` without waiting for 03:00 — scope it to the service, since a
bare `up -d` would recreate everything else in the compose project:
```bash
D=/etc/komodo/stacks/ersatztv/docker/jazz/stacks/ersatztv
@@ -158,18 +52,26 @@ docker compose -f $D/compose.yaml pull ersatztv-test
docker compose -f $D/compose.yaml up -d --no-deps ersatztv-test
```
The unversioned `/api/*` endpoints below predate the `/api/v1` surface — verify one against
`docs/endpoint-index.md` before relying on it.
```bash
# Via docker exec
docker exec ersatztv curl -s http://localhost:8409/api/ENDPOINT
```
### Read Endpoints (GET)
```
/api/v1/channels # List channels
/api/v1/collections # List collections
/api/v1/schedules # List schedules
/api/v1/playouts # List playouts
/api/v1/media-items # List media items
/api/v1/search # Search items
/api/v1/ffmpeg/profiles # FFmpeg profiles
/api/v1/settings/ffmpeg # Global FFmpeg settings — workAheadSegmenterLimit,
# initialSegmentCount, hlsSegmenterIdleTimeout
/api/v1/watermarks # Watermarks
/api/channels # List channels
/api/collections # List collections
/api/schedules # List schedules
/api/playouts # List playouts
/api/shows # List shows
/api/movies # List movies
/api/artists # List artists
/api/search # Search items
/api/ffmpeg/profiles # FFmpeg profiles
/api/watermarks # Watermarks
/iptv/channels.m3u # M3U playlist (for Jellyfin)
/iptv/xmltv.xml # XMLTV guide data
```
@@ -177,109 +79,16 @@ docker compose -f $D/compose.yaml up -d --no-deps ersatztv-test
### Mutation Endpoints (POST)
```bash
# Library scan
POST /api/v1/libraries/{id}/scan
POST /api/libraries/{id}/scan
# Scan single show
POST /api/v1/libraries/{id}/scan-show \
POST /api/libraries/{id}/scan-show \
-H "Content-Type: application/json" -d '{"ShowTitle":"Name","DeepScan":false}'
# Reset channel playout (rebuilds schedule)
POST /api/v1/channels/{channelId}/playout/reset
POST /api/channels/{channelNumber}/playout/reset
```
### Scripted Schedule API — `/api/v1/scripted/…`
For **programmatic playout building**: each call mutates one build session, addressed by `buildId`.
Documented by its own OpenAPI spec, **separate from `v1.json`** — which is why
`docs/endpoint-index.md` does not list any of it. It ships as **two** files, both served at
`/openapi/` (measured 2026-08-26 on prod: `scripted-schedule.json`, `scripted-schedule-tagged.json`
and `v1.json` all return 200). They carry the same 28 paths, so either answers "what operations
exist"; they differ only in grouping — the plain file puts everything under one `ScriptedSchedule`
tag, the `-tagged` one splits it into Scripted Content / Control / Metadata / Scheduling. Scalar's
`/docs` page renders the `-tagged` file (`Startup.cs` registers `openapi/scripted-schedule-tagged.json`),
which is why the browsable docs are grouped and a raw fetch of the plain file is not.
The base path is **`/api/v1/scripted/playout/build/{buildId}/`**, and `buildId` is routed as a GUID
(`ScriptedScheduleController.cs`). An older archived copy of this skill gave it as `/api/scripted/…`,
without the `v1`; no such route is registered.
**You cannot tell a wrong base path from a stale `buildId` by probing** — measured on prod
2026-08-26, `GET …/context` with a non-existent build id:
| | `/api/v1/scripted/…` | `/api/scripted/…` (no route) |
|---|---|---|
| no key | 401 | 401 |
| valid key | 404 | 404 |
Unauthenticated everything is 401, because the api-key filter runs before routing. Authenticated, the
correct path 404s too — the build session does not exist — so the 404 that a wrong path earns is
indistinguishable from the one a correct path earns. The bound: this holds **while the build id is
not live**. Against a real, open build session the correct path would answer 200 and the difference
would show — but that is not the situation you are in when you are probing to find out why nothing
works. Confirm the route in `ErsatzTV/Controllers/Api/ScriptedScheduleController.cs`; do not infer it
from a status code.
```
# 28 operations, derived from scripted-schedule.json on 2026-08-26 (ersatztv#755)
POST add_all {content, fillerKind, customTitle, disableWatermarks}
POST add_collection {key, collection, order}
POST add_count {content, count, fillerKind, customTitle, disableWatermarks}
POST add_duration {content, duration, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST add_marathon {key, groupBy, itemOrder, guids, searches, playAllItems, shuffleGroups}
POST add_multi_collection {key, multiCollection, order}
POST add_playlist {key, playlist, playlistGroup}
POST add_search {key, query, order}
POST add_show {key, guids, order}
POST add_smart_collection {key, smartCollection, order}
POST create_playlist {key, items}
POST graphics_off {graphics}
POST graphics_on {graphics, variables}
POST pad_to_next {content, minutes, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pad_until {content, when, tomorrow, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pad_until_exact {content, when, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pre_roll_off (no body)
POST pre_roll_on {playlist}
POST skip_items {content, count}
POST skip_to_item {content, season, episode}
POST start_epg_group {advance, customTitle}
POST stop_epg_group (no body)
POST wait_until {when, tomorrow, rewindOnReset}
POST wait_until_exact {when, rewindOnReset}
POST watermark_off {watermark}
POST watermark_on {watermark}
GET context (no body)
GET peek_next/{content} (no body)
```
Re-derive rather than trusting this table (it is prose and will drift):
```bash
# Absolute path on purpose: this skill is symlinked into ~/server-management and
# ~/media-management, where a repo-relative path would not resolve. ~/ersatztv is the
# shared checkout and can lag origin/main — use the live-instance form below to see
# what is actually deployed.
python3 -c "import json;d=json.load(open('$HOME/ersatztv/ErsatzTV/wwwroot/openapi/scripted-schedule.json'));\
print('\n'.join(f'{m.upper()} {p}' for p,i in d['paths'].items() for m in i if m in('get','post')))"
```
Without a checkout — straight off the running instance (prod; test is port 8410):
```bash
ssh timothy@192.168.1.29 'curl -s http://localhost:8409/openapi/scripted-schedule.json' \
| python3 -c "import json,sys;d=json.load(sys.stdin);\
print('\n'.join(f'{m.upper()} {p}' for p,i in d['paths'].items() for m in i if m in('get','post')))"
```
Field lists above are the request-body property names only; consult the spec for types,
required-ness and defaults. That omission matters for the three on/off pairs: `graphics_on`/
`graphics_off`, `watermark_on`/`watermark_off` and `pre_roll_on`/`pre_roll_off` are **separate
operations, not one toggle**, and the difference is not always visible as differing property names.
`graphics_*` and `pre_roll_*` differ outright. `watermark_on` and `watermark_off` both list
`{watermark}`, but only `on` marks it **required** — `watermark_off` with an **empty** list turns
*every* scripted watermark off (`SchedulingEngine.WatermarkOff`: `watermarks.Count == 0` →
`ClearChannelWatermarkIds()`; `GraphicsOff` is the same shape). Read the schema, not this table,
before sending an `_off`.
## SQLite DB Operations
```bash
@@ -297,56 +106,25 @@ docker start ersatztv
-- List channels
SELECT Id, Number, Name FROM Channel ORDER BY CAST(Number AS INTEGER);
-- List collections with item counts (CollectionItem has no Id column — use rowid)
SELECT c.Id, c.Name, COUNT(ci.rowid) as items
FROM Collection c LEFT JOIN CollectionItem ci ON ci.CollectionId = c.Id GROUP BY c.Id;
-- List collections with item counts
SELECT c.Id, c.Name, COUNT(ci.Id) as items FROM Collection c LEFT JOIN CollectionItem ci ON ci.CollectionId = c.Id GROUP BY c.Id;
-- List schedules
SELECT Id, Name FROM ProgramSchedule;
-- Playout with item count (check if playout is actually built)
SELECT p.Id, c.Number, c.Name, ps.Name as Schedule, p.ScheduleKind, COUNT(pi.Id) as items
FROM Playout p JOIN Channel c ON p.ChannelId = c.Id
LEFT JOIN ProgramSchedule ps ON p.ProgramScheduleId = ps.Id
LEFT JOIN PlayoutItem pi ON pi.PlayoutId = p.Id
GROUP BY p.Id ORDER BY CAST(c.Number AS INTEGER);
-- Playout (channel-schedule links)
SELECT p.Id, c.Number, c.Name, ps.Name as Schedule FROM Playout p JOIN Channel c ON p.ChannelId = c.Id LEFT JOIN ProgramSchedule ps ON p.ProgramScheduleId = ps.Id;
-- Media counts
SELECT 'Shows' as type, COUNT(*) FROM Show UNION ALL SELECT 'Movies', COUNT(*) FROM Movie UNION ALL SELECT 'Episodes', COUNT(*) FROM Episode UNION ALL SELECT 'MusicVideos', COUNT(*) FROM MusicVideo;
-- Collection content (via file paths — Movie table has only Id, metadata is via MediaVersion→MediaFile)
SELECT ci.MediaItemId, mf.Path
FROM CollectionItem ci
JOIN MediaVersion mv ON mv.MovieId = ci.MediaItemId
JOIN MediaFile mf ON mf.MediaVersionId = mv.Id
WHERE ci.CollectionId = <id>
ORDER BY mf.Path;
-- Jellyfin source
SELECT jms.Id, jc.Address, jms.ServerName FROM JellyfinMediaSource jms JOIN JellyfinConnection jc ON jc.JellyfinMediaSourceId = jms.Id;
-- Library sync status
SELECT l.Id, l.Name, l.MediaKind, jl.ShouldSyncItems FROM Library l JOIN JellyfinLibrary jl ON jl.Id = l.Id;
-- Music library folder breakdown
SELECT DISTINCT substr(mf.Path, 1, instr(substr(mf.Path, 13), '/') + 12) as folder, COUNT(*) as items
FROM MediaFile mf WHERE mf.Path LIKE '/data/music/%' GROUP BY folder ORDER BY folder;
```
### Table Schema Notes
**CollectionItem**: Has `CollectionId` + `MediaItemId` columns only (no `Id` column — use `rowid` for counting).
**MediaVersion**: Links to content via `MovieId`, `EpisodeId`, `MusicVideoId` columns (NOT a generic `MediaItemId`). Use `mv.MovieId = ci.MediaItemId` for movie/music video collections.
**Movie / Show / Episode / MusicVideo**: Inheritance from `MediaItem`. These tables have only an `Id` column (PK = MediaItem.Id). Titles and metadata are in separate `*Metadata` tables.
**Artwork**: Channel logos use `ArtworkKind=2` with `ChannelId` set. `Path` column is SHA256 hash (uppercase) of the image file. Files stored at `/config/cache/artwork/logos/{Path[0:2]}/{Path}`.
**ChannelWatermark**: Global watermark config (Id=1, "Channel Bug"). All channels share this via `Channel.WatermarkId=1`. This is the burn-in watermark overlay, NOT the channel logo.
**ProgramScheduleItem subtype tables**: `ProgramScheduleOneItem`, `ProgramScheduleDurationItem`, `ProgramScheduleFloodItem`, `ProgramScheduleMultipleItem`. MUST insert into the matching subtype table (usually `ProgramScheduleOneItem`).
### Channel Setup Workflow (DB)
**Show-specific channel** (single TV show, shuffled):
@@ -358,65 +136,26 @@ VALUES (<id>, 0, 0, '<name>', 1, 0, 1);
INSERT INTO ProgramScheduleItem (Id, CollectionType, FillWithGroupMode, GuideMode, "Index", MarathonGroupBy, MarathonShuffleGroups, MarathonShuffleItems, MediaItemId, PlaybackOrder, ProgramScheduleId)
VALUES (<id>, 1, 0, 0, 0, 0, 0, 0, <show_id>, 3, <schedule_id>);
INSERT INTO ProgramScheduleOneItem (Id) VALUES (<item_id>);
-- 3. Channel (StreamingMode=4 = HLS Segmenter — ETV default; works fine through Dispatcharr. See Gotchas → Streaming mode.)
-- 3. Channel
INSERT INTO Channel (Id, Categories, FFmpegProfileId, FallbackFillerId, "Group", IdleBehavior, IsEnabled, MirrorSourceChannelId, MusicVideoCreditsMode, MusicVideoCreditsTemplate, Name, Number, PlayoutMode, PlayoutOffset, PlayoutSource, PreferredAudioLanguageCode, PreferredAudioTitle, PreferredSubtitleLanguageCode, ShowInEpg, SongVideoMode, SortNumber, StreamSelector, StreamSelectorMode, StreamingMode, SubtitleMode, TranscodeMode, UniqueId, WatermarkId)
VALUES (<id>, '', 1, NULL, '<category>', 0, 1, NULL, 0, NULL, '<name>', '<number>', 0, NULL, 0, NULL, NULL, 'eng', 1, 0, <number>.0, NULL, 0, 4, 2, 0, lower(hex(randomblob(4)))||'-'||lower(hex(randomblob(2)))||'-4'||substr(lower(hex(randomblob(2))),2)||'-'||lower(hex(randomblob(2)))||'-'||lower(hex(randomblob(6))), 1);
-- 4. Playout (ScheduleKind=1 required — 0 is broken)
-- 4. Playout
INSERT INTO Playout (Id, ChannelId, ProgramScheduleId, ScheduleKind, Seed)
VALUES (<id>, <channel_id>, <schedule_id>, 1, abs(random()) % 1000000);
VALUES (<id>, <channel_id>, <schedule_id>, 0, abs(random()) % 1000000);
```
**Collection-based channel** (multiple movies/videos, shuffled):
**Collection-based channel** (multiple shows, shuffled):
```sql
-- 1. Collection + items (MediaItemId = Movie.Id from MediaVersion→MediaFile lookup)
-- 1. Collection + items (MediaItemId = Show.Id)
INSERT INTO Collection (Id, Name, UseCustomPlaybackOrder) VALUES (<id>, '<name>', 0);
INSERT INTO CollectionItem (CollectionId, MediaItemId) VALUES (<coll_id>, <movie_id>);
-- To bulk-add items from a folder:
INSERT INTO CollectionItem (CollectionId, MediaItemId)
SELECT <coll_id>, mv.MovieId FROM MediaFile mf
JOIN MediaVersion mv ON mf.MediaVersionId = mv.Id
WHERE mf.Path LIKE '/data/music/<folder>/%'
AND mv.MovieId NOT IN (SELECT MediaItemId FROM CollectionItem WHERE CollectionId = <coll_id>);
-- 2. Schedule + item (CollectionType=0, PlaybackOrder=3)
INSERT INTO ProgramSchedule (Id, FixedStartTimeBehavior, KeepMultiPartEpisodesTogether, Name, RandomStartPoint, ShuffleScheduleItems, TreatCollectionsAsShows)
VALUES (<id>, 0, 0, '<name>', 1, 1, 0);
INSERT INTO ProgramScheduleItem (Id, CollectionId, CollectionType, FillWithGroupMode, GuideMode, "Index", MarathonGroupBy, MarathonShuffleGroups, MarathonShuffleItems, PlaybackOrder, ProgramScheduleId)
VALUES (<id>, <coll_id>, 0, 0, 0, 0, 0, 0, 0, 3, <schedule_id>);
INSERT INTO ProgramScheduleOneItem (Id) VALUES (<item_id>);
-- 3-4. Channel + Playout same as show-specific (ScheduleKind=1)
INSERT INTO CollectionItem (CollectionId, MediaItemId) VALUES (<coll_id>, <show_id>);
-- 2. Schedule (same as above but CollectionType=0, CollectionId set instead of MediaItemId)
INSERT INTO ProgramScheduleItem (Id, CollectionId, CollectionType, ..., PlaybackOrder, ProgramScheduleId)
VALUES (<id>, <coll_id>, 0, ..., 3, <schedule_id>);
-- 3-4. Channel + Playout same as show-specific
```
After creating: `POST /api/v1/channels/{id}/playout/reset`
### Channel Logo Workflow
Logos are stored as `Artwork` rows (ArtworkKind=2) with images in the cache directory.
```bash
# 1. Create logo PNG (transparent background, white text)
magick -size 512x180 xc:transparent -font "DejaVu-Sans-Bold" -pointsize 48 \
-fill white -stroke black -strokewidth 2 -gravity center \
-annotate +0+0 "CHANNEL NAME" PNG32:/tmp/logo.png
# 2. Calculate SHA256 and place in ErsatzTV cache
HASH=$(sha256sum /tmp/logo.png | cut -d' ' -f1 | tr 'a-f' 'A-F')
LOGO_DIR=~/downloadswarm/ersatztv/cache/artwork/logos
sudo mkdir -p "$LOGO_DIR/${HASH:0:2}"
sudo cp /tmp/logo.png "$LOGO_DIR/${HASH:0:2}/$HASH"
# 3. Insert Artwork row (stop container first for writes)
docker stop ersatztv
sudo sqlite3 ~/downloadswarm/ersatztv/ersatztv.sqlite3 "
INSERT INTO Artwork (ArtworkKind, ChannelId, DateAdded, DateUpdated, Path)
VALUES (2, <channel_db_id>, datetime('now'), datetime('now'), '$HASH');
"
docker start ersatztv
# 4. After ETV restarts, push logos to Jellyfin (see docs/Docker/ErsatzTV.md for fix_logos.py)
```
**Important**: Channel DB Id (from Channel table) is NOT the channel number. E.g., channel #407 might have DB Id 43.
After creating: `POST /api/channels/{number}/playout/reset`
## Volume Mounts (matches Jellyfin)
@@ -431,133 +170,46 @@ docker start ersatztv
## FFmpeg & Hardware
- **QSV encode + VA-API decode on Intel (iHD)** — ErsatzTV runs on **jazz** (i7-10700K, Intel iGPU) since #633. The single `FFmpegProfile` row (`Id = 1`, referenced by all 43 channels) has `HardwareAcceleration = 1` (**Qsv**), `QsvPreferNativeDecoder = 1` (ON), `QsvExtraHardwareFrames = 64`, `VaapiDevice = /dev/dri/renderD128`. Verified live 2026-07-26. The profile is still *named* "1080p VAAPI h264 aac" — cosmetic, ignore the name.
- **The old "do NOT set QSV" rule is RETIRED — #498 fixed the blocker it was based on.** The 2026-07-20 regression was real (QSV's *decoder* is far stricter than VAAPI about malformed NAL units and failed 3 of 6 cold-starts: `Error splitting the input into NAL units`), and the stated cause was that one `HardwareAcceleration` column governed both decode and encode. **#498 added `QsvPreferNativeDecoder` (default ON, Linux-only)**, which splits them exactly like Jellyfin: decode with the tolerant VA-API decoder, encode with QSV. That is what prod runs now. Do not "fix" prod back to `3` (Vaapi) on the strength of the old note.
- **Two QSV traps already paid for, both fixed in code — don't re-derive them:**
- `QsvExtraHardwareFrames` must never be `0`: the software→QSV `hwupload` bridge has no headroom and the transcode writes **zero segments** on any unthrottled read (#523/#529). Code now floors it at 64 (`ffmpeg.qsv-extra-hw-frames-floor`).
- **HDR tonemapping never uses `vpp_qsv=tonemap`** — on this Gen9.5 iGPU that filter is a *silent no-op* (byte-identical output, exit 0, no warning), so it looked like GPU tonemapping while doing nothing. ErsatzTV now tonemaps via VA-API→OpenCL (#505, `ffmpeg.qsv-hdr-tonemap-opencl`). Same trap applies to Jellyfin's `EnableVppTonemapping` on this host — keep it off.
- Fallback if VAAPI also misbehaves (see #631, VAAPI `hwupload -22` on 10-bit): `HardwareAcceleration = 0` (software). jazz has 16 threads at load ~2, so it is affordable and maximally tolerant of imperfect sources.
- QSV (Intel Quick Sync) hardware acceleration
- Resolution: 1920x1080, H264, AAC stereo
- Device: `/dev/dri` passed through (`renderD128`)
- HardwareAccelerationKind: 0=None, 1=Qsv, 2=Nvenc, 3=Vaapi, 4=VideoToolbox, 5=Amf — **jazz uses 1 (Qsv)** with `QsvPreferNativeDecoder` ON (see above)
- jazz's iGPU is shared with Jellyfin only (Frigate stayed on bumblebee); render GID is 992 on both hosts, so `group_add: '992'` carried over unchanged
- Device: `/dev/dri` passed through
- HardwareAccelerationKind: 0=None, 1=Qsv, 2=Nvenc, 3=Vaapi, 4=VideoToolbox, 5=Amf
## Jellyfin Integration
- Secrets: `/config/jellyfin-secrets.json` (`{"Address":"http://jellyfin:8096","ApiKey":"978033be716d46678a5d3c54ae0e0ff9"}`)
- **ErsatzTV** library ids (verified 2026-07-26): Jellyfin source → Movies **10**, TV Shows **11**,
Music Videos **16**; Local source → Standup **14**. These are *ErsatzTV* ids and are **not** the same
as Jellyfin's own library ids — don't reuse one for the other. Re-derive with
`GET /api/v1/media-sources` rather than trusting this list.
- Scan a library with `POST /api/v1/libraries/{id}/scan` (there is no `PUT …/sync`).
- Libraries: Movies(10), TV Shows(11), Music Videos(8), Standup(9)
- `JellyfinLibrary.ShouldSyncItems` must be `1` for scans to work
## Gotchas
### Post-move to jazz (#633)
- **Any rsync from bumblebee's `~/downloadswarm/ersatztv/` re-reverts the QSV setting** — it overwrites `ersatztv.sqlite3`, restoring bumblebee's AMD-era values. Apply config changes **after** the final sync, then re-verify. (Same trap for Jellyfin's `encoding.xml` and `livetv.xml`.)
- **The config dir has root-owned files** (`ersatztv.sqlite3`, `cache/channel-guide/*`), so rsync needs sudo at **both** ends:
```bash
sudo rsync -a --delete -e "ssh -i /home/timothy/.ssh/id_rsa" --rsync-path="sudo rsync" \
timothy@192.168.1.99:/home/timothy/downloadswarm/ersatztv/ /home/timothy/downloadswarm/ersatztv/
```
- **Dispatcharr caches ErsatzTV's XMLTV.** Repointing its DB rows is not enough — it keeps serving a stale EPG full of dead `ersatztv:8409` artwork URLs (breaks Kodi artwork). Force a refresh (EPG source 9):
```bash
ssh timothy@192.168.1.29 'docker exec dispatcharr python manage.py shell -c \
"from apps.epg.tasks import refresh_epg_data; refresh_epg_data(9)"'
```
- **`/api/health` returns 401** (needs an API key). The Telegraf probe has no `response_string_match`, so ErsatzTV reads as **unhealthy in Grafana** — a false alarm, and **pre-existing**, not caused by the move. The container healthcheck uses the unauthenticated internal `/health` and is unaffected.
- **A Komodo deploy alone may not apply bind-mounted config changes** — containers kept serving the pre-checkout inode despite a current `deployed_hash`. `docker restart` explicitly and verify inside the container.
### Common Mistakes (check every time)
- **Playout not building**: Three things must all be correct: (1) `ProgramScheduleOneItem` row exists for the schedule item, (2) `PlaybackOrder=3` (Shuffle), (3) `ScheduleKind=1` on Playout. Missing any one results in 0 playout items — this is the most common issue.
- **Collection queries fail**: `CollectionItem` has no `Id` column — use `rowid` for counting. Content lookup goes through `MediaVersion.MovieId` → `MediaFile.Path` (not a generic MediaItemId join).
- **Channel logos forgotten**: After creating a channel, add an Artwork row (ArtworkKind=2) + logo file, then run `fix_logos.py` to push to Jellyfin. Without this, the channel shows no logo in the EPG.
- **Playout reset required**: After any schedule/collection change, run `POST /api/v1/channels/{id}/playout/reset`. Wait 5-10s for the playout to build before verifying item count.
### Streaming mode + the Dispatcharr reliability fix — #500
Consumers reach ETV **only through Dispatcharr** (`ErsatzTV → Dispatcharr → Jellyfin/Kodi`), which proxies every channel with `ffmpeg -i <etv-url> -c copy -f mpegts`. **Both HLS Segmenter (`StreamingMode=4`) and MPEG-TS (`StreamingMode=1`, `ts-legacy`) work** — Dispatcharr remuxes either to mpegts, and ETV's HLS segments are themselves mpegts with in-band SPS/PPS, so `-c copy` carries codec init either way. We run **42 channels on HLS** (ETV default; ts-legacy showed more visual glitching) + Jungle(407) on TS.
- **What the ~6 s cold-start actually was — ersatztv#350 (fixed 2026-07-20).** `-readrate 1.05` paces input at wall clock so the channel behaves like live TV, and it applies from the **first** read; with 4 s HLS segments a throttled session could not serve the playlist sooner than ~3.8 s. Only `workAheadSegmenterLimit` sessions (prod: **1**, see `/api/v1/settings/ffmpeg`) start unthrottled, so **concurrent tune-ins are the slow ones** — measured 866 ms for the slot winner vs 3845/6357 ms for two simultaneous tunes. Subtitle burn-in, source GOP length and NFS were investigated and **ruled out** (accurate-seek costs 30100 ms). Fixed with `-readrate_initial_burst` (5369 → 648 ms at the ffmpeg level); end-to-end verification tracked in `timothy/ersatztv#519`, so until that lands treat it as expected rather than confirmed. Diagnose with `docker logs ersatztv | grep "HLS cold-start"` — the line splits `setup / startup (prep + ffmpegInit + firstGop) / fill`.
- **The reliability bug was NOT the streaming mode — it was a Dispatcharr teardown race.** Any tune spins up a fresh ETV transcode (historically ~6 s cold-start, same for HLS and TS — see above). With Dispatcharr's default `channel_shutdown_delay=0`, the instant a client's open-timeout drops it the channel tears down, and the retry hits a 503 → ETV cold-starts again → death-spiral (Dispatcharr#503/#851). **Fix lives in Dispatcharr: `channel_shutdown_delay=15`** (see dispatcharr skill → Gotchas). Verified by reverting all channels to HLS while keeping the delay → reliable starts + correct audio sync (2026-06-28).
- **Corrected theory:** the first #500 pass blamed HLS for `Invalid avcC`/codec-init and switched everything to MPEG-TS. **That was wrong** — `-c copy` of mpegts HLS segments carries SPS/PPS fine; the `avcC` log line was transient/info-level and appeared on TS too. The isolation test (HLS + the delay) proved `channel_shutdown_delay` was the actual fix, and we reverted to HLS for better quality.
- Flip a channel's mode live (no restart — ETV reads it per M3U request): `UPDATE Channel SET StreamingMode=4 WHERE …;` then sync Dispatcharr's stored stream URL for that channel (`.m3u8?mode=segmenter` ↔ `.ts?mode=ts-legacy`).
- **Open / in progress:** through Dispatcharr's `-c copy` proxy, HLS showed a one-time skip-back shortly after start (Dispatcharr's `new_client_behind_seconds` repositioning the client behind live — set to 0 to test) and TS showed more glitching. Artifact tuning continues — see the dispatcharr skill and the #500 follow-up.
### Measuring what is actually deployed / what actually happened
- **The api.key file is root-owned, and an unsudo'd read fails SILENTLY.** `cat` returns nothing, the
header goes out empty, and the 401 body parses as a dict — so a naive script reports "0 channels"
rather than an auth error. If a query returns a suspiciously empty result, **check auth before
believing it.** (Cost a wrong reading on 2026-07-21.)
- **A container's OCI labels lie about what is running** — they are inherited from the base image (they
claimed `2026-06-27` on an image built minutes earlier). Tags and `StartedAt` lie too. To prove which
build is live, compare `docker inspect <c> --format '{{.Image}}'` (the manifest digest on jazz) to the
registry's `Docker-Content-Digest` header for that tag — not `.config.digest`. (ersatztv#350)
- **Container log lines carry a LOCAL-time bracket (`[18:48:13 DBG]`) while `docker logs -t` emits
UTC**, so `--since` windows silently mis-slice. For before/after measurements capture by **line
offset** instead (`wc -l` before, `tail -n +N` after).
### DB & Architecture
- DB owned by root — always use `sudo sqlite3`
- **The api.key file is root-owned too, and an unsudo'd read fails SILENTLY.** `cat` returns nothing,
the header goes out empty, and the 401 body parses as a dict — so a naive script reports "0
channels" rather than an auth error. If a query returns a suspiciously empty result, check auth
before believing it. (Cost a wrong reading on 2026-07-21.)
- WAL mode: reads OK while running, stop container for writes
- Full REST CRUD is available under `/api/v1`; prefer it over direct DB writes
- ~~No REST API for channel/collection/schedule CRUD~~ — **false since the fork's `/api/v1`**; use the
API, not DB scripting, wherever an endpoint exists
- **A container's OCI labels lie about what is running** — they are inherited from the linuxserver
base image (they claimed `2026-06-27` on an image built minutes earlier). To prove which build is
live, compare `docker inspect <c> --format '{{.Image}}'` to the registry's `Docker-Content-Digest`
for that tag
- **Container log lines carry a LOCAL-time bracket (`[18:48:13 DBG]`) while `docker logs -t` emits
UTC**, so `--since` windows silently mis-slice. For before/after measurements capture by line
offset instead (`wc -l` before, `tail -n +N` after)
- Secrets file uses PascalCase JSON (`Address`, `ApiKey`)
- Scanner is separate binary (`ErsatzTV.Scanner`) — check with `docker top ersatztv | grep Scanner`
- EF TPT inheritance: `ProgramScheduleItem` has subtype tables (`ProgramScheduleOneItem`, etc.) — inserting into the subtype table is required or EF Core won't recognize the row
- `/health` is the unauthenticated container-health gate; use an authenticated `/api/v1` read to verify the API
### Enums
- PlaybackOrder: 2=Chronological (broken for collections — produces empty playouts), 3=Shuffle, 6=SeasonEpisode — use 3 for reliable results
- CollectionType: 0=Collection, 1=Show (direct show reference via MediaItemId)
- EF TPT inheritance: `ProgramScheduleItem` has subtype tables (`ProgramScheduleOneItem`, etc.) — MUST insert into subtype table
- External URL logos work for M3U but NOT for watermark burn-in (code checks `File.Exists()`)
- `/api/health` predates the Blazor removal; verify the API with an authenticated `/api/v1/channels` instead
- PlaybackOrder enum: 3=Shuffle, 6=SeasonEpisode (use 3 for all channels)
- CollectionType enum: 0=Collection, 1=Show (direct show reference via MediaItemId)
- SubtitleMode: 0=None, 2=Burn-in. Set to 2 with PreferredSubtitleLanguageCode='eng' for non-music channels
- MediaItem.State: 0=Normal, 1=FileNotFound — clean up state=1 items by deleting cascading deps
- ScheduleKind: 0=None (broken — playout never builds), 1=Fixed — use 1
- StreamingMode: 4=HLS Segmenter (`…/channel/N.m3u8?mode=segmenter`) — **ETV default, what we run** (42 channels); 1=MPEG-TS (`…/channel/N.ts?mode=ts-legacy`, Jungle/407 only). Both work through Dispatcharr (it remuxes either to mpegts via `-c copy`). Read live per M3U request → flipping needs **no container restart**. The #500 reliability fix was a Dispatcharr setting (`channel_shutdown_delay`), NOT the mode — see "Streaming mode" gotcha.
### Channel Creation Checklist
1. Collection + CollectionItems (for collection-based) OR MediaItemId (for show-specific)
2. ProgramSchedule (all NOT NULL columns: FixedStartTimeBehavior, KeepMultiPartEpisodesTogether, RandomStartPoint, ShuffleScheduleItems, TreatCollectionsAsShows)
3. ProgramScheduleItem (PlaybackOrder=3) + ProgramScheduleOneItem subtype row
4. Channel (SongVideoMode=0, WatermarkId=1, all required columns)
5. Playout (ScheduleKind=1)
6. Artwork (ArtworkKind=2) + logo file in cache
7. `POST /api/v1/channels/{id}/playout/reset`
8. Run `fix_logos.py` to push logo to Jellyfin
### Logo System
- **External-URL logos now work for the on-screen bug too** — fixed in ersatztv#502 (2026-07-20,
`ffmpeg.external-logo-graphics-engine`). The old claim that they work for M3U but not watermark
burn-in described a `WatermarkSelector` `File.Exists()` gate that is gone; an external logo is
fetched, decode-budget-validated and stored in the image cache at **save** time
(`graphics.channel-logo-caching`), so the render path never fetches over HTTP and a bad URL fails
the save with a 422.
- **M3U/XMLTV absolute URLs are no longer stuck on the request-derived host.** They used to bake in
whatever host fetched the feed (the historical `http://localhost:8409` symptom, Gitea #1/#171),
which Jellyfin can't resolve from inside its container. Set the optional advertised base URL —
`GET`/`PUT /api/v1/settings/iptv` (`iptv.base_url`, ersatztv#340, `iptv.base-url`) — to pin them to
a fixed public origin; unset falls back byte-identical to the old behavior. The base64-upload
workaround in `docs/Docker/ErsatzTV.md` is only needed if that setting is left unset.
- **No usable logo ⇒ no on-screen bug, from every attachment point** (ersatztv#510, 2026-07-26,
`ffmpeg.watermark-resolution-unified`). A `ChannelLogo` watermark resolves through one shared
`WatermarkSelector.ResolveWatermark` whether it came from a playout item, the channel, the global
setting, **or a deco**. A missing cached file, an un-migrated external URL, and a channel with no logo
artwork each render *without* a bug and log a warning. So when debugging "this channel has a watermark
configured but no bug appears", grep the log for `has no logo artwork` / `no longer exists` before
suspecting the ffmpeg pipeline.
- Before #510 the **deco** path alone was unchecked and returned the generated-initials nameplate
(`/iptv/logos/gen`) for a logoless channel — it genuinely rendered. That fallback is now off
everywhere; reviving it via the image cache is ersatztv#652.
- **Not covered:** the song-progress overlay is built as a `WatermarkOptions` directly by the
streaming/troubleshooting handlers, bypassing the resolver, and is still unchecked — ersatztv#653.
- **`/iptv/logos/gen` is unauthenticated**, unlike the rest of `/iptv`: `ConditionalIptvAuthorizeFilter`
is a class-level attribute on `IptvController` only, and that route lives on `ArtworkController`.
Handy for probing, and the reason a container-internal self-fetch of a generated logo succeeds.
- **Seeding a deco watermark for testing is fully API-driven** (no SQLite needed): `POST /api/v1/watermarks`
(needs the full required field set — check `v1.json`), `POST /api/v1/decos/groups`, `POST /api/v1/decos`,
`PUT /api/v1/decos/{id}` (set `watermarkMode` + `watermarkIds`), then `PUT /api/v1/playouts/{id}/deco`.
Use `watermarkMode: "Override"` to make the deco watermark the only one selected. Note branding is
**not** testable through the troubleshooting-playback API (`testing.troubleshoot-path-cannot-test-branding`)
— drive a real channel playout and capture a frame.
- `logo_XX.png` files in the logos root dir are HTML garbage (broken downloads), not actual logos — ignore them
### Other
- Upstream was archived in Feb 2026; `timothy/ersatztv` is the maintained fork and release source
- ProgramSchedule required NOT NULL columns: FixedStartTimeBehavior, KeepMultiPartEpisodesTogether, RandomStartPoint, ShuffleScheduleItems, TreatCollectionsAsShows
- Channel required NOT NULL columns: SongVideoMode (set 0), plus all standard columns (see Channel table schema)
- After schedule changes, rebuild playout: `POST /api/channels/{number}/playout/reset`
- Playout `ScheduleKind` must be `1` (not `0`/None) — `0` causes "Cannot build playout type None" error
- M3U `tvg-logo` URLs hardcode `http://localhost:8409` — Jellyfin can't fetch these from inside its container. Fix by downloading logos from ETV and base64-uploading to Jellyfin (see `docs/Docker/ErsatzTV.md` for script). Tracked in issue #171
- Repo archived Feb 2026, v26.3.0 is final stable version. Maintainer welcomes forks
-1
View File
@@ -1 +0,0 @@
../../../server-management/.claude/skills/jellyfin
+120
View File
@@ -0,0 +1,120 @@
---
name: jellyfin
description: Jellyfin media server management — API for libraries, items, streaming, users. Use when managing media library or checking Jellyfin status.
---
# Jellyfin Management
Container: `jellyfin` | Port: `8096` | IP: `172.16.238.20` (may change on restart)
API Token: `978033be716d46678a5d3c54ae0e0ff9`
Web UI: `https://jellyfin.tblindustries.be` (NO Authelia — native login, password: `coup1802`)
Config: `/home/timothy/downloadswarm/jellyfin/` on jazz
## Access Pattern
```bash
docker exec jellyfin curl -s 'http://localhost:8096/ENDPOINT' \
-H 'X-Emby-Token: 978033be716d46678a5d3c54ae0e0ff9'
```
## Volume Mounts
| Host Path | Container Path | Content |
|-----------|---------------|---------|
| `/mnt/teramind/episodes` | `/data/tvshows` | TV shows |
| `/mnt/episodes` | `/data/episodes` | More episodes |
| `/mnt/media/movies` | `/data/movies` | Movies |
| `/mnt/media/standup` | `/data/standup` | Standup |
| `/mnt/media/music_videos` | `/data/music` | Music videos |
| `/mnt/media/audio/music` | `/data/audio` | Music audio (ro) |
## API Endpoints
### System
```
GET /System/Info # Server info, version
GET /System/Info/Public # Public info (no auth needed)
POST /System/Restart # Restart server
```
### Items (Search & Browse)
```bash
# Search items
GET /Items?includeItemTypes=Movie,Episode,Series&recursive=true&searchTerm=QUERY&fields=Path&limit=20
# Get item details
GET /Items?ids=ITEM_ID&fields=Path,MediaStreams,Overview
# Get all movies
GET /Items?includeItemTypes=Movie&recursive=true&fields=Path&limit=1000
# Get series
GET /Items?includeItemTypes=Series&recursive=true&fields=Path
# Get episodes for a series
GET /Shows/{seriesId}/Episodes?fields=Path,MediaStreams
# Filter by library (parentId)
GET /Items?parentId=LIBRARY_ID&recursive=true&fields=Path
```
### Libraries
```
GET /Library/VirtualFolders # List all libraries
POST /Library/Refresh # Trigger full library scan
POST /Items/{id}/Refresh # Refresh single item metadata
```
### Streaming
```bash
# Test stream URL
GET /Videos/{itemId}/stream?static=true
# Get playback info
GET /Items/{itemId}/PlaybackInfo
```
### Users
```
GET /Users # List users
GET /Users/{userId} # User details
```
## Library IDs
Check with: `curl -s -H "X-Emby-Token: TOKEN" http://localhost:8096/Library/VirtualFolders`
## Live TV
- **ErsatzTV** (channels <1000): M3U `http://ersatztv:8409/iptv/channels.m3u`, XMLTV `http://ersatztv:8409/iptv/xmltv.xml`
- **Dispatcharr** (channels 1000+): IPTV stream manager on port 9191, separate tuner
- Configured in Jellyfin Admin > Live TV
- Guide refresh task ID: `bea9b218c97bbf98c5dc1303bdb9a0ca` — trigger via `POST /ScheduledTasks/Running/{id}`
- **Logo fix after guide refresh**: ErsatzTV logos break (aspect ratio=0) because M3U uses `localhost:8409`. Fix script in `docs/Docker/ErsatzTV.md` downloads from ETV and base64-uploads to `POST /Items/{id}/Images/Primary` (body = base64, Content-Type = image/png)
- **Image upload format**: Jellyfin expects base64-encoded body (NOT raw binary) for `POST /Items/{id}/Images/Primary`
## Gotchas
- **Passwords**: `coup1802` (NOT `ded89Lm4`) — Jellyfin has native auth, no Authelia
- Auth header is `X-Emby-Token` (Jellyfin is an Emby fork)
- **Music videos are typed `MusicVideo`, NOT `Movie`** (corrected 2026-07-21, ersatztv#177). The old
"typed as Movie" note described a deliberate DB reclassification workaround that existed only because
ErsatzTV could not consume `MusicVideo` items — ersatztv#42 shipped that sync, so the workaround's
premise is gone. Verified live: the `Music Videos` library (`/data/music`, collection type
`musicvideos`) holds 1437 items typed `MusicVideo` and **zero** typed `Movie`. Query with
`includeItemTypes=MusicVideo`. (Reclassification to `Movie` may still apply to concert/standup content
in the `movies`/`mixed` libraries — that is a different set; see the server-management jellyfin skill.)
- **`Album` is not an `ItemFields` value.** It is a plain `BaseItemDto` property serialized whenever set,
so it comes back regardless of the `fields=` query param — do NOT add it to `fields` (verified: 111 of
1437 music videos returned `Album` with `fields=Path` alone). Contrast `Genres`/`People`/`Chapters`,
which ARE `ItemFields` and must be requested. Check the enum before extending `fields`.
- **`IndexNumber` is the track number; `ParentIndexNumber` is the disc/season axis.** Frequency misleads
here — on the live music video library `ParentIndexNumber` is populated on 66 items vs 4 for
`IndexNumber`, but where both exist `ParentIndexNumber` is `1` while `IndexNumber` holds the real
ordinal, and where only `ParentIndexNumber` exists it is a collection grouping tracking the album
(`Glastonbury: 2022` -> 230). `AlbumId` is always null on these items.
- Music library at `/data/music` maps to `/mnt/media/music_videos` on host (not actual music)
- Items return 404 on stream if source volume is unmounted
- Jellyfin preserves item IDs across restarts unless files are renamed
- Full library scan can take a long time — prefer targeted `/Items/{id}/Refresh`
- `ffprobe` available in container for checking media streams: `docker exec jellyfin ffprobe -v quiet -print_format json -show_streams FILE`
+1 -1
View File
@@ -3,7 +3,7 @@
"isRoot": true,
"tools": {
"jetbrains.resharper.globaltools": {
"version": "2025.3.5",
"version": "2025.3.4.1",
"commands": [
"jb"
],
-12
View File
@@ -1,12 +0,0 @@
{
"repo": "timothy/ersatztv",
"branch": "main",
"read_on": "2026-08-27",
"source": "GET /repos/timothy/ersatztv/branch_protections -> the rule governing `main` -> status_check_contexts",
"why": "ersatztv#787. The committed mirror of the required status checks on `main`. It exists because the guards that make a required context trustworthy run in `pr-checks.yml::script-tests`, which checks out with persist-credentials:false and holds no Gitea credential, so it cannot ask the server. scripts/tests/test_ci_dropped_step_guard.py DERIVES its marked-job scope from `contexts` rather than repeating it as a literal, and scripts/check-required-contexts.sh compares this list against the live one wherever a credential does exist. Editing `contexts` by hand without re-reading the server is the one move that defeats both. The `repo` field exists because the merge-consent hook fires for whatever owner/repo the merge tool was called with: without it, merging a PR in another repo from an ersatztv session compares that repo's live contexts against THIS repo's mirror and reports a confident, flatly false finding about it.",
"contexts": [
"Build ErsatzTV Image / Build & test (.NET) (pull_request)",
"Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request)",
"review-verdict/h10"
]
}
+11 -80
View File
@@ -4,69 +4,29 @@ name: Build CI Toolchain Image
# pushes it to the Gitea container registry (ersatztv#390). The toolchain jobs in
# docker-build.yml consume it via `container:`, pinned to an immutable :<sha>.
#
# push to MAIN touching docker/ci/** -> :<short-sha> + :latest
# workflow_dispatch on main -> :<short-sha> of main's HEAD + :latest
# workflow_dispatch on a branch -> :<short-sha> of that branch's HEAD ONLY (never :latest)
# schedule (weekly) -> picks up base-image security updates
# push touching docker/ci/** -> :<short-sha> (+ :latest only from main)
# workflow_dispatch -> manual rebuild
# schedule (weekly) -> picks up base-image security updates
#
# Deliberately separate from docker-build.yml: this image changes rarely (a Dockerfile edit or
# the weekly cron), while docker-build.yml runs on every push/PR. Coupling them would rebuild a
# ~2GB toolchain image on every commit.
#
# ROLLOUT NOTE: the jobs pin an immutable :<sha>, never :latest — a broken toolchain image would
# otherwise block every converted job the moment it was pushed. Bumping the toolchain is a deliberate
# two-step, and BOTH steps land in the SAME PR: publish (push the docker/ci commit as branch HEAD,
# dispatch this workflow on that branch), then commit the pin bump in docker-build.yml. Merging first
# is not available: a PR that changes docker/ci/** without moving the pin turns `ci-image-pin` red,
# and the merge-consent hook reads the COMBINED commit status, so it will not auto-grant. That much
# predates ersatztv#744 — what #744 changed is how the publish half is performed.
# See docs/ci-cd.md -> "Publishing from a branch is a dispatch, not a push".
# otherwise block every converted job the moment it was pushed. Bumping the toolchain is therefore
# a deliberate two-step: merge a docker/ci/Dockerfile change (this workflow publishes a new :<sha>),
# then update the pin in docker-build.yml in a follow-up PR whose CI proves the new image works.
# See docs/ci-cd.md -> "CI toolchain image".
#
# Like docker-build.yml: the Gitea registry is HTTP-only, so BuildKit needs the inline
# `http = true` config (it does not inherit the host daemon's insecure-registries setting).
on:
# Publishing from a branch is a DELIBERATE act, not a side effect of pushing (ersatztv#744).
# Gitea resolves a `push` workflow's definition from the pushed branch, so an unfiltered `push`
# trigger ran this file's own YAML — attacker-supplied, unreviewed, with no status check in the
# loop — on a docker-capable runner holding the credential that writes `ersatztv:prod` and the
# `ersatztv-ci:<sha>` five `container:` jobs execute.
#
# BE PRECISE ABOUT WHAT THIS BUYS, because the mechanism cuts both ways: the filter below is read
# from the pushed ref like everything else in this file, so a branch that DELETES it re-enables
# the route. What closes is the DRIVE-BY case — an ordinary push of a legitimate `docker/ci`
# change publishing an image nobody asked for, with no deliberate act anywhere. This is NOT a
# boundary against a malicious or compromised writer and must not be cited as one. That class was
# probed and ACCEPTED in ersatztv#853 (`ci.workflow-dispatch-ref-unrestricted`): Gitea 1.27.1 cannot
# restrict `workflow_dispatch` by ref, and restricting it would close nothing anyway:
# docker-build.yml's head-resolved `pull_request:` runs attacker-authored YAML, which reaches every
# secret in the store — so it covers renovate.yml's RENOVATE_TOKEN too, without dispatching
# renovate.yml at all. Only the DISPATCH third is settled; the `v*` tag push and the PR route
# itself remain open in ersatztv#885. `workflow_dispatch` is loaded from the ref it is dispatched
# on, exactly as the `branches:` filter below is loaded from the pushed ref, and is the deliberate
# publish path (docs/ci-cd.md -> "CI toolchain image").
#
# A `v*` tag push does not match this trigger either: there is no `tags:` key, and a `branches:`
# filter is compared against a branch ref. The exact matcher semantics are not probed here; the
# observable claim is the one that matters — a release cut no longer republishes the toolchain
# image as a side effect.
#
# `.gitea/workflows/ci-image.yml` is NOT in `paths:`, and it left `ci-image-pin`'s `expected` in
# the same change. That pairing is a DECIDED TRADEOFF, not a necessity: keeping it works, because
# the dispatch above can publish the ci-image.yml commit itself and the pin then matches. The
# price is what decided it — that route charges a full ~2GB publish plus a five-pin bump for
# EVERY edit to this file, comments included, and a rebase charges it again. The cost of the side
# taken is stated here and in ci-cd.md: a change to HOW the image is built that lives only in
# this file no longer republishes on its own, so pair it with a `docker/ci/**` edit.
#
# `paths:` here and `ci-image-pin`'s `expected` pathspec in pr-checks.yml MUST name the same
# sources, and nothing mechanically enforces that since the shared self-reference went —
# ersatztv#855.
workflow_dispatch:
push:
branches: [main]
paths:
- 'docker/ci/**'
- '.gitea/workflows/ci-image.yml'
schedule:
# Mondays 05:00 UTC. Gitea registers `schedule` only from the default branch (main).
#
@@ -90,21 +50,6 @@ env:
REGISTRY: 192.168.1.95:3000
CI_IMAGE: 192.168.1.95:3000/timothy/ersatztv-ci
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# This workflow's registry pushes authenticate with the scoped REGISTRY_* PAT
# (`ci.actions-credential-scoping`), so the injected GITEA_TOKEN serves only its single
# `actions/checkout`. This file was the one workflow #748 could not originally reach: editing it
# re-pointed `ci-image-pin`'s `expected` at the editing commit and reddened a BLOCKING job, and its
# own `paths:` made the edit publish an image. ersatztv#744 took this path out of both
# (`ci.toolchain-image-publish-is-a-dispatch`), so the exemption that briefly existed here is DELETED
# rather than documented — which is what ersatztv#835 asked for.
permissions:
code: read
jobs:
build:
name: Build & push CI image
@@ -113,23 +58,13 @@ jobs:
# toolchain image — the heaviest thing that ran in that lane. `small` is now
# git-only and capped at 1g per job, which would OOM this build.
#
# Rare trigger (main pushes touching docker/ci, a weekly cron, and the occasional
# branch dispatch), so it costs the ubuntu-latest lane almost nothing, and
# ci-runner (.127) runs no prod workload.
# Rare trigger (pushes touching docker/ci + a weekly cron), so it costs the
# ubuntu-latest lane almost nothing, and ci-runner (.127) runs no prod workload.
runs-on: ubuntu-latest
env:
CI_JOB_ROLE: none
steps:
- name: Checkout
uses: actions/checkout@v4
with:
# ersatztv#746's convention, applied here once #744 removed the reason it was skipped:
# without it the action leaves a write-capable Authorization header in .git/config for
# every later step. Nothing here pushes with git — the only git call is the
# `rev-parse --short HEAD` below — and the repo is public, so the clone needs no
# credential of its own. Guarded for every workflow by
# scripts/tests/test_workflow_persist_credentials.py (ersatztv#835).
persist-credentials: false
# only docker/ci/Dockerfile is needed; no git describe/log here
fetch-depth: 1
@@ -141,11 +76,7 @@ jobs:
# Always publish the immutable :<sha> — that is what docker-build.yml pins.
TAGS=("${CI_IMAGE}:${SHORT}")
# :latest is a convenience/floating pointer for humans and the weekly rebuild; jobs must
# never consume it. Only main may move it — and since #744 the `push` trigger is
# main-only, so on that path the branch check is satisfied by construction. It is now the
# SOLE protection on the one event that never exercised it before: a `workflow_dispatch`
# selects any ref, and the branch-side publish path documented in ci-cd.md runs exactly
# that. Do not simplify this away on the reasoning that the trigger is already main-only.
# never consume it. Only main may move it.
if [ "${GITHUB_REF}" = "refs/heads/main" ]; then
TAGS+=("${CI_IMAGE}:latest")
fi
-14
View File
@@ -33,27 +33,13 @@ env:
DOTNET_CLI_USE_MSBUILD_SERVER: "0"
MSBUILDDISABLENODEREUSE: "1"
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Holds no registry credential and reads nothing from the Gitea API; the injected GITEA_TOKEN serves
# only its one `actions/checkout`.
permissions:
code: read
jobs:
scan:
name: NuGet vulnerable packages
runs-on: ubuntu-latest
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Setup .NET
uses: actions/setup-dotnet@v4
+29 -396
View File
@@ -38,22 +38,9 @@ name: Build ErsatzTV Image
# report `success` in seconds — the two REQUIRED contexts (`Build & test (.NET)`, `EF migration
# integrity (SQLite + MySql)`) must keep reporting or a docs-only PR could never merge. We do NOT
# `if:`-skip a required job: on Gitea 1.25.4 a skipped job reports commit-status state `skipped`
# (verified, throwaway PR #418; re-confirmed on 1.27.1, 2026-08-28, ersatztv#747 — `Build & push
# image (amd64)` is `if:`-skipped on every PR and reported `skipped` on the two heads sampled,
# PRs #829 and #828) and we don't rely on how branch protection treats a skipped
# (verified, throwaway PR #418) and we don't rely on how branch protection treats a skipped
# REQUIRED context. See docs/ci-cd.md -> "Docs-only skip".
#
# RELEASE-PATH DELIMITER GATE (ersatztv#767): the `scan` job runs the PyYAML-based delimiter-ban
# test and is a `needs:` of `build`, so a `${{` opener in a banned job's `run:` body means `build`
# never runs. It is deliberately NOT gated by either skip below: the gate's coverage must not depend
# on a detector the gate is not allowed to trust, and it is cheap enough that gating it buys nothing.
# (Do NOT justify that with "the docs-only path still builds an image" — it does not. `Build and
# push` carries the docs_only gate too; a tag build is unaffected only because the script forces
# docs_only=false there.) Note it installs from PyPI (setup-python + pip), putting a NEW network
# dependency between a `v*` tag and its image. Not the only one on this path — `test` runs
# `dotnet restore` and `npm ci` behind actions/cache, and a cache miss reaches nuget.org/npm — but
# newly added here. Fail-closed and loud, and still a real availability dependency.
#
# ALREADY-VALIDATED SKIP (ersatztv#420): a second, sibling gate in `test`, `migrations` and
# `functional-e2e` only (NOT `build`). On a push-to-main merge commit, `id: revalidate` runs
# `scripts/ci-detect-already-validated.sh`, which emits `skip=true` only when the merged tree is
@@ -115,51 +102,7 @@ env:
DOTNET_CLI_USE_MSBUILD_SERVER: "0" # no persistent MSBuild server process
MSBUILDDISABLENODEREUSE: "1" # MSBuild worker nodes exit with the build instead of lingering
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Every credentialed thing this file does uses the scoped REGISTRY_* PAT, never the injected token:
# its registry pushes, its five `container:` image pulls, its three commit-status GET steps
# (`ETV_STATUS_AUTH` in jobs `test`, `migrations` and `functional-e2e`, each a read-only GET via
# scripts/ci-detect-already-validated.sh) and its registry tag READ (`ETV_REGISTRY_AUTH` in job
# `toolchain-preflight`, via scripts/ci-toolchain-image-resolves.sh). The injected token therefore
# serves only its eight `actions/checkout` steps. Note this file needs no `packages:` unit for that
# same reason: the `container:` blocks carry explicit `credentials:`.
# (Sites above are named by JOB, not by line number: this file is ~1150 lines, so any edit above a
# citation silently invalidates it — which is how the first version of this comment went stale two
# lines after it was written.)
permissions:
code: read
jobs:
# Answers "is the toolchain image still there?" in ONE place, so a deleted pin does not read as
# five broken jobs and a broken diff (ersatztv#772). Deliberately container-free and deliberately
# NOT a `needs:` of the jobs it diagnoses — see scripts/ci-toolchain-image-resolves.sh for both
# decisions and for the cleanup-rule root cause it cannot fix from this repo.
toolchain-preflight:
name: CI toolchain image resolves
runs-on: small
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Resolve the pinned toolchain tag in the registry
env:
ETV_REGISTRY_AUTH: ${{ secrets.REGISTRY_USER }}:${{ secrets.REGISTRY_PASSWORD }}
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark resolve
scripts/ci-toolchain-image-resolves.sh
- name: Assert every expected step executed (ersatztv#756)
run: >-
scripts/ci-step-ran.sh assert
--always resolve
test:
name: Build & test (.NET)
runs-on: ubuntu-latest
@@ -168,14 +111,10 @@ jobs:
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# git history/tags are needed by the `build` job's `git describe` (ersatztv#190) and,
# here, by the #420 revalidate step's `HEAD^2` tree comparison on a main merge commit.
fetch-depth: 2
@@ -183,23 +122,14 @@ jobs:
# ersatztv#416: is this a docs-only change? If so, every heavy step below is skipped and this
# REQUIRED job reports success in seconds. It still RUNS (never `if:`-skipped) so the required
# context keeps reporting — see the workflow header and docs/ci-cd.md -> "Docs-only skip".
# EVERY consequential `run:` step in this job marks itself as its FIRST act (ersatztv#756),
# and the trailing `Assert every expected step executed` guard fails the job when one is
# missing. This is a REQUIRED context on `main`, and a step the runner drops takes the job
# GREEN having done no work — see scripts/ci-step-ran.sh for why that is fail-OPEN here while
# the same drop in review-verdict.yml is fail-CLOSED.
- name: Detect docs-only changes
id: detect
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark detect
scripts/ci-detect-docs-only.sh
run: scripts/ci-detect-docs-only.sh
- name: Detect already-validated tree (#420)
id: revalidate
env:
ETV_STATUS_AUTH: ${{ secrets.REGISTRY_USER }}:${{ secrets.REGISTRY_PASSWORD }}
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark revalidate
scripts/ci-detect-already-validated.sh
run: scripts/ci-detect-already-validated.sh
- name: Cache NuGet packages
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
@@ -211,9 +141,7 @@ jobs:
- name: Restore
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark restore
dotnet restore
run: dotnet restore
# Replaces setup-node's built-in `cache: npm`. The toolchain image supplies node/npm, but
# the SPA's package downloads are project deps, so they stay cached per lockfile.
@@ -228,50 +156,36 @@ jobs:
- name: Install SPA dependencies
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark npm-ci
npm ci
run: npm ci
- name: Check generated SPA API client
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark check-api
npm run check:api
run: npm run check:api
- name: Lint SPA
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark lint
npm run lint
run: npm run lint
- name: Typecheck SPA
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark typecheck
npm run typecheck
run: npm run typecheck
- name: Test SPA
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test
npm test -- --run
run: npm test -- --run
- name: Build SPA
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-build
npm run build
run: npm run build
- name: Strip Scanner project ref (matches Docker build)
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark strip-scanner
sed -i '/Scanner/d' ErsatzTV/ErsatzTV.csproj
run: sed -i '/Scanner/d' ErsatzTV/ErsatzTV.csproj
# Start the true peak-anon sampler just before the memory-heavy dotnet Build/Test/Coverage so
# its high-water mark spans them (SPA build/test above are comparatively light). Paired with the
@@ -285,16 +199,13 @@ jobs:
- name: Build
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark build
dotnet build --configuration Release --no-restore
run: dotnet build --configuration Release --no-restore
- name: Test
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark dotnet-test
dotnet test --configuration Release --no-build --blame-hang-timeout "2m" --verbosity normal \
--collect:"XPlat Code Coverage" --settings coverlet.runsettings --results-directory ./coverage
run: >-
dotnet test --configuration Release --no-build --blame-hang-timeout "2m" --verbosity normal
--collect:"XPlat Code Coverage" --settings coverlet.runsettings --results-directory ./coverage
# Coverage reporting (ersatztv#15 scope item 4): coverlet.collector emits a Cobertura report
# per test project (via --collect above); ReportGenerator merges them into a human-readable
@@ -347,43 +258,6 @@ jobs:
continue-on-error: true
run: scripts/ci-peak-anon.sh report
# THE DROPPED-STEP GUARD (ersatztv#756). Every `run:` step above records that it began; this
# asserts the whole expected SET was recorded. A step the runner declines to interpolate is
# DROPPED and still concludes `success` (ersatztv#751), so without this a REQUIRED context
# reports green having done no work — fail-OPEN, and strictly worse than the fail-CLOSED
# version of the same bug that #751 fixed in review-verdict.yml.
#
# NO `if:` HERE, WHICH IS A DELIBERATE DEPARTURE FROM THE #751 GUARD and the one decision in
# this block that is easy to "fix" wrongly. That guard uses `if: always()` because its job has
# exactly one real step, so there is no ordinary red for it to talk over. Here there are
# twelve, and a genuine failure in an early one (a lint error, a failing test) SKIPS every
# later step — an `always()` guard would then announce "these steps never executed: typecheck
# web-test build dotnet-test" on top of every normal red build. That is not a dropped step, it
# is the runner doing what it is told, and a guard that cries wolf on every red build is a
# guard that gets deleted.
#
# The default `if:` is `success()`, which is exactly the condition wanted, and the invariant it
# rests on is worth stating because it is what makes the omission safe rather than lucky: this
# step is skipped ONLY when an earlier step failed, and an earlier step failing already fails
# the job. So `guard skipped => job red`, and the only path to a green job runs the guard. A
# dropped step is invisible precisely because it concludes `success`, which keeps the job green
# and therefore reaches here.
#
# ITS OWN BODY CANNOT BE DROPPED BY THE MECHANISM IT GUARDS AGAINST: it is a single command
# with no expression delimiter anywhere in the scalar, so the runner has nothing to rewrite.
# The two gate values come in through `env:`, which is interpolated PER VALUE — a bad payload
# there cannot take the body with it (`ci.workflow-run-body-no-expressions`), and both paths
# are held to naming a real context by
# test_every_workflow_expression_names_a_REAL_context_or_function.
- name: Assert every expected step executed (ersatztv#756)
env:
ETV_DOCS_ONLY: ${{ steps.detect.outputs.docs_only }}
ETV_REVALIDATE_SKIP: ${{ steps.revalidate.outputs.skip }}
run: >-
scripts/ci-step-ran.sh assert
--always detect revalidate
--gated restore npm-ci check-api lint typecheck web-test web-build strip-scanner build dotnet-test
migrations:
name: EF migration integrity (SQLite + MySql)
runs-on: ubuntu-latest
@@ -444,35 +318,24 @@ jobs:
--health-interval=5s
--health-timeout=5s
--health-retries=30
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# was the default fetch-depth: 1 (ersatztv#190); bumped to 2 so the #420 revalidate
# step's `HEAD^2` tree comparison can resolve on a main merge commit.
fetch-depth: 2
# ersatztv#416: docs-only? Skip the build + migration replay; the job still reports success in
# seconds. REQUIRED context, so it always RUNS (never `if:`-skipped). See the workflow header.
# Same per-step marker contract as the `test` job above (ersatztv#756) — this is the other
# REQUIRED context, so a dropped migration-replay step would report EF integrity green having
# replayed nothing.
- name: Detect docs-only changes
id: detect
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark detect
scripts/ci-detect-docs-only.sh
run: scripts/ci-detect-docs-only.sh
- name: Detect already-validated tree (#420)
id: revalidate
env:
ETV_STATUS_AUTH: ${{ secrets.REGISTRY_USER }}:${{ secrets.REGISTRY_PASSWORD }}
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark revalidate
scripts/ci-detect-already-validated.sh
run: scripts/ci-detect-already-validated.sh
- name: Cache NuGet packages
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
@@ -484,15 +347,11 @@ jobs:
- name: Restore
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark restore
dotnet restore
run: dotnet restore
- name: Build
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark build
dotnet build --configuration Release --no-restore
run: dotnet build --configuration Release --no-restore
# dotnet-ef is baked into the CI toolchain image (docker/ci/Dockerfile) and already on PATH
# — no per-run `dotnet tool install`. Bump its version there (ersatztv#390).
@@ -502,7 +361,6 @@ jobs:
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: |
set -euo pipefail
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark sqlite
echo "::group::SQLite model drift (has-pending-model-changes)"
dotnet ef migrations has-pending-model-changes --no-build --configuration Release \
--context TvContext --startup-project ErsatzTV --project ErsatzTV.Infrastructure.Sqlite -- --provider Sqlite
@@ -526,7 +384,6 @@ jobs:
MySql__ConnectionString: "Server=mysql;Port=3306;Database=ersatztv_migrations;Uid=root;Pwd=ersatztv;DefaultCommandTimeout=300;"
run: |
set -euo pipefail
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark mysql
echo "::group::MySql model drift (has-pending-model-changes)"
dotnet ef migrations has-pending-model-changes --no-build --configuration Release \
--context TvContext --startup-project ErsatzTV --project ErsatzTV.Infrastructure.MySql -- --provider MySql
@@ -551,52 +408,6 @@ jobs:
done
echo "::endgroup::"
# NOTE (ersatztv#491 -> #627): running the LibraryFolder dedupe fixture against the live `mysql`
# service was implemented here and then REMOVED. The coverage gap it closes is real — the two
# checks above only ever apply migrations to a fresh EMPTY database, so they execute no rows of any
# data-migration logic, and two MySql-only collation defects escaped exactly this gate. But the
# fixture proved non-deterministic in CI across three attempts (stale pooled session after a drop,
# then lost isolation from a shared database name, then a connect-before-create), and an
# intermittently-red gate is worse than none: it trains everyone to re-run instead of read, which is
# how the original defects escaped. The fixture itself is retained and is opt-in via
# ETV_TEST_MYSQL_CONNECTION (skipped, visibly, without it). Re-arming it here is tracked by #627.
# THE DROPPED-STEP GUARD (ersatztv#756). Every `run:` step above records that it began; this
# asserts the whole expected SET was recorded. A step the runner declines to interpolate is
# DROPPED and still concludes `success` (ersatztv#751), so without this a REQUIRED context
# reports green having done no work — fail-OPEN, and strictly worse than the fail-CLOSED
# version of the same bug that #751 fixed in review-verdict.yml.
#
# NO `if:` HERE, WHICH IS A DELIBERATE DEPARTURE FROM THE #751 GUARD and the one decision in
# this block that is easy to "fix" wrongly. That guard uses `if: always()` because its job has
# exactly one real step, so there is no ordinary red for it to talk over. Here a genuine
# failure in an early step (a failing `dotnet build`, a MySql replay error) SKIPS every later
# step — an `always()` guard would then announce "these steps never executed: sqlite mysql" on
# top of every normal red build. That is not a dropped step, it is the runner doing what it is
# told, and a guard that cries wolf on every red build is a guard that gets deleted.
#
# The default `if:` is `success()`, which is exactly the condition wanted, and the invariant it
# rests on is worth stating because it is what makes the omission safe rather than lucky: this
# step is skipped ONLY when an earlier step failed, and an earlier step failing already fails
# the job. So `guard skipped => job red`, and the only path to a green job runs the guard. A
# dropped step is invisible precisely because it concludes `success`, which keeps the job green
# and therefore reaches here.
#
# ITS OWN BODY CANNOT BE DROPPED BY THE MECHANISM IT GUARDS AGAINST: it is a single command
# with no expression delimiter anywhere in the scalar, so the runner has nothing to rewrite.
# The two gate values come in through `env:`, which is interpolated PER VALUE — a bad payload
# there cannot take the body with it (`ci.workflow-run-body-no-expressions`), and both paths
# are held to naming a real context by
# test_every_workflow_expression_names_a_REAL_context_or_function.
- name: Assert every expected step executed (ersatztv#756)
env:
ETV_DOCS_ONLY: ${{ steps.detect.outputs.docs_only }}
ETV_REVALIDATE_SKIP: ${{ steps.revalidate.outputs.skip }}
run: >-
scripts/ci-step-ran.sh assert
--always detect revalidate
--gated restore build sqlite mysql
functional-e2e:
name: Functional E2E (curl + UI contracts)
runs-on: ubuntu-latest
@@ -614,14 +425,10 @@ jobs:
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# bumped from 1 (ersatztv#190 default) so the #420 revalidate step's `HEAD^2` tree
# comparison can resolve on a main merge commit.
fetch-depth: 2
@@ -712,111 +519,6 @@ jobs:
# server. Its exit status is Playwright's.
scripts/e2e-ui.sh
# THE DELIMITER BAN, RE-CHECKED ON THE RELEASE PATH ITSELF (ersatztv#767).
#
# The ban that keeps `build`'s `Smoke + IPTV E2E` from being silently dropped was enforced only by
# `test_the_delimiter_banned_jobs_have_NO_expression_delimiter_in_any_run_body` in the
# `script-tests` job of pr-checks.yml — `on: pull_request`, and NOT a required context. So the ban
# was REVIEW-TIME only: nothing re-checked it on a `v*` tag push, which is precisely when the
# candidate image is published and `DeployStack jazz-media` promotes it.
#
# WHY A JOB AND NOT A STEP INSIDE `build`. A step cannot protect the thing it shares a job with:
# `build` is what publishes, so a guard step there fails OPEN if the runner drops it, and "my body
# has no opener so I cannot be dropped" is circular when the only thing enforcing that property is
# the same PR-only test being backstopped. As a `needs:` of `build`, a red here means `build` never
# runs at all — the image is not built, let alone pushed. Fail-closed by dependency, not by
# assertion.
#
# WHY IT RUNS THE REAL PYTEST rather than a bespoke scanner. The first cut of #767 hand-parsed the
# workflow YAML in stdlib Python, to avoid provisioning PyYAML on `build`'s bare runner. Two
# independent reviews found ~10 false NEGATIVES in that parser within one round (flow mappings
# `{run: …}`, a quoted `"run":` key, aliases, multiline quoted scalars) — i.e. it was strictly
# WEAKER than the check it was meant to backstop, in the one direction that matters for a security
# gate. Running the existing PyYAML-based test needs no second implementation of "what is a `run:`
# body" and therefore has no drift surface. `small` is git-only, so Python is provisioned here the
# same way `script-tests` does it.
#
# This job's OWN steps carry #756 markers and a trailing assert, so a drop inside THIS job is
# caught too. That terminates the regress at the same axiom the sibling guards already rest on —
# to fail open you must now drop the pytest step AND the assert step, rather than either one.
#
# THIS PUTS A `small`-LANE JOB BACK ON THE TAG PATH, which ersatztv#535 deliberately moved away
# from — say so rather than letting it look accidental. #535 split the git-only gates into
# pr-checks.yml because on the v26.12.0 tag they wedged in act's setup phase, were killed, and
# reported `failure` with no logs. The blast radius here is WORSE than it was then: as a `needs:`
# of `build`, that flake would not merely redden a status, it would skip the build and produce no
# release image at all.
#
# It is acceptable now for a stated reason rather than an assumed one, and the evidence is weaker
# than it first looks — so read the limits. Per `ci.small-lane-git-only`, the lane's per-job cap was
# forced to 10g by its two HEAVIEST members (this file's `build` AND ci-image.yml's toolchain
# buildx), not by `build` alone, and that cap is what pinned the lane to one slot on a 25 GiB host;
# both were moved off in server-management#639, after which the lane is git-only and runs wide and
# tiny. What has NOT been demonstrated is this lane on a TAG PUSH: `script-tests` runs there happily
# but lives in pr-checks.yml (`on: pull_request`), so it has never exercised the condition #535
# measured, and #767's own runs (1928/1929) were `workflow_dispatch` on a scratch branch. The
# lane-width argument is what carries this, not a like-for-like observation. If the wedging returns,
# move this job to `ubuntu-latest` rather than weakening the `needs:` edge — a slower gate is fine,
# an optional one is not.
scan:
name: Delimiter ban (release path)
runs-on: small
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install test dependencies
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark deps
python3 -m pip install --disable-pip-version-check --quiet pytest pyyaml
# The ban test plus the structural tests that hold this job's own shape. NOT the whole
# scripts/tests suite: that is `script-tests`'s job, it needs jq/git preflights, and an
# unrelated pytest regression must not be able to block a release.
- name: Run the delimiter-ban tests
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark ban
PYTHONPATH=. python3 -m pytest scripts/tests/test_ci_dropped_step_guard.py scripts/tests/test_ci_release_path_scan_job.py -q
# THE POSITIVE CONTROL, and it is deliberately NOT a test (ersatztv#767). The step above proves
# the ban HOLDS; it cannot prove the ban would NOTICE. Review disarmed the entire gate with one
# repo-root `pytest.ini` (`addopts = -k "not delimiter_banned"`) or `conftest.py`
# (`pytest_collection_modifyitems`), which deselects the ban test and every test guarding it,
# leaving all jobs green with a delimiter sitting in `Smoke`. Nothing inside pytest can be
# trusted to catch that, because pytest's own configuration outranks it.
#
# So this poisons the checked-out workflow, re-runs the SAME command, and fails the job if it
# PASSES. It runs in the real checkout — an isolated copy does not inherit the repo-root config
# a disarm would live in, which made the first version of this script report healthy while the
# job's real invocation was deselected. The workflow file is restored by an EXIT trap.
- name: Prove the ban would DETECT a delimiter (ersatztv#767)
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark selfcheck
scripts/ci-prove-ban-detects.sh
# No `if:` — see the sibling guards in `test`/`migrations` for why the default `success()` is
# the wanted condition. Both keys are `--always`: every step in this job is unconditional.
#
# THE MARKER-PATH RATIONALE DOES NOT TRANSFER HERE, and assuming it did would be the mistake
# `ci.required-job-step-execution-markers` itself warns about. That record says the run-id and
# attempt keying is "defence in depth" because "these jobs get a fresh container, which is the
# primary protection". This job has NO `container:` — it is on `small`, where RUNNER_TEMP is
# the shared host /tmp. So here the keying is the ONLY protection, and the residual is a
# single-job re-run that does not increment GITHUB_RUN_ATTEMPT: it would find the previous
# attempt's marker file and the assert would pass even had the pytest step been dropped.
# Identity was read off a real run rather than assumed — run 1929 printed
# `Marker identity: job=scan run=1929 attempt=1 (from the runner)`, so all three variables are
# populated on this lane.
- name: Assert every expected step executed (ersatztv#756)
run: >-
scripts/ci-step-ran.sh assert
--always deps ban selfcheck
build:
name: Build & push image (amd64)
# Moved back off `small` (server-management#639). This is the one HEAVY job that
@@ -828,24 +530,17 @@ jobs:
#
# The `ubuntu-latest` queueing that sent it to `small` in the first place
# (server-management#574: a PR-run skip stuck 31 min behind long builds) does not
# come back, because `needs: [test, migrations, scan]` means this job cannot be
# dispatched until those three have already finished — by which point the lane it
# come back, because `needs: [test, migrations]` means this job cannot be
# dispatched until those two have already finished — by which point the lane it
# was queueing behind has drained. Real builds (main/tags) get the full
# ubuntu-latest allotment: 4 CPUs / 10g on ci-runner (.127).
runs-on: ubuntu-latest
# `scan` (ersatztv#767) re-checks the delimiter ban on the release path. As a `needs:` its red
# SKIPS this job outright, so a delimiter in `Smoke + IPTV E2E` can no longer reach the point
# where an image is published and never booted.
needs: [test, migrations, scan]
needs: [test, migrations]
if: github.event_name != 'pull_request'
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: none
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
# ersatztv#416: a docs-only push to main has nothing to rebuild (docs are not in the image),
@@ -865,27 +560,7 @@ jobs:
INFO_VERSION="${VERSION}"
TAGS=("${IMAGE}:prod" "${IMAGE}:${VERSION}" "${IMAGE}:${SHORT}")
else
# `git describe` MUST resolve here, and a failure is fatal rather than defaulted
# (ersatztv#836). This job checks out `fetch-depth: 0`, so the tags are present; the
# only thing that ever stopped `describe` from seeing them was the detector step above
# grafting this complete clone shallow. The old `|| echo v0.0.0` was a fallback that
# cannot fail, so from 2026-07-17 (when #416 introduced the depth) until #836 every
# `:latest` image was published carrying
# `InformationalVersion 0.0.0-<sha>` and nothing anywhere went red — the defect was
# found by reading the string out of a running container, which is not a detector.
# Failing the job instead means no `:latest` is published at all: visible, recoverable,
# and never a mislabelled image promoted downstream. The tag path above never calls
# `describe`, so a release cut is unaffected by this.
# stderr is discarded on the CAPTURE and re-run for the diagnostic, rather than folded
# in with `2>&1`: a git warning on the SUCCESS path would otherwise land inside DESC and
# become part of the version string — the same shape of silent corruption this whole
# step is being hardened against.
if ! DESC=$(git describe --tags --abbrev=0 2>/dev/null); then
echo "is-shallow-repository=$(git rev-parse --is-shallow-repository)"
git describe --tags --abbrev=0 || true
echo "::error::git describe --tags --abbrev=0 failed, so this image would ship InformationalVersion 0.0.0-${SHORT} instead of a real version (ersatztv#836). The usual cause is a --depth fetch grafting this complete clone shallow; the two lines above say which."
exit 1
fi
DESC=$(git describe --tags --abbrev=0 2>/dev/null || echo v0.0.0)
INFO_VERSION="${DESC#v}-${SHORT}"
TAGS=("${IMAGE}:latest" "${IMAGE}:${SHORT}")
fi
@@ -931,33 +606,11 @@ jobs:
cache-from: type=registry,ref=192.168.1.95:3000/timothy/ersatztv:buildcache
cache-to: type=registry,ref=192.168.1.95:3000/timothy/ersatztv:buildcache,mode=max,ignore-error=true
# THE TWO VALUES COME IN THROUGH `env:`, NOT INLINE (ersatztv#756). This step runs AFTER
# `Build and push`, so on a `v*` tag the image is already in the registry as the release
# candidate — and it is this smoke run that decides whether the candidate was ever booted at
# all. A stray expression delimiter anywhere in this body (a comment is not inert — #751) would
# DROP the step and conclude the job `success`: a candidate published, never smoke-tested, and
# `DeployStack jazz-media` promotes exactly that image. `env:` is interpolated PER VALUE, so a
# bad payload there fails that value instead of taking the whole body with it, and with the
# body delimiter-free the class is unreachable here — held by
# test_the_delimiter_banned_jobs_have_NO_expression_delimiter_in_any_run_body.
#
# The ban IS re-checked on the release path now (ersatztv#767): the `scan` job above runs the
# PyYAML-based ban test and is a `needs:` of this job, so a delimiter here means `build` never
# runs and no image is published. Do not re-add the note that once stood here saying the ban is
# "review-time only, tracked as #767" — that was true before the `scan` job existed.
#
# This step still carries no per-step markers, and that is a genuine (smaller) residual rather
# than a dismissal: markers would additionally catch a drop caused by something OTHER than a
# delimiter. Adding them needs a bucket modelling this step's publish-ref `if:`, which the
# guard's always/gated buckets do not express. The delimiter class itself is covered.
- name: Smoke + IPTV E2E (assert key endpoints)
if: ${{ (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')) && steps.detect.outputs.docs_only != 'true' }}
env:
SMOKE_SHORT_SHA: ${{ steps.meta.outputs.short }}
SMOKE_RUN_ID: ${{ github.run_id }}
run: |
IMG="${IMAGE}:${SMOKE_SHORT_SHA}"
NAME="etv-smoke-${SMOKE_RUN_ID}"
IMG="${IMAGE}:${{ steps.meta.outputs.short }}"
NAME="etv-smoke-${{ github.run_id }}"
trap 'docker rm -f "$NAME" >/dev/null 2>&1 || true' EXIT
echo "Pulling ${IMG}"
docker pull "$IMG"
@@ -1048,28 +701,18 @@ jobs:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
if: github.event_name == 'pull_request'
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Detect API-surface changes
id: detect
run: |
base_ref="${{ github.base_ref }}"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only "origin/${base_ref}...HEAD")"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only "origin/${base_ref}...HEAD" 2>/dev/null || true)"
echo "Changed files in this PR:"; printf '%s\n' "$changed"
if printf '%s\n' "$changed" | grep -Eq '^ErsatzTV/Controllers/Api/|^ErsatzTV\.Core/Api/'; then
echo "api_changed=true" >> "$GITHUB_OUTPUT"
@@ -1153,28 +796,18 @@ jobs:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
if: github.event_name == 'pull_request'
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Detect changed C# files
id: detect
run: |
base_ref="${{ github.base_ref }}"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only --diff-filter=ACM "origin/${base_ref}...HEAD" -- '*.cs')"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only --diff-filter=ACM "origin/${base_ref}...HEAD" -- '*.cs' 2>/dev/null || true)"
echo "Changed .cs files in this PR:"; printf '%s\n' "$changed"
if [ -n "$changed" ]; then
printf '%s\n' "$changed" > /tmp/changed-cs.txt
+26 -394
View File
@@ -3,9 +3,8 @@ name: PR Gates
# Fast, git-only PR gates split out of docker-build.yml into a dedicated `on: pull_request`
# workflow (ersatztv#535) so they are NEVER created on a tag/main push.
#
# WHY THIS FILE EXISTS. These checks are cheap `checkout + git diff` gates (or, for `script-tests`,
# checkout + pytest): they carry no `container:`, run on the `small` lane (git-only, 1 GiB;
# server-management#639), and are PR-only.
# WHY THIS FILE EXISTS. These three checks are pure `checkout + git diff` gates: they carry no
# `container:`, run on the `small` lane (git-only, 1 GiB; server-management#639), and are PR-only.
# While they lived in docker-build.yml — which also triggers on push to main and on `v*` tags —
# Gitea still DISPATCHED them as runner tasks on every such push to evaluate the `if:` skip, because
# **Gitea dispatches a job as a runner task even when its `if` skips it** (docs/ci-cd.md -> the
@@ -23,10 +22,10 @@ name: PR Gates
#
# These stay on `runs-on: small` and carry NO CI toolchain image pin, so `ci-image-pin`'s grep of
# docker-build.yml still validates the five pin-bearing jobs (test/migrations/functional-e2e/
# api-docs/format) that remain there. None of these jobs are required checks — branch protection
# requires only `Build & test (.NET)`, `EF migration integrity` and `review-verdict/h10` — so
# relocating them (which changes their status-context prefix from "Build ErsatzTV Image / …" to
# "PR Gates / …") does not affect merges. See docs/ci-cd.md -> "PR gates workflow".
# api-docs/format) that remain there. None of these three are required checks — branch protection
# requires only `Build & test (.NET)` and `EF migration integrity` — so relocating them (which
# changes their status-context prefix from "Build ErsatzTV Image / …" to "PR Gates / …") does not
# affect merges. See docs/ci-cd.md -> "PR gates workflow".
on:
pull_request:
@@ -42,79 +41,40 @@ concurrency:
group: ersatztv-pr-gates-${{ github.ref }}
cancel-in-progress: true
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Holds no secrets at all and reads nothing from the Gitea API; the injected GITEA_TOKEN serves only
# its five `actions/checkout` steps.
permissions:
code: read
jobs:
# BLOCKING (ersatztv#390): the CI toolchain image pin in docker-build.yml must name the short sha of
# the last commit to touch the image's SOURCES (`docker/ci/**`). Read that as "the image ci-image.yml
# last published" only under the convention that every such commit is published — this job compares
# git shas and never queries the registry, so it cannot see a pin whose tag was never built or has
# been evicted. Existence is `toolchain-preflight`'s job, and the container jobs' pull is the backstop.
# Since ersatztv#744 publishing from a branch is a `workflow_dispatch`, so "was it published" is a
# human step this job does not observe.
#
# Without this detector, a PR that edits docker/ci/** ships a new image RECIPE while running its own
# BLOCKING (ersatztv#390): the CI toolchain image pin in docker-build.yml must name the image that
# ci-image.yml actually last published — i.e. the short sha of the last commit to touch the image's
# sources. Without this detector, a PR that edits docker/ci/** publishes a NEW image but runs its own
# jobs against the OLD pin: CI green-lights a toolchain it never executed, and once merged, main's
# Dockerfile silently disagrees with what CI runs. **Renovate actively generates exactly that PR** —
# it manages docker/ci/Dockerfile's base pins (dockerfile manager) but cannot bump an opaque
# `:<sha>` in `container.image`, so it would leave the pin behind every time.
#
# Failing here forces the documented two-step (docs/ci-cd.md -> "CI toolchain image"): get the
# Dockerfile change published as `:<sha>`, then update the pin to that sha. Since ersatztv#744 the
# publish half of that two-step is a `workflow_dispatch` on the branch rather than a side effect of
# the push — ci-image.yml's `push` trigger is now `branches: [main]`. Seconds-long git+grep -> keep
# it off the build runners.
# Failing here forces the documented two-step (docs/ci-cd.md -> "CI toolchain image"): push the
# Dockerfile change, let ci-image.yml publish `:<sha>`, then update the pin to that sha. Seconds-long
# git+grep -> keep it off the build runners.
ci-image-pin:
name: CI image pin matches docker/ci
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# need real history: `git log -- <path>` on a shallow clone can't find the last
# commit that touched the image sources
fetch-depth: 0
- name: Verify the pin matches the image-source commit
- name: Verify the pin matches the last-published image
run: |
set -euo pipefail
# ci-image.yml tags the image `git rev-parse --short HEAD` of the run that built it. Only
# its filtered `push` clause requires a `docker/ci/**` change; the weekly `schedule` and a
# `workflow_dispatch` both build the selected ref's HEAD whatever it touched. So `expected`
# is not a model of every tag in the registry — it is the one tag a PR is REQUIRED to be
# pinned to: the last commit to change the image's sources.
#
# `.gitea/workflows/ci-image.yml` is deliberately NOT part of `expected` (ersatztv#744),
# and that is a DECIDED TRADEOFF, not a necessity. Keeping it is workable — dispatch the
# branch at the ci-image.yml commit, then pin it — but it prices every edit to that file,
# comments included, at a full ~2GB publish plus a five-pin bump, redone after every
# rebase. Dropping it prices the opposite risk: a change to HOW the image is built living
# ONLY in ci-image.yml (build-args, Dockerfile path, platforms) neither republishes nor
# invalidates the pin, so CI keeps running an image built by the previous recipe. The
# second was chosen because that file is edited far more often for triggers, comments and
# runner placement than for build recipe. Make a recipe change alongside a `docker/ci/**`
# edit — a comment bump suffices, and it is the ONLY remedy: pinning the workflow-only
# commit is rejected here, because `expected` is the last `docker/ci` commit.
# Nothing MECHANICALLY couples this pathspec to `ci-image.yml`'s `on.push.paths`; before
# #744 the shared self-reference kept them in step. Divergence is silent and green in the
# dangerous direction — tracked in ersatztv#855.
# See docs/ci-cd.md -> "Publishing from a branch is a dispatch, not a push".
# ci-image.yml tags the image `git rev-parse --short HEAD` of the push that built it, and it
# only builds on pushes touching these paths — so the published image is named by the last
# commit to touch them.
#
# Compare RESOLVED FULL shas, never the abbreviations: git auto-scales abbreviation length
# with the repo's object count, so the tag built in CI from a `fetch-depth: 1` shallow clone
# is 7 chars while `%h` here (full clone) is 8. Comparing those strings would fail always.
expected="$(git log -1 --format=%H -- docker/ci)"
expected="$(git log -1 --format=%H -- docker/ci .gitea/workflows/ci-image.yml)"
mapfile -t pins < <(grep -oE 'ersatztv-ci:[0-9a-f]+' .gitea/workflows/docker-build.yml | cut -d: -f2 | sort -u)
echo "Image sources last changed in: ${expected}"
echo "Pins found in docker-build.yml: ${pins[*]} (${#pins[@]} distinct)"
@@ -146,10 +106,10 @@ jobs:
# in-repo remedy in that state: relax this length check in the same PR and say why. Note
# that ci-image.yml still tags with a plain `--short` (auto-scaled), so "always 7" is an
# empirical property of today's shallow clone, not an enforced invariant. Making the
# publisher emit `--short=7` is tracked as ersatztv#597. That is no longer blocked by this
# job at all: since ersatztv#744, editing ci-image.yml does NOT re-point `expected`, so a
# `--short=7` change lands like any other PR. It does need a deliberate republish to take
# effect — see the note on `expected` above.
# publisher emit `--short=7` is tracked as ersatztv#597. It is not blocked, just out of
# scope here: editing ci-image.yml re-points `expected` (above) at that commit, so it needs
# the branch's own publish-then-pin two-step (docs/ci-cd.md -> 'CI toolchain image') —
# ci-image.yml's push trigger has no branches: filter, so a feature branch does publish.
if [ "${#pins[0]}" -ne 7 ]; then
echo "::error::CI toolchain image pin ersatztv-ci:${pins[0]} is ${#pins[0]} chars, but ci-image.yml publishes 7-char tags (it tags with 'git rev-parse --short HEAD' from a fetch-depth:1 clone). A differently-sized abbreviation still resolves to the right commit, so this would pass every other check here — but NO such tag exists in the registry, and all five container: jobs would fail at image-pull time with 'manifest unknown'. Pin exactly: ersatztv-ci:${expected:0:7} (locally: git rev-parse --short=7 HEAD). See docs/ci-cd.md -> 'CI toolchain image'."
exit 1
@@ -160,7 +120,7 @@ jobs:
exit 1
fi
if [ "$pin_full" != "$expected" ]; then
echo "::error::CI toolchain image pin is stale: docker-build.yml pins ersatztv-ci:${pins[0]} ($pin_full), but docker/ci was last changed in $expected. Your jobs are testing an image that is NOT built from this PR's docker/ci. Publish the new :<sha> — push this commit as branch HEAD and dispatch ci-image.yml on the branch (a branch PUSH no longer publishes, ersatztv#744) — then update the pin in ALL jobs to it (docs/ci-cd.md -> 'CI toolchain image')."
echo "::error::CI toolchain image pin is stale: docker-build.yml pins ersatztv-ci:${pins[0]} ($pin_full), but docker/ci was last changed in $expected. Your jobs are testing an image that is NOT built from this PR's docker/ci. Let ci-image.yml publish the new :<sha>, then update the pin in ALL jobs to it (docs/ci-cd.md -> 'CI toolchain image')."
exit 1
fi
echo "Pin is current: ersatztv-ci:${pins[0]} resolves to $pin_full = docker/ci's last change."
@@ -173,32 +133,16 @@ jobs:
name: Docs update reminder
runs-on: small # seconds-long git diff; keep it off the build runners
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: report-only
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
# `continue-on-error` for the same reason the two steps below carry it: this whole job
# is a non-blocking nudge, and an advisory red still joins the combined status the merge gate
# reads. Unmasking the fetch (ersatztv#746) makes a broken base LOUD in the log; it must not
# also make a warn-only job merge-blocking. The three jobs that genuinely gate on this diff —
# api-docs, format, decisions lifecycle — do redden on a failed fetch, which is where that
# belongs.
- name: Warn when a screen/route change skips the parity doc
continue-on-error: true
run: |
base_ref="${{ github.base_ref }}"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only "origin/${base_ref}...HEAD")"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only "origin/${base_ref}...HEAD" 2>/dev/null || true)"
echo "Changed files in this PR:"; printf '%s\n' "$changed"
screen_or_route=no
if printf '%s\n' "$changed" | grep -Eq '^web/src/screens/.+\.tsx$|^ErsatzTV/LegacyUiRedirects\.cs$'; then
@@ -214,44 +158,9 @@ jobs:
echo "Parity-doc reminder: nothing to flag."
fi
# ersatztv#784 — ADVISORY nudge for `docs.no-session-narrative`. Deliberately NON-BLOCKING and
# deliberately in this job rather than a gate of its own: it is a string predicate over prose,
# and `docs/defect-shapes-773.md` §4 argues that class must not be load-bearing. The script
# exits 0 on every path (asserted per argument shape in scripts/tests/test_check_doc_narrative.py,
# not only in prose), so this step cannot redden the run even on a hit; if you find yourself
# wanting it to fail, read the decision record first — it says no in as many words.
# `python3` is not guaranteed on the bare `small` lane (docs/ci-cd.md), and every other
# python-using job on it declares this. Without it a missing interpreter is exit 127 — a RED
# advisory job joining the combined status, which is the one thing this step must never be.
#
# Both steps OF THIS CHECK (setup-python + the narrative step; the parity nudge above has its
# own) carry `continue-on-error` because the SCRIPT exiting 0 is not the whole invariant:
# a setup-python download failure reddens the job just as effectively as a hit would, and an
# advisory red still joins the combined status the merge gate reads (ersatztv#598). Scope,
# stated rather than implied: this covers the two steps that exist to run the check. A failed
# `Checkout` is NOT covered and deliberately so — with no tree there is nothing to check, and
# a job that cannot run is a different failure from an advisory one that ran and disagreed.
# Measured on this runner (PR#811, run 2179): the job reports `success` and the commit status
# context is `success` with both steps green under `continue-on-error`.
- name: Set up Python
uses: actions/setup-python@v5
continue-on-error: true
with:
python-version: '3.x'
- name: Warn when a doc narrates its own revision history
continue-on-error: true
run: |
base_ref="${{ github.base_ref }}"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
python3 scripts/check-doc-narrative.py --diff "origin/${base_ref}"
# BLOCKING (ersatztv#521, supersedes the ersatztv#303 H9 append-only mechanic): validates decision-
# record lifecycle invariants (metadata schema, one active record per key, reciprocal
# supersedes/superseded-by links, no rationale-prose rewrite without a Decisions-Edit: yes git
# trailer (ersatztv#609 — never a bare substring, which prose about the marker could arm), no record
# supersedes/superseded-by links, no rationale-prose rewrite without [decisions-edit], no record
# vanishing from the active set without an archive copy) and that the generated active catalog
# (docs/decisions/README.md) is in sync. Same validator the Husky pre-commit hook shim calls, so
# local and CI enforcement can't drift. Seconds-long git diff + parse -> keep it off the build runners.
@@ -259,13 +168,10 @@ jobs:
name: decisions lifecycle
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
@@ -274,283 +180,9 @@ jobs:
- name: Validate decision lifecycle
run: |
base_ref="${{ github.base_ref }}"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
git fetch --no-tags --depth=200 origin "$base_ref" || true
PYTHONPATH=. python3 scripts/decisions_validate.py --base "origin/${base_ref}" --head HEAD
- name: Active catalog in sync
run: PYTHONPATH=. python3 scripts/build_decisions_catalog.py --check
- name: Kickoff guard
run: bash scripts/check-kickoff-guard.sh
# FAILS THE RUN on a red (ersatztv#631) — like its sibling gates here it is not (yet) a required
# status check, so it reddens the PR without hard-blocking the merge button; see the header.
# Runs scripts/tests/ — the pytest suite covering the decision-corpus
# parser/validator/catalog builder, the #610 migration-equivalence harness, the merge-consent
# exemption logic and the #622 review-verdict poster. Until #631 NOTHING executed these: no
# workflow and no Husky hook invoked pytest, so the suite guarding our merge-gating machinery was
# local-only and a regression in it was caught only by luck. `decisions-guard` above runs that
# code, but never its tests.
#
# WHY ITS OWN JOB rather than a step inside decisions-guard (which the issue proposed as the
# cheapest home): `ci.decisions-lifecycle-flake` is a STANDING instruction that a lone
# `decisions lifecycle` red is a known infra flake to be ignored — "do not investigate". Folding
# the suite into that job would make a genuine pytest regression present as exactly the red every
# session is told to wave through, which is the same silently-green failure mode #631 exists to
# close. A distinct job name keeps a real failure unambiguous.
#
# Runs UNCONDITIONALLY on every PR rather than behind a `scripts/**` path filter. The suite's
# corpus tests are fixture/tmp-repo based, but several execute REAL artifacts from other top-level
# directories: test_post_review_verdict.py runs `scripts/post-review-verdict.sh`,
# test_merge_consent_exemption.py runs `.claude/hooks/pretooluse-merge-consent.sh`, and since
# ersatztv#845 test_post_review_verdict.py ALSO reads `.gitea/workflows/review-verdict.yml` —
# the writer derives the H10 allow-list from it, so editing that literal changes the suite's
# outcome. Its true input set therefore spans at least three top-level directories, and this
# enumeration is the kind that goes stale: a `scripts/**` filter would silently miss a
# `.claude/hooks/**` or `.gitea/workflows/**` edit. The reason is the INPUT SET, not the cost —
# the suite was ~10s when that was decided and is minutes now, and filtering on `scripts/**`
# would still be wrong.
prove-fix:
name: "Fix proofs (Proves trailers)"
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
# Full history: prove-fix.sh reverts each commit against its PARENT, so a shallow
# clone would leave it unable to resolve `<sha>^` and it would refuse every commit.
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install test dependencies
run: python3 -m pip install --disable-pip-version-check --quiet pytest pyyaml
# OPT-IN BY TRAILER, deliberately. Requiring `Proves:` on every commit would block
# docs, CI and refactor commits that have no code side to revert, and a gate that
# blocks ordinary work gets disabled — which is how a check ends up running nowhere
# (#631). So the trailer is the AUTHOR'S CLAIM, and this job checks claims: write
# one and it must hold. Coverage is therefore honest rather than assumed, and
# `docs/decisions/records/testing/fix-ships-a-witnessed-red-test.md` says so.
- name: Prove every commit that claims a proof
run: |
set -uo pipefail
base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}"
echo "range: $base..$head"
# Capture and VALIDATE the enumeration before looping. `for sha in $(git ...)`
# swallows a git failure: the command substitution yields nothing, the loop body
# never runs, and the job reports "0 claims" green. Fail-open enumeration in the
# thing that decides what gets checked is the defect this job exists to catch.
if ! shas="$(git rev-list "$base".."$head")"; then
echo "::error::git rev-list failed for $base..$head — cannot enumerate commits," \
"so this job cannot assert anything. Refusing to pass."
exit 1
fi
claimed=0; proven=0; failed=0
while IFS= read -r sha; do
[ -n "$sha" ] || continue
# Trim whitespace only — NOT `xargs`, which applies quote parsing and turns a
# legitimate parametrised node id like test_x[can't] into an empty selector,
# silently dropping a real claim.
# Extract with a CHECKED status. `sel="$(git show ... )"` under `set -uo
# pipefail` but no `-e` yields an empty selector when git fails, the commit is
# skipped, and the job exits 0 having been unable to inspect a possible claim —
# fail-open in the step that decides what gets checked.
if ! raw="$(git show -s --format='%(trailers:key=Proves,valueonly)' "$sha")"; then
echo "::error::git show failed for $sha — cannot read its trailers, so this" \
"job cannot assert anything about it. Refusing to pass."
exit 1
fi
# Refuse MORE THAN ONE `Proves:` here too. prove-fix.sh has this guard, but it
# only fires when it reads the trailer itself — and this job passes the selector
# explicitly, so the guard was bypassed on the one path that actually enforces.
# Measured: a commit with two trailers reported PROVEN while the second was never
# run. Fixing the script and not its twin is how a guard reads as coverage.
# Count trailer PRESENCE, not non-empty values: `%(...valueonly)` renders a bare
# `Proves:` as an empty line, so counting non-empty lines misses a commit whose
# FIRST trailer is empty — `sel` then comes out empty and the commit is skipped
# in silence, with a real second selector never checked. Fail-open in CI while
# the script is fail-closed is the same asymmetry this guard exists to remove.
present="$(git show -s --format='%(trailers:key=Proves)' "$sha")"
if [ "$(printf '%s\n' "$present" | grep -c .)" -gt 1 ]; then
claimed=$((claimed + 1)); failed=$((failed + 1))
echo "::error::commit $sha carries more than one 'Proves:' trailer; only the" \
"first would be checked, so the rest would read as proven without ever" \
"running. Use a single selector."
continue
fi
sel="$(printf '%s\n' "$raw" | head -1 | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
# A trailer that is PRESENT but empty is a claim with no selector. Refuse it
# loudly; skipping it silently would let the job report "no claims" for a PR that
# made one.
if [ -n "$present" ] && [ -z "$sel" ]; then
claimed=$((claimed + 1)); failed=$((failed + 1))
echo "::error::commit $sha carries a 'Proves:' trailer with no selector."
continue
fi
[ -n "$sel" ] || continue
claimed=$((claimed + 1))
# A merge commit has several parents, so "before this change" is ambiguous.
# prove-fix.sh refuses them; catch it here with a clearer message rather than
# letting the trailer be silently skipped (which --no-merges used to do).
if [ "$(git rev-list --parents -n 1 "$sha" | wc -w)" -gt 2 ]; then
failed=$((failed + 1))
echo "::error::commit $sha is a MERGE carrying 'Proves: $sel'. Put the trailer" \
"on the commit that carries the fix — a merge has no single 'before'."
continue
fi
echo "::group::prove $sha -> $sel"
if bash ./scripts/prove-fix.sh "$sha" "$sel"; then
proven=$((proven + 1)); echo "PROVEN $sha"
else
rc=$?
failed=$((failed + 1))
echo "::error::commit $sha claims 'Proves: $sel' but prove-fix.sh exited $rc." \
"A claimed proof that does not hold is worse than none — it reads as" \
"coverage. Strengthen the test until reverting the fix reddens it, or" \
"drop the trailer."
fi
echo "::endgroup::"
done <<< "$shas"
echo "commits claiming a proof: $claimed (proven $proven, failed $failed)"
if [ "$claimed" -eq 0 ]; then
echo "::notice::No commit in this PR carries a 'Proves:' trailer, so nothing was" \
"verified here. That is allowed — the trailer is opt-in — but it means this" \
"job asserts NOTHING about this PR. Do not read its green as fix coverage."
fi
[ "$failed" -eq 0 ]
script-tests:
name: Script lint and tests (ruff + pytest)
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
# Preflight, not an install (ersatztv#390 removed run-time `apt-get` from CI on purpose).
# Two consumers need `git`: the lint steps below derive their population from `git ls-files`,
# and test_post_review_verdict.py / test_merge_consent_exemption.py exec the REAL
# post-review-verdict.sh / pretooluse-merge-consent.sh. `curl` those tests shim on PATH; `jq`
# and `git` they do NOT. It stays AHEAD of the lint steps, not merely ahead of pytest: without
# it, a missing git reaches the lint steps as an empty population, which they report as a
# population problem. One actionable line beats a misdirected one, and beats the wall of
# unattributable assertion failures the suite produces without git.
- name: Preflight external tools
run: |
if ! command -v git >/dev/null 2>&1; then
echo "::error::script-tests needs git on PATH but it is absent. The lint steps derive" \
"their population from it and the suite execs real shell scripts that use it." \
"Bake it into the runner image rather than apt-get installing here (ersatztv#390)."
exit 1
fi
echo "Preflight OK: $(git --version)"
# ersatztv#780. Lint runs EARLY — after the git preflight it depends on, but before the test
# dependencies, the jq preflight and the ~4-minute pytest run. A style red therefore arrives in
# seconds, and, more importantly, the lint does not sit behind `Preflight jq version`: that is
# an `--expect` tripwire, so a runner jq bump would take the lint dark for as long as the jq
# contract is broken, under a red that says "jq".
#
# The version is PINNED: an unpinned ruff makes the verdict a function of whenever the job ran
# — the same environment-divergence the committed ruff.toml exists to close. Bumping it is a
# deliberate PR (new rules may fire), exactly like the jq pin below. `pytest`/`pyyaml` are
# deliberately NOT pinned: a pytest release does not add assertions to your suite, a ruff
# release adds rules to your lint.
- name: Install ruff
run: python3 -m pip install --disable-pip-version-check --quiet 'ruff==0.12.11'
# POPULATION. Both steps lint an EXPLICIT list from `git ls-files`, never `ruff check .`, and
# pass `--no-force-exclude`. Measured with ruff 0.12.11 and `exclude = ["scripts/**"]` — a
# per-FILE pattern, because `exclude` matches per file: a bare `["scripts"]` still works at the
# top level but matches nothing under `[lint]`/`[format]`. The subject is a planted tracked file
# holding an unused import, a hardcoded credential and a formatting error. GREEN means the gate
# was silently off:
#
# DISCOVERY FORM EXPLICIT FORM (what ships)
# exclude scope check . format --check . check format --check
# top-level GREEN GREEN red red
# [lint] GREEN red red red
# [format] red GREEN red red
# top + force-exclude GREEN GREEN red red <- with the flag
# GREEN GREEN <- without it
#
# Only the top-level scope empties BOTH discovery commands; `[lint]` empties `check` and
# `[format]` empties `format --check`, so in those two the job would still redden on the other
# step. `[format]` is where a line appended to ruff.toml lands, by TOML rules. `include = []`,
# `extend-exclude` and a nested `scripts/ruff.toml` behave the same way and are equally inert
# against the explicit form. The last row is the whole reason for `--no-force-exclude`:
# `force-exclude = true` re-applies excludes to explicitly-passed paths, and is the one setting
# that reaches explicitly-passed paths at all.
#
# `ruff check .` over an empty tree exits **0** with only a stderr warning, so every GREEN above
# is a gate that was switched off without a red.
#
# This also derives the population from source rather than from the filesystem
# (docs/decisions/records/testing/guard-derives-population-from-source.md) and covers
# tracked-but-gitignored files, which `ruff check .` skips. The empty-population arm is the
# anti-vacuity check: a completeness check whose population is empty reports that it proved
# everything. What it does NOT cover: an emptied RULE set. `select = []` silences every selected
# rule, so the `ruff check` step goes green over any lint violation (a syntax error still reds)
# while printing a reassuring file count.
# `ruff format --check` is unaffected, because formatting is not rule-selected. So half the
# gate is killable by a config edit, and only a human reading that edit catches it.
- name: Lint scripts (ruff check)
run: |
mapfile -d '' -t PYFILES < <(git ls-files -z '*.py' '*.pyi' '*.ipynb')
if [ "${#PYFILES[@]}" -eq 0 ]; then
echo "::error::the lint population is EMPTY — git tracks no Python files. Either the" \
"checkout is wrong or the glob is. A lint over nothing passes; see ersatztv#780."
exit 1
fi
echo "Linting ${#PYFILES[@]} tracked Python files"
python3 -m ruff check --no-force-exclude -- "${PYFILES[@]}"
- name: Lint scripts (ruff format --check)
run: |
mapfile -d '' -t PYFILES < <(git ls-files -z '*.py' '*.pyi' '*.ipynb')
if [ "${#PYFILES[@]}" -eq 0 ]; then
echo "::error::the format population is EMPTY — git tracks no Python files. See ersatztv#780."
exit 1
fi
echo "Format-checking ${#PYFILES[@]} tracked Python files"
python3 -m ruff format --check --no-force-exclude -- "${PYFILES[@]}"
# pytest + PyYAML. PyYAML is NOT a contradiction of the dependency-free decisions READ path:
# `decisions_lib._read_frontmatter` is hand-written precisely so validation runs where nothing
# is installed, but the one-shot WRITE path `migrate_decisions_split.py` uses PyYAML by
# design — and `test_migration_equivalence.py` imports that module, so the suite needs it.
# `pytest` and `yaml` are the complete third-party set, established by an AST import scan over
# all of scripts/ rather than by reading the files that seemed relevant: the first cut of this
# job claimed "pure stdlib", passed locally on a machine that happened to have PyYAML, and
# went red in CI on a collection error.
- name: Install test dependencies
run: python3 -m pip install --disable-pip-version-check --quiet pytest pyyaml
# jq gets its OWN step because its VERSION, not merely its presence, is load-bearing
# (ersatztv#648). `--expect` makes this a TRIPWIRE: scripts/tests exercises the jq 1.6 code path
# only because this runner ships 1.6, so an upgrade would silently delete that coverage — and
# the three divergences found in ersatztv#643/#647 all lived exactly there. Going red forces an
# explicit human decision instead of letting the coverage evaporate.
#
# The pin lives HERE and deliberately NOT in review-verdict.yml: that workflow writes the
# branch-protection-required `review-verdict/h10` status, so pinning a version there would turn
# any jq bump on the runner into a repo-wide merge deadlock. It gets the floor-only mode.
# See docs/ci-cd.md -> "The jq contract".
- name: Preflight jq version
run: ./scripts/jq-preflight.sh --expect 1.6
- name: Run scripts/tests
run: PYTHONPATH=. python3 -m pytest scripts/tests -q
-14
View File
@@ -45,26 +45,12 @@ concurrency:
group: ersatztv-renovate
cancel-in-progress: false
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# This workflow has no checkout step and never uses the injected GITEA_TOKEN for anything. Renovate's
# own branch/PR writes go through RENOVATE_TOKEN, a dedicated bot PAT the Actions default does not
# govern, and its container image comes from Docker Hub. Read-only is declared to STATE that the
# injected token is unused, not because any step needs it.
permissions:
code: read
jobs:
renovate:
name: Renovate
runs-on: ubuntu-latest
container:
image: renovate/renovate:43
env:
CI_JOB_ROLE: none
steps:
- name: Run Renovate
env:
File diff suppressed because it is too large Load Diff
-26
View File
@@ -10,10 +10,6 @@ project.lock.json
# Claude Code
.mcp/
.mcp.json
# Machine-local settings (DOTNET_ROOT and friends — see docs/local-lsp-tooling.md).
# Ignored here rather than relying on a personal ~/.config/git/ignore, so a second
# contributor following that doc cannot accidentally commit their own Homebrew paths.
/.claude/settings.local.json
.agents/
plugins/
nupkg/
@@ -74,11 +70,6 @@ ErsatzTV/wwwroot/app/
web/dist/
web/node_modules
# Root-level link that makes `typescript` resolvable from the repo root, which is
# the LSP workspace root — without it typescript-language-server refuses to start
# (ersatztv#777). See docs/local-lsp-tooling.md.
/node_modules/
# E2E / screenshot scratch (from Playwright/live-E2E runs) — never committed
/*.png
.playwright-mcp/
@@ -89,20 +80,3 @@ web/playwright-report/
# Per-session worktree-ownership marker (H7, ersatztv#303) — local, never committed
.claude-worktree-owner
# Codex CLI project scaffolding — a machine-local mirror of the .claude hooks, generated by
# `codex exec`. Deliberately NOT tracked even though `.claude/` is: its config.toml embeds a
# plaintext Gitea credential and absolute /Users paths, so it is neither portable nor safe to
# commit. See ersatztv#711 for the related merge-gate gap.
.codex/
# serena's per-project state, written by `activate_project` (ersatztv#799): project.yml,
# project.local.yml, a language-server cache, and memories/.
#
# This deliberately rejects serena's own versioning model. Its nested .serena/.gitignore excludes
# only `cache` and `project.local.yml`, and project.local.yml says project.yml "is intended to be
# versioned" — but activation here is per DIRECTORY, and every worktree generates a project.yml
# whose project_name is that worktree's folder (e.g. `781-tooling`). A committed copy would name
# the wrong project in every checkout but the one that produced it. memories/ is ignored with it:
# it is serena's own written notes, and this repo's durable knowledge lives in docs/ instead.
.serena/
+2 -3
View File
@@ -12,9 +12,8 @@ unset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE
# H11 (ersatztv#311): refuse to push a branch that is BEHIND origin/main — rebase, don't merge
# main in (a merge drags in files you never touched, e.g. legacy-BOM .cs, and trips the format
# hook on code that isn't yours). Fail-open; escape with ETV_SKIP_REBASE_CHECK=1. Exempts a
# tag-only push (ersatztv#719) — forward the ref lines captured above so it can tell.
printf '%s\n' "$_prepush_refs" | ./.claude/hooks/prepush-rebase-check.sh || exit 1
# hook on code that isn't yours). Fail-open; escape with ETV_SKIP_REBASE_CHECK=1.
./.claude/hooks/prepush-rebase-check.sh || exit 1
# H13 (ersatztv#416 session): refuse to push when a file in the pushed diff still has uncommitted
# working-tree/index changes — the pushed commit wouldn't match what you built/reviewed (the #416
+9 -47
View File
@@ -42,7 +42,7 @@ docker build -f docker/Dockerfile -t ersatztv:dev .
| Migrate / add / redirect a route (new `web/src/screens/*.tsx`, `LegacyUiRedirects.cs`) | `docs/blazor-route-parity.md` + `docs/domain-model.md` |
| Add / change a `/api/*` endpoint | `docs/api-conventions.md` checklist, then regenerate `v1.json` + `endpoint-index.md` via `./scripts/update-openapi.sh` |
| Change a SPA screen convention | `docs/spa-conventions.md` |
| Establish / reverse a convention or decision | a new `docs/decisions/records/<area>/<topic>.md` (filename = key; lifecycle: add record, `git mv` predecessor to `archive/<area>/`) + regenerate the catalog + the affected doc |
| Establish / reverse a convention or decision | `docs/decisions.md` (lifecycle: add record, relocate predecessor to archive/) + the affected doc |
| Add / remove / retitle a doc | `docs/README.md` index |
The `docs-reminder` CI job flags a screen/route change that skips `blazor-route-parity.md`, but it's a **non-blocking** nudge — the rule is on you, not the check.
@@ -52,44 +52,19 @@ docker build -f docker/Dockerfile -t ersatztv:dev .
- Test with **NUnit** + Shouldly + NSubstitute (the existing `*.Tests` projects); xUnit is **not** used here
- **Dependencies use Central Package Management**: versions live in the repo-root `Directory.Packages.props`; csproj reference packages by name only. Add/upgrade by editing the central `<PackageVersion>` — never put `Version=` back on a `<PackageReference>` (trips `NU1008`). See `docs/ci-cd.md` → Dependency management.
- **DB migrations target BOTH providers**: a `TvContext` model change needs a migration in `ErsatzTV.Infrastructure.Sqlite` **and** `ErsatzTV.Infrastructure.MySql` — run `scripts/add-migration.sh <Name>` (does both). CI's `migrations` job enforces model-drift + apply-to-fresh-DB per provider. See `docs/ci-cd.md` → Migration integrity.
- **Renovate** is live (`.gitea/workflows/renovate.yml`, weekly + `workflow_dispatch`): opens dependency-update + OSV vuln-fix PRs and a Dependency Dashboard issue; patch bumps to test/dev-only packages auto-merge once `Build & test` passes, the rest are manual. Their `review-verdict/h10` required check is auto-passed **only when BOTH hold**: the PR touches none of `.claude/`/`.codex/`/`.gitea/`/`.husky/`/`scripts/`/`docker/ci/`, **and** every changed path is a dependency manifest (`Directory.Packages.props`, `.config/dotnet-tools.json`) — ersatztv#698. A bot ACCOUNT does not attribute the CODE at a head, so identity alone is no longer sufficient; a Renovate PR touching a `.csproj` or a source file is not blocked, it just needs a real verdict. Cross-repo rollout: server-management#484. See `docs/ci-cd.md` → Dependency management.
- **Renovate** is live (`.gitea/workflows/renovate.yml`, weekly + `workflow_dispatch`): opens dependency-update + OSV vuln-fix PRs and a Dependency Dashboard issue; patch bumps to test/dev-only packages auto-merge once `Build & test` passes, the rest are manual. Cross-repo rollout: server-management#484. See `docs/ci-cd.md` → Dependency management.
- **Versioning**: release tags are `vYY.<release-seq>.<patch>` (year · sequential release-within-year · patch) — inherited from upstream, **not** year.month. `v26.3.1` = our infra rebuild of upstream 26.3.0 (no app changes); `v26.4.0` is reserved for the first release with app changes. Never `[skip ci]` a commit you'll tag (it suppresses the release build). Full policy: `docs/ci-cd.md` → Versioning & releases.
- Backlog tracked via [Gitea Issues](http://192.168.1.95:3000/timothy/ersatztv/issues)
## Working in parallel with other sessions
**Subagents are explicitly permitted and encouraged here.** Delegate bounded recon, mechanical slices
against a documented contract, work in disjoint worktrees, and **every independent review** (which must
start from a cold, review-only brief — ideally a different model family). Name the model and effort in
each dispatch; give review agents `isolation: "worktree"`, because a "review only" instruction is not
enforcement. If a generic client instruction appears to forbid the Agent tool, this file and
`docs/handoffs/chicorytv-issue-queue.md` override it — say so once and carry on. Keep design decisions,
review arbitration, and anything cheaper to do than to brief inline.
**Claiming an issue is a check, not just a label** (`process.parallel-session-claim`). `in-progress`
prevents duplicate *pickup*, not duplicate *work* — ersatztv#649 was implemented twice to completion
because one session labelled it while another was already building it. Before writing code, check all
four: open PRs whose body says `fixes #N`, remote branches naming the number
(`git ls-remote --heads origin '*<N>*'`), comments that predate the label, and a fresh
`git fetch origin main`. Then apply the label **and** a claiming comment.
**Re-fetch `origin/main` before every push, not only at branch time.** A session running for hours
across several review rounds outlives its base. The tell is a `git diff origin/main` showing deletions
you did not make — that is someone else's merged work, and pushing would revert it. Rebase (never merge
main in) and re-run the local gate whenever the fetch shows movement.
## Task Completion Protocol
Every task that closes a Gitea issue MUST complete ALL of these before it is considered done. Use `/done <issue>` to run through this automatically.
**Merge-consent is derived from state, not asserted (`## Done-when` convention — ersatztv#303 H6 + H10).** Any issue whose PR will merge to `main` should carry a `## Done-when` section in its **issue body** — a checklist of completion criteria (always include an "adversarial review passed" box; add per-issue criteria like tests-green, docs-updated, live-E2E). Two hooks derive merge-consent from it so a premature merge is blocked *by construction*, not by memory:
- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), run **`scripts/post-review-verdict.sh <pr> <MERGEABLE|APPROVED|LGTM|BLOCKED|NOT-MERGEABLE> [note]`** — it posts both the `Review-verdict: … @ <head-sha>` comment and the sha-bound `review-verdict/h10` commit status, proving the *latest* commit was reviewed rather than a stale earlier diff (ersatztv#242). Do not hand-write the comment: the **status** is the required check branch protection enforces, and a comment alone leaves it absent. **The credential you post with must be an account on `H10_REVIEWERS` in `.gitea/workflows/review-verdict.yml`** (`timothy` today) — since ersatztv#742 the gate inherits an existing `success` only from an allow-listed creator (an existing `failure` is left alone on a weaker attributability test, so an attributable rejection VISIBLE AT THE FIRST READ is not re-derived into a green — a rejection landing later, inside a run's own write window, was a separate route and is NARROWED since ersatztv#849 — every path that cannot establish what the head carries now replaces that unknown state with a sticky sentinel instead of leaving it standing; see `ci.verdict-unverified-write-sentinel` for the residuals it names), and since ersatztv#845 the script ENFORCES that coupling rather than assuming it: it reads its own status back and refuses, before writing the verdict comment, unless the recorded `.creator.login` is on that allow-list — so a POSITIVE verdict posted with any other account fails loudly at your terminal instead of being reported as success. The gate still re-derives such a status on the next PR event — that part is unchanged; what the check removes is the tool telling you it worked. **The membership requirement is `success`-only**, mirroring the gate: a `BLOCKED` verdict is honoured from ANY attributable account, so an off-list reviewer can still record a rejection. **The status is still written** — the check runs after the POST, because it measures the creator Gitea recorded rather than what the credential claims — and what is withheld is the verdict COMMENT, which leaves the merge hook at condition (c) with nothing to classify, i.e. an `ask`. So a refused positive verdict leaves a green `review-verdict/h10` standing on that head that the gate itself will not inherit; branch protection binds the context NAME and not its issuer, so do not read that green as consent. The allow-list is derived from the workflow by `scripts/lib/h10-reviewers.sh`; it is never restated.
- **The gate is enforced server-side, per sha (ersatztv#622).** `review-verdict/h10` is a required status check on `main`. Because a commit status belongs to one sha, a commit pushed *after* an auto-merge is scheduled clears it and blocks the merge — closing the hole where `merge_when_checks_succeed` froze consent at scheduling time and Gitea later merged an unreviewed head. Renovate-authored and docs-only PRs are auto-passed by `.gitea/workflows/review-verdict.yml`, **except** when they touch `.claude/`, `.codex/`, `.gitea/`, `.husky/`, `scripts/` or `docker/ci/`. See `docs/ci-cd.md` → Review-verdict gate.
- `.husky/pre-push``prepush-donewhen.sh` — a fail-open backstop that blocks a direct `git push origin main` whose commits `fix #N` an issue with unticked boxes. **Since ersatztv#743 that push can no longer happen at all** (see below), so this hook is now belt-and-braces for a path the server refuses.
- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), post a PR comment with a line `Review-verdict: <MERGEABLE|APPROVED|BLOCKED> @ <head-sha>` — this proves the *latest* commit was reviewed, not a stale earlier diff (ersatztv#242).
- `.husky/pre-push` `prepush-donewhen.sh` — a fail-open backstop that blocks a direct `git push origin main` whose commits `fix #N` an issue with unticked boxes.
**`main` is PR-only — there is no direct-push path any more (ersatztv#743, `release.main-direct-push-disabled`).** Branch protection carries `enable_push: false` **and** `block_admin_merge_override: true`: a direct `git push origin HEAD:main` is refused server-side at pre-receive for every account including a site admin, the contents API is refused too, and an admin cannot `force_merge` past a missing or red required context. This is what makes `review-verdict/h10` load-bearing rather than conventional — Gitea only evaluates `status_check_contexts` on the PR merge path, so before this the whole gate was skippable with no forgery. Practically: **every** change to `main` goes through a PR, including a one-line docs fix. Tag pushes are unaffected (separate mechanism), so the release cut is unchanged.
Both need Gitea read creds in the env to enforce (**`ETV_GITEA_BASICAUTH=user:pass`** or `ETV_GITEA_TOKEN`; `ETV_GITEA_URL` overrides the base). Without them the merge hook asks and the push backstop is a no-op — the gate degrades to today's manual confirmation, never a silent pass. Docs-only PRs are exempt from the *review-verdict* gate; the direct-push exemption is moot now that direct pushes are refused outright.
Both need Gitea read creds in the env to enforce (**`ETV_GITEA_BASICAUTH=user:pass`** or `ETV_GITEA_TOKEN`; `ETV_GITEA_URL` overrides the base). Without them the merge hook asks and the push backstop is a no-op — the gate degrades to today's manual confirmation, never a silent pass. Docs-only PRs/pushes are exempt.
**The 7 mandatory completion steps and the `## Closing record` comment template** live in the
`closing-an-issue` skill (`.claude/skills/closing-an-issue/SKILL.md`) — invoke it (or `/done`)
@@ -97,30 +72,17 @@ when finishing a task that closes an issue.
## Project Boundaries
**ersatztv OWNS***developing the fork*: the ErsatzTV fork code (C#/.NET), the `/api/v1` REST
surface, M3U/XMLTV generation, the `ErsatzTV.Mcp` server, CI and releases, and the **`ersatztv`
skill** — whose canonical copy is `.claude/skills/ersatztv/SKILL.md` **here**. Both
`~/server-management/.claude/skills/ersatztv` and `~/media-management/.claude/skills/ersatztv` are
symlinks to it (ersatztv#617, #755). Edit it in this repo; never fork a second copy.
**The split that is easy to get wrong** (ersatztv#755, `process.ersatztv-owns-code-not-operations`):
channel/collection/schedule *code* is owned here; **channel OPERATIONS against the running instance
are not**. Creating and editing channels, lineups, collections, schedules, playouts, logos and
overlays on the live ErsatzTV belong to `media-management`. Driving prod from here is in scope only
as *verification of a change this repo is shipping* (live-E2E, a release smoke test) — not as
day-to-day channel work.
**ersatztv OWNS**: ErsatzTV fork code (C#/.NET), channel/collection/schedule management, M3U/XMLTV generation, the ErsatzTV skill in server-management.
**ersatztv does NOT own**:
- Channel/collection/schedule/playout **operations** against a live instance → media-management
- Docker compose configs → server-management (`~/downloadswarm/stacks/ersatztv/`)
- NFS mounts, Ansible, DNS, networking → server-management
- Content sourcing (yt-dlp downloads, Sonarr/Radarr libraries) → media-management
- Jellyfin skill → server-management. `.claude/skills/jellyfin` here is a **relative symlink** to `~/server-management/.claude/skills/jellyfin` (ersatztv#617 — it had silently become a stale divergent copy). It therefore resolves only in a checkout at `~/ersatztv`, not inside a git worktree; that is inherent to the cross-repo symlink pattern server-management already uses (`beets`, `radarr`, `sonarr`, …).
- Content sourcing (yt-dlp downloads, Sonarr/Radarr libraries) → media-management (planned)
- Jellyfin skill → server-management (symlinked)
**For infrastructure changes** (Docker, NFS, ports, Authelia): open an issue in `timothy/server-management`.
**For content/media sourcing questions and channel operations** (what goes into channels, yt-dlp
pipelines, editing a live channel): open an issue in `timothy/media-management`.
**For content/media sourcing questions** (what goes into channels, yt-dlp pipelines): open an issue in `timothy/media-management` once it exists; for now, `timothy/server-management`.
**For plan/audit reviews**: open `~/adversarial-reviewer` before significant architecture changes.
+5 -5
View File
@@ -6,7 +6,7 @@
<ItemGroup>
<PackageVersion Include="AsyncFixer" Version="2.1.0" />
<PackageVersion Include="Blurhash.SkiaSharp" Version="2.0.0" />
<PackageVersion Include="CliWrap" Version="3.10.4" />
<PackageVersion Include="CliWrap" Version="3.10.2" />
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
<PackageVersion Include="Dapper" Version="2.1.79" />
<PackageVersion Include="Destructurama.Attributed" Version="5.2.0" />
@@ -29,7 +29,7 @@
<PackageVersion Include="Lucene.Net.Analysis.Common" Version="4.8.0-beta00017" />
<PackageVersion Include="Lucene.Net.QueryParser" Version="4.8.0-beta00017" />
<PackageVersion Include="MediatR" Version="[12.5.0]" />
<PackageVersion Include="Meziantou.Analyzer" Version="3.0.129" />
<PackageVersion Include="Meziantou.Analyzer" Version="3.0.115" />
<PackageVersion Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.2" />
<PackageVersion Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="10.0.2" />
<PackageVersion Include="Microsoft.Extensions.Identity.Core" Version="10.0.2" />
@@ -75,7 +75,7 @@
<PackageVersion Include="RichTextKit.Stbear" Version="0.4.167.3" />
<PackageVersion Include="Roslynator.Analyzers" Version="4.15.0" />
<PackageVersion Include="Scalar.AspNetCore" Version="2.12.32" />
<PackageVersion Include="Scriban.Signed" Version="7.2.6" />
<PackageVersion Include="Scriban.Signed" Version="7.2.5" />
<PackageVersion Include="Serilog" Version="4.3.0" />
<PackageVersion Include="Serilog.AspNetCore" Version="10.0.0" />
<PackageVersion Include="Serilog.Extensions.Hosting" Version="10.0.0" />
@@ -93,8 +93,8 @@
<PackageVersion Include="SonarAnalyzer.CSharp" Version="10.27.0.140913" />
<!-- Direct pin to override EF Core 9's transitive SQLitePCLRaw 2.1.10 (vulnerable
bundled SQLite, GHSA-2m69-gcr7-jv3q). The 3.x line ships the patched native
(lib.e_sqlite3 3.50.3); core 3.0.4 satisfies Microsoft.Data.Sqlite's `>= 2.1.10`. (#8) -->
<PackageVersion Include="SQLitePCLRaw.bundle_e_sqlite3" Version="3.0.4" />
(lib.e_sqlite3 3.50.3); core 3.0.3 satisfies Microsoft.Data.Sqlite's `>= 2.1.10`. (#8) -->
<PackageVersion Include="SQLitePCLRaw.bundle_e_sqlite3" Version="3.0.3" />
<PackageVersion Include="System.CommandLine" Version="2.0.2" />
<PackageVersion Include="TagLibSharp" Version="2.3.0" />
<PackageVersion Include="Testably.Abstractions" Version="10.0.0" />
@@ -1,43 +0,0 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
namespace ErsatzTV.Application.Channels;
/// <summary>
/// #732: the On Now / Next overlay is a default rather than an opt-in, so every newly created channel
/// gets the built-in element attached.
/// </summary>
/// <remarks>
/// This lives in one place because there is more than one channel-creation path and they diverged
/// once already: <c>CreateChannelHandler</c> had it and <c>CreateChannelFromLineupHandler</c> -- the
/// SPA's primary "Add Channel" flow, and the one Auto-Tune bulk-creates through -- did not. Any new
/// site that persists a <c>Channel</c> must call this. The third site, <c>DbInitializer</c>'s default
/// channel, needs no call: it runs before <c>AttachOnNowNextByDefault</c> in the same startup, so the
/// backfill covers it.
/// </remarks>
public static class ChannelGraphicsDefaults
{
public static async Task Attach(TvContext dbContext, Channel channel, CancellationToken cancellationToken)
{
// HLS Direct is skipped because ErsatzTV is not transcoding there -- there is no frame
// pipeline to draw into, and the editor disables the toggle for the same reason. Identity is
// the element's filename, never its user-editable Name (the #67 lesson).
if (channel.StreamingMode is StreamingMode.HttpLiveStreamingDirect)
{
return;
}
Option<int> maybeElementId =
await GraphicsElementSeeder.GetBuiltInElementId(dbContext, cancellationToken);
foreach (int elementId in maybeElementId)
{
// Add rather than assign: a future create path that carries graphics ids would otherwise
// be silently discarded here.
channel.ChannelGraphicsElements ??= [];
channel.ChannelGraphicsElements.Add(new ChannelGraphicsElement { GraphicsElementId = elementId });
}
}
}
@@ -85,7 +85,6 @@ public class CreateChannelFromLineupHandler(
await using var transaction = await dbContext.Database.BeginTransactionAsync(cancellationToken);
try
{
await ChannelGraphicsDefaults.Attach(dbContext, prepared.Channel, cancellationToken);
dbContext.Channels.Add(prepared.Channel);
if (prepared.Playlist is not null)
{
@@ -7,7 +7,6 @@ using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Infrastructure.Extensions;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Channels.ChannelValidations;
@@ -36,8 +35,7 @@ public class CreateChannelHandler(
Right: async logoPath =>
{
ApplyResolvedLogo(request, channel, logoPath);
return Right<BaseError, CreateChannelResult>(
await PersistChannel(dbContext, channel, cancellationToken));
return Right<BaseError, CreateChannelResult>(await PersistChannel(dbContext, channel));
},
Left: e => Task.FromResult(Left<BaseError, CreateChannelResult>(e)));
},
@@ -77,12 +75,8 @@ public class CreateChannelHandler(
}
}
private async Task<CreateChannelResult> PersistChannel(
TvContext dbContext,
Channel channel,
CancellationToken cancellationToken)
private async Task<CreateChannelResult> PersistChannel(TvContext dbContext, Channel channel)
{
await ChannelGraphicsDefaults.Attach(dbContext, channel, cancellationToken);
await dbContext.Channels.AddAsync(channel);
await dbContext.SaveChangesAsync();
searchTargets.SearchTargetsChanged();
@@ -595,13 +595,6 @@ public class RefreshChannelDataHandler : IRequestHandler<RefreshChannelData>
metadata.Genres ??= [];
metadata.Studios ??= [];
// Artists/AlbumArtists are NULLABLE primitive collections, so they are guarded at the read site
// rather than assigned back onto `metadata` like the navigations above (ersatztv#701/#691): they
// are scalar JSON-array columns, so `??= []` on a tracked entity would persist `[]` over NULL.
// The shipped `_song.sbntxt` only does `array.join`, but a user template is free to do anything.
List<string> songArtists = Optional(metadata.Artists).Flatten().ToList();
List<string> songAlbumArtists = Optional(metadata.AlbumArtists).Flatten().ToList();
string artworkPath = GetPrioritizedArtworkPath(metadata);
var data = new
@@ -614,8 +607,8 @@ public class RefreshChannelDataHandler : IRequestHandler<RefreshChannelData>
HasCustomTitle = hasCustomTitle,
displayItem.CustomTitle,
SongTitle = subtitle,
SongArtists = songArtists,
SongAlbumArtists = songAlbumArtists,
SongArtists = metadata.Artists,
SongAlbumArtists = metadata.AlbumArtists,
SongHasYear = metadata.Year.HasValue,
SongYear = metadata.Year,
SongGenres = metadata.Genres.Map(g => g.Name).OrderBy(n => n),
@@ -35,6 +35,4 @@ public record CreateFFmpegProfile(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup) : IRequest<Either<BaseError, CreateFFmpegProfileResult>>;
bool QsvPreferNativeDecoder) : IRequest<Either<BaseError, CreateFFmpegProfileResult>>;
@@ -50,12 +50,8 @@ public class CreateFFmpegProfileHandler :
private static Validation<BaseError, FFmpegProfile> Validate(
CreateFFmpegProfile request,
int resolutionId) =>
(ValidateName(request),
ValidateThreadCount(request),
FFmpegProfileBounds.ValidateQsvExtraHardwareFrames(request.QsvExtraHardwareFrames, stored: null),
FFmpegProfileBounds.ValidateReadRate(request.ReadRate),
FFmpegProfileBounds.ValidateReadRateCatchup(request.ReadRateCatchup, request.ReadRate))
.Apply((name, threadCount, _, _, _) =>
(ValidateName(request), ValidateThreadCount(request))
.Apply((name, threadCount) =>
{
var hwAccel = request.NormalizeVideo
? request.HardwareAcceleration
@@ -72,9 +68,11 @@ public class CreateFFmpegProfileHandler :
HardwareAcceleration = hwAccel,
VaapiDriver = request.VaapiDriver,
VaapiDevice = request.VaapiDevice,
// stored exactly as submitted: an out-of-range value was already rejected with a
// 422 naming the bound, so there is nothing left to silently rewrite (ersatztv#735)
QsvExtraHardwareFrames = request.QsvExtraHardwareFrames,
// store what the pipeline will actually use, never a pool size FFmpegState would
// floor away at render time (ersatztv#529)
QsvExtraHardwareFrames = request.QsvExtraHardwareFrames is { } frames
? Math.Max(frames, FFmpegState.MinimumQsvExtraHardwareFrames)
: null,
ResolutionId = resolutionId,
ScalingBehavior = request.ScalingBehavior,
@@ -113,9 +111,7 @@ public class CreateFFmpegProfileHandler :
NormalizeFramerate = request.NormalizeFramerate,
NormalizeColors = request.NormalizeColors,
DeinterlaceVideo = request.DeinterlaceVideo,
QsvPreferNativeDecoder = request.QsvPreferNativeDecoder,
ReadRate = request.ReadRate,
ReadRateCatchup = request.ReadRateCatchup
QsvPreferNativeDecoder = request.QsvPreferNativeDecoder
};
});
@@ -36,6 +36,4 @@ public record UpdateFFmpegProfile(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup) : IRequest<Either<BaseError, UpdateFFmpegProfileResult>>;
bool QsvPreferNativeDecoder) : IRequest<Either<BaseError, UpdateFFmpegProfileResult>>;
@@ -55,10 +55,11 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
p.VaapiDisplay = update.VaapiDisplay;
p.VaapiDriver = update.VaapiDriver;
p.VaapiDevice = update.VaapiDevice;
// stored exactly as submitted: an out-of-range NEW value was already rejected with a 422
// naming the bound. an unchanged value that predates that validation is written back as-is
// rather than rewritten, and FFmpegState floors it at render time (ersatztv#735)
p.QsvExtraHardwareFrames = update.QsvExtraHardwareFrames;
// store what the pipeline will actually use, so a profile doesn't keep displaying a pool
// size that FFmpegState floors away at render time (ersatztv#529)
p.QsvExtraHardwareFrames = update.QsvExtraHardwareFrames is { } frames
? Math.Max(frames, FFmpegState.MinimumQsvExtraHardwareFrames)
: null;
p.ResolutionId = update.ResolutionId;
p.ScalingBehavior = update.ScalingBehavior;
p.PadMode = update.PadMode;
@@ -107,8 +108,6 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
p.NormalizeColors = update.NormalizeColors;
p.DeinterlaceVideo = update.DeinterlaceVideo;
p.QsvPreferNativeDecoder = update.QsvPreferNativeDecoder;
p.ReadRate = update.ReadRate;
p.ReadRateCatchup = update.ReadRateCatchup;
// don't save invalid preset
ICollection<string> presets = FFmpegLibraryHelper.PresetsForFFmpegProfile(
@@ -132,14 +131,8 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
TvContext dbContext,
UpdateFFmpegProfile request,
FFmpegProfile profile) =>
(await ValidateName(dbContext, request),
ValidateThreadCount(request),
FFmpegProfileBounds.ValidateQsvExtraHardwareFrames(
request.QsvExtraHardwareFrames,
profile.QsvExtraHardwareFrames),
FFmpegProfileBounds.ValidateReadRate(request.ReadRate),
FFmpegProfileBounds.ValidateReadRateCatchup(request.ReadRateCatchup, request.ReadRate))
.Apply((_, _, _, _, _) => profile);
(await ValidateName(dbContext, request), ValidateThreadCount(request))
.Apply((_, _) => profile);
private static Task<Option<FFmpegProfile>> FFmpegProfileMustExist(
TvContext dbContext,
@@ -1,79 +0,0 @@
using ErsatzTV.Core;
using ErsatzTV.FFmpeg;
namespace ErsatzTV.Application.FFmpegProfiles;
/// <summary>
/// Write-path bounds for the consequential numeric FFmpeg profile fields.
/// A submitted value outside its documented range is REJECTED, naming the bound, rather than
/// accepted and silently rewritten to something the caller never sent (ersatztv#735). The
/// render-time clamps in <see cref="FFmpegState" /> stay as they are: they cover rows that
/// predate this validation or were written out of band, which is what keeps the fix
/// migration-free.
/// </summary>
internal static class FFmpegProfileBounds
{
internal static Validation<BaseError, Unit> ValidateQsvExtraHardwareFrames(int? requested, int? stored)
{
// a row stored before this validation existed may hold anything, and the SPA sends the whole
// profile back on every edit — so rejecting an UNCHANGED legacy value would make an old
// profile uneditable over a field the operator never touched (and cannot even see unless
// hardware acceleration is QSV). only a NEWLY submitted out-of-range value is rejected;
// FFmpegState.QsvExtraHardwareFrames still floors the legacy one at render time
if (requested is null || requested == stored)
{
return Success<BaseError, Unit>(Unit.Default);
}
return requested < FFmpegState.MinimumQsvExtraHardwareFrames
? BaseError.New(
$"QSV extra hardware frames must be at least {FFmpegState.MinimumQsvExtraHardwareFrames}; " +
$"{requested} leaves the QSV upload pool with too little headroom and the transcode writes nothing at all")
: Success<BaseError, Unit>(Unit.Default);
}
internal static Validation<BaseError, Unit> ValidateReadRate(double? requested)
{
if (requested is null)
{
return Success<BaseError, Unit>(Unit.Default);
}
return requested is < FFmpegState.MinimumReadRate or > FFmpegState.MaximumReadRate
? BaseError.New(
$"Read rate must be between {Format(FFmpegState.MinimumReadRate)} and {Format(FFmpegState.MaximumReadRate)}; " +
"below realtime the channel stalls, and above this the input is no longer meaningfully paced")
: Success<BaseError, Unit>(Unit.Default);
}
internal static Validation<BaseError, Unit> ValidateReadRateCatchup(double? requested, double? requestedReadRate)
{
if (requested is null)
{
return Success<BaseError, Unit>(Unit.Default);
}
if (requested is < FFmpegState.MinimumReadRateCatchup or > FFmpegState.MaximumReadRateCatchup)
{
return BaseError.New(
$"Read rate catchup must be between {Format(FFmpegState.MinimumReadRateCatchup)} and " +
$"{Format(FFmpegState.MaximumReadRateCatchup)}");
}
// catchup is the rate a LAGGING input may read at until it is level again, so a value at or
// below the base rate cannot let it recover: EQUAL is rejected too, because a catchup with
// zero headroom is functionally no catchup while still reading as configured. compared
// against the transcode default rather than the stream-copy one because that is the higher
// of the two: a value that clears it clears both, without this check having to know the
// profile's video format
double effectiveReadRate = requestedReadRate ?? FFmpegState.DefaultReadRate;
return requested <= effectiveReadRate
? BaseError.New(
$"Read rate catchup ({Format(requested.Value)}) must be greater than the read rate " +
$"({Format(effectiveReadRate)}); a lagging input cannot catch up at a rate it is already paced at")
: Success<BaseError, Unit>(Unit.Default);
}
private static string Format(double value) =>
value.ToString("0.0####", System.Globalization.CultureInfo.InvariantCulture);
}
@@ -36,6 +36,4 @@ public record FFmpegProfileViewModel(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup);
bool QsvPreferNativeDecoder);
@@ -38,9 +38,7 @@ internal static class Mapper
profile.NormalizeFramerate,
profile.NormalizeColors,
profile.DeinterlaceVideo == true,
profile.QsvPreferNativeDecoder != false,
profile.ReadRate,
profile.ReadRateCatchup);
profile.QsvPreferNativeDecoder != false);
internal static FFmpegProfileResponseModel ProjectToResponseModel(FFmpegProfile ffmpegProfile) =>
new(
@@ -84,7 +82,5 @@ internal static class Mapper
ffmpegProfile.NormalizeFramerate,
ffmpegProfile.NormalizeColors,
ffmpegProfile.DeinterlaceVideo == true,
ffmpegProfile.QsvPreferNativeDecoder != false,
ffmpegProfile.ReadRate,
ffmpegProfile.ReadRateCatchup);
ffmpegProfile.QsvPreferNativeDecoder != false);
}
@@ -1,5 +1,4 @@
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Filler.Mapper;
@@ -13,13 +12,9 @@ public class GetPagedFillerPresetsHandler(IDbContextFactory<TvContext> dbContext
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
// no filter today, but count and page are still derived from ONE query so that adding one
// cannot leave the count behind (api.paged-count-matches-page-query)
IQueryable<FillerPreset> query = dbContext.FillerPresets.AsNoTracking();
int count = await query.CountAsync(cancellationToken);
List<FillerPresetViewModel> page = await query
int count = await dbContext.FillerPresets.CountAsync(cancellationToken);
List<FillerPresetViewModel> page = await dbContext.FillerPresets
.AsNoTracking()
.OrderBy(f => f.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
+26 -34
View File
@@ -37,43 +37,23 @@ internal static class Mapper
collection.Collection is not null ? ProjectToViewModel(collection.Collection) : null,
collection.MultiCollection is not null ? ProjectToViewModel(collection.MultiCollection) : null,
collection.SmartCollection is not null ? ProjectToViewModel(collection.SmartCollection) : null,
ProjectMediaItemToViewModel(collection.MediaItem),
collection.MediaItem switch
{
Show show => MediaItems.Mapper.ProjectToViewModel(show),
Season season => MediaItems.Mapper.ProjectToViewModel(season),
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
Song song => MediaItems.Mapper.ProjectToViewModel(song),
Image image => MediaItems.Mapper.ProjectToViewModel(image),
_ => null
},
collection.FirstRunPlaybackOrder,
collection.RerunPlaybackOrder,
collection.Version);
/// <summary>
/// Flattens the <see cref="MediaItem" /> half of a selection tagged union to a named view model.
/// Shared by <see cref="RerunCollection" /> and <see cref="PlaylistItem" />, which select from an
/// identical set of media types; one copy is what stops the two drifting apart again (issue #671
/// — the same rationale as <c>ProgramScheduleItemQueryExtensions.IncludeScheduleItemDetails</c>
/// on the query side).
/// A null <paramref name="mediaItem" /> is the legitimate "this selection is not a media item"
/// case (the selection is a Collection/MultiCollection/SmartCollection instead) and maps to null.
/// An unrecognized non-null subtype keeps its id and takes a deliberately conspicuous name rather
/// than falling through to null: the id is what the editor round-trips, so returning null there
/// silently clears the user's stored selection — while throwing would fail an entire paged GET
/// over one unreadable row.
/// </summary>
private static MediaItems.NamedMediaItemViewModel ProjectMediaItemToViewModel(MediaItem mediaItem) =>
mediaItem switch
{
null => null,
Show show => MediaItems.Mapper.ProjectToViewModel(show),
Season season => MediaItems.Mapper.ProjectToViewModel(season),
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
Song song => MediaItems.Mapper.ProjectToViewModel(song),
Image image => MediaItems.Mapper.ProjectToViewModel(image),
RemoteStream remoteStream => MediaItems.Mapper.ProjectToNamedViewModel(remoteStream),
_ => new MediaItems.NamedMediaItemViewModel(
mediaItem.Id,
$"[unsupported media type: {mediaItem.GetType().Name}]")
};
internal static TraktListViewModel ProjectToViewModel(TraktList traktList) =>
new(
traktList.Id,
@@ -128,7 +108,19 @@ internal static class Mapper
playlistItem.SmartCollection is not null
? ProjectToViewModel(playlistItem.SmartCollection)
: null,
ProjectMediaItemToViewModel(playlistItem.MediaItem),
playlistItem.MediaItem switch
{
Show show => MediaItems.Mapper.ProjectToViewModel(show),
Season season => MediaItems.Mapper.ProjectToViewModel(season),
Artist artist => MediaItems.Mapper.ProjectToViewModel(artist),
Movie movie => MediaItems.Mapper.ProjectToViewModel(movie),
Episode episode => MediaItems.Mapper.ProjectToViewModel(episode),
MusicVideo musicVideo => MediaItems.Mapper.ProjectToViewModel(musicVideo),
OtherVideo otherVideo => MediaItems.Mapper.ProjectToViewModel(otherVideo),
Song song => MediaItems.Mapper.ProjectToViewModel(song),
Image image => MediaItems.Mapper.ProjectToViewModel(image),
_ => null
},
playlistItem.PlaybackOrder,
playlistItem.Count,
playlistItem.PlayAll,
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.MediaCollections.Mapper;
@@ -13,6 +13,8 @@ public class GetPagedCollectionsHandler(IDbContextFactory<TvContext> dbContextFa
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.Collections.CountAsync(cancellationToken);
IQueryable<Collection> query = dbContext.Collections.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -20,9 +22,6 @@ public class GetPagedCollectionsHandler(IDbContextFactory<TvContext> dbContextFa
query = query.Where(c => EF.Functions.Like(c.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<MediaCollectionViewModel> page = await query
.OrderBy(c => c.Name)
.Skip(request.PageNum * request.PageSize)
@@ -13,6 +13,9 @@ public class GetPagedMultiCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.MultiCollections
.CountAsync(mc => mc.OwnedByChannelId == null, cancellationToken);
IQueryable<MultiCollection> query = dbContext.MultiCollections
.AsNoTracking()
.Where(mc => mc.OwnedByChannelId == null);
@@ -22,9 +25,6 @@ public class GetPagedMultiCollectionsHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(mc => EF.Functions.Like(mc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<MultiCollectionViewModel> page = await query
.OrderBy(mc => mc.Name)
.Skip(request.PageNum * request.PageSize)
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.MediaCollections.Mapper;
@@ -13,6 +13,8 @@ public class GetPagedRerunCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.RerunCollections.CountAsync(cancellationToken);
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -20,14 +22,7 @@ public class GetPagedRerunCollectionsHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(rc => EF.Functions.Like(rc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758).
// The includes belong to the page chain only — a COUNT does not materialize the graph.
int count = await query.CountAsync(cancellationToken);
// EF applies the includes to the paged subquery, so the selection graph is loaded for at most
// PageSize rows — the per-request cost is bounded by the page, not by the table (issue #671).
List<RerunCollectionViewModel> page = await query
.IncludeSelectionDetails()
.OrderBy(rc => rc.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -13,6 +13,9 @@ public class GetPagedSmartCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.SmartCollections
.CountAsync(sc => sc.OwnedByChannelId == null, cancellationToken);
IQueryable<SmartCollection> query = dbContext.SmartCollections
.AsNoTracking()
.Where(sc => sc.OwnedByChannelId == null);
@@ -22,9 +25,6 @@ public class GetPagedSmartCollectionsHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(sc => EF.Functions.Like(sc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<SmartCollectionViewModel> page = await query
.OrderBy(s => s.Name)
.Skip(request.PageNum * request.PageSize)
@@ -1,5 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.MediaCollections.Mapper;
@@ -13,13 +12,9 @@ public class GetPagedTraktListsHandler(IDbContextFactory<TvContext> dbContextFac
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
// no filter today, but count and page are still derived from ONE query so that adding one
// cannot leave the count behind (api.paged-count-matches-page-query)
IQueryable<TraktList> query = dbContext.TraktLists.AsNoTracking();
int count = await query.CountAsync(cancellationToken);
List<TraktListViewModel> page = await query
int count = await dbContext.TraktLists.CountAsync(cancellationToken);
List<TraktListViewModel> page = await dbContext.TraktLists
.AsNoTracking()
.OrderBy(l => l.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -55,10 +55,6 @@ public class GetPlaylistItemsHandler(IDbContextFactory<TvContext> dbContextFacto
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Image).ImageMetadata)
.ThenInclude(mm => mm.Artwork)
// RemoteStream is projected by the shared ProjectMediaItemToViewModel switch as of #671;
// without its metadata the name would degrade to "???" here while every sibling type resolves.
.Include(i => i.MediaItem)
.ThenInclude(i => (i as RemoteStream).RemoteStreamMetadata)
.ToListAsync(cancellationToken);
return allItems.Map(Mapper.ProjectToViewModel).ToList();
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
using Microsoft.EntityFrameworkCore;
@@ -16,7 +16,20 @@ public class GetRerunCollectionByIdHandler(IDbContextFactory<TvContext> dbContex
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
return await dbContext.RerunCollections
.AsNoTracking()
.IncludeSelectionDetails()
.Include(c => c.Collection)
.Include(c => c.MultiCollection)
.Include(c => c.SmartCollection)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Movie).MovieMetadata)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Season).SeasonMetadata)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Season).Show)
.ThenInclude(s => s.ShowMetadata)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Show).ShowMetadata)
.Include(i => i.MediaItem)
.ThenInclude(i => (i as Artist).ArtistMetadata)
.SelectOneAsync(c => c.Id, c => c.Id == request.Id, cancellationToken)
.MapT(ProjectToViewModel);
}
@@ -1,57 +0,0 @@
using ErsatzTV.Core.Domain;
using Microsoft.EntityFrameworkCore;
namespace ErsatzTV.Application.MediaCollections;
internal static class RerunCollectionQueryExtensions
{
/// <summary>
/// The single source of truth for the navigation graph a <see cref="RerunCollection" /> needs before it
/// can be projected via <see cref="Mapper.ProjectToViewModel(RerunCollection)" />. Both the paged-list
/// and by-id handlers reload through this chain so the two cannot drift apart again (see #671 — the list
/// handler had no includes at all, so every row projected a null selection, while the by-id handler
/// covered only Movie/Season/Show/Artist and so returned a null selection for Song/OtherVideo/Image and
/// a 500 for Episode/MusicVideo).
/// Because the id and the display name are both read off these navigations, an un-included type does not
/// merely lose its label — it loses the selected id too, which is what silently cleared a stored
/// selection in the editor.
/// Deliberately narrower than the analogous playlist-item chain in <c>GetPlaylistItemsHandler</c>: the
/// rerun projection reads only each selection's id and title, never its artwork, so the
/// <c>.ThenInclude(… =&gt; …Artwork)</c> legs are omitted rather than paid for on every page.
/// </summary>
public static IQueryable<RerunCollection> IncludeSelectionDetails(this IQueryable<RerunCollection> query) =>
query
.Include(c => c.Collection)
.Include(c => c.MultiCollection)
.Include(c => c.SmartCollection)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Movie).MovieMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Show).ShowMetadata)
// No (i as Season).SeasonMetadata leg on purpose: ProjectToViewModel(Season) builds its name
// from Show.ShowMetadata and the scalar SeasonNumber, and never reads SeasonMetadata.
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Season).Show)
.ThenInclude(s => s.ShowMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Artist).ArtistMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Episode).EpisodeMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Episode).Season)
.ThenInclude(s => s.Show)
.ThenInclude(s => s.ShowMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as MusicVideo).MusicVideoMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as MusicVideo).Artist)
.ThenInclude(a => a.ArtistMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as OtherVideo).OtherVideoMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Song).SongMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as Image).ImageMetadata)
.Include(c => c.MediaItem)
.ThenInclude(i => (i as RemoteStream).RemoteStreamMetadata);
}
+16 -49
View File
@@ -1,24 +1,18 @@
using System.Globalization;
using System.Globalization;
using ErsatzTV.Core.Domain;
namespace ErsatzTV.Application.MediaItems;
internal static class Mapper
{
// Every metadata navigation below is read through Optional(...).Flatten() rather than a bare
// dereference: these projections are reached from several handlers whose Include chains differ,
// and an un-included navigation must degrade to the "???" placeholder instead of throwing an
// NRE that surfaces as a 500 on a GET (issue #671).
internal static NamedMediaItemViewModel ProjectToViewModel(Show show) =>
new(
show.Id,
Optional(show.ShowMetadata).Flatten().HeadOrNone().Map(sm => $"{sm?.Title} ({sm?.Year})").IfNone("???"));
new(show.Id, show.ShowMetadata.HeadOrNone().Map(sm => $"{sm?.Title} ({sm?.Year})").IfNone("???"));
internal static NamedMediaItemViewModel ProjectToViewModel(Season season) =>
new(season.Id, $"{ShowTitle(season)} - {SeasonDescription(season)}");
internal static NamedMediaItemViewModel ProjectToViewModel(Artist artist) =>
new(artist.Id, Optional(artist.ArtistMetadata).Flatten().HeadOrNone().Match(am => am.Title, () => "???"));
new(artist.Id, artist.ArtistMetadata.HeadOrNone().Match(am => am.Title, () => "???"));
internal static NamedMediaItemViewModel ProjectToViewModel(Movie movie) =>
new(movie.Id, MovieTitle(movie));
@@ -30,37 +24,23 @@ internal static class Mapper
new(musicVideo.Id, MusicVideoTitle(musicVideo));
internal static NamedMediaItemViewModel ProjectToViewModel(OtherVideo otherVideo) =>
new(
otherVideo.Id,
Optional(otherVideo.OtherVideoMetadata).Flatten().HeadOrNone().Match(ov => ov.Title, () => "???"));
new(otherVideo.Id, otherVideo.OtherVideoMetadata.HeadOrNone().Match(ov => ov.Title, () => "???"));
internal static NamedMediaItemViewModel ProjectToViewModel(Song song) =>
new(song.Id, SongTitle(song));
internal static NamedMediaItemViewModel ProjectToViewModel(Image image) =>
new(image.Id, Optional(image.ImageMetadata).Flatten().HeadOrNone().Match(i => i.Title, () => "???"));
new(image.Id, image.ImageMetadata.HeadOrNone().Match(i => i.Title, () => "???"));
internal static RemoteStreamViewModel ProjectToViewModel(RemoteStream remoteStream) =>
new(remoteStream.Id, remoteStream.Url, remoteStream.Script);
/// <summary>
/// The named projection for a <see cref="RemoteStream" />. This cannot be an overload of
/// <see cref="ProjectToViewModel(RemoteStream)" /> — that one already exists and returns a
/// <see cref="RemoteStreamViewModel" />, and C# will not overload on return type alone. Its
/// absence is why every selection-flattening switch dropped <c>RemoteStream</c> through a
/// <c>_ =&gt; null</c> arm (issue #671).
/// </summary>
internal static NamedMediaItemViewModel ProjectToNamedViewModel(RemoteStream remoteStream) =>
new(
remoteStream.Id,
Optional(remoteStream.RemoteStreamMetadata).Flatten().HeadOrNone().Match(rsm => rsm.Title, () => "???"));
private static string MovieTitle(Movie movie)
{
var title = "???";
var year = "???";
foreach (MovieMetadata movieMetadata in Optional(movie.MovieMetadata).Flatten().HeadOrNone())
foreach (MovieMetadata movieMetadata in movie.MovieMetadata.HeadOrNone())
{
title = movieMetadata.Title;
foreach (int y in Optional(movieMetadata.Year))
@@ -77,10 +57,7 @@ internal static class Mapper
var title = "???";
var year = "???";
// Season.Show and Show.ShowMetadata are only populated when the caller eager-loaded them.
// An un-included navigation must degrade to the "???" placeholder these helpers already
// produce for missing metadata — never an NRE, which surfaced as a 500 (issue #671).
foreach (ShowMetadata show in Optional(season.Show?.ShowMetadata).Flatten().HeadOrNone())
foreach (ShowMetadata show in season.Show.ShowMetadata.HeadOrNone())
{
title = show.Title;
foreach (int y in Optional(show.Year))
@@ -97,10 +74,10 @@ internal static class Mapper
private static string EpisodeTitle(Episode e)
{
string showTitle = Optional(e.Season?.Show?.ShowMetadata).Flatten().HeadOrNone()
string showTitle = e.Season.Show.ShowMetadata.HeadOrNone()
.Map(sm => $"{sm.Title} - ").IfNone(string.Empty);
var episodeNumbers = Optional(e.EpisodeMetadata).Flatten().Map(em => em.EpisodeNumber).ToList();
var episodeTitles = Optional(e.EpisodeMetadata).Flatten().Map(em => em.Title).ToList();
var episodeNumbers = e.EpisodeMetadata.Map(em => em.EpisodeNumber).ToList();
var episodeTitles = e.EpisodeMetadata.Map(em => em.Title).ToList();
if (episodeNumbers.Count == 0 || episodeTitles.Count == 0)
{
return "[unknown episode]";
@@ -109,34 +86,24 @@ internal static class Mapper
var numbersString = $"e{string.Join('e', episodeNumbers.Map(n => $"{n:00}"))}";
var titlesString = $"{string.Join('/', episodeTitles)}";
// "s00" conventionally means Specials, so an unloaded Season must not borrow it — that would
// fabricate plausible-looking real data. Render the season as explicitly unknown instead.
string seasonNumber = e.Season is null ? "??" : $"{e.Season.SeasonNumber:00}";
return $"{showTitle}s{seasonNumber}{numbersString} - {titlesString}";
return $"{showTitle}s{e.Season.SeasonNumber:00}{numbersString} - {titlesString}";
}
private static string MusicVideoTitle(MusicVideo mv)
{
string artistName = Optional(mv.Artist?.ArtistMetadata).Flatten().HeadOrNone()
string artistName = mv.Artist.ArtistMetadata.HeadOrNone()
.Map(am => $"{am.Title} - ").IfNone(string.Empty);
return Optional(mv.MusicVideoMetadata).Flatten().HeadOrNone()
return mv.MusicVideoMetadata.HeadOrNone()
.Map(mvm => $"{artistName}{mvm.Title}")
.IfNone("[unknown music video]");
}
private static string SongTitle(Song s)
{
// Artists is a NULLABLE primitive collection, not a navigation: a song whose tags failed to read
// is persisted by FallbackMetadataProvider with Artists never assigned, and string.Join throws
// ArgumentNullException on a null sequence. Filtering the empty case too avoids prefixing an
// artist-less song with a bare " - ".
string songArtist = Optional(s.SongMetadata).Flatten().HeadOrNone()
.Map(sm => Optional(sm.Artists).Flatten().ToList())
.Filter(artists => artists.Count > 0)
.Map(artists => $"{string.Join(", ", artists)} - ")
string songArtist = s.SongMetadata.HeadOrNone()
.Map(sm => $"{string.Join(", ", sm.Artists)} - ")
.IfNone(string.Empty);
return Optional(s.SongMetadata).Flatten().HeadOrNone()
return s.SongMetadata.HeadOrNone()
.Map(sm => $"{songArtist}{sm.Title ?? string.Empty}")
.IfNone("[unknown song]");
}
@@ -1,5 +1,4 @@
using System.Threading.Channels;
using ErsatzTV.Application.Scheduling;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Scheduling;
@@ -87,29 +86,6 @@ public class ReplacePlayoutAlternateScheduleItemsHandler(
var incoming = request.Items.Except([highest]).ToList();
// Reject an EXPLICITLY empty recurrence set before any mutation (#880). The checked set is
// `incoming` -- the exact list whose DaysOfWeek/DaysOfMonth/MonthsOfYear the loops below
// write -- so the check and its subject cannot drift apart. That EXCLUDES the highest-Index
// catch-all by construction: its recurrence is discarded along with its date range (only its
// ProgramScheduleId is read, further down), so an empty set there cannot make anything "never
// apply" and rejecting it would state a reason that is false for that item.
foreach (ReplacePlayoutAlternateSchedule item in incoming)
{
ProgramScheduleAlternate stored = existing.FirstOrDefault(e => e.Id == item.Id);
Option<BaseError> recurrenceError = RecurrenceSetBounds.Validate(
item.DaysOfWeek,
item.DaysOfMonth,
item.MonthsOfYear,
stored?.DaysOfWeek,
stored?.DaysOfMonth,
stored?.MonthsOfYear);
foreach (BaseError error in recurrenceError)
{
return error;
}
}
var toAdd = incoming.Filter(x => existing.All(e => e.Id != x.Id)).ToList();
var toRemove = existing.Filter(e => incoming.All(m => m.Id != e.Id)).ToList();
var toUpdate = incoming.Except(toAdd).ToList();
+7 -24
View File
@@ -1,6 +1,5 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
namespace ErsatzTV.Application.Playouts;
@@ -41,15 +40,9 @@ internal static class Mapper
programScheduleAlternate.Id,
programScheduleAlternate.Index,
programScheduleAlternate.ProgramScheduleId,
// ersatztv#823: these three are NULLABLE columns and a legacy row can hold NULL. Substitute the
// SAME unrestricted defaults AlternateScheduleSelector.GetScheduleForDate reads, so the DTO the
// SPA renders agrees with what actually gets scheduled -- web/src/screens/playoutTemplateCalendar.ts
// `appliesToDate` is an exact port of that method, and it would otherwise both mispreview and
// throw (`[...template.daysOfMonth]` on a null is a TypeError). Never assigned back onto the
// entity (`media.nullable-primitive-collection-mutation`).
programScheduleAlternate.DaysOfWeek ?? AlternateScheduleSelector.AllDaysOfWeek(),
programScheduleAlternate.DaysOfMonth ?? AlternateScheduleSelector.AllDaysOfMonth(),
programScheduleAlternate.MonthsOfYear ?? AlternateScheduleSelector.AllMonthsOfYear(),
programScheduleAlternate.DaysOfWeek,
programScheduleAlternate.DaysOfMonth,
programScheduleAlternate.MonthsOfYear,
programScheduleAlternate.LimitToDateRange,
programScheduleAlternate.StartMonth,
programScheduleAlternate.StartDay,
@@ -109,24 +102,14 @@ internal static class Mapper
: $"{s} ({chapterTitle})")
.IfNone("[unknown video]");
case Song s:
// SongMetadata.Artists is a NULLABLE primitive collection (FallbackMetadataProvider never
// assigns it for a song whose tags failed to read) and string.Join throws
// ArgumentNullException on a null sequence. SongMetadata IS eager-loaded on this path, so
// this was a LIVE 500 on the playout guide, not a latent one (issue #671).
string songArtist = Optional(s.SongMetadata).Flatten().HeadOrNone()
.Map(sm => Optional(sm.Artists).Flatten().ToList())
.Filter(artists => artists.Count > 0)
.Map(artists => $"{string.Join(", ", artists)} - ")
string songArtist = s.SongMetadata.HeadOrNone()
.Map(sm => $"{string.Join(", ", sm.Artists)} - ")
.IfNone(string.Empty);
return Optional(s.SongMetadata).Flatten().HeadOrNone()
return s.SongMetadata.HeadOrNone()
.Map(sm => $"{songArtist}{sm.Title ?? string.Empty}")
.Map(t => string.IsNullOrWhiteSpace(chapterTitle)
// interpolate the composed title `t`, NOT the `case Song s` entity — Song has no
// ToString() override, so `{s}` rendered a chaptered song as the literal type name
// "ErsatzTV.Core.Domain.Song (Chapter 3)". The MusicVideo/OtherVideo arms above are
// correct only because they happen to name their lambda parameter `s`.
? t
: $"{t} ({chapterTitle})")
: $"{s} ({chapterTitle})")
.IfNone("[unknown song]");
case Image i:
return i.ImageMetadata.HeadOrNone().Map(im => im.Title ?? string.Empty).IfNone("[unknown image]");
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Playouts.Mapper;
@@ -13,8 +13,13 @@ public class GetPagedPlayoutsHandler(IDbContextFactory<TvContext> dbContextFacto
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.Playouts.CountAsync(cancellationToken);
IQueryable<Playout> query = dbContext.Playouts
.AsNoTracking()
.Include(p => p.Channel)
.Include(p => p.ProgramSchedule)
.Include(p => p.BuildStatus)
.Filter(p => p.Channel != null);
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -22,15 +27,7 @@ public class GetPagedPlayoutsHandler(IDbContextFactory<TvContext> dbContextFacto
query = query.Where(p => EF.Functions.Like(p.Channel.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758).
// This is also what makes the `Channel != null` filter count, which the old unfiltered
// CountAsync over the whole DbSet did not.
int count = await query.CountAsync(cancellationToken);
List<PlayoutNameViewModel> page = await query
.Include(p => p.Channel)
.Include(p => p.ProgramSchedule)
.Include(p => p.BuildStatus)
.OrderBy(p => p.Channel.SortNumber)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.ProgramSchedules.Mapper;
@@ -13,6 +13,8 @@ public class GetPagedProgramSchedulesHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.ProgramSchedules.CountAsync(cancellationToken);
IQueryable<ProgramSchedule> query = dbContext.ProgramSchedules.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -20,9 +22,6 @@ public class GetPagedProgramSchedulesHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(ps => EF.Functions.Like(ps.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<ProgramScheduleViewModel> page = await query
.OrderBy(ps => ps.Name)
.Skip(request.PageNum * request.PageSize)
@@ -44,25 +44,6 @@ public class ReplacePlayoutTemplateItemsHandler(
List<ReplacePlayoutTemplate> incoming = request.Items;
// Same rule as the alternate-schedule path (#880), over ALL items: unlike that one, every
// template item's recurrence IS stored, so there is no catch-all to exclude here.
foreach (ReplacePlayoutTemplate item in incoming)
{
PlayoutTemplate stored = existing.FirstOrDefault(e => e.Id == item.Id);
Option<BaseError> recurrenceError = RecurrenceSetBounds.Validate(
item.DaysOfWeek,
item.DaysOfMonth,
item.MonthsOfYear,
stored?.DaysOfWeek,
stored?.DaysOfMonth,
stored?.MonthsOfYear);
if (recurrenceError.IsSome)
{
return recurrenceError;
}
}
var toAdd = incoming.Filter(x => existing.All(e => e.Id != x.Id)).ToList();
var toRemove = existing.Filter(e => incoming.All(m => m.Id != e.Id)).ToList();
var toUpdate = incoming.Except(toAdd).ToList();
+4 -11
View File
@@ -1,7 +1,6 @@
using ErsatzTV.Application.Tree;
using ErsatzTV.Application.Tree;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
namespace ErsatzTV.Application.Scheduling;
@@ -191,15 +190,9 @@ internal static class Mapper
ProjectToViewModel(playoutTemplate.Template),
ProjectToViewModel(playoutTemplate.DecoTemplate),
playoutTemplate.Index,
// ersatztv#823: these three are NULLABLE columns and a legacy row can hold NULL. Substitute the
// SAME unrestricted defaults AlternateScheduleSelector.GetScheduleForDate reads, so the DTO the
// SPA renders agrees with what actually gets scheduled -- web/src/screens/playoutTemplateCalendar.ts
// `appliesToDate` is an exact port of that method, and it would otherwise both mispreview and
// throw (`[...template.daysOfMonth]` on a null is a TypeError). Never assigned back onto the
// entity (`media.nullable-primitive-collection-mutation`).
playoutTemplate.DaysOfWeek ?? AlternateScheduleSelector.AllDaysOfWeek(),
playoutTemplate.DaysOfMonth ?? AlternateScheduleSelector.AllDaysOfMonth(),
playoutTemplate.MonthsOfYear ?? AlternateScheduleSelector.AllMonthsOfYear(),
playoutTemplate.DaysOfWeek,
playoutTemplate.DaysOfMonth,
playoutTemplate.MonthsOfYear,
playoutTemplate.LimitToDateRange,
playoutTemplate.StartMonth,
playoutTemplate.StartDay,
@@ -1,74 +0,0 @@
using ErsatzTV.Core;
namespace ErsatzTV.Application.Scheduling;
/// <summary>
/// Validates the three recurrence sets shared by <c>ProgramScheduleAlternate</c> and
/// <c>PlayoutTemplate</c> (ersatztv#880). One validator called from BOTH replace handlers, mirroring
/// <c>FFmpegProfileBounds</c> — the exemplar for `api.ffmpeg-profile-numeric-bounds`, whose shape this
/// follows deliberately.
/// </summary>
/// <remarks>
/// <para>
/// An EMPTY set is rejected because the three are read CONJUNCTIVELY by
/// <c>AlternateScheduleSelector.GetScheduleForDate</c> — a miss on any one continues to the next
/// item — so an empty one matches NO date and stores an item that can never apply. Rejecting
/// rather than substituting is the point: accept-then-rewrite would make an explicit `[]`
/// indistinguishable from an omitted field, which is the very collapse this issue removed.
/// </para>
/// <para>
/// An UNCHANGED empty set that the row ALREADY holds is let through. Both PUT paths are
/// whole-list replaces, so a hard rejection would make every OTHER item in the playout
/// uneditable over a row the operator never touched — the same reason
/// `api.ffmpeg-profile-numeric-bounds` rejects only a NEWLY submitted out-of-range value. A row
/// whose stored set is NULL is NOT exempt: null means unrestricted, so submitting `[]` for it is
/// a new emptying, not an unchanged legacy value.
/// </para>
/// <para>
/// This runs on the COMMAND, after the request records have normalized an ABSENT array to the
/// All*() sets, so an empty set reaching here is one a caller sent EXPLICITLY. That also means a
/// direct (non-HTTP) caller is held to the same rule rather than being able to write a dead row.
/// </para>
/// </remarks>
public static class RecurrenceSetBounds
{
public static Option<BaseError> Validate(
ICollection<DayOfWeek> daysOfWeek,
ICollection<int> daysOfMonth,
ICollection<int> monthsOfYear,
ICollection<DayOfWeek> storedDaysOfWeek,
ICollection<int> storedDaysOfMonth,
ICollection<int> storedMonthsOfYear)
{
if (IsNewlyEmpty(daysOfWeek, storedDaysOfWeek))
{
return Some(BaseError.New(Message("DaysOfWeek", "no day of the week")));
}
if (IsNewlyEmpty(daysOfMonth, storedDaysOfMonth))
{
return Some(BaseError.New(Message("DaysOfMonth", "no day of the month")));
}
if (IsNewlyEmpty(monthsOfYear, storedMonthsOfYear))
{
return Some(BaseError.New(Message("MonthsOfYear", "no month")));
}
return Option<BaseError>.None;
}
// "send null" rather than "omit the property": all three are listed in the schema's `required` array
// in v1.json (they are nullable, not optional), so a client generated from the published contract
// cannot omit them. Omitting also works at runtime -- Newtonsoft maps a missing property and an
// explicit null to the same thing -- but naming only that would tell a conforming client to send
// something its own schema forbids.
private static string Message(string field, string consequence) =>
$"[{field}] must not be empty; an empty set matches {consequence}, so the item would never apply. " +
"Send null to leave it unrestricted";
// A new item (no stored row) has `stored` null, so an empty set is newly empty and is rejected.
// Only a stored set that is ITSELF already empty exempts an empty submission.
private static bool IsNewlyEmpty<T>(ICollection<T> submitted, ICollection<T> stored) =>
submitted is { Count: 0 } && stored is not { Count: 0 };
}
@@ -1,6 +1,3 @@
using System.Text;
using System.Text.Json;
using Dapper;
using ErsatzTV.Core.Api.Search;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
@@ -14,62 +11,6 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
private const int DefaultLimit = 50;
private const int MaxLimit = 50;
/// <summary>
/// Rows read per round trip when walking the list-valued (JSON-array) columns on
/// <c>SongMetadata</c>, and the ceiling on rows read per request.
/// <para>
/// These count ACTUAL ROWS, and arriving at that took four tries — each earlier attempt bounded a
/// quantity that sounded like rows and was not. A fixed <c>LIMIT</c> budget bounded the RESULT, and
/// the pre-filter (allowed to over-match) starved it with rows that could not match. Keyset paging
/// with a <c>LIMIT</c> bounded CANDIDATES RETURNED — but a query matching nothing must evaluate
/// every eligible row before it can return an empty page, so rows inspected stayed unbounded. A
/// closed <c>Id</c> range bounded KEYSPACE WIDTH — but keyspace is not rows: delete 20,000
/// historical rows, put one song at <c>Id</c> 20001, and the walk burns its whole allowance on empty
/// ranges and inspects nothing.
/// </para>
/// <para>
/// What makes this one hold is that <b>the query has no RESIDUAL predicate</b> — nothing that can
/// discard a row the engine already produced. The only condition is the cursor
/// <c>Id &gt; @AfterId</c>, which is a seek on the <c>ORDER BY</c> key itself, not a filter. So the
/// page returns exactly <see cref="ListValuedBatchRows" /> rows whenever that many logical rows
/// remain, independent of how sparse the matches are or where the <c>Id</c> gaps fall.
/// </para>
/// <para>
/// <b>Be precise about what is bounded: LOGICAL ROWS RETURNED AND MATERIALIZED, and the number of
/// round trips. Not physical work, and not bytes.</b> Two things break the stronger reading, and an
/// earlier version of this comment asserted it anyway:
/// <list type="bullet">
/// <item>
/// MySQL purge lag. Deleted clustered-index records survive until purge runs, and a range
/// scan still traverses them, so returning 2,000 VISIBLE rows can touch far more index
/// records. Deletion history therefore still affects physical work — the very thing the
/// keyspace attempt was trying to make irrelevant.
/// </item>
/// <item>
/// Row width is unbounded. These columns are <c>TEXT</c>/<c>longtext</c>, which both SQLite
/// and InnoDB spill to overflow pages, so a row count implies neither a byte count nor a
/// page-read count.
/// </item>
/// </list>
/// The logical-row bound is still worth having — it is what makes the walk terminate and what caps
/// the number of rows and round trips — but do not restate it as bounded I/O, and do not restate it
/// as bounded MEMORY either: payload width is unrestricted and a single JSON array can hold
/// arbitrarily many strings, every one of which may enter the in-memory set.
/// </para>
/// <para>
/// The trade is real and deliberate: no server-side narrowing, so a query with few matches transfers
/// rows it will discard, up to <see cref="ListValuedMaxRowsRead" />. A query with enough matches
/// stops as soon as it has <c>limit</c> distinct ones, so the dense cases — including an empty
/// <c>q</c> — finish on the first page. See <c>api.search-field-values-sources</c> for the measured
/// cost and for why reintroducing a <c>LIKE</c> is not an option.
/// </para>
/// </summary>
internal const int ListValuedBatchRows = 2000;
/// <inheritdoc cref="ListValuedBatchRows" />
internal const int ListValuedMaxRowsRead = 20000;
public async Task<Option<SearchFieldValuesResponseModel>> Handle(
GetSearchFieldValues request,
CancellationToken cancellationToken)
@@ -83,22 +24,17 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
}
int limit = request.Limit <= 0 ? DefaultLimit : Math.Clamp(request.Limit, 1, MaxLimit);
string query = request.Query ?? string.Empty;
// Invariant, not current-culture: UseRequestLocalization honours Accept-Language, so a caller can select
// tr-TR and turn `q=I` into `ı` — which then matches nothing a Turkish-dotless-i-free library contains.
// This feeds the EF-translated filter, which has no StringComparison overload EF can translate.
string qLower = query.ToLowerInvariant();
string qLower = (request.Query ?? string.Empty).ToLower();
// in-memory special cases (no DB query needed)
switch (request.Name)
{
case "state":
return new SearchFieldValuesResponseModel(
FilterSortTake(Enum.GetNames<MediaItemState>(), query, limit));
FilterSortTake(Enum.GetNames<MediaItemState>(), qLower, limit));
case "video_dynamic_range":
return new SearchFieldValuesResponseModel(
FilterSortTake(["hdr", "sdr"], query, limit));
FilterSortTake(["hdr", "sdr"], qLower, limit));
}
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
@@ -106,75 +42,34 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
if (request.Name == "content_rating")
{
return new SearchFieldValuesResponseModel(
await GetContentRatingValues(dbContext, query, limit, cancellationToken));
await GetContentRatingValues(dbContext, qLower, limit, cancellationToken));
}
IQueryable<string> source = GetSource(dbContext, request.Name);
string listColumn = GetSongListValuedColumn(request.Name);
if (source is null && listColumn is null)
if (source is null)
{
return Option<SearchFieldValuesResponseModel>.None;
}
var values = new List<string>();
List<string> values = await source
.Where(v => v != null && v.ToLower().StartsWith(qLower))
.Distinct()
.OrderBy(v => v)
.Take(limit)
.ToListAsync(cancellationToken);
if (source is not null)
{
values.AddRange(
await source
.Where(v => v != null && v.ToLower().StartsWith(qLower))
.Distinct()
.OrderBy(v => v)
.Take(limit)
.ToListAsync(cancellationToken));
}
// ersatztv#668. The query above prefix-matches through SQL LOWER(), and SQLite's LOWER() folds ASCII
// ONLY -- lower('Édith') is 'Édith' unchanged -- so it cannot reach a stored value whose prefix
// carries an uppercase non-ASCII character, from ANY query. It UNDER-matches, and an under-match is
// unrecoverable downstream: no later stage can reintroduce a row SQL never returned. So for the only
// queries that can be affected (those containing a non-ASCII character) run a second, Unicode-correct
// pass and merge it in. This is ADDITIVE on purpose -- the SQL pass above still contributes, so a
// value already reachable today cannot stop being reachable.
//
// MySQL needs none of this: its LOWER() is Unicode-aware, so LOWER('Édith') really is 'édith' and the
// existing predicate reaches the row unaided. Measured on 8.4 -- and note the executed path does NOT
// over-match, even though the column collation (utf8mb4_0900_ai_ci) is accent-insensitive: the driver
// binds the LIKE pattern with a BINARY collation, so the comparison is accent-sensitive in practice.
// A hand-typed probe using a LITERAL pattern DOES over-match; that is a different query from the one
// this code runs, and mistaking the two is how an earlier revision of the decision record got it wrong.
if (source is not null && ContainsNonAscii(query) && IsSqlite(dbContext))
{
values.AddRange(
await GetUnicodeFoldedValues(dbContext, request.Name, query, limit, cancellationToken));
}
if (listColumn is not null)
{
values.AddRange(await GetSongListValuedValues(dbContext, listColumn, query, limit, cancellationToken));
}
// ORDERING IS BEST-EFFORT, NOT EXACT. Each source truncates using its own ordering — the EF source by the
// database collation (SQLite's NOCASE/BINARY is ASCII-only), the list source by primary key — and neither
// is the ordinal ordering applied here. So when a source actually truncates, a value it dropped may have
// outranked one that survived: with "Zulu" and "apple" and limit=1 the database keeps "apple" (its
// ordering is case-insensitive) while ordinal ranks "Zulu" first, so the merge never sees "Zulu".
// Below the truncation points (the normal typeahead case) the result is exact.
return new SearchFieldValuesResponseModel(FilterSortTake(values.Distinct(StringComparer.Ordinal), query, limit));
return new SearchFieldValuesResponseModel(values);
}
internal static IQueryable<string> GetSource(TvContext dbContext, string name) => name switch
private static IQueryable<string> GetSource(TvContext dbContext, string name) => name switch
{
"genre" or "show_genre" => dbContext.Set<Genre>().Select(g => g.Name),
"studio" => dbContext.Set<Studio>().Select(s => s.Name),
"director" => dbContext.Set<Director>().Select(d => d.Name),
"writer" => dbContext.Set<Writer>().Select(w => w.Name),
"actor" => dbContext.Actors.Select(a => a.Name),
// Mirrors what LuceneSearchIndex writes to the `artist` field: the music video's linked artist entity
// (ArtistMetadata.Title) plus its free-text credits (MusicVideoArtist rows). The third contributor —
// SongMetadata.Artists — is a JSON-array column and is handled by GetSongListValuedValues instead.
"artist" => dbContext.ArtistMetadata.Select(m => m.Title)
.Concat(dbContext.Set<MusicVideoArtist>().Select(a => a.Name)),
// entity artists only; free-text music-video/song artist credits are not included (known limitation)
"artist" => dbContext.ArtistMetadata.Select(m => m.Title),
"tag" => dbContext.Set<Tag>()
.Where(t => t.ExternalTypeId != Tag.NfoCountryTypeId && t.ExternalTypeId != Tag.PlexNetworkTypeId)
.Select(t => t.Name),
@@ -192,309 +87,9 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
_ => null
};
/// <summary>
/// SQL name of the invariant-uppercase fold registered by <c>SqliteUnicodeFunctions</c>. Duplicated
/// rather than referenced because Application must not depend on a provider assembly; a test asserts
/// the two constants are equal so they cannot drift.
/// </summary>
internal const string UpperFunction = "etv_upper";
/// <summary>
/// True when the value contains any character outside US-ASCII, which is exactly when SQLite's
/// ASCII-only <c>LOWER()</c> can under-match. Evaluated on the RAW query, never the lowercased copy:
/// the trigger must not be coupled to the fold.
/// </summary>
internal static bool ContainsNonAscii(string value)
{
foreach (char c in value)
{
if (c > 0x7F)
{
return true;
}
}
return false;
}
// Derived per-context rather than read from the TvContext.IsSqlite static on purpose. Nothing MECHANICALLY
// stops that read -- ProviderStaticsWiringTests only parses the two composition roots for ASSIGNMENTS, not
// readers -- but that test's scanner exemption for IsSqlite is justified in prose as "read only by
// DbInitializer + DatabaseMigratorService, both host-only", and reading it here would make that reason
// false while the test stayed green. Do not "simplify" this to IsSqlite.
private static bool IsSqlite(TvContext dbContext) =>
(dbContext.Database.ProviderName ?? string.Empty).Contains("Sqlite", StringComparison.OrdinalIgnoreCase);
/// <summary>
/// Escapes the LIKE metacharacters in a user-supplied prefix and appends the trailing wildcard. The
/// backslash MUST be escaped first, or the escapes added for <c>%</c>/<c>_</c> would themselves be
/// re-escaped. Paired with an explicit <c>ESCAPE '\'</c> in <see cref="UnicodeFoldSql" />, since raw
/// SQL gets none of the escaping EF does for <c>StartsWith</c>.
/// </summary>
internal static string EscapeLikePrefix(string value) =>
value
.Replace("\\", "\\\\", StringComparison.Ordinal)
.Replace("%", "\\%", StringComparison.Ordinal)
.Replace("_", "\\_", StringComparison.Ordinal) + "%";
/// <summary>
/// One bounded, exact prefix query using the Unicode-correct fold. Unlike the list-valued walk this
/// KEEPS its selectivity in SQL — it is a normal indexed-or-not <c>LIMIT</c>ed query exactly like the
/// EF one it supplements, not a paged walk, so there is no row budget to blow and no reason to strip
/// the discriminator predicates out of it.
/// </summary>
internal static string UnicodeFoldSql(string table, string column, string predicate)
{
var match = $"{UpperFunction}({column}) LIKE @Pattern ESCAPE '\\'";
string where = predicate is null ? match : $"({predicate}) AND {match}";
return $"SELECT DISTINCT {column} AS Value FROM {table} WHERE {where} ORDER BY {column} LIMIT @Limit";
}
/// <summary>
/// The tables/columns behind each EF-sourced field, mirroring <see cref="GetSource" /> 1:1.
/// <para>
/// The discriminator predicates must mirror EF's NULL semantics, not C#'s reading of the source.
/// EF compiles <c>t.ExternalTypeId != Tag.NfoCountryTypeId</c> with null semantics, so a row whose
/// <c>ExternalTypeId</c> is NULL IS included; plain SQL <c>&lt;&gt;</c> against NULL yields NULL and
/// would silently drop it. Hence the explicit <c>IS NULL</c> arm.
/// </para>
/// </summary>
private static IReadOnlyList<UnicodeFoldSource> GetUnicodeFoldSources(string name) => name switch
{
"genre" or "show_genre" => [new UnicodeFoldSource("Genre", "Name")],
"studio" => [new UnicodeFoldSource("Studio", "Name")],
"director" => [new UnicodeFoldSource("Director", "Name")],
"writer" => [new UnicodeFoldSource("Writer", "Name")],
"actor" => [new UnicodeFoldSource("Actor", "Name")],
"artist" =>
[
new UnicodeFoldSource("ArtistMetadata", "Title"),
new UnicodeFoldSource("MusicVideoArtist", "Name")
],
"tag" =>
[
new UnicodeFoldSource(
"Tag",
"Name",
"ExternalTypeId IS NULL OR (ExternalTypeId <> @NfoCountryTypeId AND ExternalTypeId <> @PlexNetworkTypeId)",
new Dictionary<string, object>
{
["NfoCountryTypeId"] = Tag.NfoCountryTypeId,
["PlexNetworkTypeId"] = Tag.PlexNetworkTypeId
})
],
"network" =>
[
new UnicodeFoldSource(
"Tag",
"Name",
"ExternalTypeId = @PlexNetworkTypeId",
new Dictionary<string, object> { ["PlexNetworkTypeId"] = Tag.PlexNetworkTypeId })
],
"collection" => [new UnicodeFoldSource("Collection", "Name")],
"video_codec" =>
[
new UnicodeFoldSource(
"MediaStream",
"Codec",
"MediaStreamKind = @VideoStreamKind AND Codec IS NOT NULL",
new Dictionary<string, object> { ["VideoStreamKind"] = (int)MediaStreamKind.Video })
],
"album" =>
[
new UnicodeFoldSource("MusicVideoMetadata", "Album", "Album IS NOT NULL"),
new UnicodeFoldSource("SongMetadata", "Album", "Album IS NOT NULL")
],
_ => []
};
private static async Task<List<string>> GetUnicodeFoldedValues(
TvContext dbContext,
string name,
string query,
int limit,
CancellationToken cancellationToken)
{
IReadOnlyList<UnicodeFoldSource> sources = GetUnicodeFoldSources(name);
if (sources.Count == 0)
{
return [];
}
// CreateFunction is per-connection, so registration happens here, at the one call site that needs
// the function, rather than through an EF connection interceptor: Dapper opens a closed connection
// itself and a direct ADO open does not raise EF's interceptors, so an interceptor-based seam would
// silently miss exactly this query. Opening first makes the registration order-independent.
await dbContext.Database.OpenConnectionAsync(cancellationToken);
TvContext.RegisterUnicodeCaseFunctions(dbContext.Connection);
string pattern = EscapeLikePrefix(query.ToUpperInvariant());
var values = new List<string>();
foreach (UnicodeFoldSource source in sources)
{
var parameters = new DynamicParameters();
parameters.Add("Pattern", pattern);
parameters.Add("Limit", limit);
if (source.Parameters is not null)
{
foreach ((string key, object value) in source.Parameters)
{
parameters.Add(key, value);
}
}
IEnumerable<string> rows = await dbContext.Connection.QueryAsync<string>(
new CommandDefinition(
UnicodeFoldSql(source.Table, source.Column, source.Predicate),
parameters,
cancellationToken: cancellationToken));
values.AddRange(rows.Where(v => !string.IsNullOrEmpty(v)));
}
return values;
}
private sealed record UnicodeFoldSource(
string Table,
string Column,
string Predicate = null,
IReadOnlyDictionary<string, object> Parameters = null);
/// <summary>
/// Maps a field name onto the <c>SongMetadata</c> column that backs it as an <c>IList&lt;string&gt;</c>.
/// The returned value is a compile-time constant from this switch — never caller input — so it is safe
/// to interpolate into the SQL in <see cref="ListValuedSql" />.
/// </summary>
private static string GetSongListValuedColumn(string name) => name switch
{
"artist" => "Artists",
"album_artist" => "AlbumArtists",
_ => null
};
/// <summary>
/// Reads whole values out of a <c>SongMetadata</c> <c>IList&lt;string&gt;</c> column.
/// <para>
/// EF maps these as primitive collections: one JSON array per row in a single <c>TEXT</c>/
/// <c>longtext</c> column. Neither provider can project the elements server-side — SQLite needs
/// the SQL <c>APPLY</c> operator it doesn't have, and Pomelo MySQL doesn't implement primitive
/// collections at all — so there is no server-side <c>SELECT DISTINCT</c> over the elements.
/// </para>
/// <para>
/// So the rows are walked in primary-key order, keyset-paged by row position, and split +
/// exact-filtered in memory. All selectivity is in memory — the query's only condition is the
/// cursor, a seek on the ordering key that never discards a row, so its <c>LIMIT</c> bounds the
/// LOGICAL ROWS returned. See <see cref="ListValuedBatchRows" /> for the four revisions it took to
/// get that right, and for what that bound does and does not cover.
/// </para>
/// </summary>
private static async Task<List<string>> GetSongListValuedValues(
TvContext dbContext,
string column,
string query,
int limit,
CancellationToken cancellationToken)
{
string sql = ListValuedSql(column);
var distinct = new System.Collections.Generic.HashSet<string>(StringComparer.Ordinal);
var afterId = 0;
var read = 0;
while (read < ListValuedMaxRowsRead && distinct.Count < limit)
{
int batch = Math.Min(ListValuedBatchRows, ListValuedMaxRowsRead - read);
List<ListValuedRow> rows = (await dbContext.Connection.QueryAsync<ListValuedRow>(
new CommandDefinition(
sql,
new { AfterId = afterId, Batch = batch },
cancellationToken: cancellationToken))).AsList();
if (rows.Count == 0)
{
break;
}
read += rows.Count;
afterId = rows[^1].Id;
foreach (ListValuedRow row in rows)
{
foreach (string element in ParseElements(row.Payload))
{
if (element.StartsWith(query, StringComparison.OrdinalIgnoreCase))
{
distinct.Add(element);
}
}
}
if (rows.Count < batch)
{
// With no RESIDUAL predicate -- only the cursor, which selects a range rather than discarding
// rows from it -- a short page can only mean the table is exhausted. It can never mean "this
// stretch happened to match nothing", which is precisely why the residual predicate had to go.
// Advancing from the last returned Id is safe for the same reason: nothing was filtered out
// behind it, so no row can be skipped.
break;
}
}
return distinct.ToList();
}
private static IEnumerable<string> ParseElements(string payload)
{
if (string.IsNullOrWhiteSpace(payload))
{
return [];
}
try
{
return (JsonSerializer.Deserialize<string[]>(payload) ?? []).Where(e => !string.IsNullOrEmpty(e));
}
catch (JsonException)
{
return [];
}
}
/// <summary>
/// One keyset page of rows, by ROW POSITION rather than by <c>Id</c> value.
/// <para>
/// The only condition is the cursor — deliberately <b>no RESIDUAL predicate</b>: no <c>LIKE</c>, no
/// <c>LOWER</c>, not even <c>IS NOT NULL</c>. The distinction that matters is not "no predicate"
/// (the cursor is one); it is that <c>Id &gt; @AfterId</c> is a <i>seekable predicate on the
/// ordering key</i>, which positions the scan and never discards a row, whereas a residual
/// predicate throws away rows the engine already produced. <c>LIMIT</c> only truncates what
/// survives a residual predicate, so with one present it bounds the output rather than the row
/// count — which is how every earlier revision scanned past its own bound. With none, <c>LIMIT n</c>
/// yields <c>n</c> logical rows. Null payloads are dropped in memory by
/// <see cref="ParseElements" />.
/// </para>
/// <para>
/// Note this pins the SQL string only. It cannot pin an execution plan, MVCC visibility work, or
/// payload I/O — and on MySQL, using the index to satisfy <c>ORDER BY</c> is an optimizer choice,
/// not a semantic guarantee.
/// </para>
/// </summary>
internal static string ListValuedSql(string column) =>
$"SELECT Id, {column} AS Payload FROM SongMetadata WHERE Id > @AfterId ORDER BY Id LIMIT @Batch";
private sealed class ListValuedRow
{
public int Id { get; init; }
public string Payload { get; init; }
}
private static async Task<List<string>> GetContentRatingValues(
TvContext dbContext,
string query,
string qLower,
int limit,
CancellationToken cancellationToken)
{
@@ -513,22 +108,13 @@ public class GetSearchFieldValuesHandler(IDbContextFactory<TvContext> dbContextF
.Where(cr => !string.IsNullOrEmpty(cr))
.Distinct();
return FilterSortTake(split, query, limit);
return FilterSortTake(split, qLower, limit);
}
/// <summary>
/// The one in-memory filter/sort/take every field funnels through. Both the comparison and the ordering
/// are ORDINAL on purpose: <c>UseRequestLocalization</c> honours <c>Accept-Language</c>, so the current
/// culture is caller-controlled, and <c>ToLower()</c> plus the default (linguistic)
/// <c>StartsWith(string)</c> would make the result depend on it — under <c>tr-TR</c>, <c>q=I</c> lowers
/// to <c>ı</c> and stops matching <c>Istanbul</c>. Note this is the LAST stage only: a field sourced by
/// a plain EF query has already been filtered and truncated by the database collation before it gets
/// here, which ordinal semantics downstream cannot undo (ersatztv#668).
/// </summary>
private static List<string> FilterSortTake(IEnumerable<string> values, string query, int limit) =>
private static List<string> FilterSortTake(IEnumerable<string> values, string qLower, int limit) =>
values
.Where(v => v.StartsWith(query, StringComparison.OrdinalIgnoreCase))
.OrderBy(v => v, StringComparer.Ordinal)
.Where(v => v.ToLower().StartsWith(qLower))
.OrderBy(v => v)
.Take(limit)
.ToList();
}
@@ -1,104 +0,0 @@
using System.Text.RegularExpressions;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Architecture.Tests;
/// <summary>
/// ersatztv#491: <c>TvContext</c> carries settable provider statics (<c>LastInsertedRowId</c>,
/// <c>CaseInsensitiveCollation</c>, <c>IsUniqueConstraintViolation</c>, …) that Infrastructure code
/// reads at runtime. There are TWO composition roots that execute that Infrastructure code —
/// <c>ErsatzTV/Startup.cs</c> (the host) and <c>ErsatzTV.Scanner/Program.cs</c> (a separate
/// executable launched per scan by <c>CallLibraryScannerHandler</c>) — and each wires the statics in
/// its own copy of the provider branch.
/// <para>
/// The failure mode this guards is "a static nobody assigned": #491 added
/// <c>IsUniqueConstraintViolation</c> to <c>Startup</c> only, so every production caller of
/// <c>GetOrAddFolder</c> (all of which live in the scanner) silently kept the conservative
/// <c>_ =&gt; false</c> default and the new catch was inert. Nothing about that is visible in a
/// unit test, because every test harness wires the classifier itself.
/// </para>
/// <para>
/// Source-level rather than reflective on purpose: the wiring lives inside a host-builder
/// lambda that cannot be invoked without standing up a real application, and the thing being
/// asserted is precisely that a line of code exists in both files.
/// </para>
/// </summary>
[TestFixture]
public class ProviderStaticsWiringTests
{
/// <summary>
/// Statics the host wires that the scanner deliberately does not. Add to this only with a reason:
/// the default must be provably harmless in the scanner process.
/// </summary>
private static readonly Dictionary<string, string> ScannerExemptions = new()
{
// Only read by DbInitializer / DatabaseMigratorService, which run in the host exclusively; no
// Infrastructure code on a scan path reads it. Pre-dates #491.
["IsSqlite"] = "read only by DbInitializer + DatabaseMigratorService, both host-only"
};
private static string HostSource => ReadRepoFile(Path.Combine("ErsatzTV", "Startup.cs"));
private static string ScannerSource => ReadRepoFile(Path.Combine("ErsatzTV.Scanner", "Program.cs"));
[Test]
public void Scanner_should_wire_every_TvContext_provider_static_the_host_wires()
{
HashSet<string> host = AssignedStatics(HostSource);
HashSet<string> scanner = AssignedStatics(ScannerSource);
// sanity: the parser found the wiring at all, so a rename can't turn this test into a no-op
host.ShouldContain("LastInsertedRowId");
host.ShouldContain("IsUniqueConstraintViolation");
scanner.ShouldContain("LastInsertedRowId");
List<string> missing = host
.Except(scanner)
.Except(ScannerExemptions.Keys)
.OrderBy(name => name, StringComparer.Ordinal)
.ToList();
missing.ShouldBeEmpty(
"ErsatzTV.Scanner/Program.cs does not assign TvContext static(s) that ErsatzTV/Startup.cs "
+ $"assigns: {string.Join(", ", missing)}. The scanner is a separate process, so an unassigned "
+ "static keeps its default in every library scan. Wire it in BOTH provider branches, or add "
+ "it to ScannerExemptions with a reason if the default is provably harmless there.");
}
[Test]
public void Both_hosts_should_wire_the_unique_constraint_classifier_for_both_providers()
{
// The specific #491 regression, asserted directly rather than via set arithmetic: the classifier
// must be pointed at a real provider implementation on BOTH branches of BOTH composition roots.
foreach ((string name, string source) in new[] { ("host", HostSource), ("scanner", ScannerSource) })
{
source.ShouldContain(
"TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation",
customMessage: $"{name} does not wire the Sqlite unique-constraint classifier");
source.ShouldContain(
"TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation",
customMessage: $"{name} does not wire the MySql unique-constraint classifier");
}
}
private static HashSet<string> AssignedStatics(string source) =>
Regex.Matches(source, @"\bTvContext\.(?<name>[A-Za-z_][A-Za-z0-9_]*)\s*=[^=]")
.Select(m => m.Groups["name"].Value)
.ToHashSet(StringComparer.Ordinal);
private static string ReadRepoFile(string relativePath)
{
var directory = new DirectoryInfo(AppContext.BaseDirectory);
while (directory is not null && !File.Exists(Path.Combine(directory.FullName, "ErsatzTV.sln")))
{
directory = directory.Parent;
}
directory.ShouldNotBeNull("could not locate the repository root (no ErsatzTV.sln above the test binary)");
string path = Path.Combine(directory!.FullName, relativePath);
File.Exists(path).ShouldBeTrue($"expected source file not found: {path}");
return File.ReadAllText(path);
}
}
@@ -1,129 +0,0 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.Interfaces.FFmpeg;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.FFmpeg.State;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Core.Tests.FFmpeg;
[TestFixture]
public class SongVideoGeneratorTests
{
private ITempFilePool _tempFilePool;
private IImageCache _imageCache;
private IFFmpegProcessService _ffmpegProcessService;
private ILocalFileSystem _localFileSystem;
private SongVideoGenerator _songVideoGenerator;
private string _tempSubtitleFile;
[SetUp]
public void SetUp()
{
_tempSubtitleFile = Path.Combine(Path.GetTempPath(), $"{Guid.NewGuid()}.ass");
_tempFilePool = Substitute.For<ITempFilePool>();
_tempFilePool.GetNextTempFile(Arg.Any<TempFileCategory>()).Returns(_tempSubtitleFile);
_imageCache = Substitute.For<IImageCache>();
_imageCache.GetPathForImage(Arg.Any<string>(), Arg.Any<ArtworkKind>(), Arg.Any<Option<int>>())
.Returns("/fake/watermark.png");
_ffmpegProcessService = Substitute.For<IFFmpegProcessService>();
_ffmpegProcessService.GenerateSongImage(
Arg.Any<string>(),
Arg.Any<string>(),
Arg.Any<Option<string>>(),
Arg.Any<Channel>(),
Arg.Any<MediaVersion>(),
Arg.Any<string>(),
Arg.Any<bool>(),
Arg.Any<Option<string>>(),
Arg.Any<WatermarkLocation>(),
Arg.Any<int>(),
Arg.Any<int>(),
Arg.Any<int>(),
Arg.Any<CancellationToken>())
.Returns(Either<BaseError, string>.Right("/fake/song-image.png"));
_localFileSystem = Substitute.For<ILocalFileSystem>();
_localFileSystem.GetCustomOrDefaultFile(Arg.Any<string>(), Arg.Any<string>())
.Returns("/fake/background.png");
_songVideoGenerator = new SongVideoGenerator(
_tempFilePool,
_imageCache,
_ffmpegProcessService,
_localFileSystem);
}
[TearDown]
public void TearDown()
{
if (_tempSubtitleFile is not null && File.Exists(_tempSubtitleFile))
{
File.Delete(_tempSubtitleFile);
}
}
private static Channel BuildChannel()
{
var resolution = new Resolution { Width = 1920, Height = 1080 };
FFmpegProfile ffmpegProfile = FFmpegProfile.New("test", resolution);
return new Channel(Guid.NewGuid())
{
Number = "1",
Name = "Test Channel",
FFmpegProfile = ffmpegProfile,
SongVideoMode = ChannelSongVideoMode.Default
};
}
private static Song BuildUntaggedSong()
{
// an untagged song: FallbackMetadataProvider.GetSongMetadata never assigns
// Artists/AlbumArtists, so they persist (and materialize) as null (ersatztv#691)
var metadata = new SongMetadata
{
MetadataKind = MetadataKind.Fallback,
Title = "Untagged Song",
Artwork = [],
Artists = null,
AlbumArtists = null
};
return new Song
{
SongMetadata = [metadata],
MediaVersions = []
};
}
[Test]
public async Task GenerateSongVideo_should_not_throw_when_artists_and_album_artists_are_null()
{
Song song = BuildUntaggedSong();
Channel channel = BuildChannel();
// SongVideoGenerator randomly picks between two rendering styles (and dereferences
// metadata.Artists/AlbumArtists differently in each); loop enough times that both
// branches -- including the AlbumArtists.Filter(... Artists.Contains ...) branch --
// are exercised with overwhelming probability, so the null guard is proven on both.
for (var i = 0; i < 25; i++)
{
Tuple<string, MediaVersion> result = await _songVideoGenerator.GenerateSongVideo(
song,
channel,
"/usr/bin/ffmpeg",
"/usr/bin/ffprobe",
CancellationToken.None);
result.ShouldNotBeNull();
result.Item1.ShouldBe("/fake/song-image.png");
}
}
}
@@ -1,589 +0,0 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.Images;
using ErsatzTV.Core.Interfaces.FFmpeg;
using ErsatzTV.Core.Interfaces.Images;
using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using Testably.Abstractions.Testing;
namespace ErsatzTV.Core.Tests.FFmpeg;
/// <summary>
/// Pins ersatztv#510: a watermark attached through a DECO resolves by exactly the same policy as the three
/// precedence levels (playout item, channel, global).
/// </summary>
/// <remarks>
/// Before #510 the deco path had its own copy of the image-source switch that resolved paths *unchecked*, so
/// one channel could disagree with itself about whether a bug rendered purely by how the watermark was
/// attached. The divergence covered all three <see cref="ChannelWatermarkImageSource" /> values, not just
/// <c>ChannelLogo</c>:
/// <list type="bullet">
/// <item>a missing local file was handed downstream as a dead path (and a dead LOCAL path can reach
/// ffmpeg as a bare <c>-i</c> argument via <c>CanUseFFmpegNativeWatermark</c>, so it is worse than a
/// skipped overlay);</item>
/// <item>an un-migrated external-URL logo was handed down as a renderable URL, which
/// <c>graphics.channel-logo-caching</c> (#525) forbids the render path from fetching;</item>
/// <item>a channel with no logo artwork got the generated-initials localhost URL, which a live-E2E on a
/// real transcoded frame confirmed DID render — the deco path only. #510 resolved that split in favour
/// of "no on-screen bug" everywhere.</item>
/// </list>
/// The <c>Deco_And_Channel_Level_Should_Resolve_Identically</c> cases are the structural guard: they assert
/// the two callers agree, so re-introducing a per-caller policy fails here rather than silently in prod.
/// </remarks>
[TestFixture]
public class WatermarkSelectorDecoResolutionTests
{
private const string ExternalLogoUrl = "https://cdn.example.com/logos/channel.png";
private const string LogoStoredPath = "abc123.png";
private const string LogoCachePath = "/cache/logos/ab/abc123.png";
private const string CustomStoredPath = "def456.png";
private const string CustomCachePath = "/cache/watermarks/de/def456.png";
private const string ResourceImage = "song-progress.png";
private static string ResourcePath => Path.Combine(FileSystemLayout.ResourcesCacheFolder, ResourceImage);
/// <summary>Builds a selector whose mock filesystem contains exactly <paramref name="existingFiles" />.</summary>
private static WatermarkSelector Selector(Deco playoutDeco, params string[] existingFiles)
{
// one Initialize() call, chained -- calling it per file would leave "does a second Initialize()
// preserve the first file?" untested, and a silently under-seeded filesystem makes a
// "resolves to nothing" assertion pass for the wrong reason
var mockFileSystem = new MockFileSystem();
if (existingFiles.Length > 0)
{
var initialized = mockFileSystem.Initialize().WithFile(existingFiles[0]);
foreach (string file in existingFiles.Skip(1))
{
initialized = initialized.WithFile(file);
}
}
var fakeImageCache = Substitute.For<IImageCache>();
fakeImageCache.GetPathForImage(Arg.Any<string>(), Arg.Is(ArtworkKind.Logo), Arg.Any<Option<int>>())
.Returns(_ => LogoCachePath);
fakeImageCache.GetPathForImage(Arg.Any<string>(), Arg.Is(ArtworkKind.Watermark), Arg.Any<Option<int>>())
.Returns(_ => CustomCachePath);
// Faithful to the real ImageCache.GetPathForImage, which does fileName[..2] and therefore THROWS on a
// blank/null name. Modelling that is what makes the blank-image guard tests mutation-sensitive: before
// #510 the channel and global arms had no guard and this threw out of stream startup.
fakeImageCache
.GetPathForImage(
Arg.Is<string>(s => string.IsNullOrWhiteSpace(s)),
Arg.Any<ArtworkKind>(),
Arg.Any<Option<int>>())
.Returns<string>(_ => throw new ArgumentOutOfRangeException(nameof(IImageCache.GetPathForImage)));
var decoSelector = Substitute.For<IDecoSelector>();
decoSelector.GetDecoEntries(Arg.Any<Playout>(), Arg.Any<DateTimeOffset>())
.Returns(new DecoEntries(Option<Deco>.None, Optional(playoutDeco)));
return new WatermarkSelector(
mockFileSystem,
fakeImageCache,
decoSelector,
NullLogger<WatermarkSelector>.Instance);
}
private static ChannelWatermark Watermark(ChannelWatermarkImageSource source, string image = "") =>
new()
{
Id = 7,
Name = "Deco Bug",
ImageSource = source,
Image = image,
Mode = ChannelWatermarkMode.Permanent
};
private static Deco DecoWith(ChannelWatermark watermark) =>
new()
{
Id = 1,
Name = "Test Deco",
WatermarkMode = DecoMode.Override,
UseWatermarkDuringFiller = true,
DecoWatermarks = [new DecoWatermark { WatermarkId = watermark.Id, Watermark = watermark }],
Watermarks = []
};
private static Channel ChannelWith(string logoPath, ChannelWatermark channelWatermark = null)
{
var channel = new Channel(Guid.Empty)
{
Id = 1,
Number = "1",
Name = "Test",
StreamingMode = StreamingMode.TransportStream,
Artwork = [],
Watermark = channelWatermark,
WatermarkId = channelWatermark?.Id
};
if (logoPath is not null)
{
channel.Artwork.Add(new Artwork { ArtworkKind = ArtworkKind.Logo, Path = logoPath });
}
return channel;
}
private static PlayoutItem PlayoutItem() =>
new()
{
FillerKind = FillerKind.None,
DisableWatermarks = false,
Watermarks = [],
Playout = new Playout()
};
private static List<WatermarkOptions> SelectViaDeco(
ChannelWatermark watermark,
Channel channel,
params string[] existingFiles)
{
WatermarkSelector selector = Selector(DecoWith(watermark), existingFiles);
return selector.SelectWatermarks(
Option<ChannelWatermark>.None,
channel,
PlayoutItem(),
DateTimeOffset.Now);
}
// ---- positive control: the arrangement CAN produce a watermark ------------------------------
//
// Without this, every "resolves to nothing" assertion below could pass vacuously (a broken deco
// arrangement that never reaches the resolver at all looks identical to a correct refusal).
[Test]
public void Deco_ChannelLogo_Should_Use_Cached_Path_When_Local_Logo_Exists()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
Channel channel = ChannelWith(LogoStoredPath);
List<WatermarkOptions> result = SelectViaDeco(watermark, channel, LogoCachePath);
result.Count.ShouldBe(1);
result[0].ImagePath.ShouldBe(LogoCachePath);
}
// ---- ChannelLogo: the three cases #510 was filed for ----------------------------------------
[Test]
public void Deco_ChannelLogo_Should_Be_Ignored_When_Logo_Is_An_External_Url()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
Channel channel = ChannelWith(ExternalLogoUrl);
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
result.ShouldBeEmpty();
}
[Test]
public void Deco_ChannelLogo_Should_Be_Ignored_When_Local_Logo_File_Is_Missing()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
Channel channel = ChannelWith(LogoStoredPath);
// nothing on disk
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
result.ShouldBeEmpty();
}
/// <summary>
/// The #510 policy decision: with no logo artwork the generated-initials fallback is NOT used. It
/// genuinely rendered here before (confirmed by live-E2E on a real frame), so this is a deliberate,
/// recorded behavior change — not a no-op cleanup.
/// </summary>
[Test]
public void Deco_ChannelLogo_Should_Be_Ignored_When_Channel_Has_No_Logo_Artwork()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
Channel channel = ChannelWith(null);
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
result.ShouldBeEmpty();
// Folded in from a separate test that asserted only this. On its own it was vacuous — an empty list
// trivially contains no URL — so it is a second assertion here rather than a test implying independent
// coverage. It earns its place by naming the value if this ever starts returning options again (#652).
result.Select(o => o.ImagePath)
.ShouldNotContain(ChannelLogoGenerator.GenerateChannelLogoUrl(channel));
}
// ---- Custom and Resource: the two arms #510 did not mention but that diverged too -----------
[Test]
public void Deco_Custom_Should_Use_Cached_Path_When_File_Exists()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
Channel channel = ChannelWith(LogoStoredPath);
List<WatermarkOptions> result = SelectViaDeco(watermark, channel, CustomCachePath);
result.Count.ShouldBe(1);
result[0].ImagePath.ShouldBe(CustomCachePath);
}
[Test]
public void Deco_Custom_Should_Be_Ignored_When_File_Is_Missing()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
Channel channel = ChannelWith(LogoStoredPath);
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
result.ShouldBeEmpty();
}
[Test]
public void Deco_Custom_Should_Be_Ignored_When_Image_Is_Blank()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Custom, " ");
Channel channel = ChannelWith(LogoStoredPath);
List<WatermarkOptions> result = SelectViaDeco(watermark, channel, CustomCachePath);
result.ShouldBeEmpty();
}
[Test]
public void Deco_Resource_Should_Use_Resource_Path_When_File_Exists()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Resource, ResourceImage);
Channel channel = ChannelWith(LogoStoredPath);
List<WatermarkOptions> result = SelectViaDeco(watermark, channel, ResourcePath);
result.Count.ShouldBe(1);
result[0].ImagePath.ShouldBe(ResourcePath);
}
[Test]
public void Deco_Resource_Should_Be_Ignored_When_File_Is_Missing()
{
ChannelWatermark watermark = Watermark(ChannelWatermarkImageSource.Resource, ResourceImage);
Channel channel = ChannelWith(LogoStoredPath);
List<WatermarkOptions> result = SelectViaDeco(watermark, channel);
result.ShouldBeEmpty();
}
// ---- non-deco consequences of the SAME unification -------------------------------------------
//
// These pin precedence-level behavior rather than deco behavior, but they exist because of the #510
// unification: one is the single piece of per-caller policy deliberately kept, the others are arms that
// used to throw. Without them a future refactor can delete the survivor, or re-introduce the crash, with
// a fully green suite.
/// <summary>
/// The one surviving per-caller policy: a playout-item `Custom` watermark with a blank image falls
/// THROUGH to the channel/global watermark rather than resolving to "no watermark". Unifying
/// resolution must not change which watermark WINS.
/// </summary>
[Test]
public void Blank_Custom_Playout_Item_Watermark_Should_Fall_Through_To_Channel_Watermark()
{
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.Custom, " ");
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
channelWatermark.Id = 8;
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
Option<WatermarkOptions> result = Selector(null, CustomCachePath)
.GetWatermarkOptions(channel, playoutItemWatermark, Option<ChannelWatermark>.None);
// the CHANNEL watermark wins -- not None, and not the blank playout-item one
result.IsSome.ShouldBeTrue();
WatermarkOptions options = result.IfNone(() => throw new InvalidOperationException());
options.ImagePath.ShouldBe(CustomCachePath);
options.Watermark.Id.ShouldBe(8);
}
/// <summary>
/// Before #510 the channel and global arms had no blank-image guard, so they reached
/// <c>ImageCache.GetPathForImage</c> whose <c>fileName[..2]</c> threw out of stream startup. Now a
/// warning plus no watermark.
/// </summary>
[TestCase(null)]
[TestCase("")]
[TestCase(" ")]
public void Channel_Level_Blank_Custom_Watermark_Should_Resolve_To_None_Not_Throw(string image)
{
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.Custom, image);
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
Option<WatermarkOptions> result = Should.NotThrow(
() => Selector(null).GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
Option<ChannelWatermark>.None));
result.IsNone.ShouldBeTrue();
}
/// <summary>
/// Same fall-through, but landing on the GLOBAL watermark — the channel-level variant above cannot
/// distinguish "fell through correctly" from "stopped at the channel by accident".
/// </summary>
[Test]
public void Blank_Custom_Playout_Item_Watermark_Should_Fall_Through_To_Global_Watermark()
{
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.Custom, " ");
ChannelWatermark globalWatermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
globalWatermark.Id = 9;
// no channel-level watermark, so the only remaining candidate is the global one
Channel channel = ChannelWith(LogoStoredPath);
Option<WatermarkOptions> result = Selector(null, CustomCachePath)
.GetWatermarkOptions(channel, playoutItemWatermark, globalWatermark);
result.IsSome.ShouldBeTrue();
WatermarkOptions options = result.IfNone(() => throw new InvalidOperationException());
options.ImagePath.ShouldBe(CustomCachePath);
options.Watermark.Id.ShouldBe(9);
}
/// <summary>
/// The complement of the fall-through cases: a NON-blank custom image whose file is merely missing must
/// NOT fall through — it resolves to "no watermark" and the channel watermark never gets a turn.
/// Without this, widening the blank-image guard to "any unresolvable custom" would pass unnoticed.
/// </summary>
/// <remarks>
/// The channel-level fallback is deliberately an INDEPENDENTLY RESOLVABLE `ChannelLogo` watermark whose
/// cached file exists. An earlier version of this test gave the fallback the same missing custom path as
/// the playout-item watermark, which made it unfalsifiable: a wrongly-widened guard would have fallen
/// through to a fallback that also resolved to None, so the assertion held either way.
/// </remarks>
[Test]
public void Missing_But_Named_Custom_Playout_Item_Watermark_Should_Not_Fall_Through()
{
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
channelWatermark.Id = 8;
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
// the channel logo's cached file EXISTS, so a fall-through would return it and fail this test;
// the custom watermark's file does not, so the playout-item watermark is unresolvable
Option<WatermarkOptions> result = Selector(null, LogoCachePath)
.GetWatermarkOptions(channel, playoutItemWatermark, Option<ChannelWatermark>.None);
result.IsNone.ShouldBeTrue();
}
/// <summary>
/// Positive control for the test above: the same arrangement, but with the playout-item watermark BLANK
/// rather than missing, must fall through and return the resolvable channel logo. Together the pair
/// shows the guard distinguishes blank from unresolvable, rather than both landing on None.
/// </summary>
/// <remarks>
/// Parameterized over all three blank forms because the guard is <c>IsNullOrWhiteSpace</c>: testing only
/// <c>" "</c> would let a mutation to <c>image == " "</c> pass while silently breaking fall-through
/// for <c>null</c> and <c>""</c> — and <c>null</c> is the form the API actually persists.
/// </remarks>
[TestCase(null)]
[TestCase("")]
[TestCase(" ")]
public void Blank_Custom_Playout_Item_Watermark_Should_Fall_Through_To_A_Resolvable_Channel_Logo(string image)
{
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.Custom, image);
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
channelWatermark.Id = 8;
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
Option<WatermarkOptions> result = Selector(null, LogoCachePath)
.GetWatermarkOptions(channel, playoutItemWatermark, Option<ChannelWatermark>.None);
result.IsSome.ShouldBeTrue();
result.IfNone(() => throw new InvalidOperationException()).ImagePath.ShouldBe(LogoCachePath);
}
/// <summary>
/// Pins the <c>ImageSource is Custom</c> half of the blank-image guard, which nothing else covers.
/// </summary>
/// <remarks>
/// A <c>ChannelLogo</c> watermark's <c>Image</c> is NORMALLY blank — the API persists `Image = null` for
/// every non-`Custom` source — so if the guard's `is Custom` discriminator were dropped, leaving only
/// `IsNullOrWhiteSpace(Image)`, every playout-item `ChannelLogo` watermark would fall through to
/// channel/global instead of resolving the channel's own logo. This test fails on that mutation: the
/// playout-item watermark carries a distinguishing Id, so falling through is observable even though both
/// levels would resolve to the same cached path.
/// </remarks>
[Test]
public void Blank_Image_ChannelLogo_Playout_Item_Watermark_Should_Win_And_Not_Fall_Through()
{
// Image is left blank, exactly as the API stores a ChannelLogo watermark
ChannelWatermark playoutItemWatermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
playoutItemWatermark.Id = 42;
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.ChannelLogo);
channelWatermark.Id = 8;
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
Option<WatermarkOptions> result = Selector(null, LogoCachePath)
.GetWatermarkOptions(channel, playoutItemWatermark, Option<ChannelWatermark>.None);
result.IsSome.ShouldBeTrue();
WatermarkOptions options = result.IfNone(() => throw new InvalidOperationException());
options.ImagePath.ShouldBe(LogoCachePath);
// the PLAYOUT-ITEM watermark won; a fall-through would have returned the channel's (Id 8)
options.Watermark.Id.ShouldBe(42);
}
/// <summary>
/// `CreateWatermarkHandler`/`UpdateWatermarkHandler` write `Image = null` for every non-`Custom`
/// watermark, so an API-created `Resource` watermark hits `Path.Combine(folder, null)` — an
/// `ArgumentNullException` out of stream startup. Uses the persisted shape (null), not a hand-made
/// filename, which is what the rest of the fixture would otherwise assume.
/// </summary>
[TestCase(null)]
[TestCase("")]
public void Resource_Watermark_With_No_Image_Name_Should_Resolve_To_None_Not_Throw(string image)
{
ChannelWatermark channelWatermark = Watermark(ChannelWatermarkImageSource.Resource, image);
Channel channel = ChannelWith(LogoStoredPath, channelWatermark);
Option<WatermarkOptions> result = Should.NotThrow(
() => Selector(null).GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
Option<ChannelWatermark>.None));
result.IsNone.ShouldBeTrue();
}
/// <summary>
/// Dropping an unresolvable watermark shortens the list handed to
/// <c>CanUseFFmpegNativeWatermark</c>, whose predicate includes `Count == 1`. So this is also the pin on
/// the observable routing change: two attached permanent watermarks, one missing, now yield ONE option
/// (ffmpeg-native) where they previously yielded two (graphics engine).
/// </summary>
[Test]
public void Deco_With_One_Valid_And_One_Missing_Watermark_Should_Return_Only_The_Valid_One()
{
ChannelWatermark valid = Watermark(ChannelWatermarkImageSource.ChannelLogo);
ChannelWatermark missing = Watermark(ChannelWatermarkImageSource.Custom, CustomStoredPath);
missing.Id = 8;
var deco = new Deco
{
Id = 1,
Name = "Test Deco",
WatermarkMode = DecoMode.Override,
UseWatermarkDuringFiller = true,
DecoWatermarks =
[
new DecoWatermark { WatermarkId = valid.Id, Watermark = valid },
new DecoWatermark { WatermarkId = missing.Id, Watermark = missing }
],
Watermarks = []
};
// only the channel logo's cached file exists; the custom watermark's does not
List<WatermarkOptions> result = Selector(deco, LogoCachePath).SelectWatermarks(
Option<ChannelWatermark>.None,
ChannelWith(LogoStoredPath),
PlayoutItem(),
DateTimeOffset.Now);
result.Count.ShouldBe(1);
result[0].ImagePath.ShouldBe(LogoCachePath);
// The routing claim itself, not just the filtering: call the real predicate. Asserting Count == 1 alone
// would leave the decision record's "now routes ffmpeg-native" statement unpinned, since the decision
// lives in FFmpegLibraryProcessService rather than in the selector.
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark(0, result).ShouldBeTrue();
}
/// <summary>
/// Before #510 the global arm had no <c>Resource</c> case and hit <c>default: throw</c>.
/// </summary>
[Test]
public void Global_Level_Resource_Watermark_Should_Resolve_Instead_Of_Throwing()
{
ChannelWatermark globalWatermark = Watermark(ChannelWatermarkImageSource.Resource, ResourceImage);
Channel channel = ChannelWith(LogoStoredPath);
Option<WatermarkOptions> result = Should.NotThrow(
() => Selector(null, ResourcePath).GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
globalWatermark));
result.IsSome.ShouldBeTrue();
result.IfNone(() => throw new InvalidOperationException()).ImagePath.ShouldBe(ResourcePath);
}
// ---- the structural guard: deco and channel-level must agree, case for case ------------------
private static IEnumerable<TestCaseData> ParityCases()
{
// (image source, watermark.Image, channel logo path, files that exist)
yield return new TestCaseData(
ChannelWatermarkImageSource.ChannelLogo, "", LogoStoredPath, new[] { LogoCachePath })
.SetName("ChannelLogo, local file present");
yield return new TestCaseData(
ChannelWatermarkImageSource.ChannelLogo, "", LogoStoredPath, Array.Empty<string>())
.SetName("ChannelLogo, local file missing");
yield return new TestCaseData(
ChannelWatermarkImageSource.ChannelLogo, "", ExternalLogoUrl, Array.Empty<string>())
.SetName("ChannelLogo, external URL");
yield return new TestCaseData(
ChannelWatermarkImageSource.ChannelLogo, "", null, Array.Empty<string>())
.SetName("ChannelLogo, no logo artwork");
yield return new TestCaseData(
ChannelWatermarkImageSource.Custom, CustomStoredPath, LogoStoredPath, new[] { CustomCachePath })
.SetName("Custom, file present");
yield return new TestCaseData(
ChannelWatermarkImageSource.Custom, CustomStoredPath, LogoStoredPath, Array.Empty<string>())
.SetName("Custom, file missing");
// Both sides agree here by construction (each returns nothing), which is the point: it documents that
// the blank-image fall-through asymmetry lives ONLY at the playout-item level -- covered by
// Blank_Custom_Playout_Item_Watermark_Should_Fall_Through_To_Channel_Watermark -- rather than leaving
// the omission looking like an evasion.
yield return new TestCaseData(
ChannelWatermarkImageSource.Custom, " ", LogoStoredPath, Array.Empty<string>())
.SetName("Custom, blank image");
yield return new TestCaseData(
ChannelWatermarkImageSource.Resource, ResourceImage, LogoStoredPath, new[] { ResourcePath })
.SetName("Resource, file present");
yield return new TestCaseData(
ChannelWatermarkImageSource.Resource, ResourceImage, LogoStoredPath, Array.Empty<string>())
.SetName("Resource, file missing");
}
[TestCaseSource(nameof(ParityCases))]
public void Deco_And_Channel_Level_Should_Resolve_Identically(
ChannelWatermarkImageSource source,
string image,
string logoPath,
string[] existingFiles)
{
// deco path
ChannelWatermark decoWatermark = Watermark(source, image);
List<WatermarkOptions> viaDeco = SelectViaDeco(decoWatermark, ChannelWith(logoPath), existingFiles);
// channel precedence level, same watermark definition and same channel
ChannelWatermark channelWatermark = Watermark(source, image);
Channel channel = ChannelWith(logoPath, channelWatermark);
Option<WatermarkOptions> viaChannel = Selector(null, existingFiles)
.GetWatermarkOptions(channel, Option<ChannelWatermark>.None, Option<ChannelWatermark>.None);
List<string> decoPaths = viaDeco.Select(o => o.ImagePath).ToList();
// built explicitly rather than via Option.ToList(), which yields a LanguageExt Lst<string>
var channelPaths = new List<string>();
viaChannel.IfSome(o => channelPaths.Add(o.ImagePath));
decoPaths.ShouldBe(channelPaths);
}
}
@@ -1,4 +1,3 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
using NUnit.Framework;
@@ -865,241 +864,4 @@ public static class AlternateScheduleSelectorTests
result.IsNone.ShouldBeFalse();
}
}
/// <summary>
/// ersatztv#823. <c>DaysOfWeek</c>, <c>DaysOfMonth</c> and <c>MonthsOfYear</c> on
/// <see cref="PlayoutTemplate" /> and <see cref="ProgramScheduleAlternate" /> are six
/// single-column primitive collections whose columns are <c>nullable: true</c> on both providers.
/// A NULL column materializes as CLR <c>null</c> — EF does not invoke the value converter for a
/// NULL at all — so unguarded, each <c>.Contains</c> in
/// <see cref="AlternateScheduleSelector.GetScheduleForDate{T}" /> throws
/// <see cref="NullReferenceException" />. These tests are RED without the read-site guard.
/// <para>
/// A null reads as UNRESTRICTED (the <c>All*()</c> sets), not as empty. The deciding case is
/// SQLite's <c>20240113140741_Add_PlayoutTemplate_DaysOfMonth</c>, which adds the column
/// <c>nullable: true</c> with NO default: a row inserted before it had no day-of-month
/// restriction, so reading its NULL as empty would INVERT its meaning and silently stop the
/// template applying. That is the one NULL reachable without any code writing one.
/// </para>
/// <para>
/// Reachability itself is pinned by
/// <c>ErsatzTV.Tests.Integration.SchedulingCollectionColumnNullTests</c> against a real
/// <c>TvContext</c>; these tests pin what the selector does once the null is there.
/// </para>
/// </summary>
[TestFixture]
public class GetScheduleForDate_NullCollections
{
private static readonly TimeSpan Offset = TimeSpan.FromHours(-5);
// A Wednesday in March, so no All*() member is coincidentally excluded — and deliberately the
// 20th rather than the 6th. With a day <= 12 a CROSS-WIRED substitution survives the whole
// fixture: `DaysOfMonth ?? AllMonthsOfYear()` hands back 1..12, which still contains day 6, so
// every assertion here passes while the guard substitutes the wrong set. Day 20 is outside 1..12
// and kills it.
private static readonly DateTimeOffset AnyDate = new(2024, 3, 20, 0, 0, 0, Offset);
private static PlayoutTemplate Unrestricted() =>
new()
{
DaysOfWeek = AlternateScheduleSelector.AllDaysOfWeek(),
DaysOfMonth = AlternateScheduleSelector.AllDaysOfMonth(),
MonthsOfYear = AlternateScheduleSelector.AllMonthsOfYear()
};
private static Option<PlayoutTemplate> Select(params PlayoutTemplate[] templates) =>
AlternateScheduleSelector.GetScheduleForDate(templates.ToList(), AnyDate);
[Test]
public void Null_DaysOfWeek_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
Select(template).IsSome.ShouldBeTrue(
"a NULL DaysOfWeek means no weekday restriction was recorded, so the template still applies");
}
[Test]
public void Null_DaysOfMonth_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfMonth = null!;
Select(template).IsSome.ShouldBeTrue();
}
[Test]
public void Null_MonthsOfYear_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.MonthsOfYear = null!;
Select(template).IsSome.ShouldBeTrue();
}
[Test]
public void All_Three_Null_On_ProgramScheduleAlternate_Reads_As_Unrestricted()
{
var alternate = new ProgramScheduleAlternate
{
DaysOfWeek = null!,
DaysOfMonth = null!,
MonthsOfYear = null!
};
AlternateScheduleSelector.GetScheduleForDate(
new List<ProgramScheduleAlternate> { alternate },
AnyDate)
.IsSome.ShouldBeTrue();
}
/// <summary>
/// THE DISCRIMINATING CONTROL. Every test above sets a NULL and expects the item to be selected,
/// so all of them pass equally under "NULL means unrestricted" and under the much broader
/// "any NULL makes this item match unconditionally" — a refactor that short-circuits the whole
/// date check when any dimension is null keeps them green. Here the nulled dimension is paired
/// with a RESTRICTIVE non-null one that the date fails, so only the narrow reading passes.
/// </summary>
[Test]
public void A_Null_Dimension_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
// AnyDate is in MARCH; restrict to January only.
template.MonthsOfYear = [1];
Select(template).IsNone.ShouldBeTrue(
"a NULL DaysOfWeek relaxes ONLY the weekday dimension — the January restriction still "
+ "excludes a March date");
}
/// <summary>
/// One arrangement is not enough: with only the <c>DaysOfWeek</c> case above, a PER-DIMENSION
/// mutant survives the whole fixture — e.g. <c>if (item.MonthsOfYear is null) return item;</c>
/// placed ahead of the checks is never reached by that test, because its <c>MonthsOfYear</c> is
/// non-null. So each of the three dimensions is nulled in turn against a restriction on a
/// DIFFERENT dimension.
/// </summary>
[Test]
public void A_Null_MonthsOfYear_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.MonthsOfYear = null!;
// AnyDate is a WEDNESDAY; restrict to Monday only.
template.DaysOfWeek = [DayOfWeek.Monday];
Select(template).IsNone.ShouldBeTrue(
"a NULL MonthsOfYear relaxes ONLY the month dimension — the Monday restriction still "
+ "excludes a Wednesday");
}
/// <summary>
/// The third of the per-dimension controls — see
/// <see cref="A_Null_MonthsOfYear_Does_Not_Relax_The_Other_Dimensions" /> for why one
/// arrangement is not enough. Here the nulled dimension is <c>DaysOfMonth</c> and the
/// restriction that must still bite is on <c>MonthsOfYear</c>.
/// </summary>
[Test]
public void A_Null_DaysOfMonth_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfMonth = null!;
// AnyDate is in MARCH; restrict to January only.
template.MonthsOfYear = [1];
Select(template).IsNone.ShouldBeTrue(
"a NULL DaysOfMonth relaxes ONLY the day-of-month dimension — the January restriction "
+ "still excludes a March date");
}
/// <summary>
/// A null must not be confused with an explicitly EMPTY collection. Empty is a legal, reachable
/// state meaning "matches no day", and it keeps that meaning — which is exactly why a NULL
/// cannot be normalized to it.
/// </summary>
[Test]
public void An_Explicitly_Empty_Collection_Still_Matches_Nothing()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = [];
Select(template).IsNone.ShouldBeTrue(
"an empty DaysOfWeek is a recorded restriction of NO days, unlike a NULL");
}
/// <summary>
/// The guard resolves PER ITEM: a null on the first item must not decide the second. Making the
/// nulled item genuinely non-matching is what measures that — with an unrestricted nulled item
/// at index 0 it simply wins on ordering and the second item is never evaluated, so the
/// invariant would go unmeasured while the test passed.
/// </summary>
[Test]
public void A_Null_On_One_Item_Does_Not_Decide_A_Later_Item()
{
PlayoutTemplate nulled = Unrestricted();
nulled.DaysOfWeek = null!;
nulled.MonthsOfYear = [1]; // AnyDate is in March, so this item must NOT match
nulled.Index = 0;
PlayoutTemplate second = Unrestricted();
second.Index = 1;
foreach (PlayoutTemplate selected in Select(nulled, second))
{
selected.ShouldBeSameAs(second);
return;
}
Assert.Fail("the loop stopped at the null-collection item instead of continuing to the next");
}
/// <summary>
/// ...and when the nulled item IS unrestricted it legitimately wins on ordering. Paired with the
/// test above so "index 0 wins" and "the loop continues past a non-matching null item" are
/// separately pinned.
/// </summary>
[Test]
public void An_Unrestricted_Null_Item_Wins_On_Index_Order()
{
PlayoutTemplate nulled = Unrestricted();
nulled.DaysOfWeek = null!;
nulled.Index = 0;
PlayoutTemplate second = Unrestricted();
second.Index = 1;
foreach (PlayoutTemplate selected in Select(nulled, second))
{
selected.ShouldBeSameAs(nulled);
return;
}
Assert.Fail("the null-collection item was skipped instead of read as unrestricted");
}
/// <summary>
/// The read-site guard must not be written BACK onto the item. These are single-column
/// primitive collections, so assigning the guard would flip a tracked entity to
/// <c>Modified</c> and the next <c>SaveChanges</c> would persist the substituted collection
/// over the NULL — the mechanism recorded as <c>media.nullable-primitive-collection-mutation</c>.
/// </summary>
[Test]
public void Guard_Must_Not_Be_Written_Back_Onto_The_Item()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
template.DaysOfMonth = null!;
template.MonthsOfYear = null!;
Select(template);
template.DaysOfWeek.ShouldBeNull();
template.DaysOfMonth.ShouldBeNull();
template.MonthsOfYear.ShouldBeNull();
}
}
}
@@ -37,6 +37,4 @@ public record FFmpegFullProfileResponseModel(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup);
bool QsvPreferNativeDecoder);
@@ -8,7 +8,6 @@ public record PlayoutResponseModel(
PlayoutScheduleKind ScheduleKind,
string ChannelName,
string ChannelNumber,
int ChannelId,
ChannelPlayoutMode PlayoutMode,
string ScheduleName,
string? ScheduleFile,
@@ -24,7 +23,6 @@ public record PlayoutResponseModel(
PlayoutScheduleKind scheduleKind,
string channelName,
string channelNumber,
int channelId,
ChannelPlayoutMode playoutMode,
string scheduleName,
string? scheduleFile,
@@ -39,7 +37,6 @@ public record PlayoutResponseModel(
scheduleKind,
channelName,
channelNumber,
channelId,
playoutMode,
scheduleName,
scheduleFile,
-3
View File
@@ -26,9 +26,6 @@ public class ConfigElementKey
public static ConfigElementKey ChannelTemplatesDefaultTemplateId => new("channel_templates.default_template_id");
public static ConfigElementKey WatermarkChannelBugSeeded => new("watermark.channel_bug_seeded");
public static ConfigElementKey GraphicsOnNowNextSeeded => new("graphics.on_now_next_seeded");
public static ConfigElementKey GraphicsOnNowNextDefaultAttached =>
new("graphics.on_now_next_default_attached");
public static ConfigElementKey FFmpegSegmenterTimeout => new("ffmpeg.segmenter.timeout_seconds");
public static ConfigElementKey FFmpegWorkAheadSegmenters => new("ffmpeg.segmenter.work_ahead_limit");
public static ConfigElementKey FFmpegInitialSegmentCount => new("ffmpeg.segmenter.initial_segment_count");
-2
View File
@@ -14,8 +14,6 @@ public record FFmpegProfile
public VaapiDriver VaapiDriver { get; set; }
public string VaapiDevice { get; set; }
public int? QsvExtraHardwareFrames { get; set; }
public double? ReadRate { get; set; }
public double? ReadRateCatchup { get; set; }
public bool? QsvPreferNativeDecoder { get; set; }
public int ResolutionId { get; set; }
public Resolution Resolution { get; set; }
+1 -11
View File
@@ -1,19 +1,9 @@
namespace ErsatzTV.Core.Domain;
namespace ErsatzTV.Core.Domain;
public class LibraryFolder
{
public int Id { get; set; }
public string Path { get; set; }
/// <summary>
/// SHA-256 hex of <see cref="Path" /> (<see cref="ErsatzTV.Core.PathUtils.GetPathHash" />), the
/// indexable stand-in for the unbounded <see cref="Path" /> column that backs the unique
/// <c>(LibraryPathId, PathHash)</c> constraint — the same shape as <c>MediaFile.PathHash</c>.
/// Nullable: rows created before ersatztv#491 carry <c>null</c> until a scan heals them, and a
/// unique index treats nulls as distinct so those legacy rows never collide.
/// </summary>
public string PathHash { get; set; }
public int LibraryPathId { get; set; }
public LibraryPath LibraryPath { get; set; }
public int? ParentId { get; set; }
@@ -1,4 +1,4 @@
namespace ErsatzTV.Core.Domain;
namespace ErsatzTV.Core.Domain;
public class SongMetadata : Metadata
{
@@ -610,9 +610,7 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
channel.FFmpegProfile.QsvPreferNativeDecoder != false,
Optional(channel.FFmpegProfile.ReadRate),
Optional(channel.FFmpegProfile.ReadRateCatchup));
channel.FFmpegProfile.QsvPreferNativeDecoder != false);
_logger.LogDebug("FFmpeg desired state {FrameState}", desiredState);
@@ -829,9 +827,7 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
MaybeReadRate: Optional(channel.FFmpegProfile.ReadRate),
MaybeReadRateCatchup: Optional(channel.FFmpegProfile.ReadRateCatchup));
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel);
var ffmpegSubtitleStream = new ErsatzTV.FFmpeg.MediaStream(0, "ass", StreamKind.Video);
@@ -972,9 +968,7 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
MaybeReadRate: Optional(channel.FFmpegProfile.ReadRate),
MaybeReadRateCatchup: Optional(channel.FFmpegProfile.ReadRateCatchup));
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel);
var audioInputFile = new NullAudioInputFile(audioState);
+7 -10
View File
@@ -1,4 +1,4 @@
using System.Globalization;
using System.Globalization;
using System.Text;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.FFmpeg;
@@ -85,9 +85,6 @@ public class SongVideoGenerator : ISongVideoGenerator
var sb = new StringBuilder();
List<string> artists = Optional(metadata.Artists).Flatten().ToList();
List<string> albumArtists = Optional(metadata.AlbumArtists).Flatten().ToList();
if (detailsStyle)
{
if (!string.IsNullOrWhiteSpace(metadata.Title))
@@ -95,17 +92,17 @@ public class SongVideoGenerator : ISongVideoGenerator
sb.Append(CultureInfo.InvariantCulture, $"{{\\fs{largeFontSize}}}{metadata.Title}");
}
if (artists.Count > 0)
if (metadata.Artists.Count > 0)
{
var allArtists = string.Join(", ", artists);
var allArtists = string.Join(", ", metadata.Artists);
sb.Append(CultureInfo.InvariantCulture, $"\\N{{\\fs{fontSize}}}{allArtists}");
}
}
else
{
if (artists.Count > 0)
if (metadata.Artists.Count > 0)
{
var allArtists = string.Join(", ", artists);
var allArtists = string.Join(", ", metadata.Artists);
sb.Append(allArtists);
}
@@ -114,11 +111,11 @@ public class SongVideoGenerator : ISongVideoGenerator
sb.Append(CultureInfo.InvariantCulture, $"\\N\"{metadata.Title}\"");
}
if (albumArtists.Count > 0)
if (metadata.AlbumArtists.Count > 0)
{
var allAlbumArtists = string.Join(
", ",
albumArtists.Filter(aa => !artists.Contains(aa)));
metadata.AlbumArtists.Filter(aa => !metadata.Artists.Contains(aa)));
sb.Append(CultureInfo.InvariantCulture, $"\\N{allAlbumArtists}");
}
+161 -145
View File
@@ -171,136 +171,125 @@ public class WatermarkSelector(
// check for playout item watermark
foreach (ChannelWatermark watermark in playoutItemWatermark)
{
// A custom watermark with no image at all is a bad-form-validation artifact, and it has always
// fallen THROUGH to the channel/global watermark rather than resolving to "no watermark". That
// stays true: unifying *resolution* (#510) must not change which watermark WINS.
if (watermark.ImageSource is ChannelWatermarkImageSource.Custom
&& string.IsNullOrWhiteSpace(watermark.Image))
switch (watermark.ImageSource)
{
logger.LogWarning(
"Watermark {Name} has custom image configured with no image; ignoring",
watermark.Name);
break;
}
// used for song progress overlay
case ChannelWatermarkImageSource.Resource:
string resourcePath = fileSystem.Path.Combine(
FileSystemLayout.ResourcesCacheFolder,
watermark.Image);
if (fileSystem.File.Exists(resourcePath))
{
return new WatermarkOptions(watermark, resourcePath, Option<int>.None);
}
logger.LogDebug("Watermark will come from playout item ({ImageSource})", watermark.ImageSource);
return ResolveWatermark(channel, watermark);
logger.LogWarning(
"Watermark resource no longer exists at {Path} and will be ignored",
resourcePath);
return None;
case ChannelWatermarkImageSource.Custom:
// bad form validation makes this possible
if (string.IsNullOrWhiteSpace(watermark.Image))
{
logger.LogWarning(
"Watermark {Name} has custom image configured with no image; ignoring",
watermark.Name);
break;
}
logger.LogDebug("Watermark will come from playout item (custom)");
string customPath = imageCache.GetPathForImage(
watermark.Image,
ArtworkKind.Watermark,
Option<int>.None);
if (fileSystem.File.Exists(customPath))
{
return new WatermarkOptions(watermark, customPath, None);
}
logger.LogWarning(
"Custom watermark no longer exists at {Path} and will be ignored",
customPath);
return None;
case ChannelWatermarkImageSource.ChannelLogo:
logger.LogDebug("Watermark will come from playout item (channel logo)");
return ChannelLogoWatermarkOptions(channel, watermark);
default:
throw new NotSupportedException("Unsupported watermark image source");
}
}
// check for channel watermark
if (channel.Watermark != null)
{
logger.LogDebug("Watermark will come from channel ({ImageSource})", channel.Watermark.ImageSource);
return ResolveWatermark(channel, channel.Watermark);
switch (channel.Watermark.ImageSource)
{
case ChannelWatermarkImageSource.Custom:
logger.LogDebug("Watermark will come from channel (custom)");
string customPath = imageCache.GetPathForImage(
channel.Watermark.Image,
ArtworkKind.Watermark,
Option<int>.None);
if (fileSystem.File.Exists(customPath))
{
return new WatermarkOptions(channel.Watermark, customPath, None);
}
logger.LogWarning(
"Custom watermark no longer exists at {Path} and will be ignored",
customPath);
return None;
case ChannelWatermarkImageSource.ChannelLogo:
logger.LogDebug("Watermark will come from channel (channel logo)");
return ChannelLogoWatermarkOptions(channel, channel.Watermark);
default:
throw new NotSupportedException("Unsupported watermark image source");
}
}
// check for global watermark
foreach (ChannelWatermark watermark in globalWatermark)
{
logger.LogDebug("Watermark will come from global ({ImageSource})", watermark.ImageSource);
return ResolveWatermark(channel, watermark);
switch (watermark.ImageSource)
{
case ChannelWatermarkImageSource.Custom:
logger.LogDebug("Watermark will come from global (custom)");
string customPath = imageCache.GetPathForImage(
watermark.Image,
ArtworkKind.Watermark,
Option<int>.None);
if (fileSystem.File.Exists(customPath))
{
return new WatermarkOptions(watermark, customPath, None);
}
logger.LogWarning(
"Custom watermark no longer exists at {Path} and will be ignored",
customPath);
return None;
case ChannelWatermarkImageSource.ChannelLogo:
logger.LogDebug("Watermark will come from global (channel logo)");
return ChannelLogoWatermarkOptions(channel, watermark);
default:
throw new NotSupportedException("Unsupported watermark image source");
}
}
return Option<WatermarkOptions>.None;
}
/// <summary>
/// The single place a <see cref="ChannelWatermark" /> becomes a renderable image path, shared by every
/// watermark source: the three precedence levels (playout item, channel, global) AND the deco path.
/// </summary>
/// <remarks>
/// Before #510 the deco path had its own copy of this switch that resolved paths *unchecked* — it handed
/// down a nonexistent file, an un-migrated external URL, and the generated-initials localhost URL. The
/// playout-item level checked all three sources; the channel and global levels checked
/// <c>Custom</c>/<c>ChannelLogo</c> and *threw* for <c>Resource</c> (no arm, so `default:`). So the same
/// channel could disagree with itself about whether a bug rendered, purely by how the watermark was
/// attached. Duplication is what let that drift happen (it existed in triplicate before #502), so there is
/// now one resolver. Exactly one piece of per-caller policy survives, and it lives in the CALLER rather
/// than here: a playout-item <c>Custom</c> watermark with a blank image falls through to channel/global
/// (see <see cref="GetWatermarkOptions" />). An unresolvable watermark resolves to "no on-screen bug",
/// never a dead path passed downstream: a dead LOCAL path could reach ffmpeg as a bare <c>-i</c> argument
/// via <c>CanUseFFmpegNativeWatermark</c>, which is materially worse than a skipped overlay.
/// <para>
/// Watermarks built OUTSIDE this selector are not covered — the song-progress overlay is constructed as a
/// <c>WatermarkOptions</c> directly by the streaming and troubleshooting handlers and is still unchecked
/// (#653).
/// </para>
/// </remarks>
private Option<WatermarkOptions> ResolveWatermark(Channel channel, ChannelWatermark watermark)
{
switch (watermark.ImageSource)
{
// NOT dead code and NOT only hand-edited rows: CreateWatermarkHandler/UpdateWatermarkHandler
// persist whatever ImageSource the request names, so a Resource watermark is creatable through
// the API -- always with Image = null, which is why the guard below is essential.
// Separately, the real song-progress overlay does NOT come through here: it is built directly as a
// WatermarkOptions by the streaming/troubleshooting handlers, which bypass this resolver and are
// still unchecked (#653).
case ChannelWatermarkImageSource.Resource:
// Image is NULL for every non-Custom watermark the API writes (CreateWatermarkHandler /
// UpdateWatermarkHandler both set `Image = null` unless ImageSource is Custom), so this guard is
// load-bearing, not defensive: Path.Combine(folder, null) throws ArgumentNullException, which
// would surface as a failed stream start rather than a missing overlay.
if (string.IsNullOrWhiteSpace(watermark.Image))
{
logger.LogWarning(
"Watermark {Name} uses a resource image but has no image name; ignoring",
watermark.Name);
return None;
}
string resourcePath = fileSystem.Path.Combine(
FileSystemLayout.ResourcesCacheFolder,
watermark.Image);
if (fileSystem.File.Exists(resourcePath))
{
return new WatermarkOptions(watermark, resourcePath, Option<int>.None);
}
logger.LogWarning(
"Watermark resource no longer exists at {Path} and will be ignored",
resourcePath);
return None;
case ChannelWatermarkImageSource.Custom:
// bad form validation makes this possible
if (string.IsNullOrWhiteSpace(watermark.Image))
{
logger.LogWarning(
"Watermark {Name} has custom image configured with no image; ignoring",
watermark.Name);
return None;
}
string customPath = imageCache.GetPathForImage(
watermark.Image,
ArtworkKind.Watermark,
Option<int>.None);
if (fileSystem.File.Exists(customPath))
{
return new WatermarkOptions(watermark, customPath, None);
}
logger.LogWarning(
"Custom watermark no longer exists at {Path} and will be ignored",
customPath);
return None;
case ChannelWatermarkImageSource.ChannelLogo:
return ChannelLogoWatermarkOptions(channel, watermark);
// deliberately loud: a newly-added image source must fail visibly rather than silently
// resolve to some neighbouring source's behavior
default:
throw new NotSupportedException("Unsupported watermark image source");
}
}
/// <summary>
/// Resolves a <see cref="ChannelWatermarkImageSource.ChannelLogo" /> watermark to a renderable path.
/// Since #510 this is reached from <see cref="ResolveWatermark" />, so all FOUR sources — the playout-item,
/// channel and global precedence levels AND the deco path — agree.
/// Resolves a <see cref="ChannelWatermarkImageSource.ChannelLogo" /> watermark to a renderable path,
/// shared by the playout-item, channel and global precedence levels so all three agree.
/// </summary>
/// <remarks>
/// As of #525 an external-URL logo is downloaded and cached at save time, so a URL path here can only
@@ -335,52 +324,79 @@ public class WatermarkSelector(
return None;
}
// With no logo artwork at all the only candidate is the generated-initials image, served over HTTP from
// ChannelLogoGenerator.GenerateChannelLogoUrl -- a URL that hardcodes localhost (issue #1, closed as a
// topology problem without removing the hardcode).
//
// Until #510 that URL WAS returned by the deco path, and it genuinely rendered: a live-E2E on a real
// transcoded frame confirmed the nameplate compositing through the graphics engine (the /iptv/logos/gen
// route sits on ArtworkController, which carries no auth filter, so the container-internal self-fetch
// succeeded). It never rendered at the three precedence levels. #510 resolved that split in favour of
// "no bug", because a render-time HTTP fetch inside stream startup is exactly what `graphics.channel-logo-caching`
// (#525) eliminated for logos -- so the fallback is now off everywhere rather than on for one caller.
// Reviving it properly means generating the image into the image cache so it resolves to a LOCAL path;
// that is deliberately out of scope here and tracked separately.
// with no logo artwork the only candidate is the generated-initials image, whose URL hardcodes
// localhost (ChannelLogoGenerator.GenerateChannelLogoUrl, issue #1). It has never rendered here and
// reviving it is deliberately deferred in docs/decisions.md, so it stays ignored.
logger.LogWarning(
"Channel {Channel} has no logo artwork; rendering without an on-screen bug. The generated-initials "
+ "fallback ({Url}) is deliberately not used by the render path",
channel.Number,
"Channel logo no longer exists at {Path} and will be ignored",
ChannelLogoGenerator.GenerateChannelLogoUrl(channel));
return None;
}
/// <summary>
/// Resolves the watermarks attached to a deco. Since #510 this shares <see cref="ResolveWatermark" />
/// with the three precedence levels rather than carrying its own unchecked copy of the same switch.
/// </summary>
/// <remarks>
/// Resolution is now identical to the precedence levels; what stays deco-specific is only WHICH
/// watermarks apply and whether they merge with or override the rest (handled in
/// <see cref="SelectWatermarks" />).
/// <para>
/// The routing PREDICATE is unchanged — <c>CanUseFFmpegNativeWatermark</c> still keys off the resolved
/// path alone and sends any URL to the graphics engine regardless of provenance. Its INPUT can change,
/// though: dropping an unresolvable watermark shortens this list, so a deco carrying one valid and one
/// missing permanent watermark now yields count 1 (ffmpeg-native) where it previously yielded count 2
/// (graphics engine). That is intended — the surviving watermark is a single valid permanent local image,
/// exactly what the native path is for — but it IS an observable routing change, not a no-op.
/// </para>
/// </remarks>
private List<WatermarkOptions> OptionsForWatermarks(Channel channel, IEnumerable<ChannelWatermark> watermarks)
{
var result = new List<WatermarkOptions>();
foreach (var watermark in watermarks)
{
result.AddRange(ResolveWatermark(channel, watermark));
result.AddRange(GetWatermarkOptions(channel, watermark));
}
return result;
}
private Option<WatermarkOptions> GetWatermarkOptions(Channel channel, ChannelWatermark watermark)
{
switch (watermark.ImageSource)
{
// used for song progress overlay
case ChannelWatermarkImageSource.Resource:
return new WatermarkOptions(
watermark,
Path.Combine(FileSystemLayout.ResourcesCacheFolder, watermark.Image),
Option<int>.None);
case ChannelWatermarkImageSource.Custom:
// bad form validation makes this possible
if (string.IsNullOrWhiteSpace(watermark.Image))
{
logger.LogWarning(
"Watermark {Name} has custom image configured with no image; ignoring",
watermark.Name);
break;
}
string customPath = imageCache.GetPathForImage(
watermark.Image,
ArtworkKind.Watermark,
Option<int>.None);
return new WatermarkOptions(
watermark,
customPath,
None);
case ChannelWatermarkImageSource.ChannelLogo:
// deliberately NOT ChannelLogoWatermarkOptions: the deco path has always passed its resolved
// path through unchecked, so #502's File.Exists defect never reached it and its *resolution*
// is unchanged here. Aligning its missing-file / no-artwork policy with the three precedence
// levels above is a behavior change beyond this fix — tracked in #510.
// Note this only scopes resolution: the ffmpeg-native-vs-graphics-engine routing in
// FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark keys off the resolved path alone, so a
// deco watermark resolving to a URL (an external logo, or the generated-initials URL below) is
// rerouted to the graphics engine like any other. That is intended: it is the URL-aware path.
string channelPath = ChannelLogoGenerator.GenerateChannelLogoUrl(channel);
Option<Artwork> maybeLogoArtwork =
Optional(channel.Artwork.Find(a => a.ArtworkKind == ArtworkKind.Logo));
foreach (var logoArtwork in maybeLogoArtwork)
{
channelPath = Artwork.IsExternalUrl(logoArtwork.Path)
? logoArtwork.Path
: imageCache.GetPathForImage(logoArtwork.Path, ArtworkKind.Logo, Option<int>.None);
}
return new WatermarkOptions(watermark, channelPath, None);
default:
throw new NotSupportedException("Unsupported watermark image source");
}
return Option<WatermarkOptions>.None;
}
}
@@ -4,7 +4,4 @@ public static class GraphicsElementDefaults
{
// Built-in "On Now / Next" text element; identity is by filename, never by user-editable Name.
public const string OnNowNextFileName = "on-now-next.yml";
// Display name only. Never use it for identity -- that is the filename above (#67 / #74).
public const string OnNowNextName = "On Now / Next";
}
@@ -31,28 +31,6 @@ public class TextGraphicsElement : BaseGraphicsElement
[YamlMember(Alias = "z_index", ApplyNamingConventions = false)]
public int? ZIndex { get; set; }
// Background box (ersatztv#732). Element-level, not per-style: the graphics engine renders one
// TextBlock into one bitmap, so a single box behind the whole element is the only shape the
// renderer can express. Unset background_color means no FILL; a border_color alone still draws
// an outlined box. With neither there is no box and no insets -- the pre-#732 geometry.
[YamlMember(Alias = "background_color", ApplyNamingConventions = false)]
public string BackgroundColor { get; set; }
[YamlMember(Alias = "background_opacity_percent", ApplyNamingConventions = false)]
public int? BackgroundOpacityPercent { get; set; }
[YamlMember(Alias = "background_padding", ApplyNamingConventions = false)]
public double? BackgroundPadding { get; set; }
[YamlMember(Alias = "background_corner_radius", ApplyNamingConventions = false)]
public double? BackgroundCornerRadius { get; set; }
[YamlMember(Alias = "border_color", ApplyNamingConventions = false)]
public string BorderColor { get; set; }
[YamlMember(Alias = "border_width", ApplyNamingConventions = false)]
public double? BorderWidth { get; set; }
public List<StyleDefinition> Styles { get; set; } = [];
[YamlMember(Alias = "base_style", ApplyNamingConventions = false)]
@@ -85,46 +85,19 @@ public static class AlternateScheduleSelector
}
}
// These three are NULLABLE single-column primitive collections, and a runtime null IS
// reachable (ersatztv#823, measured against a real TvContext on SQLite and MySQL 8.4): EF does
// NOT invoke the value converter for a NULL column, so it materializes as CLR null rather than
// through IntCollectionValueConverter's null-to-empty branch, which never runs on this path.
// Unguarded, each .Contains below throws NullReferenceException.
//
// A NULL reads as UNRESTRICTED -- the All*() sets -- NOT as empty. This is the whole semantic
// question and it is decided by the one NULL that is reachable WITHOUT any code writing one:
// Sqlite's 20240113140741_Add_PlayoutTemplate_DaysOfMonth adds DaysOfMonth with
// `nullable: true` and NO defaultValue, so a PlayoutTemplate row inserted before it holds NULL
// and, by construction, had NO day-of-month restriction. Reading that as empty would INVERT
// the row's meaning and silently stop the template applying at all. All*() preserves it, and
// it is how "no restriction recorded" is already represented elsewhere in this domain
// (GetPlayoutAlternateSchedulesHandler, PreviewBlockPlayoutHandler). Note what does NOT decide
// it: the API request records normalize an omitted field with `?? []`, but that is a client
// omitting a field on a WRITE and says nothing about what a legacy database NULL meant.
//
// Guarded at the READ SITE, into locals, and NEVER assigned back onto `item`: the property IS
// the column value, so writing the guard back would flip a tracked entry to Modified and
// persist the substituted collection over the NULL
// (`media.nullable-primitive-collection-mutation`). The matching substitution happens at the
// entity->DTO boundary in the two Mapper.ProjectToViewModel overloads, so the SPA's
// appliesToDate -- an exact port of this method -- previews what this actually schedules.
ICollection<DayOfWeek> itemDaysOfWeek = item.DaysOfWeek ?? AllDaysOfWeek();
ICollection<int> itemDaysOfMonth = item.DaysOfMonth ?? AllDaysOfMonth();
ICollection<int> itemMonthsOfYear = item.MonthsOfYear ?? AllMonthsOfYear();
bool daysOfWeek = itemDaysOfWeek.Contains(date.DayOfWeek);
bool daysOfWeek = item.DaysOfWeek.Contains(date.DayOfWeek);
if (!daysOfWeek)
{
continue;
}
bool daysOfMonth = itemDaysOfMonth.Contains(date.Day);
bool daysOfMonth = item.DaysOfMonth.Contains(date.Day);
if (!daysOfMonth)
{
continue;
}
bool monthOfYear = itemMonthsOfYear.Contains(date.Month);
bool monthOfYear = item.MonthsOfYear.Contains(date.Month);
if (monthOfYear)
{
return item;
@@ -2,7 +2,6 @@ using System;
using System.Collections.Generic;
using System.Text.RegularExpressions;
using ErsatzTV.FFmpeg.Capabilities;
using ErsatzTV.FFmpeg.Filter;
using ErsatzTV.FFmpeg.Format;
using ErsatzTV.FFmpeg.OutputFormat;
using ErsatzTV.FFmpeg.Pipeline;
@@ -21,7 +20,6 @@ namespace ErsatzTV.FFmpeg.Tests.Pipeline;
public class QsvPipelineBuilderTests
{
private readonly ILogger _logger = Substitute.For<ILogger>();
private Option<SubtitleInputFile> _lastSubtitleInputFile;
[Test]
public void Qsv_PreferNativeDecoder_Should_Decode_Via_Vaapi_To_Software_Then_Qsv_Encode()
@@ -124,142 +122,6 @@ public class QsvPipelineBuilderTests
command.ShouldContain("hwupload=extra_hw_frames=128");
}
// ersatztv#505. Measured on the deployed FFmpeg 8.1.2 / iHD 25.1.4 / UHD 630: a graph ending in
// "vpp_qsv=tonemap=1" returns a frame that is BYTE-IDENTICAL (same md5) to the same graph with
// no tonemap step at all — QSV VPP tonemapping needs Gen11+, and pre-Gen11 iHD ignores it with
// no warning. So the assertion that matters is not "GPU tonemap is used" but "the silent no-op
// is never emitted", which is why every case below asserts its absence.
[TestCase(true, true)]
[TestCase(true, false)]
[TestCase(false, true)]
[TestCase(false, false)]
public void Qsv_Hdr_Should_Never_Emit_The_Silently_No_Op_Vpp_Qsv_Tonemap(
bool preferNativeDecoder,
bool deinterlace)
{
string command = BuildHdrAndPrint(preferNativeDecoder, deinterlace);
// assert against the vpp_qsv OPTION, not the bare substring: "tonemap=1" alone could match
// an unrelated filter, and would miss an equivalent spelling
command.ShouldNotContain("vpp_qsv=tonemap");
Regex.IsMatch(command, @"vpp_qsv=[^,\s]*tonemap")
.ShouldBeFalse(command);
}
[Test]
public void Qsv_Hdr_NativeDecode_Should_Tonemap_On_The_Gpu_Via_OpenCL()
{
string command = BuildHdrAndPrint(preferNativeDecoder: true);
// upload to VA-API explicitly: "-filter_hw_device hw" points at the QSV device, so a bare
// hwupload here would land on a QSV surface, which cannot be mapped to OpenCL
command.ShouldContain("hwupload=derive_device=vaapi");
// scale BEFORE tonemap, on the VA-API device (tonemapping full-size costs ~50% more wall
// clock than the software tonemap this replaces)
int scaleAt = command.IndexOf("scale_vaapi", StringComparison.Ordinal);
int tonemapAt = command.IndexOf("tonemap_opencl", StringComparison.Ordinal);
scaleAt.ShouldBeGreaterThan(-1, command);
tonemapAt.ShouldBeGreaterThan(-1, command);
scaleAt.ShouldBeLessThan(tonemapAt, command);
// no vpp_qsv scale on this path — a QSV surface could not reach OpenCL afterwards
command.ShouldNotContain("vpp_qsv");
// and no CPU tonemap, which is the cost ersatztv#505 was filed about
command.ShouldNotContain("zscale");
// the hardware filters strip color info, so the output has to be re-tagged bt709 — without
// this the picture is tonemapped but still ANNOUNCES bt2020 primaries, and the player
// converts it a second time (verified against ffprobe on the Intel host)
command.ShouldContain("all=bt709");
command.ShouldContain("h264_qsv");
// pin the exact graph measured on the Intel host, in order — the assertions above would
// all still pass with setFormat off, hwdownload dropped, or the wrong tonemap output
// format, any of which breaks the validated command
command.ShouldContain(
"format=nv12|p010le|vaapi,hwupload=derive_device=vaapi," +
"scale_vaapi=1280:720:force_divisible_by=2:format=p010,setsar=1," +
"hwmap=derive_device=opencl,tonemap_opencl=tonemap=linear:format=nv12," +
"hwdownload,format=nv12");
}
[Test]
public void Qsv_Hdr_Should_Retag_Bt709_Even_When_Color_Normalization_Is_Disabled()
{
// a tonemap converts the PIXELS to SDR, so the stream must stop announcing bt2020 whether
// or not the profile asks for color normalization — otherwise the player converts twice
string command = BuildAndPrint(preferNativeDecoder: true, hdr: true, normalizeColors: false);
command.ShouldContain("tonemap_opencl");
command.ShouldContain("all=bt709");
}
[Test]
public void Qsv_Hdr_Anamorphic_Should_Fall_Back_To_Software_Tonemap()
{
// ScaleVaapiFilter multiplies by ffmpeg's runtime `sar` instead of the SAR VideoStream
// calculates, so anamorphic sources keep the software tonemap they already had
string command = BuildAndPrint(preferNativeDecoder: true, hdr: true, anamorphic: true);
command.ShouldContain("zscale");
command.ShouldNotContain("tonemap_opencl");
}
[Test]
public void Qsv_Hdr_With_Image_Subtitle_Should_Scale_The_Subtitle_To_Match_The_Video()
{
// the video is scaled by ScaleVaapiFilter on this path; if the subtitle-scaling predicate
// does not recognize it, the burned-in subtitle canvas stays at source resolution
string command = BuildAndPrint(preferNativeDecoder: true, hdr: true, imageSubtitle: true);
command.ShouldContain("scale_vaapi");
var subtitleSteps = new List<IPipelineFilterStep>();
foreach (SubtitleInputFile subtitle in _lastSubtitleInputFile)
{
subtitleSteps.AddRange(subtitle.FilterSteps);
}
subtitleSteps.ShouldContain(s => s is ScaleImageFilter, "subtitle canvas was never resized");
}
[TestCase(true, true, TestName = "Qsv_Hdr_Interlaced_Falls_Back_To_Software_Tonemap")]
[TestCase(false, false, TestName = "Qsv_Hdr_QsvDecode_Falls_Back_To_Software_Tonemap")]
public void Qsv_Hdr_Should_Fall_Back_To_Software_Tonemap_When_Frames_Cannot_Reach_OpenCL(
bool preferNativeDecoder,
bool deinterlace)
{
// both cases put frames on a QSV surface before the tonemap would run (deinterlace_qsv, or
// the QSV decoder itself), and a QSV surface maps to neither OpenCL nor VA-API. Slower on
// the CPU, but correct — unlike the vpp_qsv no-op this replaces.
string command = BuildHdrAndPrint(preferNativeDecoder, deinterlace);
command.ShouldContain("zscale");
command.ShouldNotContain("tonemap_opencl");
}
[Test]
public void Qsv_Hdr_Should_Fall_Back_To_Software_Tonemap_Without_The_OpenCL_Filter()
{
// an ffmpeg build with no tonemap_opencl must not silently skip tonemapping
string command = BuildAndPrint(preferNativeDecoder: true, hdr: true, hasOpenClTonemap: false);
command.ShouldContain("zscale");
command.ShouldNotContain("tonemap_opencl");
command.ShouldNotContain("tonemap=1");
}
private string BuildHdrAndPrint(bool preferNativeDecoder, bool deinterlace = false) =>
BuildAndPrint(
preferNativeDecoder,
maybeExtraHardwareFrames: default,
deinterlace ? ScanKind.Interlaced : ScanKind.Progressive,
deinterlace,
hdr: true);
private string BuildInterlacedAndPrint(Option<int> maybeExtraHardwareFrames = default) =>
BuildAndPrint(
preferNativeDecoder: true,
@@ -271,40 +133,21 @@ public class QsvPipelineBuilderTests
bool preferNativeDecoder,
Option<int> maybeExtraHardwareFrames = default,
ScanKind scanKind = ScanKind.Progressive,
bool deinterlace = false,
bool hdr = false,
bool hasOpenClTonemap = true,
bool normalizeColors = true,
bool anamorphic = false,
bool imageSubtitle = false)
bool deinterlace = false)
{
(VideoInputFile videoInputFile, AudioInputFile audioInputFile, FFmpegState ffmpegState, FrameState desiredState) =
BuildQsvH264Pipeline(preferNativeDecoder, scanKind, deinterlace, hdr, anamorphic);
BuildQsvH264Pipeline(preferNativeDecoder, scanKind, deinterlace);
ffmpegState = ffmpegState with { MaybeQsvExtraHardwareFrames = maybeExtraHardwareFrames };
if (!normalizeColors)
{
desiredState = desiredState with { ColorsAreBt709 = false };
}
Option<SubtitleInputFile> subtitleInputFile = imageSubtitle
? new SubtitleInputFile(
"/tmp/whatever.mkv",
new List<MediaStream> { new(2, "hdmv_pgs_subtitle", StreamKind.Subtitle) },
SubtitleMethod.Burn)
: Option<SubtitleInputFile>.None;
var builder = new QsvPipelineBuilder(
hasOpenClTonemap
? new DefaultFFmpegCapabilities(FFmpegKnownFilter.TonemapOpenCL.Name)
: new DefaultFFmpegCapabilities(),
new DefaultFFmpegCapabilities(),
new DefaultHardwareCapabilities(),
HardwareAccelerationMode.Qsv,
videoInputFile,
audioInputFile,
None,
subtitleInputFile,
None,
None,
Option<GraphicsEngineInput>.None,
"",
@@ -313,34 +156,26 @@ public class QsvPipelineBuilderTests
FFmpegPipeline result = builder.Build(ffmpegState, desiredState);
// the subtitle input's filter steps never reach CommandGenerator, so expose them for the
// subtitle-scaling assertion
_lastSubtitleInputFile = subtitleInputFile;
return PrintCommand(videoInputFile, audioInputFile, None, None, None, result);
}
private static (VideoInputFile, AudioInputFile, FFmpegState, FrameState) BuildQsvH264Pipeline(
bool preferNativeDecoder,
ScanKind scanKind,
bool deinterlace,
bool hdr = false,
bool anamorphic = false)
bool deinterlace)
{
// the real trigger: HEVC Main10, BT.2020 primaries, smpte2084 (PQ) transfer — matching the
// prod sources this was validated against on jazz
var videoInputFile = new VideoInputFile(
"/tmp/whatever.mkv",
new List<VideoStream>
{
new(
0,
hdr ? VideoFormat.Hevc : VideoFormat.H264,
VideoFormat.H264,
VideoProfile.Main,
hdr ? new PixelFormatYuv420P10Le() : new PixelFormatYuv420P(),
hdr ? new ColorParams("tv", "bt2020nc", "smpte2084", "bt2020") : ColorParams.Default,
hdr ? new FrameSize(3840, 1608) : new FrameSize(1920, 1080),
anamorphic ? "4:3" : "1:1",
new PixelFormatYuv420P(),
ColorParams.Default,
new FrameSize(1920, 1080),
"1:1",
"16:9",
FrameRate.DefaultFrameRate,
false,
@@ -377,9 +212,7 @@ public class QsvPipelineBuilderTests
2000,
4000,
90_000,
// HDR output is normalized to bt709, which is what makes the colorspace filter
// reachable at all; leaving this false would hide the output-tagging assertions
hdr,
false,
deinterlace);
var ffmpegState = new FFmpegState(
@@ -437,11 +270,11 @@ public class QsvPipelineBuilderTests
return command;
}
public class DefaultFFmpegCapabilities(params string[] filters) : FFmpegCapabilities(
public class DefaultFFmpegCapabilities() : FFmpegCapabilities(
string.Empty,
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(filters),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>());
@@ -593,143 +593,7 @@ public class PipelineBuilderBaseTests
command.ShouldNotContain("-readrate_initial_burst");
}
[Test]
public void Realtime_Input_Should_Catch_Up_When_Option_Is_Supported()
{
string command = BuildRealtimeCommand(new CatchupCapableFFmpegCapabilities());
// -readrate paces an input off its furthest-behind stream, so a sparse stream sharing the
// input pins throughput below realtime; catchup lets it recover (ersatztv#726). anchor on
// the input path so this can't be satisfied by some other input carrying the option
// this overlaps Bitmap_Subtitle_Burn_In_... by design: that one pins the #726 MECHANISM on a
// bitmap pipeline, this one pins the plain no-subtitle shape plus the uniqueness guard below
command.ShouldContain("-readrate 1.05 -readrate_initial_burst 8 -readrate_catchup 6.0 -i /tmp/whatever.mkv");
Regex.Matches(command, Regex.Escape("-readrate_catchup 6.0")).Count.ShouldBe(1);
}
[Test]
public void Realtime_Input_Should_Not_Catch_Up_A_Still_Image()
{
// mirrors the burst's still-image exclusion (ersatztv#350): the video input takes no readrate
// at all, so catchup would only reach the separate audio input and run it ahead of a graph
// that the realtime filter is already pacing. pinned so the divergence can't reappear silently
string command = BuildRealtimeCommand(new CatchupCapableFFmpegCapabilities(), stillImage: true);
// the positive anchor keeps this from passing vacuously if the helper ever stops
// producing a realtime audio input at all
command.ShouldContain("-readrate 1.05");
command.ShouldNotContain("-readrate_catchup");
}
[Test]
public void Realtime_Input_Should_Not_Catch_Up_When_Option_Is_Unsupported()
{
// an older binary silently keeps today's behavior rather than failing to start
string command = BuildRealtimeCommand(new BurstCapableFFmpegCapabilities());
// the positive anchor keeps this from passing vacuously if the helper ever stops
// producing a realtime input at all
command.ShouldContain("-readrate 1.05");
command.ShouldNotContain("-readrate_catchup");
}
[Test]
public void Concat_Should_Never_Catch_Up()
{
// concat reads already-written segments from the running segmenter at a flat 1.0; it has no
// sparse stream to lag on, and letting it catch up would gallop through the segments
var concatInputFile = new ConcatInputFile("http://localhost:8080/ffmpeg/concat/1", new FrameSize(1920, 1080));
var builder = new SoftwarePipelineBuilder(
new CatchupCapableFFmpegCapabilities(),
HardwareAccelerationMode.None,
None,
None,
None,
None,
concatInputFile,
Option<GraphicsEngineInput>.None,
"",
"",
_logger);
FFmpegPipeline result = builder.Concat(concatInputFile, FFmpegState.Concat(false, "Some Channel"));
string command = PrintCommand(None, None, None, concatInputFile, None, result);
command.ShouldContain("-readrate 1.0");
command.ShouldNotContain("-readrate_catchup");
}
[Test]
public void Bitmap_Subtitle_Burn_In_Should_Catch_Up_On_The_Shared_Video_Input()
{
// THE #726 regression test. an embedded bitmap subtitle is read through the SAME -i as the
// video (SubtitleInputFile carries the video's path and resolves to a stream specifier on
// that input), and being sparse it drags that input's pacing down to ~0.53x realtime.
// this must be built on a BITMAP subtitle: a text subtitle is fetched by the libass filter
// outside the demuxer, so the same assertions would pass vacuously while the bug is present.
string command = BuildRealtimeCommand(new CatchupCapableFFmpegCapabilities(), imageSubtitle: true);
// the mechanism itself: subtitle stream 2 resolves onto input 0 -- the VIDEO's input -- so it
// is read through the throttled demuxer that catchup is being applied to. if the subtitle
// ever moves to an input of its own this label changes and the test fails, which is the point
command.ShouldContain("[0:0][0:2]overlay");
// ...so the catchup has to be on that input
command.ShouldContain("-readrate 1.05 -readrate_initial_burst 8 -readrate_catchup 6.0 -i /tmp/whatever.mkv");
}
// ersatztv#735: the pacing values became operator-tunable profile fields. these pin that a
// configured value actually reaches the command line -- the defaults above are the OTHER half
// of the same guard, and they are what an unset profile still gets
[Test]
public void Realtime_Input_Should_Use_A_Configured_ReadRate_And_Catchup()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 1.5,
readRateCatchup: 4.0);
command.ShouldContain("-readrate 1.5 -readrate_initial_burst 8 -readrate_catchup 4.0 -i /tmp/whatever.mkv");
command.ShouldNotContain("-readrate 1.05");
command.ShouldNotContain("-readrate_catchup 6.0");
}
// the write path rejects an out-of-range value with a 422, so this only fires for a row written
// out of band -- but FFmpeg must never see the unbounded value either way
[Test]
public void Realtime_Input_Should_Clamp_An_Out_Of_Range_ReadRate()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 9.0,
readRateCatchup: 0.1);
// 9.0 clamps to the 2.0 ceiling, and 0.1 is raised to the resolved base rate, because a
// catchup below it could never let a lagging input recover
command.ShouldContain("-readrate 2.0 -readrate_initial_burst 8 -readrate_catchup 2.0 -i /tmp/whatever.mkv");
}
// ...and the catchup CEILING isolated from the base rate, which the case above cannot show:
// there both clamps land on the same 2.0, so either one alone would satisfy it
[Test]
public void Realtime_Input_Should_Clamp_An_Out_Of_Range_ReadRateCatchup()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 1.2,
readRateCatchup: 15.0);
command.ShouldContain("-readrate 1.2 -readrate_initial_burst 8 -readrate_catchup 10.0 -i /tmp/whatever.mkv");
}
private string BuildRealtimeCommand(
IFFmpegCapabilities capabilities,
bool stillImage = false,
bool imageSubtitle = false,
Option<double> readRate = default,
Option<double> readRateCatchup = default)
private string BuildRealtimeCommand(IFFmpegCapabilities capabilities, bool stillImage = false)
{
var videoInputFile = new VideoInputFile(
"/tmp/whatever.mkv",
@@ -794,9 +658,7 @@ public class PipelineBuilderBaseTests
false,
false,
"clip",
false,
MaybeReadRate: readRate,
MaybeReadRateCatchup: readRateCatchup);
false);
// a *separate* audio input matters here: for a still image the video input takes no readrate
// at all, so only a distinct audio input can prove the burst was suppressed (this is the
@@ -814,22 +676,13 @@ public class PipelineBuilderBaseTests
AudioFilter.None,
Option<double>.None));
// an embedded bitmap subtitle carries the VIDEO's path, which is how it ends up sharing the
// video's single throttled -i rather than getting one of its own (ersatztv#726)
Option<SubtitleInputFile> subtitleInputFile = imageSubtitle
? new SubtitleInputFile(
"/tmp/whatever.mkv",
new List<MediaStream> { new(2, "dvdsub", StreamKind.Subtitle) },
SubtitleMethod.Burn)
: Option<SubtitleInputFile>.None;
var builder = new SoftwarePipelineBuilder(
capabilities,
HardwareAccelerationMode.None,
videoInputFile,
audioInputFile,
None,
subtitleInputFile,
None,
None,
Option<GraphicsEngineInput>.None,
"",
@@ -882,19 +735,4 @@ public class PipelineBuilderBaseTests
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string> { FFmpegKnownOption.ReadrateInitialBurst.Name },
new System.Collections.Generic.HashSet<string>());
// a binary new enough for -readrate_catchup also has -readrate_initial_burst, so this models a
// real ffmpeg rather than an impossible catchup-without-burst one
public class CatchupCapableFFmpegCapabilities() : FFmpegCapabilities(
string.Empty,
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>
{
FFmpegKnownOption.ReadrateInitialBurst.Name,
FFmpegKnownOption.ReadrateCatchup.Name
},
new System.Collections.Generic.HashSet<string>());
}
@@ -13,15 +13,8 @@ public record FFmpegKnownOption
// ffmpeg 6.1+; lets a readrate-throttled input read flat out for an initial window
public static FFmpegKnownOption ReadrateInitialBurst => new("readrate_initial_burst");
// ffmpeg 8.0+ (added 2025-02-15 in 6232f416b, first released in 8.0); lets a readrate-throttled
// input read faster than its readrate *while it is behind*, so a sparse stream sharing that
// input cannot pin throughput below realtime (ersatztv#726). verified present in 8.1.2, the
// pinned base image — note this is NEWER than 7.1, so it is detected at runtime, never assumed
public static FFmpegKnownOption ReadrateCatchup => new("readrate_catchup");
public static IList<string> AllOptions =>
[
ReadrateInitialBurst.Name,
ReadrateCatchup.Name
ReadrateInitialBurst.Name
];
}
+1 -46
View File
@@ -28,9 +28,7 @@ public record FFmpegState(
bool IsHdrTonemap,
string TonemapAlgorithm,
bool IsTroubleshooting,
bool QsvPreferNativeDecoder = false,
Option<double> MaybeReadRate = default,
Option<double> MaybeReadRateCatchup = default)
bool QsvPreferNativeDecoder = false)
{
// the QSV upload pool needs headroom for the frames in flight through the filter graph.
// extra_hw_frames=0 leaves none, so any input that is not throttled exhausts it: the graph
@@ -44,49 +42,6 @@ public record FFmpegState(
public int QsvExtraHardwareFrames =>
Math.Max(MaybeQsvExtraHardwareFrames.IfNone(MinimumQsvExtraHardwareFrames), MinimumQsvExtraHardwareFrames);
// realtime pacing. an unset profile keeps the values these constants name, which are the ones
// the pipeline hardcoded before they became configurable (ersatztv#735)
public const double DefaultReadRate = 1.05;
public const double DefaultStreamCopyReadRate = 1.0;
// how fast a LAGGING realtime input may read until it is level again. measured on the #726
// repro (embedded dvd_subtitle -> overlay, QSV encode): 1.05 alone sustains 0.53x, catchup 2.0
// reaches 0.711x, and 6.0 restores the full 1.067x that the same pipeline achieves with no
// subtitle at all. 20.0 also measures 1.067x — i.e. the value is not a throughput dial above
// the point where the input catches up, so 6.0 is chosen as the smallest measured-sufficient
// ceiling rather than the largest that works (ersatztv#726)
public const double DefaultReadRateCatchup = 6.0;
// below realtime the process reads slower than a live client consumes and the channel stalls;
// ersatztv#726 is that failure, measured at an effective 0.53x. the ceiling is a CHOSEN bound,
// not a measured cliff: it exists so the field cannot be used to effectively disable pacing,
// which is the configuration ersatztv#529 measured to produce zero segments on a QSV pipeline
public const double MinimumReadRate = 1.0;
public const double MaximumReadRate = 2.0;
// catchup is a ceiling that applies only WHILE an input is behind, so it is bounded more
// loosely than the base rate; the same chosen-not-measured caveat applies to the ceiling.
// the FLOOR is only a write-path bound: at render time the resolved base rate is always at
// least MinimumReadRate, so Math.Max below already dominates it
public const double MinimumReadRateCatchup = 1.0;
public const double MaximumReadRateCatchup = 10.0;
// clamped for the same reason QsvExtraHardwareFrames is: a row written out of band (or before
// the write path validated the field) must not reach FFmpeg unbounded. the write path rejects
// an out-of-range value with a 422 naming the bound, so this is belt-and-braces, not the
// primary guard (ersatztv#735)
public double ReadRateFor(bool isStreamCopy) =>
MaybeReadRate.Match(
configured => Math.Clamp(configured, MinimumReadRate, MaximumReadRate),
() => isStreamCopy ? DefaultStreamCopyReadRate : DefaultReadRate);
// a catchup rate below the base rate cannot let a lagging input recover, so the resolved base
// rate is its real floor — no separate lower clamp, which would be unreachable behind this Max
public double ReadRateCatchupFor(bool isStreamCopy) =>
Math.Max(
Math.Min(MaybeReadRateCatchup.IfNone(DefaultReadRateCatchup), MaximumReadRateCatchup),
ReadRateFor(isStreamCopy));
public static FFmpegState Concat(bool saveReport, string channelName) =>
new(
saveReport,
@@ -1,30 +0,0 @@
using ErsatzTV.FFmpeg.Format;
namespace ErsatzTV.FFmpeg.Filter.Qsv;
// vpp_qsv=tonemap=1 is a SILENT no-op on pre-Gen11 Intel graphics (ersatztv#505): the frame comes
// back untouched, byte for byte, with no warning and no error, so HDR content ships untonemapped.
// The QSV pipeline therefore tonemaps through OpenCL, the same route VaapiPipelineBuilder takes.
//
// This filter always runs on VA-API frames and hands SOFTWARE frames back: QSV surfaces cannot be
// mapped to OpenCL ("Media sharing must be enabled on context creation") and cannot be mapped to
// VA-API either (hwmap returns -38, function not implemented), so the only route from a QSV-encode
// profile into tonemap_opencl is to stay on the VA-API device the QSV device was derived from.
public class TonemapOpenClQsvFilter(FFmpegState ffmpegState, IPixelFormat desiredPixelFormat) : BaseFilter
{
public override string Filter =>
$"hwmap=derive_device=opencl,tonemap_opencl=tonemap={ffmpegState.TonemapAlgorithm}:format={OutputFormat}," +
$"hwdownload,format={OutputFormat}";
private string OutputFormat =>
desiredPixelFormat.BitDepth == 10 ? FFmpegFormat.P010LE : FFmpegFormat.NV12;
public override FrameState NextState(FrameState currentState) =>
currentState with
{
FrameDataLocation = FrameDataLocation.Software,
PixelFormat = desiredPixelFormat.BitDepth == 10
? new PixelFormatP010()
: new PixelFormatNv12(desiredPixelFormat.Name)
};
}
@@ -0,0 +1,12 @@
namespace ErsatzTV.FFmpeg.Filter.Qsv;
public class TonemapQsvFilter : BaseFilter
{
public override string Filter => "vpp_qsv=tonemap=1";
public override FrameState NextState(FrameState currentState) =>
currentState with
{
FrameDataLocation = FrameDataLocation.Hardware
};
}
@@ -1,28 +1,16 @@
namespace ErsatzTV.FFmpeg.Filter.Vaapi;
namespace ErsatzTV.FFmpeg.Filter.Vaapi;
public class HardwareUploadVaapiFilter : BaseFilter
{
private readonly bool _deriveDevice;
private readonly bool _setFormat;
// deriveDevice matters only where the graph's default filter device is NOT the VA-API one: the
// QSV pipeline sets "-filter_hw_device hw" (the QSV device), so a bare hwupload there would
// upload to QSV instead of VA-API. It defaults to false so the VA-API pipeline, whose default
// filter device already is VA-API, keeps emitting exactly what it emitted before.
public HardwareUploadVaapiFilter(bool setFormat, bool deriveDevice = false)
{
_setFormat = setFormat;
_deriveDevice = deriveDevice;
}
public HardwareUploadVaapiFilter(bool setFormat) => _setFormat = setFormat;
public override string Filter
public override string Filter => _setFormat switch
{
get
{
string hwupload = _deriveDevice ? "hwupload=derive_device=vaapi" : "hwupload";
return _setFormat ? $"format=nv12|p010le|vaapi,{hwupload}" : hwupload;
}
}
false => "hwupload",
true => "format=nv12|p010le|vaapi,hwupload"
};
public override FrameState NextState(FrameState currentState) =>
currentState with { FrameDataLocation = FrameDataLocation.Hardware };
@@ -3,11 +3,10 @@ using ErsatzTV.FFmpeg.Environment;
namespace ErsatzTV.FFmpeg.InputOption;
public class ReadrateInputOption(double readRate, Option<int> initialBurstSeconds, Option<double> catchupReadRate)
: IInputOption
public class ReadrateInputOption(double readRate, Option<int> initialBurstSeconds) : IInputOption
{
public ReadrateInputOption(double readRate)
: this(readRate, Option<int>.None, Option<double>.None)
: this(readRate, Option<int>.None)
{
}
@@ -31,17 +30,6 @@ public class ReadrateInputOption(double readRate, Option<int> initialBurstSecond
result.Add(burst.ToString(CultureInfo.InvariantCulture));
}
// -readrate paces the WHOLE input off its furthest-behind stream, so one sparse stream
// (an embedded PGS/DVD bitmap subtitle feeding the overlay) drags the video down with it
// and output collapses to ~0.53x realtime. catchup lets a lagging input read faster until
// it is level again; it is a ceiling that only applies WHILE behind, never a target, so
// caught-up input still paces at readRate and cannot race ahead (ersatztv#726)
foreach (double catchup in catchupReadRate)
{
result.Add("-readrate_catchup");
result.Add(catchup.ToString("0.0####", CultureInfo.InvariantCulture));
}
return result.ToArray();
}
@@ -652,7 +652,7 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
}
//SetStillImageInfiniteLoop(videoInputFile, videoStream, ffmpegState);
SetRealtimeInput(videoInputFile, ffmpegState, desiredState);
SetRealtimeInput(videoInputFile, desiredState);
SetInfiniteLoop(videoInputFile, videoStream, ffmpegState, desiredState);
SetFrameRateOutput(desiredState, pipelineSteps);
SetVideoTrackTimescaleOutput(desiredState, pipelineSteps);
@@ -847,17 +847,14 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
}
}
private void SetRealtimeInput(VideoInputFile videoInputFile, FFmpegState ffmpegState, FrameState desiredState)
private void SetRealtimeInput(VideoInputFile videoInputFile, FrameState desiredState)
{
if (videoInputFile.StreamInputKind is StreamInputKind.Live || !desiredState.Realtime)
{
return;
}
// both defaults and both bounds live on FFmpegState, beside the profile fields that
// override them, so the pacing contract is readable in one place (ersatztv#735)
bool isStreamCopy = desiredState.VideoFormat == VideoFormat.Copy;
double readRate = ffmpegState.ReadRateFor(isStreamCopy);
double readRate = desiredState.VideoFormat == VideoFormat.Copy ? 1.0 : 1.05;
// without a burst, the readrate throttle applies from the very first read, so the first
// segment cannot be written faster than ~realtime and every start pays a multi-second wait.
@@ -874,26 +871,8 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
? InitialBurstSeconds
: Option<int>.None;
// -readrate paces an input off its furthest-behind stream. an embedded bitmap subtitle is
// read through the SAME -i as the video (its SubtitleInputFile carries the video's path and
// resolves to a stream specifier on that input), and being sparse it falls further behind
// every second, dragging video throughput to ~0.53x — well under the 1.0x a live client
// consumes at. catchup lets the lagging input recover instead of pinning the whole process.
// applied to every realtime input, not just subtitle pipelines: it is inert unless an input
// is actually behind, and any sparse stream can cause this (ersatztv#726).
//
// a still image is excluded for the SAME reason the burst above excludes it: its video input
// takes no readrate at all, so this would reach only the separate audio input and let it run
// ahead of the video, which is exactly what #350 declined. for a non-still-image item both
// inputs carry identical options, so the symmetry is preserved there. and an image-based
// subtitle always rides the video path, so this shape cannot suffer the starvation anyway
Option<double> catchupReadRate =
!isStillImage && _ffmpegCapabilities.HasOption(FFmpegKnownOption.ReadrateCatchup)
? ffmpegState.ReadRateCatchupFor(isStreamCopy)
: Option<double>.None;
_audioInputFile.Iter(a => a.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds, catchupReadRate)));
videoInputFile.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds, catchupReadRate));
_audioInputFile.Iter(a => a.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds)));
videoInputFile.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds));
}
protected static void SetStillImageLoop(
+22 -164
View File
@@ -6,7 +6,6 @@ using ErsatzTV.FFmpeg.Encoder.Qsv;
using ErsatzTV.FFmpeg.Environment;
using ErsatzTV.FFmpeg.Filter;
using ErsatzTV.FFmpeg.Filter.Qsv;
using ErsatzTV.FFmpeg.Filter.Vaapi;
using ErsatzTV.FFmpeg.Format;
using ErsatzTV.FFmpeg.GlobalOption.HardwareAcceleration;
using ErsatzTV.FFmpeg.InputOption;
@@ -19,7 +18,6 @@ namespace ErsatzTV.FFmpeg.Pipeline;
public class QsvPipelineBuilder : SoftwarePipelineBuilder
{
private readonly IFFmpegCapabilities _ffmpegCapabilities;
private readonly IHardwareCapabilities _hardwareCapabilities;
private readonly ILogger _logger;
@@ -48,7 +46,6 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
fontsFolder,
logger)
{
_ffmpegCapabilities = ffmpegCapabilities;
_hardwareCapabilities = hardwareCapabilities;
_logger = logger;
}
@@ -218,31 +215,12 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
};
}
// HDR has to be tonemapped through OpenCL on the VA-API device (ersatztv#505); when that is
// the plan the downscale has to happen in scale_vaapi rather than vpp_qsv, because a QSV
// surface can be mapped neither to OpenCL nor back to VA-API. Decided once, up front, so
// the scale and tonemap steps cannot disagree about which device the frames are on.
bool useOpenClTonemap = UseOpenClTonemap(videoStream, context, ffmpegState, currentState);
// _logger.LogDebug("After decode: {PixelFormat}", currentState.PixelFormat);
currentState = SetDeinterlace(videoInputFile, context, ffmpegState, currentState);
// _logger.LogDebug("After deinterlace: {PixelFormat}", currentState.PixelFormat);
currentState = SetScale(
videoInputFile,
videoStream,
context,
ffmpegState,
desiredState,
currentState,
useOpenClTonemap);
currentState = SetScale(videoInputFile, videoStream, context, ffmpegState, desiredState, currentState);
// _logger.LogDebug("After scale: {PixelFormat}", currentState.PixelFormat);
currentState = SetTonemap(
videoInputFile,
videoStream,
ffmpegState,
desiredState,
currentState,
useOpenClTonemap);
currentState = SetTonemap(videoInputFile, videoStream, ffmpegState, desiredState, currentState);
currentState = SetPad(videoInputFile, videoStream, desiredState, currentState);
// _logger.LogDebug("After pad: {PixelFormat}", currentState.PixelFormat);
currentState = SetCrop(videoInputFile, desiredState, currentState);
@@ -357,15 +335,9 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
IPixelFormat formatForDownload = pixelFormat;
// "did a hardware filter run", not "was it a QSV one": these all strip or rewrite the
// frame's color info, so the colorspace filter below has to re-assert it explicitly.
// The VA-API/OpenCL tonemap route (ersatztv#505) belongs here too — leaving it out
// shipped a correctly-tonemapped picture still TAGGED bt2020 primaries, which invites
// the player to convert it a second time.
bool usesVppQsv =
videoInputFile.FilterSteps.Any(f =>
f is QsvFormatFilter or ScaleQsvFilter or DeinterlaceQsvFilter
or ScaleVaapiFilter or TonemapOpenClQsvFilter);
f is QsvFormatFilter or ScaleQsvFilter or DeinterlaceQsvFilter or TonemapQsvFilter);
// if we have no filters, check whether we need to convert pixel format
// since qsv doesn't seem to like doing that at the encoder
@@ -414,15 +386,7 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
}
}
// A tonemap converted the PIXELS to SDR, so the stream must stop announcing HDR — that
// is a correctness requirement, not a normalization preference, and it holds even when
// the profile has NormalizeColors off. Without this an operator with NormalizeColors
// disabled gets tonemapped pixels still tagged bt2020 and the player converts them a
// second time. Deliberately NOT done by hoisting usesVppQsv out of the guard: a
// scale-only hardware chain on non-HDR content should still respect the preference.
bool tonemapped = videoInputFile.FilterSteps.Any(f => f is TonemapOpenClQsvFilter or TonemapFilter);
if (tonemapped || (desiredState.ColorsAreBt709 && (!videoStream.ColorParams.IsBt709 || usesVppQsv)))
if (desiredState.ColorsAreBt709 && (!videoStream.ColorParams.IsBt709 || usesVppQsv))
{
// _logger.LogDebug("Adding colorspace filter");
@@ -615,12 +579,8 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
}
}
// only scale if scaling or padding was used for main video stream.
// ScaleVaapiFilter belongs here too: the HDR/OpenCL tonemap path (ersatztv#505)
// scales the video with it, and leaving it out left the subtitle canvas at
// source resolution while the video shrank. VaapiPipelineBuilder already lists it.
if (videoInputFile.FilterSteps.Any(s =>
s is ScaleFilter or ScaleQsvFilter or ScaleVaapiFilter or PadFilter))
// only scale if scaling or padding was used for main video stream
if (videoInputFile.FilterSteps.Any(s => s is ScaleFilter or ScaleQsvFilter or PadFilter))
{
var scaleFilter = new ScaleImageFilter(desiredState.PaddedSize);
subtitle.FilterSteps.Add(scaleFilter);
@@ -680,84 +640,14 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
return currentState;
}
// The QSV pipeline can only reach tonemap_opencl through the VA-API device that its own QSV
// device is derived from, so every condition here is about that device existing and being
// reachable with software frames in hand.
private bool UseOpenClTonemap(
VideoStream videoStream,
PipelineContext context,
FFmpegState ffmpegState,
FrameState currentState)
{
if (!videoStream.ColorParams.IsHdr)
{
return false;
}
// The backstop for every case below, and for any future filter that lands ahead of the
// tonemap: the route starts with hwupload, so the frames have to actually be in software.
// Checked against the state rather than inferred from the enumeration, so a later change
// that puts frames on a surface earlier degrades to the software tonemap instead of
// emitting a second upload on top of an existing one.
if (currentState.FrameDataLocation == FrameDataLocation.Hardware)
{
return false;
}
// ffmpeg has no vaapi on Windows, so there is no device to derive OpenCL from
if (OperatingSystem.IsWindows())
{
return false;
}
// with no configured device QsvHardwareAccelerationOption emits a bare "-init_hw_device
// qsv=hw" and never initializes a VA-API device at all
if (ffmpegState.VaapiDevice.Filter(d => !string.IsNullOrWhiteSpace(d)).IsNone)
{
return false;
}
// frames from the QSV decoder are ALREADY on a QSV surface (-hwaccel_output_format qsv),
// and a QSV surface maps to neither OpenCL nor VA-API, so there is no route to the GPU
// tonemap from here. Software tonemap is slower but it is the only one that is correct.
if (ffmpegState.DecoderHardwareAccelerationMode == HardwareAccelerationMode.Qsv)
{
return false;
}
// deinterlace_qsv runs before the scale and leaves frames on a QSV surface, same problem
if (context.ShouldDeinterlace)
{
return false;
}
// ScaleQsvFilter is handed the SAR that VideoStream CALCULATES (it has a fallback for a
// missing or 0:0 SAR); ScaleVaapiFilter instead multiplies by ffmpeg's runtime `sar`, which
// is not the same value when the decoded frame leaves SAR unspecified. Rather than ship an
// anamorphic HDR graph nobody has run, keep anamorphic sources on the software tonemap —
// which is exactly what they got before this change, so it costs nothing they had.
if (videoStream.IsAnamorphic)
{
return false;
}
return _ffmpegCapabilities.HasFilter(FFmpegKnownFilter.TonemapOpenCL);
}
private static FrameState SetScale(
VideoInputFile videoInputFile,
VideoStream videoStream,
PipelineContext context,
FFmpegState ffmpegState,
FrameState desiredState,
FrameState currentState,
bool useOpenClTonemap)
FrameState currentState)
{
if (useOpenClTonemap)
{
return SetScaleVaapiForTonemap(videoInputFile, desiredState, currentState);
}
IPipelineFilterStep scaleStep;
bool useSoftwareFilter = ffmpegState is
@@ -811,40 +701,6 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
return currentState;
}
// HDR frames arrive from the VA-API decoder in system memory (the QSV pipeline deliberately
// omits -hwaccel_output_format), so upload them to the VA-API device and scale THERE. Scaling
// first matters: tonemapping the full-size frame instead costs ~50% more wall clock than the
// software tonemap it replaces, which is the difference between above and below realtime.
private static FrameState SetScaleVaapiForTonemap(
VideoInputFile videoInputFile,
FrameState desiredState,
FrameState currentState)
{
// the decoder's yuv420p10le is not a VA-API surface format; p010/nv12 are
IPixelFormat uploadFormat = currentState.PixelFormat.Map(pf => pf.BitDepth).IfNone(10) == 10
? new PixelFormatP010()
: new PixelFormatNv12(currentState.PixelFormat.Map(pf => pf.Name).IfNone(FFmpegFormat.NV12));
var upload = new HardwareUploadVaapiFilter(setFormat: true, deriveDevice: true);
currentState = upload.NextState(currentState) with { PixelFormat = Some(uploadFormat) };
videoInputFile.FilterSteps.Add(upload);
var scaleStep = new ScaleVaapiFilter(
currentState,
desiredState.ScaledSize,
desiredState.PaddedSize,
desiredState.CroppedSize,
VideoStream.IsAnamorphicEdgeCase);
if (!string.IsNullOrWhiteSpace(scaleStep.Filter))
{
currentState = scaleStep.NextState(currentState);
videoInputFile.FilterSteps.Add(scaleStep);
}
return currentState;
}
private static FrameState SetDeinterlace(
VideoInputFile videoInputFile,
PipelineContext context,
@@ -866,24 +722,26 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
VideoStream videoStream,
FFmpegState ffmpegState,
FrameState desiredState,
FrameState currentState,
bool useOpenClTonemap)
FrameState currentState)
{
if (videoStream.ColorParams.IsHdr)
{
foreach (IPixelFormat pixelFormat in desiredState.PixelFormat)
{
// NOTE: vpp_qsv=tonemap=1 is deliberately NOT an option here. On pre-Gen11 Intel
// graphics it returns the frame untouched with no warning, so it does not tonemap,
// it only LOOKS like it did (ersatztv#505). Either OpenCL tonemaps on the GPU or
// the software filter does it on the CPU; there is no silently-wrong third branch.
IPipelineFilterStep filter = useOpenClTonemap
? new TonemapOpenClQsvFilter(ffmpegState, pixelFormat)
: new TonemapFilter(ffmpegState, currentState, pixelFormat);
currentState = filter.NextState(currentState);
videoStream.ResetColorParams(ColorParams.Default);
videoInputFile.FilterSteps.Add(filter);
if (ffmpegState.DecoderHardwareAccelerationMode == HardwareAccelerationMode.Qsv)
{
var filter = new TonemapQsvFilter();
currentState = filter.NextState(currentState);
videoStream.ResetColorParams(ColorParams.Default);
videoInputFile.FilterSteps.Add(filter);
}
else
{
var filter = new TonemapFilter(ffmpegState, currentState, pixelFormat);
currentState = filter.NextState(currentState);
videoStream.ResetColorParams(ColorParams.Default);
videoInputFile.FilterSteps.Add(filter);
}
}
}
@@ -1,172 +0,0 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.MySql.Migrations
{
/// <inheritdoc />
public partial class Add_LibraryFolder_PathHash_UniqueIndex : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
// ersatztv#491 — audit/clean pre-existing duplicate LibraryFolder rows before the unique index.
// Mirrors the Sqlite migration; see it for the full rationale. The one provider difference is
// that every Path comparison here is forced BYTE-EXACT with CONVERT(... USING binary), because
// MySql's string comparison differs from Sqlite's on two independent axes and the dedupe
// deletes rows irreversibly:
// * case — the server default (utf8mb4_general_ci) is case-INsensitive, so grouping under it
// would collapse sibling folders differing only in case, legal on a case-sensitive fs;
// * trailing spaces — utf8mb4_bin, the obvious fix for the case half, is a PAD SPACE
// collation (verified on 8.4: '/media/Foo' = '/media/Foo ' is TRUE under it), so it would
// still collapse "/media/Foo" and "/media/Foo ", two distinct legal directories.
// Binary comparison is NO PAD and byte-exact, which is exactly what PathUtils.GetPathHash does
// — so the dedupe now destroys only rows the unique index would actually have rejected, and
// the two providers' migrations are semantically equivalent. (utf8mb4_0900_bin is also NO PAD
// but carries a server-version floor; CONVERT USING binary does not.)
// DROP TABLE IF EXISTS makes a retry after a partial failure safe (DDL implicitly commits on
// MySql, so the migration is not atomic).
migrationBuilder.Sql("DROP TABLE IF EXISTS `__LibraryFolderDedupe`");
migrationBuilder.Sql(
"""
CREATE TABLE `__LibraryFolderDedupe` (
LoserId INT NOT NULL PRIMARY KEY,
KeeperId INT NOT NULL
)
""");
migrationBuilder.Sql(
"""
INSERT INTO `__LibraryFolderDedupe` (LoserId, KeeperId)
SELECT l.Id, k.KeeperId
FROM LibraryFolder l
INNER JOIN (
SELECT LibraryPathId, CONVERT(Path USING binary) AS BinPath, MIN(Id) AS KeeperId
FROM LibraryFolder
GROUP BY LibraryPathId, CONVERT(Path USING binary)
) k ON k.LibraryPathId = l.LibraryPathId AND k.BinPath = CONVERT(l.Path USING binary)
WHERE l.Id <> k.KeeperId
""");
// media files recorded against a duplicate folder follow the keeper (MediaFile.LibraryFolderId
// is Restrict, so the delete below would fail otherwise)
migrationBuilder.Sql(
"""
UPDATE MediaFile
SET LibraryFolderId = (
SELECT KeeperId FROM `__LibraryFolderDedupe` WHERE LoserId = MediaFile.LibraryFolderId)
WHERE LibraryFolderId IN (SELECT LoserId FROM `__LibraryFolderDedupe`)
""");
// child folders parented on a duplicate follow the keeper (ParentId is Restrict as well)
migrationBuilder.Sql(
"""
UPDATE LibraryFolder
SET ParentId = (
SELECT KeeperId FROM `__LibraryFolderDedupe` WHERE LoserId = LibraryFolder.ParentId)
WHERE ParentId IN (SELECT LoserId FROM `__LibraryFolderDedupe`)
""");
// A folder parented on its OWN duplicate would become its own parent above. Unreachable from
// any code path today, but this is a tree the scanner walks, so remove the cycle class rather
// than reason about it.
migrationBuilder.Sql("UPDATE LibraryFolder SET ParentId = NULL WHERE ParentId = Id");
// Clear the survivor's etag. Which duplicate the scanner was actually writing to was
// arbitrary, so MIN(Id)'s etag may describe a stale view of the folder and would suppress the
// next rescan. A null etag costs exactly one rescan and cannot be wrong.
migrationBuilder.Sql(
"UPDATE LibraryFolder SET Etag = NULL WHERE Id IN (SELECT KeeperId FROM `__LibraryFolderDedupe`)");
// ImageFolderDuration is 1:1 with LibraryFolder (unique index on LibraryFolderId), so the
// duplicates' rows cannot all be repointed. Keep the keeper's own setting when it has one;
// otherwise promote exactly one loser's (lowest Id) and drop the rest.
migrationBuilder.Sql("DROP TABLE IF EXISTS `__LibraryFolderDedupeIfd`");
migrationBuilder.Sql(
"""
CREATE TABLE `__LibraryFolderDedupeIfd` (
KeeperId INT NOT NULL PRIMARY KEY,
IfdId INT NOT NULL
)
""");
migrationBuilder.Sql(
"""
INSERT INTO `__LibraryFolderDedupeIfd` (KeeperId, IfdId)
SELECT d.KeeperId, MIN(i.Id)
FROM `__LibraryFolderDedupe` d
INNER JOIN ImageFolderDuration i ON i.LibraryFolderId = d.LoserId
WHERE NOT EXISTS (
SELECT 1 FROM ImageFolderDuration ki WHERE ki.LibraryFolderId = d.KeeperId)
GROUP BY d.KeeperId
""");
migrationBuilder.Sql(
"""
DELETE FROM ImageFolderDuration
WHERE LibraryFolderId IN (SELECT LoserId FROM `__LibraryFolderDedupe`)
AND Id NOT IN (SELECT IfdId FROM `__LibraryFolderDedupeIfd`)
""");
migrationBuilder.Sql(
"""
UPDATE ImageFolderDuration
SET LibraryFolderId = (
SELECT KeeperId FROM `__LibraryFolderDedupeIfd` WHERE IfdId = ImageFolderDuration.Id)
WHERE Id IN (SELECT IfdId FROM `__LibraryFolderDedupeIfd`)
""");
migrationBuilder.Sql(
"DELETE FROM LibraryFolder WHERE Id IN (SELECT LoserId FROM `__LibraryFolderDedupe`)");
migrationBuilder.Sql("DROP TABLE `__LibraryFolderDedupeIfd`");
migrationBuilder.Sql("DROP TABLE `__LibraryFolderDedupe`");
migrationBuilder.AddColumn<string>(
name: "PathHash",
table: "LibraryFolder",
type: "varchar(64)",
maxLength: 64,
nullable: true)
.Annotation("MySql:CharSet", "utf8mb4");
// Existing rows keep a null hash on purpose: a unique index treats nulls as distinct, so the
// index applies cleanly to any database, and LibraryRepository.GetOrAddFolder heals each row
// (SHA-256 of Path) the first time a scan touches it. Those rows are deduplicated above and are
// still found by the Path lookup, so no insert can race them in the meantime.
//
// Order matters on MySql, and EF scaffolds it the other way round: InnoDB refuses to drop the
// FK's only backing index ("Cannot drop index 'IX_LibraryFolder_LibraryPathId': needed in a
// foreign key constraint"). Create the composite first — LibraryPathId is its leftmost column,
// so it takes over as the FK's backing index — then drop the now-redundant single-column one.
migrationBuilder.CreateIndex(
name: "IX_LibraryFolder_LibraryPathId_PathHash",
table: "LibraryFolder",
columns: new[] { "LibraryPathId", "PathHash" },
unique: true);
migrationBuilder.DropIndex(
name: "IX_LibraryFolder_LibraryPathId",
table: "LibraryFolder");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
// mirror of Up: restore the single-column index before dropping the composite one, so the
// foreign key is never left without a backing index
migrationBuilder.CreateIndex(
name: "IX_LibraryFolder_LibraryPathId",
table: "LibraryFolder",
column: "LibraryPathId");
migrationBuilder.DropIndex(
name: "IX_LibraryFolder_LibraryPathId_PathHash",
table: "LibraryFolder");
migrationBuilder.DropColumn(
name: "PathHash",
table: "LibraryFolder");
}
}
}
@@ -1,38 +0,0 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.MySql.Migrations
{
/// <inheritdoc />
public partial class Add_FFmpegProfile_ReadRatePacing : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<double>(
name: "ReadRate",
table: "FFmpegProfile",
type: "double",
nullable: true);
migrationBuilder.AddColumn<double>(
name: "ReadRateCatchup",
table: "FFmpegProfile",
type: "double",
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ReadRate",
table: "FFmpegProfile");
migrationBuilder.DropColumn(
name: "ReadRateCatchup",
table: "FFmpegProfile");
}
}
}
@@ -929,12 +929,6 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
.HasColumnType("tinyint(1)")
.HasDefaultValue(true);
b.Property<double?>("ReadRate")
.HasColumnType("double");
b.Property<double?>("ReadRateCatchup")
.HasColumnType("double");
b.Property<int>("ResolutionId")
.HasColumnType("int");
@@ -1359,16 +1353,11 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
b.Property<string>("Path")
.HasColumnType("longtext");
b.Property<string>("PathHash")
.HasMaxLength(64)
.HasColumnType("varchar(64)");
b.HasKey("Id");
b.HasIndex("ParentId");
b.HasIndex("LibraryPathId");
b.HasIndex("LibraryPathId", "PathHash")
.IsUnique();
b.HasIndex("ParentId");
b.ToTable("LibraryFolder", (string)null);
});
@@ -1,57 +0,0 @@
using System.Data;
using Microsoft.Data.Sqlite;
namespace ErsatzTV.Infrastructure.Sqlite.Data;
/// <summary>
/// ersatztv#668. SQLite's built-in <c>lower()</c>/<c>upper()</c> fold ASCII ONLY — <c>lower('Édith')</c>
/// returns <c>'Édith'</c> unchanged — so a facet value whose prefix carries an uppercase non-ASCII
/// character can never be matched by the prefix predicate the facet-value endpoint emits. Registering a
/// managed scalar gives that one query a Unicode-correct fold. Wired to
/// <see cref="ErsatzTV.Infrastructure.Data.TvContext.RegisterUnicodeCaseFunctions" /> at startup.
/// </summary>
public static class SqliteUnicodeFunctions
{
/// <summary>
/// SQL name of the invariant-uppercase fold. The facet-value handler interpolates this constant into
/// its SQL, so the two cannot drift apart.
/// </summary>
public const string UpperInvariantFunction = "etv_upper";
/// <summary>
/// Registers <see cref="UpperInvariantFunction" /> on <paramref name="connection" /> when it is a
/// SQLite connection, and does nothing otherwise. Idempotent — a repeat registration replaces the
/// previous delegate with an identical one — so the single call site may call it unconditionally.
/// <para>
/// The property this fold has to satisfy is ONE-SIDED: the SQL stage may over-match freely,
/// because the endpoint applies an exact <see cref="StringComparison.OrdinalIgnoreCase" /> filter
/// in memory afterwards, but it must never UNDER-match — no later stage can reintroduce a row SQL
/// never returned. <see cref="string.ToUpperInvariant" /> satisfies it because
/// <c>OrdinalIgnoreCase</c> equality is a strict SUBSET of invariant-uppercase equality, so
/// folding both sides with it yields a superset of the final filter's matches.
/// </para>
/// <para>
/// Do not restate that as "<c>OrdinalIgnoreCase</c> IS invariant-uppercase-then-ordinal" — it is
/// not, and the difference is measurable: <c>char.ToUpperInvariant('ſ')</c> (U+017F) is <c>'S'</c>,
/// yet <c>"ſweet".StartsWith("S", OrdinalIgnoreCase)</c> is <b>false</b>. That gap is precisely
/// the harmless direction — SQL returns the row, the in-memory filter drops it. The containment,
/// not any identity of the two foldings, is what makes this safe.
/// </para>
/// <para>
/// Registration is per-connection and therefore done at the one call site that uses the function,
/// not through an EF connection interceptor: Dapper opens a closed connection itself, and a direct
/// ADO open does not raise EF's interceptors — so an interceptor-based seam would silently miss
/// exactly the query that needs it.
/// </para>
/// </summary>
public static void Register(IDbConnection connection)
{
if (connection is SqliteConnection sqlite)
{
sqlite.CreateFunction(
UpperInvariantFunction,
(string? value) => value?.ToUpperInvariant(),
isDeterministic: true);
}
}
}
@@ -1,157 +0,0 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
{
/// <inheritdoc />
public partial class Add_LibraryFolder_PathHash_UniqueIndex : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
// ersatztv#491 — audit/clean pre-existing duplicate LibraryFolder rows before the unique index.
// Duplicates were reachable before #488 (the folder lookup read a scan-start in-memory snapshot,
// so a folder created earlier in the SAME scan was invisible and inserted again) and via the
// check-then-insert race the index now closes. Keep the lowest Id per (LibraryPathId, Path) and
// repoint every dependent row at it before deleting the losers. The helper tables keep the
// statements readable; DROP TABLE IF EXISTS makes a retry after a partial failure safe.
migrationBuilder.Sql("DROP TABLE IF EXISTS __LibraryFolderDedupe");
migrationBuilder.Sql(
"""
CREATE TABLE __LibraryFolderDedupe (
LoserId INTEGER NOT NULL PRIMARY KEY,
KeeperId INTEGER NOT NULL
)
""");
migrationBuilder.Sql(
"""
INSERT INTO __LibraryFolderDedupe (LoserId, KeeperId)
SELECT l.Id, k.KeeperId
FROM LibraryFolder l
INNER JOIN (
SELECT LibraryPathId, Path, MIN(Id) AS KeeperId
FROM LibraryFolder
GROUP BY LibraryPathId, Path
) k ON k.LibraryPathId = l.LibraryPathId AND k.Path = l.Path
WHERE l.Id <> k.KeeperId
""");
// media files recorded against a duplicate folder follow the keeper (MediaFile.LibraryFolderId
// is Restrict, so the delete below would fail otherwise)
migrationBuilder.Sql(
"""
UPDATE MediaFile
SET LibraryFolderId = (
SELECT KeeperId FROM __LibraryFolderDedupe WHERE LoserId = MediaFile.LibraryFolderId)
WHERE LibraryFolderId IN (SELECT LoserId FROM __LibraryFolderDedupe)
""");
// child folders parented on a duplicate follow the keeper (ParentId is Restrict as well)
migrationBuilder.Sql(
"""
UPDATE LibraryFolder
SET ParentId = (
SELECT KeeperId FROM __LibraryFolderDedupe WHERE LoserId = LibraryFolder.ParentId)
WHERE ParentId IN (SELECT LoserId FROM __LibraryFolderDedupe)
""");
// A folder parented on its OWN duplicate would become its own parent above. Unreachable from
// any code path today, but this is a tree the scanner walks, so remove the cycle class rather
// than reason about it.
migrationBuilder.Sql("UPDATE LibraryFolder SET ParentId = NULL WHERE ParentId = Id");
// Clear the survivor's etag. Which duplicate the scanner was actually writing to was
// arbitrary, so MIN(Id)'s etag may describe a stale view of the folder and would suppress the
// next rescan. A null etag costs exactly one rescan and cannot be wrong.
migrationBuilder.Sql(
"UPDATE LibraryFolder SET Etag = NULL WHERE Id IN (SELECT KeeperId FROM __LibraryFolderDedupe)");
// ImageFolderDuration is 1:1 with LibraryFolder (unique index on LibraryFolderId), so the
// duplicates' rows cannot all be repointed. Keep the keeper's own setting when it has one;
// otherwise promote exactly one loser's (lowest Id) and drop the rest.
migrationBuilder.Sql("DROP TABLE IF EXISTS __LibraryFolderDedupeIfd");
migrationBuilder.Sql(
"""
CREATE TABLE __LibraryFolderDedupeIfd (
KeeperId INTEGER NOT NULL PRIMARY KEY,
IfdId INTEGER NOT NULL
)
""");
migrationBuilder.Sql(
"""
INSERT INTO __LibraryFolderDedupeIfd (KeeperId, IfdId)
SELECT d.KeeperId, MIN(i.Id)
FROM __LibraryFolderDedupe d
INNER JOIN ImageFolderDuration i ON i.LibraryFolderId = d.LoserId
WHERE NOT EXISTS (
SELECT 1 FROM ImageFolderDuration ki WHERE ki.LibraryFolderId = d.KeeperId)
GROUP BY d.KeeperId
""");
migrationBuilder.Sql(
"""
DELETE FROM ImageFolderDuration
WHERE LibraryFolderId IN (SELECT LoserId FROM __LibraryFolderDedupe)
AND Id NOT IN (SELECT IfdId FROM __LibraryFolderDedupeIfd)
""");
migrationBuilder.Sql(
"""
UPDATE ImageFolderDuration
SET LibraryFolderId = (
SELECT KeeperId FROM __LibraryFolderDedupeIfd WHERE IfdId = ImageFolderDuration.Id)
WHERE Id IN (SELECT IfdId FROM __LibraryFolderDedupeIfd)
""");
migrationBuilder.Sql(
"DELETE FROM LibraryFolder WHERE Id IN (SELECT LoserId FROM __LibraryFolderDedupe)");
migrationBuilder.Sql("DROP TABLE __LibraryFolderDedupeIfd");
migrationBuilder.Sql("DROP TABLE __LibraryFolderDedupe");
migrationBuilder.AddColumn<string>(
name: "PathHash",
table: "LibraryFolder",
type: "TEXT",
maxLength: 64,
nullable: true);
// Existing rows keep a null hash on purpose: a unique index treats nulls as distinct, so the
// index applies cleanly to any database, and LibraryRepository.GetOrAddFolder heals each row
// (SHA-256 of Path) the first time a scan touches it. Those rows are deduplicated above and are
// still found by the Path lookup, so no insert can race them in the meantime.
//
// Create-then-drop rather than EF's scaffolded drop-then-create, matching the MySql copy, where
// InnoDB refuses to drop the foreign key's only backing index.
migrationBuilder.CreateIndex(
name: "IX_LibraryFolder_LibraryPathId_PathHash",
table: "LibraryFolder",
columns: new[] { "LibraryPathId", "PathHash" },
unique: true);
migrationBuilder.DropIndex(
name: "IX_LibraryFolder_LibraryPathId",
table: "LibraryFolder");
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateIndex(
name: "IX_LibraryFolder_LibraryPathId",
table: "LibraryFolder",
column: "LibraryPathId");
migrationBuilder.DropIndex(
name: "IX_LibraryFolder_LibraryPathId_PathHash",
table: "LibraryFolder");
migrationBuilder.DropColumn(
name: "PathHash",
table: "LibraryFolder");
}
}
}
@@ -1,38 +0,0 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
{
/// <inheritdoc />
public partial class Add_FFmpegProfile_ReadRatePacing : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<double>(
name: "ReadRate",
table: "FFmpegProfile",
type: "REAL",
nullable: true);
migrationBuilder.AddColumn<double>(
name: "ReadRateCatchup",
table: "FFmpegProfile",
type: "REAL",
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ReadRate",
table: "FFmpegProfile");
migrationBuilder.DropColumn(
name: "ReadRateCatchup",
table: "FFmpegProfile");
}
}
}
@@ -896,12 +896,6 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
.HasColumnType("INTEGER")
.HasDefaultValue(true);
b.Property<double?>("ReadRate")
.HasColumnType("REAL");
b.Property<double?>("ReadRateCatchup")
.HasColumnType("REAL");
b.Property<int>("ResolutionId")
.HasColumnType("INTEGER");
@@ -1304,16 +1298,11 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
b.Property<string>("Path")
.HasColumnType("TEXT");
b.Property<string>("PathHash")
.HasMaxLength(64)
.HasColumnType("TEXT");
b.HasKey("Id");
b.HasIndex("ParentId");
b.HasIndex("LibraryPathId");
b.HasIndex("LibraryPathId", "PathHash")
.IsUnique();
b.HasIndex("ParentId");
b.ToTable("LibraryFolder", (string)null);
});
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
@@ -10,19 +10,6 @@ public class LibraryFolderConfiguration : IEntityTypeConfiguration<LibraryFolder
{
builder.ToTable("LibraryFolder");
// ersatztv#491: GetOrAddFolder is a check-then-insert, so two callers racing the same folder both
// miss the lookup and both insert. The unique index makes the duplicate impossible at the storage
// layer (the loser gets a constraint violation the repository catches and re-reads).
// Indexed on PathHash rather than Path because Path is unbounded (MySQL longtext, which cannot be
// indexed without a prefix length, and whose default collation is case-INsensitive — a prefix
// index would also false-collide sibling folders differing only in case on a case-sensitive
// filesystem). This mirrors the existing MediaFile.Path/PathHash pair.
builder.Property(f => f.PathHash)
.HasMaxLength(64);
builder.HasIndex(f => new { f.LibraryPathId, f.PathHash })
.IsUnique();
builder.HasOne(f => f.Parent)
.WithMany(p => p.Children)
.HasForeignKey(f => f.ParentId)
@@ -1,6 +1,5 @@
using System.IO.Abstractions;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Infrastructure.Extensions;
@@ -111,35 +110,15 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
if (knownFolder.IsNone)
{
var newFolder = new LibraryFolder
{
Path = path,
PathHash = PathUtils.GetPathHash(path),
Etag = etag,
LibraryPathId = libraryPath.Id
};
try
{
await dbContext.LibraryFolders.AddAsync(newFolder);
await dbContext.SaveChangesAsync();
}
catch (DbUpdateException ex) when (TvContext.IsUniqueConstraintViolation(ex))
{
// ersatztv#491: a concurrent caller created this folder between the caller's lookup and
// this insert. The etag write is the whole point of the call, so apply it to the winner's
// row rather than failing the scan.
dbContext.Entry(newFolder).State = EntityState.Detached;
LibraryFolder winner = await GetFolder(dbContext, libraryPath.Id, path);
if (winner is null)
await dbContext.LibraryFolders.AddAsync(
new LibraryFolder
{
throw;
}
Path = path,
Etag = etag,
LibraryPathId = libraryPath.Id
});
await dbContext.Connection.ExecuteAsync(
"UPDATE LibraryFolder SET Etag = @Etag WHERE Id = @Id",
new { winner.Id, Etag = etag });
}
await dbContext.SaveChangesAsync();
}
}
@@ -195,74 +174,11 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
// local scan path (via GetLibrary) and is null on the remote (Jellyfin) sync path, which used
// to NRE every Jellyfin music-video scan here (ersatztv#488). The local scanners already hit
// the db once per folder via GetParentFolderId, so this adds no new query pattern.
LibraryFolder knownFolder = await GetFolder(dbContext, libraryPath.Id, folder);
// add new folder to library path
if (knownFolder is null)
{
LibraryFolder newFolder = CreateNewFolder(libraryPath, maybeParentFolder, folder);
try
{
await dbContext.LibraryFolders.AddAsync(newFolder);
await dbContext.SaveChangesAsync();
knownFolder = newFolder;
}
catch (DbUpdateException ex) when (TvContext.IsUniqueConstraintViolation(ex))
{
// ersatztv#491: the lookup above is not atomic with this insert, so a concurrent caller
// scanning the same folder can slip its row in between. The unique index on
// (LibraryPathId, PathHash) turns that lost race into a constraint violation instead of a
// duplicate row; adopt the winner's row rather than failing the scan. Detach first so the
// failed insert is not retried by anything reusing this context.
dbContext.Entry(newFolder).State = EntityState.Detached;
knownFolder = await GetFolder(dbContext, libraryPath.Id, folder);
if (knownFolder is null)
{
// no winner to adopt — the violation came from somewhere else, so don't swallow it
throw;
}
}
}
else if (string.IsNullOrEmpty(knownFolder.PathHash))
{
// Heal a row created before the PathHash column existed, so it participates in the unique
// index from here on (a null hash is distinct from every other value, so it does not).
//
// This is opportunistic maintenance on a hot scan path, so it must never be able to abort a
// scan. It goes through EF rather than a raw Dapper UPDATE precisely so a collision surfaces
// as a classifiable DbUpdateException instead of a bare provider exception, and a lost heal
// is simply left for the next scan. Reachable only if some other row already owns
// (LibraryPathId, hash) — a legacy duplicate the migration's dedupe could not see (e.g. one
// with a NULL Path, which `NULL = NULL` excludes from its grouping).
string pathHash = PathUtils.GetPathHash(folder);
LibraryFolder tracked = null;
try
{
// the predicate must agree with the IsNullOrEmpty guard above, or a PathHash = '' row would
// enter this branch, match nothing, and silently never heal
tracked = await dbContext.LibraryFolders
.FirstOrDefaultAsync(f => f.Id == knownFolder.Id && (f.PathHash == null || f.PathHash == ""));
if (tracked is not null)
{
tracked.PathHash = pathHash;
await dbContext.SaveChangesAsync();
knownFolder.PathHash = pathHash;
}
}
catch (DbUpdateException ex) when (
TvContext.IsUniqueConstraintViolation(ex) || ex is DbUpdateConcurrencyException)
{
// Either another row already owns this hash, or the row was deleted out from under us by a
// concurrent library edit (DbUpdateConcurrencyException derives from DbUpdateException but
// carries no provider exception, so the classifier does NOT recognize it). Both mean "the
// heal is moot" — leave the row unhealed rather than fail the scan, per the invariant above.
if (tracked is not null)
{
// drop the failed change so it cannot be replayed by a later save on this context
dbContext.Entry(tracked).State = EntityState.Detached;
}
}
}
LibraryFolder knownFolder = await dbContext.LibraryFolders
.AsNoTracking()
.Filter(f => f.LibraryPathId == libraryPath.Id && f.Path == folder)
.FirstOrDefaultAsync()
?? CreateNewFolder(libraryPath, maybeParentFolder, folder);
// update parent folder if not present
foreach (int parentFolder in maybeParentFolder)
@@ -277,6 +193,13 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
}
}
// add new folder to library path
if (knownFolder.Id < 1)
{
await dbContext.LibraryFolders.AddAsync(knownFolder);
await dbContext.SaveChangesAsync();
}
return knownFolder;
}
@@ -298,71 +221,6 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
new { Path = normalizedLibraryPath, libraryPath.Id });
}
/// <summary>
/// The in-memory half of <see cref="GetFolder" />, lifted out so the ordinal decision is pinned by a
/// test with no database at all: the collation behaviour that makes it necessary is MySQL-only, so a
/// SQLite-backed test cannot exercise it (SQLite's <c>=</c> on TEXT is already binary and never
/// returns the case-differing candidate). Given the candidates a case-INsensitive server may return,
/// pick the one whose path matches ordinally; callers pass them lowest <c>Id</c> first.
/// </summary>
public static LibraryFolder ResolveExact(IReadOnlyList<LibraryFolder> candidates, string folder)
{
for (var i = 0; i < candidates.Count; i++)
{
if (string.Equals(candidates[i].Path, folder, StringComparison.Ordinal))
{
return candidates[i];
}
}
return null;
}
/// <summary>
/// Resolve a folder by its exact path within a library path.
/// <para>
/// The SQL equality is only a *narrowing* filter, not the identity test. On MySQL, `Path` is a
/// `longtext` whose collation the schema does not pin — only the `utf8mb4` charset — so the
/// effective comparison is whatever the server defaults to, and it differs from byte equality:
/// <list type="bullet">
/// <item>
/// always case-INsensitive: both plausible defaults are `_ci` (8.4 verified:
/// `utf8mb4_0900_ai_ci`; older servers `utf8mb4_general_ci`), which is why
/// <see cref="TvContext.CaseInsensitiveCollation" /> exists at all;
/// </item>
/// <item>
/// possibly PAD SPACE, making trailing spaces insignificant — true of
/// `utf8mb4_general_ci`, but NOT of `utf8mb4_0900_ai_ci`, which is NO PAD. So this axis
/// is server-dependent rather than guaranteed, and must be tolerated rather than
/// assumed either way.
/// </item>
/// </list>
/// `Path = @folder` can therefore also match siblings differing only in case, or (on a PAD
/// SPACE server) in trailing whitespace — all legal on a case-sensitive filesystem, and all
/// preserved by the #491 migration. Crucially the SQL predicate is a *superset*: every such
/// quirk makes it more permissive, never less, so it cannot miss a byte-exact match. Identity
/// is then settled in memory by <see cref="ResolveExact" /> with an ORDINAL comparison,
/// matching <c>PathUtils.GetPathHash</c>, which hashes the exact bytes. Without this the lookup
/// and the hash disagree, and the PathHash heal could stamp one sibling's hash onto the other's
/// row.
/// </para>
/// <para>
/// Ordered by Id so the result is deterministic: an unordered <c>FirstOrDefault</c> may return a
/// different candidate run to run as the query plan changes (adding the composite index alone
/// can flip it), which would make the heal non-idempotent.
/// </para>
/// </summary>
private static async Task<LibraryFolder> GetFolder(TvContext dbContext, int libraryPathId, string folder)
{
List<LibraryFolder> candidates = await dbContext.LibraryFolders
.AsNoTracking()
.Filter(f => f.LibraryPathId == libraryPathId && f.Path == folder)
.OrderBy(f => f.Id)
.ToListAsync();
return ResolveExact(candidates, folder);
}
private static LibraryFolder CreateNewFolder(LibraryPath libraryPath, Option<int> maybeParentFolder, string folder)
{
int? parentId = null;
@@ -374,7 +232,6 @@ public class LibraryRepository(IFileSystem fileSystem, IDbContextFactory<TvConte
return new LibraryFolder
{
Path = folder,
PathHash = PathUtils.GetPathHash(folder),
Etag = null,
LibraryPathId = libraryPath.Id,
ParentId = parentId
@@ -1144,7 +1144,7 @@ public class MediaCollectionRepository : IMediaCollectionRepository
var allArtists = items.OfType<Song>()
.SelectMany(s => s.SongMetadata)
.Map(sm => Optional(sm.AlbumArtists).Flatten().HeadOrNone().Match(aa => aa, string.Empty))
.Map(sm => sm.AlbumArtists.HeadOrNone().Match(aa => aa, string.Empty))
.Distinct()
.ToList();
@@ -1157,7 +1157,7 @@ public class MediaCollectionRepository : IMediaCollectionRepository
foreach (Song song in items.OfType<Song>())
{
string firstArtist = song.SongMetadata
.SelectMany(sm => Optional(sm.AlbumArtists).Flatten())
.SelectMany(sm => sm.AlbumArtists)
.HeadOrNone()
.Match(aa => aa, string.Empty);
@@ -1,4 +1,4 @@
using Dapper;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Repositories;
@@ -171,15 +171,8 @@ public class MusicVideoRepository : IMusicVideoRepository
public async Task<int> GetMusicVideoCount(int artistId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
// count the same population GetPagedMusicVideos pages — MusicVideoMetadata, not MusicVideo.
// A music video whose metadata row is missing (a scanner failure; FindOrphanPaths models
// exactly that state) is not pageable, so counting the item table over-reports
// (api.paged-count-matches-page-query, #832).
return await dbContext.Connection.QuerySingleAsync<int>(
@"SELECT COUNT(*)
FROM MusicVideoMetadata MVM
INNER JOIN MusicVideo M on MVM.MusicVideoId = M.Id
WHERE M.ArtistId = @ArtistId",
@"SELECT COUNT(*) FROM MusicVideo WHERE ArtistId = @ArtistId",
new { ArtistId = artistId });
}
@@ -1,4 +1,4 @@
using Dapper;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
@@ -134,26 +134,9 @@ public class TelevisionRepository : ITelevisionRepository
public async Task<int> GetSeasonCount(int showId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
// GetPagedSeasons expands the requested show to EVERY show sharing its Title+Year (the same
// show present in two libraries) and pages the union, so the count must expand identically
// or it under-reports (api.paged-count-matches-page-query, #832).
Option<ShowMetadata> maybeShowMetadata = await dbContext.ShowMetadata
.SelectOneAsync(sm => sm.Id, sm => sm.ShowId == showId, CancellationToken.None);
foreach (ShowMetadata showMetadata in maybeShowMetadata)
{
List<int> showIds = await dbContext.ShowMetadata
.Filter(sm => sm.Title == showMetadata.Title && sm.Year == showMetadata.Year)
.Map(sm => sm.ShowId)
.ToListAsync();
return await dbContext.Seasons
.AsNoTracking()
.CountAsync(s => showIds.Contains(s.ShowId));
}
// no metadata for the requested show: GetPagedSeasons returns nothing, so neither does this
return 0;
return await dbContext.Seasons
.AsNoTracking()
.CountAsync(s => s.ShowId == showId);
}
public async Task<List<Season>> GetPagedSeasons(
@@ -196,12 +179,9 @@ public class TelevisionRepository : ITelevisionRepository
public async Task<int> GetEpisodeCount(int seasonId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
// count the same population GetPagedEpisodes pages — EpisodeMetadata, not Episode. An
// episode whose metadata row is missing is not pageable, so counting the item table
// over-reports (api.paged-count-matches-page-query, #832).
return await dbContext.EpisodeMetadata
return await dbContext.Episodes
.AsNoTracking()
.CountAsync(em => em.Episode.SeasonId == seasonId);
.CountAsync(e => e.SeasonId == seasonId);
}
public async Task<List<EpisodeMetadata>> GetPagedEpisodes(int seasonId, int pageNumber, int pageSize)

Some files were not shown because too many files have changed in this diff Show More