Compare commits

..
Author SHA1 Message Date
timothyandtimothy 736649b3b7 fix(812): classify the narrative sites by who-benefits; keep the detector's reach (#882)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 22s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m18s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m33s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m52s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-29 20:28:14 +00:00
timothy 1afad0851d fix(823,824): a scheduling NULL collection reads as UNRESTRICTED and is guarded at both read sites; the Elastic indexer gets its own mutation proof (#879)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 17s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 32s
Build ErsatzTV Image / Build & test (.NET) (push) Canceled after 0s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (push) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Canceled after 0s
2026-08-29 19:55:53 +00:00
timothyandClaude Opus 5 9685132ee0 fix(823): three cleanups from the coherence pass — a wrong witness, a wrong because, a duplicated paths:
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 14s
review-verdict/h10 Review-verdict: MERGEABLE @ 9685132 (base: main)
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 12s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 13s
PR Gates / decisions lifecycle (pull_request) Successful in 17s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 43s
Review verdict / Set review-verdict status (pull_request_target) Successful in 9s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 11m28s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 14m32s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 10m10s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 9m23s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 11s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 10s
Round-five review confirmed the decision record is coherent with no third
survivor of the empty reading, and returned three LOW findings. All are in
prose I wrote in the last two commits.

- The comment defending `(IsAbstract && !IsSealed)` cited
  AlternateScheduleSelectorTests as an in-repo static-fixture witness. That
  class IS static, but it merely NESTS its [TestFixture]es and declares no test
  of its own, so it would fail the sibling "declares no runnable test" assertion
  rather than demonstrating the point. The rule is right and the witness was
  wrong, which is the worse of the two failures because a wrong example is what
  a reader checks the rule against. No witness is cited now, and why is stated.

- A mis-bound `because` in `rule:`: "assigning a null and calling SaveChanges
  SUCCEEDS ... because only the HTTP request records normalize with `?? []`".
  The `?? []` clause explains how a null could REACH the entity; what makes the
  save succeed is the column being nullable. A right observation with a wrong
  cause attached. Split into the two claims.

- `signals:` carried the literal token `paths:` twice, an artifact of appending
  the #823 path list to the existing one. It degrades the field the discovery
  surface parses.

Also recorded from that review, and NOT changed: `MonthsOfYear ?? AllDaysOfMonth()`
survives the selector fixture and no date can kill it -- 1..31 contains every
valid month, so it is an EQUIVALENT mutant there rather than a coverage gap.
Its non-equivalent twin at the DTO boundary is pinned per-dimension by
RecurrenceLimitsMapperNullTests. Left alone deliberately: chasing an equivalent
mutant with a contrived date would buy nothing and cost the fixture's
readability.

Local gate: ErsatzTV.Tests 2091 passed / 6 skipped, Core.Tests 697/1 -- 0
failures. Format clean, no BOM. decisions_validate OK.

Refs #823
Refs #824

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 21:14:27 +02:00
timothyandClaude Opus 5 3951fcf516 fix(823): stop patching the record by grep — a second ?? [] survived, and my "swept it" claim was false
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 18s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 9s
PR Gates / decisions lifecycle (pull_request) Successful in 14s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
review-verdict/h10 Awaiting review verdict for 3951fcf
Review verdict / Set review-verdict status (pull_request_target) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 11s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 7m52s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 0s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Round-four review. One HIGH, again in the decision record, and the previous
commit message asserted this exact class was cleared. It was not.

THE HIGH, and the reason it recurred.

A second sentence still described the rejected reading: "The guard form is
`?? []` into a local rather than this record's Optional(x).Flatten(), a STATED
deviation". The shipped guard is `?? AllDaysOfWeek()`. That sentence is the one
that dictates guard FORM to the next implementer, so it would have taught the
`[]` reading the same record spends a paragraph calling data corruption -- and
it had already propagated into docs/decisions/README.md, the mandated entry
point, which carries `rule:` verbatim.

The mechanism, not the sentence, is the defect. I swept with a regex keyed on
"null" plus a reading word; this sentence talks about guard FORM and contains
neither, so it could not match. That is grepping the retracted WORDING instead
of sweeping the CONCEPT, which is exactly what this corpus warns about -- and
three rounds in a row have now found a defect introduced by the previous
round's targeted string edit. So the fix is not another targeted edit: the
whole `rule:` field was split into its 39 sentences and read back one by one
against the code. Everything below came out of that pass rather than a grep.

Its secondary damage is worth recording because it is the shape of a rationale
that outlives its claim: the deviation was justified by ".ToList() allocates
for nothing", which is now BOTH irrelevant to the choice AND false about the
shipped code, since AllDaysOfMonth()/AllMonthsOfYear() are themselves
Enumerable.Range(...).ToList() on exactly the null path it describes.

- The opening sentence of `rule:` prescribed Optional(x).Flatten() as THE
  read-site form. It is the sentence most likely to be read in isolation, and
  it is wrong for six of the eight columns. It now separates the universal half
  (a LOCAL, never assigned back) from the half that is not (the substituted
  value), and names where each applies.
- `signals:` had never been touched, so roughly 60% of `rule:` was unreachable
  by the discovery surface built for it -- no AlternateScheduleSelector, no
  mapper, no "unrestricted", and its paths: list named none of the files this
  work touched. It also advertised "Optional Flatten hoisted local" as the
  form, which is precisely what the six do NOT use.
- The body prose was still entirely about SongMetadata while `rule:` had grown
  a whole second subject. Added the two results that contradicted the prior
  reasoning, in prose, where a reader meets them.

A REAL BUG in my own guard, not just prose:

  fixture.IsAbstract.ShouldBeFalse(...)

A C# `static class` compiles to `abstract sealed`, and NUnit runs tests
declared in one -- this repo already has such a fixture
(AlternateScheduleSelectorTests is `public static class`). So the check I added
one commit ago to reject an un-runnable fixture would have falsely reddened a
perfectly good static one. Now rejects an abstract BASE (abstract and NOT
sealed), which is the case NUnit actually cannot instantiate.

A SURVIVING MUTANT the added controls did not kill:

AnyDate was 2024-03-06. With a day <= 12 a CROSS-WIRED substitution survives
the whole fixture -- `DaysOfMonth ?? AllMonthsOfYear()` hands back 1..12, which
still contains day 6, so every assertion passes while the guard substitutes the
wrong set. Moved to 2024-03-20, still a Wednesday in March, outside 1..12.
Measured both ways rather than reasoned: the cross-wire mutant passes the old
fixture and FAILS 2 of 11 on the new one.

Also re-witnessed, because I had modified that file and never re-proved it:
restoring `??=` in LuceneSearchIndex reddens the LUCENE fixture (1 red, 1
green) -- the exact mirror of the Elastic mutation. Extracting
ThrowOnWarningLogger did not cost #701 its proof, and the two fixtures are
independently load-bearing in both directions.

The record is now 73 prose lines, over the 60-line WARNING ceiling. Stated
rather than trimmed: it is 42nd of 42 records over that line, and the added
content is distinct findings (a second subject, a migration analysis and three
residuals), not redundancy against a sibling.

Local gate: ErsatzTV.Tests 2091 passed / 6 skipped (the three fixtures' MySQL
halves), Core.Tests 697/1 -- 0 failures. Format clean, no BOM. decisions
validate OK.

Refs #823
Refs #824

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 21:00:22 +02:00
timothyandClaude Opus 5 ea888011aa fix(823): the decision record argued BOTH readings — and the per-dimension mutant that survived the fixture
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 16s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Successful in 16s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
review-verdict/h10 Awaiting review verdict for ea88801
Review verdict / Set review-verdict status (pull_request_target) Successful in 18s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 7m4s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m55s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 4m22s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Round-three review findings. One HIGH, and it was in the durable artifact
rather than the code.

THE HIGH: the record stated the shipped reading and its inverse.

The semantic reversal (empty -> unrestricted) rewrote the residual and the
write-half of `media.nullable-primitive-collection-mutation` but left the
ORIGINAL reasoning standing two sentences earlier: "A null reads as EMPTY, so
the item matches nothing"; "the REJECTED alternative was the All*() set";
"SKIPPING the row is the conservative repair". The shipped code is
`?? AllDaysOfWeek()` -- precisely the alternative that passage calls rejected.
The previous commit then inserted residual (1), which reasons entirely FROM
the All*() reading, two sentences after the sentence denying it.

That is worse than a stale comment. A session resolving this key -- or reading
the MemPalace mirror, which carries `rule:` verbatim -- would have been told to
write the guard the other way, i.e. talked into the `[]` reading that the same
record elsewhere argues is data corruption one save later. Replaced the whole
passage, then swept the record for every other mention of the empty reading
rather than trusting the one replacement: the only survivor is the new sentence
that records EMPTY as the rejected alternative, which is the direction that
stops it being re-adopted.

THE MEDIUM: one arrangement did not close the hole it claimed to.

The discriminating control added last commit nulls DaysOfWeek against a
restrictive MonthsOfYear. It excludes "any NULL matches unconditionally" only
for that dimension. The review supplied the surviving mutant --
`if (item.MonthsOfYear is null) { return item; }` ahead of the checks -- and
traced it green through all nine tests. Verified by EXECUTION, not by reading:
applied to the previous fixture it passes; applied now it FAILS 1 of 11. Each
of the three dimensions is now nulled against a restriction on a different
dimension.

The rest, all from the same round:

- The coverage guard's test detection listed attribute TYPES, and each list
  falsely reddened whatever it omitted: TestAttribute alone missed [TestCase],
  and the three-type replacement missed [Theory]. Now decided by NUnit's own
  ITestBuilder/ISimpleTestBuilder interfaces, which cannot fall behind the
  vocabulary. It also dropped BindingFlags.Static (GetMethods() defaults to
  including it), which would have falsely reddened a static test method.
- The same guard accepted an ABSTRACT fixture -- NUnit never instantiates one.
  The indexer population already filtered IsAbstract; the fixture side now
  mirrors it.
- The record's `mechanics:` still described the old `[Test]`-only clause, in
  the same file the change edited.
- An <inheritdoc> made the ProgramScheduleAlternate empty-case test inherit a
  docstring written from the PlayoutTemplate test's viewpoint.

Two more mutations executed:
- `if (item.MonthsOfYear is null) return item;` -> 1 red, 10 green. This is the
  mutant that survived the previous head; it no longer does.
- an abstract type named in the covered set -> coverage guard red.

Local gate: ErsatzTV.Tests 2091 passed / 6 skipped (the three fixtures' MySQL
halves, skipping visibly without ETV_TEST_MYSQL_CONNECTION), Core.Tests 697/1
-- 0 failures. Format clean, no BOM on the touched set. decisions_validate OK.

Refs #823
Refs #824

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 20:43:07 +02:00
timothyandClaude Opus 5 6200713965 fix(823,824): close the review round's findings — a discriminating control, the second mapper's empty case, and honest test detection
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 19s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 14s
review-verdict/h10 Awaiting review verdict for 6200713
Review verdict / Set review-verdict status (pull_request_target) Successful in 11s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 7m53s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 7m59s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Follow-up commit (the branch is pushed, so not an amend). Two more cold
reviews landed on the previous head; both reported 0 Blocker and 0 High, and
these are their Mediums and Lows. Each fix carries its own witnessed mutation.

1. The selector fixture could not tell the fix from a much broader one.
   Every null test set a NULL and expected the item SELECTED, so all of them
   pass equally under "NULL means unrestricted" and under "any NULL makes this
   item match unconditionally" -- a refactor short-circuiting the whole date
   check on any null kept them green. Added the discriminating control: a NULL
   DaysOfWeek paired with MonthsOfYear = [1] against a MARCH date must be None.
   Only the narrow reading passes.

2. A_Null_Item_Does_Not_Disturb_Selection_Of_A_Later_Item never measured its
   own docstring. The nulled item was unrestricted and at Index 0, so it always
   won and the second item was never evaluated -- the stated invariant ("a null
   on the first item must not decide the second") went unmeasured while the
   test passed. Split into two: one where the nulled item genuinely does not
   match, which measures that the loop CONTINUES; and one that pins the
   index-order win separately.

3. The empty-preservation control existed for one of two identical mappers.
   The anti-mutant test for "empty or null becomes All*" covered only
   Playouts.Mapper; Scheduling.Mapper is a byte-identical triple in another
   file and had none, so a defensive edit to it alone would have rewritten a
   deliberately-empty user selection to 1..31 with the suite green. That is the
   one-helper-two-callers shape this repo has been bitten by. Added the
   matching test.

4. The coverage guard's [Test] clause did not check what its message claimed.
   GetMethods() without BindingFlags returns INHERITED methods, so a fixture
   that merely subclasses another satisfied it while driving the wrong indexer
   -- and Values.Distinct() cannot catch that, since the two Types differ. It
   also matched TestAttribute alone, so a future fixture written as [TestCase]
   would have falsely reddened, and it accepted an [Explicit]/[Ignore]d fixture
   that never runs, which is the "wired is not running" failure the guard
   exists to prevent. Now DeclaredOnly, the full test-method vocabulary, and
   Explicit/Ignore rejected at both method and fixture level.

5. Three residuals recorded on media.nullable-primitive-collection-mutation
   that the previous head asserted nothing about:
   - the LOUDNESS change, worst for an all-three-NULL ProgramScheduleAlternate,
     which now matches unconditionally and shadows the default schedule where
     it previously threw. Unreachable today, and a choice over an unreachable
     state rather than a measured requirement -- said plainly.
   - the normalization is ONE-WAY and WHOLE-LIST: both PUT paths are full
     replaces, so editing any row persists All*() over EVERY NULL row in that
     playout, and afterwards "the operator selected all 31" and "this is a
     legacy row" are indistinguishable. An ordinary user action closes that
     door.
   - the WRITE side disagrees with the READ side about what ABSENCE means: an
     omitted daysOfWeek normalizes to [] ("never applies") while a NULL column
     reads as unrestricted, so an API client gets HTTP 200 and a row that
     silently never fires. Filed as #880 rather than folded in here, because a
     client omitting a field on a write is a different question from what a
     legacy NULL meant.

Two more mutations executed, both witnessed:
- DaysOfWeek guard disarmed in Scheduling.Mapper -> 1 red, 3 green.
- A fixture with no DECLARED test named in the covered set -> coverage red.

Local gate (MySQL lane armed): ErsatzTV.Tests 2097 passed / 0 skipped,
Core.Tests 695/1 -- 0 failures. Format clean, no BOM on the touched set with
the population count asserted. decisions_validate OK.

Refs #823
Refs #824
Refs #880

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 20:27:33 +02:00
timothyandClaude Opus 5 95b2700f09 fix(823,824): a scheduling NULL collection reads as UNRESTRICTED and is guarded at both read sites; the Elastic indexer gets its own mutation proof
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 6s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 18s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 20s
PR Gates / decisions lifecycle (pull_request) Successful in 20s
PR Gates / Docs update reminder (pull_request) Successful in 21s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 11s
review-verdict/h10 Review-verdict: MERGEABLE @ 95b2700 (base: main)
Review verdict / Set review-verdict status (pull_request_target) Successful in 23s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 8m6s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m55s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m35s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 2m56s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Both issues are #701 deferrals, and they land together because both rewrite
the same decision record.

#823 -- can a null reach one of the six collection-valued scalar columns?

MEASURED against a real TvContext on BOTH providers (SQLite, and MySQL 8.4
on an ephemeral server), because the reasoning available beforehand pointed
the wrong way. The two converters differ on their read side --
IntCollectionValueConverter maps null-or-blank to Array.Empty<int>(), while
EnumCollectionJsonValueConverter would dereference the result of
JsonConvert.DeserializeObject -- so the expectation was that a NULL row
behaves differently per column. NEITHER RUNS: EF does not invoke a value
converter for a NULL column at all. All six materialize as CLR null, the
int converter's null-to-empty branch is dead on this path, and unguarded
each .Contains in AlternateScheduleSelector throws NullReferenceException.

A NULL reads as UNRESTRICTED -- the All*() sets -- not as empty. This is
the whole semantic question and the first draft got it backwards. It is
decided by the one NULL reachable WITHOUT any code writing one: Sqlite's
20240113140741_Add_PlayoutTemplate_DaysOfMonth adds the column with
nullable:true and NO defaultValue, so a PlayoutTemplate row inserted before
it holds NULL and by construction had no day-of-month restriction. Reading
that as empty INVERTS the row's meaning and silently stops the template
applying at all. All*() preserves it, and is how "no restriction recorded"
is already represented (GetPlayoutAlternateSchedulesHandler,
PreviewBlockPlayoutHandler). What does NOT decide it, and was wrongly cited
in the first draft: the API request records normalize an omitted field with
`?? []`, but that is a client omitting a field on a WRITE and says nothing
about what a legacy database NULL meant.

Two read sites, not one. Guarding only the selector would have left the
entity->DTO mappers unguarded, and those feed the SPA: PlayoutScheduleEditors
spreads the collection (`[...template.daysOfMonth]` -> TypeError on a JSON
null) and playoutTemplateCalendar's appliesToDate -- an exact port of
GetScheduleForDate -- calls .includes on it. Both mappers now substitute the
SAME defaults, so the preview agrees with what is actually scheduled. Neither
guard is assigned back onto the entity, which is the
media.nullable-primitive-collection-mutation mechanism.

Reachability, stated precisely rather than overclaimed. All six are
nullable:true on both providers, but a nullable column does not produce a
NULL row: five of the six were present at CreateTable, so a NULL there still
needs code to write one, and on MySQL there is NO code-path-free NULL for any
of the six. The write path ACCEPTS a null (SaveChanges succeeds, stores SQL
NULL) but no caller supplies one today -- every production construction of the
two commands goes through the request records. That is a property of the code,
not a live caller; claiming otherwise would be the banned "it's AsNoTracking
today" argument pointed the other way.

#824 -- ElasticSearchIndex.UpdateSong had no regression test

Issue option 1 (a non-network transport) shipped, and needed no new package:
Elastic.Transport.InMemoryRequestInvoker is public in the pinned version and
ElasticsearchClientSettings(NodePool, IRequestInvoker) accepts it, injected
into the private _client the way #701 injects the Lucene IndexWriter.
UpdateItems never runs `_client ??= CreateClient()`, so the injected instance
is the one used.

Two traps there are load-bearing, both measured: the canned response must
carry an `X-Elastic-Product: Elasticsearch` header or the client's product
check throws UnsupportedProductException INTO UpdateSong's catch, and an empty
body fails to deserialize the same way. Either turns the fixture into a green
measurement of the error path -- which is how it first failed here, caught by
the ThrowOnWarningLogger. The document id is asserted as the LAST PATH SEGMENT,
not by substring: the index name carries digits, so ShouldContain would stop
discriminating for a song whose id collided with one.

Six mutations executed, each disarming ITS OWN clause alone:

- `??=` restored in ElasticSearchIndex only -> the Elastic fixture reddens on
  "metadata.Artists should be null but was []" while the LUCENE fixture stays
  GREEN. The #824 hole demonstrated, not described.
- DaysOfWeek guard disarmed in the selector -> 4 red, 3 green (DaysOfMonth and
  MonthsOfYear unaffected). Each clause is independently load-bearing.
- DaysOfMonth guard disarmed in Playouts.Mapper -> 1 red, 2 green.
- Elastic dropped from the covered set / mapped to the SAME fixture as Lucene /
  mapped to a class with no [Test] -> SearchIndexMutationCoverageTests reddens
  on each.

That coverage guard is the boundary fix the issue asked for: the covered set is
compared against an ISearchIndex population DERIVED FROM THE ASSEMBLY. Its claim
stops where the check does -- no static check can establish that a named fixture
actually DRIVES its indexer, so it forces a human to look rather than proving
coverage. ThrowOnWarningLogger moved to ErsatzTV.Tests/Support so both fixtures
share it; the Lucene fixture's assertions are otherwise untouched, since it is a
witnessed proof artifact.

No production change in ElasticSearchIndex.cs -- #824 is coverage only.

Docs: testing.md gains a "Provider-parity fixtures" section naming all THREE
opt-in-MySQL fixtures and recording that CI runs none of them (#627);
docs/README.md gains the matching task signal; guard-inventory.md's
hand-written C# guard list goes from five files to six. Scheduling/Mapper.cs
loses the UTF-8 BOM it inherited, per #311 fix-as-you-touch.

Local gate (with the MySQL lane armed): ErsatzTV.Tests 2096 passed / 0 skipped,
Core.Tests 693/1, Infrastructure.Tests 114, Architecture.Tests 7, Scanner.Tests
1504 -- 0 failures in each. scripts/tests 1228 passed / 2 skipped. dotnet format
whitespace --verify-no-changes clean; BOM check over the touched set with the
population COUNT asserted, because a bare zsh loop silently checks one
concatenated filename. decisions_validate OK.

Fixes #823
Fixes #824

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 20:02:52 +02:00
timothyandtimothy 8aeacd534a fix(819): derive the SPA page-size guard population from the git index (#875)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 21s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m52s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m23s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m5s
The guard asserted EXACT completeness over a population enumerated by a directory
walk, so an untracked .ts/.tsx under web/src/ entered it and failed as unregistered
on that developer's checkout while CI — which only ever checks out tracked files —
stayed green.

The glob still supplies file CONTENT; the POPULATION is now the git index, read by
web/vite-plugins/trackedSourceFiles.ts in Vite's own Node context and handed to the
app project as a virtual module. That reaches the index without admitting
@types/node to tsconfig.app.json, the obstacle that deferred this in #818.

Three mechanisms carry the proof, each added because the previous was measured
insufficient: a closed-form restatement of the shared scope predicate (sharing no
helper at any depth with what it checks); a second independent `ls-files --others`
query cross-checking the population; and real-git tests that execute the derivation
against a temp repository.

Six residuals are stated with their MEASURED fail-directions, and
testing.guard-derives-population-from-source gains a bounded exception plus the
closed-form criterion.

fixes #819

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-29 09:14:15 +00:00
timothyandtimothy b6b3520bdb fix(809,822): isolate the suite from the production hook-fire log by construction (#874)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m32s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m12s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m14s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m32s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-29 02:32:26 +00:00
90f96c14a5 fix(803,664): fence the HEAD alias on the PR timeline's pull_push count (#873)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 15s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m16s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m39s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m37s
A force-push H1 -> H2 -> H1 spanning `pr-changed-files.sh`'s paging leaves its final
`.head.sha` comparison equal while the middle pages came from H2, so a mixed file list
could produce a docs-only exemption `success` no single head ever justified. The base
alias had been fenced since #706 by a monotonic `change_target_branch` count; the head
axis had nothing, and three contracts asserted otherwise.

`count_retargets` becomes `count_pr_mutations`: one timeline walk, two tallies, one shared
trust flag, a separate fence arm and diagnostic per axis. The advisory hook re-reads
`.head.sha` at the same hoist and off the same response as the base re-read. All three
overclaiming contracts are corrected, plus four paraphrases the first sweep missed.

Measured, not assumed: Gitea 1.27.1 still serves no `files` on `compare/{base}...{head}`;
every push is a `pull_push` event and its count cannot alias; PR #761 really went
`8798a1d -> 830a407 -> 8798a1d`; and Gitea creates the push comment BEFORE emitting the
synchronize notification, so a run cannot abstain on its own trigger.

Two pre-existing fail-opens in the shared walk were found by review and fixed: an empty
ARRAY first page was trusted on any page while the `null` arm required `page > 1`, and no
row was validated before `.type` was selected on.

NOT closed, and documented rather than overclaimed: the walk's `null` terminator is
defeatable, because Gitea pages before it filters (#870). The fence closes the ABA on a
timeline with no truncating block, not the ABA outright.

fixes #803
fixes #664
Refs #870

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 23:29:47 +00:00
timothyandtimothy 4b3f6f6c90 fix(786,789): workflows declare their own per-job metadata; guard populations derive from it (#872)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m21s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m58s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m8s
Closes #786 and #789, bundled because working either alone would build the artifact the other removes.

Every job in all six tracked workflows declares `env.CI_JOB_ROLE` (guard/report-only/none); the
`docker-build.yml` jobs also declare `env.CI_EXECUTION_CLASS` (toolchain/bare-runner). Both guard
populations derive from those markers; the `TOOLCHAIN_JOBS`/`BARE_RUNNER_JOBS` literals are deleted.
A missing or unrecognised marker is a hard failure in both checkers.

#789's literal had a real justification — set equality between two DERIVED sets is blind to a member
leaving both at once — so the marker is the anchor that replaces it, and the cost (proximity to the
`container:` block) is paid by a THIRD derivation from each job's own steps, which is also the only
check that sees the failure #789 filed: a .NET step moved into a bare-runner job, where no set
changes. The residual is disclosed: drop the block, flip the marker AND hide the tool behind a
script and all three go blind, bounded by the failure mode being a loud missing-binary crash.

#786's guard jobs join a machine-checked population: a new `test_workflow_job_guards.py` asserts set
equality both ways against a new "Workflow-job guards" table, and the four jobs with no dropped-step
guard each carry a recorded decision.

Two issue claims were refuted by measurement: #789's "editing docker-build.yml re-points the pin"
(the pathspec is `docker/ci` only) and #786's job count (17, not 15).

Four cold adversarial review rounds across two model families; rounds 1-3 BLOCKED, all findings
fixed and each fix demonstrated by reproducing the reviewer's own test. The recurring defect class
was prose drifting from code, including a mechanism claim in the decision record that execution
refuted. All five mutation proofs redden when their shipped detector is disarmed.

New decision record: `testing.workflow-declares-its-own-job-metadata`.

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 20:54:53 +00:00
timothyandtimothy e11d577193 docs(796): verification code is code under test, and the proof it was claiming an exemption from (#871)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m38s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m9s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m18s
Records `testing.verification-code-needs-its-own-proof`: the proof obligation follows the
VERDICT rather than the file, so it binds harnesses, wrappers, timeouts and checkers — not
only the files the guard population derives.

The issue asked for a stated position on whether non-guard checker scripts get mutation
proofs. The position as first written claimed `scripts/mcp_smoke.py` "cannot participate"
because driving it needs the gitignored `.mcp.json` and a cold-built language server. Cold
review refuted that by execution: it takes its config path and server name as positional
arguments. The record had failed its own headline rule on the one claim its decision rested
on, so this ships the proof instead of the exemption.

- `scripts/tests/test_mcp_smoke.py` — a hermetic stub JSON-RPC responder and six cases
  pinning the defects the checker has already had, with the positive control as a fixture
  the refusal tests depend on, so a node-id or `-k` selection cannot skip it.
- A declared clause in `mutation_manifest.py` targeting the unguessable request id, using
  the `guard=test / target=script` shape that already exists for `mutation_harness_lib.py`.
  Witnessed red: `id_init = 1` makes the pre-answer accepted at `initialize` (rc 9 -> 10),
  and only that test moves.

`mcp_smoke.py` still gets no inventory row — one is rejected as a phantom (measured). The
row goes to the test file, which joins the derived population automatically.

Five cold-review rounds, four BLOCKED. Round 2 caught a `ruff format` red that would have
failed `script-tests`. Rounds 3-5 found only hand-maintained counts and uniqueness claims in
prose, three of them created by the previous round's fix; that class was deleted rather than
corrected again, per this record's own stop-and-subtract rule.

Docs updated in the same PR: `docs/README.md` task-signal map and `docs/guard-inventory.md`
(row, summary counts, scope-limit item 6).

fixes #796
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 20:06:45 +00:00
timothy 609fd852c2 fix(763): page both /statuses/{sha} reads to a validated terminator (#868)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 20s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m45s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m10s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m2s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m9s
2026-08-28 17:34:48 +00:00
timothyandClaude Opus 5 11287a54ba fix(763): satisfy ruff — E741 and formatting on the touched test file
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 10s
review-verdict/h10 Review-verdict: MERGEABLE @ 11287a5 (base: main)
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 19s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 19s
PR Gates / Docs update reminder (pull_request) Successful in 16s
Review verdict / Set review-verdict status (pull_request_target) Successful in 20s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 6m51s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m5s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m6s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
CI's `Script lint and tests` job went red. Cause: I never ran ruff locally,
which this repo's Python convention requires after any .py change.

  - E741 twice: `l` as a comprehension variable in the sort-order guard.
  - `ruff format --check`: the file was correctly formatted on `main`; my edits
    broke it. One of them left a docstring line at column 0, which `ruff format`
    then "corrected" by over-indenting the rest of the paragraph — repaired at
    the source rather than accepting that rewrite.

Verified the way CI does: local ruff is the pinned 0.12.11, and both
`ruff check` and `ruff format --check` run under bash over the full tracked
population (`git ls-files -z '*.py' '*.pyi' '*.ipynb'`, 46 files) are clean.
The population is counted, not assumed — an empty glob would pass vacuously,
which is the failure `scripts/tests` guards against elsewhere.

`scripts/tests` 1097 passed, 2 skipped after the reformat.

refs #763

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 18:51:32 +02:00
5fb9c8537a docs(747): re-verify the Gitea 1.25.4-pinned CI claims on 1.27.1, and measure the merge-gate semantics that were source-attested (#867)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 17s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m25s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m48s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m10s
Population derived from `git ls-files`, not the issue's 9-key list (~21 claim sites).

Re-confirmed unchanged on 1.27.1: the distinct `skipped` commit-status state; `compare` serving
no `files`; no agent-side cancel route (REST route + swagger only); `branches: [main]` suppressing
the run off a non-main base.

Newly measured on four throwaway scratch bases, `main`'s rule never PATCHed: an absent required
context blocks an ORDINARY merge without needing `block_admin_merge_override` (that field governs
the FORCE path only), and `enable_bypass_allowlist` with an empty list is NOT a substitute for it.
Trap recorded: the PR API reports `mergeable: true` while such a merge is refused.

Left explicitly dated with reasons: push-supersession auto-cancel, `pull_request_target` overlap,
`--depth=1` no-merge-base, and the scope-enum/`reqRepoWriter`/403 items. Not a corpus sweep, and
`ci.actions-credential-scoping` now says so. `review-verdict.yml` untouched — #763 holds that file.

Five adversarial review rounds (21/12/9/6/2). Caveat: all same-model-family; Codex was rate-limited.

fixes #747

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 16:51:13 +00:00
timothyandClaude Opus 5 462118d98d fix(763): round 9 — assert the repair's reason positively, not by absence
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 22s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 17s
PR Gates / Docs update reminder (pull_request) Successful in 21s
review-verdict/h10 Review-verdict: MERGEABLE @ 462118d (base: main)
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 23s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Failing after 14s
Review verdict / Set review-verdict status (pull_request_target) Successful in 7s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 1m31s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
A sixth cold review found everything in round 8 clean except one line, and it
is the rule this branch keeps rediscovering: the test pinned the new
`raced_why` only by asserting the ABSENCE of the borrowed wording. Measured —
replacing the string with `zzz` left the suite green while an operator would
get `::error::… — zzz.` beside a sticky sentinel. The sibling test 330 lines
away states the rule and follows it; this one did not.

Now asserted positively, with the em-dash and full stop discriminating the
`::error::` reason from the `::warning::` text that continues ", which cannot
be true". The `zzz` mutation reddens it.

Three nits from the same review, all verified by execution rather than reading:
  - the earlier fixture's row was excluded by the strict `> $since` because the
    mark became its OWN id, not because it sat below the mark.
  - the predecessor comment said `main` "warned only on `null`". True of the two
    EMPTY shapes being contrasted; an empty body and a non-array object warned
    as well. Scoped.
  - `docs/ci-cd.md` and the record described the `::error::` as a two-way split
    (found vs unverifiable). Round 8's whole argument is that a complete read
    returning an IMPOSSIBLE answer is a third case, not a variety of the second
    — which is the operator-facing point, since it decides whether to go looking
    for an API failure that never happened. Both now say three.

The review re-verified, by comment-stripped diff, that round 8 changed no
executable line beyond the `raced_why` string and the if/elif restructure, and
independently reproduced both inertness measurements and the `origin/main`
predecessor behaviour.

Verification: `scripts/tests` 1097 passed, 2 skipped; decisions_validate and
build_decisions_catalog --check exit 0.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 18:42:22 +02:00
timothyandClaude Opus 5 4164efbc3e fix(763): round 8 — a fixture must pose the scenario it narrates
A fifth cold review confirmed the gate's behaviour is correct and proof-backed,
and blocked on three non-behavioural items. All three fixed; none touches the
shipped logic.

MEDIUM — the round-7 fixture narrated a raced human verdict it did not
construct. `null-page1-after-post` appended the row unconditionally, so it also
joined the PRE-write read and lifted the high-water mark above itself; removing
it changed nothing. The reviewer's suggested fix was to gate the append on the
post-write read. Measured after gating: still inert, because page 1 answers
`null` before any row reaches the wire.

So the row is gone rather than gated, and the prose now describes what the
fixture actually poses: a response asserting an empty history for a sha this job
wrote to must not be accepted as proof that nothing raced. Whether a verdict
really raced is not modelled and does not need to be — the response is not
evidence either way. A row the test cannot observe is decoration that reads as
coverage, which is the same class this branch has now been blocked on five
times.

LOW — the comment claimed the predecessor "at least produced a `::warning::`".
Half false, measured against `origin/main`: its `jq -e 'type == "array"'` gate
ACCEPTED `[]` silently and warned only on `null`. What is actually new is that
the paged walk reports such a read as a SUCCESS.

LOW — when the empty clause fired it set `ph_ok=no`, so the log said "could not
be read completely" beside a walk that completed on a validated terminator. The
answer was impossible, not unreadable, and an operator holding a sticky sentinel
needs to know which. It now carries its own `raced_why`, asserted by the test.

Both clauses mutation-proved: disarming the empty check, and reverting to the
borrowed wording, each redden the named test.

Verification: `scripts/tests` 1097 passed, 2 skipped; decisions_validate and
build_decisions_catalog --check exit 0.

refs #763

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 18:18:37 +02:00
timothyandClaude Opus 5 e6f04cc739 fix(763): round 7 — reject an empty post-write history, and the fourth overclaim
A fourth cold review returned NOT-MERGEABLE on two Mediums. Both fixed, plus
its three Lows.

MEDIUM, and a defect this branch introduced. Tolerating a `null`/`[]` page 1 as
"complete, zero rows" is correct for the PRE-write caller — a head nothing has
posted to genuinely has no statuses — and impossible for the POST-write one,
which has just written a row to that sha. The body is well-formed, so nothing
retries it, and the walk reports success: `raced=0` concluded from a list that
cannot be real, on the one path whose failure direction is toward SUCCESS.
Worse than the code it replaced, which at least emitted a `::warning::` — a
logged fail-open had become an unlogged one. Reviewer measured both directions.

The post-write caller now rejects an empty result itself; the walk stays
caller-agnostic because the pre-write caller genuinely needs the empty answer.
This is NOT the withdrawn currency witness: that asked whether ANY row sat above
the mark, which an unrelated newer row satisfied while the rejection stayed
hidden, and it fired on schema-valid staleness. This asks only whether the list
is EMPTY — a state no unrelated row can produce and no ordering can disguise.
It carries neither defect. Proved by fixture; disarming it reddens the named
test, and the previously-uncovered `null`-at-page-1 clause is now covered too.

MEDIUM — the fourth overclaim of the same class, in the decision record body:
"Uncertainty must fail closed at both ends … Both repair now." The page-2 probe
was DELETED, not converted; it repairs nothing. It also contradicted the
record's own `rule:` ("the two directions are NOT symmetric") and the bullet
directly beneath it. Round 5 retracted this wording in `docs/ci-cd.md` only —
the sweep was by subject, not by the retracted words.

Also fixed:
  - the record presented "an empty FIRST page is legitimate" as a property of
    the walk; it is a property of the pre-write caller.
  - `docs/ci-cd.md` called the numeric-only id comparisons a fix for mark
    inflation; they are a TYPE guard, closing the string half. A corrupt but
    genuinely numeric id still inflates the mark — not attacker-controllable,
    since ids are server-assigned, and now stated rather than implied.
  - `test_a_partial_mark_is_SAFE...`'s self-guard promised to detect that the
    fallback ran; it keys on a warning emitted by a different condition, so
    deleting the fallback left it green. Its sibling is what reddens; the
    message now says what it actually pins.
  - the order-faithful fixture appended the job's own POST after the reversal,
    serving the NEWEST row on the OLDEST page — the opposite of DESC, in the one
    fixture that exists to be ordering-faithful.
  - "twice per walk" for the wasted sleep; it is once per walk, twice per run.
  - a dead counter read in the DESC mode.

Rebased onto b16ec15d6 (the other session's #781/#799 docs work; no file
overlap, no conflicts).

Verification: `scripts/tests` 1097 passed, 2 skipped; fifteen executed mutations
across rounds 2-7; decisions_validate and build_decisions_catalog --check exit 0.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:47:06 +02:00
timothyandClaude Opus 5 89db6022d9 fix(763): round 6 — measure the partial-mark fallback under the REAL ordering
A fourth cold review (Opus, isolated worktree, tests/double/docs focus)
reported no correctness bugs in shipped behaviour but two coverage defects on
exactly the two things this change advertises. Both are closed.

The partial-mark fallback's safety is a claim ABOUT THE ORDERING — page 1 holds
the newest rows, so a walk that fails later still saw the true maximum. The
fixture pinning it served ASCENDING ids, i.e. the arrangement the design calls
unsafe, and passed anyway because the raced row's id sat above even the partial
mark. It could not distinguish safe from unsafe.

The stub now HONOURS the sort parameter: order-faithful modes serve DESC by
default and ASC when the request asks. The new fixture holds a PRE-EXISTING
base-mismatched verdict at id 7055 among 60 rows. Under DESC the salvaged mark
is 7059 and that row is below it — the exemption correctly stands. Under ASC
the mark would be 7049 and that untouched row tests as NEWER, a sticky repair
on a head nothing raced. So re-adding `sort=highestindex` now reddens by
BEHAVIOUR, not only by the structural assertion added in round 5. Measured:
re-adding it reds both tests.

Most modes stay ordering-blind on purpose and now say so: they test walk
COMPLETENESS, which is order-independent, and insertion order is what lets a
fixture place a row beyond page 1.

Also fixed:
  - `null` is accepted as an empty page. An array-only gate is the exact shape
    of #751 — `count_retargets` had one, the timeline really did return `null`
    past the end, and the fence withheld EVERY exemption from the day it
    shipped. The same narrowing here is worse, because this walk's failure is
    the STICKY sentinel: every exempt PR would need a hand-posted verdict, per
    head. Tolerating `null` cannot misread `[]`. Proved by fixture.
  - the fail-closed comment said "past the 1000-row page cap"; the bound is 950,
    as the walk's own comment and both docs already said.
  - the docs claimed "only a read returning no rows at all abandons the mark".
    False: a VALIDATED empty history yields a mark of 0 and is not abandoned —
    that is the normal first run. What abandons it is a read that both FAILED
    and returned nothing. Corrected in ci-cd.md and the record `rule:`.
  - a comment pointed at the page-2 probe "a few lines further down"; it was
    deleted, so the deixis pointed at nothing.
  - the stub claimed its logical-read counter "is only reached on a SUCCESSFUL
    page-1 serve" — measured false; it counts page-1 requests, retries included.
  - five `(round N)` markers removed. A round number is session chronology and
    does not parse for a reader who never saw it (`docs.no-session-narrative`);
    an issue number does. The four that remain predate this change.

Verification: `scripts/tests` 1096 passed, 2 skipped. Thirteen executed
mutations across rounds 2-6. The reviewer independently re-ran the earlier
matrix and confirmed it, with one correction carried here: two of those
mutations redden MORE than their named test, so "each reddening exactly its
named test" was wrong — they redden at least it.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:08 +02:00
timothyandClaude Opus 5 53ee44cbc1 fix(763): round 5 — correct the overstated motivation, and fix the string-id twin
A third cold review (Opus, isolated worktree) returned NOT MERGEABLE with one
High and three Medium. All are addressed.

HIGH — the stated motivation was wrong, and self-contradictory once round 4
landed. Under the server default (`created_unix DESC`) page 1 holds the NEWEST
rows and ids are monotonic with `created_at`, so page 1 already carried the
true maximum id AND every row newer than the mark — the only rows the
post-write check selects on. A single-page read therefore missed a raced
verdict only if more than 50 rows were created INSIDE the write window, not
merely on "a head with more than 50 rows", which the issue, the comments and
the docs all asserted. Reviewer executed an order-faithful DESC stub: a
page-1-only reader repairs identically to the full walk.

What actually removed #761's stall is retiring #751's page-2 probe, not the
paging. The walk still earns its place, for a reason now stated instead of the
false one: it stops the gate's one fail-toward-SUCCESS path depending on an
undocumented ordering the server honours only coarsely (page 1 came back
`114,112,113,111,110`). That measurement was deleted in commit 1 and is
restored, since round 4's safety argument rests on exactly it.

MEDIUM/real defect — the string-id TWIN, live on `main` and one expression
away from the fix already made: `select((.id? // 0) > $since)`. jq orders
strings above every number, so a PRE-EXISTING row with `"id": "3"` reads as
newer than any mark, is counted as having raced the write, and gets the sticky
sentinel plus a false "was overwritten" on EVERY later run — a permanent
per-sha stall no re-trigger clears. Now numeric-only, with a test.

Also fixed: a non-empty history carrying no numeric id was collapsed to a mark
of 0 (making every pre-existing row look newer); it is now reported unusable
and the check is skipped. `sleep` no longer fires after the final attempt.

Three unpinned clauses now have tests, each proved by an executed mutation:
  - the page cap is a refusal, not a terminator (1050-row fixture)
  - the `::error::` found-vs-unverifiable distinction (forcing `raced_why=human`
    reddened nothing before)
  - the walk requests no sort order — a structural guard on round 4's
    withdrawal, which nothing mechanical protected. It reads request LINES, not
    comments, since the withdrawal note names the parameter to explain it.

Honest scoping, not new code: the test double is ordering-blind, so the paging
tests prove WALK COMPLETENESS, not that a real raced verdict would otherwise be
missed — under DESC it would not be. The stub comment and the docstrings now
say so rather than implying the stronger claim.

Docs: `ci-cd.md` and the record's `rule:` carry the corrected reachability, the
DESC dependency of the partial-mark fallback, and both rejected alternatives
stated as rejected alternatives rather than as draft chronology
(`docs.no-session-narrative`).

Verification: `scripts/tests` 1094 passed, 2 skipped; eleven executed
mutations across rounds 2-5, each reddening exactly its named test;
decisions_validate and build_decisions_catalog --check exit 0.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:07 +02:00
timothyandClaude Opus 5 1eb7b41fc1 fix(763): round 4 — withdraw the ASC sort, which inverted the partial-mark fallback
Round 3 added `sort=highestindex` to close a mid-walk-insert gap: under the
server default (`created_unix DESC`) a row inserted while the walk is running
lands at position 0, on a page already read, so the walk never sees it.

That fix and the round-2 partial-mark fallback are incompatible. ASC puts the
OLDEST rows on page 1, so an incomplete walk takes its high-water mark over
the oldest rows — leaving every pre-existing row above the mark and read as
"raced". That is a spurious STICKY repair on a head nothing raced, which is
precisely the #761 failure this whole issue exists to remove. Under the
default DESC the newest row is on page 1 by construction and ids are monotonic
with `created_at` (measured), so a partial mark is at or very near the true
maximum and "lower is safe" actually holds.

Two defects from one mechanism again, so the mechanism goes rather than
getting patched: the sort is withdrawn and the mid-walk-insert residual is
ACCEPTED and documented. It is bounded — a row arriving after this job's POST
is not one this job overwrote, and being newest it wins on the combined
endpoint branch protection reads.

Both the code comment and the docs record the withdrawal and the reason, so
the next reader does not re-adopt it.

Verification: `scripts/tests` 1090 passed, 2 skipped; the partial-mark mutation
still reddens `test_a_PRE_WRITE_paging_failure_still_yields_a_usable_high_water_mark`;
decisions_validate and build_decisions_catalog --check both exit 0.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:07 +02:00
timothyandClaude Opus 5 007d2fd3df fix(763): round 2+3 — close the fail-opens the paging change introduced
Two independent cold reviews (Codex GPT-5.6 cross-family, and an isolated
Opus agent) converged on the same blocker, which is fixed here along with
everything else they found.

BLOCKER — the mark walk turned a fail-closed case into a fail-open. The
high-water mark gates the post-write race check entirely: `max_id_before=-1`
skips it. Before paging, only a failure of the single page-1 request could
reach that. Requiring a COMPLETE walk newly routed a page-2 hiccup, an
over-cap history, or one malformed id on a later page into the same hole, so
a human rejection racing the write was left green where `main` repaired.
A partial list now still yields a mark: it can only be LOWER than the true
maximum, which makes the check more eager, never blinder. Only a read
returning no rows at all abandons it — the pre-existing #849 gap, unchanged
and now asserted by a test so it stays visible.

WITHDRAWN — the "currency witness". It produced two defects from one
mechanism, which is the signal to remove rather than patch twice: counting
ANY row above the mark does not witness this job's write, so a stale-but-valid
snapshot carrying an unrelated newer row passed while hiding a rejection; and
a schema-valid stale read is not retried, so one such response turned a
transient anomaly into a permanent sentinel. The hazard has no mechanism here
either — Gitea is a single instance with no read replicas. Removing it
restores the pre-change exposure on that path, a non-regression.

Also fixed, each a fail-open with a fixture and an executed mutation:
  - `.creator` is type-tested before indexing. `.creator.login` on a non-object
    exits jq 5 and `set -e` took the step down after the green was posted and
    before the repair. Reproduced by both reviewers.
  - the mark is the max over NUMERIC ids only. jq orders strings above every
    number, so one `"id": "99999"` passed the numeric gate and inflated the
    mark until nothing looked newer.
  - an unusable `raced` count now repairs instead of "not acting on it".
  - `sort=highestindex` (ASC, measured) so a row inserted mid-walk appends at
    the end rather than at position 0 on a page already read. An unknown sort
    value silently falls back to DESC, so this is insurance, not load-bearing,
    and the comment says so.
  - `ph_ok`/`ph_rows` renamed off `read_existing_verdict`'s `st_ok`. No live
    bug, but a name collision in a 1400-line step.

Tests the reviews showed were missing, each proved by an executed mutation:
  - verdict beyond a SHORT page (a deliberately unfaithful truncated response
    — against a faithful double a short page is always the last, so the rule
    "terminate only on an EMPTY page" was unobservable)
  - pre-write paging failure still yields a usable mark
  - pre-write read returning nothing abandons the mark and says so
  - a TRANSIENT page failure is retried (the retry was unproven code: every
    other error mode fails on every attempt, so disarming it reddened nothing)
  - a string id cannot inflate the mark
  - a malformed `creator` row does not kill the job

Stub corrections, both the same class as the earlier `[]`-vs-`null` gap: it
served one flat list (so paging was unobservable) and computed its own-post id
with `max()` over mixed str/int, which raised TypeError and made the string-id
test pass because the DOUBLE crashed rather than because the mark was right.

Mutation matrix, all executed, each reddening exactly its named test: retry
disarmed; numeric-max reverted; partial-mark fallback removed; short-page
terminates; page-1-only walk; post-write fail-closed flipped open; jq
type-guard reverted. The unusable-count arm is unreachable by any fixture and
is annotated as such rather than claimed as proved.

Verification: `scripts/tests` 1090 passed, 2 skipped; decisions_validate and
build_decisions_catalog --check both exit 0; terminator, clamp, sort order and
id monotonicity all re-measured live on Gitea 1.27.1.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:07 +02:00
timothyandClaude Opus 5 4368cc8cbe fix(763): page both /statuses/{sha} reads to a validated terminator
`review-verdict.yml` read the per-POST status history twice with a single
`?limit=100` request. `limit` clamps to the server-wide `MAX_RESPONSE_ITEMS`
(measured 50), so on a head carrying more rows than the clamp both reads saw a
partial list. The high-water mark was only page 1's maximum, and — the direction
that matters — a raced human verdict beyond page 1 was invisible to the
post-write race check, leaving a forged green over a rejection.

Both reads now walk to a validated empty page (`[]` on this endpoint, measured
2026-08-28 against PR #761's 114-row head: pages 1-2 return 50, page 3 returns
14, page 4 is `[]`), never terminating on a short page, under a 20-page cap and
retrying each page once. Correctness does not depend on the cap value.

This retires #751's page-2 "assume raced" probe, which repaired every head that
outgrew one page. It fired on Renovate PR #761: an `::error::` claimed a human
verdict had been overwritten on a head carrying none, and the sticky sentinel
then refused re-exemption on every later run.

Two properties replace it. Uncertainty now fails closed at both ends — the
unreadable-history branch warned and left the exemption green while the page-2
probe repaired on the same uncertainty, one check disagreeing with itself; this
is affordable only because paging removed the common trigger. And the post-write
read must witness the job's own write: reaching a validated empty page proves the
walk finished, not that it saw a current list, so at least one row above the
pre-write mark must exist because the job just posted one.

The `::error::` now distinguishes a verdict actually found from an unverifiable
read. The sentinel description stays generic — the classification recognises it
as a fixed point, so its wording is load-bearing.

The stub gained faithful paging (50-row slices, `[]` past the end, one snapshot
per logical read so a counter mode cannot describe two different histories across
pages) and, separately, modelling of the job's own POST appearing in the history
— which it had never done, so in its world every ordinary run looked like a head
nothing had been posted to. `own-write-invisible` withholds exactly that detail
as the negative control for the currency witness.

Mutation-proved by execution, one clause at a time:
  - walk reads page 1 only -> RUNNING_PAST_PAGE_1_is_PAGED_and_the_exemption_
    STANDS, raced_verdict_on_PAGE_2_is_detected_and_repaired and both UNREADABLE
    history tests go red
  - currency-witness zero branch deleted -> CANNOT_SEE_OUR_OWN_WRITE red
  - fail-closed flipped to fail-open -> both UNREADABLE history tests red

Verification: `scripts/tests` 1085 passed, 2 skipped; decisions_validate and
build_decisions_catalog --check both exit 0.

fixes #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:07 +02:00
timothyandtimothy b16ec15d6c docs(781,799): re-measure the tooling audit from a derived population, and adopt serena (#862)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 23s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m36s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m10s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m13s
§5.3's verdicts rested on a single surface, which manufactured four false zeros: codex is driven
through `codex exec` inside Bash, security-guidance and ralph-loop expose no tool at all and run as
hooks (1,086 executions each), and feature-dev is used through its agents. The audit also compared
current enablement against historical usage — six of the eight plugins it called "genuinely unused"
were disabled for 16 of the 30 corpus days.

The retirement half of #781 is answered *no* on evidence: the zeros split six ways and only one is
grounds for removal. Eight plugins are kept by operator decision.

#799's observation was correct and its cause is now established. serena was `false` in settings.json
until 2026-08-14T12:31Z, when a concurrent session enabled it; its tools appear in no transcript
before 12:42:54Z. #799's session started at 12:01Z and never reloaded, so its probe correctly found
nothing while the settings file already said `true`. serena is adopted and documented as the third
code-intelligence surface.

Four review rounds, two independent cold reviewers (one cross-family); rounds 1-3 BLOCKED.

fixes #781
fixes #799

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 14:37:34 +00:00
timothyandtimothy 761e575836 fix(787): derive the dropped-step guard's scope, and reconcile its snapshot against the server (#861)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 21s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m45s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m22s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m56s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m33s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-27 22:37:02 +00:00
timothyandtimothy 8aebba4d89 fix(748): declare permissions: on all six workflows, and prove the declaration binds (#860)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 18s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m40s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m4s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m20s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-27 22:02:44 +00:00
timothyandtimothy 45b17e58e1 fix(744,835): ci-image.yml publishes from main only; guard persist-credentials with no exemption (#857)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 23s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m53s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m23s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m15s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m27s
Closes the push route into ci-image.yml (#744) and ships the persist-credentials guard that was waiting on it (#835).

ci-image.yml's push trigger had no branches: filter and was path-scoped to docker/ci/** AND to the workflow file itself. Gitea resolves a push workflow's definition from the pushed ref, so any branch push touching those paths ran that branch's own YAML on a docker-capable runner holding the credential that writes ersatztv:prod and the ersatztv-ci:<sha> five container: jobs execute.

Be precise about what the filter buys: it is loaded from the pushed ref like the rest of the file, so a branch that deletes it re-enables the route. This closes the DRIVE-BY case - publication as a side effect of an ordinary push - and is not a boundary against a writer who intends to run their own YAML. The wider class is #853.

The self-reference left both paths: and ci-image-pin's expected in the same change - a decided tradeoff with both prices stated, not a necessity. Branch publishing moves to workflow_dispatch, probed live: run 2340 on this branch published ersatztv-ci:43b1e45 and left :latest unchanged.

With both mechanical blockers gone, ci-image.yml's checkout takes persist-credentials: false (16 of 16) and scripts/tests/test_workflow_persist_credentials.py holds the convention with NO exemption list - git-index population, declared clause mutation re-run every suite, guard-inventory rows.

fixes #744
fixes #835

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-27 20:44:44 +00:00
timothyandtimothy d68ce42fcb fix(742): inherit an h10 verdict only from an allow-listed reviewer (#850)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 12s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m55s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m25s
`review-verdict.yml` decided whether an existing `review-verdict/h10` was worth INHERITING by
testing `.creator.login != null` — satisfied by any account's credential, including the `renovate`
bot's `RENOVATE_TOKEN`, a `write:repository` PAT that cannot be scoped down the way #697 scoped the
registry credential. The test is now membership in `H10_REVIEWERS="timothy"`, a literal in the
base-resolved definition.

The design that survived 11 cold review rounds:

* `read_existing_verdict` carries TWO flags. `ex_human` (attributable AND allow-listed) gates
  INHERITANCE; `ex_attributable` gates the last-moment re-read, which asks the opposite question and
  must stay broad. Narrowing both — the first draft — makes the job post its exemption over a
  mid-run rejection, and the post-write repair does not cover that.
* The two calls no longer compute an identical predicate, so "changed" is made explicit: the
  state/creator/description triple from the first read is snapshotted and compared.
* The allow-list governs an inherited `success` ONLY. An existing `failure` inherits on
  attributability alone, because inheriting a rejection can only withhold an exemption while
  re-deriving one can turn it green on an exempt PR. A symmetric rule was a measured fail-open.
* The post-write raced check stays broad — not because narrowing would let a rejection go green
  (a real reviewer is on the list by construction), but for the misconfiguration case.

Two mechanisms were WITHDRAWN rather than patched a third time, and both withdrawals are recorded
in `ci.exemption-provenance` so they are not re-attempted: a `::warning::` annotation that produced
three defects in three rounds, and a post-write fix whose generic `pending` would have been
re-derived anyway and which had no retry trigger.

Verified: the inheritance predicate driven against the LIVE Gitea API on a probe-named context,
both allow-list directions; every clause mutation-proven against the shipped file; `scripts/tests`
1012 passed, 2 skipped.

Follow-ups filed: #845 (post-review-verdict.sh does not check its own account is allow-listed) and
#849 (post-write verification: three routes leaving an exemption `success` over a human `failure`,
plus the retarget fence's post-POST gap, plus the prose sweep that lands with the behaviour).

fixes #742

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-27 02:52:02 +00:00
timothyandtimothy ed8b602445 feat(735): bound the numeric FFmpeg profile fields with a 422, and expose readrate pacing (#847)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 11s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m9s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m41s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m23s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 6m23s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 22:05:38 +00:00
timothyandtimothy 469d19852c fix(788): one declarative H10 verdict vocabulary, derived by both sides (#846)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m34s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m17s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m22s
The verdict words lived in two hand-written shell copies — the `case` arms of
post-review-verdict.sh (write) and the POS_RE/NEG_RE regexes of
check-review-verdict.sh (read) — held together by nothing but a comment that had
already gone stale. scripts/lib/review-verdict-vocabulary.sh now declares them
once and both sides derive; neither script enumerates a verdict word any more.

Only the WORD SET moved. The grammar stays in check-review-verdict.sh, where
every #629 false-open actually lived.

No parity test: #774 shipped one and withdrew it after six rounds, because a
regex over shell source is not a shell parser. The proof is behavioural and
graded MUTATION — the harness restores the pre-#788 hardcoded POS_RE each run and
requires it to redden.

Enforcement is a DATA dependency, not a control-flow gate. Review round 1 found a
real fail-open in the first commit: `${#arr[@]}` is nounset-safe only for a
declared-empty array, and under `set -u` that error inside a function called as
`if ! validate` skips BOTH branches — so on the reader (deliberately no `set -e`)
an explicit BLOCKED @ head classified `positive`, exit 0. Validation now sets a
sentinel on its last line and the derived views refuse without it.

Six cold review rounds; rounds 2-6 found no fail-open across differential fuzzing
(4788 / 2612 / 7560 payloads, zero divergences from origin/main's grammar),
sentinel forgery, environment poisoning, declare -p evasion on bash 5.3 and 3.2,
path/symlink resolution and probe TOCTOU. Every malformation fails closed: reader
exit 2, writer exit 1 with nothing posted.

Also corrected: CLAUDE.md and release.review-verdict-gate both enumerated the
vocabulary without LGTM, a word the code has accepted since #629.

fixes #788

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 20:58:09 +00:00
timothyandtimothy ba6a4b08aa feat(732): On Now / Next gets a background box, and is on by default (#843)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m40s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m18s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m12s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m13s
probe742/combined-newest SECOND
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 19:28:25 +00:00
timothyandtimothy f2551b778e fix(746): drop the persisted checkout credential; unmask the base-ref fetches (#842)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 24s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 30s
Build ErsatzTV Image / Build & test (.NET) (push) Failing after 1m37s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m14s
Build ErsatzTV Image / Build & push image (amd64) (push) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m1s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 18:50:56 +00:00
timothyandtimothy 40a3232d9e feat(734): field-level progressive disclosure — shared FieldHelp trigger + panel (#841)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 15s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m57s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m32s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m18s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m27s
Implements the three-level field-help pattern from #734 as a shared component: field name + optional one-sentence summary → a one-short-paragraph panel behind a consistent Info-icon trigger → a future external-docs deep link (`docsHref`, built and typed; no screen passes one yet).

Adopted on FFmpegProfilesScreen (9 fields), documented as docs/spa-conventions.md §15 with decision record `spa.field-progressive-disclosure`, and mirrored into the design-system prototype.

The panel is portalled to document.body: `.ctv-card` sets `overflow: hidden`, which clips a positioned descendant whatever its z-index, and one field's explainer rendered 12px of a 92px paragraph in every state of the Audio card.

A `::before` hover bridge was added and then WITHDRAWN — it held for a vertical descent onto the panel and failed for a diagonal one, leaving a safe sideways exit of 1.25px on an 18px icon. Hover reads the paragraph in place; the panel's interactive content is reached by pinning.

Four cold adversarial review rounds; the first three returned BLOCKED. They found five wrong copy claims across nine paragraphs and two vacuous tests in a row for the same mechanism.

Deferred with owners: #839 (placement verified by hand, not by a test) and #840 (the portal puts a docsHref link at the end of the tab order).

fixes #734

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 18:05:53 +00:00
timothyandtimothy 35affecd29 docs(755): ersatztv owns the fork code, media-management owns channel operations (#838)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 20s
Build ErsatzTV Image / Build & test (.NET) (push) Failing after 1m41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m7s
Build ErsatzTV Image / Build & push image (amd64) (push) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m5s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 17:29:16 +00:00
timothy e8e17f3844 docs(708): record the live route-2 reproduction against PR #761 (#834)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m57s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m5s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m53s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m24s
2026-08-26 09:11:23 +00:00
timothyandtimothy 08cd3a002d fix(690,758): count the same query a paged handler pages (#833)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 14s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m53s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m18s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m40s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m46s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 07:39:23 +00:00
timothyandtimothy 7453dd3a82 fix(721,740): align Auto-Tune proposal rows on a grid; guard AddItemsDialog's async searches (#831)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m35s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m38s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m58s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Failing after 1m17s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-25 21:55:23 +00:00
183 changed files with 33071 additions and 1226 deletions
+170 -22
View File
@@ -22,7 +22,7 @@
# The "## Done-when" issue-body checklist is the convention (docs/decisions.md, CLAUDE.md Task
# Completion Protocol). One box is "adversarial review passed"; the others are per-issue.
# The H10 review-verdict convention: after reviewing a PR (or its latest fix commit), post a PR
# comment carrying a line `Review-verdict: <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> @ <head-sha>`.
# comment carrying a line `Review-verdict: <MERGEABLE|APPROVED|LGTM|BLOCKED|NOT-MERGEABLE> @ <head-sha>`.
#
# Decision policy — a CONSENT gate, so it does NOT fail silently open:
# - state derivable and satisfied -> grant (auto-approve: permissionDecision "allow",
@@ -96,8 +96,9 @@ sha=$(printf '%s' "$prjson" | jq -r '.head.sha // ""' 2>/dev/null || true)
body=$(printf '%s' "$prjson" | jq -r '.body // ""' 2>/dev/null || true)
# --- Docs-only exemption: if every changed file is docs/process, skip the gate. ---
# The file list must be enumerated EXHAUSTIVELY, validated row by row, and bound to ONE head, or the
# exemption is unsafe. ALL of that now lives in scripts/pr-changed-files.sh — the single shared
# The file list must be enumerated EXHAUSTIVELY, validated row by row, and checked for head/base
# movement across the paging round trips, or the exemption is unsafe. (That check detects ONE-WAY
# movement only — this said "bound to ONE head" until 2026-08-28, ersatztv#803.) ALL of that now lives in scripts/pr-changed-files.sh — the single shared
# implementation, also called by .gitea/workflows/review-verdict.yml (ersatztv#649).
#
# Why it moved: this logic was written twice. This copy is ADVISORY (a failure produces a human
@@ -212,6 +213,48 @@ fi
# From here on both names are the freshly-confirmed base; they are equal by the check above.
base_ref=$base_now
live_base=$base_now
# THE HEAD IS RE-READ AT THE SAME HOIST, FROM THE SAME RESPONSE (ersatztv#803).
#
# `$sha` comes from the PR snapshot at the top of this hook, and until 2026-08-28 every later check
# consumed that captured value: the CI combined status, the `review-verdict/h10` status, and the
# verdict-comment classification were all evaluated against `/commits/$sha/status` and `--head $sha`.
# A push landing in the gap — which includes the docs-only enumeration's up-to-forty round trips —
# was therefore checked against the commit it had just replaced, and the hook would report "a
# positive Review-verdict references the current head" about a head that was no longer current.
#
# This is the SAME defect the base had until #778 hoisted the re-read above, and it is fixed the same
# way rather than a different way. Reading `.head.sha` off `$prjson_now` — the response the base
# check already fetched — costs NO extra round trip, and it keeps the two axes on ONE snapshot, so
# they cannot disagree about which moment they describe. Two separate reads would answer about two
# different instants while reading as one check.
#
# DENY, not ask, and for the same reason the `stale` verdict class denies: a head that moved means
# the verdict this hook is about to accept covers an OLDER commit, which is a state we have
# positively established rather than failed to establish. An UNREADABLE `.head.sha` is the different
# case and asks.
#
# WHAT THIS DOES NOT CLOSE, said here rather than left to be inferred. A push landing after this
# check still passes, exactly as a retarget does — the file's rule against a second re-read applies
# unchanged (see the branch-protection block below), because two reads only move the window rather
# than closing it. That residual is bounded server-side and this hook is not what bounds it: the new
# head has no `review-verdict/h10` status, and that context is REQUIRED on `main`, so Gitea refuses
# the merge (#622). The hook's job here is to stop CLAIMING a head is reviewed when it can see that
# it is not — an advisory gate that states something false is worse than one that asks.
if [ -n "$sha" ]; then
sha_now=$(printf '%s' "$prjson_now" | jq -r '.head.sha // ""' 2>/dev/null || true)
if [ -z "$sha_now" ]; then
decide ask "H10 merge gate: PR #$pr reports no head commit (.head.sha) on re-read, so whether the review verdict still covers the current head could not be confirmed. Check the PR, then merge."
fi
if [ "$sha_now" != "$sha" ]; then
decide deny "H6/H10 merge gate: BLOCKED — PR #$pr's head moved from ${sha:0:7} to ${sha_now:0:7} while this gate was evaluating. Every check formed against ${sha:0:7} — the changed-file enumeration, the CI status and the review verdict — describes a commit that is no longer the one being merged (ersatztv#803). Re-review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE"
fi
# From here on `$sha` is the freshly-confirmed head; the two are equal by the check above. Mirrors
# `base_ref=$base_now` a few lines up, and is written for the same reason that one is: it makes the
# value every later check consumes the one that was just re-read, so a future edit moving a
# consumer above this point fails visibly rather than silently reading the stale capture.
sha=$sha_now
fi
if [ -n "$sha" ]; then
# This is the THIRD read of this endpoint in a worst-case hook run (the ordinary-CI branch and the
# scheduled-auto-merge branch each do their own). Sharing one snapshot would close a narrow
@@ -472,25 +515,17 @@ else
# directions, and it is rare in practice: as of 2026-08-19 this repo's only rule is the plain
# name `main`, which the classifier resolves to `exact` on every run. That is a dated
# observation about mutable remote config, not a property to rely on.
bp_verdict=$(printf '%s' "$bp_list" | jq --arg b "$base_ref" -c '
def esc: gsub("(?<c>[.+?^${}()|\\[\\]\\\\])"; "\\" + .c);
def offs: [match("[*?\\[\\]{}\\\\]"; "g").offset];
def superset: . as $n | (offs) as $o
| ($n[0:$o[0]] | esc) + ".*" + ($n[($o[-1]+1):] | esc);
def nonascii: explode | any(. > 127);
. as $rules | $b as $base |
($rules | map(select((.branch_name // .rule_name // "") as $n
| (($n|offs|length) == 0)
and (($n|ascii_downcase) == ($base|ascii_downcase))))) as $exacts |
(($base|nonascii) or ($rules | any((.branch_name // .rule_name // "") as $n
| ($n|offs|length) == 0 and ($n|nonascii)))) as $unfoldable |
if ($rules | any((.branch_name // .rule_name // "") as $n
| (($n|offs|length) > 0)
and ($base | test("^" + ($n|superset) + "$")))) then {verdict:"undecidable"}
elif $unfoldable then {verdict:"undecidable"}
elif ($exacts | length) > 1 then {verdict:"undecidable"}
elif ($exacts | length) == 1 then {verdict:"exact", rule:($exacts | first)}
else {verdict:"none"} end' 2>/dev/null || true)
# The classifier is a FILE now (ersatztv#787), so its absence is a new failure mode: `jq -f` on a
# missing program exits 2 with empty stdout, which reaches the `*)` arm below and asks that "this
# repo's branch-protection rules came back in a shape this hook could not parse" — blaming the
# payload for a missing local file. That is precisely the states-a-cause-that-did-not-happen defect
# the two comments beside that arm were written to fix, so it is checked here rather than inherited.
classifier="$repo_root/scripts/lib/branch-rule-classifier.jq"
if [ ! -r "$classifier" ]; then
rm -f "$bp_file"
decide ask "H6/H10 merge gate: the shared branch-protection rule classifier is missing or unreadable at $classifier, so which rule governs '$base_ref' — and therefore whether 'review-verdict/h10' is required on it — could not be derived (ersatztv#787). Restore the file, or confirm the required checks manually."
fi
bp_verdict=$(printf '%s' "$bp_list" | jq --arg b "$base_ref" -c -f "$classifier" 2>/dev/null || true)
case $(printf '%s' "$bp_verdict" | jq -r '.verdict // ""' 2>/dev/null || true) in
exact) bp=$(printf '%s' "$bp_verdict" | jq -c '.rule' 2>/dev/null || true); bp_code=200 ;;
undecidable) rm -f "$bp_file"
@@ -614,6 +649,119 @@ case "$class" in
decide ask "H10 merge gate: unrecognized verdict classification '$class' for PR #$pr. Confirm the review covered the latest commit ($short) before merging." ;;
esac
# --- (d) Guard-scope freshness (ersatztv#787): the committed mirror of `main`'s required status
# checks must still match the server. ------------------------------------------------------
# ORDERED LAST, and that is a severity argument rather than a stylistic one. Every check above
# can DENY; this one can only ever downgrade an otherwise-satisfied auto-grant to a prompt. Run
# earlier it would preempt those verdicts and report a stale guard scope at a reader whose merge
# is blocked for a completely different and more serious reason, and it would ask on payloads the
# checks above are about to reject anyway. Placed here it is also PAST the point where the two
# merge paths converge, so it covers both without duplicating anything.
# `scripts/tests/test_ci_dropped_step_guard.py` DERIVES which jobs must carry per-step execution
# markers from `.gitea/required-status-contexts.json`, because its CI job checks out with
# `persist-credentials: false` and cannot ask Gitea. That makes the snapshot the single
# hand-maintained input in the chain: a fourth required context added on the server leaves the
# snapshot — and therefore the guard's scope — silently behind, which is the whole of #787.
#
# THIS RUNS ON BOTH MERGE PATHS, deliberately, and it is placed here rather than beside the
# branch-protection read in the scheduled-auto-merge branch for that reason.
#
# WHAT IT DOES NOT COVER, said here rather than left to be discovered: a PR whose changed files are
# all docs/process — `.gitea/` included — exits at the docs-only passthrough far above, so this arm
# never runs for it. A PR that edits ONLY `.gitea/required-status-contexts.json` is docs-only BY
# CONSTRUCTION, and that is exactly the snapshot-NARROWING direction the decision record names as
# this design's residual. Excluding that path from the allow-list would not buy the protection it
# looks like it would: this arm compares the live server against the snapshot in the LOCAL CHECKOUT,
# not against the version the PR proposes, so it cannot see a narrowing that has not landed yet.
# What does hold is that the passthrough is a passthrough — a human prompt, never an auto-grant —
# which is the `.gitea/` treatment ersatztv#317 asked for. That read is inside
# `else` (mwcs = true) and never executes on an immediate merge, which is the common case; hanging
# the freshness check off it would fire it only when an auto-merge is armed. This file already
# records that exact defect one section up — the base re-read "first landed inside the
# scheduled-auto-merge branch only", and cold review found scheduled+retarget denied while
# immediate+retarget auto-GRANTED. Same shape, so it is not repeated here.
#
# It reads `main` (the branch the snapshot names), NOT `$base_ref`. That is a DIFFERENT question
# from the one the scheduled branch asks — "is review-verdict/h10 required on the base I am merging
# into" — so this is not a second copy of that classifier and the two cannot drift into disagreeing:
# they consume different fields of different rules for different decisions.
#
# ASK, NEVER DENY. Drift does not make THIS merge unsafe: Gitea enforces the live required set
# server-side, so a newly required context with no status blocks the merge on its own. What has gone
# stale is a guard's scope — a different artifact, on a different clock. Denying would state
# something false about the change in front of the reader. Every non-`match` class asks, so a
# comparison that could not be made is surfaced rather than skipped (`unknown` is not `fine`).
# ONE base for both the checker and the snapshot, and it is `$repo_root` — derived from this file's
# own location — rather than `$CLAUDE_PROJECT_DIR`. Two reasons, and the second is the load-bearing
# one. Resolving them from different roots would let the hook classify one checkout's snapshot with
# another checkout's script, mismatched halves of a comparison whose whole job is to detect a
# mismatch. And an ENV VAR is not a sound input to a security decision: a wrong value pointing at a
# tree that happens to contain an executable checker returns `match` about a different checkout
# entirely, which silently authorizes the grant. A missing path only asks, so the failure is quiet
# exactly where it is worst.
ctx_base="$repo_root"
ctx_snapshot="$ctx_base/.gitea/required-status-contexts.json"
ctx_script="$ctx_base/scripts/check-required-contexts.sh"
# THIS ARM IS ABOUT ONE REPO, and the merge tool is not. Every other check here reads
# `$owner/$repo` from the tool input and is repo-agnostic; this one compares a HARDCODED branch
# against a snapshot committed in THIS checkout. Merging a PR in another repo from a session opened
# here would otherwise weigh that repo's live contexts against this repo's mirror and report a
# confident, flatly false finding about it — measured: server-management returns `[]`, which
# classifies as `nomatch`. So the snapshot names the repo it describes and the arm runs only for it.
# An unreadable snapshot cannot answer "is this my repo?" either, so it asks rather than skipping.
ctx_repo=$(jq -r 'if (.repo | type) == "string" then .repo else "" end' "$ctx_snapshot" 2>/dev/null || true)
if [ -z "$ctx_repo" ]; then
decide ask "H6 merge gate: $ctx_snapshot is missing, unreadable, or names no \`repo\`, so the dropped-step guard's scope could not be checked against branch protection — nor could it be established whether this snapshot even describes $owner/$repo (ersatztv#787). Restore the file, or check the required checks manually."
fi
# CASE-FOLDED, because Gitea resolves owner/repo case-insensitively: verified live, both
# `/repos/timothy/ersatztv` and `/repos/TIMOTHY/ErsatzTV` answer 200. A byte-exact compare would let
# any case variant sail through every other arm and SKIP this one, so drift would go unreported with
# no ask — the gate failing open on a spelling. The hook already treats case folding as
# decision-relevant one section up, where `MAIN` vs `main` makes the governing rule undecidable.
ctx_repo_fold=$(printf '%s' "$ctx_repo" | tr '[:upper:]' '[:lower:]')
target_repo_fold=$(printf '%s' "$owner/$repo" | tr '[:upper:]' '[:lower:]')
if [ "$ctx_repo_fold" = "$target_repo_fold" ]; then
if [ ! -x "$ctx_script" ]; then
decide ask "H6 merge gate: the required-contexts checker is missing or not executable at $ctx_script, so whether the dropped-step guard's scope still matches branch protection on 'main' could not be derived (ersatztv#787). Check it manually, or restore the script."
fi
bpf=$(mktemp) || decide ask "H6 merge gate: could not allocate a temp file to read branch protection for the guard-scope freshness check (ersatztv#787)."
if [ -n "${ETV_GITEA_TOKEN:-}" ]; then
ctx_code=$(curl -s -o "$bpf" -w '%{http_code}' -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true)
else
ctx_code=$(curl -s -o "$bpf" -w '%{http_code}' -u "$ETV_GITEA_BASICAUTH" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true)
fi
if [ "$ctx_code" = "200" ]; then
# stderr is KEPT, not sent to /dev/null. The checker exits 2 with a diagnostic on a usage error —
# an unreadable snapshot, a branch mismatch, a missing classifier — and discarding it made all of
# those arrive at the operator as the catch-all's "returned 'nothing'", which names no cause. That
# is the same states-a-cause-that-did-not-happen shape this arm was careful about elsewhere.
ctx_class=$("$ctx_script" --branch main --snapshot "$ctx_snapshot" < "$bpf" 2>"$bpf.err" || true)
ctx_diag=$(tr '\n' ' ' < "$bpf.err" 2>/dev/null | cut -c1-300 || true)
else
ctx_class=readfail
ctx_diag=""
fi
rm -f "$bpf" "$bpf.err"
case "$ctx_class" in
match) : ;;
drift)
decide ask "H6 merge gate: the required status checks on 'main' no longer match .gitea/required-status-contexts.json (ersatztv#787). scripts/tests/test_ci_dropped_step_guard.py derives its marked-job scope from that snapshot, so until it is reconciled a required context may have NO dropped-step guard — a step the runner drops would conclude success and take that check green having done no work (ersatztv#756). Re-read the live list and update the snapshot in a PR (the guard will then demand markers for any newly required job, or an ACCOUNTED_ELSEWHERE entry naming what covers it). This does not make the merge in front of you unsafe — Gitea enforces the live required set server-side — so approve if you have judged it unrelated." ;;
nomatch)
decide ask "H6 merge gate: no branch-protection rule governs 'main' at all, so the required status checks the dropped-step guard scopes itself to could not be confirmed (ersatztv#787). Branch protection on 'main' is what makes 'review-verdict/h10' load-bearing (ersatztv#743) — check it before merging." ;;
undecidable)
decide ask "H6 merge gate: a glob branch-protection rule could govern 'main', so which rule's required contexts to compare against .gitea/required-status-contexts.json is not derivable without reimplementing Gitea's matcher (ersatztv#787). Confirm the required checks manually." ;;
unreadable)
decide ask "H6 merge gate: branch protection for 'main', or .gitea/required-status-contexts.json itself, came back in a shape the required-contexts checker could not consume, so whether the dropped-step guard's scope is still current is unknown (ersatztv#787). Check the rules and the snapshot manually." ;;
readfail)
decide ask "H6 merge gate: could not read branch protection for the guard-scope freshness check (HTTP '${ctx_code:-none}' — Gitea unreachable, or these credentials lack the repo-admin scope that endpoint needs), so whether .gitea/required-status-contexts.json is still current is unknown (ersatztv#787). Confirm the required checks on 'main' manually." ;;
*)
decide ask "H6 merge gate: the required-contexts checker returned '${ctx_class:-nothing}', which is not a class this hook understands, so the dropped-step guard's scope could not be confirmed against branch protection (ersatztv#787).${ctx_diag:+ It said: ${ctx_diag}}Check scripts/check-required-contexts.sh." ;;
esac
fi # end of the guard-scope freshness arm (opened at `if [ "$ctx_repo_fold" = ... ]` above). The
# body is left unindented to match the rest of this file, which is flat throughout; the marker
# is here because the block is long enough that its extent is otherwise easy to misread.
if [ "$class" = "positive" ]; then
# (a) CI + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant.
# The reason string must not claim more than was actually checked: on the merge_when_checks_succeed
+102 -2
View File
@@ -4,8 +4,15 @@ description: "ErsatzTV custom IPTV channel management — REST API, SQLite DB, J
---
> **Canonical copy: `~/ersatztv/.claude/skills/ersatztv/SKILL.md`** (ersatztv owns this skill per that
> repo's `CLAUDE.md` → Project Boundaries). `~/server-management/.claude/skills/ersatztv` is a symlink
> to it. Edit it in the ersatztv repo; never fork a second copy (ersatztv#617).
> repo's `CLAUDE.md` → Project Boundaries and `process.ersatztv-owns-code-not-operations`). Both
> `~/server-management/.claude/skills/ersatztv` **and** `~/media-management/.claude/skills/ersatztv`
> are symlinks to it. Edit it in the ersatztv repo; never fork a second copy (ersatztv#617, #755) —
> media-management's copy had silently become a divergent fork still describing a Blazor UI that no
> longer exists, which is what made this the rule rather than a preference.
>
> **Channel OPERATIONS (create/edit a live channel, lineup, collection, schedule, playout, logo,
> overlay) are `media-management`'s job**; ersatztv owns the fork code, `/api/v1`, CI and releases.
> This skill serves both — it is the operator's reference *and* the developer's map.
# ErsatzTV Channel Management
@@ -180,6 +187,99 @@ POST /api/v1/libraries/{id}/scan-show \
POST /api/v1/channels/{channelId}/playout/reset
```
### Scripted Schedule API — `/api/v1/scripted/…`
For **programmatic playout building**: each call mutates one build session, addressed by `buildId`.
Documented by its own OpenAPI spec, **separate from `v1.json`** — which is why
`docs/endpoint-index.md` does not list any of it. It ships as **two** files, both served at
`/openapi/` (measured 2026-08-26 on prod: `scripted-schedule.json`, `scripted-schedule-tagged.json`
and `v1.json` all return 200). They carry the same 28 paths, so either answers "what operations
exist"; they differ only in grouping — the plain file puts everything under one `ScriptedSchedule`
tag, the `-tagged` one splits it into Scripted Content / Control / Metadata / Scheduling. Scalar's
`/docs` page renders the `-tagged` file (`Startup.cs` registers `openapi/scripted-schedule-tagged.json`),
which is why the browsable docs are grouped and a raw fetch of the plain file is not.
The base path is **`/api/v1/scripted/playout/build/{buildId}/`**, and `buildId` is routed as a GUID
(`ScriptedScheduleController.cs`). An older archived copy of this skill gave it as `/api/scripted/…`,
without the `v1`; no such route is registered.
**You cannot tell a wrong base path from a stale `buildId` by probing** — measured on prod
2026-08-26, `GET …/context` with a non-existent build id:
| | `/api/v1/scripted/…` | `/api/scripted/…` (no route) |
|---|---|---|
| no key | 401 | 401 |
| valid key | 404 | 404 |
Unauthenticated everything is 401, because the api-key filter runs before routing. Authenticated, the
correct path 404s too — the build session does not exist — so the 404 that a wrong path earns is
indistinguishable from the one a correct path earns. The bound: this holds **while the build id is
not live**. Against a real, open build session the correct path would answer 200 and the difference
would show — but that is not the situation you are in when you are probing to find out why nothing
works. Confirm the route in `ErsatzTV/Controllers/Api/ScriptedScheduleController.cs`; do not infer it
from a status code.
```
# 28 operations, derived from scripted-schedule.json on 2026-08-26 (ersatztv#755)
POST add_all {content, fillerKind, customTitle, disableWatermarks}
POST add_collection {key, collection, order}
POST add_count {content, count, fillerKind, customTitle, disableWatermarks}
POST add_duration {content, duration, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST add_marathon {key, groupBy, itemOrder, guids, searches, playAllItems, shuffleGroups}
POST add_multi_collection {key, multiCollection, order}
POST add_playlist {key, playlist, playlistGroup}
POST add_search {key, query, order}
POST add_show {key, guids, order}
POST add_smart_collection {key, smartCollection, order}
POST create_playlist {key, items}
POST graphics_off {graphics}
POST graphics_on {graphics, variables}
POST pad_to_next {content, minutes, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pad_until {content, when, tomorrow, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pad_until_exact {content, when, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pre_roll_off (no body)
POST pre_roll_on {playlist}
POST skip_items {content, count}
POST skip_to_item {content, season, episode}
POST start_epg_group {advance, customTitle}
POST stop_epg_group (no body)
POST wait_until {when, tomorrow, rewindOnReset}
POST wait_until_exact {when, rewindOnReset}
POST watermark_off {watermark}
POST watermark_on {watermark}
GET context (no body)
GET peek_next/{content} (no body)
```
Re-derive rather than trusting this table (it is prose and will drift):
```bash
# Absolute path on purpose: this skill is symlinked into ~/server-management and
# ~/media-management, where a repo-relative path would not resolve. ~/ersatztv is the
# shared checkout and can lag origin/main — use the live-instance form below to see
# what is actually deployed.
python3 -c "import json;d=json.load(open('$HOME/ersatztv/ErsatzTV/wwwroot/openapi/scripted-schedule.json'));\
print('\n'.join(f'{m.upper()} {p}' for p,i in d['paths'].items() for m in i if m in('get','post')))"
```
Without a checkout — straight off the running instance (prod; test is port 8410):
```bash
ssh timothy@192.168.1.29 'curl -s http://localhost:8409/openapi/scripted-schedule.json' \
| python3 -c "import json,sys;d=json.load(sys.stdin);\
print('\n'.join(f'{m.upper()} {p}' for p,i in d['paths'].items() for m in i if m in('get','post')))"
```
Field lists above are the request-body property names only; consult the spec for types,
required-ness and defaults. That omission matters for the three on/off pairs: `graphics_on`/
`graphics_off`, `watermark_on`/`watermark_off` and `pre_roll_on`/`pre_roll_off` are **separate
operations, not one toggle**, and the difference is not always visible as differing property names.
`graphics_*` and `pre_roll_*` differ outright. `watermark_on` and `watermark_off` both list
`{watermark}`, but only `on` marks it **required** — `watermark_off` with an **empty** list turns
*every* scripted watermark off (`SchedulingEngine.WatermarkOff`: `watermarks.Count == 0` →
`ClearChannelWatermarkIds()`; `GraphicsOff` is the same shape). Read the schema, not this table,
before sending an `_off`.
## SQLite DB Operations
```bash
+12
View File
@@ -0,0 +1,12 @@
{
"repo": "timothy/ersatztv",
"branch": "main",
"read_on": "2026-08-27",
"source": "GET /repos/timothy/ersatztv/branch_protections -> the rule governing `main` -> status_check_contexts",
"why": "ersatztv#787. The committed mirror of the required status checks on `main`. It exists because the guards that make a required context trustworthy run in `pr-checks.yml::script-tests`, which checks out with persist-credentials:false and holds no Gitea credential, so it cannot ask the server. scripts/tests/test_ci_dropped_step_guard.py DERIVES its marked-job scope from `contexts` rather than repeating it as a literal, and scripts/check-required-contexts.sh compares this list against the live one wherever a credential does exist. Editing `contexts` by hand without re-reading the server is the one move that defeats both. The `repo` field exists because the merge-consent hook fires for whatever owner/repo the merge tool was called with: without it, merging a PR in another repo from an ersatztv session compares that repo's live contexts against THIS repo's mirror and reports a confident, flatly false finding about it.",
"contexts": [
"Build ErsatzTV Image / Build & test (.NET) (pull_request)",
"Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request)",
"review-verdict/h10"
]
}
+74 -11
View File
@@ -4,29 +4,63 @@ name: Build CI Toolchain Image
# pushes it to the Gitea container registry (ersatztv#390). The toolchain jobs in
# docker-build.yml consume it via `container:`, pinned to an immutable :<sha>.
#
# push touching docker/ci/** -> :<short-sha> (+ :latest only from main)
# workflow_dispatch -> manual rebuild
# schedule (weekly) -> picks up base-image security updates
# push to MAIN touching docker/ci/** -> :<short-sha> + :latest
# workflow_dispatch on main -> :<short-sha> of main's HEAD + :latest
# workflow_dispatch on a branch -> :<short-sha> of that branch's HEAD ONLY (never :latest)
# schedule (weekly) -> picks up base-image security updates
#
# Deliberately separate from docker-build.yml: this image changes rarely (a Dockerfile edit or
# the weekly cron), while docker-build.yml runs on every push/PR. Coupling them would rebuild a
# ~2GB toolchain image on every commit.
#
# ROLLOUT NOTE: the jobs pin an immutable :<sha>, never :latest — a broken toolchain image would
# otherwise block every converted job the moment it was pushed. Bumping the toolchain is therefore
# a deliberate two-step: merge a docker/ci/Dockerfile change (this workflow publishes a new :<sha>),
# then update the pin in docker-build.yml in a follow-up PR whose CI proves the new image works.
# See docs/ci-cd.md -> "CI toolchain image".
# otherwise block every converted job the moment it was pushed. Bumping the toolchain is a deliberate
# two-step, and BOTH steps land in the SAME PR: publish (push the docker/ci commit as branch HEAD,
# dispatch this workflow on that branch), then commit the pin bump in docker-build.yml. Merging first
# is not available: a PR that changes docker/ci/** without moving the pin turns `ci-image-pin` red,
# and the merge-consent hook reads the COMBINED commit status, so it will not auto-grant. That much
# predates ersatztv#744 — what #744 changed is how the publish half is performed.
# See docs/ci-cd.md -> "Publishing from a branch is a dispatch, not a push".
#
# Like docker-build.yml: the Gitea registry is HTTP-only, so BuildKit needs the inline
# `http = true` config (it does not inherit the host daemon's insecure-registries setting).
on:
# Publishing from a branch is a DELIBERATE act, not a side effect of pushing (ersatztv#744).
# Gitea resolves a `push` workflow's definition from the pushed branch, so an unfiltered `push`
# trigger ran this file's own YAML — attacker-supplied, unreviewed, with no status check in the
# loop — on a docker-capable runner holding the credential that writes `ersatztv:prod` and the
# `ersatztv-ci:<sha>` five `container:` jobs execute.
#
# BE PRECISE ABOUT WHAT THIS BUYS, because the mechanism cuts both ways: the filter below is read
# from the pushed ref like everything else in this file, so a branch that DELETES it re-enables
# the route. What closes is the DRIVE-BY case — an ordinary push of a legitimate `docker/ci`
# change publishing an image nobody asked for, with no deliberate act anywhere. This is NOT a
# boundary against a malicious or compromised writer and must not be cited as one; that class is
# ersatztv#853. `workflow_dispatch` is self-supplied from the selected branch in the same way, and
# is the deliberate publish path (docs/ci-cd.md -> "CI toolchain image").
#
# A `v*` tag push does not match this trigger either: there is no `tags:` key, and a `branches:`
# filter is compared against a branch ref. The exact matcher semantics are not probed here; the
# observable claim is the one that matters — a release cut no longer republishes the toolchain
# image as a side effect.
#
# `.gitea/workflows/ci-image.yml` is NOT in `paths:`, and it left `ci-image-pin`'s `expected` in
# the same change. That pairing is a DECIDED TRADEOFF, not a necessity: keeping it works, because
# the dispatch above can publish the ci-image.yml commit itself and the pin then matches. The
# price is what decided it — that route charges a full ~2GB publish plus a five-pin bump for
# EVERY edit to this file, comments included, and a rebase charges it again. The cost of the side
# taken is stated here and in ci-cd.md: a change to HOW the image is built that lives only in
# this file no longer republishes on its own, so pair it with a `docker/ci/**` edit.
#
# `paths:` here and `ci-image-pin`'s `expected` pathspec in pr-checks.yml MUST name the same
# sources, and nothing mechanically enforces that since the shared self-reference went —
# ersatztv#855.
workflow_dispatch:
push:
branches: [main]
paths:
- 'docker/ci/**'
- '.gitea/workflows/ci-image.yml'
schedule:
# Mondays 05:00 UTC. Gitea registers `schedule` only from the default branch (main).
#
@@ -50,6 +84,21 @@ env:
REGISTRY: 192.168.1.95:3000
CI_IMAGE: 192.168.1.95:3000/timothy/ersatztv-ci
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# This workflow's registry pushes authenticate with the scoped REGISTRY_* PAT
# (`ci.actions-credential-scoping`), so the injected GITEA_TOKEN serves only its single
# `actions/checkout`. This file was the one workflow #748 could not originally reach: editing it
# re-pointed `ci-image-pin`'s `expected` at the editing commit and reddened a BLOCKING job, and its
# own `paths:` made the edit publish an image. ersatztv#744 took this path out of both
# (`ci.toolchain-image-publish-is-a-dispatch`), so the exemption that briefly existed here is DELETED
# rather than documented — which is what ersatztv#835 asked for.
permissions:
code: read
jobs:
build:
name: Build & push CI image
@@ -58,13 +107,23 @@ jobs:
# toolchain image — the heaviest thing that ran in that lane. `small` is now
# git-only and capped at 1g per job, which would OOM this build.
#
# Rare trigger (pushes touching docker/ci + a weekly cron), so it costs the
# ubuntu-latest lane almost nothing, and ci-runner (.127) runs no prod workload.
# Rare trigger (main pushes touching docker/ci, a weekly cron, and the occasional
# branch dispatch), so it costs the ubuntu-latest lane almost nothing, and
# ci-runner (.127) runs no prod workload.
runs-on: ubuntu-latest
env:
CI_JOB_ROLE: none
steps:
- name: Checkout
uses: actions/checkout@v4
with:
# ersatztv#746's convention, applied here once #744 removed the reason it was skipped:
# without it the action leaves a write-capable Authorization header in .git/config for
# every later step. Nothing here pushes with git — the only git call is the
# `rev-parse --short HEAD` below — and the repo is public, so the clone needs no
# credential of its own. Guarded for every workflow by
# scripts/tests/test_workflow_persist_credentials.py (ersatztv#835).
persist-credentials: false
# only docker/ci/Dockerfile is needed; no git describe/log here
fetch-depth: 1
@@ -76,7 +135,11 @@ jobs:
# Always publish the immutable :<sha> — that is what docker-build.yml pins.
TAGS=("${CI_IMAGE}:${SHORT}")
# :latest is a convenience/floating pointer for humans and the weekly rebuild; jobs must
# never consume it. Only main may move it.
# never consume it. Only main may move it — and since #744 the `push` trigger is
# main-only, so on that path the branch check is satisfied by construction. It is now the
# SOLE protection on the one event that never exercised it before: a `workflow_dispatch`
# selects any ref, and the branch-side publish path documented in ci-cd.md runs exactly
# that. Do not simplify this away on the reasoning that the trigger is already main-only.
if [ "${GITHUB_REF}" = "refs/heads/main" ]; then
TAGS+=("${CI_IMAGE}:latest")
fi
+14
View File
@@ -33,13 +33,27 @@ env:
DOTNET_CLI_USE_MSBUILD_SERVER: "0"
MSBUILDDISABLENODEREUSE: "1"
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Holds no registry credential and reads nothing from the Gitea API; the injected GITEA_TOKEN serves
# only its one `actions/checkout`.
permissions:
code: read
jobs:
scan:
name: NuGet vulnerable packages
runs-on: ubuntu-latest
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Setup .NET
uses: actions/setup-dotnet@v4
+71 -5
View File
@@ -38,7 +38,9 @@ name: Build ErsatzTV Image
# report `success` in seconds — the two REQUIRED contexts (`Build & test (.NET)`, `EF migration
# integrity (SQLite + MySql)`) must keep reporting or a docs-only PR could never merge. We do NOT
# `if:`-skip a required job: on Gitea 1.25.4 a skipped job reports commit-status state `skipped`
# (verified, throwaway PR #418) and we don't rely on how branch protection treats a skipped
# (verified, throwaway PR #418; re-confirmed on 1.27.1, 2026-08-28, ersatztv#747 — `Build & push
# image (amd64)` is `if:`-skipped on every PR and reported `skipped` on the two heads sampled,
# PRs #829 and #828) and we don't rely on how branch protection treats a skipped
# REQUIRED context. See docs/ci-cd.md -> "Docs-only skip".
#
# RELEASE-PATH DELIMITER GATE (ersatztv#767): the `scan` job runs the PyYAML-based delimiter-ban
@@ -113,6 +115,24 @@ env:
DOTNET_CLI_USE_MSBUILD_SERVER: "0" # no persistent MSBuild server process
MSBUILDDISABLENODEREUSE: "1" # MSBuild worker nodes exit with the build instead of lingering
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Every credentialed thing this file does uses the scoped REGISTRY_* PAT, never the injected token:
# its registry pushes, its five `container:` image pulls, its three commit-status GET steps
# (`ETV_STATUS_AUTH` in jobs `test`, `migrations` and `functional-e2e`, each a read-only GET via
# scripts/ci-detect-already-validated.sh) and its registry tag READ (`ETV_REGISTRY_AUTH` in job
# `toolchain-preflight`, via scripts/ci-toolchain-image-resolves.sh). The injected token therefore
# serves only its eight `actions/checkout` steps. Note this file needs no `packages:` unit for that
# same reason: the `container:` blocks carry explicit `credentials:`.
# (Sites above are named by JOB, not by line number: this file is ~1150 lines, so any edit above a
# citation silently invalidates it — which is how the first version of this comment went stale two
# lines after it was written.)
permissions:
code: read
jobs:
# Answers "is the toolchain image still there?" in ONE place, so a deleted pin does not read as
# five broken jobs and a broken diff (ersatztv#772). Deliberately container-free and deliberately
@@ -121,9 +141,14 @@ jobs:
toolchain-preflight:
name: CI toolchain image resolves
runs-on: small
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Resolve the pinned toolchain tag in the registry
env:
ETV_REGISTRY_AUTH: ${{ secrets.REGISTRY_USER }}:${{ secrets.REGISTRY_PASSWORD }}
@@ -143,10 +168,14 @@ jobs:
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# git history/tags are needed by the `build` job's `git describe` (ersatztv#190) and,
# here, by the #420 revalidate step's `HEAD^2` tree comparison on a main merge commit.
fetch-depth: 2
@@ -415,10 +444,14 @@ jobs:
--health-interval=5s
--health-timeout=5s
--health-retries=30
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# was the default fetch-depth: 1 (ersatztv#190); bumped to 2 so the #420 revalidate
# step's `HEAD^2` tree comparison can resolve on a main merge commit.
fetch-depth: 2
@@ -581,10 +614,14 @@ jobs:
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# bumped from 1 (ersatztv#190 default) so the #420 revalidate step's `HEAD^2` tree
# comparison can resolve on a main merge commit.
fetch-depth: 2
@@ -724,9 +761,14 @@ jobs:
scan:
name: Delimiter ban (release path)
runs-on: small
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v5
with:
@@ -796,10 +838,14 @@ jobs:
# where an image is published and never booted.
needs: [test, migrations, scan]
if: github.event_name != 'pull_request'
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: none
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
# ersatztv#416: a docs-only push to main has nothing to rebuild (docs are not in the image),
@@ -982,18 +1028,28 @@ jobs:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
if: github.event_name == 'pull_request'
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Detect API-surface changes
id: detect
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only "origin/${base_ref}...HEAD" 2>/dev/null || true)"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only "origin/${base_ref}...HEAD")"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
echo "Changed files in this PR:"; printf '%s\n' "$changed"
if printf '%s\n' "$changed" | grep -Eq '^ErsatzTV/Controllers/Api/|^ErsatzTV\.Core/Api/'; then
echo "api_changed=true" >> "$GITHUB_OUTPUT"
@@ -1077,18 +1133,28 @@ jobs:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
if: github.event_name == 'pull_request'
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Detect changed C# files
id: detect
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only --diff-filter=ACM "origin/${base_ref}...HEAD" -- '*.cs' 2>/dev/null || true)"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only --diff-filter=ACM "origin/${base_ref}...HEAD" -- '*.cs')"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
echo "Changed .cs files in this PR:"; printf '%s\n' "$changed"
if [ -n "$changed" ]; then
printf '%s\n' "$changed" > /tmp/changed-cs.txt
+93 -21
View File
@@ -42,40 +42,79 @@ concurrency:
group: ersatztv-pr-gates-${{ github.ref }}
cancel-in-progress: true
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Holds no secrets at all and reads nothing from the Gitea API; the injected GITEA_TOKEN serves only
# its five `actions/checkout` steps.
permissions:
code: read
jobs:
# BLOCKING (ersatztv#390): the CI toolchain image pin in docker-build.yml must name the image that
# ci-image.yml actually last published — i.e. the short sha of the last commit to touch the image's
# sources. Without this detector, a PR that edits docker/ci/** publishes a NEW image but runs its own
# BLOCKING (ersatztv#390): the CI toolchain image pin in docker-build.yml must name the short sha of
# the last commit to touch the image's SOURCES (`docker/ci/**`). Read that as "the image ci-image.yml
# last published" only under the convention that every such commit is published — this job compares
# git shas and never queries the registry, so it cannot see a pin whose tag was never built or has
# been evicted. Existence is `toolchain-preflight`'s job, and the container jobs' pull is the backstop.
# Since ersatztv#744 publishing from a branch is a `workflow_dispatch`, so "was it published" is a
# human step this job does not observe.
#
# Without this detector, a PR that edits docker/ci/** ships a new image RECIPE while running its own
# jobs against the OLD pin: CI green-lights a toolchain it never executed, and once merged, main's
# Dockerfile silently disagrees with what CI runs. **Renovate actively generates exactly that PR** —
# it manages docker/ci/Dockerfile's base pins (dockerfile manager) but cannot bump an opaque
# `:<sha>` in `container.image`, so it would leave the pin behind every time.
#
# Failing here forces the documented two-step (docs/ci-cd.md -> "CI toolchain image"): push the
# Dockerfile change, let ci-image.yml publish `:<sha>`, then update the pin to that sha. Seconds-long
# git+grep -> keep it off the build runners.
# Failing here forces the documented two-step (docs/ci-cd.md -> "CI toolchain image"): get the
# Dockerfile change published as `:<sha>`, then update the pin to that sha. Since ersatztv#744 the
# publish half of that two-step is a `workflow_dispatch` on the branch rather than a side effect of
# the push — ci-image.yml's `push` trigger is now `branches: [main]`. Seconds-long git+grep -> keep
# it off the build runners.
ci-image-pin:
name: CI image pin matches docker/ci
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# need real history: `git log -- <path>` on a shallow clone can't find the last
# commit that touched the image sources
fetch-depth: 0
- name: Verify the pin matches the last-published image
- name: Verify the pin matches the image-source commit
run: |
set -euo pipefail
# ci-image.yml tags the image `git rev-parse --short HEAD` of the push that built it, and it
# only builds on pushes touching these paths — so the published image is named by the last
# commit to touch them.
# ci-image.yml tags the image `git rev-parse --short HEAD` of the run that built it. Only
# its filtered `push` clause requires a `docker/ci/**` change; the weekly `schedule` and a
# `workflow_dispatch` both build the selected ref's HEAD whatever it touched. So `expected`
# is not a model of every tag in the registry — it is the one tag a PR is REQUIRED to be
# pinned to: the last commit to change the image's sources.
#
# `.gitea/workflows/ci-image.yml` is deliberately NOT part of `expected` (ersatztv#744),
# and that is a DECIDED TRADEOFF, not a necessity. Keeping it is workable — dispatch the
# branch at the ci-image.yml commit, then pin it — but it prices every edit to that file,
# comments included, at a full ~2GB publish plus a five-pin bump, redone after every
# rebase. Dropping it prices the opposite risk: a change to HOW the image is built living
# ONLY in ci-image.yml (build-args, Dockerfile path, platforms) neither republishes nor
# invalidates the pin, so CI keeps running an image built by the previous recipe. The
# second was chosen because that file is edited far more often for triggers, comments and
# runner placement than for build recipe. Make a recipe change alongside a `docker/ci/**`
# edit — a comment bump suffices, and it is the ONLY remedy: pinning the workflow-only
# commit is rejected here, because `expected` is the last `docker/ci` commit.
# Nothing MECHANICALLY couples this pathspec to `ci-image.yml`'s `on.push.paths`; before
# #744 the shared self-reference kept them in step. Divergence is silent and green in the
# dangerous direction — tracked in ersatztv#855.
# See docs/ci-cd.md -> "Publishing from a branch is a dispatch, not a push".
#
# Compare RESOLVED FULL shas, never the abbreviations: git auto-scales abbreviation length
# with the repo's object count, so the tag built in CI from a `fetch-depth: 1` shallow clone
# is 7 chars while `%h` here (full clone) is 8. Comparing those strings would fail always.
expected="$(git log -1 --format=%H -- docker/ci .gitea/workflows/ci-image.yml)"
expected="$(git log -1 --format=%H -- docker/ci)"
mapfile -t pins < <(grep -oE 'ersatztv-ci:[0-9a-f]+' .gitea/workflows/docker-build.yml | cut -d: -f2 | sort -u)
echo "Image sources last changed in: ${expected}"
echo "Pins found in docker-build.yml: ${pins[*]} (${#pins[@]} distinct)"
@@ -107,10 +146,10 @@ jobs:
# in-repo remedy in that state: relax this length check in the same PR and say why. Note
# that ci-image.yml still tags with a plain `--short` (auto-scaled), so "always 7" is an
# empirical property of today's shallow clone, not an enforced invariant. Making the
# publisher emit `--short=7` is tracked as ersatztv#597. It is not blocked, just out of
# scope here: editing ci-image.yml re-points `expected` (above) at that commit, so it needs
# the branch's own publish-then-pin two-step (docs/ci-cd.md -> 'CI toolchain image') —
# ci-image.yml's push trigger has no branches: filter, so a feature branch does publish.
# publisher emit `--short=7` is tracked as ersatztv#597. That is no longer blocked by this
# job at all: since ersatztv#744, editing ci-image.yml does NOT re-point `expected`, so a
# `--short=7` change lands like any other PR. It does need a deliberate republish to take
# effect — see the note on `expected` above.
if [ "${#pins[0]}" -ne 7 ]; then
echo "::error::CI toolchain image pin ersatztv-ci:${pins[0]} is ${#pins[0]} chars, but ci-image.yml publishes 7-char tags (it tags with 'git rev-parse --short HEAD' from a fetch-depth:1 clone). A differently-sized abbreviation still resolves to the right commit, so this would pass every other check here — but NO such tag exists in the registry, and all five container: jobs would fail at image-pull time with 'manifest unknown'. Pin exactly: ersatztv-ci:${expected:0:7} (locally: git rev-parse --short=7 HEAD). See docs/ci-cd.md -> 'CI toolchain image'."
exit 1
@@ -121,7 +160,7 @@ jobs:
exit 1
fi
if [ "$pin_full" != "$expected" ]; then
echo "::error::CI toolchain image pin is stale: docker-build.yml pins ersatztv-ci:${pins[0]} ($pin_full), but docker/ci was last changed in $expected. Your jobs are testing an image that is NOT built from this PR's docker/ci. Let ci-image.yml publish the new :<sha>, then update the pin in ALL jobs to it (docs/ci-cd.md -> 'CI toolchain image')."
echo "::error::CI toolchain image pin is stale: docker-build.yml pins ersatztv-ci:${pins[0]} ($pin_full), but docker/ci was last changed in $expected. Your jobs are testing an image that is NOT built from this PR's docker/ci. Publish the new :<sha> — push this commit as branch HEAD and dispatch ci-image.yml on the branch (a branch PUSH no longer publishes, ersatztv#744) — then update the pin in ALL jobs to it (docs/ci-cd.md -> 'CI toolchain image')."
exit 1
fi
echo "Pin is current: ersatztv-ci:${pins[0]} resolves to $pin_full = docker/ci's last change."
@@ -134,16 +173,32 @@ jobs:
name: Docs update reminder
runs-on: small # seconds-long git diff; keep it off the build runners
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: report-only
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
# `continue-on-error` for the same reason the two steps below carry it: this whole job
# is a non-blocking nudge, and an advisory red still joins the combined status the merge gate
# reads. Unmasking the fetch (ersatztv#746) makes a broken base LOUD in the log; it must not
# also make a warn-only job merge-blocking. The three jobs that genuinely gate on this diff —
# api-docs, format, decisions lifecycle — do redden on a failed fetch, which is where that
# belongs.
- name: Warn when a screen/route change skips the parity doc
continue-on-error: true
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only "origin/${base_ref}...HEAD" 2>/dev/null || true)"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only "origin/${base_ref}...HEAD")"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
echo "Changed files in this PR:"; printf '%s\n' "$changed"
screen_or_route=no
if printf '%s\n' "$changed" | grep -Eq '^web/src/screens/.+\.tsx$|^ErsatzTV/LegacyUiRedirects\.cs$'; then
@@ -169,7 +224,8 @@ jobs:
# python-using job on it declares this. Without it a missing interpreter is exit 127 — a RED
# advisory job joining the combined status, which is the one thing this step must never be.
#
# Both steps carry `continue-on-error` because the SCRIPT exiting 0 is not the whole invariant:
# Both steps OF THIS CHECK (setup-python + the narrative step; the parity nudge above has its
# own) carry `continue-on-error` because the SCRIPT exiting 0 is not the whole invariant:
# a setup-python download failure reddens the job just as effectively as a hit would, and an
# advisory red still joins the combined status the merge gate reads (ersatztv#598). Scope,
# stated rather than implied: this covers the two steps that exist to run the check. A failed
@@ -186,7 +242,10 @@ jobs:
continue-on-error: true
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=100 origin "$base_ref" || true
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
python3 scripts/check-doc-narrative.py --diff "origin/${base_ref}"
# BLOCKING (ersatztv#521, supersedes the ersatztv#303 H9 append-only mechanic): validates decision-
@@ -200,10 +259,13 @@ jobs:
name: decisions lifecycle
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
@@ -212,7 +274,10 @@ jobs:
- name: Validate decision lifecycle
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=200 origin "$base_ref" || true
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
PYTHONPATH=. python3 scripts/decisions_validate.py --base "origin/${base_ref}" --head HEAD
- name: Active catalog in sync
run: PYTHONPATH=. python3 scripts/build_decisions_catalog.py --check
@@ -245,12 +310,15 @@ jobs:
name: "Fix proofs (Proves trailers)"
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
# Full history: prove-fix.sh reverts each commit against its PARENT, so a shallow
# clone would leave it unable to resolve `<sha>^` and it would refuse every commit.
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
@@ -362,9 +430,13 @@ jobs:
name: Script lint and tests (ruff + pytest)
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v5
with:
+14
View File
@@ -45,12 +45,26 @@ concurrency:
group: ersatztv-renovate
cancel-in-progress: false
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# This workflow has no checkout step and never uses the injected GITEA_TOKEN for anything. Renovate's
# own branch/PR writes go through RENOVATE_TOKEN, a dedicated bot PAT the Actions default does not
# govern, and its container image comes from Docker Hub. Read-only is declared to STATE that the
# injected token is unused, not because any step needs it.
permissions:
code: read
jobs:
renovate:
name: Renovate
runs-on: ubuntu-latest
container:
image: renovate/renovate:43
env:
CI_JOB_ROLE: none
steps:
- name: Run Renovate
env:
File diff suppressed because it is too large Load Diff
+11
View File
@@ -95,3 +95,14 @@ web/playwright-report/
# plaintext Gitea credential and absolute /Users paths, so it is neither portable nor safe to
# commit. See ersatztv#711 for the related merge-gate gap.
.codex/
# serena's per-project state, written by `activate_project` (ersatztv#799): project.yml,
# project.local.yml, a language-server cache, and memories/.
#
# This deliberately rejects serena's own versioning model. Its nested .serena/.gitignore excludes
# only `cache` and `project.local.yml`, and project.local.yml says project.yml "is intended to be
# versioned" — but activation here is per DIRECTORY, and every worktree generates a project.yml
# whose project_name is that worktree's folder (e.g. `781-tooling`). A committed copy would name
# the wrong project in every checkout but the one that produced it. memories/ is ignored with it:
# it is serena's own written notes, and this repo's durable knowledge lives in docs/ instead.
.serena/
+17 -4
View File
@@ -83,7 +83,7 @@ main in) and re-run the local gate whenever the fetch shows movement.
Every task that closes a Gitea issue MUST complete ALL of these before it is considered done. Use `/done <issue>` to run through this automatically.
**Merge-consent is derived from state, not asserted (`## Done-when` convention — ersatztv#303 H6 + H10).** Any issue whose PR will merge to `main` should carry a `## Done-when` section in its **issue body** — a checklist of completion criteria (always include an "adversarial review passed" box; add per-issue criteria like tests-green, docs-updated, live-E2E). Two hooks derive merge-consent from it so a premature merge is blocked *by construction*, not by memory:
- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), run **`scripts/post-review-verdict.sh <pr> <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> [note]`** — it posts both the `Review-verdict: … @ <head-sha>` comment and the sha-bound `review-verdict/h10` commit status, proving the *latest* commit was reviewed rather than a stale earlier diff (ersatztv#242). Do not hand-write the comment: the **status** is the required check branch protection enforces, and a comment alone leaves it absent.
- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), run **`scripts/post-review-verdict.sh <pr> <MERGEABLE|APPROVED|LGTM|BLOCKED|NOT-MERGEABLE> [note]`** — it posts both the `Review-verdict: … @ <head-sha>` comment and the sha-bound `review-verdict/h10` commit status, proving the *latest* commit was reviewed rather than a stale earlier diff (ersatztv#242). Do not hand-write the comment: the **status** is the required check branch protection enforces, and a comment alone leaves it absent. **The credential you post with must be an account on `H10_REVIEWERS` in `.gitea/workflows/review-verdict.yml`** (`timothy` today) — since ersatztv#742 the gate inherits an existing `success` only from an allow-listed creator (an existing `failure` is left alone on a weaker attributability test, so an attributable rejection VISIBLE AT THE FIRST READ is not re-derived into a green — a rejection landing later, inside a run's own write window, is a separate and still-open route, ersatztv#849), so a verdict posted with any other account is written, reported as success by the script, and then silently re-derived on the next PR event (ersatztv#845).
- **The gate is enforced server-side, per sha (ersatztv#622).** `review-verdict/h10` is a required status check on `main`. Because a commit status belongs to one sha, a commit pushed *after* an auto-merge is scheduled clears it and blocks the merge — closing the hole where `merge_when_checks_succeed` froze consent at scheduling time and Gitea later merged an unreviewed head. Renovate-authored and docs-only PRs are auto-passed by `.gitea/workflows/review-verdict.yml`, **except** when they touch `.claude/`, `.codex/`, `.gitea/`, `.husky/`, `scripts/` or `docker/ci/`. See `docs/ci-cd.md` → Review-verdict gate.
- `.husky/pre-push``prepush-donewhen.sh` — a fail-open backstop that blocks a direct `git push origin main` whose commits `fix #N` an issue with unticked boxes. **Since ersatztv#743 that push can no longer happen at all** (see below), so this hook is now belt-and-braces for a path the server refuses.
@@ -97,17 +97,30 @@ when finishing a task that closes an issue.
## Project Boundaries
**ersatztv OWNS**: ErsatzTV fork code (C#/.NET), channel/collection/schedule management, M3U/XMLTV generation, and the **`ersatztv` skill** — whose canonical copy is `.claude/skills/ersatztv/SKILL.md` **here**; `~/server-management/.claude/skills/ersatztv` is a symlink to it (ersatztv#617). Edit it in this repo; never fork a second copy.
**ersatztv OWNS***developing the fork*: the ErsatzTV fork code (C#/.NET), the `/api/v1` REST
surface, M3U/XMLTV generation, the `ErsatzTV.Mcp` server, CI and releases, and the **`ersatztv`
skill** — whose canonical copy is `.claude/skills/ersatztv/SKILL.md` **here**. Both
`~/server-management/.claude/skills/ersatztv` and `~/media-management/.claude/skills/ersatztv` are
symlinks to it (ersatztv#617, #755). Edit it in this repo; never fork a second copy.
**The split that is easy to get wrong** (ersatztv#755, `process.ersatztv-owns-code-not-operations`):
channel/collection/schedule *code* is owned here; **channel OPERATIONS against the running instance
are not**. Creating and editing channels, lineups, collections, schedules, playouts, logos and
overlays on the live ErsatzTV belong to `media-management`. Driving prod from here is in scope only
as *verification of a change this repo is shipping* (live-E2E, a release smoke test) — not as
day-to-day channel work.
**ersatztv does NOT own**:
- Channel/collection/schedule/playout **operations** against a live instance → media-management
- Docker compose configs → server-management (`~/downloadswarm/stacks/ersatztv/`)
- NFS mounts, Ansible, DNS, networking → server-management
- Content sourcing (yt-dlp downloads, Sonarr/Radarr libraries) → media-management (planned)
- Content sourcing (yt-dlp downloads, Sonarr/Radarr libraries) → media-management
- Jellyfin skill → server-management. `.claude/skills/jellyfin` here is a **relative symlink** to `~/server-management/.claude/skills/jellyfin` (ersatztv#617 — it had silently become a stale divergent copy). It therefore resolves only in a checkout at `~/ersatztv`, not inside a git worktree; that is inherent to the cross-repo symlink pattern server-management already uses (`beets`, `radarr`, `sonarr`, …).
**For infrastructure changes** (Docker, NFS, ports, Authelia): open an issue in `timothy/server-management`.
**For content/media sourcing questions** (what goes into channels, yt-dlp pipelines): open an issue in `timothy/media-management` once it exists; for now, `timothy/server-management`.
**For content/media sourcing questions and channel operations** (what goes into channels, yt-dlp
pipelines, editing a live channel): open an issue in `timothy/media-management`.
**For plan/audit reviews**: open `~/adversarial-reviewer` before significant architecture changes.
+1 -1
View File
@@ -6,7 +6,7 @@
<ItemGroup>
<PackageVersion Include="AsyncFixer" Version="2.1.0" />
<PackageVersion Include="Blurhash.SkiaSharp" Version="2.0.0" />
<PackageVersion Include="CliWrap" Version="3.10.5" />
<PackageVersion Include="CliWrap" Version="3.10.4" />
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
<PackageVersion Include="Dapper" Version="2.1.79" />
<PackageVersion Include="Destructurama.Attributed" Version="5.2.0" />
@@ -0,0 +1,43 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
namespace ErsatzTV.Application.Channels;
/// <summary>
/// #732: the On Now / Next overlay is a default rather than an opt-in, so every newly created channel
/// gets the built-in element attached.
/// </summary>
/// <remarks>
/// This lives in one place because there is more than one channel-creation path and they diverged
/// once already: <c>CreateChannelHandler</c> had it and <c>CreateChannelFromLineupHandler</c> -- the
/// SPA's primary "Add Channel" flow, and the one Auto-Tune bulk-creates through -- did not. Any new
/// site that persists a <c>Channel</c> must call this. The third site, <c>DbInitializer</c>'s default
/// channel, needs no call: it runs before <c>AttachOnNowNextByDefault</c> in the same startup, so the
/// backfill covers it.
/// </remarks>
public static class ChannelGraphicsDefaults
{
public static async Task Attach(TvContext dbContext, Channel channel, CancellationToken cancellationToken)
{
// HLS Direct is skipped because ErsatzTV is not transcoding there -- there is no frame
// pipeline to draw into, and the editor disables the toggle for the same reason. Identity is
// the element's filename, never its user-editable Name (the #67 lesson).
if (channel.StreamingMode is StreamingMode.HttpLiveStreamingDirect)
{
return;
}
Option<int> maybeElementId =
await GraphicsElementSeeder.GetBuiltInElementId(dbContext, cancellationToken);
foreach (int elementId in maybeElementId)
{
// Add rather than assign: a future create path that carries graphics ids would otherwise
// be silently discarded here.
channel.ChannelGraphicsElements ??= [];
channel.ChannelGraphicsElements.Add(new ChannelGraphicsElement { GraphicsElementId = elementId });
}
}
}
@@ -85,6 +85,7 @@ public class CreateChannelFromLineupHandler(
await using var transaction = await dbContext.Database.BeginTransactionAsync(cancellationToken);
try
{
await ChannelGraphicsDefaults.Attach(dbContext, prepared.Channel, cancellationToken);
dbContext.Channels.Add(prepared.Channel);
if (prepared.Playlist is not null)
{
@@ -7,6 +7,7 @@ using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Infrastructure.Extensions;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Channels.ChannelValidations;
@@ -35,7 +36,8 @@ public class CreateChannelHandler(
Right: async logoPath =>
{
ApplyResolvedLogo(request, channel, logoPath);
return Right<BaseError, CreateChannelResult>(await PersistChannel(dbContext, channel));
return Right<BaseError, CreateChannelResult>(
await PersistChannel(dbContext, channel, cancellationToken));
},
Left: e => Task.FromResult(Left<BaseError, CreateChannelResult>(e)));
},
@@ -75,8 +77,12 @@ public class CreateChannelHandler(
}
}
private async Task<CreateChannelResult> PersistChannel(TvContext dbContext, Channel channel)
private async Task<CreateChannelResult> PersistChannel(
TvContext dbContext,
Channel channel,
CancellationToken cancellationToken)
{
await ChannelGraphicsDefaults.Attach(dbContext, channel, cancellationToken);
await dbContext.Channels.AddAsync(channel);
await dbContext.SaveChangesAsync();
searchTargets.SearchTargetsChanged();
@@ -35,4 +35,6 @@ public record CreateFFmpegProfile(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder) : IRequest<Either<BaseError, CreateFFmpegProfileResult>>;
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup) : IRequest<Either<BaseError, CreateFFmpegProfileResult>>;
@@ -50,8 +50,12 @@ public class CreateFFmpegProfileHandler :
private static Validation<BaseError, FFmpegProfile> Validate(
CreateFFmpegProfile request,
int resolutionId) =>
(ValidateName(request), ValidateThreadCount(request))
.Apply((name, threadCount) =>
(ValidateName(request),
ValidateThreadCount(request),
FFmpegProfileBounds.ValidateQsvExtraHardwareFrames(request.QsvExtraHardwareFrames, stored: null),
FFmpegProfileBounds.ValidateReadRate(request.ReadRate),
FFmpegProfileBounds.ValidateReadRateCatchup(request.ReadRateCatchup, request.ReadRate))
.Apply((name, threadCount, _, _, _) =>
{
var hwAccel = request.NormalizeVideo
? request.HardwareAcceleration
@@ -68,11 +72,9 @@ public class CreateFFmpegProfileHandler :
HardwareAcceleration = hwAccel,
VaapiDriver = request.VaapiDriver,
VaapiDevice = request.VaapiDevice,
// store what the pipeline will actually use, never a pool size FFmpegState would
// floor away at render time (ersatztv#529)
QsvExtraHardwareFrames = request.QsvExtraHardwareFrames is { } frames
? Math.Max(frames, FFmpegState.MinimumQsvExtraHardwareFrames)
: null,
// stored exactly as submitted: an out-of-range value was already rejected with a
// 422 naming the bound, so there is nothing left to silently rewrite (ersatztv#735)
QsvExtraHardwareFrames = request.QsvExtraHardwareFrames,
ResolutionId = resolutionId,
ScalingBehavior = request.ScalingBehavior,
@@ -111,7 +113,9 @@ public class CreateFFmpegProfileHandler :
NormalizeFramerate = request.NormalizeFramerate,
NormalizeColors = request.NormalizeColors,
DeinterlaceVideo = request.DeinterlaceVideo,
QsvPreferNativeDecoder = request.QsvPreferNativeDecoder
QsvPreferNativeDecoder = request.QsvPreferNativeDecoder,
ReadRate = request.ReadRate,
ReadRateCatchup = request.ReadRateCatchup
};
});
@@ -36,4 +36,6 @@ public record UpdateFFmpegProfile(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder) : IRequest<Either<BaseError, UpdateFFmpegProfileResult>>;
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup) : IRequest<Either<BaseError, UpdateFFmpegProfileResult>>;
@@ -55,11 +55,10 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
p.VaapiDisplay = update.VaapiDisplay;
p.VaapiDriver = update.VaapiDriver;
p.VaapiDevice = update.VaapiDevice;
// store what the pipeline will actually use, so a profile doesn't keep displaying a pool
// size that FFmpegState floors away at render time (ersatztv#529)
p.QsvExtraHardwareFrames = update.QsvExtraHardwareFrames is { } frames
? Math.Max(frames, FFmpegState.MinimumQsvExtraHardwareFrames)
: null;
// stored exactly as submitted: an out-of-range NEW value was already rejected with a 422
// naming the bound. an unchanged value that predates that validation is written back as-is
// rather than rewritten, and FFmpegState floors it at render time (ersatztv#735)
p.QsvExtraHardwareFrames = update.QsvExtraHardwareFrames;
p.ResolutionId = update.ResolutionId;
p.ScalingBehavior = update.ScalingBehavior;
p.PadMode = update.PadMode;
@@ -108,6 +107,8 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
p.NormalizeColors = update.NormalizeColors;
p.DeinterlaceVideo = update.DeinterlaceVideo;
p.QsvPreferNativeDecoder = update.QsvPreferNativeDecoder;
p.ReadRate = update.ReadRate;
p.ReadRateCatchup = update.ReadRateCatchup;
// don't save invalid preset
ICollection<string> presets = FFmpegLibraryHelper.PresetsForFFmpegProfile(
@@ -131,8 +132,14 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
TvContext dbContext,
UpdateFFmpegProfile request,
FFmpegProfile profile) =>
(await ValidateName(dbContext, request), ValidateThreadCount(request))
.Apply((_, _) => profile);
(await ValidateName(dbContext, request),
ValidateThreadCount(request),
FFmpegProfileBounds.ValidateQsvExtraHardwareFrames(
request.QsvExtraHardwareFrames,
profile.QsvExtraHardwareFrames),
FFmpegProfileBounds.ValidateReadRate(request.ReadRate),
FFmpegProfileBounds.ValidateReadRateCatchup(request.ReadRateCatchup, request.ReadRate))
.Apply((_, _, _, _, _) => profile);
private static Task<Option<FFmpegProfile>> FFmpegProfileMustExist(
TvContext dbContext,
@@ -0,0 +1,79 @@
using ErsatzTV.Core;
using ErsatzTV.FFmpeg;
namespace ErsatzTV.Application.FFmpegProfiles;
/// <summary>
/// Write-path bounds for the consequential numeric FFmpeg profile fields.
/// A submitted value outside its documented range is REJECTED, naming the bound, rather than
/// accepted and silently rewritten to something the caller never sent (ersatztv#735). The
/// render-time clamps in <see cref="FFmpegState" /> stay as they are: they cover rows that
/// predate this validation or were written out of band, which is what keeps the fix
/// migration-free.
/// </summary>
internal static class FFmpegProfileBounds
{
internal static Validation<BaseError, Unit> ValidateQsvExtraHardwareFrames(int? requested, int? stored)
{
// a row stored before this validation existed may hold anything, and the SPA sends the whole
// profile back on every edit — so rejecting an UNCHANGED legacy value would make an old
// profile uneditable over a field the operator never touched (and cannot even see unless
// hardware acceleration is QSV). only a NEWLY submitted out-of-range value is rejected;
// FFmpegState.QsvExtraHardwareFrames still floors the legacy one at render time
if (requested is null || requested == stored)
{
return Success<BaseError, Unit>(Unit.Default);
}
return requested < FFmpegState.MinimumQsvExtraHardwareFrames
? BaseError.New(
$"QSV extra hardware frames must be at least {FFmpegState.MinimumQsvExtraHardwareFrames}; " +
$"{requested} leaves the QSV upload pool with too little headroom and the transcode writes nothing at all")
: Success<BaseError, Unit>(Unit.Default);
}
internal static Validation<BaseError, Unit> ValidateReadRate(double? requested)
{
if (requested is null)
{
return Success<BaseError, Unit>(Unit.Default);
}
return requested is < FFmpegState.MinimumReadRate or > FFmpegState.MaximumReadRate
? BaseError.New(
$"Read rate must be between {Format(FFmpegState.MinimumReadRate)} and {Format(FFmpegState.MaximumReadRate)}; " +
"below realtime the channel stalls, and above this the input is no longer meaningfully paced")
: Success<BaseError, Unit>(Unit.Default);
}
internal static Validation<BaseError, Unit> ValidateReadRateCatchup(double? requested, double? requestedReadRate)
{
if (requested is null)
{
return Success<BaseError, Unit>(Unit.Default);
}
if (requested is < FFmpegState.MinimumReadRateCatchup or > FFmpegState.MaximumReadRateCatchup)
{
return BaseError.New(
$"Read rate catchup must be between {Format(FFmpegState.MinimumReadRateCatchup)} and " +
$"{Format(FFmpegState.MaximumReadRateCatchup)}");
}
// catchup is the rate a LAGGING input may read at until it is level again, so a value at or
// below the base rate cannot let it recover: EQUAL is rejected too, because a catchup with
// zero headroom is functionally no catchup while still reading as configured. compared
// against the transcode default rather than the stream-copy one because that is the higher
// of the two: a value that clears it clears both, without this check having to know the
// profile's video format
double effectiveReadRate = requestedReadRate ?? FFmpegState.DefaultReadRate;
return requested <= effectiveReadRate
? BaseError.New(
$"Read rate catchup ({Format(requested.Value)}) must be greater than the read rate " +
$"({Format(effectiveReadRate)}); a lagging input cannot catch up at a rate it is already paced at")
: Success<BaseError, Unit>(Unit.Default);
}
private static string Format(double value) =>
value.ToString("0.0####", System.Globalization.CultureInfo.InvariantCulture);
}
@@ -36,4 +36,6 @@ public record FFmpegProfileViewModel(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder);
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup);
@@ -38,7 +38,9 @@ internal static class Mapper
profile.NormalizeFramerate,
profile.NormalizeColors,
profile.DeinterlaceVideo == true,
profile.QsvPreferNativeDecoder != false);
profile.QsvPreferNativeDecoder != false,
profile.ReadRate,
profile.ReadRateCatchup);
internal static FFmpegProfileResponseModel ProjectToResponseModel(FFmpegProfile ffmpegProfile) =>
new(
@@ -82,5 +84,7 @@ internal static class Mapper
ffmpegProfile.NormalizeFramerate,
ffmpegProfile.NormalizeColors,
ffmpegProfile.DeinterlaceVideo == true,
ffmpegProfile.QsvPreferNativeDecoder != false);
ffmpegProfile.QsvPreferNativeDecoder != false,
ffmpegProfile.ReadRate,
ffmpegProfile.ReadRateCatchup);
}
@@ -1,4 +1,5 @@
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Filler.Mapper;
@@ -12,9 +13,13 @@ public class GetPagedFillerPresetsHandler(IDbContextFactory<TvContext> dbContext
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.FillerPresets.CountAsync(cancellationToken);
List<FillerPresetViewModel> page = await dbContext.FillerPresets
.AsNoTracking()
// no filter today, but count and page are still derived from ONE query so that adding one
// cannot leave the count behind (api.paged-count-matches-page-query)
IQueryable<FillerPreset> query = dbContext.FillerPresets.AsNoTracking();
int count = await query.CountAsync(cancellationToken);
List<FillerPresetViewModel> page = await query
.OrderBy(f => f.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.MediaCollections.Mapper;
@@ -13,8 +13,6 @@ public class GetPagedCollectionsHandler(IDbContextFactory<TvContext> dbContextFa
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.Collections.CountAsync(cancellationToken);
IQueryable<Collection> query = dbContext.Collections.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -22,6 +20,9 @@ public class GetPagedCollectionsHandler(IDbContextFactory<TvContext> dbContextFa
query = query.Where(c => EF.Functions.Like(c.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<MediaCollectionViewModel> page = await query
.OrderBy(c => c.Name)
.Skip(request.PageNum * request.PageSize)
@@ -13,9 +13,6 @@ public class GetPagedMultiCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.MultiCollections
.CountAsync(mc => mc.OwnedByChannelId == null, cancellationToken);
IQueryable<MultiCollection> query = dbContext.MultiCollections
.AsNoTracking()
.Where(mc => mc.OwnedByChannelId == null);
@@ -25,6 +22,9 @@ public class GetPagedMultiCollectionsHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(mc => EF.Functions.Like(mc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<MultiCollectionViewModel> page = await query
.OrderBy(mc => mc.Name)
.Skip(request.PageNum * request.PageSize)
@@ -13,18 +13,21 @@ public class GetPagedRerunCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.RerunCollections.CountAsync(cancellationToken);
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking().IncludeSelectionDetails();
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
{
query = query.Where(rc => EF.Functions.Like(rc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758).
// The includes belong to the page chain only — a COUNT does not materialize the graph.
int count = await query.CountAsync(cancellationToken);
// EF applies the includes to the paged subquery, so the selection graph is loaded for at most
// PageSize rows — the per-request cost is bounded by the page, not by the table (issue #671).
List<RerunCollectionViewModel> page = await query
.IncludeSelectionDetails()
.OrderBy(rc => rc.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -13,9 +13,6 @@ public class GetPagedSmartCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.SmartCollections
.CountAsync(sc => sc.OwnedByChannelId == null, cancellationToken);
IQueryable<SmartCollection> query = dbContext.SmartCollections
.AsNoTracking()
.Where(sc => sc.OwnedByChannelId == null);
@@ -25,6 +22,9 @@ public class GetPagedSmartCollectionsHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(sc => EF.Functions.Like(sc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<SmartCollectionViewModel> page = await query
.OrderBy(s => s.Name)
.Skip(request.PageNum * request.PageSize)
@@ -1,4 +1,5 @@
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.MediaCollections.Mapper;
@@ -12,9 +13,13 @@ public class GetPagedTraktListsHandler(IDbContextFactory<TvContext> dbContextFac
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.TraktLists.CountAsync(cancellationToken);
List<TraktListViewModel> page = await dbContext.TraktLists
.AsNoTracking()
// no filter today, but count and page are still derived from ONE query so that adding one
// cannot leave the count behind (api.paged-count-matches-page-query)
IQueryable<TraktList> query = dbContext.TraktLists.AsNoTracking();
int count = await query.CountAsync(cancellationToken);
List<TraktListViewModel> page = await query
.OrderBy(l => l.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
+10 -3
View File
@@ -1,5 +1,6 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
namespace ErsatzTV.Application.Playouts;
@@ -40,9 +41,15 @@ internal static class Mapper
programScheduleAlternate.Id,
programScheduleAlternate.Index,
programScheduleAlternate.ProgramScheduleId,
programScheduleAlternate.DaysOfWeek,
programScheduleAlternate.DaysOfMonth,
programScheduleAlternate.MonthsOfYear,
// ersatztv#823: these three are NULLABLE columns and a legacy row can hold NULL. Substitute the
// SAME unrestricted defaults AlternateScheduleSelector.GetScheduleForDate reads, so the DTO the
// SPA renders agrees with what actually gets scheduled -- web/src/screens/playoutTemplateCalendar.ts
// `appliesToDate` is an exact port of that method, and it would otherwise both mispreview and
// throw (`[...template.daysOfMonth]` on a null is a TypeError). Never assigned back onto the
// entity (`media.nullable-primitive-collection-mutation`).
programScheduleAlternate.DaysOfWeek ?? AlternateScheduleSelector.AllDaysOfWeek(),
programScheduleAlternate.DaysOfMonth ?? AlternateScheduleSelector.AllDaysOfMonth(),
programScheduleAlternate.MonthsOfYear ?? AlternateScheduleSelector.AllMonthsOfYear(),
programScheduleAlternate.LimitToDateRange,
programScheduleAlternate.StartMonth,
programScheduleAlternate.StartDay,
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Playouts.Mapper;
@@ -13,13 +13,8 @@ public class GetPagedPlayoutsHandler(IDbContextFactory<TvContext> dbContextFacto
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.Playouts.CountAsync(cancellationToken);
IQueryable<Playout> query = dbContext.Playouts
.AsNoTracking()
.Include(p => p.Channel)
.Include(p => p.ProgramSchedule)
.Include(p => p.BuildStatus)
.Filter(p => p.Channel != null);
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -27,7 +22,15 @@ public class GetPagedPlayoutsHandler(IDbContextFactory<TvContext> dbContextFacto
query = query.Where(p => EF.Functions.Like(p.Channel.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758).
// This is also what makes the `Channel != null` filter count, which the old unfiltered
// CountAsync over the whole DbSet did not.
int count = await query.CountAsync(cancellationToken);
List<PlayoutNameViewModel> page = await query
.Include(p => p.Channel)
.Include(p => p.ProgramSchedule)
.Include(p => p.BuildStatus)
.OrderBy(p => p.Channel.SortNumber)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.ProgramSchedules.Mapper;
@@ -13,8 +13,6 @@ public class GetPagedProgramSchedulesHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.ProgramSchedules.CountAsync(cancellationToken);
IQueryable<ProgramSchedule> query = dbContext.ProgramSchedules.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -22,6 +20,9 @@ public class GetPagedProgramSchedulesHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(ps => EF.Functions.Like(ps.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<ProgramScheduleViewModel> page = await query
.OrderBy(ps => ps.Name)
.Skip(request.PageNum * request.PageSize)
+11 -4
View File
@@ -1,6 +1,7 @@
using ErsatzTV.Application.Tree;
using ErsatzTV.Application.Tree;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
namespace ErsatzTV.Application.Scheduling;
@@ -190,9 +191,15 @@ internal static class Mapper
ProjectToViewModel(playoutTemplate.Template),
ProjectToViewModel(playoutTemplate.DecoTemplate),
playoutTemplate.Index,
playoutTemplate.DaysOfWeek,
playoutTemplate.DaysOfMonth,
playoutTemplate.MonthsOfYear,
// ersatztv#823: these three are NULLABLE columns and a legacy row can hold NULL. Substitute the
// SAME unrestricted defaults AlternateScheduleSelector.GetScheduleForDate reads, so the DTO the
// SPA renders agrees with what actually gets scheduled -- web/src/screens/playoutTemplateCalendar.ts
// `appliesToDate` is an exact port of that method, and it would otherwise both mispreview and
// throw (`[...template.daysOfMonth]` on a null is a TypeError). Never assigned back onto the
// entity (`media.nullable-primitive-collection-mutation`).
playoutTemplate.DaysOfWeek ?? AlternateScheduleSelector.AllDaysOfWeek(),
playoutTemplate.DaysOfMonth ?? AlternateScheduleSelector.AllDaysOfMonth(),
playoutTemplate.MonthsOfYear ?? AlternateScheduleSelector.AllMonthsOfYear(),
playoutTemplate.LimitToDateRange,
playoutTemplate.StartMonth,
playoutTemplate.StartDay,
@@ -1,3 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
using NUnit.Framework;
@@ -864,4 +865,241 @@ public static class AlternateScheduleSelectorTests
result.IsNone.ShouldBeFalse();
}
}
/// <summary>
/// ersatztv#823. <c>DaysOfWeek</c>, <c>DaysOfMonth</c> and <c>MonthsOfYear</c> on
/// <see cref="PlayoutTemplate" /> and <see cref="ProgramScheduleAlternate" /> are six
/// single-column primitive collections whose columns are <c>nullable: true</c> on both providers.
/// A NULL column materializes as CLR <c>null</c> — EF does not invoke the value converter for a
/// NULL at all — so unguarded, each <c>.Contains</c> in
/// <see cref="AlternateScheduleSelector.GetScheduleForDate{T}" /> throws
/// <see cref="NullReferenceException" />. These tests are RED without the read-site guard.
/// <para>
/// A null reads as UNRESTRICTED (the <c>All*()</c> sets), not as empty. The deciding case is
/// SQLite's <c>20240113140741_Add_PlayoutTemplate_DaysOfMonth</c>, which adds the column
/// <c>nullable: true</c> with NO default: a row inserted before it had no day-of-month
/// restriction, so reading its NULL as empty would INVERT its meaning and silently stop the
/// template applying. That is the one NULL reachable without any code writing one.
/// </para>
/// <para>
/// Reachability itself is pinned by
/// <c>ErsatzTV.Tests.Integration.SchedulingCollectionColumnNullTests</c> against a real
/// <c>TvContext</c>; these tests pin what the selector does once the null is there.
/// </para>
/// </summary>
[TestFixture]
public class GetScheduleForDate_NullCollections
{
private static readonly TimeSpan Offset = TimeSpan.FromHours(-5);
// A Wednesday in March, so no All*() member is coincidentally excluded — and deliberately the
// 20th rather than the 6th. With a day <= 12 a CROSS-WIRED substitution survives the whole
// fixture: `DaysOfMonth ?? AllMonthsOfYear()` hands back 1..12, which still contains day 6, so
// every assertion here passes while the guard substitutes the wrong set. Day 20 is outside 1..12
// and kills it.
private static readonly DateTimeOffset AnyDate = new(2024, 3, 20, 0, 0, 0, Offset);
private static PlayoutTemplate Unrestricted() =>
new()
{
DaysOfWeek = AlternateScheduleSelector.AllDaysOfWeek(),
DaysOfMonth = AlternateScheduleSelector.AllDaysOfMonth(),
MonthsOfYear = AlternateScheduleSelector.AllMonthsOfYear()
};
private static Option<PlayoutTemplate> Select(params PlayoutTemplate[] templates) =>
AlternateScheduleSelector.GetScheduleForDate(templates.ToList(), AnyDate);
[Test]
public void Null_DaysOfWeek_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
Select(template).IsSome.ShouldBeTrue(
"a NULL DaysOfWeek means no weekday restriction was recorded, so the template still applies");
}
[Test]
public void Null_DaysOfMonth_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfMonth = null!;
Select(template).IsSome.ShouldBeTrue();
}
[Test]
public void Null_MonthsOfYear_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.MonthsOfYear = null!;
Select(template).IsSome.ShouldBeTrue();
}
[Test]
public void All_Three_Null_On_ProgramScheduleAlternate_Reads_As_Unrestricted()
{
var alternate = new ProgramScheduleAlternate
{
DaysOfWeek = null!,
DaysOfMonth = null!,
MonthsOfYear = null!
};
AlternateScheduleSelector.GetScheduleForDate(
new List<ProgramScheduleAlternate> { alternate },
AnyDate)
.IsSome.ShouldBeTrue();
}
/// <summary>
/// THE DISCRIMINATING CONTROL. Every test above sets a NULL and expects the item to be selected,
/// so all of them pass equally under "NULL means unrestricted" and under the much broader
/// "any NULL makes this item match unconditionally" — a refactor that short-circuits the whole
/// date check when any dimension is null keeps them green. Here the nulled dimension is paired
/// with a RESTRICTIVE non-null one that the date fails, so only the narrow reading passes.
/// </summary>
[Test]
public void A_Null_Dimension_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
// AnyDate is in MARCH; restrict to January only.
template.MonthsOfYear = [1];
Select(template).IsNone.ShouldBeTrue(
"a NULL DaysOfWeek relaxes ONLY the weekday dimension — the January restriction still "
+ "excludes a March date");
}
/// <summary>
/// One arrangement is not enough: with only the <c>DaysOfWeek</c> case above, a PER-DIMENSION
/// mutant survives the whole fixture — e.g. <c>if (item.MonthsOfYear is null) return item;</c>
/// placed ahead of the checks is never reached by that test, because its <c>MonthsOfYear</c> is
/// non-null. So each of the three dimensions is nulled in turn against a restriction on a
/// DIFFERENT dimension.
/// </summary>
[Test]
public void A_Null_MonthsOfYear_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.MonthsOfYear = null!;
// AnyDate is a WEDNESDAY; restrict to Monday only.
template.DaysOfWeek = [DayOfWeek.Monday];
Select(template).IsNone.ShouldBeTrue(
"a NULL MonthsOfYear relaxes ONLY the month dimension — the Monday restriction still "
+ "excludes a Wednesday");
}
/// <summary>
/// The third of the per-dimension controls — see
/// <see cref="A_Null_MonthsOfYear_Does_Not_Relax_The_Other_Dimensions" /> for why one
/// arrangement is not enough. Here the nulled dimension is <c>DaysOfMonth</c> and the
/// restriction that must still bite is on <c>MonthsOfYear</c>.
/// </summary>
[Test]
public void A_Null_DaysOfMonth_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfMonth = null!;
// AnyDate is in MARCH; restrict to January only.
template.MonthsOfYear = [1];
Select(template).IsNone.ShouldBeTrue(
"a NULL DaysOfMonth relaxes ONLY the day-of-month dimension — the January restriction "
+ "still excludes a March date");
}
/// <summary>
/// A null must not be confused with an explicitly EMPTY collection. Empty is a legal, reachable
/// state meaning "matches no day", and it keeps that meaning — which is exactly why a NULL
/// cannot be normalized to it.
/// </summary>
[Test]
public void An_Explicitly_Empty_Collection_Still_Matches_Nothing()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = [];
Select(template).IsNone.ShouldBeTrue(
"an empty DaysOfWeek is a recorded restriction of NO days, unlike a NULL");
}
/// <summary>
/// The guard resolves PER ITEM: a null on the first item must not decide the second. Making the
/// nulled item genuinely non-matching is what measures that — with an unrestricted nulled item
/// at index 0 it simply wins on ordering and the second item is never evaluated, so the
/// invariant would go unmeasured while the test passed.
/// </summary>
[Test]
public void A_Null_On_One_Item_Does_Not_Decide_A_Later_Item()
{
PlayoutTemplate nulled = Unrestricted();
nulled.DaysOfWeek = null!;
nulled.MonthsOfYear = [1]; // AnyDate is in March, so this item must NOT match
nulled.Index = 0;
PlayoutTemplate second = Unrestricted();
second.Index = 1;
foreach (PlayoutTemplate selected in Select(nulled, second))
{
selected.ShouldBeSameAs(second);
return;
}
Assert.Fail("the loop stopped at the null-collection item instead of continuing to the next");
}
/// <summary>
/// ...and when the nulled item IS unrestricted it legitimately wins on ordering. Paired with the
/// test above so "index 0 wins" and "the loop continues past a non-matching null item" are
/// separately pinned.
/// </summary>
[Test]
public void An_Unrestricted_Null_Item_Wins_On_Index_Order()
{
PlayoutTemplate nulled = Unrestricted();
nulled.DaysOfWeek = null!;
nulled.Index = 0;
PlayoutTemplate second = Unrestricted();
second.Index = 1;
foreach (PlayoutTemplate selected in Select(nulled, second))
{
selected.ShouldBeSameAs(nulled);
return;
}
Assert.Fail("the null-collection item was skipped instead of read as unrestricted");
}
/// <summary>
/// The read-site guard must not be written BACK onto the item. These are single-column
/// primitive collections, so assigning the guard would flip a tracked entity to
/// <c>Modified</c> and the next <c>SaveChanges</c> would persist the substituted collection
/// over the NULL — the mechanism recorded as <c>media.nullable-primitive-collection-mutation</c>.
/// </summary>
[Test]
public void Guard_Must_Not_Be_Written_Back_Onto_The_Item()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
template.DaysOfMonth = null!;
template.MonthsOfYear = null!;
Select(template);
template.DaysOfWeek.ShouldBeNull();
template.DaysOfMonth.ShouldBeNull();
template.MonthsOfYear.ShouldBeNull();
}
}
}
@@ -37,4 +37,6 @@ public record FFmpegFullProfileResponseModel(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder);
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup);
+3
View File
@@ -26,6 +26,9 @@ public class ConfigElementKey
public static ConfigElementKey ChannelTemplatesDefaultTemplateId => new("channel_templates.default_template_id");
public static ConfigElementKey WatermarkChannelBugSeeded => new("watermark.channel_bug_seeded");
public static ConfigElementKey GraphicsOnNowNextSeeded => new("graphics.on_now_next_seeded");
public static ConfigElementKey GraphicsOnNowNextDefaultAttached =>
new("graphics.on_now_next_default_attached");
public static ConfigElementKey FFmpegSegmenterTimeout => new("ffmpeg.segmenter.timeout_seconds");
public static ConfigElementKey FFmpegWorkAheadSegmenters => new("ffmpeg.segmenter.work_ahead_limit");
public static ConfigElementKey FFmpegInitialSegmentCount => new("ffmpeg.segmenter.initial_segment_count");
+2
View File
@@ -14,6 +14,8 @@ public record FFmpegProfile
public VaapiDriver VaapiDriver { get; set; }
public string VaapiDevice { get; set; }
public int? QsvExtraHardwareFrames { get; set; }
public double? ReadRate { get; set; }
public double? ReadRateCatchup { get; set; }
public bool? QsvPreferNativeDecoder { get; set; }
public int ResolutionId { get; set; }
public Resolution Resolution { get; set; }
@@ -610,7 +610,9 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
channel.FFmpegProfile.QsvPreferNativeDecoder != false);
channel.FFmpegProfile.QsvPreferNativeDecoder != false,
Optional(channel.FFmpegProfile.ReadRate),
Optional(channel.FFmpegProfile.ReadRateCatchup));
_logger.LogDebug("FFmpeg desired state {FrameState}", desiredState);
@@ -827,7 +829,9 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel);
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
MaybeReadRate: Optional(channel.FFmpegProfile.ReadRate),
MaybeReadRateCatchup: Optional(channel.FFmpegProfile.ReadRateCatchup));
var ffmpegSubtitleStream = new ErsatzTV.FFmpeg.MediaStream(0, "ass", StreamKind.Video);
@@ -968,7 +972,9 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel);
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
MaybeReadRate: Optional(channel.FFmpegProfile.ReadRate),
MaybeReadRateCatchup: Optional(channel.FFmpegProfile.ReadRateCatchup));
var audioInputFile = new NullAudioInputFile(audioState);
@@ -4,4 +4,7 @@ public static class GraphicsElementDefaults
{
// Built-in "On Now / Next" text element; identity is by filename, never by user-editable Name.
public const string OnNowNextFileName = "on-now-next.yml";
// Display name only. Never use it for identity -- that is the filename above (#67 / #74).
public const string OnNowNextName = "On Now / Next";
}
@@ -31,6 +31,28 @@ public class TextGraphicsElement : BaseGraphicsElement
[YamlMember(Alias = "z_index", ApplyNamingConventions = false)]
public int? ZIndex { get; set; }
// Background box (ersatztv#732). Element-level, not per-style: the graphics engine renders one
// TextBlock into one bitmap, so a single box behind the whole element is the only shape the
// renderer can express. Unset background_color means no FILL; a border_color alone still draws
// an outlined box. With neither there is no box and no insets -- the pre-#732 geometry.
[YamlMember(Alias = "background_color", ApplyNamingConventions = false)]
public string BackgroundColor { get; set; }
[YamlMember(Alias = "background_opacity_percent", ApplyNamingConventions = false)]
public int? BackgroundOpacityPercent { get; set; }
[YamlMember(Alias = "background_padding", ApplyNamingConventions = false)]
public double? BackgroundPadding { get; set; }
[YamlMember(Alias = "background_corner_radius", ApplyNamingConventions = false)]
public double? BackgroundCornerRadius { get; set; }
[YamlMember(Alias = "border_color", ApplyNamingConventions = false)]
public string BorderColor { get; set; }
[YamlMember(Alias = "border_width", ApplyNamingConventions = false)]
public double? BorderWidth { get; set; }
public List<StyleDefinition> Styles { get; set; } = [];
[YamlMember(Alias = "base_style", ApplyNamingConventions = false)]
@@ -85,19 +85,46 @@ public static class AlternateScheduleSelector
}
}
bool daysOfWeek = item.DaysOfWeek.Contains(date.DayOfWeek);
// These three are NULLABLE single-column primitive collections, and a runtime null IS
// reachable (ersatztv#823, measured against a real TvContext on SQLite and MySQL 8.4): EF does
// NOT invoke the value converter for a NULL column, so it materializes as CLR null rather than
// through IntCollectionValueConverter's null-to-empty branch, which never runs on this path.
// Unguarded, each .Contains below throws NullReferenceException.
//
// A NULL reads as UNRESTRICTED -- the All*() sets -- NOT as empty. This is the whole semantic
// question and it is decided by the one NULL that is reachable WITHOUT any code writing one:
// Sqlite's 20240113140741_Add_PlayoutTemplate_DaysOfMonth adds DaysOfMonth with
// `nullable: true` and NO defaultValue, so a PlayoutTemplate row inserted before it holds NULL
// and, by construction, had NO day-of-month restriction. Reading that as empty would INVERT
// the row's meaning and silently stop the template applying at all. All*() preserves it, and
// it is how "no restriction recorded" is already represented elsewhere in this domain
// (GetPlayoutAlternateSchedulesHandler, PreviewBlockPlayoutHandler). Note what does NOT decide
// it: the API request records normalize an omitted field with `?? []`, but that is a client
// omitting a field on a WRITE and says nothing about what a legacy database NULL meant.
//
// Guarded at the READ SITE, into locals, and NEVER assigned back onto `item`: the property IS
// the column value, so writing the guard back would flip a tracked entry to Modified and
// persist the substituted collection over the NULL
// (`media.nullable-primitive-collection-mutation`). The matching substitution happens at the
// entity->DTO boundary in the two Mapper.ProjectToViewModel overloads, so the SPA's
// appliesToDate -- an exact port of this method -- previews what this actually schedules.
ICollection<DayOfWeek> itemDaysOfWeek = item.DaysOfWeek ?? AllDaysOfWeek();
ICollection<int> itemDaysOfMonth = item.DaysOfMonth ?? AllDaysOfMonth();
ICollection<int> itemMonthsOfYear = item.MonthsOfYear ?? AllMonthsOfYear();
bool daysOfWeek = itemDaysOfWeek.Contains(date.DayOfWeek);
if (!daysOfWeek)
{
continue;
}
bool daysOfMonth = item.DaysOfMonth.Contains(date.Day);
bool daysOfMonth = itemDaysOfMonth.Contains(date.Day);
if (!daysOfMonth)
{
continue;
}
bool monthOfYear = item.MonthsOfYear.Contains(date.Month);
bool monthOfYear = itemMonthsOfYear.Contains(date.Month);
if (monthOfYear)
{
return item;
@@ -680,10 +680,56 @@ public class PipelineBuilderBaseTests
command.ShouldContain("-readrate 1.05 -readrate_initial_burst 8 -readrate_catchup 6.0 -i /tmp/whatever.mkv");
}
// ersatztv#735: the pacing values became operator-tunable profile fields. these pin that a
// configured value actually reaches the command line -- the defaults above are the OTHER half
// of the same guard, and they are what an unset profile still gets
[Test]
public void Realtime_Input_Should_Use_A_Configured_ReadRate_And_Catchup()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 1.5,
readRateCatchup: 4.0);
command.ShouldContain("-readrate 1.5 -readrate_initial_burst 8 -readrate_catchup 4.0 -i /tmp/whatever.mkv");
command.ShouldNotContain("-readrate 1.05");
command.ShouldNotContain("-readrate_catchup 6.0");
}
// the write path rejects an out-of-range value with a 422, so this only fires for a row written
// out of band -- but FFmpeg must never see the unbounded value either way
[Test]
public void Realtime_Input_Should_Clamp_An_Out_Of_Range_ReadRate()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 9.0,
readRateCatchup: 0.1);
// 9.0 clamps to the 2.0 ceiling, and 0.1 is raised to the resolved base rate, because a
// catchup below it could never let a lagging input recover
command.ShouldContain("-readrate 2.0 -readrate_initial_burst 8 -readrate_catchup 2.0 -i /tmp/whatever.mkv");
}
// ...and the catchup CEILING isolated from the base rate, which the case above cannot show:
// there both clamps land on the same 2.0, so either one alone would satisfy it
[Test]
public void Realtime_Input_Should_Clamp_An_Out_Of_Range_ReadRateCatchup()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 1.2,
readRateCatchup: 15.0);
command.ShouldContain("-readrate 1.2 -readrate_initial_burst 8 -readrate_catchup 10.0 -i /tmp/whatever.mkv");
}
private string BuildRealtimeCommand(
IFFmpegCapabilities capabilities,
bool stillImage = false,
bool imageSubtitle = false)
bool imageSubtitle = false,
Option<double> readRate = default,
Option<double> readRateCatchup = default)
{
var videoInputFile = new VideoInputFile(
"/tmp/whatever.mkv",
@@ -748,7 +794,9 @@ public class PipelineBuilderBaseTests
false,
false,
"clip",
false);
false,
MaybeReadRate: readRate,
MaybeReadRateCatchup: readRateCatchup);
// a *separate* audio input matters here: for a still image the video input takes no readrate
// at all, so only a distinct audio input can prove the burst was suppressed (this is the
+46 -1
View File
@@ -28,7 +28,9 @@ public record FFmpegState(
bool IsHdrTonemap,
string TonemapAlgorithm,
bool IsTroubleshooting,
bool QsvPreferNativeDecoder = false)
bool QsvPreferNativeDecoder = false,
Option<double> MaybeReadRate = default,
Option<double> MaybeReadRateCatchup = default)
{
// the QSV upload pool needs headroom for the frames in flight through the filter graph.
// extra_hw_frames=0 leaves none, so any input that is not throttled exhausts it: the graph
@@ -42,6 +44,49 @@ public record FFmpegState(
public int QsvExtraHardwareFrames =>
Math.Max(MaybeQsvExtraHardwareFrames.IfNone(MinimumQsvExtraHardwareFrames), MinimumQsvExtraHardwareFrames);
// realtime pacing. an unset profile keeps the values these constants name, which are the ones
// the pipeline hardcoded before they became configurable (ersatztv#735)
public const double DefaultReadRate = 1.05;
public const double DefaultStreamCopyReadRate = 1.0;
// how fast a LAGGING realtime input may read until it is level again. measured on the #726
// repro (embedded dvd_subtitle -> overlay, QSV encode): 1.05 alone sustains 0.53x, catchup 2.0
// reaches 0.711x, and 6.0 restores the full 1.067x that the same pipeline achieves with no
// subtitle at all. 20.0 also measures 1.067x — i.e. the value is not a throughput dial above
// the point where the input catches up, so 6.0 is chosen as the smallest measured-sufficient
// ceiling rather than the largest that works (ersatztv#726)
public const double DefaultReadRateCatchup = 6.0;
// below realtime the process reads slower than a live client consumes and the channel stalls;
// ersatztv#726 is that failure, measured at an effective 0.53x. the ceiling is a CHOSEN bound,
// not a measured cliff: it exists so the field cannot be used to effectively disable pacing,
// which is the configuration ersatztv#529 measured to produce zero segments on a QSV pipeline
public const double MinimumReadRate = 1.0;
public const double MaximumReadRate = 2.0;
// catchup is a ceiling that applies only WHILE an input is behind, so it is bounded more
// loosely than the base rate; the same chosen-not-measured caveat applies to the ceiling.
// the FLOOR is only a write-path bound: at render time the resolved base rate is always at
// least MinimumReadRate, so Math.Max below already dominates it
public const double MinimumReadRateCatchup = 1.0;
public const double MaximumReadRateCatchup = 10.0;
// clamped for the same reason QsvExtraHardwareFrames is: a row written out of band (or before
// the write path validated the field) must not reach FFmpeg unbounded. the write path rejects
// an out-of-range value with a 422 naming the bound, so this is belt-and-braces, not the
// primary guard (ersatztv#735)
public double ReadRateFor(bool isStreamCopy) =>
MaybeReadRate.Match(
configured => Math.Clamp(configured, MinimumReadRate, MaximumReadRate),
() => isStreamCopy ? DefaultStreamCopyReadRate : DefaultReadRate);
// a catchup rate below the base rate cannot let a lagging input recover, so the resolved base
// rate is its real floor — no separate lower clamp, which would be unreachable behind this Max
public double ReadRateCatchupFor(bool isStreamCopy) =>
Math.Max(
Math.Min(MaybeReadRateCatchup.IfNone(DefaultReadRateCatchup), MaximumReadRateCatchup),
ReadRateFor(isStreamCopy));
public static FFmpegState Concat(bool saveReport, string channelName) =>
new(
saveReport,
@@ -22,14 +22,6 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
// an operator who raises that setting above 2 gets less of the benefit (ersatztv#350)
private const int InitialBurstSeconds = OutputFormatHls.SegmentSeconds * 2;
// how fast a LAGGING realtime input may read until it is level again. measured on the #726
// repro (embedded dvd_subtitle -> overlay, QSV encode): 1.05 alone sustains 0.53x, catchup 2.0
// reaches 0.711x, and 6.0 restores the full 1.067x that the same pipeline achieves with no
// subtitle at all. 20.0 also measures 1.067x — i.e. the value is not a throughput dial above
// the point where the input catches up, so 6.0 is chosen as the smallest measured-sufficient
// ceiling rather than the largest that works (ersatztv#726)
private const double CatchupReadRate = 6.0;
private readonly Option<AudioInputFile> _audioInputFile;
private readonly Option<ConcatInputFile> _concatInputFile;
private readonly IFFmpegCapabilities _ffmpegCapabilities;
@@ -660,7 +652,7 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
}
//SetStillImageInfiniteLoop(videoInputFile, videoStream, ffmpegState);
SetRealtimeInput(videoInputFile, desiredState);
SetRealtimeInput(videoInputFile, ffmpegState, desiredState);
SetInfiniteLoop(videoInputFile, videoStream, ffmpegState, desiredState);
SetFrameRateOutput(desiredState, pipelineSteps);
SetVideoTrackTimescaleOutput(desiredState, pipelineSteps);
@@ -855,14 +847,17 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
}
}
private void SetRealtimeInput(VideoInputFile videoInputFile, FrameState desiredState)
private void SetRealtimeInput(VideoInputFile videoInputFile, FFmpegState ffmpegState, FrameState desiredState)
{
if (videoInputFile.StreamInputKind is StreamInputKind.Live || !desiredState.Realtime)
{
return;
}
double readRate = desiredState.VideoFormat == VideoFormat.Copy ? 1.0 : 1.05;
// both defaults and both bounds live on FFmpegState, beside the profile fields that
// override them, so the pacing contract is readable in one place (ersatztv#735)
bool isStreamCopy = desiredState.VideoFormat == VideoFormat.Copy;
double readRate = ffmpegState.ReadRateFor(isStreamCopy);
// without a burst, the readrate throttle applies from the very first read, so the first
// segment cannot be written faster than ~realtime and every start pays a multi-second wait.
@@ -894,7 +889,7 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
// subtitle always rides the video path, so this shape cannot suffer the starvation anyway
Option<double> catchupReadRate =
!isStillImage && _ffmpegCapabilities.HasOption(FFmpegKnownOption.ReadrateCatchup)
? CatchupReadRate
? ffmpegState.ReadRateCatchupFor(isStreamCopy)
: Option<double>.None;
_audioInputFile.Iter(a => a.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds, catchupReadRate)));
@@ -0,0 +1,38 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.MySql.Migrations
{
/// <inheritdoc />
public partial class Add_FFmpegProfile_ReadRatePacing : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<double>(
name: "ReadRate",
table: "FFmpegProfile",
type: "double",
nullable: true);
migrationBuilder.AddColumn<double>(
name: "ReadRateCatchup",
table: "FFmpegProfile",
type: "double",
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ReadRate",
table: "FFmpegProfile");
migrationBuilder.DropColumn(
name: "ReadRateCatchup",
table: "FFmpegProfile");
}
}
}
@@ -929,6 +929,12 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
.HasColumnType("tinyint(1)")
.HasDefaultValue(true);
b.Property<double?>("ReadRate")
.HasColumnType("double");
b.Property<double?>("ReadRateCatchup")
.HasColumnType("double");
b.Property<int>("ResolutionId")
.HasColumnType("int");
@@ -0,0 +1,38 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
{
/// <inheritdoc />
public partial class Add_FFmpegProfile_ReadRatePacing : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<double>(
name: "ReadRate",
table: "FFmpegProfile",
type: "REAL",
nullable: true);
migrationBuilder.AddColumn<double>(
name: "ReadRateCatchup",
table: "FFmpegProfile",
type: "REAL",
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ReadRate",
table: "FFmpegProfile");
migrationBuilder.DropColumn(
name: "ReadRateCatchup",
table: "FFmpegProfile");
}
}
}
@@ -896,6 +896,12 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
.HasColumnType("INTEGER")
.HasDefaultValue(true);
b.Property<double?>("ReadRate")
.HasColumnType("REAL");
b.Property<double?>("ReadRateCatchup")
.HasColumnType("REAL");
b.Property<int>("ResolutionId")
.HasColumnType("INTEGER");
@@ -1,4 +1,4 @@
using Dapper;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Repositories;
@@ -171,8 +171,15 @@ public class MusicVideoRepository : IMusicVideoRepository
public async Task<int> GetMusicVideoCount(int artistId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
// count the same population GetPagedMusicVideos pages — MusicVideoMetadata, not MusicVideo.
// A music video whose metadata row is missing (a scanner failure; FindOrphanPaths models
// exactly that state) is not pageable, so counting the item table over-reports
// (api.paged-count-matches-page-query, #832).
return await dbContext.Connection.QuerySingleAsync<int>(
@"SELECT COUNT(*) FROM MusicVideo WHERE ArtistId = @ArtistId",
@"SELECT COUNT(*)
FROM MusicVideoMetadata MVM
INNER JOIN MusicVideo M on MVM.MusicVideoId = M.Id
WHERE M.ArtistId = @ArtistId",
new { ArtistId = artistId });
}
@@ -1,4 +1,4 @@
using Dapper;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
@@ -134,9 +134,26 @@ public class TelevisionRepository : ITelevisionRepository
public async Task<int> GetSeasonCount(int showId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
return await dbContext.Seasons
.AsNoTracking()
.CountAsync(s => s.ShowId == showId);
// GetPagedSeasons expands the requested show to EVERY show sharing its Title+Year (the same
// show present in two libraries) and pages the union, so the count must expand identically
// or it under-reports (api.paged-count-matches-page-query, #832).
Option<ShowMetadata> maybeShowMetadata = await dbContext.ShowMetadata
.SelectOneAsync(sm => sm.Id, sm => sm.ShowId == showId, CancellationToken.None);
foreach (ShowMetadata showMetadata in maybeShowMetadata)
{
List<int> showIds = await dbContext.ShowMetadata
.Filter(sm => sm.Title == showMetadata.Title && sm.Year == showMetadata.Year)
.Map(sm => sm.ShowId)
.ToListAsync();
return await dbContext.Seasons
.AsNoTracking()
.CountAsync(s => showIds.Contains(s.ShowId));
}
// no metadata for the requested show: GetPagedSeasons returns nothing, so neither does this
return 0;
}
public async Task<List<Season>> GetPagedSeasons(
@@ -179,9 +196,12 @@ public class TelevisionRepository : ITelevisionRepository
public async Task<int> GetEpisodeCount(int seasonId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
return await dbContext.Episodes
// count the same population GetPagedEpisodes pages — EpisodeMetadata, not Episode. An
// episode whose metadata row is missing is not pageable, so counting the item table
// over-reports (api.paged-count-matches-page-query, #832).
return await dbContext.EpisodeMetadata
.AsNoTracking()
.CountAsync(e => e.SeasonId == seasonId);
.CountAsync(em => em.Episode.SeasonId == seasonId);
}
public async Task<List<EpisodeMetadata>> GetPagedEpisodes(int seasonId, int pageNumber, int pageSize)
@@ -4,12 +4,16 @@ using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Graphics;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Infrastructure.Streaming.Graphics;
public static class GraphicsElementSeeder
{
private const string OnNowNextYaml =
// The pre-#732 default, kept verbatim so an installation still carrying it byte-for-byte can
// be recognised as unmodified and upgraded. Never edit an entry here -- it is a fingerprint of
// what we shipped, not a template. Add a new entry when the current default changes again.
private const string OnNowNextYamlV1 =
"""
name: On Now / Next
epg_entries: 2
@@ -51,18 +55,88 @@ public static class GraphicsElementSeeder
{{ if (array.size Epg) > 1 }}[next]NEXT {{ Epg[1].Title }}[/next]{{ end }}
""";
public static async Task SeedOnNowNext(TvContext context, IFileSystem fileSystem, CancellationToken cancellationToken)
{
string seededKey = ConfigElementKey.GraphicsOnNowNextSeeded.Key;
bool alreadySeeded = await context.ConfigElements.AnyAsync(c => c.Key == seededKey, cancellationToken);
if (alreadySeeded)
{
return;
}
private const string OnNowNextYaml =
"""
name: On Now / Next
epg_entries: 2
location: BottomLeft
horizontal_margin_percent: 4
vertical_margin_percent: 8
width_percent: 42
text_fit: Wrap
text_align: Left
z_index: 100
# transparent until 4s in, fade in 1s, hold 6s, fade out 1s
opacity_expression: "LinearFadeDuration(content_seconds, 4, 1, 6)"
# #732: a translucent box carries legibility over both bright and dark content. The halo is
# cut from 2 to 1 rather than dropped -- the box is translucent, so bright content still
# shows through behind the glyphs, but 2px of halo ON TOP of a box over-darkens the text.
background_color: "#000000"
background_opacity_percent: 65
background_padding: 14
background_corner_radius: 8
# A translucent black box vanishes into dark content, so the box needs an edge of its own.
# Low-alpha white reads as a hairline on dark frames without becoming a hard line on bright ones.
border_color: "#59FFFFFF"
border_width: 1
base_style: now
styles:
- name: now
font_family: "Noto Sans"
font_size: 30
font_weight: 700
text_color: "#FFFFFF"
halo_color: "#000000"
halo_width: 1
- name: sub
font_family: "Noto Sans"
font_size: 22
font_weight: 400
text_color: "#DDDDDD"
halo_color: "#000000"
halo_width: 1
- name: next
font_family: "Noto Sans"
font_size: 22
font_weight: 400
text_color: "#DDDDDD"
halo_color: "#000000"
halo_width: 1
text: |
[now]NOW {{ Epg[0].Title }}[/now]
{{ if Epg[0].SubTitle }}[sub]{{ Epg[0].SubTitle }}[/sub]{{ end }}
{{ if (array.size Epg) > 1 }}[next]NEXT {{ Epg[1].Title }}[/next]{{ end }}
""";
// Every default we have ever shipped, most recent first. A file matching one of these was
// written by us and never touched, so replacing it is an upgrade rather than a clobber.
private static readonly string[] SupersededDefaults = [OnNowNextYamlV1];
public static async Task SeedOnNowNext(
TvContext context,
IFileSystem fileSystem,
ILogger logger,
CancellationToken cancellationToken)
{
string folder = FileSystemLayout.GraphicsElementsTextTemplatesFolder;
string target = fileSystem.Path.Combine(folder, GraphicsElementDefaults.OnNowNextFileName);
string seededKey = ConfigElementKey.GraphicsOnNowNextSeeded.Key;
bool alreadySeeded = await context.ConfigElements.AnyAsync(c => c.Key == seededKey, cancellationToken);
if (alreadySeeded)
{
// Already-seeded installations never revisit the file, so a change to the default would
// otherwise reach new databases only. Upgrade the ones still carrying an untouched
// earlier default; anything an operator edited no longer matches and is left alone.
//
// Deliberately no CreateDirectory on this branch: before #732 it touched the filesystem
// not at all, so an already-seeded install stays bootable on a read-only /config.
await UpgradeUnmodifiedTemplate(fileSystem, target, logger, cancellationToken);
await EnsureBuiltInElementRow(context, fileSystem, target, cancellationToken);
return;
}
if (!fileSystem.Directory.Exists(folder))
{
fileSystem.Directory.CreateDirectory(folder);
@@ -78,5 +152,218 @@ public static class GraphicsElementSeeder
new ConfigElement { Key = seededKey, Value = "true" },
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
await EnsureBuiltInElementRow(context, fileSystem, target, cancellationToken);
}
/// <summary>
/// `RefreshGraphicsElements` is what normally turns a template file into a `GraphicsElement` row,
/// but it runs on the scheduler/stream-start path -- long after startup. Creating the row here
/// removes that ordering dependency, so `AttachOnNowNextByDefault` below can never mark itself
/// done against an element that simply had not been discovered yet.
/// </summary>
private static async Task EnsureBuiltInElementRow(
TvContext context,
IFileSystem fileSystem,
string target,
CancellationToken cancellationToken)
{
if (!fileSystem.File.Exists(target))
{
return;
}
bool exists = await context.GraphicsElements.AnyAsync(e => e.Path == target, cancellationToken);
if (exists)
{
return;
}
// Name is display-only (identity is the filename), but leaving it null sorts the built-in
// element into the unnamed bucket at the bottom of the SPA list until the first refresh.
await context.GraphicsElements.AddAsync(
new Core.Domain.GraphicsElement
{
Path = target,
Kind = GraphicsElementKind.Text,
Name = GraphicsElementDefaults.OnNowNextName
},
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
}
/// <summary>
/// #732: the On Now / Next overlay is a default, not an opt-in. Existing channels predate that
/// decision, so attach the built-in element to them once.
/// </summary>
/// <remarks>
/// The marker is written only once the built-in element RESOLVES, so an install whose row does
/// not exist yet is retried on the next startup rather than stranded permanently. Once written,
/// no channel is ever re-attached. While still armed the backfill cannot tell a deliberately
/// cleared channel from an untouched one -- a single global flag cannot express both
/// properties; see <c>graphics.on-now-next-on-by-default</c> for why that trade is made this
/// way. Every channel created after the marker gets the element from
/// <c>ChannelGraphicsDefaults.Attach</c> instead, which BOTH create paths call.
/// </remarks>
public static async Task AttachOnNowNextByDefault(TvContext context, CancellationToken cancellationToken)
{
string key = ConfigElementKey.GraphicsOnNowNextDefaultAttached.Key;
if (await context.ConfigElements.AnyAsync(c => c.Key == key, cancellationToken))
{
return;
}
Option<int> maybeElementId = await GetBuiltInElementId(context, cancellationToken);
if (maybeElementId.IsNone)
{
// Nothing to attach TO. Writing the marker here would strand every channel permanently
// on the one population this exists for, so stay armed and try again next startup.
return;
}
foreach (int elementId in maybeElementId)
{
// HLS Direct has no frame pipeline to draw into, so an attachment there would be inert
// while still reading as "on" in the editor.
List<int> channelIds = await context.Channels
.Where(c => c.StreamingMode != StreamingMode.HttpLiveStreamingDirect)
.Where(c => c.ChannelGraphicsElements.All(cge => cge.GraphicsElementId != elementId))
.Select(c => c.Id)
.ToListAsync(cancellationToken);
foreach (int channelId in channelIds)
{
await context.AddAsync(
new ChannelGraphicsElement { ChannelId = channelId, GraphicsElementId = elementId },
cancellationToken);
}
}
await context.ConfigElements.AddAsync(
new ConfigElement { Key = key, Value = "true" },
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
}
/// <summary>
/// Identity is the filename, never the user-editable Name (the #67 lesson carried into #74).
/// The Kind is part of it: the five template folders are separate namespaces, so an unrelated
/// image/motion/subtitle/script element may legitimately be named `on-now-next.yml` too, and
/// filename alone would hand back whichever row the unordered query happened to return first.
/// </summary>
public static async Task<Option<int>> GetBuiltInElementId(
TvContext context,
CancellationToken cancellationToken)
{
List<(int Id, string Path)> candidates = await context.GraphicsElements
.Where(e => e.Kind == GraphicsElementKind.Text)
.Select(e => new { e.Id, e.Path })
.ToListAsync(cancellationToken)
.Map(rows => rows.Select(r => (r.Id, r.Path)).ToList());
var matches = candidates
.Where(c => System.IO.Path.GetFileName(c.Path) == GraphicsElementDefaults.OnNowNextFileName)
.OrderBy(c => c.Id)
.ToList();
// Lowest id wins if two text templates somehow share the filename, so the choice is stable
// across restarts rather than dependent on query order.
return matches.Count == 0 ? Option<int>.None : matches[0].Id;
}
private static async Task UpgradeUnmodifiedTemplate(
IFileSystem fileSystem,
string target,
ILogger logger,
CancellationToken cancellationToken)
{
// This runs inside the blocking database-startup path, ahead of DatabaseIsReady(). Before
// #732 the already-seeded branch never touched the filesystem at all, so an unreadable or
// read-only template is a state that used to boot fine -- it must not become a failure to
// start. Cosmetic upgrade, best effort.
try
{
if (!fileSystem.File.Exists(target))
{
return;
}
string existing = await fileSystem.File.ReadAllTextAsync(target, cancellationToken);
if (!SupersededDefaults.Any(d => IsSameTemplate(existing, d)))
{
return;
}
// Write-then-move, never write in place. WriteAllTextAsync truncates first, so an
// interrupted write (disk full, IO fault, cancellation) would leave a partial file that
// matches no fingerprint and is therefore never repaired on a later boot -- the overlay
// would just be gone, permanently, on every channel carrying it.
//
// The temp name is random per call. A fixed one is shared by two containers on the same
// config volume, where one can truncate it while the other is mid-write and then rename
// the partial file over the live template. The process id is NOT good enough here: the
// image uses an exec-form ENTRYPOINT, so every container's PID namespace makes this
// process 1 and every container computes the same name. A random name also means a temp
// left by a crashed earlier boot is never reused. Only ever delete the path this call
// created.
string temp = $"{target}.{fileSystem.Path.GetRandomFileName()}.upgrade.tmp";
try
{
await fileSystem.File.WriteAllTextAsync(temp, OnNowNextYaml, cancellationToken);
// Deliberately NO in-place fallback when this throws. rename(2) onto a mountpoint
// is EBUSY, so a single-file bind mount of this template will not be upgraded --
// accepted, because reaching that case needs a pinned file that is ALSO byte-identical
// to a shipped default, and the alternative is reintroducing the truncation this
// whole dance exists to prevent, on every IO fault rather than just that one.
fileSystem.File.Move(temp, target, true);
}
finally
{
// Cleanup must never REPLACE the exception that brought us here. Without this inner
// catch, a delete that throws while unwinding a cancellation swaps the
// OperationCanceledException for an IOException, which the outer filter then
// swallows -- so a real shutdown would be silently downgraded to a warning.
try
{
if (fileSystem.File.Exists(temp))
{
fileSystem.File.Delete(temp);
}
}
catch (Exception cleanupEx)
{
logger.LogDebug(cleanupEx, "Could not remove the temporary upgrade file {Path}", temp);
}
}
}
// Recoverable filesystem faults only. Catching everything would swallow genuinely fatal
// runtime failures (OutOfMemory and friends) and continue booting a compromised process;
// letting IO escape would turn a file permission into a restart loop. Cancellation
// propagates so shutdown is not swallowed.
// OperationCanceledException is deliberately absent from this list so a real shutdown
// propagates -- but only a real one: an OCE raised while the token is NOT cancelled is just
// another faulty read, and letting it escape is the restart loop this catch exists to stop.
catch (Exception ex) when ((ex is IOException
or UnauthorizedAccessException
or NotSupportedException
or System.Security.SecurityException)
|| (ex is OperationCanceledException
&& !cancellationToken.IsCancellationRequested))
{
logger.LogWarning(
ex,
"Could not upgrade the built-in graphics template at {Path}; leaving it as-is",
target);
}
}
// Compare on content, ignoring the line endings and trailing whitespace an editor or a volume
// mount may rewrite. This is a fingerprint check, not a parse: anything that is not one of our
// own shipped defaults must fall through untouched.
private static bool IsSameTemplate(string left, string right) =>
string.Equals(Normalize(left), Normalize(right), StringComparison.Ordinal);
private static string Normalize(string value) =>
value.Replace("\r\n", "\n", StringComparison.Ordinal).TrimEnd();
}
@@ -14,6 +14,10 @@ public partial class TextElement(
ILogger logger)
: GraphicsElement, IDisposable
{
// Far larger than any sane overlay on an 8K frame, and small enough that every downstream
// int cast stays well inside range.
private const float MaxBoxDimension = 10_000f;
private static readonly Regex StylePattern = StyleRegex();
private SKBitmap _image;
private SKPointI _location;
@@ -62,30 +66,96 @@ public partial class TextElement(
}
}
BackgroundBox box = BuildBackgroundBox();
RichTextKit.TextBlock textBlock = BuildTextBlock(textElement.Text);
if (textElement.WidthPercent.HasValue)
// Padding and border sit OUTSIDE the laid-out text on every side, so they shrink the
// space the text may occupy and grow the bitmap that holds it. Zero when there is no
// box, which reproduces the pre-#732 geometry exactly.
//
// Round ONCE, here, and use the same integer on both sides: the bitmap grows by
// 2 * insetPixels, so subtracting the unrounded inset from the wrap budget would let a
// fractional padding push the finished box a pixel past width_percent.
var insetPixels = (int)Math.Ceiling(box?.Inset ?? 0f);
// Bound the inset against the FRAME even when there is no width_percent. Sanitize caps
// each field individually, but padding and border add up, and without a budget nothing
// else clamps them -- a two-field fat-finger would otherwise allocate a bitmap far
// larger than the frame it is drawn onto. Pre-#732 no config value could inflate the
// bitmap independently of the measured text.
int frameInsetCap = Math.Max(0, Math.Min(context.FrameSize.Width, context.FrameSize.Height) / 2);
if (insetPixels > frameInsetCap)
{
logger.LogWarning(
"Background padding/border of {Inset}px exceeds the frame; clamping to {Clamped}px",
insetPixels,
frameInsetCap);
insetPixels = frameInsetCap;
box = box?.ClampedTo(frameInsetCap);
}
// A width_percent of 1e300 makes maxWidth Infinity, and every int cast below it is then
// unspecified. Treat a non-finite budget as "no budget", which is what an absent
// width_percent already means.
if (textElement.WidthPercent.HasValue
&& float.IsFinite((float)(textElement.WidthPercent.Value / 100.0 * context.FrameSize.Width)))
{
var maxWidth = (float)Math.Round(textElement.WidthPercent.Value / 100.0 * context.FrameSize.Width);
// A padding wider than the budget itself cannot be honoured AND stay inside it.
// Clamp the inset rather than squeezing the text to 1px: an unclamped floor turns a
// fat-fingered background_padding into a box several times the requested width.
int maxInset = Math.Max(0, (int)Math.Floor((maxWidth - 1) / 2));
if (insetPixels > maxInset)
{
logger.LogWarning(
"Background padding/border of {Inset}px does not fit within width_percent "
+ "({MaxWidth}px); clamping to {Clamped}px",
insetPixels,
maxWidth,
maxInset);
// Clamp the BOX, not just the bitmap's inset. Shrinking insetPixels alone leaves
// DrawBackgroundBox stroking at the original border width, which is centred on a
// rect that no longer has room for it -- the stroke then floods the element.
insetPixels = maxInset;
box = box?.ClampedTo(maxInset);
}
// width_percent bounds the ELEMENT, so the text gets what is left after the insets.
// With no box the budget is passed through untouched -- not through Math.Max -- so a
// width_percent that rounds to 0 keeps its exact pre-#732 behavior.
float textMaxWidth = insetPixels == 0
? maxWidth
: Math.Max(1f, maxWidth - (2 * insetPixels));
switch (textElement.Fit)
{
case TextFit.Wrap:
textBlock.MaxWidth = maxWidth;
textBlock.MaxWidth = textMaxWidth;
break;
case TextFit.Scale:
FitTextBlock(textBlock, maxWidth);
FitTextBlock(textBlock, textMaxWidth);
break;
}
}
_image = new SKBitmap(
(int)Math.Ceiling(textBlock.MeasuredWidth),
(int)Math.Ceiling(textBlock.MeasuredHeight));
(int)Math.Ceiling(textBlock.MeasuredWidth) + (2 * insetPixels),
(int)Math.Ceiling(textBlock.MeasuredHeight) + (2 * insetPixels));
using (var canvas = new SKCanvas(_image))
{
canvas.Clear(SKColors.Transparent);
textBlock.Paint(canvas, new SKPoint(0, 0));
if (box is not null)
{
DrawBackgroundBox(canvas, box, _image.Width, _image.Height);
}
textBlock.Paint(canvas, new SKPoint(insetPixels, insetPixels));
}
var horizontalMargin =
@@ -134,6 +204,158 @@ public partial class TextElement(
: new ValueTask<Option<PreparedElementImage>>(new PreparedElementImage(_image, _location, opacity, ZIndex, false));
}
// A background box is drawn only when a colour actually parses. An unparseable colour is
// warned about and skipped rather than substituted, so a typo never silently changes the
// look into something that appears deliberate.
private BackgroundBox BuildBackgroundBox()
{
SKColor? fill = ParseOptionalColor(textElement.BackgroundColor, "background_color");
if (fill.HasValue)
{
fill = ApplyOpacityPercent(fill.Value, textElement.BackgroundOpacityPercent);
}
SKColor? border = ParseOptionalColor(textElement.BorderColor, "border_color");
// A border colour with no explicit width means a hairline border, not an invisible one:
// "border_color set, nothing drawn" is the more confusing of the two readings.
float borderWidth = Sanitize(textElement.BorderWidth ?? 1, "border_width");
if (!border.HasValue)
{
borderWidth = 0;
}
if (!fill.HasValue && borderWidth <= 0)
{
return null;
}
return new BackgroundBox(
fill,
border,
borderWidth,
Sanitize(textElement.BackgroundCornerRadius, "background_corner_radius"),
Sanitize(textElement.BackgroundPadding, "background_padding"));
}
// YAML happily yields 1e100 or NaN. Cast to float those become Infinity/NaN, and
// (int)Math.Ceiling(Infinity) is an unspecified value -- in practice int.MinValue, which sails
// straight past every `> maxInset` clamp and can wrap 2 * inset back to zero. Sanitize at the
// boundary so no downstream arithmetic ever sees a non-finite value.
private float Sanitize(double? value, string fieldName)
{
if (value is not { } raw)
{
return 0f;
}
if (double.IsNaN(raw) || raw < 0)
{
logger.LogWarning("Ignoring out-of-range {Field} value {Value}", fieldName, raw);
return 0f;
}
if (raw > MaxBoxDimension)
{
logger.LogWarning(
"Clamping {Field} value {Value} to {Max}",
fieldName,
raw,
MaxBoxDimension);
return MaxBoxDimension;
}
return (float)raw;
}
private SKColor? ParseOptionalColor(string value, string fieldName)
{
if (string.IsNullOrWhiteSpace(value))
{
return null;
}
if (SKColor.TryParse(value, out SKColor parsed))
{
return parsed;
}
logger.LogWarning(
"Unable to parse {Field} value {Value}; that part of the background box will not be drawn",
fieldName,
value);
return null;
}
private static SKColor ApplyOpacityPercent(SKColor color, int? opacityPercent)
{
if (opacityPercent is not { } percent)
{
return color;
}
int clamped = Math.Clamp(percent, 0, 100);
return color.WithAlpha((byte)Math.Round(color.Alpha * clamped / 100.0));
}
private static void DrawBackgroundBox(SKCanvas canvas, BackgroundBox box, int width, int height)
{
// Skia strokes centred on the path, so half the border would fall outside the bitmap and
// be clipped. Inset the rect by half the width to keep the whole border visible.
float half = box.BorderWidth / 2f;
var rect = new SKRect(half, half, width - half, height - half);
// A radius larger than half the shorter side is not expressible as a rounded rect.
float radius = Math.Min(box.CornerRadius, Math.Min(rect.Width, rect.Height) / 2f);
radius = Math.Max(0, radius);
if (box.Fill is { } fill)
{
using var fillPaint = new SKPaint
{
Color = fill,
Style = SKPaintStyle.Fill,
IsAntialias = true
};
canvas.DrawRoundRect(rect, radius, radius, fillPaint);
}
if (box.Border is { } border && box.BorderWidth > 0)
{
using var borderPaint = new SKPaint
{
Color = border,
Style = SKPaintStyle.Stroke,
StrokeWidth = box.BorderWidth,
IsAntialias = true
};
canvas.DrawRoundRect(rect, radius, radius, borderPaint);
}
}
private sealed record BackgroundBox(
SKColor? Fill,
SKColor? Border,
float BorderWidth,
float CornerRadius,
float Padding)
{
public float Inset => Padding + BorderWidth;
// Border first, then whatever is left goes to padding: a border that cannot be drawn inside
// the bitmap is worse than a thin one, and padding degrades gracefully to zero.
public BackgroundBox ClampedTo(float maxInset)
{
float borderWidth = Math.Min(BorderWidth, maxInset);
float padding = Math.Max(0, maxInset - borderWidth);
return this with { BorderWidth = borderWidth, Padding = padding };
}
}
private RichTextKit.TextBlock BuildTextBlock(string textToRender)
{
var textBlock = new RichTextKit.TextBlock
@@ -211,6 +433,17 @@ public partial class TextElement(
finalStyle.TextColor = parsedColor;
}
// Halo is per-style in the schema and was being dropped here, so a non-base style's
// halo_* silently inherited the base style's. The seeded template only looked correct
// because all three of its styles declare the same halo.
finalStyle.HaloWidth = s.HaloWidth ?? finalStyle.HaloWidth;
finalStyle.HaloBlur = s.HaloBlur ?? finalStyle.HaloBlur;
if (s.HaloColor != null && SKColor.TryParse(s.HaloColor, out SKColor parsedHalo))
{
finalStyle.HaloColor = parsedHalo;
}
styles[s.Name] = finalStyle;
}
@@ -286,6 +519,11 @@ public partial class TextElement(
foreach ((string text, RichTextKit.IStyle style) in originalContent)
{
// Carry across every property the YAML schema can set, not just the ones the scale
// needs (the rest are RichTextKit defaults we never touch). Halo and
// line height were being dropped here, which only mattered once #732 gave the box
// insets that can push a previously-fitting element into the Scale path: adding a
// background would then silently remove the halo it sits behind.
var newStyle = new RichTextKit.Style
{
FontFamily = style.FontFamily,
@@ -294,7 +532,11 @@ public partial class TextElement(
FontWidth = style.FontWidth,
FontWeight = style.FontWeight,
LetterSpacing = style.LetterSpacing,
TextColor = style.TextColor
LineHeight = style.LineHeight,
TextColor = style.TextColor,
HaloColor = style.HaloColor,
HaloWidth = style.HaloWidth,
HaloBlur = style.HaloBlur
};
float newSize = newStyle.FontSize * scale;
@@ -0,0 +1,92 @@
using ErsatzTV.Application.Channels;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Graphics;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
using Microsoft.EntityFrameworkCore;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Application.Channels;
/// <summary>
/// #732: the On Now / Next overlay is a default rather than an opt-in, so a channel created after
/// that decision gets the built-in element without the operator toggling anything.
/// </summary>
[TestFixture]
public class CreateChannelDefaultGraphicsElementTests : ChannelHandlerTestBase
{
private CreateChannelHandler MakeHandler() => new(Worker, Db.Factory, SearchTargets, RemoteLogoCacher);
private async Task<int> SeedBuiltInElement()
{
await using TvContext context = Db.CreateContext();
var element = new GraphicsElement
{
Path = $"/templates/text/{GraphicsElementDefaults.OnNowNextFileName}",
Kind = GraphicsElementKind.Text
};
context.GraphicsElements.Add(element);
await context.SaveChangesAsync();
return element.Id;
}
private async Task<List<int>> AttachedElementIds(int channelId)
{
await using TvContext context = Db.CreateContext();
Channel reloaded = await context.Channels
.Include(c => c.ChannelGraphicsElements)
.SingleAsync(c => c.Id == channelId);
return reloaded.ChannelGraphicsElements.Select(x => x.GraphicsElementId).ToList();
}
[Test]
public async Task Attaches_The_Built_In_Element_To_A_New_Channel()
{
await SeedFFmpegProfile();
int elementId = await SeedBuiltInElement();
Either<BaseError, CreateChannelResult> result =
await MakeHandler().Handle(MakeCreate(), CancellationToken.None);
result.IsRight.ShouldBeTrue();
int channelId = result.RightToSeq().Head().ChannelId;
(await AttachedElementIds(channelId)).ShouldBe([elementId]);
}
[Test]
public async Task Leaves_An_Hls_Direct_Channel_Alone_Because_Nothing_Can_Render_There()
{
await SeedFFmpegProfile();
await SeedBuiltInElement();
Either<BaseError, CreateChannelResult> result = await MakeHandler().Handle(
MakeCreate(streamingMode: StreamingMode.HttpLiveStreamingDirect),
CancellationToken.None);
result.IsRight.ShouldBeTrue();
int channelId = result.RightToSeq().Head().ChannelId;
(await AttachedElementIds(channelId)).ShouldBeEmpty();
}
[Test]
public async Task Creates_The_Channel_Even_When_The_Built_In_Element_Does_Not_Exist()
{
await SeedFFmpegProfile();
Either<BaseError, CreateChannelResult> result =
await MakeHandler().Handle(MakeCreate(), CancellationToken.None);
result.IsRight.ShouldBeTrue();
int channelId = result.RightToSeq().Head().ChannelId;
(await AttachedElementIds(channelId)).ShouldBeEmpty();
}
}
@@ -9,9 +9,11 @@ using ErsatzTV.Core.Api.LibraryBrowse;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Graphics;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Core.Scheduling;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
@@ -964,4 +966,66 @@ public class CreateChannelFromLineupHandlerTests
private static TR RightOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => throw new AssertionException($"Expected a Right result, got {e.Value}"), Right: r => r);
// #732: this is the SPA's primary "Add Channel" flow and the one Auto-Tune bulk-creates through.
// It was the channel-creation site the default attach originally missed, so a channel made here
// would silently never get the overlay once the one-time backfill marker had landed.
[Test]
public async Task Should_Attach_The_Built_In_On_Now_Next_Element()
{
await SeedTemplateDependencies();
await SeedTemplate();
await SeedMovie(42);
int elementId = await SeedBuiltInGraphicsElement();
Either<BaseError, CreateChannelFromLineupResponseModel> result =
await MakeHandler().Handle(MakeRequest(), CancellationToken.None);
CreateChannelFromLineupResponseModel response = RightOf(result);
await using TvContext context = _db.CreateContext();
DomainChannel channel = await context.Channels
.Include(c => c.ChannelGraphicsElements)
.SingleAsync(c => c.Id == response.ChannelId);
channel.ChannelGraphicsElements.Select(x => x.GraphicsElementId).ShouldBe([elementId]);
}
[Test]
public async Task Should_Not_Attach_The_Overlay_To_An_Hls_Direct_Channel()
{
await SeedTemplateDependencies();
await SeedTemplate();
await SeedMovie(42);
await SeedBuiltInGraphicsElement();
Either<BaseError, CreateChannelFromLineupResponseModel> result = await MakeHandler().Handle(
MakeRequest(advanced: new CreateChannelFromLineupAdvancedOptions(
PlaybackOrder.Shuffle,
StreamingMode: StreamingMode.HttpLiveStreamingDirect)),
CancellationToken.None);
CreateChannelFromLineupResponseModel response = RightOf(result);
await using TvContext context = _db.CreateContext();
DomainChannel channel = await context.Channels
.Include(c => c.ChannelGraphicsElements)
.SingleAsync(c => c.Id == response.ChannelId);
channel.ChannelGraphicsElements.ShouldBeEmpty();
}
private async Task<int> SeedBuiltInGraphicsElement()
{
await using TvContext context = _db.CreateContext();
var element = new GraphicsElement
{
Path = $"/templates/text/{GraphicsElementDefaults.OnNowNextFileName}",
Kind = GraphicsElementKind.Text
};
context.GraphicsElements.Add(element);
await context.SaveChangesAsync();
return element.Id;
}
}
@@ -5,6 +5,9 @@ using ErsatzTV.Core.Errors;
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.FFmpeg;
using ErsatzTV.FFmpeg.OutputFormat;
using Microsoft.EntityFrameworkCore;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
@@ -134,15 +137,30 @@ public class FFmpegProfileHandlerTests
persisted.QsvPreferNativeDecoder.ShouldBe(false);
}
// ersatztv#529: a stored 0 reached ffmpeg as hwupload=extra_hw_frames=0, leaving the QSV pool no
// headroom; FFmpegState floors it at render time, and these pin that the stored row converges too
// so the profile never keeps displaying a value the pipeline would override.
[TestCase(0, 64)]
[TestCase(-8, 64)]
[TestCase(63, 64)]
[TestCase(64, 64)]
[TestCase(128, 128)]
public async Task Create_Should_Floor_QsvExtraHardwareFrames(int configured, int expected)
// ersatztv#735: the write path used to accept an out-of-range pool size and store the floored
// value instead, so a client that PUT 0 got a 200 and read back 64. it is now rejected, naming
// the bound; FFmpegState still floors at render time for rows that predate this.
[TestCase(0)]
[TestCase(-8)]
[TestCase(63)]
public async Task Create_Should_Reject_QsvExtraHardwareFrames_Below_Minimum(int configured)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, qsvExtraHardwareFrames: configured),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("at least 64");
await using TvContext context = _db.CreateContext();
(await context.FFmpegProfiles.CountAsync()).ShouldBe(0);
}
[TestCase(64)]
[TestCase(128)]
public async Task Create_Should_Store_QsvExtraHardwareFrames_Exactly_As_Submitted(int configured)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
@@ -155,15 +173,15 @@ public class FFmpegProfileHandlerTests
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.QsvExtraHardwareFrames.ShouldBe(expected);
persisted.QsvExtraHardwareFrames.ShouldBe(configured);
}
[TestCase(0, 64)]
[TestCase(-8, 64)]
[TestCase(128, 128)]
public async Task Update_Should_Floor_QsvExtraHardwareFrames(int configured, int expected)
[TestCase(0)]
[TestCase(-8)]
[TestCase(63)]
public async Task Update_Should_Reject_A_Newly_Submitted_QsvExtraHardwareFrames_Below_Minimum(int configured)
{
await SeedProfile(1);
await SeedProfile(1, qsvExtraHardwareFrames: 128);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
@@ -171,11 +189,59 @@ public class FFmpegProfileHandlerTests
MakeUpdate(1, qsvExtraHardwareFrames: configured),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("at least 64");
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.QsvExtraHardwareFrames.ShouldBe(128);
}
// the other half of the same rule: the SPA sends the whole profile back on every edit, so a row
// stored before this validation existed must stay editable over fields the operator did touch.
// an UNCHANGED out-of-range value is written back as-is and floored at render time instead
[Test]
public async Task Update_Should_Accept_An_Unchanged_Legacy_QsvExtraHardwareFrames()
{
await SeedProfile(1, qsvExtraHardwareFrames: 0);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, UpdateFFmpegProfileResult> result = await handler.Handle(
MakeUpdate(1, qsvExtraHardwareFrames: 0),
CancellationToken.None);
RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.QsvExtraHardwareFrames.ShouldBe(expected);
persisted.QsvExtraHardwareFrames.ShouldBe(0);
new FFmpegState(
false,
HardwareAccelerationMode.None,
HardwareAccelerationMode.None,
None,
None,
None,
None,
false,
None,
None,
None,
None,
None,
OutputFormatKind.MpegTs,
None,
None,
None,
None,
TimeSpan.Zero,
None,
Optional(persisted.QsvExtraHardwareFrames),
false,
false,
"linear",
false)
.QsvExtraHardwareFrames.ShouldBe(FFmpegState.MinimumQsvExtraHardwareFrames);
}
// null means "unconfigured" and FFmpegState already resolves it to the same 64; it must stay
@@ -196,6 +262,133 @@ public class FFmpegProfileHandlerTests
persisted.QsvExtraHardwareFrames.ShouldBeNull();
}
// ersatztv#735: readrate pacing is an operator-tunable bounded field. out of band it is a dead
// channel either way — below realtime the client starves, above the ceiling the input is no
// longer meaningfully paced (which is the unthrottled read #529 measured to write no segments)
[TestCase(0.9)]
[TestCase(0.0)]
[TestCase(2.5)]
public async Task Create_Should_Reject_ReadRate_Outside_Bounds(double configured)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, readRate: configured),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("Read rate must be between 1.0 and 2.0");
}
[TestCase(0.9)]
[TestCase(10.5)]
public async Task Create_Should_Reject_ReadRateCatchup_Outside_Bounds(double configured)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, readRateCatchup: configured),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("Read rate catchup must be between 1.0 and 10.0");
}
// a catchup rate inside its own band can still be at or below the base rate, where it cannot
// let a lagging input recover — the cross-field bound is the one a per-field check cannot see.
// the EQUAL cases matter: zero headroom is functionally no catchup, while still reading as a
// configured one
[TestCase(null, 1.0)]
[TestCase(null, 1.05)]
[TestCase(1.5, 1.2)]
[TestCase(1.5, 1.5)]
[TestCase(2.0, 2.0)]
public async Task Create_Should_Reject_ReadRateCatchup_At_Or_Below_The_ReadRate(double? readRate, double catchup)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, readRate: readRate, readRateCatchup: catchup),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("must be greater than the read rate");
}
[Test]
public async Task Create_Should_Persist_ReadRate_Pacing()
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, readRate: 1.2, readRateCatchup: 4.0),
CancellationToken.None);
CreateFFmpegProfileResult created = RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.ReadRate.ShouldBe(1.2);
persisted.ReadRateCatchup.ShouldBe(4.0);
}
// unset is the default posture and must stay null: FFmpegState resolves null to the values the
// pipeline used before the fields existed, so an untouched profile paces exactly as it did
[Test]
public async Task Create_Should_Leave_Unset_ReadRate_Pacing_Null()
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result =
await handler.Handle(MakeCreate(1), CancellationToken.None);
CreateFFmpegProfileResult created = RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.ReadRate.ShouldBeNull();
persisted.ReadRateCatchup.ShouldBeNull();
}
[Test]
public async Task Update_Should_Reject_ReadRate_Outside_Bounds()
{
await SeedProfile(1);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, UpdateFFmpegProfileResult> result = await handler.Handle(
MakeUpdate(1, readRate: 3.0),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("Read rate must be between 1.0 and 2.0");
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.ReadRate.ShouldBeNull();
}
[Test]
public async Task Update_Should_Persist_ReadRate_Pacing()
{
await SeedProfile(1);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, UpdateFFmpegProfileResult> result = await handler.Handle(
MakeUpdate(1, readRate: 1.5, readRateCatchup: 8.0),
CancellationToken.None);
RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.ReadRate.ShouldBe(1.5);
persisted.ReadRateCatchup.ShouldBe(8.0);
}
private static TR RightOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => throw new AssertionException($"Expected a Right result, got {e}"), Right: r => r);
@@ -209,12 +402,13 @@ public class FFmpegProfileHandlerTests
await context.SaveChangesAsync();
}
private async Task SeedProfile(int id)
private async Task SeedProfile(int id, int? qsvExtraHardwareFrames = null)
{
await using TvContext context = _db.CreateContext();
context.FFmpegProfiles.Add(new FFmpegProfile
{
Id = id,
QsvExtraHardwareFrames = qsvExtraHardwareFrames,
Name = "Default",
ThreadCount = 1,
NormalizeAudio = true,
@@ -249,7 +443,9 @@ public class FFmpegProfileHandlerTests
private static CreateFFmpegProfile MakeCreate(
int resolutionId,
bool qsvPreferNativeDecoder = true,
int? qsvExtraHardwareFrames = null) =>
int? qsvExtraHardwareFrames = null,
double? readRate = null,
double? readRateCatchup = null) =>
new(
"Default",
1,
@@ -281,13 +477,17 @@ public class FFmpegProfileHandlerTests
false,
false,
false,
qsvPreferNativeDecoder);
qsvPreferNativeDecoder,
readRate,
readRateCatchup);
private static UpdateFFmpegProfile MakeUpdate(
int id,
int resolutionId = 1,
bool qsvPreferNativeDecoder = true,
int? qsvExtraHardwareFrames = null) =>
int? qsvExtraHardwareFrames = null,
double? readRate = null,
double? readRateCatchup = null) =>
new(
id,
"Default",
@@ -320,5 +520,7 @@ public class FFmpegProfileHandlerTests
false,
false,
false,
qsvPreferNativeDecoder);
qsvPreferNativeDecoder,
readRate,
readRateCatchup);
}
@@ -0,0 +1,166 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Data.Repositories;
using ErsatzTV.Tests.Support;
using Microsoft.Extensions.Logging.Abstractions;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Application.Paging;
/// <summary>
/// The MediaCards count/page pairs live in two separate repository methods rather than in one
/// handler, so `api.paged-count-matches-page-query` cannot be satisfied structurally there — the
/// two must be kept in agreement and pinned by a test instead. Each case below constructs the
/// divergence the count used to miss and asserts count == pageable rows.
/// Expected values are pinned literals, never re-derived from the method's own predicate.
/// </summary>
[TestFixture]
public class MediaCardsCountMatchesPageTests
{
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private TelevisionRepository TelevisionRepo =>
new(_db.Factory, NullLogger<TelevisionRepository>.Instance);
[Test]
public async Task GetSeasonCount_Should_Expand_To_The_Same_Shows_GetPagedSeasons_Pages()
{
// the same show present in two libraries: same Title+Year, different Show rows.
// GetPagedSeasons pages the union (2 + 3), so the count must be 5, not 2.
await using (TvContext context = _db.CreateContext())
{
context.Shows.Add(new Show { Id = 1 });
context.Shows.Add(new Show { Id = 2 });
context.ShowMetadata.Add(new ShowMetadata { Id = 201, ShowId = 1, Title = "Star Trek", Year = 1966 });
context.ShowMetadata.Add(new ShowMetadata { Id = 202, ShowId = 2, Title = "Star Trek", Year = 1966 });
// an unrelated show that must NOT be swept in
context.Shows.Add(new Show { Id = 3 });
context.ShowMetadata.Add(new ShowMetadata { Id = 203, ShowId = 3, Title = "Star Trek", Year = 1987 });
context.Seasons.Add(new Season { Id = 19, ShowId = 3, SeasonNumber = 1 });
context.Seasons.Add(new Season { Id = 11, ShowId = 1, SeasonNumber = 1 });
context.Seasons.Add(new Season { Id = 12, ShowId = 1, SeasonNumber = 2 });
context.Seasons.Add(new Season { Id = 13, ShowId = 2, SeasonNumber = 1 });
context.Seasons.Add(new Season { Id = 14, ShowId = 2, SeasonNumber = 2 });
context.Seasons.Add(new Season { Id = 15, ShowId = 2, SeasonNumber = 3 });
await context.SaveChangesAsync();
}
int count = await TelevisionRepo.GetSeasonCount(1);
List<Season> page = await TelevisionRepo.GetPagedSeasons(1, 1, 50, CancellationToken.None);
count.ShouldBe(5);
page.Count.ShouldBe(5);
count.ShouldBe(page.Count);
// pin WHICH rows, not just how many — a count and a page can agree on the wrong set
page.Select(s => s.Id).OrderBy(id => id).ShouldBe([11, 12, 13, 14, 15]);
}
[Test]
public async Task GetSeasonCount_Should_Be_Zero_When_The_Show_Has_No_Metadata()
{
// GetPagedSeasons returns nothing without a ShowMetadata row to expand from, so the count
// must agree rather than reporting the show's seasons
await using (TvContext context = _db.CreateContext())
{
context.Shows.Add(new Show { Id = 1 });
context.Seasons.Add(new Season { Id = 11, ShowId = 1, SeasonNumber = 1 });
await context.SaveChangesAsync();
}
int count = await TelevisionRepo.GetSeasonCount(1);
List<Season> page = await TelevisionRepo.GetPagedSeasons(1, 1, 50, CancellationToken.None);
count.ShouldBe(0);
page.ShouldBeEmpty();
}
[Test]
public async Task GetEpisodeCount_Should_Count_Episodes_That_Have_Metadata()
{
// 3 episodes, one of which lost its metadata row to a scanner failure. GetPagedEpisodes
// pages EpisodeMetadata, so only 2 are reachable and the count must say 2.
await using (TvContext context = _db.CreateContext())
{
// GetPagedEpisodes's include chain reaches Episode -> Season -> Show through REQUIRED
// reference navs, which EF emits as INNER JOINs, so a missing Season or Show row drops
// every row and would make the page 0 for a reason unrelated to the count under test.
// The ShowMetadata leg is a COLLECTION nav (LEFT JOIN) and drops nothing — the row below
// is incidental, seeded only to keep the graph realistic.
context.Shows.Add(new Show { Id = 1 });
context.ShowMetadata.Add(new ShowMetadata { Id = 201, ShowId = 1, Title = "Show", Year = 2000 });
context.Seasons.Add(new Season { Id = 11, ShowId = 1, SeasonNumber = 1 });
for (var i = 21; i <= 23; i++)
{
context.Episodes.Add(new Episode { Id = i, SeasonId = 11 });
}
context.EpisodeMetadata.Add(new EpisodeMetadata { Id = 221, EpisodeId = 21, EpisodeNumber = 1 });
context.EpisodeMetadata.Add(new EpisodeMetadata { Id = 222, EpisodeId = 22, EpisodeNumber = 2 });
// an episode in a different season must not be swept in
context.Seasons.Add(new Season { Id = 12, ShowId = 1, SeasonNumber = 2 });
context.Episodes.Add(new Episode { Id = 29, SeasonId = 12 });
context.EpisodeMetadata.Add(new EpisodeMetadata { Id = 229, EpisodeId = 29, EpisodeNumber = 1 });
await context.SaveChangesAsync();
}
int count = await TelevisionRepo.GetEpisodeCount(11);
List<EpisodeMetadata> page = await TelevisionRepo.GetPagedEpisodes(11, 1, 50);
count.ShouldBe(2);
page.Count.ShouldBe(2);
count.ShouldBe(page.Count);
// the two episodes WITH metadata, and not the other season's
page.Select(em => em.EpisodeId).OrderBy(id => id).ShouldBe([21, 22]);
}
[Test]
public async Task GetMusicVideoCount_Should_Count_Music_Videos_That_Have_Metadata()
{
// 3 music videos for the artist, one without a metadata row; GetPagedMusicVideos pages
// MusicVideoMetadata, so the count must be 2
await using (TvContext context = _db.CreateContext())
{
context.Artists.Add(new Artist { Id = 41 });
for (var i = 31; i <= 33; i++)
{
context.MusicVideos.Add(new MusicVideo { Id = i, ArtistId = 41 });
}
context.MusicVideoMetadata.Add(new MusicVideoMetadata { Id = 231, MusicVideoId = 31, Title = "A" });
context.MusicVideoMetadata.Add(new MusicVideoMetadata { Id = 232, MusicVideoId = 32, Title = "B" });
// another artist's video must not be swept in
context.Artists.Add(new Artist { Id = 42 });
context.MusicVideos.Add(new MusicVideo { Id = 39, ArtistId = 42 });
context.MusicVideoMetadata.Add(new MusicVideoMetadata { Id = 239, MusicVideoId = 39, Title = "C" });
await context.SaveChangesAsync();
}
var repo = new MusicVideoRepository(_db.Factory);
int count = await repo.GetMusicVideoCount(41);
List<MusicVideoMetadata> page = await repo.GetPagedMusicVideos(41, 1, 50);
count.ShouldBe(2);
page.Count.ShouldBe(2);
count.ShouldBe(page.Count);
// this artist's two videos with metadata, and not the other artist's
page.Select(m => m.Title).OrderBy(x => x).ShouldBe(["A", "B"]);
}
}
@@ -0,0 +1,286 @@
using ErsatzTV.Application.MediaCollections;
using ErsatzTV.Application.Playouts;
using ErsatzTV.Application.ProgramSchedules;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using NUnit.Framework;
using Shouldly;
using DomainChannel = ErsatzTV.Core.Domain.Channel;
namespace ErsatzTV.Tests.Application.Paging;
/// <summary>
/// Every paged handler whose page query applies a filter must compute its TotalCount from the SAME
/// query, or a filtered page reports the unfiltered total and the SPA paginates to pages that can
/// never contain anything (issues #690, #758).
/// Expected counts here are PINNED LITERALS derived from the seeded set by hand — never recomputed
/// by re-applying the handler's own predicate, which would pass whatever the handler happens to do.
/// </summary>
[TestFixture]
public class PagedQueryTotalCountTests
{
// 5 seeded rows, of which exactly these 2 contain "Alpha"
private const int SeededRows = 5;
private const int MatchingAlpha = 2;
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private static readonly string[] Names =
["Alpha One", "Beta", "Alpha Two", "Gamma", "Delta"];
[Test]
public async Task GetPagedCollections_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.Collections.Add(new Collection { Id = i + 1, Name = Names[i], MediaItems = [] });
}
await context.SaveChangesAsync();
}
var handler = new GetPagedCollectionsHandler(_db.Factory);
PagedMediaCollectionsViewModel unfiltered =
await handler.Handle(new GetPagedCollections(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedMediaCollectionsViewModel filtered =
await handler.Handle(new GetPagedCollections("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(c => c.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedMultiCollections_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.MultiCollections.Add(new MultiCollection { Id = i + 1, Name = Names[i] });
}
// channel-owned rows are excluded from BOTH the page and the count, filter or no filter
context.MultiCollections.Add(
new MultiCollection { Id = 99, Name = "Alpha Owned", OwnedByChannelId = 7 });
await context.SaveChangesAsync();
}
var handler = new GetPagedMultiCollectionsHandler(_db.Factory);
PagedMultiCollectionsViewModel unfiltered =
await handler.Handle(new GetPagedMultiCollections(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedMultiCollectionsViewModel filtered =
await handler.Handle(new GetPagedMultiCollections("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(mc => mc.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedSmartCollections_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.SmartCollections.Add(
new SmartCollection { Id = i + 1, Name = Names[i], Query = "tag:family" });
}
context.SmartCollections.Add(
new SmartCollection
{
Id = 99,
Name = "Alpha Owned",
Query = "tag:family",
OwnedByChannelId = 7
});
await context.SaveChangesAsync();
}
var handler = new GetPagedSmartCollectionsHandler(_db.Factory);
PagedSmartCollectionsViewModel unfiltered =
await handler.Handle(new GetPagedSmartCollections(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedSmartCollectionsViewModel filtered =
await handler.Handle(new GetPagedSmartCollections("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(sc => sc.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedRerunCollections_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.RerunCollections.Add(
new RerunCollection
{
Id = i + 1,
Name = Names[i],
CollectionType = CollectionType.Collection,
CollectionId = i + 1
});
context.Collections.Add(new Collection { Id = i + 1, Name = Names[i], MediaItems = [] });
}
await context.SaveChangesAsync();
}
var handler = new GetPagedRerunCollectionsHandler(_db.Factory);
PagedRerunCollectionsViewModel unfiltered =
await handler.Handle(new GetPagedRerunCollections(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedRerunCollectionsViewModel filtered =
await handler.Handle(new GetPagedRerunCollections("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(rc => rc.Name).ShouldBe(["Alpha One", "Alpha Two"]);
// the selection graph still loads for the page — moving IncludeSelectionDetails off the
// counted query must not stop the page from projecting it (issue #671)
filtered.Page.Select(rc => rc.Collection?.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedProgramSchedules_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.ProgramSchedules.Add(new ProgramSchedule { Id = i + 1, Name = Names[i] });
}
await context.SaveChangesAsync();
}
var handler = new GetPagedProgramSchedulesHandler(_db.Factory);
PagedProgramSchedulesViewModel unfiltered =
await handler.Handle(new GetPagedProgramSchedules(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedProgramSchedulesViewModel filtered =
await handler.Handle(new GetPagedProgramSchedules("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(ps => ps.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedPlayouts_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.Channels.Add(NewChannel(i + 1, $"{i + 1}", Names[i]));
context.Playouts.Add(
new Playout
{
Id = i + 1,
ChannelId = i + 1,
ScheduleKind = PlayoutScheduleKind.Classic
});
}
// a playout whose channel row does not exist: excluded from the page by the
// `Channel != null` filter, so it must be excluded from the count too
context.Playouts.Add(
new Playout { Id = 99, ChannelId = 4242, ScheduleKind = PlayoutScheduleKind.Classic });
await context.SaveChangesAsync();
}
var handler = new GetPagedPlayoutsHandler(_db.Factory);
PagedPlayoutsViewModel unfiltered =
await handler.Handle(new GetPagedPlayouts(string.Empty, 0, 10), CancellationToken.None);
// 5, NOT 6 — the orphaned playout is filtered out of the page, so it is not part of the total
unfiltered.TotalCount.ShouldBe(SeededRows);
unfiltered.Page.Count.ShouldBe(SeededRows);
PagedPlayoutsViewModel filtered =
await handler.Handle(new GetPagedPlayouts("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(p => p.ChannelName).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task Filtered_Count_Should_Drive_A_Second_Page()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.Collections.Add(new Collection { Id = i + 1, Name = Names[i], MediaItems = [] });
}
await context.SaveChangesAsync();
}
var handler = new GetPagedCollectionsHandler(_db.Factory);
// pageSize 1 over the 2 matching rows: this is the property the issues are about — the count
// is what tells the client a SECOND page exists, and each page holds exactly its own row
PagedMediaCollectionsViewModel first =
await handler.Handle(new GetPagedCollections("Alpha", 0, 1), CancellationToken.None);
first.TotalCount.ShouldBe(MatchingAlpha);
first.Page.Select(c => c.Name).ShouldBe(["Alpha One"]);
PagedMediaCollectionsViewModel second =
await handler.Handle(new GetPagedCollections("Alpha", 1, 1), CancellationToken.None);
second.TotalCount.ShouldBe(MatchingAlpha);
second.Page.Select(c => c.Name).ShouldBe(["Alpha Two"]);
// and the page AFTER the last matching row is empty — with the pre-fix count of 5 the client
// would have been told to fetch three more pages that can never contain anything
PagedMediaCollectionsViewModel past =
await handler.Handle(new GetPagedCollections("Alpha", 2, 1), CancellationToken.None);
past.TotalCount.ShouldBe(MatchingAlpha);
past.Page.ShouldBeEmpty();
}
private static DomainChannel NewChannel(int id, string number, string name) =>
new(Guid.NewGuid())
{
Id = id,
Number = number,
SortNumber = id,
Name = name,
Group = "ErsatzTV",
Categories = string.Empty,
FFmpegProfileId = 1,
StreamSelector = string.Empty,
PreferredAudioLanguageCode = string.Empty,
PreferredAudioTitle = string.Empty,
PreferredSubtitleLanguageCode = string.Empty,
MusicVideoCreditsTemplate = string.Empty,
StreamingMode = StreamingMode.TransportStreamHybrid,
PlayoutSource = ChannelPlayoutSource.Generated,
PlayoutMode = ChannelPlayoutMode.Continuous
};
}
@@ -0,0 +1,151 @@
using ErsatzTV.Application.Playouts;
using ErsatzTV.Application.Scheduling;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
using NUnit.Framework;
using Shouldly;
using PlayoutsMapper = ErsatzTV.Application.Playouts.Mapper;
using SchedulingMapper = ErsatzTV.Application.Scheduling.Mapper;
namespace ErsatzTV.Tests.Application.Scheduling;
/// <summary>
/// ersatztv#823. Guarding <see cref="AlternateScheduleSelector" /> alone would have left the OTHER read
/// of the same six columns unguarded — the entity→view-model mappers, which feed
/// <c>PlayoutController</c>'s response models and therefore the SPA.
/// <para>
/// Two things break without the guard, and neither is a C# exception, which is why the selector
/// tests cannot see them. <c>web/src/screens/PlayoutScheduleEditors.tsx</c> spreads the collection
/// (<c>daysOfMonth: [...template.daysOfMonth]</c>) and throws <c>TypeError: not iterable</c> on a
/// JSON <c>null</c>; and <c>web/src/screens/playoutTemplateCalendar.ts</c>'s <c>appliesToDate</c> —
/// an exact TypeScript port of <see cref="AlternateScheduleSelector.GetScheduleForDate{T}" /> —
/// calls <c>.includes</c> on it.
/// </para>
/// <para>
/// So the mappers substitute the SAME unrestricted defaults the selector reads. That agreement is
/// the point: a DTO that said "empty" while the selector scheduled "unrestricted" would make the
/// preview calendar disagree with the playout it is previewing.
/// </para>
/// </summary>
[TestFixture]
public class RecurrenceLimitsMapperNullTests
{
private static Template MinimalTemplate() =>
new()
{
Id = 7,
Name = "T",
TemplateGroupId = 1,
TemplateGroup = new TemplateGroup { Name = "G" },
Items = []
};
[Test]
public void ProgramScheduleAlternate_Null_Collections_Map_To_Unrestricted()
{
var alternate = new ProgramScheduleAlternate
{
Id = 1,
Index = 0,
ProgramScheduleId = 2,
DaysOfWeek = null!,
DaysOfMonth = null!,
MonthsOfYear = null!
};
PlayoutAlternateScheduleViewModel vm = PlayoutsMapper.ProjectToViewModel(alternate);
vm.DaysOfWeek.ShouldBe(AlternateScheduleSelector.AllDaysOfWeek());
vm.DaysOfMonth.ShouldBe(AlternateScheduleSelector.AllDaysOfMonth());
vm.MonthsOfYear.ShouldBe(AlternateScheduleSelector.AllMonthsOfYear());
// Never assigned back: these are single-column primitive collections, so writing the guard onto a
// tracked entity would persist the substituted set over the NULL
// (media.nullable-primitive-collection-mutation).
alternate.DaysOfWeek.ShouldBeNull();
alternate.DaysOfMonth.ShouldBeNull();
alternate.MonthsOfYear.ShouldBeNull();
}
[Test]
public void PlayoutTemplate_Null_Collections_Map_To_Unrestricted()
{
var template = new PlayoutTemplate
{
Id = 1,
Index = 0,
Template = MinimalTemplate(),
DecoTemplate = null,
DaysOfWeek = null!,
DaysOfMonth = null!,
MonthsOfYear = null!
};
PlayoutTemplateViewModel vm = SchedulingMapper.ProjectToViewModel(template);
vm.DaysOfWeek.ShouldBe(AlternateScheduleSelector.AllDaysOfWeek());
vm.DaysOfMonth.ShouldBe(AlternateScheduleSelector.AllDaysOfMonth());
vm.MonthsOfYear.ShouldBe(AlternateScheduleSelector.AllMonthsOfYear());
template.DaysOfWeek.ShouldBeNull();
template.DaysOfMonth.ShouldBeNull();
template.MonthsOfYear.ShouldBeNull();
}
/// <summary>
/// An explicitly EMPTY collection is a recorded restriction of no days and must survive the mapper
/// unchanged. Without this, a guard written as "empty or null becomes All*" would pass the two tests
/// above while silently rewriting real user data on the way out.
/// <para>
/// There is one of these per MAPPER, not one in total. The two overloads are byte-identical
/// triples in different files, so a defensive edit to one alone is exactly the "one helper, two
/// callers" shape this repo has been bitten by: covering only the Playouts mapper would leave
/// the PlayoutTemplate one free to acquire an `empty-or-null` guard with the suite still green.
/// </para>
/// </summary>
[Test]
public void An_Explicitly_Empty_Collection_Is_Not_Rewritten_By_The_PlayoutTemplate_Mapper()
{
var template = new PlayoutTemplate
{
Id = 1,
Index = 0,
Template = MinimalTemplate(),
DecoTemplate = null,
DaysOfWeek = [],
DaysOfMonth = [],
MonthsOfYear = []
};
PlayoutTemplateViewModel vm = SchedulingMapper.ProjectToViewModel(template);
vm.DaysOfWeek.ShouldBeEmpty();
vm.DaysOfMonth.ShouldBeEmpty();
vm.MonthsOfYear.ShouldBeEmpty();
}
/// <summary>
/// The <c>ProgramScheduleAlternate</c> half of the same pair — see the PlayoutTemplate one above
/// for why there is one per MAPPER rather than one in total.
/// </summary>
[Test]
public void An_Explicitly_Empty_Collection_Is_Not_Rewritten()
{
var alternate = new ProgramScheduleAlternate
{
Id = 1,
Index = 0,
ProgramScheduleId = 2,
DaysOfWeek = [],
DaysOfMonth = [],
MonthsOfYear = []
};
PlayoutAlternateScheduleViewModel vm = PlayoutsMapper.ProjectToViewModel(alternate);
vm.DaysOfWeek.ShouldBeEmpty();
vm.DaysOfMonth.ShouldBeEmpty();
vm.MonthsOfYear.ShouldBeEmpty();
}
}
@@ -217,7 +217,9 @@ public class FFmpegProfileControllerTests
false,
true,
false,
true);
true,
null,
null);
private static CreateFFmpegProfileRequest MakeCreateRequest() =>
new(
@@ -0,0 +1,674 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Graphics;
using ErsatzTV.Core.Interfaces.Streaming;
using ErsatzTV.FFmpeg;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using NUnit.Framework;
using Shouldly;
using LanguageExt;
using SkiaSharp;
namespace ErsatzTV.Tests.Infrastructure.Graphics;
/// <summary>
/// Pixel-level cover for the #732 background box. These assert the RENDERED BITMAP rather than the
/// parsed model, because every failure mode this feature has is a silent no-op: the YAML parses, the
/// element initializes, and nothing is drawn. Geometry assertions are all RELATIVE to a no-box
/// baseline so they do not depend on which typeface the host resolves.
/// </summary>
[TestFixture]
public class TextElementBackgroundBoxTests
{
private const int FrameWidth = 1920;
// Derived from the documented rule, not a hardcoded 1080p result: hardcoding it makes correct
// production code fail the moment the test frame size changes.
private static int FrameInsetCap => Math.Min(FrameWidth, FrameHeight) / 2;
private const int FrameHeight = 1080;
private static TextGraphicsElement BaseElement() =>
new()
{
Name = "test",
Location = WatermarkLocation.BottomLeft,
BaseStyle = "body",
Styles =
[
new StyleDefinition
{
Name = "body",
// Required: a null font_family makes CustomFontMapper throw on a null dictionary
// key, which TextElement swallows into "disable for this content" (the #570 trap).
// The family need not resolve -- an unknown one falls back to Skia's default.
FontFamily = "Roboto",
FontSize = 40,
TextColor = "#FFFFFF"
}
],
Text = "Hello"
};
// TextElement swallows every initialization failure into a logged warning, so a broken render
// would otherwise surface as a null bitmap with no explanation. Capture the warning and rethrow.
private sealed class ThrowingLogger : ILogger
{
public IDisposable BeginScope<TState>(TState state) where TState : notnull => null!;
public bool IsEnabled(LogLevel logLevel) => true;
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception exception,
Func<TState, Exception, string> formatter)
{
if (logLevel >= LogLevel.Warning && exception is not null)
{
throw new InvalidOperationException(formatter(state, exception), exception);
}
}
}
private static SKBitmap Render(TextGraphicsElement element)
{
var fonts = new GraphicsEngineFonts(new CustomFontMapper(NullLogger<CustomFontMapper>.Instance));
var textElement = new TextElement(fonts, element, new ThrowingLogger());
var context = new GraphicsEngineContext(
"1",
null,
[],
new Dictionary<string, object>(),
new Resolution { Width = FrameWidth, Height = FrameHeight },
new Resolution { Width = FrameWidth, Height = FrameHeight },
new FrameRate("30"),
DateTimeOffset.UnixEpoch,
DateTimeOffset.UnixEpoch,
TimeSpan.Zero,
TimeSpan.FromMinutes(1),
TimeSpan.FromMinutes(1));
textElement.InitializeAsync(context, CancellationToken.None).GetAwaiter().GetResult();
Option<PreparedElementImage> maybeImage = textElement
.PrepareImage(TimeSpan.Zero, TimeSpan.Zero, TimeSpan.FromMinutes(1), TimeSpan.Zero, CancellationToken.None)
.AsTask().GetAwaiter().GetResult();
PreparedElementImage prepared = maybeImage.IfNone(() => throw new InvalidOperationException(
"the element produced no image; initialization failed"));
return prepared.Image;
}
// The whole suite is meaningless if the host cannot lay out any text at all, so prove the
// baseline is non-degenerate rather than letting a 0x0 bitmap pass every relative assertion.
[Test]
public void Baseline_Renders_A_Non_Empty_Bitmap()
{
SKBitmap baseline = Render(BaseElement());
baseline.Width.ShouldBeGreaterThan(0);
baseline.Height.ShouldBeGreaterThan(0);
}
[Test]
public void No_Background_Fields_Leaves_Corner_Transparent()
{
SKBitmap baseline = Render(BaseElement());
baseline.GetPixel(0, 0).Alpha.ShouldBe((byte)0);
baseline.GetPixel(baseline.Width - 1, baseline.Height - 1).Alpha.ShouldBe((byte)0);
}
[Test]
public void Background_Color_Fills_The_Box()
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
SKBitmap rendered = Render(element);
SKColor corner = rendered.GetPixel(0, 0);
corner.Alpha.ShouldBe((byte)255);
corner.Red.ShouldBe((byte)255);
corner.Green.ShouldBe((byte)0);
corner.Blue.ShouldBe((byte)0);
}
[Test]
public void Background_Padding_Grows_The_Bitmap_On_Every_Side()
{
const int Padding = 12;
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#000000";
element.BackgroundPadding = Padding;
SKBitmap padded = Render(element);
padded.Width.ShouldBe(baseline.Width + (2 * Padding));
padded.Height.ShouldBe(baseline.Height + (2 * Padding));
}
// Growing the bitmap is not the same as moving the text into it. If the text were still painted
// at (0,0) the geometry assertions above would all still hold while the glyphs sat on the box
// edge, so pin the padding band itself as pure background.
[Test]
public void Background_Padding_Actually_Insets_The_Text()
{
const int Padding = 16;
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = Padding;
SKBitmap rendered = Render(element);
// A full-width band inside the top padding must contain nothing but the fill colour.
for (var x = 0; x < rendered.Width; x++)
{
SKColor pixel = rendered.GetPixel(x, Padding / 2);
pixel.Red.ShouldBe((byte)255, $"pixel at x={x} in the padding band is not the fill colour");
pixel.Green.ShouldBe((byte)0, $"pixel at x={x} in the padding band is not the fill colour");
pixel.Blue.ShouldBe((byte)0, $"pixel at x={x} in the padding band is not the fill colour");
}
}
[Test]
public void Background_Opacity_Percent_Scales_The_Alpha()
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundOpacityPercent = 50;
SKBitmap rendered = Render(element);
// 255 * 0.5, rounded. Skia stores premultiplied alpha, so assert the alpha channel only.
rendered.GetPixel(0, 0).Alpha.ShouldBe((byte)128);
}
[Test]
public void Corner_Radius_Rounds_The_Corner_Away()
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = 20;
element.BackgroundCornerRadius = 20;
SKBitmap rendered = Render(element);
// The extreme corner falls outside a 20px radius, the middle of the left edge does not.
rendered.GetPixel(0, 0).Alpha.ShouldBe((byte)0);
rendered.GetPixel(0, rendered.Height / 2).Alpha.ShouldBe((byte)255);
}
[Test]
public void Border_Color_Draws_A_Border_Distinct_From_The_Fill()
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = 20;
element.BorderColor = "#00FF00";
element.BorderWidth = 4;
SKBitmap rendered = Render(element);
// On the border ring...
SKColor edge = rendered.GetPixel(1, rendered.Height / 2);
edge.Green.ShouldBeGreaterThan((byte)200);
edge.Red.ShouldBeLessThan((byte)100);
// ...and inside it, still the fill.
SKColor inside = rendered.GetPixel(10, rendered.Height / 2);
inside.Red.ShouldBe((byte)255);
inside.Green.ShouldBe((byte)0);
}
[Test]
public void Border_Width_Is_Included_In_The_Bitmap_So_The_Border_Is_Not_Clipped()
{
const int BorderWidth = 6;
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BorderColor = "#00FF00";
element.BorderWidth = BorderWidth;
SKBitmap bordered = Render(element);
bordered.Width.ShouldBe(baseline.Width + (2 * BorderWidth));
bordered.Height.ShouldBe(baseline.Height + (2 * BorderWidth));
}
[Test]
public void Unparseable_Background_Color_Draws_No_Box_Rather_Than_Substituting_One()
{
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "not-a-color";
element.BackgroundPadding = 25;
SKBitmap rendered = Render(element);
// No box means no padding either: the geometry is the untouched baseline.
rendered.Width.ShouldBe(baseline.Width);
rendered.Height.ShouldBe(baseline.Height);
rendered.GetPixel(0, 0).Alpha.ShouldBe((byte)0);
}
[Test]
public void Border_Color_Without_An_Explicit_Width_Draws_A_Hairline()
{
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BorderColor = "#00FF00";
SKBitmap rendered = Render(element);
rendered.Width.ShouldBe(baseline.Width + 2);
rendered.GetPixel(0, rendered.Height / 2).Green.ShouldBeGreaterThan((byte)200);
}
// A box with no padding or border must not move anything: this is the seam where "adding a
// background" could silently change an existing overlay's geometry.
[Test]
public void A_Fill_With_No_Padding_Or_Border_Leaves_The_Geometry_Untouched()
{
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
SKBitmap filled = Render(element);
filled.Width.ShouldBe(baseline.Width);
filled.Height.ShouldBe(baseline.Height);
}
// The inset is rounded up to a whole pixel and the SAME integer is used on both sides. Subtracting
// the unrounded value from the wrap budget while adding its ceiling to the bitmap overflows
// width_percent by a rounding remainder, which an integer padding cannot expose.
[Test]
public void Fractional_Padding_Still_Respects_Width_Percent()
{
const double WidthPercent = 20;
foreach (double padding in new[] { 12.3, 14.0, 29.7, 0.5 })
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog and keeps running past the edge";
element.Fit = TextFit.Wrap;
element.WidthPercent = WidthPercent;
element.BackgroundColor = "#000000";
element.BackgroundPadding = padding;
SKBitmap rendered = Render(element);
var budget = (int)Math.Round(WidthPercent / 100.0 * FrameWidth);
rendered.Width.ShouldBeLessThanOrEqualTo(budget, $"padding {padding} overflowed the budget");
}
}
// An inset wider than the budget cannot be honoured AND stay inside it, so the INSET is clamped
// rather than the text being squeezed to nothing. Asserting a width bound here would be wrong:
// wrapping cannot break below one glyph, so a narrow width_percent overflows with or without a
// box (pre-existing). What the clamp guarantees is that the box's own contribution stops growing
// at the budget -- so a runaway padding renders identically to the largest one that fits.
[Test]
public void An_Oversized_Padding_Is_Clamped_To_The_Largest_That_Fits()
{
const double WidthPercent = 20;
var budget = (int)Math.Round(WidthPercent / 100.0 * FrameWidth);
int maxInset = (budget - 1) / 2;
static TextGraphicsElement WithPadding(double padding)
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Wrap;
element.WidthPercent = WidthPercent;
element.BackgroundColor = "#000000";
element.BackgroundPadding = padding;
return element;
}
SKBitmap runaway = Render(WithPadding(400));
SKBitmap clamped = Render(WithPadding(maxInset));
runaway.Width.ShouldBe(clamped.Width);
runaway.Height.ShouldBe(clamped.Height);
// and the clamp actually bit -- an unclamped 400px padding would add 800px of box
runaway.Width.ShouldBeLessThan(budget + (2 * maxInset));
}
// FitTextBlock rebuilds every style from scratch. It used to drop the halo, which only became
// reachable once a box's insets could push a previously-fitting element into the Scale path --
// adding a background would then silently remove the halo behind the text.
[Test]
public void The_Scale_Path_Preserves_The_Halo()
{
static TextGraphicsElement Scaled(bool withHalo)
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Scale;
element.WidthPercent = 15;
element.BackgroundColor = "#000000";
element.BackgroundPadding = 12;
element.Styles[0].HaloColor = withHalo ? "#00FF00" : null;
element.Styles[0].HaloWidth = withHalo ? 3 : null;
return element;
}
SKBitmap withHalo = Render(Scaled(true));
SKBitmap withoutHalo = Render(Scaled(false));
withHalo.Width.ShouldBe(withoutHalo.Width);
withHalo.Height.ShouldBe(withoutHalo.Height);
var differing = 0;
for (var x = 0; x < withHalo.Width; x++)
{
for (var y = 0; y < withHalo.Height; y++)
{
if (withHalo.GetPixel(x, y) != withoutHalo.GetPixel(x, y))
{
differing++;
}
}
}
// If the Scale path dropped the halo, the two renders would be pixel-identical.
differing.ShouldBeGreaterThan(0, "the halo made no difference through the Scale path");
}
// width_percent bounds the ELEMENT. If the insets were not subtracted from the wrap width the
// box would overflow the budget by 2*inset, which is exactly the bug this pins.
[Test]
public void Width_Percent_Bounds_The_Whole_Box_Including_Padding()
{
const double WidthPercent = 20;
const int Padding = 30;
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog and keeps on running well past the edge";
element.Fit = TextFit.Wrap;
element.WidthPercent = WidthPercent;
element.BackgroundColor = "#000000";
element.BackgroundPadding = Padding;
SKBitmap rendered = Render(element);
var budget = (int)Math.Round(WidthPercent / 100.0 * FrameWidth);
rendered.Width.ShouldBeLessThanOrEqualTo(budget);
}
// YAML yields doubles, so 1e100 and NaN are reachable from a config file. Cast to float they
// become Infinity/NaN, and (int)Math.Ceiling of those is an unspecified value that sails past
// every clamp and can wrap the doubled inset back to zero.
[Test]
public void Non_Finite_And_Absurd_Box_Values_Do_Not_Corrupt_The_Geometry()
{
SKBitmap baseline = Render(BaseElement());
foreach (double bad in new[] { 1e100, double.NaN, double.PositiveInfinity, -5.0 })
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = bad;
SKBitmap rendered = Render(element);
rendered.Width.ShouldBeGreaterThan(0, $"padding {bad} produced a degenerate width");
rendered.Height.ShouldBeGreaterThan(0, $"padding {bad} produced a degenerate height");
rendered.Width.ShouldBeGreaterThanOrEqualTo(baseline.Width, $"padding {bad} shrank the element");
rendered.Width.ShouldBeLessThanOrEqualTo(baseline.Width + (2 * FrameInsetCap), $"padding {bad} was not clamped");
}
}
[Test]
public void A_Non_Finite_Width_Percent_Is_Treated_As_No_Budget()
{
SKBitmap baseline = Render(BaseElement());
// This pins the OUTCOME -- a non-finite budget behaves like no budget -- not the
// float.IsFinite guard, which no mutation can distinguish because .NET saturates float-to-int
// conversion. See the "Not covered by any mutation" note in docs/graphics-elements.md.
const int Padding = 10;
TextGraphicsElement element = BaseElement();
element.Fit = TextFit.Wrap;
element.WidthPercent = 1e300;
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = Padding;
SKBitmap rendered = Render(element);
rendered.Width.ShouldBe(baseline.Width + (2 * Padding));
rendered.Height.ShouldBe(baseline.Height + (2 * Padding));
}
// Clamping insetPixels alone leaves DrawBackgroundBox stroking at the ORIGINAL border width,
// centred on a rect that no longer has room for it -- the stroke then floods the whole element
// and paints over the interior. The box's own fields must be clamped too.
[Test]
public void An_Oversized_Border_Is_Clamped_And_Does_Not_Flood_The_Element()
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Wrap;
element.WidthPercent = 20;
element.BackgroundColor = "#0000FF";
element.BorderColor = "#FF0000";
element.BorderWidth = 400;
SKBitmap rendered = Render(element);
// Somewhere inside the element there must still be fill, not solid border.
var fillPixels = 0;
for (var x = 0; x < rendered.Width; x++)
{
for (var y = 0; y < rendered.Height; y++)
{
SKColor px = rendered.GetPixel(x, y);
if (px.Blue > 200 && px.Red < 100)
{
fillPixels++;
}
}
}
fillPixels.ShouldBeGreaterThan(0, "the border flooded the element; no fill survived");
}
// Sanitize bounds each field on its own, but padding and border ADD UP, and with no
// width_percent nothing else bounded them -- two fields could allocate a bitmap far larger than
// the frame it is drawn onto. This pins the frame cap itself; the non-finite test above cannot,
// because its bound is looser than the growth this clause prevents.
[Test]
public void The_Inset_Is_Capped_Against_The_Frame_Even_With_No_Width_Percent()
{
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = 4000;
element.BorderColor = "#00FF00";
element.BorderWidth = 4000;
SKBitmap rendered = Render(element);
// 8000px of requested inset collapses to the frame cap, not to 16000px of bitmap growth.
rendered.Width.ShouldBe(baseline.Width + (2 * FrameInsetCap));
rendered.Height.ShouldBe(baseline.Height + (2 * FrameInsetCap));
}
// halo_* is documented as a per-style field. The merge loop used to rebuild each style from the
// base and then override only font/colour, so a non-base style's halo silently inherited the
// base one -- invisible in the seeded template, whose three styles declare identical halos.
[Test]
public void A_Non_Base_Style_Uses_Its_Own_Halo_Not_The_Base_Styles()
{
static TextGraphicsElement WithSubHalo(float haloWidth)
{
TextGraphicsElement element = BaseElement();
element.Styles.Add(new StyleDefinition
{
Name = "sub",
FontFamily = "Roboto",
FontSize = 40,
TextColor = "#FFFFFF",
HaloColor = "#00FF00",
HaloWidth = haloWidth
});
element.Text = "[sub]Hello[/sub]";
return element;
}
SKBitmap thin = Render(WithSubHalo(1));
SKBitmap thick = Render(WithSubHalo(6));
static int HaloPixels(SKBitmap b)
{
var n = 0;
for (var x = 0; x < b.Width; x++)
{
for (var y = 0; y < b.Height; y++)
{
SKColor px = b.GetPixel(x, y);
if (px.Green > 150 && px.Red < 120)
{
n++;
}
}
}
return n;
}
// If the style's own halo were ignored, both renders would use the base style's (none) and
// neither would contain halo pixels.
HaloPixels(thin).ShouldBeGreaterThan(0, "the style's own halo was not applied");
HaloPixels(thick).ShouldBeGreaterThan(HaloPixels(thin), "halo_width had no effect per style");
}
private static int PixelsMatching(SKBitmap b, Func<SKColor, bool> predicate)
{
var n = 0;
for (var x = 0; x < b.Width; x++)
{
for (var y = 0; y < b.Height; y++)
{
if (predicate(b.GetPixel(x, y)))
{
n++;
}
}
}
return n;
}
// halo_blur is a per-style field too, and the merge loop drops it just as silently as halo_color
// did. Blur spreads the halo over more pixels at lower alpha, so a blurred halo covers more area.
[Test]
public void A_Non_Base_Style_Uses_Its_Own_Halo_Blur()
{
static TextGraphicsElement WithSubBlur(float blur)
{
TextGraphicsElement element = BaseElement();
element.Styles.Add(new StyleDefinition
{
Name = "sub",
FontFamily = "Roboto",
FontSize = 40,
TextColor = "#FFFFFF",
HaloColor = "#00FF00",
HaloWidth = 3,
HaloBlur = blur
});
element.Text = "[sub]Hello[/sub]";
return element;
}
SKBitmap sharp = Render(WithSubBlur(0));
SKBitmap blurred = Render(WithSubBlur(5));
static bool Greenish(SKColor px) => px.Green > 60 && px.Red < 140 && px.Alpha > 0;
PixelsMatching(blurred, Greenish)
.ShouldNotBe(PixelsMatching(sharp, Greenish), "halo_blur had no effect per style");
}
// FitTextBlock rebuilds every style from scratch on the Scale path. line_height and halo_blur are
// carried across there; without them a scaled element silently loses line spacing and halo blur.
[Test]
public void The_Scale_Path_Preserves_Line_Height()
{
static TextGraphicsElement Scaled(float lineHeight)
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Scale;
element.WidthPercent = 15;
element.Styles[0].LineHeight = lineHeight;
return element;
}
SKBitmap tight = Render(Scaled(1.0f));
SKBitmap loose = Render(Scaled(2.5f));
loose.Height.ShouldBeGreaterThan(tight.Height, "line_height was dropped by the Scale path");
}
[Test]
public void The_Scale_Path_Preserves_Halo_Blur()
{
static TextGraphicsElement Scaled(float blur)
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Scale;
element.WidthPercent = 15;
element.Styles[0].HaloColor = "#00FF00";
element.Styles[0].HaloWidth = 3;
element.Styles[0].HaloBlur = blur;
return element;
}
SKBitmap sharp = Render(Scaled(0));
SKBitmap blurred = Render(Scaled(5));
static bool Greenish(SKColor px) => px.Green > 60 && px.Red < 140 && px.Alpha > 0;
PixelsMatching(blurred, Greenish)
.ShouldNotBe(PixelsMatching(sharp, Greenish), "halo_blur was dropped by the Scale path");
}
// The documented range is 0-100. The sole opacity test used 50, so Math.Clamp could be removed
// with everything green while an out-of-range value wrapped to an unrelated alpha.
[Test]
public void Background_Opacity_Percent_Is_Clamped_To_Its_Documented_Range()
{
static SKColor CornerAt(int percent)
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundOpacityPercent = percent;
return Render(element).GetPixel(0, 0);
}
CornerAt(-50).Alpha.ShouldBe((byte)0, "a negative opacity did not clamp to 0");
CornerAt(400).Alpha.ShouldBe((byte)255, "an opacity over 100 did not clamp to 100");
}
}
@@ -0,0 +1,288 @@
using ErsatzTV.Core.Domain;
using GraphicsElement = ErsatzTV.Core.Domain.GraphicsElement;
using ErsatzTV.Core.Graphics;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using System.IO.Abstractions;
using Microsoft.Extensions.Logging.Abstractions;
using Testably.Abstractions.Testing;
using ErsatzTV.Core;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Infrastructure;
/// <summary>
/// #732 part 2: the On Now / Next overlay is a default rather than an opt-in. Channels that predate
/// that decision are backfilled once -- and only once, so a channel an operator deliberately clears
/// is never silently re-attached on the next restart.
/// </summary>
[TestFixture]
public class GraphicsElementDefaultAttachTests
{
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private static async Task<int> SeedBuiltInElement(TvContext context)
{
var element = new GraphicsElement
{
Path = $"/templates/text/{GraphicsElementDefaults.OnNowNextFileName}",
Name = "On Now / Next",
Kind = GraphicsElementKind.Text
};
await context.GraphicsElements.AddAsync(element);
await context.SaveChangesAsync();
return element.Id;
}
private static async Task<Channel> SeedChannel(
TvContext context,
string number,
StreamingMode mode = StreamingMode.HttpLiveStreamingSegmenter)
{
var channel = new Channel(Guid.NewGuid())
{
Name = $"Channel {number}",
Number = number,
StreamingMode = mode,
ChannelGraphicsElements = []
};
await context.Channels.AddAsync(channel);
await context.SaveChangesAsync();
return channel;
}
private static async Task<List<int>> AttachedElementIds(TvContext context, int channelId) =>
await context.Set<ChannelGraphicsElement>()
.AsNoTracking()
.Where(cge => cge.ChannelId == channelId)
.Select(cge => cge.GraphicsElementId)
.ToListAsync();
[Test]
public async Task Attaches_The_Built_In_Element_To_Existing_Channels()
{
await using TvContext context = _db.CreateContext();
int elementId = await SeedBuiltInElement(context);
Channel one = await SeedChannel(context, "1");
Channel two = await SeedChannel(context, "2");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, one.Id)).ShouldBe([elementId]);
(await AttachedElementIds(context, two.Id)).ShouldBe([elementId]);
}
[Test]
public async Task Skips_Hls_Direct_Channels_Where_The_Overlay_Cannot_Render()
{
await using TvContext context = _db.CreateContext();
await SeedBuiltInElement(context);
Channel direct = await SeedChannel(context, "1", StreamingMode.HttpLiveStreamingDirect);
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, direct.Id)).ShouldBeEmpty();
}
[Test]
public async Task Does_Not_Duplicate_An_Existing_Attachment()
{
await using TvContext context = _db.CreateContext();
int elementId = await SeedBuiltInElement(context);
Channel channel = await SeedChannel(context, "1");
await context.AddAsync(
new ChannelGraphicsElement { ChannelId = channel.Id, GraphicsElementId = elementId });
await context.SaveChangesAsync();
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).Count.ShouldBe(1);
}
// The load-bearing property: a default must not fight the operator.
[Test]
public async Task Does_Not_Re_Attach_After_An_Operator_Clears_It()
{
await using TvContext context = _db.CreateContext();
await SeedBuiltInElement(context);
Channel channel = await SeedChannel(context, "1");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).Count.ShouldBe(1);
// operator turns the overlay off for this channel
context.Set<ChannelGraphicsElement>()
.RemoveRange(context.Set<ChannelGraphicsElement>().Where(cge => cge.ChannelId == channel.Id));
await context.SaveChangesAsync();
// ...and the app restarts
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).ShouldBeEmpty();
}
// The upgrade population this backfill exists for -- an install seeding the template for the
// first time on this boot -- must not be stranded. The GraphicsElement row is normally created
// by RefreshGraphicsElements, which runs long after startup, so the seeder ensures it itself.
// Without that, the marker would be written against an unresolved element and every pre-existing
// channel would go permanently unattached.
[Test]
public async Task Backfills_On_The_Same_Boot_That_First_Seeds_The_Template()
{
var fs = new MockFileSystem();
await using TvContext context = _db.CreateContext();
Channel channel = await SeedChannel(context, "1");
// no ConfigElement markers and no GraphicsElement row: a pre-#74 install meeting #732
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).Count.ShouldBe(1);
}
// Filename alone is ambiguous: the five template folders are separate namespaces, so an element
// of another kind may legitimately carry the same filename.
[Test]
public async Task Ignores_A_Same_Named_Element_Of_A_Different_Kind()
{
await using TvContext context = _db.CreateContext();
await context.GraphicsElements.AddAsync(
new GraphicsElement
{
Path = $"/templates/image/{GraphicsElementDefaults.OnNowNextFileName}",
Kind = GraphicsElementKind.Image
});
await context.SaveChangesAsync();
Channel channel = await SeedChannel(context, "1");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).ShouldBeEmpty();
}
[Test]
public async Task Ignores_A_Non_Built_In_Element_With_A_Different_Filename()
{
await using TvContext context = _db.CreateContext();
await context.GraphicsElements.AddAsync(
new GraphicsElement { Path = "/templates/text/something-else.yml", Kind = GraphicsElementKind.Text });
await context.SaveChangesAsync();
Channel channel = await SeedChannel(context, "1");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).ShouldBeEmpty();
}
// The marker is permanent, so writing it with nothing resolved would strand every channel. Stay
// armed instead and pick the work up once the element exists.
[Test]
public async Task Stays_Armed_When_There_Is_No_Built_In_Element_To_Attach()
{
await using TvContext context = _db.CreateContext();
Channel channel = await SeedChannel(context, "1");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await context.ConfigElements
.AnyAsync(c => c.Key == ConfigElementKey.GraphicsOnNowNextDefaultAttached.Key))
.ShouldBeFalse("the marker was written with nothing to attach");
// the element turns up later; the backfill must still do its job
await SeedBuiltInElement(context);
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).Count.ShouldBe(1);
}
// The already-seeded branch of SeedOnNowNext has its own EnsureBuiltInElementRow call. Removing
// it reddened nothing until this test existed -- the other tests all exercise a FRESH seed.
[Test]
public async Task An_Already_Seeded_Install_Missing_Its_Element_Row_Gets_One()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
string target = Path.Combine(
FileSystemLayout.GraphicsElementsTextTemplatesFolder,
GraphicsElementDefaults.OnNowNextFileName);
await fs.File.WriteAllTextAsync(target, "name: On Now / Next\n");
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
(await context.GraphicsElements.CountAsync()).ShouldBe(0);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
// Assert the ROW, not just that one exists: a lookup-only assertion is satisfied by a row
// with a null Name, which sorts the built-in element into the unnamed bucket in the SPA.
List<GraphicsElement> rows = await context.GraphicsElements.ToListAsync();
rows.Count.ShouldBe(1);
rows[0].Path.ShouldBe(target);
rows[0].Kind.ShouldBe(GraphicsElementKind.Text);
rows[0].Name.ShouldBe(GraphicsElementDefaults.OnNowNextName);
(await GraphicsElementSeeder.GetBuiltInElementId(context, CancellationToken.None)).IsSome.ShouldBeTrue();
}
// The armed path is a real, reachable state: an operator deletes the template, refresh reaps the
// row, and the backfill then has nothing to resolve. Pin what happens when the element comes
// back -- a single global marker cannot both avoid stranding and avoid re-adding, and this is
// the half we accept. See graphics.on-now-next-on-by-default.
[Test]
public async Task While_Armed_A_Restored_Element_Is_Attached_To_Every_Eligible_Channel()
{
await using TvContext context = _db.CreateContext();
Channel kept = await SeedChannel(context, "1");
Channel cleared = await SeedChannel(context, "2");
// nothing to resolve yet -> stays armed, no marker
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await context.ConfigElements
.AnyAsync(c => c.Key == ConfigElementKey.GraphicsOnNowNextDefaultAttached.Key))
.ShouldBeFalse();
// The element reappears and is attached to BOTH channels; the operator then clears `cleared`.
// Doing the attach-then-remove for real matters: seeding `cleared` with no join at all would
// only prove an untouched channel gets backfilled, which is not the claim.
int elementId = await SeedBuiltInElement(context);
await context.AddAsync(new ChannelGraphicsElement { ChannelId = kept.Id, GraphicsElementId = elementId });
await context.AddAsync(new ChannelGraphicsElement { ChannelId = cleared.Id, GraphicsElementId = elementId });
await context.SaveChangesAsync();
(await AttachedElementIds(context, cleared.Id)).Count.ShouldBe(1);
context.Set<ChannelGraphicsElement>()
.RemoveRange(context.Set<ChannelGraphicsElement>().Where(cge => cge.ChannelId == cleared.Id));
await context.SaveChangesAsync();
(await AttachedElementIds(context, cleared.Id)).ShouldBeEmpty();
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
// documented consequence: the still-armed backfill cannot see that deliberate clear
(await AttachedElementIds(context, kept.Id)).Count.ShouldBe(1);
(await AttachedElementIds(context, cleared.Id)).Count.ShouldBe(1);
// ...but it is now marked, so it never fires again
(await context.ConfigElements
.AnyAsync(c => c.Key == ConfigElementKey.GraphicsOnNowNextDefaultAttached.Key))
.ShouldBeTrue();
}
}
@@ -6,6 +6,7 @@ using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using NUnit.Framework;
using Shouldly;
using YamlDotNet.Serialization;
@@ -36,7 +37,7 @@ public class GraphicsElementSeederTests
var fs = new MockFileSystem();
await using TvContext context = _db.CreateContext();
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
fs.File.Exists(_seededPath).ShouldBeTrue();
fs.File.ReadAllText(_seededPath).ShouldContain("epg_entries: 2");
@@ -51,7 +52,7 @@ public class GraphicsElementSeederTests
await fs.File.WriteAllTextAsync(_seededPath, "name: Operator Custom\nepg_entries: 2\n");
await using TvContext context = _db.CreateContext();
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
fs.File.ReadAllText(_seededPath).ShouldContain("Operator Custom");
}
@@ -62,9 +63,9 @@ public class GraphicsElementSeederTests
var fs = new MockFileSystem();
await using TvContext context = _db.CreateContext();
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
fs.File.Delete(_seededPath);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
fs.File.Exists(_seededPath).ShouldBeFalse();
}
@@ -77,7 +78,7 @@ public class GraphicsElementSeederTests
{
var fs = new MockFileSystem();
await using TvContext context = _db.CreateContext();
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
string yaml = await fs.File.ReadAllTextAsync(_seededPath);
IDeserializer deserializer = new DeserializerBuilder()
@@ -0,0 +1,365 @@
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Graphics;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Tests.Support;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.EntityFrameworkCore;
using NUnit.Framework;
using Shouldly;
using Testably.Abstractions.Testing;
using Testably.Abstractions.Testing.FileSystem;
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.NamingConventions;
namespace ErsatzTV.Tests.Infrastructure;
/// <summary>
/// #732: the seeder writes the On Now / Next template once and never revisits it, so a change to the
/// shipped default would reach new databases only. These pin the upgrade path that fixes that, and
/// the boundary that keeps it from clobbering an operator's edits.
/// </summary>
[TestFixture]
public class GraphicsElementSeederUpgradeTests
{
// Byte-for-byte the default shipped before #732. Verified 2026-08-26 against the live prod
// install at 192.168.1.29 (md5 ef9afc088cf6dba252f725babbf3334f), so this is a real
// fingerprint rather than a copy of the constant it is meant to detect.
private const string OnNowNextYamlV1 =
"""
name: On Now / Next
epg_entries: 2
location: BottomLeft
horizontal_margin_percent: 4
vertical_margin_percent: 8
width_percent: 42
text_fit: Wrap
text_align: Left
z_index: 100
# transparent until 4s in, fade in 1s, hold 6s, fade out 1s
opacity_expression: "LinearFadeDuration(content_seconds, 4, 1, 6)"
base_style: now
styles:
- name: now
font_family: "Noto Sans"
font_size: 30
font_weight: 700
text_color: "#FFFFFF"
halo_color: "#000000"
halo_width: 2
- name: sub
font_family: "Noto Sans"
font_size: 22
font_weight: 400
text_color: "#DDDDDD"
halo_color: "#000000"
halo_width: 2
- name: next
font_family: "Noto Sans"
font_size: 22
font_weight: 400
text_color: "#DDDDDD"
halo_color: "#000000"
halo_width: 2
text: |
[now]NOW {{ Epg[0].Title }}[/now]
{{ if Epg[0].SubTitle }}[sub]{{ Epg[0].SubTitle }}[/sub]{{ end }}
{{ if (array.size Epg) > 1 }}[next]NEXT {{ Epg[1].Title }}[/next]{{ end }}
""";
private InMemoryTvContext _db = null!;
private string _target = null!;
[SetUp]
public async Task SetUp()
{
_db = await InMemoryTvContext.CreateAsync();
_target = Path.Combine(
FileSystemLayout.GraphicsElementsTextTemplatesFolder,
GraphicsElementDefaults.OnNowNextFileName);
}
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private async Task<MockFileSystem> RunSeederOverAlreadySeededDatabase(string existingContent)
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
if (existingContent is not null)
{
await fs.File.WriteAllTextAsync(_target, existingContent);
}
await using TvContext context = _db.CreateContext();
// The fixture DB is shared across calls within a test, so only seed the marker once.
string key = ConfigElementKey.GraphicsOnNowNextSeeded.Key;
if (!context.ConfigElements.Any(c => c.Key == key))
{
context.ConfigElements.Add(new ConfigElement { Key = key, Value = "true" });
await context.SaveChangesAsync();
}
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
return fs;
}
[Test]
public async Task Upgrades_An_Untouched_Previous_Default()
{
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(OnNowNextYamlV1);
string result = await fs.File.ReadAllTextAsync(_target);
result.ShouldNotBe(OnNowNextYamlV1);
result.ShouldContain("background_color");
result.ShouldContain("background_padding");
}
[Test]
public async Task Upgrades_An_Untouched_Previous_Default_With_Windows_Line_Endings()
{
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(
OnNowNextYamlV1.Replace("\n", "\r\n"));
string result = await fs.File.ReadAllTextAsync(_target);
result.ShouldContain("background_color");
}
[Test]
public async Task Leaves_An_Operator_Modified_File_Alone()
{
// One changed value is enough to stop matching the fingerprint.
string edited = OnNowNextYamlV1.Replace("width_percent: 42", "width_percent: 30");
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(edited);
string result = await fs.File.ReadAllTextAsync(_target);
result.ShouldBe(edited);
result.ShouldNotContain("background_color");
}
[Test]
public async Task Leaves_The_Current_Default_Alone_So_The_Upgrade_Is_Idempotent()
{
MockFileSystem first = await RunSeederOverAlreadySeededDatabase(OnNowNextYamlV1);
string upgraded = await first.File.ReadAllTextAsync(_target);
MockFileSystem second = await RunSeederOverAlreadySeededDatabase(upgraded);
string again = await second.File.ReadAllTextAsync(_target);
again.ShouldBe(upgraded);
}
// The upgrade runs inside DatabaseMigratorService, ahead of DatabaseIsReady(). Before #732 the
// already-seeded branch touched the filesystem not at all, so a template the app cannot read --
// e.g. edited as root via `docker exec` while the app runs as PUID/PGID -- used to boot fine.
// It must not become a failure to start.
[Test]
public async Task An_Unwritable_Template_Does_Not_Fail_Startup()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
// the file matches a shipped default, so the upgrade WILL try to rewrite it -- and that write
// is denied, standing in for a root-owned or read-only template
var intercepted = 0;
fs.Intercept.Changing(
FileSystemTypes.File,
_ =>
{
intercepted++;
throw new UnauthorizedAccessException("simulated permission denial");
});
await using TvContext context = _db.CreateContext();
string key = ConfigElementKey.GraphicsOnNowNextSeeded.Key;
context.ConfigElements.Add(new ConfigElement { Key = key, Value = "true" });
await context.SaveChangesAsync();
await Should.NotThrowAsync(
() => GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None));
// Without this the test would pass just as happily if the upgrade never reached the write.
intercepted.ShouldBeGreaterThan(0, "the write interceptor never fired");
// and the original template survives the denied write
(await fs.File.ReadAllTextAsync(_target)).ShouldBe(OnNowNextYamlV1);
}
// The write path is not the only one that can fault. A template the app cannot READ used to be
// harmless on an already-seeded install; it must stay that way. An exclusive lock produces a
// genuine ReadAllTextAsync failure rather than an intercepted write dressed up as one.
[Test]
public async Task A_Read_Failure_On_The_Template_Does_Not_Fail_Startup()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
await using Stream exclusive = fs.File.Open(_target, FileMode.Open, FileAccess.Read, FileShare.None);
// prove the lock actually denies a read, so the test cannot pass by never hitting one
Should.Throw<IOException>(() => fs.File.ReadAllText(_target));
await Should.NotThrowAsync(
() => GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None));
}
[Test]
public async Task Does_Not_Create_The_File_When_It_Is_Absent()
{
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(null);
fs.File.Exists(_target).ShouldBeFalse();
}
[Test]
public async Task The_Upgraded_Template_Still_Deserializes_With_A_Resolvable_Base_Style()
{
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(OnNowNextYamlV1);
string yaml = await fs.File.ReadAllTextAsync(_target);
IDeserializer deserializer = new DeserializerBuilder()
.WithNamingConvention(CamelCaseNamingConvention.Instance)
.Build();
var element = deserializer.Deserialize<TextGraphicsElement>(yaml);
element.ShouldNotBeNull();
element.BackgroundColor.ShouldBe("#000000");
element.BackgroundOpacityPercent.ShouldBe(65);
element.BackgroundPadding.ShouldBe(14);
element.BackgroundCornerRadius.ShouldBe(8);
// The border is what makes the box visible over dark content; without it the translucent
// black fill is indistinguishable from the frame behind it.
element.BorderColor.ShouldBe("#59FFFFFF");
element.BorderWidth.ShouldBe(1);
// #570: every style needs a font_family, and base_style must resolve.
element.Styles.ShouldNotBeEmpty();
element.Styles.ShouldAllBe(s => s.FontFamily != null);
element.Styles.ShouldContain(s => s.Name == element.BaseStyle);
}
// Without the duplicate guard in EnsureBuiltInElementRow the already-seeded branch inserts a
// fresh row on EVERY boot: RefreshGraphicsElements will neither reap them (the file exists) nor
// dedupe them, so the row set grows without bound. Idempotence of the FILE is not idempotence
// of the ROW, and the existing idempotence test only looks at the file.
[Test]
public async Task Repeated_Seeding_Does_Not_Accumulate_Element_Rows()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
for (var i = 0; i < 3; i++)
{
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
}
(await context.GraphicsElements.ToListAsync()).Count.ShouldBe(1);
}
// A failed write must not leave a truncated template behind: it would match no fingerprint, so
// the upgrade could never repair it, and the loader rejects malformed YAML outright.
[Test]
public async Task A_Failed_Write_Leaves_The_Original_Template_Intact()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
// Capture WHICH path the write targets. Asserting only "the original survived" cannot tell
// an atomic write from an in-place one here: Testably raises the interception BEFORE it
// truncates, so a plain WriteAllTextAsync(target) would leave the file intact too -- on a
// real filesystem it would not. The path is what actually distinguishes them.
var writtenPaths = new List<string>();
fs.Intercept.Changing(
FileSystemTypes.File,
c =>
{
writtenPaths.Add(c.Path);
throw new IOException("simulated disk full");
});
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
await Should.NotThrowAsync(
() => GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None));
writtenPaths.ShouldNotBeEmpty("the write interceptor never fired");
writtenPaths.ShouldAllBe(path => path.EndsWith(".upgrade.tmp"), "the upgrade wrote the live template in place instead of a temp file");
(await fs.File.ReadAllTextAsync(_target)).ShouldBe(OnNowNextYamlV1);
fs.Directory.GetFiles(FileSystemLayout.GraphicsElementsTextTemplatesFolder, "*.upgrade.tmp")
.ShouldBeEmpty("a temp file was left behind");
}
// The sibling test faults on the FIRST write, so it never reaches File.Move or the cleanup. Fault
// the replace instead, after a complete temp write: that is the path where a non-atomic
// implementation would already have truncated the live template.
[Test]
public async Task A_Failed_Replace_After_A_Complete_Temp_Write_Leaves_The_Original_Intact()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
var seenPaths = new List<string>();
fs.Intercept.Event(
c =>
{
seenPaths.Add($"{c.ChangeType}:{c.Path}");
// let the temp file be written in full; fail only when the live template is touched
if (c.Path == _target)
{
throw new IOException("simulated replace failure");
}
},
_ => true);
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
await Should.NotThrowAsync(
() => GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None));
seenPaths.ShouldContain(path => path.EndsWith(".upgrade.tmp"), "no temp file was ever written");
// Assert the replace is a RENAME, not merely "the target was touched after the temp was".
// A File.Copy(temp, target, true) also touches both in that order and would leave the
// original intact under this mock (interception runs before the change), so path ordering
// alone cannot tell an atomic replace from a truncating one -- the change TYPE can.
seenPaths.ShouldContain($"Renamed:{_target}", "the replace was not an atomic rename");
// The whole point of write-then-move: the live template is untouched by a failed replace.
(await fs.File.ReadAllTextAsync(_target)).ShouldBe(OnNowNextYamlV1);
// and nothing this call created is left behind
fs.Directory.GetFiles(FileSystemLayout.GraphicsElementsTextTemplatesFolder, "*.upgrade.tmp")
.ShouldBeEmpty("a temp file was left behind");
}
}
@@ -0,0 +1,299 @@
using System.Reflection;
using System.Text;
using Elastic.Clients.Elasticsearch;
using Elastic.Transport;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Search;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
using ErsatzTV.Infrastructure.Search;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
/// <summary>
/// ersatztv#824 — the gap ersatztv#701 named rather than papered over.
/// <para>
/// <c>ElasticSearchIndex.UpdateSong</c> holds an INDEPENDENT copy of the logic
/// <see cref="SongIndexerMetadataMutationTests" /> pins on <c>LuceneSearchIndex</c>. #701 removed
/// <c>metadata.AlbumArtists ??= []; metadata.Artists ??= [];</c> from both, but only Lucene gained
/// a regression test — so reintroducing the mutation in the Elastic copy ALONE left the whole
/// suite green. This fixture closes that: the assertions are the same three, driven through the
/// real <c>ElasticSearchIndex</c> against a real <see cref="TvContext" />.
/// </para>
/// <para>
/// <b>Why a stubbed transport rather than a live server.</b> #824 listed "inject a non-network
/// <c>ElasticsearchClient</c> transport" as option 1 and it is what shipped:
/// <c>Elastic.Transport.InMemoryRequestInvoker</c> is public in the pinned Elastic.Transport, and
/// <c>ElasticsearchClientSettings(NodePool, IRequestInvoker)</c> accepts it. No server, no socket,
/// no new package.
/// </para>
/// <para>
/// <b>The canned response body is load-bearing, not decoration.</b> A bare
/// <c>InMemoryRequestInvoker()</c> answers with an EMPTY body, which the client cannot deserialize
/// into an <c>IndexResponse</c>. That throw lands in <c>UpdateSong</c>'s catch, which logs a
/// warning and assigns <c>metadata.Song = null</c> — so the fixture would measure the ERROR path
/// while every "did not mutate" assertion below still passed, vacuously. The
/// <see cref="ThrowOnWarningLogger{T}" /> is the belt to that brace: it fails the test if the
/// catch ran at all.
/// </para>
/// <para>
/// The client is injected into the private <c>_client</c> field rather than obtained normally,
/// because <c>CreateClient</c> reads the process-wide static <c>ElasticSearchIndex.Uri</c> and
/// would open a real socket. <c>UpdateItems</c> — unlike <c>IndexExists</c> and
/// <c>Initialize</c> — never runs <c>_client ??= CreateClient()</c>, so the injected instance is
/// the one used and an uninjected one would simply be null.
/// </para>
/// </summary>
[TestFixture]
[NonParallelizable]
public class ElasticSongIndexerMetadataMutationTests
{
private const string TestIndexName = "etv-824-test";
private string? _originalIndexName;
/// <summary>
/// <c>ElasticSearchIndex.IndexName</c> is a process-wide static. Only <c>Startup</c> reads it today,
/// so leaving it set leaks nothing that currently runs — but a static this fixture writes and never
/// restores is a cross-test hazard waiting for the first test that does read it.
/// </summary>
[SetUp]
public void SetUp() => _originalIndexName = ElasticSearchIndex.IndexName;
[TearDown]
public void TearDown() => ElasticSearchIndex.IndexName = _originalIndexName;
/// <summary>
/// A well-formed <c>IndexResponse</c>. See the fixture docstring: an empty body diverts the run
/// into <c>UpdateSong</c>'s catch and makes every assertion below vacuous.
/// </summary>
private const string IndexResponseBody =
"""
{"_index":"etv-824-test","_id":"1","_version":1,"result":"created",
"_shards":{"total":1,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
""";
[Test]
public async Task UpdateSong_Must_Not_Mutate_Nullable_Artists_On_A_Tracked_Entity()
{
await using var harness = await InMemoryTvContext.CreateAsync();
int metadataId;
int songId;
await using (TvContext context = harness.CreateContext())
{
var library = new LocalLibrary { Name = "Music", MediaKind = LibraryMediaKind.Songs };
context.Add(library);
await context.SaveChangesAsync();
var libraryPath = new LibraryPath { Path = "/music", LibraryId = library.Id };
context.Add(libraryPath);
await context.SaveChangesAsync();
var song = new Song
{
LibraryPathId = libraryPath.Id,
MediaVersions = [],
SongMetadata =
[
new SongMetadata
{
MetadataKind = MetadataKind.Fallback,
Title = "Untagged Track",
SortTitle = "untagged track",
DateAdded = new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc),
// The shape FallbackMetadataProvider.GetSongMetadata leaves behind: it never
// assigns either primitive collection, so both columns persist as NULL.
Artists = null!,
AlbumArtists = null!,
Genres = [],
Tags = [],
Studios = [],
Actors = [],
Artwork = [],
Guids = []
}
]
};
context.Add(song);
await context.SaveChangesAsync();
metadataId = song.SongMetadata[0].Id;
songId = song.Id;
}
// The seed must actually have produced NULL columns, or every assertion below is vacuous.
(await ReadRawArtists(harness, metadataId)).ShouldBeNull();
await using (TvContext context = harness.CreateContext())
{
// Deliberately TRACKED -- the indexer's own contract is what is being pinned, not the
// AsNoTracking() habit of today's two callers. See SongIndexerMetadataMutationTests.
Song tracked = await context.Songs
.IncludeForSearch()
.AsSplitQuery()
.SingleAsync();
SongMetadata metadata = tracked.SongMetadata[0];
metadata.Artists.ShouldBeNull("EF must materialize the NULL column as null, not as an empty list");
var logger = new ThrowOnWarningLogger<ElasticSearchIndex>();
var index = new ElasticSearchIndex(
new SearchQueryParser(
Substitute.For<ISmartCollectionCache>(),
Substitute.For<ILogger<SearchQueryParser>>()),
logger);
var invoker = new CapturingRequestInvoker(
new InMemoryRequestInvoker(
Encoding.UTF8.GetBytes(IndexResponseBody),
200,
exception: null,
contentType: "application/json",
// The X-Elastic-Product header is REQUIRED, not cosmetic. The client runs a product
// check on its first response and throws UnsupportedProductException ("the server is
// not a supported distribution of Elasticsearch") without it -- which lands in
// UpdateSong's catch and makes the fixture measure the error path. Measured: this is
// exactly how this fixture first failed.
headers: ProductCheckHeaders()));
var settings = new ElasticsearchClientSettings(
new SingleNodePool(new Uri("http://localhost:9200")),
invoker)
.DefaultIndex(TestIndexName);
ElasticSearchIndex.IndexName = TestIndexName;
typeof(ElasticSearchIndex)
.GetField("_client", BindingFlags.NonPublic | BindingFlags.Instance)!
.SetValue(index, new ElasticsearchClient(settings));
// A bare substitute returns null from GetAllLanguageCodes, which NPEs inside AddLanguages and
// would divert the run into UpdateSong's catch.
var languageCodeService = Substitute.For<ILanguageCodeService>();
languageCodeService.GetAllLanguageCodes(Arg.Any<List<string>>()).Returns([]);
languageCodeService.GetAllLanguageCodes(Arg.Any<string>()).Returns([]);
await index.UpdateItems(
Substitute.For<ISearchRepository>(),
Substitute.For<IFallbackMetadataProvider>(),
languageCodeService,
[tracked]);
// Surfacing the exception rather than asserting ShouldBeNull: the catch is the fixture's
// most likely failure mode (see the canned-response note above), and "expected null but was
// <Exception>" without the message sends the next reader hunting for a cause the fixture
// already had in its hand.
if (logger.Failure is not null)
{
Assert.Fail("UpdateSong threw and its catch ran, so this probe measured the error path "
+ $"rather than the indexing path: {logger.Failure}");
}
// POSITIVE CONTROL, and it is not optional: every assertion below says something did NOT
// happen, so all of them hold vacuously if UpdateSong never ran. The Lucene fixture uses
// `writer.NumDocs == 1` for exactly this; the transport-level equivalent is that the indexer
// actually issued the index request for THIS song. Like NumDocs, it proves the song-indexing
// path ran -- it does NOT prove the artist reads specifically ran.
invoker.Requests.Count.ShouldBe(
1,
"UpdateSong did not issue exactly one index request, so the assertions below would pass "
+ $"without exercising the code under test. Captured: [{string.Join(", ", invoker.Requests)}]");
// The document id is compared as the LAST PATH SEGMENT, not with ShouldContain. A substring
// test is a false-pass vector here: the index name itself carries digits ("etv-824-test"), so
// ShouldContain("2") or ShouldContain("4") would be satisfied by the index name alone for a
// song whose id happened to be 2 or 4, and the assertion would stop discriminating without
// ever failing.
string path = invoker.Requests[0].Split(' ')[^1].Split('?')[0];
path.Split('/')[^1].ShouldBe(
songId.ToString(),
$"the index request was not for the seeded song. Captured: {invoker.Requests[0]}");
// 1. The indexer left the entity alone.
metadata.Artists.ShouldBeNull();
metadata.AlbumArtists.ShouldBeNull();
// 2. ...so EF has nothing to persist. This is the assertion that fails loudly the day the
// mutation returns, even if a later refactor stopped the value from being observable above.
context.Entry(metadata).State.ShouldBe(EntityState.Unchanged);
// 3. And the save that a real caller would go on to make does not rewrite the column.
await context.SaveChangesAsync();
}
(await ReadRawArtists(harness, metadataId)).ShouldBeNull();
}
private static Dictionary<string, IEnumerable<string>> ProductCheckHeaders() =>
new(StringComparer.OrdinalIgnoreCase) { ["x-elastic-product"] = ["Elasticsearch"] };
/// <summary>
/// Records every request the client actually issues, so the fixture can prove the code under test
/// ran. Delegates the answering to a real <see cref="InMemoryRequestInvoker" /> rather than
/// hand-building a response.
/// </summary>
private sealed class CapturingRequestInvoker(InMemoryRequestInvoker inner) : IRequestInvoker
{
public List<string> Requests { get; } = [];
public ResponseFactory ResponseFactory => inner.ResponseFactory;
public TResponse Request<TResponse>(
Endpoint endpoint,
BoundConfiguration boundConfiguration,
PostData? postData)
where TResponse : TransportResponse, new()
{
Requests.Add($"{endpoint.Method} {endpoint.PathAndQuery}");
return inner.Request<TResponse>(endpoint, boundConfiguration, postData);
}
public Task<TResponse> RequestAsync<TResponse>(
Endpoint endpoint,
BoundConfiguration boundConfiguration,
PostData? postData,
CancellationToken cancellationToken)
where TResponse : TransportResponse, new()
{
Requests.Add($"{endpoint.Method} {endpoint.PathAndQuery}");
return inner.RequestAsync<TResponse>(endpoint, boundConfiguration, postData, cancellationToken);
}
// IRequestInvoker extends IDisposable, but InMemoryRequestInvoker holds no disposable state and
// exposes no Dispose of its own -- there is nothing to forward to.
public void Dispose()
{
}
}
private static async Task<object?> ReadRawArtists(InMemoryTvContext harness, int metadataId)
{
await using TvContext context = harness.CreateContext();
await using var command = context.Database.GetDbConnection().CreateCommand();
command.CommandText = $"SELECT Artists FROM SongMetadata WHERE Id = {metadataId}";
object? value = await command.ExecuteScalarAsync();
// ExecuteScalar returns CLR null both for "the column is NULL" and for "there is no such row",
// and the second is reachable: UpdateSong's catch assigns metadata.Song = null, which severs a
// required relationship and cascades the row to Deleted, so a SaveChanges on the error path
// DELETES it and a plain null check would pass for the wrong reason.
if (value is null)
{
Assert.Fail($"SongMetadata row {metadataId} no longer exists, so its Artists column cannot "
+ "be read -- the probe measured a deleted row rather than a preserved NULL.");
}
return value is DBNull ? null : value;
}
}
@@ -0,0 +1,388 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Infrastructure;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.MySql.Data;
using ErsatzTV.Infrastructure.Sqlite.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using MySqlConnector;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
/// <summary>
/// ersatztv#823 — the question ersatztv#701 split out rather than answered: can a runtime null reach
/// one of the SIX collection-valued scalar columns (<c>DaysOfMonth</c>, <c>MonthsOfYear</c>,
/// <c>DaysOfWeek</c> on <see cref="ProgramScheduleAlternate" /> and <see cref="PlayoutTemplate" />)?
/// <para>
/// <b>Measured, not reasoned about</b>, because the reasoning available beforehand pointed the
/// wrong way. The two converters differ on their read side —
/// <c>IntCollectionValueConverter</c> maps null-or-blank to <c>Array.Empty&lt;int&gt;()</c> while
/// <c>EnumCollectionJsonValueConverter</c> would dereference the result of
/// <c>JsonConvert.DeserializeObject</c> — so the expectation was that the two behave differently
/// on a NULL row. They do not: <b>EF does not invoke a value converter for a NULL column at
/// all</b>, so all six materialize as CLR <c>null</c> and the int converter's null-to-empty
/// branch is dead on this path. That is the measurement this fixture pins.
/// </para>
/// <para>
/// It also pins the WRITE half: assigning <c>null</c> to one of the six and calling
/// <c>SaveChanges</c> SUCCEEDS and stores SQL NULL — the converter is skipped on the way out too.
/// State that precisely, because the overclaim is tempting: this is a property of the CODE, not a
/// live caller. NO caller supplies a null today — every production construction of
/// <c>ReplacePlayoutAlternateSchedule</c> / <c>ReplacePlayoutTemplate</c> goes through the HTTP
/// request records, which normalize with <c>?? []</c>. What makes it a latent gun is that
/// <c>ReplacePlayoutAlternateScheduleItemsHandler</c> / <c>ReplacePlayoutTemplateItemsHandler</c>
/// assign the command value straight onto the entity, so nothing in the write path itself refuses.
/// </para>
/// <para>
/// The LEGACY route is narrower than "the columns are nullable", and conflating the two is the easy
/// error: all six are <c>nullable: true</c> on both providers, but five of the six were present at
/// <c>CreateTable</c>, so a NULL there still needs code to write one. EXACTLY ONE case is
/// code-path-free — SQLite's <c>20240113140741_Add_PlayoutTemplate_DaysOfMonth</c> is an
/// <c>AddColumn</c> with <c>nullable: true</c> and NO <c>defaultValue</c>, so <c>PlayoutTemplate</c>
/// rows inserted before it hold NULL, and by construction those rows had no day-of-month
/// restriction. On MySQL <c>PlayoutTemplate</c> arrived whole in
/// <c>20240114034944_Add_BlockScheduling</c>, so there is no code-path-free NULL for any of the six
/// there. This fixture manufactures its NULL with a raw <c>UPDATE</c>, which is a code path — it
/// measures MATERIALIZATION, and the legacy route above is established by reading the migrations.
/// </para>
/// <para>
/// What the selector then does with it is pinned by
/// <c>ErsatzTV.Core.Tests.Scheduling.AlternateScheduleSelectorTests.GetScheduleForDate_NullCollections</c>:
/// unguarded, <c>.Contains</c> throws <see cref="NullReferenceException" />.
/// </para>
/// <para>
/// Runs against BOTH providers from ONE fixture body, because the question is about provider
/// materialization and a SQLite-only answer would not have settled it. MySQL needs a live server,
/// supplied via <c>ETV_TEST_MYSQL_CONNECTION</c>; without it the MySQL fixture <b>ignores</b> — a
/// visible skip, never a silent pass. <c>ETV_REQUIRE_MYSQL_TESTS</c> turns that skip into a hard
/// failure for a runner that is supposed to have one. This mirrors
/// <see cref="LibraryFolderDedupeMigrationTests" />, which established the pattern.
/// </para>
/// <para>
/// <b>Cost, stated.</b> Each test migrates a FRESH database (per-test isolation by construction —
/// ersatztv#491 measured what a shared database and a wipe-that-must-succeed cost). Replaying every
/// migration is the expensive part and it is deliberate: a reachability fixture should stand on the
/// SHIPPED schema, not on one <c>EnsureCreated</c> builds from the current model.
/// </para>
/// </summary>
[TestFixture(TestProvider.Sqlite)]
[TestFixture(TestProvider.MySql)]
[NonParallelizable]
public class SchedulingCollectionColumnNullTests(TestProvider provider)
{
private const string MySqlConnectionVariable = "ETV_TEST_MYSQL_CONNECTION";
private const string MySqlRequiredVariable = "ETV_REQUIRE_MYSQL_TESTS";
private string _databasePath = null!;
private string? _mySqlConnectionString;
private DbContextOptions<TvContext> _options = null!;
private string _originalLastInsertedRowId = null!;
private string _originalCollation = null!;
private bool _originalIsSqlite;
private Func<DbUpdateException, bool> _originalUniqueViolation = null!;
/// <summary>
/// <c>TvContext</c>'s provider statics are process-wide, and the MySQL arm sets them to MySQL
/// values. Restoring them is NOT belt-and-braces: <see cref="InMemoryTvContext" /> — the harness
/// most of this suite uses — resets only three of the five (<c>IsSqlite</c>,
/// <c>IsUniqueConstraintViolation</c>, <c>RegisterUnicodeCaseFunctions</c>) and leaves
/// <c>LastInsertedRowId</c> and <c>CaseInsensitiveCollation</c> alone. So without this, a MySQL arm
/// running before a SQLite test leaves <c>last_insert_id()</c> in place for a SQLite connection,
/// which is an order-dependent failure in a fixture that never mentions MySQL.
/// <c>[NonParallelizable]</c> serialises execution; it does not restore state.
/// </summary>
[SetUp]
public async Task SetUp()
{
_originalLastInsertedRowId = TvContext.LastInsertedRowId;
_originalCollation = TvContext.CaseInsensitiveCollation;
_originalIsSqlite = TvContext.IsSqlite;
_originalUniqueViolation = TvContext.IsUniqueConstraintViolation;
if (provider is TestProvider.Sqlite)
{
TvContext.IsSqlite = true;
TvContext.LastInsertedRowId = "last_insert_rowid()";
TvContext.CaseInsensitiveCollation = "NOCASE";
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
_databasePath = Path.Combine(Path.GetTempPath(), $"etv823-{Guid.NewGuid():N}.sqlite3");
_options = new DbContextOptionsBuilder<TvContext>()
.UseSqlite(
$"Data Source={_databasePath};Foreign Keys=False",
o => o.MigrationsAssembly("ErsatzTV.Infrastructure.Sqlite"))
.Options;
await using TvContext sqlite = Create(_options);
await sqlite.Database.MigrateAsync();
return;
}
string? baseConnectionString = Environment.GetEnvironmentVariable(MySqlConnectionVariable);
if (string.IsNullOrWhiteSpace(baseConnectionString))
{
string message =
$"{MySqlConnectionVariable} is not set, so the MySql half of the #823 measurement cannot "
+ "run. Whether a NULL column materializes as CLR null is a provider question, so a "
+ "SQLite-only answer does not settle it.";
if (IsTrue(Environment.GetEnvironmentVariable(MySqlRequiredVariable)))
{
Assert.Fail($"{message} {MySqlRequiredVariable} is set, so this is a failure, not a skip.");
}
Assert.Ignore($"{message} Set it to run this locally.");
}
// A database of our own, and a FRESH one per test: isolation by construction, per #491's finding
// that a shared name trades isolation for a wipe that has to succeed.
_mySqlConnectionString =
new MySqlConnectionStringBuilder(baseConnectionString) { Database = $"etv823_{Guid.NewGuid():N}" }
.ConnectionString;
TvContext.IsSqlite = false;
TvContext.LastInsertedRowId = "last_insert_id()";
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
ServerVersion serverVersion = ServerVersion.AutoDetect(_mySqlConnectionString);
_options = new DbContextOptionsBuilder<TvContext>()
.UseMySql(
_mySqlConnectionString,
serverVersion,
o => o.MigrationsAssembly("ErsatzTV.Infrastructure.MySql"))
.Options;
// NUnit does not run [TearDown] when [SetUp] throws, and by this point the database exists — the
// migration itself created it. Without this, a migration that fails part way strands a database
// and its connection pool on a SHARED server, once per attempt.
try
{
await using TvContext mysql = Create(_options);
await mysql.Database.MigrateAsync();
}
catch
{
await DropMySqlDatabase();
_mySqlConnectionString = null;
throw;
}
}
private async Task DropMySqlDatabase()
{
try
{
await using (TvContext context = Create(_options))
{
await context.Database.EnsureDeletedAsync();
}
await using var probe = new MySqlConnection(_mySqlConnectionString);
await MySqlConnection.ClearPoolAsync(probe);
}
catch (Exception ex)
{
// Never mask the original failure with a cleanup failure, and never fail a PASSING test in
// teardown because the server went away -- but say so, because a silent cleanup failure is how
// a leak becomes invisible.
await TestContext.Out.WriteLineAsync(
$"WARNING: could not drop the MySql test database {_mySqlConnectionString}: {ex.Message}");
}
}
[TearDown]
public async Task TearDown()
{
TvContext.LastInsertedRowId = _originalLastInsertedRowId;
TvContext.CaseInsensitiveCollation = _originalCollation;
TvContext.IsSqlite = _originalIsSqlite;
TvContext.IsUniqueConstraintViolation = _originalUniqueViolation;
if (provider is TestProvider.Sqlite)
{
Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools();
foreach (string path in new[] { _databasePath, $"{_databasePath}-wal", $"{_databasePath}-shm" })
{
if (File.Exists(path))
{
File.Delete(path);
}
}
return;
}
if (_mySqlConnectionString is not null)
{
await DropMySqlDatabase();
_mySqlConnectionString = null;
}
}
/// <summary>
/// A row whose three columns are NULL materializes as CLR null on every one of them — including
/// the two <c>IntCollectionValueConverter</c> columns, whose converter would have produced
/// <c>Array.Empty&lt;int&gt;()</c> had it been invoked.
/// </summary>
[Test]
public async Task A_Null_Column_Materializes_As_Clr_Null_On_All_Six()
{
await using (TvContext context = Create(_options))
{
await DisableForeignKeys(context);
context.ProgramScheduleAlternates.Add(NewAlternate(playoutId: 1));
context.PlayoutTemplates.Add(NewTemplate(playoutId: 1));
await context.SaveChangesAsync();
// Out of band, so the row is byte-identical to a legacy one. This UPDATE is itself a code
// path and does NOT demonstrate the legacy route -- see the class docstring for which single
// (column, provider) case is genuinely code-path-free. What is being measured here is what EF
// MATERIALIZES from such a row, which is the same regardless of how the NULL got there.
await context.Database.ExecuteSqlRawAsync(
"UPDATE ProgramScheduleAlternate SET DaysOfWeek = NULL, DaysOfMonth = NULL, MonthsOfYear = NULL");
await context.Database.ExecuteSqlRawAsync(
"UPDATE PlayoutTemplate SET DaysOfWeek = NULL, DaysOfMonth = NULL, MonthsOfYear = NULL");
}
// ANTI-VACUITY: a read that found no row would leave every ShouldBeNull below trivially true,
// so both reads are Single and would throw on an empty table.
await using (TvContext context = Create(_options))
{
ProgramScheduleAlternate alternate =
await context.ProgramScheduleAlternates.AsNoTracking().SingleAsync();
alternate.DaysOfWeek.ShouldBeNull(
"EnumCollectionJsonValueConverter must not be invoked for a NULL column");
alternate.DaysOfMonth.ShouldBeNull(
"IntCollectionValueConverter's null-to-empty branch must not run — EF skips the "
+ "converter for a NULL column, so this is null rather than an empty array");
alternate.MonthsOfYear.ShouldBeNull();
PlayoutTemplate template = await context.PlayoutTemplates.AsNoTracking().SingleAsync();
template.DaysOfWeek.ShouldBeNull();
template.DaysOfMonth.ShouldBeNull();
template.MonthsOfYear.ShouldBeNull();
}
}
/// <summary>
/// The write half. A caller that hands one of the six a null — which the two Replace* handlers do
/// verbatim from their command, and only the HTTP request records guard against — gets a stored
/// SQL NULL and no error at all.
/// </summary>
[Test]
public async Task Assigning_Null_Persists_Sql_Null_Rather_Than_Throwing()
{
await using (TvContext context = Create(_options))
{
await DisableForeignKeys(context);
ProgramScheduleAlternate alternate = NewAlternate(playoutId: 2);
alternate.DaysOfWeek = null!;
alternate.DaysOfMonth = null!;
alternate.MonthsOfYear = null!;
context.ProgramScheduleAlternates.Add(alternate);
// Not Should.NotThrow: the point is that this is the SHIPPED behaviour of the write path, so
// the assertion is that the round-trip below finds NULL, not merely that nothing blew up.
await context.SaveChangesAsync();
}
await using (TvContext context = Create(_options))
{
ProgramScheduleAlternate reloaded =
await context.ProgramScheduleAlternates.AsNoTracking().SingleAsync();
reloaded.DaysOfWeek.ShouldBeNull();
reloaded.DaysOfMonth.ShouldBeNull();
reloaded.MonthsOfYear.ShouldBeNull();
}
// ...and it really is SQL NULL in the column, not an empty string the converter round-trips.
// ExecuteScalar hands back CLR null both for a NULL column and for NO SUCH ROW, so the row is
// counted first — otherwise a fixture that silently deleted its row would report the same thing.
await using (TvContext context = Create(_options))
{
(await ScalarAsync(context, "SELECT COUNT(*) FROM ProgramScheduleAlternate"))
.ShouldNotBeNull();
Convert.ToInt32(await ScalarAsync(context, "SELECT COUNT(*) FROM ProgramScheduleAlternate"))
.ShouldBe(1, "the row is gone, so a null read below would prove nothing");
(await ScalarAsync(context, "SELECT DaysOfWeek FROM ProgramScheduleAlternate")).ShouldBeNull();
(await ScalarAsync(context, "SELECT DaysOfMonth FROM ProgramScheduleAlternate")).ShouldBeNull();
(await ScalarAsync(context, "SELECT MonthsOfYear FROM ProgramScheduleAlternate")).ShouldBeNull();
}
}
private static ProgramScheduleAlternate NewAlternate(int playoutId) =>
new()
{
PlayoutId = playoutId,
ProgramScheduleId = 1,
Index = 0,
DaysOfWeek = [],
DaysOfMonth = [],
MonthsOfYear = [],
StartMonth = 1,
StartDay = 1,
EndMonth = 12,
EndDay = 31
};
private static PlayoutTemplate NewTemplate(int playoutId) =>
new()
{
PlayoutId = playoutId,
TemplateId = 1,
Index = 0,
DaysOfWeek = [],
DaysOfMonth = [],
MonthsOfYear = [],
StartMonth = 1,
StartDay = 1,
EndMonth = 12,
EndDay = 31
};
/// <summary>
/// The rows here are deliberately partial graphs (a <c>PlayoutId</c> pointing at no Playout), so
/// foreign keys are off. SQLite takes it as a connection-string keyword; MySQL's
/// <c>foreign_key_checks</c> is a SESSION variable, so it is set on the context's own connection
/// and lives as long as that context does.
/// </summary>
private async Task DisableForeignKeys(TvContext context)
{
if (provider is TestProvider.MySql)
{
await context.Database.OpenConnectionAsync();
await context.Database.ExecuteSqlRawAsync("SET SESSION foreign_key_checks = 0");
}
}
private static async Task<object?> ScalarAsync(TvContext context, string sql)
{
await context.Database.OpenConnectionAsync();
await using System.Data.Common.DbCommand command = context.Database.GetDbConnection().CreateCommand();
command.CommandText = sql;
object? value = await command.ExecuteScalarAsync();
return value is DBNull ? null : value;
}
private static bool IsTrue(string? value) =>
!string.IsNullOrWhiteSpace(value)
&& (value == "1" || value.Equals("true", StringComparison.OrdinalIgnoreCase));
private static TvContext Create(DbContextOptions<TvContext> options) =>
new(
options,
NullLoggerFactory.Instance,
new SlowQueryInterceptor(NullLogger<SlowQueryInterceptor>.Instance));
}
@@ -0,0 +1,129 @@
using System.Reflection;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Infrastructure.Search;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
/// <summary>
/// ersatztv#824. The defect that produced #824 was not that <c>ElasticSearchIndex</c> was hard to
/// test — it was that NOTHING NOTICED it had no cover. #701 fixed two independent copies of the same
/// <c>UpdateSong</c> logic and pinned one; the suite stayed green, and the gap survived on a
/// hand-written list of what had been covered (namely, one entry).
/// <para>
/// So the covered set is compared against a population DERIVED FROM THE ASSEMBLY rather than
/// restated: a third <see cref="ISearchIndex" /> implementation reddens this test until it is
/// given a mutation fixture of its own. That is
/// <c>testing.guard-derives-population-from-source</c> applied to a test population instead of a
/// file population.
/// </para>
/// <para>
/// <b>Scope, stated rather than implied.</b> The derivation is over the assembly that declares
/// both indexers (<c>ErsatzTV.Infrastructure</c>). An implementation added in a DIFFERENT
/// assembly is outside what this sees — it is not covered and this test cannot say so. Both
/// implementations have lived here since the interface existed, so the narrower scope buys a
/// guard that cannot be defeated by an unrelated assembly load order; widening it to every
/// loaded assembly would be the false-confidence version of the same check.
/// </para>
/// </summary>
[TestFixture]
public class SearchIndexMutationCoverageTests
{
/// <summary>
/// The indexers whose <c>UpdateSong</c> is pinned against the ersatztv#701 mutation. Referenced by
/// TYPE, so renaming an indexer or deleting a fixture is a compile error rather than a silent
/// divergence.
/// </summary>
private static readonly Dictionary<Type, Type> CoveredBy = new()
{
[typeof(LuceneSearchIndex)] = typeof(SongIndexerMetadataMutationTests),
[typeof(ElasticSearchIndex)] = typeof(ElasticSongIndexerMetadataMutationTests)
};
[Test]
public void Every_ISearchIndex_Implementation_Has_A_Metadata_Mutation_Fixture()
{
List<Type> implementations = typeof(LuceneSearchIndex).Assembly
.GetTypes()
.Where(t => t is { IsAbstract: false, IsInterface: false })
.Where(t => typeof(ISearchIndex).IsAssignableFrom(t))
.OrderBy(t => t.FullName, StringComparer.Ordinal)
.ToList();
// The set comparison below would ALREADY fail on an empty derivation, because the expected side
// is non-empty -- so this floor is not load-bearing for correctness and saying it is would be a
// false claim about a check. It is a DIAGNOSTIC: it separates "the reflection stopped finding
// types" (a moved type, a renamed interface) from "someone added an indexer", which the set
// comparison alone reports identically.
implementations.Count.ShouldBeGreaterThanOrEqualTo(
2,
"the ISearchIndex population derivation found almost nothing -- this is the derivation "
+ "breaking, not an indexer being added");
implementations.ShouldBe(
CoveredBy.Keys.OrderBy(t => t.FullName, StringComparer.Ordinal),
ignoreOrder: false,
"every ISearchIndex implementation needs an UpdateSong metadata-mutation fixture — see "
+ "ersatztv#824, where a second copy of the same logic went uncovered and an Elastic-only "
+ "reintroduction of `metadata.Artists ??= []` left the whole suite green");
// Comparing the KEYS alone would leave the mapping half-checked: a third indexer could be pointed
// at an EXISTING fixture, or at a fixture class holding no runnable test, and the set comparison
// above would still pass. Both are closed here. What NO static check can establish is that the
// named fixture actually DRIVES its indexer -- that is stated as a residual on this guard rather
// than implied away, and it is why the record claims a third implementation cannot be added
// WITHOUT NOTICE, not that it cannot be mis-covered.
CoveredBy.Values.Distinct().Count().ShouldBe(
CoveredBy.Count,
"two indexers are mapped to the SAME fixture, so one of them is not actually covered");
foreach ((Type indexer, Type fixture) in CoveredBy)
{
// Test-ness is decided by NUnit's INTERFACES, not by a hand-listed set of attribute types.
// A list has to be kept in step with NUnit and silently falsely-reddens whatever it omits --
// an earlier revision listed TestAttribute alone (so a [TestCase]-only fixture failed), then
// three types (so a [Theory] one did). ITestBuilder/ISimpleTestBuilder is what NUnit itself
// dispatches on, so it cannot fall behind the vocabulary.
//
// DeclaredOnly is load-bearing: without it a fixture that merely SUBCLASSES another inherits
// its [Test] and satisfies this while driving the wrong indexer -- and Values.Distinct()
// cannot catch that, since the two Types differ. Static is included because NUnit runs a
// public static test method in a non-static fixture, and omitting the flag would falsely
// redden one.
const BindingFlags TestMethods =
BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly;
static bool NeverRuns(object[] attributes) =>
attributes.Any(a => a is ExplicitAttribute or IgnoreAttribute);
bool declaresARunnableTest = fixture
.GetMethods(TestMethods)
.Select(m => m.GetCustomAttributes(inherit: true))
.Any(attrs =>
attrs.Any(a => a is ITestBuilder or ISimpleTestBuilder) && !NeverRuns(attrs));
declaresARunnableTest.ShouldBeTrue(
$"{fixture.Name} is named as the mutation fixture for {indexer.Name} but DECLARES no "
+ "runnable test method of its own, so it proves nothing");
// The same "wired is not running" failure at fixture level, in its three forms: an abstract
// class NUnit will not instantiate, and [Explicit]/[Ignore]. The indexer population at the top
// of this method already filters IsAbstract; the fixture side needs the mirror of that.
// `IsAbstract` ALONE is wrong here and would have been a false red: a C# `static class`
// compiles to `abstract sealed`, and NUnit runs a test declared in one. What must be rejected
// is an abstract BASE (abstract and NOT sealed), which NUnit cannot instantiate. Deliberately
// no in-repo witness is cited: `AlternateScheduleSelectorTests` is a static class, but it
// merely NESTS its `[TestFixture]`es and declares no test of its own, so it would fail the
// sibling assertion above instead -- naming it here would have been a wrong example attached
// to a right rule.
(fixture.IsAbstract && !fixture.IsSealed).ShouldBeFalse(
$"{fixture.Name} is named as the mutation fixture for {indexer.Name} but is an abstract "
+ "base class, so NUnit never runs it");
NeverRuns(fixture.GetCustomAttributes(inherit: true)).ShouldBeFalse(
$"{fixture.Name} is named as the mutation fixture for {indexer.Name} but is [Explicit] or "
+ "[Ignore]d, so it never runs and proves nothing");
}
}
}
@@ -186,28 +186,6 @@ public class SongIndexerMetadataMutationTests
(await ReadRawArtists(harness, metadataId)).ShouldBeNull();
}
private sealed class ThrowOnWarningLogger<T> : ILogger<T>
{
public Exception? Failure { get; private set; }
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
public bool IsEnabled(LogLevel logLevel) => true;
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter)
{
if (logLevel >= LogLevel.Warning)
{
Failure ??= exception ?? new InvalidOperationException(formatter(state, exception));
}
}
}
private static async Task<object?> ReadRawArtists(InMemoryTvContext harness, int metadataId)
{
await using TvContext context = harness.CreateContext();
@@ -88,7 +88,8 @@ public abstract class ChannelHandlerTestBase
bool showInEpg = false,
string logoPath = "",
string name = "Test",
string group = "ErsatzTV") =>
string group = "ErsatzTV",
StreamingMode streamingMode = StreamingMode.TransportStreamHybrid) =>
new(
name,
number,
@@ -105,7 +106,7 @@ public abstract class ChannelHandlerTestBase
ChannelPlayoutMode.Continuous,
null,
null,
StreamingMode.TransportStreamHybrid,
streamingMode,
null,
null,
string.Empty,
@@ -0,0 +1,37 @@
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Tests.Support;
/// <summary>
/// Captures the first warning-or-worse a component logs, so a fixture can fail on it instead of
/// silently measuring an error path.
/// <para>
/// Both search indexers wrap each <c>Update*</c> body in a <c>catch</c> that logs a warning and
/// assigns <c>metadata.Song = null</c> — which severs a required relationship and cascades the
/// metadata row to <see cref="Microsoft.EntityFrameworkCore.EntityState.Deleted" />. A fixture
/// that let that catch run quietly would report the wrong cause for every assertion after it, and
/// on the first run of the ersatztv#701 probe it did exactly that (a bare
/// <c>ILanguageCodeService</c> substitute NPEs inside <c>AddLanguages</c>).
/// </para>
/// </summary>
public sealed class ThrowOnWarningLogger<T> : ILogger<T>
{
public Exception? Failure { get; private set; }
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
public bool IsEnabled(LogLevel logLevel) => true;
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter)
{
if (logLevel >= LogLevel.Warning)
{
Failure ??= exception ?? new InvalidOperationException(formatter(state, exception));
}
}
}
@@ -1,3 +1,4 @@
using System.ComponentModel;
using ErsatzTV.Application.FFmpegProfiles;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
@@ -13,6 +14,11 @@ public record CreateFFmpegProfileRequest(
string VaapiDisplay,
VaapiDriver VaapiDriver,
string VaapiDevice,
[property: Description(
"Extra surfaces in the QSV upload pool. Must be at least 64 when set; a smaller pool "
+ "leaves no headroom for frames in flight and the transcode writes nothing at all. On update, a "
+ "value equal to the one already stored is accepted unchanged, so a profile written before this "
+ "validation existed stays editable.")]
int? QsvExtraHardwareFrames,
int ResolutionId,
ScalingBehavior ScalingBehavior,
@@ -35,7 +41,16 @@ public record CreateFFmpegProfileRequest(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool? QsvPreferNativeDecoder = null)
bool? QsvPreferNativeDecoder = null,
[property: Description(
"Realtime pacing multiplier for the input. Unset keeps the built-in pacing "
+ "(1.05, or 1.0 for a stream copy). Must be between 1.0 and 2.0 when set.")]
double? ReadRate = null,
[property: Description(
"Rate a lagging realtime input may read at until it is level again. Unset keeps the "
+ "built-in 6.0. Must be between 1.0 and 10.0, and GREATER than the read rate — equal is "
+ "zero headroom, which is functionally no catchup.")]
double? ReadRateCatchup = null)
{
public CreateFFmpegProfile ToCommand() =>
new(
@@ -69,5 +84,7 @@ public record CreateFFmpegProfileRequest(
NormalizeFramerate,
NormalizeColors,
DeinterlaceVideo,
QsvPreferNativeDecoder ?? true);
QsvPreferNativeDecoder ?? true,
ReadRate,
ReadRateCatchup);
}
@@ -1,3 +1,4 @@
using System.ComponentModel;
using ErsatzTV.Application.FFmpegProfiles;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
@@ -13,6 +14,11 @@ public record UpdateFFmpegProfileRequest(
string VaapiDisplay,
VaapiDriver VaapiDriver,
string VaapiDevice,
[property: Description(
"Extra surfaces in the QSV upload pool. Must be at least 64 when set; a smaller pool "
+ "leaves no headroom for frames in flight and the transcode writes nothing at all. On update, a "
+ "value equal to the one already stored is accepted unchanged, so a profile written before this "
+ "validation existed stays editable.")]
int? QsvExtraHardwareFrames,
int ResolutionId,
ScalingBehavior ScalingBehavior,
@@ -35,7 +41,16 @@ public record UpdateFFmpegProfileRequest(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool? QsvPreferNativeDecoder = null)
bool? QsvPreferNativeDecoder = null,
[property: Description(
"Realtime pacing multiplier for the input. Unset keeps the built-in pacing "
+ "(1.05, or 1.0 for a stream copy). Must be between 1.0 and 2.0 when set.")]
double? ReadRate = null,
[property: Description(
"Rate a lagging realtime input may read at until it is level again. Unset keeps the "
+ "built-in 6.0. Must be between 1.0 and 10.0, and GREATER than the read rate — equal is "
+ "zero headroom, which is functionally no catchup.")]
double? ReadRateCatchup = null)
{
public UpdateFFmpegProfile ToCommand(int id) =>
new(
@@ -70,5 +85,7 @@ public record UpdateFFmpegProfileRequest(
NormalizeFramerate,
NormalizeColors,
DeinterlaceVideo,
QsvPreferNativeDecoder ?? true);
QsvPreferNativeDecoder ?? true,
ReadRate,
ReadRateCatchup);
}
@@ -90,7 +90,8 @@ public class DatabaseMigratorService : BackgroundService
await DbInitializer.Initialize(dbContext, stoppingToken);
var fileSystem = scope.ServiceProvider.GetRequiredService<System.IO.Abstractions.IFileSystem>();
await GraphicsElementSeeder.SeedOnNowNext(dbContext, fileSystem, stoppingToken);
await GraphicsElementSeeder.SeedOnNowNext(dbContext, fileSystem, _logger, stoppingToken);
await GraphicsElementSeeder.AttachOnNowNextByDefault(dbContext, stoppingToken);
_systemStartup.DatabaseIsReady();
+51 -1
View File
@@ -25391,6 +25391,7 @@
"null",
"integer"
],
"description": "Extra surfaces in the QSV upload pool. Must be at least 64 when set; a smaller pool leaves no headroom for frames in flight and the transcode writes nothing at all. On update, a value equal to the one already stored is accepted unchanged, so a profile written before this validation existed stays editable.",
"format": "int32"
},
"resolutionId": {
@@ -25478,6 +25479,22 @@
"null",
"boolean"
]
},
"readRate": {
"type": [
"null",
"number"
],
"description": "Realtime pacing multiplier for the input. Unset keeps the built-in pacing (1.05, or 1.0 for a stream copy). Must be between 1.0 and 2.0 when set.",
"format": "double"
},
"readRateCatchup": {
"type": [
"null",
"number"
],
"description": "Rate a lagging realtime input may read at until it is level again. Unset keeps the built-in 6.0. Must be between 1.0 and 10.0, and GREATER than the read rate — equal is zero headroom, which is functionally no catchup.",
"format": "double"
}
}
},
@@ -26485,7 +26502,9 @@
"normalizeFramerate",
"normalizeColors",
"deinterlaceVideo",
"qsvPreferNativeDecoder"
"qsvPreferNativeDecoder",
"readRate",
"readRateCatchup"
],
"type": "object",
"properties": {
@@ -26604,6 +26623,20 @@
},
"qsvPreferNativeDecoder": {
"type": "boolean"
},
"readRate": {
"type": [
"null",
"number"
],
"format": "double"
},
"readRateCatchup": {
"type": [
"null",
"number"
],
"format": "double"
}
}
},
@@ -32016,6 +32049,7 @@
"null",
"integer"
],
"description": "Extra surfaces in the QSV upload pool. Must be at least 64 when set; a smaller pool leaves no headroom for frames in flight and the transcode writes nothing at all. On update, a value equal to the one already stored is accepted unchanged, so a profile written before this validation existed stays editable.",
"format": "int32"
},
"resolutionId": {
@@ -32103,6 +32137,22 @@
"null",
"boolean"
]
},
"readRate": {
"type": [
"null",
"number"
],
"description": "Realtime pacing multiplier for the input. Unset keeps the built-in pacing (1.05, or 1.0 for a stream copy). Must be between 1.0 and 2.0 when set.",
"format": "double"
},
"readRateCatchup": {
"type": [
"null",
"number"
],
"description": "Rate a lagging realtime input may read at until it is level again. Unset keeps the built-in 6.0. Must be between 1.0 and 10.0, and GREATER than the read rate — equal is zero headroom, which is functionally no catchup.",
"format": "double"
}
}
},
@@ -816,7 +816,7 @@
<div key={r.name} style={{ borderTop: i ? "1px solid var(--border-hairline)" : "none", opacity: r.alreadyExists ? 0.55 : 1 }}>
<div style={{ display: "flex", alignItems: "center", gap: 12, padding: "9px 14px" }}>
<Checkbox checked={on} disabled={r.alreadyExists} onChange={() => toggle(r.name)} />
<span style={{ ...mono, minWidth: 42, font: "var(--text-sm) var(--font-mono)", color: "var(--text-secondary)" }}>{r.number}</span>
<span style={{ ...mono, minWidth: 42, textAlign: "right", font: "var(--text-sm) var(--font-mono)", color: "var(--text-secondary)" }}>{r.number}</span>
<Bug name={name} initials={ov.bug && ov.bug.initials} ci={ov.bug && ov.bug.ci} size={28} />
<button type="button" onClick={() => !r.alreadyExists && toggleExp(r.name)} style={{ flex: 1, minWidth: 0, textAlign: "left", background: "transparent", border: "none", padding: 0, cursor: r.alreadyExists ? "default" : "pointer" }}>
<div style={{ font: "var(--weight-semibold) var(--text-sm)/1.2 var(--font-sans)", color: "var(--text-primary)", whiteSpace: "nowrap", overflow: "hidden", textOverflow: "ellipsis" }}>{name}</div>
@@ -816,7 +816,7 @@
<div key={r.name} style={{ borderTop: i ? "1px solid var(--border-hairline)" : "none", opacity: r.alreadyExists ? 0.55 : 1 }}>
<div style={{ display: "flex", alignItems: "center", gap: 12, padding: "9px 14px" }}>
<Checkbox checked={on} disabled={r.alreadyExists} onChange={() => toggle(r.name)} />
<span style={{ ...mono, minWidth: 42, font: "var(--text-sm) var(--font-mono)", color: "var(--text-secondary)" }}>{r.number}</span>
<span style={{ ...mono, minWidth: 42, textAlign: "right", font: "var(--text-sm) var(--font-mono)", color: "var(--text-secondary)" }}>{r.number}</span>
<Bug name={name} initials={ov.bug && ov.bug.initials} ci={ov.bug && ov.bug.ci} size={28} />
<button type="button" onClick={() => !r.alreadyExists && toggleExp(r.name)} style={{ flex: 1, minWidth: 0, textAlign: "left", background: "transparent", border: "none", padding: 0, cursor: r.alreadyExists ? "default" : "pointer" }}>
<div style={{ font: "var(--weight-semibold) var(--text-sm)/1.2 var(--font-sans)", color: "var(--text-primary)", whiteSpace: "nowrap", overflow: "hidden", textOverflow: "ellipsis" }}>{name}</div>
@@ -56,7 +56,71 @@
{ label: "loudnorm", value: "LoudNorm" },
];
function Row({ label, help, control = 320, first = false, children }) {
// Level-2 explainer copy for the field-level progressive-disclosure pattern (ersatztv#734).
// Mirrors web/src/screens/FFmpegProfilesScreen.tsx's FIELD_HELP record; see
// docs/spa-conventions.md §15 for the contract this shape is mirroring.
const FIELD_HELP = {
threadCount:
"Caps the worker threads FFmpeg uses per transcode. 0 lets FFmpeg decide, which is usually right; a low fixed value keeps one channel from starving the others on a busy host, at the cost of falling behind realtime on heavy content.",
scalingBehavior:
"Decides what happens when the source aspect ratio does not match the preferred resolution. Scale and Pad keeps the whole picture and adds bars; Crop fills the frame and cuts whatever overflows; Stretch fills it by distorting the image, which is why it is rarely what you want.",
videoBitrate:
"Target output bitrate. Too low and the encoder throws away detail on motion; too high and clients on slow links buffer. Buffer size is the companion setting — it bounds how far the encoder may deviate from this target.",
videoBufferSize:
"How much bitrate deviation the encoder may bank before it has to correct. Roughly 2x the bitrate is the usual starting point. Very small values force a near-constant bitrate and hurt quality on scene changes.",
hardwareAcceleration:
"Offloads decode and encode to the GPU. The list only offers what this FFmpeg build supports, so an unsupported kind never appears here. What nothing checks when you save is whether the device itself is present and passed through to the container — that is the mismatch that fails at playback time.",
normalizeLoudnessMode:
"Levels volume across content from different sources. Off leaves each item at its own level, so volume jumps between them; loudnorm retargets everything to one integrated loudness, which evens that out at the cost of an extra filter in the graph and a less faithful dynamic range.",
};
// Mockup of the shared `FieldHelp` primitive (web/src/components/fieldHelp.tsx). The prototype
// reproduces the two opening signals that read in a static review (tap-pins and hover); the shipped
// component additionally handles keyboard focus, Escape, outside-press dismissal, the hover
// bridge, and portalling the panel out of the card's clip.
function FieldHelp({ label, detail }) {
const [pinned, setPinned] = React.useState(false);
const [hovered, setHovered] = React.useState(false);
const open = pinned || hovered;
return (
<span
style={{ position: "relative", display: "inline-flex", verticalAlign: "middle", marginLeft: 5 }}
onMouseEnter={() => setHovered(true)}
onMouseLeave={() => setHovered(false)}
>
<button
type="button"
aria-label={`More about ${label}`}
aria-expanded={open}
onClick={(e) => { e.preventDefault(); setPinned((v) => !v); }}
style={{
display: "inline-flex", alignItems: "center", justifyContent: "center",
width: 18, height: 18, border: "none", borderRadius: "var(--radius-xs)",
background: "none", padding: 0, color: "var(--text-disabled)", cursor: "help",
}}
>
<Ico.Info aria-hidden="true" size={13} />
</button>
{open && (
<span
role="note"
style={{
position: "absolute", zIndex: 60, left: 0, top: "calc(100% + 7px)",
width: "max-content", maxWidth: 320,
border: "1px solid var(--border-control)", borderRadius: "var(--radius-sm)",
background: "var(--ctv-surface-3)", boxShadow: "var(--shadow-pop)",
padding: "9px 11px", color: "var(--text-primary)",
font: "var(--text-xs)/1.5 var(--font-sans)", whiteSpace: "normal", textAlign: "left",
}}
>
{detail}
</span>
)}
</span>
);
}
function Row({ label, help, detail, control = 320, first = false, children }) {
return (
<div
style={{
@@ -68,7 +132,10 @@
}}
>
<div style={{ flex: "1 1 auto", minWidth: 0, paddingTop: 5 }}>
<div style={{ font: "var(--weight-medium) var(--text-sm)/1.3 var(--font-sans)", color: "var(--text-primary)" }}>{label}</div>
<div style={{ font: "var(--weight-medium) var(--text-sm)/1.3 var(--font-sans)", color: "var(--text-primary)" }}>
{label}
{detail && <FieldHelp detail={detail} label={label} />}
</div>
{help && <div style={{ marginTop: 3, font: "var(--text-xs)/1.4 var(--font-sans)", color: "var(--text-secondary)", maxWidth: 460 }}>{help}</div>}
</div>
<div style={{ flex: `0 0 ${control}px` }}>{children}</div>
@@ -177,7 +244,7 @@
<Row first label="Name" control={360}>
<Input size="sm" value={isAdd ? "" : profile.name} placeholder="e.g. 1080p30 Software" />
</Row>
<Row label="Thread count" control={200}>
<Row label="Thread count" control={200} detail={FIELD_HELP.threadCount} help="0 lets FFmpeg pick a thread count.">
<Input size="sm" type="number" trailing="threads" value="0" />
</Row>
<Row label="Normalize audio" control={200}>
@@ -191,7 +258,7 @@
</Row>
{normalizeVideo && (
<React.Fragment>
<Row label="Scaling behavior" control={360}>
<Row label="Scaling behavior" control={360} detail={FIELD_HELP.scalingBehavior} help="What to do when the source aspect ratio does not match.">
<Select value="ScaleAndPad" options={SCALING_OPTIONS} />
</Row>
<Row control={360} label="Pad mode" help="Hardware padding only applies with VAAPI; otherwise software padding is used.">
@@ -219,13 +286,13 @@
<Row label="Bit depth" control={360}>
<Select value="EightBit" options={BIT_DEPTH_OPTIONS} />
</Row>
<Row label="Bitrate" control={220}>
<Row label="Bitrate" control={220} detail={FIELD_HELP.videoBitrate}>
<Input size="sm" type="number" trailing="kBit/s" value="2000" />
</Row>
<Row label="Buffer size" control={220}>
<Row label="Buffer size" control={220} detail={FIELD_HELP.videoBufferSize} help="Usually about twice the bitrate.">
<Input size="sm" type="number" trailing="kBit" value="4000" />
</Row>
<Row label="Hardware acceleration" control={360}>
<Row label="Hardware acceleration" control={360} detail={FIELD_HELP.hardwareAcceleration} help="Requires the device to be passed through to the container.">
<Select value={hwaccel} options={HWACCEL_OPTIONS} onChange={(e) => setHwaccel(e.target.value)} />
</Row>
{hwaccel === "Vaapi" && (
@@ -290,7 +357,7 @@
<Row label="Sample rate" control={220}>
<Input size="sm" type="number" trailing="kHz" value="48" />
</Row>
<Row label="Normalize loudness" control={360}>
<Row label="Normalize loudness" control={360} detail={FIELD_HELP.normalizeLoudnessMode}>
<Select value={loudness} options={LOUDNESS_OPTIONS} onChange={(e) => setLoudness(e.target.value)} />
</Row>
{loudness === "LoudNorm" && (
+31 -1
View File
@@ -9,7 +9,37 @@ WORKDIR /source
COPY design-system/. ./design-system/
COPY web/. ./web/
WORKDIR /source/web
RUN npm run lint && npm run typecheck && npm test -- --run && npm run build
# The SPA suite runs here except for two files (ersatztv#819), excluded for OVERLAPPING reasons —
# one needs the git binary, the other needs the binary AND a checkout. Reading them as two separate
# reasons is what broke this stage once already:
# * `web/src/api/pageSizeCallSites.guard.test.ts` needs a git CHECKOUT — and, through it, the
# binary. It derives its file population from `git ls-files` rather than a directory walk, and
# refuses rather than falling back. This stage's context is `web/` + `design-system/` only, so
# there is no `.git`.
# * `web/vite-plugins/trackedSourceFiles.realgit.test.ts` needs the BINARY but no checkout: it
# builds its own temp repository to prove the derivation by executing it.
# `node:22-bookworm-slim` ships no git (`command -v git` -> not found), so it dies with
# `spawnSync git ENOENT`.
# So this is NOT checkout-versus-binary, and adding a `COPY .git` would not let either run here —
# the binary would still be missing. Excluding only the first is not enough either, and a replica
# that merely deletes `.git` cannot show that: verify any change here with the git binary off
# `PATH`, not just with the directory absent.
# Everything else — all but those two files — runs fine gitless and is kept, rather than dropping
# the whole suite for one file as an earlier fix here did.
# The excluded pair is not skipped overall: `docker-build.yml`'s `Build & test (.NET)` job runs the
# whole suite on a real checkout, and `build` (the job that invokes this Dockerfile) carries
# `needs: [test, migrations, scan]`. State that chain precisely, because the `needs:` edge is not
# all of it: `Test SPA` is also gated on `docs_only` and on the #420 revalidate skip, and `build` is
# not gated on `revalidate`. On a push whose tree is byte-identical to an already-green head the
# suite is skipped and the image still builds — carried there by #420's byte-identical-tree
# argument. The `docs_only` arm cannot ship an image at all (`Build and push` is gated on it too),
# and `ci-detect-docs-only.sh` classifies by PATH SHAPE (`docs/` or `*.md`), not by directory, so a
# `web/*.md` would count as docs — there are none today, but do not restate it as "any `web/**`".
RUN npm run lint && npm run typecheck && \
npm test -- --run \
--exclude 'src/api/pageSizeCallSites.guard.test.ts' \
--exclude 'vite-plugins/trackedSourceFiles.realgit.test.ts' && \
npm run build
FROM --platform=linux/amd64 192.168.1.95:3000/timothy/ersatztv-ffmpeg:8.1.2 AS runtime-base
COPY --from=dotnet-runtime /usr/share/dotnet /usr/share/dotnet
+20 -9
View File
@@ -20,20 +20,26 @@ doc below, or that changes which sections a task signal points to.**
| Named-issue pickup | Skip queue selection; go straight to focused retrieval — see "Knowledge retrieval" below, then the issue body |
| Adding/changing a `/api/*` endpoint | `docs/api-conventions.md` checklist + `docs/endpoint-index.md` |
| Adding a ChicoryTV SPA screen | `docs/spa-conventions.md` |
| Explaining a consequential settings field in the SPA (summary → hover/tap panel → docs link) | `docs/spa-conventions.md` §15 — use the shared `FieldHelp` trigger and put the copy in the screen's own `FIELD_HELP` record; the icon, the gesture and the a11y contract are fixed |
| Graphics element / overlay work (text bug, On Now / Next, watermark-vs-`[vge]`) | `docs/graphics-elements.md`, then decisions catalog rows keyed `graphics.*` |
| Scheduling / playout engine work | `docs/domain-model.md` + decisions catalog rows keyed `sched.*` (`docs/decisions/README.md`) |
| Adding or changing a paged list handler (a page plus a `TotalCount`) | Resolve `api.paged-count-matches-page-query` via `docs/decisions/README.md` — for an EF-backed filtered list, count the SAME query you page, with includes appended to the page chain only; where the count and the page are separate methods, a test pins their agreement. Then `api.paging-zero-based` for the `pageNum`/`pageSize` contract |
| Concurrency / optimistic-locking work | `docs/api-conventions.md` §7a/b/c + `docs/decisions/optimistic-concurrency.md` |
| Auth / security-surface work | `docs/decisions/api-auth-security.md` |
| CI / release pipeline work | `docs/ci-cd.md` + `docs/decisions/release-ci-governance.md` |
| Proposing a new guard / CI check / regression test convention | `docs/defect-shapes-773.md` §4 (detector menu + the classes where no detector is plausible), then the three rules every guard must satisfy: `docs/decisions/records/testing/guard-derives-population-from-source.md`, `…/guard-ships-with-mutation-proof.md` and `…/mutation-claims-are-executed.md` (a `MUTATION` grade carries a DECLARED clause mutation that is re-run every suite) |
| Proposing a new guard / CI check / regression test convention | `docs/defect-shapes-773.md` §4 (detector menu + the classes where no detector is plausible), then the rules every guard must satisfy: `docs/decisions/records/testing/guard-derives-population-from-source.md`, `…/guard-ships-with-mutation-proof.md` and `…/mutation-claims-are-executed.md` (a `MUTATION` grade carries a DECLARED clause mutation that is re-run every suite) — plus `…/verification-code-needs-its-own-proof.md`, which extends the same obligation BEYOND guards to the harness, wrapper or checker doing the checking, and says where its proof lives when the checker holds no row |
| Adding or bounding a consequential numeric config field (an FFmpeg profile tunable, a pipeline knob) | `docs/api-conventions.md` §3d — reject out of range with a 422 naming the bound and its consequence, never accept-then-rewrite; validate against the constants the renderer reads, keep the render-time clamp for pre-existing rows, and let an UNCHANGED legacy value through on update. Then `api.ffmpeg-profile-numeric-bounds` |
| Testing a surface gated by config / an env var / a credential | `docs/decisions/records/testing/deny-path-at-production-config-value.md` — cover the setting absent, at its production value, and each opt-out, and assert the DENY branch |
| Touching a full-replace write path or a hand-built request object | `docs/decisions/records/testing/full-replace-asserts-field-list.md` — derive the field list from the DTO and assert set equality; reconcile by id where child state exists. In the SPA the same rule is enforced by the type system: `docs/spa-conventions.md` §4b — build the body as `Complete<T>`, annotating BOTH the wrapper parameter and every construction site |
| Writing or editing any doc, or answering a review finding in prose | `docs/decisions/records/docs/no-session-narrative.md` — the doc records the END STATE; the path to it goes in the commit message. Apply the who-benefits test, and read the carve-out before you cut (dated measurements, stated snapshot boundaries and tested-and-rejected results stay) |
| Adding, renaming or removing a workflow JOB | `docs/ci-cd.md` → "Per-job declarations" — every job declares `env.CI_JOB_ROLE` (and, in `docker-build.yml`, `env.CI_EXECUTION_CLASS`); a missing or unknown value fails `scripts/tests/test_workflow_job_guards.py` / `…/test_ci_image_pin_population.py`, and a `guard` **or `report-only`** job also needs a row in `docs/guard-inventory.md` → "Workflow-job guards". Rationale: `docs/decisions/records/testing/workflow-declares-its-own-job-metadata.md` |
| Adding / changing / deleting a guard file | `docs/guard-inventory.md` — every guard's row is machine-checked by `scripts/tests/test_guard_inventory.py`, so a new guard must acquire a row before the suite goes green, and a row graded `MUTATION` must also acquire a declared clause in `scripts/tests/mutation_manifest.py` |
| Writing code that reads live Gitea/remote state and then acts on it | `docs/decisions/records/process/check-and-use-pins-a-version.md`, then `docs/remote-state-inventory.md` — a new executable under `scripts/` (**excluding `scripts/tests/`**), `.claude/hooks/`, `.husky/` or `.gitea/workflows/` must acquire a row there before `scripts/tests/test_remote_state_inventory.py` goes green |
| Finding every site that references a symbol (multi-site fix/sweep) | `docs/local-lsp-tooling.md` — which surface answers, and why a delegated agent must be pointed at the `csharp-lsp` MCP tools rather than the `LSP` tool |
| Finding every site that references a symbol (multi-site fix/sweep) | `docs/local-lsp-tooling.md` — which of the three surfaces answers, and why a delegated agent must be pointed at an MCP server (`csharp-lsp`, or `serena` after an `activate_project`) rather than the `LSP` tool, which no subagent has been observed to reach |
| Live local run / Playwright-MCP verification | `docs/e2e-local.md` + `scripts/e2e-local.sh` |
| Adding/changing a UI-E2E browser flow | `docs/e2e-local.md` → "UI-E2E harness" + `scripts/e2e-ui.sh` |
| What does a test suite cover | `docs/testing.md` |
| Writing a test whose behaviour is PROVIDER-SPECIFIC (collation, a value converter, data-migration DML) | `docs/testing.md` → "Provider-parity fixtures (opt-in MySQL)" — run one fixture body against both providers via `ETV_TEST_MYSQL_CONNECTION`; without it the MySQL arm `Assert.Ignore`s visibly, and CI does not currently run it (ersatztv#627) |
| Legacy Blazor route lookup | `docs/blazor-route-parity.md` (historical #91 phase (b) inventory) |
| "Why do we do X this way" / challenging a convention | **Catalog-first**: `docs/decisions/README.md` (active rows) → follow the row's link to `docs/decisions/records/<area>/<topic>.md` for full rationale. `docs/decisions/archive/<area>/` only for "what did the rule used to be." |
@@ -73,11 +79,11 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K
- **`docs/spa-conventions.md`** — playbook for adding a screen to the ChicoryTV React SPA.
- **`docs/e2e-local.md`** (+ `scripts/e2e-local.sh`) — how to run a live local instance for manual
or Playwright-MCP verification.
- **`docs/local-lsp-tooling.md`** — the code-intelligence surfaces (the `LSP` tool's three servers and
the `csharp-lsp` MCP server): how each is configured, which one a **subagent** can actually reach,
the traps (a cold server answers the first query with a confidently partial result), and
`scripts/check-local-lsp.sh` to verify the preconditions. Read before briefing an agent to find
every site referencing a symbol.
- **`docs/local-lsp-tooling.md`** — the code-intelligence surfaces (the `LSP` tool's three servers,
the `csharp-lsp` MCP server, and `serena`): how each is configured, which ones a **subagent** can
actually reach, the traps (a cold server answers the first query with a confidently partial result;
serena needs an `activate_project` per directory), and `scripts/check-local-lsp.sh` to verify the
preconditions. Read before briefing an agent to find every site referencing a symbol.
- **`docs/testing.md`** — testing map: what each `*.Tests` project / `web` suite covers,
golden-file nets, the timezone-independence rule, how to run subsets, the per-PR verification
gate.
@@ -98,6 +104,10 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K
Largely superseded day-to-day by `docs/api-conventions.md`; read this for the original rationale.
- **`docs/mcp.md`** — the `ErsatzTV.Mcp` stdio JSON-RPC MCP server (#58): how it wraps `/api/v1` as
read + cautious-write tools, its config/env vars, auth, security posture, and the tool catalog.
- **`docs/graphics-elements.md`** — graphics element (overlay) schema reference: how elements are
discovered and attached, the YAML parsing traps (an unknown key disables the element outright), the
full text-element field table including the #732 background box, and why `[vge]` in a filter graph
does not imply a graphics element is bound.
- **`docs/channels.md`** — Channel entity field reference.
- **`docs/m3u-xmltv.md`** — M3U/XMLTV generation overview (`ChannelPlaylist`, `GetChannelGuideHandler`).
- **`docs/fork-strategy.md`** — divergence policy vs upstream ErsatzTV.
@@ -127,8 +137,9 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K
detector, since the class has no plausible linter (`docs/defect-shapes-773.md` §4 detector D).
- **`docs/guard-inventory.md`** — every executable guard file, what it blocks, whether it is a
`GUARD` or `TOOLING`, and whether it ships a mutation proof (`MUTATION` / `BEHAVIOUR-ONLY` /
`NONE`) with a `file::function` ref. The population is derived from the filesystem and the
workflow/hook call sites and compared for set equality by `scripts/tests/test_guard_inventory.py`,
`NONE`) with a `file::function` ref. The population is derived from the GIT INDEX (not a
filesystem walk, since ersatztv#806) and the workflow/hook call sites, and compared for set
equality by `scripts/tests/test_guard_inventory.py`,
so a new guard cannot ship unclassified and a renamed test cannot leave a row claiming coverage it
has lost. Guards implemented inline in workflow YAML are deliberately outside that population —
the doc states the limit rather than implying coverage.
+46
View File
@@ -54,6 +54,16 @@ Exemplars:
members, 1000 `search/all-items`). `OpenApiPagingContractTests` pins this and names the expected
set of paged operations, so a new paged endpoint fails until it is added there **with**
descriptions. See `api.paging-zero-based`.
**The handler behind it must compute its total from the SAME query it pages.** Build one
`IQueryable`, apply every filter to it, then take both `CountAsync` and the page from that object —
never `dbContext.<Set>.CountAsync(ct)` beside a separate page query, and never a second
`CountAsync(pred, ct)` restating the predicate. Take the shape even before the first filter exists —
a handler with no filter today is where the drift is introduced tomorrow. The drifted state is silent and shaped like working
software: the page is right, the total is wrong, and the client believes the total, so the SPA
paginates to pages that can never fill and an MCP caller pages toward a completeness target it
cannot reach. Eager-loading `.Include(...)` belongs on the page chain only, appended after the
count — a `COUNT` does not materialize the graph. See `api.paged-count-matches-page-query`
(ersatztv#690, #758).
- **Sortable GET with allow-listed sort params**: same file — `sortField`/`sortDirection` are
normalized against a fixed allow-list (`AllowedSortFields`) rather than trusted or rejected with
a 422: an unrecognized `sortField` silently falls back to the default field, an unrecognized
@@ -337,6 +347,42 @@ and "synced". See `docs/decisions.md` 2026-07-11 (#202) for the fuller rationale
bug this pattern corrected (Blazor's Plex `Unlock` ordering released the library lock before a
dependent second message ran).
### 3d. Bound a consequential numeric field with a 422 — never accept-then-rewrite
A write path that accepts an out-of-range number, stores a *different* one and returns `200` teaches
the caller nothing and leaves the stored config no longer describing the behavior: the SPA keeps
rendering what was typed while the pipeline uses the substitute, and a machine client that `PUT`s a
value reads back another. **Validate to the documented range and return 422 naming the bound**, with
the consequence of exceeding it in the message ("…leaves the QSV upload pool with too little headroom
and the transcode writes nothing at all"), so the error teaches the bound instead of hiding it.
Three rules that come with it (exemplar: `ErsatzTV.Application/FFmpegProfiles/FFmpegProfileBounds.cs`,
ersatztv#735):
- **Put the constants where the renderer reads them, and validate against those** — `FFmpegState`
owns `MinimumQsvExtraHardwareFrames`, `Minimum/MaximumReadRate` and the defaults, and the
write-path validator reads those symbols rather than restating numbers. **The SPA cannot: it
restates each bound as a literal** (`web/src/screens/ffmpegProfileDraft.ts`), and nothing pins the
two together, so raising a server bound leaves every test green while the form keeps enforcing the
old one. Mirror the value AND the wording of the server's message, and treat the drift as a known
residual rather than assuming the literal is checked.
- **Keep the render-time clamp as well.** It is what makes the change migration-free: rows written
before the validation existed, or out of band, still cannot reach FFmpeg unbounded. Validation is
the primary guard; the clamp is belt-and-braces, and both need a test.
- **On update, reject a NEWLY submitted out-of-range value, not an unchanged legacy one.** The SPA
sends the whole profile back on every edit, so rejecting a stored-but-out-of-range value would
make an old row uneditable over a field the operator never touched — and, when the field is
conditionally rendered, cannot even see. Compare against the stored value and let an unchanged one
through — **and mirror the exemption in the client**, or the form blocks a save the server would
have accepted and the row is uneditable in the surface that matters (`validate(draft, stored)` in
`ffmpegProfileDraft.ts`; the add/copy path passes no stored draft and stays strict, matching the
create handler's `stored: null`).
`null` keeps meaning **unset**, resolved by the renderer to the value it used before the field was
configurable — never materialized into a stored number on save, so an untouched profile behaves
identically. Document the range in the schema with `[property: Description("…")]` on the request
record's positional parameter (`System.ComponentModel`); it renders into `v1.json`.
## 4. Artwork contract
API response DTOs return **rooted, directly-usable artwork URLs** — e.g. `/artwork/posters/...`,
+3 -4
View File
@@ -49,9 +49,8 @@ parameterized routes. Together they cover every SPA-ready route (all rows former
| `/settings/ui` | `/app/settings/general` |
| `/settings/xmltv` | `/app/settings/xmltv` |
Note: **the settings sub-routes above are already redirected**, not Blazor-only as an earlier draft
of this table implied — `LegacyUiRedirects.cs` covers all seven `/settings/*` pages that have SPA
equivalents. There is no Blazor-only settings sub-route left except the ones with no SPA screen at
Note: **the settings sub-routes above are already redirected**`LegacyUiRedirects.cs` covers all
seven `/settings/*` pages that have SPA equivalents. There is no Blazor-only settings sub-route left except the ones with no SPA screen at
all (there are none currently — every Blazor `Settings/*.razor` page has both an SPA screen and a
redirect).
@@ -227,7 +226,7 @@ Nothing remains here. The playback troubleshooting screen (`/app/troubleshooting
### Media sources — #202 DONE (2026-07-11)
Nothing remains here. Found by the #91 cold review (2026-07-09): the SPA `/app/libraries` screen
Nothing remains here. As of the #91 sweep (2026-07-09): the SPA `/app/libraries` screen
listed sources/libraries and could trigger scans, but source add/edit, connection parameters,
path replacements, and library enable/disable had **no SPA UI and no write REST API**
(`MediaSourcesController` was GET-only; "Add Source" was a hardcoded disabled stub) — a fresh
+672 -86
View File
@@ -147,6 +147,272 @@ A backup, fetch, or migration-smoke failure aborts before the live container is
**Also avoid firing several pushes back-to-back** (e.g. a `[skip ci]` commit, then `main`, then a tag, all within ~1s). Observed once on this Gitea instance: the later events were silently dropped — no `ActionRun` records created at all, even though the runner was online and the workflow `active`. Pushing again, spaced out, created the runs normally. If a push/tag doesn't produce a run, re-push (or push an empty commit) rather than assuming the runner is broken.
## Workflow token scope (`permissions:`, ersatztv#748)
Every workflow in this repo declares `permissions:`. Five declare `code: read` at the top level;
`review-verdict.yml` declares `code: write` + `issues: read` + `pull-requests: read` on its
`set-verdict-status` job and nothing at the top level. That is the whole convention — a new workflow
should declare the narrowest set it needs, and say at the declaration why it needs write if it does.
**Declaring `permissions:` is exhaustive, not additive.** A unit omitted is a unit not granted, and
that holds *at any owner-level default* — it is not conditional on the instance default being
Restricted. This is the property that makes the block load-bearing rather than decorative, and it is
also what makes a wrong one dangerous: the gate job's API calls are `curl -sf` under
`set -euo pipefail`, so a missing unit is a 403 that kills the step, and `review-verdict/h10` is
never posted. That blocks **every** merge to `main`, including the PR that would repair it, and
since ersatztv#743 (`block_admin_merge_override: true`) there is no force-merge escape.
`code: write` and not something narrower because **Gitea has no `statuses` scope**.
`POST /repos/{o}/{r}/statuses/{sha}` is gated by `reqRepoWriter(unit.TypeCode)`, so status-write *is*
Code-write (`ci.actions-credential-scoping`). `contents: write` would also work but additionally
grants `releases: write`, which nothing here uses. The two read units are not spare: the job GETs
`/issues/{n}/timeline` (the ersatztv#706 retarget fence) and, through
`scripts/pr-changed-files.sh`, `/pulls/{n}` and `/pulls/{n}/files` (the ersatztv#698 exemption path).
No workflow needs a `packages:` unit. All five `container:` blocks in `docker-build.yml` carry
explicit `credentials:` using the scoped `REGISTRY_*` PAT, so no job pulls the toolchain image with
the injected token.
### The binding was measured, not assumed (2026-08-27, Gitea 1.27.1)
`ci.actions-credential-scoping` recorded for three weeks that whether `permissions:` is honored on
this instance was **unprobed** — 1.26.0 was the version it was said to start binding at, from
release notes rather than from an experiment. It was settled with the scratch-base probe below, run
as a matched pair that differed **only** in one unit:
| Control | Gate job unit | Result |
|---|---|---|
| Positive | `code: write` | job `success`; `Posted review-verdict/h10-PROBE=success`; `creator: null` |
| Negative | `code: read` | job `failure`; `curl` exit 22 (HTTP ≥400 under `-f`) at the POST; **no status written** |
Both runs reached the classify step's start marker, so neither failure was the ersatztv#751
dropped-step shape. What shows each read unit was exercised is a different observation for each, and
they are not interchangeable:
- **`issues: read`** — both runs logged `Retarget fence: 0 retarget event(s) … (trusted=yes)`, which
requires the timeline walk to have read `/issues/{n}/timeline`. That log line and the function
behind it were renamed by ersatztv#803, which added the head axis to the same walk: the line now
reads `Mutation fence: N retarget event(s) and M push event(s) … (trusted=…)` and the function is
`count_pr_mutations`. The observation above is left as it was recorded — it is what those runs
printed — but grep for the new strings when reproducing it.
- **`pull-requests: read`** — the positive control posted `=success`, and a `success` is reachable
only through `complete=yes`, which requires `scripts/pr-changed-files.sh` to have paged
`/pulls/{n}` and `/pulls/{n}/files` to a validated end. The fence and the start marker do **not**
show this; the `success` does.
The negative control is the load-bearing half: it is what shows the declaration *binds* rather than
being a no-op that happens to sit next to a permissive default. Both controls carried the block **at
job level**, on `set-verdict-status` — the same level and the same job as the shipped
`review-verdict.yml`, so the gate's own shape is what was measured.
Both ran while the owner default was still `permissive`, so the binding holds **today**, without any
flip. One scope limit, stated rather than glossed: the other five workflows declare `permissions:` at
**top level**, a form the probe did not exercise. That is inferred, not measured — but the failure
direction is safe (a top-level key Gitea ignored would be a security no-op, never a brick), and
`docker-build.yml` and `pr-checks.yml` are `on: pull_request`, so this change's own CI exercises
theirs.
**What the probe does NOT establish — read this before flipping the owner default.** It says nothing
about what a declared `code: write` receives under a **Restricted** default. GitHub's semantics let
`permissions:` only *narrow*, never widen past the default; if Gitea copies that, Restricted would
cap the gate job at read and `review-verdict/h10` would become unwritable — precisely the failure
this section warns about, with no force-merge escape. Both controls ran under `permissive`, so that
case is untested. server-management#714 stays open for exactly this: re-run the scratch-base probe
**under Restricted** before trusting the flip.
`creator: null` on the positive control matters separately: it is the provenance signal
`review-verdict.yml` relies on to tell a machine-written status from a reviewer's
(`ci.exemption-provenance`, ersatztv#742). Approach A — scoping the built-in token — preserves it.
Moving status writes onto a PAT (approach B) would not, which is why B stays blocked.
**What this does and does not close.** Scoping bounds what a job may *do*; it never bounds whether
attacker YAML *runs* — that needs a trigger filter (ersatztv#744). And `review-verdict.yml` still
holds repo-write. The property gained is not "workflows can no longer assert the verdict"; it is
that *only a workflow visibly opting into repo-write can*, instead of every job on the instance
holding write implicitly. Real, and weaker than closed.
⚠️ **Changing `review-verdict.yml`'s block requires the scratch-base probe**, not CI-green on the
editing PR — that workflow is base-resolved and is never exercised by its own PR, so its own checks
say nothing about the edit. See the probe recipe in the review-verdict gate section below.
The other five split two ways, and only one half is covered by an editing PR's own CI:
- `docker-build.yml` and `pr-checks.yml` run on `pull_request`, so an editing PR exercises their
blocks directly. (`docker-build.yml`'s `build` job carries
`if: github.event_name != 'pull_request'`, so its registry-push path is first exercised on the
post-merge push to `main` — that reddens `main`, not the gate, and is recoverable.)
- `ci-image.yml` (`push` to `main` only since ersatztv#744, plus `schedule`/`workflow_dispatch`),
`dependency-scan.yml` and `renovate.yml` are **not** exercised by an editing PR at all — their
first run is after merge or on the next schedule. None is a required check, so a wrong block there reddens a non-blocking run rather than the gate, but do not read an
editing PR's green as covering them.
**`ci-image.yml` is included, and only just.** It was the one workflow this change could not
originally reach: editing it re-pointed `ci-image-pin`'s `expected` at the editing commit and
reddened a **blocking** job, and its own `paths:` made the edit publish a toolchain image. ersatztv#748 briefly
carried a documented one-name exemption for it. ersatztv#744 (`ci.toolchain-image-publish-is-a-dispatch`)
took that path out of both, so the exemption was **deleted rather than documented** — which is the
form ersatztv#835 asked for, and it is why `test_pr_changed_files.py` asserts over the whole derived
population with nothing carved out.
## Checkouts drop the persisted credential (ersatztv#746)
**Every** `actions/checkout` in this repo sets `persist-credentials: false` — with no exemption,
since ersatztv#744 removed the one that existed (see "The last exemption, and how it closed" below),
and ersatztv#835 now holds the rule mechanically rather than by convention:
`scripts/tests/test_workflow_persist_credentials.py` derives the workflow population from the git
index and asserts the flag on every checkout step it finds. Without it the action leaves an
`Authorization` header in
`.git/config`, and a head-resolved job then hands that credential to every script it runs, not just
to the checkout. That header is write-capable *only while the job's token is*: this instance's Actions default
token permission is `permissive`, where it stands after a Restricted trial was reverted on
2026-08-05 — but since ersatztv#748 that default no longer decides it. Every workflow declares
`permissions:` (see the section above), so every persisted header in this repo is read-only by
declaration. Re-applying the owner-level default is server-management#714. No job that checks out
loses anything: none of them push with git, and anonymous access covers the fetches because the repo
is public — `private: false`, and `GET /info/refs?service=git-upload-pack` answers unauthenticated,
measured 2026-08-26 from the LAN. That measures the repository, not a runner's network path to it,
and the runner path is deliberately not asserted here: it does not need to be, because the fetches
in the jobs below are unmasked: an anonymous fetch that failed from a runner reddens `api-docs`,
`Formatting` and `decisions lifecycle` instead of being swallowed. (The two masked fetches further
down are covered by their own argument, not by this one.) Renovate does push branches, but
`renovate.yml` has no `actions/checkout` step at all and carries its own `RENOVATE_TOKEN`.
The base-ref fetches that feed the changed-file skip logic — in `docs-reminder`, `decisions
lifecycle`, `api-docs` and `Formatting` — therefore run **unmasked**: a failed `git fetch` — and a failed
`git diff` at the three steps that run one inline — exit the step with an `::error::` instead of
yielding an empty changed set. That ordering is the general rule, not a fact about one
change: unmask a dependent fetch *before* you drop a persisted credential anywhere. Under `|| true`
a credential regression looked like "nothing changed" on the three steps that also masked
`git diff`, so those checks skipped themselves silently on a green job; on the other two it handed
a possibly-absent `origin/<base>` to a Python validator. Neither shape is one you want a credential
change to land on top of.
**What this does and does not make fail-closed.** A failed *fetch* now reddens `api-docs`,
`Formatting` and `decisions lifecycle`, which is the credential/network case this change exists for.
It does not make the two validator steps fail-closed in general: they run no inline `git diff`, and
if the base resolves badly despite a successful fetch, `scripts/decisions_validate.py` emits a
`::warning::` and skips its diff-dependent checks while `scripts/check-doc-narrative.py` prints
`SCANNED NOTHING` and exits 0. Both fail open there by their own design and are out of scope here —
do not read a green `decisions lifecycle` as proof its diff-dependent checks ran. Note which side of
that line the `Decisions-Edit: yes` trailer check falls on: `_edit_trailer_armed` is reached only
after the merge base resolves, so it goes quiet with the rest of them rather than surviving.
**Loud is not the same as blocking, and the split is by job.** `api-docs`, `Formatting` and
`decisions lifecycle` gate on this diff, so a failed fetch reddens them. `docs-reminder` does not —
it is a non-blocking nudge, and an advisory red still joins the combined status the merge gate
reads. All three of its non-checkout steps therefore carry `continue-on-error: true` — the parity
nudge, `Set up Python` and the doc-narrative warning — so the `::error::` reaches the log without
turning a warning into a merge block. `Checkout` is deliberately still blocking; with no tree there
is nothing to warn about.
**None of these fetches passes `--depth`.** All five run in jobs that check out `fetch-depth: 0`,
and `git fetch --depth=N` on a *complete* clone grafts it shallow: the history is still there but
cut off, so `git diff origin/<base>...HEAD` fails with `no merge base` once the branch point is
further back than N. Measured 2026-08-26: a 150-commit repo branched 130 back, `--depth=100`,
`.git/shallow` created, three-dot diff `fatal: no merge base`; the same fetch without `--depth`
returns the correct file list. Under the old `|| true` that surfaced as a silent skip; unmasked it
would have reddened `api-docs` and `Formatting` on any branch not rebased recently, sent
`docs-reminder`'s parity nudge advisory-red, and left `decisions lifecycle` green with its
diff-dependent checks silently skipped — so the depth came off with the mask. The same graft is why `:latest` images ship `InformationalVersion 0.0.0-<sha>` (ersatztv#836).
**Two masked fetches that stay masked, deliberately.** `scripts/ci-detect-docs-only.sh` and
`scripts/ci-detect-already-validated.sh` both swallow a failed fetch — and should. In both, a failed
fetch can only leave the base revision unresolvable, and an unresolvable base emits `false`: *run
the full matrix / full validation*. Neither can turn a failed fetch into a skip. Masking there costs
runner time and nothing else, which is the opposite direction from the workflow steps above, where
the same masking bought a silently skipped check. Judge these by which way the failure falls, not by
the `|| true`.
That clears their **masking** and nothing else. Their `--depth` is a separate question with a
separate answer: `ci-detect-docs-only.sh`'s push-path `--depth=2` runs inside the `build` job's
`fetch-depth: 0` checkout and grafts it shallow, which is why the `git describe` in the immediately
following step returns nothing and `:latest` ships `InformationalVersion 0.0.0-<sha>` — measured on
the live test container as `0.0.0-5a7a4ed7` (ersatztv#836). That depth is correct for the script's
other three consumers — `test`, `migrations` and `functional-e2e` — whose checkouts really are
shallow (`fetch-depth: 2`). `build` is the only one that checks out complete, which is why it is the
only one that grafts.
### The last exemption, and how it closed
`ci-image.yml`'s checkout was the one left unset, and the reason was mechanical rather than a
judgement that it was safe: any commit touching `.gitea/workflows/ci-image.yml` re-pointed
`ci-image-pin`'s `expected` at itself and reddened that blocking job, and the file's unfiltered
`push` trigger published a toolchain image from whatever branch the edit landed on. Both are gone
(ersatztv#744): the trigger is `branches: [main]` and the file is no longer one of `expected`'s
paths, so the flag went on in the same change. **What this costs the guard is nothing, and that is
the point** — ersatztv#835 was deliberately held back rather than shipped with a one-entry
`ci-image.yml` exemption, because such an entry outlives its reason silently: once the flag went on
there the exemption would still have passed and the guard would have been permanently blind to the
file it was written for.
## Per-job declarations (`env.CI_EXECUTION_CLASS`, `env.CI_JOB_ROLE` — ersatztv#786/#789)
**Every job in every tracked workflow declares `CI_JOB_ROLE`; the jobs in `docker-build.yml` also
declare `CI_EXECUTION_CLASS`** (the scopes differ — see the table). They are metadata, not runtime
inputs: nothing reads them at execution time. They exist so that a guard comparing sets of jobs
derives its expected set from the workflow instead of from a literal list in the checker.
| Marker | Values | Scope | Derived by |
| --- | --- | --- | --- |
| `CI_EXECUTION_CLASS` | `toolchain`, `bare-runner` | `docker-build.yml` only — the sole workflow using the toolchain image | `scripts/tests/test_ci_image_pin_population.py` |
| `CI_JOB_ROLE` | `guard`, `report-only`, `none` | every tracked workflow | `scripts/tests/test_workflow_job_guards.py` |
**A missing marker and an unrecognised value both fail the suite.** Neither defaults: a scheme whose
absent value reads as some safe class stops applying the moment a job is added and nobody notices.
**`CI_JOB_ROLE` is decided by what the job PRODUCES.** A `guard` job's output is a **verdict**; a
`none` job's output is an **artifact**, and a red there means the build did not work rather than that
an invariant was violated. `docker-build.yml::build` publishes an image and smoke-tests it, and is
`none`. Which jobs are `none` is read off the markers rather than listed here — a list in prose is a
second copy of the workflow that rots on the next job added. Every `guard`/`report-only` job holds a
row in `docs/guard-inventory.md` → "Workflow-job guards", cross-checked both ways.
`report-only` is the third value: a verdict the job **cannot enforce**, because every check step
carries `continue-on-error: true`.
**A rejected alternative, so it is not re-adopted:** "a guard enforces an invariant about the
*repository*, so a job exercising the *product* is `none`" reads as more principled and is wrong —
it puts `test` and `migrations` outside the population, and those are the two **required** status
contexts on `main`, i.e. precisely where a failure to fire is fail-open against branch protection.
**Why not a literal list**, since one stood here until #789 with a real argument behind it: set
equality between two *derived* sets is blind to a member leaving both at once, so a job that loses
its `container:` block leaves the declared and the pinned set together and the comparison stays
balanced. That needs an anchor that does not move with the block — and a reviewed literal was the
only one available. The marker is the same anchor, better placed: it lives in the job it describes,
so it is reachable by anyone reading that job and it moves with the job when the job is renamed.
(Not "in the same hunk" — measured, the marker sits 67 lines from the `container:` block in four
jobs and **58** in `migrations`, where the `services:` block sits between them. Proximity is a
convenience; the property the guard rests on is that the marker does not move when the block does.)
**Its cost is proximity**, so the scheme does not stop there. A literal in a distant file survives a
careless workflow edit; a marker a handful of lines from the `container:` block can go with it. So
`test_no_job_needs_the_toolchain_without_declaring_it` derives the requirement a third time — from
each job's own step bodies, looking for tools that exist only in the CI image (`dotnet`,
`dotnet-ef`, `node`, `npm`, `npx`, `ffmpeg`, `ffprobe`, `playwright`, `reportgenerator` — the list
lives in `TOOLCHAIN_ONLY_TOOLS`, and this sentence is a summary of it, not a second copy to keep in
sync). `git`/`jq`/`python3`/`curl`/`tar` are **excluded** because they do not discriminate: the
bare-runner lanes provide or provision them too. That check depends on neither the marker nor the
block, and it is the only one that catches a .NET step **moved** into a bare-runner job, where no
set changes at all.
It is a **necessary condition only**: a job whose *only* toolchain use sat behind a script would be
invisible to it. Measured — the blind-spot set is **empty today**: all five declared toolchain jobs
are detected directly, `functional-e2e` included (it runs `dotnet restore` and `npm ci` in its own
steps before handing off to `scripts/e2e-local.sh`). The converse is still not asserted, because a
future job could be written that way and a red on a correct tree is how a guard gets deleted.
**The residual, stated because it is the one that matters.** All three checks go BLIND together
(none of them reddens) under one plausible edit: drop the `container:` block, flip the marker, *and* move the invocation into a
script. Both set comparisons then stay balanced and the step scan sees no token. What bounds this is
the failure **mode**, not its likelihood — the job runs a missing binary and dies with
`dotnet: command not found`, loud and immediate, where ersatztv#774's original defect sent a
*required* check green on the bare runner. Trading a silent pass for a noisy crash is the point;
it is not a claim the hole is closed.
Full rationale: `docs/decisions/records/testing/workflow-declares-its-own-job-metadata.md`.
## The workflow: `.gitea/workflows/docker-build.yml`
Single workflow. Gating jobs `test` + `migrations` run in parallel and gate `build`; a
@@ -427,9 +693,10 @@ the image build.
job (exit 137); excluding it cuts the instrumented surface ~126× (2.5M→20k coverable
lines in the whole-solution `Architecture.Tests` process) and makes the percentage reflect
authored code.
- **Shallow checkout**: `fetch-depth: 1` (ersatztv#190) — this job never runs `git
describe`/`git log`, only `build` needs full history/tags for version computation, so
`test` and `migrations` both check out shallow. `build`'s checkout stays `fetch-depth: 0`.
- **Shallow checkout**: `fetch-depth: 2` — this job never runs `git describe`/`git log`, and only
`build` needs full history/tags for version computation, so `test`, `migrations` and
`functional-e2e` all check out shallow. It was 1 (ersatztv#190) until ersatztv#420's revalidate
step needed `HEAD^2` to resolve on a main merge commit. `build`'s checkout stays `fetch-depth: 0`.
- **NuGet package cache**: both `test` and `migrations` cache `~/.nuget/packages` via
`actions/cache@v4`, keyed on `hashFiles('Directory.Packages.props', 'global.json')` with
a `restore-keys` OS-level fallback (ersatztv#190). Avoids a from-scratch `dotnet restore`
@@ -548,10 +815,11 @@ files) has nothing for the heavy jobs to validate. Before this, such a change ra
is gated `if: steps.detect.outputs.docs_only != 'true'`. On a docs-only change the job runs only
checkout + detect and **reports `success` in seconds**.
**Shallow-checkout safe (the change-set diff).** `test`/`migrations` check out `fetch-depth: 1`, and
**Shallow-checkout safe (the change-set diff).** `test`, `migrations` and `functional-e2e` all check
out shallow (`fetch-depth: 2`), and
a shallow clone has **no `origin/<base>` tracking ref and no merge-base** — so a three-dot
`origin/main...HEAD` diff *errors*, the fail-safe returns `docs_only=false`, and the skip silently
never fires (the first cut shipped this bug — every docs-only PR still ran the full matrix; caught by
never fires (this bug shipped once — every docs-only PR still ran the full matrix; caught by
#416's "verify on a real PR" box). The script therefore fetches the base and diffs against
**`FETCH_HEAD`** (always written by `git fetch`, resolves in a shallow clone) with a **two-dot** tree
diff (`git diff --no-renames FETCH_HEAD HEAD`) — no merge-base required. (`api-docs`/`format` avoided
@@ -565,8 +833,9 @@ The jobs are **not** `if:`-skipped. That is deliberate and it is the whole trap
or an `if:`-skipped job), those contexts would never report and the PR could **never merge** — the
naive fix *bricks* docs PRs rather than speeding them up.
- On Gitea **1.25.4** an `if:`-skipped job reports commit-status state **`skipped`**, a distinct
state (verified with a throwaway probe, PR #418) — not `success`. We do **not** rely on how branch
protection treats a `skipped` **required** context. Keeping the job running and gating its *steps*
state (verified with a throwaway probe, PR #418) — not `success`. **Still true on 1.27.1**
(re-confirmed 2026-08-28, ersatztv#747, from production PR heads rather than a new probe PR). We do
**not** rely on how branch protection treats a `skipped` **required** context. Keeping the job running and gating its *steps*
makes the required context report `success` unconditionally, which is safe by construction.
- Non-required jobs may skip freely: production already proves a `skipped` **non-required** context
does not block merge (`build` is `skipped` on every PR). So `build` skips its image steps on a
@@ -674,12 +943,12 @@ Two independent mechanisms hold it, and neither is redundant:
**Why a job and not a step inside `build`.** A step cannot protect the job it lives in. `build` is
what publishes, so a guard step there fails **open** if the runner drops it — and the defence
("the guard's own body has no opener, so it cannot be dropped") is circular when the only thing
enforcing that property is the same PR-only test being backstopped. This was the first design and
two independent reviews rejected it for exactly that.
enforcing that property is the same PR-only test being backstopped. That was the first design, and
it was rejected for exactly that.
**Why it runs the real pytest and not a bespoke scanner.** The same first cut hand-parsed the
workflow YAML in stdlib Python, to avoid provisioning PyYAML on `build`'s bare runner. Review found
~10 **false negatives** in that parser in one round — flow mappings (`{run: …}`), a quoted
workflow YAML in stdlib Python, to avoid provisioning PyYAML on `build`'s bare runner. That parser
yielded ~10 **false negatives** in one round — flow mappings (`{run: …}`), a quoted
`"run":` key, aliases, multiline quoted scalars — making it strictly *weaker* than the check it
backstopped, in the only direction that matters for a security gate. Running the existing test
needs no second definition of "what is a `run:` body", so it has no drift surface at all. `scan`
@@ -710,13 +979,13 @@ Two independent mechanisms hold it, and neither is redundant:
command" (the step also runs the wiring tests; the script runs only the ban file) and not merely
"fails if it passes": any other outcome — exit 5 because everything was deselected, exit 2 on a
collection error, or a failure in some unrelated test — is a **refusal**, not a pass. That
exhaustiveness is the load-bearing part, because each weaker reading was a live bug in an earlier
draft: "non-zero means it noticed" vouched for the gate on a collection error, and the deselection
disarm this script exists to catch exits **5**, not 1.
exhaustiveness is the load-bearing part, because each weaker reading was a live bug here:
"non-zero means it noticed" vouched for the gate on a collection error, and the deselection disarm this script
exists to catch exits **5**, not 1.
It runs in the **real checkout** — an isolated copy does not inherit the repo-root config a disarm
would live in, which is exactly how the first version of that script reported healthy while the
real invocation was deselected. The workflow file is
would live in, so a copy-based run reports healthy while the real invocation is deselected — which
has happened here. The workflow file is
restored by an `EXIT` trap, and the step carries a `selfcheck` marker so dropping it reddens the
job.
@@ -739,9 +1008,9 @@ Two independent mechanisms hold it, and neither is redundant:
interpolation drop. It moves the terminal assumption — to fail open you must now drop the pytest
step **and** the assert step, rather than either one alone.
Measuring a guard on this path does **not** require cutting a release, and an earlier draft here
claiming it would was simply wrong: `build` runs on every push to `main`
(`if: github.event_name != 'pull_request'`), and a `workflow_dispatch` on any other ref runs the
Measuring a guard on this path does **not** require cutting a release: `build` runs on every
push to `main` (`if: github.event_name != 'pull_request'`), and a `workflow_dispatch` on any
other ref runs the
job while `Build and push` publishes nothing (its `push:` is gated on `main`/`v*`). That is how
#767 was verified — see the decision record for the run ids.
@@ -809,8 +1078,8 @@ resolved identity on stdout**, where capture is not in question, and the answer
this change's own run: `Marker identity: job=test run=1916 attempt=1 (from the runner)`, and the same
for `migrations`. Both required jobs, on the lane that matters.
That measurement is what promoted it from warn-and-default to required, and it is why the residual
this paragraph used to describe — a rerun inheriting attempt 1's markers — no longer exists. The
That measurement is what promoted it from warn-and-default to required, and it is what rules out a
rerun inheriting attempt 1's markers. The
identity line stays, as the standing evidence a future reader checks first if the keying is ever
doubted again.
@@ -867,9 +1136,11 @@ exists for.
Because this step needs an interpreter, the job DOES carry `actions/setup-python` — the one
exception to the no-setup-actions note below, since `python3` is not guaranteed on the bare `small`
lane. Both it and the script step carry `continue-on-error: true`: a script that exits 0 does not by
itself keep the JOB green, and a setup-action download failure would redden an advisory check just
as effectively as a hit would.
lane. It, the script step and the parity-doc step above it all carry `continue-on-error: true`: a script
that exits 0 does not by itself keep the JOB green, a setup-action download failure would redden an
advisory check just as effectively as a hit would, and the parity step's base-ref fetch fails loudly
(ersatztv#746) — which must reach the log without reddening a warn-only job. `Checkout` is the one
step left blocking.
The job deliberately has **no** `setup-dotnet`/`setup-node` (and
thus no `actions/cache`), so it can't hit the cache-save hangs seen on the VM-127 runner
@@ -1102,10 +1373,19 @@ entire gate was skippable with no forgery at all, which was cheaper than every r
- `block_admin_merge_override: true` — without it (the default is `false`), a repo admin could
`POST /pulls/{n}/merge` with `force_merge: true` and merge straight past a missing or red
`review-verdict/h10`. Disabling push alone just moves the bypass from the push path to the merge
path, since `timothy` is admin and is the identity every session already uses. **Source-attested,
not probed** (Gitea 1.27 `CanBypassBranchProtection`): verifying it by experiment means merging an
unreviewed PR, so the field was set rather than measured. Setting it is safe under either
semantics; re-confirming the bypass itself rides with ersatztv#747.
path, since `timothy` is admin and is the identity every session already uses. **Measured
2026-08-28 (ersatztv#747)**, replacing the earlier source-attested-only note. Two throwaway scratch
bases, each with its own rule, head and PR — differing by construction, not by mutating one rule in
place: on `scratch/747a-base` with the field `false`, an admin `force_merge` past an ABSENT required
context MERGED (HTTP 200); on `scratch/747b-base`, whose rule was identical but for the field set
`true`, the same call was refused (HTTP 405 `Not all required status checks successful`). The pair is
the evidence — arm B's 405 has no attribution control of its own. It never required merging an
unreviewed PR into `main`: a scratch BASE branch carries its own rule, so the experiment is
disposable. Full arm table, including the ordinary-merge arm and its control:
`release.main-direct-push-disabled`. `enable_bypass_allowlist` is **not** a
substitute in the one configuration tested: enabled with an EMPTY username list (and
`block_admin_merge_override: false`) the admin force-merge still returned HTTP 200. A non-empty
list was not tested.
**Operator recovery when a required context gets stuck.** `block_admin_merge_override: true` removes
the "Merge (admin)" / `force_merge: true` escape that used to unstick a PR whose required context was
@@ -1122,6 +1402,38 @@ and the client-side Husky guards (H6/H11/H13) remain useful friction but were ne
they are fail-open and `--no-verify` bypasses them. Tag pushes are unaffected (separate mechanism;
`tag_protections` is empty), so the release cut in "Cutting a release" still works unchanged.
**A fourth arm: guard-scope freshness (ersatztv#787).** `scripts/tests/test_ci_dropped_step_guard.py`
decides which CI jobs must carry per-step execution markers by DERIVING that scope from
`.gitea/required-status-contexts.json`, the committed mirror of `main`'s required status checks. It
derives rather than repeating a literal because its own CI job, `pr-checks.yml::script-tests`, checks
out with `persist-credentials: false` and cannot ask Gitea — which leaves the snapshot as the one
hand-maintained input in the chain. So the merge hook reconciles it: it reads
`branch_protections`, hands the payload to `scripts/check-required-contexts.sh`, and **asks** (never
denies) when the live list no longer matches. The comparison is against the EFFECTIVE required set:
a rule with `enable_status_check: false` requires nothing whatever its `status_check_contexts`
still holds, so reading the list without the flag would certify disabled protection as current.
Two consequences worth knowing before you meet them. **The immediate merge path now reads a
repo-admin-scoped endpoint**: before #787 `branch_protections` was fetched only when arming a
scheduled auto-merge, so a credential without repo-admin scope kept its auto-grant on the common
path and now gets a prompt there instead. That is the fail-closed direction and it is loud, but it
is a real change in what the gate needs. **The arm runs only for the repo the snapshot names**
(its `repo` field): the merge tool is called with whatever owner/repo is being merged, and judging
another repo by this one's mirror would report a confident finding about a repo the mirror does not
describe.
And what it does not reach: a docs/process-only PR exits at the hook's docs-only passthrough before
this arm, so a PR editing only `.gitea/required-status-contexts.json` — the snapshot-narrowing
direction — never triggers it. That passthrough is a human prompt rather than an auto-grant, and the
arm could not have judged such a PR anyway, since it compares the live server against the snapshot in
the local checkout rather than the one the PR proposes. Ask rather than deny because drift does not make that
merge unsafe — Gitea enforces the live required set server-side — while a stale guard scope means a
required context may have no dropped-step guard at all. The arm runs **last**, so it can only
downgrade an otherwise-satisfied auto-grant and never masks a more serious refusal, and it runs on
**both** merge paths: the pre-existing branch-protection read sits inside the
`merge_when_checks_succeed` branch and never fires on an immediate merge. Rule selection is shared,
not copied — both the hook and the checker load `scripts/lib/branch-rule-classifier.jq`.
**The hole it closes.** `pretooluse-merge-consent.sh` proves its three consent conditions at the
moment the merge tool is called. Pass `merge_when_checks_succeed=true` and Gitea performs the merge
*later*, against whatever head is green then — while the Done-when and review-verdict checks were
@@ -1156,6 +1468,33 @@ hook's condition (c)) and the `review-verdict/h10` status on the same sha. `BLOC
a commit landed mid-flight it writes **no** status and exits non-zero rather than retargeting your
verdict at a commit you never read.
**The verdict words are declared once** in `scripts/lib/review-verdict-vocabulary.sh`, and both
sides derive from it: `post-review-verdict.sh` classifies through `etv_verdict_class`, and
`check-review-verdict.sh` builds its `POS_RE`/`NEG_RE` from `etv_verdict_alternation`. Adding a word
is a one-line edit to that file and it reaches both sides by construction (ersatztv#788,
`release.verdict-vocabulary-shared`). Only the WORD SET is shared — the grammar the classifier
applies around it (the column-0 anchor, the whole-word boundary, the anchored `@ <sha>` field) stays
in `check-review-verdict.sh`, where every #629 false-open actually lived. Run
`scripts/post-review-verdict.sh` with no arguments to print the live vocabulary.
Two things follow that are easy to get wrong. **Do not reintroduce a parity test** that compares the
two scripts' sources: #774 shipped one and withdrew it after six review rounds, each finding another
shell construction that escaped the extractor or reddened it on a correct tree. And a vocabulary
that cannot be loaded or does not validate yields **no words on either side** — the reader exits 2
(which callers treat as an unreadable input and fail closed on), the writer exits 1 having posted
nothing. Words are interpolated into a regex, so a `.*` in the positive list would classify an
explicit `BLOCKED` as `positive`; validation restricts every word to `[a-z][a-z-]*`.
**What enforces that is a data dependency, not the `if` around the validator** — worth knowing
before editing either script, because the obvious reading is wrong and the first version of #788
shipped the hole. Under `set -u`, an unbound-variable error inside a function aborts the function
but not the script, and in an `if ! validate` form neither branch runs, so the `exit 2` is skipped.
The reader has no `set -e`, so unvalidated words reached the regex and an explicit `BLOCKED @ head`
classified `positive`, exit 0. Validation now sets a sentinel on its last line and the derived views
refuse without it. Likewise a top-level `exit` in a sourced library terminates the sourcing script
where it stands, so both consumers probe-source the library in a subshell and require an
end-of-file marker before trusting it.
The status description also records the base branch — `Review-verdict: MERGEABLE @ abc1234 (base:
main)` — and the merge-consent hook denies when that no longer matches the PR's live `base.ref`
(ersatztv#632). Retargeting a PR changes the effective diff without moving the head sha, so the
@@ -1185,24 +1524,83 @@ dependency PR is unexpectedly asking for a verdict, this is why**; the status de
The two exemptions are evaluated as **independent predicates**, never as an `elif` chain: a Renovate
PR touching only `docs/` still gets the docs-only exemption on its own merits.
An existing `review-verdict/h10` on the head is **only** left alone when it is positively identifiable
as a human verdict — a non-null `.creator.login` **and** a `Review-verdict:` description, which is what
`post-review-verdict.sh` writes. Anything else, including any shape the workflow does not recognise, is
**re-derived** rather than inherited. (Measured: a status POSTed with a user credential carries a
creator; one POSTed by an Actions job carries `"creator": null`.) Without this, an exemption obtained
once was accepted unchanged on every later run. This is a *provenance* check, not an authentication
one — someone who can POST statuses directly can still impersonate a verdict (ersatztv#697). That
provenance asymmetry is *why* the credential scoping in `ci.actions-credential-scoping` mattered: a
forgery through a **user** credential inherits as a human verdict, while one through a job's
`GITEA_TOKEN` carries `creator: null` and is re-derived, so it must win a race. CI's registry secret
was a user credential — the admin account — and no longer carries status-write. **`RENOVATE_TOKEN`
still is one** (`write:repository`, a real bot account), and secrets are a per-repo store any
PR-added workflow can reference, so that route is narrowed rather than closed; tightening this check
from "non-null creator" to an allow-list of approved reviewers is what would close it
(ersatztv#742). A collaborator's own personal token still can, and no repo-side change closes that.
Note also that re-derivation is **not** a race the attacker can lose: it fires only on the trigger's
`types`, and posting a status is not one of them, so a POST timed after the last PR event stands
until the next one.
An existing `review-verdict/h10` **`success`** on the head is **only** left alone when it is positively
identifiable as a verdict from an approved reviewer — a `.creator.login` in the workflow's
`H10_REVIEWERS` allow-list **and** a `Review-verdict:` description, which is what
`post-review-verdict.sh` writes. Anything else, including any shape the workflow does not recognise,
is **re-derived** rather than inherited. An existing **`failure`** is left alone on a weaker test —
attributable (`Review-verdict:`, a non-null creator, and a base matching this PR's) but **not**
necessarily allow-listed — see the state asymmetry below. (Measured: a status POSTed with a user credential carries a creator; one POSTed by an
Actions job carries `"creator": null`.) Without this, an exemption obtained once was accepted
unchanged on every later run.
**The allow-list replaced a "non-null creator" test (ersatztv#742), and how wide that old test
really was is the point.**
Non-null proved only that *some* user credential POSTed the status, and this repo stores credentials
for accounts that are not reviewers. `RENOVATE_TOKEN` is a `write:repository` PAT of the `renovate`
bot; unlike the registry credential it **cannot be scoped down** (Renovate needs repo write to push
branches and open PRs), and secrets are a per-repo *store* rather than a per-workflow grant, so a
PR-added workflow can reference it and POST a `Review-verdict:` success that reads back
`creator: renovate` — non-null, and therefore inheritable. (Inferred, not observed: `renovate` has
never actually posted a commit status in this repo — every row on all six Renovate-authored PR heads
carries `creator: null`. The user-credential-vs-Actions-token asymmetry underneath it IS measured.)
Membership makes every non-reviewer account's
**`success`** re-derive exactly as an Actions-token status does. The list is a **literal in the workflow**, not a
repo `vars:` entry or a secret: those are settable without a PR, i.e. without this gate, while
`.gitea/` is on the PROTECTED path list so the edit cannot ride a docs-only exemption either.
**The allow-list governs an inherited `success`. It does NOT govern a `failure`**, and the asymmetry is
deliberate: an off-list `failure` is left alone, because inheriting a rejection can only ever withhold
an exemption while re-deriving one can turn it green on an exempt PR. The worst a forged `failure`
achieves is a stall, and `post-review-verdict.sh` POSTs unconditionally, so any human clears it in one
command.
**Beyond that, two other readers of `.creator` stay broad, and the split is load-bearing.** `read_existing_verdict` carries two flags: `ex_attributable` (an attributable
`Review-verdict:` row for this base, whoever wrote it) and `ex_human` (`ex_attributable` **plus**
membership).
- The **inheritance short-circuit** reads `ex_human`. Narrowing is safe there: an unrecognised account
falls through to re-derivation.
- The **last-moment re-read** reads `ex_attributable`, because it asks the opposite question — "did a
reviewer post a verdict while we were classifying". Narrowing it makes the job stop abstaining and
post its exemption over the row, and the post-write repair does not cover that: it is skipped
whenever the high-water mark could not be established. It additionally requires the
state/creator/description triple to have *changed* since the first read, because the two calls no
longer compute an identical predicate and "changed" can no longer be inferred from "fired".
- The **post-write raced check** stays broad on `creator != null`. Not because narrowing it would let
a rejection go green — a real reviewer is on the list by construction — but for the
misconfiguration case, a second human account added to the repo and forgotten from the literal.
Its reach is narrower than it looks: it covers such a verdict landing inside the **write window**
and nothing else, since a verdict already on the head sits below the high-water mark and is
invisible to it. Within that window the errors are asymmetric — a repair to `pending` is
recoverable, a wrongly-standing `success` is not — so it stays broad. The cost: any verdict-shaped
row landing in that window trips the sticky repair sentinel and costs that head its exemption until
a human clears it.
Pinned by `test_the_MID_RUN_abstain_still_fires_for_a_NON_allowlisted_creator` and
`test_the_RACED_verdict_check_still_counts_a_NON_allowlisted_creator`.
**If a reviewer's verdicts stop being honoured, this is why.** Whenever the job declines to inherit an
existing status it logs the creator, the allow-list and the description together, and says to add the
account to `H10_REVIEWERS` in `.gitea/workflows/review-verdict.yml`. (A separate `::warning::`
annotation carrying the same three values was tried and withdrawn: its only delta was a severity whose
rendering could not be probed, and it needed a guard of its own to stop contradicting the line above
it.) `post-review-verdict.sh` posts
with whatever account owns `ETV_GITEA_TOKEN`/`ETV_GITEA_BASICAUTH` and does **not** check that it is an
allow-listed one (ersatztv#845). A changed writer credential does not fail loudly — it reports success,
and each *subsequent* PR event re-derives the status: to `pending` on a PR that needs a verdict, or to
an exemption **`success`** on a docs-only or Renovate-manifest one, which is the quieter and worse
half. A verdict posted after the last event stands until the next one, so the symptom is a PR that
keeps losing its verdict rather than an immediate refusal.
This remains a *provenance* check, not an authentication one — an **allow-listed** collaborator's own
write-scoped token can still POST a verdict they did not form, branch protection binds the context
*name* rather than its issuer, and no repo-side change closes that (ersatztv#697). That provenance
asymmetry is *why* the credential scoping in `ci.actions-credential-scoping` mattered: a forgery
through an allow-listed **user** credential inherits, while one through a job's `GITEA_TOKEN` carries
`creator: null` and is re-derived, so it must win a race. Note also that re-derivation is **not** a
race the attacker can lose: it fires only on the trigger's `types`, and posting a status is not one of
them, so a POST timed after the last PR event stands until the next one.
Deciding either exemption requires the PR's **complete** changed-file list, which the workflow does
not compute itself: it calls `scripts/pr-changed-files.sh`, the single shared implementation also
@@ -1228,7 +1626,7 @@ payload, which a retarget cannot rewrite, and `edited` is in `types:` so a retar
`edited` gives **detection, not atomicity**: runs are not serialized, so a stale run could still post
`success` after the reclassifying run posted `pending`.
**That residual is now fenced (ersatztv#706).** Runs are still not serialized — instead a run that was
**That residual is now fenced — NARROWED, not resolved (ersatztv#706; correction ersatztv#849).** The fence never re-counts *after* the POST, so a retarget landing between its final pre-write count and the write still yields a **permanent** forged green: the successor run consumes the `edited` event and exits on the existing status, and the stale run then posts last with nothing left to correct it. Runs are still not serialized — instead a run that was
overtaken *declines to write*. The job counts `change_target_branch` events on the PR's issue timeline
at start and again immediately before its POST, and posts **nothing** if the count moved. The count is
the key precisely because the branch *name* is ABA-vulnerable: `main → scratch → main` reads `main` at
@@ -1237,8 +1635,10 @@ a PR, because every retarget fires `edited` — the event that makes one run abs
its successor.
If the count can't be established (unreadable timeline, paging that never reached a validated empty
page), only the exemption `success` is withheld; `pending` still posts, since `pending` cannot turn an
unreviewed head green and withholding it would strand ordinary PRs for nothing. **If an exempt PR is
page), only the exemption `success` is withheld; `pending` still posts, since it blocks the merge
immediately and withholding it would strand ordinary PRs whenever the timeline is unreadable. That is
the right trade but not a free one — a generic `pending` can mask a rejection landing in its own write
window and be re-derived into an exemption `success` later (ersatztv#849). **If an exempt PR is
unexpectedly missing its status after a retarget, this is why** — the job log names the counts.
Worth knowing before reaching for the obvious alternative: **a concurrency group does not work here**,
@@ -1249,11 +1649,112 @@ measured rather than assumed. Gitea 1.25.4 auto-cancels superseded `push` runs o
statusless with nothing left to re-trigger it. Full measurements and the two surviving residuals:
`ci.verdict-write-retarget-fence`.
Since ersatztv#803 that fence counts **two** event types on the one timeline walk:
`change_target_branch` for the base alias (`main -> S -> main`) and `pull_push` for the HEAD alias
(a force-push `H1 -> H2 -> H1` spanning `pr-changed-files.sh`'s paging, which leaves every sha
comparison equal). Either count moving withholds the write. The run log line is
`Mutation fence: N retarget event(s) and M push event(s) … (trusted=…)`.
The `push`-supersession half of that claim stays **1.25.4-dated on purpose** (ersatztv#747,
2026-08-28). It is not unobservable, but it is no longer reproducible ON DEMAND. No workflow triggers
on a push to a *non-`main`* branch — `docker-build.yml` filters its `push` trigger to `main` plus `v*`
tags, `ci-image.yml` to `main` and `docker/ci/**` with deliberately no `tags:` key
(`ci.toolchain-image-publish-is-a-dispatch`), and the other four have no `push` trigger at all — and
`main` itself refuses direct pushes since `release.main-direct-push-disabled`. A PR merge IS a push to
`main`, so two merges landing close together remain a route; a `v*` tag push triggers `docker-build.yml`
but is **not** one, because supersession is per-ref and a release tag is pushed once. What is gone is the
cheap disposable probe — the scratch-branch route the other measurements used. Re-running it would mean
arranging two real merges, or adding a throwaway trigger; both cost more than the claim is worth,
because nothing branches on it. The `pull_request_target` half the fence actually relies on was **not**
re-measured either and is 1.25.4-dated too.
Separately, after posting an exemption `success` the job re-reads the per-POST status history and, if
a human `Review-verdict:` row appeared during the write window, overwrites its own status with
`pending` and logs an error — so a human `BLOCKED` can never be silently turned green. The repair is
`pending` and logs an error. **That check runs only for a `success` write**, so it does not extend to
a generic `pending` write: that write is not verified, and a rejection it masks is re-derived green
by a later run (ersatztv#849). The repair is
`pending`, never a copy of the human's verdict, which would attribute a human decision to the job.
**Both `/statuses/{sha}` reads are PAGED** (ersatztv#763). The history is read twice — before the write
for the high-water mark, after it for the race check — and `limit` clamps to `MAX_RESPONSE_ITEMS`
(measured 50), so a single read of a busy head returned a partial list.
**Be precise about what that cost** — ersatztv#763's framing of it is too strong. Under the server
default (`created_unix DESC`) page 1 holds the *newest* rows and ids are monotonic with `created_at`,
so page 1 already carried both the true maximum id and every row newer than the mark — the only rows
the post-write check selects on. A single-page read therefore missed a raced verdict only if **more
than 50 rows were created inside the write window**, not merely on "a head with more than 50 rows".
What actually removed PR #761's stall is the deletion of the page-2 probe below, not the paging.
The walk earns its place for a different reason: it removes the gate's one fail-toward-SUCCESS path
from depending on an **undocumented ordering** the server honours only coarsely (page 1 came back
`114,112,113,111,110` — newest-first, but not strictly, and nothing in the API contract promises even
that). Both reads walk to a **validated empty page**, never terminating on a short one, retrying each
page once. The termination rule is page-size independent; the *capacity* is not — 20 requests of 50
validate at most 950 rows, since the twentieth must be the empty terminator. The terminator is measured
per endpoint and they differ: `/statuses/{sha}` returns `[]`, `/issues/{n}/timeline` a bare `null`,
`/commits/{sha}/status` an object with `statuses: null`.
**The page-2 probe is gone.** ersatztv#751 treated "there are rows I did not read" as "assume raced".
That fired on Renovate PR #761: a head that grew past one page over ordinary CI re-runs had its
exemption repaired away, with an `::error::` asserting a human verdict was overwritten when the head
carried none, and the sticky sentinel then refused re-exemption on every later run.
The two directions are **not** symmetric, and the asymmetry is deliberate:
- **Post-write, uncertainty fails closed.** An unreadable history, an over-cap history, or a count that
does not come back as a number all repair to `pending`. Previously an unreadable history warned and
left the exemption green while the page-2 probe repaired on the same uncertainty — one check
disagreeing with itself.
- **Pre-write, a partial list still yields a mark.** The mark gates the post-write check entirely, so
refusing one *skips* the check and is itself a fail-open. This rests on the DESC default above: the
newest row, carrying the maximum id, is on page 1, so a walk that fails later still saw it. A
*validated* empty history is a different case and is not abandoned: it yields a mark of 0, which is
correct, since every later row is newer — that is the normal first run. What abandons the mark is a
read that both **failed** and returned nothing, the pre-existing gap tracked as ersatztv#849 and
unchanged here. A non-empty history carrying no numeric id is reported as unusable rather than
collapsed to 0.
Two id comparisons are **numeric-only**, closing the *string* half of a fail-open (a type guard, not
a value bound — a corrupt but genuinely numeric id would still inflate the mark, which is not
attacker-controllable since ids are server-assigned): jq orders strings above every number, so one
`"id": "99999"` would inflate the high-water mark until nothing looked newer, and
`.id > $since` would read any string id as newer than any mark — making a pre-existing base-mismatched
verdict look raced on every run, a permanent stall. `.creator` is likewise type-tested before it is
indexed, because `.creator.login` on a non-object exits jq 5 and, under `set -e`, took the step down
after the green was posted and before the repair.
**Rejected alternatives**, recorded so they are not re-adopted:
- **A currency witness** (require a row above the mark, proving the read reflects the POST just made).
Counting *any* row above the mark does not witness this job's write, so a stale-but-valid snapshot
carrying an unrelated newer row still passes; and a schema-valid stale read is not retried, so one
such response turns a transient anomaly into a permanent sentinel. Gitea here is a single instance
with no read replicas, so the hazard has no mechanism behind it.
- **`sort=highestindex`** (index ASC, measured), which would close the mid-walk-insert gap below. ASC
puts the *oldest* rows on page 1, which inverts the partial-mark fallback: the incomplete walk's
maximum becomes the max of the oldest rows, leaving every pre-existing row above the mark and read as
raced. That is the #761 failure, re-introduced to close a smaller one. A test asserts the request
carries no sort parameter.
**Accepted residual: a row inserted mid-walk can be missed.** These are independent offset-paginated
GETs with no snapshot token, so under DESC a new row lands at position 0 — on a page already read —
while everything else shifts down. It is bounded: a row arriving after this job's POST is not one the
job overwrote, and being newest it wins on the combined endpoint branch protection reads.
**An empty post-write history is rejected.** The walk terminates on an empty page — correct before the
write, impossible after it, since this job has just POSTed and the endpoint returns one row per POST. A
well-formed "no statuses exist" is not retried (the body is valid), so accepting it would conclude
`raced=0` from a list that cannot be real, silently. This is *not* the withdrawn currency witness: that
asked whether ANY row sat above the mark, which an unrelated newer row satisfied; this asks only whether
the list is empty, which no unrelated row can produce.
The `::error::` names its own cause, and there are three: a verdict actually **found**; a read that
could not be **completed**; and a read that completed but returned an **impossible** answer. The third
is not a variety of the second — an operator holding a sticky sentinel needs to know whether to go
looking for an API failure that never happened. The sentinel *description* stays generic, because the
classification recognises it as a fixed point.
Three properties of this workflow are security-relevant and are **structurally** asserted by tests in
`scripts/tests/test_pr_changed_files.py` — those tests pin the workflow's shape, which is not the same
as establishing that the gate cannot be forged (see the residual below, and ersatztv#697/#698):
@@ -1365,9 +1866,8 @@ Three things now hold the line, and they are deliberately different in kind:
historical defect (`pr number`) and a nonexistent context, but **not** a syntactically invalid
payload whose tokens are all known (`${{ github.ref == }}` passes), nor a renamed output
(`steps.metadata.outputs.shortsha` passes — every token after the first is preceded by `.` and is
skipped), nor an unclosed opener. Catching those needs an expression parser. An earlier draft of
this section claimed it caught "a payload that cannot evaluate, wherever it sits"; that was false,
and the corrected claim is the one to rely on.
skipped), nor an unclosed opener. Catching those needs an expression parser — it does **not**
catch "a payload that cannot evaluate, wherever it sits".
Worth knowing why nothing caught this for three days: every *other* workflow-shape test in that file
reads `_code_lines()`, which strips comments. That is correct for what it was for, but it encodes the
@@ -1376,7 +1876,8 @@ assumption this bug falsifies. The strict test reads the raw scalar, and must ne
⚠️ **A page past the end of `/issues/{n}/timeline` is JSON `null`, not `[]`** — and this instance is
not consistent between endpoints (`/issues/{n}/comments` returns `[]` when empty). The retarget
fence's `count_retargets` gated on `type == "array"`, so it read the real terminator as *unreadable*:
fence's timeline walk (`count_retargets`, renamed `count_pr_mutations` by #803) gated on
`type == "array"`, so it read the real terminator as *unreadable*:
the walk never reached a validated empty page, `rt_ok` was never `yes` for **any** PR, and the fence
therefore withheld **every** exemption `success`. Renovate and docs-only PRs got no status at all —
the same user-visible outcome as the dropped step above, by a completely unrelated route. So fixing
@@ -1392,21 +1893,21 @@ Two things kept it invisible, and both are worth generalising:
`array`-only gate was never exercised by the suite either. With the double corrected and the old
gate restored, **most of the fence suite fails** — 18 tests when first measured at `c710db4a1`, 21
once three more fence-dependent tests existed. The invariant is the point, not the count: they had
all been passing for the wrong reason. (Given as a range on purpose — an earlier draft cited a bare
"18", which was stale two commits later, inside a section about stale claims.) When a double claims
all been passing for the wrong reason. (Given as a range on purpose: a bare count goes stale within
a couple of commits, inside a section about stale claims.) When a double claims
fidelity, that claim is a test assertion and needs re-measuring like any other.
The type is now read as a value (`case` over `jq -r 'type'`) rather than through `jq -e`, whose
exit-status semantics already bit this workflow once at jq 1.6, and both `null` and `[]` terminate the
walk. The regression test is parameterised over both shapes because both are live on this server.
`null` is accepted as exhaustion only from **page 2 on** — every real PR's first page carries events
An empty page — `null` OR `[]`, which take the SAME rule since ersatztv#803 — is accepted as
exhaustion only from **page 2 on** — every real PR's first page carries events
(spot-checked non-empty across #752/#753/#749/#739/#717; the counts are deliberately not recorded here
because timelines grow and an earlier draft's five figures were stale within days), so a `null` first
page is anomalous rather
than empty, and the walk should not certify "no retarget happened" from a response it cannot explain.
because timelines grow and such figures go stale within days), so an empty first
page is ANOMALOUS rather than EXHAUSTED, and the walk should not certify "no retarget happened" from a
response it cannot explain.
**The same nil-slice shape bites `/commits/{sha}/status`** — a third instance, found by cold review of
the fix for the second. A head with no statuses yet returns
**The same nil-slice shape bites `/commits/{sha}/status`** — a third instance. A head with no statuses yet returns
`{"state":"pending","total_count":0,"statuses":null}` (measured on PR #739's head). `read_existing_verdict`
gated on `.statuses | type == "array"`, so it hit its `exit 1` and posted nothing at all — fail-closed,
same user-visible outcome. `null` is now accepted there only when `total_count` is 0, so a body that
@@ -1428,7 +1929,7 @@ proved empty:
at the server-wide `MAX_RESPONSE_ITEMS`, **measured at 50** (`/issues?limit=100` returns 50). A
response can therefore never carry 100 rows, and the comparison was **dead code**. The repo already
documented that cap in `scripts/pr-changed-files.sh`, two test files and `ci.script-tests-job`; the
guard was written against 100 anyway, and a cold review caught it. Hardcoding 50 instead would
guard was written against 100 anyway. Hardcoding 50 instead would
re-break the day the setting changes.
So the job **asks the server, and only when it matters**: if the `review-verdict/h10` row is on page 1
@@ -1500,7 +2001,7 @@ so a missing system library fails the image build rather than a CI run.
⚠️ **`ARG PLAYWRIGHT_VERSION` must equal `web/package.json`'s `@playwright/test` pin**, which is
deliberately **EXACT** (no caret): Playwright ties a browser *revision* to the package version, so a
mismatch leaves no usable browser. Renovate bumps the npm pin but cannot know about this ARG — when it
does, bump the ARG, let `ci-image.yml` publish the new `:<sha>`, then update all five container pins.
does, bump the ARG, publish the new `:<sha>` (step 1 below), then update all five container pins.
`scripts/e2e-ui.sh` guards the drift by *launching* a browser up front and failing with exactly that
instruction (it probes by launch, not by path, because `chromium.executablePath()` reports the
full-chromium path that a headless-shell-only image deliberately lacks).
@@ -1513,11 +2014,15 @@ SDK matches the base's glibc/ICU. Keep the ffmpeg tag on that `FROM` equal to th
ersatztv#299 seeded-media/scanner E2E follow-ups will need.
**Bumping the toolchain is a deliberate two-step.** The jobs pin an immutable `:<sha>`, never
`:latest`, so a bad toolchain push cannot break every job at once:
`:latest`, so a bad toolchain push cannot break every job at once. Both steps land in the **same
PR** — a PR that changes `docker/ci/**` without moving the pin turns `ci-image-pin` red, and the
merge-consent hook reads the combined status, so it will not auto-grant:
1. Merge a `docker/ci/Dockerfile` change. `ci-image.yml` publishes `ersatztv-ci:<sha>` (+ `:latest`
from `main` only — a human pointer; **jobs must never consume it**).
2. In a follow-up PR, update the pin in `docker-build.yml`**all five jobs together**. That PR's own
1. **Publish.** Push the branch with the `docker/ci/Dockerfile` change as HEAD, then dispatch
`ci-image.yml` on that branch; it publishes `ersatztv-ci:<sha>`. (`:latest` moves only from
`main` — a human pointer; **jobs must never consume it**.) Since ersatztv#744 a branch *push*
does not publish; see "Publishing from a branch is a dispatch, not a push" below.
2. **Pin.** Update the pin in `docker-build.yml`**all five jobs together** — on top. That PR's own
CI is what proves the new image works. The pin is repeated per job because
`jobs.<id>.container.image` cannot read the workflow `env` context.
@@ -1542,8 +2047,9 @@ registry outage rather than a one-character pin error. `ci-image-pin` therefore
> emits 8, the correct pin becomes 8, and the gate's hardcoded 7 goes permanently red demanding a tag
> with no image behind it. Making the publisher emit `--short=7` is tracked as ersatztv#597.
`ci-image.yml` triggers on pushes touching `docker/ci/**`, `workflow_dispatch`, and a weekly Monday
05:00 UTC cron (base-image security updates; Gitea registers `schedule` only from `main`). It runs on
`ci-image.yml` triggers on pushes **to `main`** touching `docker/ci/**`, `workflow_dispatch` on any
branch, and a weekly Monday 05:00 UTC cron (base-image security updates; Gitea registers `schedule`
only from `main`). It runs on
`ubuntu-latest` — it was on `small` until server-management#639, where "docker-only" was found to be
a poor proxy for "small": this is a full buildx of the .NET toolchain image, the heaviest job in that
lane. Like `docker-build.yml`, it needs BuildKit's inline `http = true` for the HTTP
@@ -1568,17 +2074,96 @@ registry. Renovate tracks the Dockerfile's image pins (`dockerfile` manager, see
⚠️ **A REBASE invalidates the pin.** The pin must equal the short sha of the commit that touched
`docker/ci/**`, and a rebase **rewrites that commit's sha** — so `ci-image-pin` goes red on a branch
that was green before, with a pin that still resolves to a real (now-orphaned) commit and an image
that still exists in the registry. Worse, the force-push usually does **not** rebuild: `ci-image.yml`
filters on `paths: docker/ci/**`, and a rebase that doesn't change the Dockerfile's *content*
produces no diff for that path, so nothing republishes. And you cannot simply re-dispatch it —
`ci-image.yml` tags `git rev-parse --short HEAD`, i.e. whatever the branch HEAD is when it runs, not
the commit that touched `docker/ci`. Those two coincide only when the `docker/ci` commit *is* HEAD.
that still exists in the registry.
Recovery (ersatztv#445 hit this): make the `docker/ci` commit be HEAD again — push a commit that
really does change `docker/ci/**`, let `ci-image.yml` publish `:<its short sha>`, then bump the pin in
a follow-up commit. That is the same two-step below, just re-run after the rebase. The cheapest way to
avoid it entirely is to **land a toolchain-image change on its own, before** the work that consumes
it, so the consuming branch never carries the `docker/ci` commit through a rebase.
### Publishing from a branch is a dispatch, not a push (ersatztv#744)
**A branch push of this file's committed form no longer publishes anything.** `ci-image.yml`'s
`push` trigger is `branches: [main]`, because Gitea resolves a `push` workflow's definition from the
pushed branch — so an unfiltered trigger meant any branch push touching those paths executed that
branch's own YAML, unreviewed, on a docker-capable runner holding the credential that writes
`ersatztv:prod` and the toolchain image five `container:` jobs run. Credential scoping
(`ci.actions-credential-scoping`) bounds what such a job may *do*; it never bounded whether it runs.
**Read that heading exactly as narrow as it is written.** The filter is loaded from the pushed ref
like every other line in the file, so a branch that *deletes* it re-enables the route in one edit.
What is closed is the **drive-by** case: an ordinary push of a legitimate `docker/ci` change
publishing an image nobody asked for, with no deliberate act anywhere in the loop. Against someone
who intends to run their own YAML this is not a boundary at all, and citing it as one is the mistake
to avoid — see #853.
**What this does not close — beginning with this file's own filter, then sorted by REF CLASS
rather than by which keyword a trigger carries.**
`docker-build.yml` has `branches: [main]`, so a keyword test passes it; it also has `tags: ['v*']`
and `pull_request:`, both head-resolved and both reaching the same registry credential. A tag push
is outside `release.main-direct-push-disabled`, so `git push origin <commit>:refs/tags/v99.9.9` runs
that commit's `docker-build.yml` and can publish `ersatztv:prod` — no dispatch needed. Separately,
`ci-image.yml`, `docker-build.yml`, `dependency-scan.yml` and `renovate.yml` all carry an
unrestricted `workflow_dispatch`, which loads its definition from the selected ref. The filter here
removes the *drive-by* route, where merely pushing a branch was enough; it is not a boundary against
a malicious writer. The whole remaining class is tracked in #853.
**A `v*` tag push does not fire `ci-image.yml`.** There is no `tags:` key, and a `branches:` filter
is compared against a branch ref — so a release cut no longer republishes the toolchain image as a
side effect. The exact matcher semantics are unprobed; the observable claim is the one stated.
The capability removed is real and is **not** assumed away — it is replaced by `workflow_dispatch`,
which Gitea loads from the branch you select. The constraint to plan around is that `ci-image.yml`
tags `git rev-parse --short HEAD`, i.e. **the branch HEAD at dispatch time**, not the commit that
touched `docker/ci`. So the branch-side two-step is:
1. Push the branch with the `docker/ci/**` change as **HEAD**. Anything that changes how the image is
built — a `ci-image.yml` edit to build-args, Dockerfile path or platforms — must be **in that
commit or below it**, because the dispatch builds HEAD's whole tree. Commit it afterwards
and the published image was built by the *old* recipe while the pin and `expected` both name the
`docker/ci` commit, so `ci-image-pin` is green over an image the merged recipe would not produce.
2. Dispatch `ci-image.yml` on that branch (Actions → *Build CI Toolchain Image**Run workflow*
pick the branch). It publishes `:<short sha of that HEAD>`; `:latest` stays put, because only
`main` may move it.
3. Commit the pin bump in `docker-build.yml` on top. HEAD moves, but nothing needs republishing —
`expected` is the `docker/ci` commit, which the pin now names.
After a rebase, redo all three: the sha is new, so the image must be republished under it. That is the
loop ersatztv#445 paid twice when `main` moved mid-review.
> **Probed on this instance 2026-08-27**, because the whole branch-side path rests on it. Dispatched
> `ci-image.yml` on `fix/744-ci-image-branch-filter` at `43b1e45` (run 2340, `conclusion: success`).
> Gitea bound the job to `head_branch: fix/744-ci-image-branch-filter` / `head_sha: 43b1e45b5554…`,
> so it loads both the definition and the tree from the selected non-default ref, and the run
> published `ersatztv-ci:43b1e45` — it tags the branch HEAD at dispatch time, as described above.
> `:latest` still resolved to `sha256:7baf403c…`, the digest it carried before the dispatch. Note the
> limit of that last check: `docker/ci` was unchanged, so the rebuild was content-identical and the
> new `:<sha>` carries the same digest as `:latest`. It therefore shows `:latest` did not CHANGE, and
> cannot by itself distinguish "not pushed" from "pushed the same digest" — the `refs/heads/main`
> guard in the workflow is what makes that decision.
The cheapest way to avoid the rebase loop entirely is still to **land a toolchain-image change on its
own, before** the work that consumes it, so the consuming branch never carries the `docker/ci` commit
through a rebase.
> **`docker/ci/**` has no cheap-edit path, comments included.** Editing any byte there re-points
> `ci-image-pin`'s `expected` at the editing commit, so the five `container:` pins go stale and a
> blocking job fails — a one-line comment fix costs a full publish-and-repin, and a rebase charges it
> again. Ride prose fixes in that directory with the next genuine toolchain bump. The TWO stale
> publish instructions still in `docker/ci/Dockerfile` — the `PLAYWRIGHT_VERSION` bump comment, and
> the rebase warning's "re-dispatching does not help", which is now the opposite of the documented
> recovery — are #854, deferred on exactly these grounds.
**A change that lives only in `ci-image.yml` publishes nothing.** That file is no longer one of its
own trigger paths, nor one of `ci-image-pin`'s `expected` paths. That pairing is a **decided
tradeoff, not a necessity** — keeping the file in `expected` still works, since the branch dispatch
can publish the ci-image.yml commit itself and the pin then matches. What it costs is the reason it
went: that route charges a full ~2 GB publish and a five-pin bump for *every* edit to the file, a
comment included, and a rebase charges it again. That price reverted the first attempt (#697) and is
why this change was landed alone — also a priced decision, not a rule. The chosen side pays the
opposite risk instead, below.
The consequence to live with: if you change **how** the image is built (build-args, Dockerfile path,
platforms) without touching `docker/ci/**`, CI keeps running an image built by the previous recipe
and nothing goes red. **Make the change alongside a `docker/ci/**` edit** — a comment bump in the
Dockerfile is enough. That is not one option of two: publishing after the merge and then pinning
does not work, because `expected` is the last `docker/ci` commit, so a pin naming the workflow-only
commit fails `ci-image-pin` and leaving the pin where it is keeps consuming the old image. The
`docker/ci` touch is what gives the new recipe a commit the pin is allowed to name.
### When the pinned tag disappears
@@ -1591,8 +2176,8 @@ three different claims, and existence is the one that is not ours to make: the r
server-management, and an owner-level Gitea *package cleanup rule* there (`keep_count` 15,
`remove_days` 1, `remove_pattern` `.*`, and a `keep_pattern` that no 7-hex sha can match) deletes any
sha tag once 15 newer versions of the package exist. `ci-image.yml` publishes a new `:<sha>` weekly
and on every push touching `docker/ci/**` or the workflow file, while the pin only moves when a human
bumps it — so a pin ages toward eviction on its own. That is what happened between 2026-08-11 and
and on every push to `main` touching `docker/ci/**` (plus every branch dispatch), while the pin only
moves when a human bumps it — so a pin ages toward eviction on its own. That is what happened between 2026-08-11 and
2026-08-13 (ersatztv#772): the tag vanished, and every `container:` job — **both required contexts
included** — died after 12s with
@@ -1709,12 +2294,13 @@ what it establishes is the auth + HTTP-registry push path, not the toolchain bui
**Bumping the pin is enforced, not remembered.** The `ci-image-pin` job (blocking, PR-only; defined
in `pr-checks.yml`, but it greps `docker-build.yml` where the pins live) fails if
`docker-build.yml`'s pin isn't the short sha of the last commit to touch `docker/ci/**` or
`ci-image.yml`, if that pin isn't exactly 7 chars long (see above), or if the five jobs ever pin
different tags. This exists because **Renovate manages
`docker-build.yml`'s pin isn't the short sha of the last commit to touch `docker/ci/**` (since
ersatztv#744 that path alone — `ci-image.yml` is deliberately not part of it), if that pin isn't
exactly 7 chars long (see above), or if the five jobs ever pin different tags. This exists because **Renovate manages
`docker/ci/Dockerfile`'s base pins but cannot bump an opaque `:<sha>`** in `container.image` — so a
Renovate base bump would otherwise publish a new image, test the *old* one, and merge with the
Dockerfile disagreeing with the pin. A red `ci-image-pin` means: let `ci-image.yml` publish the new
Dockerfile disagreeing with the pin. A red `ci-image-pin` means: dispatch `ci-image.yml` on the
branch to publish the new
`:<sha>`, then update all five pins to it.
**What it is and isn't worth.** Measured honestly (ersatztv#390): the image saves ~1540s per job
File diff suppressed because one or more lines are too long
@@ -0,0 +1,75 @@
---
key: api.ffmpeg-profile-numeric-bounds
title: '2026-08-26 — Consequential numeric FFmpeg profile fields are REJECTED out of range, not accepted and rewritten; readrate pacing becomes a bounded profile field (#735)'
status: active
since: '2026-08-26'
supersedes: none
superseded-by: none
rule: 'A write path that receives an out-of-range value for a consequential numeric FFmpeg profile field returns 422 naming the bound AND the consequence of exceeding it, instead of storing a substitute and returning 200. `FFmpegProfileBounds` (ErsatzTV.Application/FFmpegProfiles) is the single validator, called from both the create and the update handler, and it validates against constants declared on `FFmpegState` beside the render-time resolution rather than restating numbers — `MinimumQsvExtraHardwareFrames`, `Minimum/MaximumReadRate` and `MaximumReadRateCatchup` are read by BOTH the validator and the renderer, while `MinimumReadRateCatchup` is write-path-only (at render time the resolved base rate is always at least `MinimumReadRate`, so it can never be the binding floor). THE RENDER-TIME CLAMPS STAY: they cover rows written before this validation existed or out of band, and keeping them is what makes the change migration-free. ON UPDATE, only a NEWLY submitted out-of-range value is rejected — an UNCHANGED legacy value is written back as-is, because the SPA sends the whole profile on every edit and rejecting it would make an old row uneditable over a field the operator never touched and, when hardware acceleration is not QSV, cannot see. Separately, the readrate pacing that `PipelineBuilderBase` hardcoded is now two nullable profile fields, `ReadRate` and `ReadRateCatchup`; `null` means unset and resolves to the values the pipeline used before they were configurable, so an untouched profile paces identically. `-readrate_catchup` stays ON by default and capability-gated in code — this makes it tunable, not optional.'
signals: 'silent transform of a submitted value · PUT 0 returns 200 and reads back 64 · 422 naming the bound · extra_hw_frames floor · readrate · readrate_catchup · pacing is not exposed to an operator · bounded numeric profile field · legacy row stays editable · paths: `ErsatzTV.Application/FFmpegProfiles/FFmpegProfileBounds.cs`, `ErsatzTV.FFmpeg/FFmpegState.cs`, `ErsatzTV.FFmpeg/Pipeline/PipelineBuilderBase.cs`, `ErsatzTV/Controllers/Api/Requests/CreateFFmpegProfileRequest.cs`, `web/src/screens/ffmpegProfileDraft.ts`, `web/src/screens/FFmpegProfilesScreen.tsx` · issues: #735, #726, #529, #350'
mechanics: '`FFmpegProfileBounds.Validate{QsvExtraHardwareFrames,ReadRate,ReadRateCatchup}` return `Validation<BaseError, Unit>` and are applied alongside the name/thread-count checks in both handlers; the payload is `Unit` because LanguageExt `Validation.Success` throws on a null-valued `Nullable<T>`. `FFmpegState.ReadRateFor(isStreamCopy)` / `ReadRateCatchupFor(isStreamCopy)` resolve and clamp at render time. Pinned by `FFmpegProfileHandlerTests` (rejection, the unchanged-legacy acceptance, and the persisted-null default), `PipelineBuilderBaseTests` (a configured value reaches the command line; an out-of-range one is clamped) and `web/src/screens/ffmpegProfileDraft.test.ts` (the SPA mirrors the same bounds and the same legacy exemption). The SPA constants are hand-mirrored literals with NO cross-language pin — see the record body.'
---
- **The wart was the silence, not the floor.** `ffmpeg.qsv-extra-hw-frames-floor` recorded, as an
accepted residual, that a client which `PUT`s `0` gets a `200` and then reads back `64`. The floor
itself was right — `extra_hw_frames=0` is a channel that serves nothing at all — but an accepted
value that is stored as a different one leaves the config no longer describing the behavior, and
the OpenAPI document never advertised the substitution. A rejection with a reason teaches the
bound; a rewrite hides it. The floor is now the *second* line rather than the first.
- **Both halves are needed, and each has its own failure.** Validation alone would leave every row
written before it existed unbounded at render time; the render clamp alone is what produced the
silent transform in the first place. So: validate on write, clamp on render, and test both paths.
The clamp is also the reason no backfill migration is required, which was the deliberate #529
trade-off and still holds.
- **An unchanged legacy value is accepted — deliberately.** The obvious strict rule ("reject any
out-of-range value") makes an old profile uneditable: the SPA round-trips the whole profile, so
renaming a pre-#529 profile would 422 over `qsvExtraHardwareFrames`, a field the editor only
renders when hardware acceleration is QSV. The validator therefore compares against the stored
value and rejects only a change. The cost is a contract that is not purely a function of the
request — the same body can be accepted or rejected depending on stored state — and that is
stated in the OpenAPI description rather than left to be discovered.
- **The pacing values were unreachable, and that cost a whole diagnosis.** During #726 the only way
to try a different `-readrate` was to rebuild the app; the diagnosis had to be done by replaying
command lines by hand on the Docker host. They are now `ReadRate` / `ReadRateCatchup` on the
profile. The defaults are unchanged and live on `FFmpegState` beside the bounds, so "what does an
unset profile do" is answerable in one place.
- **Bounded, not free-form, and the ceilings are chosen rather than measured.** The floors are
evidence-backed: below realtime the process reads slower than a live client consumes and the
channel stalls, which is #726 measured at an effective 0.53x, and a catchup below the base rate
cannot let a lagging input recover at all. The ceilings (2.0 and 10.0) are *chosen* — they exist so
the field cannot be used to effectively disable pacing, which is the unthrottled-read condition
#529 measured to produce zero segments on a QSV pipeline. Say so rather than implying a cliff was
found there.
- **What is deliberately NOT exposed.** `-readrate_initial_burst` stays derived from the HLS segment
length (`OutputFormatHls.SegmentSeconds * 2`) rather than becoming a third field: it is a function
of the segmenter's own configuration, not an independent dial. The concat/wrap-segmenter wrapper's
bare `-readrate 1.0` also stays hardcoded: it reads ErsatzTV's own loopback output rather than a
media file's demuxer, so it is not the input #726 was about. (`FFmpegState.Concat` takes no
profile, but that is a consequence of the decision, not the reason for it — both call sites have
`channel.FFmpegProfile` in scope and could pass it.) And a raw-args passthrough remains out of
scope here (#736 tracks it as a separate advanced-gated feature with its own safety posture):
these are known-dangerous knobs, which is exactly the case for bounding and explaining them.
- **Two residuals, stated rather than implied.** (1) `FFmpegProfileRepository.Copy` clones every
current value (`CurrentValues.Clone()`), including these fields, and `CopyFFmpegProfileHandler`
validates only the name — so it can propagate a legacy out-of-range value into a new row. That is
consistent with the unchanged-value exemption (Copy takes no operator-supplied number, so it can
never *introduce* one), and the command has no route today: no controller action, no MCP tool, and
the SPA's copy goes through `POST` to the validated create handler. If Copy is ever given a route,
route it through `FFmpegProfileBounds` first. (2) The SPA's mirrored bounds are hand-written
literals in `ffmpegProfileDraft.ts` with nothing pinning them to `FFmpegState`; raising a server
bound leaves every test green while the form keeps enforcing the old one. Left unpinned on
purpose: a C#-constant-versus-TS-literal guard is a string predicate over two languages, and the
drift it would catch is a needlessly strict form, not a bad value reaching FFmpeg. Mirror the
value AND the wording instead, and re-read this residual when a bound moves.
- **The SPA's numeric bound is checked whatever the acceleration is, matching the server.** The
editor only *renders* `qsvExtraHardwareFrames` under QSV, but `validate()` does not gate on that:
copying a legacy QSV profile and switching acceleration to None would otherwise submit a draft the
create handler rejects over a field the form is no longer showing. A conditionally-rendered field
still needs its unconditional check.
@@ -0,0 +1,83 @@
---
key: api.paged-count-matches-page-query
title: '2026-08-26 — A paged total is computed from the SAME query it pages — one IQueryable in a handler, a test-pinned pair where the count and page are separate methods (#690, #758)'
status: active
since: '2026-08-26'
supersedes: none
superseded-by: none
rule: 'A handler that returns a page plus a total count builds ONE `IQueryable`, applies every filter to it, and then derives BOTH the count and the page from that single object — `int count = await query.CountAsync(ct)` followed by `query.Include(...).OrderBy(...).Skip(...).Take(...)`. Counting the `DbSet` directly, or re-stating the predicate in a second `CountAsync(pred, ct)`, is the defect: the two expressions are then free to drift and nothing reports it. This is not a style preference — the drifted state is SILENT and shaped like working software. The page is correct, the count is wrong, and the client trusts the count: the SPA paginates on `TotalCount`, so 40 rows with 3 matching a search renders 4 pages of which 3 are permanently empty (#690), and an MCP agent paging to a completeness target reads a `totalCount` its own page can never reach (#758). Scope that harm honestly — of the six, only `GetPagedRerunCollections`, `GetPagedMultiCollections` and `GetPagedPlayouts` reach a controller today; `GetPagedCollections`, `GetPagedSmartCollections` and `GetPagedProgramSchedules` have no production caller (their REST routes use unpaged `GetAll*` queries), so they were latent, not live. Both named issues were ONE mechanism at six sites, of which the issues named two: `GetPagedCollections`, `GetPagedMultiCollections`, `GetPagedRerunCollections`, `GetPagedSmartCollections`, `GetPagedPlayouts`, `GetPagedProgramSchedules`. THE FILTER IS NOT ONLY THE SEARCH STRING — `GetPagedPlayouts` also applies `Filter(p => p.Channel != null)` to the page, and counting the DbSet missed that too; that clause is DEFENSIVE rather than a live defect, because `Playout.ChannelId` is non-nullable with `DeleteBehavior.Cascade` and both production connection strings set `foreign keys=true`, so the orphan state is unreachable while the FK holds. Three corollaries. (1) INCLUDES BELONG TO THE PAGE CHAIN, not to the shared filtered query: a COUNT does not materialize the graph, so `.Include(...)`/`IncludeSelectionDetails()` are appended after the count is taken, which keeps `api.selection-projection-include-chain` intact while leaving one predicate source. (2) A HANDLER WITH NO FILTER STILL TAKES THE SHAPE — `GetPagedFillerPresets` and `GetPagedTraktLists` take no `Query` parameter, so their `DbSet` counts were not WRONG, but leaving them counting one expression while paging another preserves exactly the drift this record is about for whoever adds the first filter. They derive both from one query too. (3) THE POPULATION IS DERIVED FROM THE SHAPE, NOT FROM THE `GetPaged*` NAME — three further count+page producers in `ErsatzTV.Application/MediaCards` (`GetTelevisionSeasonCards`, `GetTelevisionEpisodeCards`, `GetMusicVideoCards`) carry the same drift across a REPOSITORY boundary, where the count and the page are two interface methods rather than two expressions, so the structural fix cannot apply and they are pinned by a test instead (`MediaCardsCountMatchesPageTests`). `GetSeasonCount` now expands to the same Title+Year show set `GetPagedSeasons` pages; `GetEpisodeCount` and `GetMusicVideoCount` now count the METADATA table their pages are taken from, so a media item whose metadata row is missing no longer inflates the total. Their 1-based `pageNumber` is a separate defect against `api.paging-zero-based` and stays open in #832.'
signals: 'TotalCount ignores the search query · filtered page reports the unfiltered total · SPA renders empty pages after a search · agent pages to a completeness target it can never reach · count the same query you page · one predicate applied to both so they cannot drift · CountAsync on the DbSet · Channel != null missing from the count · includes belong to the page chain not the counted query · an unfiltered paged handler takes the shape too · a repository count and its page are two methods that must be pinned by a test · paths: `ErsatzTV.Application/MediaCollections/Queries/GetPagedCollectionsHandler.cs`, `ErsatzTV.Application/MediaCollections/Queries/GetPagedMultiCollectionsHandler.cs`, `ErsatzTV.Application/MediaCollections/Queries/GetPagedRerunCollectionsHandler.cs`, `ErsatzTV.Application/MediaCollections/Queries/GetPagedSmartCollectionsHandler.cs`, `ErsatzTV.Application/Playouts/Queries/GetPagedPlayoutsHandler.cs`, `ErsatzTV.Application/ProgramSchedules/Queries/GetPagedProgramSchedulesHandler.cs`, `ErsatzTV.Application/Filler/Queries/GetPagedFillerPresetsHandler.cs`, `ErsatzTV.Application/MediaCollections/Queries/GetPagedTraktListsHandler.cs`, `ErsatzTV.Infrastructure/Data/Repositories/TelevisionRepository.cs`, `ErsatzTV.Infrastructure/Data/Repositories/MusicVideoRepository.cs`, `ErsatzTV.Tests/Application/Paging/PagedQueryTotalCountTests.cs`, `ErsatzTV.Tests/Application/Paging/MediaCardsCountMatchesPageTests.cs` · issues: #758, #690, #671, #757'
mechanics: 'TWO enforcement modes, because the rule has two shapes. Where the count and the page are expressions in one handler, the STRUCTURE carries it (one `IQueryable`) and `PagedQueryTotalCountTests` pins at least one test per filtering paged handler. Where they are separate repository methods (`MediaCards`), nothing structural is available and `MediaCardsCountMatchesPageTests` pins their agreement instead. Every case asserts LITERAL expected counts AND the literal identities on the page — counts alone would let a count and a page agree on the WRONG SET and stay green. No repo-wide detector is proposed — see the record body for why the obvious one is not reliable.'
---
**The two issues were filed as separate bugs on separate entities, and they are one mechanism.**
#690 (rerun collections) and #758 (playouts) each describe a handler that counts
`dbContext.<Set>.CountAsync(ct)` and then pages a differently-filtered `IQueryable`. Auditing only
the two named handlers would have fixed two of six sites and left the same defect live on
collections, multi-collections, smart collections and program schedules — which is
`an issue's file list is not the population` in its most ordinary form: the reporter found the
instance that bit them, not the class.
**Why the fix is structural rather than "add the missing `Where` to the count".** The natural repair
is to give `CountAsync` a predicate matching the page's. That restores today's correctness and
preserves the defect: two expressions stating one intent, which the next person to add a filter has
to remember to update in both places. `GetPagedMultiCollections` and `GetPagedSmartCollections` were
already in exactly that half-state — their counts carried the `OwnedByChannelId == null` clause,
faithfully, and silently omitted the `Query` clause added later. The predicate that drifts is the one
added after the count was written, so no amount of care in the existing line reaches it. Deriving
both from one object removes the possibility rather than asserting its absence.
**The three `MediaCards` sites are the reason the population is stated by shape**, and the reason the
rule needs a second enforcement mode. Their count and their page are two different *repository
methods*, not two expressions in one handler, so "derive both from one `IQueryable`" has nothing to
attach to: `GetSeasonCount` counted `ShowId == showId` while `GetPagedSeasons` pages every show
sharing a Title+Year, and the episode and music-video pairs counted the item table while paging the
*metadata* table, so a media item whose metadata row was lost to a scanner failure inflated the
total. Where the structure cannot carry the invariant, a test does: `MediaCardsCountMatchesPageTests`
constructs each divergence and asserts `count == pageable rows`.
**Two things that surfaced only by writing those tests, and are the reason they are worth keeping.**
The seasons count has a THIRD answer nobody would guess from the count alone — with no `ShowMetadata`
row there is nothing to expand from, so `GetPagedSeasons` returns nothing and the count must be 0
rather than the show's season total; that case is pinned separately. And an include chain can filter more narrowly than
the count, in ALL THREE pairs rather than the one it was first noticed in: a REQUIRED reference
`Include` is emitted as an INNER JOIN, so `GetPagedEpisodes` (`Episode -> Season -> Show`),
`GetPagedSeasons` (`Include(s => s.Show)`) and `GetPagedMusicVideos` (`ThenInclude(mv => mv.Artist)`)
each return nothing when the principal row is absent, while the corrected count still counts. A
COLLECTION `Include` such as `Show.ShowMetadata` is a LEFT JOIN and drops nothing — the distinction
is the whole mechanism, so do not read "an include filters" as a blanket claim. Scope the consequence
honestly, the same way the `Channel != null` clause above is scoped: `Episode.SeasonId`, `Season.ShowId` and
`MusicVideo.ArtistId` are all non-nullable with `DeleteBehavior.Cascade` and production enforces the FK,
so count-N / page-0 is a corruption-only state no user can reach. It is stated because it makes
"count the table the page reads" necessary and NOT sufficient as a general rule, not because a live
defect is being left open; #832 carries it.
**`GetBlockPlayoutHistory`, `GetFuturePlayoutItemsById` and `GetLibraryBrowseItems` already did
this** — they build the filtered query, count it, then page it. The idiom was in the repo; the six
defective handlers predate it or were written beside it. That is the reason this is written down as a
convention: the correct shape existing somewhere did not stop six handlers from taking the other one.
**Why no repo-wide detector.** The plausible check is "a handler containing both `CountAsync` and a
conditional `Where` must count a variable, not a `DbSet`". It cannot distinguish the legitimate cases:
`GetPlayoutWarningsCount` is a bare count with a predicate that pages nothing,
`DeleteFFmpegProfileHandler` counts rows to decide whether a delete is allowed, and
`GetLibraryBrowseItems` sums five independently-filtered counts across entity types in a way no
single-query rule describes. It would also miss the three `MediaCards` sites entirely, since there
the count and the page are not in the same file at all. A detector that flags those reads
as noise and gets suppressed. The population is instead enumerated by SHAPE — every handler
returning a page plus a count, found by reading the git index for `CountAsync`/`.Skip(`/`TotalCount`
rather than for the `GetPaged*` name — and pinned by at least one test per filtering instance. That
distinction is not pedantic: the name-derived population is eight handlers and misses the three
`MediaCards` sites entirely — they were found, and fixed, only because the population was re-derived
by shape. That is the failure this record's own first paragraph names, committed once inside the fix
for it. (#832 carries what is deliberately left there: 1-based paging and delete-or-keep.)
**The tests assert pinned literals, not filter-derived expectations.** Each seeds five matchable rows
of which exactly two contain `"Alpha"` — plus, where the handler carries a non-search clause, one row
that clause must exclude from BOTH sides (a channel-owned collection, an orphaned playout) — then
asserts `TotalCount.ShouldBe(2)` and the page's names against the literal pair. Recomputing the expectation by re-applying the handler's own predicate would pass
whatever the handler does. The mutation proof is recorded in the PR, in both modes: restoring the
pre-fix `CountAsync` clause at all six handler sites turns the six per-handler
`PagedQueryTotalCountTests` cases red, and each of those constructs exactly one handler, so every
test is shown to detect its own site rather than a neighbour's; restoring all three pre-fix
repository counts turns all four `MediaCardsCountMatchesPageTests` red.
@@ -5,8 +5,8 @@ status: active
since: '2026-08-05'
supersedes: none
superseded-by: none
rule: 'Any credential reachable from an Actions job is scoped to what that job needs. The container-registry secret `REGISTRY_PASSWORD` is a personal access token scoped `write:package` + `read:repository` — never an account PASSWORD. This matters because Gitea has NO `status` token scope: `POST /repos/{o}/{r}/statuses/{sha}` is gated by `reqRepoWriter(unit.TypeCode)`, so ANY credential that can write the repository can forge `review-verdict/h10`, the required context that is supposed to make merge-consent derived rather than assertable. Package-write IS a separate scope, so the registry credential can be made status-incapable at no cost: `scripts/ci-detect-already-validated.sh` only GETs. Do NOT add a `permissions:` key to constrain the injected `GITEA_TOKEN` on the assumption that it binds — below Gitea 1.26.0 it is silently a NO-OP, which is worse than absent because it reads in review as a constraint. That version precondition NO LONGER HOLDS: this instance was upgraded 1.25.4 -> 1.27.1 on 2026-08-05. What has NOT changed is that the consequence is unverified — whether `permissions:` is honored here, and what this instance''s default Actions token permission is, were both left UNPROBED (there is still no API surface: `/api/v1/settings/actions` 404s at 1.27.1). Probe before relying on it; do not read the upgrade alone as the constraint now working. Scoping is necessary and not sufficient: it bounds what a job may DO, never whether attacker YAML runs at all, so a self-referencing trigger needs its own filter (`ci-image.yml`, tracked in #744 — deliberately NOT bundled here, because editing that file re-points `ci-image-pin` at the editing commit and reddens a blocking job). This record closes ONE route. It does not close the class, and four later sections say exactly what survives — read them before citing this record as a mitigation.'
signals: 'admin password in CI secrets, registry credential scope, ETV_STATUS_AUTH can write statuses, forge review-verdict/h10, head-resolved workflow holds credentials, Gitea token scopes, no status scope, write:package vs write:repository, permissions key no-op, GITEA_TOKEN default read/write, Restricted default token permissions, orphan secret, deploy key in secret store, toolchain image overwrite, prod floating tag write · paths: `.gitea/workflows/docker-build.yml`, `.gitea/workflows/ci-image.yml`, `.gitea/workflows/renovate.yml`, `scripts/ci-detect-already-validated.sh` · issues: #697, #672, #698, #742, #743, #420, server-management#714'
rule: 'Any credential reachable from an Actions job is scoped to what that job needs. The container-registry secret `REGISTRY_PASSWORD` is a personal access token scoped `write:package` + `read:repository` — never an account PASSWORD. This matters because Gitea has NO `status` token scope: `POST /repos/{o}/{r}/statuses/{sha}` is gated by `reqRepoWriter(unit.TypeCode)`, so ANY credential that can write the repository can forge `review-verdict/h10`, the required context that is supposed to make merge-consent derived rather than assertable. Package-write IS a separate scope, so the registry credential can be made status-incapable at no cost: `scripts/ci-detect-already-validated.sh` only GETs. `permissions:` on a workflow/job DOES bind on this instance — MEASURED 2026-08-27 on 1.27.1 by matched scratch-base probe PRs differing only in one unit, the block carried at JOB level on `set-verdict-status` (`code: write` posted the probe status; `code: read` 403ed the POST, curl exit 22, no status written), so all six workflows here now declare it (#748). Two properties that make it usable: the declaration is EXHAUSTIVE, not additive — a unit omitted is not granted — and it binds while the owner-level default is `permissive`, which is what makes the five `code: read` declarations effective TODAY rather than only after a flip. NOT established: what a declared `code: write` does under a RESTRICTED default. GitHub semantics let `permissions:` only narrow, never widen past the default, and if Gitea copies that, Restricted would cap the gate job at read and `review-verdict/h10` would stop being writable — the exact catastrophic case. The probe ran under `permissive` and CANNOT rule that out. Flip the owner default only behind the scratch-base probe re-run under Restricted (server-management#714, still open for this reason). The earlier form of this rule said the opposite ("do NOT add a `permissions:` key on the assumption that it binds — below Gitea 1.26.0 it is silently a NO-OP"); that was correct at 1.25.4 and is retained here so a reader meeting the old advice recognises it as superseded. There is still no API surface for the owner-level default (`/api/v1/settings/actions` 404s at 1.27.1). The instance default HAS since been probed and is NOT unknown: it was set to Restricted, verified, and reverted to `permissive` on 2026-08-05 (#748), which is where it stands. Probe before relying on it; do not read the upgrade alone as the constraint now working. Scoping is necessary and not sufficient: it bounds what a job may DO, never whether attacker YAML runs at all, so a self-referencing trigger needs its own filter. That landed for `ci-image.yml` in #744 (`ci.toolchain-image-publish-is-a-dispatch`) — deliberately NOT bundled here, because until it also removed the file from `ci-image-pin`''s `expected`, editing it re-pointed that job at the editing commit and reddened a blocking check. A second, separate consequence of the same boundary: `actions/checkout` persists whatever the job token can do into `.git/config` unless `persist-credentials: false` is set — read-only everywhere since #748 declared `permissions:` on all six workflows — and all 16 of this repo''s checkouts now set it — 15 in #746 and `ci-image.yml`''s in #744, once `ci.toolchain-image-publish-is-a-dispatch` removed the two mechanical reasons it was excluded; the convention is held with no exemption list by `scripts/tests/test_workflow_persist_credentials.py` (#835). Ordering is part of the rule: unmask the dependent fetches FIRST, because until then a credential regression presents as an empty changed-file set and a silently skipped check rather than a red job. This record closes ONE route. It does not close the class, and the later sections say exactly what survives — read them before citing this record as a mitigation.'
signals: 'admin password in CI secrets, registry credential scope, ETV_STATUS_AUTH can write statuses, forge review-verdict/h10, head-resolved workflow holds credentials, persist-credentials on actions/checkout, credential left in .git/config, masked git fetch yields empty changed set, Gitea token scopes, no status scope, write:package vs write:repository, permissions key no-op, GITEA_TOKEN default read/write, Restricted default token permissions, orphan secret, deploy key in secret store, toolchain image overwrite, prod floating tag write · paths: `.gitea/workflows/docker-build.yml`, `.gitea/workflows/pr-checks.yml`, `.gitea/workflows/dependency-scan.yml`, `.gitea/workflows/ci-image.yml`, `.gitea/workflows/renovate.yml`, `scripts/ci-detect-already-validated.sh` · issues: #697, #672, #698, #742, #743, #746, #748, #835, #420, server-management#714'
mechanics: 'PAT `ci-registry-scoped-697`, scopes `write:package,read:repository`, stored as repo Actions secret `REGISTRY_PASSWORD`; `REGISTRY_USER` remains `timothy`. Verified 2026-08-05 on Gitea 1.25.4: registry push of a probe tag SUCCEEDED; `GET /commits/{sha}/status` 200; `POST /statuses/{sha}` REFUSED HTTP 403 `token does not have at least one of required scope(s), required=[write:repository], token scope=write:package,read:repository`. Probe artifacts deleted, confirmed 404. NOT measured with this token: the `container:` pull, the buildcache write and the base-image pull. Those rest on Gitea''s scope model (write implies read per category, read at tag `v1.25.4`) — INFERRED. Note WHICH run proves which: only the `container:` pull is exercised by a PR. `cache-to`/`cache-from` and the base-image pull are confined to the `build` job, which carries `if: github.event_name != ''pull_request''`, so they are first exercised on the post-merge push to `main` — AFTER the merge gate has passed. A wrong inference there reddens main, not the PR.'
---
@@ -17,13 +17,14 @@ instance.
**Why the credential and not only the triggers.** Patching triggers enumerates *instances* of "a
ref-resolved workflow obtains status-capable credentials", and adding a new workflow file is itself a
route, so that enumeration never completes. But it is not either/or: `ci-image.yml`'s unfiltered
`push:` is path-scoped to itself, so any branch push runs attacker YAML on a docker-capable runner
with no PR. Scoping bounds what a job may DO; only a filter bounds whether it RUNS. That filter is
**#744**, not this record: editing `ci-image.yml` re-points `ci-image-pin`'s `expected` at the editing
commit and staleness-fails a **blocking** job. That is a toll, not a wall — the documented two-step
(publish `:<short sha>`, then bump all five pins) clears it — but a rebase rewrites the sha and charges
it again, so it lands alone (`land-toolchain-image-change-separately`).
route, so that enumeration never completes. But it is not either/or: `ci-image.yml`'s `push:` was
unfiltered and path-scoped to itself, so any branch push ran attacker YAML on a docker-capable runner
with no PR. Scoping bounds what a job may DO; only a filter bounds whether it RUNS. That filter landed
in **#744**, not this record, and its own decision is `ci.toolchain-image-publish-is-a-dispatch`:
`branches: [main]`, with branch publishing moved to a deliberate `workflow_dispatch`. It had to land
alone (`land-toolchain-image-change-separately`) because, until it also removed the file from
`ci-image-pin`'s `expected` paths, editing `ci-image.yml` re-pointed that job at the editing commit
and staleness-failed a **blocking** check on every rebase.
**What the scoped token still reaches — not "just a registry credential".** `write:package` over owner
`timothy` writes `ersatztv:prod` (the floating tag prod's `jazz-media` stack follows) and
@@ -40,9 +41,24 @@ this, but is not free: packages live in a user namespace only its owner and admi
halves INFERRED, neither probed.
**Provenance, corrected.** `review-verdict.yml` leaves an existing `h10` alone only when it is
positively identifiable as human — non-null `.creator.login` plus a `Review-verdict:` description
(`release.verdict-status-check`). A user credential posts with a real creator and is INHERITED; an
Actions job posts `creator: null` and is re-derived. **That asymmetry is not protection.** Re-derivation
positively identifiable as an approved reviewer's verdict — a `.creator.login` in the workflow's
`H10_REVIEWERS` allow-list plus a `Review-verdict:` description (`release.verdict-status-check`,
`ci.exemption-provenance`). An allow-listed user credential posts with a matching creator and is
INHERITED; an Actions job posts `creator: null` and is re-derived, and so now is every other account.
**The `RENOVATE_TOKEN` INHERITANCE route is closed by the allow-list, not by scoping (#742),
because scoping was not available — and inheritance is the only half it closes.** A status POSTed
with that token still satisfies branch protection if it is timed after the last PR event; what it can
no longer do is survive as a verdict across every later run of the workflow. That token needs `write:repository` to do its job — pushing branches and opening PRs
— so there is no `write:package`-shaped separation to exploit the way there was for the registry
credential, and secrets are a per-repo store any workflow can reference. The test was therefore
tightened at the *reader* instead: `creator IN <allow-list>` rather than `creator != null`, which
re-derives a `renovate`-posted verdict exactly as it re-derives an Actions-token one. Scoping and
allow-listing close the same class from opposite ends; where a credential cannot be narrowed, narrow
what the consumer will accept from it. **The creator-shape asymmetry — an ALLOW-LISTED creator's `success` INHERITED, everything else
re-derived — is not protection.** (Since #742 "a real creator" is no longer the test: only
`H10_REVIEWERS` membership inherits a `success`, while a `failure` inherits on attributability
alone.) Re-derivation
fires only on `opened|reopened|synchronize|ready_for_review|edited`, and posting a status is none of
them, so a POST timed after the last event stands until the attacker merges. The gain here is that PR
code can no longer escalate to instance admin — NOT that the durable forgery route is closed.
@@ -51,28 +67,112 @@ code can no longer escalate to instance admin — NOT that the durable forgery r
bound — auditing it rather than the workflow set is what found `RENOVATE_TOKEN` and
`SERVERMGMT_DEPLOY_KEY` below, since any PR-added workflow can reference any secret. But
`GITEA_TOKEN` is injected and never in the store; nor is the credential
`actions/checkout` persists into `.git/config` (`docker-build.yml` omits `persist-credentials: false`);
and jobs reach the runner's docker daemon.
`actions/checkout` persists into `.git/config` (dropped on 15 of the repo's 16 checkouts in #746 and
on the sixteenth in #744 — see below); and jobs
reach the runner's docker daemon.
**The persisted checkout credential (#746).** `actions/checkout` writes an `Authorization` header
into `.git/config` unless `persist-credentials: false` is set, so a job hands that credential to
every script it runs, not only to the checkout — and it is write-capable while this instance's
Actions default token permission is `permissive` — where it stands after a Restricted trial was
reverted on 2026-08-05 (#748). All 16 of the repo's checkouts
now set it: `docker-build.yml` (8), `pr-checks.yml` (5), `dependency-scan.yml` (1),
`review-verdict.yml` (1, pre-existing) and `ci-image.yml` (1, added in #744). No job that checks out pushes with git, so none of them
loses a capability; `renovate.yml` does push, but has no checkout step and uses `RENOVATE_TOKEN`.
**Unmask the dependent fetches BEFORE dropping a persisted credential** — the rule, not a note about
one change. Five steps fed `git fetch ... origin "$base_ref" || true` into changed-file skip logic,
where a credential regression reddens nothing: on the three that also masked `git diff` it yields an
empty changed set the skip logic reads as "nothing changed", and on the other two it hands a
possibly-absent `origin/<base>` to a validator. Anonymous fetch does cover this repo (public; `git-upload-pack` answers
unauthenticated, measured 2026-08-26), which is a reason to make the failure loud rather than a
reason to trust it silently. The same step is where `--depth` came off those fetches: their jobs
check out `fetch-depth: 0`, and `git fetch --depth=N` grafts a complete clone shallow, so a
three-dot diff against a branch point further back than N fails `no merge base` — invisible under
`|| true`, and without it a red `api-docs`/`Formatting` (`decisions lifecycle` would stay green with
its diff-dependent checks skipped, because its validator fails open on an unresolvable base).
`ci-image.yml`'s single checkout was the one exception, and never because it was safe — its
unfiltered `push` trigger made it head-resolved in the sense that matters. It was excluded because
editing that file re-pointed `ci-image-pin`'s `expected` at the editing commit and reddened a blocking
job, and because the same trigger would publish a toolchain image from the editing branch. #744
removed both (`ci.toolchain-image-publish-is-a-dispatch`) and set the flag in the same change, so the
count is now **16 of 16 with no exemption**. That is what let #835 ship the guard —
`scripts/tests/test_workflow_persist_credentials.py`, population derived from the git index — with no
exemption list at all: a one-entry list justified by a temporary condition elsewhere keeps passing
after that condition expires, leaving the guard permanently blind to the file it was written for.
**Measured vs inferred.** Measured here: the `v1.25.4` scope enum (`access_token_scope.go`) has no
`status` entry; the `reqRepoWriter` gate (`routers/api/v1/api.go`); the probes in `mechanics`. Read from
docs, NOT verified (2026-08-05): `permissions:` landed in 1.26.0 (Gitea PR #36173); no `app.ini` lever
at any version; Gitea rejects GitHub's `statuses`/`checks` scopes.
**MEASURED 2026-08-27 on 1.27.1 (#748) — this block is NOT covered by the 1.25.4 caveat below.** It
re-probes exactly ONE of that caveat's items, the `permissions:` question, and supersedes the
"whether `permissions:` is honored here is UNPROBED" claim. It does NOT discharge the caveat: the
scope enum, the `reqRepoWriter` gate and the `write:package` 403 are still 1.25.4-dated and
un-re-run. Method: a matched pair of scratch-base probe PRs, identical except for one permission
unit, the block carried **at job level on `set-verdict-status`** — the same level and job as the
shipped `review-verdict.yml`, so the gate's own shape is what was measured.
- `code: write` -> job success, `Posted review-verdict/h10-PROBE=success`, status `creator: null`.
- `code: read` -> job failure at the POST, `curl` exit 22 (HTTP >=400 under `-f`), NO status written.
What each observation licenses, kept separate. `issues: read` was exercised: both runs logged
`Retarget fence: 0 retarget event(s) ... (trusted=yes)`, which requires the timeline walk to have
read `/issues/{n}/timeline`. (That line and `count_retargets` were renamed by #803, which put the
head axis on the same walk: `Mutation fence: N retarget event(s) and M push event(s) ...` and
`count_pr_mutations`. The quoted string is what those runs actually printed and is kept verbatim;
reproduce against the current names.) `pull-requests: read` was exercised: the positive control posted
`=success`, reachable only through `complete=yes`, which requires `scripts/pr-changed-files.sh` to
have paged `/pulls/{n}` and `/pulls/{n}/files` to a validated end — the fence and the start marker do
NOT show this, the `success` does. Both reached the classify step's start marker, so neither was the
#751 dropped-step shape.
The negative control is the load-bearing half — without it a green positive is equally consistent
with `permissions:` being a no-op sitting beside a permissive default.
NOT established: what a declared `code: write` receives under a RESTRICTED owner default (both
controls ran under `permissive`); and the TOP-LEVEL declaration form the other five workflows use,
which is inferred rather than measured. That inference fails safe (a top-level key Gitea ignored is a
security no-op, never a brick). Probe branches and PRs (#851, #852) deleted after reading.
**Version caveat — this record's measurements are pinned to 1.25.4, the instance is now 1.27.1.**
The instance was upgraded mid-session on 2026-08-05 (#743). Everything above measured on 1.25.4 is
therefore a *dated* claim, not a current one: the scope enum, the `reqRepoWriter` gate and the 403
probe were all taken pre-upgrade and have NOT been re-run. They are recorded honestly as of their
date and are the best evidence available, but do not cite them as current behaviour without
re-probing. Re-verification of the 1.25.4-pinned claims across the CI docs is tracked separately.
re-probing. #747 re-probed SOME of the 1.25.4-pinned CI claims on 2026-08-28 —
`ci.docs-only-skip-steps`, `ci.shared-pr-file-enumeration`, the REST-route half of
`ci.batch-pushes-no-cancel-route`, the `compare` half of `ci.exemption-provenance` and the
`branches: [main]` item in `ci.gate-trigger-base-resolved` — and it was **not** a sweep of the
corpus. Other 1.25.4-dated CI claims remain un-re-run, the `creator`-attribution measurement the H10
allow-list rests on (`review-verdict.yml`) among them. It **did not** discharge this record's own
remaining items either: the `v1.25.4` scope enum, the `reqRepoWriter` gate and the `write:package` 403
all stay 1.25.4-dated. Only the 403 needs a scoped PAT minted against the live instance; the scope enum
and the `reqRepoWriter` gate were established by reading Gitea's source at the version tag (see
"Measured vs inferred" above), so re-running those needs only a source read at `v1.27.1`. #747 did
neither. What #747 did settle elsewhere is the merge-path half — see `release.main-direct-push-disabled`
for the measured
`block_admin_merge_override` semantics and for why `enable_bypass_allowlist` is not a substitute.
**Surviving routes — this record is not a mitigation for any of them.** `RENOVATE_TOKEN` is a
`write:repository` bot PAT in the same store, posting with a real creator, and cannot be scoped down
because Renovate needs repo write (#742). The injected `GITEA_TOKEN` is write-capable in every job;
only Gitea >=1.26 with the Actions default set to **Restricted** binds it (server-management#714) —
the version half of that condition is now satisfied (1.27.1) but the *default* half is unverified, so
treat this route as still open until probed. A
collaborator's own token always can. `docker-build.yml` publishes `:prod` from a `v*` tag push and a tag
`write:repository` bot PAT in the same store and still cannot be scoped down, because Renovate needs
repo write; what changed is that a verdict posted with it is no longer *inherited*`renovate` is not
in `H10_REVIEWERS`, so `review-verdict.yml` re-derives it (#742). It remains status-capable, so it can
still overwrite a status; it can no longer impersonate a reviewer. The injected `GITEA_TOKEN` was write-capable in every job.
That is no longer the mechanism: #748 declared `permissions:` on all six workflows (2026-08-27), and
because the declaration binds under the current `permissive` default, five hold `code: read` and only
`review-verdict.yml`'s gate job holds write. So the route is narrowed NOW, without the owner-level
flip. The owner-level Restricted default (server-management#714) remains worth applying as the
instance-wide floor for repos that declare nothing — but it is NOT yet shown safe here: whether a
declared `code: write` survives a Restricted default is unmeasured, and if Gitea caps at the default
the way GitHub does, the flip would make the gate unwritable. What is NOT closed: `review-verdict.yml`
still holds repo-write, so the property is "only a workflow visibly opting into write can assert the
verdict", not "no workflow can". Scoping still bounds what a job may DO, never whether attacker YAML
RUNS. A
collaborator's own token always can — and after #742 an ALLOW-LISTED collaborator's still is
inherited, which is the residual the allow-list explicitly does not touch. `docker-build.yml` publishes `:prod` from a `v*` tag push and a tag
may point at ANY commit — a prod image with no PR, review or status (tag protections are empty).
**And none of it was necessary: direct pushes to `main` were server-side permitted, so the gate was
bypassable with no forgery at all (#743).** That route is now closed — `main` carries
@@ -1,19 +1,26 @@
---
key: ci.batch-pushes-no-cancel-route
title: '2026-07-21 — Batch your pushes: there is no agent-side cancel route on Gitea 1.25.4 (#542)'
title: '2026-07-21 — Batch your pushes: there is no agent-side cancel route on Gitea 1.25.4; REST route re-probed on 1.27.1 (#542)'
status: active
since: '2026-07-21'
supersedes: none
superseded-by: none
rule: Hold review fixes, doc corrections and format fixes locally and push **once** — a superseded run cannot be cancelled from the agent side and holds a runner slot until it finishes.
signals: 'cancel run 404 · Gitea 1.25.4 · `POST /api/v1/.../actions/runs/{id}/cancel` · MCP `actions_run_write` `cancel_run` · session+CSRF · `dispatch_workflow` · orphaned run · 4-slot runner · paths: n/a · issues: #542'
signals: 'cancel run 404 · Gitea 1.25.4 · Gitea 1.27.1 · `POST /api/v1/.../actions/runs/{id}/cancel` · MCP `actions_run_write` `cancel_run` · session+CSRF · `dispatch_workflow` · orphaned run · 4-slot runner · paths: n/a · issues: #542'
mechanics: Gitea Actions REST + MCP `actions_run_write`; operator-only cancel in the browser UI.
---
Cancellation is impossible from the agent side on this Gitea (**1.25.4**):
Cancellation is impossible from the agent side on this Gitea (measured on **1.25.4**):
`POST /api/v1/.../actions/runs/{id}/cancel` returns **404**, MCP `actions_run_write`'s `cancel_run`
returns **404**, and the web-UI route needs a session + CSRF that does not script (login 303s with no
session cookie). **Only the operator can cancel, in the browser** — so if you must supersede a live
session cookie).
**Re-probed on 1.27.1, 2026-08-28 (#747) — the REST route only.** `POST .../cancel` still returns
**404**, and the live swagger shows the absence is structural rather than one 404: the only POST routes
under `actions/runs/{run}` are `rerun`, `rerun-failed-jobs` and `jobs/{job_id}/rerun`. The MCP
`cancel_run` and web-UI/CSRF findings above were **not** re-run and stay 1.25.4-dated. Swagger also
lists `DELETE /actions/runs/{run}` (delete a run record); it was not probed — do not assume it cancels
a run or frees a runner slot. **Only the operator can cancel, in the browser** — so if you must supersede a live
run, say so explicitly instead of leaving it burning. (`dispatch_workflow` is a different route and
still works for re-triggering a **main** run.)
@@ -11,7 +11,9 @@ mechanics: '`docs/ci-cd.md` → "Docs-only skip"; verified via a real shallow `f
---
The #416 docs-only skip shipped (#422) safe but **ineffective**: every docs-only PR still ran the full
matrix. Root cause — `test`/`migrations` check out `fetch-depth: 1`, and in a shallow clone
matrix. Root cause — the consumer jobs check out shallow (`fetch-depth: 1` then, 2 now, and the
consumers are `test`, `migrations` and `functional-e2e`; both depths are shallow, so the rule is
unaffected by the change), and in a shallow clone
`origin/<base>` has no remote-tracking ref and there is no merge-base, so the detect script's
three-dot `git diff origin/main...HEAD` errored → `|| true` → empty diff → the fail-safe returned
`docs_only=false` → full matrix. Confirmed in a real shallow `file://` clone (`origin/main` did not
@@ -20,7 +20,10 @@ docs-only PR produced **no run** for them, those contexts never report and the P
— the naive fix bricks docs PRs instead of speeding them. A probe (throwaway PR #418) confirmed that
on Gitea **1.25.4** an `if:`-skipped job reports commit-status state **`skipped`** (a distinct state,
not `success`); how branch protection treats a `skipped` *required* context is not something we rely
on.
on. **Re-confirmed unchanged on 1.27.1, 2026-08-28 (#747)**, without a new probe PR — `Build & push
image (amd64)` is `if:`-skipped on every PR, and reports `skipped` — a state distinct from `success`
— on the heads of PR #829 (`e5a01dc84100`) and PR #828 (`02bb0393`). The distinct state survived the
upgrade, so the reason this decision gates STEPS rather than jobs still holds.
**The decision.** Each heavy job (`test`, `migrations`, `functional-e2e`, `build`) runs
`scripts/ci-detect-docs-only.sh` as its first post-checkout step (`id: detect`) and gates every real
@@ -5,9 +5,9 @@ status: active
since: '2026-07-29'
supersedes: none
superseded-by: none
rule: 'The three inputs the exemption decision rests on must each be bound to something the judged PR cannot mutate. (1) BASE — `scripts/pr-changed-files.sh` takes the expected base BRANCH as a REQUIRED 5th argument and re-reads it before and after paging, because `/pulls/{n}/files` diffs against the PR''s live base and retargeting moves the answer without moving the head sha; the workflow passes `github.event.pull_request.base.ref` from the `pull_request_target` payload, which a retarget cannot rewrite. (2) BOT EXEMPTION — an author match is necessary but never sufficient: `pull_request.user.login` is the PR''s immutable CREATOR while its head is not, so the exemption additionally requires EVERY changed path to be a dependency manifest (`Directory.Packages.props` or `.config/dotnet-tools.json`, and ONLY those — the npm manifests are excluded because `package.json` `scripts` are executed by CI). (3) INHERITED SUCCESS — the never-overwrite short-circuit fires only for a status POSITIVELY identified as a human verdict for THIS base, meaning a non-null `.creator.login` AND a `Review-verdict:` description AND, when that description records a base (`(base: …)`, `release.verdict-status-check`), a base matching the PR''s — tested by requiring the description to END with the exact literal `(base: <base>)` and to contain exactly ONE such marker, never by extracting a value (see below); a present-but-different base is rejected, an absent one is not, since verdicts predating that convention carry none; every other shape, including any unrecognised one, is re-derived rather than trusted. The bot and docs-only exemptions are evaluated as INDEPENDENT predicates and the decision made afterwards, never as an `elif` chain. `edited` is in the workflow''s `types:` so a retarget reclassifies — which gives DETECTION, not atomicity: status writes are not serialized, so a stale run can still post over a fresher one. That residual is now FENCED rather than merely tracked — the job refuses to write at all if the PR''s timeline retarget COUNT moved while it was classifying (`ci.verdict-write-retarget-fence`, #706) — leaving only the sub-round-trip window that no API without compare-and-set can close. The PROTECTED path list additionally covers `CLAUDE.md` and `AGENTS.md` (#751) — they are not prose but the documents DEFINING the completion protocol, the merge-consent convention and the H10 rule, so protecting `.claude/` while the file specifying what it enforces stayed docs-only-exempt was the same self-exemption one directory over; driving the real classify body with a lone `CLAUDE.md` change produced an exemption `success`. `README.md` is deliberately not listed. It also covers `.codex/` (#711), which mirrors `.claude/hooks/` byte for byte including the merge-consent hook — latent while that directory is untracked, live the moment it is tracked; the list stays ENUMERATIVE rather than derived, because a derived rule would have to be evaluated against the very file list being classified. Reading the CURRENT status for input (3) must tolerate `statuses: null`: `GET /commits/{sha}/status` serialises a nil slice as `null`, not `[]`, on a head with no statuses yet, and an `array`-only gate made `read_existing_verdict` `exit 1` and post nothing at all (#751, `ci.workflow-run-body-no-expressions`) — `null` is accepted only when `total_count` is 0, so a body that merely lost its array is still refused. Path predicates are evaluated by COUNTING with `grep -c`, never `| grep -q` (SIGPIPE inversion) and never a here-string (temp-space failure) — see `ci.grep-q-pipefail-inversion`.'
signals: 'forged review-verdict exemption, retarget race against the docs-only classifier, PR base changed mid-run, hijacked Renovate branch, bot exemption on a code change, machine-written success inherited as a verdict, status creator null vs user, never overwrite a human verdict, exemption chain skips docs-only for bots, why is my Renovate PR asking for a verdict, base ref binding on pr-changed-files.sh · paths: `.gitea/workflows/review-verdict.yml`, `scripts/pr-changed-files.sh`, `.claude/hooks/pretooluse-merge-consent.sh`, `scripts/tests/test_pr_changed_files.py` · issues: #698, #697, #672, #663, #649, #632'
mechanics: '`scripts/pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha> <expected-base-ref>` (5 args; a 4-arg call exits 2); workflow env `BASE_REF: ${{ github.event.pull_request.base.ref }}`; `BOT_MANIFESTS` anchored allow-list; short-circuit requires `.creator.login` non-null AND description matching `^Review-verdict:`; `types: [opened, reopened, synchronize, ready_for_review, edited]`'
rule: 'The three inputs the exemption decision rests on must each be bound to something the judged PR cannot mutate. (1) BASE — `scripts/pr-changed-files.sh` takes the expected base BRANCH as a REQUIRED 5th argument and re-reads it before and after paging, because `/pulls/{n}/files` diffs against the PR''s live base and retargeting moves the answer without moving the head sha; the workflow passes `github.event.pull_request.base.ref` from the `pull_request_target` payload, which a retarget cannot rewrite. (2) BOT EXEMPTION — an author match is necessary but never sufficient: `pull_request.user.login` is the PR''s immutable CREATOR while its head is not, so the exemption additionally requires EVERY changed path to be a dependency manifest (`Directory.Packages.props` or `.config/dotnet-tools.json`, and ONLY those — the npm manifests are excluded because `package.json` `scripts` are executed by CI). (3) INHERITED SUCCESS — the never-overwrite short-circuit fires for a `success` only when it is POSITIVELY identified as an APPROVED REVIEWER''s verdict for THIS base, meaning a `.creator.login` that is a member of the workflow''s `H10_REVIEWERS` allow-list (an existing `failure` takes the weaker ATTRIBUTABLE test and needs no membership, because inheriting a rejection can only withhold an exemption while re-deriving one can turn it green on an exempt PR) — non-null was too weak, since any account''s credential satisfies it and `RENOVATE_TOKEN` is one that cannot be scoped down (#742) — AND a `Review-verdict:` description AND, when that description records a base (`(base: …)`, `release.verdict-status-check`), a base matching the PR''s — tested by requiring the description to END with the exact literal `(base: <base>)` and to contain exactly ONE such marker, never by extracting a value (see below); a present-but-different base is rejected, an absent one is not, since verdicts predating that convention carry none; every other shape, including any unrecognised one, is re-derived rather than trusted. The bot and docs-only exemptions are evaluated as INDEPENDENT predicates and the decision made afterwards, never as an `elif` chain. `edited` is in the workflow''s `types:` so a retarget reclassifies — which gives DETECTION, not atomicity: status writes are not serialized, so a stale run can still post over a fresher one. That residual is now FENCED rather than merely tracked (NARROWED, not resolved — a retarget between the fence''s final count and the POST still leaves a PERMANENT forged green, #849) — the job refuses to write at all if the PR''s timeline retarget COUNT moved while it was classifying (`ci.verdict-write-retarget-fence`, #706) — leaving TWO windows, not one: the sub-round-trip window that no API without compare-and-set can close, AND the post-final-count/pre-POST window, which is not sub-round-trip and yields a PERMANENT green because the successor can consume the `edited` event and exit before the stale run posts last (#849). The PROTECTED path list additionally covers `CLAUDE.md` and `AGENTS.md` (#751) — they are not prose but the documents DEFINING the completion protocol, the merge-consent convention and the H10 rule, so protecting `.claude/` while the file specifying what it enforces stayed docs-only-exempt was the same self-exemption one directory over; driving the real classify body with a lone `CLAUDE.md` change produced an exemption `success`. `README.md` is deliberately not listed. It also covers `.codex/` (#711), which mirrors `.claude/hooks/` byte for byte including the merge-consent hook — latent while that directory is untracked, live the moment it is tracked; the list stays ENUMERATIVE rather than derived, because a derived rule would have to be evaluated against the very file list being classified. Reading the CURRENT status for input (3) must tolerate `statuses: null`: `GET /commits/{sha}/status` serialises a nil slice as `null`, not `[]`, on a head with no statuses yet, and an `array`-only gate made `read_existing_verdict` `exit 1` and post nothing at all (#751, `ci.workflow-run-body-no-expressions`) — `null` is accepted only when `total_count` is 0, so a body that merely lost its array is still refused. Path predicates are evaluated by COUNTING with `grep -c`, never `| grep -q` (SIGPIPE inversion) and never a here-string (temp-space failure) — see `ci.grep-q-pipefail-inversion`.'
signals: 'verdict inherited from a bot token, RENOVATE_TOKEN posts a review verdict, who may post an h10 verdict, H10_REVIEWERS allow-list, forged review-verdict exemption, retarget race against the docs-only classifier, PR base changed mid-run, hijacked Renovate branch, bot exemption on a code change, machine-written success inherited as a verdict, status creator null vs user, never overwrite a human verdict, exemption chain skips docs-only for bots, why is my Renovate PR asking for a verdict, base ref binding on pr-changed-files.sh · paths: `.gitea/workflows/review-verdict.yml`, `scripts/pr-changed-files.sh`, `.claude/hooks/pretooluse-merge-consent.sh`, `scripts/tests/test_pr_changed_files.py` · issues: #849, #845, #742, #698, #697, #672, #663, #649, #632'
mechanics: '`scripts/pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha> <expected-base-ref>` (5 args; a 4-arg call exits 2); workflow env `BASE_REF: ${{ github.event.pull_request.base.ref }}`; `BOT_MANIFESTS` anchored allow-list; `H10_REVIEWERS="timothy"` declared as a literal in the classify body (a repo `vars:` entry or secret is settable without a PR); membership by exact POSIX `[ = ]` comparison over word-split entries, never an UNQUOTED right-hand side — `[[ $rv == $ex_creator ]]` makes it a pattern, while a quoted RHS under `[[ ]]` is literal (probed, bash 5.3); POSIX `[ = ]` cannot glob at all, which is why it is used and never `grep -xF` (an API-supplied creator containing a newline reads as several patterns); `read_existing_verdict` sets `ex_attributable` (creator non-null AND `^Review-verdict:` AND base OK) and derives `ex_human` from it by membership; the INHERITANCE short-circuit reads `ex_human`, the LAST-MOMENT RE-READ reads `ex_attributable` plus a change against the state/creator/description triple snapshotted at the first read, and the post-write raced `jq` stays broad on `creator != null`; `types: [opened, reopened, synchronize, ready_for_review, edited]`'
---
`ci.gate-trigger-base-resolved` stopped a PR supplying the gate's own *definition*. This closes the
@@ -21,7 +21,9 @@ mid-flight. The job enumerated against the moved base, read docs-only, and poste
reclassified** (`created_at == updated_at`), leaving a PR into `main` whose diff carried a C# file
behind a green required check. Closed unmerged, branches deleted, no forged `h10` left anywhere.
**Why a base BINDING and not a pinned diff.** Diffing two immutable shas would close it outright;
Gitea 1.25.4 cannot serve that — measured: `compare/{base}...{head}` returns no `files`, and a
Gitea cannot serve that: `compare/{base}...{head}` returns an object whose only keys are `commits`
and `total_commits`, no `files` — measured on 1.25.4 and **re-confirmed on 1.27.1, 2026-08-28 (#747)**,
probed on `b16ec15d6c1e...9b4844c1a8a9`, 5 commits. Separately, and **not re-probed since 1.25.4**, a
`--depth=1` fetch of the two shas has no merge base, so three-dot is impossible and two-dot
over-reports everything `main` gained since the branch point. So the base is read before the first page
and after the last, and **the gap is stated plainly**: a retarget opening *and* closing strictly
@@ -36,7 +38,7 @@ than described as fixed. `edited` and re-derivation remain one fix — `edited`
on the existing `success`; re-derivation alone never gets a second run — but together they are
mitigation, not a guarantee.
**Resolved 2026-08-03 (#706), and worth recording that the guarantee finally came from somewhere else
**Narrowed 2026-08-03 (#706) an earlier heading here said RESOLVED, corrected 2026-08-27 (#849): the fence never re-counts after its final pre-write read, so the post-final-count/pre-POST window remains, and is PERMANENT rather than transient. Worth recording that the narrowing finally came from somewhere else
entirely.** The missing piece was never ordering: `ci.verdict-write-retarget-fence` leaves the runs as
unserialized as they ever were and instead makes a run that was overtaken decline to write, keyed on
the timeline's monotonic retarget COUNT — the one signal the `main → scratch → main` ABA cannot make
@@ -45,7 +47,10 @@ overlap live (probe PR #722, the older run finishing 20s after the newer one sta
non-cancelling concurrency group — the fix this record's residual implied and #706 proposed — was
measured doing nothing at all. The paragraph above stands as written; only its last sentence is
overtaken, and the sub-round-trip window it describes survives, because Gitea's status API has no
compare-and-set.
compare-and-set. **That is not the only window** — see the correction to
`ci.verdict-write-retarget-fence` (2026-08-27, #849): the fence never re-counts after its final
pre-write read, so a retarget landing between that read and the POST leaves a PERMANENT forged green
rather than a transient one, whenever the successor run consumes the `edited` event and exits first.
**Route 2 — a bot ACCOUNT does not attribute the CODE.** `pull_request.user.login` is the PR's
immutable *creator*; its head is not. Push application code onto an open Renovate branch and the PR is
@@ -90,5 +95,154 @@ a PR, because `pull_request_target` resolves this definition from `main` — the
executed-behaviour tests in `scripts/tests/test_pr_changed_files.py`, each verified by mutation; the
live re-check happens on `main` right after merge.
**Route 2 was reproduced live 2026-08-26 (#708), and it is REFUSED.** Application code was pushed onto
the branch of Renovate PR #761 (guarded first with a `WIP:` title prefix, confirmed `draft: true` /
`mergeable: false`, so the hijacked head could not merge), then the branch was force-restored to its
recorded sha and the PR verified byte-identical across title, head sha, base ref, draft, mergeable,
state, labels, body hash, author and milestone. With `AUTHOR: renovate` and a changed set of
`Directory.Packages.props` **and** `ErsatzTV.Core/Probe708.cs`, the classifier logged
`Decision: state=pending — authored by the 'renovate' bot account, but changes files outside the
dependency-manifest set`, and posted `review-verdict/h10=pending`. Before the manifest allow-list the
same head would have been `success` on identity alone. The classify step's start marker was present,
so this is a real classification and not the #751 silent no-op.
**Where the refusal reason surfaces — the log, NOT the status description.** The route-2 explanation
lives in `reason`, which is only `echo`ed as `Decision: state=… — <reason>`. Of the three description
branches, two are distinctive — `Exempt: <reason>` and the raced-repair sentinel `REPAIR_DESC` — and
every *other* non-exempt head falls to the generic `Awaiting review verdict for <sha7>`. So a Route-2
bot refusal and an ordinary awaiting-verdict head are indistinguishable from the status alone;
confirming *which branch* the classifier took means reading the job log. #708's own text expected the
reason in the description; it is not there.
**Cost observed on the same PR, on its ORIGINAL head — not a defect in this record's mechanism.**
Re-running the gate on `8798a1d2` (the unmutated head, re-triggered by the `WIP:` title edit via
`types: [… edited]`, and five minutes *before* the code push) repaired its exemption to `pending` via
#751's conservative page-2 rule: that head's status history runs past `MAX_RESPONSE_ITEMS`, so *any*
page-2 rows are assumed raced. It could not have happened on the probe head — the page-2 check sits on
the `exempt=yes` path, which a Route-2 refusal never reaches. That is the blunt-instrument case `#763`
already predicts, now with a live instance of it; see that issue.
**Tightened 2026-08-26 (#742): non-null creator -> an allow-list.** The route-3 test asked "was this
POSTed with a user credential", and *any* account's credential answered yes. `RENOVATE_TOKEN` is a
`write:repository` PAT of the `renovate` bot: unlike `#697`'s registry credential it **cannot** be
scoped off status-write, because Renovate genuinely needs repo write to push branches and open PRs,
and secrets are a per-repo *store* rather than a per-workflow grant — so a workflow can reference it
and POST a `Review-verdict:` success that reads back `creator: renovate` — non-null, and therefore
inheritable. **Inferred from a measurement, not an observed event:** the user-credential-vs-Actions-token
asymmetry is measured, but `renovate` has never actually posted a commit status in this repo — every
status row on all six Renovate-authored PR heads carries `creator: null`. An earlier draft's past tense
("was inherited") described something nobody saw.
Membership in `H10_REVIEWERS` restores the intended set for an inherited `success`: every other
account's, recognised or not, re-derives exactly as an Actions-token status already does. (An existing
`failure` is governed by the weaker attributability test instead — see the state asymmetry below.) The list is a literal in this
base-resolved definition rather than a repo `vars:` entry, because a `vars:` entry is settable without
a PR — i.e. without this gate — while `.gitea/` sits on the PROTECTED path list, so editing the
allow-list cannot ride a docs-only exemption.
**`read_existing_verdict` answers TWO questions and therefore carries TWO flags.** It is called
twice, and the callers want opposite widths. `ex_attributable` — an attributable `Review-verdict:` row
for THIS base, whoever wrote it — is what the LAST-MOMENT RE-READ uses, because that site asks "did a
reviewer post a verdict while we were classifying"; narrowing it does not make it stricter, it makes
the job stop abstaining and post its exemption straight over the row, and the post-write repair does
NOT cover that, being skipped whenever the high-water mark could not be established. `ex_human` is
`ex_attributable` plus membership, and only the INHERITANCE short-circuit uses it. The first draft
gated the single flag and so narrowed both sites; cold review caught it, and
`test_the_MID_RUN_abstain_still_fires_for_a_NON_allowlisted_creator` pins it.
**The two calls must compute a predicate that "changed" is measurable against.** Before the split they
were identical, so anything firing at the second call and not the first had arrived in between — the
mid-run test got that for free. Splitting the flags broke it: a pre-existing row the job declined to
INHERIT is still attributable, so the re-read abstained on the very row the classification had just
decided to overwrite, left the forgery standing and posted nothing at all. The state/creator/
description triple seen at the first read is therefore snapshotted and compared, making "changed"
explicit. A REPLACEMENT counts as a mid-run event, not only an arrival: the combined endpoint returns
one row per context, so a newer POST replaces rather than adds — MEASURED on this instance 2026-08-26, not assumed, because the
comparison made that load-bearing where presence alone had not: two POSTs of context
`probe742/combined-newest` to one sha (`pending`, then `success`) came back as ONE row on
`/commits/{sha}/status`, the newest, and as BOTH rows on `/statuses/{sha}`.
**The allow-list governs `success` ONLY. Treating the two states alike is a fail-open.** Trace an off-list `failure` under a symmetric rule: a real reviewer left off
the literal posts `BLOCKED`; `ex_human=no`, so the short-circuit declines; the PR is docs-only, so the
classifier exempts it; the re-read triple never changed, so the mid-run abstain declines; the row
predates the high-water mark, so the post-write repair cannot see it — and the job posts `success` over
an explicit rejection, reached by the very misconfiguration the allow-list's own diagnostic is about.
The asymmetry costs nothing the allow-list bought: inheriting a `failure` cannot green anything, the
worst a forged one achieves is a stall, and `post-review-verdict.sh` POSTs unconditionally so any human
clears it in one command. Pinned by `test_an_off_list_FAILURE_is_LEFT_ALONE_even_on_an_EXEMPT_PR`,
which uses a DOCS-ONLY file list on purpose — with a code file the job posts `pending` anyway and the
test would pass measuring nothing.
**Why the post-write raced check stays broad — and NOT for the reason first written.** The first draft
justified it as "narrowing it would leave an exemption success standing over a rejection". That is not a checked reason: a genuine reviewer is on `H10_REVIEWERS` by construction, so
membership there would still count every real verdict, and an off-list row is one the job deliberately
overwrote a few lines earlier. What is left once `failure` is handled at the short-circuit is a
NARROWER net than an earlier draft claimed: it covers a misconfigured reviewer's verdict landing inside
the WRITE WINDOW, and nothing else — a verdict already on the head when the job starts sits below the
high-water mark and is structurally invisible to it. Saying it is "the last net before a green stands"
overstated that, and the sentence is corrected rather than kept. It stays broad because within its
window the errors are not symmetric: a repair to `pending` is recoverable by re-posting, a
wrongly-standing `success` on an unreviewed head is not. The cost is stated at the site rather than
implied: any verdict-shaped row landing inside that window trips the STICKY `REPAIR_DESC` sentinel and
costs that head its exemption until a human clears it.
Both directions are mutation-proven against the shipped file — reverting the membership clause reddens
`test_a_verdict_shaped_SUCCESS_from_a_NON_allowlisted_account_is_RE_DERIVED` and
`test_an_account_that_merely_RESEMBLES_an_allowlisted_login_is_RE_DERIVED`; pointing the mid-run
abstain back at `ex_human`, or dropping the changed-row comparison, reddens the mid-run and
inheritance cases respectively. Counts are deliberately not quoted: `test_pr_changed_files.py` is
graded `PROOF | NONE`, so nothing re-executes a number written here and one added `parametrize` case
would falsify it silently.
**A guard being byte-identical does not make its REACHABILITY unchanged — the post-write residuals
are #849, and this change makes them reachable from a new class.** Before this change, an attributable
`success` from ANY account short-circuited and the job wrote NOTHING. Re-deriving an off-list `success`
is the POINT of the change, and it turns "no write" into "a write" — which is what reaches the paths
where post-write verification cannot verify. The guards themselves are untouched here; their
reachability is not, and "pre-existing" was the wrong answer to that.
**Two of them were fixed on this branch and then WITHDRAWN, which is the more useful record.** The fix
withheld the exemption when no high-water mark could be established, and repaired to the sentinel when
the post-write history was unreadable. Re-review found both wanting, on grounds that generalise:
- The withheld exemption wrote a GENERIC `pending`, and a generic `pending` is exactly what a later run
re-derives into `success`. It did not close the burial route, it moved which run greens it. Closing it
properly needs a distinct sticky "unverified write" sentinel, which is a new state in a state machine
that already has three.
- That `pending` also has NO RETRY PATH. The workflow triggers only on `pull_request_target` types —
no schedule, no `workflow_dispatch` — so a transient API failure on a PR's LAST event leaves an
exempt PR stalled until a human nudges it. Trading a race that needs BOTH a read failure and a
reviewer posting inside the write window, for a stall that needs only the read failure, is not
obviously the safe direction, and it was described as a "one-run cost" when it is not.
So the whole of post-write verification stays in #849, where it can be designed once: every write
verified when a mark exists, a sticky sentinel when it does not, and a CHAINED test feeding run N's
real output into run N+1. Two lines inside a provenance change was the wrong size for it — the attempt
is recorded because the next reader will otherwise re-attempt it the same way.
**A coupling this record does NOT assert, tracked as #845 instead.** `post-review-verdict.sh` posts with
whatever account owns `ETV_GITEA_TOKEN`/`ETV_GITEA_BASICAUTH` and never checks whose it is. If that
ever stops being an allow-listed account, every verdict silently stops being inheritable and each
later event re-derives it — to `pending` on a PR that needs a verdict, or to an exemption `success` on
a docs-only or Renovate-manifest one, which is the quieter and worse half. Either way the verdict is
gone, and the only diagnostic is a line in a job log.
Correct today — verified live 2026-08-26, reproduce with `GET /api/v1/user` (login `timothy`) and
`GET /api/v1/admin/users?limit=50` (exactly `renovate`, `timothy`) — but nothing keeps it correct
(#845). The job DOES name the creator and the allow-list whenever it declines to inherit — but in the
WORKFLOW RUN, while the mistake is made at the terminal running `post-review-verdict.sh`, which
reports success either way. That gap is what #845 is about, and it is why a louder annotation was not
the answer: an earlier draft added a `::warning::` alongside that line, and it was withdrawn after
producing three defects in three rounds (a message that contradicted itself for an allow-listed
creator, an unescaped API value interpolated into a workflow command, and a test asserting its absence
on a path that exits two guards earlier and so could never fail). The signal has to reach the WRITER,
not be repeated more loudly to the reader.
**What #742 does NOT close, restated because the allow-list invites the wrong reading.** An
allow-listed collaborator's own write-scoped token can still POST a verdict nobody formed. Branch
protection binds the context NAME, not its issuer, and Gitea offers no way to restrict who may post a
context. `h10` remains a process guard against mistakes and drift, not a security boundary against
push access. The advisory hook `.claude/hooks/pretooluse-merge-consent.sh` checks no identity at all
and is deliberately left that way: it runs as the operator, on the operator's machine, and its
strictest outcome is a human prompt.
**Separate defect found reviewing this change:** `ci.grep-q-pipefail-inversion` — a pre-existing
SIGPIPE inversion that let a large PR skip the `PROTECTED` guard entirely. Fixed in the same PR.
@@ -5,7 +5,7 @@ status: active
since: '2026-07-28'
supersedes: none
superseded-by: none
rule: 'The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR''s own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch''s gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree''s scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class: Gitea injects a write-capable `GITEA_TOKEN` into EVERY job, so any ref-resolved workflow — and a collaborator''s own API token, since branch protection binds the context and not its issuer — can still forge `review-verdict/h10`. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI''s registry secret was the ADMIN account''s basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential''s route (a user credential''s forgery carries a real `creator` and is inherited as a human verdict; an Actions job''s carries `creator: null` and is re-derived — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger''s `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The injected token stays write-capable until Gitea >=1.26 with a Restricted default (server-management#714), and a collaborator''s own token remains unfixable; the exemption path has its own separate defects in #698.'
rule: 'The workflow that writes the branch-protection-required `review-verdict/h10` status triggers on `pull_request_target` with `branches: [main]`, never on plain `pull_request`. Gitea resolves a `pull_request` workflow DEFINITION from the PR''s own head commit, so under that trigger a PR editing `.gitea/workflows/review-verdict.yml` ran its own rewritten copy and could post `h10=success` for itself; `pull_request_target` resolves the definition from the base instead. The `branches: [main]` filter is part of the rule, not a refinement of it: base resolution only relocates the rewrite from the head to the base, so without the filter a PR opened into an attacker-pushed base branch runs that branch''s gate. `pull_request_target` is safe HERE only because this job never checks out or executes head-supplied code — it checks out `base.sha` and runs only that tree''s scripts (`ci.shared-pr-file-enumeration`); reintroducing a head checkout under this trigger would be worse than the bug it fixed. This closes the rewrite route through THIS workflow and does NOT close the class. Gitea injects `GITEA_TOKEN` into EVERY job and it USED to be write-capable everywhere, so any ref-resolved workflow could forge `review-verdict/h10`; since #748 (2026-08-27) all six workflows here declare `permissions:` — five `code: read`, and `review-verdict.yml`''s gate job `code: write`, deliberately, because it IS the gate (the five rest on the TOP-LEVEL form, which is INFERRED rather than probed — the gate job''s JOB-LEVEL form is the one measured; see `ci.actions-credential-scoping`). That narrows the INJECTED token and only that: deliberately NO "only X can post a status" summary is stated here, because any such summary generalizes from the injected token, which `permissions:` scopes, to every credential a job can reach, which it does not touch — `RENOVATE_TOKEN` is in the same secret store and is referenced by `renovate.yml`, one of the five holding `code: read`. `ci-image.yml` was the last holdout — its unfiltered `push:` made it the worst-placed of the six — and #744 closed that trigger route (`ci.toolchain-image-publish-is-a-dispatch`) while #748 scoped its token. So the route is NARROWED, not closed, and the SHAPE is open — a newly added workflow declaring write, or omitting a declaration and inheriting the owner default, is a route again, which is why `test_pr_changed_files.py` asserts every tracked workflow declares a block. And a collaborator''s own API token still can, since branch protection binds the context and not its issuer. The credential half is now RESOLVED in `ci.actions-credential-scoping` (#697): CI''s registry secret was the ADMIN account''s basic auth and is now a PAT that cannot post a status, which removes the ADMIN escalation and that credential''s route (an ALLOW-LISTED user credential''s forgery carries a matching `creator` and is inherited as a verdict; an Actions job''s carries `creator: null` and is re-derived, as since #742 is every account outside `H10_REVIEWERS` — but do NOT read that asymmetry as protection: re-derivation fires only on the trigger''s `types`, and posting a status is not one of them, so a POST timed after the last PR event simply stands). It does not remove EVERY route: `RENOVATE_TOKEN` is a `write:repository` bot PAT in the same secret store, reachable by any PR-added workflow. The owner-level Restricted default (server-management#714) remains a separate OPEN lever — its effect on a declared `code: write` is unmeasured, and the version half of that condition has been satisfied since the 1.25.4 -> 1.27.1 upgrade on 2026-08-05, so it is not something to wait on a Gitea release for. A collaborator''s own token remains unfixable; the exemption path has its own separate defects in #698.'
signals: 'workflow definition resolved from head, PR rewrites the gate that judges it, self-approve a required status check, pull_request_target vs pull_request, gate trigger branches filter, attacker-supplied base branch, how to test a change to review-verdict.yml, workflow not exercised by its own PR, gate edit goes live only on merge, required_approvals 0 does not bind an author, forged commit status inherited by sha · paths: `.gitea/workflows/review-verdict.yml`, `scripts/tests/test_pr_changed_files.py` · issues: #672, #663, #649, #622'
mechanics: '`on: pull_request_target: {branches: [main], types: [opened, reopened, synchronize, ready_for_review, edited]}` (`edited` added by `ci.exemption-provenance` so a retarget reclassifies); asserted by `test_the_workflow_trigger_is_pull_request_TARGET_scoped_to_main` in `scripts/tests/test_pr_changed_files.py`; the job''s own context is renamed to `... (pull_request_target)` and must stay OUT of branch protection''s required list'
---
@@ -28,6 +28,11 @@ and the base definition posted `h10=pending` on `opened` and `synchronize` alike
establishing the residual below — is counted because omitting it turns an honest partial into an
overclaim. Probes posted only probe-named contexts, never a forged `h10`. Full results in #699.
That third item was **re-confirmed at the STATUS level on 1.27.1, 2026-08-28 (#747)**, as a by-product
of an unrelated probe (PR #863): a PR from a scratch head into a scratch base carried thirteen `pull_request`
contexts and no review-verdict context of any kind. The run list itself was not re-enumerated, so the
observation is also consistent with a run that started and posted nothing.
**Why `branches: [main]` is load-bearing rather than tidy.** The *base branch* supplies the
definition, and anyone who can push a branch can make it a base — so dropping the filter trades a
head-supplied gate for a base-supplied one and closes nothing. Worse than lateral: a commit status is
@@ -50,14 +55,31 @@ authority the judged thing can assert is not a gate, not because an attack was e
**The class is NOT closed, and this record must not be read as claiming otherwise.** This fixed one
instance of "a ref-resolved workflow can obtain credentials that POST a commit status", and that
inventory is not a short list: Gitea injects `GITEA_TOKEN` into **every** job, defaulting to
read/**write**, so head-resolved, `push`-triggered and `workflow_dispatch` workflows alike are routes
(1.24+ loads a dispatched definition from the selected branch). A collaborator's own API token is a
inventory is not a short list. Gitea injects `GITEA_TOKEN` into **every** job, and it used to
default to read/**write**, so head-resolved, `push`-triggered and `workflow_dispatch` workflows alike
were routes (1.24+ loads a dispatched definition from the selected branch). Narrowed since #748
(2026-08-27): all six workflows here now declare `permissions:` explicitly — five `code: read`, and
`review-verdict.yml`'s gate job `code: write`, deliberately, because it IS the gate. (The five rest
on the TOP-LEVEL declaration form, which is inferred rather than probed; the gate job's JOB-LEVEL
form is the one that was measured — see `ci.actions-credential-scoping`.) `ci-image.yml` was the last
holdout: its unfiltered `push:` trigger made it the worst-placed of the six, and #744 closed that
route (`ci.toolchain-image-publish-is-a-dispatch`) in the same week #748 scoped its token.
**What that narrows is the INJECTED token, and only that.** No one-line "only X can post a status"
summary belongs here: every such summary quietly generalizes from the injected token, which
`permissions:` scopes, to every credential a job can reach, which it does not touch. The falsifying
case is concrete and current — `RENOVATE_TOKEN` still writes statuses and is referenced by
`renovate.yml`, one of the five holding `code: read`. The SHAPE is open too: a new workflow that
declares write, or omits a declaration and inherits the owner default, is a route again, which is why
`test_pr_changed_files.py` asserts every tracked workflow declares a block. A collaborator's own API token is a
route with no workflow at all — branch protection binds the *context*, not its issuer. Full inventory
in `#697`, whose credential half is resolved in `ci.actions-credential-scoping` — the registry secret
no longer carries status-write. That does NOT leave the workflow routes provenance-free: any
PR-added workflow can reference `RENOVATE_TOKEN`, a `write:repository` bot PAT in the same store,
whose status carries a real creator and IS inherited (`#742`). The exemption path's
PR-added workflow can reference `RENOVATE_TOKEN`, a `write:repository` bot PAT in the same store. Its
status carries a real creator, and until `#742` that alone made it INHERITED as a verdict; the
provenance test is now membership in `H10_REVIEWERS`, so a `renovate`-posted verdict is re-derived
like an Actions-token one. The token still writes statuses — what it lost is durability, not the
ability to POST. The exemption path's
own defects are `#698`. No in-repository test can establish
status-authority isolation: the sibling guard added here catches only plain-text naming of the
context.
@@ -5,9 +5,9 @@ status: active
since: '2026-07-26'
supersedes: none
superseded-by: none
rule: 'A PR''s complete set of changed file paths is computed by exactly one implementation, `scripts/pr-changed-files.sh`, called by both `.claude/hooks/pretooluse-merge-consent.sh` (advisory — a failure falls through to a human prompt) and `.gitea/workflows/review-verdict.yml` (enforced — a failure must fail closed, because a match here posts the branch-protection-required `review-verdict/h10` status with nobody in the loop). The script owns exhaustiveness (pagination, rename/path validation, head-sha binding, base-ref binding — see `ci.exemption-provenance` — and base-TIP binding, #707: the ref answers "did this PR RETARGET", the tip answers "did the base ADVANCE mid-enumeration", and only the second can see `/pulls/{n}/files` recomputing each offset-paged page against a moved base and dropping a path out of an already-consumed range; both ends of the window are bound, and an advance BEFORE the window is deliberately not an error, or ordinary churn on `main` would fail every open PR) and returns exit 0 only for a verified-complete list; it does NOT classify paths — each caller keeps its own docs-only allow-list, and the two allow-lists differ on purpose and stay separate.'
rule: 'A PR''s complete set of changed file paths is computed by exactly one implementation, `scripts/pr-changed-files.sh`, called by both `.claude/hooks/pretooluse-merge-consent.sh` (advisory — a failure falls through to a human prompt) and `.gitea/workflows/review-verdict.yml` (enforced — a failure must fail closed, because a match here posts the branch-protection-required `review-verdict/h10` status with nobody in the loop). The script owns exhaustiveness (pagination, rename/path validation, head-sha and base-ref MOVEMENT DETECTION — one-way, never a binding: an A->B->A alias on either axis passes, #664/#803 — see `ci.exemption-provenance` — and base-TIP binding, #707: the ref answers "did this PR RETARGET", the tip answers "did the base ADVANCE mid-enumeration", and only the second can see `/pulls/{n}/files` recomputing each offset-paged page against a moved base and dropping a path out of an already-consumed range; both ends of the window are read and compared, and an advance BEFORE the window is deliberately not an error, or ordinary churn on `main` would fail every open PR) and returns exit 0 only for a verified-complete list; it does NOT classify paths — each caller keeps its own docs-only allow-list, and the two allow-lists differ on purpose and stay separate.'
signals: 'duplicated PR file enumeration, enforced gate weaker than advisory hook, docs-only allow-list drift, shared mechanism not shared policy, pr-changed-files.sh, checkout base ref not PR head, gate judging its own PR, exhaustiveness bug in a security predicate · paths: `scripts/pr-changed-files.sh`, `.claude/hooks/pretooluse-merge-consent.sh`, `.gitea/workflows/review-verdict.yml` · issues: #643, #648, #649'
mechanics: '`scripts/pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha> <expected-base-ref>` -> stdout newline-delimited paths, exit 0 only if complete and bound to BOTH the given sha and the given base branch; the 5th argument is REQUIRED and a 4-arg call exits 2 (`ci.exemption-provenance`); callers: `.claude/hooks/pretooluse-merge-consent.sh`, `.gitea/workflows/review-verdict.yml`'
mechanics: '`scripts/pr-changed-files.sh <owner> <repo> <pr> <expected-head-sha> <expected-base-ref>` -> stdout newline-delimited paths, exit 0 only if complete AND neither the given sha nor the given base branch was observed to move across the paging round trips — ONE-WAY detection, not a binding: an A->B->A alias on either axis passes (#664/#803), which is why the enforced caller adds a monotonic event-count fence; the 5th argument is REQUIRED and a 4-arg call exits 2 (`ci.exemption-provenance`); callers: `.claude/hooks/pretooluse-merge-consent.sh`, `.gitea/workflows/review-verdict.yml`'
---
Before #649, the PR changed-file enumeration existed as two independent implementations. That would
@@ -34,7 +34,8 @@ there is no mechanism that would surface a second drift before it mattered. Extr
atomically.
**Mechanism, not policy — the two allow-lists stay separate on purpose.** The extracted script
answers exactly one question: "what is the complete set of paths this PR touches, at one head, or can
answers exactly one question: "what is the complete set of paths this PR touches, with no head or base
movement observable from inside the enumeration, or can
we not tell?" It does not decide whether that set makes the PR docs-only. Each caller keeps its own
classification:
@@ -55,7 +56,14 @@ in each caller:
- CR/LF rejection and `..` rejection on every path.
- A closed `.status` allow-list — `added`/`deleted`/`changed`/`modified`/`renamed`/`copied`, not an
open denylist. Note `changed` and `deleted` are the values live Gitea 1.25.4 actually emits;
open denylist. The emitted vocabulary was re-derived on 1.27.1, 2026-08-28 (#747) over every file row
of the 200 most recently updated PRs. That is a large *sample*, not a constructed case — worth stating
because a 30-PR pass over the same corpus showed only `changed`/`added` and would have licensed a
false negative; widening it, not constructing a case, is what surfaced the rest. Four values appear in
that corpus: `changed` (1262), `added` (489), `renamed` (2), `deleted` (2) — all four already in the
allow-list, so those four are evidenced rather than presumed. `copied` and `modified` did NOT appear,
which does not show Gitea never emits them: no PR was constructed to force either. The earlier 1.25.4
measurement recorded here saw only `changed` and `deleted`; the 1.27.1 corpus is wider.
`modified` is accepted alongside `changed` because a closed list built from the wrong vocabulary
would gate every genuine docs-only PR. GitHub's `removed` is deliberately **not** in the list — an
earlier draft of this record said it was, which would have sent a maintainer looking for a value
@@ -71,9 +79,20 @@ in each caller:
would otherwise yield a list belonging to no single commit. **This detects ONE-WAY movement only.**
An A→B→A force-push round trip restores the expected sha, so the binding holds while the pages came
from two different states — see #664. Closing that needs a commit-pinned files endpoint (Gitea has
none) or a local diff, not a tighter check here; the guarantee is stated narrowly rather than left
none — re-probed at 1.27.1 on 2026-08-28: `compare/{base}...{head}` still returns no `files` key)
or a local diff, not a tighter check here; the guarantee is stated narrowly rather than left
to read as complete.
**Fenced at the ENFORCED caller since 2026-08-28 (#803), and only there.** The script's contract is
unchanged and still one-way, because nothing checkable inside it can do better. What changed is that
`review-verdict.yml` — the caller whose match posts a green required status with nobody in the loop —
now refuses to write if the PR timeline's `pull_push` count moved while it classified, a monotonic
key an alias cannot defeat (`ci.verdict-write-retarget-fence`). The advisory hook is deliberately not
given that fence: its failure mode is a human prompt, and it pays for the gap differently, by
re-reading `.head.sha` off the same response as its base re-read and DENYING if the head moved. So
the mechanism stays shared and single while the two callers keep buying different amounts of
protection with it — the same mechanism-not-policy split this record is about.
**Base-ref checkout — binds the SCRIPTS to the base, not the workflow itself.** `review-verdict.yml`
checks out the PR's BASE ref (`ref: ${{ github.event.pull_request.base.sha }}`,
`persist-credentials: false`), never the head, so the *scripts the job executes* — above all
@@ -0,0 +1,75 @@
---
key: ci.toolchain-image-publish-is-a-dispatch
title: '2026-08-27 — `ci-image.yml` publishes on `push` to main only; from a branch it is a `workflow_dispatch` (#744)'
status: active
since: '2026-08-27'
supersedes: none
superseded-by: none
rule: 'A `push` trigger reachable from any ref other than `main` — that is `branches:` AND `tags:`, judged by the ref class it admits and never by which keyword is present — executes ref-supplied YAML, because Gitea resolves a `push` workflow''s definition from the pushed ref — so `ci-image.yml` is `branches: [main]`, and publishing a toolchain image from a feature branch is a deliberate `workflow_dispatch` on that branch rather than a side effect of pushing. Be precise about what a `branches:` filter buys: it is loaded from the pushed ref like the rest of the file, so a branch that deletes it re-enables the route — this removes the DRIVE-BY case, and is not a boundary against a writer who intends to run their own YAML. The self-referencing trigger path `.gitea/workflows/ci-image.yml` came out of BOTH its own `paths:` and `ci-image-pin`''s `expected` in the same change — a DECIDED tradeoff, not a necessity: keeping it is workable via the branch dispatch, but prices every edit to that file, comments included, at a ~2GB publish plus a five-pin bump, redone after every rebase. The cost is stated, not assumed away — a change to HOW the image is built that lives only in `ci-image.yml` no longer republishes, and the ONLY remedy is to make it alongside a `docker/ci/**` edit: publishing after the merge and then pinning cannot work, because `expected` is the last `docker/ci` commit and would reject that pin. This closes the push route INTO THIS FILE, not the class: `docker-build.yml` remains reachable from an arbitrary ref by a `v*` tag push and by `pull_request`, and four workflows carry an unrestricted `workflow_dispatch` (#853).'
signals: 'unfiltered push trigger, branches filter on push, attacker YAML on a runner, feature branch publishes an image, self-referencing workflow trigger, ci-image-pin expected goes stale, publish-then-pin two-step, toolchain image recovery after a rebase, workflow_dispatch on a branch · paths: `.gitea/workflows/ci-image.yml`, `.gitea/workflows/pr-checks.yml`, `.gitea/workflows/docker-build.yml`, `docker/ci/Dockerfile`, `docs/ci-cd.md`, `docs/decisions/records/ci/actions-credential-scoping.md`, `scripts/e2e-ui.sh` · issues: #744, #853, #697, #746, #835, #445, #390, #594, #597'
mechanics: '`docs/ci-cd.md` -> "CI toolchain image" -> "Publishing from a branch is a dispatch, not a push"; branch-dispatch probe 2026-08-27 (run 2340, published `ersatztv-ci:43b1e45`)'
---
- **Credential scoping bounds what a job may DO; it never bounded whether the job RUNS.** That is the
seam `ci.actions-credential-scoping` explicitly leaves open, and this record closes one route through
it. `ci-image.yml`'s `push` trigger was path-scoped but unfiltered by branch, and Gitea resolves a
`push` workflow's definition from the pushed branch — so any branch push touching `docker/ci/**` or
the workflow file ran that branch's own YAML, with no PR, no review and no status check in the loop,
on a docker-capable runner holding the credential that writes `ersatztv:prod` and the
`ersatztv-ci:<sha>` that five `container:` jobs execute. `branches: [main]` closes the **drive-by**
route, and only that: the filter is loaded from the pushed ref like every other line in the file, so
a branch that deletes it re-enables the route in one edit. What goes away is publication as a side
effect of an ordinary push of a legitimate change — a real and frequently-trodden path, and not a
boundary against someone who intends to run their own YAML. Do not cite this record as that
boundary. `workflow_dispatch` is self-supplied from the selected branch in the same way, and is now
the deliberate publish path.
- **The capability removed was real, so it is replaced rather than argued away.** An earlier attempt
(#697) justified the filter with "publishing from a feature branch was not load-bearing", which is
false: a PR changing `docker/ci/**` without moving the pin turns `ci-image-pin` red, and the
merge-consent hook reads the COMBINED commit status, so it will not auto-grant — the job is not
itself a branch-protection required context, but the effect is the same. The only way to move the
pin was to publish from the branch first. The replacement is the same
two-step with a dispatch in place of the push — push the `docker/ci` commit as HEAD, dispatch
`ci-image.yml` on that branch (it tags `git rev-parse --short HEAD`, so the two must coincide), then
commit the pin bump on top. A rebase rewrites the sha, so all three are redone; that was already true.
Probed rather than reasoned: dispatching on `fix/744-ci-image-branch-filter` at `43b1e45` succeeded
(run 2340), the job bound to that branch and sha, and it published `ersatztv-ci:43b1e45`; `:latest`
still resolved to its pre-dispatch digest, a check whose stated limit lives with the probe
(2026-08-27, `docs/ci-cd.md`).
- **Dropping the self-path is a DECIDED TRADEOFF, and calling it forced would be wrong.** `ci-image-pin`
derives `expected` from the last commit touching the image sources; while `.gitea/workflows/ci-image.yml`
was one of them, editing that file re-pointed `expected` at the editing commit. Retaining it stays
workable *because* the branch dispatch survives — push the ci-image.yml commit as HEAD, dispatch,
pin it — so the necessity argument does not hold. What does hold is the price: that route charges a
full ~2GB publish plus a five-pin bump for **every** edit to the file, a comment included, and a
rebase charges it again. That price is what reverted #697's attempt and what made landing this change
alone the cheaper option. Dropping it prices the opposite risk instead (the bullet below). The second was chosen
because `ci-image.yml` is edited far more often for triggers, comments and runner placement than for
build recipe, so the first cost is paid constantly and the second only on a change that can be
paired with a `docker/ci` touch deliberately.
- **What that costs, recorded because it is invisible otherwise:** a change to how the image is built
that lives only in `ci-image.yml` — build-args, Dockerfile path, platforms — neither republishes nor
invalidates the pin, so CI keeps running an image built by the previous recipe and nothing goes red.
The remedy is a `docker/ci/**` edit in the same change — a Dockerfile comment bump suffices — and it
is the only one: publishing after the merge and then pinning is not an alternative, because
`expected` remains the last `docker/ci` commit and rejects a pin naming the workflow-only commit,
while leaving the pin alone keeps consuming the old image. The weekly cron does not cover it either
— that moves `:latest`, never a pin. A corollary worth its own line: **`docker/ci/**` has no
cheap-edit path**, so even a comment fix there costs a publish-and-repin and should ride the next
genuine toolchain bump (#854).
- **This closes ONE route into ONE file, not the class — and the residual is NOT all dispatch.**
Sort by the ref class a trigger admits, not by which keyword it carries, or the enumeration
mis-sorts: `docker-build.yml` has `branches: [main]` and would pass a keyword test, yet it also
has `tags: ['v*']` and `pull_request:`, both head-resolved and both holding the same registry
credential — and a tag push is explicitly outside `release.main-direct-push-disabled`, so
`git push origin <any-commit>:refs/tags/v99.9.9` runs that commit's workflow and can publish
`ersatztv:prod` with no dispatch at all. On top of that, four workflows carry an unrestricted
`workflow_dispatch`. What #744 changed is that none of this is any longer a side effect of an
ordinary branch push. Read it as protection against an accidental or drive-by push, **not** as a
boundary against a malicious or compromised writer; the whole remaining class is tracked in #853.
- **The persist-credentials exemption was downstream of this, and closed with it.** `ci-image.yml`'s
checkout was the one left without `persist-credentials: false` (#746), for exactly the two mechanical
reasons above. Both gone, the flag went on in the same change, which is what let #835 ship its guard
with **no exemption list** — an exemption whose only justification is a temporary condition elsewhere
keeps passing after that condition expires, leaving the guard permanently blind to the file it was
written for.
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -68,3 +68,11 @@ stored value moves too (harmless — only the QSV path ever reads it — but it
on a field the user didn't touch); and a machine client that `PUT`s `0` gets a `200` and then reads
back `64`, which is a silent transform of a submitted value that the OpenAPI description does not
advertise.
**The residual's write-path half was closed by #735** (`api.ffmpeg-profile-numeric-bounds`): the
create/update handlers no longer normalize on save — a newly submitted value below the floor is
rejected with a 422 naming the bound, and the schema documents it. The render-time floor described
above is unchanged and still authoritative, because it is what covers rows written before that
validation existed (an UNCHANGED legacy value is still accepted on update, precisely so an old
profile stays editable). So the accepted residual now reads: a *stored* `0` still displays as `0`
while FFmpeg receives 64 — but no new write can create one.
@@ -0,0 +1,81 @@
---
key: graphics.on-now-next-on-by-default
title: 2026-08-26 — The On Now / Next overlay is on by default, backfilled exactly once (#732)
status: active
since: '2026-08-26'
supersedes: none
superseded-by: none
rule: 'The built-in On Now / Next element is attached to new channels by `ChannelGraphicsDefaults.Attach`, called from BOTH create paths, and to pre-existing channels by a one-time `AttachOnNowNextByDefault` backfill guarded by `graphics.on_now_next_default_attached`. The marker is written only once the built-in element RESOLVES, so an install whose element row does not exist yet is retried rather than stranded; the cost is that while the backfill is still armed it cannot tell a deliberately cleared channel from an untouched one. HLS Direct is excluded at both sites.'
signals: 'AttachOnNowNextByDefault, GraphicsOnNowNextDefaultAttached, ChannelGraphicsDefaults.Attach, ChannelGraphicsElement default, On Now Next default on, create-time graphics default, CreateChannelFromLineup graphics · paths: `ErsatzTV.Infrastructure/Streaming/Graphics/GraphicsElementSeeder.cs`, `ErsatzTV.Application/Channels/ChannelGraphicsDefaults.cs`, `ErsatzTV.Application/Channels/Commands/CreateChannelHandler.cs`, `ErsatzTV.Application/Channels/Commands/CreateChannelFromLineupHandler.cs` · issues: #732, #74'
---
#74 shipped the overlay as a per-channel opt-in toggle. #732 made it a default. The binding level is
`ChannelGraphicsElement` — the base layer added at `GraphicsElementSelector`'s final fall-through, so a
deco in `Override`/`Disable` mode still suppresses it exactly as before (`graphics.channel-level-attachment`).
Two mechanisms, because they answer different questions:
- **New channels**`ChannelGraphicsDefaults.Attach` is called as the channel is persisted. The
create requests carry no graphics field and the SPA's channel editor is update-only, so the server
is the only place a create-time default can live.
- **Pre-existing channels**`AttachOnNowNextByDefault` runs once at startup, after the seeder.
**The create-time mechanism has more than one call site, and enumerating them from the source is the
only reliable way to find them.** There are three places that persist a `Channel`:
`CreateChannelHandler` (the SPA's "New blank channel"), `CreateChannelFromLineupHandler` (the SPA's
primary "Add Channel" flow, and what Auto-Tune bulk-creates through), and `DbInitializer`'s default
channel. The first two both call the shared helper — they diverged once, with only the first covered,
which silently excluded the busier path. The third needs no call because it runs before
`AttachOnNowNextByDefault` in the same startup, so the backfill picks it up. A new persisting site
must call the helper; `grep` for `Channels.Add` rather than trusting this list to stay complete.
**The marker is written only once the element resolves**, and the built-in `GraphicsElement` row is
created by the seeder (`EnsureBuiltInElementRow`) rather than waited for. Those two go together.
Writing the marker unconditionally is the tempting simplification, and it is wrong: on an install
upgrading from before #74 the template is seeded on the same boot, so a marker written while the row
is merely *undiscovered* strands every pre-existing channel permanently — the exact population the
backfill exists for. Creating the row in the seeder removes that ordering dependency for every normal
path, and skipping the marker when nothing resolves covers the rest.
**This is a real trade, not a free win, and a single global marker cannot represent both halves.**
The two properties wanted here are "never re-add to a channel the operator cleared" and "never strand
a channel that had no element to attach". A boolean that means *the backfill has run* can express one
or the other, never both:
- While the backfill is **armed** (nothing has resolved yet — the template file is absent, e.g. an
operator deleted it and `RefreshGraphicsElements` reaped the row), a channel cleared by the
operator is indistinguishable from one never considered. If the element is later restored, the
next boot attaches it to every eligible channel, including that one.
- Once the marker is **written**, no channel is ever re-attached, on any restart.
The armed window is narrow (it requires the built-in template to be absent at startup) and the
failure is visible and reversible — an overlay reappears — whereas stranding is silent and permanent.
That is why it is resolved this way. Closing it properly needs per-channel "considered / opted-out"
state rather than one global flag, which is a schema change and is tracked separately.
**It also cannot reconstruct pre-upgrade operator intent.** On an install that predates #732, an
operator who enabled the overlay and later turned it off left no record that survives — the join row
is simply absent, indistinguishable from never having enabled it — so the one-time backfill
re-attaches it. That is inherent to "enable it on all channels by default" rather than a defect. The
never-re-attach guarantee therefore holds *from the marker onwards*, not across the upgrade boundary.
Identity is the element's **filename** (`GraphicsElementDefaults.OnNowNextFileName`), never the
user-editable `Name` — the #67 lesson carried through #74.
**HLS Direct is excluded at both sites.** ErsatzTV is not transcoding there, so `GraphicsElementSelector`
returns empty and the editor disables the toggle; an attachment would be inert while still reading as
"on". Excluding it keeps the stored state honest rather than merely harmless.
HLS Direct is **not** the only inert case: `FFmpegLibraryProcessService` gates the graphics engine on
`videoFormat != VideoFormat.Copy`, so a channel whose FFmpeg profile is set to `Copy` also renders
nothing while showing the toggle on. That one is deliberately *not* excluded here — the profile is
mutable and shared, so the streaming-mode exclusion is a property of the channel while the `Copy`
gate is a property of a setting that can change under it. The render-site gate is the only correct
place for it.
**Residual risk, stated rather than reassured away.** On 2026-08-26 all 43 channels on the live
install already carried the element (measured by reading `graphicsElementIds` from
`GET /api/v1/channels/{id}` for every id in `GET /api/v1/channels`). The backfill is therefore a no-op
in the only place it has ever run, and its behaviour against real pre-existing data is covered by
tests and a local live run, not by production evidence.
@@ -0,0 +1,67 @@
---
key: graphics.seeded-template-upgrade-by-fingerprint
title: 2026-08-26 — A seeded graphics template is upgraded only when it still matches a shipped default (#732)
status: active
since: '2026-08-26'
supersedes: none
superseded-by: none
rule: '`GraphicsElementSeeder` keeps every default it has ever shipped as a verbatim fingerprint; on an already-seeded database it rewrites the on-disk template only when the file still matches one of them (line-endings and trailing whitespace normalised), so an untouched install gets the new default while any operator edit is left alone.'
signals: 'GraphicsElementSeeder, OnNowNextYamlV1, SupersededDefaults, UpgradeUnmodifiedTemplate, on-now-next.yml upgrade, graphics.on_now_next_seeded · paths: `ErsatzTV.Infrastructure/Streaming/Graphics/GraphicsElementSeeder.cs` · issues: #732, #74'
---
#74 seeded `on-now-next.yml` behind the `graphics.on_now_next_seeded` `ConfigElement` marker, writing
the file only when absent so operator edits are never clobbered. That is the right rule for *content*,
but it has a consequence nobody stated at the time: **an already-seeded installation never revisits the
file at all**, so a change to the shipped default reaches new databases only. #732 hit this directly —
the background box is useless if the one install that has the overlay keeps rendering the pre-#732
template forever.
**Decision: upgrade by fingerprint, not by version number or by marker bump.** Every default we have
shipped stays in the source as a verbatim constant (`OnNowNextYamlV1`, …) collected in
`SupersededDefaults`. On an already-seeded database the seeder reads the file and rewrites it **only**
if it still normalises equal to one of those. The comparison ignores line endings and trailing
whitespace, because a volume mount or an editor rewrites those without the operator touching content.
Why this shape:
- **The safety property is derived, not asserted.** "Did the operator edit this file?" is answered by
comparing bytes against what we wrote, rather than assumed — which is exactly the assumption #74
refused to make, and which a marker bump would have to make.
The comparison is not byte-exact in one direction: line endings are normalised and the result is
`TrimEnd`ed, so an edit consisting *only* of trailing whitespace at end-of-file does not opt the
file out and would be overwritten. That is deliberate (a volume mount or editor rewrites those
without operator intent) and the cost is bounded to whitespace nobody can see. Every edit with any
visible effect opts out permanently.
- **It is self-limiting, so it needs no new marker.** After the rewrite the content equals the
*current* default, which is not in `SupersededDefaults`, so the next startup is a no-op.
- **A fingerprint entry is not a template.** Never edit an entry in `SupersededDefaults` — it is a
record of what we shipped. Changing the current default means adding a new constant and pushing the
outgoing one into the list; editing an existing entry silently un-recognises every install carrying
it, and the failure is invisible (the upgrade just never fires).
**The rewrite is write-then-move, with no in-place fallback.** `WriteAllTextAsync` truncates before
it writes, so an interrupted write would leave a partial file matching no fingerprint — never
repairable by a later boot, and rejected outright by the loader. The temp name is random per call,
because a fixed one is shared by two containers on the same config volume — and a process id is not
random enough: the image's ENTRYPOINT is exec-form, so every container is PID 1 in its own namespace
and would compute the same name. The accepted cost: a
template bind-mounted as a single file cannot be replaced by `rename(2)` (EBUSY), so that install
never receives the upgrade. Reaching that needs a pinned file that is *also* byte-identical to a
shipped default, and the alternative — falling back to an in-place copy — reintroduces the truncation
on every IO fault rather than just that one.
**The upgrade is a one-way door, and a downgrade is lossy.** `GraphicsElementLoader.FromYaml<T>` does
not set `IgnoreUnmatchedProperties`, so an unknown YAML key throws and the element is dropped with
only a logged warning. Once a template has been upgraded, rolling ErsatzTV back to a build that does
not know the new keys makes that element fail to load on **every** channel it is attached to — which,
since `graphics.on-now-next-on-by-default`, is every eligible channel that still has it (HLS Direct
was never attached, and an operator may have cleared individual channels). The symptom is "the overlay vanished
everywhere" and recovery is hand-editing the YAML back. Rollback is a supported operation here
(`:prod` is a floating tag promoted manually), so this belongs in the release notes of any version
that adds fields to a seeded template, not only in this record.
**Verify the fingerprint against a real install, not against the constant it was copied from.** The
#732 V1 entry was checked byte-for-byte against the live install (md5 `ef9afc088cf6dba252f725babbf3334f`,
2026-08-26) before being trusted. A fingerprint that does not match anything in the field is a
permanent silent no-op, and no test written from the same source can detect that.
File diff suppressed because one or more lines are too long
@@ -86,10 +86,13 @@ same sentence appeared in the inventory and was rewritten there first, which is
survives a fix round. A row claiming otherwise would be exactly the overclaim this record warns
about, and it is recorded here because the record's own deliverable made it on the first pass.
This is the same gap `testing.guard-derives-population-from-source` already flagged one directory
over — `MARKED_JOBS` in `test_ci_dropped_step_guard.py` is a hand-written mirror of those same
required contexts, annotated with a date. A dated comment is a claim about the past. Two
independent guards now depend on that configuration; one of them checks it.
`testing.guard-derives-population-from-source` flagged the same configuration one directory over —
`MARKED_JOBS` in `test_ci_dropped_step_guard.py` mirrored those same required contexts behind a
date. A dated comment is a claim about the past. #787 closed that half: the mirror is now a
committed artifact the guard derives its scope from, and `scripts/check-required-contexts.sh`
reconciles it against the live `status_check_contexts` from the merge-consent hook, on both merge
paths. Two independent guards depend on that configuration and both now check it — this row's own
residual is the one that remains, and it is stated above rather than closed.
**The deliverable's own population was wrong three times, and that is the most transferable part of
this record.** The inventory is the detector, so its population *is* the guard. Round one filtered

Some files were not shown because too many files have changed in this diff Show More