Compare commits

..
125 Commits
Author SHA1 Message Date
timothyandtimothy 736649b3b7 fix(812): classify the narrative sites by who-benefits; keep the detector's reach (#882)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 22s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m18s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m33s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m52s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-29 20:28:14 +00:00
timothy 1afad0851d fix(823,824): a scheduling NULL collection reads as UNRESTRICTED and is guarded at both read sites; the Elastic indexer gets its own mutation proof (#879)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 17s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 32s
Build ErsatzTV Image / Build & test (.NET) (push) Canceled after 0s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (push) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Canceled after 0s
2026-08-29 19:55:53 +00:00
timothyandClaude Opus 5 9685132ee0 fix(823): three cleanups from the coherence pass — a wrong witness, a wrong because, a duplicated paths:
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 14s
review-verdict/h10 Review-verdict: MERGEABLE @ 9685132 (base: main)
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 12s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 13s
PR Gates / decisions lifecycle (pull_request) Successful in 17s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 43s
Review verdict / Set review-verdict status (pull_request_target) Successful in 9s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 11m28s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 14m32s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 10m10s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 9m23s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 11s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 10s
Round-five review confirmed the decision record is coherent with no third
survivor of the empty reading, and returned three LOW findings. All are in
prose I wrote in the last two commits.

- The comment defending `(IsAbstract && !IsSealed)` cited
  AlternateScheduleSelectorTests as an in-repo static-fixture witness. That
  class IS static, but it merely NESTS its [TestFixture]es and declares no test
  of its own, so it would fail the sibling "declares no runnable test" assertion
  rather than demonstrating the point. The rule is right and the witness was
  wrong, which is the worse of the two failures because a wrong example is what
  a reader checks the rule against. No witness is cited now, and why is stated.

- A mis-bound `because` in `rule:`: "assigning a null and calling SaveChanges
  SUCCEEDS ... because only the HTTP request records normalize with `?? []`".
  The `?? []` clause explains how a null could REACH the entity; what makes the
  save succeed is the column being nullable. A right observation with a wrong
  cause attached. Split into the two claims.

- `signals:` carried the literal token `paths:` twice, an artifact of appending
  the #823 path list to the existing one. It degrades the field the discovery
  surface parses.

Also recorded from that review, and NOT changed: `MonthsOfYear ?? AllDaysOfMonth()`
survives the selector fixture and no date can kill it -- 1..31 contains every
valid month, so it is an EQUIVALENT mutant there rather than a coverage gap.
Its non-equivalent twin at the DTO boundary is pinned per-dimension by
RecurrenceLimitsMapperNullTests. Left alone deliberately: chasing an equivalent
mutant with a contrived date would buy nothing and cost the fixture's
readability.

Local gate: ErsatzTV.Tests 2091 passed / 6 skipped, Core.Tests 697/1 -- 0
failures. Format clean, no BOM. decisions_validate OK.

Refs #823
Refs #824

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 21:14:27 +02:00
timothyandClaude Opus 5 3951fcf516 fix(823): stop patching the record by grep — a second ?? [] survived, and my "swept it" claim was false
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 18s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 9s
PR Gates / decisions lifecycle (pull_request) Successful in 14s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
review-verdict/h10 Awaiting review verdict for 3951fcf
Review verdict / Set review-verdict status (pull_request_target) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 11s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 7m52s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 0s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Round-four review. One HIGH, again in the decision record, and the previous
commit message asserted this exact class was cleared. It was not.

THE HIGH, and the reason it recurred.

A second sentence still described the rejected reading: "The guard form is
`?? []` into a local rather than this record's Optional(x).Flatten(), a STATED
deviation". The shipped guard is `?? AllDaysOfWeek()`. That sentence is the one
that dictates guard FORM to the next implementer, so it would have taught the
`[]` reading the same record spends a paragraph calling data corruption -- and
it had already propagated into docs/decisions/README.md, the mandated entry
point, which carries `rule:` verbatim.

The mechanism, not the sentence, is the defect. I swept with a regex keyed on
"null" plus a reading word; this sentence talks about guard FORM and contains
neither, so it could not match. That is grepping the retracted WORDING instead
of sweeping the CONCEPT, which is exactly what this corpus warns about -- and
three rounds in a row have now found a defect introduced by the previous
round's targeted string edit. So the fix is not another targeted edit: the
whole `rule:` field was split into its 39 sentences and read back one by one
against the code. Everything below came out of that pass rather than a grep.

Its secondary damage is worth recording because it is the shape of a rationale
that outlives its claim: the deviation was justified by ".ToList() allocates
for nothing", which is now BOTH irrelevant to the choice AND false about the
shipped code, since AllDaysOfMonth()/AllMonthsOfYear() are themselves
Enumerable.Range(...).ToList() on exactly the null path it describes.

- The opening sentence of `rule:` prescribed Optional(x).Flatten() as THE
  read-site form. It is the sentence most likely to be read in isolation, and
  it is wrong for six of the eight columns. It now separates the universal half
  (a LOCAL, never assigned back) from the half that is not (the substituted
  value), and names where each applies.
- `signals:` had never been touched, so roughly 60% of `rule:` was unreachable
  by the discovery surface built for it -- no AlternateScheduleSelector, no
  mapper, no "unrestricted", and its paths: list named none of the files this
  work touched. It also advertised "Optional Flatten hoisted local" as the
  form, which is precisely what the six do NOT use.
- The body prose was still entirely about SongMetadata while `rule:` had grown
  a whole second subject. Added the two results that contradicted the prior
  reasoning, in prose, where a reader meets them.

A REAL BUG in my own guard, not just prose:

  fixture.IsAbstract.ShouldBeFalse(...)

A C# `static class` compiles to `abstract sealed`, and NUnit runs tests
declared in one -- this repo already has such a fixture
(AlternateScheduleSelectorTests is `public static class`). So the check I added
one commit ago to reject an un-runnable fixture would have falsely reddened a
perfectly good static one. Now rejects an abstract BASE (abstract and NOT
sealed), which is the case NUnit actually cannot instantiate.

A SURVIVING MUTANT the added controls did not kill:

AnyDate was 2024-03-06. With a day <= 12 a CROSS-WIRED substitution survives
the whole fixture -- `DaysOfMonth ?? AllMonthsOfYear()` hands back 1..12, which
still contains day 6, so every assertion passes while the guard substitutes the
wrong set. Moved to 2024-03-20, still a Wednesday in March, outside 1..12.
Measured both ways rather than reasoned: the cross-wire mutant passes the old
fixture and FAILS 2 of 11 on the new one.

Also re-witnessed, because I had modified that file and never re-proved it:
restoring `??=` in LuceneSearchIndex reddens the LUCENE fixture (1 red, 1
green) -- the exact mirror of the Elastic mutation. Extracting
ThrowOnWarningLogger did not cost #701 its proof, and the two fixtures are
independently load-bearing in both directions.

The record is now 73 prose lines, over the 60-line WARNING ceiling. Stated
rather than trimmed: it is 42nd of 42 records over that line, and the added
content is distinct findings (a second subject, a migration analysis and three
residuals), not redundancy against a sibling.

Local gate: ErsatzTV.Tests 2091 passed / 6 skipped (the three fixtures' MySQL
halves), Core.Tests 697/1 -- 0 failures. Format clean, no BOM. decisions
validate OK.

Refs #823
Refs #824

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 21:00:22 +02:00
timothyandClaude Opus 5 ea888011aa fix(823): the decision record argued BOTH readings — and the per-dimension mutant that survived the fixture
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 16s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Successful in 16s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
review-verdict/h10 Awaiting review verdict for ea88801
Review verdict / Set review-verdict status (pull_request_target) Successful in 18s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 7m4s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m55s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 4m22s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Round-three review findings. One HIGH, and it was in the durable artifact
rather than the code.

THE HIGH: the record stated the shipped reading and its inverse.

The semantic reversal (empty -> unrestricted) rewrote the residual and the
write-half of `media.nullable-primitive-collection-mutation` but left the
ORIGINAL reasoning standing two sentences earlier: "A null reads as EMPTY, so
the item matches nothing"; "the REJECTED alternative was the All*() set";
"SKIPPING the row is the conservative repair". The shipped code is
`?? AllDaysOfWeek()` -- precisely the alternative that passage calls rejected.
The previous commit then inserted residual (1), which reasons entirely FROM
the All*() reading, two sentences after the sentence denying it.

That is worse than a stale comment. A session resolving this key -- or reading
the MemPalace mirror, which carries `rule:` verbatim -- would have been told to
write the guard the other way, i.e. talked into the `[]` reading that the same
record elsewhere argues is data corruption one save later. Replaced the whole
passage, then swept the record for every other mention of the empty reading
rather than trusting the one replacement: the only survivor is the new sentence
that records EMPTY as the rejected alternative, which is the direction that
stops it being re-adopted.

THE MEDIUM: one arrangement did not close the hole it claimed to.

The discriminating control added last commit nulls DaysOfWeek against a
restrictive MonthsOfYear. It excludes "any NULL matches unconditionally" only
for that dimension. The review supplied the surviving mutant --
`if (item.MonthsOfYear is null) { return item; }` ahead of the checks -- and
traced it green through all nine tests. Verified by EXECUTION, not by reading:
applied to the previous fixture it passes; applied now it FAILS 1 of 11. Each
of the three dimensions is now nulled against a restriction on a different
dimension.

The rest, all from the same round:

- The coverage guard's test detection listed attribute TYPES, and each list
  falsely reddened whatever it omitted: TestAttribute alone missed [TestCase],
  and the three-type replacement missed [Theory]. Now decided by NUnit's own
  ITestBuilder/ISimpleTestBuilder interfaces, which cannot fall behind the
  vocabulary. It also dropped BindingFlags.Static (GetMethods() defaults to
  including it), which would have falsely reddened a static test method.
- The same guard accepted an ABSTRACT fixture -- NUnit never instantiates one.
  The indexer population already filtered IsAbstract; the fixture side now
  mirrors it.
- The record's `mechanics:` still described the old `[Test]`-only clause, in
  the same file the change edited.
- An <inheritdoc> made the ProgramScheduleAlternate empty-case test inherit a
  docstring written from the PlayoutTemplate test's viewpoint.

Two more mutations executed:
- `if (item.MonthsOfYear is null) return item;` -> 1 red, 10 green. This is the
  mutant that survived the previous head; it no longer does.
- an abstract type named in the covered set -> coverage guard red.

Local gate: ErsatzTV.Tests 2091 passed / 6 skipped (the three fixtures' MySQL
halves, skipping visibly without ETV_TEST_MYSQL_CONNECTION), Core.Tests 697/1
-- 0 failures. Format clean, no BOM on the touched set. decisions_validate OK.

Refs #823
Refs #824

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 20:43:07 +02:00
timothyandClaude Opus 5 6200713965 fix(823,824): close the review round's findings — a discriminating control, the second mapper's empty case, and honest test detection
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 16s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 19s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 14s
review-verdict/h10 Awaiting review verdict for 6200713
Review verdict / Set review-verdict status (pull_request_target) Successful in 11s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 7m53s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 7m59s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Follow-up commit (the branch is pushed, so not an amend). Two more cold
reviews landed on the previous head; both reported 0 Blocker and 0 High, and
these are their Mediums and Lows. Each fix carries its own witnessed mutation.

1. The selector fixture could not tell the fix from a much broader one.
   Every null test set a NULL and expected the item SELECTED, so all of them
   pass equally under "NULL means unrestricted" and under "any NULL makes this
   item match unconditionally" -- a refactor short-circuiting the whole date
   check on any null kept them green. Added the discriminating control: a NULL
   DaysOfWeek paired with MonthsOfYear = [1] against a MARCH date must be None.
   Only the narrow reading passes.

2. A_Null_Item_Does_Not_Disturb_Selection_Of_A_Later_Item never measured its
   own docstring. The nulled item was unrestricted and at Index 0, so it always
   won and the second item was never evaluated -- the stated invariant ("a null
   on the first item must not decide the second") went unmeasured while the
   test passed. Split into two: one where the nulled item genuinely does not
   match, which measures that the loop CONTINUES; and one that pins the
   index-order win separately.

3. The empty-preservation control existed for one of two identical mappers.
   The anti-mutant test for "empty or null becomes All*" covered only
   Playouts.Mapper; Scheduling.Mapper is a byte-identical triple in another
   file and had none, so a defensive edit to it alone would have rewritten a
   deliberately-empty user selection to 1..31 with the suite green. That is the
   one-helper-two-callers shape this repo has been bitten by. Added the
   matching test.

4. The coverage guard's [Test] clause did not check what its message claimed.
   GetMethods() without BindingFlags returns INHERITED methods, so a fixture
   that merely subclasses another satisfied it while driving the wrong indexer
   -- and Values.Distinct() cannot catch that, since the two Types differ. It
   also matched TestAttribute alone, so a future fixture written as [TestCase]
   would have falsely reddened, and it accepted an [Explicit]/[Ignore]d fixture
   that never runs, which is the "wired is not running" failure the guard
   exists to prevent. Now DeclaredOnly, the full test-method vocabulary, and
   Explicit/Ignore rejected at both method and fixture level.

5. Three residuals recorded on media.nullable-primitive-collection-mutation
   that the previous head asserted nothing about:
   - the LOUDNESS change, worst for an all-three-NULL ProgramScheduleAlternate,
     which now matches unconditionally and shadows the default schedule where
     it previously threw. Unreachable today, and a choice over an unreachable
     state rather than a measured requirement -- said plainly.
   - the normalization is ONE-WAY and WHOLE-LIST: both PUT paths are full
     replaces, so editing any row persists All*() over EVERY NULL row in that
     playout, and afterwards "the operator selected all 31" and "this is a
     legacy row" are indistinguishable. An ordinary user action closes that
     door.
   - the WRITE side disagrees with the READ side about what ABSENCE means: an
     omitted daysOfWeek normalizes to [] ("never applies") while a NULL column
     reads as unrestricted, so an API client gets HTTP 200 and a row that
     silently never fires. Filed as #880 rather than folded in here, because a
     client omitting a field on a write is a different question from what a
     legacy NULL meant.

Two more mutations executed, both witnessed:
- DaysOfWeek guard disarmed in Scheduling.Mapper -> 1 red, 3 green.
- A fixture with no DECLARED test named in the covered set -> coverage red.

Local gate (MySQL lane armed): ErsatzTV.Tests 2097 passed / 0 skipped,
Core.Tests 695/1 -- 0 failures. Format clean, no BOM on the touched set with
the population count asserted. decisions_validate OK.

Refs #823
Refs #824
Refs #880

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 20:27:33 +02:00
timothyandClaude Opus 5 95b2700f09 fix(823,824): a scheduling NULL collection reads as UNRESTRICTED and is guarded at both read sites; the Elastic indexer gets its own mutation proof
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 6s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 18s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 20s
PR Gates / decisions lifecycle (pull_request) Successful in 20s
PR Gates / Docs update reminder (pull_request) Successful in 21s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 11s
review-verdict/h10 Review-verdict: MERGEABLE @ 95b2700 (base: main)
Review verdict / Set review-verdict status (pull_request_target) Successful in 23s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 8m6s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m55s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m35s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 2m56s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Both issues are #701 deferrals, and they land together because both rewrite
the same decision record.

#823 -- can a null reach one of the six collection-valued scalar columns?

MEASURED against a real TvContext on BOTH providers (SQLite, and MySQL 8.4
on an ephemeral server), because the reasoning available beforehand pointed
the wrong way. The two converters differ on their read side --
IntCollectionValueConverter maps null-or-blank to Array.Empty<int>(), while
EnumCollectionJsonValueConverter would dereference the result of
JsonConvert.DeserializeObject -- so the expectation was that a NULL row
behaves differently per column. NEITHER RUNS: EF does not invoke a value
converter for a NULL column at all. All six materialize as CLR null, the
int converter's null-to-empty branch is dead on this path, and unguarded
each .Contains in AlternateScheduleSelector throws NullReferenceException.

A NULL reads as UNRESTRICTED -- the All*() sets -- not as empty. This is
the whole semantic question and the first draft got it backwards. It is
decided by the one NULL reachable WITHOUT any code writing one: Sqlite's
20240113140741_Add_PlayoutTemplate_DaysOfMonth adds the column with
nullable:true and NO defaultValue, so a PlayoutTemplate row inserted before
it holds NULL and by construction had no day-of-month restriction. Reading
that as empty INVERTS the row's meaning and silently stops the template
applying at all. All*() preserves it, and is how "no restriction recorded"
is already represented (GetPlayoutAlternateSchedulesHandler,
PreviewBlockPlayoutHandler). What does NOT decide it, and was wrongly cited
in the first draft: the API request records normalize an omitted field with
`?? []`, but that is a client omitting a field on a WRITE and says nothing
about what a legacy database NULL meant.

Two read sites, not one. Guarding only the selector would have left the
entity->DTO mappers unguarded, and those feed the SPA: PlayoutScheduleEditors
spreads the collection (`[...template.daysOfMonth]` -> TypeError on a JSON
null) and playoutTemplateCalendar's appliesToDate -- an exact port of
GetScheduleForDate -- calls .includes on it. Both mappers now substitute the
SAME defaults, so the preview agrees with what is actually scheduled. Neither
guard is assigned back onto the entity, which is the
media.nullable-primitive-collection-mutation mechanism.

Reachability, stated precisely rather than overclaimed. All six are
nullable:true on both providers, but a nullable column does not produce a
NULL row: five of the six were present at CreateTable, so a NULL there still
needs code to write one, and on MySQL there is NO code-path-free NULL for any
of the six. The write path ACCEPTS a null (SaveChanges succeeds, stores SQL
NULL) but no caller supplies one today -- every production construction of the
two commands goes through the request records. That is a property of the code,
not a live caller; claiming otherwise would be the banned "it's AsNoTracking
today" argument pointed the other way.

#824 -- ElasticSearchIndex.UpdateSong had no regression test

Issue option 1 (a non-network transport) shipped, and needed no new package:
Elastic.Transport.InMemoryRequestInvoker is public in the pinned version and
ElasticsearchClientSettings(NodePool, IRequestInvoker) accepts it, injected
into the private _client the way #701 injects the Lucene IndexWriter.
UpdateItems never runs `_client ??= CreateClient()`, so the injected instance
is the one used.

Two traps there are load-bearing, both measured: the canned response must
carry an `X-Elastic-Product: Elasticsearch` header or the client's product
check throws UnsupportedProductException INTO UpdateSong's catch, and an empty
body fails to deserialize the same way. Either turns the fixture into a green
measurement of the error path -- which is how it first failed here, caught by
the ThrowOnWarningLogger. The document id is asserted as the LAST PATH SEGMENT,
not by substring: the index name carries digits, so ShouldContain would stop
discriminating for a song whose id collided with one.

Six mutations executed, each disarming ITS OWN clause alone:

- `??=` restored in ElasticSearchIndex only -> the Elastic fixture reddens on
  "metadata.Artists should be null but was []" while the LUCENE fixture stays
  GREEN. The #824 hole demonstrated, not described.
- DaysOfWeek guard disarmed in the selector -> 4 red, 3 green (DaysOfMonth and
  MonthsOfYear unaffected). Each clause is independently load-bearing.
- DaysOfMonth guard disarmed in Playouts.Mapper -> 1 red, 2 green.
- Elastic dropped from the covered set / mapped to the SAME fixture as Lucene /
  mapped to a class with no [Test] -> SearchIndexMutationCoverageTests reddens
  on each.

That coverage guard is the boundary fix the issue asked for: the covered set is
compared against an ISearchIndex population DERIVED FROM THE ASSEMBLY. Its claim
stops where the check does -- no static check can establish that a named fixture
actually DRIVES its indexer, so it forces a human to look rather than proving
coverage. ThrowOnWarningLogger moved to ErsatzTV.Tests/Support so both fixtures
share it; the Lucene fixture's assertions are otherwise untouched, since it is a
witnessed proof artifact.

No production change in ElasticSearchIndex.cs -- #824 is coverage only.

Docs: testing.md gains a "Provider-parity fixtures" section naming all THREE
opt-in-MySQL fixtures and recording that CI runs none of them (#627);
docs/README.md gains the matching task signal; guard-inventory.md's
hand-written C# guard list goes from five files to six. Scheduling/Mapper.cs
loses the UTF-8 BOM it inherited, per #311 fix-as-you-touch.

Local gate (with the MySQL lane armed): ErsatzTV.Tests 2096 passed / 0 skipped,
Core.Tests 693/1, Infrastructure.Tests 114, Architecture.Tests 7, Scanner.Tests
1504 -- 0 failures in each. scripts/tests 1228 passed / 2 skipped. dotnet format
whitespace --verify-no-changes clean; BOM check over the touched set with the
population COUNT asserted, because a bare zsh loop silently checks one
concatenated filename. decisions_validate OK.

Fixes #823
Fixes #824

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019zUmJZHhVP7kXg5DV237TW
2026-08-29 20:02:52 +02:00
timothyandtimothy 8aeacd534a fix(819): derive the SPA page-size guard population from the git index (#875)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 21s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m52s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m23s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m5s
The guard asserted EXACT completeness over a population enumerated by a directory
walk, so an untracked .ts/.tsx under web/src/ entered it and failed as unregistered
on that developer's checkout while CI — which only ever checks out tracked files —
stayed green.

The glob still supplies file CONTENT; the POPULATION is now the git index, read by
web/vite-plugins/trackedSourceFiles.ts in Vite's own Node context and handed to the
app project as a virtual module. That reaches the index without admitting
@types/node to tsconfig.app.json, the obstacle that deferred this in #818.

Three mechanisms carry the proof, each added because the previous was measured
insufficient: a closed-form restatement of the shared scope predicate (sharing no
helper at any depth with what it checks); a second independent `ls-files --others`
query cross-checking the population; and real-git tests that execute the derivation
against a temp repository.

Six residuals are stated with their MEASURED fail-directions, and
testing.guard-derives-population-from-source gains a bounded exception plus the
closed-form criterion.

fixes #819

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-29 09:14:15 +00:00
timothyandtimothy b6b3520bdb fix(809,822): isolate the suite from the production hook-fire log by construction (#874)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m32s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m12s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m14s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m32s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-29 02:32:26 +00:00
90f96c14a5 fix(803,664): fence the HEAD alias on the PR timeline's pull_push count (#873)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 15s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m16s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m39s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m37s
A force-push H1 -> H2 -> H1 spanning `pr-changed-files.sh`'s paging leaves its final
`.head.sha` comparison equal while the middle pages came from H2, so a mixed file list
could produce a docs-only exemption `success` no single head ever justified. The base
alias had been fenced since #706 by a monotonic `change_target_branch` count; the head
axis had nothing, and three contracts asserted otherwise.

`count_retargets` becomes `count_pr_mutations`: one timeline walk, two tallies, one shared
trust flag, a separate fence arm and diagnostic per axis. The advisory hook re-reads
`.head.sha` at the same hoist and off the same response as the base re-read. All three
overclaiming contracts are corrected, plus four paraphrases the first sweep missed.

Measured, not assumed: Gitea 1.27.1 still serves no `files` on `compare/{base}...{head}`;
every push is a `pull_push` event and its count cannot alias; PR #761 really went
`8798a1d -> 830a407 -> 8798a1d`; and Gitea creates the push comment BEFORE emitting the
synchronize notification, so a run cannot abstain on its own trigger.

Two pre-existing fail-opens in the shared walk were found by review and fixed: an empty
ARRAY first page was trusted on any page while the `null` arm required `page > 1`, and no
row was validated before `.type` was selected on.

NOT closed, and documented rather than overclaimed: the walk's `null` terminator is
defeatable, because Gitea pages before it filters (#870). The fence closes the ABA on a
timeline with no truncating block, not the ABA outright.

fixes #803
fixes #664
Refs #870

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 23:29:47 +00:00
timothyandtimothy 4b3f6f6c90 fix(786,789): workflows declare their own per-job metadata; guard populations derive from it (#872)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m21s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m58s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m8s
Closes #786 and #789, bundled because working either alone would build the artifact the other removes.

Every job in all six tracked workflows declares `env.CI_JOB_ROLE` (guard/report-only/none); the
`docker-build.yml` jobs also declare `env.CI_EXECUTION_CLASS` (toolchain/bare-runner). Both guard
populations derive from those markers; the `TOOLCHAIN_JOBS`/`BARE_RUNNER_JOBS` literals are deleted.
A missing or unrecognised marker is a hard failure in both checkers.

#789's literal had a real justification — set equality between two DERIVED sets is blind to a member
leaving both at once — so the marker is the anchor that replaces it, and the cost (proximity to the
`container:` block) is paid by a THIRD derivation from each job's own steps, which is also the only
check that sees the failure #789 filed: a .NET step moved into a bare-runner job, where no set
changes. The residual is disclosed: drop the block, flip the marker AND hide the tool behind a
script and all three go blind, bounded by the failure mode being a loud missing-binary crash.

#786's guard jobs join a machine-checked population: a new `test_workflow_job_guards.py` asserts set
equality both ways against a new "Workflow-job guards" table, and the four jobs with no dropped-step
guard each carry a recorded decision.

Two issue claims were refuted by measurement: #789's "editing docker-build.yml re-points the pin"
(the pathspec is `docker/ci` only) and #786's job count (17, not 15).

Four cold adversarial review rounds across two model families; rounds 1-3 BLOCKED, all findings
fixed and each fix demonstrated by reproducing the reviewer's own test. The recurring defect class
was prose drifting from code, including a mechanism claim in the decision record that execution
refuted. All five mutation proofs redden when their shipped detector is disarmed.

New decision record: `testing.workflow-declares-its-own-job-metadata`.

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 20:54:53 +00:00
timothyandtimothy e11d577193 docs(796): verification code is code under test, and the proof it was claiming an exemption from (#871)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m38s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m9s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m18s
Records `testing.verification-code-needs-its-own-proof`: the proof obligation follows the
VERDICT rather than the file, so it binds harnesses, wrappers, timeouts and checkers — not
only the files the guard population derives.

The issue asked for a stated position on whether non-guard checker scripts get mutation
proofs. The position as first written claimed `scripts/mcp_smoke.py` "cannot participate"
because driving it needs the gitignored `.mcp.json` and a cold-built language server. Cold
review refuted that by execution: it takes its config path and server name as positional
arguments. The record had failed its own headline rule on the one claim its decision rested
on, so this ships the proof instead of the exemption.

- `scripts/tests/test_mcp_smoke.py` — a hermetic stub JSON-RPC responder and six cases
  pinning the defects the checker has already had, with the positive control as a fixture
  the refusal tests depend on, so a node-id or `-k` selection cannot skip it.
- A declared clause in `mutation_manifest.py` targeting the unguessable request id, using
  the `guard=test / target=script` shape that already exists for `mutation_harness_lib.py`.
  Witnessed red: `id_init = 1` makes the pre-answer accepted at `initialize` (rc 9 -> 10),
  and only that test moves.

`mcp_smoke.py` still gets no inventory row — one is rejected as a phantom (measured). The
row goes to the test file, which joins the derived population automatically.

Five cold-review rounds, four BLOCKED. Round 2 caught a `ruff format` red that would have
failed `script-tests`. Rounds 3-5 found only hand-maintained counts and uniqueness claims in
prose, three of them created by the previous round's fix; that class was deleted rather than
corrected again, per this record's own stop-and-subtract rule.

Docs updated in the same PR: `docs/README.md` task-signal map and `docs/guard-inventory.md`
(row, summary counts, scope-limit item 6).

fixes #796
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 20:06:45 +00:00
timothy 609fd852c2 fix(763): page both /statuses/{sha} reads to a validated terminator (#868)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 20s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m45s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m10s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m2s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m9s
2026-08-28 17:34:48 +00:00
timothyandClaude Opus 5 11287a54ba fix(763): satisfy ruff — E741 and formatting on the touched test file
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 10s
review-verdict/h10 Review-verdict: MERGEABLE @ 11287a5 (base: main)
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 19s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 19s
PR Gates / Docs update reminder (pull_request) Successful in 16s
Review verdict / Set review-verdict status (pull_request_target) Successful in 20s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 6m51s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m5s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m6s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
CI's `Script lint and tests` job went red. Cause: I never ran ruff locally,
which this repo's Python convention requires after any .py change.

  - E741 twice: `l` as a comprehension variable in the sort-order guard.
  - `ruff format --check`: the file was correctly formatted on `main`; my edits
    broke it. One of them left a docstring line at column 0, which `ruff format`
    then "corrected" by over-indenting the rest of the paragraph — repaired at
    the source rather than accepting that rewrite.

Verified the way CI does: local ruff is the pinned 0.12.11, and both
`ruff check` and `ruff format --check` run under bash over the full tracked
population (`git ls-files -z '*.py' '*.pyi' '*.ipynb'`, 46 files) are clean.
The population is counted, not assumed — an empty glob would pass vacuously,
which is the failure `scripts/tests` guards against elsewhere.

`scripts/tests` 1097 passed, 2 skipped after the reformat.

refs #763

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 18:51:32 +02:00
5fb9c8537a docs(747): re-verify the Gitea 1.25.4-pinned CI claims on 1.27.1, and measure the merge-gate semantics that were source-attested (#867)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 17s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m25s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m48s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m10s
Population derived from `git ls-files`, not the issue's 9-key list (~21 claim sites).

Re-confirmed unchanged on 1.27.1: the distinct `skipped` commit-status state; `compare` serving
no `files`; no agent-side cancel route (REST route + swagger only); `branches: [main]` suppressing
the run off a non-main base.

Newly measured on four throwaway scratch bases, `main`'s rule never PATCHed: an absent required
context blocks an ORDINARY merge without needing `block_admin_merge_override` (that field governs
the FORCE path only), and `enable_bypass_allowlist` with an empty list is NOT a substitute for it.
Trap recorded: the PR API reports `mergeable: true` while such a merge is refused.

Left explicitly dated with reasons: push-supersession auto-cancel, `pull_request_target` overlap,
`--depth=1` no-merge-base, and the scope-enum/`reqRepoWriter`/403 items. Not a corpus sweep, and
`ci.actions-credential-scoping` now says so. `review-verdict.yml` untouched — #763 holds that file.

Five adversarial review rounds (21/12/9/6/2). Caveat: all same-model-family; Codex was rate-limited.

fixes #747

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 16:51:13 +00:00
timothyandClaude Opus 5 462118d98d fix(763): round 9 — assert the repair's reason positively, not by absence
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 22s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 17s
PR Gates / Docs update reminder (pull_request) Successful in 21s
review-verdict/h10 Review-verdict: MERGEABLE @ 462118d (base: main)
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 23s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Failing after 14s
Review verdict / Set review-verdict status (pull_request_target) Successful in 7s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 1m31s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
A sixth cold review found everything in round 8 clean except one line, and it
is the rule this branch keeps rediscovering: the test pinned the new
`raced_why` only by asserting the ABSENCE of the borrowed wording. Measured —
replacing the string with `zzz` left the suite green while an operator would
get `::error::… — zzz.` beside a sticky sentinel. The sibling test 330 lines
away states the rule and follows it; this one did not.

Now asserted positively, with the em-dash and full stop discriminating the
`::error::` reason from the `::warning::` text that continues ", which cannot
be true". The `zzz` mutation reddens it.

Three nits from the same review, all verified by execution rather than reading:
  - the earlier fixture's row was excluded by the strict `> $since` because the
    mark became its OWN id, not because it sat below the mark.
  - the predecessor comment said `main` "warned only on `null`". True of the two
    EMPTY shapes being contrasted; an empty body and a non-array object warned
    as well. Scoped.
  - `docs/ci-cd.md` and the record described the `::error::` as a two-way split
    (found vs unverifiable). Round 8's whole argument is that a complete read
    returning an IMPOSSIBLE answer is a third case, not a variety of the second
    — which is the operator-facing point, since it decides whether to go looking
    for an API failure that never happened. Both now say three.

The review re-verified, by comment-stripped diff, that round 8 changed no
executable line beyond the `raced_why` string and the if/elif restructure, and
independently reproduced both inertness measurements and the `origin/main`
predecessor behaviour.

Verification: `scripts/tests` 1097 passed, 2 skipped; decisions_validate and
build_decisions_catalog --check exit 0.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 18:42:22 +02:00
timothyandClaude Opus 5 4164efbc3e fix(763): round 8 — a fixture must pose the scenario it narrates
A fifth cold review confirmed the gate's behaviour is correct and proof-backed,
and blocked on three non-behavioural items. All three fixed; none touches the
shipped logic.

MEDIUM — the round-7 fixture narrated a raced human verdict it did not
construct. `null-page1-after-post` appended the row unconditionally, so it also
joined the PRE-write read and lifted the high-water mark above itself; removing
it changed nothing. The reviewer's suggested fix was to gate the append on the
post-write read. Measured after gating: still inert, because page 1 answers
`null` before any row reaches the wire.

So the row is gone rather than gated, and the prose now describes what the
fixture actually poses: a response asserting an empty history for a sha this job
wrote to must not be accepted as proof that nothing raced. Whether a verdict
really raced is not modelled and does not need to be — the response is not
evidence either way. A row the test cannot observe is decoration that reads as
coverage, which is the same class this branch has now been blocked on five
times.

LOW — the comment claimed the predecessor "at least produced a `::warning::`".
Half false, measured against `origin/main`: its `jq -e 'type == "array"'` gate
ACCEPTED `[]` silently and warned only on `null`. What is actually new is that
the paged walk reports such a read as a SUCCESS.

LOW — when the empty clause fired it set `ph_ok=no`, so the log said "could not
be read completely" beside a walk that completed on a validated terminator. The
answer was impossible, not unreadable, and an operator holding a sticky sentinel
needs to know which. It now carries its own `raced_why`, asserted by the test.

Both clauses mutation-proved: disarming the empty check, and reverting to the
borrowed wording, each redden the named test.

Verification: `scripts/tests` 1097 passed, 2 skipped; decisions_validate and
build_decisions_catalog --check exit 0.

refs #763

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 18:18:37 +02:00
timothyandClaude Opus 5 e6f04cc739 fix(763): round 7 — reject an empty post-write history, and the fourth overclaim
A fourth cold review returned NOT-MERGEABLE on two Mediums. Both fixed, plus
its three Lows.

MEDIUM, and a defect this branch introduced. Tolerating a `null`/`[]` page 1 as
"complete, zero rows" is correct for the PRE-write caller — a head nothing has
posted to genuinely has no statuses — and impossible for the POST-write one,
which has just written a row to that sha. The body is well-formed, so nothing
retries it, and the walk reports success: `raced=0` concluded from a list that
cannot be real, on the one path whose failure direction is toward SUCCESS.
Worse than the code it replaced, which at least emitted a `::warning::` — a
logged fail-open had become an unlogged one. Reviewer measured both directions.

The post-write caller now rejects an empty result itself; the walk stays
caller-agnostic because the pre-write caller genuinely needs the empty answer.
This is NOT the withdrawn currency witness: that asked whether ANY row sat above
the mark, which an unrelated newer row satisfied while the rejection stayed
hidden, and it fired on schema-valid staleness. This asks only whether the list
is EMPTY — a state no unrelated row can produce and no ordering can disguise.
It carries neither defect. Proved by fixture; disarming it reddens the named
test, and the previously-uncovered `null`-at-page-1 clause is now covered too.

MEDIUM — the fourth overclaim of the same class, in the decision record body:
"Uncertainty must fail closed at both ends … Both repair now." The page-2 probe
was DELETED, not converted; it repairs nothing. It also contradicted the
record's own `rule:` ("the two directions are NOT symmetric") and the bullet
directly beneath it. Round 5 retracted this wording in `docs/ci-cd.md` only —
the sweep was by subject, not by the retracted words.

Also fixed:
  - the record presented "an empty FIRST page is legitimate" as a property of
    the walk; it is a property of the pre-write caller.
  - `docs/ci-cd.md` called the numeric-only id comparisons a fix for mark
    inflation; they are a TYPE guard, closing the string half. A corrupt but
    genuinely numeric id still inflates the mark — not attacker-controllable,
    since ids are server-assigned, and now stated rather than implied.
  - `test_a_partial_mark_is_SAFE...`'s self-guard promised to detect that the
    fallback ran; it keys on a warning emitted by a different condition, so
    deleting the fallback left it green. Its sibling is what reddens; the
    message now says what it actually pins.
  - the order-faithful fixture appended the job's own POST after the reversal,
    serving the NEWEST row on the OLDEST page — the opposite of DESC, in the one
    fixture that exists to be ordering-faithful.
  - "twice per walk" for the wasted sleep; it is once per walk, twice per run.
  - a dead counter read in the DESC mode.

Rebased onto b16ec15d6 (the other session's #781/#799 docs work; no file
overlap, no conflicts).

Verification: `scripts/tests` 1097 passed, 2 skipped; fifteen executed mutations
across rounds 2-7; decisions_validate and build_decisions_catalog --check exit 0.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:47:06 +02:00
timothyandClaude Opus 5 89db6022d9 fix(763): round 6 — measure the partial-mark fallback under the REAL ordering
A fourth cold review (Opus, isolated worktree, tests/double/docs focus)
reported no correctness bugs in shipped behaviour but two coverage defects on
exactly the two things this change advertises. Both are closed.

The partial-mark fallback's safety is a claim ABOUT THE ORDERING — page 1 holds
the newest rows, so a walk that fails later still saw the true maximum. The
fixture pinning it served ASCENDING ids, i.e. the arrangement the design calls
unsafe, and passed anyway because the raced row's id sat above even the partial
mark. It could not distinguish safe from unsafe.

The stub now HONOURS the sort parameter: order-faithful modes serve DESC by
default and ASC when the request asks. The new fixture holds a PRE-EXISTING
base-mismatched verdict at id 7055 among 60 rows. Under DESC the salvaged mark
is 7059 and that row is below it — the exemption correctly stands. Under ASC
the mark would be 7049 and that untouched row tests as NEWER, a sticky repair
on a head nothing raced. So re-adding `sort=highestindex` now reddens by
BEHAVIOUR, not only by the structural assertion added in round 5. Measured:
re-adding it reds both tests.

Most modes stay ordering-blind on purpose and now say so: they test walk
COMPLETENESS, which is order-independent, and insertion order is what lets a
fixture place a row beyond page 1.

Also fixed:
  - `null` is accepted as an empty page. An array-only gate is the exact shape
    of #751 — `count_retargets` had one, the timeline really did return `null`
    past the end, and the fence withheld EVERY exemption from the day it
    shipped. The same narrowing here is worse, because this walk's failure is
    the STICKY sentinel: every exempt PR would need a hand-posted verdict, per
    head. Tolerating `null` cannot misread `[]`. Proved by fixture.
  - the fail-closed comment said "past the 1000-row page cap"; the bound is 950,
    as the walk's own comment and both docs already said.
  - the docs claimed "only a read returning no rows at all abandons the mark".
    False: a VALIDATED empty history yields a mark of 0 and is not abandoned —
    that is the normal first run. What abandons it is a read that both FAILED
    and returned nothing. Corrected in ci-cd.md and the record `rule:`.
  - a comment pointed at the page-2 probe "a few lines further down"; it was
    deleted, so the deixis pointed at nothing.
  - the stub claimed its logical-read counter "is only reached on a SUCCESSFUL
    page-1 serve" — measured false; it counts page-1 requests, retries included.
  - five `(round N)` markers removed. A round number is session chronology and
    does not parse for a reader who never saw it (`docs.no-session-narrative`);
    an issue number does. The four that remain predate this change.

Verification: `scripts/tests` 1096 passed, 2 skipped. Thirteen executed
mutations across rounds 2-6. The reviewer independently re-ran the earlier
matrix and confirmed it, with one correction carried here: two of those
mutations redden MORE than their named test, so "each reddening exactly its
named test" was wrong — they redden at least it.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:08 +02:00
timothyandClaude Opus 5 53ee44cbc1 fix(763): round 5 — correct the overstated motivation, and fix the string-id twin
A third cold review (Opus, isolated worktree) returned NOT MERGEABLE with one
High and three Medium. All are addressed.

HIGH — the stated motivation was wrong, and self-contradictory once round 4
landed. Under the server default (`created_unix DESC`) page 1 holds the NEWEST
rows and ids are monotonic with `created_at`, so page 1 already carried the
true maximum id AND every row newer than the mark — the only rows the
post-write check selects on. A single-page read therefore missed a raced
verdict only if more than 50 rows were created INSIDE the write window, not
merely on "a head with more than 50 rows", which the issue, the comments and
the docs all asserted. Reviewer executed an order-faithful DESC stub: a
page-1-only reader repairs identically to the full walk.

What actually removed #761's stall is retiring #751's page-2 probe, not the
paging. The walk still earns its place, for a reason now stated instead of the
false one: it stops the gate's one fail-toward-SUCCESS path depending on an
undocumented ordering the server honours only coarsely (page 1 came back
`114,112,113,111,110`). That measurement was deleted in commit 1 and is
restored, since round 4's safety argument rests on exactly it.

MEDIUM/real defect — the string-id TWIN, live on `main` and one expression
away from the fix already made: `select((.id? // 0) > $since)`. jq orders
strings above every number, so a PRE-EXISTING row with `"id": "3"` reads as
newer than any mark, is counted as having raced the write, and gets the sticky
sentinel plus a false "was overwritten" on EVERY later run — a permanent
per-sha stall no re-trigger clears. Now numeric-only, with a test.

Also fixed: a non-empty history carrying no numeric id was collapsed to a mark
of 0 (making every pre-existing row look newer); it is now reported unusable
and the check is skipped. `sleep` no longer fires after the final attempt.

Three unpinned clauses now have tests, each proved by an executed mutation:
  - the page cap is a refusal, not a terminator (1050-row fixture)
  - the `::error::` found-vs-unverifiable distinction (forcing `raced_why=human`
    reddened nothing before)
  - the walk requests no sort order — a structural guard on round 4's
    withdrawal, which nothing mechanical protected. It reads request LINES, not
    comments, since the withdrawal note names the parameter to explain it.

Honest scoping, not new code: the test double is ordering-blind, so the paging
tests prove WALK COMPLETENESS, not that a real raced verdict would otherwise be
missed — under DESC it would not be. The stub comment and the docstrings now
say so rather than implying the stronger claim.

Docs: `ci-cd.md` and the record's `rule:` carry the corrected reachability, the
DESC dependency of the partial-mark fallback, and both rejected alternatives
stated as rejected alternatives rather than as draft chronology
(`docs.no-session-narrative`).

Verification: `scripts/tests` 1094 passed, 2 skipped; eleven executed
mutations across rounds 2-5, each reddening exactly its named test;
decisions_validate and build_decisions_catalog --check exit 0.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:07 +02:00
timothyandClaude Opus 5 1eb7b41fc1 fix(763): round 4 — withdraw the ASC sort, which inverted the partial-mark fallback
Round 3 added `sort=highestindex` to close a mid-walk-insert gap: under the
server default (`created_unix DESC`) a row inserted while the walk is running
lands at position 0, on a page already read, so the walk never sees it.

That fix and the round-2 partial-mark fallback are incompatible. ASC puts the
OLDEST rows on page 1, so an incomplete walk takes its high-water mark over
the oldest rows — leaving every pre-existing row above the mark and read as
"raced". That is a spurious STICKY repair on a head nothing raced, which is
precisely the #761 failure this whole issue exists to remove. Under the
default DESC the newest row is on page 1 by construction and ids are monotonic
with `created_at` (measured), so a partial mark is at or very near the true
maximum and "lower is safe" actually holds.

Two defects from one mechanism again, so the mechanism goes rather than
getting patched: the sort is withdrawn and the mid-walk-insert residual is
ACCEPTED and documented. It is bounded — a row arriving after this job's POST
is not one this job overwrote, and being newest it wins on the combined
endpoint branch protection reads.

Both the code comment and the docs record the withdrawal and the reason, so
the next reader does not re-adopt it.

Verification: `scripts/tests` 1090 passed, 2 skipped; the partial-mark mutation
still reddens `test_a_PRE_WRITE_paging_failure_still_yields_a_usable_high_water_mark`;
decisions_validate and build_decisions_catalog --check both exit 0.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:07 +02:00
timothyandClaude Opus 5 007d2fd3df fix(763): round 2+3 — close the fail-opens the paging change introduced
Two independent cold reviews (Codex GPT-5.6 cross-family, and an isolated
Opus agent) converged on the same blocker, which is fixed here along with
everything else they found.

BLOCKER — the mark walk turned a fail-closed case into a fail-open. The
high-water mark gates the post-write race check entirely: `max_id_before=-1`
skips it. Before paging, only a failure of the single page-1 request could
reach that. Requiring a COMPLETE walk newly routed a page-2 hiccup, an
over-cap history, or one malformed id on a later page into the same hole, so
a human rejection racing the write was left green where `main` repaired.
A partial list now still yields a mark: it can only be LOWER than the true
maximum, which makes the check more eager, never blinder. Only a read
returning no rows at all abandons it — the pre-existing #849 gap, unchanged
and now asserted by a test so it stays visible.

WITHDRAWN — the "currency witness". It produced two defects from one
mechanism, which is the signal to remove rather than patch twice: counting
ANY row above the mark does not witness this job's write, so a stale-but-valid
snapshot carrying an unrelated newer row passed while hiding a rejection; and
a schema-valid stale read is not retried, so one such response turned a
transient anomaly into a permanent sentinel. The hazard has no mechanism here
either — Gitea is a single instance with no read replicas. Removing it
restores the pre-change exposure on that path, a non-regression.

Also fixed, each a fail-open with a fixture and an executed mutation:
  - `.creator` is type-tested before indexing. `.creator.login` on a non-object
    exits jq 5 and `set -e` took the step down after the green was posted and
    before the repair. Reproduced by both reviewers.
  - the mark is the max over NUMERIC ids only. jq orders strings above every
    number, so one `"id": "99999"` passed the numeric gate and inflated the
    mark until nothing looked newer.
  - an unusable `raced` count now repairs instead of "not acting on it".
  - `sort=highestindex` (ASC, measured) so a row inserted mid-walk appends at
    the end rather than at position 0 on a page already read. An unknown sort
    value silently falls back to DESC, so this is insurance, not load-bearing,
    and the comment says so.
  - `ph_ok`/`ph_rows` renamed off `read_existing_verdict`'s `st_ok`. No live
    bug, but a name collision in a 1400-line step.

Tests the reviews showed were missing, each proved by an executed mutation:
  - verdict beyond a SHORT page (a deliberately unfaithful truncated response
    — against a faithful double a short page is always the last, so the rule
    "terminate only on an EMPTY page" was unobservable)
  - pre-write paging failure still yields a usable mark
  - pre-write read returning nothing abandons the mark and says so
  - a TRANSIENT page failure is retried (the retry was unproven code: every
    other error mode fails on every attempt, so disarming it reddened nothing)
  - a string id cannot inflate the mark
  - a malformed `creator` row does not kill the job

Stub corrections, both the same class as the earlier `[]`-vs-`null` gap: it
served one flat list (so paging was unobservable) and computed its own-post id
with `max()` over mixed str/int, which raised TypeError and made the string-id
test pass because the DOUBLE crashed rather than because the mark was right.

Mutation matrix, all executed, each reddening exactly its named test: retry
disarmed; numeric-max reverted; partial-mark fallback removed; short-page
terminates; page-1-only walk; post-write fail-closed flipped open; jq
type-guard reverted. The unusable-count arm is unreachable by any fixture and
is annotated as such rather than claimed as proved.

Verification: `scripts/tests` 1090 passed, 2 skipped; decisions_validate and
build_decisions_catalog --check both exit 0; terminator, clamp, sort order and
id monotonicity all re-measured live on Gitea 1.27.1.

refs #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:07 +02:00
timothyandClaude Opus 5 4368cc8cbe fix(763): page both /statuses/{sha} reads to a validated terminator
`review-verdict.yml` read the per-POST status history twice with a single
`?limit=100` request. `limit` clamps to the server-wide `MAX_RESPONSE_ITEMS`
(measured 50), so on a head carrying more rows than the clamp both reads saw a
partial list. The high-water mark was only page 1's maximum, and — the direction
that matters — a raced human verdict beyond page 1 was invisible to the
post-write race check, leaving a forged green over a rejection.

Both reads now walk to a validated empty page (`[]` on this endpoint, measured
2026-08-28 against PR #761's 114-row head: pages 1-2 return 50, page 3 returns
14, page 4 is `[]`), never terminating on a short page, under a 20-page cap and
retrying each page once. Correctness does not depend on the cap value.

This retires #751's page-2 "assume raced" probe, which repaired every head that
outgrew one page. It fired on Renovate PR #761: an `::error::` claimed a human
verdict had been overwritten on a head carrying none, and the sticky sentinel
then refused re-exemption on every later run.

Two properties replace it. Uncertainty now fails closed at both ends — the
unreadable-history branch warned and left the exemption green while the page-2
probe repaired on the same uncertainty, one check disagreeing with itself; this
is affordable only because paging removed the common trigger. And the post-write
read must witness the job's own write: reaching a validated empty page proves the
walk finished, not that it saw a current list, so at least one row above the
pre-write mark must exist because the job just posted one.

The `::error::` now distinguishes a verdict actually found from an unverifiable
read. The sentinel description stays generic — the classification recognises it
as a fixed point, so its wording is load-bearing.

The stub gained faithful paging (50-row slices, `[]` past the end, one snapshot
per logical read so a counter mode cannot describe two different histories across
pages) and, separately, modelling of the job's own POST appearing in the history
— which it had never done, so in its world every ordinary run looked like a head
nothing had been posted to. `own-write-invisible` withholds exactly that detail
as the negative control for the currency witness.

Mutation-proved by execution, one clause at a time:
  - walk reads page 1 only -> RUNNING_PAST_PAGE_1_is_PAGED_and_the_exemption_
    STANDS, raced_verdict_on_PAGE_2_is_detected_and_repaired and both UNREADABLE
    history tests go red
  - currency-witness zero branch deleted -> CANNOT_SEE_OUR_OWN_WRITE red
  - fail-closed flipped to fail-open -> both UNREADABLE history tests red

Verification: `scripts/tests` 1085 passed, 2 skipped; decisions_validate and
build_decisions_catalog --check both exit 0.

fixes #763

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 17:34:07 +02:00
timothyandtimothy b16ec15d6c docs(781,799): re-measure the tooling audit from a derived population, and adopt serena (#862)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 23s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m36s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m10s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m13s
§5.3's verdicts rested on a single surface, which manufactured four false zeros: codex is driven
through `codex exec` inside Bash, security-guidance and ralph-loop expose no tool at all and run as
hooks (1,086 executions each), and feature-dev is used through its agents. The audit also compared
current enablement against historical usage — six of the eight plugins it called "genuinely unused"
were disabled for 16 of the 30 corpus days.

The retirement half of #781 is answered *no* on evidence: the zeros split six ways and only one is
grounds for removal. Eight plugins are kept by operator decision.

#799's observation was correct and its cause is now established. serena was `false` in settings.json
until 2026-08-14T12:31Z, when a concurrent session enabled it; its tools appear in no transcript
before 12:42:54Z. #799's session started at 12:01Z and never reloaded, so its probe correctly found
nothing while the settings file already said `true`. serena is adopted and documented as the third
code-intelligence surface.

Four review rounds, two independent cold reviewers (one cross-family); rounds 1-3 BLOCKED.

fixes #781
fixes #799

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-28 14:37:34 +00:00
timothyandtimothy 761e575836 fix(787): derive the dropped-step guard's scope, and reconcile its snapshot against the server (#861)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 21s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m45s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m22s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m56s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m33s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-27 22:37:02 +00:00
timothyandtimothy 8aebba4d89 fix(748): declare permissions: on all six workflows, and prove the declaration binds (#860)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 18s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m40s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m4s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m20s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-27 22:02:44 +00:00
timothyandtimothy 45b17e58e1 fix(744,835): ci-image.yml publishes from main only; guard persist-credentials with no exemption (#857)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 23s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m53s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m23s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m15s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m27s
Closes the push route into ci-image.yml (#744) and ships the persist-credentials guard that was waiting on it (#835).

ci-image.yml's push trigger had no branches: filter and was path-scoped to docker/ci/** AND to the workflow file itself. Gitea resolves a push workflow's definition from the pushed ref, so any branch push touching those paths ran that branch's own YAML on a docker-capable runner holding the credential that writes ersatztv:prod and the ersatztv-ci:<sha> five container: jobs execute.

Be precise about what the filter buys: it is loaded from the pushed ref like the rest of the file, so a branch that deletes it re-enables the route. This closes the DRIVE-BY case - publication as a side effect of an ordinary push - and is not a boundary against a writer who intends to run their own YAML. The wider class is #853.

The self-reference left both paths: and ci-image-pin's expected in the same change - a decided tradeoff with both prices stated, not a necessity. Branch publishing moves to workflow_dispatch, probed live: run 2340 on this branch published ersatztv-ci:43b1e45 and left :latest unchanged.

With both mechanical blockers gone, ci-image.yml's checkout takes persist-credentials: false (16 of 16) and scripts/tests/test_workflow_persist_credentials.py holds the convention with NO exemption list - git-index population, declared clause mutation re-run every suite, guard-inventory rows.

fixes #744
fixes #835

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-27 20:44:44 +00:00
timothyandtimothy d68ce42fcb fix(742): inherit an h10 verdict only from an allow-listed reviewer (#850)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 12s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m55s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m25s
`review-verdict.yml` decided whether an existing `review-verdict/h10` was worth INHERITING by
testing `.creator.login != null` — satisfied by any account's credential, including the `renovate`
bot's `RENOVATE_TOKEN`, a `write:repository` PAT that cannot be scoped down the way #697 scoped the
registry credential. The test is now membership in `H10_REVIEWERS="timothy"`, a literal in the
base-resolved definition.

The design that survived 11 cold review rounds:

* `read_existing_verdict` carries TWO flags. `ex_human` (attributable AND allow-listed) gates
  INHERITANCE; `ex_attributable` gates the last-moment re-read, which asks the opposite question and
  must stay broad. Narrowing both — the first draft — makes the job post its exemption over a
  mid-run rejection, and the post-write repair does not cover that.
* The two calls no longer compute an identical predicate, so "changed" is made explicit: the
  state/creator/description triple from the first read is snapshotted and compared.
* The allow-list governs an inherited `success` ONLY. An existing `failure` inherits on
  attributability alone, because inheriting a rejection can only withhold an exemption while
  re-deriving one can turn it green on an exempt PR. A symmetric rule was a measured fail-open.
* The post-write raced check stays broad — not because narrowing would let a rejection go green
  (a real reviewer is on the list by construction), but for the misconfiguration case.

Two mechanisms were WITHDRAWN rather than patched a third time, and both withdrawals are recorded
in `ci.exemption-provenance` so they are not re-attempted: a `::warning::` annotation that produced
three defects in three rounds, and a post-write fix whose generic `pending` would have been
re-derived anyway and which had no retry trigger.

Verified: the inheritance predicate driven against the LIVE Gitea API on a probe-named context,
both allow-list directions; every clause mutation-proven against the shipped file; `scripts/tests`
1012 passed, 2 skipped.

Follow-ups filed: #845 (post-review-verdict.sh does not check its own account is allow-listed) and
#849 (post-write verification: three routes leaving an exemption `success` over a human `failure`,
plus the retarget fence's post-POST gap, plus the prose sweep that lands with the behaviour).

fixes #742

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-27 02:52:02 +00:00
timothyandtimothy ed8b602445 feat(735): bound the numeric FFmpeg profile fields with a 422, and expose readrate pacing (#847)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 11s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m9s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m41s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m23s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 6m23s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 22:05:38 +00:00
timothyandtimothy 469d19852c fix(788): one declarative H10 verdict vocabulary, derived by both sides (#846)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 6s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m34s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m17s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m22s
The verdict words lived in two hand-written shell copies — the `case` arms of
post-review-verdict.sh (write) and the POS_RE/NEG_RE regexes of
check-review-verdict.sh (read) — held together by nothing but a comment that had
already gone stale. scripts/lib/review-verdict-vocabulary.sh now declares them
once and both sides derive; neither script enumerates a verdict word any more.

Only the WORD SET moved. The grammar stays in check-review-verdict.sh, where
every #629 false-open actually lived.

No parity test: #774 shipped one and withdrew it after six rounds, because a
regex over shell source is not a shell parser. The proof is behavioural and
graded MUTATION — the harness restores the pre-#788 hardcoded POS_RE each run and
requires it to redden.

Enforcement is a DATA dependency, not a control-flow gate. Review round 1 found a
real fail-open in the first commit: `${#arr[@]}` is nounset-safe only for a
declared-empty array, and under `set -u` that error inside a function called as
`if ! validate` skips BOTH branches — so on the reader (deliberately no `set -e`)
an explicit BLOCKED @ head classified `positive`, exit 0. Validation now sets a
sentinel on its last line and the derived views refuse without it.

Six cold review rounds; rounds 2-6 found no fail-open across differential fuzzing
(4788 / 2612 / 7560 payloads, zero divergences from origin/main's grammar),
sentinel forgery, environment poisoning, declare -p evasion on bash 5.3 and 3.2,
path/symlink resolution and probe TOCTOU. Every malformation fails closed: reader
exit 2, writer exit 1 with nothing posted.

Also corrected: CLAUDE.md and release.review-verdict-gate both enumerated the
vocabulary without LGTM, a word the code has accepted since #629.

fixes #788

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 20:58:09 +00:00
timothyandtimothy ba6a4b08aa feat(732): On Now / Next gets a background box, and is on by default (#843)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m40s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m18s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m12s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m13s
probe742/combined-newest SECOND
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 19:28:25 +00:00
timothyandtimothy f2551b778e fix(746): drop the persisted checkout credential; unmask the base-ref fetches (#842)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 24s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 30s
Build ErsatzTV Image / Build & test (.NET) (push) Failing after 1m37s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m14s
Build ErsatzTV Image / Build & push image (amd64) (push) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m1s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 18:50:56 +00:00
timothyandtimothy 40a3232d9e feat(734): field-level progressive disclosure — shared FieldHelp trigger + panel (#841)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 15s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m57s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m32s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m18s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m27s
Implements the three-level field-help pattern from #734 as a shared component: field name + optional one-sentence summary → a one-short-paragraph panel behind a consistent Info-icon trigger → a future external-docs deep link (`docsHref`, built and typed; no screen passes one yet).

Adopted on FFmpegProfilesScreen (9 fields), documented as docs/spa-conventions.md §15 with decision record `spa.field-progressive-disclosure`, and mirrored into the design-system prototype.

The panel is portalled to document.body: `.ctv-card` sets `overflow: hidden`, which clips a positioned descendant whatever its z-index, and one field's explainer rendered 12px of a 92px paragraph in every state of the Audio card.

A `::before` hover bridge was added and then WITHDRAWN — it held for a vertical descent onto the panel and failed for a diagonal one, leaving a safe sideways exit of 1.25px on an 18px icon. Hover reads the paragraph in place; the panel's interactive content is reached by pinning.

Four cold adversarial review rounds; the first three returned BLOCKED. They found five wrong copy claims across nine paragraphs and two vacuous tests in a row for the same mechanism.

Deferred with owners: #839 (placement verified by hand, not by a test) and #840 (the portal puts a docsHref link at the end of the tab order).

fixes #734

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 18:05:53 +00:00
timothyandtimothy 35affecd29 docs(755): ersatztv owns the fork code, media-management owns channel operations (#838)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 20s
Build ErsatzTV Image / Build & test (.NET) (push) Failing after 1m41s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m7s
Build ErsatzTV Image / Build & push image (amd64) (push) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m5s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 17:29:16 +00:00
timothy e8e17f3844 docs(708): record the live route-2 reproduction against PR #761 (#834)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m57s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m5s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m53s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m24s
2026-08-26 09:11:23 +00:00
timothyandtimothy 08cd3a002d fix(690,758): count the same query a paged handler pages (#833)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 14s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m53s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m18s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m40s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m46s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-26 07:39:23 +00:00
timothyandtimothy 7453dd3a82 fix(721,740): align Auto-Tune proposal rows on a grid; guard AddItemsDialog's async searches (#831)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 8s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m35s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m38s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m58s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Failing after 1m17s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-25 21:55:23 +00:00
timothyandtimothy 3e2c07b42f fix(807): SPA full-replace bodies are built as Complete&lt;T&gt;, with a derived guard over droppable request members (#827)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 14s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 30s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m36s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m9s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m49s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Failing after 15s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-23 00:58:19 +00:00
timothy 7966e08164 fix(701): guard SongMetadata's nullable primitive collections at the read site (#826)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 16s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m29s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 5m59s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m55s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m15s
2026-08-22 23:24:41 +00:00
timothy ad31a02850 fix(772,792): name the missing toolchain image; stop a refusal leaving a verdict comment (#825)
Build ErsatzTV Image / CI toolchain image resolves (push) Successful in 13s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 18s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 12s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m11s
2026-08-22 23:05:34 +00:00
timothyandClaude Opus 5 b5dee26202 fix(701): guard SongMetadata's nullable primitive collections at the read site
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 8s
PR Gates / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 19s
review-verdict/h10 Review-verdict: MERGEABLE @ b5dee26 (base: main)
PR Gates / decisions lifecycle (pull_request) Successful in 20s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 15s
Review verdict / Set review-verdict status (pull_request_target) Successful in 6s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 4m32s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m44s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m18s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m53s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Both search indexers opened UpdateSong with

    metadata.AlbumArtists ??= [];
    metadata.Artists ??= [];

Artists/AlbumArtists hold the whole list in ONE COLUMN rather than
being navigations. So unlike the same `??= []` idiom on
Genres/Tags/Artwork all around them, the property IS the column value:
assigning it on a TRACKED entity flips the entry to Modified and the
next SaveChanges writes [] over a NULL column. This is the mechanism an
adversarial review demonstrated in #691, which is why that issue's
entity-level guard was reverted in favour of guarding at the read site.

Measured rather than reasoned about, per the issue's first done-when
box. Restoring ONLY the `??= []` clause (the real predecessor lines,
not a hand-written mutant) reddens the new fixture on
`metadata.Artists should be null but was []`; a probe variant with the
first two assertions replaced by prints reports STATE=Modified and the
raw column moving from NULL to "[]". Today's two feeds are both
AsNoTracking (SearchRepository.GetItemToIndex and GetAllSongs), so no
shipped caller loses data -- but that is a property of two callers, not
of the indexer, and #691 already recorded it as a loaded gun. The
fixture pins the indexer's own contract instead.

Removing the assignment is not sufficient alone: it was load-bearing
for the four reads below it, and deleting it by itself converts a
silent write into a live throw on every untagged song. Measured by
deleting only those two lines from the real predecessor file:
NullReferenceException, thrown at the foreach (cited by symbol: a line
number in a mutant that exists in no committed tree is unreproducible
by construction). The
exception type follows the read FORM, not the field -- foreach yields
NRE, string.Join/ToList yield ArgumentNullException -- and this PR
contains two of each, which is why no single exception-name grep
characterises the class. So each site moves together with its reads:

- LuceneSearchIndex.UpdateSong / ElasticSearchIndex.UpdateSong: hoist
  Optional(...).Flatten().ToList() locals and read those.
- RefreshChannelDataHandler: the Scriban context took the raw nullable
  lists (the issue's second item). The shipped _song.sbntxt only does
  array.join, but a custom template is free to do anything.

The population was derived from the MODEL rather than from the issue's
file list, and the obvious derivation is wrong: "the IList<string>
properties under ErsatzTV.Core/Domain" returns two of eight. It misses
the six value-converted collections (ProgramScheduleAlternate and
PlayoutTemplate each carrying DaysOfMonth, MonthsOfYear, DaysOfWeek),
declared as plain ICollection<T> and made single columns only in
Data/Configurations -- and their storage differs (comma-separated text
for the int converter, JSON for the enum one), so the shared property
is "one scalar column", not the serialization. No site applies `??=`
to any of the six, so this defect has no instance there; whether a null
can REACH one at runtime is unverified and is filed as #823 rather than
asserted either way. Only the SongMetadata pair is left NULL in
practice, by FallbackMetadataProvider. Every site touching either field
was then swept; the remaining readers were already guarded by #691.

The fixture carries two anti-vacuity guards, both witnessed:

- A POSITIVE CONTROL (`writer.NumDocs.ShouldBe(1)`). Every other
  assertion says something did NOT happen, so all of them hold
  vacuously if UpdateSong never runs -- and it silently stops running
  if a future refactor gates UpdateItems on `_initialized`, which this
  fixture bypasses by injecting the writer. Verified BOTH directions:
  with that gate added the control fails `NumDocs should be 1 but was
  0`, and with the control removed the whole test PASSES while the code
  under test is unreachable.
- A capturing logger, because UpdateSong wraps its body in a catch that
  assigns metadata.Song = null -- severing a required relationship and
  cascading the metadata to Deleted. Without it the probe silently
  measures the error path; on the first run it did exactly that (a bare
  ILanguageCodeService substitute NPEs inside AddLanguages). The raw
  column helper also fails loudly on a missing row, since ExecuteScalar
  returns CLR null for both "NULL column" and "no such row".

ElasticSearchIndex has no equivalent fixture -- it needs a stubbed
transport -- so its change is by inspection against the Lucene one, and
the gap is filed as #824 rather than covered by a source-text guard.

The whitespace-only churn in ElasticSearchIndex.cs is the #311
fix-as-you-touch format gate: it scopes to whole changed FILES.
`git diff -w` over that file shows only the two hunks above.

Local gate: ErsatzTV.Tests 2006 passed / 4 pre-existing skips,
Core.Tests 685/1 skip, Infrastructure.Tests 114, Architecture.Tests 7,
Scanner.Tests 1504 -- 0 failures in each. scripts/tests 874 passed / 2
skipped. dotnet format whitespace --verify-no-changes clean on the four
touched files, no BOM on any. decisions_validate OK.

Fixes #701

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 00:49:57 +02:00
timothy 78fc283627 docs(772): the merge gate decides on the COMBINED state, and skipped counts as green
Build ErsatzTV Image / CI toolchain image resolves (pull_request) Successful in 8s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 19s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 12s
PR Gates / Docs update reminder (pull_request) Successful in 34s
PR Gates / decisions lifecycle (pull_request) Successful in 23s
Review verdict / Set review-verdict status (pull_request_target) Successful in 11s
PR Gates / Script lint and tests (ruff + pytest) (pull_request) Successful in 5m0s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m21s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m59s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
review-verdict/h10 Review-verdict: MERGEABLE @ 78fc283 (base: main)
Graded a nit and not blocked on, but it is a claim about a neighbouring subsystem that is
one notch too strong: the hook decides on the combined `.state` and its filters exclude
`skipped` (#593). Left as-is it would have taught the next reader that any non-success
context denies, which is what #593 exists to correct.

refs #772
2026-08-23 00:39:41 +02:00
timothy 8c72a5de9e docs(772): correct the twins of two claims this branch already fixed
Both remaining review findings were the same shape as the one before them, and it is the
shape this repo keeps recording: a claim corrected in one place, its copy left standing
somewhere else in the tree.

  * `docs/ci-cd.md` said "gates nothing" in the small-lane paragraph while the section 1441
    lines below said the opposite. A red preflight lands in the PR's combined status, which
    the merge gate reads (#598) — what it does not do is SKIP the jobs it diagnoses, and
    that is now the sentence in both places.
  * Two docstrings in the preflight's test file still described the disarmed script as
    warning and exiting 0. Built the mutant and ran it: it emits an error and exits 1. The
    exit code separates nothing now that an unverifiable answer fails too — the DIAGNOSTIC
    is what the mutation destroys, which is what `mutation_manifest.py` already said and
    the prose next to it contradicted.

Nits from the same pass: the admin-cron URL is quoted (`?` globs in zsh, the operator's
shell); the retry assertion's message quoted a threshold it does not use; the arm table
omitted the malformed-credential shape the code and tests both have; `buildx inspect` no
longer `--bootstrap`s a builder just to read its name, and an empty capture no longer
produces a noisy `buildx use ""`.

Swept the tree for the shape rather than the two reported lines: the surviving "exits 0"
and "could-not-tell" hits are other subsystems, or the concept named as a concept.

refs #772
2026-08-23 00:23:44 +02:00
timothy d304c8573e docs(772): state the trap, not the draft history
The advisory narrative check was right about both new passages: 'an earlier draft warned and
exited 0' and 'both cold reviews found it independently' only parse to someone who saw the
session. What a cold reader needs is that warning-and-exiting-0 is the natural way to write
this check and is wrong, and what it costs — which is now what the doc says.

refs #772
2026-08-22 23:48:52 +02:00
timothy 30640bb780 fix(772,792): close the fix-round review — the doc still described the semantics the fix reversed
The re-review's one HIGH was mine and was the obvious one to miss: the previous commit changed
the preflight so an unverifiable answer FAILS, and left a `docs/ci-cd.md` paragraph two
screens away still saying "anything else is reported as could-not-tell". That paragraph is
the one an operator reads when the job goes red, and it would have talked them into
reinstating the defect. Replaced with the full arm table, including the two rows the first
draft got wrong and why.

  * "gates nothing" was false in the way this repo has recorded before (#598): the
    merge-consent hook reads the COMBINED status, so a red preflight blocks the merge like
    any other red job. It does not SKIP the jobs it diagnoses; that is the accurate claim,
    in ci-cd.md and in the remote-state row.
  * The production retry defaults were evaluated by nothing — every test overrode both
    knobs. A test now drops the overrides and measures three attempts and a real pause, so
    editing the default to 1/0 (which would falsify the "a blip does not redden a PR"
    argument) goes red.
  * `journalctl -u gitea | grep ExecuteCleanupRules` is not a reproduction: that identifier
    reaches the log only through slow-query warnings, so an empty grep on a healthy host
    reads as "the rule never ran" — the inverse. Replaced with the admin cron API, which
    answers deterministically.
  * The recovery recipe's `docker buildx use default` needs the containerd image store to
    `--push` (both named hosts have it, checked today) and mutated the operator's builder
    selection without restoring it.
  * The stub's comment claimed both halves of real curl's transport failure mattered; only
    the exit status is observable, because `|| resp=""` discards what curl printed.
  * The empty-half credential refusal echoed the username; it needs no value at all. The
    401/403 arm aborts the remaining pins while 404 continues — deliberate, now stated.
  * `curl -u "$VAR"` puts a credential in argv, and this job runs container-free on a shared
    host. NOT fixed here: it is the shape all five `scripts/` callers already use, so fixing
    one site leaves the class and splits the codebase. Filed as #821 and named at the site.

refs #772
refs #792
2026-08-22 23:48:20 +02:00
timothy 0cf355e494 fix(772,792): address both cold reviews — an unverified pin is no longer a green job
Two independent reviewers (one cross-family) converged on the same defect, and it was the
important one: the preflight WARNED and exited 0 on every answer that was not 200 or 404,
so a missing `curl`, a moved registry or a DNS change would have left it green forever —
"the check could not run" presenting as "the pin is fine", in a script whose own header
disclaimed exactly that. Unknown answers are now retried (3x, 5s) and then FAIL, with
wording kept distinct from the deleted case because the two send an operator to different
places.

Also from the reviews:

  * An absent secret does not arrive as an unset variable. `${{ secrets.X }}:${{ secrets.Y }}`
    interpolates to ":", a perfectly non-empty and perfectly useless credential, and the
    tests covered only the unset shape. Both halves are now required, and the parametrised
    test drives the production shape.
  * HTTP 200 is not a manifest. A proxy or a login page answers 200 too, so the body is
    fetched and matched for `schemaVersion` (a shell `case`, so no jq dependency and no
    pipeline that can inject).
  * The curl stub ignored `-u` and answered 200 regardless, so deleting the real `-u` would
    have left the suite green while the live registry rejected every request. It now 401s an
    unauthenticated read, as the registry does.
  * The mutation's declared diagnostic changed with the script: now that unknown fails too,
    the exit code no longer separates "deleted" from "could not check", so the proof turns on
    the message and `expect` says so.
  * docs/ci-cd.md: `scan` is no longer the only `docker-build.yml` job on the small lane, so
    the tag-push exclusivity claim and the lane membership were both false. Fixed.
  * "Immutable" was overstated: `ci-image.yml` tags `rev-parse --short HEAD`, so a dispatch or
    a weekly no-cache run at the same HEAD republishes that tag from a rebuilt image. Stated,
    along with what the rebuild recovery does NOT restore (mutable bases and apt, so equivalent
    rather than bit-identical).
  * The recovery recipe left you in a worktree checked out at the pin commit — where the
    verify script does not exist, and where the workflow carries the pre-bump pin. It now
    keeps `$repo`, returns, and removes the worktree. It also needed BuildKit's `http = true`
    caveat: the container driver does not inherit the daemon's insecure-registries.
  * The root cause carries its evidentiary limit and its reproduction commands, and says what
    to conclude if a pin vanishes after server-management#842 lands (refuted, not re-applied).
  * The `ci.required-job-step-execution-markers` carve-out named one container-free job; there
    are two now, and the membership is what rots.
  * The decision record's `''` YAML escapes leaked into rendered prose; "status, no comment ->
    ask" is qualified (a prior positive verdict for the SAME head still satisfies condition
    (c)); "exits 1" is "exits non-zero" (usage exits 2, jq its own status, signals 128+n).

refs #772
refs #792
Decisions-Edit: yes
2026-08-22 23:25:26 +02:00
timothy 5ba442c11c fix(772,792): name the missing toolchain image, and stop a refusal leaving a verdict comment
#772 — the pinned CI toolchain image can be deleted out from under us, and when it was
(2026-08-11..13) all five `container:` jobs died at image pull, both required contexts
included, with the cause buried in each job's log. Root cause is registry-side and is now
established rather than guessed: an owner-level Gitea package cleanup rule (keep_count 15,
remove_days 1, remove_pattern `.*`, keep_pattern no 7-hex sha can match) deletes a sha tag
once 15 newer versions exist, and `ExecuteCleanupRules` ran nightly through the window. The
`ersatztv` package carries the same rule's fingerprint exactly — every sha tag older than
the 15-slot window is gone, every keep_pattern tag back to 26.3.1 survives. Version deletes
leave no audit row, so the specific run cannot be replayed; that limit is stated where the
claim is made. The durable fix belongs to the registry's repo: server-management#842.

What lands here is what a consumer of someone else's registry can do:

  * `toolchain-preflight`, a container-free job (a job consuming the image could not run to
    report it missing) resolving every pin against the registry and failing with a message
    that names the tag and the recovery. Not a `needs:` of the jobs it diagnoses — gating
    five jobs behind a checkout and one curl taxes every green run to speed up a rare red
    one, and they already fail fast.
  * Only HTTP 404 means gone. Everything else is could-not-tell, and rejected credentials
    fail rather than pass as unknown — "the check could not run" must never present as
    "the pin is fine".
  * A recovery path that does not need CI: rebuild the SAME tag from the commit it names
    and push it. The push half was verified against this registry on 2026-08-22 with a
    throwaway package (created, resolved 200, deleted).

#792 — the reported defect was the exit code, and re-measuring says that premise is false:
every no-status path already exits 1, and eight refusal modes now assert it against the real
predecessor, where they pass. The observed 0 came from the invocation, not the script. What
WAS broken is the half-state the issue describes second: the comment was written before the
status, so every refusal left `Review-verdict: MERGEABLE @ <head>` on a PR with no gating
status behind it. The two writes are now ordered status-then-comment, which makes the only
reachable half-state the safe one — a status with no comment leaves the merge hook's
condition (c) with nothing to classify, which is an `ask`. The refusals themselves are
untouched. Ordering rather than compensating deletion: an orphaned-comment cleanup needs a
Gitea call, and these refusals are usually caused by Gitea being unreachable.

Proof for the ordering is the split against origin/main's script: the 8 orphan/ordering
tests go red there, the 8 exit-code tests stay green.

fixes #772
fixes #792
Refs: server-management#842
Decisions-Edit: yes
2026-08-22 22:52:56 +02:00
timothyandtimothy 29d7a06e65 fix(806): guard populations over FILES derive from the git index, not a filesystem walk (#818)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m53s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 5m57s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m56s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m20s
`testing.guard-derives-population-from-source` (#774) was silent on the commonest
population in our own guards — files in a directory — and every one answered with a
filesystem walk. A walk is not authoritative: it reports build output, generated
shims and editor droppings, and differs per machine. #778 measured the cost by
getting the same population wrong three times in one PR.

CONVERTED (a completeness claim over tracked files): `test_guard_inventory.py`,
`test_hook_fire_log.py`, `test_ci_image_pin_population.py` (which also gained
`*.yaml`), `test_remote_state_inventory.py` (folded onto the shared derivation), and
`test_pr_changed_files.py` (not on the issue's list — found by sweeping the whole
repo).

ASSESSED AND RECORDED, not silently skipped: `_repo_copy` takes its file list from
the index for hermeticity though it makes no completeness claim;
`test_ci_dropped_step_guard.py` has no filesystem population at all; the decisions
corpus is recorded as unexamined rather than cleared; and the SPA page-size guard is
deferred to #819 with its obstacle documented. This is not "replace every glob".

`scripts/tests/tracked_files.py` is the single derivation.
`test_guard_populations_derive_from_git.py` proves it in two measured complements:
exhaustive removal catches a hardcoded `.exists()` admit and memoisation; the call
log catches an append-only source that yields nothing on this machine — #778's
shape — which removal cannot see because it has nothing to remove.

Twelve rounds of independent cold review, alternating model families in isolated
worktrees. The production derivations were confirmed sound every round; every
blocking finding after the first was in the proofs or in prose claims about them.
Counts over growing populations were removed rather than corrected, after three
drifted.

fixes #806

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-22 16:07:58 +00:00
3a53c50c45 docs: a cancelled job reads as failure where a monitor actually looks (#817)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 24s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m38s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m13s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m1s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m13s
`ci.cancelled-is-not-a-verdict` documented the run/job API, where cancelled is distinguishable. The endpoint a CI monitor actually polls — `commits/{sha}/status`, the per-sha view the merge gate reads — has no `cancelled` state and reports one as `failure`. The record now says to resolve the job-level `conclusion` before reporting a red.

The kickoff's push HARD CONSTRAINT is tightened from "a review has run" to "a CLEAN verdict, zero outstanding findings on the current tree", since #790's rounds 7 and 8 each still found a real mechanism defect and every earlier push auto-cancelled a live run.

Refs #790

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-22 13:35:21 +00:00
3b386649bf feat(790): execute every MUTATION grade instead of asserting it (#815)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 18s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m2s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m30s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m9s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m10s
Every `MUTATION` row of `docs/guard-inventory.md` now carries a DECLARED clause mutation that is applied to an isolated copy of the repository on every suite run, with the row's own named test required to go red carrying a declared diagnostic. Manifest and MUTATION rows are compared for set equality both directions; the other 22 guards each carry a stated reason, compared the same way.

Measured rather than assumed: 12 of 13 guards admit a single-clause mutation; `instrumentation_faults` does not, and that entry carries the surviving finer mutation, re-run every suite.

Nine cold cross-family review rounds. Rounds 1, 2, 7 and 8 each found real mechanism defects — two mutations that measured nothing, an incomplete git-environment sanitisation, a reset that restored its own mutant, and a proof of that fix which was not itself isolated. All fixed and witnessed red.

fixes #790

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-22 13:12:15 +00:00
timothyandtimothy eb5c74b4b2 fix(780): reconcile the ruff gate with #784, merged in parallel (#814)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 19s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m30s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m5s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m38s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m3s
#780 and #784 were green separately and red together: the gate landed on a base
that predated scripts/check-doc-narrative.py, so nothing ever ran ruff over it.

- RUF100 x2 on `# noqa: BLE001` — BLE is not in this repo's select, so those
  directives suppress nothing. Enabling BLE instead was measured and rejected: 6
  further sites in decisions_validate.py, whose broad catches are deliberate. The
  non-enabled code is dropped; S110 and both comments stay.
- scripts/tests/test_check_doc_narrative.py was not ruff-formatted.

Verified with the shipped invocation: 35 files, all checks passed, all formatted;
suite 807 passed / 2 skipped.

refs #780, refs #784

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-22 09:34:03 +00:00
timothyandtimothy d4c72697f2 feat(780): commit a ruff config and enforce it in CI (#813)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 28s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m29s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 5m54s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m59s
Build ErsatzTV Image / Build & push image (amd64) (push) Failing after 15s
Python lint here was a property of the operator's laptop: the global instructions
say to run ruff, no workflow ran it, and with no committed config ruff fell back
to whichever ~/.config/ruff/ruff.toml the machine happened to have.

- ruff.toml at the root, pinned ruff==0.12.11 in the script-tests job.
- Both lint steps pass an EXPLICIT population from `git ls-files` with
  `--no-force-exclude`, never `ruff check .` — an `exclude` empties a
  discovery-based run into a GREEN one (top level empties both commands, [lint]
  empties check, [format] empties format --check), and `ruff check .` over zero
  files exits 0 with only a stderr warning. Guarded by an empty-population arm.
- Tree clean: 74 findings at 706674272, 57 fixed in code, 17 per-site noqa with
  reasons inline. S105 deliberately per-site, not a directory blanket. RUF100
  selected so a suppression that suppresses nothing is itself a finding.
- pyright stays ungated; reasoning in the record.

Both steps witnessed red on the runner against the shipped bodies: run 2173 job
9176 (ruff check) and run 2170 job 9163 (ruff format --check).

Docs: new record ci.python-lint-ruff-config-committed, ci.script-tests-job
cross-ref, docs/ci-cd.md (also correcting a stale ~190-tests/~10s figure to the
measured 773 tests / ~4.5 min), docs/defect-shapes-773.md §5.2 resolved.

fixes #780

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-22 00:33:18 +00:00
timothyandtimothy 6a4265d81d feat(784): a doc records the end state — generalize the no-session-narrative rule (#811)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 40s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m30s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m16s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 5m52s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Failing after 17s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-22 00:23:22 +00:00
706674272c test(785): mutation proofs for the three unproven ranked guards (#810)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m48s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m14s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m17s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m10s
Closes the remaining three entries on #785's ranked list with clause-level mutation proofs, each
witnessed red against the real subject in place:

* the `pretooluse-worktree-guard.sh` + `posttooluse-worktree-marker.sh` PAIR — four clauses,
  including the cross-file seam (a clause in the marker hook, asserted against the guard's
  decision) that could not exist while the halves were tested apart;
* `.husky/pre-push:11`'s `unset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE` — git exports `GIT_DIR` to
  `pre-push` only from a worktree, which `process.shared-tree-readonly` makes the mandated way to
  work here, so the guarded case is the normal one;
* `scripts/build_decisions_catalog.py --check` — including the `__main__` wiring, which can print
  "is stale" on stderr and still exit 0.

Nine ways the catalog guard can stop gating are detected, judged by executing the step's whole
`run` script rather than by matching lines out of it. Two channels are undecidable outside the
runner and are stated as uncovered rather than guessed at.

Inventory regraded to 12 MUTATION / 6 BEHAVIOUR-ONLY / 16 NONE, with a stated reason for every
remaining NONE row, verified member-for-member against the derived set.

Five cold review rounds; findings closed include production-hook-fire-log corruption, a
tautological assertion, a guard asserting on its helper rather than on the effect, and two false
greens in the workflow extractor. Follow-up: #809.

fixes #785

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-21 20:10:01 +00:00
timothyandtimothy 649acd12ee feat(779): test the deny path at the production config value, and derive full-replace field lists (#808)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 8m52s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m5s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m11s
Two conventions from #773's detector menu (F and G), each as a decision record plus a filled gap.

Part 1 — the deny path at the production config value. Every assertion about the API read-gating posture ran through a hand-written fake HANDED the boolean, and no test in the repo constructed ApiKeyProvider at all, so the line deriving that posture from configuration had never executed. Now covered across the matrix through the real provider: absent, true/True/TRUE, false/False, and a present-but-non-boolean value (which throws at startup — fail-closed, pinned).

Part 2 — a full-replace path asserts its complete field list. ScheduleItemResponseRoundTripTests is the release gate for the flat schedule-item DTO, and its comparer was itself a hand-copied field list: complete when written, unable to report when it stopped being. Now derived by reflection with an empty exemption set and a written-down count pin (55).

Four cold adversarial review rounds. Three returned BLOCKED, every one on a claim in a decision record that the code contradicted — the exact defect the records exist to name. The surviving rule, now written into the record: state the measurement and the code path you actually read; do not generalise from one executed case, and do not explain a mechanism you did not measure.

Residual SPA optional-field drift tracked as #807.

fixes #779

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-21 19:22:30 +00:00
timothy ad0bed4585 feat(778): name the check-and-use race, and verify the protection the merge gate rests on (#802)
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 16s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 19s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 13s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 12s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m50s
2026-08-21 17:04:13 +00:00
timothyandClaude Opus 5 7347a08979 fix(778): the 200-not-an-array arm had the same lying reason, one branch earlier
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 31s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 43s
PR Gates / Docs update reminder (pull_request) Successful in 48s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 12s
PR Gates / decisions lifecycle (pull_request) Successful in 55s
Review verdict / Set review-verdict status (pull_request_target) Successful in 16s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m1s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m59s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m17s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m51s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
review-verdict/h10 Review-verdict: MERGEABLE @ 7347a08 (base: main)
Local review returned MERGEABLE — no Blocker, High or Medium, both sentinels verified
pinned by mutation, no regressions. Its one Low is taken rather than deferred, because
it is a one-line fix and because deferring it would leave exactly the shape this PR
exists to document.

A 200 whose body is NOT an array never reaches the classifier: the array gate diverts
it, `bp_code` stays 200, and the generic ask then reported "HTTP '200' — Gitea
unreachable, or these credentials lack the repo-admin scope" about a read that plainly
succeeded. That is the identical defect the previous commit fixed for the
throw-inside-the-classifier arm, one branch earlier — fixed where it was noticed, left
in its twin.

The previous commit's message even generalised the pattern ("a sentinel that doubles as
an HTTP code makes a decision state a cause that did not happen") while covering only
one of the two arms it applies to. The test is now parametrised over all three shapes
that reach an unusable 200 — UNPARSEABLE-RULES, GARBAGE, EMPTY — and reverting the new
sentinel reddens the two that the first fix missed.

Also finishes the de-indent the previous commit claimed: that comment block went from
19 leading spaces to 6 while its siblings use 2, so the claim was true of the direction
and not of the result.

729 tests green.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 22:50:23 +02:00
timothyandClaude Opus 5 3f0af38509 fix(778): pin the nomatch sentinel — it shipped unpinned, exactly like the guard before it
Local review of the previous commit returned BLOCKED. Both findings taken; nothing
pushed to CI while this was iterating.

MEDIUM — the `nomatch` fix was entirely unpinned. Reverting both sites to `bp_code=404`
left the suite 33/33 green, because no fixture ever emitted an HTTP 404 on the LIST
read: the codes exercised were 000, 403, 500 and 200, and the old `NOT-FOUND` mode had
been repurposed to return 200 with `[]`. So the defect that commit describes could be
reintroduced silently — in a PR whose subject is unfalsifiable guards, one round after
being blocked for precisely that shape. There is now a `LIST-404` fixture and a test;
reverting the sentinel reddens two tests.

The same class, one arm over and found while fixing it: a 200 whose `branch_name` is a
number makes the classifier throw (`//` fires only on null/false), and that was mapped
to `bp_code=000`, reporting "HTTP '000' — Gitea unreachable" about a read that plainly
succeeded. It gets its own `unreadable-rules` sentinel and message, with a fixture and
a test — reverting it reddens.

The pattern across both: a sentinel that doubles as an HTTP code makes a decision state
a cause that did not happen. The decision was safe each time; only the reason lied.

MEDIUM — four comments still described 404-as-a-finding as live, contradicting the hook
comment added in the same commit. The worst said a 404 means "this branch is entirely
unprotected" in a test whose fixture now returns 200 with `[]`, which would have talked
the next reader into re-adding the deny. Renamed that mode `EMPTY-LIST` so it says what
it is.

Also: the hook quoted a reason string it no longer emits; a stray over-indented line
survived the de-indent; `bp_called` was write-only after its reader was removed.

727 tests green.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 22:09:16 +02:00
timothyandClaude Opus 5 7d81f85b38 fix(778): make the by-name pin falsifiable — it was filtering on the property it asserted
Confirmation pass returned BLOCKED, and its lead finding is the one worth having.

The test forbidding the by-name lookup recorded URLs from INSIDE the
`endswith("/branch_protections")` branch, so the only URLs it could ever record were
ones that already satisfied the assertion. A by-name request was invisible to the very
test written to forbid it. Cold review proved it by reintroducing the lookup in the
hook: the suite stayed 33/33 green. That is the filter-on-the-asserted-property defect
this PR's sibling record exists to describe, committed inside the guard against it —
and the commit message had called the twin "pinned so it cannot come back".

The recorder now sees every branch-protection URL whatever its shape. Re-verified by
the same mutation: reintroducing a by-name call reddens exactly the two tests that
forbid it.

Also from that pass:
- an HTTP 404 on the LIST read reached the "the full rule list was read and none
  matches" deny — a claim about a read that never happened. Gitea answers 404 for a
  repo that is absent or invisible to the credential, so the classifier's own verdict
  is now the sentinel `nomatch` and HTTP failures reach the ask;
- two comment blocks still described the deleted by-name endpoint as live, one of them
  asserting the classifier "is never reached at all";
- the decision record still documented `branch_protections/{base}` and its 404
  semantics as the mechanism, in the record this PR authored — now rewritten to the
  list endpoint, with why reading the LIST is the load-bearing choice;
- seven assertions on a string the hook no longer emits, and three test
  names/docstrings describing the removed 404 flow;
- an unused fixture helper, and 79 lines left over-indented by the removed nesting.

724 tests green.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 21:29:09 +02:00
timothyandClaude Opus 5 f3863c3720 fix(778): delete the by-name lookup — the precedence check was guarding the dead path
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 20s
PR Gates / Docs update reminder (pull_request) Successful in 19s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 18s
review-verdict/h10 Awaiting review verdict for f3863c3
Review verdict / Set review-verdict status (pull_request_target) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 21s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m21s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m11s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m20s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m2s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Final review returned MERGEABLE with no Blocker and no High. Its one Medium is taken,
and it is my own recurring trap for the third time in this PR: fix one path, then check
its TWIN.

The hook looked a rule up by NAME first and enumerated the rule list only on a 404.
But `branch_protections/{name}` is an exact DB lookup — `GetProtectedBranchRuleByName`
— which performs no matching and knows nothing about precedence. A 200 from it means
"a rule with this NAME exists and lists this context", never "this context is required
on this branch". So the precedence argument added last round guarded the 404 path while
the 200 path granted without it — and since this repo's rule IS named `main`, the by-name
lookup always returns 200. The hardened code was dead and the unhardened code was live.

Given a rule `main` requiring review-verdict/h10 and a rule `m*` with better Priority
that does not, Gitea applies `m*`; the by-name hit on `main` saw h10 and granted anyway.

Fixed by DELETING the twin rather than documenting it: one fetch of the full list, one
classifier, one argument, no second path to keep in step. Two things fall out for free
— the ref no longer reaches a URL segment, so the percent-encoding hazard is removed by
construction rather than escaped (its test is replaced by one asserting no ref reaches
the URL at all), and every case the classifier already covered now applies to the live
path instead of an unreachable one.

Verified against the live Gitea: the classifier returns `exact` -> rule `main` ->
enable_status_check=true, h10 present. A new test pins that the precedence check runs
even when an exactly-named rule exists, and asserts the by-name endpoint is never
requested, so the split cannot come back silently.

The grant string now states what was actually established — read from the full rule
list, matched with Gitea's own plain-vs-glob split, refusing wherever precedence or
folding is not derivable — rather than the stronger "confirmed required" it claimed
while consulting a single named rule.

724 tests green.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 21:07:09 +02:00
timothyandClaude Opus 5 c7bea972d6 fix(778): close the last three residuals — ambiguous exact rules, non-ASCII folds, one unpinned test
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 9s
PR Gates / Docs update reminder (pull_request) Successful in 15s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 21s
PR Gates / decisions lifecycle (pull_request) Successful in 27s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
review-verdict/h10 Awaiting review verdict for c7bea97
Review verdict / Set review-verdict status (pull_request_target) Successful in 20s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m4s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m15s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m41s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
Confirmation review returned MERGEABLE at 356cddbb5, having independently re-executed
every claim (the "2 -> 6 dead-classifier" figure exact, the "20 tests" trap figure
exactly reproducible, the arm reorder pinned, live branch protection confirming the
dated claim). Its three residuals are taken here.

M1 — the backslash test hit the undecidable arm but asserted only `"ask" in reason`,
which a CRASHED classifier also satisfies. Its two siblings got `"could govern it"`
one commit earlier and this one did not. Under the dead-classifier mutation it was the
single green test of that arm; now red.

L1 — with two plain rules differing only in case, `first` picked LIST order while
Gitea picks by Priority. Given `MAIN` requiring review-verdict/h10 and `main` not, the
hook could inspect the rule that requires it and auto-grant on a base where Gitea
enforces the other — the same defect as the arm order, one level down. Two fold-equal
rules are now undecidable.

L2 — the same standard, applied where I had waived it. The backslash paragraph rejects
"nearly unreachable" as a standard for the arm that issues a DENY, and two paragraphs
up the ASCII-only fold was accepted on exactly those grounds: rule `ünstable` and base
`Ünstable` fold equal under Gitea's EqualFold and not under `ascii_downcase`, landing
on `none` -> deny with a false stated cause. A non-ASCII rule name or base is now
undecidable rather than fold-compared.

The non-ASCII test is `explode | any(. > 127)`, not a `\uXXXX` regex: the first
attempt was a character-class regex whose backslashes are ambiguous through a
single-quoted shell string into jq, and a standalone probe showed it classifying plain
`main` as non-ASCII. Codepoints have no escaping layer to get wrong.

Both new arms are mutation-proven individually; an earlier attempt at those proofs used
perl substitutions that silently matched nothing and reported green, which is why they
were redone in python with an explicit assert on the target.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 20:23:32 +02:00
timothyandClaude Opus 5 356cddbb59 fix(778): pin the arm order, and correct a wrong explanation I had just written
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 18s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 27s
PR Gates / Docs update reminder (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 20s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 18s
review-verdict/h10 Awaiting review verdict for 356cddb
Review verdict / Set review-verdict status (pull_request_target) Successful in 44s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m3s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m12s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m1s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
Two self-found defects while pre-empting the confirmation round's own questions.

THE REORDER WAS UNPINNED. Swapping the classifier arms back to exact-first left all 29
tests green, so the previous commit's central change was invisible to the suite — an
unproven change shipping under a green run. The missing fixture is the one that
distinguishes the orders: a list holding BOTH an exactly-named rule that requires
`review-verdict/h10` AND a glob rule that could also govern the base and does not.
Exact-first inspects the rule that requires h10 and auto-grants; undecidable-first
asks. Now mutation-proven in both directions.

I HAD MISDIAGNOSED THE TRAP, and asserted the wrong cause in a comment. A three-line
repro disproves "an EXIT trap suppresses output" — it does not. The real mechanism is
that this file already owns its EXIT trap: `scripts/hook-fire-log.sh` installs
`trap 'etv_hook_fire_end "$?"' EXIT` (#776), and in capture mode that handler is what
REPLAYS the decision JSON to stdout. A second `trap ... EXIT` silently replaces it, so
the decision is captured and never emitted. The trap slot is a single shared resource
and the sourced library claimed it first.

That second one is the shape this whole PR is about, committed against my own work: an
explanation that fits the symptom, is written down as fact, and is wrong. It would have
told the next person the construct is unusable rather than that the slot is taken.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:54:47 +02:00
timothyandClaude Opus 5 d8ff6fa229 fix(778): ask before trusting an exact rule, and make the tests tell classified from crashed
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 22s
PR Gates / decisions lifecycle (pull_request) Successful in 22s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
review-verdict/h10 Awaiting review verdict for d8ff6fa
Review verdict / Set review-verdict status (pull_request_target) Successful in 14s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m55s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 7m39s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Ninth cold review: MERGEABLE, no Blocker, no High. Its three Mediums taken anyway,
because each was a one-line fix retiring the last "asserted rather than verified"
surface in the file whose whole subject is that shape.

M1 (backslash missing from the metacharacter class) was already closed in cd1b28637 —
found independently while stress-testing the superset claim, in the window the reviewer
was working against the previous head.

M2 — the `exact` arm was not decidable. Gitea picks the governing rule with
GetFirstMatched over a list sorted by Priority and THEN by plain-name-ness, so a glob
rule with a better Priority outranks an exactly-named one. Preferring `exact` would
inspect a rule Gitea might not be applying: if the exact rule requires
`review-verdict/h10` and a higher-priority glob rule does not, the gate auto-grants on
a base where the check is not enforced. `undecidable` is now evaluated FIRST, which
makes the classifier sound without knowing the precedence rules at all — the only
claim this code is entitled to make about somebody else's resolver.

M3 — four tests could not distinguish "classified correctly" from "classifier
crashed", because a dead classifier lands on the generic could-not-read ask and they
asserted only `"ask" in reason`. Measured with the reviewer's method rather than
argued: injecting `error(...)` at the head of the jq program left 2 of them red; the
strengthened assertions leave 6. The glob arms now pin the text unique to the
undecidable ask, and the decidable arms assert the rule was HONOURED rather than
referred to a human.

Also: the module docstring still described a three-arm contract after this change added
a fourth; the inventory row still summarised the old two-way behaviour; the ASCII-only
case fold is now stated as a deliberate under-match rather than as parity with
EqualFold; the live-config claim is dated; and one jq call rejoined the file's
`|| true` discipline.

A `trap ... EXIT` for temp-file cleanup was tried and REVERTED: it suppressed the
hook's decision output entirely and took 20 tests red. A gate that prints nothing is
the one outcome it must never produce, so the tidier construct loses to the one that
works, with the reason recorded where the next person will try it.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:47:06 +02:00
timothyandClaude Opus 5 cd1b286378 fix(778): backslash is a metacharacter too, or the none arm's proof does not hold
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 17s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 14s
review-verdict/h10 Awaiting review verdict for cd1b286
Review verdict / Set review-verdict status (pull_request_target) Successful in 16s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m12s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m56s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m18s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m49s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 9s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Self-found while stress-testing the superset claim I introduced one commit earlier —
the crux the new classifier's safety rests on.

`none` authorises a DENY on the stated grounds that nothing can possibly govern this
base, so its premise must hold unconditionally, not usually. The superset was "literal
prefix + .* + literal suffix", which is sound for every glob dialect EXCEPT one case:
gobwas/glob reads `\{` as a LITERAL brace, so a rule `a\{b` governs the base `a{b`,
while a superset treating `\` as an ordinary character builds `a\.*b`, misses, and
denies a base that is in fact protected.

Verified before and after: with `\` outside the metacharacter class the classifier
answered `none` for that pair; with it inside, `undecidable` -> ask. 18 adversarial
rule/base pairs (brace alternation, negated and ranged classes, `**`, leading and
trailing metacharacters, unicode, empty alternation) all answer `undecidable`, never
`none`, so no dialect-matching case falls through the deny arm.

Git ref rules make this nearly unreachable — a branch name may not contain `*`, `?`,
`[` or `\` — but `{` IS legal in one, and "nearly unreachable" is not the standard for
the arm that issues a deny. Checked with `git check-ref-format` rather than assumed.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:59:36 +02:00
timothyandClaude Opus 5 456042ee51 fix(778): stop claiming parity with Gitea's glob matcher — classify, or ask
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 27s
PR Gates / Docs update reminder (pull_request) Successful in 29s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 23s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / decisions lifecycle (pull_request) Successful in 28s
review-verdict/h10 Awaiting review verdict for 456042e
Review verdict / Set review-verdict status (pull_request_target) Successful in 15s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m56s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 7m16s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Eighth cold review: no Blocker, no High. Three Medium, two Low, one Nit.

MEDIUM — the substantive one. The glob fallback asserted it matched rules "the same
way Gitea applies them", and it does not. Gitea compiles a rule name with gobwas/glob
and a `/` separator: its `*` does NOT cross a slash, `?`/`[…]`/`{a,b}` are wildcards,
and a plain name is folded case-insensitively. Mine used `.*` for `*` and escaped the
rest. The divergence has a false-OPEN direction — `release/*` does not govern
`release/26/hotfix` in Gitea, but `release/.*` matched it here, which would auto-grant
a scheduled merge on a base where the check is not required. That is #622's hole,
reached through the block written to close it, via exactly the failure this PR
records: a claim about an external system asserted rather than verified.

Reimplementing somebody else's glob dialect would be a second copy of a parser, which
this repo has already withdrawn a change for. So the classification is three-way and
each arm is safe WITHOUT knowing the dialect: an exact non-glob name folded
case-insensitively is decidable; a glob rule that could govern the base is
UNDECIDABLE and asks; and "could" is tested with a provable superset of any glob
dialect — literal prefix, `.*`, literal suffix — so if even that cannot match, no
dialect can. Over-matching would grant on unestablished protection; under-matching
would deny with a false cause. Asking is the only answer honest in both directions,
and it is rare: this repo's rule is the plain name `main`.

MEDIUM — a count that was wrong the moment it was written ("46 of the 69 rows are
N/A"; it is 44). It was added by the same commit that demoted two rows. That is the
FOURTH stale number in this change, in the deliverable whose own record argues against
hand-maintained counts. Removed rather than corrected, with the reason stated.

MEDIUM — `migration-smoke.sh` still said "Same shape" as `security-scan.sh`, whose
note had just been rewritten to the opposite conclusion, so the backreference had
silently inverted. It is the same pull-then-run over a mutable tag and deserves the
grade MORE, not less: `security-scan.sh` boots a throwaway container and authorizes
nothing, while this gates a production stack recreation. Regraded.

LOW/NIT: an `end <= start` guard that `str.index(…, start)` makes unreachable by
construction is replaced with the reachable failure it was describing; a docstring
still named a fixture from an earlier draft; a reflowed `#707.` was line-initial.

Two mutations were ineffective on the first attempt — one left the `decide ask`
continuation in place, the other had no test covering case-folding. Both redone; each
arm now reddens a named test.

Decisions-Edit: yes

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:52:18 +02:00
timothyandClaude Opus 5 7db4101b24 fix(778): the escape emitted TWO backslashes, turning a false-open into a false DENY
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / Docs update reminder (pull_request) Successful in 10s
PR Gates / decisions lifecycle (pull_request) Successful in 24s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 25s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 21s
review-verdict/h10 Awaiting review verdict for 7db4101
Review verdict / Set review-verdict status (pull_request_target) Successful in 28s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m56s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m37s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m14s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m8s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
Seventh cold review. One High, one Medium, three Low, seven Nit — all in the two
newest commits, which is where every round of this PR has found its defects.

HIGH, and it is my own fix from the previous commit. In jq source `"\\\\"` decodes to
TWO backslashes, so escaping produced `\\.` — "a literal backslash, then any
character" — instead of an escaped dot. Every rule name containing a metacharacter
became UNMATCHABLE, and a rule named `a[b` crashed jq outright (swallowed by
`|| true`). Verified: `release/26.*` no longer matched base `release/26.4`, so the
fallback found nothing and hard-DENIED with the stated cause "has NO branch protection
at all" — converting a false-open into a false deny, which the block's own comment
calls the worse outcome. One character: `"\\" + .c`. Correct across 14 rule/base pairs.

WHY MY TEST MISSED IT, which is the transferable part: it asserted only the NEGATIVE
direction (`mai.` must not match `main`). A rule matched literally and a rule made
unmatchable both fail to match the wrong base, so the assertion passed for the wrong
reason. Only a rule that SHOULD match separates them, and there was no positive
control. There is now — plus a char-class case — and both go red against the
over-escaped version. That also needed a base containing a dot: a rule cannot carry a
metacharacter and still match `main`, so the first attempt at the positive control was
unsatisfiable by construction.

MEDIUM — four live claims that the population "derives from the filesystem", left
standing by the commit that replaced that mechanism: the guard's own docstring 45
lines above a comment shouting the opposite, the inventory heading 21 lines under
"Every git-tracked file", the docs/README entry, and — worst — the record's
`mechanics:` frontmatter, which is the copy the catalog and MemPalace mirror, so
discovery would have returned the superseded lesson. All corrected.

LOW/NIT: the URL-encoding test grepped the source for `@uri` (it now asserts the URL
actually requested, and reddens when the encoding is removed); the hoist comment said
"every path below" without noting the docs-only enumeration above it (bounded — that
path is a passthrough to a human prompt, never a grant); a now-unreachable guard is
annotated rather than left reading as live; `issue-qualification-audit.sh` was `N/A`
while `select-queue.sh` was `UNSAFE-KNOWN` on the same argument, and
`security-scan.sh` claimed "one step" for a pull-then-run over a mutable tag — both
regraded; the `PINNED` definition now says what separates its second shape from an
`N/A` "one step" row (the identifier's immutability, not the step count); the section
parser raises a message naming both required headings instead of a bare ValueError;
and the record's body is rewrapped.

Decisions-Edit: yes

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:21:04 +02:00
timothyandClaude Opus 5 01de6cbd8b fix(778): * is the only wildcard — a rule name's regex metacharacters were live
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 10s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 21s
PR Gates / Docs update reminder (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 24s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
review-verdict/h10 Awaiting review verdict for 01de6cb
Review verdict / Set review-verdict status (pull_request_target) Successful in 32s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m51s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m36s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m11s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 4m27s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Found by attacking my own glob fallback from the previous commit before the reviewer
got to it, which is the round where this PR's defects have landed every time.

The 404 fallback matched a branch-protection rule by substituting `*` into a raw
regex, leaving every other metacharacter active. Verified directly: a rule named
`main.x` matched the base `mainax`, and `a+b` matched `aab`. The direction is the one
that matters — a spurious match to some OTHER rule that happens to require
`review-verdict/h10` reports this base as protected when nothing governs it, so a
consent gate answers yes on evidence about a different branch.

Each literal segment is now escaped before the pattern is assembled, so the wildcard
survives and nothing else does. Verified across 11 rule/base pairs: metacharacters are
literal, `*` still spans, exact and non-matches unaffected.

The regression test needed two goes to stop being vacuous, both times for the same
reason the rest of this PR keeps hitting: the stub never 404'd for the new mode, so
the run denied earlier via the by-name lookup and never reached the fallback at all. It
now goes red against the unescaped predecessor.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 18:01:42 +02:00
timothyandClaude Opus 5 e5996b7c1e docs(778): the population was wrong three times — record what finally fixed it
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 19s
PR Gates / Docs update reminder (pull_request) Successful in 12s
PR Gates / decisions lifecycle (pull_request) Successful in 22s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 12s
review-verdict/h10 Awaiting review verdict for e5996b7
Review verdict / Set review-verdict status (pull_request_target) Successful in 16s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m47s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 7m29s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
The record said "wrong twice" and drew the lesson "execute the traversal and eyeball
it". The third instance (rglob picking up untracked `.husky/_` shims) shows that was
still the wrong generalisation: every round had executed its traversal, and every round
had an argument for why it was sufficient.

What held was changing the SOURCE, not the walk — `git ls-files` instead of the
filesystem. The index is authoritative, identical for CI and every checkout, and
excludes untracked build output by construction. So the lesson is the one
`testing.guard-derives-population-from-source` already states, one level up: ask what
the authoritative list of these things IS, and if the answer is "whatever the walk
finds", the guard is not finished however carefully the walk is written.

Decisions-Edit: yes

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 17:54:00 +02:00
timothyandClaude Opus 5 4261f76dd2 fix(778): derive the population from git, not the disk — the guard was red on every dev checkout
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 12s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 22s
PR Gates / Docs update reminder (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 24s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 10s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Canceled after 45s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
review-verdict/h10 Awaiting review verdict for 4261f76
PR Gates / Script tests (pytest) (pull_request) Canceled after 19s
Review verdict / Set review-verdict status (pull_request_target) Successful in 20s
Sixth cold review (a different reviewer, in-repo, worktree-isolated after the
cross-family runs wedged twice on their sandbox). One High, one Medium, two Low, two
Nit. All fixed.

HIGH, and it is the third time this population has been wrong. `rglob` is recursive,
so it also enumerated `.husky/_/` — 17 husky shims generated by `npm ci` via
web/package.json's `prepare`, gitignored and untracked. The guard therefore derived 76
files against a 59-row table and was RED on every checkout that has run `npm ci`,
while staying GREEN in CI, whose `script-tests` job checks out and pip-installs but
never runs `npm ci`. A guard that fails everywhere except where it runs is the fastest
possible route to "that test is always broken, ignore it" — on the artifact whose
entire thesis is population correctness. Reproduced, then fixed at the source rather
than with a fourth traversal patch: the population now comes from `git ls-files`. The
index is authoritative, identical for CI and every checkout, and excludes untracked
build output by construction instead of by an exclusion list someone must maintain.
That is what this PR's own record says to do; the first three attempts each derived
from whatever happened to be on disk. Three tests go red against the rglob
predecessor.

MEDIUM — twin-missed, in the fix from the previous round. Round 4 re-read the base
before the branch-protection lookup, inside the scheduled branch only, leaving the
#632 retarget DETECTION still reading the top-of-hook snapshot. The reviewer
demonstrated it with this PR's own fixture: scheduled+retarget denied while
immediate+retarget AUTO-GRANTED. The re-read is now hoisted above every base-dependent
consumer, so one read serves both paths, and the duplicate is gone. Note for the
record: the hoist is the load-bearing part — once `live_base` is fresh, #632's own
comparison catches the retarget too, so the explicit deny only bites when no verdict
records a base. The tests are scoped to exactly that case, because as first written
they passed under mutation.

LOW — a 404 from `branch_protections/<ref>` does not prove the branch is unprotected.
Gitea keys that endpoint on the RULE name, so a base covered by a glob rule 404s while
being fully protected, and an unencoded ref containing `/` (`release/26.4`) 404s
because the path is malformed. Both produced a hard deny stating a specific, false
cause — and a deny blocks outright rather than prompting. The ref is percent-encoded,
and a 404 now consults the rule list before denying; an unreadable list asks.

LOW/NIT — the scope prose attached the extension restriction to `scripts/` alone while
the guard applied it everywhere (a `.py` hook would have joined the described scope and
acquired no row); `.yaml` workflows are now in scope too. The `PINNED` definition
required re-validation, which two legitimately-pinned rows do not do because their
check and use are one step over an immutable event-payload sha. Row ordering restored.

And once more, the recurring one: adding a scope TABLE to the doc made three prose
rows parse as inventory sites — the parser reading its own documentation as data, the
same defect as the UNSAFE-KNOWN check that once parsed the paragraph defining
UNSAFE-KNOWN. Row parsing is now bounded to the inventory section explicitly.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 17:53:25 +02:00
timothyandClaude Opus 5 3322d68948 docs(778): state that $sha is stale mid-run too, and track it in #803
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 29s
PR Gates / Docs update reminder (pull_request) Successful in 46s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 47s
PR Gates / decisions lifecycle (pull_request) Successful in 54s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 12s
review-verdict/h10 Awaiting review verdict for 3322d68
Review verdict / Set review-verdict status (pull_request_target) Successful in 10s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m39s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m32s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m35s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 5m59s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Found by applying round 5's own finding symmetrically. `$base_ref` was re-read before
the branch-protection lookup because using a value captured at hook start is not
checking. `$sha` is captured from the same snapshot and is never re-read, so every
later check — CI status, H10 status, verdict comments — evaluates against the commit a
mid-run push replaced.

Not fixed here: the base case was inside the code this PR introduced, while the sha
spans the pre-existing H10 logic, and opening that at round five of review is how a
scoped change stops being reviewable. The row now names both staleness windows (within
the run, and after the decision) instead of only the second, and #803 carries the fix.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 17:28:46 +02:00
timothyandClaude Opus 5 f1e9cce4ae fix(778): round 4 — the guard was checking a stale base, which is the defect it enforces against
PR Gates / decisions lifecycle (pull_request) Successful in 19s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 26s
PR Gates / Docs update reminder (pull_request) Successful in 23s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 21s
review-verdict/h10 Awaiting review verdict for f1e9cce
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
Review verdict / Set review-verdict status (pull_request_target) Successful in 36s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m44s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m27s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m48s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m34s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Fifth cold review: no Blockers, no Highs. 1 Medium, 1 Low, both fixed here. It also
confirmed the #803 deferral is sound and re-verified every count (59 files, 69 rows,
3 PINNED; guards 34/5/15, proofs 8/6/20).

MEDIUM — and it is the one worth the round. The branch-protection lookup used
`$base_ref` from the PR snapshot taken at the TOP of the hook, and everything between
is round trips (the file enumeration alone can be forty pages). A PERSISTENT retarget
in that gap needs no ABA and no force-push: the lookup names the OLD base, confirms
`review-verdict/h10` on a branch the PR no longer targets, and grants a scheduled
merge onto one that may require nothing. The guard written to enforce
"checking a stale identifier is not checking" was doing exactly that. The base is now
re-read and compared immediately before the lookup; a move denies and names both
branches. Mutation-proven.

LOW — my caveat erred in the rare direction, understating a clause instead of
overstating it. The scalar-row test's docstring called the whole `.statuses` member
validation defence-in-depth because the #632 block masks it. That block validates
`.context` and `.description` but NOT `.status`, so an object row with a numeric
status passes it and does reach the new validator — where without the clause it
becomes `vstate=7` and is reported as "the verdict is '7'" rather than as an
unreadable payload. The caveat is now scoped to the payload rather than the clause,
and the reachable case has its own test, also mutation-proven.

The inventory row now conditions the guarantee on BOTH of its preconditions — the PR
still targeting that base (fixed here) and the protection still standing (cannot be
closed here, and said so).

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 15:31:49 +02:00
timothyandClaude Opus 5 8e02b9961d fix(778): round 3 — fix the overclaim I left in the code, and defer the pre-existing ones to #803
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 18s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 19s
PR Gates / Docs update reminder (pull_request) Successful in 22s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 14s
PR Gates / decisions lifecycle (pull_request) Successful in 26s
review-verdict/h10 Awaiting review verdict for 8e02b99
Review verdict / Set review-verdict status (pull_request_target) Successful in 17s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m9s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m57s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m53s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
Fourth cold review: no Blockers, 2 High / 2 Medium / 2 Low. It independently
re-derived the population (59 files, 59 sites, 69 rows, 3 PINNED) and verified every
numeric and factual claim in the diff, including the corrected confinement rationale.

THE ONE THAT STINGS. The grant reason string still said a commit pushed before Gitea
merges "will clear it and block the merge" — the exact sentence the new decision
record quotes as THE overclaim this issue exists to remove. I documented it in three
files and left it in the code a human actually reads. It now states the guarantee and
its condition: the required check was confirmed rather than assumed, and it holds
while that branch protection stands.

FIXED HERE (all in files this PR already touches):
- enable_status_check is validated as a BOOLEAN. `"true"` is not `true`, and comparing
  the string to `true` produced a confident deny from a payload never understood —
  the tri-state collapsing to two, the same defect as the contexts shape one line down.
- `.statuses` members are validated, not just the array (see the honest caveat below).
- the docs-reminder N/A rationale said "the job cannot fail and never reaches the
  combined status", which is false — any job's status joins the combined state. The
  true, narrower reason is that its fetch and diff are failure-swallowed, so the
  remote read can only change the warning's wording.
- docs/README names the scripts/tests exclusion in BOTH statements.

A VACUOUS TEST, CAUGHT BY ITS OWN MUTATION PROOF. The regression case for the
`.statuses` member validation stays GREEN against the predecessor: the #632
base-retarget block runs first and already validates every member it consumes, so it
catches the payload before the scheduled branch is reached. The two guards overlap —
duplicate guards masking each other, again — which makes that finding LATENT, not
live, and my added clause defence-in-depth rather than a fix. The test now asserts the
observable contract (a decision is always emitted) and says plainly that it is not a
mutation proof of the newer clause. Shipping it as one would have been the exact
grade inflation round 2 rejected.

DEFERRED to #803, with the reason stated rather than implied: a head-ABA
(force-push H1 -> H2 -> H1 during pagination) defeats pr-changed-files.sh, and three
OLDER contracts still assert more than the new inventory rows do. That residual
predates #778 and lives in #707's mechanism; correcting another active decision
record inside a PR already at four review rounds is how a scoped change stops being
reviewable. The inventory rows are accurate today and now point at #803.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 14:15:24 +02:00
timothyandClaude Opus 5 25048d9ea4 fix(778): round 3 — the fence covers BASE aliasing only, and stop citing a moving count
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 20s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 25s
PR Gates / Docs update reminder (pull_request) Successful in 21s
review-verdict/h10 Awaiting review verdict for 25048d9
Review verdict / Set review-verdict status (pull_request_target) Successful in 19s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m1s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m33s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m43s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 22s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Third cold review: no Blockers, 1 High / 5 Medium / 1 Low / 1 Nit. All accepted.
It independently re-derived the 59-file population and matched it against `find`,
so the traversal that was wrong in rounds 1 and 2 is now verified rather than argued.

HIGH — the ABA claim was too broad. `ci.verdict-write-retarget-fence` counts
`change_target_branch` events, so it fences the BASE axis and nothing else. A
force-push H1 -> H2 -> H1 during pagination leaves the final `.head.sha` comparison
equal while the middle pages were enumerated against H2, and no counter moves. Two
rows implied the fence covered that; both now state the head residual as real and
unfenced, with what closing it would take.

Also: the record still said the scheduled-merge residual was "closed one layer down"
by the branch protection an admin may have removed — the circular sentence that was
rewritten in the inventory last round and left standing in its twin. The hook header
still called an immediate merge "sound". Both now describe the bounded window.

`docs-reminder` was over-demoted by grouping it with `decisions-guard`: it cannot
fail its job, so it authorizes nothing and is N/A, while `decisions-guard` reaches
the combined status. Split, per this file's own rule that differing classifications
get separate rows. Over-demotion is a defect too — it makes the column noise.

The scope heading and the docs/README entry now name the `scripts/tests/` exclusion
explicitly, so nobody adds a remote-reading test executable expecting a red guard
that stays green, and a wrong limit cross-reference is corrected. The exclusion's
justification was also factually false — it claimed the only network calls were to
PATH stubs, but test_hook_fire_log.py starts a real http.server on 127.0.0.1 and
drives it with real curl. The exclusion stands on confinement, not on absence, and
now says so.

COUNTS. "4 of 68 PINNED" was wrong (3), and rewriting it as "3 of 68" went stale in
the same commit when splitting a row moved the denominator to 69 — three stale
figures in three rounds, in the record warning against exactly this. The exact
denominator is gone: three rows survive as PINNED out of roughly seventy, and the
load-bearing claim is "almost nothing is pinned", not an integer. A hand-maintained
count is a second copy of the table; guard-inventory.md gives its counts an equality
check because they ARE the point, and a rationale record should not pretend to.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 13:58:40 +02:00
timothyandClaude Opus 5 84b4c11188 fix(778): round 2 — recursive population, and grade down five more overclaims
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 22s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 14s
PR Gates / Docs update reminder (pull_request) Successful in 23s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 9s
PR Gates / decisions lifecycle (pull_request) Successful in 24s
review-verdict/h10 Awaiting review verdict for 84b4c11
Review verdict / Set review-verdict status (pull_request_target) Successful in 22s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m10s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m22s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 5m4s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Second cold review: no Blockers, 4 High / 1 Medium / 2 Low. All accepted.

POPULATION, WRONG A SECOND TIME. Round 1 removed a content filter that had
omitted `git fetch`. Round 2 found the replacement traversal used non-recursive
`Path.glob`, so four nested files were still outside it — including
scripts/scripted-schedules/entrypoint.py, which calls get_context() against a live
ErsatzTV server and then drives define_content/reset_playout/build_playout off the
result. Now rglob, with scripts/tests/ as the single stated DIRECTORY-level
exclusion (a scope choice, reviewable in one line; not a predicate over content).
Population 55 -> 59, rows 63 -> 68.

The generalisation is in the record, because the deliverable made the same mistake
twice: the scope may be hand-written, but anything narrowing the POPULATION has to
be executed and its output compared against the filesystem — the members it drops
are invisible by construction. That is the #774 rule turned on the artifact meant
to enforce it.

FIVE MORE OVERCLAIMS GRADED DOWN. Both merge-consent head/base rows (the hook
returns `allow` and a separate call merges, so the window is small, not absent —
"no async window" was simply false); the release smoke pull and the ci-image verify
(the concurrency group is PER-REF, so a branch build and a tag build of one commit
can publish the same :<short-sha>); and the workflow base-fetch rows, which are not
advisory — the merge hook reads the COMBINED status, so any red context blocks the
auto-grant. Also fixed a stale cross-reference where the enumeration row still said
it "inherits that row's pins" from a row graded down in the same commit.

Four PINNED rows survive out of 68. That ratio is the honest finding.

CIRCULAR JUSTIFICATION REMOVED. The scheduled-merge row said its residual was
"closed one layer down" by the very branch protection an admin may have removed.
It is not closed, it is BOUNDED by a trust assumption, and the row now says so.

Low: jq's `//` fires on `false` as well as null, so `status_check_contexts: false`
was defaulted to [] and produced a confident deny from a shape never understood —
absent and null are now defaulted explicitly, everything else is "unknown". And the
title sentence claimed "every executable in this repo" while the guard covers four
directories; both it and the docs/README entry now say what is actually enforced.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 13:44:10 +02:00
timothyandClaude Opus 5 a0849eec58 fix(778): address the cold review — drop the population filter, grade down three overclaims
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 7s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 24s
PR Gates / Docs update reminder (pull_request) Successful in 9s
review-verdict/h10 Awaiting review verdict for a0849ee
Review verdict / Set review-verdict status (pull_request_target) Successful in 17s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 13s
PR Gates / Script tests (pytest) (pull_request) Successful in 3m58s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m43s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m30s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m29s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Independent cross-family review (Codex, cold brief, read-only worktree) returned
BLOCKED with 9 findings. All 9 accepted; #5 partially, keeping one row PINNED with
its justification stated.

POPULATION (the finding that mattered most). The derivation filtered the scope by an
outbound-network token list and argued that was a scope choice rather than a
population filter. It omitted `git fetch` — this repo's most common remote read — so
prepush-rebase-check.sh, which fetches origin/main and derives a PUSH DECISION, was
structurally invisible to a guard claiming to cover "every executable that reads live
remote state", along with three others. The defence offered was that over-inclusion is
the safe direction; the filter also under-included. The content filter is gone: the
population is now all 55 files in the scoped directories, and a file that reads no
remote state carries an explicit N/A row.

OVERCLAIMS, graded down. Three rows asserted more than the code does:
- the scheduled-merge path was PINNED while the hook's own comment concedes the
  branch-protection read pins nothing → UNSAFE-KNOWN, "preflight, not a pin";
- pr-changed-files.sh was PINNED and claimed "any movement fails", but
  before-and-after equality is ABA-vulnerable (main → scratch → main) → UNSAFE-KNOWN,
  pointing at the caller-side event-count fence that does close it;
- the CI toolchain image was PINNED on a mutable TAG, against this file's own
  definition naming a digest → UNSAFE-KNOWN. The release smoke pull stays PINNED: it
  pulls the tag the same concurrency-serialized job just pushed.

The guard-inventory MUTATION regrade is reverted to BEHAVIOUR-ONLY (8/6/20). The
review is right on species: the test feeds the real script an input the clause
rejects, which this table explicitly defines as behaviour-only and has already
regraded three rows for. A manually-executed disarm does not change what the test is.

TWO REAL FAIL-OPENS FIXED:
- jq `index()` on a STRING is substring search, so a status_check_contexts arriving
  as "prefix-review-verdict/h10-suffix" answered yes and would auto-grant. Membership
  is now exact equality over a value first proven to be an array of strings.
- post-review-verdict.sh guarded both re-read comparisons with `[ -n "$x" ] &&`, so a
  2xx body that merely omitted .head.sha or .base.ref made the check a no-op and the
  status was posted having confirmed nothing.

That second fix carries a lesson worth the line: the obvious mutation (disarm the new
`-z` arm) stays GREEN, because the unconditional `!=` also rejects empty — the two
overlap, exactly the duplicate-guards-mask-each-other shape. The proof is taken
against the REAL predecessor with the `-n` conjunct restored, which goes red showing
returncode=0 and a status written.

Also: 404 is now separated from 403/transport (an unprotected branch is the strongest
form of the finding; `curl -sf` collapses both to an empty string), and the positive
control asserts the decision is `allow` and that the endpoint was actually reached,
rather than the absence of one phrase.

refs #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 13:20:42 +02:00
timothyandClaude Opus 5 fe578c5864 feat(778): name the check-and-use race, and verify the protection the merge gate rests on
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 20s
PR Gates / Docs update reminder (pull_request) Successful in 24s
PR Gates / Fix proofs (Proves trailers) (pull_request) Successful in 26s
PR Gates / decisions lifecycle (pull_request) Successful in 31s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 10s
review-verdict/h10 Awaiting review verdict for fe578c5
Review verdict / Set review-verdict status (pull_request_target) Successful in 14s
PR Gates / Script tests (pytest) (pull_request) Successful in 4m8s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m34s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m38s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m19s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 4s
#773 §3 Family D is the one class its taxonomy had no bucket for: a check and the
action it authorizes are separated in time over state that can change in between,
with nothing pinning a version (#536, #622, #632, #706, #707).

The repo had already solved this twice without noticing it was one problem — a
compare-exchange claim in-process (ffmpeg.work-ahead-slot-atomic) and RFC 7232
If-Match across /api/v1 (concurrency.ifmatch-rfc7232) — and then solved it a third
time from scratch for the tooling at #706/#707. Hence a class-level record rather
than a fourth per-instance one.

What the enumeration actually found, which none of the five records predicted:
the merge-consent hook's scheduled-auto-merge path is safe only because
`review-verdict/h10` is a REQUIRED status check on main. That is branch-protection
CONFIG, it lives outside this repo, nothing compared the two, and the hook asserted
it in a comment AND in the grant reason a human reads. Switch the context off and
every word of that sentence is false while the hook keeps printing it and keeps
auto-granting. The hook now reads the branch protection and treats it as three
outcomes: present proceeds, unreadable asks, absent denies.

Two defects were caught by the new checks themselves rather than by review:
- the population test found .gitea/workflows/dependency-scan.yml absent from the
  first draft of the inventory (a sixth workflow the recon slice never listed);
- self-review found the guard denying with a confident wrong reason when jq errors
  one level down on a malformed contexts member, so the word is now matched
  exhaustively rather than compared against "yes". Same swallow that survived the
  first fix in the #632 base-change guard.

Detector D has no plausible linter, so the detector is detector A applied to an
enumerated inventory: docs/remote-state-inventory.md classifies every in-scope
executable, and scripts/tests/test_remote_state_inventory.py derives the population
from the filesystem and asserts set equality both ways.

Deferred with reasons stated in the inventory: select-queue.sh (advisory, authorizes
no write), ci-detect-already-validated.sh (skip elides re-validation only, the image
still builds), review-verdict.yml's status POST (Gitea offers no conditional write;
already fenced by #706's retarget counter).

Mutation proofs witnessed for both new guards, clause-level, not whole-file.

fixes #778

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 12:57:04 +02:00
15d2439915 feat(794): witness a fix's test failing BEFORE the fix, and check the claim in CI (#801)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 19s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m31s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m14s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m30s
Mechanises the defect that took #776 and #793 six review rounds each: a fix's test
written to confirm the fix, not to discriminate against its absence.
testing.guard-ships-with-mutation-proof generalised from guards to fixes.

prove-fix.sh runs the selector at the commit (control, must be GREEN) and again in a
separate fresh worktree with the non-test files reverted (must be RED = pytest exit 1
exactly; 2/3/4/5/143 are refused, and --continue-on-collection-errors keeps add-a-file
fixes provable). pytest's status comes from a marker written only after it returns,
because ( cd X && pytest ); rc=$? returns the SUBSHELL's status. Opt-in by a Proves:
trailer; CI checks every commit that carries one and says out loud when a PR has none.

THE TOOL REJECTED ITS OWN AUTHOR. Three commits on the branch claimed
Proves: scripts/tests/test_prove_fix.py; the job returned UNPROVEN for all three,
because reverting the script restored a working earlier version the suite also passed.
Two had been "verified" against hand-written mutants that did not match the code that
actually shipped. The tests were rewritten until both go RED against 587edbecc — whose
script emits "red without it (pytest exit 2)", a witnessed false PROVEN.

This branch deliberately carries no Proves: trailer: the only one that would pass does
so because reverting deletes prove-fix.sh, an add-file smoke check rather than a proof
of its logic. The logic proof is a clause-level mutation that re-runs the unchanged
refusal test against a mutant and witnesses it red (graded MUTATION).

fixes #794

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-16 10:24:59 +00:00
107716fa4f docs(781): re-measure §5.3 from a derived population — the zeros split four ways (#800)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m21s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m49s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m24s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m42s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
The §5.3 table was hand-assembled; deriving the population from config surfaces
seven enabled plugins it omits, including serena. The corpus was undercounted too
(964 transcripts via rglob, not 811 — a top-level glob sees 209 and manufactures
false zeros; positive control 24,762 Bash).

Two corrections: mempalace is not dead (31 calls, last seen 2026-08-14 — the gap
was a snapshot artifact), and codex is the third-heaviest tool in the corpus at
113 `codex exec` calls across 17 sessions.

The issue's framing does not survive: "retire what is enabled and never invoked"
reads a zero as uselessness, but these zeros split four ways — broken (#777),
unreachable (serena, #799), just enabled, and measured on the wrong surface.
Establishing WHY a counter is zero is a precondition for acting on it.

The one supported removal was gitea's PROJECT copy, not the "more specific" one:
server-management and homelab-docs have no .mcp.json and depend on user scope.

The dated 2026-08-13 table is kept, with the re-measurement stated against it.

refs #781

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-16 08:02:15 +00:00
timothyandtimothy 499dd348ab feat(776): every hook reports that it fired, and the report is measured (#795)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 25s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m37s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m19s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-14 19:11:05 +00:00
timothyandtimothy 84165ab755 fix(797): the BOM guard was fail-open wherever xxd is not installed (#798)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m19s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m38s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m25s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m27s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-14 16:50:59 +00:00
4bead26326 fix(777): repair both broken LSPs, and name the surface a subagent can actually reach (#793)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 26s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m34s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m35s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m22s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 5m12s
Both C#/TS language servers and the csharp-lsp MCP server were dead; all three are
fixed and each demonstrated with a real find-all-references call in this repo.

Root causes were one shape — a config naming a path this machine does not have,
with nothing checking. None returned a wrong answer; each refused to start:
- csharp-ls: MSBuildLocator needs a dotnet root owning host/fxr; Homebrew's bin
  has none, libexec does.
- typescript-language-server: the LSP workspace root is the repo root but
  `typescript` lives in web/node_modules, and the plugin cannot pass a tsserver
  path (v5 dropped --tsserver-path; lspServers cannot set initializationOptions).
- the csharp-lsp MCP server: .mcp.json named a dotnet install that no longer
  existed, while ~/.codex/config.toml's copy of the same server had been migrated.
  Both files are gitignored, so nothing could compare them.

Corrects defect-shapes-773.md §5.1: the "workflow agents must use csharp-lsp" note
names the MCP server's tools, which subagents DO reach — it was dead because the
server could not start, not because agents cannot call it. The LSP tool is the one
no subagent has been observed to resolve.

Six cold review rounds. Five false greens were found in this PR's own verification
code, each introduced by the fix for the previous one — extracted as #796.

fixes #777

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-14 15:55:20 +00:00
timothy b552e569bb Merge pull request 'chore(deps): update dependency scriban.signed to 7.2.6' (#762) from renovate/scriban.signed-7.x into main
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 21s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 12s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 11s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Canceled after 0s
Reviewed-on: #762
2026-08-14 15:26:37 +00:00
timothy f8f2dca2ce Merge branch 'main' into renovate/scriban.signed-7.x
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 8s
PR Gates / Docs update reminder (pull_request) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 18s
PR Gates / decisions lifecycle (pull_request) Successful in 16s
review-verdict/h10 Exempt: authored by the 'renovate' bot account, touches no protected path, and changes only dependency manifests
Review verdict / Set review-verdict status (pull_request_target) Successful in 17s
PR Gates / Script tests (pytest) (pull_request) Successful in 2m26s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m41s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m12s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
2026-08-14 13:45:55 +00:00
timothy 945328fa24 feat(774,775): one rule for guard populations, one for guard proofs — both enforced (#791)
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 18s
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 23s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 11s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 11s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m24s
2026-08-13 23:17:57 +00:00
timothyandClaude Opus 5 57030d5168 docs(774): the enumeration table RELABELLED two lost invariants instead of listing them
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 10s
PR Gates / Docs update reminder (pull_request) Successful in 14s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 21s
PR Gates / decisions lifecycle (pull_request) Successful in 15s
Review verdict / Set review-verdict status (pull_request_target) Successful in 12s
PR Gates / Script tests (pytest) (pull_request) Successful in 2m23s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m53s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m43s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m20s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 6s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
review-verdict/h10 Review-verdict: MERGEABLE @ 57030d5 (base: main)
Final review round. One blocker, and it is the table committing the failure the table exists
to prevent.

The withdrawn parity test asserted two DISJOINTNESS properties — no read-side word in both
`POS_RE` and `NEG_RE`, no write-side word in both `case` arms. The enumeration listed rows
called "read-side polarity" and "write-side polarity" and pointed them at the two guards
added in the rescue. But polarity is not disjointness, so those rows described the
REPLACEMENTS while quietly dropping the originals from the ledger. Enumerating what a
removal cost is the whole job of that table, and relabelling a lost invariant as a narrower
surviving one is precisely how the previous two removals lost something.

Both are now listed as LOST, and the two added guards moved to a separate table that says
what they actually pin. The gap is stated with its demonstration rather than asserted:
`MERGEABLE` in BOTH write-side arms leaves every polarity assertion green, because the
success arm wins — the withdrawn test failed that mutation. What the added guards DO catch
is the dangerous direction, a token meant as BLOCKED reading or posting as approval, which
writes a green `review-verdict/h10`.

Documentation only; no code changed. Review confirmed everything else clean: both new tests
load-bearing (BLOCKED added to the success arm, and LGTM moved to failure, each reddens),
fixture usage correct, ten cases collecting with no skips or collisions, and both names and
docstrings accurately disclaiming disjointness and parity.

584 script-tests pass, decisions-validate OK, inventory parses to 47 rows unchanged.
(--no-verify: pre-commit hook exceeds the tool timeout; its checks were run explicitly.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 23:52:31 +02:00
timothyandClaude Opus 5 fb258522ac fix(774): cover the WRITE side too, and name the read-side test what it actually is
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 9s
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 19s
PR Gates / Docs update reminder (pull_request) Successful in 21s
PR Gates / decisions lifecycle (pull_request) Successful in 27s
review-verdict/h10 Awaiting review verdict for fb25852
Review verdict / Set review-verdict status (pull_request_target) Successful in 9s
PR Gates / Script tests (pytest) (pull_request) Successful in 2m30s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m35s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 4m47s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Cold review of the rescue returned BLOCKED on two, both fair.

THE SUBSTANTIVE ONE: the deleted parity test checked disjointness on BOTH scripts; the
rescue covered only the read side. Review demonstrated the gap rather than asserting it —
adding `BLOCKED` to post-review-verdict.sh's SUCCESS arm produced an overlap the deleted
test caught and the rescue did not, because the rescue never executes that script. That was
a real, undisclosed loss, and it is the second time in two commits that removing something
dropped an invariant nobody enumerated. So:

  test_post_review_verdict.py::test_each_verdict_word_posts_its_established_polarity

`case` takes the FIRST matching arm, so a token in both arms is not ambiguous — it resolves
to whichever comes first, exactly as `is_pos` wins on the read side. Same consequence, and
it is the one that matters: a word a reviewer means as BLOCKED posting `success` writes a
GREEN `review-verdict/h10`, the required context branch protection honours. Mutation-proved
with the exact case review cited: `BLOCKED` in the success arm -> the test names it and
reddens.

THE NAMING ONE, and it is the mistake I keep repeating: the read-side test called itself a
disjointness test and its docstring said "no word may be in both vocabularies", while it
pins the observable classification of five hardcoded tokens. For a UNIVERSAL property an
omitted token is not a vacuous pass, it is precisely the untested member — the record's own
warning. Renamed to test_each_verdict_word_retains_its_established_polarity and the
docstring now scopes itself to the five words. Both surviving tests are polarity
regressions, not disjointness and not parity.

The inventory now enumerates all seven invariants the withdrawn file asserted and says where
each went — five retired to #788, two rescued as per-script polarity. Enumerating on removal
is `process.enumerate-workaround-behaviors-before-deleting`, which this branch has now
failed twice and should stop failing.

584 script-tests pass, pyright clean, decisions-validate OK. ruff reports one S105 in
test_post_review_verdict.py:103 — PRE-EXISTING and a known false positive on a test stub
(identical on origin/main, my additions start at line 335); it is #780's territory.
(--no-verify: pre-commit hook exceeds the tool timeout; its checks were run explicitly.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 23:31:34 +02:00
timothyandClaude Opus 5 b8f60bdec4 fix(774): rescue the disjointness invariant the deletion silently took with it
Build ErsatzTV Image / Delimiter ban (release path) (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 20s
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / decisions lifecycle (pull_request) Successful in 29s
review-verdict/h10 Awaiting review verdict for b8f60bd
Review verdict / Set review-verdict status (pull_request_target) Successful in 12s
PR Gates / Script tests (pytest) (pull_request) Successful in 2m17s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m12s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Canceled after 1m58s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Canceled after 0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Canceled after 0s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Canceled after 0s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Canceled after 0s
Cold review of the deletion caught what the deletion itself missed: the withdrawn parity
test carried a SECOND, separable invariant. `test_no_word_is_both_positive_and_negative`
had nothing to do with parsing shell — it prevented a verdict token belonging to both
vocabularies, which matters because `check-review-verdict.sh` sets `is_pos` and `is_neg`
from two INDEPENDENT `grep -iqE` calls. Deleting the file took it along, undisclosed. That
is `process.enumerate-workaround-behaviors-before-deleting`, and I did not enumerate.

Rescued BEHAVIOURALLY, which is why it survives where its parent could not: it EXECUTES the
real classifier rather than reading its source, so no shell construction can fool it. The
word list is a literal, and per `testing.guard-derives-population-from-source` that is
legitimate here — the property is PER-MEMBER ("each of these classifies as exactly one
thing"), not a completeness claim, so a word missing from the list is untested rather than
concealed. It is explicitly NOT a claim that these are the only words the scripts accept;
proving that still needs #788.

THE MUTATION RUN CORRECTED THE DOCSTRING, which had been written first — the wrong order,
and the third time this session that running a mutation contradicted something already
asserted. Adding `mergeable` to NEG_RE left the test GREEN. Reading
check-review-verdict.sh:212 explains it: `if [ "$is_pos" = 1 ]; then head_pos=1; else
head_neg=1; fi` means `is_pos` wins, so that edit has NO observable effect — NEG_RE is
shadowed by POS_RE for any overlapping word. The real direction is the reverse: adding
`blocked` to POS_RE makes `BLOCKED` classify `positive`, and the test goes red naming it.
Both mutations are now recorded in the docstring as measured, with which one is caught and
why the other has nothing to catch.

Also closed from the same review: issue #788's BODY still described the parity test as the
live interim measure with an unticked "delete it" box, while only a later comment recorded
the withdrawal. The body now carries a status banner, strikes the superseded line and ticks
the box — fixed on the issue, since a stale first bookkeeping surface is the same defect
class this branch fixed in post-review-verdict.sh.

ruff clean, pyright clean, decisions-validate OK, 579 script-tests pass.
(--no-verify: the pre-commit hook exceeds the tool timeout; its checks were run explicitly.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 23:18:51 +02:00
timothyandClaude Opus 5 8f02e4a284 fix(774): withdraw the vocabulary parity test — six rounds is the answer, not a seventh
Round six returned BLOCKED on the same file again: a column-zero `esac` inside a string
truncates the scoped match and silently drops a real arm, and a heredoc inside the block
still false-reds. Both correct. Both the sixth distinct shell construction found in six
rounds.

That is no longer a sequence of bugs, it is a result. A regex over shell source is not a
shell parser and cannot be made into one, and each round's fix was locally right while the
sequence converged on nothing. The file's own docstring told the next session not to get on
this treadmill; the honest reading is that it should not have been built.

DELETED rather than patched again. The reasoning is this change's own thesis, applied to
itself: `testing.guard-derives-population-from-source` says the answer to a missing
authoritative source is to CREATE one, never to approximate it with a predicate over text —
and detector C says two copies of one rule get deduped, not compared. The right fix was
available from the start and is #788. What I built instead was the weak detector the record
warns against, and six rounds of a reviewer falsifying its prose is the empirical proof.

A guard whose accompanying prose can be falsified every round is worse than no guard,
because by this record's own argument a guard described as sound stops being re-examined.

WHAT IS LOST, stated plainly: the duplication is real and is now UNMITIGATED. The two
vocabularies in post-review-verdict.sh and check-review-verdict.sh can drift, and only a
comment says they must not. That comment now says so explicitly, names #788 as the fix, and
no longer claims a test is holding them together.

WHAT IS KEPT: the finding itself (the duplication, the stale breadcrumb pointing at the
merge-consent hook that carries no copy), the corrected comment, #788, and a new section in
the #774 record recording this as the worked example of "a weak detector is itself the
symptom-keyed mistake" — demonstrated rather than argued.

Inventory updated: 31 guards / 4 tooling / 12 proof, 4 MUTATION / 6 BEHAVIOUR-ONLY / 21 NONE.
The withdrawal is recorded IN the inventory, since that is where a future session will look
for the guard and find it missing. Its count-parity guard verified the new numbers itself.

decisions-validate OK, 574 script-tests pass (six removed with the file).
(--no-verify: the pre-commit hook exceeds the tool timeout; its checks were run explicitly.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 23:07:35 +02:00
timothyandClaude Opus 5 49fe17336b fix(774): scope the scan to the case block — fixing the INPUT, not adding a fifth epicycle
Round five. The confirmation review closed Q1 (the unquoted-value escape is gone) and found
two things left: the file still described every loose match as a case ARM, and
`_ANY_CASE_ARM` could false-RED on a `<word>) state=` inside a heredoc, a compact comment or
an unrelated case statement.

Both had one cause, and it was not the regex. Both patterns read the WHOLE FILE when their
subject is a single `case` block. No amount of widening or narrowing fixes a wrong input —
that is the treadmill this file's own docstring told the next session not to get on, and
round five would have been the first step of it.

The scan now reads only between `case "$verdict" in` and its `esac`. That removes the entire
false-positive class at once, and it makes the "every case arm" language TRUE rather than
nearly true — the overclaim and the false red were the same defect described from two sides.
If the block cannot be located the helper REFUSES: falling back to the whole file would
silently restore the false reds, and returning empty would make every assertion vacuous.

Also from the same review: comparison is now a MULTISET rather than a set, so two arms
sharing a label cannot let an unparsed occurrence hide behind a parsed twin — the same rule
as testing.enumerating-guard-identity-not-position. And the failure message no longer offers
two causes that scoping has since eliminated.

Proven both directions: a heredoc containing `SHIP-IT) state=success ;;` leaves the suite
green; the same line inside the case block reddens it.

The limits list is updated, and records the removed class deliberately — it shows which
fixes are worth making. What remains are same-line-shape misses, which really are regex-bound;
the false-positive family was an input-scope bug wearing a regex costume.

ruff clean, pyright clean, 580 script-tests pass.
(Committed with --no-verify: the pre-commit hook exceeded the tool timeout on the previous
commit; decisions-validate and the full suite were run explicitly above.)

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 22:58:22 +02:00
timothyandClaude Opus 5 f89da6a197 fix(774): strip comments before the loose scan — the widening could false-RED a correct tree
Self-audit of the round-four fix, before its confirmation review returned. Making
`_ANY_CASE_ARM` permissive by construction closed the miss and opened the opposite failure:
`#FOO) state=bar` — a comment with no space after the hash — satisfies the loose pattern and
not the strict one, so it would be reported as an unparsed case arm on a completely correct
tree.

That direction matters as much as the miss did. A guard that reddens a correct tree gets
deleted, and then catches nothing at all — which costs more than the construction the
widening was for. Comments are now stripped before both scans, the same treatment the hook
wiring check already needed for the same reason.

Narrow: `# FOO) state=bar` with a space never matched, and the real file contains no such
line today. Fixed anyway, because "narrow" is how each of the previous four rounds started.

Proven three ways: a comment mentioning a hypothetical arm leaves the suite green; a real
unquoted `SHIP-IT) state=success ;;` arm still reddens; clean tree green.

ruff clean, pyright clean, 580 script-tests pass.

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 22:52:54 +02:00
timothyandClaude Opus 5 96a67c7e76 fix(774): the loose extractor shared its subject's blind spot; stop claiming the limit list is exhaustive
Round four, one Medium, and it lands on the defence rather than the code: the change argued
that its residue was acceptable BECAUSE it was accurately disclosed, and the disclosure was
wrong within one round.

`SHIP-IT) state=success ;;` is valid shell selecting `success`. Both extractors required the
double-quoted spelling `state="success"`, so the LOOSE one missed it too — `unparsed` stayed
empty, the vocabularies stayed equal, everything stayed green. A completeness check that
shares its subject's blind spot is not a completeness check.

The bug was structural, not about quoting. A loose counterpart must be permissive BY
CONSTRUCTION; mine was merely a little wider than the strict pattern, so the one thing it
could not see was the one thing it existed to find. It now matches `state=` with any value
form and lets the strict pattern's failures surface as a difference. Proven on three arms —
unquoted, single-quoted, and a differently-named double-quoted one — each red, clean tree
green.

The disclosure is corrected too, and this is the part worth keeping. It said "KNOWN LIMITS,
ENUMERATED", which reads as exhaustive and was false one round later. It now says the list
is NOT exhaustive, records that this very entry was the one it missed, and ends with
"whatever the next round finds. Assume this list is one short." Four rounds have each
produced another construction; claiming completeness over a regex on shell source is the
overclaim the whole change argues against.

Test renamed to test_the_strict_extractor_consumed_EVERY_case_arm_THE_LOOSE_ONE_FOUND, since
the old name asserted more than the code could deliver — and the inventory guard immediately
went red on the now-stale proof ref, which is exactly the drift it was built to catch,
catching its own author one commit after being written.

ruff clean, pyright clean, decisions-validate OK, 580 script-tests pass.

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 22:48:53 +02:00
timothyandClaude Opus 5 e746ba5504 fix(775): resolve the record's two competing standards; close the templated-image hole; state the predicate limits
Third review round. Of the eight findings from round two, five were closed; this addresses
what remained, and the split between "fixed" and "stated" is deliberate.

FIXED — the record could not adjudicate. Its frontmatter `rule` required disarming the
guard's clause; the body added an input-mutation standard for guards that ARE tests. Two
incompatible criteria in one active record means one reviewer rejects the self-referencing
MUTATION rows on the frontmatter and another accepts them on the body. The exception is now
IN the rule with its limits: admissible only for checker-guards, only when executed and
witnessed, never a licence to grade a script-guard MUTATION for having a bad-input test,
and a file-level grade covers the clause its cited case mutates rather than every assertion
that later lands in the file.

FIXED — a matrix-templated image bypassed the cross-workflow check. `_PIN.match` requires a
literal tag, so `image: <repo>:${{ matrix.tag }}` in another workflow ran on the toolchain
image while the check reported none. Now keyed on the image REPOSITORY, so a templated tag
is reported rather than skipped — it is a fault in its own right, since nothing could then
verify which image ran. Proven: a probe workflow with exactly that construction is caught,
removed, green.

STATED, NOT PATCHED — the remaining three findings are all one shape: a regex over shell
source cannot be made complete. Each round found another construction (a case arm whose
first command is not the assignment, an indented reassignment, a basename inside `: #
... disabled`), and a fourth round would find a fifth. This repo has already paid three
rounds for exactly this class at #629, #633 and #698. So the limits are now enumerated in
the files themselves rather than left for the next reader:

  - the parity extractors list the three constructions that escape them, say what they DO
    catch (the realistic same-style edit on one side only), and say plainly that this is
    not a proof of semantic equality between two shell programs;
  - the inventory records that hook wiring is a substring test for the basename, so it
    catches deletion but not deliberate disablement.

Both name the issue that removes the underlying duplication (#788), and the parity file
tells the next session NOT to invest another widening round.

Also outstanding and tracked, not silently dropped: PROOF/GUARD roles and MUTATION grades
are per FILE, so a standalone invariant added to a PROOF file inherits its classification
and a self-referencing grade does not cover clauses added later. That is clause-level
inventory, which is #790.

ruff clean, pyright clean, decisions-validate OK, 580 script-tests pass.

Refs #774
Refs #775

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 22:36:08 +02:00
timothyandClaude Opus 5 0c5938dd60 fix(774): the wiring check counted a MENTION, reproducing inside the fix the defect it closed
Self-audit before re-review, and it found one. `wired_hook_files()` was added to stop hook
EXISTENCE standing in for hook WIRING — but it substring-matched the filename against the
whole husky text, and `.husky/pre-commit:7` reads

    # CI where a base ref exists). Fail-open shim — see .claude/hooks/decisions-guard.sh.

one line above the real invocation. Delete line 8, keep line 7, and the hook still reads as
wired. That is mention-for-invocation, which is the exact substitution the function exists
to prevent, one line inside the fix for it. Comment lines are now stripped from the husky
hooks first; settings.json needs no stripping because JSON has no comments.

Proven both ways: with the invocation removed and the comment left, the guard names
decisions-guard.sh as unwired; clean tree stays green.

Also verified rather than assumed, since a fix round is where adjacent defects live:
  - a stale SELF-referencing proof ref is still caught (the self-reference escape hatch
    skips only the PROOF-row classification check, not the def-existence check);
  - a reworded summary is LOUD, not vacuous — an unparsed summary fails with a message
    saying so, rather than silently checking nothing.

ruff clean, pyright clean (0 errors) on the three new files. Deliberately NOT ruff-format-ed:
the pre-existing scripts/tests corpus is not formatted either, so reformatting only these
three would diverge them from every sibling and bake in a format derived from an
un-versioned config on one machine — which is the divergence #780 exists to settle.

580 script-tests pass.

Refs #774

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 22:26:25 +02:00
timothyandClaude Opus 5 3473a6c889 fix(774,775): close the cold-review findings — including three the change inflicted on itself
Two independent cold reviews (Codex GPT-5.6 cross-family; Fable 5 on the patch) both
returned BLOCKED. They agreed on the counts error and the extractor hole; each found
things the other did not. Fixes, with what each was:

THE INVENTORY DID NOT COVER ITS OWN NEW GUARDS. `_SCRIPT_REF` matched `scripts/name.py`
but not `scripts/tests/*.py`, so the three guard files this change introduced had no rows
and the completeness check stayed green. A completeness guard blind to its author's new
guards is precisely the defect being legislated against. The population now globs
`scripts/tests/test_*.py` — which is how they actually run, since pr-checks.yml invokes
the directory. 32 rows -> 48.

That forced a third Kind. Once test files are in the population, every mutation proof
becomes a row wanting a proof of its own, forever. `PROOF` marks a file whose job is to
prove another guard; a scripts/tests file enforcing a repo invariant with no separate
guard behind it stays GUARD and may cite a mutation case in its own file.

HOOK EXISTENCE WAS STANDING IN FOR HOOK WIRING. Deleting a hook's registration from
.claude/settings.json left the population and the table unchanged, so the row went on
describing a guard that no longer ran — #631's shape one level down. Now derived from
settings.json plus the husky hooks.

THE SUMMARY COUNTS WERE A HAND-KEPT MIRROR AND WERE WRONG ON ARRIVAL: "28 guards, 4
tooling ... 19 have none" against a table holding 27/5/6/3/18. Both reviewers found it
independently. The prose is now parsed and asserted against the table.

TWO FALSE MUTATION GRADES, each with a concrete disarm:
  - test_full_first_page_alone_does_not_end_enumeration sends 50 docs paths then one more
    docs path; disarm pagination to treat a full page as final and it is still all-docs,
    still exempt, still green. Re-pointed at test_protected_path_on_a_LATER_page_is_still_seen,
    which does go red under that mutation.
  - test_the_scan_job_runs_the_out_of_pytest_positive_control asserts only that the script
    exists, is executable, is referenced and is marked; replace its logic with `exit 0` and
    all four pass. ci-prove-ban-detects.sh regraded NONE.

The MUTATION column was also being applied as a curve: three rows graded MUTATION fed the
real script an input only that clause rejects, which is what the rows eight lines away are
graded BEHAVIOUR-ONLY for. Definition sharpened to *witnessed* rather than plausible, and
those regraded. 5 MUTATION / 6 BEHAVIOUR-ONLY / 21 NONE across 32 guards.

THE VOCABULARY EXTRACTOR COULD RETURN A PARTIAL SET. `[A-Z|-]` cannot match `SHIP*)`, so
adding that arm leaves the extracted set non-empty AND equal to the read side — parity
green while the gate desyncs. Emptiness checks cannot see partial degradation. A loose
counterpart now asserts the strict pattern consumed every arm; proven red on exactly that
attack and green on a clean tree. Also: each verdict pattern must be assigned once, since
the extractor unions assignments while the classifier runs the last.

Also: docker-build.yml was itself an unchecked scope mirror (now asserted to be the only
workflow with toolchain container jobs, by parsing container.image rather than grepping —
ci-image.yml names the image because it builds it); the mutant floor is an equality;
e2e-functional.sh reclassified GUARD (it exits 1 on a failed contract assertion);
design-sync-reminder.sh does block the first Stop. The doc now states all six excluded
classes instead of one.

Not done here, filed instead: workflow-owned execution-class metadata to replace
TOOLCHAIN_JOBS, a single shared verdict vocabulary, and an executable clause-level
mutation harness. Each touches a CI-gating or merge-gate path and wants its own review.

580 script-tests pass.

Refs #774
Refs #775

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 22:20:51 +02:00
timothyandClaude Opus 5 0bd59b0b6e feat(774,775): one rule for guard populations, one for guard proofs — both enforced
#773's analysis found that the largest recorded failure family is reasoning about a
representative instead of the population (39% of process-failure records), and that the
most common is a check that never ran at all (25%). Both rules had been reinvented
repeatedly and written down nowhere.

Two decision records:

  testing.guard-derives-population-from-source (#774) — a guard enumerates its population
  from a machine-readable authoritative source and asserts set equality both ways. States
  the boundary that keeps it honest: filtering to select the SUBJECT of a per-member
  property is fine; filtering the population before a COMPLETENESS claim is the defect.
  Also separates guard SCOPE (a reviewable policy choice) from guard POPULATION (always
  derived).

  testing.guard-ships-with-mutation-proof (#775) — disarm that clause alone and a named
  test must go red. Behaviour-only coverage is graded separately, because it proves the
  guard reacts, never that it is connected.

Audit findings fixed:

  ci-image-pin stated an invariant it did not check. Its error text says "Every container:
  job must pin ersatztv-ci:<7-char-sha>"; what it asserts is that `grep … | sort -u` yields
  one DISTINCT value. Distinctness is a property of the pins present, so deleting the
  container: block from `test` leaves four pins, one distinct value, and a REQUIRED context
  silently running on the bare runner. test_ci_image_pin_population.py adds the population
  check, keyed on a reviewed registry cross-checked both ways — set equality between two
  DERIVED sets could not see this, because both sides shrink together.

  The verdict vocabulary was written down twice with no cross-check — post-review-verdict.sh
  (write) and check-review-verdict.sh (read). A word in one and not the other sends the
  required status green while the hook still denies. Both vocabularies are now extracted
  from their own source and compared as sets; a test that restated the words would just be
  a third copy. The write side's comment pointing at pretooluse-merge-consent.sh was also
  stale — the hook carries no copy and delegates.

Mechanical enforcement, answered explicitly for both:

  No to a filter-shaped-guard lint. The token is not the defect — ToolCatalogTests filters
  correctly eight lines from a completeness assertion that must not — and it would be a
  string predicate over source, which this repo's record says takes 3+ rounds. Building it
  would be #774 violating #774.

  Yes to enforcing the bookkeeping. docs/guard-inventory.md classifies all 32 guard files;
  test_guard_inventory.py derives the population from the filesystem and call sites,
  asserts set equality both ways, and resolves every claimed proof ref to a real def. A new
  guard cannot ship unclassified; a renamed test cannot leave a row claiming lost coverage.
  What it does NOT check — whether a MUTATION claim is true — is stated, not implied.

Measured: 28 guards, 4 tooling. 6 mutation-proved, 3 behaviour-only, 19 unproven.

Every guard added here was mutation-proved by execution before being believed: neutering
pin_population_faults turned 20 of 25 red; the inventory guard was driven red three ways
(deleted row, new unclassified hook, stale proof ref) and restored green.

573 script-tests pass. Scope limit stated in the doc: inline workflow-job guards are not
in the machine-checked population.

Refs #774
Refs #775

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 21:56:52 +02:00
timothyandtimothy e1937674cd docs(773): root-cause the recurring defect shapes across the full closed-issue corpus (#782)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 23s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m42s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m23s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m38s
Analysis over all 349 closed issues (95 carry a `## Closing record`), classified by five
independent raters against a written taxonomy that permitted `NEW:<name>`, with two controls: a
blind inter-rater re-rating (12/15 agreement) and a backward-generalization sample over the 254
pre-convention issues.

Findings that change the picture #773 started from:

- The ranking reverses. Vacuous verification is the most common shape (17/69), not twin-missed
  (14/69) — #773's 50-issue sample had it the other way.
- #773's central hypothesis holds and extends: twin-missed, vacuous-by-sampling and symptom-keyed
  guards are one error (reasoning about a representative instead of a population), 27/69 (39%),
  one detector — already reinvented six times in this repo under six names.
- The shapes predate the closing-record convention (#1, #215, #232, #403, #473), so they are not an
  artifact of recent guard-building. That confound was tested and refuted, not assumed away.
- A class the taxonomy missed entirely: check-and-use races over mutable state (#536, #622, #632,
  #706, #707).
- Overclaim drops to 4% as a primary cause — a modifier, not a class. Round-churn likewise: 33 of 69
  records narrate >=3 review rounds, spread across every family, only 2 in the class named after it.

Part 2, measured rather than assumed: csharp-lsp cannot initialize and typescript-lsp cannot resolve
typescript, the LSP tool has 0 calls across 811 transcripts, ruff/pyright are enforced nowhere
despite the global instruction, no hook scripts are dead — but PreToolUse/PostToolUse execution
leaves no durable trace, so we cannot tell whether our own guards fire.

Names the classes where no mechanical detector is plausible rather than inventing weak ones, and
strikes one proposed tool (shellcheck) after testing showed it does not catch the case it was
proposed for.

Provenance, kept here rather than in the document because a reader never saw the earlier drafts: six
cold review rounds, worktree-isolated. Rounds 2-5 each found a real defect in the text written to fix
the round before — two fabricated quotes, a Family A/C double-count, a miscited precedent (#711
argues FOR the enumeration it was cited as rejecting), a false floor-rounding claim, and a
round-count built by pattern-matching that undercounted by half. Every one landed in new prose, never
in the text under review, which is the document's own thesis operating on its author and the reason
the final pass was whole-file rather than delta-scoped.

Spawns #774-#781 and #784, tracked in the "Defect-shape hardening" milestone.

fixes #773

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-13 19:16:50 +00:00
timothyandtimothy cfdab63a3a fix(767): gate the release path on the delimiter ban with a prerequisite job (#770)
Build ErsatzTV Image / Delimiter ban (release path) (push) Successful in 19s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m13s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m40s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m30s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m27s
The delimiter ban protecting `build`'s `Smoke + IPTV E2E` was enforced only by a pytest in `script-tests` — `on: pull_request`, not a required context — so nothing re-checked it on a `v*` tag push, which is exactly when the candidate image is published. A `scan` job now runs the ban test and `build` lists it in `needs:`, so a red `scan` skips `build` and no image is built.

Measured both directions without cutting a release: run 1928 (poisoned Smoke) → scan failed, `Build & push` skipped; run 1929 (control) → scan green, build ran.

The gate rests on three different KINDS of check, because each single kind was defeated in review: the ban test; an execution probe against a poisoned copy with all three `env:` tiers layered; and `scripts/ci-prove-ban-detects.sh`, which is not a test — it poisons the real checkout and vouches only for the ban test's `build` parametrisation failing. Eight review rounds; rounds 1-5 each found a real defect in the previous fix.

Refs: #767
Decisions-Edit: yes
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-13 18:04:52 +00:00
timothyandtimothy f9f8f65cee docs(720): name all three Komodo stacks and the label-based resolution rule (#769)
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 7m0s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m31s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 6m9s
A live Komodo stack literally named `ersatztv` owns the TEST channel, not prod. `DeployStack ersatztv` succeeds, looks healthy, and promotes nothing — silent and plausible. Extends the existing callout with all three stack names and the resolution rule: identify the prod stack by the container's `com.docker.compose.project.config_files` label, not by stack name.

Container labels re-verified live on jazz 2026-08-11; the verification date is scoped to what was actually re-measured, after review flagged the stamp as covering unchecked values.

The server-management half (the `komodo` skill still uses the dead `media-servers` as its worked deploy target) cannot land in this repo and is tracked as server-management#743. #720's box 2 was re-scoped to that hand-off rather than ticked as though the skill were fixed.

fixes #720

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-11 19:26:07 +00:00
timothyandtimothy 884ac8a7e9 fix(756): extend the dropped-step guard to docker-build.yml's required jobs, where a drop is fail-OPEN (#768)
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m0s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m24s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Failing after 6m8s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m42s
A `run:` body the runner declines to interpolate is dropped, and the job still
concludes `success` (#751). #751 fixed that in review-verdict.yml, where the
failure is fail-CLOSED. This closes the two places where it is fail-OPEN:
`Build & test (.NET)` and `EF migration integrity (SQLite + MySql)` are the
other two required contexts on `main`, so a dropped step there sends a required
check green having done no work.

Per-STEP markers, not per-job as proposed: a marker on the first step only
proves the job began, while the drop that costs something is `Test`, `Build` or
a migration replay. The trailing guard carries no `if:` — with a dozen steps,
`always()` would announce a false "these steps never executed" on every ordinary
red build; the default `success()` is correct because guard-skipped implies
job-red. Plus a ban on the raw `${{` opener in `test`, `migrations` and `build`,
which makes the class unreachable rather than merely caught. `build` is included
because its Smoke step runs AFTER the image is pushed.

Measured live on the build lane in both directions: probe #765 (drop caught,
sole failure in the job) and #766 (a failing continue-on-error step does not
skip the guard). 510 tests, 30 mutations killed across two harnesses, five cold
review rounds across two model families.

Residual tracked as #767: the `build` ban is review-time only, not fail-closed
on the release path.

fixes #756

Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-10 23:35:08 +00:00
timothy 9a5d34e888 fix(751): a stray expression delimiter in a COMMENT killed the verdict gate (#764)
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 16s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 13s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 14s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m31s
The classify step in review-verdict.yml stopped executing on 2026-08-03 and the job
reported success anyway, so the branch-protection-required review-verdict/h10 was posted
by nothing but a human hand for three days and both exemption classes silently died.

Three independent defects, each alone sufficient:

* A ${{ }} sequence in a SHELL COMMENT. The runner scans the whole run: scalar for the
  expression opener and rewrites the entire body into one format(...) call; `pr number`
  does not parse, so it drops the step and concludes the job green. The prose documenting
  a fix disabled the fix.
* The retarget fence never trusted its count: a page past the end of the timeline is JSON
  `null`, not `[]`, so rt_ok was never yes for ANY PR and every exemption success was
  withheld. Fixing the first alone would not have restored the exemptions.
* The same nil-slice shape on /commits/{sha}/status, which made read_existing_verdict
  exit 1 and post nothing.

A nil Go slice serialises to `null`, so every list-shaped field on this API is suspect and
only a per-endpoint measurement settles it — timeline returns bare null, the combined
status returns {"statuses": null}, comments and pulls/{n}/files return [], and
/statuses/{sha} returns []. Four endpoints, three shapes.

The silent green is the actual defect, so a start-marker guard now fails the job when the
classifier did not execute, and two static guards reject the delimiter at review time.
CLAUDE.md and AGENTS.md became PROTECTED paths: they define the H10 rule and were
docs-only-exemptible, reachable again precisely because this restores the exemptions.

Verified by a live scratch-base probe pair with a negative control, 460 tests, and 29
mutations across six rounds. Five cold review rounds, alternating model families; none
found a path to a green review-verdict/h10 on an unreviewed head, and every one found a
defect beside the fix — including that round 2's guard was dead code against a page limit
of 100 on a server that caps at 50.

Deferred: #756 (docker-build's required jobs, where a dropped step is fail-OPEN) and #763
(paging both /statuses/{sha} reads).

fixes #751
2026-08-10 19:33:33 +00:00
timothyandClaude Opus 5 20b117dabf fix(751): round-5 findings — a rationale that was itself vacuous, and a third regex round
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 13s
PR Gates / Docs update reminder (pull_request) Successful in 15s
PR Gates / decisions lifecycle (pull_request) Successful in 18s
Review verdict / Set review-verdict status (pull_request_target) Successful in 6s
PR Gates / Script tests (pytest) (pull_request) Successful in 2m14s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m34s
review-verdict/h10 Review-verdict: MERGEABLE @ 20b117d (base: main)
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m41s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m27s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 10s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 7s
Fifth cold review: MERGEABLE, no Blocker, no High. Four Low findings, none behavioural.
Fixing all four rather than accepting them, because two are the exact class this issue
exists to retire: text that reads as a checked reason and is not.

A VACUOUS RATIONALE, on the branch about vacuous rationales. The comment on the history
stub's `page` guard said it sits ahead of the read-counting modes "so the page-2 probe
cannot shift 'raced row appears on read N'", by analogy with the combined endpoint.
Measured: moving that guard AFTER the counter modes reddens NOTHING, because no history
mode that counts reads ever issues a page-2 request — `raced=1` on page 1 short-circuits
the probe. The real reason is the other half: page 2 must terminate for modes that
describe page 1 only, and dropping just that `print("[]")` reddens
`test_a_PRE_EXISTING_human_row_does_NOT_trigger_a_repair`. Comment now says which half is
load-bearing and which was wrong. (The COMBINED endpoint's guard genuinely is
counter-related — moving it reddens three mid-run-race tests.)

THE FALSE REPAIR IS STICKY, and the previous commit undersold it as "a stall a reviewer
can clear". It writes `$REPAIR_DESC`, which the classification refuses to grant an
exemption over and re-writes as a fixed point on every later run — so a spurious repair
removes that head's exemption PERMANENTLY, not for one run, and only a human verdict
clears it. Still the right direction against a forged green over a rejection, but it is a
per-sha loss of the exemption, and that is the argument for real paging (#763) rather
than living with this. Said in the comment now.

CORRECTING THE PREVIOUS COMMIT MESSAGE, which over-generalised: "uncertainty resolves to
a stall … never to leaving green" is true of the page-2 probe and NOT of the enclosing
path. An unreadable page 1, or a non-numeric high-water mark, still leaves the exemption
`success` standing unverified. The workflow's own comments state that correctly; the
message did not.

THIRD ROUND ON ONE REGEX, which is the documented budget for a string-matching predicate.
Assertion C started as `\w+\s*\(\)\s*\{`, gained `function\s+\w+` when review found
`function mk {` slipped it, and STILL missed the union form `function mk() {` — the
natural next spelling once the previous one is caught. Now
`^\s*(function\s+)?\w+\s*(\(\s*\))?\s*\{`, verified against all seven spellings.

THE COMPLETENESS COUNT, restored properly. Relaxing `len(bodies) >= 3` to `assert bodies`
fixed a false red but threw away the only check that the walk reached ALL run-bearing
steps: `max(len) > 5000` proves it reached the classifier and nothing about the short
ones, so a helper that silently stopped yielding them would pass an unscanned delimiter.
Now counted against the job's own step list, read directly rather than through the helper
under test — which catches a helper reading the wrong key or dropping steps, while still
tolerating a step being legitimately added or removed.

Verification: 460 green. Three mutations, each as intended — the union spelling `function
mk() {` (red, previously passed), a walk that drops the short steps (red, the property the
count guard restores), and a legitimate step deletion (PASSES, confirming the false red it
replaced stays fixed). Twenty-nine mutations across six rounds.

Refs: #751
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 21:08:10 +02:00
timothyandClaude Opus 5 e133c11fde fix(751): rebase onto #760, close the fail-OPEN twin, and retire four claims that had rotted
Fourth cold review round: no Blocker, no new path to a green `review-verdict/h10` on an
unreviewed head, and it independently re-measured 14 claims in the diff. It also caught
that this branch was about to revert someone else's work, and found the one remaining
place where the nil-slice/clamp lesson had not been applied.

REBASED ONTO 9881d1ff8 (#760), which landed while this was in review. The tell was the
one CLAUDE.md documents: `git diff origin/main HEAD` showed deletions I never made —
`docs/decisions/records/mcp/tool-schema-openapi-parity.md` and edits to `docs/mcp.md`.
Pushing would have reverted them. The generated catalog was regenerated rather than
trusted to the rebase, and verified to carry BOTH records.

THE FAIL-OPEN TWIN, one function further on than the last round reached.
`/statuses/{sha}?limit=100` is read twice — for the high-water mark and for the
post-write race check — and neither pages, while `limit` clamps to 50. So "no raced row
on page 1" does not establish "no race": a human `BLOCKED` landing in the write window
can sit on a page the job never reads, leaving a forged green over a rejection. This is
the ONE path in the design whose failure direction is toward SUCCESS.

Measured rather than argued: a probe head reached 33 rows after ~5 runs against a cap of
50, and the ordering is only coarsely newest-first (`33,32,31,30,28,29,27,…`), so a few
CI reruns reach it and the row's position cannot be relied on — which this workflow's own
comment already disclaimed. That comment ALSO claimed order-independence flatly; false
once the page clamps, so it now says what actually holds and what saves us.

The mitigation is conservative rather than complete: if page 1 shows no race, page 2 is
read, and any rows there — or an unreadable page 2 — count as "assume raced" and repair
to `pending`. Uncertainty resolves to a stall a reviewer can clear, never to leaving
green. Real paging of both reads, including the high-water mark, is #763.

A THIRD empty shape turned up while modelling it: `/statuses/{sha}` past the end returns
`[]`, where `/commits/{sha}/status` returns `{"statuses": null}` and the timeline returns
bare `null`. Three endpoints, three shapes, one server. The code tolerates both here
because guessing per endpoint has now been wrong twice.

MY OWN COVERAGE GAP, found by mutation rather than by reading: inverting the
unreadable-history-page-2 branch reddened NOTHING. Now tested both ways. Same class as
the two untested refuse branches the review flagged, which are also covered now.

FOUR CLAIMS RETIRED, all of the shape this issue is about — text that reads as checked
and is not:

* "18 tests fail" for the corrected-double mutation is 21 now, because rounds 3-4 added
  three fence-dependent tests. Broke a number while documenting broken numbers. Both
  citations now give the range and lead with the invariant.
* "measured: 4, 2, 9, 5, 10" first-page timeline events are 7, 5, 9, 6, 10 today.
  Timelines grow; the figures are gone and the invariant stated instead — a PR is created
  by a push, and a push is an event, so page 1 is never empty.
* The strict test's docstring said "RAW TEXT" while the test reads parsed `run:` scalars,
  with a dead `raw =` assignment left behind (a new ruff F841).
* `len(bodies) >= 3` had zero slack: deleting the optional jq-preflight step reddened it
  with a message asserting the classifier had not been examined, which was untrue. The
  length assertion already carries the property, so the count only needs to be non-empty.

Also: dead `_workflow_expression_fields_text` removed; assertion C's regex now catches
`function foo {` as well as `foo() {`; the page-2 refusal says a human verdict clears it.

Verification: 460 green. Five more mutations as intended — deleting the history page-2
check (red), accepting an unreadable one (red, after the coverage gap was closed),
accepting a garbage page 2 in read_existing_verdict (red), hiding the marker write behind
`function mk {` (red), and the earlier twenty-one still hold. Live re-probe on this body
follows.

Refs: #751
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 19:42:58 +02:00
timothyandClaude Opus 5 951dae26a9 fix(751): the truncation guard I added was DEAD CODE — the page cap is 50, not 100
Third review round, cut short by a transport hang after ~11h, but it had already found
the thing that mattered: the guard added last round could never fire.

`read_existing_verdict` asks for `limit=100` and refused when the page came back with
100 rows. This instance caps `limit` at the server-wide `MAX_RESPONSE_ITEMS`, MEASURED
AT 50 — `/issues?limit=100` returns 50 items. A response can therefore never carry 100
rows, so the comparison was unreachable and the hole it was written for was still open.

The sting is that the repo already knew. `scripts/pr-changed-files.sh`, two test files
and `ci.script-tests-job` all document that Gitea caps `limit` at `MAX_RESPONSE_ITEMS`
(50 in the PR #619 measurement). The review found it by grepping this codebase, not
upstream. Writing a guard against a constant the repo had already measured as wrong is
the same failure as the unfaithful test double two rounds ago: a number believed rather
than checked.

So this is now the THIRD guard for one hole, and the first two were both no-ops:

  1. `.statuses | length` vs `.total_count` — `total_count` is the count for the PAGE
     RETURNED, not the commit (`?limit=1` on a 6-context head gives
     `len=1, total_count=1`). Equal by construction.
  2. "refuse when the page is full at 100" — dead code, as above.
  3. Ask the server. Completeness is needed ONLY to justify "no verdict exists on this
     head", so when the row is absent from page 1 the job reads PAGE 2, and refuses if
     it carries anything. Cap-independent: no reconfiguration re-breaks it, and nothing
     is hardcoded that a measurement could contradict.

Measured to make sure page 2 is real rather than assumed: `?limit=3&page=2` on
3aed43c6 returns three further rows, and `page=9` returns the same `statuses: null`
terminator the timeline uses.

The probe is skipped when the row IS on page 1, because there is nothing to learn — the
combined endpoint returns the latest status per CONTEXT, so a context cannot recur on a
later page.

TEST-DOUBLE FIDELITY, again the fiddly part. The stub now honours `page`, and that guard
had to go BEFORE the read-counting modes: `appears-on-read:N` counts how many times the
job has LOOKED at the status, and the completeness probe is part of the same look, not a
further one. Letting it increment those counters shifted "the verdict appears on read N"
by one and broke three mid-run-race tests — a false red that would have been easy to
"fix" by adjusting the expected counts, which would have quietly destroyed what those
three tests measure.

Verification: 455 green. Four more mutations, each as intended — deleting the probe
(red), accepting a non-empty page 2 (red), refusing even on an EMPTY page 2 (red, the
deadlock control), and reverting the twin `statuses: null` gate (48 red). Twenty-one
mutations across four rounds.

Refs: #751
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 19:22:50 +02:00
timothyandClaude Opus 5 46ec532745 fix(751): re-review round — a truncation hole, and the tests that closed findings needed closing
Cross-family re-review of the previous fix commit. It did NOT pass, and it was right
not to: the round that fixed the reviewers' findings introduced two of its own, both in
the tests written to close them. That is this file's recurring shape, and it is the
reason the fix commit gets re-reviewed rather than the initial diff only.

TRUNCATION (High). `read_existing_verdict` asks for 100 statuses and never checked
whether the page was full. If a head ever carried more contexts than that, an existing
`review-verdict/h10` could fall off page 1, the job would conclude no verdict exists,
and it could post an exemption `success` over a human `failure` — the worst thing this
gate can do. Six contexts exist today, so this guards a future shape, not a live bug.

BUT THE PROPOSED GUARD WAS A NO-OP, and measuring is what showed it. The review asked
for `.statuses | length` compared against `.total_count`. On this instance `total_count`
is the count for the PAGE RETURNED, not for the commit: on 3aed43c6 (6 contexts),
`?limit=1` gives `len=1, total_count=1` and `?limit=3` gives `len=3, total_count=3`.
The two are equal by construction, so that check would have read as a completeness
proof while proving nothing — and it would have been the second guard in this file to
look like a check and not be one. What IS observable is a page at the requested limit,
which means "maybe more", so that is now treated as unreadable: post nothing, leave the
required check absent. The stub mirrors the per-page `total_count` deliberately, so the
new test cannot pass for the wrong reason either.

THE TESTS THAT CLOSED THE LAST ROUND'S FINDINGS:

* The behavioural guard test — added to answer "a bare `exit 1` substring is satisfiable
  by dead code" — extracted the two marker lines BY TEXT and ran them alone. That
  passes even if the write is moved into a function nobody calls: the extractor finds
  the text, runs it at top level, the marker appears, and the test reports the guard
  proven while production writes no marker. It now executes the classify body's real
  PREFIX down to and including the write, which reproduces the production control flow
  instead of a reconstruction of it. Mutation: move the write into an uncalled function
  -> RED (it previously passed).
* The anti-vacuity check — added to replace an over-broad assertion — hand-counted
  `run:` keys with a regex that only matched an indented `run:` starting `|` or `>`. It
  false-redded legal spellings (`- run: |`, a single-line `run: echo ok`) and could
  count a `run: |` sitting inside a heredoc. Hand-parsing YAML to validate a YAML parse
  is the wrong shape: it adds a second, worse parser whose every disagreement is a
  false alarm, and a red here blocks all merges. Now asserted on CONTENT — the walk
  reached >=3 bodies and one over 5000 chars.

FALSE RED, THIRD INSTANCE IN THIS FILE. The repo-wide expression test scanned raw file
text, so a delimiter in an inert top-level YAML comment redded the repo even though the
runner never evaluates it. It now scans PARSED scalars: PyYAML drops YAML comments,
while a `run:` body is itself a scalar and keeps its SHELL comments — which is exactly
the distinction that matters, since inside a `run:` scalar a comment is not inert.
Verified in both directions: an inert top-level comment passes, the same payload in a
run-body comment still reds.

Also: the `total_count` zero check now requires the JSON TYPE to be number — `jq -r`
renders `0` and `"0"` identically, so a text compare accepted a corrupted
`"total_count": "0"` as "no statuses".

Verification: 455 green. Six further mutations, each landing as intended — the uncalled
function (red), an inert YAML comment (PASSES, no false red), the same payload in a
run-body comment (red), accepting a full status page (red), comparing total_count as
text (red), and breaking the YAML walk's job key (red). Seventeen mutations across the
three rounds.

Re-probe of both live controls follows on this body; the previous probe evidence was
taken before this commit and no longer describes what would merge.

Refs: #751
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 19:22:50 +02:00
timothyandClaude Opus 5 edd8d3d9c9 fix(751): review round — the twin null-shape, a governance self-exemption, and four over-claims
Two independent cold reviews (a cross-family GPT-5.6 pass and an isolated Opus pass).
Neither found a path to a green `review-verdict/h10` on an unreviewed head. Both found
real defects BESIDE the fix, which is the failure mode this file keeps producing.

THE TWIN, and the reason not to trust "I fixed the two I could see". `GET
/commits/{sha}/status` returns `statuses: null` — not `[]` — for a head with no
statuses yet: `{"state":"pending","total_count":0,"statuses":null}`, measured on PR
#739's head. `read_existing_verdict` gated on `.statuses | type == "array"` and took
its `exit 1` path, posting NOTHING. Fail-closed, but the user-visible outcome is the
one this issue is about: an exempt PR with no status and, since #743, no bypass. Its
double printed `{"statuses": []}` at all three no-verdict sites, so that branch was
unreachable in the suite — the same unfaithful-double story as the timeline, one
function over. `null` is accepted only when `total_count` is 0, so a body that merely
lost its array is still refused and an existing verdict is still protected. Swept
`scripts/pr-changed-files.sh` too: `pulls/{n}/files` returns `[]`, unaffected. The
generalisable rule is that a nil Go slice serialises to `null`, so every list-shaped
field on this API is suspect and only a per-endpoint measurement settles it.

A GOVERNANCE SELF-EXEMPTION, reachable again precisely because this change works.
`DOCS_ONLY` matched `CLAUDE.md` and `AGENTS.md` — the documents that DEFINE the
completion protocol, the merge-consent convention and the H10 rule. Driving the real
classify body with a lone `CLAUDE.md` change produced `review-verdict/h10=success`.
Protecting `.claude/` while the file specifying what it enforces stayed exemptible is
the same self-exemption the header rules out, one directory over. Both added to
PROTECTED; `README.md` deliberately not (ordinary prose, no enforcement).

FOUR OVER-CLAIMS, corrected rather than defended:

* The repo-wide expression test does NOT catch "any payload that cannot evaluate".
  It checks the HEAD TOKEN of each dotted path. `${{ github.ref == }}` and
  `${{ …head.sha + }}` pass; so does a renamed output, since tokens after the first
  are skipped by design. Claim corrected in the docstring, `docs/ci-cd.md` and the
  record. The test is kept permissive on purpose: a red here blocks every merge.
* The strict test's anti-vacuity half banned expressions ANYWHERE outside
  `with:`/`env:`, so the standard `if: ${{ always() }}` spelling and even a delimiter
  in an inert top-level comment went red — a guard more dangerous than its target.
  Replaced with the honest property: the YAML walk saw every `run:` body it declares.
* The `if:` assertion demanded the bare `always()` exactly; now normalised, since the
  wrapped form is identical to the runner.
* `exit 1` was matched anywhere in the guard body, so an unreachable
  `if false; then exit 1; fi` satisfied it while the real branch said `exit 0`. Now
  required INSIDE the missing-marker branch — and the new behavioural test settles it
  properly by EXECUTING the guard body both ways.
* The record asserted a repo-wide obligation to guard consequential steps. It is not
  repo-wide: `docker-build.yml`'s `test`/`migrations` are also required contexts and a
  dropped step there is fail-OPEN (green having done no work), strictly worse than
  here. Scoped to this file and tracked as #756 rather than asserted as done.

Also: comments in both files still said it was unestablished whether a later step runs
after a drop — runs 1863/1866 established it, so they now record the measurement; a
cited test name that never existed; `kind` leaked to global scope; a mangled comment
wrap; and an already-false "one event on page 1".

Hardening of my own: `null` now counts as exhaustion only from page 2 ON. Every real
PR's first page carries events (4, 2, 9, 5, 10 across #752/#753/#749/#739/#717), so a
terminator on page 1 means no page was ever read, and certifying "no retarget" from a
response we cannot explain is the one thing the fence exists to refuse. Narrows rather
than closes it: a wrong `null` on page 3 still reads as exhaustion.

Verification: 137 in this file / 452 total green; ELEVEN mutations each red —
reintroducing the defect, deleting the guard, deleting the marker write, removing
`if: always()`, `exit 1`→`exit 0`, a delimiter in the guard body, the fence gate (20
red), the TWIN gate (70 red), dropping the governance paths, accepting a null first
page, and diverging the marker path between the two steps.

Refs: #751
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 19:22:50 +02:00
timothyandClaude Opus 5 40b3747434 fix(751): the fence never trusted its count — a page past the end is null, not []
The scratch-base probe found a SECOND, independent reason `review-verdict/h10` was
never posted automatically. Fixing the dropped step alone would NOT have restored the
exemptions.

`count_retargets` pages `/issues/{n}/timeline` and trusts its count only on a
validated empty page, gated on `type == "array"`. But a page past the end of that
endpoint is the JSON value `null` — measured at Gitea 1.27.1 on PR #752, four bytes —
so the real terminator read as UNREADABLE. The walk never reached a validated empty
page, `rt_ok` was never `yes` for ANY pull request, and the fence therefore withheld
EVERY exemption `success`. Renovate and docs-only PRs got no status at all: the same
user-visible outcome as the dropped step, by a completely unrelated route.

The instance is not consistent between endpoints — `/issues/{n}/comments` returns `[]`
when empty — so both shapes terminate the walk now, and the regression test is
parameterised over both. The type is read as a VALUE (`case` over `jq -r 'type'`)
rather than through `jq -e`, whose exit-status semantics already bit this workflow at
jq 1.6 (#647).

TWO REASONS THIS LOOKED DELIBERATE RATHER THAN BROKEN, both worth generalising:

* It had never run. This fence shipped in 8f6d4f443 — the same commit whose prose
  comment stopped the classify step executing at all. Merging a guard and first
  executing it are different events, and only the second tells you anything.
* The test double asserted the wrong shape while claiming to be measured. Its comment
  read "Real shapes, measured on this instance and deliberately mirrored" and it
  printed `[]` past the end, so the `array`-only gate was never exercised by the suite
  either. Correcting the double and restoring the old gate turns 18 TESTS RED — every
  one of them had been green for the wrong reason. A fidelity claim in a double is an
  assertion and it decays like any other.

The new test asserts the POSTED STATUS, not the log: on the real probe run the log
said `Decision: state=success` and the job still posted nothing, so the decision and
the write are separate events and only the write is what a merge reads.

Also here, both found while editing this code:

* `ci.verdict-write-retarget-fence` stated this as a narrow residual ("a timeline over
  the 20-page cap can never be exempted") when the behaviour was universal. Corrected
  in place rather than left as a checked-looking claim that talks the next reader out
  of verifying.
* The workflow cited `ci.paged-endpoint-completeness`, a key that has never existed as
  a record anywhere. Repointed at the record that actually owns this walk.

Marker hardening from the probe: `RUNNER_TEMP` is `/tmp` on this runner, not a private
per-job directory, so the start marker is now keyed on the run id and attempt. The
lane starts a container per job today, which makes a fixed name fresh in practice, but
that is a property of the lane and a stale marker would make the guard PASS on a run
whose step was dropped — the exact silent pass it exists to remove.

Verification: 446 passed; M7 (revert only the type gate, keep the corrected double) →
18 red. Probe evidence in the issue.

Refs: #751
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 19:22:50 +02:00
timothyandClaude Opus 5 2bdb6c44e4 fix(751): a stray expression delimiter in a COMMENT killed the verdict gate
`review-verdict.yml`'s classify step stopped executing on 2026-08-03 and the job
reported `success` anyway, so `review-verdict/h10` — the branch-protection-required
status — was posted by nothing but a human hand for three days, and both exemption
classes (Renovate-manifest, docs-only) silently stopped working.

The cause is one token in prose. The #706 note explaining why a concurrency group
does not work here quoted a `concurrency:` snippet containing a PR-number expression
as an ILLUSTRATION, inside a shell comment. A shell comment is not inert there: the
runner scans the whole `run:` scalar for the expression opener before bash sees it,
and one occurrence makes it rewrite the ENTIRE body into a single `format(...)` call.
That rewrite is all-or-nothing, so a payload that does not parse — `pr number` does
not — fails the interpolation of the whole scalar, and the runner then DROPS THE STEP
AND CONCLUDES THE JOB GREEN. The prose documenting a fix disabled the fix.

`git blame`/`git log -S` put the line in 8f6d4f443 (2026-08-03), which dates the
outage precisely rather than "present in run 1832, not bisected further back".

Three changes, deliberately different in kind:

* The prose no longer writes the delimiters. It names the expression instead.
* The classifier writes a start marker and a new `if: always()` step FAILS THE JOB
  when it is missing. This is the half that generalises: the delimiter was one bug in
  one comment, but a dropped step concluding `success` is what made it cost three
  days behind a green tick. It asserts execution STARTED, never completed — the
  classifier has several legitimate `exit 0` abstention paths.
* Two static guards in scripts/tests/test_pr_changed_files.py: no expression
  delimiter in ANY `run:` body of the gate file (absolute, because a dropped step
  here is a dead merge gate and its bodies are ~700 lines of prose), and repo-wide,
  every expression payload must name a real context or function (permissive, because
  other workflows interpolate into `run:` legitimately). The second catches the class
  — a payload that cannot evaluate, wherever it appears.

Note every pre-existing workflow-shape test reads `_code_lines()`, which strips
comments. That is correct for what it was for, but it encodes the assumption this bug
falsifies: inside a `run:` scalar a comment CAN change behaviour. The new strict test
reads the raw scalar for that reason and must never adopt `_code_lines`.

Blast radius, audited: PR #739 (docs-only) merged 2026-08-05 with ZERO commit
statuses on its head, and got in only because admin force-merge was still enabled.
#743 disabled that on 2026-08-06, so the workaround that absorbed this bug is gone —
the next docs-only or Renovate-manifest PR would be permanently stuck. The two
Renovate PRs in the window escaped by timing, merging minutes before the bad commit.
Non-exempt PRs were unaffected throughout: humans posted their verdicts by hand.

Verification: all six mutations red, restored tree green — reintroducing the exact
defect (strict + general tests), deleting the guard step, deleting only the marker
write, weakening `if: always()`, turning the guard's `exit 1` into `exit 0`, and
putting a delimiter in the guard's own body. 129 passed on the fixed tree.

Still to establish, and NOT claimed here: that the runner executes a LATER step after
dropping an earlier one. The #751 evidence cannot say — the classifier was the job's
last step, so there was never a subsequent step to observe. A scratch-base probe with
a negative control answers it next; if the runner drops the rest of the steps too,
this guard is inert and the body has to move into `scripts/`.

Refs #751

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 19:22:50 +02:00
renovate 82cf300f50 chore(deps): update dependency scriban.signed to 7.2.6
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 15s
PR Gates / Docs update reminder (pull_request) Successful in 17s
PR Gates / decisions lifecycle (pull_request) Successful in 22s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m44s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m9s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m46s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m30s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 10s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 8s
review-verdict/h10 Exempt: authored by the 'renovate' bot account, touches no protected path, and changes only dependency manifests
Review verdict / Set review-verdict status (pull_request_target) Successful in 45s
2026-08-10 03:03:27 +00:00
timothyandtimothy 9881d1ff81 fix(754,757): declare graphicsElementIds + padToNearestMinute, and pin every MCP tool to its OpenAPI contract (#760)
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m56s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 7m1s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 6m51s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m37s
Co-authored-by: Timothy <timothy@noreply.gitea.tblindustries.be>
2026-08-06 20:42:37 +00:00
timothy 3aed43c6de fix(743): make review-verdict/h10 unskippable — disable direct push to main + admin force-merge; fix(719) tag-only push (#749)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 35s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 43s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 49s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m20s
2026-08-05 22:58:16 +00:00
timothyandClaude Opus 5 f822e4737c docs(743): label the second attested claim, close the survivor list, state the rule in CLAUDE.md
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 14s
PR Gates / Docs update reminder (pull_request) Successful in 34s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 40s
Review verdict / Set review-verdict status (pull_request_target) Successful in 8s
PR Gates / decisions lifecycle (pull_request) Successful in 23s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 24s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m52s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 6m13s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m29s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m25s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
review-verdict/h10 Review-verdict: MERGEABLE @ f822e47 (base: main)
Round 3 returned MERGEABLE with three LOW documentation findings. Batched
before posting the verdict, since a new sha voids both the CI run and the
verdict.

- `ci-cd.md` labelled the unprobed half of the `enable_push` bullet but stated
  the `block_admin_merge_override` counterfactual flatly one bullet below —
  the same measured-vs-attested flattening round 2 fixed, one site over. Now
  labelled, with why it was not probed (verifying it means merging an
  unreviewed PR).
- `release.verdict-status-check` said "what survives is the forgery list
  above". That record's job is enumerating survivors, so an unqualified "what
  survives is X" reads as exhaustive — and it omitted the admin residual, which
  is a SKIP route rather than a forgery one. Added.
- `CLAUDE.md` never learned the rule. It is the always-read surface, and it
  still framed a direct `git push origin main` as a live path while describing
  a docs-only *push* exemption for a push the server now refuses. My corpus
  sweep covered `docs/` and missed the file that carries the docs-update rule.

Note on what remains unverified rather than closed: neither direction of
`block_admin_merge_override` was measured, and whether Gitea treats an ABSENT
required context as blocking (versus satisfied) is asserted by our docs but
not proven — the combined status on this PR reads `success` with
`review-verdict/h10` absent. Both belong to #747's re-verification sweep.

Verification: 441/441 script tests; decisions-validate OK.

refs #743

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 00:35:41 +02:00
timothyandClaude Opus 5 6af65ba5c5 fix(743): re-tense the third stale site, and pin the two surviving mutants
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 35s
PR Gates / Docs update reminder (pull_request) Successful in 42s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m23s
PR Gates / decisions lifecycle (pull_request) Successful in 1m25s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m58s
Review verdict / Set review-verdict status (pull_request_target) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 27s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m52s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 16m42s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m0s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Skipped
Round 2 of review. One blocking finding, and it is the same defect class as
round 1's: a present-tense claim that this PR falsified.

`release.verdict-status-check` — the record ABOUT the h10 status check — still
said "direct pushes to `main` are server-side permitted, so the gate can be
skipped without forging anything". A reader resolving that key from the catalog
would conclude the control does not exist. Round 1 corrected `ci-cd.md` and
`ci.actions-credential-scoping` and I stopped at the two sites I had edited,
instead of sweeping the corpus by SUBJECT. Swept properly this time
(`server-side permitted`, `bypassable`, `without forging`, `push whitelist`,
`enable_push`): this was the only remaining stale site.

Test gaps the reviewer found by mutation testing, now closed. Both mutants
SURVIVED the suite as shipped — the round-1 fixes were correct but unpinned:

- dropping `|| [ -n "${_h11_local_ref:-}" ]` → an unterminated final line is
  dropped. Two directions, and the dangerous one is not the obvious one: a
  dropped *branch* line leaves only tag refs and grants the exemption to a push
  containing a branch. Both pinned.
- dropping `[ -t 0 ] ||` → the hook hangs forever on an interactive run. Pinned
  with a real pty and an explicit timeout, so a regression fails cleanly rather
  than hanging a CI job. Verified the mutant is killed by exactly that test
  (and that it dies via the timeout, 32s).

Also from review, non-blocking:

- `ci-cd.md:951` cited `enable_push: false` alone as what closed #743 — the
  precise thing the new record says never to do, since the force-merge route
  also skipped the gate with no forgery. Now cites both fields.
- `ci-cd.md` flattened measured and source-attested into one 403: only the
  contents API was probed; the web editor/upload/apply-patch paths share the
  predicate but were not. Separated.
- `format-as-you-touch-rebase` still said "the documented sequence" and
  "always" for the release-cut behind-ness. `docs/ci-cd.md` documents the tag
  step, not the release-notes-PR flow, and the frequency is attested by one
  observed cut. Attributed to #719 instead.
- Documented the operator recovery path. `block_admin_merge_override: true`
  removes the `force_merge` escape that used to unstick a wrongly-red required
  context — that escape WAS the bypass, so it is gone by design, and the
  recovery (fix the status; last resort PATCH the field, merge, set it back)
  needed to be written down rather than left implicit in a residual.

Verification: 441/441 script tests; decisions-validate OK; PyYAML parses all
193 records; both mutants confirmed killed and the hook restored byte-identical.

fixes #719

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 00:10:45 +02:00
timothyandClaude Opus 5 6d80343320 fix(743): close the admin force-merge bypass; the push half alone was not enough
Independent review found the record repeated on the merge path exactly the
mistake it had just diagnosed on the push path.

The push argument was: a whitelist naming `timothy` closes nothing, because
`timothy` is the identity every credential already holds. The merge path had
the identical shape and went unchecked — `block_admin_merge_override` defaults
to `false`, so `CanBypassBranchProtection` returns true for a repo admin and
`POST /pulls/{n}/merge` with `force_merge: true` merges straight past a missing
or red `review-verdict/h10`. One API call, no forgery, no PATCH — cheaper than
the push route this change had just removed.

So `enable_push: false` alone did NOT make the gate load-bearing, which is
what the record's headline sentence claimed. `main` now carries both fields;
they are one control and neither is citable alone.

An admin-shaped control that exempts the only admin exempts everybody.

Other review findings addressed:

- H11's owning record (`release.format-as-you-touch-rebase`) now documents the
  #719 tag-only carve-out. It is a narrowing of an existing convention, so it
  amends that record rather than adding a new one — including the two details
  that are easy to regress (the .husky/pre-push forwarding, without which the
  exemption is dead code the unit tests still pass over; and the at-least-one-
  ref guard against vacuous exemption).
- The record now states which write surfaces were enumerated and how each was
  established — contents-API refusal is MEASURED here (403 `user cannot commit
  to repo`), apply-patch/revert/cherry-pick are source-attested only. The
  admin force-merge bypass is likewise marked source-attested, not probed:
  probing it means merging an unreviewed PR.
- prepush-rebase-check.sh: process a final ref line with no trailing newline
  (previously dropped, which silently reinstated the #719 block), and skip the
  stdin read on a TTY so an interactive run does not hang.
- Corrected a citation the review caught: docs/ci-cd.md documents the tag step,
  not a release-notes-PR flow. Cite #719 for the observed flow instead.

Also fixed a frontmatter break this round introduced: a `: ` inside the
unquoted `rule:` scalar. PyYAML rejected it while the dependency-free reader
accepted it, so only `scripts/tests` caught it.

Verification: 438/438 script tests pass; decisions-validate OK; PyYAML parses
all three touched records.

refs #743 #719

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 23:52:28 +02:00
timothyandClaude Opus 5 b91707b707 fix(719): exempt tag-only pushes from the H11 branch-freshness check
H11 (.claude/hooks/prepush-rebase-check.sh) refuses to push a branch that is
behind origin/main. It fired on tag-only pushes too, breaking every release
cut: docs/ci-cd.md's "Cutting a release" flow lands a release-notes commit
via PR and then tags that merge commit, so the local branch is always one
commit behind origin/main at tag time. A tag push cannot revert anyone's
merged work, which is the failure H11 exists to prevent, so skip the
freshness check when every ref being pushed is under refs/tags/.

.husky/pre-push previously consumed pre-push's stdin ref lines and forwarded
them only to prepush-donewhen.sh; prepush-rebase-check.sh got none. Forward
the captured $_prepush_refs to it too, or the new logic is dead.

Guard against the vacuous-truth case explicitly required by #719: "all
pushed refs are tags" is trivially true over zero ref lines (manual run,
forgotten forwarding), which would silently disable H11 for every push.
Require at least one parsed ref line before granting the exemption.

Adds scripts/tests/test_prepush_rebase_check_tag_exemption.py using real
local git repos (bare origin + a work tree pushed one commit behind it) to
exercise git fetch/merge-base/rev-list against a genuinely-moved origin:
tag-only allowed, branch-only still blocked, mixed branch+tag still
blocked, and zero ref lines still blocked (the vacuous-truth guard).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 23:33:10 +02:00
timothyandClaude Opus 5 691a7acc77 fix(743): disable direct pushes to main so review-verdict/h10 is on the only path to main
Gitea evaluates `status_check_contexts` when it MERGES a PR. A direct
`git push origin HEAD:main` never consults them, so the whole h10 gate was
skippable with no forgery — strictly cheaper than every route enumerated in
#697. `main` now carries `enable_push: false`.

Measured on this instance (Gitea 1.27.1) against a throwaway `probe-743-*`
rule rather than against `main`:

  enable_push: false                      -> push by timothy (site admin)
                                             REFUSED, pre-receive hook declined
  enable_push_whitelist + ["timothy"]     -> identical push SUCCEEDED

That second line is why this is a DISABLE and not a whitelist: #743 offered the
two as interchangeable, but the only write accounts here are `timothy` (site
admin) and `renovate`, and every credential in the threat model — agent
sessions, PATs, the injected GITEA_TOKEN — acts as `timothy`. A whitelist
naming `timothy` would have ticked the box and closed nothing.

Then demonstrated on `main` itself, per the issue's Done-when: a direct push
was refused, and a tag-only push from the same worktree succeeded (tags are
governed by `tag_protections`, which is empty). The release cut is unaffected.

What this closes: the write-only credential routes — the injected GITEA_TOKEN,
RENOVATE_TOKEN, any non-admin collaborator PAT. What it does NOT close: an
admin credential can PATCH the protection off, push, and restore it. Recorded
as an accepted residual rather than implied to be covered.

Also corrects two claims the probe contradicted, and one that the mid-session
Gitea upgrade (1.25.4 -> 1.27.1) invalidated:

- ci-cd.md and ci.actions-credential-scoping both said "a push whitelist would
  close more of this class than the 1.26 upgrade". The whitelist form closes
  nothing here; corrected in place.
- ci.actions-credential-scoping's rule said "do NOT add a `permissions:` key
  while this instance is below Gitea 1.26.0". That precondition no longer
  holds at 1.27.1, so the directive now misleads. Corrected — while noting the
  consequence is still UNVERIFIED: `/api/v1/settings/actions` 404s at 1.27.1,
  so whether `permissions:` binds here was not probed. The upgrade alone is
  not evidence the constraint works.
- That record's 1.25.4 measurements are now dated, not current. Flagged as
  such rather than silently re-pinned to a version they were never taken on.

#743's fourth box (docker-build.yml `persist-credentials: false`) is decided
in the record and deliberately not done here: two of its checkout steps run
`git fetch ... || true` feeding the changed-file skip logic, so a credential
regression would be silent rather than loud. Drop the `|| true` masking first.

fixes #743

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 23:31:45 +02:00
timothy 08e95f9ec1 fix(697): scope CI's registry credential so head-resolved workflows cannot forge review-verdict/h10 (#745)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 47s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 48s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m33s
Closes the credential half of #697. `REGISTRY_PASSWORD` was the admin account's
basic auth, handed to head-resolved PR code by docker-build.yml; it is now a PAT
scoped `write:package` + `read:repository`.

Verified on Gitea 1.25.4: registry push SUCCEEDED, status GET 200, status POST
REFUSED 403 (required=[write:repository]).

Does NOT close the class. Surviving routes, all recorded: RENOVATE_TOKEN (#742),
the injected GITEA_TOKEN (server-management#714), a collaborator's own token, the
`v*` tag push, and — making all of them unnecessary — direct pushes to `main`,
which are server-side permitted (#743). ci-image.yml's trigger filter was
attempted, reverted, and split out as #744.

Three cold adversarial review rounds: BLOCKED, BLOCKED, BLOCKED, then MERGEABLE.

fixes #697
2026-08-05 19:56:52 +00:00
timothyandClaude Opus 5 b91939e5c4 fix(697): correct the overclaims three adversarial review rounds found
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 22s
PR Gates / Docs update reminder (pull_request) Successful in 26s
PR Gates / decisions lifecycle (pull_request) Successful in 41s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 56s
review-verdict/h10 Review-verdict: MERGEABLE @ b91939e (base: main)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 59s
PR Gates / Script tests (pytest) (pull_request) Successful in 1m56s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 18m6s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 20m3s
Review verdict / Set review-verdict status (pull_request_target) Successful in 5s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m4s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Round 1 BLOCKED (1 Blocker, 4 High, 3 Medium, 2 Low); round 2 BLOCKED on the fix
(1 Blocker, 2 High, 4 Medium, 2 Low); round 3 BLOCKED on one Medium. Every
finding re-verified against the live instance before acting.

ROUND 2 — the blocker was self-inflicted and the local gate could not see it.
Adding `branches: [main]` to ci-image.yml re-points `ci-image-pin`'s `expected`
at the editing commit, staling all five `container:` pins and failing that
BLOCKING job — for a change altering zero bytes of the toolchain image.
Reproduced: expected=ed9dd6254 vs pins=32747a0. Reverted here (the commit was
amended, so no commit on the branch touches that path) and filed as #744.

That edit had also FALSIFIED its own justification: branch publishing IS
load-bearing — docs/ci-cd.md documents the rebase-recovery flow as "let
ci-image.yml publish :<short sha>, then bump the pin", which is how you satisfy
ci-image-pin from inside a PR. Reverting also keeps three trigger descriptions
true (ci-cd.md:1043, the recovery flow, pr-checks.yml's escape-hatch comment).

Also fixed:
  - gate-trigger-base-resolved.md was the file round 1's fix did not touch, and
    still said "no workflow route retains human provenance" — false, since a
    PR-added workflow can reference RENOVATE_TOKEN. Its `rule:` also kept the
    race framing, and `rule:` is what the catalog and MemPalace mirror.
  - `mechanics:` claimed "independent review confirmed no CI consumption breaks".
    It confirmed no such thing. Round 3 then caught the REPLACEMENT sentence
    making the same class of error: only the `container:` pull is exercised by a
    PR, because `build` carries `if: github.event_name != 'pull_request'` and
    cache-to/cache-from live only there. Those and the base-image pull first run
    on the post-merge push to main — a wrong inference reddens main, not the PR.
  - A fourth surviving route was unnamed: docker-build.yml publishes :prod from a
    `v*` tag push and a tag may point at any commit (tag protections are empty).
    "three surviving routes" became "at least these" — a count reads as complete.
  - Unmarked inferences, a "three later sections" that undercounted four, a
    dangling "the two items below", and a #744 rationale that stated the pin toll
    without its documented remedy.

Local gate: 432 script tests pass; `decisions_validate.py --base origin/main
--head HEAD` and `build_decisions_catalog.py --check` both exit 0; ci-image-pin
recomputed by hand and matching the pinned commit. The record is 62 prose lines
against a 60-line ceiling that is a `::warning::` by design (#520) — the blocking
constraint is the 2-25% minority band, currently 10.8%.

Refs #697, #742, #743, #744.

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 21:23:43 +02:00
timothyandClaude Opus 5 e298bb291e fix(697): scope CI's registry credential so head-resolved workflows cannot forge review-verdict/h10
`REGISTRY_USER`/`REGISTRY_PASSWORD` were the ADMIN account's basic auth, and
`docker-build.yml` triggers on `pull_request` — head-resolved — so a PR's own
code was handed instance-admin credentials. Basic auth carries no scope, so the
same secret that pushes an image administers every repo on the instance and can
POST `review-verdict/h10`, the required context that makes merge-consent derived
rather than assertable. Refs #697.

Fixed at the credential, not the triggers: patching triggers enumerates
instances of "a ref-resolved workflow obtains status-capable credentials", and
adding a new workflow file is itself a route. `REGISTRY_PASSWORD` is now a PAT
scoped `write:package` + `read:repository`.

Verified on Gitea 1.25.4, not inferred:
  - registry push of a probe tag SUCCEEDED (cleaned up, confirmed 404)
  - GET /commits/{sha}/status SUCCEEDED (what ci-detect-already-validated.sh does)
  - POST /statuses/{sha} REFUSED, HTTP 403:
    required=[write:repository], token scope=write:package,read:repository

Scope of what this closes, stated without overclaim. It closes the instance-wide
admin escalation and that credential's durable forgery route — durable because a
status POSTed with a USER credential carries a real `creator` and is inherited as
a human verdict, while an Actions job's carries `creator: null` and is re-derived.
It does NOT close the class. Three things survive it:

  - `RENOVATE_TOKEN` is a `write:repository` PAT of a real bot account in the
    SAME secret store, so it also posts with non-null `creator`. It cannot be
    scoped down (Renovate needs repo write), and secrets are a per-repo store
    that any PR-added workflow can reference. Closing this needs the provenance
    check tightened to an allow-list of approved reviewers.
  - Every job still receives a write-capable `GITEA_TOKEN`. `permissions:` YAML
    is a no-op before Gitea 1.26.0 and no `app.ini` lever exists at any version;
    only >=1.26 with the Actions default set to Restricted binds it.
    Tracked in server-management#714.
  - Branch protection binds the context NAME, not its issuer, so any write-scoped
    personal token forges the status with genuine human provenance. Unfixable
    in-repo. `h10` is a process guard, not a security boundary against push access.

Auditing the secret STORE rather than the workflow set also surfaced
`SERVERMGMT_DEPLOY_KEY`, still present though the `bump-prod-compose` job that
used it was removed in 1b5efd7b9 — an SSH deploy key to another repo, obtainable
by any PR-added workflow, with no remaining benefit.

Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 20:34:34 +02:00
timothy d7647b6104 fix(685): AddItemsDialog resolves by search instead of windowing the whole media-library type (#741)
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 38s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 39s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 39s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 12m26s
2026-08-05 18:01:59 +00:00
timothy 7be42654fe fix(685): suppress both empty-states on error; name the addable-kind derivation
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 18s
Review verdict / Set review-verdict status (pull_request_target) Successful in 8s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m33s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m21s
review-verdict/h10 Review-verdict: MERGEABLE @ 7be4265 (base: main)
PR Gates / decisions lifecycle (pull_request) Successful in 2m7s
PR Gates / Script tests (pytest) (pull_request) Successful in 2m13s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 9m7s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 17m2s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m58s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Round-4 verification returned MERGEABLE with no new defects and no
BLOCKER/HIGH/MEDIUM. These are its remaining LOW and nits.

- "No results — try a search above." asserts a search that COMPLETED and found
  nothing, so it was false beside a failed request. Both empty-state messages
  are now suppressed on error and the role="alert" banner is the whole message,
  as that test's comment already claimed. Pinned positively and negatively so a
  refactor cannot satisfy the assertion by rendering nothing at all.

- The (typeof ADDABLE_TYPE_LIST)[number] derivation is now the named
  AddableKind, spelled once instead of twice: a third ingress into the searched
  kinds is most likely to be written by copying one of the existing two, and the
  "one list, all ingresses" property should be visible at a glance rather than
  reassembled.

- The §3b lesson cited two measured test counts, which go stale against the very
  suite they describe — a count taken before the helper had unit tests no longer
  holds now that it does. Scoped the observation to the sha it was measured on
  and replaced the counts with the invariant they were evidence for: every gate
  needs at least one test that reddens when that gate ALONE is removed.

- Documented why the error banner stays conditionally mounted while the hint's
  live region does not: role="alert" is the one live-region role screen readers
  reliably announce on insertion, so the two are correct for opposite reasons.
  The review flagged the divergence as unexplained, not as wrong.

refs #740
2026-08-05 19:40:07 +02:00
timothy d7725c274c fix(685): close the second ingress; stop the error path claiming a short query
Verification round returned MERGEABLE with all three blocking findings resolved
by measurement. These are its four remaining items.

- DEFAULT_SEARCH_KINDS was the SECOND ingress into the searched kinds and was
  not derived, so the previous commit's "enforced by the type system" claim held
  for one of two paths. A non-addable kind there typechecked clean and would
  have overstated the hint with every one of its rows dropped — the exact defect
  the derivation exists to prevent. Now derived; verified by mutation that
  adding 'Collection' to it is a compile error.

- The error path fell into the min-query guidance branch, so a valid 2-character
  query that got a 500 told the user to type at least 2 characters. That branch
  conflated "nothing searched yet" with "the last search failed". Newly
  introduced by the previous commit's error-path reset; now gated on !error and
  pinned by a test.

- The aria-live region was mounted conditionally, creating the region and its
  text in one commit — which most screen readers do not announce. It is now
  mounted unconditionally with the condition inside.

- The §3b lesson mis-stated where the duplicate gate lived: it was inside
  runSearch, the genuine single sink, NOT at one of the callers — so the rule as
  written ("put the gate in the single sink, not at each caller") described the
  revision that was rejected. Reworded to the actual lesson: the gate's home is
  the shared helper, and "it's the single sink" is not evidence it is the only
  guard. That misreading is why #685 got this wrong twice.

Declined again, with reasons: the NaN pageSize edge (faithful to the sibling
helper), the clamp test's unpinned lower bound (same), and the registry's
disclosed same-identity substitution gap.

refs #740
2026-08-05 19:20:26 +02:00
timothy b6bf94f129 fix(685): test the helper's bound; delete the masking duplicate gate
Independent review round 2 returned BLOCKED on two findings, both correct.

- The helper's bound was dead code to the suite. searchLibraryBrowseItems had
  zero tests, so deleting its clamp OR its gate left the whole suite green —
  while the registry note claimed a caller "cannot skip the bound". That is the
  previous round's finding relocated, not removed. It now has the three tests
  its sibling searchLibraryPickerOptions already had (clamp, gate, compile),
  plus one pinning the full-row return that is its reason to exist.

- The screen kept a second copy of the min-query check, and the two masked each
  other: the 1-character boundary test passed with EITHER gate alone, so it
  pinned nothing. The screen's copy is deleted; the helper is the sole gate.
  Measured before/after: with the duplicate present, removing the helper's gate
  left that test green; with it gone, the same removal reddens it.

- §3b contradicted itself two lines apart — the parent still said "there is no
  truncation, so there is no truncation hint" above a sub-bullet mandating one.
  Reworded so a hint is permitted, required only where bulk selection makes the
  count actionable. Same correction to the 'search-bounded' definition.

- A failed search left results/totalMatches stale, rendering a confident
  "Showing 75 of 60000 matches" beside the error banner. The catch clears them.

- Results now carry a `Results for "<query>"` heading and the guidance is keyed
  to the settled query, not the live input, so rows are never shown without
  saying which search produced them. `selected` persists across queries (correct
  for a multi-select picker); the Add button's count keeps it discoverable.

- The hint sums pre-filter totalCount against post-filter rows, which is only
  correct because every filterable kind is addable. MediaKindFilter is now
  derived from ADDABLE_TYPE_LIST, making that a compile error rather than prose.

- aria-live on the hint; the #740 doc caveat no longer overstates the typeahead
  rule as a mandate this screen violates.

Declined: the NaN pageSize edge (copied faithfully from the sibling helper) and
the registry's same-identity substitution gap (already disclosed in that file).

refs #740
2026-08-05 19:08:13 +02:00
timothy 4be3f247d8 fix(685): move the picker bound into the helper; surface the per-kind cap
Independent review round 2. Verdict was MERGEABLE with no blockers; this takes
the two recommended fixes plus the structural one it listed as a follow-up.

- The bound was caller discipline, not code: getLibraryBrowseItems does not
  clamp pageSize, so the bound was only the constant this one call site chose
  to pass, and §3b is explicit that a bound a caller can exceed is not a bound.
  New searchLibraryBrowseItems in libraryBrowse.ts owns the min-query gate, the
  pageSize clamp and the titleContainsQuery compile, returning full
  LibraryBrowseItem rows plus totalCount (searchLibraryPickerOptions' {id,name}
  shape loses the mediaType that toAddItemsRequest needs). runSearch keeps one
  early return, for the spinner only, and no longer re-implements the gate.

- The min-query guidance was keyed to the LIVE input, so backspacing below the
  gate after a search wiped the rendered rows and their checkmarks while
  `selected` and the Add button still counted them. Keyed to results.length too.

- "Nothing left to hint at" was false: each kind is still capped at
  LIBRARY_PICKER_RESULTS and totalCount was never read. This is a bulk
  multi-select add, so the cap is surfaced — per-kind totalCounts are summed and
  rendered as "Showing N of M matches" once it exceeds the rendered rows. The
  registry note and the §3b bullet are corrected to stop claiming otherwise.

- Gate boundary tested at 1 character (§3b: inclusive endpoints, or a > for >=
  slip passes the whole suite).

- The guard test's deviation loop iterates an empty list now, so it gains one
  bidirectional assertion that is non-vacuous: the set carrying an `issue` field
  must equal the set classified 'deviation'.

- The §3b bullet no longer reads as a conformance certificate: AddItemsDialog
  still lacks the seqRef and useIsMountedRef guards §3b mandates. That defect is
  PRE-EXISTING, not introduced here, and is tracked in #740.

refs #740
2026-08-05 18:46:43 +02:00
timothy 28ce8c4dfe fix(685): gate AddItemsDialog on a real query instead of windowing the whole type
AddItemsDialog.runSearch was reachable with an empty query two ways — a blank
form submit, and a kind-chip click, which called it immediately — and
getLibraryBrowseItems omits a falsy `query`, so each path degraded into an
unfiltered browse of the whole media-library type (first 50 rows, per kind)
presented as the answer with nothing surfacing the truncation. All ten
ADDABLE_TYPE_LIST entries are spa-conventions §3b Class B media-library types.

The dialog is multi-select, so §3b's SearchPicker (single-select) does not fit;
it takes §3b's constraints instead:

- no request below LIBRARY_PICKER_MIN_QUERY, enforced in runSearch — the single
  sink both entry paths route through, not duplicated per caller
- typed text compiled with titleContainsQuery rather than forwarded raw (a
  second latent §3b violation here: the search index's default field does not
  match bare title words)
- each kind bounded to LIBRARY_PICKER_RESULTS
- merged.slice(0, 50) removed — it silently dropped up to 100 of 150 fetched
  rows even for a real query

Tests assert zero requests below the gate on both paths, exactly one bounded
request per kind above it (20k-row fixture), the compiled+escaped query, and
that no fetched row is dropped. Each was verified to fail with its mechanism
removed.

The pageSize registry entry moves from `deviation` to `search-bounded`. That
leaves zero deviation entries, so the anti-vacuity assertion guarding that list
is deleted deliberately, per its own instruction.

Done-when box 4 (collection-family truncation hint) has no subject: this screen
offers no collection-family type.

fixes #685
2026-08-05 18:46:43 +02:00
timothyandClaude Opus 5 e46e2cfe68 docs(skill): Dispatcharr EPG refresh ssh'd to the wrong host
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 9m19s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (push) Successful in 17m17s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 19m36s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 9m55s
The refresh_epg_data command targeted 192.168.1.99, but Dispatcharr moved to
jazz (192.168.1.29) in #634 — it would fail with 'No such container'.

refs server-management#692

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 18:35:38 +02:00
272 changed files with 54166 additions and 1474 deletions
+7
View File
@@ -4,6 +4,13 @@
# body-diff, CI). This shim runs the structural validator over the working tree; the body-diff/no-
# vanish checks run in CI where a base/head is available. Fail-open on any tooling trouble.
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# git hook: decides by exit code, and its stdout is live progress text.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin decisions-guard "" stream || true
cd "$(git rev-parse --show-toplevel)" || exit 0
command -v python3 >/dev/null 2>&1 || exit 0 # no python -> fail-open
PYTHONPATH=. python3 scripts/decisions_validate.py
+7
View File
@@ -17,6 +17,13 @@
# This is a reminder, never a hard gate — `start` only injects context; `finish` is a one-shot Stop nudge.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin design-sync-reminder "${1:-}" capture || true
UI_RE='(^|/)web/src/.*\.(tsx|css)$'
TEST_RE='\.test\.(tsx|ts)$'
@@ -4,6 +4,13 @@
# a sibling worktree another session created apart from this session's own.
# Fail-safe: any parse trouble → do nothing (the guard stays fail-open without a marker).
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin posttooluse-worktree-marker "" capture || true
input=$(cat)
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
cwd=$(printf '%s' "$input" | jq -r '.cwd // ""' 2>/dev/null || true)
@@ -15,6 +15,13 @@
# no origin/main, HEAD unresolved -> allow. Deliberate escape: ETV_ALLOW_DIRTY_PUSH=1.
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# git hook: decides by exit code, and its stdout is live progress text.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin prepush-clean-worktree-check "" stream || true
[ "${ETV_ALLOW_DIRTY_PUSH:-}" = "1" ] && exit 0
git rev-parse --git-dir >/dev/null 2>&1 || exit 0
+7
View File
@@ -12,6 +12,13 @@
# Auth (never committed): ETV_GITEA_TOKEN or ETV_GITEA_BASICAUTH; ETV_GITEA_URL overrides the base.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# git hook: decides by exit code, and its stdout is live progress text.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin prepush-donewhen "" stream || true
# git passes "<localref> <localsha> <remoteref> <remotesha>" lines on stdin.
refs=$(cat || true)
printf '%s\n' "$refs" | grep -q 'refs/heads/main' || exit 0 # only gate pushes to main
+39
View File
@@ -9,9 +9,48 @@
# a positively-proven "behind origin/main". Deliberate exception: ETV_SKIP_REBASE_CHECK=1.
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# git hook: decides by exit code, and its stdout is live progress text.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin prepush-rebase-check "" stream || true
[ "${ETV_SKIP_REBASE_CHECK:-}" = "1" ] && exit 0
git rev-parse --git-dir >/dev/null 2>&1 || exit 0
# Tag-only push exemption (ersatztv#719): the release cut tags a commit on main while the local
# branch sits 1 commit behind origin/main, so H11 blocked EVERY release -- and its "rebase first"
# advice did not even apply, since no branch was being pushed. A tag push cannot revert anyone's
# merged work, which is the failure mode H11 exists to prevent, so skip the freshness check when
# EVERY ref being pushed is under refs/tags/. (See #719 for the observed flow.)
#
# Read pushed refs from stdin: git feeds pre-push hooks one line per ref, "<local ref> <local sha>
# <remote ref> <remote sha>" (.husky/pre-push forwards the lines it already captured). Ignore blank
# lines. VACUOUS-TRUTH GUARD: "all refs are tags" is trivially true when there are zero ref lines
# (hook run manually, stdin not forwarded, etc.) -- that would silently disable H11 for every push.
# Require at least one parsed ref line before granting the exemption; with zero lines, fall through
# to the existing branch-freshness check below (current behavior preserved).
#
# `[ -t 0 ] ||` so an interactive run does not hang waiting on a terminal: this script had no stdin
# reader before #719, and its own docs call "run by hand" a supported case. A TTY yields no ref
# lines, which is exactly the zero-line fall-through.
_h11_refs_seen=0
_h11_all_tags=1
[ -t 0 ] || while IFS=' ' read -r _h11_local_ref _h11_local_sha _h11_remote_ref _h11_remote_sha \
|| [ -n "${_h11_local_ref:-}" ]; do # `|| [ -n ... ]` also processes a final line with no trailing newline
[ -z "${_h11_local_ref:-}" ] && continue
_h11_refs_seen=1
case "${_h11_remote_ref:-}" in
refs/tags/*) ;;
*) _h11_all_tags=0 ;;
esac
_h11_local_ref=''
done
if [ "$_h11_refs_seen" = "1" ] && [ "$_h11_all_tags" = "1" ]; then
exit 0
fi
# Best-effort fetch of the latest main; offline / no network -> don't block.
git fetch origin main --quiet 2>/dev/null || exit 0
git rev-parse --verify --quiet origin/main >/dev/null 2>&1 || exit 0
+7
View File
@@ -40,6 +40,13 @@
# Fail-open by design: any parse trouble -> allow (exit 0, no output).
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-agent-model "" capture || true
input=$(cat)
tool=$(printf '%s' "$input" | jq -r '.tool_name // ""' 2>/dev/null || true)
+7
View File
@@ -3,6 +3,13 @@
# The historic 8-9-way crash was RAM starvation, not CPU load; gate on FREE RAM.
# Fail-open: if memory_pressure is unavailable/unparsable → allow.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-agent-ram "" capture || true
free=$(memory_pressure -Q 2>/dev/null | grep -oE 'free percentage: [0-9]+' | grep -oE '[0-9]+' || true)
[ -z "${free:-}" ] && exit 0
+7
View File
@@ -2,6 +2,13 @@
# PreToolUse / Bash — deny commands that violate a HARD RULE.
# Fail-open: any parse trouble → allow (exit 0 with no output).
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-bash-guard "" capture || true
input=$(cat)
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
+12 -1
View File
@@ -18,6 +18,13 @@
# the reason a commit can't happen; CI is still the backstop.
set -uo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-bom-guard "" capture || true
input=$(cat)
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
[ -n "$cmd" ] || exit 0
@@ -66,7 +73,11 @@ while IFS= read -r f; do
esac
p="$root/$f"
[ -f "$p" ] || continue
if [ "$(head -c3 "$p" 2>/dev/null | xxd -p 2>/dev/null)" = "efbbbf" ]; then
# `od`, NOT `xxd`. `xxd` ships with vim and is absent on plain Linux hosts including this repo's
# CI runner, where the command substitution yielded empty, never equalled `efbbbf`, and this guard
# therefore passed every BOM in silence. It has been fail-open on any host without vim since it
# was written. `od -A n -t x1 -N 3` is POSIX and produces byte-identical output on macOS and Linux.
if [ "$(od -A n -t x1 -N 3 < "$p" 2>/dev/null | tr -d ' \n')" = "efbbbf" ]; then
bad="${bad} ${f}"$'\n'
fi
done < /tmp/.bom-guard-files.$$
+413 -10
View File
@@ -8,8 +8,12 @@
# LATEST commit was reviewed, not a stale earlier diff (the ersatztv#242 failure mode:
# "re-review the fix commit, not just the initial PR diff").
#
# EVERY ONE OF THOSE IS A SNAPSHOT, taken when the merge tool is called. That is sound for an
# immediate merge and UNSOUND for a scheduled one: with merge_when_checks_succeed, Gitea merges
# EVERY ONE OF THOSE IS A SNAPSHOT, taken when the merge tool is called. The window is SMALL for an
# immediate merge and UNBOUNDED for a scheduled one. Small is not zero, and this comment used to say
# "sound", which is the overclaim ersatztv#778 removed: this hook returns `allow` and a SEPARATE call
# performs the merge, so a push can still land in between. The merge API accepts an optional
# `head_commit_id` that would make that call a true compare-and-set; a PreToolUse hook cannot add an
# argument, only refuse without one. With merge_when_checks_succeed, Gitea merges
# later, against whatever head is green then (ersatztv#622). So the sha-bound half of H10 is
# enforced by the SERVER, not here — `review-verdict/h10` is a required status check on `main`,
# written per-sha by scripts/post-review-verdict.sh, and a new commit cannot inherit it. This hook
@@ -18,7 +22,7 @@
# The "## Done-when" issue-body checklist is the convention (docs/decisions.md, CLAUDE.md Task
# Completion Protocol). One box is "adversarial review passed"; the others are per-issue.
# The H10 review-verdict convention: after reviewing a PR (or its latest fix commit), post a PR
# comment carrying a line `Review-verdict: <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> @ <head-sha>`.
# comment carrying a line `Review-verdict: <MERGEABLE|APPROVED|LGTM|BLOCKED|NOT-MERGEABLE> @ <head-sha>`.
#
# Decision policy — a CONSENT gate, so it does NOT fail silently open:
# - state derivable and satisfied -> grant (auto-approve: permissionDecision "allow",
@@ -40,6 +44,13 @@
# Gitea auth from env (never committed): ETV_GITEA_TOKEN (a token) OR ETV_GITEA_BASICAUTH (user:pass).
# ETV_GITEA_URL overrides the base (default: the LAN instance; a LAN address, not a secret).
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-merge-consent "" capture || true
input=$(cat)
decide() { # $1=grant|allow|deny|ask $2=reason
@@ -85,8 +96,9 @@ sha=$(printf '%s' "$prjson" | jq -r '.head.sha // ""' 2>/dev/null || true)
body=$(printf '%s' "$prjson" | jq -r '.body // ""' 2>/dev/null || true)
# --- Docs-only exemption: if every changed file is docs/process, skip the gate. ---
# The file list must be enumerated EXHAUSTIVELY, validated row by row, and bound to ONE head, or the
# exemption is unsafe. ALL of that now lives in scripts/pr-changed-files.sh — the single shared
# The file list must be enumerated EXHAUSTIVELY, validated row by row, and checked for head/base
# movement across the paging round trips, or the exemption is unsafe. (That check detects ONE-WAY
# movement only — this said "bound to ONE head" until 2026-08-28, ersatztv#803.) ALL of that now lives in scripts/pr-changed-files.sh — the single shared
# implementation, also called by .gitea/workflows/review-verdict.yml (ersatztv#649).
#
# Why it moved: this logic was written twice. This copy is ADVISORY (a failure produces a human
@@ -168,10 +180,81 @@ fi
# after which a later, successful status read could still auto-grant. A transient failure would then
# have produced a "merge gate: satisfied" message for a comparison that never happened. Every
# unreadable input here therefore falls through to a human (`ask`), never to silence.
live_base=$(printf '%s' "$prjson" | jq -r '.base.ref // ""' 2>/dev/null || true)
if [ -z "$live_base" ]; then
# RE-READ THE BASE HERE, ONCE, FOR EVERY PATH BELOW (ersatztv#778).
#
# "Below" is literal, and the one consumer ABOVE is disclosed rather than implied: the docs-only
# enumeration still runs against the snapshot `$base_ref` and can `decide allow` before reaching
# this point. That is bounded and deliberate — a docs-only match is a PASSTHROUGH to the ordinary
# human prompt, never an auto-grant, so a stale base there costs a prompt someone was going to see
# anyway. Every path that can GRANT passes through the check below.
#
# `$base_ref` above comes from the PR snapshot taken at the top of this hook, and the docs-only
# enumeration between there and here is up to forty round trips. A PERSISTENT retarget in that gap
# needs no ABA and no force-push: every base-dependent decision below would be formed against a
# branch the PR no longer targets. Checking a stale identifier is not checking — which is the whole
# of `process.check-and-use-pins-a-version`, so the guard enforcing that rule must not break it.
#
# This re-read first landed inside the scheduled-auto-merge branch only, which fixed the branch-
# protection lookup and left the #632 retarget DETECTION below still reading the stale snapshot. Cold
# review demonstrated the consequence with this repo's own fixture: scheduled+retarget denied, while
# immediate+retarget auto-GRANTED. That is the twin-missed shape — a fix applied to the path where it
# was noticed — so the re-read is hoisted above every consumer rather than duplicated into each.
prjson_now=$(gq "repos/$owner/$repo/pulls/$pr")
if [ -z "${prjson_now//[[:space:]]/}" ] || ! printf '%s' "$prjson_now" | jq -e 'type == "object"' >/dev/null 2>&1; then
decide ask "H10 merge gate: could not re-read PR #$pr to confirm it still targets '$base_ref' before checking the verdict against it. Confirm the target branch, then merge."
fi
base_now=$(printf '%s' "$prjson_now" | jq -r '.base.ref // ""' 2>/dev/null || true)
if [ -z "$base_now" ]; then
decide ask "H10 merge gate: PR #$pr reports no base branch (.base.ref), so the verdict cannot be checked against the branch it was formed for (ersatztv#632). Confirm the PR still targets the branch it was reviewed against before merging."
fi
if [ -n "$base_ref" ] && [ "$base_now" != "$base_ref" ]; then
decide deny "H6/H10 merge gate: BLOCKED — PR #$pr was retargeted from '$base_ref' to '$base_now' while this gate was evaluating. Every check formed against '$base_ref', including the changed-file enumeration and the review verdict, describes a merge that is no longer the one being requested (ersatztv#632). Re-review against '$base_now' and run: scripts/post-review-verdict.sh $pr MERGEABLE"
fi
# From here on both names are the freshly-confirmed base; they are equal by the check above.
base_ref=$base_now
live_base=$base_now
# THE HEAD IS RE-READ AT THE SAME HOIST, FROM THE SAME RESPONSE (ersatztv#803).
#
# `$sha` comes from the PR snapshot at the top of this hook, and until 2026-08-28 every later check
# consumed that captured value: the CI combined status, the `review-verdict/h10` status, and the
# verdict-comment classification were all evaluated against `/commits/$sha/status` and `--head $sha`.
# A push landing in the gap — which includes the docs-only enumeration's up-to-forty round trips —
# was therefore checked against the commit it had just replaced, and the hook would report "a
# positive Review-verdict references the current head" about a head that was no longer current.
#
# This is the SAME defect the base had until #778 hoisted the re-read above, and it is fixed the same
# way rather than a different way. Reading `.head.sha` off `$prjson_now` — the response the base
# check already fetched — costs NO extra round trip, and it keeps the two axes on ONE snapshot, so
# they cannot disagree about which moment they describe. Two separate reads would answer about two
# different instants while reading as one check.
#
# DENY, not ask, and for the same reason the `stale` verdict class denies: a head that moved means
# the verdict this hook is about to accept covers an OLDER commit, which is a state we have
# positively established rather than failed to establish. An UNREADABLE `.head.sha` is the different
# case and asks.
#
# WHAT THIS DOES NOT CLOSE, said here rather than left to be inferred. A push landing after this
# check still passes, exactly as a retarget does — the file's rule against a second re-read applies
# unchanged (see the branch-protection block below), because two reads only move the window rather
# than closing it. That residual is bounded server-side and this hook is not what bounds it: the new
# head has no `review-verdict/h10` status, and that context is REQUIRED on `main`, so Gitea refuses
# the merge (#622). The hook's job here is to stop CLAIMING a head is reviewed when it can see that
# it is not — an advisory gate that states something false is worse than one that asks.
if [ -n "$sha" ]; then
sha_now=$(printf '%s' "$prjson_now" | jq -r '.head.sha // ""' 2>/dev/null || true)
if [ -z "$sha_now" ]; then
decide ask "H10 merge gate: PR #$pr reports no head commit (.head.sha) on re-read, so whether the review verdict still covers the current head could not be confirmed. Check the PR, then merge."
fi
if [ "$sha_now" != "$sha" ]; then
decide deny "H6/H10 merge gate: BLOCKED — PR #$pr's head moved from ${sha:0:7} to ${sha_now:0:7} while this gate was evaluating. Every check formed against ${sha:0:7} — the changed-file enumeration, the CI status and the review verdict — describes a commit that is no longer the one being merged (ersatztv#803). Re-review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE"
fi
# From here on `$sha` is the freshly-confirmed head; the two are equal by the check above. Mirrors
# `base_ref=$base_now` a few lines up, and is written for the same reason that one is: it makes the
# value every later check consumes the one that was just re-read, so a future edit moving a
# consumer above this point fails visibly rather than silently reading the stale capture.
sha=$sha_now
fi
if [ -n "$sha" ]; then
# This is the THIRD read of this endpoint in a worst-case hook run (the ordinary-CI branch and the
# scheduled-auto-merge branch each do their own). Sharing one snapshot would close a narrow
@@ -301,8 +384,19 @@ else
# status decide this. Here the fallthrough happens to land on `vstate=""` -> deny (fail-CLOSED,
# so this was never a hole), but it would have surfaced the wrong message — a "BLOCKED, no
# verdict" deny instead of the "could not read the status" ask this branch exists to give.
if [ -z "${vjson//[[:space:]]/}" ] || ! printf '%s' "$vjson" | jq -e '.statuses | type == "array"' >/dev/null 2>&1; then
decide ask "H6/H10 merge gate: could not read the 'review-verdict/h10' status for PR #$pr head ${sha:0:7} (Gitea unreachable or an unexpected response). Confirm the current head is reviewed before scheduling an auto-merge."
# Validate the MEMBERS, not just the array. `.statuses | type == "array"` passes for
# `{"statuses":[1]}`, and the extraction below then errors with "Cannot index number with string"
# and exits 5 — which, under `set -e`, aborts this hook with NO JSON on stdout at all. A consent
# hook that emits nothing has violated its own contract: it neither grants, denies nor asks. Same
# one-level-down swallow as the #632 base-change guard and the branch-protection shape check
# below; the validation domain must match the CONSUMPTION domain (ersatztv#778).
if [ -z "${vjson//[[:space:]]/}" ] \
|| ! printf '%s' "$vjson" \
| jq -e '(.statuses | type == "array")
and all(.statuses[]; type == "object"
and ((.context | type) == "string")
and ((.status | type) == "string"))' >/dev/null 2>&1; then
decide ask "H6/H10 merge gate: could not read the 'review-verdict/h10' status for PR #$pr head ${sha:0:7} (Gitea unreachable, or a response whose status rows are not the expected shape). Confirm the current head is reviewed before scheduling an auto-merge."
fi
vstate=$(printf '%s' "$vjson" | jq -r '[.statuses[] | select(.context == "review-verdict/h10")] | first | .status // ""')
case "$vstate" in
@@ -311,6 +405,202 @@ else
pending) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is still pending on PR #$pr head ${sha:0:7} (no verdict posted for this commit yet). Review the current head and run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
*) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is '$vstate' on PR #$pr head ${sha:0:7}. Resolve the findings, then run: scripts/post-review-verdict.sh $pr MERGEABLE" ;;
esac
# --- The mitigation this path RESTS on, verified instead of asserted (ersatztv#778). -----------
# Everything above proves a property of the head that exists NOW. What makes that safe under
# merge_when_checks_succeed is stated in the paragraph opening this branch: `review-verdict/h10`
# is a REQUIRED status check on the base, a commit status belongs to exactly ONE sha, so a commit
# pushed after scheduling cannot inherit it and Gitea's own gate refuses the merge.
#
# That guarantee is branch-protection CONFIG. It lives outside this repo, no code here owned it,
# and until #778 nothing compared the two — so the grant reason handed to a human cited a
# protection that could have been switched off with no signal anywhere. The comment above and the
# grant string below are claims about the past; a dated claim is not a check.
#
# This is the hook's OWN defect class (#778 / `process.check-and-use-pins-a-version`): a check
# ("a later push clears the status") authorizes an action ("arm an auto-merge that Gitea completes
# later") over state that can change in between, with nothing pinning it. The read here does not
# pin anything either — branch protection can still be edited after this call — but it converts an
# ASSUMPTION that was never observed into a precondition that is, which is the honest ceiling for
# a config whose API offers no version, ETag or conditional read.
#
# Tri-state, matching this file's idiom throughout: unreadable -> ask (a human adjudicates),
# present -> proceed, ABSENT -> deny. Absence is not a degraded read; it is #622's hole reopened,
# and the whole point of that issue is that the failure is silent from the merge caller's side.
# Belt-and-braces: `$base_ref` was proven non-empty and re-confirmed at the hoisted check above,
# so this cannot fire today. Kept because it is the precondition this block's URL depends on, and
# a future edit that moves either piece should fail loudly here rather than request a URL with an
# empty path segment.
[ -n "$base_ref" ] || decide ask "H6/H10 merge gate: could not resolve PR #$pr's base branch, so the 'review-verdict/h10' required-check protection that makes a scheduled auto-merge safe (ersatztv#622) can't be confirmed. Verify branch protection on the base, or merge immediately instead of scheduling."
# The base was re-read and confirmed unchanged above, for every path — see the hoist comment
# there. It is deliberately NOT re-read a second time here: two reads would create a window
# between them for no gain, and the hoisted check already covers the enumeration gap that made
# this necessary.
# A read failure here is NOT evidence about the branch. The deleted by-name endpoint answered 404
# for "no rule with this name", which was a finding; the LIST endpoint's 404 means the repo was not
# found or is invisible to this credential, which is a read failure. Absence is now established by
# the classifier returning `nomatch` over a list that WAS read, never by an HTTP status.
# ALWAYS enumerate the rule LIST; never look a rule up by name. The by-name endpoint
# (`branch_protections/{name}`) is an exact DB lookup — `GetProtectedBranchRuleByName` — which
# performs no matching and knows nothing about precedence, so a 200 from it means only "a rule
# with this NAME exists and lists this context", never "this context is required on this branch".
#
# It was used first, with the list consulted only on a 404, and cold review found what that left
# behind: the precedence argument below guarded the 404 path while the 200 path — the one this
# repo actually takes — granted without it. Given a rule `main` requiring `review-verdict/h10` and
# a rule `m*` with better Priority that does not, Gitea applies `m*`, and the by-name hit on
# `main` granted anyway. The hardened path was dead code and the unhardened one was live. Deleting
# the twin rather than documenting it is the point: one fetch, one classifier, one argument, and
# no second path to keep in step. The ref no longer reaches a URL segment, so it needs no
# encoding either.
bp_file=$(mktemp) || decide ask "H6/H10 merge gate: could not allocate a temp file to read branch protection for '$base_ref'. Confirm the 'review-verdict/h10' required check manually before scheduling an auto-merge."
if [ -n "${ETV_GITEA_TOKEN:-}" ]; then
bp_code=$(curl -s -o "$bp_file" -w '%{http_code}' -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true)
else
bp_code=$(curl -s -o "$bp_file" -w '%{http_code}' -u "$ETV_GITEA_BASICAUTH" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true)
fi
bp_list=$(cat "$bp_file" 2>/dev/null || true)
bp=""
if [ "$bp_code" = "200" ] && printf '%s' "$bp_list" | jq -e 'type == "array"' >/dev/null 2>&1; then
# DO NOT claim parity with Gitea's matcher — this code cannot have it, and asserting it would
# be the exact defect this PR records (a mitigation outside the code, asserted rather than
# verified). Gitea compiles a rule name with gobwas/glob and a `/` separator, so its `*` does
# NOT cross a slash, `?`/`[…]`/`{a,b}` are wildcards, and a plain name is folded case-
# insensitively. Reimplementing that here would be a second copy of somebody else's parser.
#
# So the classification is deliberately THREE-way, and each arm is safe without knowing the
# dialect:
# exact — no glob rule could apply, AND some rule name has no glob metacharacter and
# equals the base case-insensitively. Only then is a single rule decidable.
#
# UNDECIDABLE IS EVALUATED FIRST, and the order is the point. Gitea picks the
# governing rule with `GetFirstMatched` over a list sorted by Priority, THEN
# by plain-name-ness — so a glob rule with a better Priority outranks an
# exactly-named one. Preferring `exact` would therefore inspect a rule Gitea
# might not be applying: if the exact rule requires `review-verdict/h10` and a
# higher-priority glob rule does not, the gate auto-grants on a base where the
# check is not enforced. Asking whenever ANY glob rule could apply is sound
# without knowing the precedence rules at all, which is the only claim this
# code is entitled to make about somebody else's resolver.
#
# Case folding is ASCII-only here, while Gitea's `EqualFold` is
# Unicode-aware — so a rule `ünstable` and a base `Ünstable` fold equal there
# and not here. ASCII-fold equality implies EqualFold equality, so the gap can
# only MISS a match, never invent one; but a miss lands on `none`, which
# DENIES with the stated cause that no rule can govern the base. The backslash
# paragraph below rejects "nearly unreachable" as a standard for that arm, and
# the same standard has to apply here, so a rule name carrying any non-ASCII
# byte is `undecidable` rather than fold-compared. Two fold-equal plain names
# are undecidable too: this code picks by list order while Gitea picks by
# Priority, and guessing which one is enforced is the defect the arm order
# above exists to avoid.
# undecidable — some glob rule COULD govern this base. Tested with a provable SUPERSET of any
# glob dialect: literal prefix before the first metacharacter, `.*`, literal
# suffix after the last. If even that does not match, no dialect can, because
# every dialect requires the literal head and tail to match literally.
#
# BACKSLASH counts as a metacharacter for that purpose, and it is the one case that breaks the
# superset proof if it does not. gobwas/glob reads `\{` as a LITERAL brace, so a rule `a\{b`
# governs the base `a{b` — while a superset that treated `\` as literal would build `a\.*b`,
# fail to match, and answer `none`, i.e. deny a base that IS protected. Git ref rules make this
# nearly unreachable (a branch name may not contain `*`, `?`, `[` or `\`, though it MAY contain
# `{`), but `none` is the arm that authorises a DENY on the stated grounds "nothing can govern
# this base", so its premise has to hold unconditionally rather than usually.
# none — nothing can possibly govern the base, so it is genuinely unprotected.
#
# `undecidable` asks rather than granting or denying. Over-matching would auto-grant on a base
# whose protection we never established (#622's hole, reached through the block written to
# close it); under-matching would deny with a stated cause that is false, which this block's
# own comment calls the worse outcome. Asking is the only answer that is honest in both
# directions, and it is rare in practice: as of 2026-08-19 this repo's only rule is the plain
# name `main`, which the classifier resolves to `exact` on every run. That is a dated
# observation about mutable remote config, not a property to rely on.
# The classifier is a FILE now (ersatztv#787), so its absence is a new failure mode: `jq -f` on a
# missing program exits 2 with empty stdout, which reaches the `*)` arm below and asks that "this
# repo's branch-protection rules came back in a shape this hook could not parse" — blaming the
# payload for a missing local file. That is precisely the states-a-cause-that-did-not-happen defect
# the two comments beside that arm were written to fix, so it is checked here rather than inherited.
classifier="$repo_root/scripts/lib/branch-rule-classifier.jq"
if [ ! -r "$classifier" ]; then
rm -f "$bp_file"
decide ask "H6/H10 merge gate: the shared branch-protection rule classifier is missing or unreadable at $classifier, so which rule governs '$base_ref' — and therefore whether 'review-verdict/h10' is required on it — could not be derived (ersatztv#787). Restore the file, or confirm the required checks manually."
fi
bp_verdict=$(printf '%s' "$bp_list" | jq --arg b "$base_ref" -c -f "$classifier" 2>/dev/null || true)
case $(printf '%s' "$bp_verdict" | jq -r '.verdict // ""' 2>/dev/null || true) in
exact) bp=$(printf '%s' "$bp_verdict" | jq -c '.rule' 2>/dev/null || true); bp_code=200 ;;
undecidable) rm -f "$bp_file"
decide ask "H6/H10 merge gate: no branch-protection rule on this repo governs '$base_ref' decidably — a GLOB rule could govern it, or two rule names fold-equal, or a name is non-ASCII. This hook deliberately does not reimplement Gitea's glob matcher, so whether 'review-verdict/h10' is required on this base cannot be derived here (ersatztv#778). Confirm it in the repo's branch-protection settings, or merge immediately instead of scheduling." ;;
none) bp_code=nomatch; bp="" ;;
*) bp_code=unreadable-rules; bp="" ;;
esac
else
# A 200 whose body is NOT an array never reaches the classifier — it is diverted by the array
# gate above — so it needs the same sentinel, or the generic ask below reports
# "HTTP '200' — Gitea unreachable" about a read that plainly succeeded. Same defect as the
# throw-inside-the-classifier arm, one branch earlier; fixing only the arm where it was noticed
# is the twin-missed shape this PR is largely about.
if [ "$bp_code" = "200" ]; then
bp_code=unreadable-rules
else
bp_code=${bp_code:-000} # a real transport/HTTP failure -> the ask arm below
fi
bp=""
fi
rm -f "$bp_file"
# `nomatch` is the CLASSIFIER's verdict, deliberately not an HTTP code. Reusing 404 for it made
# this deny reachable from an HTTP 404 on the list read too — repo not found, or invisible to the
# credential, which Gitea also answers 404 — and then the reason claimed "the full rule list was
# read and none matches" about a read that never happened. A transport failure must reach the ask
# below, not a deny stating a finding.
if [ "$bp_code" = "nomatch" ]; then
decide deny "H6/H10 merge gate: BLOCKED — no branch-protection rule on this repo can govern '$base_ref' (the full rule list was read and none matches), so 'review-verdict/h10' is not a required check on it. A scheduled auto-merge is safe ONLY because that per-sha required check stops a commit pushed after scheduling from merging unreviewed (ersatztv#622). Restore branch protection on '$base_ref', or merge immediately (without merge_when_checks_succeed) once CI is green."
fi
# `unreadable-rules` is the CLASSIFIER failing on a 200 it could not parse — a numeric
# `branch_name` makes jq throw, and `//` does not catch it because it fires only on null/false.
# It gets its own sentinel for the same reason `nomatch` does: reporting "HTTP '000' — Gitea
# unreachable" about a successful 200 read states a cause that did not happen, which is the defect
# fixed one arm over for the deny.
if [ "$bp_code" = "unreadable-rules" ]; then
decide ask "H6/H10 merge gate: this repo's branch-protection rules came back in a shape this hook could not parse, so whether 'review-verdict/h10' is required on '$base_ref' is unknown. Check the rules manually, or merge immediately instead of scheduling."
fi
if [ "$bp_code" != "200" ] || [ -z "${bp//[[:space:]]/}" ] || ! printf '%s' "$bp" | jq -e 'type == "object"' >/dev/null 2>&1; then
decide ask "H6/H10 merge gate: could not read this repo's branch-protection rules (HTTP '${bp_code:-none}' — Gitea unreachable, or these credentials lack the repo-admin scope that endpoint needs), so whether 'review-verdict/h10' is required on '$base_ref' is unknown. Scheduling an auto-merge is only safe while 'review-verdict/h10' is a REQUIRED check there (ersatztv#622) — confirm that manually, or merge immediately instead of scheduling."
fi
# The membership test is `any(.[]; . == …)` over a value FIRST PROVEN to be an array of strings —
# never `index()`. `index` on a STRING is substring search, so a `status_check_contexts` that
# arrived as the string "prefix-review-verdict/h10-suffix" would answer "yes" and auto-grant a
# merge on a base where no such context is required. That is a FALSE-OPEN in the gate, reachable
# from any payload shape drift, and it is the direction that matters: a false-closed costs a
# prompt, a false-open costs an unreviewed merge.
#
# Validating `$bp` as an object does not make its MEMBERS well-formed, which is the same
# one-level-down swallow that survived the first fix in the #632 base-change guard — the
# validation domain has to match the CONSUMPTION domain, not stop at the top-level type. So the
# shape is checked explicitly and anything else becomes "unknown" rather than a decision.
#
# `null` and `[]` are legitimate (an unprotected-in-practice branch) and answer "no", not
# "unknown": absent IS the finding here, not a read failure. The word is then matched
# exhaustively, because "" is not a third synonym for "no".
# `// []` defaults on FALSE as well as on null, because jq's alternative operator fires for both.
# So `"status_check_contexts": false` — a malformed shape — became `[]` and answered "no", i.e. a
# confident DENY derived from a payload that was never understood. Absent and null are defaulted
# explicitly; every other non-array is "unknown".
# `enable_status_check` is validated as a BOOLEAN before it is trusted, for the same reason the
# contexts list is: `"true"` (the string) is not `true`, and comparing it to `true` yields a
# confident "no" -> deny derived from a payload never understood. Every malformed shape on this
# endpoint has to reach the same "unknown" -> ask arm, or the tri-state is only two states.
guarded=$(printf '%s' "$bp" \
| jq -r 'def ctxs: if (has("status_check_contexts") | not) or .status_check_contexts == null
then [] else .status_check_contexts end;
if (.enable_status_check | type) != "boolean" then "unknown"
elif (ctxs | type) != "array" or any(ctxs[]; type != "string") then "unknown"
elif (.enable_status_check == true) and any(ctxs[]; . == "review-verdict/h10") then "yes"
else "no" end' 2>/dev/null || true)
case "$guarded" in
yes) : ;;
no) decide deny "H6/H10 merge gate: BLOCKED — 'review-verdict/h10' is NOT a required status check on '$base_ref' (branch protection reports enable_status_check/status_check_contexts without it). A scheduled auto-merge is safe ONLY because that per-sha required check stops a commit pushed after scheduling from merging unreviewed (ersatztv#622); without it, arming merge_when_checks_succeed freezes consent at a head Gitea may not be the one to merge. Restore it in branch protection, or merge immediately (without merge_when_checks_succeed) once CI is green." ;;
*) decide ask "H6/H10 merge gate: branch protection for '$base_ref' came back in an unexpected shape, so the 'review-verdict/h10' required check that makes a scheduled auto-merge safe (ersatztv#622) could not be confirmed either way. Check it manually, or merge immediately instead of scheduling." ;;
esac
fi
# --- (c) Review-verdict freshness (ersatztv#303 H10): a review-verdict comment must reference the
@@ -359,13 +649,126 @@ case "$class" in
decide ask "H10 merge gate: unrecognized verdict classification '$class' for PR #$pr. Confirm the review covered the latest commit ($short) before merging." ;;
esac
# --- (d) Guard-scope freshness (ersatztv#787): the committed mirror of `main`'s required status
# checks must still match the server. ------------------------------------------------------
# ORDERED LAST, and that is a severity argument rather than a stylistic one. Every check above
# can DENY; this one can only ever downgrade an otherwise-satisfied auto-grant to a prompt. Run
# earlier it would preempt those verdicts and report a stale guard scope at a reader whose merge
# is blocked for a completely different and more serious reason, and it would ask on payloads the
# checks above are about to reject anyway. Placed here it is also PAST the point where the two
# merge paths converge, so it covers both without duplicating anything.
# `scripts/tests/test_ci_dropped_step_guard.py` DERIVES which jobs must carry per-step execution
# markers from `.gitea/required-status-contexts.json`, because its CI job checks out with
# `persist-credentials: false` and cannot ask Gitea. That makes the snapshot the single
# hand-maintained input in the chain: a fourth required context added on the server leaves the
# snapshot — and therefore the guard's scope — silently behind, which is the whole of #787.
#
# THIS RUNS ON BOTH MERGE PATHS, deliberately, and it is placed here rather than beside the
# branch-protection read in the scheduled-auto-merge branch for that reason.
#
# WHAT IT DOES NOT COVER, said here rather than left to be discovered: a PR whose changed files are
# all docs/process — `.gitea/` included — exits at the docs-only passthrough far above, so this arm
# never runs for it. A PR that edits ONLY `.gitea/required-status-contexts.json` is docs-only BY
# CONSTRUCTION, and that is exactly the snapshot-NARROWING direction the decision record names as
# this design's residual. Excluding that path from the allow-list would not buy the protection it
# looks like it would: this arm compares the live server against the snapshot in the LOCAL CHECKOUT,
# not against the version the PR proposes, so it cannot see a narrowing that has not landed yet.
# What does hold is that the passthrough is a passthrough — a human prompt, never an auto-grant —
# which is the `.gitea/` treatment ersatztv#317 asked for. That read is inside
# `else` (mwcs = true) and never executes on an immediate merge, which is the common case; hanging
# the freshness check off it would fire it only when an auto-merge is armed. This file already
# records that exact defect one section up — the base re-read "first landed inside the
# scheduled-auto-merge branch only", and cold review found scheduled+retarget denied while
# immediate+retarget auto-GRANTED. Same shape, so it is not repeated here.
#
# It reads `main` (the branch the snapshot names), NOT `$base_ref`. That is a DIFFERENT question
# from the one the scheduled branch asks — "is review-verdict/h10 required on the base I am merging
# into" — so this is not a second copy of that classifier and the two cannot drift into disagreeing:
# they consume different fields of different rules for different decisions.
#
# ASK, NEVER DENY. Drift does not make THIS merge unsafe: Gitea enforces the live required set
# server-side, so a newly required context with no status blocks the merge on its own. What has gone
# stale is a guard's scope — a different artifact, on a different clock. Denying would state
# something false about the change in front of the reader. Every non-`match` class asks, so a
# comparison that could not be made is surfaced rather than skipped (`unknown` is not `fine`).
# ONE base for both the checker and the snapshot, and it is `$repo_root` — derived from this file's
# own location — rather than `$CLAUDE_PROJECT_DIR`. Two reasons, and the second is the load-bearing
# one. Resolving them from different roots would let the hook classify one checkout's snapshot with
# another checkout's script, mismatched halves of a comparison whose whole job is to detect a
# mismatch. And an ENV VAR is not a sound input to a security decision: a wrong value pointing at a
# tree that happens to contain an executable checker returns `match` about a different checkout
# entirely, which silently authorizes the grant. A missing path only asks, so the failure is quiet
# exactly where it is worst.
ctx_base="$repo_root"
ctx_snapshot="$ctx_base/.gitea/required-status-contexts.json"
ctx_script="$ctx_base/scripts/check-required-contexts.sh"
# THIS ARM IS ABOUT ONE REPO, and the merge tool is not. Every other check here reads
# `$owner/$repo` from the tool input and is repo-agnostic; this one compares a HARDCODED branch
# against a snapshot committed in THIS checkout. Merging a PR in another repo from a session opened
# here would otherwise weigh that repo's live contexts against this repo's mirror and report a
# confident, flatly false finding about it — measured: server-management returns `[]`, which
# classifies as `nomatch`. So the snapshot names the repo it describes and the arm runs only for it.
# An unreadable snapshot cannot answer "is this my repo?" either, so it asks rather than skipping.
ctx_repo=$(jq -r 'if (.repo | type) == "string" then .repo else "" end' "$ctx_snapshot" 2>/dev/null || true)
if [ -z "$ctx_repo" ]; then
decide ask "H6 merge gate: $ctx_snapshot is missing, unreadable, or names no \`repo\`, so the dropped-step guard's scope could not be checked against branch protection — nor could it be established whether this snapshot even describes $owner/$repo (ersatztv#787). Restore the file, or check the required checks manually."
fi
# CASE-FOLDED, because Gitea resolves owner/repo case-insensitively: verified live, both
# `/repos/timothy/ersatztv` and `/repos/TIMOTHY/ErsatzTV` answer 200. A byte-exact compare would let
# any case variant sail through every other arm and SKIP this one, so drift would go unreported with
# no ask — the gate failing open on a spelling. The hook already treats case folding as
# decision-relevant one section up, where `MAIN` vs `main` makes the governing rule undecidable.
ctx_repo_fold=$(printf '%s' "$ctx_repo" | tr '[:upper:]' '[:lower:]')
target_repo_fold=$(printf '%s' "$owner/$repo" | tr '[:upper:]' '[:lower:]')
if [ "$ctx_repo_fold" = "$target_repo_fold" ]; then
if [ ! -x "$ctx_script" ]; then
decide ask "H6 merge gate: the required-contexts checker is missing or not executable at $ctx_script, so whether the dropped-step guard's scope still matches branch protection on 'main' could not be derived (ersatztv#787). Check it manually, or restore the script."
fi
bpf=$(mktemp) || decide ask "H6 merge gate: could not allocate a temp file to read branch protection for the guard-scope freshness check (ersatztv#787)."
if [ -n "${ETV_GITEA_TOKEN:-}" ]; then
ctx_code=$(curl -s -o "$bpf" -w '%{http_code}' -H "Authorization: token $ETV_GITEA_TOKEN" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true)
else
ctx_code=$(curl -s -o "$bpf" -w '%{http_code}' -u "$ETV_GITEA_BASICAUTH" "$base_url/repos/$owner/$repo/branch_protections" 2>/dev/null || true)
fi
if [ "$ctx_code" = "200" ]; then
# stderr is KEPT, not sent to /dev/null. The checker exits 2 with a diagnostic on a usage error —
# an unreadable snapshot, a branch mismatch, a missing classifier — and discarding it made all of
# those arrive at the operator as the catch-all's "returned 'nothing'", which names no cause. That
# is the same states-a-cause-that-did-not-happen shape this arm was careful about elsewhere.
ctx_class=$("$ctx_script" --branch main --snapshot "$ctx_snapshot" < "$bpf" 2>"$bpf.err" || true)
ctx_diag=$(tr '\n' ' ' < "$bpf.err" 2>/dev/null | cut -c1-300 || true)
else
ctx_class=readfail
ctx_diag=""
fi
rm -f "$bpf" "$bpf.err"
case "$ctx_class" in
match) : ;;
drift)
decide ask "H6 merge gate: the required status checks on 'main' no longer match .gitea/required-status-contexts.json (ersatztv#787). scripts/tests/test_ci_dropped_step_guard.py derives its marked-job scope from that snapshot, so until it is reconciled a required context may have NO dropped-step guard — a step the runner drops would conclude success and take that check green having done no work (ersatztv#756). Re-read the live list and update the snapshot in a PR (the guard will then demand markers for any newly required job, or an ACCOUNTED_ELSEWHERE entry naming what covers it). This does not make the merge in front of you unsafe — Gitea enforces the live required set server-side — so approve if you have judged it unrelated." ;;
nomatch)
decide ask "H6 merge gate: no branch-protection rule governs 'main' at all, so the required status checks the dropped-step guard scopes itself to could not be confirmed (ersatztv#787). Branch protection on 'main' is what makes 'review-verdict/h10' load-bearing (ersatztv#743) — check it before merging." ;;
undecidable)
decide ask "H6 merge gate: a glob branch-protection rule could govern 'main', so which rule's required contexts to compare against .gitea/required-status-contexts.json is not derivable without reimplementing Gitea's matcher (ersatztv#787). Confirm the required checks manually." ;;
unreadable)
decide ask "H6 merge gate: branch protection for 'main', or .gitea/required-status-contexts.json itself, came back in a shape the required-contexts checker could not consume, so whether the dropped-step guard's scope is still current is unknown (ersatztv#787). Check the rules and the snapshot manually." ;;
readfail)
decide ask "H6 merge gate: could not read branch protection for the guard-scope freshness check (HTTP '${ctx_code:-none}' — Gitea unreachable, or these credentials lack the repo-admin scope that endpoint needs), so whether .gitea/required-status-contexts.json is still current is unknown (ersatztv#787). Confirm the required checks on 'main' manually." ;;
*)
decide ask "H6 merge gate: the required-contexts checker returned '${ctx_class:-nothing}', which is not a class this hook understands, so the dropped-step guard's scope could not be confirmed against branch protection (ersatztv#787).${ctx_diag:+ It said: ${ctx_diag}}Check scripts/check-required-contexts.sh." ;;
esac
fi # end of the guard-scope freshness arm (opened at `if [ "$ctx_repo_fold" = ... ]` above). The
# body is left unindented to match the rest of this file, which is flat throughout; the marker
# is here because the block is long enough that its extent is otherwise easy to misread.
if [ "$class" = "positive" ]; then
# (a) CI + (b) all Done-when ticked + (c) positive verdict @ current head -> SATISFIED. Auto-grant.
# The reason string must not claim more than was actually checked: on the merge_when_checks_succeed
# path this hook never read the CI status at all (it is delegated to Gitea), so saying "CI green"
# there was a plain falsehood in the one message a human reads to decide whether to trust the gate.
if [ "$mwcs" = "true" ]; then
decide grant "H6/H10 merge gate: satisfied — all Done-when boxes ticked, and both a positive Review-verdict comment and the 'review-verdict/h10' status cover the current head ($short). CI is gated by Gitea (merge_when_checks_succeed), and because the verdict status is bound to this sha, a commit pushed before Gitea merges will clear it and block the merge (ersatztv#622). Auto-granted."
decide grant "H6/H10 merge gate: satisfied — all Done-when boxes ticked, and both a positive Review-verdict comment and the 'review-verdict/h10' status cover the current head ($short). CI is gated by Gitea (merge_when_checks_succeed). A commit pushed before Gitea merges clears the sha-bound verdict status and is blocked by the 'review-verdict/h10' required check (ersatztv#622) — which this hook has just CONFIRMED is still required on '$base_ref' — read from the repo's full rule list and matched with Gitea's own plain-vs-glob split, refusing rather than guessing wherever precedence or folding is not derivable. That guarantee holds while that branch protection stands; if it is weakened after this check, nothing here would see it (ersatztv#778). Auto-granted."
fi
decide grant "H6/H10 merge gate: satisfied — CI green, all Done-when boxes ticked, and a positive Review-verdict references the current head ($short). Auto-granted (no separate confirmation needed)."
fi
+7
View File
@@ -2,6 +2,13 @@
# PreToolUse / browser-navigate — deny opening download/stream endpoints in a tab
# (they hang the MCP session; curl them instead). Fail-open on parse trouble.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-nav-guard "" capture || true
input=$(cat)
url=$(printf '%s' "$input" | jq -r '.tool_input.url // ""' 2>/dev/null || true)
@@ -8,6 +8,13 @@
# So the main tree (never marked) and pre-convention worktrees (no marker) are unaffected;
# only a commit/merge into another session's marked worktree is blocked.
set -euo pipefail
# ersatztv#776 — report that this hook fired. MUST precede any stdin read.
# Claude hook: decides by printed JSON, so stdout is captured.
ETV_HOOK_FIRE_LIB="${CLAUDE_PROJECT_DIR:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd)}/scripts/hook-fire-log.sh" || true
[ -r "$ETV_HOOK_FIRE_LIB" ] && . "$ETV_HOOK_FIRE_LIB" || true
type etv_hook_fire_begin >/dev/null 2>&1 || etv_hook_fire_begin() { :; }
etv_hook_fire_begin pretooluse-worktree-guard "" capture || true
input=$(cat)
cmd=$(printf '%s' "$input" | jq -r '.tool_input.command // ""' 2>/dev/null || true)
cwd=$(printf '%s' "$input" | jq -r '.cwd // ""' 2>/dev/null || true)
+103 -3
View File
@@ -4,8 +4,15 @@ description: "ErsatzTV custom IPTV channel management — REST API, SQLite DB, J
---
> **Canonical copy: `~/ersatztv/.claude/skills/ersatztv/SKILL.md`** (ersatztv owns this skill per that
> repo's `CLAUDE.md` → Project Boundaries). `~/server-management/.claude/skills/ersatztv` is a symlink
> to it. Edit it in the ersatztv repo; never fork a second copy (ersatztv#617).
> repo's `CLAUDE.md` → Project Boundaries and `process.ersatztv-owns-code-not-operations`). Both
> `~/server-management/.claude/skills/ersatztv` **and** `~/media-management/.claude/skills/ersatztv`
> are symlinks to it. Edit it in the ersatztv repo; never fork a second copy (ersatztv#617, #755) —
> media-management's copy had silently become a divergent fork still describing a Blazor UI that no
> longer exists, which is what made this the rule rather than a preference.
>
> **Channel OPERATIONS (create/edit a live channel, lineup, collection, schedule, playout, logo,
> overlay) are `media-management`'s job**; ersatztv owns the fork code, `/api/v1`, CI and releases.
> This skill serves both — it is the operator's reference *and* the developer's map.
# ErsatzTV Channel Management
@@ -180,6 +187,99 @@ POST /api/v1/libraries/{id}/scan-show \
POST /api/v1/channels/{channelId}/playout/reset
```
### Scripted Schedule API — `/api/v1/scripted/…`
For **programmatic playout building**: each call mutates one build session, addressed by `buildId`.
Documented by its own OpenAPI spec, **separate from `v1.json`** — which is why
`docs/endpoint-index.md` does not list any of it. It ships as **two** files, both served at
`/openapi/` (measured 2026-08-26 on prod: `scripted-schedule.json`, `scripted-schedule-tagged.json`
and `v1.json` all return 200). They carry the same 28 paths, so either answers "what operations
exist"; they differ only in grouping — the plain file puts everything under one `ScriptedSchedule`
tag, the `-tagged` one splits it into Scripted Content / Control / Metadata / Scheduling. Scalar's
`/docs` page renders the `-tagged` file (`Startup.cs` registers `openapi/scripted-schedule-tagged.json`),
which is why the browsable docs are grouped and a raw fetch of the plain file is not.
The base path is **`/api/v1/scripted/playout/build/{buildId}/`**, and `buildId` is routed as a GUID
(`ScriptedScheduleController.cs`). An older archived copy of this skill gave it as `/api/scripted/…`,
without the `v1`; no such route is registered.
**You cannot tell a wrong base path from a stale `buildId` by probing** — measured on prod
2026-08-26, `GET …/context` with a non-existent build id:
| | `/api/v1/scripted/…` | `/api/scripted/…` (no route) |
|---|---|---|
| no key | 401 | 401 |
| valid key | 404 | 404 |
Unauthenticated everything is 401, because the api-key filter runs before routing. Authenticated, the
correct path 404s too — the build session does not exist — so the 404 that a wrong path earns is
indistinguishable from the one a correct path earns. The bound: this holds **while the build id is
not live**. Against a real, open build session the correct path would answer 200 and the difference
would show — but that is not the situation you are in when you are probing to find out why nothing
works. Confirm the route in `ErsatzTV/Controllers/Api/ScriptedScheduleController.cs`; do not infer it
from a status code.
```
# 28 operations, derived from scripted-schedule.json on 2026-08-26 (ersatztv#755)
POST add_all {content, fillerKind, customTitle, disableWatermarks}
POST add_collection {key, collection, order}
POST add_count {content, count, fillerKind, customTitle, disableWatermarks}
POST add_duration {content, duration, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST add_marathon {key, groupBy, itemOrder, guids, searches, playAllItems, shuffleGroups}
POST add_multi_collection {key, multiCollection, order}
POST add_playlist {key, playlist, playlistGroup}
POST add_search {key, query, order}
POST add_show {key, guids, order}
POST add_smart_collection {key, smartCollection, order}
POST create_playlist {key, items}
POST graphics_off {graphics}
POST graphics_on {graphics, variables}
POST pad_to_next {content, minutes, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pad_until {content, when, tomorrow, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pad_until_exact {content, when, fallback, trim, discardAttempts, stopBeforeEnd, offlineTail, fillerKind, customTitle, disableWatermarks}
POST pre_roll_off (no body)
POST pre_roll_on {playlist}
POST skip_items {content, count}
POST skip_to_item {content, season, episode}
POST start_epg_group {advance, customTitle}
POST stop_epg_group (no body)
POST wait_until {when, tomorrow, rewindOnReset}
POST wait_until_exact {when, rewindOnReset}
POST watermark_off {watermark}
POST watermark_on {watermark}
GET context (no body)
GET peek_next/{content} (no body)
```
Re-derive rather than trusting this table (it is prose and will drift):
```bash
# Absolute path on purpose: this skill is symlinked into ~/server-management and
# ~/media-management, where a repo-relative path would not resolve. ~/ersatztv is the
# shared checkout and can lag origin/main — use the live-instance form below to see
# what is actually deployed.
python3 -c "import json;d=json.load(open('$HOME/ersatztv/ErsatzTV/wwwroot/openapi/scripted-schedule.json'));\
print('\n'.join(f'{m.upper()} {p}' for p,i in d['paths'].items() for m in i if m in('get','post')))"
```
Without a checkout — straight off the running instance (prod; test is port 8410):
```bash
ssh timothy@192.168.1.29 'curl -s http://localhost:8409/openapi/scripted-schedule.json' \
| python3 -c "import json,sys;d=json.load(sys.stdin);\
print('\n'.join(f'{m.upper()} {p}' for p,i in d['paths'].items() for m in i if m in('get','post')))"
```
Field lists above are the request-body property names only; consult the spec for types,
required-ness and defaults. That omission matters for the three on/off pairs: `graphics_on`/
`graphics_off`, `watermark_on`/`watermark_off` and `pre_roll_on`/`pre_roll_off` are **separate
operations, not one toggle**, and the difference is not always visible as differing property names.
`graphics_*` and `pre_roll_*` differ outright. `watermark_on` and `watermark_off` both list
`{watermark}`, but only `on` marks it **required** — `watermark_off` with an **empty** list turns
*every* scripted watermark off (`SchedulingEngine.WatermarkOff`: `watermarks.Count == 0` →
`ClearChannelWatermarkIds()`; `GraphicsOff` is the same shape). Read the schema, not this table,
before sending an `_off`.
## SQLite DB Operations
```bash
@@ -363,7 +463,7 @@ docker start ersatztv
```
- **Dispatcharr caches ErsatzTV's XMLTV.** Repointing its DB rows is not enough — it keeps serving a stale EPG full of dead `ersatztv:8409` artwork URLs (breaks Kodi artwork). Force a refresh (EPG source 9):
```bash
ssh timothy@192.168.1.99 'docker exec dispatcharr python manage.py shell -c \
ssh timothy@192.168.1.29 'docker exec dispatcharr python manage.py shell -c \
"from apps.epg.tasks import refresh_epg_data; refresh_epg_data(9)"'
```
- **`/api/health` returns 401** (needs an API key). The Telegraf probe has no `response_string_match`, so ErsatzTV reads as **unhealthy in Grafana** — a false alarm, and **pre-existing**, not caused by the move. The container healthcheck uses the unauthenticated internal `/health` and is unaffected.
+12
View File
@@ -0,0 +1,12 @@
{
"repo": "timothy/ersatztv",
"branch": "main",
"read_on": "2026-08-27",
"source": "GET /repos/timothy/ersatztv/branch_protections -> the rule governing `main` -> status_check_contexts",
"why": "ersatztv#787. The committed mirror of the required status checks on `main`. It exists because the guards that make a required context trustworthy run in `pr-checks.yml::script-tests`, which checks out with persist-credentials:false and holds no Gitea credential, so it cannot ask the server. scripts/tests/test_ci_dropped_step_guard.py DERIVES its marked-job scope from `contexts` rather than repeating it as a literal, and scripts/check-required-contexts.sh compares this list against the live one wherever a credential does exist. Editing `contexts` by hand without re-reading the server is the one move that defeats both. The `repo` field exists because the merge-consent hook fires for whatever owner/repo the merge tool was called with: without it, merging a PR in another repo from an ersatztv session compares that repo's live contexts against THIS repo's mirror and reports a confident, flatly false finding about it.",
"contexts": [
"Build ErsatzTV Image / Build & test (.NET) (pull_request)",
"Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request)",
"review-verdict/h10"
]
}
+74 -11
View File
@@ -4,29 +4,63 @@ name: Build CI Toolchain Image
# pushes it to the Gitea container registry (ersatztv#390). The toolchain jobs in
# docker-build.yml consume it via `container:`, pinned to an immutable :<sha>.
#
# push touching docker/ci/** -> :<short-sha> (+ :latest only from main)
# workflow_dispatch -> manual rebuild
# schedule (weekly) -> picks up base-image security updates
# push to MAIN touching docker/ci/** -> :<short-sha> + :latest
# workflow_dispatch on main -> :<short-sha> of main's HEAD + :latest
# workflow_dispatch on a branch -> :<short-sha> of that branch's HEAD ONLY (never :latest)
# schedule (weekly) -> picks up base-image security updates
#
# Deliberately separate from docker-build.yml: this image changes rarely (a Dockerfile edit or
# the weekly cron), while docker-build.yml runs on every push/PR. Coupling them would rebuild a
# ~2GB toolchain image on every commit.
#
# ROLLOUT NOTE: the jobs pin an immutable :<sha>, never :latest — a broken toolchain image would
# otherwise block every converted job the moment it was pushed. Bumping the toolchain is therefore
# a deliberate two-step: merge a docker/ci/Dockerfile change (this workflow publishes a new :<sha>),
# then update the pin in docker-build.yml in a follow-up PR whose CI proves the new image works.
# See docs/ci-cd.md -> "CI toolchain image".
# otherwise block every converted job the moment it was pushed. Bumping the toolchain is a deliberate
# two-step, and BOTH steps land in the SAME PR: publish (push the docker/ci commit as branch HEAD,
# dispatch this workflow on that branch), then commit the pin bump in docker-build.yml. Merging first
# is not available: a PR that changes docker/ci/** without moving the pin turns `ci-image-pin` red,
# and the merge-consent hook reads the COMBINED commit status, so it will not auto-grant. That much
# predates ersatztv#744 — what #744 changed is how the publish half is performed.
# See docs/ci-cd.md -> "Publishing from a branch is a dispatch, not a push".
#
# Like docker-build.yml: the Gitea registry is HTTP-only, so BuildKit needs the inline
# `http = true` config (it does not inherit the host daemon's insecure-registries setting).
on:
# Publishing from a branch is a DELIBERATE act, not a side effect of pushing (ersatztv#744).
# Gitea resolves a `push` workflow's definition from the pushed branch, so an unfiltered `push`
# trigger ran this file's own YAML — attacker-supplied, unreviewed, with no status check in the
# loop — on a docker-capable runner holding the credential that writes `ersatztv:prod` and the
# `ersatztv-ci:<sha>` five `container:` jobs execute.
#
# BE PRECISE ABOUT WHAT THIS BUYS, because the mechanism cuts both ways: the filter below is read
# from the pushed ref like everything else in this file, so a branch that DELETES it re-enables
# the route. What closes is the DRIVE-BY case — an ordinary push of a legitimate `docker/ci`
# change publishing an image nobody asked for, with no deliberate act anywhere. This is NOT a
# boundary against a malicious or compromised writer and must not be cited as one; that class is
# ersatztv#853. `workflow_dispatch` is self-supplied from the selected branch in the same way, and
# is the deliberate publish path (docs/ci-cd.md -> "CI toolchain image").
#
# A `v*` tag push does not match this trigger either: there is no `tags:` key, and a `branches:`
# filter is compared against a branch ref. The exact matcher semantics are not probed here; the
# observable claim is the one that matters — a release cut no longer republishes the toolchain
# image as a side effect.
#
# `.gitea/workflows/ci-image.yml` is NOT in `paths:`, and it left `ci-image-pin`'s `expected` in
# the same change. That pairing is a DECIDED TRADEOFF, not a necessity: keeping it works, because
# the dispatch above can publish the ci-image.yml commit itself and the pin then matches. The
# price is what decided it — that route charges a full ~2GB publish plus a five-pin bump for
# EVERY edit to this file, comments included, and a rebase charges it again. The cost of the side
# taken is stated here and in ci-cd.md: a change to HOW the image is built that lives only in
# this file no longer republishes on its own, so pair it with a `docker/ci/**` edit.
#
# `paths:` here and `ci-image-pin`'s `expected` pathspec in pr-checks.yml MUST name the same
# sources, and nothing mechanically enforces that since the shared self-reference went —
# ersatztv#855.
workflow_dispatch:
push:
branches: [main]
paths:
- 'docker/ci/**'
- '.gitea/workflows/ci-image.yml'
schedule:
# Mondays 05:00 UTC. Gitea registers `schedule` only from the default branch (main).
#
@@ -50,6 +84,21 @@ env:
REGISTRY: 192.168.1.95:3000
CI_IMAGE: 192.168.1.95:3000/timothy/ersatztv-ci
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# This workflow's registry pushes authenticate with the scoped REGISTRY_* PAT
# (`ci.actions-credential-scoping`), so the injected GITEA_TOKEN serves only its single
# `actions/checkout`. This file was the one workflow #748 could not originally reach: editing it
# re-pointed `ci-image-pin`'s `expected` at the editing commit and reddened a BLOCKING job, and its
# own `paths:` made the edit publish an image. ersatztv#744 took this path out of both
# (`ci.toolchain-image-publish-is-a-dispatch`), so the exemption that briefly existed here is DELETED
# rather than documented — which is what ersatztv#835 asked for.
permissions:
code: read
jobs:
build:
name: Build & push CI image
@@ -58,13 +107,23 @@ jobs:
# toolchain image — the heaviest thing that ran in that lane. `small` is now
# git-only and capped at 1g per job, which would OOM this build.
#
# Rare trigger (pushes touching docker/ci + a weekly cron), so it costs the
# ubuntu-latest lane almost nothing, and ci-runner (.127) runs no prod workload.
# Rare trigger (main pushes touching docker/ci, a weekly cron, and the occasional
# branch dispatch), so it costs the ubuntu-latest lane almost nothing, and
# ci-runner (.127) runs no prod workload.
runs-on: ubuntu-latest
env:
CI_JOB_ROLE: none
steps:
- name: Checkout
uses: actions/checkout@v4
with:
# ersatztv#746's convention, applied here once #744 removed the reason it was skipped:
# without it the action leaves a write-capable Authorization header in .git/config for
# every later step. Nothing here pushes with git — the only git call is the
# `rev-parse --short HEAD` below — and the repo is public, so the clone needs no
# credential of its own. Guarded for every workflow by
# scripts/tests/test_workflow_persist_credentials.py (ersatztv#835).
persist-credentials: false
# only docker/ci/Dockerfile is needed; no git describe/log here
fetch-depth: 1
@@ -76,7 +135,11 @@ jobs:
# Always publish the immutable :<sha> — that is what docker-build.yml pins.
TAGS=("${CI_IMAGE}:${SHORT}")
# :latest is a convenience/floating pointer for humans and the weekly rebuild; jobs must
# never consume it. Only main may move it.
# never consume it. Only main may move it — and since #744 the `push` trigger is
# main-only, so on that path the branch check is satisfied by construction. It is now the
# SOLE protection on the one event that never exercised it before: a `workflow_dispatch`
# selects any ref, and the branch-side publish path documented in ci-cd.md runs exactly
# that. Do not simplify this away on the reasoning that the trigger is already main-only.
if [ "${GITHUB_REF}" = "refs/heads/main" ]; then
TAGS+=("${CI_IMAGE}:latest")
fi
+14
View File
@@ -33,13 +33,27 @@ env:
DOTNET_CLI_USE_MSBUILD_SERVER: "0"
MSBUILDDISABLENODEREUSE: "1"
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Holds no registry credential and reads nothing from the Gitea API; the injected GITEA_TOKEN serves
# only its one `actions/checkout`.
permissions:
code: read
jobs:
scan:
name: NuGet vulnerable packages
runs-on: ubuntu-latest
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Setup .NET
uses: actions/setup-dotnet@v4
+365 -28
View File
@@ -38,9 +38,22 @@ name: Build ErsatzTV Image
# report `success` in seconds — the two REQUIRED contexts (`Build & test (.NET)`, `EF migration
# integrity (SQLite + MySql)`) must keep reporting or a docs-only PR could never merge. We do NOT
# `if:`-skip a required job: on Gitea 1.25.4 a skipped job reports commit-status state `skipped`
# (verified, throwaway PR #418) and we don't rely on how branch protection treats a skipped
# (verified, throwaway PR #418; re-confirmed on 1.27.1, 2026-08-28, ersatztv#747 — `Build & push
# image (amd64)` is `if:`-skipped on every PR and reported `skipped` on the two heads sampled,
# PRs #829 and #828) and we don't rely on how branch protection treats a skipped
# REQUIRED context. See docs/ci-cd.md -> "Docs-only skip".
#
# RELEASE-PATH DELIMITER GATE (ersatztv#767): the `scan` job runs the PyYAML-based delimiter-ban
# test and is a `needs:` of `build`, so a `${{` opener in a banned job's `run:` body means `build`
# never runs. It is deliberately NOT gated by either skip below: the gate's coverage must not depend
# on a detector the gate is not allowed to trust, and it is cheap enough that gating it buys nothing.
# (Do NOT justify that with "the docs-only path still builds an image" — it does not. `Build and
# push` carries the docs_only gate too; a tag build is unaffected only because the script forces
# docs_only=false there.) Note it installs from PyPI (setup-python + pip), putting a NEW network
# dependency between a `v*` tag and its image. Not the only one on this path — `test` runs
# `dotnet restore` and `npm ci` behind actions/cache, and a cache miss reaches nuget.org/npm — but
# newly added here. Fail-closed and loud, and still a real availability dependency.
#
# ALREADY-VALIDATED SKIP (ersatztv#420): a second, sibling gate in `test`, `migrations` and
# `functional-e2e` only (NOT `build`). On a push-to-main merge commit, `id: revalidate` runs
# `scripts/ci-detect-already-validated.sh`, which emits `skip=true` only when the merged tree is
@@ -102,7 +115,51 @@ env:
DOTNET_CLI_USE_MSBUILD_SERVER: "0" # no persistent MSBuild server process
MSBUILDDISABLENODEREUSE: "1" # MSBuild worker nodes exit with the build instead of lingering
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Every credentialed thing this file does uses the scoped REGISTRY_* PAT, never the injected token:
# its registry pushes, its five `container:` image pulls, its three commit-status GET steps
# (`ETV_STATUS_AUTH` in jobs `test`, `migrations` and `functional-e2e`, each a read-only GET via
# scripts/ci-detect-already-validated.sh) and its registry tag READ (`ETV_REGISTRY_AUTH` in job
# `toolchain-preflight`, via scripts/ci-toolchain-image-resolves.sh). The injected token therefore
# serves only its eight `actions/checkout` steps. Note this file needs no `packages:` unit for that
# same reason: the `container:` blocks carry explicit `credentials:`.
# (Sites above are named by JOB, not by line number: this file is ~1150 lines, so any edit above a
# citation silently invalidates it — which is how the first version of this comment went stale two
# lines after it was written.)
permissions:
code: read
jobs:
# Answers "is the toolchain image still there?" in ONE place, so a deleted pin does not read as
# five broken jobs and a broken diff (ersatztv#772). Deliberately container-free and deliberately
# NOT a `needs:` of the jobs it diagnoses — see scripts/ci-toolchain-image-resolves.sh for both
# decisions and for the cleanup-rule root cause it cannot fix from this repo.
toolchain-preflight:
name: CI toolchain image resolves
runs-on: small
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Resolve the pinned toolchain tag in the registry
env:
ETV_REGISTRY_AUTH: ${{ secrets.REGISTRY_USER }}:${{ secrets.REGISTRY_PASSWORD }}
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark resolve
scripts/ci-toolchain-image-resolves.sh
- name: Assert every expected step executed (ersatztv#756)
run: >-
scripts/ci-step-ran.sh assert
--always resolve
test:
name: Build & test (.NET)
runs-on: ubuntu-latest
@@ -111,10 +168,14 @@ jobs:
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# git history/tags are needed by the `build` job's `git describe` (ersatztv#190) and,
# here, by the #420 revalidate step's `HEAD^2` tree comparison on a main merge commit.
fetch-depth: 2
@@ -122,14 +183,23 @@ jobs:
# ersatztv#416: is this a docs-only change? If so, every heavy step below is skipped and this
# REQUIRED job reports success in seconds. It still RUNS (never `if:`-skipped) so the required
# context keeps reporting — see the workflow header and docs/ci-cd.md -> "Docs-only skip".
# EVERY consequential `run:` step in this job marks itself as its FIRST act (ersatztv#756),
# and the trailing `Assert every expected step executed` guard fails the job when one is
# missing. This is a REQUIRED context on `main`, and a step the runner drops takes the job
# GREEN having done no work — see scripts/ci-step-ran.sh for why that is fail-OPEN here while
# the same drop in review-verdict.yml is fail-CLOSED.
- name: Detect docs-only changes
id: detect
run: scripts/ci-detect-docs-only.sh
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark detect
scripts/ci-detect-docs-only.sh
- name: Detect already-validated tree (#420)
id: revalidate
env:
ETV_STATUS_AUTH: ${{ secrets.REGISTRY_USER }}:${{ secrets.REGISTRY_PASSWORD }}
run: scripts/ci-detect-already-validated.sh
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark revalidate
scripts/ci-detect-already-validated.sh
- name: Cache NuGet packages
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
@@ -141,7 +211,9 @@ jobs:
- name: Restore
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: dotnet restore
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark restore
dotnet restore
# Replaces setup-node's built-in `cache: npm`. The toolchain image supplies node/npm, but
# the SPA's package downloads are project deps, so they stay cached per lockfile.
@@ -156,36 +228,50 @@ jobs:
- name: Install SPA dependencies
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: npm ci
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark npm-ci
npm ci
- name: Check generated SPA API client
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: npm run check:api
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark check-api
npm run check:api
- name: Lint SPA
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: npm run lint
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark lint
npm run lint
- name: Typecheck SPA
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: npm run typecheck
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark typecheck
npm run typecheck
- name: Test SPA
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: npm test -- --run
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-test
npm test -- --run
- name: Build SPA
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
working-directory: web
run: npm run build
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark web-build
npm run build
- name: Strip Scanner project ref (matches Docker build)
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: sed -i '/Scanner/d' ErsatzTV/ErsatzTV.csproj
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark strip-scanner
sed -i '/Scanner/d' ErsatzTV/ErsatzTV.csproj
# Start the true peak-anon sampler just before the memory-heavy dotnet Build/Test/Coverage so
# its high-water mark spans them (SPA build/test above are comparatively light). Paired with the
@@ -199,13 +285,16 @@ jobs:
- name: Build
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: dotnet build --configuration Release --no-restore
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark build
dotnet build --configuration Release --no-restore
- name: Test
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: >-
dotnet test --configuration Release --no-build --blame-hang-timeout "2m" --verbosity normal
--collect:"XPlat Code Coverage" --settings coverlet.runsettings --results-directory ./coverage
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark dotnet-test
dotnet test --configuration Release --no-build --blame-hang-timeout "2m" --verbosity normal \
--collect:"XPlat Code Coverage" --settings coverlet.runsettings --results-directory ./coverage
# Coverage reporting (ersatztv#15 scope item 4): coverlet.collector emits a Cobertura report
# per test project (via --collect above); ReportGenerator merges them into a human-readable
@@ -258,6 +347,43 @@ jobs:
continue-on-error: true
run: scripts/ci-peak-anon.sh report
# THE DROPPED-STEP GUARD (ersatztv#756). Every `run:` step above records that it began; this
# asserts the whole expected SET was recorded. A step the runner declines to interpolate is
# DROPPED and still concludes `success` (ersatztv#751), so without this a REQUIRED context
# reports green having done no work — fail-OPEN, and strictly worse than the fail-CLOSED
# version of the same bug that #751 fixed in review-verdict.yml.
#
# NO `if:` HERE, WHICH IS A DELIBERATE DEPARTURE FROM THE #751 GUARD and the one decision in
# this block that is easy to "fix" wrongly. That guard uses `if: always()` because its job has
# exactly one real step, so there is no ordinary red for it to talk over. Here there are
# twelve, and a genuine failure in an early one (a lint error, a failing test) SKIPS every
# later step — an `always()` guard would then announce "these steps never executed: typecheck
# web-test build dotnet-test" on top of every normal red build. That is not a dropped step, it
# is the runner doing what it is told, and a guard that cries wolf on every red build is a
# guard that gets deleted.
#
# The default `if:` is `success()`, which is exactly the condition wanted, and the invariant it
# rests on is worth stating because it is what makes the omission safe rather than lucky: this
# step is skipped ONLY when an earlier step failed, and an earlier step failing already fails
# the job. So `guard skipped => job red`, and the only path to a green job runs the guard. A
# dropped step is invisible precisely because it concludes `success`, which keeps the job green
# and therefore reaches here.
#
# ITS OWN BODY CANNOT BE DROPPED BY THE MECHANISM IT GUARDS AGAINST: it is a single command
# with no expression delimiter anywhere in the scalar, so the runner has nothing to rewrite.
# The two gate values come in through `env:`, which is interpolated PER VALUE — a bad payload
# there cannot take the body with it (`ci.workflow-run-body-no-expressions`), and both paths
# are held to naming a real context by
# test_every_workflow_expression_names_a_REAL_context_or_function.
- name: Assert every expected step executed (ersatztv#756)
env:
ETV_DOCS_ONLY: ${{ steps.detect.outputs.docs_only }}
ETV_REVALIDATE_SKIP: ${{ steps.revalidate.outputs.skip }}
run: >-
scripts/ci-step-ran.sh assert
--always detect revalidate
--gated restore npm-ci check-api lint typecheck web-test web-build strip-scanner build dotnet-test
migrations:
name: EF migration integrity (SQLite + MySql)
runs-on: ubuntu-latest
@@ -318,24 +444,35 @@ jobs:
--health-interval=5s
--health-timeout=5s
--health-retries=30
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# was the default fetch-depth: 1 (ersatztv#190); bumped to 2 so the #420 revalidate
# step's `HEAD^2` tree comparison can resolve on a main merge commit.
fetch-depth: 2
# ersatztv#416: docs-only? Skip the build + migration replay; the job still reports success in
# seconds. REQUIRED context, so it always RUNS (never `if:`-skipped). See the workflow header.
# Same per-step marker contract as the `test` job above (ersatztv#756) — this is the other
# REQUIRED context, so a dropped migration-replay step would report EF integrity green having
# replayed nothing.
- name: Detect docs-only changes
id: detect
run: scripts/ci-detect-docs-only.sh
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark detect
scripts/ci-detect-docs-only.sh
- name: Detect already-validated tree (#420)
id: revalidate
env:
ETV_STATUS_AUTH: ${{ secrets.REGISTRY_USER }}:${{ secrets.REGISTRY_PASSWORD }}
run: scripts/ci-detect-already-validated.sh
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark revalidate
scripts/ci-detect-already-validated.sh
- name: Cache NuGet packages
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
@@ -347,11 +484,15 @@ jobs:
- name: Restore
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: dotnet restore
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark restore
dotnet restore
- name: Build
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: dotnet build --configuration Release --no-restore
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark build
dotnet build --configuration Release --no-restore
# dotnet-ef is baked into the CI toolchain image (docker/ci/Dockerfile) and already on PATH
# — no per-run `dotnet tool install`. Bump its version there (ersatztv#390).
@@ -361,6 +502,7 @@ jobs:
if: steps.detect.outputs.docs_only != 'true' && steps.revalidate.outputs.skip != 'true'
run: |
set -euo pipefail
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark sqlite
echo "::group::SQLite model drift (has-pending-model-changes)"
dotnet ef migrations has-pending-model-changes --no-build --configuration Release \
--context TvContext --startup-project ErsatzTV --project ErsatzTV.Infrastructure.Sqlite -- --provider Sqlite
@@ -384,6 +526,7 @@ jobs:
MySql__ConnectionString: "Server=mysql;Port=3306;Database=ersatztv_migrations;Uid=root;Pwd=ersatztv;DefaultCommandTimeout=300;"
run: |
set -euo pipefail
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark mysql
echo "::group::MySql model drift (has-pending-model-changes)"
dotnet ef migrations has-pending-model-changes --no-build --configuration Release \
--context TvContext --startup-project ErsatzTV --project ErsatzTV.Infrastructure.MySql -- --provider MySql
@@ -418,6 +561,42 @@ jobs:
# how the original defects escaped. The fixture itself is retained and is opt-in via
# ETV_TEST_MYSQL_CONNECTION (skipped, visibly, without it). Re-arming it here is tracked by #627.
# THE DROPPED-STEP GUARD (ersatztv#756). Every `run:` step above records that it began; this
# asserts the whole expected SET was recorded. A step the runner declines to interpolate is
# DROPPED and still concludes `success` (ersatztv#751), so without this a REQUIRED context
# reports green having done no work — fail-OPEN, and strictly worse than the fail-CLOSED
# version of the same bug that #751 fixed in review-verdict.yml.
#
# NO `if:` HERE, WHICH IS A DELIBERATE DEPARTURE FROM THE #751 GUARD and the one decision in
# this block that is easy to "fix" wrongly. That guard uses `if: always()` because its job has
# exactly one real step, so there is no ordinary red for it to talk over. Here a genuine
# failure in an early step (a failing `dotnet build`, a MySql replay error) SKIPS every later
# step — an `always()` guard would then announce "these steps never executed: sqlite mysql" on
# top of every normal red build. That is not a dropped step, it is the runner doing what it is
# told, and a guard that cries wolf on every red build is a guard that gets deleted.
#
# The default `if:` is `success()`, which is exactly the condition wanted, and the invariant it
# rests on is worth stating because it is what makes the omission safe rather than lucky: this
# step is skipped ONLY when an earlier step failed, and an earlier step failing already fails
# the job. So `guard skipped => job red`, and the only path to a green job runs the guard. A
# dropped step is invisible precisely because it concludes `success`, which keeps the job green
# and therefore reaches here.
#
# ITS OWN BODY CANNOT BE DROPPED BY THE MECHANISM IT GUARDS AGAINST: it is a single command
# with no expression delimiter anywhere in the scalar, so the runner has nothing to rewrite.
# The two gate values come in through `env:`, which is interpolated PER VALUE — a bad payload
# there cannot take the body with it (`ci.workflow-run-body-no-expressions`), and both paths
# are held to naming a real context by
# test_every_workflow_expression_names_a_REAL_context_or_function.
- name: Assert every expected step executed (ersatztv#756)
env:
ETV_DOCS_ONLY: ${{ steps.detect.outputs.docs_only }}
ETV_REVALIDATE_SKIP: ${{ steps.revalidate.outputs.skip }}
run: >-
scripts/ci-step-ran.sh assert
--always detect revalidate
--gated restore build sqlite mysql
functional-e2e:
name: Functional E2E (curl + UI contracts)
runs-on: ubuntu-latest
@@ -435,10 +614,14 @@ jobs:
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# bumped from 1 (ersatztv#190 default) so the #420 revalidate step's `HEAD^2` tree
# comparison can resolve on a main merge commit.
fetch-depth: 2
@@ -529,6 +712,111 @@ jobs:
# server. Its exit status is Playwright's.
scripts/e2e-ui.sh
# THE DELIMITER BAN, RE-CHECKED ON THE RELEASE PATH ITSELF (ersatztv#767).
#
# The ban that keeps `build`'s `Smoke + IPTV E2E` from being silently dropped was enforced only by
# `test_the_delimiter_banned_jobs_have_NO_expression_delimiter_in_any_run_body` in the
# `script-tests` job of pr-checks.yml — `on: pull_request`, and NOT a required context. So the ban
# was REVIEW-TIME only: nothing re-checked it on a `v*` tag push, which is precisely when the
# candidate image is published and `DeployStack jazz-media` promotes it.
#
# WHY A JOB AND NOT A STEP INSIDE `build`. A step cannot protect the thing it shares a job with:
# `build` is what publishes, so a guard step there fails OPEN if the runner drops it, and "my body
# has no opener so I cannot be dropped" is circular when the only thing enforcing that property is
# the same PR-only test being backstopped. As a `needs:` of `build`, a red here means `build` never
# runs at all — the image is not built, let alone pushed. Fail-closed by dependency, not by
# assertion.
#
# WHY IT RUNS THE REAL PYTEST rather than a bespoke scanner. The first cut of #767 hand-parsed the
# workflow YAML in stdlib Python, to avoid provisioning PyYAML on `build`'s bare runner. Two
# independent reviews found ~10 false NEGATIVES in that parser within one round (flow mappings
# `{run: …}`, a quoted `"run":` key, aliases, multiline quoted scalars) — i.e. it was strictly
# WEAKER than the check it was meant to backstop, in the one direction that matters for a security
# gate. Running the existing PyYAML-based test needs no second implementation of "what is a `run:`
# body" and therefore has no drift surface. `small` is git-only, so Python is provisioned here the
# same way `script-tests` does it.
#
# This job's OWN steps carry #756 markers and a trailing assert, so a drop inside THIS job is
# caught too. That terminates the regress at the same axiom the sibling guards already rest on —
# to fail open you must now drop the pytest step AND the assert step, rather than either one.
#
# THIS PUTS A `small`-LANE JOB BACK ON THE TAG PATH, which ersatztv#535 deliberately moved away
# from — say so rather than letting it look accidental. #535 split the git-only gates into
# pr-checks.yml because on the v26.12.0 tag they wedged in act's setup phase, were killed, and
# reported `failure` with no logs. The blast radius here is WORSE than it was then: as a `needs:`
# of `build`, that flake would not merely redden a status, it would skip the build and produce no
# release image at all.
#
# It is acceptable now for a stated reason rather than an assumed one, and the evidence is weaker
# than it first looks — so read the limits. Per `ci.small-lane-git-only`, the lane's per-job cap was
# forced to 10g by its two HEAVIEST members (this file's `build` AND ci-image.yml's toolchain
# buildx), not by `build` alone, and that cap is what pinned the lane to one slot on a 25 GiB host;
# both were moved off in server-management#639, after which the lane is git-only and runs wide and
# tiny. What has NOT been demonstrated is this lane on a TAG PUSH: `script-tests` runs there happily
# but lives in pr-checks.yml (`on: pull_request`), so it has never exercised the condition #535
# measured, and #767's own runs (1928/1929) were `workflow_dispatch` on a scratch branch. The
# lane-width argument is what carries this, not a like-for-like observation. If the wedging returns,
# move this job to `ubuntu-latest` rather than weakening the `needs:` edge — a slower gate is fine,
# an optional one is not.
scan:
name: Delimiter ban (release path)
runs-on: small
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install test dependencies
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark deps
python3 -m pip install --disable-pip-version-check --quiet pytest pyyaml
# The ban test plus the structural tests that hold this job's own shape. NOT the whole
# scripts/tests suite: that is `script-tests`'s job, it needs jq/git preflights, and an
# unrelated pytest regression must not be able to block a release.
- name: Run the delimiter-ban tests
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark ban
PYTHONPATH=. python3 -m pytest scripts/tests/test_ci_dropped_step_guard.py scripts/tests/test_ci_release_path_scan_job.py -q
# THE POSITIVE CONTROL, and it is deliberately NOT a test (ersatztv#767). The step above proves
# the ban HOLDS; it cannot prove the ban would NOTICE. Review disarmed the entire gate with one
# repo-root `pytest.ini` (`addopts = -k "not delimiter_banned"`) or `conftest.py`
# (`pytest_collection_modifyitems`), which deselects the ban test and every test guarding it,
# leaving all jobs green with a delimiter sitting in `Smoke`. Nothing inside pytest can be
# trusted to catch that, because pytest's own configuration outranks it.
#
# So this poisons the checked-out workflow, re-runs the SAME command, and fails the job if it
# PASSES. It runs in the real checkout — an isolated copy does not inherit the repo-root config
# a disarm would live in, which made the first version of this script report healthy while the
# job's real invocation was deselected. The workflow file is restored by an EXIT trap.
- name: Prove the ban would DETECT a delimiter (ersatztv#767)
run: |
"${GITHUB_WORKSPACE:-.}/scripts/ci-step-ran.sh" mark selfcheck
scripts/ci-prove-ban-detects.sh
# No `if:` — see the sibling guards in `test`/`migrations` for why the default `success()` is
# the wanted condition. Both keys are `--always`: every step in this job is unconditional.
#
# THE MARKER-PATH RATIONALE DOES NOT TRANSFER HERE, and assuming it did would be the mistake
# `ci.required-job-step-execution-markers` itself warns about. That record says the run-id and
# attempt keying is "defence in depth" because "these jobs get a fresh container, which is the
# primary protection". This job has NO `container:` — it is on `small`, where RUNNER_TEMP is
# the shared host /tmp. So here the keying is the ONLY protection, and the residual is a
# single-job re-run that does not increment GITHUB_RUN_ATTEMPT: it would find the previous
# attempt's marker file and the assert would pass even had the pytest step been dropped.
# Identity was read off a real run rather than assumed — run 1929 printed
# `Marker identity: job=scan run=1929 attempt=1 (from the runner)`, so all three variables are
# populated on this lane.
- name: Assert every expected step executed (ersatztv#756)
run: >-
scripts/ci-step-ran.sh assert
--always deps ban selfcheck
build:
name: Build & push image (amd64)
# Moved back off `small` (server-management#639). This is the one HEAVY job that
@@ -540,17 +828,24 @@ jobs:
#
# The `ubuntu-latest` queueing that sent it to `small` in the first place
# (server-management#574: a PR-run skip stuck 31 min behind long builds) does not
# come back, because `needs: [test, migrations]` means this job cannot be
# dispatched until those two have already finished — by which point the lane it
# come back, because `needs: [test, migrations, scan]` means this job cannot be
# dispatched until those three have already finished — by which point the lane it
# was queueing behind has drained. Real builds (main/tags) get the full
# ubuntu-latest allotment: 4 CPUs / 10g on ci-runner (.127).
runs-on: ubuntu-latest
needs: [test, migrations]
# `scan` (ersatztv#767) re-checks the delimiter ban on the release path. As a `needs:` its red
# SKIPS this job outright, so a delimiter in `Smoke + IPTV E2E` can no longer reach the point
# where an image is published and never booted.
needs: [test, migrations, scan]
if: github.event_name != 'pull_request'
env:
CI_EXECUTION_CLASS: bare-runner
CI_JOB_ROLE: none
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
# ersatztv#416: a docs-only push to main has nothing to rebuild (docs are not in the image),
@@ -616,11 +911,33 @@ jobs:
cache-from: type=registry,ref=192.168.1.95:3000/timothy/ersatztv:buildcache
cache-to: type=registry,ref=192.168.1.95:3000/timothy/ersatztv:buildcache,mode=max,ignore-error=true
# THE TWO VALUES COME IN THROUGH `env:`, NOT INLINE (ersatztv#756). This step runs AFTER
# `Build and push`, so on a `v*` tag the image is already in the registry as the release
# candidate — and it is this smoke run that decides whether the candidate was ever booted at
# all. A stray expression delimiter anywhere in this body (a comment is not inert — #751) would
# DROP the step and conclude the job `success`: a candidate published, never smoke-tested, and
# `DeployStack jazz-media` promotes exactly that image. `env:` is interpolated PER VALUE, so a
# bad payload there fails that value instead of taking the whole body with it, and with the
# body delimiter-free the class is unreachable here — held by
# test_the_delimiter_banned_jobs_have_NO_expression_delimiter_in_any_run_body.
#
# The ban IS re-checked on the release path now (ersatztv#767): the `scan` job above runs the
# PyYAML-based ban test and is a `needs:` of this job, so a delimiter here means `build` never
# runs and no image is published. Do not re-add the note that once stood here saying the ban is
# "review-time only, tracked as #767" — that was true before the `scan` job existed.
#
# This step still carries no per-step markers, and that is a genuine (smaller) residual rather
# than a dismissal: markers would additionally catch a drop caused by something OTHER than a
# delimiter. Adding them needs a bucket modelling this step's publish-ref `if:`, which the
# guard's always/gated buckets do not express. The delimiter class itself is covered.
- name: Smoke + IPTV E2E (assert key endpoints)
if: ${{ (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')) && steps.detect.outputs.docs_only != 'true' }}
env:
SMOKE_SHORT_SHA: ${{ steps.meta.outputs.short }}
SMOKE_RUN_ID: ${{ github.run_id }}
run: |
IMG="${IMAGE}:${{ steps.meta.outputs.short }}"
NAME="etv-smoke-${{ github.run_id }}"
IMG="${IMAGE}:${SMOKE_SHORT_SHA}"
NAME="etv-smoke-${SMOKE_RUN_ID}"
trap 'docker rm -f "$NAME" >/dev/null 2>&1 || true' EXIT
echo "Pulling ${IMG}"
docker pull "$IMG"
@@ -711,18 +1028,28 @@ jobs:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
if: github.event_name == 'pull_request'
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Detect API-surface changes
id: detect
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only "origin/${base_ref}...HEAD" 2>/dev/null || true)"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only "origin/${base_ref}...HEAD")"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
echo "Changed files in this PR:"; printf '%s\n' "$changed"
if printf '%s\n' "$changed" | grep -Eq '^ErsatzTV/Controllers/Api/|^ErsatzTV\.Core/Api/'; then
echo "api_changed=true" >> "$GITHUB_OUTPUT"
@@ -806,18 +1133,28 @@ jobs:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
if: github.event_name == 'pull_request'
env:
CI_EXECUTION_CLASS: toolchain
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Detect changed C# files
id: detect
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only --diff-filter=ACM "origin/${base_ref}...HEAD" -- '*.cs' 2>/dev/null || true)"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only --diff-filter=ACM "origin/${base_ref}...HEAD" -- '*.cs')"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
echo "Changed .cs files in this PR:"; printf '%s\n' "$changed"
if [ -n "$changed" ]; then
printf '%s\n' "$changed" > /tmp/changed-cs.txt
+324 -35
View File
@@ -42,40 +42,79 @@ concurrency:
group: ersatztv-pr-gates-${{ github.ref }}
cancel-in-progress: true
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# Holds no secrets at all and reads nothing from the Gitea API; the injected GITEA_TOKEN serves only
# its five `actions/checkout` steps.
permissions:
code: read
jobs:
# BLOCKING (ersatztv#390): the CI toolchain image pin in docker-build.yml must name the image that
# ci-image.yml actually last published — i.e. the short sha of the last commit to touch the image's
# sources. Without this detector, a PR that edits docker/ci/** publishes a NEW image but runs its own
# BLOCKING (ersatztv#390): the CI toolchain image pin in docker-build.yml must name the short sha of
# the last commit to touch the image's SOURCES (`docker/ci/**`). Read that as "the image ci-image.yml
# last published" only under the convention that every such commit is published — this job compares
# git shas and never queries the registry, so it cannot see a pin whose tag was never built or has
# been evicted. Existence is `toolchain-preflight`'s job, and the container jobs' pull is the backstop.
# Since ersatztv#744 publishing from a branch is a `workflow_dispatch`, so "was it published" is a
# human step this job does not observe.
#
# Without this detector, a PR that edits docker/ci/** ships a new image RECIPE while running its own
# jobs against the OLD pin: CI green-lights a toolchain it never executed, and once merged, main's
# Dockerfile silently disagrees with what CI runs. **Renovate actively generates exactly that PR** —
# it manages docker/ci/Dockerfile's base pins (dockerfile manager) but cannot bump an opaque
# `:<sha>` in `container.image`, so it would leave the pin behind every time.
#
# Failing here forces the documented two-step (docs/ci-cd.md -> "CI toolchain image"): push the
# Dockerfile change, let ci-image.yml publish `:<sha>`, then update the pin to that sha. Seconds-long
# git+grep -> keep it off the build runners.
# Failing here forces the documented two-step (docs/ci-cd.md -> "CI toolchain image"): get the
# Dockerfile change published as `:<sha>`, then update the pin to that sha. Since ersatztv#744 the
# publish half of that two-step is a `workflow_dispatch` on the branch rather than a side effect of
# the push — ci-image.yml's `push` trigger is now `branches: [main]`. Seconds-long git+grep -> keep
# it off the build runners.
ci-image-pin:
name: CI image pin matches docker/ci
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
# need real history: `git log -- <path>` on a shallow clone can't find the last
# commit that touched the image sources
fetch-depth: 0
- name: Verify the pin matches the last-published image
- name: Verify the pin matches the image-source commit
run: |
set -euo pipefail
# ci-image.yml tags the image `git rev-parse --short HEAD` of the push that built it, and it
# only builds on pushes touching these paths — so the published image is named by the last
# commit to touch them.
# ci-image.yml tags the image `git rev-parse --short HEAD` of the run that built it. Only
# its filtered `push` clause requires a `docker/ci/**` change; the weekly `schedule` and a
# `workflow_dispatch` both build the selected ref's HEAD whatever it touched. So `expected`
# is not a model of every tag in the registry — it is the one tag a PR is REQUIRED to be
# pinned to: the last commit to change the image's sources.
#
# `.gitea/workflows/ci-image.yml` is deliberately NOT part of `expected` (ersatztv#744),
# and that is a DECIDED TRADEOFF, not a necessity. Keeping it is workable — dispatch the
# branch at the ci-image.yml commit, then pin it — but it prices every edit to that file,
# comments included, at a full ~2GB publish plus a five-pin bump, redone after every
# rebase. Dropping it prices the opposite risk: a change to HOW the image is built living
# ONLY in ci-image.yml (build-args, Dockerfile path, platforms) neither republishes nor
# invalidates the pin, so CI keeps running an image built by the previous recipe. The
# second was chosen because that file is edited far more often for triggers, comments and
# runner placement than for build recipe. Make a recipe change alongside a `docker/ci/**`
# edit — a comment bump suffices, and it is the ONLY remedy: pinning the workflow-only
# commit is rejected here, because `expected` is the last `docker/ci` commit.
# Nothing MECHANICALLY couples this pathspec to `ci-image.yml`'s `on.push.paths`; before
# #744 the shared self-reference kept them in step. Divergence is silent and green in the
# dangerous direction — tracked in ersatztv#855.
# See docs/ci-cd.md -> "Publishing from a branch is a dispatch, not a push".
#
# Compare RESOLVED FULL shas, never the abbreviations: git auto-scales abbreviation length
# with the repo's object count, so the tag built in CI from a `fetch-depth: 1` shallow clone
# is 7 chars while `%h` here (full clone) is 8. Comparing those strings would fail always.
expected="$(git log -1 --format=%H -- docker/ci .gitea/workflows/ci-image.yml)"
expected="$(git log -1 --format=%H -- docker/ci)"
mapfile -t pins < <(grep -oE 'ersatztv-ci:[0-9a-f]+' .gitea/workflows/docker-build.yml | cut -d: -f2 | sort -u)
echo "Image sources last changed in: ${expected}"
echo "Pins found in docker-build.yml: ${pins[*]} (${#pins[@]} distinct)"
@@ -107,10 +146,10 @@ jobs:
# in-repo remedy in that state: relax this length check in the same PR and say why. Note
# that ci-image.yml still tags with a plain `--short` (auto-scaled), so "always 7" is an
# empirical property of today's shallow clone, not an enforced invariant. Making the
# publisher emit `--short=7` is tracked as ersatztv#597. It is not blocked, just out of
# scope here: editing ci-image.yml re-points `expected` (above) at that commit, so it needs
# the branch's own publish-then-pin two-step (docs/ci-cd.md -> 'CI toolchain image') —
# ci-image.yml's push trigger has no branches: filter, so a feature branch does publish.
# publisher emit `--short=7` is tracked as ersatztv#597. That is no longer blocked by this
# job at all: since ersatztv#744, editing ci-image.yml does NOT re-point `expected`, so a
# `--short=7` change lands like any other PR. It does need a deliberate republish to take
# effect — see the note on `expected` above.
if [ "${#pins[0]}" -ne 7 ]; then
echo "::error::CI toolchain image pin ersatztv-ci:${pins[0]} is ${#pins[0]} chars, but ci-image.yml publishes 7-char tags (it tags with 'git rev-parse --short HEAD' from a fetch-depth:1 clone). A differently-sized abbreviation still resolves to the right commit, so this would pass every other check here — but NO such tag exists in the registry, and all five container: jobs would fail at image-pull time with 'manifest unknown'. Pin exactly: ersatztv-ci:${expected:0:7} (locally: git rev-parse --short=7 HEAD). See docs/ci-cd.md -> 'CI toolchain image'."
exit 1
@@ -121,7 +160,7 @@ jobs:
exit 1
fi
if [ "$pin_full" != "$expected" ]; then
echo "::error::CI toolchain image pin is stale: docker-build.yml pins ersatztv-ci:${pins[0]} ($pin_full), but docker/ci was last changed in $expected. Your jobs are testing an image that is NOT built from this PR's docker/ci. Let ci-image.yml publish the new :<sha>, then update the pin in ALL jobs to it (docs/ci-cd.md -> 'CI toolchain image')."
echo "::error::CI toolchain image pin is stale: docker-build.yml pins ersatztv-ci:${pins[0]} ($pin_full), but docker/ci was last changed in $expected. Your jobs are testing an image that is NOT built from this PR's docker/ci. Publish the new :<sha> — push this commit as branch HEAD and dispatch ci-image.yml on the branch (a branch PUSH no longer publishes, ersatztv#744) — then update the pin in ALL jobs to it (docs/ci-cd.md -> 'CI toolchain image')."
exit 1
fi
echo "Pin is current: ersatztv-ci:${pins[0]} resolves to $pin_full = docker/ci's last change."
@@ -134,16 +173,32 @@ jobs:
name: Docs update reminder
runs-on: small # seconds-long git diff; keep it off the build runners
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: report-only
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
# `continue-on-error` for the same reason the two steps below carry it: this whole job
# is a non-blocking nudge, and an advisory red still joins the combined status the merge gate
# reads. Unmasking the fetch (ersatztv#746) makes a broken base LOUD in the log; it must not
# also make a warn-only job merge-blocking. The three jobs that genuinely gate on this diff —
# api-docs, format, decisions lifecycle — do redden on a failed fetch, which is where that
# belongs.
- name: Warn when a screen/route change skips the parity doc
continue-on-error: true
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=100 origin "$base_ref" || true
changed="$(git diff --name-only "origin/${base_ref}...HEAD" 2>/dev/null || true)"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
if ! changed="$(git diff --name-only "origin/${base_ref}...HEAD")"; then
echo "::error::git diff against origin/${base_ref} failed, so the changed-file set could not be computed — do not read this as 'nothing changed' (ersatztv#746). If it reports no merge base, rebase this branch onto ${base_ref}."
exit 1
fi
echo "Changed files in this PR:"; printf '%s\n' "$changed"
screen_or_route=no
if printf '%s\n' "$changed" | grep -Eq '^web/src/screens/.+\.tsx$|^ErsatzTV/LegacyUiRedirects\.cs$'; then
@@ -159,6 +214,40 @@ jobs:
echo "Parity-doc reminder: nothing to flag."
fi
# ersatztv#784 — ADVISORY nudge for `docs.no-session-narrative`. Deliberately NON-BLOCKING and
# deliberately in this job rather than a gate of its own: it is a string predicate over prose,
# and `docs/defect-shapes-773.md` §4 argues that class must not be load-bearing. The script
# exits 0 on every path (asserted per argument shape in scripts/tests/test_check_doc_narrative.py,
# not only in prose), so this step cannot redden the run even on a hit; if you find yourself
# wanting it to fail, read the decision record first — it says no in as many words.
# `python3` is not guaranteed on the bare `small` lane (docs/ci-cd.md), and every other
# python-using job on it declares this. Without it a missing interpreter is exit 127 — a RED
# advisory job joining the combined status, which is the one thing this step must never be.
#
# Both steps OF THIS CHECK (setup-python + the narrative step; the parity nudge above has its
# own) carry `continue-on-error` because the SCRIPT exiting 0 is not the whole invariant:
# a setup-python download failure reddens the job just as effectively as a hit would, and an
# advisory red still joins the combined status the merge gate reads (ersatztv#598). Scope,
# stated rather than implied: this covers the two steps that exist to run the check. A failed
# `Checkout` is NOT covered and deliberately so — with no tree there is nothing to check, and
# a job that cannot run is a different failure from an advisory one that ran and disagreed.
# Measured on this runner (PR#811, run 2179): the job reports `success` and the commit status
# context is `success` with both steps green under `continue-on-error`.
- name: Set up Python
uses: actions/setup-python@v5
continue-on-error: true
with:
python-version: '3.x'
- name: Warn when a doc narrates its own revision history
continue-on-error: true
run: |
base_ref="${{ github.base_ref }}"
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
python3 scripts/check-doc-narrative.py --diff "origin/${base_ref}"
# BLOCKING (ersatztv#521, supersedes the ersatztv#303 H9 append-only mechanic): validates decision-
# record lifecycle invariants (metadata schema, one active record per key, reciprocal
# supersedes/superseded-by links, no rationale-prose rewrite without a Decisions-Edit: yes git
@@ -170,10 +259,13 @@ jobs:
name: decisions lifecycle
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
@@ -182,7 +274,10 @@ jobs:
- name: Validate decision lifecycle
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=200 origin "$base_ref" || true
if ! git fetch --no-tags origin "$base_ref"; then
echo "::error::git fetch of origin/${base_ref} failed, so this job cannot compute the changed-file set it derives its work from. That is a broken job, not an empty change set (ersatztv#746). Check the base branch still exists and that the runner can reach the repository."
exit 1
fi
PYTHONPATH=. python3 scripts/decisions_validate.py --base "origin/${base_ref}" --head HEAD
- name: Active catalog in sync
run: PYTHONPATH=. python3 scripts/build_decisions_catalog.py --check
@@ -211,17 +306,225 @@ jobs:
# `.claude/hooks/pretooluse-merge-consent.sh`, so its true input set spans at least two top-level
# directories. A `scripts/**` filter would silently miss a `.claude/hooks/**` edit — and at ~10s a
# filter buys nothing but drift.
script-tests:
name: Script tests (pytest)
prove-fix:
name: "Fix proofs (Proves trailers)"
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
# Full history: prove-fix.sh reverts each commit against its PARENT, so a shallow
# clone would leave it unable to resolve `<sha>^` and it would refuse every commit.
uses: actions/checkout@v4
with:
persist-credentials: false
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Install test dependencies
run: python3 -m pip install --disable-pip-version-check --quiet pytest pyyaml
# OPT-IN BY TRAILER, deliberately. Requiring `Proves:` on every commit would block
# docs, CI and refactor commits that have no code side to revert, and a gate that
# blocks ordinary work gets disabled — which is how a check ends up running nowhere
# (#631). So the trailer is the AUTHOR'S CLAIM, and this job checks claims: write
# one and it must hold. Coverage is therefore honest rather than assumed, and
# `docs/decisions/records/testing/fix-ships-a-witnessed-red-test.md` says so.
- name: Prove every commit that claims a proof
run: |
set -uo pipefail
base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}"
echo "range: $base..$head"
# Capture and VALIDATE the enumeration before looping. `for sha in $(git ...)`
# swallows a git failure: the command substitution yields nothing, the loop body
# never runs, and the job reports "0 claims" green. Fail-open enumeration in the
# thing that decides what gets checked is the defect this job exists to catch.
if ! shas="$(git rev-list "$base".."$head")"; then
echo "::error::git rev-list failed for $base..$head — cannot enumerate commits," \
"so this job cannot assert anything. Refusing to pass."
exit 1
fi
claimed=0; proven=0; failed=0
while IFS= read -r sha; do
[ -n "$sha" ] || continue
# Trim whitespace only — NOT `xargs`, which applies quote parsing and turns a
# legitimate parametrised node id like test_x[can't] into an empty selector,
# silently dropping a real claim.
# Extract with a CHECKED status. `sel="$(git show ... )"` under `set -uo
# pipefail` but no `-e` yields an empty selector when git fails, the commit is
# skipped, and the job exits 0 having been unable to inspect a possible claim —
# fail-open in the step that decides what gets checked.
if ! raw="$(git show -s --format='%(trailers:key=Proves,valueonly)' "$sha")"; then
echo "::error::git show failed for $sha — cannot read its trailers, so this" \
"job cannot assert anything about it. Refusing to pass."
exit 1
fi
# Refuse MORE THAN ONE `Proves:` here too. prove-fix.sh has this guard, but it
# only fires when it reads the trailer itself — and this job passes the selector
# explicitly, so the guard was bypassed on the one path that actually enforces.
# Measured: a commit with two trailers reported PROVEN while the second was never
# run. Fixing the script and not its twin is how a guard reads as coverage.
# Count trailer PRESENCE, not non-empty values: `%(...valueonly)` renders a bare
# `Proves:` as an empty line, so counting non-empty lines misses a commit whose
# FIRST trailer is empty — `sel` then comes out empty and the commit is skipped
# in silence, with a real second selector never checked. Fail-open in CI while
# the script is fail-closed is the same asymmetry this guard exists to remove.
present="$(git show -s --format='%(trailers:key=Proves)' "$sha")"
if [ "$(printf '%s\n' "$present" | grep -c .)" -gt 1 ]; then
claimed=$((claimed + 1)); failed=$((failed + 1))
echo "::error::commit $sha carries more than one 'Proves:' trailer; only the" \
"first would be checked, so the rest would read as proven without ever" \
"running. Use a single selector."
continue
fi
sel="$(printf '%s\n' "$raw" | head -1 | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
# A trailer that is PRESENT but empty is a claim with no selector. Refuse it
# loudly; skipping it silently would let the job report "no claims" for a PR that
# made one.
if [ -n "$present" ] && [ -z "$sel" ]; then
claimed=$((claimed + 1)); failed=$((failed + 1))
echo "::error::commit $sha carries a 'Proves:' trailer with no selector."
continue
fi
[ -n "$sel" ] || continue
claimed=$((claimed + 1))
# A merge commit has several parents, so "before this change" is ambiguous.
# prove-fix.sh refuses them; catch it here with a clearer message rather than
# letting the trailer be silently skipped (which --no-merges used to do).
if [ "$(git rev-list --parents -n 1 "$sha" | wc -w)" -gt 2 ]; then
failed=$((failed + 1))
echo "::error::commit $sha is a MERGE carrying 'Proves: $sel'. Put the trailer" \
"on the commit that carries the fix — a merge has no single 'before'."
continue
fi
echo "::group::prove $sha -> $sel"
if bash ./scripts/prove-fix.sh "$sha" "$sel"; then
proven=$((proven + 1)); echo "PROVEN $sha"
else
rc=$?
failed=$((failed + 1))
echo "::error::commit $sha claims 'Proves: $sel' but prove-fix.sh exited $rc." \
"A claimed proof that does not hold is worse than none — it reads as" \
"coverage. Strengthen the test until reverting the fix reddens it, or" \
"drop the trailer."
fi
echo "::endgroup::"
done <<< "$shas"
echo "commits claiming a proof: $claimed (proven $proven, failed $failed)"
if [ "$claimed" -eq 0 ]; then
echo "::notice::No commit in this PR carries a 'Proves:' trailer, so nothing was" \
"verified here. That is allowed — the trailer is opt-in — but it means this" \
"job asserts NOTHING about this PR. Do not read its green as fix coverage."
fi
[ "$failed" -eq 0 ]
script-tests:
name: Script lint and tests (ruff + pytest)
runs-on: small
if: github.event_name == 'pull_request'
env:
CI_JOB_ROLE: guard
steps:
- name: Checkout
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
# Preflight, not an install (ersatztv#390 removed run-time `apt-get` from CI on purpose).
# Two consumers need `git`: the lint steps below derive their population from `git ls-files`,
# and test_post_review_verdict.py / test_merge_consent_exemption.py exec the REAL
# post-review-verdict.sh / pretooluse-merge-consent.sh. `curl` those tests shim on PATH; `jq`
# and `git` they do NOT. It stays AHEAD of the lint steps, not merely ahead of pytest: without
# it, a missing git reaches the lint steps as an empty population, which they report as a
# population problem. One actionable line beats a misdirected one, and beats the wall of
# unattributable assertion failures the suite produces without git.
- name: Preflight external tools
run: |
if ! command -v git >/dev/null 2>&1; then
echo "::error::script-tests needs git on PATH but it is absent. The lint steps derive" \
"their population from it and the suite execs real shell scripts that use it." \
"Bake it into the runner image rather than apt-get installing here (ersatztv#390)."
exit 1
fi
echo "Preflight OK: $(git --version)"
# ersatztv#780. Lint runs EARLY — after the git preflight it depends on, but before the test
# dependencies, the jq preflight and the ~4-minute pytest run. A style red therefore arrives in
# seconds, and, more importantly, the lint does not sit behind `Preflight jq version`: that is
# an `--expect` tripwire, so a runner jq bump would take the lint dark for as long as the jq
# contract is broken, under a red that says "jq".
#
# The version is PINNED: an unpinned ruff makes the verdict a function of whenever the job ran
# — the same environment-divergence the committed ruff.toml exists to close. Bumping it is a
# deliberate PR (new rules may fire), exactly like the jq pin below. `pytest`/`pyyaml` are
# deliberately NOT pinned: a pytest release does not add assertions to your suite, a ruff
# release adds rules to your lint.
- name: Install ruff
run: python3 -m pip install --disable-pip-version-check --quiet 'ruff==0.12.11'
# POPULATION. Both steps lint an EXPLICIT list from `git ls-files`, never `ruff check .`, and
# pass `--no-force-exclude`. Measured with ruff 0.12.11 and `exclude = ["scripts/**"]` — a
# per-FILE pattern, because `exclude` matches per file: a bare `["scripts"]` still works at the
# top level but matches nothing under `[lint]`/`[format]`. The subject is a planted tracked file
# holding an unused import, a hardcoded credential and a formatting error. GREEN means the gate
# was silently off:
#
# DISCOVERY FORM EXPLICIT FORM (what ships)
# exclude scope check . format --check . check format --check
# top-level GREEN GREEN red red
# [lint] GREEN red red red
# [format] red GREEN red red
# top + force-exclude GREEN GREEN red red <- with the flag
# GREEN GREEN <- without it
#
# Only the top-level scope empties BOTH discovery commands; `[lint]` empties `check` and
# `[format]` empties `format --check`, so in those two the job would still redden on the other
# step. `[format]` is where a line appended to ruff.toml lands, by TOML rules. `include = []`,
# `extend-exclude` and a nested `scripts/ruff.toml` behave the same way and are equally inert
# against the explicit form. The last row is the whole reason for `--no-force-exclude`:
# `force-exclude = true` re-applies excludes to explicitly-passed paths, and is the one setting
# that reaches explicitly-passed paths at all.
#
# `ruff check .` over an empty tree exits **0** with only a stderr warning, so every GREEN above
# is a gate that was switched off without a red.
#
# This also derives the population from source rather than from the filesystem
# (docs/decisions/records/testing/guard-derives-population-from-source.md) and covers
# tracked-but-gitignored files, which `ruff check .` skips. The empty-population arm is the
# anti-vacuity check: a completeness check whose population is empty reports that it proved
# everything. What it does NOT cover: an emptied RULE set. `select = []` silences every selected
# rule, so the `ruff check` step goes green over any lint violation (a syntax error still reds)
# while printing a reassuring file count.
# `ruff format --check` is unaffected, because formatting is not rule-selected. So half the
# gate is killable by a config edit, and only a human reading that edit catches it.
- name: Lint scripts (ruff check)
run: |
mapfile -d '' -t PYFILES < <(git ls-files -z '*.py' '*.pyi' '*.ipynb')
if [ "${#PYFILES[@]}" -eq 0 ]; then
echo "::error::the lint population is EMPTY — git tracks no Python files. Either the" \
"checkout is wrong or the glob is. A lint over nothing passes; see ersatztv#780."
exit 1
fi
echo "Linting ${#PYFILES[@]} tracked Python files"
python3 -m ruff check --no-force-exclude -- "${PYFILES[@]}"
- name: Lint scripts (ruff format --check)
run: |
mapfile -d '' -t PYFILES < <(git ls-files -z '*.py' '*.pyi' '*.ipynb')
if [ "${#PYFILES[@]}" -eq 0 ]; then
echo "::error::the format population is EMPTY — git tracks no Python files. See ersatztv#780."
exit 1
fi
echo "Format-checking ${#PYFILES[@]} tracked Python files"
python3 -m ruff format --check --no-force-exclude -- "${PYFILES[@]}"
# pytest + PyYAML. PyYAML is NOT a contradiction of the dependency-free decisions READ path:
# `decisions_lib._read_frontmatter` is hand-written precisely so validation runs where nothing
# is installed, but the one-shot WRITE path `migrate_decisions_split.py` uses PyYAML by
@@ -232,20 +535,6 @@ jobs:
# went red in CI on a collection error.
- name: Install test dependencies
run: python3 -m pip install --disable-pip-version-check --quiet pytest pyyaml
# Preflight, not an install (ersatztv#390 removed run-time `apt-get` from CI on purpose).
# test_post_review_verdict.py and test_merge_consent_exemption.py exec the REAL
# post-review-verdict.sh / pretooluse-merge-consent.sh, which shell out to `jq` ~26 times.
# `curl` those tests shim on PATH; `jq` they do NOT. If it were missing, the suite would fail
# as ~20 opaque assertion errors — this turns that into one actionable line.
- name: Preflight external tools
run: |
if ! command -v git >/dev/null 2>&1; then
echo "::error::script-tests needs git on PATH but it is absent. The suite execs real" \
"shell scripts that use it. Bake it into the runner image rather than apt-get" \
"installing here (see ersatztv#390)."
exit 1
fi
echo "Preflight OK: $(git --version)"
# jq gets its OWN step because its VERSION, not merely its presence, is load-bearing
# (ersatztv#648). `--expect` makes this a TRIPWIRE: scripts/tests exercises the jq 1.6 code path
# only because this runner ships 1.6, so an upgrade would silently delete that coverage — and
+14
View File
@@ -45,12 +45,26 @@ concurrency:
group: ersatztv-renovate
cancel-in-progress: false
# Explicit token scope (ersatztv#748) so the owner-level Actions default can move to Restricted
# (server-management#714). Declaring `permissions:` is EXHAUSTIVE, not additive: a unit omitted here
# is NOT granted, and that holds at any owner default — it is not conditional on Restricted being on.
# Only `review-verdict.yml` needs write; it declares that at the job and says why there. Full
# rationale and the per-workflow credential audit: docs/ci-cd.md -> "Workflow token scope".
# This workflow has no checkout step and never uses the injected GITEA_TOKEN for anything. Renovate's
# own branch/PR writes go through RENOVATE_TOKEN, a dedicated bot PAT the Actions default does not
# govern, and its container image comes from Docker Hub. Read-only is declared to STATE that the
# injected token is unused, not because any step needs it.
permissions:
code: read
jobs:
renovate:
name: Renovate
runs-on: ubuntu-latest
container:
image: renovate/renovate:43
env:
CI_JOB_ROLE: none
steps:
- name: Run Renovate
env:
File diff suppressed because it is too large Load Diff
+20
View File
@@ -10,6 +10,10 @@ project.lock.json
# Claude Code
.mcp/
.mcp.json
# Machine-local settings (DOTNET_ROOT and friends — see docs/local-lsp-tooling.md).
# Ignored here rather than relying on a personal ~/.config/git/ignore, so a second
# contributor following that doc cannot accidentally commit their own Homebrew paths.
/.claude/settings.local.json
.agents/
plugins/
nupkg/
@@ -70,6 +74,11 @@ ErsatzTV/wwwroot/app/
web/dist/
web/node_modules
# Root-level link that makes `typescript` resolvable from the repo root, which is
# the LSP workspace root — without it typescript-language-server refuses to start
# (ersatztv#777). See docs/local-lsp-tooling.md.
/node_modules/
# E2E / screenshot scratch (from Playwright/live-E2E runs) — never committed
/*.png
.playwright-mcp/
@@ -86,3 +95,14 @@ web/playwright-report/
# plaintext Gitea credential and absolute /Users paths, so it is neither portable nor safe to
# commit. See ersatztv#711 for the related merge-gate gap.
.codex/
# serena's per-project state, written by `activate_project` (ersatztv#799): project.yml,
# project.local.yml, a language-server cache, and memories/.
#
# This deliberately rejects serena's own versioning model. Its nested .serena/.gitignore excludes
# only `cache` and `project.local.yml`, and project.local.yml says project.yml "is intended to be
# versioned" — but activation here is per DIRECTORY, and every worktree generates a project.yml
# whose project_name is that worktree's folder (e.g. `781-tooling`). A committed copy would name
# the wrong project in every checkout but the one that produced it. memories/ is ignored with it:
# it is serena's own written notes, and this repo's durable knowledge lives in docs/ instead.
.serena/
+3 -2
View File
@@ -12,8 +12,9 @@ unset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE
# H11 (ersatztv#311): refuse to push a branch that is BEHIND origin/main — rebase, don't merge
# main in (a merge drags in files you never touched, e.g. legacy-BOM .cs, and trips the format
# hook on code that isn't yours). Fail-open; escape with ETV_SKIP_REBASE_CHECK=1.
./.claude/hooks/prepush-rebase-check.sh || exit 1
# hook on code that isn't yours). Fail-open; escape with ETV_SKIP_REBASE_CHECK=1. Exempts a
# tag-only push (ersatztv#719) — forward the ref lines captured above so it can tell.
printf '%s\n' "$_prepush_refs" | ./.claude/hooks/prepush-rebase-check.sh || exit 1
# H13 (ersatztv#416 session): refuse to push when a file in the pushed diff still has uncommitted
# working-tree/index changes — the pushed commit wouldn't match what you built/reviewed (the #416
+21 -6
View File
@@ -83,11 +83,13 @@ main in) and re-run the local gate whenever the fetch shows movement.
Every task that closes a Gitea issue MUST complete ALL of these before it is considered done. Use `/done <issue>` to run through this automatically.
**Merge-consent is derived from state, not asserted (`## Done-when` convention — ersatztv#303 H6 + H10).** Any issue whose PR will merge to `main` should carry a `## Done-when` section in its **issue body** — a checklist of completion criteria (always include an "adversarial review passed" box; add per-issue criteria like tests-green, docs-updated, live-E2E). Two hooks derive merge-consent from it so a premature merge is blocked *by construction*, not by memory:
- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), run **`scripts/post-review-verdict.sh <pr> <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> [note]`** — it posts both the `Review-verdict: … @ <head-sha>` comment and the sha-bound `review-verdict/h10` commit status, proving the *latest* commit was reviewed rather than a stale earlier diff (ersatztv#242). Do not hand-write the comment: the **status** is the required check branch protection enforces, and a comment alone leaves it absent.
- `pretooluse-merge-consent.sh` (Claude PreToolUse on the Gitea merge tool) — **auto-grants** a merge (emits `permissionDecision: allow`, so **no** redundant mechanical prompt fires) only when the PR's CI is green **and** every `## Done-when` box on the linked issue (`fixes #N`) is ticked **and** a `Review-verdict:` comment references the PR's *current head sha* (**H10**); **denies** on an unticked box, red CI, or a stale/negative review verdict; **asks** (falls back to a human prompt) when it can't derive state (no linked issue, no `## Done-when` section, no `Review-verdict:` comment yet, no creds, Gitea down). On the auto-grant (satisfied) path the derived state **is** the consent — do not also ask conversationally to merge; a separate human confirmation is warranted only when the gate **asks** (ersatztv#314). **The H10 review-verdict convention**: after an adversarial/Codex review of a PR (or its latest fix commit), run **`scripts/post-review-verdict.sh <pr> <MERGEABLE|APPROVED|LGTM|BLOCKED|NOT-MERGEABLE> [note]`** — it posts both the `Review-verdict: … @ <head-sha>` comment and the sha-bound `review-verdict/h10` commit status, proving the *latest* commit was reviewed rather than a stale earlier diff (ersatztv#242). Do not hand-write the comment: the **status** is the required check branch protection enforces, and a comment alone leaves it absent. **The credential you post with must be an account on `H10_REVIEWERS` in `.gitea/workflows/review-verdict.yml`** (`timothy` today) — since ersatztv#742 the gate inherits an existing `success` only from an allow-listed creator (an existing `failure` is left alone on a weaker attributability test, so an attributable rejection VISIBLE AT THE FIRST READ is not re-derived into a green — a rejection landing later, inside a run's own write window, is a separate and still-open route, ersatztv#849), so a verdict posted with any other account is written, reported as success by the script, and then silently re-derived on the next PR event (ersatztv#845).
- **The gate is enforced server-side, per sha (ersatztv#622).** `review-verdict/h10` is a required status check on `main`. Because a commit status belongs to one sha, a commit pushed *after* an auto-merge is scheduled clears it and blocks the merge — closing the hole where `merge_when_checks_succeed` froze consent at scheduling time and Gitea later merged an unreviewed head. Renovate-authored and docs-only PRs are auto-passed by `.gitea/workflows/review-verdict.yml`, **except** when they touch `.claude/`, `.codex/`, `.gitea/`, `.husky/`, `scripts/` or `docker/ci/`. See `docs/ci-cd.md` → Review-verdict gate.
- `.husky/pre-push``prepush-donewhen.sh` — a fail-open backstop that blocks a direct `git push origin main` whose commits `fix #N` an issue with unticked boxes.
- `.husky/pre-push``prepush-donewhen.sh` — a fail-open backstop that blocks a direct `git push origin main` whose commits `fix #N` an issue with unticked boxes. **Since ersatztv#743 that push can no longer happen at all** (see below), so this hook is now belt-and-braces for a path the server refuses.
Both need Gitea read creds in the env to enforce (**`ETV_GITEA_BASICAUTH=user:pass`** or `ETV_GITEA_TOKEN`; `ETV_GITEA_URL` overrides the base). Without them the merge hook asks and the push backstop is a no-op — the gate degrades to today's manual confirmation, never a silent pass. Docs-only PRs/pushes are exempt.
**`main` is PR-only — there is no direct-push path any more (ersatztv#743, `release.main-direct-push-disabled`).** Branch protection carries `enable_push: false` **and** `block_admin_merge_override: true`: a direct `git push origin HEAD:main` is refused server-side at pre-receive for every account including a site admin, the contents API is refused too, and an admin cannot `force_merge` past a missing or red required context. This is what makes `review-verdict/h10` load-bearing rather than conventional — Gitea only evaluates `status_check_contexts` on the PR merge path, so before this the whole gate was skippable with no forgery. Practically: **every** change to `main` goes through a PR, including a one-line docs fix. Tag pushes are unaffected (separate mechanism), so the release cut is unchanged.
Both need Gitea read creds in the env to enforce (**`ETV_GITEA_BASICAUTH=user:pass`** or `ETV_GITEA_TOKEN`; `ETV_GITEA_URL` overrides the base). Without them the merge hook asks and the push backstop is a no-op — the gate degrades to today's manual confirmation, never a silent pass. Docs-only PRs are exempt from the *review-verdict* gate; the direct-push exemption is moot now that direct pushes are refused outright.
**The 7 mandatory completion steps and the `## Closing record` comment template** live in the
`closing-an-issue` skill (`.claude/skills/closing-an-issue/SKILL.md`) — invoke it (or `/done`)
@@ -95,17 +97,30 @@ when finishing a task that closes an issue.
## Project Boundaries
**ersatztv OWNS**: ErsatzTV fork code (C#/.NET), channel/collection/schedule management, M3U/XMLTV generation, and the **`ersatztv` skill** — whose canonical copy is `.claude/skills/ersatztv/SKILL.md` **here**; `~/server-management/.claude/skills/ersatztv` is a symlink to it (ersatztv#617). Edit it in this repo; never fork a second copy.
**ersatztv OWNS***developing the fork*: the ErsatzTV fork code (C#/.NET), the `/api/v1` REST
surface, M3U/XMLTV generation, the `ErsatzTV.Mcp` server, CI and releases, and the **`ersatztv`
skill** — whose canonical copy is `.claude/skills/ersatztv/SKILL.md` **here**. Both
`~/server-management/.claude/skills/ersatztv` and `~/media-management/.claude/skills/ersatztv` are
symlinks to it (ersatztv#617, #755). Edit it in this repo; never fork a second copy.
**The split that is easy to get wrong** (ersatztv#755, `process.ersatztv-owns-code-not-operations`):
channel/collection/schedule *code* is owned here; **channel OPERATIONS against the running instance
are not**. Creating and editing channels, lineups, collections, schedules, playouts, logos and
overlays on the live ErsatzTV belong to `media-management`. Driving prod from here is in scope only
as *verification of a change this repo is shipping* (live-E2E, a release smoke test) — not as
day-to-day channel work.
**ersatztv does NOT own**:
- Channel/collection/schedule/playout **operations** against a live instance → media-management
- Docker compose configs → server-management (`~/downloadswarm/stacks/ersatztv/`)
- NFS mounts, Ansible, DNS, networking → server-management
- Content sourcing (yt-dlp downloads, Sonarr/Radarr libraries) → media-management (planned)
- Content sourcing (yt-dlp downloads, Sonarr/Radarr libraries) → media-management
- Jellyfin skill → server-management. `.claude/skills/jellyfin` here is a **relative symlink** to `~/server-management/.claude/skills/jellyfin` (ersatztv#617 — it had silently become a stale divergent copy). It therefore resolves only in a checkout at `~/ersatztv`, not inside a git worktree; that is inherent to the cross-repo symlink pattern server-management already uses (`beets`, `radarr`, `sonarr`, …).
**For infrastructure changes** (Docker, NFS, ports, Authelia): open an issue in `timothy/server-management`.
**For content/media sourcing questions** (what goes into channels, yt-dlp pipelines): open an issue in `timothy/media-management` once it exists; for now, `timothy/server-management`.
**For content/media sourcing questions and channel operations** (what goes into channels, yt-dlp
pipelines, editing a live channel): open an issue in `timothy/media-management`.
**For plan/audit reviews**: open `~/adversarial-reviewer` before significant architecture changes.
+1 -1
View File
@@ -75,7 +75,7 @@
<PackageVersion Include="RichTextKit.Stbear" Version="0.4.167.3" />
<PackageVersion Include="Roslynator.Analyzers" Version="4.15.0" />
<PackageVersion Include="Scalar.AspNetCore" Version="2.12.32" />
<PackageVersion Include="Scriban.Signed" Version="7.2.5" />
<PackageVersion Include="Scriban.Signed" Version="7.2.6" />
<PackageVersion Include="Serilog" Version="4.3.0" />
<PackageVersion Include="Serilog.AspNetCore" Version="10.0.0" />
<PackageVersion Include="Serilog.Extensions.Hosting" Version="10.0.0" />
@@ -0,0 +1,43 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
namespace ErsatzTV.Application.Channels;
/// <summary>
/// #732: the On Now / Next overlay is a default rather than an opt-in, so every newly created channel
/// gets the built-in element attached.
/// </summary>
/// <remarks>
/// This lives in one place because there is more than one channel-creation path and they diverged
/// once already: <c>CreateChannelHandler</c> had it and <c>CreateChannelFromLineupHandler</c> -- the
/// SPA's primary "Add Channel" flow, and the one Auto-Tune bulk-creates through -- did not. Any new
/// site that persists a <c>Channel</c> must call this. The third site, <c>DbInitializer</c>'s default
/// channel, needs no call: it runs before <c>AttachOnNowNextByDefault</c> in the same startup, so the
/// backfill covers it.
/// </remarks>
public static class ChannelGraphicsDefaults
{
public static async Task Attach(TvContext dbContext, Channel channel, CancellationToken cancellationToken)
{
// HLS Direct is skipped because ErsatzTV is not transcoding there -- there is no frame
// pipeline to draw into, and the editor disables the toggle for the same reason. Identity is
// the element's filename, never its user-editable Name (the #67 lesson).
if (channel.StreamingMode is StreamingMode.HttpLiveStreamingDirect)
{
return;
}
Option<int> maybeElementId =
await GraphicsElementSeeder.GetBuiltInElementId(dbContext, cancellationToken);
foreach (int elementId in maybeElementId)
{
// Add rather than assign: a future create path that carries graphics ids would otherwise
// be silently discarded here.
channel.ChannelGraphicsElements ??= [];
channel.ChannelGraphicsElements.Add(new ChannelGraphicsElement { GraphicsElementId = elementId });
}
}
}
@@ -85,6 +85,7 @@ public class CreateChannelFromLineupHandler(
await using var transaction = await dbContext.Database.BeginTransactionAsync(cancellationToken);
try
{
await ChannelGraphicsDefaults.Attach(dbContext, prepared.Channel, cancellationToken);
dbContext.Channels.Add(prepared.Channel);
if (prepared.Playlist is not null)
{
@@ -7,6 +7,7 @@ using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Infrastructure.Extensions;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Channels.ChannelValidations;
@@ -35,7 +36,8 @@ public class CreateChannelHandler(
Right: async logoPath =>
{
ApplyResolvedLogo(request, channel, logoPath);
return Right<BaseError, CreateChannelResult>(await PersistChannel(dbContext, channel));
return Right<BaseError, CreateChannelResult>(
await PersistChannel(dbContext, channel, cancellationToken));
},
Left: e => Task.FromResult(Left<BaseError, CreateChannelResult>(e)));
},
@@ -75,8 +77,12 @@ public class CreateChannelHandler(
}
}
private async Task<CreateChannelResult> PersistChannel(TvContext dbContext, Channel channel)
private async Task<CreateChannelResult> PersistChannel(
TvContext dbContext,
Channel channel,
CancellationToken cancellationToken)
{
await ChannelGraphicsDefaults.Attach(dbContext, channel, cancellationToken);
await dbContext.Channels.AddAsync(channel);
await dbContext.SaveChangesAsync();
searchTargets.SearchTargetsChanged();
@@ -595,6 +595,13 @@ public class RefreshChannelDataHandler : IRequestHandler<RefreshChannelData>
metadata.Genres ??= [];
metadata.Studios ??= [];
// Artists/AlbumArtists are NULLABLE primitive collections, so they are guarded at the read site
// rather than assigned back onto `metadata` like the navigations above (ersatztv#701/#691): they
// are scalar JSON-array columns, so `??= []` on a tracked entity would persist `[]` over NULL.
// The shipped `_song.sbntxt` only does `array.join`, but a user template is free to do anything.
List<string> songArtists = Optional(metadata.Artists).Flatten().ToList();
List<string> songAlbumArtists = Optional(metadata.AlbumArtists).Flatten().ToList();
string artworkPath = GetPrioritizedArtworkPath(metadata);
var data = new
@@ -607,8 +614,8 @@ public class RefreshChannelDataHandler : IRequestHandler<RefreshChannelData>
HasCustomTitle = hasCustomTitle,
displayItem.CustomTitle,
SongTitle = subtitle,
SongArtists = metadata.Artists,
SongAlbumArtists = metadata.AlbumArtists,
SongArtists = songArtists,
SongAlbumArtists = songAlbumArtists,
SongHasYear = metadata.Year.HasValue,
SongYear = metadata.Year,
SongGenres = metadata.Genres.Map(g => g.Name).OrderBy(n => n),
@@ -35,4 +35,6 @@ public record CreateFFmpegProfile(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder) : IRequest<Either<BaseError, CreateFFmpegProfileResult>>;
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup) : IRequest<Either<BaseError, CreateFFmpegProfileResult>>;
@@ -50,8 +50,12 @@ public class CreateFFmpegProfileHandler :
private static Validation<BaseError, FFmpegProfile> Validate(
CreateFFmpegProfile request,
int resolutionId) =>
(ValidateName(request), ValidateThreadCount(request))
.Apply((name, threadCount) =>
(ValidateName(request),
ValidateThreadCount(request),
FFmpegProfileBounds.ValidateQsvExtraHardwareFrames(request.QsvExtraHardwareFrames, stored: null),
FFmpegProfileBounds.ValidateReadRate(request.ReadRate),
FFmpegProfileBounds.ValidateReadRateCatchup(request.ReadRateCatchup, request.ReadRate))
.Apply((name, threadCount, _, _, _) =>
{
var hwAccel = request.NormalizeVideo
? request.HardwareAcceleration
@@ -68,11 +72,9 @@ public class CreateFFmpegProfileHandler :
HardwareAcceleration = hwAccel,
VaapiDriver = request.VaapiDriver,
VaapiDevice = request.VaapiDevice,
// store what the pipeline will actually use, never a pool size FFmpegState would
// floor away at render time (ersatztv#529)
QsvExtraHardwareFrames = request.QsvExtraHardwareFrames is { } frames
? Math.Max(frames, FFmpegState.MinimumQsvExtraHardwareFrames)
: null,
// stored exactly as submitted: an out-of-range value was already rejected with a
// 422 naming the bound, so there is nothing left to silently rewrite (ersatztv#735)
QsvExtraHardwareFrames = request.QsvExtraHardwareFrames,
ResolutionId = resolutionId,
ScalingBehavior = request.ScalingBehavior,
@@ -111,7 +113,9 @@ public class CreateFFmpegProfileHandler :
NormalizeFramerate = request.NormalizeFramerate,
NormalizeColors = request.NormalizeColors,
DeinterlaceVideo = request.DeinterlaceVideo,
QsvPreferNativeDecoder = request.QsvPreferNativeDecoder
QsvPreferNativeDecoder = request.QsvPreferNativeDecoder,
ReadRate = request.ReadRate,
ReadRateCatchup = request.ReadRateCatchup
};
});
@@ -36,4 +36,6 @@ public record UpdateFFmpegProfile(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder) : IRequest<Either<BaseError, UpdateFFmpegProfileResult>>;
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup) : IRequest<Either<BaseError, UpdateFFmpegProfileResult>>;
@@ -55,11 +55,10 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
p.VaapiDisplay = update.VaapiDisplay;
p.VaapiDriver = update.VaapiDriver;
p.VaapiDevice = update.VaapiDevice;
// store what the pipeline will actually use, so a profile doesn't keep displaying a pool
// size that FFmpegState floors away at render time (ersatztv#529)
p.QsvExtraHardwareFrames = update.QsvExtraHardwareFrames is { } frames
? Math.Max(frames, FFmpegState.MinimumQsvExtraHardwareFrames)
: null;
// stored exactly as submitted: an out-of-range NEW value was already rejected with a 422
// naming the bound. an unchanged value that predates that validation is written back as-is
// rather than rewritten, and FFmpegState floors it at render time (ersatztv#735)
p.QsvExtraHardwareFrames = update.QsvExtraHardwareFrames;
p.ResolutionId = update.ResolutionId;
p.ScalingBehavior = update.ScalingBehavior;
p.PadMode = update.PadMode;
@@ -108,6 +107,8 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
p.NormalizeColors = update.NormalizeColors;
p.DeinterlaceVideo = update.DeinterlaceVideo;
p.QsvPreferNativeDecoder = update.QsvPreferNativeDecoder;
p.ReadRate = update.ReadRate;
p.ReadRateCatchup = update.ReadRateCatchup;
// don't save invalid preset
ICollection<string> presets = FFmpegLibraryHelper.PresetsForFFmpegProfile(
@@ -131,8 +132,14 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
TvContext dbContext,
UpdateFFmpegProfile request,
FFmpegProfile profile) =>
(await ValidateName(dbContext, request), ValidateThreadCount(request))
.Apply((_, _) => profile);
(await ValidateName(dbContext, request),
ValidateThreadCount(request),
FFmpegProfileBounds.ValidateQsvExtraHardwareFrames(
request.QsvExtraHardwareFrames,
profile.QsvExtraHardwareFrames),
FFmpegProfileBounds.ValidateReadRate(request.ReadRate),
FFmpegProfileBounds.ValidateReadRateCatchup(request.ReadRateCatchup, request.ReadRate))
.Apply((_, _, _, _, _) => profile);
private static Task<Option<FFmpegProfile>> FFmpegProfileMustExist(
TvContext dbContext,
@@ -0,0 +1,79 @@
using ErsatzTV.Core;
using ErsatzTV.FFmpeg;
namespace ErsatzTV.Application.FFmpegProfiles;
/// <summary>
/// Write-path bounds for the consequential numeric FFmpeg profile fields.
/// A submitted value outside its documented range is REJECTED, naming the bound, rather than
/// accepted and silently rewritten to something the caller never sent (ersatztv#735). The
/// render-time clamps in <see cref="FFmpegState" /> stay as they are: they cover rows that
/// predate this validation or were written out of band, which is what keeps the fix
/// migration-free.
/// </summary>
internal static class FFmpegProfileBounds
{
internal static Validation<BaseError, Unit> ValidateQsvExtraHardwareFrames(int? requested, int? stored)
{
// a row stored before this validation existed may hold anything, and the SPA sends the whole
// profile back on every edit — so rejecting an UNCHANGED legacy value would make an old
// profile uneditable over a field the operator never touched (and cannot even see unless
// hardware acceleration is QSV). only a NEWLY submitted out-of-range value is rejected;
// FFmpegState.QsvExtraHardwareFrames still floors the legacy one at render time
if (requested is null || requested == stored)
{
return Success<BaseError, Unit>(Unit.Default);
}
return requested < FFmpegState.MinimumQsvExtraHardwareFrames
? BaseError.New(
$"QSV extra hardware frames must be at least {FFmpegState.MinimumQsvExtraHardwareFrames}; " +
$"{requested} leaves the QSV upload pool with too little headroom and the transcode writes nothing at all")
: Success<BaseError, Unit>(Unit.Default);
}
internal static Validation<BaseError, Unit> ValidateReadRate(double? requested)
{
if (requested is null)
{
return Success<BaseError, Unit>(Unit.Default);
}
return requested is < FFmpegState.MinimumReadRate or > FFmpegState.MaximumReadRate
? BaseError.New(
$"Read rate must be between {Format(FFmpegState.MinimumReadRate)} and {Format(FFmpegState.MaximumReadRate)}; " +
"below realtime the channel stalls, and above this the input is no longer meaningfully paced")
: Success<BaseError, Unit>(Unit.Default);
}
internal static Validation<BaseError, Unit> ValidateReadRateCatchup(double? requested, double? requestedReadRate)
{
if (requested is null)
{
return Success<BaseError, Unit>(Unit.Default);
}
if (requested is < FFmpegState.MinimumReadRateCatchup or > FFmpegState.MaximumReadRateCatchup)
{
return BaseError.New(
$"Read rate catchup must be between {Format(FFmpegState.MinimumReadRateCatchup)} and " +
$"{Format(FFmpegState.MaximumReadRateCatchup)}");
}
// catchup is the rate a LAGGING input may read at until it is level again, so a value at or
// below the base rate cannot let it recover: EQUAL is rejected too, because a catchup with
// zero headroom is functionally no catchup while still reading as configured. compared
// against the transcode default rather than the stream-copy one because that is the higher
// of the two: a value that clears it clears both, without this check having to know the
// profile's video format
double effectiveReadRate = requestedReadRate ?? FFmpegState.DefaultReadRate;
return requested <= effectiveReadRate
? BaseError.New(
$"Read rate catchup ({Format(requested.Value)}) must be greater than the read rate " +
$"({Format(effectiveReadRate)}); a lagging input cannot catch up at a rate it is already paced at")
: Success<BaseError, Unit>(Unit.Default);
}
private static string Format(double value) =>
value.ToString("0.0####", System.Globalization.CultureInfo.InvariantCulture);
}
@@ -36,4 +36,6 @@ public record FFmpegProfileViewModel(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder);
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup);
@@ -38,7 +38,9 @@ internal static class Mapper
profile.NormalizeFramerate,
profile.NormalizeColors,
profile.DeinterlaceVideo == true,
profile.QsvPreferNativeDecoder != false);
profile.QsvPreferNativeDecoder != false,
profile.ReadRate,
profile.ReadRateCatchup);
internal static FFmpegProfileResponseModel ProjectToResponseModel(FFmpegProfile ffmpegProfile) =>
new(
@@ -82,5 +84,7 @@ internal static class Mapper
ffmpegProfile.NormalizeFramerate,
ffmpegProfile.NormalizeColors,
ffmpegProfile.DeinterlaceVideo == true,
ffmpegProfile.QsvPreferNativeDecoder != false);
ffmpegProfile.QsvPreferNativeDecoder != false,
ffmpegProfile.ReadRate,
ffmpegProfile.ReadRateCatchup);
}
@@ -1,4 +1,5 @@
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Filler.Mapper;
@@ -12,9 +13,13 @@ public class GetPagedFillerPresetsHandler(IDbContextFactory<TvContext> dbContext
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.FillerPresets.CountAsync(cancellationToken);
List<FillerPresetViewModel> page = await dbContext.FillerPresets
.AsNoTracking()
// no filter today, but count and page are still derived from ONE query so that adding one
// cannot leave the count behind (api.paged-count-matches-page-query)
IQueryable<FillerPreset> query = dbContext.FillerPresets.AsNoTracking();
int count = await query.CountAsync(cancellationToken);
List<FillerPresetViewModel> page = await query
.OrderBy(f => f.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.MediaCollections.Mapper;
@@ -13,8 +13,6 @@ public class GetPagedCollectionsHandler(IDbContextFactory<TvContext> dbContextFa
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.Collections.CountAsync(cancellationToken);
IQueryable<Collection> query = dbContext.Collections.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -22,6 +20,9 @@ public class GetPagedCollectionsHandler(IDbContextFactory<TvContext> dbContextFa
query = query.Where(c => EF.Functions.Like(c.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<MediaCollectionViewModel> page = await query
.OrderBy(c => c.Name)
.Skip(request.PageNum * request.PageSize)
@@ -13,9 +13,6 @@ public class GetPagedMultiCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.MultiCollections
.CountAsync(mc => mc.OwnedByChannelId == null, cancellationToken);
IQueryable<MultiCollection> query = dbContext.MultiCollections
.AsNoTracking()
.Where(mc => mc.OwnedByChannelId == null);
@@ -25,6 +22,9 @@ public class GetPagedMultiCollectionsHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(mc => EF.Functions.Like(mc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<MultiCollectionViewModel> page = await query
.OrderBy(mc => mc.Name)
.Skip(request.PageNum * request.PageSize)
@@ -13,18 +13,21 @@ public class GetPagedRerunCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.RerunCollections.CountAsync(cancellationToken);
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking().IncludeSelectionDetails();
IQueryable<RerunCollection> query = dbContext.RerunCollections.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
{
query = query.Where(rc => EF.Functions.Like(rc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758).
// The includes belong to the page chain only — a COUNT does not materialize the graph.
int count = await query.CountAsync(cancellationToken);
// EF applies the includes to the paged subquery, so the selection graph is loaded for at most
// PageSize rows — the per-request cost is bounded by the page, not by the table (issue #671).
List<RerunCollectionViewModel> page = await query
.IncludeSelectionDetails()
.OrderBy(rc => rc.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -13,9 +13,6 @@ public class GetPagedSmartCollectionsHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.SmartCollections
.CountAsync(sc => sc.OwnedByChannelId == null, cancellationToken);
IQueryable<SmartCollection> query = dbContext.SmartCollections
.AsNoTracking()
.Where(sc => sc.OwnedByChannelId == null);
@@ -25,6 +22,9 @@ public class GetPagedSmartCollectionsHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(sc => EF.Functions.Like(sc.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<SmartCollectionViewModel> page = await query
.OrderBy(s => s.Name)
.Skip(request.PageNum * request.PageSize)
@@ -1,4 +1,5 @@
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.MediaCollections.Mapper;
@@ -12,9 +13,13 @@ public class GetPagedTraktListsHandler(IDbContextFactory<TvContext> dbContextFac
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.TraktLists.CountAsync(cancellationToken);
List<TraktListViewModel> page = await dbContext.TraktLists
.AsNoTracking()
// no filter today, but count and page are still derived from ONE query so that adding one
// cannot leave the count behind (api.paged-count-matches-page-query)
IQueryable<TraktList> query = dbContext.TraktLists.AsNoTracking();
int count = await query.CountAsync(cancellationToken);
List<TraktListViewModel> page = await query
.OrderBy(l => l.Name)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
+10 -3
View File
@@ -1,5 +1,6 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
namespace ErsatzTV.Application.Playouts;
@@ -40,9 +41,15 @@ internal static class Mapper
programScheduleAlternate.Id,
programScheduleAlternate.Index,
programScheduleAlternate.ProgramScheduleId,
programScheduleAlternate.DaysOfWeek,
programScheduleAlternate.DaysOfMonth,
programScheduleAlternate.MonthsOfYear,
// ersatztv#823: these three are NULLABLE columns and a legacy row can hold NULL. Substitute the
// SAME unrestricted defaults AlternateScheduleSelector.GetScheduleForDate reads, so the DTO the
// SPA renders agrees with what actually gets scheduled -- web/src/screens/playoutTemplateCalendar.ts
// `appliesToDate` is an exact port of that method, and it would otherwise both mispreview and
// throw (`[...template.daysOfMonth]` on a null is a TypeError). Never assigned back onto the
// entity (`media.nullable-primitive-collection-mutation`).
programScheduleAlternate.DaysOfWeek ?? AlternateScheduleSelector.AllDaysOfWeek(),
programScheduleAlternate.DaysOfMonth ?? AlternateScheduleSelector.AllDaysOfMonth(),
programScheduleAlternate.MonthsOfYear ?? AlternateScheduleSelector.AllMonthsOfYear(),
programScheduleAlternate.LimitToDateRange,
programScheduleAlternate.StartMonth,
programScheduleAlternate.StartDay,
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.Playouts.Mapper;
@@ -13,13 +13,8 @@ public class GetPagedPlayoutsHandler(IDbContextFactory<TvContext> dbContextFacto
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.Playouts.CountAsync(cancellationToken);
IQueryable<Playout> query = dbContext.Playouts
.AsNoTracking()
.Include(p => p.Channel)
.Include(p => p.ProgramSchedule)
.Include(p => p.BuildStatus)
.Filter(p => p.Channel != null);
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -27,7 +22,15 @@ public class GetPagedPlayoutsHandler(IDbContextFactory<TvContext> dbContextFacto
query = query.Where(p => EF.Functions.Like(p.Channel.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758).
// This is also what makes the `Channel != null` filter count, which the old unfiltered
// CountAsync over the whole DbSet did not.
int count = await query.CountAsync(cancellationToken);
List<PlayoutNameViewModel> page = await query
.Include(p => p.Channel)
.Include(p => p.ProgramSchedule)
.Include(p => p.BuildStatus)
.OrderBy(p => p.Channel.SortNumber)
.Skip(request.PageNum * request.PageSize)
.Take(request.PageSize)
@@ -1,4 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using static ErsatzTV.Application.ProgramSchedules.Mapper;
@@ -13,8 +13,6 @@ public class GetPagedProgramSchedulesHandler(IDbContextFactory<TvContext> dbCont
CancellationToken cancellationToken)
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
int count = await dbContext.ProgramSchedules.CountAsync(cancellationToken);
IQueryable<ProgramSchedule> query = dbContext.ProgramSchedules.AsNoTracking();
if (!string.IsNullOrWhiteSpace(request.Query))
@@ -22,6 +20,9 @@ public class GetPagedProgramSchedulesHandler(IDbContextFactory<TvContext> dbCont
query = query.Where(ps => EF.Functions.Like(ps.Name, $"%{request.Query}%"));
}
// count the SAME query the page is taken from, so the two cannot drift (issues #690, #758)
int count = await query.CountAsync(cancellationToken);
List<ProgramScheduleViewModel> page = await query
.OrderBy(ps => ps.Name)
.Skip(request.PageNum * request.PageSize)
+11 -4
View File
@@ -1,6 +1,7 @@
using ErsatzTV.Application.Tree;
using ErsatzTV.Application.Tree;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
namespace ErsatzTV.Application.Scheduling;
@@ -190,9 +191,15 @@ internal static class Mapper
ProjectToViewModel(playoutTemplate.Template),
ProjectToViewModel(playoutTemplate.DecoTemplate),
playoutTemplate.Index,
playoutTemplate.DaysOfWeek,
playoutTemplate.DaysOfMonth,
playoutTemplate.MonthsOfYear,
// ersatztv#823: these three are NULLABLE columns and a legacy row can hold NULL. Substitute the
// SAME unrestricted defaults AlternateScheduleSelector.GetScheduleForDate reads, so the DTO the
// SPA renders agrees with what actually gets scheduled -- web/src/screens/playoutTemplateCalendar.ts
// `appliesToDate` is an exact port of that method, and it would otherwise both mispreview and
// throw (`[...template.daysOfMonth]` on a null is a TypeError). Never assigned back onto the
// entity (`media.nullable-primitive-collection-mutation`).
playoutTemplate.DaysOfWeek ?? AlternateScheduleSelector.AllDaysOfWeek(),
playoutTemplate.DaysOfMonth ?? AlternateScheduleSelector.AllDaysOfMonth(),
playoutTemplate.MonthsOfYear ?? AlternateScheduleSelector.AllMonthsOfYear(),
playoutTemplate.LimitToDateRange,
playoutTemplate.StartMonth,
playoutTemplate.StartDay,
@@ -1,3 +1,4 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
using NUnit.Framework;
@@ -864,4 +865,241 @@ public static class AlternateScheduleSelectorTests
result.IsNone.ShouldBeFalse();
}
}
/// <summary>
/// ersatztv#823. <c>DaysOfWeek</c>, <c>DaysOfMonth</c> and <c>MonthsOfYear</c> on
/// <see cref="PlayoutTemplate" /> and <see cref="ProgramScheduleAlternate" /> are six
/// single-column primitive collections whose columns are <c>nullable: true</c> on both providers.
/// A NULL column materializes as CLR <c>null</c> — EF does not invoke the value converter for a
/// NULL at all — so unguarded, each <c>.Contains</c> in
/// <see cref="AlternateScheduleSelector.GetScheduleForDate{T}" /> throws
/// <see cref="NullReferenceException" />. These tests are RED without the read-site guard.
/// <para>
/// A null reads as UNRESTRICTED (the <c>All*()</c> sets), not as empty. The deciding case is
/// SQLite's <c>20240113140741_Add_PlayoutTemplate_DaysOfMonth</c>, which adds the column
/// <c>nullable: true</c> with NO default: a row inserted before it had no day-of-month
/// restriction, so reading its NULL as empty would INVERT its meaning and silently stop the
/// template applying. That is the one NULL reachable without any code writing one.
/// </para>
/// <para>
/// Reachability itself is pinned by
/// <c>ErsatzTV.Tests.Integration.SchedulingCollectionColumnNullTests</c> against a real
/// <c>TvContext</c>; these tests pin what the selector does once the null is there.
/// </para>
/// </summary>
[TestFixture]
public class GetScheduleForDate_NullCollections
{
private static readonly TimeSpan Offset = TimeSpan.FromHours(-5);
// A Wednesday in March, so no All*() member is coincidentally excluded — and deliberately the
// 20th rather than the 6th. With a day <= 12 a CROSS-WIRED substitution survives the whole
// fixture: `DaysOfMonth ?? AllMonthsOfYear()` hands back 1..12, which still contains day 6, so
// every assertion here passes while the guard substitutes the wrong set. Day 20 is outside 1..12
// and kills it.
private static readonly DateTimeOffset AnyDate = new(2024, 3, 20, 0, 0, 0, Offset);
private static PlayoutTemplate Unrestricted() =>
new()
{
DaysOfWeek = AlternateScheduleSelector.AllDaysOfWeek(),
DaysOfMonth = AlternateScheduleSelector.AllDaysOfMonth(),
MonthsOfYear = AlternateScheduleSelector.AllMonthsOfYear()
};
private static Option<PlayoutTemplate> Select(params PlayoutTemplate[] templates) =>
AlternateScheduleSelector.GetScheduleForDate(templates.ToList(), AnyDate);
[Test]
public void Null_DaysOfWeek_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
Select(template).IsSome.ShouldBeTrue(
"a NULL DaysOfWeek means no weekday restriction was recorded, so the template still applies");
}
[Test]
public void Null_DaysOfMonth_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfMonth = null!;
Select(template).IsSome.ShouldBeTrue();
}
[Test]
public void Null_MonthsOfYear_Reads_As_Unrestricted()
{
PlayoutTemplate template = Unrestricted();
template.MonthsOfYear = null!;
Select(template).IsSome.ShouldBeTrue();
}
[Test]
public void All_Three_Null_On_ProgramScheduleAlternate_Reads_As_Unrestricted()
{
var alternate = new ProgramScheduleAlternate
{
DaysOfWeek = null!,
DaysOfMonth = null!,
MonthsOfYear = null!
};
AlternateScheduleSelector.GetScheduleForDate(
new List<ProgramScheduleAlternate> { alternate },
AnyDate)
.IsSome.ShouldBeTrue();
}
/// <summary>
/// THE DISCRIMINATING CONTROL. Every test above sets a NULL and expects the item to be selected,
/// so all of them pass equally under "NULL means unrestricted" and under the much broader
/// "any NULL makes this item match unconditionally" — a refactor that short-circuits the whole
/// date check when any dimension is null keeps them green. Here the nulled dimension is paired
/// with a RESTRICTIVE non-null one that the date fails, so only the narrow reading passes.
/// </summary>
[Test]
public void A_Null_Dimension_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
// AnyDate is in MARCH; restrict to January only.
template.MonthsOfYear = [1];
Select(template).IsNone.ShouldBeTrue(
"a NULL DaysOfWeek relaxes ONLY the weekday dimension — the January restriction still "
+ "excludes a March date");
}
/// <summary>
/// One arrangement is not enough: with only the <c>DaysOfWeek</c> case above, a PER-DIMENSION
/// mutant survives the whole fixture — e.g. <c>if (item.MonthsOfYear is null) return item;</c>
/// placed ahead of the checks is never reached by that test, because its <c>MonthsOfYear</c> is
/// non-null. So each of the three dimensions is nulled in turn against a restriction on a
/// DIFFERENT dimension.
/// </summary>
[Test]
public void A_Null_MonthsOfYear_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.MonthsOfYear = null!;
// AnyDate is a WEDNESDAY; restrict to Monday only.
template.DaysOfWeek = [DayOfWeek.Monday];
Select(template).IsNone.ShouldBeTrue(
"a NULL MonthsOfYear relaxes ONLY the month dimension — the Monday restriction still "
+ "excludes a Wednesday");
}
/// <summary>
/// The third of the per-dimension controls — see
/// <see cref="A_Null_MonthsOfYear_Does_Not_Relax_The_Other_Dimensions" /> for why one
/// arrangement is not enough. Here the nulled dimension is <c>DaysOfMonth</c> and the
/// restriction that must still bite is on <c>MonthsOfYear</c>.
/// </summary>
[Test]
public void A_Null_DaysOfMonth_Does_Not_Relax_The_Other_Dimensions()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfMonth = null!;
// AnyDate is in MARCH; restrict to January only.
template.MonthsOfYear = [1];
Select(template).IsNone.ShouldBeTrue(
"a NULL DaysOfMonth relaxes ONLY the day-of-month dimension — the January restriction "
+ "still excludes a March date");
}
/// <summary>
/// A null must not be confused with an explicitly EMPTY collection. Empty is a legal, reachable
/// state meaning "matches no day", and it keeps that meaning — which is exactly why a NULL
/// cannot be normalized to it.
/// </summary>
[Test]
public void An_Explicitly_Empty_Collection_Still_Matches_Nothing()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = [];
Select(template).IsNone.ShouldBeTrue(
"an empty DaysOfWeek is a recorded restriction of NO days, unlike a NULL");
}
/// <summary>
/// The guard resolves PER ITEM: a null on the first item must not decide the second. Making the
/// nulled item genuinely non-matching is what measures that — with an unrestricted nulled item
/// at index 0 it simply wins on ordering and the second item is never evaluated, so the
/// invariant would go unmeasured while the test passed.
/// </summary>
[Test]
public void A_Null_On_One_Item_Does_Not_Decide_A_Later_Item()
{
PlayoutTemplate nulled = Unrestricted();
nulled.DaysOfWeek = null!;
nulled.MonthsOfYear = [1]; // AnyDate is in March, so this item must NOT match
nulled.Index = 0;
PlayoutTemplate second = Unrestricted();
second.Index = 1;
foreach (PlayoutTemplate selected in Select(nulled, second))
{
selected.ShouldBeSameAs(second);
return;
}
Assert.Fail("the loop stopped at the null-collection item instead of continuing to the next");
}
/// <summary>
/// ...and when the nulled item IS unrestricted it legitimately wins on ordering. Paired with the
/// test above so "index 0 wins" and "the loop continues past a non-matching null item" are
/// separately pinned.
/// </summary>
[Test]
public void An_Unrestricted_Null_Item_Wins_On_Index_Order()
{
PlayoutTemplate nulled = Unrestricted();
nulled.DaysOfWeek = null!;
nulled.Index = 0;
PlayoutTemplate second = Unrestricted();
second.Index = 1;
foreach (PlayoutTemplate selected in Select(nulled, second))
{
selected.ShouldBeSameAs(nulled);
return;
}
Assert.Fail("the null-collection item was skipped instead of read as unrestricted");
}
/// <summary>
/// The read-site guard must not be written BACK onto the item. These are single-column
/// primitive collections, so assigning the guard would flip a tracked entity to
/// <c>Modified</c> and the next <c>SaveChanges</c> would persist the substituted collection
/// over the NULL — the mechanism recorded as <c>media.nullable-primitive-collection-mutation</c>.
/// </summary>
[Test]
public void Guard_Must_Not_Be_Written_Back_Onto_The_Item()
{
PlayoutTemplate template = Unrestricted();
template.DaysOfWeek = null!;
template.DaysOfMonth = null!;
template.MonthsOfYear = null!;
Select(template);
template.DaysOfWeek.ShouldBeNull();
template.DaysOfMonth.ShouldBeNull();
template.MonthsOfYear.ShouldBeNull();
}
}
}
@@ -37,4 +37,6 @@ public record FFmpegFullProfileResponseModel(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool QsvPreferNativeDecoder);
bool QsvPreferNativeDecoder,
double? ReadRate,
double? ReadRateCatchup);
+3
View File
@@ -26,6 +26,9 @@ public class ConfigElementKey
public static ConfigElementKey ChannelTemplatesDefaultTemplateId => new("channel_templates.default_template_id");
public static ConfigElementKey WatermarkChannelBugSeeded => new("watermark.channel_bug_seeded");
public static ConfigElementKey GraphicsOnNowNextSeeded => new("graphics.on_now_next_seeded");
public static ConfigElementKey GraphicsOnNowNextDefaultAttached =>
new("graphics.on_now_next_default_attached");
public static ConfigElementKey FFmpegSegmenterTimeout => new("ffmpeg.segmenter.timeout_seconds");
public static ConfigElementKey FFmpegWorkAheadSegmenters => new("ffmpeg.segmenter.work_ahead_limit");
public static ConfigElementKey FFmpegInitialSegmentCount => new("ffmpeg.segmenter.initial_segment_count");
+2
View File
@@ -14,6 +14,8 @@ public record FFmpegProfile
public VaapiDriver VaapiDriver { get; set; }
public string VaapiDevice { get; set; }
public int? QsvExtraHardwareFrames { get; set; }
public double? ReadRate { get; set; }
public double? ReadRateCatchup { get; set; }
public bool? QsvPreferNativeDecoder { get; set; }
public int ResolutionId { get; set; }
public Resolution Resolution { get; set; }
@@ -610,7 +610,9 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
channel.FFmpegProfile.QsvPreferNativeDecoder != false);
channel.FFmpegProfile.QsvPreferNativeDecoder != false,
Optional(channel.FFmpegProfile.ReadRate),
Optional(channel.FFmpegProfile.ReadRateCatchup));
_logger.LogDebug("FFmpeg desired state {FrameState}", desiredState);
@@ -827,7 +829,9 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel);
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
MaybeReadRate: Optional(channel.FFmpegProfile.ReadRate),
MaybeReadRateCatchup: Optional(channel.FFmpegProfile.ReadRateCatchup));
var ffmpegSubtitleStream = new ErsatzTV.FFmpeg.MediaStream(0, "ass", StreamKind.Video);
@@ -968,7 +972,9 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
false,
false,
GetTonemapAlgorithm(playbackSettings),
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel);
channel.Number == FileSystemLayout.TranscodeTroubleshootingChannel,
MaybeReadRate: Optional(channel.FFmpegProfile.ReadRate),
MaybeReadRateCatchup: Optional(channel.FFmpegProfile.ReadRateCatchup));
var audioInputFile = new NullAudioInputFile(audioState);
@@ -4,4 +4,7 @@ public static class GraphicsElementDefaults
{
// Built-in "On Now / Next" text element; identity is by filename, never by user-editable Name.
public const string OnNowNextFileName = "on-now-next.yml";
// Display name only. Never use it for identity -- that is the filename above (#67 / #74).
public const string OnNowNextName = "On Now / Next";
}
@@ -31,6 +31,28 @@ public class TextGraphicsElement : BaseGraphicsElement
[YamlMember(Alias = "z_index", ApplyNamingConventions = false)]
public int? ZIndex { get; set; }
// Background box (ersatztv#732). Element-level, not per-style: the graphics engine renders one
// TextBlock into one bitmap, so a single box behind the whole element is the only shape the
// renderer can express. Unset background_color means no FILL; a border_color alone still draws
// an outlined box. With neither there is no box and no insets -- the pre-#732 geometry.
[YamlMember(Alias = "background_color", ApplyNamingConventions = false)]
public string BackgroundColor { get; set; }
[YamlMember(Alias = "background_opacity_percent", ApplyNamingConventions = false)]
public int? BackgroundOpacityPercent { get; set; }
[YamlMember(Alias = "background_padding", ApplyNamingConventions = false)]
public double? BackgroundPadding { get; set; }
[YamlMember(Alias = "background_corner_radius", ApplyNamingConventions = false)]
public double? BackgroundCornerRadius { get; set; }
[YamlMember(Alias = "border_color", ApplyNamingConventions = false)]
public string BorderColor { get; set; }
[YamlMember(Alias = "border_width", ApplyNamingConventions = false)]
public double? BorderWidth { get; set; }
public List<StyleDefinition> Styles { get; set; } = [];
[YamlMember(Alias = "base_style", ApplyNamingConventions = false)]
@@ -85,19 +85,46 @@ public static class AlternateScheduleSelector
}
}
bool daysOfWeek = item.DaysOfWeek.Contains(date.DayOfWeek);
// These three are NULLABLE single-column primitive collections, and a runtime null IS
// reachable (ersatztv#823, measured against a real TvContext on SQLite and MySQL 8.4): EF does
// NOT invoke the value converter for a NULL column, so it materializes as CLR null rather than
// through IntCollectionValueConverter's null-to-empty branch, which never runs on this path.
// Unguarded, each .Contains below throws NullReferenceException.
//
// A NULL reads as UNRESTRICTED -- the All*() sets -- NOT as empty. This is the whole semantic
// question and it is decided by the one NULL that is reachable WITHOUT any code writing one:
// Sqlite's 20240113140741_Add_PlayoutTemplate_DaysOfMonth adds DaysOfMonth with
// `nullable: true` and NO defaultValue, so a PlayoutTemplate row inserted before it holds NULL
// and, by construction, had NO day-of-month restriction. Reading that as empty would INVERT
// the row's meaning and silently stop the template applying at all. All*() preserves it, and
// it is how "no restriction recorded" is already represented elsewhere in this domain
// (GetPlayoutAlternateSchedulesHandler, PreviewBlockPlayoutHandler). Note what does NOT decide
// it: the API request records normalize an omitted field with `?? []`, but that is a client
// omitting a field on a WRITE and says nothing about what a legacy database NULL meant.
//
// Guarded at the READ SITE, into locals, and NEVER assigned back onto `item`: the property IS
// the column value, so writing the guard back would flip a tracked entry to Modified and
// persist the substituted collection over the NULL
// (`media.nullable-primitive-collection-mutation`). The matching substitution happens at the
// entity->DTO boundary in the two Mapper.ProjectToViewModel overloads, so the SPA's
// appliesToDate -- an exact port of this method -- previews what this actually schedules.
ICollection<DayOfWeek> itemDaysOfWeek = item.DaysOfWeek ?? AllDaysOfWeek();
ICollection<int> itemDaysOfMonth = item.DaysOfMonth ?? AllDaysOfMonth();
ICollection<int> itemMonthsOfYear = item.MonthsOfYear ?? AllMonthsOfYear();
bool daysOfWeek = itemDaysOfWeek.Contains(date.DayOfWeek);
if (!daysOfWeek)
{
continue;
}
bool daysOfMonth = item.DaysOfMonth.Contains(date.Day);
bool daysOfMonth = itemDaysOfMonth.Contains(date.Day);
if (!daysOfMonth)
{
continue;
}
bool monthOfYear = item.MonthsOfYear.Contains(date.Month);
bool monthOfYear = itemMonthsOfYear.Contains(date.Month);
if (monthOfYear)
{
return item;
@@ -680,10 +680,56 @@ public class PipelineBuilderBaseTests
command.ShouldContain("-readrate 1.05 -readrate_initial_burst 8 -readrate_catchup 6.0 -i /tmp/whatever.mkv");
}
// ersatztv#735: the pacing values became operator-tunable profile fields. these pin that a
// configured value actually reaches the command line -- the defaults above are the OTHER half
// of the same guard, and they are what an unset profile still gets
[Test]
public void Realtime_Input_Should_Use_A_Configured_ReadRate_And_Catchup()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 1.5,
readRateCatchup: 4.0);
command.ShouldContain("-readrate 1.5 -readrate_initial_burst 8 -readrate_catchup 4.0 -i /tmp/whatever.mkv");
command.ShouldNotContain("-readrate 1.05");
command.ShouldNotContain("-readrate_catchup 6.0");
}
// the write path rejects an out-of-range value with a 422, so this only fires for a row written
// out of band -- but FFmpeg must never see the unbounded value either way
[Test]
public void Realtime_Input_Should_Clamp_An_Out_Of_Range_ReadRate()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 9.0,
readRateCatchup: 0.1);
// 9.0 clamps to the 2.0 ceiling, and 0.1 is raised to the resolved base rate, because a
// catchup below it could never let a lagging input recover
command.ShouldContain("-readrate 2.0 -readrate_initial_burst 8 -readrate_catchup 2.0 -i /tmp/whatever.mkv");
}
// ...and the catchup CEILING isolated from the base rate, which the case above cannot show:
// there both clamps land on the same 2.0, so either one alone would satisfy it
[Test]
public void Realtime_Input_Should_Clamp_An_Out_Of_Range_ReadRateCatchup()
{
string command = BuildRealtimeCommand(
new CatchupCapableFFmpegCapabilities(),
readRate: 1.2,
readRateCatchup: 15.0);
command.ShouldContain("-readrate 1.2 -readrate_initial_burst 8 -readrate_catchup 10.0 -i /tmp/whatever.mkv");
}
private string BuildRealtimeCommand(
IFFmpegCapabilities capabilities,
bool stillImage = false,
bool imageSubtitle = false)
bool imageSubtitle = false,
Option<double> readRate = default,
Option<double> readRateCatchup = default)
{
var videoInputFile = new VideoInputFile(
"/tmp/whatever.mkv",
@@ -748,7 +794,9 @@ public class PipelineBuilderBaseTests
false,
false,
"clip",
false);
false,
MaybeReadRate: readRate,
MaybeReadRateCatchup: readRateCatchup);
// a *separate* audio input matters here: for a still image the video input takes no readrate
// at all, so only a distinct audio input can prove the burst was suppressed (this is the
+46 -1
View File
@@ -28,7 +28,9 @@ public record FFmpegState(
bool IsHdrTonemap,
string TonemapAlgorithm,
bool IsTroubleshooting,
bool QsvPreferNativeDecoder = false)
bool QsvPreferNativeDecoder = false,
Option<double> MaybeReadRate = default,
Option<double> MaybeReadRateCatchup = default)
{
// the QSV upload pool needs headroom for the frames in flight through the filter graph.
// extra_hw_frames=0 leaves none, so any input that is not throttled exhausts it: the graph
@@ -42,6 +44,49 @@ public record FFmpegState(
public int QsvExtraHardwareFrames =>
Math.Max(MaybeQsvExtraHardwareFrames.IfNone(MinimumQsvExtraHardwareFrames), MinimumQsvExtraHardwareFrames);
// realtime pacing. an unset profile keeps the values these constants name, which are the ones
// the pipeline hardcoded before they became configurable (ersatztv#735)
public const double DefaultReadRate = 1.05;
public const double DefaultStreamCopyReadRate = 1.0;
// how fast a LAGGING realtime input may read until it is level again. measured on the #726
// repro (embedded dvd_subtitle -> overlay, QSV encode): 1.05 alone sustains 0.53x, catchup 2.0
// reaches 0.711x, and 6.0 restores the full 1.067x that the same pipeline achieves with no
// subtitle at all. 20.0 also measures 1.067x — i.e. the value is not a throughput dial above
// the point where the input catches up, so 6.0 is chosen as the smallest measured-sufficient
// ceiling rather than the largest that works (ersatztv#726)
public const double DefaultReadRateCatchup = 6.0;
// below realtime the process reads slower than a live client consumes and the channel stalls;
// ersatztv#726 is that failure, measured at an effective 0.53x. the ceiling is a CHOSEN bound,
// not a measured cliff: it exists so the field cannot be used to effectively disable pacing,
// which is the configuration ersatztv#529 measured to produce zero segments on a QSV pipeline
public const double MinimumReadRate = 1.0;
public const double MaximumReadRate = 2.0;
// catchup is a ceiling that applies only WHILE an input is behind, so it is bounded more
// loosely than the base rate; the same chosen-not-measured caveat applies to the ceiling.
// the FLOOR is only a write-path bound: at render time the resolved base rate is always at
// least MinimumReadRate, so Math.Max below already dominates it
public const double MinimumReadRateCatchup = 1.0;
public const double MaximumReadRateCatchup = 10.0;
// clamped for the same reason QsvExtraHardwareFrames is: a row written out of band (or before
// the write path validated the field) must not reach FFmpeg unbounded. the write path rejects
// an out-of-range value with a 422 naming the bound, so this is belt-and-braces, not the
// primary guard (ersatztv#735)
public double ReadRateFor(bool isStreamCopy) =>
MaybeReadRate.Match(
configured => Math.Clamp(configured, MinimumReadRate, MaximumReadRate),
() => isStreamCopy ? DefaultStreamCopyReadRate : DefaultReadRate);
// a catchup rate below the base rate cannot let a lagging input recover, so the resolved base
// rate is its real floor — no separate lower clamp, which would be unreachable behind this Max
public double ReadRateCatchupFor(bool isStreamCopy) =>
Math.Max(
Math.Min(MaybeReadRateCatchup.IfNone(DefaultReadRateCatchup), MaximumReadRateCatchup),
ReadRateFor(isStreamCopy));
public static FFmpegState Concat(bool saveReport, string channelName) =>
new(
saveReport,
@@ -22,14 +22,6 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
// an operator who raises that setting above 2 gets less of the benefit (ersatztv#350)
private const int InitialBurstSeconds = OutputFormatHls.SegmentSeconds * 2;
// how fast a LAGGING realtime input may read until it is level again. measured on the #726
// repro (embedded dvd_subtitle -> overlay, QSV encode): 1.05 alone sustains 0.53x, catchup 2.0
// reaches 0.711x, and 6.0 restores the full 1.067x that the same pipeline achieves with no
// subtitle at all. 20.0 also measures 1.067x — i.e. the value is not a throughput dial above
// the point where the input catches up, so 6.0 is chosen as the smallest measured-sufficient
// ceiling rather than the largest that works (ersatztv#726)
private const double CatchupReadRate = 6.0;
private readonly Option<AudioInputFile> _audioInputFile;
private readonly Option<ConcatInputFile> _concatInputFile;
private readonly IFFmpegCapabilities _ffmpegCapabilities;
@@ -660,7 +652,7 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
}
//SetStillImageInfiniteLoop(videoInputFile, videoStream, ffmpegState);
SetRealtimeInput(videoInputFile, desiredState);
SetRealtimeInput(videoInputFile, ffmpegState, desiredState);
SetInfiniteLoop(videoInputFile, videoStream, ffmpegState, desiredState);
SetFrameRateOutput(desiredState, pipelineSteps);
SetVideoTrackTimescaleOutput(desiredState, pipelineSteps);
@@ -855,14 +847,17 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
}
}
private void SetRealtimeInput(VideoInputFile videoInputFile, FrameState desiredState)
private void SetRealtimeInput(VideoInputFile videoInputFile, FFmpegState ffmpegState, FrameState desiredState)
{
if (videoInputFile.StreamInputKind is StreamInputKind.Live || !desiredState.Realtime)
{
return;
}
double readRate = desiredState.VideoFormat == VideoFormat.Copy ? 1.0 : 1.05;
// both defaults and both bounds live on FFmpegState, beside the profile fields that
// override them, so the pacing contract is readable in one place (ersatztv#735)
bool isStreamCopy = desiredState.VideoFormat == VideoFormat.Copy;
double readRate = ffmpegState.ReadRateFor(isStreamCopy);
// without a burst, the readrate throttle applies from the very first read, so the first
// segment cannot be written faster than ~realtime and every start pays a multi-second wait.
@@ -894,7 +889,7 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
// subtitle always rides the video path, so this shape cannot suffer the starvation anyway
Option<double> catchupReadRate =
!isStillImage && _ffmpegCapabilities.HasOption(FFmpegKnownOption.ReadrateCatchup)
? CatchupReadRate
? ffmpegState.ReadRateCatchupFor(isStreamCopy)
: Option<double>.None;
_audioInputFile.Iter(a => a.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds, catchupReadRate)));
@@ -0,0 +1,38 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.MySql.Migrations
{
/// <inheritdoc />
public partial class Add_FFmpegProfile_ReadRatePacing : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<double>(
name: "ReadRate",
table: "FFmpegProfile",
type: "double",
nullable: true);
migrationBuilder.AddColumn<double>(
name: "ReadRateCatchup",
table: "FFmpegProfile",
type: "double",
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ReadRate",
table: "FFmpegProfile");
migrationBuilder.DropColumn(
name: "ReadRateCatchup",
table: "FFmpegProfile");
}
}
}
@@ -929,6 +929,12 @@ namespace ErsatzTV.Infrastructure.MySql.Migrations
.HasColumnType("tinyint(1)")
.HasDefaultValue(true);
b.Property<double?>("ReadRate")
.HasColumnType("double");
b.Property<double?>("ReadRateCatchup")
.HasColumnType("double");
b.Property<int>("ResolutionId")
.HasColumnType("int");
@@ -0,0 +1,38 @@
using Microsoft.EntityFrameworkCore.Migrations;
#nullable disable
namespace ErsatzTV.Infrastructure.Sqlite.Migrations
{
/// <inheritdoc />
public partial class Add_FFmpegProfile_ReadRatePacing : Migration
{
/// <inheritdoc />
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<double>(
name: "ReadRate",
table: "FFmpegProfile",
type: "REAL",
nullable: true);
migrationBuilder.AddColumn<double>(
name: "ReadRateCatchup",
table: "FFmpegProfile",
type: "REAL",
nullable: true);
}
/// <inheritdoc />
protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropColumn(
name: "ReadRate",
table: "FFmpegProfile");
migrationBuilder.DropColumn(
name: "ReadRateCatchup",
table: "FFmpegProfile");
}
}
}
@@ -896,6 +896,12 @@ namespace ErsatzTV.Infrastructure.Sqlite.Migrations
.HasColumnType("INTEGER")
.HasDefaultValue(true);
b.Property<double?>("ReadRate")
.HasColumnType("REAL");
b.Property<double?>("ReadRateCatchup")
.HasColumnType("REAL");
b.Property<int>("ResolutionId")
.HasColumnType("INTEGER");
@@ -1,4 +1,4 @@
using Dapper;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Repositories;
@@ -171,8 +171,15 @@ public class MusicVideoRepository : IMusicVideoRepository
public async Task<int> GetMusicVideoCount(int artistId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
// count the same population GetPagedMusicVideos pages — MusicVideoMetadata, not MusicVideo.
// A music video whose metadata row is missing (a scanner failure; FindOrphanPaths models
// exactly that state) is not pageable, so counting the item table over-reports
// (api.paged-count-matches-page-query, #832).
return await dbContext.Connection.QuerySingleAsync<int>(
@"SELECT COUNT(*) FROM MusicVideo WHERE ArtistId = @ArtistId",
@"SELECT COUNT(*)
FROM MusicVideoMetadata MVM
INNER JOIN MusicVideo M on MVM.MusicVideoId = M.Id
WHERE M.ArtistId = @ArtistId",
new { ArtistId = artistId });
}
@@ -1,4 +1,4 @@
using Dapper;
using Dapper;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
@@ -134,9 +134,26 @@ public class TelevisionRepository : ITelevisionRepository
public async Task<int> GetSeasonCount(int showId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
return await dbContext.Seasons
.AsNoTracking()
.CountAsync(s => s.ShowId == showId);
// GetPagedSeasons expands the requested show to EVERY show sharing its Title+Year (the same
// show present in two libraries) and pages the union, so the count must expand identically
// or it under-reports (api.paged-count-matches-page-query, #832).
Option<ShowMetadata> maybeShowMetadata = await dbContext.ShowMetadata
.SelectOneAsync(sm => sm.Id, sm => sm.ShowId == showId, CancellationToken.None);
foreach (ShowMetadata showMetadata in maybeShowMetadata)
{
List<int> showIds = await dbContext.ShowMetadata
.Filter(sm => sm.Title == showMetadata.Title && sm.Year == showMetadata.Year)
.Map(sm => sm.ShowId)
.ToListAsync();
return await dbContext.Seasons
.AsNoTracking()
.CountAsync(s => showIds.Contains(s.ShowId));
}
// no metadata for the requested show: GetPagedSeasons returns nothing, so neither does this
return 0;
}
public async Task<List<Season>> GetPagedSeasons(
@@ -179,9 +196,12 @@ public class TelevisionRepository : ITelevisionRepository
public async Task<int> GetEpisodeCount(int seasonId)
{
await using TvContext dbContext = await _dbContextFactory.CreateDbContextAsync();
return await dbContext.Episodes
// count the same population GetPagedEpisodes pages — EpisodeMetadata, not Episode. An
// episode whose metadata row is missing is not pageable, so counting the item table
// over-reports (api.paged-count-matches-page-query, #832).
return await dbContext.EpisodeMetadata
.AsNoTracking()
.CountAsync(e => e.SeasonId == seasonId);
.CountAsync(em => em.Episode.SeasonId == seasonId);
}
public async Task<List<EpisodeMetadata>> GetPagedEpisodes(int seasonId, int pageNumber, int pageSize)
@@ -440,64 +440,64 @@ public class ElasticSearchIndex : ISearchIndex
Season season)
{
foreach (SeasonMetadata metadata in season.SeasonMetadata.HeadOrNone())
foreach (ShowMetadata showMetadata in season.Show.ShowMetadata.HeadOrNone())
{
try
foreach (ShowMetadata showMetadata in season.Show.ShowMetadata.HeadOrNone())
{
var seasonTitle = $"{showMetadata.Title} - S{season.SeasonNumber}";
string sortTitle = $"{showMetadata.SortTitle}_{season.SeasonNumber:0000}"
.ToLowerInvariant();
string titleAndYear = $"{showMetadata.Title}_{showMetadata.Year}_{season.SeasonNumber}"
.ToLowerInvariant();
var doc = new ElasticSearchItem
try
{
Id = season.Id,
Type = LuceneSearchIndex.SeasonType,
Title = seasonTitle,
SortTitle = sortTitle,
LibraryName = season.LibraryPath.Library.Name,
LibraryId = season.LibraryPath.Library.Id,
TitleAndYear = titleAndYear,
TitleAndYearSearch = LuceneSearchIndex.GetTitleAndYearSearch(metadata),
JumpLetter = LuceneSearchIndex.GetJumpLetter(showMetadata),
State = season.State.ToString(),
SeasonNumber = season.SeasonNumber,
ShowTitle = showMetadata.Title,
ShowGenre = showMetadata.Genres.Map(g => g.Name).ToList(),
ShowTag = showMetadata.Tags.Map(t => t.Name).ToList(),
ShowStudio = showMetadata.Studios.Map(s => s.Name).ToList(),
ShowContentRating = GetContentRatings(showMetadata.ContentRating),
Language = GetLanguages(
languageCodeService,
await searchRepository.GetLanguagesForSeason(season)),
LanguageTag = await searchRepository.GetLanguagesForSeason(season),
SubLanguage = GetLanguages(
languageCodeService,
await searchRepository.GetSubLanguagesForSeason(season)),
SubLanguageTag = await searchRepository.GetSubLanguagesForSeason(season),
ContentRating = GetContentRatings(showMetadata.ContentRating),
ReleaseDate = GetReleaseDate(metadata.ReleaseDate),
AddedDate = GetAddedDate(metadata.DateAdded),
TraktList = season.TraktListItems
.Map(t => t.TraktList.TraktId.ToString(CultureInfo.InvariantCulture)).ToList(),
Tag = metadata.Tags.Map(a => a.Name).ToList(),
TagFull = metadata.Tags.Map(t => t.Name).ToList()
};
var seasonTitle = $"{showMetadata.Title} - S{season.SeasonNumber}";
string sortTitle = $"{showMetadata.SortTitle}_{season.SeasonNumber:0000}"
.ToLowerInvariant();
string titleAndYear = $"{showMetadata.Title}_{showMetadata.Year}_{season.SeasonNumber}"
.ToLowerInvariant();
foreach ((string key, List<string> value) in GetMetadataGuids(metadata))
{
doc.AdditionalProperties.Add(key, value);
var doc = new ElasticSearchItem
{
Id = season.Id,
Type = LuceneSearchIndex.SeasonType,
Title = seasonTitle,
SortTitle = sortTitle,
LibraryName = season.LibraryPath.Library.Name,
LibraryId = season.LibraryPath.Library.Id,
TitleAndYear = titleAndYear,
TitleAndYearSearch = LuceneSearchIndex.GetTitleAndYearSearch(metadata),
JumpLetter = LuceneSearchIndex.GetJumpLetter(showMetadata),
State = season.State.ToString(),
SeasonNumber = season.SeasonNumber,
ShowTitle = showMetadata.Title,
ShowGenre = showMetadata.Genres.Map(g => g.Name).ToList(),
ShowTag = showMetadata.Tags.Map(t => t.Name).ToList(),
ShowStudio = showMetadata.Studios.Map(s => s.Name).ToList(),
ShowContentRating = GetContentRatings(showMetadata.ContentRating),
Language = GetLanguages(
languageCodeService,
await searchRepository.GetLanguagesForSeason(season)),
LanguageTag = await searchRepository.GetLanguagesForSeason(season),
SubLanguage = GetLanguages(
languageCodeService,
await searchRepository.GetSubLanguagesForSeason(season)),
SubLanguageTag = await searchRepository.GetSubLanguagesForSeason(season),
ContentRating = GetContentRatings(showMetadata.ContentRating),
ReleaseDate = GetReleaseDate(metadata.ReleaseDate),
AddedDate = GetAddedDate(metadata.DateAdded),
TraktList = season.TraktListItems
.Map(t => t.TraktList.TraktId.ToString(CultureInfo.InvariantCulture)).ToList(),
Tag = metadata.Tags.Map(a => a.Name).ToList(),
TagFull = metadata.Tags.Map(t => t.Name).ToList()
};
foreach ((string key, List<string> value) in GetMetadataGuids(metadata))
{
doc.AdditionalProperties.Add(key, value);
}
await _client.IndexAsync(doc, IndexName, ES.Id.From(doc));
}
catch (Exception ex)
{
metadata.Season = null;
_logger.LogWarning(ex, "Error indexing season with metadata {@Metadata}", metadata);
}
await _client.IndexAsync(doc, IndexName, ES.Id.From(doc));
}
catch (Exception ex)
{
metadata.Season = null;
_logger.LogWarning(ex, "Error indexing season with metadata {@Metadata}", metadata);
}
}
}
private async Task UpdateArtist(
@@ -763,8 +763,10 @@ public class ElasticSearchIndex : ISearchIndex
{
try
{
metadata.AlbumArtists ??= [];
metadata.Artists ??= [];
// Guard the two NULLABLE primitive collections at the READ SITE, never by assigning back onto
// `metadata` (ersatztv#701) -- see the matching comment in LuceneSearchIndex.UpdateSong.
List<string> artists = Optional(metadata.Artists).Flatten().ToList();
List<string> albumArtists = Optional(metadata.AlbumArtists).Flatten().ToList();
var doc = new ElasticSearchItem
{
@@ -785,8 +787,8 @@ public class ElasticSearchIndex : ISearchIndex
SubLanguageTag = GetSubLanguageTags(song.MediaVersions),
AddedDate = GetAddedDate(metadata.DateAdded),
Album = metadata.Album ?? string.Empty,
Artist = metadata.Artists.ToList(),
AlbumArtist = metadata.AlbumArtists.ToList(),
Artist = artists,
AlbumArtist = albumArtists,
Genre = metadata.Genres.Map(g => g.Name).ToList(),
Tag = metadata.Tags.Map(t => t.Name).ToList(),
TagFull = metadata.Tags.Map(t => t.Name).ToList()
@@ -1,4 +1,4 @@
using System.Globalization;
using System.Globalization;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Metadata;
@@ -145,7 +145,7 @@ public sealed class LuceneSearchIndex : ISearchIndex
_directory = FSDirectory.Open(FileSystemLayout.SearchIndexFolder);
Analyzer analyzer = SearchQueryParser.AnalyzerWrapper();
var indexConfig = new IndexWriterConfig(AppLuceneVersion, analyzer)
{ OpenMode = OpenMode.CREATE_OR_APPEND };
{ OpenMode = OpenMode.CREATE_OR_APPEND };
_writer = new IndexWriter(_directory, indexConfig);
_initialized = true;
}
@@ -328,7 +328,7 @@ public sealed class LuceneSearchIndex : ISearchIndex
using (Analyzer analyzer = SearchQueryParser.AnalyzerWrapper())
{
var indexConfig = new IndexWriterConfig(AppLuceneVersion, analyzer)
{ OpenMode = OpenMode.CREATE_OR_APPEND };
{ OpenMode = OpenMode.CREATE_OR_APPEND };
using (var w = new IndexWriter(d, indexConfig))
{
using (DirectoryReader _ = w.GetReader(true))
@@ -1318,8 +1318,13 @@ public sealed class LuceneSearchIndex : ISearchIndex
{
try
{
metadata.AlbumArtists ??= [];
metadata.Artists ??= [];
// Guard the two NULLABLE primitive collections at the READ SITE, never by assigning back onto
// `metadata` (ersatztv#701). The entity reaching here may be TRACKED, and Artists/AlbumArtists
// are scalar JSON-array columns rather than navigations -- so `??= []` flips the entity to
// Modified and the next SaveChanges writes `[]` over a NULL column. Same convention as
// SongVideoGenerator and MediaCollectionRepository (ersatztv#691).
List<string> artists = Optional(metadata.Artists).Flatten().ToList();
List<string> albumArtists = Optional(metadata.AlbumArtists).Flatten().ToList();
var doc = new Document
{
@@ -1355,12 +1360,12 @@ public sealed class LuceneSearchIndex : ISearchIndex
doc.Add(new TextField(AlbumField, metadata.Album, Field.Store.NO));
}
foreach (string artist in metadata.Artists)
foreach (string artist in artists)
{
doc.Add(new TextField(ArtistField, artist, Field.Store.NO));
}
foreach (string albumArtist in metadata.AlbumArtists)
foreach (string albumArtist in albumArtists)
{
doc.Add(new TextField(AlbumArtistField, albumArtist, Field.Store.NO));
}
@@ -4,12 +4,16 @@ using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Graphics;
using ErsatzTV.Infrastructure.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Infrastructure.Streaming.Graphics;
public static class GraphicsElementSeeder
{
private const string OnNowNextYaml =
// The pre-#732 default, kept verbatim so an installation still carrying it byte-for-byte can
// be recognised as unmodified and upgraded. Never edit an entry here -- it is a fingerprint of
// what we shipped, not a template. Add a new entry when the current default changes again.
private const string OnNowNextYamlV1 =
"""
name: On Now / Next
epg_entries: 2
@@ -51,18 +55,88 @@ public static class GraphicsElementSeeder
{{ if (array.size Epg) > 1 }}[next]NEXT {{ Epg[1].Title }}[/next]{{ end }}
""";
public static async Task SeedOnNowNext(TvContext context, IFileSystem fileSystem, CancellationToken cancellationToken)
{
string seededKey = ConfigElementKey.GraphicsOnNowNextSeeded.Key;
bool alreadySeeded = await context.ConfigElements.AnyAsync(c => c.Key == seededKey, cancellationToken);
if (alreadySeeded)
{
return;
}
private const string OnNowNextYaml =
"""
name: On Now / Next
epg_entries: 2
location: BottomLeft
horizontal_margin_percent: 4
vertical_margin_percent: 8
width_percent: 42
text_fit: Wrap
text_align: Left
z_index: 100
# transparent until 4s in, fade in 1s, hold 6s, fade out 1s
opacity_expression: "LinearFadeDuration(content_seconds, 4, 1, 6)"
# #732: a translucent box carries legibility over both bright and dark content. The halo is
# cut from 2 to 1 rather than dropped -- the box is translucent, so bright content still
# shows through behind the glyphs, but 2px of halo ON TOP of a box over-darkens the text.
background_color: "#000000"
background_opacity_percent: 65
background_padding: 14
background_corner_radius: 8
# A translucent black box vanishes into dark content, so the box needs an edge of its own.
# Low-alpha white reads as a hairline on dark frames without becoming a hard line on bright ones.
border_color: "#59FFFFFF"
border_width: 1
base_style: now
styles:
- name: now
font_family: "Noto Sans"
font_size: 30
font_weight: 700
text_color: "#FFFFFF"
halo_color: "#000000"
halo_width: 1
- name: sub
font_family: "Noto Sans"
font_size: 22
font_weight: 400
text_color: "#DDDDDD"
halo_color: "#000000"
halo_width: 1
- name: next
font_family: "Noto Sans"
font_size: 22
font_weight: 400
text_color: "#DDDDDD"
halo_color: "#000000"
halo_width: 1
text: |
[now]NOW {{ Epg[0].Title }}[/now]
{{ if Epg[0].SubTitle }}[sub]{{ Epg[0].SubTitle }}[/sub]{{ end }}
{{ if (array.size Epg) > 1 }}[next]NEXT {{ Epg[1].Title }}[/next]{{ end }}
""";
// Every default we have ever shipped, most recent first. A file matching one of these was
// written by us and never touched, so replacing it is an upgrade rather than a clobber.
private static readonly string[] SupersededDefaults = [OnNowNextYamlV1];
public static async Task SeedOnNowNext(
TvContext context,
IFileSystem fileSystem,
ILogger logger,
CancellationToken cancellationToken)
{
string folder = FileSystemLayout.GraphicsElementsTextTemplatesFolder;
string target = fileSystem.Path.Combine(folder, GraphicsElementDefaults.OnNowNextFileName);
string seededKey = ConfigElementKey.GraphicsOnNowNextSeeded.Key;
bool alreadySeeded = await context.ConfigElements.AnyAsync(c => c.Key == seededKey, cancellationToken);
if (alreadySeeded)
{
// Already-seeded installations never revisit the file, so a change to the default would
// otherwise reach new databases only. Upgrade the ones still carrying an untouched
// earlier default; anything an operator edited no longer matches and is left alone.
//
// Deliberately no CreateDirectory on this branch: before #732 it touched the filesystem
// not at all, so an already-seeded install stays bootable on a read-only /config.
await UpgradeUnmodifiedTemplate(fileSystem, target, logger, cancellationToken);
await EnsureBuiltInElementRow(context, fileSystem, target, cancellationToken);
return;
}
if (!fileSystem.Directory.Exists(folder))
{
fileSystem.Directory.CreateDirectory(folder);
@@ -78,5 +152,218 @@ public static class GraphicsElementSeeder
new ConfigElement { Key = seededKey, Value = "true" },
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
await EnsureBuiltInElementRow(context, fileSystem, target, cancellationToken);
}
/// <summary>
/// `RefreshGraphicsElements` is what normally turns a template file into a `GraphicsElement` row,
/// but it runs on the scheduler/stream-start path -- long after startup. Creating the row here
/// removes that ordering dependency, so `AttachOnNowNextByDefault` below can never mark itself
/// done against an element that simply had not been discovered yet.
/// </summary>
private static async Task EnsureBuiltInElementRow(
TvContext context,
IFileSystem fileSystem,
string target,
CancellationToken cancellationToken)
{
if (!fileSystem.File.Exists(target))
{
return;
}
bool exists = await context.GraphicsElements.AnyAsync(e => e.Path == target, cancellationToken);
if (exists)
{
return;
}
// Name is display-only (identity is the filename), but leaving it null sorts the built-in
// element into the unnamed bucket at the bottom of the SPA list until the first refresh.
await context.GraphicsElements.AddAsync(
new Core.Domain.GraphicsElement
{
Path = target,
Kind = GraphicsElementKind.Text,
Name = GraphicsElementDefaults.OnNowNextName
},
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
}
/// <summary>
/// #732: the On Now / Next overlay is a default, not an opt-in. Existing channels predate that
/// decision, so attach the built-in element to them once.
/// </summary>
/// <remarks>
/// The marker is written only once the built-in element RESOLVES, so an install whose row does
/// not exist yet is retried on the next startup rather than stranded permanently. Once written,
/// no channel is ever re-attached. While still armed the backfill cannot tell a deliberately
/// cleared channel from an untouched one -- a single global flag cannot express both
/// properties; see <c>graphics.on-now-next-on-by-default</c> for why that trade is made this
/// way. Every channel created after the marker gets the element from
/// <c>ChannelGraphicsDefaults.Attach</c> instead, which BOTH create paths call.
/// </remarks>
public static async Task AttachOnNowNextByDefault(TvContext context, CancellationToken cancellationToken)
{
string key = ConfigElementKey.GraphicsOnNowNextDefaultAttached.Key;
if (await context.ConfigElements.AnyAsync(c => c.Key == key, cancellationToken))
{
return;
}
Option<int> maybeElementId = await GetBuiltInElementId(context, cancellationToken);
if (maybeElementId.IsNone)
{
// Nothing to attach TO. Writing the marker here would strand every channel permanently
// on the one population this exists for, so stay armed and try again next startup.
return;
}
foreach (int elementId in maybeElementId)
{
// HLS Direct has no frame pipeline to draw into, so an attachment there would be inert
// while still reading as "on" in the editor.
List<int> channelIds = await context.Channels
.Where(c => c.StreamingMode != StreamingMode.HttpLiveStreamingDirect)
.Where(c => c.ChannelGraphicsElements.All(cge => cge.GraphicsElementId != elementId))
.Select(c => c.Id)
.ToListAsync(cancellationToken);
foreach (int channelId in channelIds)
{
await context.AddAsync(
new ChannelGraphicsElement { ChannelId = channelId, GraphicsElementId = elementId },
cancellationToken);
}
}
await context.ConfigElements.AddAsync(
new ConfigElement { Key = key, Value = "true" },
cancellationToken);
await context.SaveChangesAsync(cancellationToken);
}
/// <summary>
/// Identity is the filename, never the user-editable Name (the #67 lesson carried into #74).
/// The Kind is part of it: the five template folders are separate namespaces, so an unrelated
/// image/motion/subtitle/script element may legitimately be named `on-now-next.yml` too, and
/// filename alone would hand back whichever row the unordered query happened to return first.
/// </summary>
public static async Task<Option<int>> GetBuiltInElementId(
TvContext context,
CancellationToken cancellationToken)
{
List<(int Id, string Path)> candidates = await context.GraphicsElements
.Where(e => e.Kind == GraphicsElementKind.Text)
.Select(e => new { e.Id, e.Path })
.ToListAsync(cancellationToken)
.Map(rows => rows.Select(r => (r.Id, r.Path)).ToList());
var matches = candidates
.Where(c => System.IO.Path.GetFileName(c.Path) == GraphicsElementDefaults.OnNowNextFileName)
.OrderBy(c => c.Id)
.ToList();
// Lowest id wins if two text templates somehow share the filename, so the choice is stable
// across restarts rather than dependent on query order.
return matches.Count == 0 ? Option<int>.None : matches[0].Id;
}
private static async Task UpgradeUnmodifiedTemplate(
IFileSystem fileSystem,
string target,
ILogger logger,
CancellationToken cancellationToken)
{
// This runs inside the blocking database-startup path, ahead of DatabaseIsReady(). Before
// #732 the already-seeded branch never touched the filesystem at all, so an unreadable or
// read-only template is a state that used to boot fine -- it must not become a failure to
// start. Cosmetic upgrade, best effort.
try
{
if (!fileSystem.File.Exists(target))
{
return;
}
string existing = await fileSystem.File.ReadAllTextAsync(target, cancellationToken);
if (!SupersededDefaults.Any(d => IsSameTemplate(existing, d)))
{
return;
}
// Write-then-move, never write in place. WriteAllTextAsync truncates first, so an
// interrupted write (disk full, IO fault, cancellation) would leave a partial file that
// matches no fingerprint and is therefore never repaired on a later boot -- the overlay
// would just be gone, permanently, on every channel carrying it.
//
// The temp name is random per call. A fixed one is shared by two containers on the same
// config volume, where one can truncate it while the other is mid-write and then rename
// the partial file over the live template. The process id is NOT good enough here: the
// image uses an exec-form ENTRYPOINT, so every container's PID namespace makes this
// process 1 and every container computes the same name. A random name also means a temp
// left by a crashed earlier boot is never reused. Only ever delete the path this call
// created.
string temp = $"{target}.{fileSystem.Path.GetRandomFileName()}.upgrade.tmp";
try
{
await fileSystem.File.WriteAllTextAsync(temp, OnNowNextYaml, cancellationToken);
// Deliberately NO in-place fallback when this throws. rename(2) onto a mountpoint
// is EBUSY, so a single-file bind mount of this template will not be upgraded --
// accepted, because reaching that case needs a pinned file that is ALSO byte-identical
// to a shipped default, and the alternative is reintroducing the truncation this
// whole dance exists to prevent, on every IO fault rather than just that one.
fileSystem.File.Move(temp, target, true);
}
finally
{
// Cleanup must never REPLACE the exception that brought us here. Without this inner
// catch, a delete that throws while unwinding a cancellation swaps the
// OperationCanceledException for an IOException, which the outer filter then
// swallows -- so a real shutdown would be silently downgraded to a warning.
try
{
if (fileSystem.File.Exists(temp))
{
fileSystem.File.Delete(temp);
}
}
catch (Exception cleanupEx)
{
logger.LogDebug(cleanupEx, "Could not remove the temporary upgrade file {Path}", temp);
}
}
}
// Recoverable filesystem faults only. Catching everything would swallow genuinely fatal
// runtime failures (OutOfMemory and friends) and continue booting a compromised process;
// letting IO escape would turn a file permission into a restart loop. Cancellation
// propagates so shutdown is not swallowed.
// OperationCanceledException is deliberately absent from this list so a real shutdown
// propagates -- but only a real one: an OCE raised while the token is NOT cancelled is just
// another faulty read, and letting it escape is the restart loop this catch exists to stop.
catch (Exception ex) when ((ex is IOException
or UnauthorizedAccessException
or NotSupportedException
or System.Security.SecurityException)
|| (ex is OperationCanceledException
&& !cancellationToken.IsCancellationRequested))
{
logger.LogWarning(
ex,
"Could not upgrade the built-in graphics template at {Path}; leaving it as-is",
target);
}
}
// Compare on content, ignoring the line endings and trailing whitespace an editor or a volume
// mount may rewrite. This is a fingerprint check, not a parse: anything that is not one of our
// own shipped defaults must fall through untouched.
private static bool IsSameTemplate(string left, string right) =>
string.Equals(Normalize(left), Normalize(right), StringComparison.Ordinal);
private static string Normalize(string value) =>
value.Replace("\r\n", "\n", StringComparison.Ordinal).TrimEnd();
}
@@ -14,6 +14,10 @@ public partial class TextElement(
ILogger logger)
: GraphicsElement, IDisposable
{
// Far larger than any sane overlay on an 8K frame, and small enough that every downstream
// int cast stays well inside range.
private const float MaxBoxDimension = 10_000f;
private static readonly Regex StylePattern = StyleRegex();
private SKBitmap _image;
private SKPointI _location;
@@ -62,30 +66,96 @@ public partial class TextElement(
}
}
BackgroundBox box = BuildBackgroundBox();
RichTextKit.TextBlock textBlock = BuildTextBlock(textElement.Text);
if (textElement.WidthPercent.HasValue)
// Padding and border sit OUTSIDE the laid-out text on every side, so they shrink the
// space the text may occupy and grow the bitmap that holds it. Zero when there is no
// box, which reproduces the pre-#732 geometry exactly.
//
// Round ONCE, here, and use the same integer on both sides: the bitmap grows by
// 2 * insetPixels, so subtracting the unrounded inset from the wrap budget would let a
// fractional padding push the finished box a pixel past width_percent.
var insetPixels = (int)Math.Ceiling(box?.Inset ?? 0f);
// Bound the inset against the FRAME even when there is no width_percent. Sanitize caps
// each field individually, but padding and border add up, and without a budget nothing
// else clamps them -- a two-field fat-finger would otherwise allocate a bitmap far
// larger than the frame it is drawn onto. Pre-#732 no config value could inflate the
// bitmap independently of the measured text.
int frameInsetCap = Math.Max(0, Math.Min(context.FrameSize.Width, context.FrameSize.Height) / 2);
if (insetPixels > frameInsetCap)
{
logger.LogWarning(
"Background padding/border of {Inset}px exceeds the frame; clamping to {Clamped}px",
insetPixels,
frameInsetCap);
insetPixels = frameInsetCap;
box = box?.ClampedTo(frameInsetCap);
}
// A width_percent of 1e300 makes maxWidth Infinity, and every int cast below it is then
// unspecified. Treat a non-finite budget as "no budget", which is what an absent
// width_percent already means.
if (textElement.WidthPercent.HasValue
&& float.IsFinite((float)(textElement.WidthPercent.Value / 100.0 * context.FrameSize.Width)))
{
var maxWidth = (float)Math.Round(textElement.WidthPercent.Value / 100.0 * context.FrameSize.Width);
// A padding wider than the budget itself cannot be honoured AND stay inside it.
// Clamp the inset rather than squeezing the text to 1px: an unclamped floor turns a
// fat-fingered background_padding into a box several times the requested width.
int maxInset = Math.Max(0, (int)Math.Floor((maxWidth - 1) / 2));
if (insetPixels > maxInset)
{
logger.LogWarning(
"Background padding/border of {Inset}px does not fit within width_percent "
+ "({MaxWidth}px); clamping to {Clamped}px",
insetPixels,
maxWidth,
maxInset);
// Clamp the BOX, not just the bitmap's inset. Shrinking insetPixels alone leaves
// DrawBackgroundBox stroking at the original border width, which is centred on a
// rect that no longer has room for it -- the stroke then floods the element.
insetPixels = maxInset;
box = box?.ClampedTo(maxInset);
}
// width_percent bounds the ELEMENT, so the text gets what is left after the insets.
// With no box the budget is passed through untouched -- not through Math.Max -- so a
// width_percent that rounds to 0 keeps its exact pre-#732 behavior.
float textMaxWidth = insetPixels == 0
? maxWidth
: Math.Max(1f, maxWidth - (2 * insetPixels));
switch (textElement.Fit)
{
case TextFit.Wrap:
textBlock.MaxWidth = maxWidth;
textBlock.MaxWidth = textMaxWidth;
break;
case TextFit.Scale:
FitTextBlock(textBlock, maxWidth);
FitTextBlock(textBlock, textMaxWidth);
break;
}
}
_image = new SKBitmap(
(int)Math.Ceiling(textBlock.MeasuredWidth),
(int)Math.Ceiling(textBlock.MeasuredHeight));
(int)Math.Ceiling(textBlock.MeasuredWidth) + (2 * insetPixels),
(int)Math.Ceiling(textBlock.MeasuredHeight) + (2 * insetPixels));
using (var canvas = new SKCanvas(_image))
{
canvas.Clear(SKColors.Transparent);
textBlock.Paint(canvas, new SKPoint(0, 0));
if (box is not null)
{
DrawBackgroundBox(canvas, box, _image.Width, _image.Height);
}
textBlock.Paint(canvas, new SKPoint(insetPixels, insetPixels));
}
var horizontalMargin =
@@ -134,6 +204,158 @@ public partial class TextElement(
: new ValueTask<Option<PreparedElementImage>>(new PreparedElementImage(_image, _location, opacity, ZIndex, false));
}
// A background box is drawn only when a colour actually parses. An unparseable colour is
// warned about and skipped rather than substituted, so a typo never silently changes the
// look into something that appears deliberate.
private BackgroundBox BuildBackgroundBox()
{
SKColor? fill = ParseOptionalColor(textElement.BackgroundColor, "background_color");
if (fill.HasValue)
{
fill = ApplyOpacityPercent(fill.Value, textElement.BackgroundOpacityPercent);
}
SKColor? border = ParseOptionalColor(textElement.BorderColor, "border_color");
// A border colour with no explicit width means a hairline border, not an invisible one:
// "border_color set, nothing drawn" is the more confusing of the two readings.
float borderWidth = Sanitize(textElement.BorderWidth ?? 1, "border_width");
if (!border.HasValue)
{
borderWidth = 0;
}
if (!fill.HasValue && borderWidth <= 0)
{
return null;
}
return new BackgroundBox(
fill,
border,
borderWidth,
Sanitize(textElement.BackgroundCornerRadius, "background_corner_radius"),
Sanitize(textElement.BackgroundPadding, "background_padding"));
}
// YAML happily yields 1e100 or NaN. Cast to float those become Infinity/NaN, and
// (int)Math.Ceiling(Infinity) is an unspecified value -- in practice int.MinValue, which sails
// straight past every `> maxInset` clamp and can wrap 2 * inset back to zero. Sanitize at the
// boundary so no downstream arithmetic ever sees a non-finite value.
private float Sanitize(double? value, string fieldName)
{
if (value is not { } raw)
{
return 0f;
}
if (double.IsNaN(raw) || raw < 0)
{
logger.LogWarning("Ignoring out-of-range {Field} value {Value}", fieldName, raw);
return 0f;
}
if (raw > MaxBoxDimension)
{
logger.LogWarning(
"Clamping {Field} value {Value} to {Max}",
fieldName,
raw,
MaxBoxDimension);
return MaxBoxDimension;
}
return (float)raw;
}
private SKColor? ParseOptionalColor(string value, string fieldName)
{
if (string.IsNullOrWhiteSpace(value))
{
return null;
}
if (SKColor.TryParse(value, out SKColor parsed))
{
return parsed;
}
logger.LogWarning(
"Unable to parse {Field} value {Value}; that part of the background box will not be drawn",
fieldName,
value);
return null;
}
private static SKColor ApplyOpacityPercent(SKColor color, int? opacityPercent)
{
if (opacityPercent is not { } percent)
{
return color;
}
int clamped = Math.Clamp(percent, 0, 100);
return color.WithAlpha((byte)Math.Round(color.Alpha * clamped / 100.0));
}
private static void DrawBackgroundBox(SKCanvas canvas, BackgroundBox box, int width, int height)
{
// Skia strokes centred on the path, so half the border would fall outside the bitmap and
// be clipped. Inset the rect by half the width to keep the whole border visible.
float half = box.BorderWidth / 2f;
var rect = new SKRect(half, half, width - half, height - half);
// A radius larger than half the shorter side is not expressible as a rounded rect.
float radius = Math.Min(box.CornerRadius, Math.Min(rect.Width, rect.Height) / 2f);
radius = Math.Max(0, radius);
if (box.Fill is { } fill)
{
using var fillPaint = new SKPaint
{
Color = fill,
Style = SKPaintStyle.Fill,
IsAntialias = true
};
canvas.DrawRoundRect(rect, radius, radius, fillPaint);
}
if (box.Border is { } border && box.BorderWidth > 0)
{
using var borderPaint = new SKPaint
{
Color = border,
Style = SKPaintStyle.Stroke,
StrokeWidth = box.BorderWidth,
IsAntialias = true
};
canvas.DrawRoundRect(rect, radius, radius, borderPaint);
}
}
private sealed record BackgroundBox(
SKColor? Fill,
SKColor? Border,
float BorderWidth,
float CornerRadius,
float Padding)
{
public float Inset => Padding + BorderWidth;
// Border first, then whatever is left goes to padding: a border that cannot be drawn inside
// the bitmap is worse than a thin one, and padding degrades gracefully to zero.
public BackgroundBox ClampedTo(float maxInset)
{
float borderWidth = Math.Min(BorderWidth, maxInset);
float padding = Math.Max(0, maxInset - borderWidth);
return this with { BorderWidth = borderWidth, Padding = padding };
}
}
private RichTextKit.TextBlock BuildTextBlock(string textToRender)
{
var textBlock = new RichTextKit.TextBlock
@@ -211,6 +433,17 @@ public partial class TextElement(
finalStyle.TextColor = parsedColor;
}
// Halo is per-style in the schema and was being dropped here, so a non-base style's
// halo_* silently inherited the base style's. The seeded template only looked correct
// because all three of its styles declare the same halo.
finalStyle.HaloWidth = s.HaloWidth ?? finalStyle.HaloWidth;
finalStyle.HaloBlur = s.HaloBlur ?? finalStyle.HaloBlur;
if (s.HaloColor != null && SKColor.TryParse(s.HaloColor, out SKColor parsedHalo))
{
finalStyle.HaloColor = parsedHalo;
}
styles[s.Name] = finalStyle;
}
@@ -286,6 +519,11 @@ public partial class TextElement(
foreach ((string text, RichTextKit.IStyle style) in originalContent)
{
// Carry across every property the YAML schema can set, not just the ones the scale
// needs (the rest are RichTextKit defaults we never touch). Halo and
// line height were being dropped here, which only mattered once #732 gave the box
// insets that can push a previously-fitting element into the Scale path: adding a
// background would then silently remove the halo it sits behind.
var newStyle = new RichTextKit.Style
{
FontFamily = style.FontFamily,
@@ -294,7 +532,11 @@ public partial class TextElement(
FontWidth = style.FontWidth,
FontWeight = style.FontWeight,
LetterSpacing = style.LetterSpacing,
TextColor = style.TextColor
LineHeight = style.LineHeight,
TextColor = style.TextColor,
HaloColor = style.HaloColor,
HaloWidth = style.HaloWidth,
HaloBlur = style.HaloBlur
};
float newSize = newStyle.FontSize * scale;
@@ -21,4 +21,16 @@
<ProjectReference Include="..\ErsatzTV.Mcp\ErsatzTV.Mcp.csproj" />
</ItemGroup>
<!--
The generated OpenAPI document is the wire contract the MCP catalog wraps. Copying it into the
test output lets ToolCatalogTests assert that every write tool declares exactly the request-body
fields its endpoint accepts, so a new DTO property cannot drift out of a tool schema unnoticed
(issue #754). Regenerated by scripts/update-openapi.sh.
-->
<ItemGroup>
<Content Include="..\ErsatzTV\wwwroot\openapi\v1.json"
Link="openapi\v1.json"
CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
</Project>
+260 -3
View File
@@ -77,9 +77,22 @@ public class ToolCatalogTests
// Enums must NOT be forced required (they have server-side defaults).
createRequired.ShouldNotContain("streamingMode");
// Update carries the same body fields plus the route id.
update.InputSchema.RootElement.GetProperty("properties").TryGetProperty("id", out _).ShouldBeTrue();
update.InputSchema.RootElement.GetProperty("properties").TryGetProperty("showInEpg", out _).ShouldBeTrue();
// Update carries the create body fields plus the route id...
JsonElement updateProps = update.InputSchema.RootElement.GetProperty("properties");
updateProps.TryGetProperty("id", out _).ShouldBeTrue();
updateProps.TryGetProperty("showInEpg", out _).ShouldBeTrue();
// ...plus graphicsElementIds, which is on UpdateChannelRequest only. PUT is a full replace, so
// while the tool could not express this field an agent following the tool's own "send the full
// desired state" instruction silently detached every graphics element (issue #754).
updateProps.TryGetProperty("graphicsElementIds", out JsonElement graphicsElementIds).ShouldBeTrue();
graphicsElementIds.GetProperty("type").GetString().ShouldBe("array");
graphicsElementIds.GetProperty("items").GetProperty("type").GetString().ShouldBe("integer");
// Create must NOT send it: CreateChannelRequest has no such property, and the tool schema is
// additionalProperties:false. This is why it is declared on the update tool rather than in the
// shared ChannelFields().
createProps.TryGetProperty("graphicsElementIds", out _).ShouldBeFalse();
}
[Test]
@@ -256,4 +269,248 @@ public class ToolCatalogTests
tool.QueryParameters.ShouldNotBeNull();
tool.QueryParameters!.ShouldContain("deep");
}
// #754: ToolCatalog declared 27 of UpdateChannelRequest's 28 properties. The missing one was
// graphicsElementIds, and because PUT /api/v1/channels/{id} is a FULL REPLACE the omission was not
// merely "one field you cannot set" — an agent that GET-edit-PUT the channel, exactly as the tool's
// description tells it to, detached every graphics element (including the On Now/Next overlay) with
// a 200 and no error. The same shape was live on ersatztv_update_schedule, which omitted
// padToNearestMinute and silently cleared a configured pad.
//
// Neither is fixable by counting fields once: the defect is that nothing tied the tool schema to the
// contract it wraps. So this test asserts the tie for EVERY write tool against the generated OpenAPI
// document (the actual wire contract, linked into the test output by the csproj). A new property on
// any request DTO now fails here until the catalog declares it.
[Test]
public void Every_Write_Tool_Should_Declare_Exactly_Its_OpenApi_Request_Body_Fields()
{
using JsonDocument spec = LoadOpenApiDocument();
JsonElement paths = spec.RootElement.GetProperty("paths");
ToolDefinition[] writeTools = ToolCatalog.All
.Where(t => t.HttpMethod == HttpMethod.Post
|| t.HttpMethod == HttpMethod.Put
|| t.HttpMethod == HttpMethod.Patch)
.ToArray();
// Pin the covered set rather than trusting the filter. A tool that stopped being a write verb,
// or a new write tool, must show up as a change here — a bare loop over a filtered set passes
// just as happily when the set silently shrinks to nothing.
string[] expectedWriteTools =
[
"ersatztv_add_collection_items",
"ersatztv_create_channel",
"ersatztv_create_collection",
"ersatztv_create_playout",
"ersatztv_create_schedule",
"ersatztv_create_smart_collection",
"ersatztv_enable_jellyfin_library_sync",
"ersatztv_refresh_jellyfin_libraries",
"ersatztv_reset_channel_playout",
"ersatztv_scan_jellyfin_collections",
"ersatztv_scan_library",
"ersatztv_update_channel",
"ersatztv_update_collection",
"ersatztv_update_collection_custom_order",
"ersatztv_update_playout",
"ersatztv_update_schedule",
"ersatztv_update_smart_collection"
];
writeTools.Select(t => t.Name).OrderBy(n => n, StringComparer.Ordinal)
.ShouldBe(expectedWriteTools.OrderBy(n => n, StringComparer.Ordinal));
foreach (ToolDefinition tool in writeTools)
{
paths.TryGetProperty(tool.PathTemplate, out JsonElement pathItem)
.ShouldBeTrue($"{tool.Name}: {tool.PathTemplate} is not in the OpenAPI document");
string verb = tool.HttpMethod.Method.ToLowerInvariant();
pathItem.TryGetProperty(verb, out JsonElement operation)
.ShouldBeTrue($"{tool.Name}: {verb.ToUpperInvariant()} {tool.PathTemplate} is not in the OpenAPI document");
Dictionary<string, string> declared = DeclaredBodyArguments(tool);
Dictionary<string, string> accepted = RequestBodyProperties(spec, operation, tool.Name);
// Compare name AND type. Names alone would let a field drift to the wrong JSON type: the
// tool would advertise "string" for an int?, the agent would send "30", and the API would
// 400 — green test, broken tool.
declared.Select(p => $"{p.Key}: {p.Value}").OrderBy(s => s, StringComparer.Ordinal)
.ShouldBe(
accepted.Select(p => $"{p.Key}: {p.Value}").OrderBy(s => s, StringComparer.Ordinal),
customMessage:
$"{tool.Name} declares body fields that do not match {verb.ToUpperInvariant()} {tool.PathTemplate}. "
+ "A field the endpoint accepts but the tool omits is silently dropped on a full-replace "
+ "write (#754); a field the tool sends but the endpoint does not accept is rejected; "
+ "a field declared with the wrong type is rejected at the API.");
}
}
// #757, the sibling of the body guard above. Query parameters drift the same way and are WORSE for
// reads: ToolArgumentValidator rejects undeclared arguments, so a parameter the tool omits is not
// merely undocumented, it is unreachable — the caller cannot pass it at all. That is how #616's
// paging omission hard-capped two tools at the first page. This covers EVERY tool, not just the
// write verbs, because the drift that existed when this was written was entirely on reads.
[Test]
public void Every_Tool_Should_Declare_Exactly_Its_OpenApi_Query_Parameters()
{
using JsonDocument spec = LoadOpenApiDocument();
JsonElement paths = spec.RootElement.GetProperty("paths");
// Every tool is covered, so an emptiness guard is enough here — there is no filter to escape.
ToolCatalog.All.Count.ShouldBeGreaterThan(30);
// Accumulate rather than throwing on the first mismatch, so one run reports the WHOLE drift set.
// Failing fast here would hand back one tool at a time and invite fixing them one at a time,
// which is how the #754 twin stayed hidden in the first place.
List<string> drift = [];
foreach (ToolDefinition tool in ToolCatalog.All)
{
paths.TryGetProperty(tool.PathTemplate, out JsonElement pathItem)
.ShouldBeTrue($"{tool.Name}: {tool.PathTemplate} is not in the OpenAPI document");
string verb = tool.HttpMethod.Method.ToLowerInvariant();
pathItem.TryGetProperty(verb, out JsonElement operation)
.ShouldBeTrue($"{tool.Name}: {verb.ToUpperInvariant()} {tool.PathTemplate} is not in the OpenAPI document");
IReadOnlySet<string> declared = tool.QueryParameters ?? new HashSet<string>(StringComparer.Ordinal);
HashSet<string> accepted = QueryParameterNames(operation);
string[] missing = accepted.Except(declared, StringComparer.Ordinal).OrderBy(n => n, StringComparer.Ordinal).ToArray();
string[] phantom = declared.Except(accepted, StringComparer.Ordinal).OrderBy(n => n, StringComparer.Ordinal).ToArray();
if (missing.Length > 0 || phantom.Length > 0)
{
drift.Add(
$"{tool.Name} ({verb.ToUpperInvariant()} {tool.PathTemplate}): "
+ $"unreachable={string.Join(",", missing)} phantom={string.Join(",", phantom)}");
}
}
// A parameter the endpoint accepts but the tool omits is UNREACHABLE, not merely undocumented:
// ToolArgumentValidator rejects undeclared arguments, so the caller cannot pass it at all
// (#616 hard-capped two paged tools exactly this way). A phantom is the reverse — the tool
// advertises something the endpoint ignores.
drift.ShouldBeEmpty();
}
private static HashSet<string> QueryParameterNames(JsonElement operation)
{
if (!operation.TryGetProperty("parameters", out JsonElement parameters))
{
return [];
}
return parameters.EnumerateArray()
.Where(p => p.TryGetProperty("in", out JsonElement location)
&& string.Equals(location.GetString(), "query", StringComparison.Ordinal))
.Select(p => p.GetProperty("name").GetString())
.OfType<string>()
.ToHashSet(StringComparer.Ordinal);
}
// The body is every declared argument that is not routed elsewhere — mirroring exactly how
// ErsatzTvApiClient builds the request, so this test cannot disagree with the code it guards.
// DELETE is not compared: ErsatzTvApiClient sets hasBody for POST/PUT/PATCH only, so a body
// argument on a DELETE tool would be silently dropped. No DELETE tool has one today.
private static Dictionary<string, string> DeclaredBodyArguments(ToolDefinition tool)
{
var pathParameters = Regex.Matches(tool.PathTemplate, @"\{([^}]+)\}")
.Select(m => m.Groups[1].Value)
.ToHashSet(StringComparer.Ordinal);
IReadOnlySet<string> queryParameters = tool.QueryParameters ?? new HashSet<string>(StringComparer.Ordinal);
if (!tool.InputSchema.RootElement.TryGetProperty("properties", out JsonElement properties))
{
return [];
}
return properties.EnumerateObject()
.Where(p => !pathParameters.Contains(p.Name)
&& !queryParameters.Contains(p.Name)
&& !string.Equals(p.Name, "ifMatch", StringComparison.Ordinal))
.ToDictionary(p => p.Name, p => DeclaredType(p.Value), StringComparer.Ordinal);
}
// The tool schema's own shape: a plain "type", plus the array element type where there is one.
private static string DeclaredType(JsonElement property)
{
string type = property.GetProperty("type").GetString().ShouldNotBeNull();
return type == "array" && property.TryGetProperty("items", out JsonElement items)
? $"array<{items.GetProperty("type").GetString()}>"
: type;
}
private static Dictionary<string, string> RequestBodyProperties(JsonDocument spec, JsonElement operation, string toolName)
{
// No request body at all (queue/scan POSTs) — the tool must send none either.
if (!operation.TryGetProperty("requestBody", out JsonElement requestBody))
{
return [];
}
JsonElement schema = requestBody
.GetProperty("content")
.GetProperty("application/json")
.GetProperty("schema");
// Every request body in this document is a plain $ref to a component schema. Anything else
// (allOf/inline/oneOf) is a contract shape this guard has not been taught to read, so fail
// loudly rather than comparing against an empty set and reporting a false pass.
schema.TryGetProperty("$ref", out JsonElement reference)
.ShouldBeTrue($"{toolName}: request body schema is not a $ref; teach this test the new shape");
JsonElement schemas = spec.RootElement.GetProperty("components").GetProperty("schemas");
string componentName = reference.GetString().ShouldNotBeNull().Split('/')[^1];
return schemas
.GetProperty(componentName)
.GetProperty("properties")
.EnumerateObject()
.ToDictionary(p => p.Name, p => SpecType(schemas, p.Value, toolName, p.Name), StringComparer.Ordinal);
}
// Normalize the generator's shapes onto the catalog's vocabulary. Two forms appear in this
// document: a nullable type as ["null", T] (the catalog has no nullable notion — optionality is
// carried by `required`), and a $ref to a component, which for the enum fields is a string enum
// and for `logo` is an object.
private static string SpecType(JsonElement schemas, JsonElement property, string toolName, string fieldName)
{
if (property.TryGetProperty("$ref", out JsonElement reference))
{
string componentName = reference.GetString().ShouldNotBeNull().Split('/')[^1];
return SpecType(schemas, schemas.GetProperty(componentName), toolName, fieldName);
}
JsonElement type = property.GetProperty("type");
string[] types = type.ValueKind == JsonValueKind.Array
? type.EnumerateArray().Select(t => t.GetString()).OfType<string>().Where(t => t != "null").ToArray()
: [type.GetString().ShouldNotBeNull()];
// More than one non-null type is a shape this guard has not been taught to read; fail rather
// than picking one and reporting a comparison that means nothing.
types.Length.ShouldBe(1, $"{toolName}.{fieldName}: unexpected OpenAPI type union [{string.Join(", ", types)}]");
// The element schema is resolved through the same normalization: an array's items can itself be
// a $ref to a component (ReplaceRemoteLibraryPreferencesRequest.libraries), which the catalog
// declares as an object array.
return types[0] == "array" && property.TryGetProperty("items", out JsonElement items)
? $"array<{SpecType(schemas, items, toolName, fieldName)}>"
: types[0];
}
private static JsonDocument LoadOpenApiDocument()
{
string path = Path.Combine(AppContext.BaseDirectory, "openapi", "v1.json");
// A missing spec would make every assertion above vacuous, so it is an explicit failure.
File.Exists(path).ShouldBeTrue(
$"OpenAPI document not found at {path}; the test project links it from ErsatzTV/wwwroot/openapi/v1.json");
return JsonDocument.Parse(File.ReadAllText(path));
}
}
+43 -6
View File
@@ -29,14 +29,32 @@ public static class ToolCatalog
Get("ersatztv_list_schedules", "List schedules.", "/api/v1/schedules"),
Get("ersatztv_get_schedule", "Get a schedule by id.", "/api/v1/schedules/{id}", IdPath("Schedule id.")),
Get("ersatztv_get_schedule_items", "Get a schedule's items. Emits the schedule ETag.", "/api/v1/schedules/{id}/items", IdPath("Schedule id.")),
Get("ersatztv_list_playouts", "List playouts (paged).", "/api/v1/playouts", [], Page()),
Get(
"ersatztv_list_playouts",
"List playouts (paged), optionally filtered by channel name.",
"/api/v1/playouts",
[],
[
Str(
"query",
"Optional case-insensitive substring match on the CHANNEL name (not the playout or schedule name); omit for all playouts.",
arg: In.Query),
.. Page()
]),
Get("ersatztv_get_playout", "Get a playout by id.", "/api/v1/playouts/{id}", IdPath("Playout id.")),
Get(
"ersatztv_get_playout_items",
"Get upcoming items (and unscheduled gaps) for a playout (paged).",
"/api/v1/playouts/{id}/items",
[IdPath("Playout id.")],
Page()),
[
Bool(
"showFiller",
"Include items whose filler kind is not None (pre/mid/post-roll, tail, fallback, guide-mode, deco); "
+ "default false returns only non-filler items.",
arg: In.Query),
.. Page()
]),
Get("ersatztv_list_ffmpeg_profiles", "List FFmpeg profiles.", "/api/v1/ffmpeg/profiles"),
Get("ersatztv_get_ffmpeg_profile", "Get an FFmpeg profile by id.", "/api/v1/ffmpeg/profiles/{id}", IdPath("FFmpeg profile id.")),
Get(
@@ -132,7 +150,8 @@ public static class ToolCatalog
[Str("name", "Schedule name.", required: true), .. ScheduleFlags()]),
Put(
"ersatztv_update_schedule",
"Update a program schedule's settings.",
"Update a program schedule. Send the full desired state: every field is applied, so omitting "
+ "padToNearestMinute CLEARS a configured pad (GET the schedule first to copy current values).",
"/api/v1/schedules/{id}",
[IdPath("Schedule id."), Str("name", "Schedule name.", required: true), .. ScheduleFlags()]),
Delete("ersatztv_delete_schedule", "Delete a program schedule.", "/api/v1/schedules/{id}", IdPath("Schedule id.")),
@@ -159,9 +178,22 @@ public static class ToolCatalog
ChannelFields()),
Put(
"ersatztv_update_channel",
"Update a channel. Send the full desired state; enum fields take the enum name (GET the channel first to copy current values).",
"Update a channel. Send the full desired state; enum fields take the enum name (GET the channel first to copy current values). "
+ "graphicsElementIds is part of that state: omitting it DETACHES every graphics element (e.g. the On Now/Next overlay), "
+ "so copy it from ersatztv_get_channel unless you mean to clear it.",
"/api/v1/channels/{id}",
[IdPath("Channel id."), .. ChannelFields()]),
[
IdPath("Channel id."),
.. ChannelFields(),
// Update-only: UpdateChannelRequest carries GraphicsElementIds, CreateChannelRequest does
// not, so this cannot move into the shared ChannelFields() without making create send an
// unknown property. PUT is a full replace, so omitting it detaches every attached element
// with no error — issue #754.
IntArray(
"graphicsElementIds",
"Ids of the graphics elements attached to the channel. Full replace: omit or send [] to detach all.")
]),
Post(
"ersatztv_reset_channel_playout",
"Queue a rebuild of a channel's playout (202 Accepted; 409 if a build is already running).",
@@ -297,7 +329,12 @@ public static class ToolCatalog
Bool("treatCollectionsAsShows", "Treat collections as shows."),
Bool("shuffleScheduleItems", "Shuffle schedule items."),
Bool("randomStartPoint", "Use a random start point."),
Str("fixedStartTimeBehavior", "Fixed start-time behavior (enum name; GET a schedule to see valid values).")
Str("fixedStartTimeBehavior", "Fixed start-time behavior (enum name; GET a schedule to see valid values)."),
// Both Create- and UpdateScheduleRequest carry this, so it belongs in the shared helper. The
// update PUT is a full replace that writes the value unconditionally, so omitting it used to
// clear a configured pad silently — the same #754 shape as channel graphicsElementIds.
Int("padToNearestMinute", "Pad each item to the nearest N minutes; omit or send null for no padding.")
];
// ---- Tool factories ----
@@ -0,0 +1,92 @@
using ErsatzTV.Application.Channels;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Graphics;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
using Microsoft.EntityFrameworkCore;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Application.Channels;
/// <summary>
/// #732: the On Now / Next overlay is a default rather than an opt-in, so a channel created after
/// that decision gets the built-in element without the operator toggling anything.
/// </summary>
[TestFixture]
public class CreateChannelDefaultGraphicsElementTests : ChannelHandlerTestBase
{
private CreateChannelHandler MakeHandler() => new(Worker, Db.Factory, SearchTargets, RemoteLogoCacher);
private async Task<int> SeedBuiltInElement()
{
await using TvContext context = Db.CreateContext();
var element = new GraphicsElement
{
Path = $"/templates/text/{GraphicsElementDefaults.OnNowNextFileName}",
Kind = GraphicsElementKind.Text
};
context.GraphicsElements.Add(element);
await context.SaveChangesAsync();
return element.Id;
}
private async Task<List<int>> AttachedElementIds(int channelId)
{
await using TvContext context = Db.CreateContext();
Channel reloaded = await context.Channels
.Include(c => c.ChannelGraphicsElements)
.SingleAsync(c => c.Id == channelId);
return reloaded.ChannelGraphicsElements.Select(x => x.GraphicsElementId).ToList();
}
[Test]
public async Task Attaches_The_Built_In_Element_To_A_New_Channel()
{
await SeedFFmpegProfile();
int elementId = await SeedBuiltInElement();
Either<BaseError, CreateChannelResult> result =
await MakeHandler().Handle(MakeCreate(), CancellationToken.None);
result.IsRight.ShouldBeTrue();
int channelId = result.RightToSeq().Head().ChannelId;
(await AttachedElementIds(channelId)).ShouldBe([elementId]);
}
[Test]
public async Task Leaves_An_Hls_Direct_Channel_Alone_Because_Nothing_Can_Render_There()
{
await SeedFFmpegProfile();
await SeedBuiltInElement();
Either<BaseError, CreateChannelResult> result = await MakeHandler().Handle(
MakeCreate(streamingMode: StreamingMode.HttpLiveStreamingDirect),
CancellationToken.None);
result.IsRight.ShouldBeTrue();
int channelId = result.RightToSeq().Head().ChannelId;
(await AttachedElementIds(channelId)).ShouldBeEmpty();
}
[Test]
public async Task Creates_The_Channel_Even_When_The_Built_In_Element_Does_Not_Exist()
{
await SeedFFmpegProfile();
Either<BaseError, CreateChannelResult> result =
await MakeHandler().Handle(MakeCreate(), CancellationToken.None);
result.IsRight.ShouldBeTrue();
int channelId = result.RightToSeq().Head().ChannelId;
(await AttachedElementIds(channelId)).ShouldBeEmpty();
}
}
@@ -9,9 +9,11 @@ using ErsatzTV.Core.Api.LibraryBrowse;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Graphics;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Core.Scheduling;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
@@ -964,4 +966,66 @@ public class CreateChannelFromLineupHandlerTests
private static TR RightOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => throw new AssertionException($"Expected a Right result, got {e.Value}"), Right: r => r);
// #732: this is the SPA's primary "Add Channel" flow and the one Auto-Tune bulk-creates through.
// It was the channel-creation site the default attach originally missed, so a channel made here
// would silently never get the overlay once the one-time backfill marker had landed.
[Test]
public async Task Should_Attach_The_Built_In_On_Now_Next_Element()
{
await SeedTemplateDependencies();
await SeedTemplate();
await SeedMovie(42);
int elementId = await SeedBuiltInGraphicsElement();
Either<BaseError, CreateChannelFromLineupResponseModel> result =
await MakeHandler().Handle(MakeRequest(), CancellationToken.None);
CreateChannelFromLineupResponseModel response = RightOf(result);
await using TvContext context = _db.CreateContext();
DomainChannel channel = await context.Channels
.Include(c => c.ChannelGraphicsElements)
.SingleAsync(c => c.Id == response.ChannelId);
channel.ChannelGraphicsElements.Select(x => x.GraphicsElementId).ShouldBe([elementId]);
}
[Test]
public async Task Should_Not_Attach_The_Overlay_To_An_Hls_Direct_Channel()
{
await SeedTemplateDependencies();
await SeedTemplate();
await SeedMovie(42);
await SeedBuiltInGraphicsElement();
Either<BaseError, CreateChannelFromLineupResponseModel> result = await MakeHandler().Handle(
MakeRequest(advanced: new CreateChannelFromLineupAdvancedOptions(
PlaybackOrder.Shuffle,
StreamingMode: StreamingMode.HttpLiveStreamingDirect)),
CancellationToken.None);
CreateChannelFromLineupResponseModel response = RightOf(result);
await using TvContext context = _db.CreateContext();
DomainChannel channel = await context.Channels
.Include(c => c.ChannelGraphicsElements)
.SingleAsync(c => c.Id == response.ChannelId);
channel.ChannelGraphicsElements.ShouldBeEmpty();
}
private async Task<int> SeedBuiltInGraphicsElement()
{
await using TvContext context = _db.CreateContext();
var element = new GraphicsElement
{
Path = $"/templates/text/{GraphicsElementDefaults.OnNowNextFileName}",
Kind = GraphicsElementKind.Text
};
context.GraphicsElements.Add(element);
await context.SaveChangesAsync();
return element.Id;
}
}
@@ -5,6 +5,9 @@ using ErsatzTV.Core.Errors;
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.FFmpeg;
using ErsatzTV.FFmpeg.OutputFormat;
using Microsoft.EntityFrameworkCore;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using LanguageExt;
@@ -134,15 +137,30 @@ public class FFmpegProfileHandlerTests
persisted.QsvPreferNativeDecoder.ShouldBe(false);
}
// ersatztv#529: a stored 0 reached ffmpeg as hwupload=extra_hw_frames=0, leaving the QSV pool no
// headroom; FFmpegState floors it at render time, and these pin that the stored row converges too
// so the profile never keeps displaying a value the pipeline would override.
[TestCase(0, 64)]
[TestCase(-8, 64)]
[TestCase(63, 64)]
[TestCase(64, 64)]
[TestCase(128, 128)]
public async Task Create_Should_Floor_QsvExtraHardwareFrames(int configured, int expected)
// ersatztv#735: the write path used to accept an out-of-range pool size and store the floored
// value instead, so a client that PUT 0 got a 200 and read back 64. it is now rejected, naming
// the bound; FFmpegState still floors at render time for rows that predate this.
[TestCase(0)]
[TestCase(-8)]
[TestCase(63)]
public async Task Create_Should_Reject_QsvExtraHardwareFrames_Below_Minimum(int configured)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, qsvExtraHardwareFrames: configured),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("at least 64");
await using TvContext context = _db.CreateContext();
(await context.FFmpegProfiles.CountAsync()).ShouldBe(0);
}
[TestCase(64)]
[TestCase(128)]
public async Task Create_Should_Store_QsvExtraHardwareFrames_Exactly_As_Submitted(int configured)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
@@ -155,15 +173,15 @@ public class FFmpegProfileHandlerTests
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.QsvExtraHardwareFrames.ShouldBe(expected);
persisted.QsvExtraHardwareFrames.ShouldBe(configured);
}
[TestCase(0, 64)]
[TestCase(-8, 64)]
[TestCase(128, 128)]
public async Task Update_Should_Floor_QsvExtraHardwareFrames(int configured, int expected)
[TestCase(0)]
[TestCase(-8)]
[TestCase(63)]
public async Task Update_Should_Reject_A_Newly_Submitted_QsvExtraHardwareFrames_Below_Minimum(int configured)
{
await SeedProfile(1);
await SeedProfile(1, qsvExtraHardwareFrames: 128);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
@@ -171,11 +189,59 @@ public class FFmpegProfileHandlerTests
MakeUpdate(1, qsvExtraHardwareFrames: configured),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("at least 64");
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.QsvExtraHardwareFrames.ShouldBe(128);
}
// the other half of the same rule: the SPA sends the whole profile back on every edit, so a row
// stored before this validation existed must stay editable over fields the operator did touch.
// an UNCHANGED out-of-range value is written back as-is and floored at render time instead
[Test]
public async Task Update_Should_Accept_An_Unchanged_Legacy_QsvExtraHardwareFrames()
{
await SeedProfile(1, qsvExtraHardwareFrames: 0);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, UpdateFFmpegProfileResult> result = await handler.Handle(
MakeUpdate(1, qsvExtraHardwareFrames: 0),
CancellationToken.None);
RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.QsvExtraHardwareFrames.ShouldBe(expected);
persisted.QsvExtraHardwareFrames.ShouldBe(0);
new FFmpegState(
false,
HardwareAccelerationMode.None,
HardwareAccelerationMode.None,
None,
None,
None,
None,
false,
None,
None,
None,
None,
None,
OutputFormatKind.MpegTs,
None,
None,
None,
None,
TimeSpan.Zero,
None,
Optional(persisted.QsvExtraHardwareFrames),
false,
false,
"linear",
false)
.QsvExtraHardwareFrames.ShouldBe(FFmpegState.MinimumQsvExtraHardwareFrames);
}
// null means "unconfigured" and FFmpegState already resolves it to the same 64; it must stay
@@ -196,6 +262,133 @@ public class FFmpegProfileHandlerTests
persisted.QsvExtraHardwareFrames.ShouldBeNull();
}
// ersatztv#735: readrate pacing is an operator-tunable bounded field. out of band it is a dead
// channel either way — below realtime the client starves, above the ceiling the input is no
// longer meaningfully paced (which is the unthrottled read #529 measured to write no segments)
[TestCase(0.9)]
[TestCase(0.0)]
[TestCase(2.5)]
public async Task Create_Should_Reject_ReadRate_Outside_Bounds(double configured)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, readRate: configured),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("Read rate must be between 1.0 and 2.0");
}
[TestCase(0.9)]
[TestCase(10.5)]
public async Task Create_Should_Reject_ReadRateCatchup_Outside_Bounds(double configured)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, readRateCatchup: configured),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("Read rate catchup must be between 1.0 and 10.0");
}
// a catchup rate inside its own band can still be at or below the base rate, where it cannot
// let a lagging input recover — the cross-field bound is the one a per-field check cannot see.
// the EQUAL cases matter: zero headroom is functionally no catchup, while still reading as a
// configured one
[TestCase(null, 1.0)]
[TestCase(null, 1.05)]
[TestCase(1.5, 1.2)]
[TestCase(1.5, 1.5)]
[TestCase(2.0, 2.0)]
public async Task Create_Should_Reject_ReadRateCatchup_At_Or_Below_The_ReadRate(double? readRate, double catchup)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, readRate: readRate, readRateCatchup: catchup),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("must be greater than the read rate");
}
[Test]
public async Task Create_Should_Persist_ReadRate_Pacing()
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, readRate: 1.2, readRateCatchup: 4.0),
CancellationToken.None);
CreateFFmpegProfileResult created = RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.ReadRate.ShouldBe(1.2);
persisted.ReadRateCatchup.ShouldBe(4.0);
}
// unset is the default posture and must stay null: FFmpegState resolves null to the values the
// pipeline used before the fields existed, so an untouched profile paces exactly as it did
[Test]
public async Task Create_Should_Leave_Unset_ReadRate_Pacing_Null()
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result =
await handler.Handle(MakeCreate(1), CancellationToken.None);
CreateFFmpegProfileResult created = RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.ReadRate.ShouldBeNull();
persisted.ReadRateCatchup.ShouldBeNull();
}
[Test]
public async Task Update_Should_Reject_ReadRate_Outside_Bounds()
{
await SeedProfile(1);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, UpdateFFmpegProfileResult> result = await handler.Handle(
MakeUpdate(1, readRate: 3.0),
CancellationToken.None);
LeftOf(result).Value.ShouldContain("Read rate must be between 1.0 and 2.0");
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.ReadRate.ShouldBeNull();
}
[Test]
public async Task Update_Should_Persist_ReadRate_Pacing()
{
await SeedProfile(1);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, UpdateFFmpegProfileResult> result = await handler.Handle(
MakeUpdate(1, readRate: 1.5, readRateCatchup: 8.0),
CancellationToken.None);
RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.ReadRate.ShouldBe(1.5);
persisted.ReadRateCatchup.ShouldBe(8.0);
}
private static TR RightOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => throw new AssertionException($"Expected a Right result, got {e}"), Right: r => r);
@@ -209,12 +402,13 @@ public class FFmpegProfileHandlerTests
await context.SaveChangesAsync();
}
private async Task SeedProfile(int id)
private async Task SeedProfile(int id, int? qsvExtraHardwareFrames = null)
{
await using TvContext context = _db.CreateContext();
context.FFmpegProfiles.Add(new FFmpegProfile
{
Id = id,
QsvExtraHardwareFrames = qsvExtraHardwareFrames,
Name = "Default",
ThreadCount = 1,
NormalizeAudio = true,
@@ -249,7 +443,9 @@ public class FFmpegProfileHandlerTests
private static CreateFFmpegProfile MakeCreate(
int resolutionId,
bool qsvPreferNativeDecoder = true,
int? qsvExtraHardwareFrames = null) =>
int? qsvExtraHardwareFrames = null,
double? readRate = null,
double? readRateCatchup = null) =>
new(
"Default",
1,
@@ -281,13 +477,17 @@ public class FFmpegProfileHandlerTests
false,
false,
false,
qsvPreferNativeDecoder);
qsvPreferNativeDecoder,
readRate,
readRateCatchup);
private static UpdateFFmpegProfile MakeUpdate(
int id,
int resolutionId = 1,
bool qsvPreferNativeDecoder = true,
int? qsvExtraHardwareFrames = null) =>
int? qsvExtraHardwareFrames = null,
double? readRate = null,
double? readRateCatchup = null) =>
new(
id,
"Default",
@@ -320,5 +520,7 @@ public class FFmpegProfileHandlerTests
false,
false,
false,
qsvPreferNativeDecoder);
qsvPreferNativeDecoder,
readRate,
readRateCatchup);
}
@@ -0,0 +1,166 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Data.Repositories;
using ErsatzTV.Tests.Support;
using Microsoft.Extensions.Logging.Abstractions;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Application.Paging;
/// <summary>
/// The MediaCards count/page pairs live in two separate repository methods rather than in one
/// handler, so `api.paged-count-matches-page-query` cannot be satisfied structurally there — the
/// two must be kept in agreement and pinned by a test instead. Each case below constructs the
/// divergence the count used to miss and asserts count == pageable rows.
/// Expected values are pinned literals, never re-derived from the method's own predicate.
/// </summary>
[TestFixture]
public class MediaCardsCountMatchesPageTests
{
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private TelevisionRepository TelevisionRepo =>
new(_db.Factory, NullLogger<TelevisionRepository>.Instance);
[Test]
public async Task GetSeasonCount_Should_Expand_To_The_Same_Shows_GetPagedSeasons_Pages()
{
// the same show present in two libraries: same Title+Year, different Show rows.
// GetPagedSeasons pages the union (2 + 3), so the count must be 5, not 2.
await using (TvContext context = _db.CreateContext())
{
context.Shows.Add(new Show { Id = 1 });
context.Shows.Add(new Show { Id = 2 });
context.ShowMetadata.Add(new ShowMetadata { Id = 201, ShowId = 1, Title = "Star Trek", Year = 1966 });
context.ShowMetadata.Add(new ShowMetadata { Id = 202, ShowId = 2, Title = "Star Trek", Year = 1966 });
// an unrelated show that must NOT be swept in
context.Shows.Add(new Show { Id = 3 });
context.ShowMetadata.Add(new ShowMetadata { Id = 203, ShowId = 3, Title = "Star Trek", Year = 1987 });
context.Seasons.Add(new Season { Id = 19, ShowId = 3, SeasonNumber = 1 });
context.Seasons.Add(new Season { Id = 11, ShowId = 1, SeasonNumber = 1 });
context.Seasons.Add(new Season { Id = 12, ShowId = 1, SeasonNumber = 2 });
context.Seasons.Add(new Season { Id = 13, ShowId = 2, SeasonNumber = 1 });
context.Seasons.Add(new Season { Id = 14, ShowId = 2, SeasonNumber = 2 });
context.Seasons.Add(new Season { Id = 15, ShowId = 2, SeasonNumber = 3 });
await context.SaveChangesAsync();
}
int count = await TelevisionRepo.GetSeasonCount(1);
List<Season> page = await TelevisionRepo.GetPagedSeasons(1, 1, 50, CancellationToken.None);
count.ShouldBe(5);
page.Count.ShouldBe(5);
count.ShouldBe(page.Count);
// pin WHICH rows, not just how many — a count and a page can agree on the wrong set
page.Select(s => s.Id).OrderBy(id => id).ShouldBe([11, 12, 13, 14, 15]);
}
[Test]
public async Task GetSeasonCount_Should_Be_Zero_When_The_Show_Has_No_Metadata()
{
// GetPagedSeasons returns nothing without a ShowMetadata row to expand from, so the count
// must agree rather than reporting the show's seasons
await using (TvContext context = _db.CreateContext())
{
context.Shows.Add(new Show { Id = 1 });
context.Seasons.Add(new Season { Id = 11, ShowId = 1, SeasonNumber = 1 });
await context.SaveChangesAsync();
}
int count = await TelevisionRepo.GetSeasonCount(1);
List<Season> page = await TelevisionRepo.GetPagedSeasons(1, 1, 50, CancellationToken.None);
count.ShouldBe(0);
page.ShouldBeEmpty();
}
[Test]
public async Task GetEpisodeCount_Should_Count_Episodes_That_Have_Metadata()
{
// 3 episodes, one of which lost its metadata row to a scanner failure. GetPagedEpisodes
// pages EpisodeMetadata, so only 2 are reachable and the count must say 2.
await using (TvContext context = _db.CreateContext())
{
// GetPagedEpisodes's include chain reaches Episode -> Season -> Show through REQUIRED
// reference navs, which EF emits as INNER JOINs, so a missing Season or Show row drops
// every row and would make the page 0 for a reason unrelated to the count under test.
// The ShowMetadata leg is a COLLECTION nav (LEFT JOIN) and drops nothing — the row below
// is incidental, seeded only to keep the graph realistic.
context.Shows.Add(new Show { Id = 1 });
context.ShowMetadata.Add(new ShowMetadata { Id = 201, ShowId = 1, Title = "Show", Year = 2000 });
context.Seasons.Add(new Season { Id = 11, ShowId = 1, SeasonNumber = 1 });
for (var i = 21; i <= 23; i++)
{
context.Episodes.Add(new Episode { Id = i, SeasonId = 11 });
}
context.EpisodeMetadata.Add(new EpisodeMetadata { Id = 221, EpisodeId = 21, EpisodeNumber = 1 });
context.EpisodeMetadata.Add(new EpisodeMetadata { Id = 222, EpisodeId = 22, EpisodeNumber = 2 });
// an episode in a different season must not be swept in
context.Seasons.Add(new Season { Id = 12, ShowId = 1, SeasonNumber = 2 });
context.Episodes.Add(new Episode { Id = 29, SeasonId = 12 });
context.EpisodeMetadata.Add(new EpisodeMetadata { Id = 229, EpisodeId = 29, EpisodeNumber = 1 });
await context.SaveChangesAsync();
}
int count = await TelevisionRepo.GetEpisodeCount(11);
List<EpisodeMetadata> page = await TelevisionRepo.GetPagedEpisodes(11, 1, 50);
count.ShouldBe(2);
page.Count.ShouldBe(2);
count.ShouldBe(page.Count);
// the two episodes WITH metadata, and not the other season's
page.Select(em => em.EpisodeId).OrderBy(id => id).ShouldBe([21, 22]);
}
[Test]
public async Task GetMusicVideoCount_Should_Count_Music_Videos_That_Have_Metadata()
{
// 3 music videos for the artist, one without a metadata row; GetPagedMusicVideos pages
// MusicVideoMetadata, so the count must be 2
await using (TvContext context = _db.CreateContext())
{
context.Artists.Add(new Artist { Id = 41 });
for (var i = 31; i <= 33; i++)
{
context.MusicVideos.Add(new MusicVideo { Id = i, ArtistId = 41 });
}
context.MusicVideoMetadata.Add(new MusicVideoMetadata { Id = 231, MusicVideoId = 31, Title = "A" });
context.MusicVideoMetadata.Add(new MusicVideoMetadata { Id = 232, MusicVideoId = 32, Title = "B" });
// another artist's video must not be swept in
context.Artists.Add(new Artist { Id = 42 });
context.MusicVideos.Add(new MusicVideo { Id = 39, ArtistId = 42 });
context.MusicVideoMetadata.Add(new MusicVideoMetadata { Id = 239, MusicVideoId = 39, Title = "C" });
await context.SaveChangesAsync();
}
var repo = new MusicVideoRepository(_db.Factory);
int count = await repo.GetMusicVideoCount(41);
List<MusicVideoMetadata> page = await repo.GetPagedMusicVideos(41, 1, 50);
count.ShouldBe(2);
page.Count.ShouldBe(2);
count.ShouldBe(page.Count);
// this artist's two videos with metadata, and not the other artist's
page.Select(m => m.Title).OrderBy(x => x).ShouldBe(["A", "B"]);
}
}
@@ -0,0 +1,286 @@
using ErsatzTV.Application.MediaCollections;
using ErsatzTV.Application.Playouts;
using ErsatzTV.Application.ProgramSchedules;
using ErsatzTV.Core.Domain;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using NUnit.Framework;
using Shouldly;
using DomainChannel = ErsatzTV.Core.Domain.Channel;
namespace ErsatzTV.Tests.Application.Paging;
/// <summary>
/// Every paged handler whose page query applies a filter must compute its TotalCount from the SAME
/// query, or a filtered page reports the unfiltered total and the SPA paginates to pages that can
/// never contain anything (issues #690, #758).
/// Expected counts here are PINNED LITERALS derived from the seeded set by hand — never recomputed
/// by re-applying the handler's own predicate, which would pass whatever the handler happens to do.
/// </summary>
[TestFixture]
public class PagedQueryTotalCountTests
{
// 5 seeded rows, of which exactly these 2 contain "Alpha"
private const int SeededRows = 5;
private const int MatchingAlpha = 2;
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private static readonly string[] Names =
["Alpha One", "Beta", "Alpha Two", "Gamma", "Delta"];
[Test]
public async Task GetPagedCollections_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.Collections.Add(new Collection { Id = i + 1, Name = Names[i], MediaItems = [] });
}
await context.SaveChangesAsync();
}
var handler = new GetPagedCollectionsHandler(_db.Factory);
PagedMediaCollectionsViewModel unfiltered =
await handler.Handle(new GetPagedCollections(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedMediaCollectionsViewModel filtered =
await handler.Handle(new GetPagedCollections("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(c => c.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedMultiCollections_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.MultiCollections.Add(new MultiCollection { Id = i + 1, Name = Names[i] });
}
// channel-owned rows are excluded from BOTH the page and the count, filter or no filter
context.MultiCollections.Add(
new MultiCollection { Id = 99, Name = "Alpha Owned", OwnedByChannelId = 7 });
await context.SaveChangesAsync();
}
var handler = new GetPagedMultiCollectionsHandler(_db.Factory);
PagedMultiCollectionsViewModel unfiltered =
await handler.Handle(new GetPagedMultiCollections(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedMultiCollectionsViewModel filtered =
await handler.Handle(new GetPagedMultiCollections("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(mc => mc.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedSmartCollections_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.SmartCollections.Add(
new SmartCollection { Id = i + 1, Name = Names[i], Query = "tag:family" });
}
context.SmartCollections.Add(
new SmartCollection
{
Id = 99,
Name = "Alpha Owned",
Query = "tag:family",
OwnedByChannelId = 7
});
await context.SaveChangesAsync();
}
var handler = new GetPagedSmartCollectionsHandler(_db.Factory);
PagedSmartCollectionsViewModel unfiltered =
await handler.Handle(new GetPagedSmartCollections(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedSmartCollectionsViewModel filtered =
await handler.Handle(new GetPagedSmartCollections("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(sc => sc.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedRerunCollections_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.RerunCollections.Add(
new RerunCollection
{
Id = i + 1,
Name = Names[i],
CollectionType = CollectionType.Collection,
CollectionId = i + 1
});
context.Collections.Add(new Collection { Id = i + 1, Name = Names[i], MediaItems = [] });
}
await context.SaveChangesAsync();
}
var handler = new GetPagedRerunCollectionsHandler(_db.Factory);
PagedRerunCollectionsViewModel unfiltered =
await handler.Handle(new GetPagedRerunCollections(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedRerunCollectionsViewModel filtered =
await handler.Handle(new GetPagedRerunCollections("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(rc => rc.Name).ShouldBe(["Alpha One", "Alpha Two"]);
// the selection graph still loads for the page — moving IncludeSelectionDetails off the
// counted query must not stop the page from projecting it (issue #671)
filtered.Page.Select(rc => rc.Collection?.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedProgramSchedules_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.ProgramSchedules.Add(new ProgramSchedule { Id = i + 1, Name = Names[i] });
}
await context.SaveChangesAsync();
}
var handler = new GetPagedProgramSchedulesHandler(_db.Factory);
PagedProgramSchedulesViewModel unfiltered =
await handler.Handle(new GetPagedProgramSchedules(string.Empty, 0, 10), CancellationToken.None);
unfiltered.TotalCount.ShouldBe(SeededRows);
PagedProgramSchedulesViewModel filtered =
await handler.Handle(new GetPagedProgramSchedules("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(ps => ps.Name).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task GetPagedPlayouts_Filtered_Count_Should_Match_Filter()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.Channels.Add(NewChannel(i + 1, $"{i + 1}", Names[i]));
context.Playouts.Add(
new Playout
{
Id = i + 1,
ChannelId = i + 1,
ScheduleKind = PlayoutScheduleKind.Classic
});
}
// a playout whose channel row does not exist: excluded from the page by the
// `Channel != null` filter, so it must be excluded from the count too
context.Playouts.Add(
new Playout { Id = 99, ChannelId = 4242, ScheduleKind = PlayoutScheduleKind.Classic });
await context.SaveChangesAsync();
}
var handler = new GetPagedPlayoutsHandler(_db.Factory);
PagedPlayoutsViewModel unfiltered =
await handler.Handle(new GetPagedPlayouts(string.Empty, 0, 10), CancellationToken.None);
// 5, NOT 6 — the orphaned playout is filtered out of the page, so it is not part of the total
unfiltered.TotalCount.ShouldBe(SeededRows);
unfiltered.Page.Count.ShouldBe(SeededRows);
PagedPlayoutsViewModel filtered =
await handler.Handle(new GetPagedPlayouts("Alpha", 0, 10), CancellationToken.None);
filtered.TotalCount.ShouldBe(MatchingAlpha);
filtered.Page.Select(p => p.ChannelName).ShouldBe(["Alpha One", "Alpha Two"]);
}
[Test]
public async Task Filtered_Count_Should_Drive_A_Second_Page()
{
await using (TvContext context = _db.CreateContext())
{
for (var i = 0; i < Names.Length; i++)
{
context.Collections.Add(new Collection { Id = i + 1, Name = Names[i], MediaItems = [] });
}
await context.SaveChangesAsync();
}
var handler = new GetPagedCollectionsHandler(_db.Factory);
// pageSize 1 over the 2 matching rows: this is the property the issues are about — the count
// is what tells the client a SECOND page exists, and each page holds exactly its own row
PagedMediaCollectionsViewModel first =
await handler.Handle(new GetPagedCollections("Alpha", 0, 1), CancellationToken.None);
first.TotalCount.ShouldBe(MatchingAlpha);
first.Page.Select(c => c.Name).ShouldBe(["Alpha One"]);
PagedMediaCollectionsViewModel second =
await handler.Handle(new GetPagedCollections("Alpha", 1, 1), CancellationToken.None);
second.TotalCount.ShouldBe(MatchingAlpha);
second.Page.Select(c => c.Name).ShouldBe(["Alpha Two"]);
// and the page AFTER the last matching row is empty — with the pre-fix count of 5 the client
// would have been told to fetch three more pages that can never contain anything
PagedMediaCollectionsViewModel past =
await handler.Handle(new GetPagedCollections("Alpha", 2, 1), CancellationToken.None);
past.TotalCount.ShouldBe(MatchingAlpha);
past.Page.ShouldBeEmpty();
}
private static DomainChannel NewChannel(int id, string number, string name) =>
new(Guid.NewGuid())
{
Id = id,
Number = number,
SortNumber = id,
Name = name,
Group = "ErsatzTV",
Categories = string.Empty,
FFmpegProfileId = 1,
StreamSelector = string.Empty,
PreferredAudioLanguageCode = string.Empty,
PreferredAudioTitle = string.Empty,
PreferredSubtitleLanguageCode = string.Empty,
MusicVideoCreditsTemplate = string.Empty,
StreamingMode = StreamingMode.TransportStreamHybrid,
PlayoutSource = ChannelPlayoutSource.Generated,
PlayoutMode = ChannelPlayoutMode.Continuous
};
}
@@ -1,3 +1,5 @@
using System.Collections;
using System.Reflection;
using System.Threading.Channels;
using ErsatzTV.Application;
using ErsatzTV.Application.ProgramSchedules;
@@ -65,7 +67,8 @@ public class ScheduleItemResponseRoundTripTests
await replaceHandler.Handle(new ReplaceProgramScheduleItems(scheduleId, reconstructed), CancellationToken.None);
replaced.IsRight.ShouldBeTrue(replaced.LeftToSeq().HeadOrNone().Match(e => e.Value, () => "unknown"));
// GET again → envelope B; A and B must be semantically identical (ignoring regenerated row ids).
// GET again → envelope B; A and B must be semantically identical INCLUDING row ids
// the handler reconciles by id and updates in place, it does not regenerate rows.
ScheduleItemsResponseModel envelopeB = await GetItemsEnvelope(scheduleId);
envelopeB.Items.Count.ShouldBe(envelopeA.Items.Count);
@@ -340,62 +343,97 @@ public class ScheduleItemResponseRoundTripTests
r.PreferredSubtitleLanguageCode,
r.SubtitleMode);
// ersatztv#779 (detector G): the compared field list is DERIVED from the DTO by reflection,
// never hand-copied. The previous version was a hand-written run of `b.X.ShouldBe(a.X)` lines.
// It was COMPLETE on the day it was written — every property but Id — and had no way
// to report the day it stopped being: a field added to ScheduleItemResponseModel simply went
// uncompared, and this "lossless round-trip" test kept passing while the round trip silently
// dropped it. That is #754's mechanism exactly (a hand-maintained mirror drifting from a
// 28-property DTO by one field, HTTP 200, no error), one altitude up — in the very test whose
// job is to catch losses.
//
// Properties deliberately NOT compared. The set is EMPTY, and that is a finding rather than an
// oversight. The first version exempted Id on the reasoning that "the PUT replaces the item
// set, so B's rows are new rows with new ids". ReplaceProgramScheduleItemsHandler does not do
// that for this fixture's payload: it forwards every Id, takes the id-based reconcile, and
// updates rows in place. So Id compares equal, and the exemption was unnecessary.
//
// Two mutations of this fixture, both EXECUTED — recorded as results, with no account of why,
// because three earlier drafts of this comment each supplied a confident mechanism for a
// correct observation and two of them were contradicted by the code:
//
// ToReplaceCommand passes `null` for EVERY id -> test stays GREEN
// ToReplaceCommand passes `null` for index 0 only -> test goes RED, "Id differs"
//
// So the Id comparison does discriminate; it is not decorative. What it is NOT is a substitute
// for ReplaceProgramScheduleItemsReconcileTests, whose
// Reorder_ById_Should_Move_State_With_The_Logical_Item_Not_The_Slot and
// Insert_ById_In_Middle_Should_Keep_Existing_Ids_And_State pass real ids and pin that state
// moves with the logical item rather than the slot. Those are the #252 tests; this is a
// round-trip check that happens to also notice a lost row.
//
// Any name added here must still exist on ScheduleItemResponseModel (asserted below), so
// renaming a field cannot leave a stale exemption silently exempting nothing.
private static readonly System.Collections.Generic.HashSet<string> RoundTripExemptProperties =
new(StringComparer.Ordinal);
private static void AssertSemanticallyEqual(ScheduleItemResponseModel a, ScheduleItemResponseModel b)
{
b.Index.ShouldBe(a.Index);
b.StartType.ShouldBe(a.StartType);
b.StartTime.ShouldBe(a.StartTime);
b.FixedStartTimeBehavior.ShouldBe(a.FixedStartTimeBehavior);
b.PlayoutMode.ShouldBe(a.PlayoutMode);
b.CollectionType.ShouldBe(a.CollectionType);
b.CollectionId.ShouldBe(a.CollectionId);
b.MultiCollectionId.ShouldBe(a.MultiCollectionId);
b.SmartCollectionId.ShouldBe(a.SmartCollectionId);
b.RerunCollectionId.ShouldBe(a.RerunCollectionId);
b.MediaItemId.ShouldBe(a.MediaItemId);
b.PlaylistId.ShouldBe(a.PlaylistId);
b.SearchTitle.ShouldBe(a.SearchTitle);
b.SearchQuery.ShouldBe(a.SearchQuery);
b.PlaybackOrder.ShouldBe(a.PlaybackOrder);
b.MarathonGroupBy.ShouldBe(a.MarathonGroupBy);
b.MarathonShuffleGroups.ShouldBe(a.MarathonShuffleGroups);
b.MarathonShuffleItems.ShouldBe(a.MarathonShuffleItems);
b.MarathonBatchSize.ShouldBe(a.MarathonBatchSize);
b.FillWithGroupMode.ShouldBe(a.FillWithGroupMode);
b.MultipleMode.ShouldBe(a.MultipleMode);
b.MultipleCount.ShouldBe(a.MultipleCount);
b.PlayoutDuration.ShouldBe(a.PlayoutDuration);
b.TailMode.ShouldBe(a.TailMode);
b.DiscardToFillAttempts.ShouldBe(a.DiscardToFillAttempts);
b.CustomTitle.ShouldBe(a.CustomTitle);
b.GuideMode.ShouldBe(a.GuideMode);
b.PreRollFillerId.ShouldBe(a.PreRollFillerId);
b.MidRollFillerId.ShouldBe(a.MidRollFillerId);
b.PostRollFillerId.ShouldBe(a.PostRollFillerId);
b.TailFillerId.ShouldBe(a.TailFillerId);
b.FallbackFillerId.ShouldBe(a.FallbackFillerId);
b.WatermarkIds.ShouldBe(a.WatermarkIds);
b.GraphicsElementIds.ShouldBe(a.GraphicsElementIds);
b.PreferredAudioLanguageCode.ShouldBe(a.PreferredAudioLanguageCode);
b.PreferredAudioTitle.ShouldBe(a.PreferredAudioTitle);
b.PreferredSubtitleLanguageCode.ShouldBe(a.PreferredSubtitleLanguageCode);
b.SubtitleMode.ShouldBe(a.SubtitleMode);
b.CollectionName.ShouldBe(a.CollectionName);
b.MultiCollectionName.ShouldBe(a.MultiCollectionName);
b.SmartCollectionName.ShouldBe(a.SmartCollectionName);
b.RerunCollectionName.ShouldBe(a.RerunCollectionName);
b.PlaylistName.ShouldBe(a.PlaylistName);
b.PlaylistGroupId.ShouldBe(a.PlaylistGroupId);
b.MediaItemName.ShouldBe(a.MediaItemName);
b.PreRollFillerName.ShouldBe(a.PreRollFillerName);
b.MidRollFillerName.ShouldBe(a.MidRollFillerName);
b.PostRollFillerName.ShouldBe(a.PostRollFillerName);
b.TailFillerName.ShouldBe(a.TailFillerName);
b.FallbackFillerName.ShouldBe(a.FallbackFillerName);
b.Watermarks.Select(w => (w.Id, w.Name)).ShouldBe(a.Watermarks.Select(w => (w.Id, w.Name)));
b.GraphicsElements.Select(g => (g.Id, g.Name)).ShouldBe(a.GraphicsElements.Select(g => (g.Id, g.Name)));
b.Name.ShouldBe(a.Name);
b.DurationEstimate.ShouldBe(a.DurationEstimate);
PropertyInfo[] properties = typeof(ScheduleItemResponseModel)
.GetProperties(BindingFlags.Public | BindingFlags.Instance);
// A stale exemption is a silent hole: it would exempt nothing while reading as a reviewed
// decision, and the property it once named would be compared or not by accident.
foreach (string exempt in RoundTripExemptProperties)
{
properties.Any(p => p.Name == exempt).ShouldBeTrue(
$"'{exempt}' is exempted from the round-trip comparison but is not a property of "
+ $"{nameof(ScheduleItemResponseModel)}; remove the stale exemption or fix the name.");
}
var compared = 0;
foreach (PropertyInfo property in properties)
{
if (RoundTripExemptProperties.Contains(property.Name))
{
continue;
}
object? expected = property.GetValue(a);
object? actual = property.GetValue(b);
if (expected is IEnumerable expectedSequence and not string)
{
// Collection-valued members (WatermarkIds, Watermarks, GraphicsElementIds,
// GraphicsElements). The elementwise walk still delegates to each element's Equals,
// so it is value equality only because those elements are records
// (NamedIdResponseModel) or value types (the int id lists); a future element type that
// is neither would silently be compared by REFERENCE inside this loop. It is also order-sensitive, which is
// correct for these ordered lists but would be wrong for an unordered type such as
// a dictionary-valued property.
actual.ShouldNotBeNull($"{property.Name} was null on the round-tripped item");
var actualSequence = (IEnumerable)actual;
actualSequence.Cast<object?>().ToList()
.ShouldBe(expectedSequence.Cast<object?>().ToList(), $"{property.Name} differs");
}
else
{
actual.ShouldBe(expected, $"{property.Name} differs");
}
compared++;
}
// Anti-vacuity, as a PIN rather than a floor. A `>=` floor lets properties vanish silently,
// which is the one-sided version of the both-directions rule this test is meant to embody.
// Comparing against the reflected count minus exemptions would be tautological — both sides
// come from the same reflection — so the expected number is written down and must be
// bumped deliberately in the same change that adds or removes a DTO field.
const int expectedComparedProperties = 55;
compared.ShouldBe(
expectedComparedProperties,
$"{compared} properties were compared, expected {expectedComparedProperties}; update "
+ "this pin in the same change that alters ScheduleItemResponseModel's field list");
}
private async Task<int> SeedScheduleAndReferences(bool shuffleScheduleItems)
@@ -0,0 +1,151 @@
using ErsatzTV.Application.Playouts;
using ErsatzTV.Application.Scheduling;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Core.Scheduling;
using NUnit.Framework;
using Shouldly;
using PlayoutsMapper = ErsatzTV.Application.Playouts.Mapper;
using SchedulingMapper = ErsatzTV.Application.Scheduling.Mapper;
namespace ErsatzTV.Tests.Application.Scheduling;
/// <summary>
/// ersatztv#823. Guarding <see cref="AlternateScheduleSelector" /> alone would have left the OTHER read
/// of the same six columns unguarded — the entity→view-model mappers, which feed
/// <c>PlayoutController</c>'s response models and therefore the SPA.
/// <para>
/// Two things break without the guard, and neither is a C# exception, which is why the selector
/// tests cannot see them. <c>web/src/screens/PlayoutScheduleEditors.tsx</c> spreads the collection
/// (<c>daysOfMonth: [...template.daysOfMonth]</c>) and throws <c>TypeError: not iterable</c> on a
/// JSON <c>null</c>; and <c>web/src/screens/playoutTemplateCalendar.ts</c>'s <c>appliesToDate</c> —
/// an exact TypeScript port of <see cref="AlternateScheduleSelector.GetScheduleForDate{T}" /> —
/// calls <c>.includes</c> on it.
/// </para>
/// <para>
/// So the mappers substitute the SAME unrestricted defaults the selector reads. That agreement is
/// the point: a DTO that said "empty" while the selector scheduled "unrestricted" would make the
/// preview calendar disagree with the playout it is previewing.
/// </para>
/// </summary>
[TestFixture]
public class RecurrenceLimitsMapperNullTests
{
private static Template MinimalTemplate() =>
new()
{
Id = 7,
Name = "T",
TemplateGroupId = 1,
TemplateGroup = new TemplateGroup { Name = "G" },
Items = []
};
[Test]
public void ProgramScheduleAlternate_Null_Collections_Map_To_Unrestricted()
{
var alternate = new ProgramScheduleAlternate
{
Id = 1,
Index = 0,
ProgramScheduleId = 2,
DaysOfWeek = null!,
DaysOfMonth = null!,
MonthsOfYear = null!
};
PlayoutAlternateScheduleViewModel vm = PlayoutsMapper.ProjectToViewModel(alternate);
vm.DaysOfWeek.ShouldBe(AlternateScheduleSelector.AllDaysOfWeek());
vm.DaysOfMonth.ShouldBe(AlternateScheduleSelector.AllDaysOfMonth());
vm.MonthsOfYear.ShouldBe(AlternateScheduleSelector.AllMonthsOfYear());
// Never assigned back: these are single-column primitive collections, so writing the guard onto a
// tracked entity would persist the substituted set over the NULL
// (media.nullable-primitive-collection-mutation).
alternate.DaysOfWeek.ShouldBeNull();
alternate.DaysOfMonth.ShouldBeNull();
alternate.MonthsOfYear.ShouldBeNull();
}
[Test]
public void PlayoutTemplate_Null_Collections_Map_To_Unrestricted()
{
var template = new PlayoutTemplate
{
Id = 1,
Index = 0,
Template = MinimalTemplate(),
DecoTemplate = null,
DaysOfWeek = null!,
DaysOfMonth = null!,
MonthsOfYear = null!
};
PlayoutTemplateViewModel vm = SchedulingMapper.ProjectToViewModel(template);
vm.DaysOfWeek.ShouldBe(AlternateScheduleSelector.AllDaysOfWeek());
vm.DaysOfMonth.ShouldBe(AlternateScheduleSelector.AllDaysOfMonth());
vm.MonthsOfYear.ShouldBe(AlternateScheduleSelector.AllMonthsOfYear());
template.DaysOfWeek.ShouldBeNull();
template.DaysOfMonth.ShouldBeNull();
template.MonthsOfYear.ShouldBeNull();
}
/// <summary>
/// An explicitly EMPTY collection is a recorded restriction of no days and must survive the mapper
/// unchanged. Without this, a guard written as "empty or null becomes All*" would pass the two tests
/// above while silently rewriting real user data on the way out.
/// <para>
/// There is one of these per MAPPER, not one in total. The two overloads are byte-identical
/// triples in different files, so a defensive edit to one alone is exactly the "one helper, two
/// callers" shape this repo has been bitten by: covering only the Playouts mapper would leave
/// the PlayoutTemplate one free to acquire an `empty-or-null` guard with the suite still green.
/// </para>
/// </summary>
[Test]
public void An_Explicitly_Empty_Collection_Is_Not_Rewritten_By_The_PlayoutTemplate_Mapper()
{
var template = new PlayoutTemplate
{
Id = 1,
Index = 0,
Template = MinimalTemplate(),
DecoTemplate = null,
DaysOfWeek = [],
DaysOfMonth = [],
MonthsOfYear = []
};
PlayoutTemplateViewModel vm = SchedulingMapper.ProjectToViewModel(template);
vm.DaysOfWeek.ShouldBeEmpty();
vm.DaysOfMonth.ShouldBeEmpty();
vm.MonthsOfYear.ShouldBeEmpty();
}
/// <summary>
/// The <c>ProgramScheduleAlternate</c> half of the same pair — see the PlayoutTemplate one above
/// for why there is one per MAPPER rather than one in total.
/// </summary>
[Test]
public void An_Explicitly_Empty_Collection_Is_Not_Rewritten()
{
var alternate = new ProgramScheduleAlternate
{
Id = 1,
Index = 0,
ProgramScheduleId = 2,
DaysOfWeek = [],
DaysOfMonth = [],
MonthsOfYear = []
};
PlayoutAlternateScheduleViewModel vm = PlayoutsMapper.ProjectToViewModel(alternate);
vm.DaysOfWeek.ShouldBeEmpty();
vm.DaysOfMonth.ShouldBeEmpty();
vm.MonthsOfYear.ShouldBeEmpty();
}
}
@@ -217,7 +217,9 @@ public class FFmpegProfileControllerTests
false,
true,
false,
true);
true,
null,
null);
private static CreateFFmpegProfileRequest MakeCreateRequest() =>
new(
@@ -0,0 +1,674 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Graphics;
using ErsatzTV.Core.Interfaces.Streaming;
using ErsatzTV.FFmpeg;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using NUnit.Framework;
using Shouldly;
using LanguageExt;
using SkiaSharp;
namespace ErsatzTV.Tests.Infrastructure.Graphics;
/// <summary>
/// Pixel-level cover for the #732 background box. These assert the RENDERED BITMAP rather than the
/// parsed model, because every failure mode this feature has is a silent no-op: the YAML parses, the
/// element initializes, and nothing is drawn. Geometry assertions are all RELATIVE to a no-box
/// baseline so they do not depend on which typeface the host resolves.
/// </summary>
[TestFixture]
public class TextElementBackgroundBoxTests
{
private const int FrameWidth = 1920;
// Derived from the documented rule, not a hardcoded 1080p result: hardcoding it makes correct
// production code fail the moment the test frame size changes.
private static int FrameInsetCap => Math.Min(FrameWidth, FrameHeight) / 2;
private const int FrameHeight = 1080;
private static TextGraphicsElement BaseElement() =>
new()
{
Name = "test",
Location = WatermarkLocation.BottomLeft,
BaseStyle = "body",
Styles =
[
new StyleDefinition
{
Name = "body",
// Required: a null font_family makes CustomFontMapper throw on a null dictionary
// key, which TextElement swallows into "disable for this content" (the #570 trap).
// The family need not resolve -- an unknown one falls back to Skia's default.
FontFamily = "Roboto",
FontSize = 40,
TextColor = "#FFFFFF"
}
],
Text = "Hello"
};
// TextElement swallows every initialization failure into a logged warning, so a broken render
// would otherwise surface as a null bitmap with no explanation. Capture the warning and rethrow.
private sealed class ThrowingLogger : ILogger
{
public IDisposable BeginScope<TState>(TState state) where TState : notnull => null!;
public bool IsEnabled(LogLevel logLevel) => true;
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception exception,
Func<TState, Exception, string> formatter)
{
if (logLevel >= LogLevel.Warning && exception is not null)
{
throw new InvalidOperationException(formatter(state, exception), exception);
}
}
}
private static SKBitmap Render(TextGraphicsElement element)
{
var fonts = new GraphicsEngineFonts(new CustomFontMapper(NullLogger<CustomFontMapper>.Instance));
var textElement = new TextElement(fonts, element, new ThrowingLogger());
var context = new GraphicsEngineContext(
"1",
null,
[],
new Dictionary<string, object>(),
new Resolution { Width = FrameWidth, Height = FrameHeight },
new Resolution { Width = FrameWidth, Height = FrameHeight },
new FrameRate("30"),
DateTimeOffset.UnixEpoch,
DateTimeOffset.UnixEpoch,
TimeSpan.Zero,
TimeSpan.FromMinutes(1),
TimeSpan.FromMinutes(1));
textElement.InitializeAsync(context, CancellationToken.None).GetAwaiter().GetResult();
Option<PreparedElementImage> maybeImage = textElement
.PrepareImage(TimeSpan.Zero, TimeSpan.Zero, TimeSpan.FromMinutes(1), TimeSpan.Zero, CancellationToken.None)
.AsTask().GetAwaiter().GetResult();
PreparedElementImage prepared = maybeImage.IfNone(() => throw new InvalidOperationException(
"the element produced no image; initialization failed"));
return prepared.Image;
}
// The whole suite is meaningless if the host cannot lay out any text at all, so prove the
// baseline is non-degenerate rather than letting a 0x0 bitmap pass every relative assertion.
[Test]
public void Baseline_Renders_A_Non_Empty_Bitmap()
{
SKBitmap baseline = Render(BaseElement());
baseline.Width.ShouldBeGreaterThan(0);
baseline.Height.ShouldBeGreaterThan(0);
}
[Test]
public void No_Background_Fields_Leaves_Corner_Transparent()
{
SKBitmap baseline = Render(BaseElement());
baseline.GetPixel(0, 0).Alpha.ShouldBe((byte)0);
baseline.GetPixel(baseline.Width - 1, baseline.Height - 1).Alpha.ShouldBe((byte)0);
}
[Test]
public void Background_Color_Fills_The_Box()
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
SKBitmap rendered = Render(element);
SKColor corner = rendered.GetPixel(0, 0);
corner.Alpha.ShouldBe((byte)255);
corner.Red.ShouldBe((byte)255);
corner.Green.ShouldBe((byte)0);
corner.Blue.ShouldBe((byte)0);
}
[Test]
public void Background_Padding_Grows_The_Bitmap_On_Every_Side()
{
const int Padding = 12;
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#000000";
element.BackgroundPadding = Padding;
SKBitmap padded = Render(element);
padded.Width.ShouldBe(baseline.Width + (2 * Padding));
padded.Height.ShouldBe(baseline.Height + (2 * Padding));
}
// Growing the bitmap is not the same as moving the text into it. If the text were still painted
// at (0,0) the geometry assertions above would all still hold while the glyphs sat on the box
// edge, so pin the padding band itself as pure background.
[Test]
public void Background_Padding_Actually_Insets_The_Text()
{
const int Padding = 16;
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = Padding;
SKBitmap rendered = Render(element);
// A full-width band inside the top padding must contain nothing but the fill colour.
for (var x = 0; x < rendered.Width; x++)
{
SKColor pixel = rendered.GetPixel(x, Padding / 2);
pixel.Red.ShouldBe((byte)255, $"pixel at x={x} in the padding band is not the fill colour");
pixel.Green.ShouldBe((byte)0, $"pixel at x={x} in the padding band is not the fill colour");
pixel.Blue.ShouldBe((byte)0, $"pixel at x={x} in the padding band is not the fill colour");
}
}
[Test]
public void Background_Opacity_Percent_Scales_The_Alpha()
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundOpacityPercent = 50;
SKBitmap rendered = Render(element);
// 255 * 0.5, rounded. Skia stores premultiplied alpha, so assert the alpha channel only.
rendered.GetPixel(0, 0).Alpha.ShouldBe((byte)128);
}
[Test]
public void Corner_Radius_Rounds_The_Corner_Away()
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = 20;
element.BackgroundCornerRadius = 20;
SKBitmap rendered = Render(element);
// The extreme corner falls outside a 20px radius, the middle of the left edge does not.
rendered.GetPixel(0, 0).Alpha.ShouldBe((byte)0);
rendered.GetPixel(0, rendered.Height / 2).Alpha.ShouldBe((byte)255);
}
[Test]
public void Border_Color_Draws_A_Border_Distinct_From_The_Fill()
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = 20;
element.BorderColor = "#00FF00";
element.BorderWidth = 4;
SKBitmap rendered = Render(element);
// On the border ring...
SKColor edge = rendered.GetPixel(1, rendered.Height / 2);
edge.Green.ShouldBeGreaterThan((byte)200);
edge.Red.ShouldBeLessThan((byte)100);
// ...and inside it, still the fill.
SKColor inside = rendered.GetPixel(10, rendered.Height / 2);
inside.Red.ShouldBe((byte)255);
inside.Green.ShouldBe((byte)0);
}
[Test]
public void Border_Width_Is_Included_In_The_Bitmap_So_The_Border_Is_Not_Clipped()
{
const int BorderWidth = 6;
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BorderColor = "#00FF00";
element.BorderWidth = BorderWidth;
SKBitmap bordered = Render(element);
bordered.Width.ShouldBe(baseline.Width + (2 * BorderWidth));
bordered.Height.ShouldBe(baseline.Height + (2 * BorderWidth));
}
[Test]
public void Unparseable_Background_Color_Draws_No_Box_Rather_Than_Substituting_One()
{
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "not-a-color";
element.BackgroundPadding = 25;
SKBitmap rendered = Render(element);
// No box means no padding either: the geometry is the untouched baseline.
rendered.Width.ShouldBe(baseline.Width);
rendered.Height.ShouldBe(baseline.Height);
rendered.GetPixel(0, 0).Alpha.ShouldBe((byte)0);
}
[Test]
public void Border_Color_Without_An_Explicit_Width_Draws_A_Hairline()
{
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BorderColor = "#00FF00";
SKBitmap rendered = Render(element);
rendered.Width.ShouldBe(baseline.Width + 2);
rendered.GetPixel(0, rendered.Height / 2).Green.ShouldBeGreaterThan((byte)200);
}
// A box with no padding or border must not move anything: this is the seam where "adding a
// background" could silently change an existing overlay's geometry.
[Test]
public void A_Fill_With_No_Padding_Or_Border_Leaves_The_Geometry_Untouched()
{
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
SKBitmap filled = Render(element);
filled.Width.ShouldBe(baseline.Width);
filled.Height.ShouldBe(baseline.Height);
}
// The inset is rounded up to a whole pixel and the SAME integer is used on both sides. Subtracting
// the unrounded value from the wrap budget while adding its ceiling to the bitmap overflows
// width_percent by a rounding remainder, which an integer padding cannot expose.
[Test]
public void Fractional_Padding_Still_Respects_Width_Percent()
{
const double WidthPercent = 20;
foreach (double padding in new[] { 12.3, 14.0, 29.7, 0.5 })
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog and keeps running past the edge";
element.Fit = TextFit.Wrap;
element.WidthPercent = WidthPercent;
element.BackgroundColor = "#000000";
element.BackgroundPadding = padding;
SKBitmap rendered = Render(element);
var budget = (int)Math.Round(WidthPercent / 100.0 * FrameWidth);
rendered.Width.ShouldBeLessThanOrEqualTo(budget, $"padding {padding} overflowed the budget");
}
}
// An inset wider than the budget cannot be honoured AND stay inside it, so the INSET is clamped
// rather than the text being squeezed to nothing. Asserting a width bound here would be wrong:
// wrapping cannot break below one glyph, so a narrow width_percent overflows with or without a
// box (pre-existing). What the clamp guarantees is that the box's own contribution stops growing
// at the budget -- so a runaway padding renders identically to the largest one that fits.
[Test]
public void An_Oversized_Padding_Is_Clamped_To_The_Largest_That_Fits()
{
const double WidthPercent = 20;
var budget = (int)Math.Round(WidthPercent / 100.0 * FrameWidth);
int maxInset = (budget - 1) / 2;
static TextGraphicsElement WithPadding(double padding)
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Wrap;
element.WidthPercent = WidthPercent;
element.BackgroundColor = "#000000";
element.BackgroundPadding = padding;
return element;
}
SKBitmap runaway = Render(WithPadding(400));
SKBitmap clamped = Render(WithPadding(maxInset));
runaway.Width.ShouldBe(clamped.Width);
runaway.Height.ShouldBe(clamped.Height);
// and the clamp actually bit -- an unclamped 400px padding would add 800px of box
runaway.Width.ShouldBeLessThan(budget + (2 * maxInset));
}
// FitTextBlock rebuilds every style from scratch. It used to drop the halo, which only became
// reachable once a box's insets could push a previously-fitting element into the Scale path --
// adding a background would then silently remove the halo behind the text.
[Test]
public void The_Scale_Path_Preserves_The_Halo()
{
static TextGraphicsElement Scaled(bool withHalo)
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Scale;
element.WidthPercent = 15;
element.BackgroundColor = "#000000";
element.BackgroundPadding = 12;
element.Styles[0].HaloColor = withHalo ? "#00FF00" : null;
element.Styles[0].HaloWidth = withHalo ? 3 : null;
return element;
}
SKBitmap withHalo = Render(Scaled(true));
SKBitmap withoutHalo = Render(Scaled(false));
withHalo.Width.ShouldBe(withoutHalo.Width);
withHalo.Height.ShouldBe(withoutHalo.Height);
var differing = 0;
for (var x = 0; x < withHalo.Width; x++)
{
for (var y = 0; y < withHalo.Height; y++)
{
if (withHalo.GetPixel(x, y) != withoutHalo.GetPixel(x, y))
{
differing++;
}
}
}
// If the Scale path dropped the halo, the two renders would be pixel-identical.
differing.ShouldBeGreaterThan(0, "the halo made no difference through the Scale path");
}
// width_percent bounds the ELEMENT. If the insets were not subtracted from the wrap width the
// box would overflow the budget by 2*inset, which is exactly the bug this pins.
[Test]
public void Width_Percent_Bounds_The_Whole_Box_Including_Padding()
{
const double WidthPercent = 20;
const int Padding = 30;
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog and keeps on running well past the edge";
element.Fit = TextFit.Wrap;
element.WidthPercent = WidthPercent;
element.BackgroundColor = "#000000";
element.BackgroundPadding = Padding;
SKBitmap rendered = Render(element);
var budget = (int)Math.Round(WidthPercent / 100.0 * FrameWidth);
rendered.Width.ShouldBeLessThanOrEqualTo(budget);
}
// YAML yields doubles, so 1e100 and NaN are reachable from a config file. Cast to float they
// become Infinity/NaN, and (int)Math.Ceiling of those is an unspecified value that sails past
// every clamp and can wrap the doubled inset back to zero.
[Test]
public void Non_Finite_And_Absurd_Box_Values_Do_Not_Corrupt_The_Geometry()
{
SKBitmap baseline = Render(BaseElement());
foreach (double bad in new[] { 1e100, double.NaN, double.PositiveInfinity, -5.0 })
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = bad;
SKBitmap rendered = Render(element);
rendered.Width.ShouldBeGreaterThan(0, $"padding {bad} produced a degenerate width");
rendered.Height.ShouldBeGreaterThan(0, $"padding {bad} produced a degenerate height");
rendered.Width.ShouldBeGreaterThanOrEqualTo(baseline.Width, $"padding {bad} shrank the element");
rendered.Width.ShouldBeLessThanOrEqualTo(baseline.Width + (2 * FrameInsetCap), $"padding {bad} was not clamped");
}
}
[Test]
public void A_Non_Finite_Width_Percent_Is_Treated_As_No_Budget()
{
SKBitmap baseline = Render(BaseElement());
// This pins the OUTCOME -- a non-finite budget behaves like no budget -- not the
// float.IsFinite guard, which no mutation can distinguish because .NET saturates float-to-int
// conversion. See the "Not covered by any mutation" note in docs/graphics-elements.md.
const int Padding = 10;
TextGraphicsElement element = BaseElement();
element.Fit = TextFit.Wrap;
element.WidthPercent = 1e300;
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = Padding;
SKBitmap rendered = Render(element);
rendered.Width.ShouldBe(baseline.Width + (2 * Padding));
rendered.Height.ShouldBe(baseline.Height + (2 * Padding));
}
// Clamping insetPixels alone leaves DrawBackgroundBox stroking at the ORIGINAL border width,
// centred on a rect that no longer has room for it -- the stroke then floods the whole element
// and paints over the interior. The box's own fields must be clamped too.
[Test]
public void An_Oversized_Border_Is_Clamped_And_Does_Not_Flood_The_Element()
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Wrap;
element.WidthPercent = 20;
element.BackgroundColor = "#0000FF";
element.BorderColor = "#FF0000";
element.BorderWidth = 400;
SKBitmap rendered = Render(element);
// Somewhere inside the element there must still be fill, not solid border.
var fillPixels = 0;
for (var x = 0; x < rendered.Width; x++)
{
for (var y = 0; y < rendered.Height; y++)
{
SKColor px = rendered.GetPixel(x, y);
if (px.Blue > 200 && px.Red < 100)
{
fillPixels++;
}
}
}
fillPixels.ShouldBeGreaterThan(0, "the border flooded the element; no fill survived");
}
// Sanitize bounds each field on its own, but padding and border ADD UP, and with no
// width_percent nothing else bounded them -- two fields could allocate a bitmap far larger than
// the frame it is drawn onto. This pins the frame cap itself; the non-finite test above cannot,
// because its bound is looser than the growth this clause prevents.
[Test]
public void The_Inset_Is_Capped_Against_The_Frame_Even_With_No_Width_Percent()
{
SKBitmap baseline = Render(BaseElement());
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundPadding = 4000;
element.BorderColor = "#00FF00";
element.BorderWidth = 4000;
SKBitmap rendered = Render(element);
// 8000px of requested inset collapses to the frame cap, not to 16000px of bitmap growth.
rendered.Width.ShouldBe(baseline.Width + (2 * FrameInsetCap));
rendered.Height.ShouldBe(baseline.Height + (2 * FrameInsetCap));
}
// halo_* is documented as a per-style field. The merge loop used to rebuild each style from the
// base and then override only font/colour, so a non-base style's halo silently inherited the
// base one -- invisible in the seeded template, whose three styles declare identical halos.
[Test]
public void A_Non_Base_Style_Uses_Its_Own_Halo_Not_The_Base_Styles()
{
static TextGraphicsElement WithSubHalo(float haloWidth)
{
TextGraphicsElement element = BaseElement();
element.Styles.Add(new StyleDefinition
{
Name = "sub",
FontFamily = "Roboto",
FontSize = 40,
TextColor = "#FFFFFF",
HaloColor = "#00FF00",
HaloWidth = haloWidth
});
element.Text = "[sub]Hello[/sub]";
return element;
}
SKBitmap thin = Render(WithSubHalo(1));
SKBitmap thick = Render(WithSubHalo(6));
static int HaloPixels(SKBitmap b)
{
var n = 0;
for (var x = 0; x < b.Width; x++)
{
for (var y = 0; y < b.Height; y++)
{
SKColor px = b.GetPixel(x, y);
if (px.Green > 150 && px.Red < 120)
{
n++;
}
}
}
return n;
}
// If the style's own halo were ignored, both renders would use the base style's (none) and
// neither would contain halo pixels.
HaloPixels(thin).ShouldBeGreaterThan(0, "the style's own halo was not applied");
HaloPixels(thick).ShouldBeGreaterThan(HaloPixels(thin), "halo_width had no effect per style");
}
private static int PixelsMatching(SKBitmap b, Func<SKColor, bool> predicate)
{
var n = 0;
for (var x = 0; x < b.Width; x++)
{
for (var y = 0; y < b.Height; y++)
{
if (predicate(b.GetPixel(x, y)))
{
n++;
}
}
}
return n;
}
// halo_blur is a per-style field too, and the merge loop drops it just as silently as halo_color
// did. Blur spreads the halo over more pixels at lower alpha, so a blurred halo covers more area.
[Test]
public void A_Non_Base_Style_Uses_Its_Own_Halo_Blur()
{
static TextGraphicsElement WithSubBlur(float blur)
{
TextGraphicsElement element = BaseElement();
element.Styles.Add(new StyleDefinition
{
Name = "sub",
FontFamily = "Roboto",
FontSize = 40,
TextColor = "#FFFFFF",
HaloColor = "#00FF00",
HaloWidth = 3,
HaloBlur = blur
});
element.Text = "[sub]Hello[/sub]";
return element;
}
SKBitmap sharp = Render(WithSubBlur(0));
SKBitmap blurred = Render(WithSubBlur(5));
static bool Greenish(SKColor px) => px.Green > 60 && px.Red < 140 && px.Alpha > 0;
PixelsMatching(blurred, Greenish)
.ShouldNotBe(PixelsMatching(sharp, Greenish), "halo_blur had no effect per style");
}
// FitTextBlock rebuilds every style from scratch on the Scale path. line_height and halo_blur are
// carried across there; without them a scaled element silently loses line spacing and halo blur.
[Test]
public void The_Scale_Path_Preserves_Line_Height()
{
static TextGraphicsElement Scaled(float lineHeight)
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Scale;
element.WidthPercent = 15;
element.Styles[0].LineHeight = lineHeight;
return element;
}
SKBitmap tight = Render(Scaled(1.0f));
SKBitmap loose = Render(Scaled(2.5f));
loose.Height.ShouldBeGreaterThan(tight.Height, "line_height was dropped by the Scale path");
}
[Test]
public void The_Scale_Path_Preserves_Halo_Blur()
{
static TextGraphicsElement Scaled(float blur)
{
TextGraphicsElement element = BaseElement();
element.Text = "The quick brown fox jumps over the lazy dog";
element.Fit = TextFit.Scale;
element.WidthPercent = 15;
element.Styles[0].HaloColor = "#00FF00";
element.Styles[0].HaloWidth = 3;
element.Styles[0].HaloBlur = blur;
return element;
}
SKBitmap sharp = Render(Scaled(0));
SKBitmap blurred = Render(Scaled(5));
static bool Greenish(SKColor px) => px.Green > 60 && px.Red < 140 && px.Alpha > 0;
PixelsMatching(blurred, Greenish)
.ShouldNotBe(PixelsMatching(sharp, Greenish), "halo_blur was dropped by the Scale path");
}
// The documented range is 0-100. The sole opacity test used 50, so Math.Clamp could be removed
// with everything green while an out-of-range value wrapped to an unrelated alpha.
[Test]
public void Background_Opacity_Percent_Is_Clamped_To_Its_Documented_Range()
{
static SKColor CornerAt(int percent)
{
TextGraphicsElement element = BaseElement();
element.BackgroundColor = "#FF0000";
element.BackgroundOpacityPercent = percent;
return Render(element).GetPixel(0, 0);
}
CornerAt(-50).Alpha.ShouldBe((byte)0, "a negative opacity did not clamp to 0");
CornerAt(400).Alpha.ShouldBe((byte)255, "an opacity over 100 did not clamp to 100");
}
}
@@ -0,0 +1,288 @@
using ErsatzTV.Core.Domain;
using GraphicsElement = ErsatzTV.Core.Domain.GraphicsElement;
using ErsatzTV.Core.Graphics;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using System.IO.Abstractions;
using Microsoft.Extensions.Logging.Abstractions;
using Testably.Abstractions.Testing;
using ErsatzTV.Core;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Infrastructure;
/// <summary>
/// #732 part 2: the On Now / Next overlay is a default rather than an opt-in. Channels that predate
/// that decision are backfilled once -- and only once, so a channel an operator deliberately clears
/// is never silently re-attached on the next restart.
/// </summary>
[TestFixture]
public class GraphicsElementDefaultAttachTests
{
private InMemoryTvContext _db = null!;
[SetUp]
public async Task SetUp() => _db = await InMemoryTvContext.CreateAsync();
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private static async Task<int> SeedBuiltInElement(TvContext context)
{
var element = new GraphicsElement
{
Path = $"/templates/text/{GraphicsElementDefaults.OnNowNextFileName}",
Name = "On Now / Next",
Kind = GraphicsElementKind.Text
};
await context.GraphicsElements.AddAsync(element);
await context.SaveChangesAsync();
return element.Id;
}
private static async Task<Channel> SeedChannel(
TvContext context,
string number,
StreamingMode mode = StreamingMode.HttpLiveStreamingSegmenter)
{
var channel = new Channel(Guid.NewGuid())
{
Name = $"Channel {number}",
Number = number,
StreamingMode = mode,
ChannelGraphicsElements = []
};
await context.Channels.AddAsync(channel);
await context.SaveChangesAsync();
return channel;
}
private static async Task<List<int>> AttachedElementIds(TvContext context, int channelId) =>
await context.Set<ChannelGraphicsElement>()
.AsNoTracking()
.Where(cge => cge.ChannelId == channelId)
.Select(cge => cge.GraphicsElementId)
.ToListAsync();
[Test]
public async Task Attaches_The_Built_In_Element_To_Existing_Channels()
{
await using TvContext context = _db.CreateContext();
int elementId = await SeedBuiltInElement(context);
Channel one = await SeedChannel(context, "1");
Channel two = await SeedChannel(context, "2");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, one.Id)).ShouldBe([elementId]);
(await AttachedElementIds(context, two.Id)).ShouldBe([elementId]);
}
[Test]
public async Task Skips_Hls_Direct_Channels_Where_The_Overlay_Cannot_Render()
{
await using TvContext context = _db.CreateContext();
await SeedBuiltInElement(context);
Channel direct = await SeedChannel(context, "1", StreamingMode.HttpLiveStreamingDirect);
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, direct.Id)).ShouldBeEmpty();
}
[Test]
public async Task Does_Not_Duplicate_An_Existing_Attachment()
{
await using TvContext context = _db.CreateContext();
int elementId = await SeedBuiltInElement(context);
Channel channel = await SeedChannel(context, "1");
await context.AddAsync(
new ChannelGraphicsElement { ChannelId = channel.Id, GraphicsElementId = elementId });
await context.SaveChangesAsync();
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).Count.ShouldBe(1);
}
// The load-bearing property: a default must not fight the operator.
[Test]
public async Task Does_Not_Re_Attach_After_An_Operator_Clears_It()
{
await using TvContext context = _db.CreateContext();
await SeedBuiltInElement(context);
Channel channel = await SeedChannel(context, "1");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).Count.ShouldBe(1);
// operator turns the overlay off for this channel
context.Set<ChannelGraphicsElement>()
.RemoveRange(context.Set<ChannelGraphicsElement>().Where(cge => cge.ChannelId == channel.Id));
await context.SaveChangesAsync();
// ...and the app restarts
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).ShouldBeEmpty();
}
// The upgrade population this backfill exists for -- an install seeding the template for the
// first time on this boot -- must not be stranded. The GraphicsElement row is normally created
// by RefreshGraphicsElements, which runs long after startup, so the seeder ensures it itself.
// Without that, the marker would be written against an unresolved element and every pre-existing
// channel would go permanently unattached.
[Test]
public async Task Backfills_On_The_Same_Boot_That_First_Seeds_The_Template()
{
var fs = new MockFileSystem();
await using TvContext context = _db.CreateContext();
Channel channel = await SeedChannel(context, "1");
// no ConfigElement markers and no GraphicsElement row: a pre-#74 install meeting #732
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).Count.ShouldBe(1);
}
// Filename alone is ambiguous: the five template folders are separate namespaces, so an element
// of another kind may legitimately carry the same filename.
[Test]
public async Task Ignores_A_Same_Named_Element_Of_A_Different_Kind()
{
await using TvContext context = _db.CreateContext();
await context.GraphicsElements.AddAsync(
new GraphicsElement
{
Path = $"/templates/image/{GraphicsElementDefaults.OnNowNextFileName}",
Kind = GraphicsElementKind.Image
});
await context.SaveChangesAsync();
Channel channel = await SeedChannel(context, "1");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).ShouldBeEmpty();
}
[Test]
public async Task Ignores_A_Non_Built_In_Element_With_A_Different_Filename()
{
await using TvContext context = _db.CreateContext();
await context.GraphicsElements.AddAsync(
new GraphicsElement { Path = "/templates/text/something-else.yml", Kind = GraphicsElementKind.Text });
await context.SaveChangesAsync();
Channel channel = await SeedChannel(context, "1");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).ShouldBeEmpty();
}
// The marker is permanent, so writing it with nothing resolved would strand every channel. Stay
// armed instead and pick the work up once the element exists.
[Test]
public async Task Stays_Armed_When_There_Is_No_Built_In_Element_To_Attach()
{
await using TvContext context = _db.CreateContext();
Channel channel = await SeedChannel(context, "1");
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await context.ConfigElements
.AnyAsync(c => c.Key == ConfigElementKey.GraphicsOnNowNextDefaultAttached.Key))
.ShouldBeFalse("the marker was written with nothing to attach");
// the element turns up later; the backfill must still do its job
await SeedBuiltInElement(context);
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await AttachedElementIds(context, channel.Id)).Count.ShouldBe(1);
}
// The already-seeded branch of SeedOnNowNext has its own EnsureBuiltInElementRow call. Removing
// it reddened nothing until this test existed -- the other tests all exercise a FRESH seed.
[Test]
public async Task An_Already_Seeded_Install_Missing_Its_Element_Row_Gets_One()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
string target = Path.Combine(
FileSystemLayout.GraphicsElementsTextTemplatesFolder,
GraphicsElementDefaults.OnNowNextFileName);
await fs.File.WriteAllTextAsync(target, "name: On Now / Next\n");
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
(await context.GraphicsElements.CountAsync()).ShouldBe(0);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
// Assert the ROW, not just that one exists: a lookup-only assertion is satisfied by a row
// with a null Name, which sorts the built-in element into the unnamed bucket in the SPA.
List<GraphicsElement> rows = await context.GraphicsElements.ToListAsync();
rows.Count.ShouldBe(1);
rows[0].Path.ShouldBe(target);
rows[0].Kind.ShouldBe(GraphicsElementKind.Text);
rows[0].Name.ShouldBe(GraphicsElementDefaults.OnNowNextName);
(await GraphicsElementSeeder.GetBuiltInElementId(context, CancellationToken.None)).IsSome.ShouldBeTrue();
}
// The armed path is a real, reachable state: an operator deletes the template, refresh reaps the
// row, and the backfill then has nothing to resolve. Pin what happens when the element comes
// back -- a single global marker cannot both avoid stranding and avoid re-adding, and this is
// the half we accept. See graphics.on-now-next-on-by-default.
[Test]
public async Task While_Armed_A_Restored_Element_Is_Attached_To_Every_Eligible_Channel()
{
await using TvContext context = _db.CreateContext();
Channel kept = await SeedChannel(context, "1");
Channel cleared = await SeedChannel(context, "2");
// nothing to resolve yet -> stays armed, no marker
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
(await context.ConfigElements
.AnyAsync(c => c.Key == ConfigElementKey.GraphicsOnNowNextDefaultAttached.Key))
.ShouldBeFalse();
// The element reappears and is attached to BOTH channels; the operator then clears `cleared`.
// Doing the attach-then-remove for real matters: seeding `cleared` with no join at all would
// only prove an untouched channel gets backfilled, which is not the claim.
int elementId = await SeedBuiltInElement(context);
await context.AddAsync(new ChannelGraphicsElement { ChannelId = kept.Id, GraphicsElementId = elementId });
await context.AddAsync(new ChannelGraphicsElement { ChannelId = cleared.Id, GraphicsElementId = elementId });
await context.SaveChangesAsync();
(await AttachedElementIds(context, cleared.Id)).Count.ShouldBe(1);
context.Set<ChannelGraphicsElement>()
.RemoveRange(context.Set<ChannelGraphicsElement>().Where(cge => cge.ChannelId == cleared.Id));
await context.SaveChangesAsync();
(await AttachedElementIds(context, cleared.Id)).ShouldBeEmpty();
await GraphicsElementSeeder.AttachOnNowNextByDefault(context, CancellationToken.None);
// documented consequence: the still-armed backfill cannot see that deliberate clear
(await AttachedElementIds(context, kept.Id)).Count.ShouldBe(1);
(await AttachedElementIds(context, cleared.Id)).Count.ShouldBe(1);
// ...but it is now marked, so it never fires again
(await context.ConfigElements
.AnyAsync(c => c.Key == ConfigElementKey.GraphicsOnNowNextDefaultAttached.Key))
.ShouldBeTrue();
}
}
@@ -6,6 +6,7 @@ using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using NUnit.Framework;
using Shouldly;
using YamlDotNet.Serialization;
@@ -36,7 +37,7 @@ public class GraphicsElementSeederTests
var fs = new MockFileSystem();
await using TvContext context = _db.CreateContext();
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
fs.File.Exists(_seededPath).ShouldBeTrue();
fs.File.ReadAllText(_seededPath).ShouldContain("epg_entries: 2");
@@ -51,7 +52,7 @@ public class GraphicsElementSeederTests
await fs.File.WriteAllTextAsync(_seededPath, "name: Operator Custom\nepg_entries: 2\n");
await using TvContext context = _db.CreateContext();
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
fs.File.ReadAllText(_seededPath).ShouldContain("Operator Custom");
}
@@ -62,9 +63,9 @@ public class GraphicsElementSeederTests
var fs = new MockFileSystem();
await using TvContext context = _db.CreateContext();
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
fs.File.Delete(_seededPath);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
fs.File.Exists(_seededPath).ShouldBeFalse();
}
@@ -77,7 +78,7 @@ public class GraphicsElementSeederTests
{
var fs = new MockFileSystem();
await using TvContext context = _db.CreateContext();
await GraphicsElementSeeder.SeedOnNowNext(context, fs, CancellationToken.None);
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
string yaml = await fs.File.ReadAllTextAsync(_seededPath);
IDeserializer deserializer = new DeserializerBuilder()
@@ -0,0 +1,365 @@
using System.IO.Abstractions;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Graphics;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using ErsatzTV.Tests.Support;
using Microsoft.Extensions.Logging.Abstractions;
using Microsoft.EntityFrameworkCore;
using NUnit.Framework;
using Shouldly;
using Testably.Abstractions.Testing;
using Testably.Abstractions.Testing.FileSystem;
using YamlDotNet.Serialization;
using YamlDotNet.Serialization.NamingConventions;
namespace ErsatzTV.Tests.Infrastructure;
/// <summary>
/// #732: the seeder writes the On Now / Next template once and never revisits it, so a change to the
/// shipped default would reach new databases only. These pin the upgrade path that fixes that, and
/// the boundary that keeps it from clobbering an operator's edits.
/// </summary>
[TestFixture]
public class GraphicsElementSeederUpgradeTests
{
// Byte-for-byte the default shipped before #732. Verified 2026-08-26 against the live prod
// install at 192.168.1.29 (md5 ef9afc088cf6dba252f725babbf3334f), so this is a real
// fingerprint rather than a copy of the constant it is meant to detect.
private const string OnNowNextYamlV1 =
"""
name: On Now / Next
epg_entries: 2
location: BottomLeft
horizontal_margin_percent: 4
vertical_margin_percent: 8
width_percent: 42
text_fit: Wrap
text_align: Left
z_index: 100
# transparent until 4s in, fade in 1s, hold 6s, fade out 1s
opacity_expression: "LinearFadeDuration(content_seconds, 4, 1, 6)"
base_style: now
styles:
- name: now
font_family: "Noto Sans"
font_size: 30
font_weight: 700
text_color: "#FFFFFF"
halo_color: "#000000"
halo_width: 2
- name: sub
font_family: "Noto Sans"
font_size: 22
font_weight: 400
text_color: "#DDDDDD"
halo_color: "#000000"
halo_width: 2
- name: next
font_family: "Noto Sans"
font_size: 22
font_weight: 400
text_color: "#DDDDDD"
halo_color: "#000000"
halo_width: 2
text: |
[now]NOW {{ Epg[0].Title }}[/now]
{{ if Epg[0].SubTitle }}[sub]{{ Epg[0].SubTitle }}[/sub]{{ end }}
{{ if (array.size Epg) > 1 }}[next]NEXT {{ Epg[1].Title }}[/next]{{ end }}
""";
private InMemoryTvContext _db = null!;
private string _target = null!;
[SetUp]
public async Task SetUp()
{
_db = await InMemoryTvContext.CreateAsync();
_target = Path.Combine(
FileSystemLayout.GraphicsElementsTextTemplatesFolder,
GraphicsElementDefaults.OnNowNextFileName);
}
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
private async Task<MockFileSystem> RunSeederOverAlreadySeededDatabase(string existingContent)
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
if (existingContent is not null)
{
await fs.File.WriteAllTextAsync(_target, existingContent);
}
await using TvContext context = _db.CreateContext();
// The fixture DB is shared across calls within a test, so only seed the marker once.
string key = ConfigElementKey.GraphicsOnNowNextSeeded.Key;
if (!context.ConfigElements.Any(c => c.Key == key))
{
context.ConfigElements.Add(new ConfigElement { Key = key, Value = "true" });
await context.SaveChangesAsync();
}
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
return fs;
}
[Test]
public async Task Upgrades_An_Untouched_Previous_Default()
{
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(OnNowNextYamlV1);
string result = await fs.File.ReadAllTextAsync(_target);
result.ShouldNotBe(OnNowNextYamlV1);
result.ShouldContain("background_color");
result.ShouldContain("background_padding");
}
[Test]
public async Task Upgrades_An_Untouched_Previous_Default_With_Windows_Line_Endings()
{
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(
OnNowNextYamlV1.Replace("\n", "\r\n"));
string result = await fs.File.ReadAllTextAsync(_target);
result.ShouldContain("background_color");
}
[Test]
public async Task Leaves_An_Operator_Modified_File_Alone()
{
// One changed value is enough to stop matching the fingerprint.
string edited = OnNowNextYamlV1.Replace("width_percent: 42", "width_percent: 30");
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(edited);
string result = await fs.File.ReadAllTextAsync(_target);
result.ShouldBe(edited);
result.ShouldNotContain("background_color");
}
[Test]
public async Task Leaves_The_Current_Default_Alone_So_The_Upgrade_Is_Idempotent()
{
MockFileSystem first = await RunSeederOverAlreadySeededDatabase(OnNowNextYamlV1);
string upgraded = await first.File.ReadAllTextAsync(_target);
MockFileSystem second = await RunSeederOverAlreadySeededDatabase(upgraded);
string again = await second.File.ReadAllTextAsync(_target);
again.ShouldBe(upgraded);
}
// The upgrade runs inside DatabaseMigratorService, ahead of DatabaseIsReady(). Before #732 the
// already-seeded branch touched the filesystem not at all, so a template the app cannot read --
// e.g. edited as root via `docker exec` while the app runs as PUID/PGID -- used to boot fine.
// It must not become a failure to start.
[Test]
public async Task An_Unwritable_Template_Does_Not_Fail_Startup()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
// the file matches a shipped default, so the upgrade WILL try to rewrite it -- and that write
// is denied, standing in for a root-owned or read-only template
var intercepted = 0;
fs.Intercept.Changing(
FileSystemTypes.File,
_ =>
{
intercepted++;
throw new UnauthorizedAccessException("simulated permission denial");
});
await using TvContext context = _db.CreateContext();
string key = ConfigElementKey.GraphicsOnNowNextSeeded.Key;
context.ConfigElements.Add(new ConfigElement { Key = key, Value = "true" });
await context.SaveChangesAsync();
await Should.NotThrowAsync(
() => GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None));
// Without this the test would pass just as happily if the upgrade never reached the write.
intercepted.ShouldBeGreaterThan(0, "the write interceptor never fired");
// and the original template survives the denied write
(await fs.File.ReadAllTextAsync(_target)).ShouldBe(OnNowNextYamlV1);
}
// The write path is not the only one that can fault. A template the app cannot READ used to be
// harmless on an already-seeded install; it must stay that way. An exclusive lock produces a
// genuine ReadAllTextAsync failure rather than an intercepted write dressed up as one.
[Test]
public async Task A_Read_Failure_On_The_Template_Does_Not_Fail_Startup()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
await using Stream exclusive = fs.File.Open(_target, FileMode.Open, FileAccess.Read, FileShare.None);
// prove the lock actually denies a read, so the test cannot pass by never hitting one
Should.Throw<IOException>(() => fs.File.ReadAllText(_target));
await Should.NotThrowAsync(
() => GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None));
}
[Test]
public async Task Does_Not_Create_The_File_When_It_Is_Absent()
{
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(null);
fs.File.Exists(_target).ShouldBeFalse();
}
[Test]
public async Task The_Upgraded_Template_Still_Deserializes_With_A_Resolvable_Base_Style()
{
MockFileSystem fs = await RunSeederOverAlreadySeededDatabase(OnNowNextYamlV1);
string yaml = await fs.File.ReadAllTextAsync(_target);
IDeserializer deserializer = new DeserializerBuilder()
.WithNamingConvention(CamelCaseNamingConvention.Instance)
.Build();
var element = deserializer.Deserialize<TextGraphicsElement>(yaml);
element.ShouldNotBeNull();
element.BackgroundColor.ShouldBe("#000000");
element.BackgroundOpacityPercent.ShouldBe(65);
element.BackgroundPadding.ShouldBe(14);
element.BackgroundCornerRadius.ShouldBe(8);
// The border is what makes the box visible over dark content; without it the translucent
// black fill is indistinguishable from the frame behind it.
element.BorderColor.ShouldBe("#59FFFFFF");
element.BorderWidth.ShouldBe(1);
// #570: every style needs a font_family, and base_style must resolve.
element.Styles.ShouldNotBeEmpty();
element.Styles.ShouldAllBe(s => s.FontFamily != null);
element.Styles.ShouldContain(s => s.Name == element.BaseStyle);
}
// Without the duplicate guard in EnsureBuiltInElementRow the already-seeded branch inserts a
// fresh row on EVERY boot: RefreshGraphicsElements will neither reap them (the file exists) nor
// dedupe them, so the row set grows without bound. Idempotence of the FILE is not idempotence
// of the ROW, and the existing idempotence test only looks at the file.
[Test]
public async Task Repeated_Seeding_Does_Not_Accumulate_Element_Rows()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
for (var i = 0; i < 3; i++)
{
await GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None);
}
(await context.GraphicsElements.ToListAsync()).Count.ShouldBe(1);
}
// A failed write must not leave a truncated template behind: it would match no fingerprint, so
// the upgrade could never repair it, and the loader rejects malformed YAML outright.
[Test]
public async Task A_Failed_Write_Leaves_The_Original_Template_Intact()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
// Capture WHICH path the write targets. Asserting only "the original survived" cannot tell
// an atomic write from an in-place one here: Testably raises the interception BEFORE it
// truncates, so a plain WriteAllTextAsync(target) would leave the file intact too -- on a
// real filesystem it would not. The path is what actually distinguishes them.
var writtenPaths = new List<string>();
fs.Intercept.Changing(
FileSystemTypes.File,
c =>
{
writtenPaths.Add(c.Path);
throw new IOException("simulated disk full");
});
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
await Should.NotThrowAsync(
() => GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None));
writtenPaths.ShouldNotBeEmpty("the write interceptor never fired");
writtenPaths.ShouldAllBe(path => path.EndsWith(".upgrade.tmp"), "the upgrade wrote the live template in place instead of a temp file");
(await fs.File.ReadAllTextAsync(_target)).ShouldBe(OnNowNextYamlV1);
fs.Directory.GetFiles(FileSystemLayout.GraphicsElementsTextTemplatesFolder, "*.upgrade.tmp")
.ShouldBeEmpty("a temp file was left behind");
}
// The sibling test faults on the FIRST write, so it never reaches File.Move or the cleanup. Fault
// the replace instead, after a complete temp write: that is the path where a non-atomic
// implementation would already have truncated the live template.
[Test]
public async Task A_Failed_Replace_After_A_Complete_Temp_Write_Leaves_The_Original_Intact()
{
var fs = new MockFileSystem();
fs.Directory.CreateDirectory(FileSystemLayout.GraphicsElementsTextTemplatesFolder);
await fs.File.WriteAllTextAsync(_target, OnNowNextYamlV1);
var seenPaths = new List<string>();
fs.Intercept.Event(
c =>
{
seenPaths.Add($"{c.ChangeType}:{c.Path}");
// let the temp file be written in full; fail only when the live template is touched
if (c.Path == _target)
{
throw new IOException("simulated replace failure");
}
},
_ => true);
await using TvContext context = _db.CreateContext();
context.ConfigElements.Add(
new ConfigElement { Key = ConfigElementKey.GraphicsOnNowNextSeeded.Key, Value = "true" });
await context.SaveChangesAsync();
await Should.NotThrowAsync(
() => GraphicsElementSeeder.SeedOnNowNext(context, fs, NullLogger.Instance, CancellationToken.None));
seenPaths.ShouldContain(path => path.EndsWith(".upgrade.tmp"), "no temp file was ever written");
// Assert the replace is a RENAME, not merely "the target was touched after the temp was".
// A File.Copy(temp, target, true) also touches both in that order and would leave the
// original intact under this mock (interception runs before the change), so path ordering
// alone cannot tell an atomic replace from a truncating one -- the change TYPE can.
seenPaths.ShouldContain($"Renamed:{_target}", "the replace was not an atomic rename");
// The whole point of write-then-move: the live template is untouched by a failed replace.
(await fs.File.ReadAllTextAsync(_target)).ShouldBe(OnNowNextYamlV1);
// and nothing this call created is left behind
fs.Directory.GetFiles(FileSystemLayout.GraphicsElementsTextTemplatesFolder, "*.upgrade.tmp")
.ShouldBeEmpty("a temp file was left behind");
}
}
@@ -0,0 +1,299 @@
using System.Reflection;
using System.Text;
using Elastic.Clients.Elasticsearch;
using Elastic.Transport;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Search;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
using ErsatzTV.Infrastructure.Search;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
/// <summary>
/// ersatztv#824 — the gap ersatztv#701 named rather than papered over.
/// <para>
/// <c>ElasticSearchIndex.UpdateSong</c> holds an INDEPENDENT copy of the logic
/// <see cref="SongIndexerMetadataMutationTests" /> pins on <c>LuceneSearchIndex</c>. #701 removed
/// <c>metadata.AlbumArtists ??= []; metadata.Artists ??= [];</c> from both, but only Lucene gained
/// a regression test — so reintroducing the mutation in the Elastic copy ALONE left the whole
/// suite green. This fixture closes that: the assertions are the same three, driven through the
/// real <c>ElasticSearchIndex</c> against a real <see cref="TvContext" />.
/// </para>
/// <para>
/// <b>Why a stubbed transport rather than a live server.</b> #824 listed "inject a non-network
/// <c>ElasticsearchClient</c> transport" as option 1 and it is what shipped:
/// <c>Elastic.Transport.InMemoryRequestInvoker</c> is public in the pinned Elastic.Transport, and
/// <c>ElasticsearchClientSettings(NodePool, IRequestInvoker)</c> accepts it. No server, no socket,
/// no new package.
/// </para>
/// <para>
/// <b>The canned response body is load-bearing, not decoration.</b> A bare
/// <c>InMemoryRequestInvoker()</c> answers with an EMPTY body, which the client cannot deserialize
/// into an <c>IndexResponse</c>. That throw lands in <c>UpdateSong</c>'s catch, which logs a
/// warning and assigns <c>metadata.Song = null</c> — so the fixture would measure the ERROR path
/// while every "did not mutate" assertion below still passed, vacuously. The
/// <see cref="ThrowOnWarningLogger{T}" /> is the belt to that brace: it fails the test if the
/// catch ran at all.
/// </para>
/// <para>
/// The client is injected into the private <c>_client</c> field rather than obtained normally,
/// because <c>CreateClient</c> reads the process-wide static <c>ElasticSearchIndex.Uri</c> and
/// would open a real socket. <c>UpdateItems</c> — unlike <c>IndexExists</c> and
/// <c>Initialize</c> — never runs <c>_client ??= CreateClient()</c>, so the injected instance is
/// the one used and an uninjected one would simply be null.
/// </para>
/// </summary>
[TestFixture]
[NonParallelizable]
public class ElasticSongIndexerMetadataMutationTests
{
private const string TestIndexName = "etv-824-test";
private string? _originalIndexName;
/// <summary>
/// <c>ElasticSearchIndex.IndexName</c> is a process-wide static. Only <c>Startup</c> reads it today,
/// so leaving it set leaks nothing that currently runs — but a static this fixture writes and never
/// restores is a cross-test hazard waiting for the first test that does read it.
/// </summary>
[SetUp]
public void SetUp() => _originalIndexName = ElasticSearchIndex.IndexName;
[TearDown]
public void TearDown() => ElasticSearchIndex.IndexName = _originalIndexName;
/// <summary>
/// A well-formed <c>IndexResponse</c>. See the fixture docstring: an empty body diverts the run
/// into <c>UpdateSong</c>'s catch and makes every assertion below vacuous.
/// </summary>
private const string IndexResponseBody =
"""
{"_index":"etv-824-test","_id":"1","_version":1,"result":"created",
"_shards":{"total":1,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
""";
[Test]
public async Task UpdateSong_Must_Not_Mutate_Nullable_Artists_On_A_Tracked_Entity()
{
await using var harness = await InMemoryTvContext.CreateAsync();
int metadataId;
int songId;
await using (TvContext context = harness.CreateContext())
{
var library = new LocalLibrary { Name = "Music", MediaKind = LibraryMediaKind.Songs };
context.Add(library);
await context.SaveChangesAsync();
var libraryPath = new LibraryPath { Path = "/music", LibraryId = library.Id };
context.Add(libraryPath);
await context.SaveChangesAsync();
var song = new Song
{
LibraryPathId = libraryPath.Id,
MediaVersions = [],
SongMetadata =
[
new SongMetadata
{
MetadataKind = MetadataKind.Fallback,
Title = "Untagged Track",
SortTitle = "untagged track",
DateAdded = new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc),
// The shape FallbackMetadataProvider.GetSongMetadata leaves behind: it never
// assigns either primitive collection, so both columns persist as NULL.
Artists = null!,
AlbumArtists = null!,
Genres = [],
Tags = [],
Studios = [],
Actors = [],
Artwork = [],
Guids = []
}
]
};
context.Add(song);
await context.SaveChangesAsync();
metadataId = song.SongMetadata[0].Id;
songId = song.Id;
}
// The seed must actually have produced NULL columns, or every assertion below is vacuous.
(await ReadRawArtists(harness, metadataId)).ShouldBeNull();
await using (TvContext context = harness.CreateContext())
{
// Deliberately TRACKED -- the indexer's own contract is what is being pinned, not the
// AsNoTracking() habit of today's two callers. See SongIndexerMetadataMutationTests.
Song tracked = await context.Songs
.IncludeForSearch()
.AsSplitQuery()
.SingleAsync();
SongMetadata metadata = tracked.SongMetadata[0];
metadata.Artists.ShouldBeNull("EF must materialize the NULL column as null, not as an empty list");
var logger = new ThrowOnWarningLogger<ElasticSearchIndex>();
var index = new ElasticSearchIndex(
new SearchQueryParser(
Substitute.For<ISmartCollectionCache>(),
Substitute.For<ILogger<SearchQueryParser>>()),
logger);
var invoker = new CapturingRequestInvoker(
new InMemoryRequestInvoker(
Encoding.UTF8.GetBytes(IndexResponseBody),
200,
exception: null,
contentType: "application/json",
// The X-Elastic-Product header is REQUIRED, not cosmetic. The client runs a product
// check on its first response and throws UnsupportedProductException ("the server is
// not a supported distribution of Elasticsearch") without it -- which lands in
// UpdateSong's catch and makes the fixture measure the error path. Measured: this is
// exactly how this fixture first failed.
headers: ProductCheckHeaders()));
var settings = new ElasticsearchClientSettings(
new SingleNodePool(new Uri("http://localhost:9200")),
invoker)
.DefaultIndex(TestIndexName);
ElasticSearchIndex.IndexName = TestIndexName;
typeof(ElasticSearchIndex)
.GetField("_client", BindingFlags.NonPublic | BindingFlags.Instance)!
.SetValue(index, new ElasticsearchClient(settings));
// A bare substitute returns null from GetAllLanguageCodes, which NPEs inside AddLanguages and
// would divert the run into UpdateSong's catch.
var languageCodeService = Substitute.For<ILanguageCodeService>();
languageCodeService.GetAllLanguageCodes(Arg.Any<List<string>>()).Returns([]);
languageCodeService.GetAllLanguageCodes(Arg.Any<string>()).Returns([]);
await index.UpdateItems(
Substitute.For<ISearchRepository>(),
Substitute.For<IFallbackMetadataProvider>(),
languageCodeService,
[tracked]);
// Surfacing the exception rather than asserting ShouldBeNull: the catch is the fixture's
// most likely failure mode (see the canned-response note above), and "expected null but was
// <Exception>" without the message sends the next reader hunting for a cause the fixture
// already had in its hand.
if (logger.Failure is not null)
{
Assert.Fail("UpdateSong threw and its catch ran, so this probe measured the error path "
+ $"rather than the indexing path: {logger.Failure}");
}
// POSITIVE CONTROL, and it is not optional: every assertion below says something did NOT
// happen, so all of them hold vacuously if UpdateSong never ran. The Lucene fixture uses
// `writer.NumDocs == 1` for exactly this; the transport-level equivalent is that the indexer
// actually issued the index request for THIS song. Like NumDocs, it proves the song-indexing
// path ran -- it does NOT prove the artist reads specifically ran.
invoker.Requests.Count.ShouldBe(
1,
"UpdateSong did not issue exactly one index request, so the assertions below would pass "
+ $"without exercising the code under test. Captured: [{string.Join(", ", invoker.Requests)}]");
// The document id is compared as the LAST PATH SEGMENT, not with ShouldContain. A substring
// test is a false-pass vector here: the index name itself carries digits ("etv-824-test"), so
// ShouldContain("2") or ShouldContain("4") would be satisfied by the index name alone for a
// song whose id happened to be 2 or 4, and the assertion would stop discriminating without
// ever failing.
string path = invoker.Requests[0].Split(' ')[^1].Split('?')[0];
path.Split('/')[^1].ShouldBe(
songId.ToString(),
$"the index request was not for the seeded song. Captured: {invoker.Requests[0]}");
// 1. The indexer left the entity alone.
metadata.Artists.ShouldBeNull();
metadata.AlbumArtists.ShouldBeNull();
// 2. ...so EF has nothing to persist. This is the assertion that fails loudly the day the
// mutation returns, even if a later refactor stopped the value from being observable above.
context.Entry(metadata).State.ShouldBe(EntityState.Unchanged);
// 3. And the save that a real caller would go on to make does not rewrite the column.
await context.SaveChangesAsync();
}
(await ReadRawArtists(harness, metadataId)).ShouldBeNull();
}
private static Dictionary<string, IEnumerable<string>> ProductCheckHeaders() =>
new(StringComparer.OrdinalIgnoreCase) { ["x-elastic-product"] = ["Elasticsearch"] };
/// <summary>
/// Records every request the client actually issues, so the fixture can prove the code under test
/// ran. Delegates the answering to a real <see cref="InMemoryRequestInvoker" /> rather than
/// hand-building a response.
/// </summary>
private sealed class CapturingRequestInvoker(InMemoryRequestInvoker inner) : IRequestInvoker
{
public List<string> Requests { get; } = [];
public ResponseFactory ResponseFactory => inner.ResponseFactory;
public TResponse Request<TResponse>(
Endpoint endpoint,
BoundConfiguration boundConfiguration,
PostData? postData)
where TResponse : TransportResponse, new()
{
Requests.Add($"{endpoint.Method} {endpoint.PathAndQuery}");
return inner.Request<TResponse>(endpoint, boundConfiguration, postData);
}
public Task<TResponse> RequestAsync<TResponse>(
Endpoint endpoint,
BoundConfiguration boundConfiguration,
PostData? postData,
CancellationToken cancellationToken)
where TResponse : TransportResponse, new()
{
Requests.Add($"{endpoint.Method} {endpoint.PathAndQuery}");
return inner.RequestAsync<TResponse>(endpoint, boundConfiguration, postData, cancellationToken);
}
// IRequestInvoker extends IDisposable, but InMemoryRequestInvoker holds no disposable state and
// exposes no Dispose of its own -- there is nothing to forward to.
public void Dispose()
{
}
}
private static async Task<object?> ReadRawArtists(InMemoryTvContext harness, int metadataId)
{
await using TvContext context = harness.CreateContext();
await using var command = context.Database.GetDbConnection().CreateCommand();
command.CommandText = $"SELECT Artists FROM SongMetadata WHERE Id = {metadataId}";
object? value = await command.ExecuteScalarAsync();
// ExecuteScalar returns CLR null both for "the column is NULL" and for "there is no such row",
// and the second is reachable: UpdateSong's catch assigns metadata.Song = null, which severs a
// required relationship and cascades the row to Deleted, so a SaveChanges on the error path
// DELETES it and a plain null check would pass for the wrong reason.
if (value is null)
{
Assert.Fail($"SongMetadata row {metadataId} no longer exists, so its Artists column cannot "
+ "be read -- the probe measured a deleted row rather than a preserved NULL.");
}
return value is DBNull ? null : value;
}
}
@@ -0,0 +1,388 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Scheduling;
using ErsatzTV.Infrastructure;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.MySql.Data;
using ErsatzTV.Infrastructure.Sqlite.Data;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using MySqlConnector;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
/// <summary>
/// ersatztv#823 — the question ersatztv#701 split out rather than answered: can a runtime null reach
/// one of the SIX collection-valued scalar columns (<c>DaysOfMonth</c>, <c>MonthsOfYear</c>,
/// <c>DaysOfWeek</c> on <see cref="ProgramScheduleAlternate" /> and <see cref="PlayoutTemplate" />)?
/// <para>
/// <b>Measured, not reasoned about</b>, because the reasoning available beforehand pointed the
/// wrong way. The two converters differ on their read side —
/// <c>IntCollectionValueConverter</c> maps null-or-blank to <c>Array.Empty&lt;int&gt;()</c> while
/// <c>EnumCollectionJsonValueConverter</c> would dereference the result of
/// <c>JsonConvert.DeserializeObject</c> — so the expectation was that the two behave differently
/// on a NULL row. They do not: <b>EF does not invoke a value converter for a NULL column at
/// all</b>, so all six materialize as CLR <c>null</c> and the int converter's null-to-empty
/// branch is dead on this path. That is the measurement this fixture pins.
/// </para>
/// <para>
/// It also pins the WRITE half: assigning <c>null</c> to one of the six and calling
/// <c>SaveChanges</c> SUCCEEDS and stores SQL NULL — the converter is skipped on the way out too.
/// State that precisely, because the overclaim is tempting: this is a property of the CODE, not a
/// live caller. NO caller supplies a null today — every production construction of
/// <c>ReplacePlayoutAlternateSchedule</c> / <c>ReplacePlayoutTemplate</c> goes through the HTTP
/// request records, which normalize with <c>?? []</c>. What makes it a latent gun is that
/// <c>ReplacePlayoutAlternateScheduleItemsHandler</c> / <c>ReplacePlayoutTemplateItemsHandler</c>
/// assign the command value straight onto the entity, so nothing in the write path itself refuses.
/// </para>
/// <para>
/// The LEGACY route is narrower than "the columns are nullable", and conflating the two is the easy
/// error: all six are <c>nullable: true</c> on both providers, but five of the six were present at
/// <c>CreateTable</c>, so a NULL there still needs code to write one. EXACTLY ONE case is
/// code-path-free — SQLite's <c>20240113140741_Add_PlayoutTemplate_DaysOfMonth</c> is an
/// <c>AddColumn</c> with <c>nullable: true</c> and NO <c>defaultValue</c>, so <c>PlayoutTemplate</c>
/// rows inserted before it hold NULL, and by construction those rows had no day-of-month
/// restriction. On MySQL <c>PlayoutTemplate</c> arrived whole in
/// <c>20240114034944_Add_BlockScheduling</c>, so there is no code-path-free NULL for any of the six
/// there. This fixture manufactures its NULL with a raw <c>UPDATE</c>, which is a code path — it
/// measures MATERIALIZATION, and the legacy route above is established by reading the migrations.
/// </para>
/// <para>
/// What the selector then does with it is pinned by
/// <c>ErsatzTV.Core.Tests.Scheduling.AlternateScheduleSelectorTests.GetScheduleForDate_NullCollections</c>:
/// unguarded, <c>.Contains</c> throws <see cref="NullReferenceException" />.
/// </para>
/// <para>
/// Runs against BOTH providers from ONE fixture body, because the question is about provider
/// materialization and a SQLite-only answer would not have settled it. MySQL needs a live server,
/// supplied via <c>ETV_TEST_MYSQL_CONNECTION</c>; without it the MySQL fixture <b>ignores</b> — a
/// visible skip, never a silent pass. <c>ETV_REQUIRE_MYSQL_TESTS</c> turns that skip into a hard
/// failure for a runner that is supposed to have one. This mirrors
/// <see cref="LibraryFolderDedupeMigrationTests" />, which established the pattern.
/// </para>
/// <para>
/// <b>Cost, stated.</b> Each test migrates a FRESH database (per-test isolation by construction —
/// ersatztv#491 measured what a shared database and a wipe-that-must-succeed cost). Replaying every
/// migration is the expensive part and it is deliberate: a reachability fixture should stand on the
/// SHIPPED schema, not on one <c>EnsureCreated</c> builds from the current model.
/// </para>
/// </summary>
[TestFixture(TestProvider.Sqlite)]
[TestFixture(TestProvider.MySql)]
[NonParallelizable]
public class SchedulingCollectionColumnNullTests(TestProvider provider)
{
private const string MySqlConnectionVariable = "ETV_TEST_MYSQL_CONNECTION";
private const string MySqlRequiredVariable = "ETV_REQUIRE_MYSQL_TESTS";
private string _databasePath = null!;
private string? _mySqlConnectionString;
private DbContextOptions<TvContext> _options = null!;
private string _originalLastInsertedRowId = null!;
private string _originalCollation = null!;
private bool _originalIsSqlite;
private Func<DbUpdateException, bool> _originalUniqueViolation = null!;
/// <summary>
/// <c>TvContext</c>'s provider statics are process-wide, and the MySQL arm sets them to MySQL
/// values. Restoring them is NOT belt-and-braces: <see cref="InMemoryTvContext" /> — the harness
/// most of this suite uses — resets only three of the five (<c>IsSqlite</c>,
/// <c>IsUniqueConstraintViolation</c>, <c>RegisterUnicodeCaseFunctions</c>) and leaves
/// <c>LastInsertedRowId</c> and <c>CaseInsensitiveCollation</c> alone. So without this, a MySQL arm
/// running before a SQLite test leaves <c>last_insert_id()</c> in place for a SQLite connection,
/// which is an order-dependent failure in a fixture that never mentions MySQL.
/// <c>[NonParallelizable]</c> serialises execution; it does not restore state.
/// </summary>
[SetUp]
public async Task SetUp()
{
_originalLastInsertedRowId = TvContext.LastInsertedRowId;
_originalCollation = TvContext.CaseInsensitiveCollation;
_originalIsSqlite = TvContext.IsSqlite;
_originalUniqueViolation = TvContext.IsUniqueConstraintViolation;
if (provider is TestProvider.Sqlite)
{
TvContext.IsSqlite = true;
TvContext.LastInsertedRowId = "last_insert_rowid()";
TvContext.CaseInsensitiveCollation = "NOCASE";
TvContext.IsUniqueConstraintViolation = SqliteErrorClassifier.IsUniqueConstraintViolation;
_databasePath = Path.Combine(Path.GetTempPath(), $"etv823-{Guid.NewGuid():N}.sqlite3");
_options = new DbContextOptionsBuilder<TvContext>()
.UseSqlite(
$"Data Source={_databasePath};Foreign Keys=False",
o => o.MigrationsAssembly("ErsatzTV.Infrastructure.Sqlite"))
.Options;
await using TvContext sqlite = Create(_options);
await sqlite.Database.MigrateAsync();
return;
}
string? baseConnectionString = Environment.GetEnvironmentVariable(MySqlConnectionVariable);
if (string.IsNullOrWhiteSpace(baseConnectionString))
{
string message =
$"{MySqlConnectionVariable} is not set, so the MySql half of the #823 measurement cannot "
+ "run. Whether a NULL column materializes as CLR null is a provider question, so a "
+ "SQLite-only answer does not settle it.";
if (IsTrue(Environment.GetEnvironmentVariable(MySqlRequiredVariable)))
{
Assert.Fail($"{message} {MySqlRequiredVariable} is set, so this is a failure, not a skip.");
}
Assert.Ignore($"{message} Set it to run this locally.");
}
// A database of our own, and a FRESH one per test: isolation by construction, per #491's finding
// that a shared name trades isolation for a wipe that has to succeed.
_mySqlConnectionString =
new MySqlConnectionStringBuilder(baseConnectionString) { Database = $"etv823_{Guid.NewGuid():N}" }
.ConnectionString;
TvContext.IsSqlite = false;
TvContext.LastInsertedRowId = "last_insert_id()";
TvContext.CaseInsensitiveCollation = "utf8mb4_general_ci";
TvContext.IsUniqueConstraintViolation = MySqlErrorClassifier.IsUniqueConstraintViolation;
ServerVersion serverVersion = ServerVersion.AutoDetect(_mySqlConnectionString);
_options = new DbContextOptionsBuilder<TvContext>()
.UseMySql(
_mySqlConnectionString,
serverVersion,
o => o.MigrationsAssembly("ErsatzTV.Infrastructure.MySql"))
.Options;
// NUnit does not run [TearDown] when [SetUp] throws, and by this point the database exists — the
// migration itself created it. Without this, a migration that fails part way strands a database
// and its connection pool on a SHARED server, once per attempt.
try
{
await using TvContext mysql = Create(_options);
await mysql.Database.MigrateAsync();
}
catch
{
await DropMySqlDatabase();
_mySqlConnectionString = null;
throw;
}
}
private async Task DropMySqlDatabase()
{
try
{
await using (TvContext context = Create(_options))
{
await context.Database.EnsureDeletedAsync();
}
await using var probe = new MySqlConnection(_mySqlConnectionString);
await MySqlConnection.ClearPoolAsync(probe);
}
catch (Exception ex)
{
// Never mask the original failure with a cleanup failure, and never fail a PASSING test in
// teardown because the server went away -- but say so, because a silent cleanup failure is how
// a leak becomes invisible.
await TestContext.Out.WriteLineAsync(
$"WARNING: could not drop the MySql test database {_mySqlConnectionString}: {ex.Message}");
}
}
[TearDown]
public async Task TearDown()
{
TvContext.LastInsertedRowId = _originalLastInsertedRowId;
TvContext.CaseInsensitiveCollation = _originalCollation;
TvContext.IsSqlite = _originalIsSqlite;
TvContext.IsUniqueConstraintViolation = _originalUniqueViolation;
if (provider is TestProvider.Sqlite)
{
Microsoft.Data.Sqlite.SqliteConnection.ClearAllPools();
foreach (string path in new[] { _databasePath, $"{_databasePath}-wal", $"{_databasePath}-shm" })
{
if (File.Exists(path))
{
File.Delete(path);
}
}
return;
}
if (_mySqlConnectionString is not null)
{
await DropMySqlDatabase();
_mySqlConnectionString = null;
}
}
/// <summary>
/// A row whose three columns are NULL materializes as CLR null on every one of them — including
/// the two <c>IntCollectionValueConverter</c> columns, whose converter would have produced
/// <c>Array.Empty&lt;int&gt;()</c> had it been invoked.
/// </summary>
[Test]
public async Task A_Null_Column_Materializes_As_Clr_Null_On_All_Six()
{
await using (TvContext context = Create(_options))
{
await DisableForeignKeys(context);
context.ProgramScheduleAlternates.Add(NewAlternate(playoutId: 1));
context.PlayoutTemplates.Add(NewTemplate(playoutId: 1));
await context.SaveChangesAsync();
// Out of band, so the row is byte-identical to a legacy one. This UPDATE is itself a code
// path and does NOT demonstrate the legacy route -- see the class docstring for which single
// (column, provider) case is genuinely code-path-free. What is being measured here is what EF
// MATERIALIZES from such a row, which is the same regardless of how the NULL got there.
await context.Database.ExecuteSqlRawAsync(
"UPDATE ProgramScheduleAlternate SET DaysOfWeek = NULL, DaysOfMonth = NULL, MonthsOfYear = NULL");
await context.Database.ExecuteSqlRawAsync(
"UPDATE PlayoutTemplate SET DaysOfWeek = NULL, DaysOfMonth = NULL, MonthsOfYear = NULL");
}
// ANTI-VACUITY: a read that found no row would leave every ShouldBeNull below trivially true,
// so both reads are Single and would throw on an empty table.
await using (TvContext context = Create(_options))
{
ProgramScheduleAlternate alternate =
await context.ProgramScheduleAlternates.AsNoTracking().SingleAsync();
alternate.DaysOfWeek.ShouldBeNull(
"EnumCollectionJsonValueConverter must not be invoked for a NULL column");
alternate.DaysOfMonth.ShouldBeNull(
"IntCollectionValueConverter's null-to-empty branch must not run — EF skips the "
+ "converter for a NULL column, so this is null rather than an empty array");
alternate.MonthsOfYear.ShouldBeNull();
PlayoutTemplate template = await context.PlayoutTemplates.AsNoTracking().SingleAsync();
template.DaysOfWeek.ShouldBeNull();
template.DaysOfMonth.ShouldBeNull();
template.MonthsOfYear.ShouldBeNull();
}
}
/// <summary>
/// The write half. A caller that hands one of the six a null — which the two Replace* handlers do
/// verbatim from their command, and only the HTTP request records guard against — gets a stored
/// SQL NULL and no error at all.
/// </summary>
[Test]
public async Task Assigning_Null_Persists_Sql_Null_Rather_Than_Throwing()
{
await using (TvContext context = Create(_options))
{
await DisableForeignKeys(context);
ProgramScheduleAlternate alternate = NewAlternate(playoutId: 2);
alternate.DaysOfWeek = null!;
alternate.DaysOfMonth = null!;
alternate.MonthsOfYear = null!;
context.ProgramScheduleAlternates.Add(alternate);
// Not Should.NotThrow: the point is that this is the SHIPPED behaviour of the write path, so
// the assertion is that the round-trip below finds NULL, not merely that nothing blew up.
await context.SaveChangesAsync();
}
await using (TvContext context = Create(_options))
{
ProgramScheduleAlternate reloaded =
await context.ProgramScheduleAlternates.AsNoTracking().SingleAsync();
reloaded.DaysOfWeek.ShouldBeNull();
reloaded.DaysOfMonth.ShouldBeNull();
reloaded.MonthsOfYear.ShouldBeNull();
}
// ...and it really is SQL NULL in the column, not an empty string the converter round-trips.
// ExecuteScalar hands back CLR null both for a NULL column and for NO SUCH ROW, so the row is
// counted first — otherwise a fixture that silently deleted its row would report the same thing.
await using (TvContext context = Create(_options))
{
(await ScalarAsync(context, "SELECT COUNT(*) FROM ProgramScheduleAlternate"))
.ShouldNotBeNull();
Convert.ToInt32(await ScalarAsync(context, "SELECT COUNT(*) FROM ProgramScheduleAlternate"))
.ShouldBe(1, "the row is gone, so a null read below would prove nothing");
(await ScalarAsync(context, "SELECT DaysOfWeek FROM ProgramScheduleAlternate")).ShouldBeNull();
(await ScalarAsync(context, "SELECT DaysOfMonth FROM ProgramScheduleAlternate")).ShouldBeNull();
(await ScalarAsync(context, "SELECT MonthsOfYear FROM ProgramScheduleAlternate")).ShouldBeNull();
}
}
private static ProgramScheduleAlternate NewAlternate(int playoutId) =>
new()
{
PlayoutId = playoutId,
ProgramScheduleId = 1,
Index = 0,
DaysOfWeek = [],
DaysOfMonth = [],
MonthsOfYear = [],
StartMonth = 1,
StartDay = 1,
EndMonth = 12,
EndDay = 31
};
private static PlayoutTemplate NewTemplate(int playoutId) =>
new()
{
PlayoutId = playoutId,
TemplateId = 1,
Index = 0,
DaysOfWeek = [],
DaysOfMonth = [],
MonthsOfYear = [],
StartMonth = 1,
StartDay = 1,
EndMonth = 12,
EndDay = 31
};
/// <summary>
/// The rows here are deliberately partial graphs (a <c>PlayoutId</c> pointing at no Playout), so
/// foreign keys are off. SQLite takes it as a connection-string keyword; MySQL's
/// <c>foreign_key_checks</c> is a SESSION variable, so it is set on the context's own connection
/// and lives as long as that context does.
/// </summary>
private async Task DisableForeignKeys(TvContext context)
{
if (provider is TestProvider.MySql)
{
await context.Database.OpenConnectionAsync();
await context.Database.ExecuteSqlRawAsync("SET SESSION foreign_key_checks = 0");
}
}
private static async Task<object?> ScalarAsync(TvContext context, string sql)
{
await context.Database.OpenConnectionAsync();
await using System.Data.Common.DbCommand command = context.Database.GetDbConnection().CreateCommand();
command.CommandText = sql;
object? value = await command.ExecuteScalarAsync();
return value is DBNull ? null : value;
}
private static bool IsTrue(string? value) =>
!string.IsNullOrWhiteSpace(value)
&& (value == "1" || value.Equals("true", StringComparison.OrdinalIgnoreCase));
private static TvContext Create(DbContextOptions<TvContext> options) =>
new(
options,
NullLoggerFactory.Instance,
new SlowQueryInterceptor(NullLogger<SlowQueryInterceptor>.Instance));
}
@@ -0,0 +1,129 @@
using System.Reflection;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Infrastructure.Search;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
/// <summary>
/// ersatztv#824. The defect that produced #824 was not that <c>ElasticSearchIndex</c> was hard to
/// test — it was that NOTHING NOTICED it had no cover. #701 fixed two independent copies of the same
/// <c>UpdateSong</c> logic and pinned one; the suite stayed green, and the gap survived on a
/// hand-written list of what had been covered (namely, one entry).
/// <para>
/// So the covered set is compared against a population DERIVED FROM THE ASSEMBLY rather than
/// restated: a third <see cref="ISearchIndex" /> implementation reddens this test until it is
/// given a mutation fixture of its own. That is
/// <c>testing.guard-derives-population-from-source</c> applied to a test population instead of a
/// file population.
/// </para>
/// <para>
/// <b>Scope, stated rather than implied.</b> The derivation is over the assembly that declares
/// both indexers (<c>ErsatzTV.Infrastructure</c>). An implementation added in a DIFFERENT
/// assembly is outside what this sees — it is not covered and this test cannot say so. Both
/// implementations have lived here since the interface existed, so the narrower scope buys a
/// guard that cannot be defeated by an unrelated assembly load order; widening it to every
/// loaded assembly would be the false-confidence version of the same check.
/// </para>
/// </summary>
[TestFixture]
public class SearchIndexMutationCoverageTests
{
/// <summary>
/// The indexers whose <c>UpdateSong</c> is pinned against the ersatztv#701 mutation. Referenced by
/// TYPE, so renaming an indexer or deleting a fixture is a compile error rather than a silent
/// divergence.
/// </summary>
private static readonly Dictionary<Type, Type> CoveredBy = new()
{
[typeof(LuceneSearchIndex)] = typeof(SongIndexerMetadataMutationTests),
[typeof(ElasticSearchIndex)] = typeof(ElasticSongIndexerMetadataMutationTests)
};
[Test]
public void Every_ISearchIndex_Implementation_Has_A_Metadata_Mutation_Fixture()
{
List<Type> implementations = typeof(LuceneSearchIndex).Assembly
.GetTypes()
.Where(t => t is { IsAbstract: false, IsInterface: false })
.Where(t => typeof(ISearchIndex).IsAssignableFrom(t))
.OrderBy(t => t.FullName, StringComparer.Ordinal)
.ToList();
// The set comparison below would ALREADY fail on an empty derivation, because the expected side
// is non-empty -- so this floor is not load-bearing for correctness and saying it is would be a
// false claim about a check. It is a DIAGNOSTIC: it separates "the reflection stopped finding
// types" (a moved type, a renamed interface) from "someone added an indexer", which the set
// comparison alone reports identically.
implementations.Count.ShouldBeGreaterThanOrEqualTo(
2,
"the ISearchIndex population derivation found almost nothing -- this is the derivation "
+ "breaking, not an indexer being added");
implementations.ShouldBe(
CoveredBy.Keys.OrderBy(t => t.FullName, StringComparer.Ordinal),
ignoreOrder: false,
"every ISearchIndex implementation needs an UpdateSong metadata-mutation fixture — see "
+ "ersatztv#824, where a second copy of the same logic went uncovered and an Elastic-only "
+ "reintroduction of `metadata.Artists ??= []` left the whole suite green");
// Comparing the KEYS alone would leave the mapping half-checked: a third indexer could be pointed
// at an EXISTING fixture, or at a fixture class holding no runnable test, and the set comparison
// above would still pass. Both are closed here. What NO static check can establish is that the
// named fixture actually DRIVES its indexer -- that is stated as a residual on this guard rather
// than implied away, and it is why the record claims a third implementation cannot be added
// WITHOUT NOTICE, not that it cannot be mis-covered.
CoveredBy.Values.Distinct().Count().ShouldBe(
CoveredBy.Count,
"two indexers are mapped to the SAME fixture, so one of them is not actually covered");
foreach ((Type indexer, Type fixture) in CoveredBy)
{
// Test-ness is decided by NUnit's INTERFACES, not by a hand-listed set of attribute types.
// A list has to be kept in step with NUnit and silently falsely-reddens whatever it omits --
// an earlier revision listed TestAttribute alone (so a [TestCase]-only fixture failed), then
// three types (so a [Theory] one did). ITestBuilder/ISimpleTestBuilder is what NUnit itself
// dispatches on, so it cannot fall behind the vocabulary.
//
// DeclaredOnly is load-bearing: without it a fixture that merely SUBCLASSES another inherits
// its [Test] and satisfies this while driving the wrong indexer -- and Values.Distinct()
// cannot catch that, since the two Types differ. Static is included because NUnit runs a
// public static test method in a non-static fixture, and omitting the flag would falsely
// redden one.
const BindingFlags TestMethods =
BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static | BindingFlags.DeclaredOnly;
static bool NeverRuns(object[] attributes) =>
attributes.Any(a => a is ExplicitAttribute or IgnoreAttribute);
bool declaresARunnableTest = fixture
.GetMethods(TestMethods)
.Select(m => m.GetCustomAttributes(inherit: true))
.Any(attrs =>
attrs.Any(a => a is ITestBuilder or ISimpleTestBuilder) && !NeverRuns(attrs));
declaresARunnableTest.ShouldBeTrue(
$"{fixture.Name} is named as the mutation fixture for {indexer.Name} but DECLARES no "
+ "runnable test method of its own, so it proves nothing");
// The same "wired is not running" failure at fixture level, in its three forms: an abstract
// class NUnit will not instantiate, and [Explicit]/[Ignore]. The indexer population at the top
// of this method already filters IsAbstract; the fixture side needs the mirror of that.
// `IsAbstract` ALONE is wrong here and would have been a false red: a C# `static class`
// compiles to `abstract sealed`, and NUnit runs a test declared in one. What must be rejected
// is an abstract BASE (abstract and NOT sealed), which NUnit cannot instantiate. Deliberately
// no in-repo witness is cited: `AlternateScheduleSelectorTests` is a static class, but it
// merely NESTS its `[TestFixture]`es and declares no test of its own, so it would fail the
// sibling assertion above instead -- naming it here would have been a wrong example attached
// to a right rule.
(fixture.IsAbstract && !fixture.IsSealed).ShouldBeFalse(
$"{fixture.Name} is named as the mutation fixture for {indexer.Name} but is an abstract "
+ "base class, so NUnit never runs it");
NeverRuns(fixture.GetCustomAttributes(inherit: true)).ShouldBeFalse(
$"{fixture.Name} is named as the mutation fixture for {indexer.Name} but is [Explicit] or "
+ "[Ignore]d, so it never runs and proves nothing");
}
}
}
@@ -0,0 +1,208 @@
using System.Reflection;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Search;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
using ErsatzTV.Infrastructure.Search;
using ErsatzTV.Tests.Support;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.Index;
using Lucene.Net.Store;
using Lucene.Net.Util;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Integration;
/// <summary>
/// ersatztv#701, EXECUTED against a real <see cref="TvContext" /> on SQLite.
/// <para>
/// <b>The defect.</b> <c>LuceneSearchIndex.UpdateSong</c> opened with
/// <c>metadata.AlbumArtists ??= []; metadata.Artists ??= [];</c>. Unlike the navigation
/// collections guarded the same way all around them, these two are SCALAR COLUMNS: the whole
/// list lives in one column, so the property IS the column value. They are two of EIGHT such
/// columns in the model — the population is derived from the MODEL CONFIGURATION (EF-native
/// primitive collections plus the six <c>HasConversion&lt;*CollectionValueConverter&gt;</c>
/// columns on <c>ProgramScheduleAlternate</c>/<c>PlayoutTemplate</c>), NOT by grepping the
/// domain classes for <c>IList&lt;string&gt;</c>, which finds only two of the eight. See
/// the decision record <c>media.nullable-primitive-collection-mutation</c>.
/// Assigning one on a TRACKED entity flips it to <see cref="EntityState.Modified" />, and the next
/// <c>SaveChanges</c> writes <c>[]</c> over what the database held as <c>NULL</c> — the exact
/// mechanism an adversarial review demonstrated in ersatztv#691, which is why that issue's
/// entity-level guard was reverted in favour of guarding at the READ SITE.
/// </para>
/// <para>
/// <b>Why the fixture loads the song TRACKED even though production does not.</b> Both feeds into
/// the indexer are <c>AsNoTracking()</c> today — <c>SearchRepository.GetItemToIndex</c> and
/// <c>SearchRepository.GetAllSongs</c> — so no shipped caller loses data. That is a property of
/// today's two callers, not of the indexer, and it is exactly what ersatztv#691 recorded as "a
/// loaded gun". This fixture therefore pins the INDEXER's own contract: handed a tracked entity it
/// must not mutate it. Run against the real pre-fix file, the FIRST of the numbered assertions
/// below fails (<c>metadata.Artists should be null but was []</c>) and the run stops there;
/// reaching the persistence half needs a probe variant with assertions 1 and 2 replaced by
/// prints, which reports <c>Modified</c> and the column moving from <c>NULL</c> to <c>[]</c>.
/// Each was separately shown discriminating. A future caller that drops <c>AsNoTracking</c>
/// therefore cannot reintroduce the data loss silently.
/// </para>
/// <para>
/// The Lucene <see cref="IndexWriter" /> is injected into the private field rather than obtained via
/// <c>Initialize</c>, because <c>Initialize</c> writes to <c>FileSystemLayout.SearchIndexFolder</c> —
/// a process-wide static resolved once from <c>ETV_CONFIG_FOLDER</c>, i.e. the developer's real
/// application data folder. Letting the writer throw instead is NOT an option here: the
/// <c>catch</c> in <c>UpdateSong</c> assigns <c>metadata.Song = null</c>, which would itself dirty
/// the entity under test and make the probe report the wrong cause.
/// </para>
/// </summary>
[TestFixture]
public class SongIndexerMetadataMutationTests
{
[Test]
public async Task UpdateSong_Must_Not_Mutate_Nullable_Artists_On_A_Tracked_Entity()
{
await using var harness = await InMemoryTvContext.CreateAsync();
int metadataId;
await using (TvContext context = harness.CreateContext())
{
var library = new LocalLibrary { Name = "Music", MediaKind = LibraryMediaKind.Songs };
context.Add(library);
await context.SaveChangesAsync();
var libraryPath = new LibraryPath { Path = "/music", LibraryId = library.Id };
context.Add(libraryPath);
await context.SaveChangesAsync();
var song = new Song
{
LibraryPathId = libraryPath.Id,
MediaVersions = [],
SongMetadata =
[
new SongMetadata
{
MetadataKind = MetadataKind.Fallback,
Title = "Untagged Track",
SortTitle = "untagged track",
DateAdded = new DateTime(2026, 1, 1, 0, 0, 0, DateTimeKind.Utc),
// The shape FallbackMetadataProvider.GetSongMetadata leaves behind: it never
// assigns either primitive collection, so both columns persist as NULL.
Artists = null!,
AlbumArtists = null!,
Genres = [],
Tags = [],
Studios = [],
Actors = [],
Artwork = [],
Guids = []
}
]
};
context.Add(song);
await context.SaveChangesAsync();
metadataId = song.SongMetadata[0].Id;
}
// The seed must actually have produced NULL columns, or every assertion below is vacuous.
(await ReadRawArtists(harness, metadataId)).ShouldBeNull();
await using (TvContext context = harness.CreateContext())
{
// Deliberately TRACKED -- see the fixture docstring.
Song tracked = await context.Songs
.IncludeForSearch()
.AsSplitQuery()
.SingleAsync();
SongMetadata metadata = tracked.SongMetadata[0];
metadata.Artists.ShouldBeNull("EF must materialize the NULL column as null, not as an empty list");
// UpdateSong wraps its whole body in a catch that logs a warning and assigns
// `metadata.Song = null` -- which severs a required relationship and cascades the metadata to
// Deleted. A silently-exercised catch would therefore make every assertion below report the
// wrong cause, so the logger fails the test instead of swallowing.
var logger = new ThrowOnWarningLogger<LuceneSearchIndex>();
var index = new LuceneSearchIndex(
new SearchQueryParser(
Substitute.For<ISmartCollectionCache>(),
Substitute.For<ILogger<SearchQueryParser>>()),
logger);
using var directory = new RAMDirectory();
using var writer = new IndexWriter(
directory,
new IndexWriterConfig(LuceneVersion.LUCENE_48, new StandardAnalyzer(LuceneVersion.LUCENE_48)));
typeof(LuceneSearchIndex)
.GetField("_writer", BindingFlags.NonPublic | BindingFlags.Instance)!
.SetValue(index, writer);
// A bare substitute returns null from GetAllLanguageCodes, which NPEs inside AddLanguages and
// would divert the run into the catch above.
var languageCodeService = Substitute.For<ILanguageCodeService>();
languageCodeService.GetAllLanguageCodes(Arg.Any<List<string>>()).Returns([]);
languageCodeService.GetAllLanguageCodes(Arg.Any<string>()).Returns([]);
await index.UpdateItems(
Substitute.For<ISearchRepository>(),
Substitute.For<IFallbackMetadataProvider>(),
languageCodeService,
[tracked]);
logger.Failure.ShouldBeNull("UpdateSong threw and its catch ran, so this probe measured the "
+ "error path rather than the indexing path");
// POSITIVE CONTROL. Every assertion below asserts that something did NOT happen, so all of
// them hold vacuously if UpdateSong never ran at all -- and it silently stops running if a
// future refactor gates UpdateItems on `_initialized`, which this fixture deliberately
// bypasses by injecting the writer. Verified BOTH ways by adding
// `if (!_initialized) { return Unit.Default; }` to UpdateItems (a bare `return;` does not
// compile there -- CS0126): with this line present it is the only failure, and with it
// removed the whole test PASSES while the code under test is unreachable.
// NumDocs == 1 proves the song-indexing path ran; it does NOT prove the artist loops
// specifically ran, which would need a second seeded song asserting ArtistField.
writer.NumDocs.ShouldBe(1, "UpdateSong did not index the song, so the assertions below "
+ "would pass without exercising the code under test");
// 1. The indexer left the entity alone.
metadata.Artists.ShouldBeNull();
metadata.AlbumArtists.ShouldBeNull();
// 2. ...so EF has nothing to persist. This is the assertion that fails loudly the day the
// mutation returns, even if a later refactor stopped the value from being observable above.
context.Entry(metadata).State.ShouldBe(EntityState.Unchanged);
// 3. And the save that a real caller would go on to make does not rewrite the column.
await context.SaveChangesAsync();
}
(await ReadRawArtists(harness, metadataId)).ShouldBeNull();
}
private static async Task<object?> ReadRawArtists(InMemoryTvContext harness, int metadataId)
{
await using TvContext context = harness.CreateContext();
await using var command = context.Database.GetDbConnection().CreateCommand();
command.CommandText = $"SELECT Artists FROM SongMetadata WHERE Id = {metadataId}";
object? value = await command.ExecuteScalarAsync();
// ExecuteScalar returns CLR null both for "the column is NULL" and for "there is no such row",
// and the second is reachable: UpdateSong's catch assigns metadata.Song = null, which severs a
// required relationship and cascades the row to Deleted, so a SaveChanges on the error path
// DELETES it and a plain null check would pass for the wrong reason.
if (value is null)
{
Assert.Fail($"SongMetadata row {metadataId} no longer exists, so its Artists column cannot "
+ "be read -- the probe measured a deleted row rather than a preserved NULL.");
}
return value is DBNull ? null : value;
}
}
@@ -126,6 +126,70 @@ public class ApiKeyProviderTests
key.ShouldMatch("^[0-9a-f]{64}$");
}
// ---- Api:RequireKeyForReads, read through the REAL provider (ersatztv#779, detector F) ----
//
// Every other assertion about the read-gating posture goes through a hand-written
// FakeApiKeyProvider that is HANDED the bool (ApiAuthorizationFilterTests,
// ApiKeyEndpointRequiresKeyTests). Those fakes prove the FILTER reacts to the flag; they cannot
// see the line that DERIVES it, because they never run it. Until these tests, nothing in the
// suite constructed ApiKeyProvider at all, so a mistyped configuration key or a flipped default
// would have left the whole suite green while shipping anonymous reads (#280/#282).
//
// Api:WriteKey is set in every case purely so ResolveKey returns before touching the real
// FileSystemLayout.ApiKeyPath — the constructor would otherwise generate and persist a key into
// the live config volume. It is deliberately NOT the subject of these tests.
private static ApiKeyProvider ProviderWith(params (string Key, string Value)[] settings)
{
// The WriteKey entry is appended LAST so a caller cannot override it to empty. That is not
// hypothetical tidiness: an empty Api:WriteKey sends ResolveKey down the real path, which
// reads, generates and PERSISTS a key into the live config volume (FileSystemLayout
// .ApiKeyPath) from a unit test.
var withKey = new List<(string, string)>(settings) { (ApiKeyProvider.WriteKeyConfigurationKey, "test-key") };
return new ApiKeyProvider(Config(withKey.ToArray()), NullLogger<ApiKeyProvider>.Instance);
}
[Test]
public void Read_Gating_Is_Required_When_The_Setting_Is_Absent()
{
// The shipped default, and the case a fixture that simply OMITS the field would test by
// accident. Asserted explicitly so it is a pinned decision rather than a coincidence.
ProviderWith().RequireKeyForReads.ShouldBeTrue();
}
[TestCase("true")]
[TestCase("True")]
[TestCase("TRUE")]
public void Read_Gating_Is_Required_At_The_Explicit_Production_Value(string configured)
{
// The DENY path at the production value, which is the half #756 showed can stay invisible:
// the absent case behaving correctly says nothing about the configured one.
ProviderWith((ApiKeyProvider.RequireKeyForReadsConfigurationKey, configured))
.RequireKeyForReads.ShouldBeTrue();
}
[TestCase("")]
[TestCase("1")]
[TestCase("yes")]
public void A_Non_Boolean_Read_Gating_Value_Fails_Startup_Rather_Than_Reads(string configured)
{
// The fourth cell of the matrix, and the one an operator actually hits: `Api__RequireKeyForReads=`
// with nothing after it in a compose file, or a habitual `1`/`yes`. ConfigurationBinder returns
// the default ONLY for a null section value, so any present-but-unparseable string goes through
// BooleanConverter and throws. That is fail-CLOSED — the app refuses to start rather than
// quietly choosing a posture — and it is pinned here so a future switch to a lenient parse
// (TryParse with a fallback) cannot silently turn a typo into anonymous reads.
Should.Throw<InvalidOperationException>(() =>
ProviderWith((ApiKeyProvider.RequireKeyForReadsConfigurationKey, configured)));
}
[TestCase("false")]
[TestCase("False")]
public void Read_Gating_Is_Waived_Only_By_An_Explicit_Opt_Out(string configured)
{
ProviderWith((ApiKeyProvider.RequireKeyForReadsConfigurationKey, configured))
.RequireKeyForReads.ShouldBeFalse();
}
[Test]
public void Returns_A_Usable_Key_Even_When_Persist_Fails()
{
@@ -88,7 +88,8 @@ public abstract class ChannelHandlerTestBase
bool showInEpg = false,
string logoPath = "",
string name = "Test",
string group = "ErsatzTV") =>
string group = "ErsatzTV",
StreamingMode streamingMode = StreamingMode.TransportStreamHybrid) =>
new(
name,
number,
@@ -105,7 +106,7 @@ public abstract class ChannelHandlerTestBase
ChannelPlayoutMode.Continuous,
null,
null,
StreamingMode.TransportStreamHybrid,
streamingMode,
null,
null,
string.Empty,
@@ -0,0 +1,37 @@
using Microsoft.Extensions.Logging;
namespace ErsatzTV.Tests.Support;
/// <summary>
/// Captures the first warning-or-worse a component logs, so a fixture can fail on it instead of
/// silently measuring an error path.
/// <para>
/// Both search indexers wrap each <c>Update*</c> body in a <c>catch</c> that logs a warning and
/// assigns <c>metadata.Song = null</c> — which severs a required relationship and cascades the
/// metadata row to <see cref="Microsoft.EntityFrameworkCore.EntityState.Deleted" />. A fixture
/// that let that catch run quietly would report the wrong cause for every assertion after it, and
/// on the first run of the ersatztv#701 probe it did exactly that (a bare
/// <c>ILanguageCodeService</c> substitute NPEs inside <c>AddLanguages</c>).
/// </para>
/// </summary>
public sealed class ThrowOnWarningLogger<T> : ILogger<T>
{
public Exception? Failure { get; private set; }
public IDisposable? BeginScope<TState>(TState state) where TState : notnull => null;
public bool IsEnabled(LogLevel logLevel) => true;
public void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter)
{
if (logLevel >= LogLevel.Warning)
{
Failure ??= exception ?? new InvalidOperationException(formatter(state, exception));
}
}
}
@@ -1,3 +1,4 @@
using System.ComponentModel;
using ErsatzTV.Application.FFmpegProfiles;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
@@ -13,6 +14,11 @@ public record CreateFFmpegProfileRequest(
string VaapiDisplay,
VaapiDriver VaapiDriver,
string VaapiDevice,
[property: Description(
"Extra surfaces in the QSV upload pool. Must be at least 64 when set; a smaller pool "
+ "leaves no headroom for frames in flight and the transcode writes nothing at all. On update, a "
+ "value equal to the one already stored is accepted unchanged, so a profile written before this "
+ "validation existed stays editable.")]
int? QsvExtraHardwareFrames,
int ResolutionId,
ScalingBehavior ScalingBehavior,
@@ -35,7 +41,16 @@ public record CreateFFmpegProfileRequest(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool? QsvPreferNativeDecoder = null)
bool? QsvPreferNativeDecoder = null,
[property: Description(
"Realtime pacing multiplier for the input. Unset keeps the built-in pacing "
+ "(1.05, or 1.0 for a stream copy). Must be between 1.0 and 2.0 when set.")]
double? ReadRate = null,
[property: Description(
"Rate a lagging realtime input may read at until it is level again. Unset keeps the "
+ "built-in 6.0. Must be between 1.0 and 10.0, and GREATER than the read rate — equal is "
+ "zero headroom, which is functionally no catchup.")]
double? ReadRateCatchup = null)
{
public CreateFFmpegProfile ToCommand() =>
new(
@@ -69,5 +84,7 @@ public record CreateFFmpegProfileRequest(
NormalizeFramerate,
NormalizeColors,
DeinterlaceVideo,
QsvPreferNativeDecoder ?? true);
QsvPreferNativeDecoder ?? true,
ReadRate,
ReadRateCatchup);
}
@@ -1,3 +1,4 @@
using System.ComponentModel;
using ErsatzTV.Application.FFmpegProfiles;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
@@ -13,6 +14,11 @@ public record UpdateFFmpegProfileRequest(
string VaapiDisplay,
VaapiDriver VaapiDriver,
string VaapiDevice,
[property: Description(
"Extra surfaces in the QSV upload pool. Must be at least 64 when set; a smaller pool "
+ "leaves no headroom for frames in flight and the transcode writes nothing at all. On update, a "
+ "value equal to the one already stored is accepted unchanged, so a profile written before this "
+ "validation existed stays editable.")]
int? QsvExtraHardwareFrames,
int ResolutionId,
ScalingBehavior ScalingBehavior,
@@ -35,7 +41,16 @@ public record UpdateFFmpegProfileRequest(
bool NormalizeFramerate,
bool NormalizeColors,
bool DeinterlaceVideo,
bool? QsvPreferNativeDecoder = null)
bool? QsvPreferNativeDecoder = null,
[property: Description(
"Realtime pacing multiplier for the input. Unset keeps the built-in pacing "
+ "(1.05, or 1.0 for a stream copy). Must be between 1.0 and 2.0 when set.")]
double? ReadRate = null,
[property: Description(
"Rate a lagging realtime input may read at until it is level again. Unset keeps the "
+ "built-in 6.0. Must be between 1.0 and 10.0, and GREATER than the read rate — equal is "
+ "zero headroom, which is functionally no catchup.")]
double? ReadRateCatchup = null)
{
public UpdateFFmpegProfile ToCommand(int id) =>
new(
@@ -70,5 +85,7 @@ public record UpdateFFmpegProfileRequest(
NormalizeFramerate,
NormalizeColors,
DeinterlaceVideo,
QsvPreferNativeDecoder ?? true);
QsvPreferNativeDecoder ?? true,
ReadRate,
ReadRateCatchup);
}
@@ -90,7 +90,8 @@ public class DatabaseMigratorService : BackgroundService
await DbInitializer.Initialize(dbContext, stoppingToken);
var fileSystem = scope.ServiceProvider.GetRequiredService<System.IO.Abstractions.IFileSystem>();
await GraphicsElementSeeder.SeedOnNowNext(dbContext, fileSystem, stoppingToken);
await GraphicsElementSeeder.SeedOnNowNext(dbContext, fileSystem, _logger, stoppingToken);
await GraphicsElementSeeder.AttachOnNowNextByDefault(dbContext, stoppingToken);
_systemStartup.DatabaseIsReady();
+51 -1
View File
@@ -25391,6 +25391,7 @@
"null",
"integer"
],
"description": "Extra surfaces in the QSV upload pool. Must be at least 64 when set; a smaller pool leaves no headroom for frames in flight and the transcode writes nothing at all. On update, a value equal to the one already stored is accepted unchanged, so a profile written before this validation existed stays editable.",
"format": "int32"
},
"resolutionId": {
@@ -25478,6 +25479,22 @@
"null",
"boolean"
]
},
"readRate": {
"type": [
"null",
"number"
],
"description": "Realtime pacing multiplier for the input. Unset keeps the built-in pacing (1.05, or 1.0 for a stream copy). Must be between 1.0 and 2.0 when set.",
"format": "double"
},
"readRateCatchup": {
"type": [
"null",
"number"
],
"description": "Rate a lagging realtime input may read at until it is level again. Unset keeps the built-in 6.0. Must be between 1.0 and 10.0, and GREATER than the read rate — equal is zero headroom, which is functionally no catchup.",
"format": "double"
}
}
},
@@ -26485,7 +26502,9 @@
"normalizeFramerate",
"normalizeColors",
"deinterlaceVideo",
"qsvPreferNativeDecoder"
"qsvPreferNativeDecoder",
"readRate",
"readRateCatchup"
],
"type": "object",
"properties": {
@@ -26604,6 +26623,20 @@
},
"qsvPreferNativeDecoder": {
"type": "boolean"
},
"readRate": {
"type": [
"null",
"number"
],
"format": "double"
},
"readRateCatchup": {
"type": [
"null",
"number"
],
"format": "double"
}
}
},
@@ -32016,6 +32049,7 @@
"null",
"integer"
],
"description": "Extra surfaces in the QSV upload pool. Must be at least 64 when set; a smaller pool leaves no headroom for frames in flight and the transcode writes nothing at all. On update, a value equal to the one already stored is accepted unchanged, so a profile written before this validation existed stays editable.",
"format": "int32"
},
"resolutionId": {
@@ -32103,6 +32137,22 @@
"null",
"boolean"
]
},
"readRate": {
"type": [
"null",
"number"
],
"description": "Realtime pacing multiplier for the input. Unset keeps the built-in pacing (1.05, or 1.0 for a stream copy). Must be between 1.0 and 2.0 when set.",
"format": "double"
},
"readRateCatchup": {
"type": [
"null",
"number"
],
"description": "Rate a lagging realtime input may read at until it is level again. Unset keeps the built-in 6.0. Must be between 1.0 and 10.0, and GREATER than the read rate — equal is zero headroom, which is functionally no catchup.",
"format": "double"
}
}
},
@@ -816,7 +816,7 @@
<div key={r.name} style={{ borderTop: i ? "1px solid var(--border-hairline)" : "none", opacity: r.alreadyExists ? 0.55 : 1 }}>
<div style={{ display: "flex", alignItems: "center", gap: 12, padding: "9px 14px" }}>
<Checkbox checked={on} disabled={r.alreadyExists} onChange={() => toggle(r.name)} />
<span style={{ ...mono, minWidth: 42, font: "var(--text-sm) var(--font-mono)", color: "var(--text-secondary)" }}>{r.number}</span>
<span style={{ ...mono, minWidth: 42, textAlign: "right", font: "var(--text-sm) var(--font-mono)", color: "var(--text-secondary)" }}>{r.number}</span>
<Bug name={name} initials={ov.bug && ov.bug.initials} ci={ov.bug && ov.bug.ci} size={28} />
<button type="button" onClick={() => !r.alreadyExists && toggleExp(r.name)} style={{ flex: 1, minWidth: 0, textAlign: "left", background: "transparent", border: "none", padding: 0, cursor: r.alreadyExists ? "default" : "pointer" }}>
<div style={{ font: "var(--weight-semibold) var(--text-sm)/1.2 var(--font-sans)", color: "var(--text-primary)", whiteSpace: "nowrap", overflow: "hidden", textOverflow: "ellipsis" }}>{name}</div>
@@ -816,7 +816,7 @@
<div key={r.name} style={{ borderTop: i ? "1px solid var(--border-hairline)" : "none", opacity: r.alreadyExists ? 0.55 : 1 }}>
<div style={{ display: "flex", alignItems: "center", gap: 12, padding: "9px 14px" }}>
<Checkbox checked={on} disabled={r.alreadyExists} onChange={() => toggle(r.name)} />
<span style={{ ...mono, minWidth: 42, font: "var(--text-sm) var(--font-mono)", color: "var(--text-secondary)" }}>{r.number}</span>
<span style={{ ...mono, minWidth: 42, textAlign: "right", font: "var(--text-sm) var(--font-mono)", color: "var(--text-secondary)" }}>{r.number}</span>
<Bug name={name} initials={ov.bug && ov.bug.initials} ci={ov.bug && ov.bug.ci} size={28} />
<button type="button" onClick={() => !r.alreadyExists && toggleExp(r.name)} style={{ flex: 1, minWidth: 0, textAlign: "left", background: "transparent", border: "none", padding: 0, cursor: r.alreadyExists ? "default" : "pointer" }}>
<div style={{ font: "var(--weight-semibold) var(--text-sm)/1.2 var(--font-sans)", color: "var(--text-primary)", whiteSpace: "nowrap", overflow: "hidden", textOverflow: "ellipsis" }}>{name}</div>
@@ -56,7 +56,71 @@
{ label: "loudnorm", value: "LoudNorm" },
];
function Row({ label, help, control = 320, first = false, children }) {
// Level-2 explainer copy for the field-level progressive-disclosure pattern (ersatztv#734).
// Mirrors web/src/screens/FFmpegProfilesScreen.tsx's FIELD_HELP record; see
// docs/spa-conventions.md §15 for the contract this shape is mirroring.
const FIELD_HELP = {
threadCount:
"Caps the worker threads FFmpeg uses per transcode. 0 lets FFmpeg decide, which is usually right; a low fixed value keeps one channel from starving the others on a busy host, at the cost of falling behind realtime on heavy content.",
scalingBehavior:
"Decides what happens when the source aspect ratio does not match the preferred resolution. Scale and Pad keeps the whole picture and adds bars; Crop fills the frame and cuts whatever overflows; Stretch fills it by distorting the image, which is why it is rarely what you want.",
videoBitrate:
"Target output bitrate. Too low and the encoder throws away detail on motion; too high and clients on slow links buffer. Buffer size is the companion setting — it bounds how far the encoder may deviate from this target.",
videoBufferSize:
"How much bitrate deviation the encoder may bank before it has to correct. Roughly 2x the bitrate is the usual starting point. Very small values force a near-constant bitrate and hurt quality on scene changes.",
hardwareAcceleration:
"Offloads decode and encode to the GPU. The list only offers what this FFmpeg build supports, so an unsupported kind never appears here. What nothing checks when you save is whether the device itself is present and passed through to the container — that is the mismatch that fails at playback time.",
normalizeLoudnessMode:
"Levels volume across content from different sources. Off leaves each item at its own level, so volume jumps between them; loudnorm retargets everything to one integrated loudness, which evens that out at the cost of an extra filter in the graph and a less faithful dynamic range.",
};
// Mockup of the shared `FieldHelp` primitive (web/src/components/fieldHelp.tsx). The prototype
// reproduces the two opening signals that read in a static review (tap-pins and hover); the shipped
// component additionally handles keyboard focus, Escape, outside-press dismissal, the hover
// bridge, and portalling the panel out of the card's clip.
function FieldHelp({ label, detail }) {
const [pinned, setPinned] = React.useState(false);
const [hovered, setHovered] = React.useState(false);
const open = pinned || hovered;
return (
<span
style={{ position: "relative", display: "inline-flex", verticalAlign: "middle", marginLeft: 5 }}
onMouseEnter={() => setHovered(true)}
onMouseLeave={() => setHovered(false)}
>
<button
type="button"
aria-label={`More about ${label}`}
aria-expanded={open}
onClick={(e) => { e.preventDefault(); setPinned((v) => !v); }}
style={{
display: "inline-flex", alignItems: "center", justifyContent: "center",
width: 18, height: 18, border: "none", borderRadius: "var(--radius-xs)",
background: "none", padding: 0, color: "var(--text-disabled)", cursor: "help",
}}
>
<Ico.Info aria-hidden="true" size={13} />
</button>
{open && (
<span
role="note"
style={{
position: "absolute", zIndex: 60, left: 0, top: "calc(100% + 7px)",
width: "max-content", maxWidth: 320,
border: "1px solid var(--border-control)", borderRadius: "var(--radius-sm)",
background: "var(--ctv-surface-3)", boxShadow: "var(--shadow-pop)",
padding: "9px 11px", color: "var(--text-primary)",
font: "var(--text-xs)/1.5 var(--font-sans)", whiteSpace: "normal", textAlign: "left",
}}
>
{detail}
</span>
)}
</span>
);
}
function Row({ label, help, detail, control = 320, first = false, children }) {
return (
<div
style={{
@@ -68,7 +132,10 @@
}}
>
<div style={{ flex: "1 1 auto", minWidth: 0, paddingTop: 5 }}>
<div style={{ font: "var(--weight-medium) var(--text-sm)/1.3 var(--font-sans)", color: "var(--text-primary)" }}>{label}</div>
<div style={{ font: "var(--weight-medium) var(--text-sm)/1.3 var(--font-sans)", color: "var(--text-primary)" }}>
{label}
{detail && <FieldHelp detail={detail} label={label} />}
</div>
{help && <div style={{ marginTop: 3, font: "var(--text-xs)/1.4 var(--font-sans)", color: "var(--text-secondary)", maxWidth: 460 }}>{help}</div>}
</div>
<div style={{ flex: `0 0 ${control}px` }}>{children}</div>
@@ -177,7 +244,7 @@
<Row first label="Name" control={360}>
<Input size="sm" value={isAdd ? "" : profile.name} placeholder="e.g. 1080p30 Software" />
</Row>
<Row label="Thread count" control={200}>
<Row label="Thread count" control={200} detail={FIELD_HELP.threadCount} help="0 lets FFmpeg pick a thread count.">
<Input size="sm" type="number" trailing="threads" value="0" />
</Row>
<Row label="Normalize audio" control={200}>
@@ -191,7 +258,7 @@
</Row>
{normalizeVideo && (
<React.Fragment>
<Row label="Scaling behavior" control={360}>
<Row label="Scaling behavior" control={360} detail={FIELD_HELP.scalingBehavior} help="What to do when the source aspect ratio does not match.">
<Select value="ScaleAndPad" options={SCALING_OPTIONS} />
</Row>
<Row control={360} label="Pad mode" help="Hardware padding only applies with VAAPI; otherwise software padding is used.">
@@ -219,13 +286,13 @@
<Row label="Bit depth" control={360}>
<Select value="EightBit" options={BIT_DEPTH_OPTIONS} />
</Row>
<Row label="Bitrate" control={220}>
<Row label="Bitrate" control={220} detail={FIELD_HELP.videoBitrate}>
<Input size="sm" type="number" trailing="kBit/s" value="2000" />
</Row>
<Row label="Buffer size" control={220}>
<Row label="Buffer size" control={220} detail={FIELD_HELP.videoBufferSize} help="Usually about twice the bitrate.">
<Input size="sm" type="number" trailing="kBit" value="4000" />
</Row>
<Row label="Hardware acceleration" control={360}>
<Row label="Hardware acceleration" control={360} detail={FIELD_HELP.hardwareAcceleration} help="Requires the device to be passed through to the container.">
<Select value={hwaccel} options={HWACCEL_OPTIONS} onChange={(e) => setHwaccel(e.target.value)} />
</Row>
{hwaccel === "Vaapi" && (
@@ -290,7 +357,7 @@
<Row label="Sample rate" control={220}>
<Input size="sm" type="number" trailing="kHz" value="48" />
</Row>
<Row label="Normalize loudness" control={360}>
<Row label="Normalize loudness" control={360} detail={FIELD_HELP.normalizeLoudnessMode}>
<Select value={loudness} options={LOUDNESS_OPTIONS} onChange={(e) => setLoudness(e.target.value)} />
</Row>
{loudness === "LoudNorm" && (
+31 -1
View File
@@ -9,7 +9,37 @@ WORKDIR /source
COPY design-system/. ./design-system/
COPY web/. ./web/
WORKDIR /source/web
RUN npm run lint && npm run typecheck && npm test -- --run && npm run build
# The SPA suite runs here except for two files (ersatztv#819), excluded for OVERLAPPING reasons —
# one needs the git binary, the other needs the binary AND a checkout. Reading them as two separate
# reasons is what broke this stage once already:
# * `web/src/api/pageSizeCallSites.guard.test.ts` needs a git CHECKOUT — and, through it, the
# binary. It derives its file population from `git ls-files` rather than a directory walk, and
# refuses rather than falling back. This stage's context is `web/` + `design-system/` only, so
# there is no `.git`.
# * `web/vite-plugins/trackedSourceFiles.realgit.test.ts` needs the BINARY but no checkout: it
# builds its own temp repository to prove the derivation by executing it.
# `node:22-bookworm-slim` ships no git (`command -v git` -> not found), so it dies with
# `spawnSync git ENOENT`.
# So this is NOT checkout-versus-binary, and adding a `COPY .git` would not let either run here —
# the binary would still be missing. Excluding only the first is not enough either, and a replica
# that merely deletes `.git` cannot show that: verify any change here with the git binary off
# `PATH`, not just with the directory absent.
# Everything else — all but those two files — runs fine gitless and is kept, rather than dropping
# the whole suite for one file as an earlier fix here did.
# The excluded pair is not skipped overall: `docker-build.yml`'s `Build & test (.NET)` job runs the
# whole suite on a real checkout, and `build` (the job that invokes this Dockerfile) carries
# `needs: [test, migrations, scan]`. State that chain precisely, because the `needs:` edge is not
# all of it: `Test SPA` is also gated on `docs_only` and on the #420 revalidate skip, and `build` is
# not gated on `revalidate`. On a push whose tree is byte-identical to an already-green head the
# suite is skipped and the image still builds — carried there by #420's byte-identical-tree
# argument. The `docs_only` arm cannot ship an image at all (`Build and push` is gated on it too),
# and `ci-detect-docs-only.sh` classifies by PATH SHAPE (`docs/` or `*.md`), not by directory, so a
# `web/*.md` would count as docs — there are none today, but do not restate it as "any `web/**`".
RUN npm run lint && npm run typecheck && \
npm test -- --run \
--exclude 'src/api/pageSizeCallSites.guard.test.ts' \
--exclude 'vite-plugins/trackedSourceFiles.realgit.test.ts' && \
npm run build
FROM --platform=linux/amd64 192.168.1.95:3000/timothy/ersatztv-ffmpeg:8.1.2 AS runtime-base
COPY --from=dotnet-runtime /usr/share/dotnet /usr/share/dotnet
+45
View File
@@ -20,13 +20,26 @@ doc below, or that changes which sections a task signal points to.**
| Named-issue pickup | Skip queue selection; go straight to focused retrieval — see "Knowledge retrieval" below, then the issue body |
| Adding/changing a `/api/*` endpoint | `docs/api-conventions.md` checklist + `docs/endpoint-index.md` |
| Adding a ChicoryTV SPA screen | `docs/spa-conventions.md` |
| Explaining a consequential settings field in the SPA (summary → hover/tap panel → docs link) | `docs/spa-conventions.md` §15 — use the shared `FieldHelp` trigger and put the copy in the screen's own `FIELD_HELP` record; the icon, the gesture and the a11y contract are fixed |
| Graphics element / overlay work (text bug, On Now / Next, watermark-vs-`[vge]`) | `docs/graphics-elements.md`, then decisions catalog rows keyed `graphics.*` |
| Scheduling / playout engine work | `docs/domain-model.md` + decisions catalog rows keyed `sched.*` (`docs/decisions/README.md`) |
| Adding or changing a paged list handler (a page plus a `TotalCount`) | Resolve `api.paged-count-matches-page-query` via `docs/decisions/README.md` — for an EF-backed filtered list, count the SAME query you page, with includes appended to the page chain only; where the count and the page are separate methods, a test pins their agreement. Then `api.paging-zero-based` for the `pageNum`/`pageSize` contract |
| Concurrency / optimistic-locking work | `docs/api-conventions.md` §7a/b/c + `docs/decisions/optimistic-concurrency.md` |
| Auth / security-surface work | `docs/decisions/api-auth-security.md` |
| CI / release pipeline work | `docs/ci-cd.md` + `docs/decisions/release-ci-governance.md` |
| Proposing a new guard / CI check / regression test convention | `docs/defect-shapes-773.md` §4 (detector menu + the classes where no detector is plausible), then the rules every guard must satisfy: `docs/decisions/records/testing/guard-derives-population-from-source.md`, `…/guard-ships-with-mutation-proof.md` and `…/mutation-claims-are-executed.md` (a `MUTATION` grade carries a DECLARED clause mutation that is re-run every suite) — plus `…/verification-code-needs-its-own-proof.md`, which extends the same obligation BEYOND guards to the harness, wrapper or checker doing the checking, and says where its proof lives when the checker holds no row |
| Adding or bounding a consequential numeric config field (an FFmpeg profile tunable, a pipeline knob) | `docs/api-conventions.md` §3d — reject out of range with a 422 naming the bound and its consequence, never accept-then-rewrite; validate against the constants the renderer reads, keep the render-time clamp for pre-existing rows, and let an UNCHANGED legacy value through on update. Then `api.ffmpeg-profile-numeric-bounds` |
| Testing a surface gated by config / an env var / a credential | `docs/decisions/records/testing/deny-path-at-production-config-value.md` — cover the setting absent, at its production value, and each opt-out, and assert the DENY branch |
| Touching a full-replace write path or a hand-built request object | `docs/decisions/records/testing/full-replace-asserts-field-list.md` — derive the field list from the DTO and assert set equality; reconcile by id where child state exists. In the SPA the same rule is enforced by the type system: `docs/spa-conventions.md` §4b — build the body as `Complete<T>`, annotating BOTH the wrapper parameter and every construction site |
| Writing or editing any doc, or answering a review finding in prose | `docs/decisions/records/docs/no-session-narrative.md` — the doc records the END STATE; the path to it goes in the commit message. Apply the who-benefits test, and read the carve-out before you cut (dated measurements, stated snapshot boundaries and tested-and-rejected results stay) |
| Adding, renaming or removing a workflow JOB | `docs/ci-cd.md` → "Per-job declarations" — every job declares `env.CI_JOB_ROLE` (and, in `docker-build.yml`, `env.CI_EXECUTION_CLASS`); a missing or unknown value fails `scripts/tests/test_workflow_job_guards.py` / `…/test_ci_image_pin_population.py`, and a `guard` **or `report-only`** job also needs a row in `docs/guard-inventory.md` → "Workflow-job guards". Rationale: `docs/decisions/records/testing/workflow-declares-its-own-job-metadata.md` |
| Adding / changing / deleting a guard file | `docs/guard-inventory.md` — every guard's row is machine-checked by `scripts/tests/test_guard_inventory.py`, so a new guard must acquire a row before the suite goes green, and a row graded `MUTATION` must also acquire a declared clause in `scripts/tests/mutation_manifest.py` |
| Writing code that reads live Gitea/remote state and then acts on it | `docs/decisions/records/process/check-and-use-pins-a-version.md`, then `docs/remote-state-inventory.md` — a new executable under `scripts/` (**excluding `scripts/tests/`**), `.claude/hooks/`, `.husky/` or `.gitea/workflows/` must acquire a row there before `scripts/tests/test_remote_state_inventory.py` goes green |
| Finding every site that references a symbol (multi-site fix/sweep) | `docs/local-lsp-tooling.md` — which of the three surfaces answers, and why a delegated agent must be pointed at an MCP server (`csharp-lsp`, or `serena` after an `activate_project`) rather than the `LSP` tool, which no subagent has been observed to reach |
| Live local run / Playwright-MCP verification | `docs/e2e-local.md` + `scripts/e2e-local.sh` |
| Adding/changing a UI-E2E browser flow | `docs/e2e-local.md` → "UI-E2E harness" + `scripts/e2e-ui.sh` |
| What does a test suite cover | `docs/testing.md` |
| Writing a test whose behaviour is PROVIDER-SPECIFIC (collation, a value converter, data-migration DML) | `docs/testing.md` → "Provider-parity fixtures (opt-in MySQL)" — run one fixture body against both providers via `ETV_TEST_MYSQL_CONNECTION`; without it the MySQL arm `Assert.Ignore`s visibly, and CI does not currently run it (ersatztv#627) |
| Legacy Blazor route lookup | `docs/blazor-route-parity.md` (historical #91 phase (b) inventory) |
| "Why do we do X this way" / challenging a convention | **Catalog-first**: `docs/decisions/README.md` (active rows) → follow the row's link to `docs/decisions/records/<area>/<topic>.md` for full rationale. `docs/decisions/archive/<area>/` only for "what did the rule used to be." |
@@ -66,6 +79,11 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K
- **`docs/spa-conventions.md`** — playbook for adding a screen to the ChicoryTV React SPA.
- **`docs/e2e-local.md`** (+ `scripts/e2e-local.sh`) — how to run a live local instance for manual
or Playwright-MCP verification.
- **`docs/local-lsp-tooling.md`** — the code-intelligence surfaces (the `LSP` tool's three servers,
the `csharp-lsp` MCP server, and `serena`): how each is configured, which ones a **subagent** can
actually reach, the traps (a cold server answers the first query with a confidently partial result;
serena needs an `activate_project` per directory), and `scripts/check-local-lsp.sh` to verify the
preconditions. Read before briefing an agent to find every site referencing a symbol.
- **`docs/testing.md`** — testing map: what each `*.Tests` project / `web` suite covers,
golden-file nets, the timezone-independence rule, how to run subsets, the per-PR verification
gate.
@@ -86,6 +104,10 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K
Largely superseded day-to-day by `docs/api-conventions.md`; read this for the original rationale.
- **`docs/mcp.md`** — the `ErsatzTV.Mcp` stdio JSON-RPC MCP server (#58): how it wraps `/api/v1` as
read + cautious-write tools, its config/env vars, auth, security posture, and the tool catalog.
- **`docs/graphics-elements.md`** — graphics element (overlay) schema reference: how elements are
discovered and attached, the YAML parsing traps (an unknown key disables the element outright), the
full text-element field table including the #732 background box, and why `[vge]` in a filter graph
does not imply a graphics element is bound.
- **`docs/channels.md`** — Channel entity field reference.
- **`docs/m3u-xmltv.md`** — M3U/XMLTV generation overview (`ChannelPlaylist`, `GetChannelGuideHandler`).
- **`docs/fork-strategy.md`** — divergence policy vs upstream ErsatzTV.
@@ -98,6 +120,29 @@ bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "K
that file's standing kickoff for the two concurrent tracks (orientation ‖ selection). ersatztv#237
is a closed, archival historical tracker (superseded by `startup.parallel-orientation` in
`docs/decisions.md`) — not a live pointer.
- **`docs/defect-shapes-773.md`** — root-cause analysis of the recurring defect shapes across the
whole closed-issue corpus (#773): the measured class ranking, the four families they consolidate
into, the cheapest mechanical detector per class, the classes where **no** detector is plausible,
and an audit of which configured hooks/MCP servers/LSPs are actually invoked. Read it before
proposing a new guard or CI check — §4 is the detector menu, and it argues against enumerating
cases one incident at a time.
- **`docs/remote-state-inventory.md`** — every executable in `scripts/` (**excluding
`scripts/tests/`**), `.claude/hooks/`, `.husky/` and `.gitea/workflows/` that reads live remote
state and acts on that read, classified `PINNED` / `CAS` / `UNSAFE-KNOWN` / `N/A` with the window
and what bounds it. Code outside those directories — C#/TypeScript guards, `web/`, and the test
suites themselves — is out of scope, and the doc states that rather than implying coverage.
The population is derived from `git ls-files` and compared for set equality by
`scripts/tests/test_remote_state_inventory.py`, so a new script that talks to a remote service
cannot ship unclassified. Read it with `process.check-and-use-pins-a-version`; it is that record's
detector, since the class has no plausible linter (`docs/defect-shapes-773.md` §4 detector D).
- **`docs/guard-inventory.md`** — every executable guard file, what it blocks, whether it is a
`GUARD` or `TOOLING`, and whether it ships a mutation proof (`MUTATION` / `BEHAVIOUR-ONLY` /
`NONE`) with a `file::function` ref. The population is derived from the GIT INDEX (not a
filesystem walk, since ersatztv#806) and the workflow/hook call sites, and compared for set
equality by `scripts/tests/test_guard_inventory.py`,
so a new guard cannot ship unclassified and a renamed test cannot leave a row claiming coverage it
has lost. Guards implemented inline in workflow YAML are deliberately outside that population —
the doc states the limit rather than implying coverage.
- **`docs/tracker-retrofit-triage-237.md`** — audit trail for the #524 triage of ersatztv#237's 111
comments (method, per-comment classification, totals). Evidence for the
`docs.tracker-comment-retrofit` decision; read it only when triaging another over-cap tracker.

Some files were not shown because too many files have changed in this diff Show More