Compare commits

..
77 Commits
Author SHA1 Message Date
timothy b52c938888 Merge pull request 'docs(release): v26.12.0 headline — ErsatzTV MCP server (#58) + external-logo download (#525)' (#531) from release/v26.12.0-notes into main
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build CI Toolchain Image / Build & push CI image (push) Successful in 2m36s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 6m20s
Build ErsatzTV Image / decisions lifecycle (push) Failing after 13m1s
Build ErsatzTV Image / Docs update reminder (push) Failing after 14m13s
Build ErsatzTV Image / CI image pin matches docker/ci (push) Failing after 14m14s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 19m15s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m25s
2026-07-21 14:49:42 +00:00
timothyandClaude Opus 4.8 6148805e37 docs(release): v26.12.0 headline — ErsatzTV MCP server (#58) + external-logo download (#525)
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 11s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 16s
Build ErsatzTV Image / decisions lifecycle (pull_request) Successful in 16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 11s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 12s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 35s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 35s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 36s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 16:47:20 +02:00
timothy 2834605d63 Merge pull request 'fix(529): floor QSV extra hardware frames so an unthrottled read can't exhaust the pool' (#530) from fix/529-qsv-extra-hw-frames-floor into main
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions lifecycle (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 34s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 35s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 36s
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
2026-07-21 14:41:15 +00:00
timothy 11b78bfcbd fix(529): round-two review — cover the handlers with tests, validate in the SPA [decisions-edit]
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 9s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 11s
Build ErsatzTV Image / decisions lifecycle (pull_request) Successful in 25s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m23s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m20s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m50s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m38s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m59s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Second review returned MERGEABLE with one Medium and three Lows. Addressed all four:

- Medium: the save-time normalization had zero test coverage, so a later refactor
  dropping Math.Max would leave the suite green (the FFmpegState floor keeps the
  pipeline correct, hiding the regression until someone reads a stored 0 back).
  Added Create/Update_Should_Floor_QsvExtraHardwareFrames over 0, -8, 63, 64 and 128,
  plus Create_Should_Leave_Null_QsvExtraHardwareFrames_Null for the null-passthrough
  branch, following the existing QsvPreferNativeDecoder tests' seed/handle/re-read
  shape. Negative-controlled: reverting both handlers fails exactly 5.

- Low: the SPA `min` was cosmetic. Input does forward it to the DOM, but there is no
  <form> — save is an onClick gated only on validate(), which had no branch for this
  field, so a typed 10 submitted fine and was silently changed to 64 with a 200 and no
  message. validate() now rejects it client-side.

- Low: the warning fires at the top of SetAccelState, before we know whether the
  pipeline uploads at all, so a fully-hardware path could be told "using 64 instead"
  when nothing consumed either value. Reworded to "will use ... wherever frames are
  uploaded".

- Low: recorded in the decision entry that the save-time normalization is
  unconditional on hardwareAcceleration (a non-QSV profile's stored value moves too),
  and that a client PUTting 0 reads back 64 — a transform the OpenAPI description does
  not advertise.

Verified in production, not just asserted. Set prod's profile to 64 (operator-approved)
and drove the exposed pipeline myself via the troubleshooting playback API on an mpeg4
.avi, which forces software decode + hwupload:

  hwupload=extra_hw_frames=64,vpp_qsv=w=1875:h=1080   exit 0, speed 12.0x, 0 ENOMEM

Then the negative control on prod's own hardware, same command, only the pool differing:

  extra_hw_frames=64 -> exit 0,   8 segments, 0 ENOMEM
  extra_hw_frames=0  -> exit 244, 0 segments, 3 ENOMEM

which reproduces the six overnight production failures and confirms the fix.

Full suite green: 4095 .NET, 891 web.

Refs #350, #516, #519.
2026-07-21 16:21:18 +02:00
timothy b27c950943 fix(529): address review — decision schema, #350 forward pointer, warning log, save-time normalization [decisions-edit]
Cold adversarial review returned BLOCKED on the documentation half. Addressed:

- The new decision record carried no lifecycle metadata block, taking the repo from
  82/82 to 83/82 and making it invisible to the by-key catalog lookup that #521
  established the same day. Added key/status/since/supersedes/superseded-by
  (ffmpeg.qsv-extra-hw-frames-floor) and regenerated docs/decisions/README.md;
  decisions_validate.py now reports OK with no legacy-unmigrated notice.
- The entry claimed to correct the #350 record but left that record untouched, so the
  stale "the burst is bounded" claim stayed authoritative for anyone resolving
  ffmpeg.hls-cold-start-burst. Added a forward-pointing correction note there (hence
  the [decisions-edit] token on this commit).
- The floor was applied silently. QsvPipelineBuilder.SetAccelState now logs a warning
  naming both the configured and applied value, because raising a deliberately small
  pool costs additional surfaces (64 NV12 1080p surfaces is roughly 190 MiB, 760 MiB
  at 4K) on memory-constrained iGPUs.
- Narrowed an overstated claim in the entry: 1..63 are untested, not known-bad. We
  raise them because the risk is a channel serving nothing, not because asking for
  less is illegitimate. Recorded as a deliberate over-reach with a stated cost.
- Corrected a factual error: SubtitleScaleQsvFilter also formats extra_hw_frames but
  is dead code with no construction site, so it is NOT covered by the guard.
- Config-vs-behavior mismatch: Create/UpdateFFmpegProfileHandler now normalize on
  save so stored rows converge on what the pipeline runs, and the SPA field carries
  min=64 rather than defaulting the display to 0. Render-time flooring is kept as the
  net that fixes existing deployments with no migration; the remaining gap for
  un-resaved rows is recorded as an accepted residual.
- Tests strengthened: pinned to the literal measured 64 rather than to the constant
  (so lowering the floor cannot quietly satisfy them), added a negative-value case,
  added a deinterlace-upload case, and replaced the narrow ShouldNotContain with a
  regex asserting EVERY extra_hw_frames occurrence in the command is >= the minimum.

Negative control re-run against the strengthened tests: reverting the floor fails 5,
with the build verified succeeded first. Full suite green (4086 .NET, 891 web).

Review finding that needed no change: the "single point" claim was independently
verified — no bypass exists, every FFmpegState construction routes through
MaybeQsvExtraHardwareFrames.

Refs #350, #516, #519.
2026-07-21 16:06:35 +02:00
timothy bbd7356f81 fix(529): floor QSV extra hardware frames so an unthrottled read can't exhaust the pool
A stored qsvExtraHardwareFrames of 0 reached FFmpeg as hwupload=extra_hw_frames=0,
leaving the QSV upload pool no headroom for frames in flight through the filter graph.
Any input that is not throttled then exhausts it: the graph fails with -12 (Cannot
allocate memory), h264_qsv reports "Could not open encoder before EOF", and zero
segments are written.

Measured against the deployed FFmpeg 8.1.2, one real logged command, only the marked
tokens differing:

  readrate 1.05, no burst   + frames 0  -> exit 0,   14 segments
  readrate 1.05 + burst 2/4/8 + frames 0 -> exit 244, ENOMEM, 0 segments
  no readrate at all        + frames 0  -> exit 244, ENOMEM, 0 segments
  readrate 1.05 + burst 8   + frames 64 -> exit 0,   14 segments
  no readrate at all        + frames 64 -> exit 0,   14 segments

So the defect predates #350's cold-start burst: a work-ahead start takes no -readrate
and was already failing on a profile with 0. The burst removed the throttle on every
realtime session, turning an intermittent failure into a near-deterministic one, which
is how it surfaced. Input throttling was doing load-bearing allocation-bounding work
that nobody had written down -- which is why the FFmpeg-level benchmark in #350 and the
argument-generation tests in #516 were both green and neither could see it.

Fixed at FFmpegState.QsvExtraHardwareFrames, the single point every QSV upload site
reads, so one guard covers HardwareUploadFilter, HardwareUploadQsvFilter,
WatermarkHardwareUploadFilter, ScaleQsvFilter, DeinterlaceQsvFilter and
SubtitleScaleQsvFilter rather than six call sites that can drift apart. Values above
the floor are still honored.

Tests negative-controlled: reverting the floor fails exactly the three sub-minimum
cases, with the build verified succeeded first.

Refs #350, #516, #519.
2026-07-21 15:52:57 +02:00
timothy 6b1bd9cf4b Merge pull request 'feat(525): external channel-logo URLs download + cache at save time' (#528) from feat/525-external-logo-download-on-save into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / decisions lifecycle (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 19s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 28s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 27s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 10m16s
2026-07-21 13:36:15 +00:00
timothy aa76994825 docs(525): correct logo-download rejection status to 422, not 400
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 8s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 12s
Build ErsatzTV Image / decisions lifecycle (pull_request) Successful in 24s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m21s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m21s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m34s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 13m59s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m49s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Live-E2E of the write path (scripts/e2e-local.sh) confirmed the feature end to
end — good URL -> cached /iptv/logos/<hash>, M3U emits the cached URL, cached logo
serves back as PNG, rejected save leaves the prior logo intact — and caught that a
bad/non-image/oversized logo rejects as 422 (BaseError -> ValidationProblemDetails
via ToErrorResult), not 400. 400 remains model-binding-only. Corrected channels.md,
api-conventions.md, and the graphics.channel-logo-caching decision Rule + catalog.
2026-07-21 13:19:07 +02:00
timothy 8930972a0a fix(525): address final-review findings (migration host-crash guard, multi-error 400s)
- Medium-1: wrap ExternalLogoMigratorService.ExecuteAsync in try/catch — a DB
  exception (e.g. a channel deleted mid-migration -> DbUpdateConcurrencyException)
  no longer trips BackgroundServiceExceptionBehavior.StopHost and kills the app;
  it logs and self-heals on the next boot. Caller-cancel path handled separately.
- Low-2: CreateChannelHandler/UpdateChannelHandler validation failure now returns
  errors.Join() (all accumulated errors) not errors.Head (first only), restoring
  the repo-wide convention; regression test added.
- Low-4: corrected the Startup registration comment (migrator self-awaits
  WaitForDatabase; order is not load-bearing).

Final whole-branch review: MERGEABLE @ 6d5f6b24 (fable). Carried Minors adjudicated
acceptable-defer.
2026-07-21 13:18:48 +02:00
timothy 56a18cd5dc docs(525): record download-on-save decision; update channels + api-conventions
New lifecycle record graphics.channel-logo-caching (supersedes: none — narrows
for YAML image elements). Regenerated the active catalog. channels.md + api-
conventions.md updated for the new save-time download + 400s.
2026-07-21 13:18:48 +02:00
timothy 3fb2145e50 feat(525): SPA previews saved logos, drops stale external-URL copy 2026-07-21 13:13:21 +02:00
timothy b54c06b5ec feat(525): startup migration converts existing URL logo rows to cache 2026-07-21 13:13:21 +02:00
timothy 36375157ad feat(525): render path no longer fetches a URL logo; degrades to no bug 2026-07-21 13:13:21 +02:00
timothy a5b4783b13 feat(525): budget-check direct artwork uploads (close the upload gap) 2026-07-21 13:13:21 +02:00
timothyandClaude Opus 4.8 93188eeb5e feat(525): download external-url logo on channel create + create-from-lineup
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 13:13:21 +02:00
timothy 1f21f0f30e feat(525): download external-url logo on channel update 2026-07-21 13:13:21 +02:00
timothy 5985bef577 feat(525): add RemoteLogoCacher (fetch + validate + cache a logo URL) 2026-07-21 13:13:21 +02:00
timothy 65a41dbf8d refactor(525): extract RemoteImageValidator; render path delegates to static 2026-07-21 13:13:21 +02:00
timothy 7ee436241a refactor(525): extract RemoteImageDecodeBudget from ImageElementBase 2026-07-21 13:13:21 +02:00
timothy 21a4e4d34e docs(plan): resolve IRemoteImageValidator layering (Core has no ImageSharp)
Core interface returns Task (throws-on-invalid); render path uses a static
Infra method returning Image. Drops the GraphicsEngine threading. refs #525
2026-07-21 13:13:21 +02:00
timothy 185de69930 docs(plan): external-logo download-on-save implementation plan
10 tasks, TDD, no schema change. Extracts RemoteImageDecodeBudget (Core) +
RemoteImageValidator (Infra), adds IRemoteLogoCacher, wires the 3 channel
handlers + UploadArtworkHandler, a startup migration for existing URL rows,
and the SPA preview/copy changes.

refs #525
2026-07-21 13:13:21 +02:00
timothy f1e7ee17ee docs(spec): resolve open questions — fold in upload budget check, keep fetcher namespace
refs #511
2026-07-21 13:13:21 +02:00
timothy 44199d3dba docs(spec): external channel-logo URLs become download-on-save
Design for reversing #511's render-time fetch: an external logo URL is
downloaded once at save time into the artwork cache (content-hash name,
same as an upload), validated against #511's decode budgets, and stored
so nothing downstream knows it came from a URL. Bad URL fails in the
editor, not at 3am. No refresh button — re-add the URL. One-time startup
migration converts existing URL rows; failures leave the row and warn.

refs #511 #502
2026-07-21 13:13:21 +02:00
timothy 063017584d Merge pull request 'feat(520,521): full decision-corpus migration + parallel-orientation startup rewrite (PR2 of 2)' (#527) from feat/520-521-migration-kickoff into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / decisions lifecycle (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 13s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 27s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 28s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m12s
2026-07-21 10:29:18 +00:00
timothyandClaude Opus 4.8 1b207db87c fix(520,521): clear stale migration-map + narrow convention-docs-session-start rule to the task-map protocol [decisions-edit]
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 13s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 16s
Build ErsatzTV Image / decisions lifecycle (pull_request) Successful in 37s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m22s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m20s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m19s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m22s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 12:04:35 +02:00
timothyandClaude Opus 4.8 fb6720ea27 fix(521): de-dup 6 overlapped records; guard duplicate metadata blocks; exclude retrieval-eval; complete eval bank [decisions-edit]
- Exclude docs/decisions/retrieval-eval.md from active decision parsing
  (_NON_DECISION_FILES); its `## N.` eval-question headings were being
  miscounted as 7 legacy-unmigrated records.
- Add decisions_lib.metadata_line_count() + a decisions_validate guard
  that fails a record with more than one `key:` metadata line, so a
  stacked-metadata-block migration bug (which the parser silently
  tolerated by reading only the first block) can't recur unnoticed.
  TDD: test_duplicate_metadata_block_fails / test_single_metadata_block_passes.
- De-duplicate the 6 docs/decisions.md records left with two stacked
  metadata blocks (scan.getoraddfolder-db-lookup #488,
  scan.musicvideo-reconciliation #494, scan.jellyfin-mixed-content-library
  #489, iptv.logo-drives-bug-preset #67, ffmpeg.qsv-decode-encode-split
  #498, ci.small-lane-git-only server-management#639), merging the union
  of Signals/paths/issues/Mechanics from both blocks and keeping the
  richer Rule wording; rationale prose untouched.
- Fill in the deferred Q6b row in docs/decisions/retrieval-eval.md now
  that startup.parallel-orientation is active in docs/decisions.md,
  scoring it as a real active-vs-superseded question against the
  archived docs.queue-state-gitea-tracker.
- Regenerate docs/decisions/README.md via build_decisions_catalog.py.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:54:52 +02:00
timothy 4ac18e06bc merge(520): parallel-orientation startup + #237 retirement [decisions-edit] 2026-07-21 11:49:32 +02:00
timothy 4a07dd4373 merge(521): retrieval-eval bank 2026-07-21 11:49:32 +02:00
timothyandClaude Opus 4.8 962dc2a31a feat(520): parallel orientation+selection startup; retire #237 as live state; #642 retrieval bullets [decisions-edit]
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:48:14 +02:00
timothyandClaude Opus 4.8 c3b2b4d4bd docs(521): retrieval-eval question bank (active-record selection + citation)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:39:20 +02:00
timothyandClaude Opus 4.8 1e79f62402 feat(521): migrate final 11 scheduling/CI gap records; corpus fully migrated [decisions-edit]
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:33:40 +02:00
timothy dbf842f8fb merge(521): migrate decisions.md range C tail [decisions-edit] 2026-07-21 11:30:25 +02:00
timothy ad814fec40 merge(521): migrate decisions.md range B [decisions-edit] 2026-07-21 11:30:14 +02:00
timothy 4d6ab394cd merge(521): migrate decisions.md range A [decisions-edit] 2026-07-21 11:30:14 +02:00
timothy f66c9b5c4b merge(521): migrate 4 topic files [decisions-edit] 2026-07-21 11:30:14 +02:00
timothyandClaude Opus 4.8 2e34d9c47b feat(521): migrate decisions.md range B: #385 through #502 decision records to lifecycle schema [decisions-edit]
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:29:05 +02:00
timothyandClaude Opus 4.8 aae2c418ad feat(521): migrate decisions.md range A: through #340 decision records to lifecycle schema [decisions-edit]
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:27:41 +02:00
timothyandClaude Opus 4.8 63a6fe80fb feat(521): migrate topic-file decision records to lifecycle schema [decisions-edit]
Adds metadata blocks (key/status/since/supersedes/superseded-by + Rule/Signals/Mechanics)
to all 26 legacy-unmigrated ## records across optimistic-concurrency.md (7, concurrency.*),
api-auth-security.md (9, security.*), release-ci-governance.md (7 remaining legacy records,
release.*), and spa-modularization.md (3, spa.*). Heading text and rationale prose are
byte-identical; only metadata was inserted, per the key-table assignments.

security.session-auth-dual-credential (#295 PR1) gets a one-line cross-reference noting it
narrows the OIDC-inert sub-claim of security.blazor-removal-auth-posture (#206), per
orchestrator adjudication — both stay status: active with supersedes/superseded-by: none.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:26:25 +02:00
timothyandClaude Opus 4.8 97062cc439 feat(521): migrate decisions.md range C: tail #488+/#350/#58/#511 decision records to lifecycle schema [decisions-edit]
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:25:50 +02:00
timothyandClaude Opus 4.8 ab2d80b47a fix(521): reciprocity covers archive<->archive pairs; skip topic-file Contents heading
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:14:00 +02:00
timothyandClaude Opus 4.8 7bdb83fb16 docs(520): carry #642 orientation bullets into PR2 plan (from a814dd16)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 11:11:01 +02:00
timothy f4bebd77b8 Merge pull request 'feat(521): decision-lifecycle machinery — keyed records, validator, generated catalog (PR1 of 2)' (#526) from feat/520-521-decision-lifecycle into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions lifecycle (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 15s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 28s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 9m33s
2026-07-21 09:08:51 +00:00
timothyandClaude Opus 4.8 46ea8f0e92 docs(521): carry archive<->archive reciprocity gap into PR2 Task 6 (PR1 whole-branch review, non-blocking)
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 12s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 21s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 22s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m22s
Build ErsatzTV Image / decisions lifecycle (pull_request) Successful in 1m20s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 17m44s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 21m47s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 25m34s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:00:15 +02:00
timothyandClaude Opus 4.8 d58b373463 docs(520): record MemPalace issue-discovery vs queue-state boundary + closing-record-as-artifact (PR2 retrieval contract)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:00:15 +02:00
timothyandClaude Opus 4.8 f93458c76c fix(521): whole-branch review — correct Gitea anchors, guard archive+demotion, reciprocal links, stale append-only refs, budget warning [decisions-edit]
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:00:15 +02:00
timothyandClaude Opus 4.8 b1e7e08884 feat(521): rework decisions guard — lifecycle validator + narrow [decisions-edit]; header + CI rewrite [decisions-edit]
Retire the line-level append-only mechanic (ersatztv#303 H9) in favor of the lifecycle validator
built in Tasks 1-5. .claude/hooks/decisions-guard.sh is now a thin fail-open shim around
scripts/decisions_validate.py; .husky/pre-commit calls it for the structural (working-tree) checks,
.husky/commit-msg drops the old staged/[decisions-edit]-deletion block and keeps only the
Co-Authored-By check. The Gitea decisions-guard job is renamed "decisions lifecycle" and now runs
decisions_validate.py --base/--head (structural + body-diff + no-vanish) and
build_decisions_catalog.py --check (active catalog drift), with actions/setup-python@v5 added since
the bare `small` lane doesn't guarantee python3; the old 1800-line consolidation-floor step is
removed (replaced by the validator's aggregate active-corpus budget). docs/decisions.md's header is
rewritten from append-only to lifecycle framing (metadata schema, statuses, generated catalog,
archive, same-PR supersession); [decisions-edit] is re-scoped (not removed) to rationale-prose
edits/factual corrections only. docs/ci-cd.md's release ritual and hook/job descriptions are
rewritten to match.

Also fixes a pre-existing validator false-positive surfaced while sanity-checking against
origin/main: Task 6's #303 H9/H3 split (commit d09be57e) renamed the archived record's heading
away from the pre-split original, which the validator's heading-based relocation check reads as
"removed without an archive copy." Restored the archived heading to match the original text
(functionally unchanged — still status: superseded, same key) and updated the two prose
cross-references (migration-map.md, release-ci-governance.md) that pointed at the old anchor.

PR1 scope only (per brief): does NOT wire the kickoff-guard CI step or touch the
kickoff/README/select-queue docs — that's Task 8/PR2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:00:15 +02:00
timothyandClaude Opus 4.8 417279c072 fix(521): split #303 H9/H3 bundle — H3 root-screenshot guard stays active, only H9 archived [decisions-edit]
The archived docs.append-only-guard record bundled the genuinely-superseded
H9 (decisions.md append-only) with the independent, still-active H3
(root-screenshot pre-commit guard), relocating both into archive/ and
burying an active decision in the history wing. Split into two records:
docs.append-only-guard stays in archive/release-ci-governance.md,
status: superseded, H3 content removed and replaced with a forward
pointer; ci.root-screenshot-guard is a new status: active record in the
active docs/decisions/release-ci-governance.md, with the H3 rationale
paragraph lifted byte-identical from the original bundle. Updated
migration-map.md (new row + a "Migration conventions" note: never
relocate an active decision to archive/) and regenerated
docs/decisions/README.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:00:15 +02:00
timothyandClaude Opus 4.8 eada44deb1 feat(521): reconcile #390/#406 + #411 + append-only supersession; seed exemplars [decisions-edit]
Migrate #406 (ci.runner-placement) and #412 (ci.peak-anon-measurement) to the lifecycle
schema as exemplars; #390 and #411 confirmed prose-only (no standalone record). Dogfood the
append-only->lifecycle supersession: docs.append-only-guard moved to archive/, superseded by
new active docs.decision-lifecycle. Stand up docs/decisions/archive/ + migration-map.md.

Refs #521

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 03:00:15 +02:00
timothyandClaude Opus 4.8 458ab2111f fix(521): catalog _anchor collapses punctuation runs; single trailing newline
_anchor() mapped each space/hyphen to its own '-' without collapsing runs,
so the standard heading separator " — " (space, em-dash, space) produced a
double hyphen in every generated anchor. Since nearly every real decision
record heading uses that separator, this made the catalog emit a dead link
for essentially every row. Fix: after building the char list, collapse
consecutive '-' into one and strip leading/trailing '-' via re.sub, matching
how Goldmark/GitHub/Gitea sluggers behave.

Also fixed main() writing an extra trailing newline (want already ends in
"\n", then "+ \n" appended a second one) so docs/decisions/README.md now
ends with exactly one trailing newline; --check still compares via .strip().

Added test_anchor_collapses_em_dash_and_keeps_underscore to pin the anchor
behavior against the reported iptv.base_url case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 67619b2bf6 feat(521): active-catalog generator
Adds scripts/build_decisions_catalog.py, which renders docs/decisions/README.md
as a compact table of only 'active' decision records (sorted by key), and its
test scripts/tests/test_build_catalog.py. Supports --check for CI drift
detection. No decision records are migrated yet, so the generated catalog is
currently empty (banner + header only) — expected at this stage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 bdcc59ff80 test(521): add non-vacuous append-smuggle regression test for _rationale bound
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 7b8ac751d7 fix(521): bound _rationale strip window; restore REQUIRED_META; fail-open _run; archive-placement invariant
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 9266437d68 feat(521): decision lifecycle validator
Adds scripts/decisions_validate.py: lifecycle invariant checks (unique
active key, key format, reciprocal supersession, removed-without-archive,
rationale-rewrite-without-token, catalog staleness, corpus budget) plus
git-diff helpers for merge-base-based CI checks. Deviates from the task
brief in one spot: REQUIRED_META narrowed to (key, status) — the brief's
(key, status, since, supersedes, superseded_by) makes its own
test_clean_corpus_passes fail, since since/supersedes/superseded_by
default to None on bare Record() instances built without going through
the markdown parser.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 428ebc0c81 docs(520,521): record #642 MemPalace conventions + archive-placement/bounded-strip invariants
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 2cd78aa0bf style(521): lint/type cleanup on decisions_lib parser + test
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 ba39ca65ae feat(521): decision-record parser (decisions_lib)
Adds scripts/decisions_lib.py, the shared parser for ErsatzTV decision
records (docs/decisions.md + docs/decisions/*.md). Parses H2 sections
into Record dataclasses, distinguishing migrated records (visible
metadata block: key/status/since/supersedes/superseded-by + Rule/
Signals/Mechanics) from legacy-unmigrated ones with no metadata line.

scripts/ is now an importable package (scripts/__init__.py,
scripts/tests/__init__.py) so later tools can `import scripts.decisions_lib`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 404ec95479 docs(520,521): fold Fable review — body-diff guard, two-PR split, key/backstop fixes
Refs #520 #521

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 0637fa46ac docs(520,521): implementation plan — decision lifecycle + startup rewrite
Refs #520 #521

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothyandClaude Opus 4.8 2af3cd024e docs(520,521): design spec — decision lifecycle + retrieval-efficient startup
Design for the decision-knowledge lifecycle schema/catalog/validator (#521) and
the #237-retirement / parallel-startup kickoff rewrite (#520), as one arc.

Refs #520 #521

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 02:58:58 +02:00
timothy be0e2b1f1f Merge pull request 'fix(511): bound remote graphics-engine image fetches (timeout, size cap, decode cap, redirects, pooling)' (#518) from fix/511-remote-image-hardening into main
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 12s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 26s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 26s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m4s
2026-07-20 23:47:20 +00:00
timothy 5616fa6de5 fix(511): don't let the header pre-pass break animated PNG logos
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 11s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 15s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 13s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m21s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m22s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m38s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m24s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m42s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Fourth adversarial pass cleared the security design — all three earlier
bypasses are dead, DecoderOptions.MaxFrames is honored by every decoder
that can produce multiple frames (GIF/WebP/TIFF exactly N, APNG N-1),
and it bounds PEAK allocation, not just the final frame count (measured:
65 MiB capped vs 2.41 GiB uncapped on the same 600-frame GIF).

But it caught a functional regression this PR introduced: a *default*
`Image.IdentifyAsync` throws InvalidImageContentException on most APNGs
that `Image.Load` reads back perfectly — including files ImageSharp's
own PngEncoder wrote. Reproduced independently: 13 of 16 shapes throw,
and `MaxFrames = 1` on the Identify fixes all 16 with dimensions intact.
Since #502 routes ordinary channel-logo watermarks through this path, an
admin with an animated PNG logo would have silently lost their watermark
to a log line — a hardening change breaking working content.

The existing tests could not see it: they use 64x64, which happens to be
one of the few shapes a default Identify handles. Now pinned with a
288x288 shape that asserts the default Identify DOES fail and that
DecodeRemoteImage decodes it anyway, in full.

Also, from the same pass:
- document the REAL enforced peak (up to 3x the nominal 50 MP budget,
  since detecting "over the limit" means decoding past it) instead of
  restating the nominal number. Tightening the single-frame allowance to
  budget/3 would reject legitimate 8K stills, so the overshoot is
  deliberate; it is ~600 MB against the ~36 GiB it replaces
- correct the MaxFrames off-by-one claim: N-1 is APNG-specific, not
  universal, so the stated rationale for +2 was wrong for three of the
  four animated formats
2026-07-21 01:28:13 +02:00
timothy d4e112f1e9 fix(511): bound the DECODER, not the header's frame count
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 15s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 20s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m34s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m15s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 18m39s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Second adversarial re-review defeated the product budget too, and the
mechanism generalizes: the budget was enforced on a number the decoder
does not honor.

Measured on ImageSharp 3.1.12 (reproduced independently before fixing):

  600-frame APNG  ->  Identify: FrameMetadataCollection.Count = 0
                      Load:     Frames.Count = 600

So EnsureDecodeAffordable(w, h, 0) charged Math.Max(0,1) = 1 frame —
the most permissive possible reading. A 4000x4000 x600 APNG is ~134 KiB
on the wire, is charged 16 MP, and decodes to ~36 GiB: 2.5x worse than
the GIF the previous commit exists to stop, at half the wire size. The
retention budget could not backstop it — that runs after LoadAsync, so
the process OOMs first, killing every concurrent stream.

GIF, WebP and TIFF report honestly; PNG/APNG is the sole divergence,
which is the point: you cannot audit every format, so the header cannot
be the source of truth.

DecodeRemoteImage now:
- checks header DIMENSIONS only (trustworthy; a GIF image descriptor
  exceeding its logical screen is clamped by the decoder, verified)
- derives how many frames of that size the budget affords
- passes that to DecoderOptions.MaxFrames, which the DECODER enforces
  whatever the header claimed. Measured: MaxFrames = N yields N-1
  frames, so it asks for affordable + 2 — decoding one more than allowed
  is what distinguishes "at the limit" from "over it" without silently
  truncating a legitimate animation
- re-verifies the real image.Frames.Count after decoding, disposing and
  rejecting if over

Also adds wiring coverage for the retention budget (M4): deleting its
call site now fails a test — negative-controlled, build verified before
trusting the result.

docs/decisions.md records both failed attempts, because the lesson is
the generalizable part: independent caps do not compose into a budget,
and a limit the decoder does not enforce is not a limit.
2026-07-21 01:04:25 +02:00
timothy 9a2096f340 fix(511): budget decode by the PRODUCT, not by independent caps
Adversarial re-review of the first fix defeated its decode guard with a
measured payload: a 2500x2500 x600-frame GIF is ~60 KiB on the wire,
passes the 50 MP dimension check (6.25 MP) AND the 600-frame check
(exactly 600), and costs ~14 GiB to decode — strictly worse than the
30000x30000 PNG the guard was added to stop, at 1/60th the wire size.
Checking dimensions and frames independently never bounded the decode.

- decode budget is now width x height x frames <= 50 MP, as one product;
  a zero frame count is charged as one so an unenumerable header cannot
  zero it out
- new retention budget: frames x scaledWidth x scaledHeight <= 200 MP.
  Independent of the decode budget in both directions — a 100x100 source
  is trivial to decode but retains ~5 GB of SKBitmap once every frame is
  scaled to 1920x1080, since LoadImage clones and resizes each frame to
  output resolution and keeps them
- both budgets are pure functions (EnsureDecodeAffordable,
  EnsureScaledFramesAffordable) so the arithmetic is tested at every
  boundary without materializing multi-gigabyte images
- the frame guard had NO coverage before; it does now
- fail loudly on a non-seekable fetcher stream instead of letting
  Position throw NotSupportedException into the blanket catch
- test the copy over-read against the ACTUAL rented buffer length
  (ArrayPool.Rent(81920) returns 131072), not the requested 81920

docs/decisions.md corrected: it claimed the byte cap bounded the
decode-bomb surface and that the header check closed the class. Both
overstated. An append-only file that is confidently wrong is worse than
one with a gap.
2026-07-21 01:04:25 +02:00
timothy e132c422bb fix(511): bound remote graphics-engine image fetches
`ImageElementBase.LoadImage` fetched http(s) images with a throwaway
`new HttpClient()` + `GetStreamAsync`: no timeout override (the 100s
default), no size cap, unbounded redirects, no pooling — all inside
stream startup, while ffmpeg waits on the pipe. #502 routed ordinary
channel-logo watermarks onto that path, widening a pre-existing weakness.

Introduce `IRemoteImageFetcher` / `HttpRemoteImageFetcher`, modelled on
the neighbouring `IRemoteStreamProber`:

- deadline covers headers AND body (linked CTS + `CancelAfter`, client
  `Timeout = InfiniteTimeSpan`) — under `ResponseHeadersRead` the body
  read falls outside `HttpClient.Timeout` (the #289 lesson)
- 10 MiB cap enforced during the copy; `Content-Length` is only a cheap
  early reject, since it can be absent or a lie
- permissive content-type check (rejects an HTML error page, allows a
  missing type and octet-stream)
- pooled via `IHttpClientFactory`; redirects capped at 3, not 50

A byte cap does NOT bound decoding, so `DecodeRemoteImage` additionally
reads declared dimensions + frame count from the header and rejects
before `Image.LoadAsync` allocates (50 MP / 600 frames). A 4 KB PNG
declaring 30000x30000 costs ~3.6 GB to decode and passes every wire-size
check — caught by adversarial review of the first version of this change,
which capped bytes and wrongly claimed that was decode-bomb protection.

Not cached and SSRF not mitigated — both deliberate, with the reasoning
recorded in docs/decisions.md.

fixes #511
2026-07-21 01:04:25 +02:00
timothy d97e1dece2 Merge pull request 'feat(#58): ErsatzTV.Mcp — read + cautious-write MCP server over /api/v1' (#517) from feat/58-mcp-v1 into main
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 11s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 29s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 30s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 13m18s
2026-07-20 23:02:36 +00:00
timothyandClaude Opus 4.8 a88240dcec feat(#58): ErsatzTV.Mcp — read + cautious-write MCP server over /api/v1
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 11s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 17s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 16s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 19s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 6m20s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m38s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m29s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Fresh stdio JSON-RPC MCP server wrapping the frozen /api/v1 surface,
superseding the closed read-only PR #76. 26 read tools (six families +
search/all-items & search/artists discovery) and cautious-write CRUD:
collections (incl. idempotent membership adds for #487), smart collections,
schedules, playouts, channels (create/update/delete/reset), and a
Jellyfin-focused media-source sync/scan slice. Writes gated behind
ERSATZTV_ALLOW_WRITES (default false, runtime-enforced).

Security baseline carried forward from PR #76/#289: read-only backstop,
JSON-RPC DoS guards + bounded stdin reader, per-request CTS over
headers+body, response-size cap, arg validation vs InputSchema,
reverse-proxy prefix preservation. Machine-key auth (X-Api-Key,
CSRF-exempt). If-Match/ETag round-trip for the one replace-all PUT that
honors it.

Cold-review fixes folded in:
- HIGH: reject control chars (CR/LF) in the ifMatch value before it reaches
  TryAddWithoutValidation — SocketsHttpHandler writes it verbatim, so a
  crafted value could smuggle headers onto the X-Api-Key request.
- Cache the empty-args JsonDocument (no per-call pooled-doc leak).
- Accept explicit JSON null for optional fields so a nullable API field
  (e.g. dailyRebuildTime) can be cleared as documented.

Deferred (documented): the ~40-field replace-list writes and redesign
workflow tools (#63-#68).

Docs: docs/mcp.md, docs/README.md index, docs/decisions.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 00:42:00 +02:00
timothy 34d76095a2 Merge pull request 'fix(350): burst-read the first HLS segments so cold start isn't readrate-bound' (#516) from fix/350-coldstart-seek into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 27s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 12s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 25s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 12m24s
2026-07-20 22:33:39 +00:00
timothy 0e3b0ca309 test(350): harden the burst tests against silent regressions
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 8s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 11s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 16s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 5m54s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 8s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 6s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m13s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 20m32s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Re-review of the fix commit flagged two ways the new tests could pass
vacuously in future: the burst assertion was a bare substring (satisfiable
by any input carrying the option) and the still-image test asserted only an
absence. Anchor the first on the input path plus an occurrence count, and
give the second a positive anchor.
2026-07-21 00:07:35 +02:00
timothy 1359bb6135 fix(350): burst-read the first HLS segments so cold start isn't readrate-bound
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 11s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 16s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 15s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m19s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 5m52s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m14s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m12s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
`-readrate 1.05` paces input reading at wall clock so a channel behaves like
live TV, but it applies from the very first read. With 4s HLS segments and the
segmenter waiting for the first one, the playlist could not appear sooner than
~4/1.05 = 3.8s, so every tune-in that did not win a work-ahead slot paid a
multi-second wait.

Add `-readrate_initial_burst` (FFmpeg >= 6.1) next to `-readrate` on the normal
playback path, gated on runtime capability detection via the existing
`FFmpegKnownOption`/`HasOption` machinery, whose option list had simply been
empty. Measured on real prod media: time-to-first-playlist 5369/5344ms ->
648/649ms.

Root cause detail: the cold-start bimodality earlier rounds could not explain
was never about the media. `HlsSessionWorker` grants an unthrottled start only
while `_workAheadCount < work_ahead_limit` (prod: 1), so concurrent tune-ins
fall back to the throttled path. Confirmed on prod with three concurrent tunes:
firstGop 866ms for the slot winner vs 3845ms and 6357ms. This also falsifies the
issue's ranked #1 driver — accurate-seek decode-discard measures 30-100ms on
real media, and probe caps 20-50ms; neither can account for seconds.

Still images are excluded: their video input is paced by the realtime filter and
takes no readrate, so a burst would only run a song's separate audio input ahead
of the video. Concat/WrapSegmenter keep the unburst single-arg constructor.

fixes #350
2026-07-20 23:59:42 +02:00
timothy 64de59376f Merge pull request 'fix(502): render the on-screen bug for external-URL channel logos' (#513) from fix/502-external-url-watermark into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 5m49s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 13m47s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 18m8s
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 4m2s
2026-07-20 21:33:50 +00:00
timothy 99b1ccadcc Merge pull request 'docs(release): Komodo stack is jazz-media; no auto-update fallback' (#514) from docs/komodo-stack-name into main
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 12s
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 11s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 27s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Build & push image (amd64) (push) Successful in 8s
2026-07-20 21:07:09 +00:00
timothy 7248416bd6 docs(release): Komodo stack is jazz-media; no auto-update fallback
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 10s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 11s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 13s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 20s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 21s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 17s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 20s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 16s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Verified live while promoting v26.11.0. Two corrections:

1. `DeployStack media-servers` targets a DEAD stack. The Komodo stack name
   changed to `jazz-media` with the move to jazz; the compose PROJECT is still
   `media-servers` (which is what container labels show, so the labels don't
   catch this). A `media-servers` stack still exists on bumblebee in state
   `unhealthy` — the stopped migration leftovers — so the documented command
   silently deploys the wrong, dead thing.

2. There is no Global Auto Update fallback: `jazz-media` has
   auto_update=false (poll_for_updates=true only). Promotion is manual, full
   stop, and the 'don't cut a tag near the 03:00 run' caveat is obsolete.

The pre-deploy safety chain is intact and jazz-aware (#635) — verified by
reading the deploy's Pre Deploy stage: image-change trigger fired, 286M backup
with integrity_check=ok plus off-box PBS, migration smoke PASS against the
prod-copy, then only ersatztv recreated.
2026-07-20 23:04:34 +02:00
timothy 66448e1abf fix(502): correct the deco-scoping claim, extract + test the routing guard
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 13s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 13s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 14s
Build CI Toolchain Image / Build & push CI image (push) Successful in 1m39s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 17s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 16s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m56s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 15m2s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 17m48s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Adversarial review found a documentation defect, not a code one: both
docs/decisions.md and the WatermarkSelector comment asserted the deco path
was unaffected by this change. That is true of the *resolution* half and
false of the *routing* half. SelectWatermarks puts deco-derived options
into the same list the routing guard filters, so a deco watermark whose
resolved path is a URL is rerouted to the graphics engine too — including
the generated-initials localhost URL, which only the deco path still emits
and which plausibly rendered through ffmpeg before.

That reroute is intended (routing by what the path is beats routing by
provenance, which would drift), so the fix is to say so accurately rather
than to narrow the guard. Also records the accepted per-frame cost
asymmetry the entry previously argued on correctness grounds alone.

The guard is extracted as CanUseFFmpegNativeWatermark so it can be tested
directly — review's highest-value gap was that the half of the fix which
decides whether pixels appear had no automated coverage, only the one-off
live E2E. Nine cases pin it, including the localhost-fallback reroute.

Both deferrals now point at real issues instead of an unverifiable
"tracked separately": #510 (deco vs precedence-level missing-logo policy)
and #511 (remote-fetch hardening — timeout, size cap, redirects, pooling,
caching, SSRF).

Also pins scheme-case insensitivity in the selector.
2026-07-20 23:00:27 +02:00
timothy f9bd245158 fix(502): render the on-screen bug for external-URL channel logos
A channel whose logo is an external URL never rendered a watermark, even
with an ImageSource=ChannelLogo watermark attached. WatermarkSelector
resolved the URL correctly and then existence-checked it on the
filesystem — File.Exists("https://…") is always false — so all three
precedence levels (playout item, channel, global) logged "Channel logo
no longer exists" and returned None. The channel editor advertises the
URL as winning over an uploaded logo, which was true for the guide
listing and silently false for the bug.

External artwork passes through rather than being downloaded into the
image cache: that is already the convention everywhere else (M3U, XMLTV,
SPA JSON all emit the raw URL), no fetch->SaveArtworkToCache glue exists,
and the render path does not need it — ImageElementBase.LoadImage already
fetches an http(s) path with HttpClient and decodes it for real pixel
dimensions.

A remote-URL watermark is therefore forced onto the graphics engine
instead of the ffmpeg-native shortcut, which would otherwise hand the URL
to ffprobe and ffmpeg as a bare -i argument, putting an unbounded network
fetch inside stream startup.

The three gated precedence levels now share one ChannelLogoWatermarkOptions
helper — the triplicated block is what let the defect exist three times
over. Scope held narrow: the generated-initials localhost fallback (#1)
stays disabled behind an explicit comment and a scope-guard test, and the
deco path keeps its own long-standing unchecked policy.

Verified by live-E2E against a real channel playout with an external-URL
logo: origin/main renders 0 logo pixels and logs the "no longer exists"
warning verbatim; this branch renders the logo in the expected region.

Whitespace-only reformatting in FFmpegLibraryProcessService.cs is the
fix-as-you-touch format gate on pre-existing violations, plus a BOM strip.

fixes #502
2026-07-20 23:00:27 +02:00
timothy 64414be1ec Merge pull request '#508: move the two docker build jobs off the small lane' (#509)
Build ErsatzTV Image / CI image pin matches docker/ci (push) Has been skipped
Build ErsatzTV Image / Docs update reminder (push) Has been skipped
Build ErsatzTV Image / decisions.md append-only (push) Has been skipped
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (push) Has been skipped
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (push) Has been skipped
Build ErsatzTV Image / Functional E2E (curl contracts) (push) Successful in 33s
Build ErsatzTV Image / Build & test (.NET) (push) Successful in 34s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (push) Successful in 35s
Build CI Toolchain Image / Build & push CI image (push) Successful in 2m6s
Build ErsatzTV Image / Build & push image (amd64) (push) Has been cancelled
ci(508): move both `docker build` jobs off the `small` runner lane

Fixes #508. `small` is now git-only (ci-image-pin, docs-reminder,
decisions-guard); both heavy `build` jobs (docker-build image push +
ci-image toolchain buildx) move to ubuntu-latest so the lane can widen
to 4 slots across two hosts while committing less RAM than its single
10 GiB slot did. #574's skip-task queueing can't recur (build keeps
needs: [test, migrations]). Runner-side half: server-management#639.
2026-07-20 20:56:19 +00:00
timothyandClaude Opus 4.8 5709bf5a2c ci(508): bump the CI toolchain pin to the image this branch published
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Successful in 11s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 11s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 14s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 5m50s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 22s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 16s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 5m18s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 7m20s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
Editing .gitea/workflows/ci-image.yml is itself a trigger path for ci-image.yml, so
the previous commit republished the toolchain image at its own sha. `ci-image-pin`
then correctly failed: the pin still named 07048b8 while
`git log -1 -- docker/ci .gitea/workflows/ci-image.yml` resolved to 4263cf79.

The image content is unchanged — that commit only moved the job's `runs-on:` — but
the pin's contract is "the pin equals the last commit that touched the image
sources", not "the bytes differ", so it has to move. Verified 4263cf7 is actually
published to the registry before pinning it; a pin that doesn't resolve to a real
image would break every `container:` job at once.

This is why the bump lands as a SEPARATE commit: the tag is the short sha of the
pushed branch tip, so a single commit could never contain its own sha. Splitting it
makes the branch self-consistent — the source-touching commit stays the last one to
touch those paths, and this one carries the matching pin.

refs #508

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 22:29:56 +02:00
timothyandClaude Opus 4.8 4263cf7919 ci(508): move both docker build jobs off the small lane
Build ErsatzTV Image / CI image pin matches docker/ci (pull_request) Failing after 9s
Build ErsatzTV Image / decisions.md append-only (pull_request) Successful in 14s
Build ErsatzTV Image / Docs update reminder (pull_request) Successful in 14s
Build CI Toolchain Image / Build & push CI image (push) Successful in 2m37s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 7s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 5s
Build ErsatzTV Image / Functional E2E (curl contracts) (pull_request) Successful in 14m54s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 19m11s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 19m27s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
`runs-on: small` carried two jobs that are not small: docker-build.yml's image
build, and ci-image.yml's toolchain buildx. The second reads as lightweight
because it is "docker-only, no toolchain needed — it *builds* the toolchain",
which is true and yet describes the heaviest job in the lane.

A lane's per-job memory cap is set by its worst member, not its median, so these
two pinned `small` at --memory=10g. On bumblebee's 25 GiB — also the prod media
host — that permits exactly ONE slot, and four jobs shared it. So "widen the
lane" and "keep the heavy jobs" were never simultaneously available.

The symptom that forces the issue is not queue wait. A saturated lane also wedges
DISPATCHED jobs in act's setup phase: >10 min in_progress, no log file written at
all, then failure, before Checkout runs. That is where "decisions.md is a known
flake, just rerun it" came from — the rerun works only because it lands after load
clears, so a capacity problem read as a bug in the guard.

With both builds on ubuntu-latest, `small` is a checkout plus a `git diff` and
server-management#639 caps it at 1 GiB, widening it to 4 slots across two hosts
while committing LESS RAM to CI than the single slot did.

so it cannot be dispatched until the jobs it would queue behind have finished.

refs #508

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 22:24:12 +02:00
103 changed files with 11158 additions and 680 deletions
+11 -53
View File
@@ -1,54 +1,12 @@
#!/usr/bin/env bash
# ersatztv#303 H9 — docs/decisions.md is append-only. This blocks a commit / PR that DELETES or
# MODIFIES an existing line of that file; pure INSERTIONS anywhere are always allowed (adding a new
# entry inserts a TOC line near the top AND appends a block at the bottom — both are insertions, so
# numstat reports 0 deleted lines). A genuine factual fix to a past entry is the one legitimate edit:
# put the literal token [decisions-edit] in the commit message to override.
#
# Fail-open: any tooling trouble (unknown mode, non-numeric numstat, missing refs) -> allow. The point
# is to catch the accidental rewrite-history case, never to wedge a legitimate commit.
#
# Assumes decisions.md ends with a trailing newline (it does; .editorconfig enforces it). If that final
# newline were ever dropped, git would render the next append as a modify of the last line (deleted=1)
# and this would false-block the append until the author adds [decisions-edit] — cheap and self-correcting.
#
# Modes:
# staged <msgfile> pre-commit/commit-msg — staged diff vs HEAD; trailer read from <msgfile>
# range <base> <head> CI (PR) — merge-base diff base...head; trailer scanned across base..head msgs
set -euo pipefail
FILE="docs/decisions.md"
mode="${1:-}"
case "$mode" in
staged)
deleted=$(git diff --cached --numstat -- "$FILE" 2>/dev/null | awk '{print $2}' | head -1)
msg=$(cat "${2:-/dev/null}" 2>/dev/null || true)
;;
range)
base="${2:-}"; head="${3:-}"
[ -n "$base" ] && [ -n "$head" ] || exit 0 # missing refs -> fail-open
deleted=$(git diff --numstat "$base...$head" -- "$FILE" 2>/dev/null | awk '{print $2}' | head -1)
msg=$(git log --format='%B' "$base..$head" 2>/dev/null || true)
;;
*)
exit 0 # unknown mode -> fail-open
;;
esac
# Empty (no change to the file) or '-' (binary) -> treat as 0 (fail-open / nothing to guard).
deleted="${deleted:-0}"
case "$deleted" in ''|*[!0-9]*) deleted=0 ;; esac
[ "$deleted" -gt 0 ] || exit 0 # pure insertion / no change -> allow
# Explicit override for a documented factual fix.
if printf '%s' "$msg" | grep -qiF '[decisions-edit]'; then
exit 0
fi
{
echo "decisions-guard (ersatztv#303 H9): docs/decisions.md is append-only — this change deletes/modifies ${deleted} existing line(s)."
echo " Append new entries at the bottom (plus a TOC line in the Index); do not rewrite settled entries."
echo " To fix a genuine factual error in a past entry, add the token [decisions-edit] to the commit message."
} >&2
exit 1
# ersatztv#521 — the line-level append-only mechanic is retired. Decision integrity is now enforced by
# the lifecycle validator. `[decisions-edit]` survives ONLY for rationale-prose edits (validator
# body-diff, CI). This shim runs the structural validator over the working tree; the body-diff/no-
# vanish checks run in CI where a base/head is available. Fail-open on any tooling trouble.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)" || exit 0
command -v python3 >/dev/null 2>&1 || exit 0 # no python -> fail-open
PYTHONPATH=. python3 scripts/decisions_validate.py
rc=$?
[ "$rc" -eq 1 ] && exit 1 # only a real validation failure blocks
exit 0 # crashes/other codes -> fail-open
+8 -3
View File
@@ -53,9 +53,14 @@ env:
jobs:
build:
name: Build & push CI image
# `small` = the small-jobs runner lane. This is a docker-only job (no toolchain needed —
# it *builds* the toolchain), same as docker-build.yml's `build` job.
runs-on: small
# Moved off `small` with docker-build.yml's `build` (server-management#639). Being
# "docker-only" made it look lightweight, but it is a full buildx of the .NET
# toolchain image — the heaviest thing that ran in that lane. `small` is now
# git-only and capped at 1g per job, which would OOM this build.
#
# Rare trigger (pushes touching docker/ci + a weekly cron), so it costs the
# ubuntu-latest lane almost nothing, and ci-runner (.127) runs no prod workload.
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
+37 -30
View File
@@ -25,7 +25,7 @@ name: Build ErsatzTV Image
# cannot read the workflow `env` context. **Bump all five together**; see docs/ci-cd.md ->
# "CI toolchain image" for the two-step procedure.
#
# CI image pin: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
# CI image pin: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7
#
# DOCS-ONLY SKIP (ersatztv#416): a change that touches only docs/** or *.md has nothing for the
# heavy jobs to validate. `test`, `migrations`, `functional-e2e` and `build` each run
@@ -103,7 +103,7 @@ jobs:
name: Build & test (.NET)
runs-on: ubuntu-latest
container:
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -258,7 +258,7 @@ jobs:
name: EF migration integrity (SQLite + MySql)
runs-on: ubuntu-latest
container:
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -417,7 +417,7 @@ jobs:
# v* tag builds.
if: github.event_name == 'pull_request' || github.ref == 'refs/heads/main'
container:
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -496,12 +496,20 @@ jobs:
build:
name: Build & push image (amd64)
# `small` = the dedicated small-jobs runner lane (server-management#574).
# On PR runs this job only resolves its skip, but Gitea still dispatches it
# as a task — on the ubuntu-latest runners that skip queued behind long
# builds (observed 31 min). Real builds (main/tags) run on bumblebee,
# capped at 4 CPUs / 10g.
runs-on: small
# Moved back off `small` (server-management#639). This is the one HEAVY job that
# was still in that lane, and its 10g requirement was what pinned the lane's
# per-job cap at 10g — which in turn capped the lane at ONE slot on a 25 GiB
# host. Four jobs sharing one slot is what starved the git-only checks in act's
# setup phase (>10 min, no logs, then fail). With this job gone, `small` is
# git-only and can run wide and tiny on two hosts.
#
# The `ubuntu-latest` queueing that sent it to `small` in the first place
# (server-management#574: a PR-run skip stuck 31 min behind long builds) does not
# come back, because `needs: [test, migrations]` means this job cannot be
# dispatched until those two have already finished — by which point the lane it
# was queueing behind has drained. Real builds (main/tags) get the full
# ubuntu-latest allotment: 4 CPUs / 10g on ci-runner (.127).
runs-on: ubuntu-latest
needs: [test, migrations]
if: github.event_name != 'pull_request'
steps:
@@ -718,12 +726,14 @@ jobs:
echo "Parity-doc reminder: nothing to flag."
fi
# BLOCKING (ersatztv#303 H9): docs/decisions.md is an append-only log. Fails a PR that deletes or
# rewrites a settled entry (numstat reports >0 deleted lines) unless a commit in the range carries
# the [decisions-edit] override token for a documented factual fix. Same script the Husky commit-msg
# hook calls, so local and CI enforcement can't drift. Seconds-long git diff -> keep it off the build runners.
# BLOCKING (ersatztv#521, supersedes the ersatztv#303 H9 append-only mechanic): validates decision-
# record lifecycle invariants (metadata schema, one active record per key, reciprocal
# supersedes/superseded-by links, no rationale-prose rewrite without [decisions-edit], no record
# vanishing from the active set without an archive copy) and that the generated active catalog
# (docs/decisions/README.md) is in sync. Same validator the Husky pre-commit hook shim calls, so
# local and CI enforcement can't drift. Seconds-long git diff + parse -> keep it off the build runners.
decisions-guard:
name: decisions.md append-only
name: decisions lifecycle
runs-on: small
if: github.event_name == 'pull_request'
steps:
@@ -731,22 +741,19 @@ jobs:
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Enforce append-only
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.x'
- name: Validate decision lifecycle
run: |
base_ref="${{ github.base_ref }}"
git fetch --no-tags --depth=200 origin "$base_ref" || true
./.claude/hooks/decisions-guard.sh range "origin/${base_ref}" HEAD
- name: Consolidation-floor reminder (non-blocking)
run: |
# Consolidation is primarily a release step; this is the between-releases floor. The metric is
# the file's LINE COUNT — the context an agent actually burns reading the log — not entry count.
# Floor 1800 keeps the whole log inside one default 2000-line Read (headroom for the reader's
# own overhead). Nudge (never fail) past it so append-only can't grow past what agents can read.
n=$(wc -l < docs/decisions.md | tr -d ' ')
echo "docs/decisions.md is ${n} lines (consolidation floor: 1800; one Read caps at 2000)."
if [ "${n:-0}" -gt 1800 ]; then
echo "::warning::docs/decisions.md is ${n} lines (>1800) — larger than agents can comfortably read in one pass. Do a consolidation pass (prune/merge superseded entries with [decisions-edit]); don't wait for the next release. See the decisions.md header."
fi
PYTHONPATH=. python3 scripts/decisions_validate.py --base "origin/${base_ref}" --head HEAD
- name: Active catalog in sync
run: PYTHONPATH=. python3 scripts/build_decisions_catalog.py --check
- name: Kickoff guard
run: bash scripts/check-kickoff-guard.sh
# BLOCKING (unlike docs-reminder): the mechanizable half of the "docs-update in the
# same PR" rule for the API contract (ersatztv#303 H4/H5). If a PR touches the API
@@ -777,7 +784,7 @@ jobs:
# 48 GiB at capacity 4 + a bumblebee overflow slot), which fixes the queue at the source.
runs-on: ubuntu-latest
container:
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -872,7 +879,7 @@ jobs:
# move to a lighter lane is a server-management capacity call (#604).
runs-on: ubuntu-latest
container:
image: 192.168.1.95:3000/timothy/ersatztv-ci:07048b8
image: 192.168.1.95:3000/timothy/ersatztv-ci:4263cf7
credentials:
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
-5
View File
@@ -8,8 +8,3 @@ grep -q '^Co-Authored-By:' "$1" || {
echo 'husky - commit message missing Co-Authored-By trailer'
exit 1
}
# H9 (ersatztv#303) — docs/decisions.md is append-only. Block a commit that rewrites a settled
# entry unless the message carries [decisions-edit]. commit-msg runs after the index is final, so
# the staged diff is what's being committed; the message file ($1) supplies the override token.
./.claude/hooks/decisions-guard.sh staged "$1" || exit 1
+6
View File
@@ -1,6 +1,12 @@
cd web && npx lint-staged || exit 1
cd ..
# ersatztv#521 — decision-record lifecycle structural validator (replaces the old H9 append-only
# line guard). Runs the same validator the CI `decisions lifecycle` job uses, over the working
# tree (no base/head here, so only structural checks run; the body-diff/no-vanish checks run in
# CI where a base ref exists). Fail-open shim — see .claude/hooks/decisions-guard.sh.
./.claude/hooks/decisions-guard.sh || exit 1
# H3 (ersatztv#303) — never commit a screenshot dropped at the repo root. Belt-and-suspenders with
# .gitignore (catches a forced `git add -f`). Root-level *.png only; nested paths are legit assets.
root_png=$(git diff --cached --name-only --diff-filter=ACM | grep -iE '^[^/]+\.png$' || true)
+17 -4
View File
@@ -38,7 +38,7 @@ Custom IPTV channel server for Jellyfin. Forked from [ErsatzTV/ErsatzTV](https:/
- **Docker host**: **jazz (192.168.1.29)**, container `ersatztv`, port 8409. Media transcoders (Jellyfin, `ersatztv`, `ersatztv-test`) moved here from bumblebee on 2026-07-20 (server-management#633); bumblebee (192.168.1.99) still hosts the **CI runners** and the rest of the stacks. **Name-reuse trap**: `jazz` was an *earlier* name for the .99 host, so pre-2026-07-20 docs/commits saying "jazz" mean today's **bumblebee** — go by the IP, not the name.
- **Config volume**: `~/downloadswarm/ersatztv/` on jazz → `/config` in container
- **SQLite DB**: `/config/ersatztv.sqlite3` (WAL mode, root-owned)
- **Images** (our fork, built by `.gitea/workflows/docker-build.yml``192.168.1.95:3000/timothy/ersatztv`): push to `main``:latest` + `:<sha>` (test image); push `v*` tag → `:prod` + `:<version>` + `:<sha>`. Prod's **Komodo GitOps** `media-servers` stack follows floating `:prod`; after the immutable `:<version>` candidate passes the release scans, manually deploy the stack (Global Auto Update is the daily fallback). Both paths run the fail-closed pre-deploy backup and prod-copy migration smoke before recreation. Test tracks `:latest`. Pipeline details: `docs/ci-cd.md`.
- **Images** (our fork, built by `.gitea/workflows/docker-build.yml``192.168.1.95:3000/timothy/ersatztv`): push to `main``:latest` + `:<sha>` (test image); push `v*` tag → `:prod` + `:<version>` + `:<sha>`. Prod's **Komodo GitOps** stack — named **`jazz-media`** (the compose *project* is still `media-servers`; a dead `media-servers` stack lingers on bumblebee) — follows floating `:prod`; after the immutable `:<version>` candidate passes the release scans, manually `DeployStack jazz-media`. There is **no** auto-update fallback (`auto_update: false`) — promotion is manual. Both paths run the fail-closed pre-deploy backup and prod-copy migration smoke before recreation. Test tracks `:latest`. Pipeline details: `docs/ci-cd.md`.
## Development
@@ -56,7 +56,7 @@ docker build -f docker/Dockerfile -t ersatztv:dev .
## Conventions
- **Read [`docs/contributing.md`](docs/contributing.md)** before non-trivial changes — it documents the established patterns (layering, CQRS handlers, LanguageExt, the ChicoryTV SPA, EF Core + dual-provider migrations, the FFmpeg pipeline, analyzers, testing) and the **deviation policy**: match the established style; diverge only with a concrete, stated reason.
- **Docs-first is a HARD RULE — read before you explore**: before ANY API / SPA / E2E / parity / scheduling work, read `docs/README.md` (index) → the convention docs (`api-conventions`, `spa-conventions`, `e2e-local`, `domain-model`, `blazor-route-parity`, `decisions`). **Do NOT reverse-engineer conventions from source (Grep/Read) before reading these** — they exist precisely so you don't. Only recon the task-specific delta the docs deliberately don't freeze (a merged endpoint's exact DTO, a Blazor page's field list). **This applies to delegated subagents too**: tell each agent which doc section to read; never let one re-derive conventions from code.
- **Docs-first is a HARD RULE — read before you explore**: before ANY API / SPA / E2E / parity / scheduling work, read the `docs/README.md` **task-signal map** and only the sections it points to for your task — not the whole corpus. **Do NOT reverse-engineer conventions from source (Grep/Read) before reading these** — they exist precisely so you don't. Only recon the task-specific delta the docs deliberately don't freeze (a merged endpoint's exact DTO, a Blazor page's field list). **This applies to delegated subagents too**: tell each agent which doc section to read; never let one re-derive conventions from code. **Decision/convention lookups start at the active catalog**, `docs/decisions/README.md` — resolve by topic/key, never by chasing a file path named in a historical comment (the breadcrumb rule; see `docs/README.md` → "Knowledge retrieval").
- **Docs-update is part of "done" — same PR, never a follow-up**: any PR that changes a convention, adds/migrates/redirects a route, adds/changes a `/api/*` endpoint, or reverses a decision MUST update the relevant doc in that same PR:
| Change | Update in the same PR |
@@ -64,7 +64,7 @@ docker build -f docker/Dockerfile -t ersatztv:dev .
| Migrate / add / redirect a route (new `web/src/screens/*.tsx`, `LegacyUiRedirects.cs`) | `docs/blazor-route-parity.md` + `docs/domain-model.md` |
| Add / change a `/api/*` endpoint | `docs/api-conventions.md` checklist, then regenerate `v1.json` + `endpoint-index.md` via `./scripts/update-openapi.sh` |
| Change a SPA screen convention | `docs/spa-conventions.md` |
| Establish / reverse a convention or decision | `docs/decisions.md` (append-only) + the affected doc |
| Establish / reverse a convention or decision | `docs/decisions.md` (lifecycle: add record, relocate predecessor to archive/) + the affected doc |
| Add / remove / retitle a doc | `docs/README.md` index |
The `docs-reminder` CI job flags a screen/route change that skips `blazor-route-parity.md`, but it's a **non-blocking** nudge — the rule is on you, not the check.
@@ -91,11 +91,24 @@ Both need Gitea read creds in the env to enforce (**`ETV_GITEA_BASICAUTH=user:pa
1. **Root cause** (bug fixes / incidents only): Document WHY the problem existed, not just what was changed. If root cause is unknown, say so explicitly and open a follow-up investigation issue. Fixing symptoms without understanding causes creates recurring problems.
2. **Comment on issues** as you work — what you found, what approach you're taking, any deviations from the suggested fix.
3. **Push changes**: `git push` all commits before closing. Use `fixes #N` in commit messages to auto-close where appropriate.
4. **Close comment**: Add a structured closing comment on the issue covering: what was done, root cause (if applicable), files changed, anything deferred, follow-up issues created, and which docs were updated.
4. **Close comment**: Add a structured `## Closing record` comment on the issue (template below).
5. **Close the issue** via API or `fixes #N` commit. Leave open with a comment only if partially addressed.
6. **Update docs**: If the change affects operational behavior, update the relevant Obsidian docs (`~/homelab-docs/`), MEMORY.md, or CLAUDE.md inline — not as a follow-up.
7. **Reply to reviewer** (if from adversarial review): Summary of done/deferred/questions. This triggers the next review cycle.
**`## Closing record` template** (step 4 — this is both the human-readable summary and the per-issue unit MemPalace mines for retrieval; see `docs/handoffs/chicorytv-issue-queue.md` → "Knowledge retrieval" for the retrieval contract this feeds):
```markdown
## Closing record
**Outcome:** <what shipped / what didn't; PR link>
**Root cause:** <for bug fixes/incidents — why the problem existed, or "unknown, see follow-up #N">
**Decisions/conventions changed:** <keys added/superseded in docs/decisions.md, or "none">
**Reusable knowledge:** <a fact/gotcha worth surfacing to a future session or MemPalace search>
**Verification:** <tests run, live-E2E, CI status>
**Deferred:** <anything explicitly punted, with a follow-up issue link, or "none">
**Docs updated:** <which docs/*.md files changed in this PR, or "none required and why">
```
## Project Boundaries
**ersatztv OWNS**: ErsatzTV fork code (C#/.NET), channel/collection/schedule management, M3U/XMLTV generation, the ErsatzTV skill in server-management.
@@ -9,8 +9,13 @@ namespace ErsatzTV.Application.Artworks;
public class UploadArtworkHandler : IRequestHandler<UploadArtwork, Either<BaseError, ArtworkUploadResponseModel>>
{
private readonly IImageCache _imageCache;
private readonly IRemoteImageValidator _validator;
public UploadArtworkHandler(IImageCache imageCache) => _imageCache = imageCache;
public UploadArtworkHandler(IImageCache imageCache, IRemoteImageValidator validator)
{
_imageCache = imageCache;
_validator = validator;
}
public async Task<Either<BaseError, ArtworkUploadResponseModel>> Handle(
UploadArtwork request,
@@ -38,6 +43,22 @@ public class UploadArtworkHandler : IRequestHandler<UploadArtwork, Either<BaseEr
string contentType = maybeContentType.IfNone(string.Empty);
// One rule: anything entering the logo cache is decode-budget-checked. A supported format is
// not enough — a small header can declare a multi-gigabyte canvas (a decompression bomb), so
// reject it here before it lands in the cache. The synthetic upload:// Uri is only for the
// exception message text. (ersatztv#525)
using (var probe = new MemoryStream(bytes, writable: false))
{
try
{
await _validator.Validate(probe, new Uri("upload://artwork"), cancellationToken);
}
catch (Exception ex)
{
return BaseError.New($"Image cannot be used: {ex.Message}");
}
}
using var toCache = new MemoryStream(bytes, writable: false);
Either<BaseError, string> maybeFileName = await _imageCache.SaveArtworkToCache(
toCache,
@@ -8,6 +8,7 @@ using ErsatzTV.Core.Api.LibraryBrowse;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Core.Scheduling;
using ErsatzTV.Infrastructure.Data;
@@ -21,6 +22,7 @@ public class CreateChannelFromLineupHandler(
ChannelWriter<IBackgroundServiceRequest> workerChannel,
IDbContextFactory<TvContext> dbContextFactory,
ISearchTargets searchTargets,
IRemoteLogoCacher remoteLogoCacher,
ILogger<CreateChannelFromLineupHandler> logger)
: IRequestHandler<CreateChannelFromLineup, Either<BaseError, CreateChannelFromLineupResponseModel>>
{
@@ -37,7 +39,42 @@ public class CreateChannelFromLineupHandler(
Either<BaseError, PreparedCreate> validation = await Validate(dbContext, request, cancellationToken);
return await validation.Match(
Left: error => Task.FromResult<Either<BaseError, CreateChannelFromLineupResponseModel>>(error),
Right: prepared => PersistAndDispatch(dbContext, prepared, cancellationToken));
Right: async prepared =>
{
Either<BaseError, PreparedCreate> resolved =
await ResolveExternalLogo(request, prepared, cancellationToken);
return await resolved.Match(
Left: error => Task.FromResult<Either<BaseError, CreateChannelFromLineupResponseModel>>(error),
Right: p => PersistAndDispatch(dbContext, p, cancellationToken));
});
}
// The lineup logo artwork is built (in BuildChannel) with the raw request path. When that path is
// an external http(s) URL, download + cache it and swap the cache name onto the logo artwork before
// persisting (a cacher Left fails the whole create); a blank or already-local/cached path is left
// unchanged. (ersatztv#525)
private async Task<Either<BaseError, PreparedCreate>> ResolveExternalLogo(
CreateChannelFromLineup request,
PreparedCreate prepared,
CancellationToken cancellationToken)
{
string path = request.Logo?.Path ?? string.Empty;
if (!Artwork.IsExternalUrl(path))
{
return prepared;
}
Either<BaseError, string> cached = await remoteLogoCacher.CacheFromUrl(new Uri(path), cancellationToken);
return cached.Map(name =>
{
foreach (Artwork logo in prepared.Channel.Artwork.Where(a => a.ArtworkKind == ArtworkKind.Logo))
{
logo.Path = name;
}
return prepared;
});
}
private async Task<Either<BaseError, CreateChannelFromLineupResponseModel>> PersistAndDispatch(
@@ -1,9 +1,10 @@
using System.Globalization;
using System.Globalization;
using System.Text.RegularExpressions;
using System.Threading.Channels;
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
@@ -16,7 +17,8 @@ namespace ErsatzTV.Application.Channels;
public class CreateChannelHandler(
ChannelWriter<IBackgroundServiceRequest> workerChannel,
IDbContextFactory<TvContext> dbContextFactory,
ISearchTargets searchTargets)
ISearchTargets searchTargets,
IRemoteLogoCacher remoteLogoCacher)
: IRequestHandler<CreateChannel, Either<BaseError, CreateChannelResult>>
{
public async Task<Either<BaseError, CreateChannelResult>> Handle(
@@ -25,7 +27,52 @@ public class CreateChannelHandler(
{
await using TvContext dbContext = await dbContextFactory.CreateDbContextAsync(cancellationToken);
Validation<BaseError, Channel> validation = await Validate(dbContext, request, cancellationToken);
return await validation.Apply(c => PersistChannel(dbContext, c));
return await validation.Match(
Succ: async channel =>
{
Either<BaseError, string> resolvedLogo = await ResolveLogoPath(request, cancellationToken);
return await resolvedLogo.Match(
Right: async logoPath =>
{
ApplyResolvedLogo(request, channel, logoPath);
return Right<BaseError, CreateChannelResult>(await PersistChannel(dbContext, channel));
},
Left: e => Task.FromResult(Left<BaseError, CreateChannelResult>(e)));
},
Fail: errors => Task.FromResult(Left<BaseError, CreateChannelResult>(errors.Join())));
}
// Resolve the incoming logo path into a value safe to persist. An external http(s) URL is
// downloaded and cached (a cacher Left fails the whole save); an empty path or an
// already-local/cached path passes through unchanged. (ersatztv#525)
private async Task<Either<BaseError, string>> ResolveLogoPath(
CreateChannel request,
CancellationToken cancellationToken)
{
string path = request.Logo?.Path ?? string.Empty;
if (!Artwork.IsExternalUrl(path))
{
return path;
}
Either<BaseError, string> cached = await remoteLogoCacher.CacheFromUrl(new Uri(path), cancellationToken);
return cached;
}
// When the incoming logo was an external URL, swap the downloaded cache name onto the logo
// artwork built during validation so no URL is ever persisted in Artwork.Path. (ersatztv#525)
private static void ApplyResolvedLogo(CreateChannel request, Channel channel, string resolvedLogoPath)
{
if (!Artwork.IsExternalUrl(request.Logo?.Path ?? string.Empty))
{
return;
}
foreach (Artwork logo in channel.Artwork.Where(a => a.ArtworkKind == ArtworkKind.Logo))
{
logo.Path = resolvedLogoPath;
}
}
private async Task<CreateChannelResult> PersistChannel(TvContext dbContext, Channel channel)
@@ -6,6 +6,7 @@ using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
@@ -19,7 +20,8 @@ namespace ErsatzTV.Application.Channels;
public class UpdateChannelHandler(
ChannelWriter<IBackgroundServiceRequest> workerChannel,
IDbContextFactory<TvContext> dbContextFactory,
ISearchTargets searchTargets)
ISearchTargets searchTargets,
IRemoteLogoCacher remoteLogoCacher)
: IRequestHandler<UpdateChannel, Either<BaseError, ChannelViewModel>>
{
public async Task<Either<BaseError, ChannelViewModel>> Handle(
@@ -39,17 +41,45 @@ public class UpdateChannelHandler(
{
Validation<BaseError, Channel> validation =
await Validate(dbContext, request, channel, cancellationToken);
return await validation.Apply(c => ApplyUpdateRequest(dbContext, c, request, cancellationToken));
return await validation.Match(
Succ: async c =>
{
Either<BaseError, string> resolvedLogo = await ResolveLogoPath(request, cancellationToken);
return await resolvedLogo.Match(
Right: async logoPath => Right<BaseError, ChannelViewModel>(
await ApplyUpdateRequest(dbContext, c, request, logoPath, cancellationToken)),
Left: e => Task.FromResult(Left<BaseError, ChannelViewModel>(e)));
},
Fail: errors => Task.FromResult(Left<BaseError, ChannelViewModel>(errors.Join())));
},
None: () => Task.FromResult(
Left<BaseError, ChannelViewModel>(
new NotFoundError($"Channel {request.ChannelId} does not exist."))));
}
// Resolve the incoming logo path into a value safe to persist. An external http(s) URL is
// downloaded and cached (a cacher Left fails the whole save); an empty path (logo removal) or an
// already-local/cached path passes through unchanged. (ersatztv#525)
private async Task<Either<BaseError, string>> ResolveLogoPath(
UpdateChannel request,
CancellationToken cancellationToken)
{
string path = request.Logo?.Path ?? string.Empty;
if (!Artwork.IsExternalUrl(path))
{
return path;
}
Either<BaseError, string> cached = await remoteLogoCacher.CacheFromUrl(new Uri(path), cancellationToken);
return cached;
}
private async Task<ChannelViewModel> ApplyUpdateRequest(
TvContext dbContext,
Channel c,
UpdateChannel update,
string resolvedLogoPath,
CancellationToken cancellationToken)
{
bool hasEpgChange = c.PlayoutSource != update.PlayoutSource || c.ShowInEpg != update.ShowInEpg;
@@ -76,9 +106,9 @@ public class UpdateChannelHandler(
c.ShowInEpg = update.IsEnabled && update.ShowInEpg;
c.Artwork ??= [];
if (!string.IsNullOrWhiteSpace(update.Logo?.Path))
if (!string.IsNullOrWhiteSpace(resolvedLogoPath))
{
string logo = update.Logo.Path;
string logo = resolvedLogoPath;
if (logo.StartsWith("iptv/logos/", StringComparison.Ordinal))
{
logo = logo.Replace("iptv/logos/", string.Empty);
@@ -2,6 +2,7 @@ using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.FFmpeg;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
using Microsoft.EntityFrameworkCore;
@@ -67,7 +68,11 @@ public class CreateFFmpegProfileHandler :
HardwareAcceleration = hwAccel,
VaapiDriver = request.VaapiDriver,
VaapiDevice = request.VaapiDevice,
QsvExtraHardwareFrames = request.QsvExtraHardwareFrames,
// store what the pipeline will actually use, never a pool size FFmpegState would
// floor away at render time (ersatztv#529)
QsvExtraHardwareFrames = request.QsvExtraHardwareFrames is { } frames
? Math.Max(frames, FFmpegState.MinimumQsvExtraHardwareFrames)
: null,
ResolutionId = resolutionId,
ScalingBehavior = request.ScalingBehavior,
@@ -3,6 +3,7 @@ using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.FFmpeg;
using ErsatzTV.FFmpeg.Preset;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Infrastructure.Extensions;
@@ -54,7 +55,11 @@ public class UpdateFFmpegProfileHandler(IDbContextFactory<TvContext> dbContextFa
p.VaapiDisplay = update.VaapiDisplay;
p.VaapiDriver = update.VaapiDriver;
p.VaapiDevice = update.VaapiDevice;
p.QsvExtraHardwareFrames = update.QsvExtraHardwareFrames;
// store what the pipeline will actually use, so a profile doesn't keep displaying a pool
// size that FFmpegState floors away at render time (ersatztv#529)
p.QsvExtraHardwareFrames = update.QsvExtraHardwareFrames is { } frames
? Math.Max(frames, FFmpegState.MinimumQsvExtraHardwareFrames)
: null;
p.ResolutionId = update.ResolutionId;
p.ScalingBehavior = update.ScalingBehavior;
p.PadMode = update.PadMode;
@@ -0,0 +1,77 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Core.Tests.FFmpeg;
/// <summary>
/// Pins which watermarks ffmpeg may carry natively and which must go to the graphics engine.
/// The remote-URL rule is the second half of the #502 fix: resolving the URL is useless if the
/// resolved path is then handed to ffmpeg as a bare <c>-i</c> argument.
/// </summary>
[TestFixture]
public class FFmpegNativeWatermarkRoutingTests
{
private const string LocalPath = "/cache/logos/ab/abc123.png";
private static WatermarkOptions Options(
string imagePath,
ChannelWatermarkMode mode = ChannelWatermarkMode.Permanent) =>
new(new ChannelWatermark { Id = 1, Name = "wm", Mode = mode }, imagePath, Option<int>.None);
[Test]
public void Local_Path_Single_Permanent_Watermark_Uses_FFmpeg()
{
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark(0, [Options(LocalPath)])
.ShouldBeTrue();
}
[TestCase("https://cdn.example.com/logos/channel.png")]
[TestCase("http://cdn.example.com/logos/channel.png")]
public void Remote_Url_Watermark_Goes_To_Graphics_Engine(string url)
{
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark(0, [Options(url)])
.ShouldBeFalse();
}
/// <summary>
/// The generated-initials fallback is a localhost URL. Only the deco path still emits it, and it is
/// routed by its resolved path like any other URL — see the #502 entry in docs/decisions.md and #510.
/// </summary>
[Test]
public void Generated_Localhost_Logo_Url_Goes_To_Graphics_Engine()
{
FFmpegLibraryProcessService
.CanUseFFmpegNativeWatermark(0, [Options("http://localhost:8409/iptv/logos/gen?text=Test")])
.ShouldBeFalse();
}
[Test]
public void Graphics_Elements_Present_Goes_To_Graphics_Engine()
{
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark(1, [Options(LocalPath)])
.ShouldBeFalse();
}
[Test]
public void Multiple_Watermarks_Go_To_Graphics_Engine()
{
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark(0, [Options(LocalPath), Options(LocalPath)])
.ShouldBeFalse();
}
[Test]
public void No_Watermarks_Does_Not_Use_FFmpeg()
{
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark(0, []).ShouldBeFalse();
}
[TestCase(ChannelWatermarkMode.Intermittent)]
[TestCase(ChannelWatermarkMode.None)]
public void Non_Permanent_Watermark_Goes_To_Graphics_Engine(ChannelWatermarkMode mode)
{
FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark(0, [Options(LocalPath, mode)])
.ShouldBeFalse();
}
}
@@ -0,0 +1,207 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.Interfaces.FFmpeg;
using ErsatzTV.Core.Interfaces.Images;
using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using Testably.Abstractions.Testing;
namespace ErsatzTV.Core.Tests.FFmpeg;
/// <summary>
/// Covers <see cref="ChannelWatermarkImageSource.ChannelLogo" /> resolution at all three watermark
/// precedence levels (playout item, channel, global). The shared fixture in
/// <see cref="WatermarkSelectorTests" /> deliberately makes every watermark file exist, so it cannot
/// express the "logo is an external URL" or "logo file is gone" cases this fixture exists for (#502).
/// </summary>
[TestFixture]
public class WatermarkSelectorChannelLogoTests
{
private const string ExternalLogoUrl = "https://cdn.example.com/logos/channel.png";
private const string LocalLogoPath = "abc123.png";
private const string LocalLogoCachePath = "/cache/logos/ab/abc123.png";
private WatermarkSelector _selector;
[SetUp]
public void SetUp()
{
var mockFileSystem = new MockFileSystem();
mockFileSystem.Initialize().WithFile(LocalLogoCachePath);
var fakeImageCache = Substitute.For<IImageCache>();
fakeImageCache.GetPathForImage(Arg.Any<string>(), Arg.Is(ArtworkKind.Logo), Arg.Any<Option<int>>())
.Returns(_ => LocalLogoCachePath);
_selector = new WatermarkSelector(
mockFileSystem,
fakeImageCache,
Substitute.For<IDecoSelector>(),
NullLogger<WatermarkSelector>.Instance);
}
private static ChannelWatermark ChannelLogoWatermark(int id, string name) =>
new()
{
Id = id,
Name = name,
ImageSource = ChannelWatermarkImageSource.ChannelLogo,
Mode = ChannelWatermarkMode.Permanent
};
private static Channel ChannelWithLogo(string logoPath, ChannelWatermark channelWatermark = null)
{
var channel = new Channel(Guid.Empty)
{
Id = 1,
Number = "1",
Name = "Test",
StreamingMode = StreamingMode.TransportStream,
Artwork = [],
Watermark = channelWatermark,
WatermarkId = channelWatermark?.Id
};
if (logoPath is not null)
{
channel.Artwork.Add(new Artwork { ArtworkKind = ArtworkKind.Logo, Path = logoPath });
}
return channel;
}
// ---- external URL logo: render path must degrade to no bug, never fetch (#525) --------------
//
// As of #525 an external-URL logo is downloaded and cached at save time, so a URL path can only be a
// row that failed migration. The render/watermark path must NOT fetch at compositing time: it degrades
// to None (no on-screen bug) with a warning, rather than handing the URL downstream as a renderable
// ImagePath (the #502 behavior these tests previously pinned).
[Test]
public void PlayoutItemWatermark_Should_Ignore_External_Url_Channel_Logo()
{
ChannelWatermark watermark = ChannelLogoWatermark(1, "PlayoutItem");
Channel channel = ChannelWithLogo(ExternalLogoUrl);
Option<WatermarkOptions> result = _selector.GetWatermarkOptions(
channel,
watermark,
Option<ChannelWatermark>.None);
result.IsNone.ShouldBeTrue();
}
[Test]
public void ChannelWatermark_Should_Ignore_External_Url_Channel_Logo()
{
ChannelWatermark watermark = ChannelLogoWatermark(2, "Channel");
Channel channel = ChannelWithLogo(ExternalLogoUrl, watermark);
Option<WatermarkOptions> result = _selector.GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
Option<ChannelWatermark>.None);
result.IsNone.ShouldBeTrue();
// never hand the URL downstream as a renderable path
result.IfSome(o => o.ImagePath.ShouldNotBe(ExternalLogoUrl));
}
[Test]
public void GlobalWatermark_Should_Ignore_External_Url_Channel_Logo()
{
ChannelWatermark watermark = ChannelLogoWatermark(3, "Global");
Channel channel = ChannelWithLogo(ExternalLogoUrl);
Option<WatermarkOptions> result = _selector.GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
watermark);
result.IsNone.ShouldBeTrue();
}
/// <summary>
/// Scheme comparison goes through <see cref="Uri" />, which lower-cases it. Pinned because the fix
/// turns on <c>Artwork.IsExternalUrl</c>, and a case-sensitive check would silently fall back to the
/// existence-gated branch and re-introduce the defect for an oddly-cased URL.
/// </summary>
[Test]
public void ChannelWatermark_Should_Ignore_External_Url_Channel_Logo_Regardless_Of_Scheme_Case()
{
const string UpperCaseUrl = "HTTPS://cdn.example.com/logos/channel.png";
ChannelWatermark watermark = ChannelLogoWatermark(2, "Channel");
Channel channel = ChannelWithLogo(UpperCaseUrl, watermark);
Option<WatermarkOptions> result = _selector.GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
Option<ChannelWatermark>.None);
result.IsNone.ShouldBeTrue();
}
// ---- regressions: local-file behavior must not change ---------------------------------------
[Test]
public void ChannelWatermark_Should_Use_Cached_Path_For_Local_Channel_Logo()
{
ChannelWatermark watermark = ChannelLogoWatermark(2, "Channel");
Channel channel = ChannelWithLogo(LocalLogoPath, watermark);
Option<WatermarkOptions> result = _selector.GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
Option<ChannelWatermark>.None);
result.IsSome.ShouldBeTrue();
result.IfNone(() => throw new InvalidOperationException()).ImagePath.ShouldBe(LocalLogoCachePath);
}
[Test]
public void ChannelWatermark_Should_Be_Ignored_When_Local_Channel_Logo_File_Is_Missing()
{
var mockFileSystem = new MockFileSystem(); // nothing on disk
var fakeImageCache = Substitute.For<IImageCache>();
fakeImageCache.GetPathForImage(Arg.Any<string>(), Arg.Is(ArtworkKind.Logo), Arg.Any<Option<int>>())
.Returns(_ => LocalLogoCachePath);
var selector = new WatermarkSelector(
mockFileSystem,
fakeImageCache,
Substitute.For<IDecoSelector>(),
NullLogger<WatermarkSelector>.Instance);
ChannelWatermark watermark = ChannelLogoWatermark(2, "Channel");
Channel channel = ChannelWithLogo(LocalLogoPath, watermark);
Option<WatermarkOptions> result = selector.GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
Option<ChannelWatermark>.None);
result.IsNone.ShouldBeTrue();
}
/// <summary>
/// Scope guard for #502: with no logo artwork at all, the resolved path is the generated-initials
/// URL from <see cref="Images.ChannelLogoGenerator.GenerateChannelLogoUrl" />, which hardcodes
/// localhost (issue #1). That fallback stays disabled here — reviving it is deliberately deferred
/// in docs/decisions.md and is not part of this fix.
/// </summary>
[Test]
public void ChannelWatermark_Should_Be_Ignored_When_Channel_Has_No_Logo_Artwork()
{
ChannelWatermark watermark = ChannelLogoWatermark(2, "Channel");
Channel channel = ChannelWithLogo(null, watermark);
Option<WatermarkOptions> result = _selector.GetWatermarkOptions(
channel,
Option<ChannelWatermark>.None,
Option<ChannelWatermark>.None);
result.IsNone.ShouldBeTrue();
}
}
@@ -0,0 +1,52 @@
using ErsatzTV.Core.Images;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Core.Tests.Images;
[TestFixture]
public class RemoteImageDecodeBudgetTests
{
private static readonly Uri Uri = new("https://example.com/logo.png");
// the product is the real bound: 2500x2500 x600 is affordable on each axis alone but not together
[Test]
public void Should_Reject_Dimensions_And_Frames_Affordable_Alone_But_Not_Together()
{
((long)2500 * 2500).ShouldBeLessThanOrEqualTo(RemoteImageDecodeBudget.MaxRemoteDecodedPixels);
600.ShouldBeLessThanOrEqualTo(RemoteImageDecodeBudget.MaxRemoteFrames);
InvalidOperationException ex = Should.Throw<InvalidOperationException>(
() => RemoteImageDecodeBudget.EnsureDecodeAffordable(2500, 2500, 600, Uri));
ex.Message.ShouldContain("pixel limit");
}
[Test]
public void Should_Reject_Too_Many_Frames_Even_When_Each_Is_Tiny() =>
Should.Throw<InvalidOperationException>(
() => RemoteImageDecodeBudget.EnsureDecodeAffordable(8, 8, RemoteImageDecodeBudget.MaxRemoteFrames + 1, Uri))
.Message.ShouldContain("frame limit");
[Test]
public void Should_Reject_A_Single_Oversized_Frame() =>
Should.Throw<InvalidOperationException>(
() => RemoteImageDecodeBudget.EnsureDimensionsAffordable(30000, 30000, Uri))
.Message.ShouldContain("pixel limit");
[Test]
public void Should_Allow_A_Single_Large_Still_Within_Budget() =>
Should.NotThrow(() => RemoteImageDecodeBudget.EnsureDecodeAffordable(7680, 4320, 1, Uri));
[Test]
public void Should_Charge_At_Least_One_Frame_When_Header_Reports_None() =>
Should.Throw<InvalidOperationException>(
() => RemoteImageDecodeBudget.EnsureDecodeAffordable(30000, 30000, 0, Uri));
[Test]
public void Should_Afford_Fewer_Frames_As_Frames_Get_Larger()
{
RemoteImageDecodeBudget.AffordableFrames(8, 8).ShouldBe(RemoteImageDecodeBudget.MaxRemoteFrames);
RemoteImageDecodeBudget.AffordableFrames(1000, 1000).ShouldBe(50);
RemoteImageDecodeBudget.AffordableFrames(7000, 7000).ShouldBe(1);
}
}
@@ -68,6 +68,29 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
_logger = logger;
}
/// <summary>
/// Whether ffmpeg can carry the watermark itself (a single permanent watermark and no other graphics),
/// rather than handing it to the graphics engine.
/// </summary>
/// <remarks>
/// A watermark whose resolved path is a remote URL is always refused here (#502). ffmpeg would receive it
/// as a bare <c>-i</c> argument — and <c>ffprobe</c> the same string for animation detection — putting an
/// unbounded network fetch inside stream startup, with no timeout, redirect or auth handling of ours, and
/// with the image's real dimensions never probed. The graphics engine fetches remote images deliberately
/// (<c>ImageElementBase.LoadImage</c>) and decodes them for their true size.
/// <para>
/// This is decided by the resolved <see cref="WatermarkOptions.ImagePath" /> alone, so it applies
/// uniformly however the watermark was selected — channel, global, playout item <em>or</em> deco. Only
/// `ChannelLogo` watermarks can produce a URL: `Custom` resolves through the image cache and
/// `Resource` through the resources folder, so neither is ever rerouted.
/// </para>
/// </remarks>
internal static bool CanUseFFmpegNativeWatermark(int graphicsElementCount, List<WatermarkOptions> watermarks) =>
graphicsElementCount == 0
&& watermarks.Count == 1
&& watermarks.All(wm => wm.Watermark.Mode is ChannelWatermarkMode.Permanent
&& !Artwork.IsExternalUrl(wm.ImagePath));
public async Task<PlayoutItemResult> ForPlayoutItem(
string ffmpegPath,
string ffprobePath,
@@ -379,8 +402,7 @@ public class FFmpegLibraryProcessService : IFFmpegProcessService
Option<GraphicsEngineContext> graphicsEngineContext = Option<GraphicsEngineContext>.None;
List<GraphicsElementContext> graphicsElementContexts = [];
// use ffmpeg for single permanent watermark, graphics engine for all others
if (graphicsElements.Count == 0 && watermarks.Count == 1 && watermarks.All(wm => wm.Watermark.Mode is ChannelWatermarkMode.Permanent))
if (CanUseFFmpegNativeWatermark(graphicsElements.Count, watermarks))
{
foreach (var wm in watermarks)
{
+57 -57
View File
@@ -216,25 +216,7 @@ public class WatermarkSelector(
case ChannelWatermarkImageSource.ChannelLogo:
logger.LogDebug("Watermark will come from playout item (channel logo)");
string channelPath = ChannelLogoGenerator.GenerateChannelLogoUrl(channel);
Option<Artwork> maybeLogoArtwork =
Optional(channel.Artwork.Find(a => a.ArtworkKind == ArtworkKind.Logo));
foreach (var logoArtwork in maybeLogoArtwork)
{
channelPath = Artwork.IsExternalUrl(logoArtwork.Path)
? logoArtwork.Path
: imageCache.GetPathForImage(logoArtwork.Path, ArtworkKind.Logo, Option<int>.None);
}
if (fileSystem.File.Exists(channelPath))
{
return new WatermarkOptions(watermark, channelPath, None);
}
logger.LogWarning(
"Channel logo no longer exists at {Path} and will be ignored",
channelPath);
return None;
return ChannelLogoWatermarkOptions(channel, watermark);
default:
throw new NotSupportedException("Unsupported watermark image source");
}
@@ -265,25 +247,7 @@ public class WatermarkSelector(
case ChannelWatermarkImageSource.ChannelLogo:
logger.LogDebug("Watermark will come from channel (channel logo)");
string channelPath = ChannelLogoGenerator.GenerateChannelLogoUrl(channel);
Option<Artwork> maybeLogoArtwork =
Optional(channel.Artwork.Find(a => a.ArtworkKind == ArtworkKind.Logo));
foreach (var logoArtwork in maybeLogoArtwork)
{
channelPath = Artwork.IsExternalUrl(logoArtwork.Path)
? logoArtwork.Path
: imageCache.GetPathForImage(logoArtwork.Path, ArtworkKind.Logo, Option<int>.None);
}
if (fileSystem.File.Exists(channelPath))
{
return new WatermarkOptions(channel.Watermark, channelPath, None);
}
logger.LogWarning(
"Channel logo no longer exists at {Path} and will be ignored",
channelPath);
return None;
return ChannelLogoWatermarkOptions(channel, channel.Watermark);
default:
throw new NotSupportedException("Unsupported watermark image source");
}
@@ -314,25 +278,7 @@ public class WatermarkSelector(
case ChannelWatermarkImageSource.ChannelLogo:
logger.LogDebug("Watermark will come from global (channel logo)");
string channelPath = ChannelLogoGenerator.GenerateChannelLogoUrl(channel);
Option<Artwork> maybeLogoArtwork =
Optional(channel.Artwork.Find(a => a.ArtworkKind == ArtworkKind.Logo));
foreach (var logoArtwork in maybeLogoArtwork)
{
channelPath = Artwork.IsExternalUrl(logoArtwork.Path)
? logoArtwork.Path
: imageCache.GetPathForImage(logoArtwork.Path, ArtworkKind.Logo, Option<int>.None);
}
if (fileSystem.File.Exists(channelPath))
{
return new WatermarkOptions(watermark, channelPath, None);
}
logger.LogWarning(
"Channel logo no longer exists at {Path} and will be ignored",
channelPath);
return None;
return ChannelLogoWatermarkOptions(channel, watermark);
default:
throw new NotSupportedException("Unsupported watermark image source");
}
@@ -341,6 +287,52 @@ public class WatermarkSelector(
return Option<WatermarkOptions>.None;
}
/// <summary>
/// Resolves a <see cref="ChannelWatermarkImageSource.ChannelLogo" /> watermark to a renderable path,
/// shared by the playout-item, channel and global precedence levels so all three agree.
/// </summary>
/// <remarks>
/// As of #525 an external-URL logo is downloaded and cached at save time, so a URL path here can only
/// be a row that failed migration. The render path must never fetch at compositing time, so such a row
/// degrades to no watermark (no on-screen bug) with a warning rather than being handed downstream as a
/// renderable URL (the #502 behavior). Other consumers (M3U, XMLTV, SPA JSON) still emit the raw URL for
/// a not-yet-migrated row; only this render/watermark path changed.
/// </remarks>
private Option<WatermarkOptions> ChannelLogoWatermarkOptions(Channel channel, ChannelWatermark watermark)
{
foreach (var logoArtwork in Optional(channel.Artwork.Find(a => a.ArtworkKind == ArtworkKind.Logo)))
{
if (Artwork.IsExternalUrl(logoArtwork.Path))
{
// As of #525 an external-URL logo is downloaded and cached at save time, so a URL here
// means a row that failed migration. Do not fetch at render time; degrade to no bug.
logger.LogWarning(
"Channel logo for channel {Channel} is still an un-downloaded URL {Url}; re-save the "
+ "channel to download it. Rendering without an on-screen bug.",
channel.Number,
logoArtwork.Path);
return None;
}
string cachedPath = imageCache.GetPathForImage(logoArtwork.Path, ArtworkKind.Logo, Option<int>.None);
if (fileSystem.File.Exists(cachedPath))
{
return new WatermarkOptions(watermark, cachedPath, None);
}
logger.LogWarning("Channel logo no longer exists at {Path} and will be ignored", cachedPath);
return None;
}
// with no logo artwork the only candidate is the generated-initials image, whose URL hardcodes
// localhost (ChannelLogoGenerator.GenerateChannelLogoUrl, issue #1). It has never rendered here and
// reviving it is deliberately deferred in docs/decisions.md, so it stays ignored.
logger.LogWarning(
"Channel logo no longer exists at {Path} and will be ignored",
ChannelLogoGenerator.GenerateChannelLogoUrl(channel));
return None;
}
private List<WatermarkOptions> OptionsForWatermarks(Channel channel, IEnumerable<ChannelWatermark> watermarks)
{
var result = new List<WatermarkOptions>();
@@ -382,6 +374,14 @@ public class WatermarkSelector(
customPath,
None);
case ChannelWatermarkImageSource.ChannelLogo:
// deliberately NOT ChannelLogoWatermarkOptions: the deco path has always passed its resolved
// path through unchecked, so #502's File.Exists defect never reached it and its *resolution*
// is unchanged here. Aligning its missing-file / no-artwork policy with the three precedence
// levels above is a behavior change beyond this fix — tracked in #510.
// Note this only scopes resolution: the ffmpeg-native-vs-graphics-engine routing in
// FFmpegLibraryProcessService.CanUseFFmpegNativeWatermark keys off the resolved path alone, so a
// deco watermark resolving to a URL (an external logo, or the generated-initials URL below) is
// rerouted to the graphics engine like any other. That is intended: it is the URL-aware path.
string channelPath = ChannelLogoGenerator.GenerateChannelLogoUrl(channel);
Option<Artwork> maybeLogoArtwork =
Optional(channel.Artwork.Find(a => a.ArtworkKind == ArtworkKind.Logo));
@@ -0,0 +1,54 @@
namespace ErsatzTV.Core.Images;
/// <summary>
/// The decode-budget policy for a remote image, as pure arithmetic so it can be enforced both at
/// render time (graphics engine) and at save time (logo download) without materializing
/// multi-gigabyte images. Extracted from ImageElementBase for reuse. (ersatztv#525, from #511.)
/// </summary>
public static class RemoteImageDecodeBudget
{
/// <summary>
/// Ceiling on TOTAL decoded pixels — width x height x frames, as one product. Checking
/// dimensions and frame count independently does not bound the decode: a 60 KiB 2500x2500 x600
/// GIF passes both a 50 MP dimension check and a 600 frame check and costs ~14 GiB.
/// </summary>
public const long MaxRemoteDecodedPixels = 50_000_000;
/// <summary>Frame ceiling, a cheap legible guard against absurd counts of tiny frames.</summary>
public const int MaxRemoteFrames = 600;
public static void EnsureDimensionsAffordable(int width, int height, Uri uri)
{
long pixels = (long)width * height;
if (pixels > MaxRemoteDecodedPixels)
{
throw new InvalidOperationException(
$"Remote image {uri} is {width}x{height} ({pixels} pixels), over the "
+ $"{MaxRemoteDecodedPixels} pixel limit");
}
}
public static int AffordableFrames(int width, int height)
{
long perFrame = Math.Max((long)width * height, 1);
return (int)Math.Clamp(MaxRemoteDecodedPixels / perFrame, 1, MaxRemoteFrames);
}
public static void EnsureDecodeAffordable(int width, int height, int frameCount, Uri uri)
{
int frames = Math.Max(frameCount, 1);
if (frames > MaxRemoteFrames)
{
throw new InvalidOperationException(
$"Remote image {uri} has {frames} frames, over the {MaxRemoteFrames} frame limit");
}
long totalPixels = (long)width * height * frames;
if (totalPixels > MaxRemoteDecodedPixels)
{
throw new InvalidOperationException(
$"Remote image {uri} decodes to {width}x{height} x{frames} frames "
+ $"({totalPixels} pixels), over the {MaxRemoteDecodedPixels} pixel limit");
}
}
}
@@ -0,0 +1,13 @@
namespace ErsatzTV.Core.Interfaces.Images;
/// <summary>
/// Validates that a stream is a decodable image within the decode budget, throwing if not.
/// Used by the logo save path and the artwork upload path (neither needs the decoded pixels,
/// only "is this safe to cache"). The graphics engine uses the static
/// RemoteImageValidator.DecodeAndValidate instead, which returns the Image it composites.
/// (ersatztv#525)
/// </summary>
public interface IRemoteImageValidator
{
Task Validate(Stream stream, Uri uri, CancellationToken cancellationToken);
}
@@ -0,0 +1,14 @@
using ErsatzTV.Core;
using LanguageExt;
namespace ErsatzTV.Core.Interfaces.Images;
/// <summary>
/// Fetches an external logo URL, validates it against the decode budget, and stores it in the
/// image cache — turning a URL into a cache name so it is thereafter identical to an uploaded
/// logo. Errors are returned, not thrown, so a save handler can surface a 400. (ersatztv#525)
/// </summary>
public interface IRemoteLogoCacher
{
Task<Either<BaseError, string>> CacheFromUrl(Uri uri, CancellationToken cancellationToken);
}
@@ -0,0 +1,24 @@
namespace ErsatzTV.Core.Interfaces.Streaming;
/// <summary>
/// Fetches a remote (http/https) image for the graphics engine, under a bounded timeout and a
/// bounded response size.
/// </summary>
public interface IRemoteImageFetcher
{
/// <summary>
/// Fetches <paramref name="uri" /> fully into memory.
/// </summary>
/// <returns>
/// A seekable, fully-buffered stream positioned at zero. The caller owns and must dispose it.
/// </returns>
/// <remarks>
/// Throws rather than returning a failure value: every call site already wraps element
/// initialization in a catch that disables the element, so a throw degrades to "no watermark"
/// rather than a killed stream. An implementation's own deadline should surface as
/// <see cref="TimeoutException" /> and caller cancellation as
/// <see cref="OperationCanceledException" /> — though note the current call sites catch both
/// alike, so today the distinction only sharpens the log message.
/// </remarks>
Task<Stream> Fetch(Uri uri, CancellationToken cancellationToken);
}
@@ -1,5 +1,6 @@
using System;
using System.Collections.Generic;
using System.Text.RegularExpressions;
using ErsatzTV.FFmpeg.Capabilities;
using ErsatzTV.FFmpeg.Format;
using ErsatzTV.FFmpeg.OutputFormat;
@@ -55,26 +56,7 @@ public class QsvPipelineBuilderTests
[Test]
public void Qsv_PreferNativeDecoder_Interlaced_Should_Hwupload_Before_Deinterlace_Qsv()
{
(VideoInputFile videoInputFile, AudioInputFile audioInputFile, FFmpegState ffmpegState, FrameState desiredState) =
BuildQsvH264Pipeline(preferNativeDecoder: true, scanKind: ScanKind.Interlaced, deinterlace: true);
var builder = new QsvPipelineBuilder(
new DefaultFFmpegCapabilities(),
new DefaultHardwareCapabilities(),
HardwareAccelerationMode.Qsv,
videoInputFile,
audioInputFile,
None,
None,
None,
Option<GraphicsEngineInput>.None,
"",
"",
_logger);
FFmpegPipeline result = builder.Build(ffmpegState, desiredState);
string command = PrintCommand(videoInputFile, audioInputFile, None, None, None, result);
string command = BuildInterlacedAndPrint();
// VA-API decode, software frames
command.ShouldContain("-hwaccel vaapi");
@@ -90,10 +72,73 @@ public class QsvPipelineBuilderTests
command.ShouldContain("h264_qsv");
}
private string BuildAndPrint(bool preferNativeDecoder)
// ersatztv#529: a stored qsvExtraHardwareFrames of 0 produced hwupload=extra_hw_frames=0, which
// leaves the QSV pool no headroom. Measured on the deployed FFmpeg 8.1.2: with 0 the filter graph
// fails with -12 and writes zero segments as soon as the input is not throttled (a work-ahead
// start, or #350's cold-start burst); with 64 the same command writes segments either way.
[TestCase(-1)]
[TestCase(0)]
[TestCase(1)]
[TestCase(63)]
public void Qsv_Should_Never_Upload_With_Less_Than_Minimum_Extra_Hardware_Frames(int configured)
{
string command = BuildAndPrint(preferNativeDecoder: true, maybeExtraHardwareFrames: configured);
// pinned to the literal value measured against the deployed FFmpeg, not to the constant, so
// that lowering the floor in code cannot quietly satisfy this test
command.ShouldContain("hwupload=extra_hw_frames=64");
// every upload site in the whole command, not just the one this pipeline happens to emit
ShouldNeverUploadBelowMinimum(command);
}
[Test]
public void Qsv_Interlaced_Should_Never_Deinterlace_Upload_Below_Minimum_Extra_Hardware_Frames()
{
// the deinterlace upload is a separate formatter fed from QsvPipelineBuilder, so it needs
// its own guard (ersatztv#529)
string command = BuildInterlacedAndPrint(maybeExtraHardwareFrames: 0);
command.ShouldContain("hwupload=extra_hw_frames=64,deinterlace_qsv");
ShouldNeverUploadBelowMinimum(command);
}
private static void ShouldNeverUploadBelowMinimum(string command)
{
MatchCollection matches = Regex.Matches(command, @"extra_hw_frames=(-?\d+)");
matches.Count.ShouldBeGreaterThan(0, "expected the pipeline to upload to QSV at all");
foreach (Match match in matches)
{
int.Parse(match.Groups[1].Value)
.ShouldBeGreaterThanOrEqualTo(FFmpegState.MinimumQsvExtraHardwareFrames, command);
}
}
[Test]
public void Qsv_Should_Honor_Extra_Hardware_Frames_Above_The_Minimum()
{
string command = BuildAndPrint(preferNativeDecoder: true, maybeExtraHardwareFrames: 128);
command.ShouldContain("hwupload=extra_hw_frames=128");
}
private string BuildInterlacedAndPrint(Option<int> maybeExtraHardwareFrames = default) =>
BuildAndPrint(
preferNativeDecoder: true,
maybeExtraHardwareFrames,
ScanKind.Interlaced,
deinterlace: true);
private string BuildAndPrint(
bool preferNativeDecoder,
Option<int> maybeExtraHardwareFrames = default,
ScanKind scanKind = ScanKind.Progressive,
bool deinterlace = false)
{
(VideoInputFile videoInputFile, AudioInputFile audioInputFile, FFmpegState ffmpegState, FrameState desiredState) =
BuildQsvH264Pipeline(preferNativeDecoder, ScanKind.Progressive, false);
BuildQsvH264Pipeline(preferNativeDecoder, scanKind, deinterlace);
ffmpegState = ffmpegState with { MaybeQsvExtraHardwareFrames = maybeExtraHardwareFrames };
var builder = new QsvPipelineBuilder(
new DefaultFFmpegCapabilities(),
@@ -1,5 +1,6 @@
using System;
using System;
using System.Collections.Generic;
using System.Text.RegularExpressions;
using ErsatzTV.FFmpeg.Capabilities;
using ErsatzTV.FFmpeg.Encoder;
using ErsatzTV.FFmpeg.Format;
@@ -531,6 +532,168 @@ public class PipelineBuilderBaseTests
"-nostdin -hide_banner -nostats -loglevel error -i /test/input/file.png -vf scale=-1:200:force_original_aspect_ratio=decrease /test/output/file.jpg");
}
[Test]
public void Realtime_Input_Should_Burst_Initial_Segments_When_Option_Is_Supported()
{
string command = BuildRealtimeCommand(new BurstCapableFFmpegCapabilities());
// burst covers the first two 4s segments, then the 1.05 throttle resumes (ersatztv#350).
// anchor on the input path so this can't be satisfied by some other input carrying the
// burst; audio and video share one file here, so exactly one input is expected
command.ShouldContain("-readrate 1.05 -readrate_initial_burst 8 -i /tmp/whatever.mkv");
Regex.Matches(command, "-readrate_initial_burst 8").Count.ShouldBe(1);
}
[Test]
public void Realtime_Input_Should_Not_Burst_When_Option_Is_Unsupported()
{
string command = BuildRealtimeCommand(new DefaultFFmpegCapabilities());
command.ShouldContain("-readrate 1.05 -i");
command.ShouldNotContain("-readrate_initial_burst");
}
[Test]
public void Realtime_Input_Should_Not_Burst_A_Still_Image()
{
// a still image is paced by the realtime filter, so bursting would only run audio ahead
string command = BuildRealtimeCommand(new BurstCapableFFmpegCapabilities(), stillImage: true);
// the positive anchor keeps this from passing vacuously if the helper ever stops
// producing a realtime audio input at all
command.ShouldContain("-readrate 1.05");
command.ShouldNotContain("-readrate_initial_burst");
}
[Test]
public void Concat_Should_Never_Burst()
{
// concat reads already-written segments from the running segmenter; bursting would
// gallop through them
var concatInputFile = new ConcatInputFile("http://localhost:8080/ffmpeg/concat/1", new FrameSize(1920, 1080));
var builder = new SoftwarePipelineBuilder(
new BurstCapableFFmpegCapabilities(),
HardwareAccelerationMode.None,
None,
None,
None,
None,
concatInputFile,
Option<GraphicsEngineInput>.None,
"",
"",
_logger);
FFmpegPipeline result = builder.Concat(concatInputFile, FFmpegState.Concat(false, "Some Channel"));
string command = PrintCommand(None, None, None, concatInputFile, None, result);
command.ShouldContain("-readrate 1.0");
command.ShouldNotContain("-readrate_initial_burst");
}
private string BuildRealtimeCommand(IFFmpegCapabilities capabilities, bool stillImage = false)
{
var videoInputFile = new VideoInputFile(
"/tmp/whatever.mkv",
new List<VideoStream>
{
new(
0,
VideoFormat.H264,
VideoProfile.Main,
new PixelFormatYuv420P(),
ColorParams.Default,
new FrameSize(1920, 1080),
"1:1",
"16:9",
FrameRate.DefaultFrameRate,
stillImage,
ScanKind.Progressive)
});
var desiredState = new FrameState(
true,
false,
VideoFormat.Hevc,
VideoProfile.Main,
VideoPreset.Unset,
false,
new PixelFormatYuv420P(),
new FrameSize(1920, 1080),
new FrameSize(1920, 1080),
Option<FrameSize>.None,
FFmpegFilterMode.HardwareIfPossible,
false,
Option<FrameRate>.None,
2000,
4000,
90_000,
false,
false);
var ffmpegState = new FFmpegState(
false,
HardwareAccelerationMode.None,
HardwareAccelerationMode.None,
Option<string>.None,
Option<string>.None,
TimeSpan.FromSeconds(1),
Option<TimeSpan>.None,
false,
Option<string>.None,
Option<string>.None,
Option<string>.None,
Option<string>.None,
Option<string>.None,
OutputFormatKind.MpegTs,
Option<string>.None,
Option<string>.None,
Option<string>.None,
Option<string>.None,
TimeSpan.Zero,
Option<int>.None,
Option<int>.None,
false,
false,
"clip",
false);
// a *separate* audio input matters here: for a still image the video input takes no readrate
// at all, so only a distinct audio input can prove the burst was suppressed (this is the
// song shape — cover art plus an audio file)
var audioInputFile = new AudioInputFile(
stillImage ? "/tmp/whatever.mp3" : "/tmp/whatever.mkv",
new List<AudioStream> { new(1, AudioFormat.Aac, 2) },
new AudioState(
AudioFormat.Aac,
2,
320,
640,
48,
false,
AudioFilter.None,
Option<double>.None));
var builder = new SoftwarePipelineBuilder(
capabilities,
HardwareAccelerationMode.None,
videoInputFile,
audioInputFile,
None,
None,
None,
Option<GraphicsEngineInput>.None,
"",
"",
_logger);
FFmpegPipeline result = builder.Build(ffmpegState, desiredState);
return PrintCommand(videoInputFile, audioInputFile, None, None, None, result);
}
private static string PrintCommand(
Option<VideoInputFile> videoInputFile,
Option<AudioInputFile> audioInputFile,
@@ -563,4 +726,13 @@ public class PipelineBuilderBaseTests
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>());
public class BurstCapableFFmpegCapabilities() : FFmpegCapabilities(
string.Empty,
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string>(),
new System.Collections.Generic.HashSet<string> { FFmpegKnownOption.ReadrateInitialBurst.Name },
new System.Collections.Generic.HashSet<string>());
}
@@ -10,7 +10,11 @@ public record FFmpegKnownOption
public string Name { get; }
// ffmpeg 6.1+; lets a readrate-throttled input read flat out for an initial window
public static FFmpegKnownOption ReadrateInitialBurst => new("readrate_initial_burst");
public static IList<string> AllOptions =>
[
ReadrateInitialBurst.Name
];
}
+11 -1
View File
@@ -30,7 +30,17 @@ public record FFmpegState(
bool IsTroubleshooting,
bool QsvPreferNativeDecoder = false)
{
public int QsvExtraHardwareFrames => MaybeQsvExtraHardwareFrames.IfNone(64);
// the QSV upload pool needs headroom for the frames in flight through the filter graph.
// extra_hw_frames=0 leaves none, so any input that is not throttled exhausts it: the graph
// fails with -12 (Cannot allocate memory), h264_qsv reports "Could not open encoder before
// EOF", and the output file gets no packets at all. Input throttling was the only thing
// hiding it — a work-ahead start (no -readrate) and #350's cold-start burst both remove that
// throttle, so the channel simply dies. A stored 0 is therefore treated as "no pool
// configured" rather than honored literally (ersatztv#529)
public const int MinimumQsvExtraHardwareFrames = 64;
public int QsvExtraHardwareFrames =>
Math.Max(MaybeQsvExtraHardwareFrames.IfNone(MinimumQsvExtraHardwareFrames), MinimumQsvExtraHardwareFrames);
public static FFmpegState Concat(bool saveReport, string channelName) =>
new(
@@ -1,19 +1,37 @@
using System.Globalization;
using System.Globalization;
using ErsatzTV.FFmpeg.Environment;
namespace ErsatzTV.FFmpeg.InputOption;
public class ReadrateInputOption(double readRate) : IInputOption
public class ReadrateInputOption(double readRate, Option<int> initialBurstSeconds) : IInputOption
{
public ReadrateInputOption(double readRate)
: this(readRate, Option<int>.None)
{
}
public EnvironmentVariable[] EnvironmentVariables => [];
public string[] GlobalOptions => [];
public string[] InputOptions(InputFile inputFile) =>
[
"-readrate",
readRate.ToString("0.0####", CultureInfo.InvariantCulture)
];
public string[] InputOptions(InputFile inputFile)
{
var result = new List<string>
{
"-readrate",
readRate.ToString("0.0####", CultureInfo.InvariantCulture)
};
// burst-read this much input before the readrate throttle kicks in, so a cold start doesn't
// have to wait ~realtime for the first segment to be written (ersatztv#350)
foreach (int burst in initialBurstSeconds)
{
result.Add("-readrate_initial_burst");
result.Add(burst.ToString(CultureInfo.InvariantCulture));
}
return result.ToArray();
}
public string[] FilterOptions => [];
public string[] OutputOptions => [];
@@ -17,6 +17,11 @@ namespace ErsatzTV.FFmpeg.Pipeline;
public abstract class PipelineBuilderBase : IPipelineBuilder
{
// enough input to cover the first couple of HLS segments; the segmenter waits for
// ffmpeg.segmenter.initial_segment_count (default 1) of them before serving the playlist.
// an operator who raises that setting above 2 gets less of the benefit (ersatztv#350)
private const int InitialBurstSeconds = OutputFormatHls.SegmentSeconds * 2;
private readonly Option<AudioInputFile> _audioInputFile;
private readonly Option<ConcatInputFile> _concatInputFile;
private readonly IFFmpegCapabilities _ffmpegCapabilities;
@@ -850,8 +855,24 @@ public abstract class PipelineBuilderBase : IPipelineBuilder
}
double readRate = desiredState.VideoFormat == VideoFormat.Copy ? 1.0 : 1.05;
_audioInputFile.Iter(a => a.AddOption(new ReadrateInputOption(readRate)));
videoInputFile.AddOption(new ReadrateInputOption(readRate));
// without a burst, the readrate throttle applies from the very first read, so the first
// segment cannot be written faster than ~realtime and every start pays a multi-second wait.
// burst enough input to cover the first segments, then settle to readRate. note this is
// per ffmpeg process, i.e. per playout item, not only on the session's cold start
// (ersatztv#350)
//
// a still image is paced by the realtime filter instead, and its video input takes no
// readrate at all, so bursting there would only run the audio input ahead of the video
bool isStillImage = videoInputFile.VideoStreams.Any(s => s.StillImage);
Option<int> initialBurstSeconds =
!isStillImage && _ffmpegCapabilities.HasOption(FFmpegKnownOption.ReadrateInitialBurst)
? InitialBurstSeconds
: Option<int>.None;
_audioInputFile.Iter(a => a.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds)));
videoInputFile.AddOption(new ReadrateInputOption(readRate, initialBurstSeconds));
}
protected static void SetStillImageLoop(
@@ -62,6 +62,21 @@ public class QsvPipelineBuilder : SoftwarePipelineBuilder
PipelineContext context,
ICollection<IPipelineStep> pipelineSteps)
{
// surface the floor rather than applying it silently: a profile that deliberately asked for
// a smaller pool now gets a larger one, which costs additional surfaces (ersatztv#529)
foreach (int configured in ffmpegState.MaybeQsvExtraHardwareFrames
.Filter(f => f < FFmpegState.MinimumQsvExtraHardwareFrames))
{
// this fires per pipeline build, before we know whether this particular pipeline uploads
// at all — a fully-hardware path may consume neither value — so word it conditionally
_logger.LogWarning(
"QSV extra hardware frames is configured as {Configured}, which leaves the upload pool too "
+ "little headroom and fails transcoding on any unthrottled read; will use {Applied} "
+ "wherever frames are uploaded",
configured,
FFmpegState.MinimumQsvExtraHardwareFrames);
}
FFmpegCapability decodeCapability = _hardwareCapabilities.CanDecode(
videoStream.Codec,
videoStream.Profile,
@@ -0,0 +1,151 @@
using System.Buffers.Binary;
using ErsatzTV.Core.Images;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Infrastructure.Images;
using NUnit.Framework;
using Shouldly;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Png;
using SixLabors.ImageSharp.PixelFormats;
using Image = SixLabors.ImageSharp.Image;
namespace ErsatzTV.Infrastructure.Tests.Images;
[TestFixture]
public class RemoteImageValidatorTests
{
private static readonly Uri Uri = new("https://example.com/logo.png");
// decode cases exercise the static method (used by the render path)
[Test]
public async Task Should_Decode_A_Normal_Image()
{
await using MemoryStream stream = await RealPng(64, 32);
using Image image = await RemoteImageValidator.DecodeAndValidate(stream, Uri, CancellationToken.None);
image.Width.ShouldBe(64);
image.Height.ShouldBe(32);
}
[Test]
public async Task Should_Reject_A_Decompression_Bomb_By_Declared_Dimensions()
{
await using MemoryStream stream = PngHeaderDeclaring(30000, 30000);
InvalidOperationException ex = await Should.ThrowAsync<InvalidOperationException>(
() => RemoteImageValidator.DecodeAndValidate(stream, Uri, CancellationToken.None));
ex.Message.ShouldContain("pixel limit");
}
[Test]
public async Task Should_Reject_An_Apng_Whose_Header_Under_Reports_Its_Frames()
{
await using MemoryStream stream = Apng(64, 64, RemoteImageDecodeBudget.MaxRemoteFrames + 100);
InvalidOperationException ex = await Should.ThrowAsync<InvalidOperationException>(
() => RemoteImageValidator.DecodeAndValidate(stream, Uri, CancellationToken.None));
ex.Message.ShouldContain("frame limit");
}
[Test]
public async Task Should_Decode_An_Apng_That_A_Default_Identify_Cannot_Read()
{
await using MemoryStream stream = Apng(288, 288, 60);
stream.Position = 0;
await Should.ThrowAsync<Exception>(() => Image.IdentifyAsync(stream));
stream.Position = 0;
using Image image = await RemoteImageValidator.DecodeAndValidate(stream, Uri, CancellationToken.None);
image.Frames.Count.ShouldBe(60);
}
// the Core interface Validate() is the save/upload contract: throws on invalid, returns on valid,
// never surfaces an ImageSharp type
[Test]
public async Task Validate_Returns_On_A_Good_Image()
{
IRemoteImageValidator validator = new RemoteImageValidator();
await using MemoryStream stream = await RealPng(64, 32);
await Should.NotThrowAsync(() => validator.Validate(stream, Uri, CancellationToken.None));
}
[Test]
public async Task Validate_Throws_On_A_Bomb()
{
IRemoteImageValidator validator = new RemoteImageValidator();
await using MemoryStream stream = PngHeaderDeclaring(30000, 30000);
await Should.ThrowAsync<InvalidOperationException>(
() => validator.Validate(stream, Uri, CancellationToken.None));
}
/// <summary>A real multi-frame APNG. Small on the wire, many frames — the shape that matters.</summary>
private static MemoryStream Apng(int width, int height, int frames)
{
using var image = new Image<Rgba32>(width, height);
for (var i = 1; i < frames; i++)
{
image.Frames.CreateFrame();
}
var stream = new MemoryStream();
image.Save(stream, new PngEncoder { ColorType = PngColorType.RgbWithAlpha });
stream.Position = 0;
return stream;
}
// PNG chunk CRC-32 (IEEE, reflected). hand-rolled because the repo does not reference
// System.IO.Hashing, and ImageSharp validates the CRC of critical chunks like IHDR.
private static uint Crc32(ReadOnlySpan<byte> data)
{
uint crc = 0xFFFFFFFF;
foreach (byte b in data)
{
crc ^= b;
for (var i = 0; i < 8; i++)
{
crc = (crc & 1) != 0 ? (crc >> 1) ^ 0xEDB88320 : crc >> 1;
}
}
return crc ^ 0xFFFFFFFF;
}
/// <summary>A real, decodable PNG.</summary>
private static async Task<MemoryStream> RealPng(int width, int height)
{
using var image = new Image<Rgba32>(width, height);
var stream = new MemoryStream();
await image.SaveAsync(stream, new PngEncoder());
stream.Position = 0;
return stream;
}
/// <summary>
/// A PNG signature plus a single valid IHDR chunk declaring <paramref name="width" /> x
/// <paramref name="height" /> and nothing else — enough for Identify, far too little to
/// decode. This is what a decompression bomb looks like at the point we have to reject it.
/// </summary>
private static MemoryStream PngHeaderDeclaring(int width, int height)
{
var stream = new MemoryStream();
stream.Write([0x89, (byte)'P', (byte)'N', (byte)'G', 0x0D, 0x0A, 0x1A, 0x0A]);
var ihdr = new byte[17];
"IHDR"u8.CopyTo(ihdr);
BinaryPrimitives.WriteInt32BigEndian(ihdr.AsSpan(4), width);
BinaryPrimitives.WriteInt32BigEndian(ihdr.AsSpan(8), height);
ihdr[12] = 8; // bit depth
ihdr[13] = 6; // color type: truecolor + alpha
ihdr[14] = 0; // compression
ihdr[15] = 0; // filter
ihdr[16] = 0; // interlace
var length = new byte[4];
BinaryPrimitives.WriteInt32BigEndian(length, 13);
stream.Write(length);
stream.Write(ihdr);
var crc = new byte[4];
BinaryPrimitives.WriteUInt32BigEndian(crc, Crc32(ihdr));
stream.Write(crc);
stream.Position = 0;
return stream;
}
}
@@ -0,0 +1,79 @@
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Streaming;
using ErsatzTV.Infrastructure.Images;
using LanguageExt;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Png;
using SixLabors.ImageSharp.PixelFormats;
using static LanguageExt.Prelude;
using Image = SixLabors.ImageSharp.Image;
namespace ErsatzTV.Infrastructure.Tests.Images;
[TestFixture]
public class RemoteLogoCacherTests
{
private static readonly Uri Uri = new("https://example.com/logo.png");
[Test]
public async Task Should_Fetch_Validate_And_Cache_Returning_The_Name()
{
MemoryStream png = await RealPng(64, 64);
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Uri, Arg.Any<CancellationToken>()).Returns(png);
var validator = Substitute.For<IRemoteImageValidator>();
validator.Validate(png, Uri, Arg.Any<CancellationToken>()).Returns(Task.CompletedTask);
var cache = Substitute.For<IImageCache>();
cache.SaveArtworkToCache(Arg.Any<Stream>(), ArtworkKind.Logo).Returns(Right<BaseError, string>("abc123"));
var cacher = new RemoteLogoCacher(fetcher, validator, cache);
Either<BaseError, string> result = await cacher.CacheFromUrl(Uri, CancellationToken.None);
result.IsRight.ShouldBeTrue();
result.IfRight(name => name.ShouldBe("abc123"));
}
[Test]
public async Task Should_Return_Left_When_The_Fetch_Throws()
{
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Uri, Arg.Any<CancellationToken>()).Returns<Stream>(_ => throw new TimeoutException("timed out"));
var cacher = new RemoteLogoCacher(fetcher, Substitute.For<IRemoteImageValidator>(), Substitute.For<IImageCache>());
Either<BaseError, string> result = await cacher.CacheFromUrl(Uri, CancellationToken.None);
result.IsLeft.ShouldBeTrue();
result.IfLeft(e => e.Value.ShouldContain("timed out"));
}
[Test]
public async Task Should_Return_Left_When_Validation_Rejects_A_Bomb()
{
MemoryStream png = await RealPng(64, 64);
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Uri, Arg.Any<CancellationToken>()).Returns(png);
var validator = Substitute.For<IRemoteImageValidator>();
validator.Validate(png, Uri, Arg.Any<CancellationToken>())
.Returns<Task>(_ => throw new InvalidOperationException("over the 50000000 pixel limit"));
var cacher = new RemoteLogoCacher(fetcher, validator, Substitute.For<IImageCache>());
Either<BaseError, string> result = await cacher.CacheFromUrl(Uri, CancellationToken.None);
result.IsLeft.ShouldBeTrue();
result.IfLeft(e => e.Value.ShouldContain("pixel limit"));
}
private static async Task<MemoryStream> RealPng(int w, int h)
{
using var img = new Image<Rgba32>(w, h);
var ms = new MemoryStream();
await img.SaveAsync(ms, new PngEncoder());
ms.Position = 0;
return ms;
}
}
@@ -0,0 +1,220 @@
using System.Buffers.Binary;
using ErsatzTV.Core.Images;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using NUnit.Framework;
using Shouldly;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Png;
using SixLabors.ImageSharp.PixelFormats;
using Image = SixLabors.ImageSharp.Image;
namespace ErsatzTV.Infrastructure.Tests.Streaming.Graphics;
/// <summary>
/// The byte cap in <c>HttpRemoteImageFetcher</c> does not bound decoding: a decompression bomb
/// is tiny on the wire and enormous in memory. These pin the header-first check that does.
/// (ersatztv#511)
/// </summary>
[TestFixture]
public class RemoteImageDecodeLimitTests
{
private static readonly Uri ImageUri = new("https://example.com/logo.png");
[Test]
public async Task Should_Decode_A_Normal_Image()
{
await using MemoryStream stream = await RealPng(64, 32);
using Image image = await ImageElementBase.DecodeRemoteImage(stream, ImageUri, CancellationToken.None);
image.Width.ShouldBe(64);
image.Height.ShouldBe(32);
}
// the bomb: a few dozen bytes on the wire, ~3.6 GB if decoded. it sails through the byte cap,
// the content-type check and the Content-Length reject -- only the header dimensions catch it.
[Test]
public async Task Should_Reject_An_Image_Whose_Declared_Dimensions_Are_A_Decompression_Bomb()
{
await using MemoryStream stream = PngHeaderDeclaring(30000, 30000);
stream.Length.ShouldBeLessThan(100, "the point is that this is tiny on the wire");
InvalidOperationException ex = await Should.ThrowAsync<InvalidOperationException>(
() => ImageElementBase.DecodeRemoteImage(stream, ImageUri, CancellationToken.None));
ex.Message.ShouldContain("pixel limit");
}
[Test]
public async Task Should_Accept_Dimensions_Exactly_At_The_Limit()
{
// 10000 x 5000 = 50,000,000 -- exactly the budget, so it must NOT be rejected. the decode
// then fails on the truncated body, which proves the check let it through. NOTE this test
// would also pass with the guard deleted entirely; deletion is covered by the bomb test
// above, and the boundary arithmetic by RemoteImageDecodeBudgetTests (ErsatzTV.Core.Tests).
await using MemoryStream stream = PngHeaderDeclaring(10000, 5000);
Exception ex = await Should.ThrowAsync<Exception>(
() => ImageElementBase.DecodeRemoteImage(stream, ImageUri, CancellationToken.None));
ex.Message.ShouldNotContain("pixel limit");
}
// the retained-frame budget is INDEPENDENT of the decode budget: this source is trivial to
// decode (6 MP total) but retains ~5 GB of SKBitmap once every frame is scaled to 1080p
[Test]
public void Should_Reject_Cheap_Frames_That_Are_Expensive_Once_Scaled()
{
Should.NotThrow(() => RemoteImageDecodeBudget.EnsureDecodeAffordable(100, 100, 600, ImageUri));
InvalidOperationException ex = Should.Throw<InvalidOperationException>(
() => ImageElementBase.EnsureScaledFramesAffordable(600, 1920, 1080, ImageUri));
ex.Message.ShouldContain("pixel limit");
}
[Test]
public void Should_Allow_A_Scaled_Watermark_Sized_Animation()
{
// a 10%-width logo on a 1080p frame, animated
Should.NotThrow(() => ImageElementBase.EnsureScaledFramesAffordable(600, 192, 108, ImageUri));
}
// --- B1 regression: the header's frame count is a LIE for APNG ---
// ImageSharp 3.1.12 reports FrameMetadataCollection.Count == 0 for an APNG while the decoder
// produces every frame. A budget derived from that header count is enforced on a number the
// decoder does not honor — this exact payload shape, at 4000x4000, is ~134 KiB on the wire and
// ~36 GiB decoded. The bound therefore has to be imposed ON THE DECODER (DecoderOptions.
// MaxFrames) and re-verified against the real frame count. (ersatztv#511, second re-review.)
[Test]
public async Task Should_Reject_An_Animation_Whose_Header_Under_Reports_Its_Frames()
{
await using MemoryStream stream = Apng(64, 64, RemoteImageDecodeBudget.MaxRemoteFrames + 100);
// the premise: the header really does under-report, so a header-derived budget waves it through
stream.Position = 0;
ImageInfo info = await Image.IdentifyAsync(stream);
info.FrameMetadataCollection.Count.ShouldBe(0, "the APNG header under-reports; that is the whole point");
Should.NotThrow(() => RemoteImageDecodeBudget.EnsureDecodeAffordable(64, 64, info.FrameMetadataCollection.Count, ImageUri));
stream.Position = 0;
InvalidOperationException ex = await Should.ThrowAsync<InvalidOperationException>(
() => ImageElementBase.DecodeRemoteImage(stream, ImageUri, CancellationToken.None));
ex.Message.ShouldContain("frame limit");
}
// ...and an animation within budget still decodes IN FULL -- the decoder cap must not silently
// truncate legitimate content by a frame
[Test]
public async Task Should_Decode_An_Animation_Within_Budget_Without_Truncating_It()
{
await using MemoryStream stream = Apng(64, 64, 300);
using Image image = await ImageElementBase.DecodeRemoteImage(stream, ImageUri, CancellationToken.None);
image.Frames.Count.ShouldBe(300);
}
// H2 regression: a default `Image.Identify` throws InvalidImageContentException on most APNGs
// (measured: 13 of 16 shapes, including ones ImageSharp's own encoder wrote) even though
// `Image.Load` reads them back perfectly. 288x288 is one of the throwing shapes; 64x64 x300 --
// used by the tests above -- happens NOT to be, which is exactly why they could not see this.
// Without the MaxFrames=1 workaround on the Identify, every animated-PNG logo that worked
// before this change would be silently disabled. (ersatztv#511, fourth re-review.)
[Test]
public async Task Should_Decode_An_Apng_That_A_Default_Identify_Cannot_Read()
{
await using MemoryStream stream = Apng(288, 288, 60);
// the premise: a default Identify really does fail on this file
stream.Position = 0;
await Should.ThrowAsync<Exception>(() => Image.IdentifyAsync(stream));
stream.Position = 0;
using Image image = await ImageElementBase.DecodeRemoteImage(stream, ImageUri, CancellationToken.None);
image.Width.ShouldBe(288);
image.Height.ShouldBe(288);
image.Frames.Count.ShouldBe(60);
}
/// <summary>A real multi-frame APNG. Small on the wire, many frames — the shape that matters.</summary>
private static MemoryStream Apng(int width, int height, int frames)
{
using var image = new Image<Rgba32>(width, height);
for (var i = 1; i < frames; i++)
{
image.Frames.CreateFrame();
}
var stream = new MemoryStream();
image.Save(stream, new PngEncoder { ColorType = PngColorType.RgbWithAlpha });
stream.Position = 0;
return stream;
}
// PNG chunk CRC-32 (IEEE, reflected). hand-rolled because the repo does not reference
// System.IO.Hashing, and ImageSharp validates the CRC of critical chunks like IHDR.
private static uint Crc32(ReadOnlySpan<byte> data)
{
uint crc = 0xFFFFFFFF;
foreach (byte b in data)
{
crc ^= b;
for (var i = 0; i < 8; i++)
{
crc = (crc & 1) != 0 ? (crc >> 1) ^ 0xEDB88320 : crc >> 1;
}
}
return crc ^ 0xFFFFFFFF;
}
/// <summary>A real, decodable PNG.</summary>
private static async Task<MemoryStream> RealPng(int width, int height)
{
using var image = new Image<Rgba32>(width, height);
var stream = new MemoryStream();
await image.SaveAsync(stream, new PngEncoder());
stream.Position = 0;
return stream;
}
/// <summary>
/// A PNG signature plus a single valid IHDR chunk declaring <paramref name="width" /> x
/// <paramref name="height" /> and nothing else — enough for Identify, far too little to
/// decode. This is what a decompression bomb looks like at the point we have to reject it.
/// </summary>
private static MemoryStream PngHeaderDeclaring(int width, int height)
{
var stream = new MemoryStream();
stream.Write([0x89, (byte)'P', (byte)'N', (byte)'G', 0x0D, 0x0A, 0x1A, 0x0A]);
var ihdr = new byte[17];
"IHDR"u8.CopyTo(ihdr);
BinaryPrimitives.WriteInt32BigEndian(ihdr.AsSpan(4), width);
BinaryPrimitives.WriteInt32BigEndian(ihdr.AsSpan(8), height);
ihdr[12] = 8; // bit depth
ihdr[13] = 6; // color type: truecolor + alpha
ihdr[14] = 0; // compression
ihdr[15] = 0; // filter
ihdr[16] = 0; // interlace
var length = new byte[4];
BinaryPrimitives.WriteInt32BigEndian(length, 13);
stream.Write(length);
stream.Write(ihdr);
var crc = new byte[4];
BinaryPrimitives.WriteUInt32BigEndian(crc, Crc32(ihdr));
stream.Write(crc);
stream.Position = 0;
return stream;
}
}
@@ -0,0 +1,183 @@
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.FFmpeg;
using ErsatzTV.Core.Interfaces.Streaming;
using ErsatzTV.FFmpeg;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Streaming.Graphics;
using LanguageExt;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Png;
using SixLabors.ImageSharp.PixelFormats;
using Microsoft.Extensions.Logging;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Infrastructure.Tests.Streaming.Graphics;
/// <summary>
/// The degradation contract for a remote watermark image: whatever the fetcher throws, the
/// element disables itself and the stream survives. (ersatztv#511)
/// </summary>
[TestFixture]
public class WatermarkElementRemoteImageTests
{
private const string RemoteLogo = "https://example.com/logo.png";
private static readonly Exception[] FetchFailures =
[
new TimeoutException("timed out fetching remote image"),
new InvalidOperationException("remote image exceeds the byte limit"),
new HttpRequestException("no route to host")
];
[TestCaseSource(nameof(FetchFailures))]
public async Task Should_Disable_The_Watermark_When_The_Fetch_Fails(Exception failure)
{
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Arg.Any<Uri>(), Arg.Any<CancellationToken>()).Returns<Task<Stream>>(_ => throw failure);
var element = new WatermarkElement(
RemoteWatermarkOptions(),
fetcher,
Substitute.For<ILogger>());
// must not throw -- a killed graphics element must never propagate into the stream
await element.InitializeAsync(Context(), CancellationToken.None);
element.IsFinished.ShouldBeTrue();
}
// the counterpart: caller cancellation is not a fetch failure, but it must still not escape
// into the streaming pipeline as an unhandled exception
[Test]
public async Task Should_Disable_The_Watermark_When_The_Caller_Cancels()
{
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns<Task<Stream>>(_ => throw new OperationCanceledException());
var element = new WatermarkElement(RemoteWatermarkOptions(), fetcher, Substitute.For<ILogger>());
await element.InitializeAsync(Context(), CancellationToken.None);
element.IsFinished.ShouldBeTrue();
}
// a remote path really does route to the fetcher -- without this the tests above would pass
// even if LoadImage stopped recognising http(s) urls
[Test]
public async Task Should_Route_A_Remote_Path_Through_The_Fetcher()
{
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns<Task<Stream>>(_ => throw new TimeoutException());
var element = new WatermarkElement(RemoteWatermarkOptions(), fetcher, Substitute.For<ILogger>());
await element.InitializeAsync(Context(), CancellationToken.None);
await fetcher.Received(1).Fetch(new Uri(RemoteLogo), Arg.Any<CancellationToken>());
}
// ...and a local path must not touch the network at all
[Test]
public async Task Should_Not_Use_The_Fetcher_For_A_Local_Path()
{
var fetcher = Substitute.For<IRemoteImageFetcher>();
var options = new WatermarkOptions(Watermark(), "/no/such/logo.png", Option<int>.None);
var element = new WatermarkElement(options, fetcher, Substitute.For<ILogger>());
await element.InitializeAsync(Context(), CancellationToken.None);
await fetcher.DidNotReceive().Fetch(Arg.Any<Uri>(), Arg.Any<CancellationToken>());
element.IsFinished.ShouldBeTrue();
}
// --- the RETENTION budget is wired in, and is remote-only (M4 from re-review) ---
//
// These two differ ONLY in the scale percent, so together they detect deletion of the
// `if (isRemoteUri) EnsureScaledFramesAffordable(...)` call site: without it the first case
// would succeed. The source is trivial to decode (300 x 64x64 = 1.2 MP, well inside the decode
// budget) but retains ~2.5 GB of SKBitmap once every frame is scaled to 1080p.
[Test]
public async Task Should_Disable_The_Watermark_When_Scaled_Frames_Blow_The_Retention_Budget()
{
var element = new WatermarkElement(
RemoteWatermarkOptions(widthPercent: 100),
FetcherReturning(Apng(64, 64, 300)),
Substitute.For<ILogger>());
await element.InitializeAsync(Context(), CancellationToken.None);
element.IsFinished.ShouldBeTrue();
}
[Test]
public async Task Should_Keep_The_Watermark_When_The_Same_Animation_Is_Scaled_Small()
{
var element = new WatermarkElement(
RemoteWatermarkOptions(widthPercent: 10),
FetcherReturning(Apng(64, 64, 300)),
Substitute.For<ILogger>());
await element.InitializeAsync(Context(), CancellationToken.None);
element.IsFinished.ShouldBeFalse();
}
private static IRemoteImageFetcher FetcherReturning(Stream stream)
{
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Arg.Any<Uri>(), Arg.Any<CancellationToken>()).Returns(stream);
return fetcher;
}
private static MemoryStream Apng(int width, int height, int frames)
{
using var image = new Image<Rgba32>(width, height);
for (var i = 1; i < frames; i++)
{
image.Frames.CreateFrame();
}
var stream = new MemoryStream();
image.Save(stream, new PngEncoder { ColorType = PngColorType.RgbWithAlpha });
stream.Position = 0;
return stream;
}
private static WatermarkOptions RemoteWatermarkOptions(double widthPercent = 10) =>
new(Watermark(widthPercent), RemoteLogo, Option<int>.None);
private static ChannelWatermark Watermark(double widthPercent = 10) =>
new()
{
Name = "test",
Mode = ChannelWatermarkMode.Permanent,
Location = WatermarkLocation.BottomRight,
Size = WatermarkSize.Scaled,
WidthPercent = widthPercent,
HorizontalMarginPercent = 5,
VerticalMarginPercent = 5,
Opacity = 100
};
private static GraphicsEngineContext Context() =>
new(
"1",
null,
[],
[],
new Resolution { Width = 1920, Height = 1080 },
new Resolution { Width = 1920, Height = 1080 },
new FrameRate("30"),
DateTimeOffset.UnixEpoch,
DateTimeOffset.UnixEpoch,
TimeSpan.Zero,
TimeSpan.FromMinutes(1),
TimeSpan.FromMinutes(1));
}
@@ -0,0 +1,179 @@
using System.Buffers;
using System.Net;
using System.Net.Http.Headers;
using ErsatzTV.Infrastructure.Streaming;
using Microsoft.Extensions.Logging;
using Microsoft.IO;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Infrastructure.Tests.Streaming;
[TestFixture]
public class HttpRemoteImageFetcherTests
{
private static readonly Uri ImageUri = new("https://example.com/logo.png");
[Test]
public async Task Should_Return_The_Buffered_Body_On_Success()
{
byte[] payload = [1, 2, 3, 4, 5];
HttpRemoteImageFetcher fetcher = FetcherReturning(Ok(payload, "image/png"));
await using Stream result = await fetcher.Fetch(ImageUri, CancellationToken.None);
result.Position.ShouldBe(0);
var read = new byte[payload.Length];
await result.ReadExactlyAsync(read);
read.ShouldBe(payload);
}
[Test]
public async Task Should_Throw_On_An_Error_Status()
{
HttpRemoteImageFetcher fetcher = FetcherReturning(new HttpResponseMessage(HttpStatusCode.NotFound));
await Should.ThrowAsync<HttpRequestException>(() => fetcher.Fetch(ImageUri, CancellationToken.None));
}
// an html error page served with a 200 must never reach the decoder
[Test]
public async Task Should_Reject_A_Non_Image_Content_Type()
{
HttpRemoteImageFetcher fetcher = FetcherReturning(Ok([1, 2, 3], "text/html"));
await Should.ThrowAsync<InvalidOperationException>(() => fetcher.Fetch(ImageUri, CancellationToken.None));
}
// hosts that omit the header, and static file servers that default to octet-stream, are common
// enough that rejecting them would break working logos for no security gain -- ImageSharp
// decodes by magic bytes, and the wire-size + decode-budget caps are the real protection.
[TestCase(null)]
[TestCase("application/octet-stream")]
public async Task Should_Accept_A_Missing_Or_Generic_Content_Type(string mediaType)
{
HttpRemoteImageFetcher fetcher = FetcherReturning(Ok([1, 2, 3], mediaType));
await using Stream result = await fetcher.Fetch(ImageUri, CancellationToken.None);
result.Length.ShouldBe(3);
}
// the advertised length is the cheap reject: the body must not be pulled at all
[Test]
public async Task Should_Reject_An_Oversized_Content_Length_Without_Reading_The_Body()
{
var body = new TrackingStream(HttpRemoteImageFetcher.MaxImageBytes + 1);
var response = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StreamContent(body) };
response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/png");
response.Content.Headers.ContentLength = HttpRemoteImageFetcher.MaxImageBytes + 1;
HttpRemoteImageFetcher fetcher = FetcherReturning(response);
await Should.ThrowAsync<InvalidOperationException>(() => fetcher.Fetch(ImageUri, CancellationToken.None));
body.BytesRead.ShouldBe(0);
}
// ...and a host that lies about (or omits) Content-Length is still capped, because the copy
// itself counts bytes. (the DECODE bomb -- small on the wire, huge in memory -- is a different
// problem, bounded by the decode/retention budgets in ImageElementBase, not by this.)
[Test]
public async Task Should_Cap_A_Body_That_Does_Not_Advertise_Its_Length()
{
var body = new ChunkedZeroStream(HttpRemoteImageFetcher.MaxImageBytes * 2);
var response = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StreamContent(body) };
response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/png");
response.Content.Headers.ContentLength = null;
HttpRemoteImageFetcher fetcher = FetcherReturning(response);
await Should.ThrowAsync<InvalidOperationException>(() => fetcher.Fetch(ImageUri, CancellationToken.None));
// proves this went through the COPY cap and not the Content-Length early reject: the body
// really was read, and it stopped at the limit rather than draining all 20 MiB.
body.Position.ShouldBeGreaterThan(0);
// the over-read bound is ONE RENTED BUFFER, and ArrayPool.Rent(81920) actually hands back
// 131072 -- deriving it keeps this honest if the request size or the pool bucketing changes.
byte[] rented = ArrayPool<byte>.Shared.Rent(81920);
ArrayPool<byte>.Shared.Return(rented);
body.Position.ShouldBeLessThanOrEqualTo(HttpRemoteImageFetcher.MaxImageBytes + rented.Length);
}
// a host that accepts the connection and then hangs must not stall stream startup forever
[Test]
public async Task Should_Time_Out_A_Hanging_Host()
{
var fetcher = new HttpRemoteImageFetcher(
new StubHttpClientFactory(new HangingHttpMessageHandler()),
new RecyclableMemoryStreamManager(),
Substitute.For<ILogger<HttpRemoteImageFetcher>>(),
TimeSpan.FromMilliseconds(100));
await Should.ThrowAsync<TimeoutException>(() => fetcher.Fetch(ImageUri, CancellationToken.None));
}
// THE headline claim: under ResponseHeadersRead the body read falls outside HttpClient.Timeout,
// so a host that returns headers promptly and then drips the body must still hit our deadline.
// the hanging-host test above only covers the pre-headers case, which a plain HttpClient.Timeout
// would already bound -- this is the one that pins the actual design.
[Test]
public async Task Should_Time_Out_A_Slow_Drip_Body()
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StreamContent(new SlowDripStream())
};
response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/png");
var fetcher = new HttpRemoteImageFetcher(
new StubHttpClientFactory(new FixedResponseHttpMessageHandler(response)),
new RecyclableMemoryStreamManager(),
Substitute.For<ILogger<HttpRemoteImageFetcher>>(),
TimeSpan.FromMilliseconds(200));
await Should.ThrowAsync<TimeoutException>(() => fetcher.Fetch(ImageUri, CancellationToken.None));
}
[Test]
public async Task Should_Throw_On_Transport_Failure()
{
var fetcher = new HttpRemoteImageFetcher(
new StubHttpClientFactory(new ThrowingHttpMessageHandler(new HttpRequestException("no route to host"))),
new RecyclableMemoryStreamManager(),
Substitute.For<ILogger<HttpRemoteImageFetcher>>());
await Should.ThrowAsync<HttpRequestException>(() => fetcher.Fetch(ImageUri, CancellationToken.None));
}
// caller cancellation (shutdown / client disconnect) is a genuine signal and must stay an
// OperationCanceledException rather than being relabelled as our timeout
[Test]
public async Task Should_Propagate_Caller_Cancellation()
{
var fetcher = new HttpRemoteImageFetcher(
new StubHttpClientFactory(new HangingHttpMessageHandler()),
new RecyclableMemoryStreamManager(),
Substitute.For<ILogger<HttpRemoteImageFetcher>>());
using var cts = new CancellationTokenSource();
await cts.CancelAsync();
await Should.ThrowAsync<OperationCanceledException>(() => fetcher.Fetch(ImageUri, cts.Token));
}
private static HttpResponseMessage Ok(byte[] payload, string mediaType)
{
var response = new HttpResponseMessage(HttpStatusCode.OK) { Content = new ByteArrayContent(payload) };
response.Content.Headers.ContentType = mediaType is null ? null : new MediaTypeHeaderValue(mediaType);
return response;
}
private static HttpRemoteImageFetcher FetcherReturning(HttpResponseMessage response) =>
new(
new StubHttpClientFactory(new FixedResponseHttpMessageHandler(response)),
new RecyclableMemoryStreamManager(),
Substitute.For<ILogger<HttpRemoteImageFetcher>>());
}
@@ -161,87 +161,4 @@ public class HttpRemoteStreamProberTests
new(
new StubHttpClientFactory(new StatusCodeHttpMessageHandler(statusCode, finalUri)),
Substitute.For<ILogger<HttpRemoteStreamProber>>());
private sealed class StubHttpClientFactory(HttpMessageHandler handler) : IHttpClientFactory
{
public HttpClient CreateClient(string name) => new(handler, disposeHandler: false);
}
private sealed class StatusCodeHttpMessageHandler(HttpStatusCode statusCode, string finalUri = null)
: HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
// HttpClient rewrites RequestMessage.RequestUri to the final hop when it follows a
// redirect; finalUri lets a test stand in for "the media server answered this".
if (finalUri is not null)
{
request.RequestUri = new Uri(finalUri);
}
return Task.FromResult(new HttpResponseMessage(statusCode) { RequestMessage = request });
}
}
private sealed class FixedResponseHttpMessageHandler(HttpResponseMessage response) : HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
response.RequestMessage = request;
return Task.FromResult(response);
}
}
/// <summary>A readable stream that records how many bytes were actually pulled from it.</summary>
private sealed class TrackingStream(long length) : Stream
{
public int BytesRead { get; private set; }
public override bool CanRead => true;
public override bool CanSeek => false;
public override bool CanWrite => false;
public override long Length => length;
public override long Position
{
get => BytesRead;
set => throw new NotSupportedException();
}
public override int Read(byte[] buffer, int offset, int count)
{
if (BytesRead >= length)
{
return 0;
}
int toRead = (int)Math.Min(count, length - BytesRead);
Array.Clear(buffer, offset, toRead);
BytesRead += toRead;
return toRead;
}
public override void Flush()
{
}
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}
private sealed class ThrowingHttpMessageHandler(Exception exception) : HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken) =>
Task.FromException<HttpResponseMessage>(exception);
}
}
@@ -0,0 +1,186 @@
using System.Diagnostics;
using System.Net;
namespace ErsatzTV.Infrastructure.Tests.Streaming;
/// <summary>
/// Shared fakes for the HTTP-backed streaming services. Extracted from
/// <see cref="HttpRemoteStreamProberTests" /> when <see cref="HttpRemoteImageFetcherTests" />
/// needed the same harness. (ersatztv#511)
/// </summary>
internal sealed class StubHttpClientFactory(HttpMessageHandler handler) : IHttpClientFactory
{
public HttpClient CreateClient(string name) => new(handler, disposeHandler: false);
}
internal sealed class StatusCodeHttpMessageHandler(HttpStatusCode statusCode, string finalUri = null)
: HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
// HttpClient rewrites RequestMessage.RequestUri to the final hop when it follows a
// redirect; finalUri lets a test stand in for "the media server answered this".
if (finalUri is not null)
{
request.RequestUri = new Uri(finalUri);
}
return Task.FromResult(new HttpResponseMessage(statusCode) { RequestMessage = request });
}
}
internal sealed class FixedResponseHttpMessageHandler(HttpResponseMessage response) : HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
response.RequestMessage = request;
return Task.FromResult(response);
}
}
/// <summary>A readable stream that records how many bytes were actually pulled from it.</summary>
internal sealed class TrackingStream(long length) : Stream
{
public int BytesRead { get; private set; }
public override bool CanRead => true;
public override bool CanSeek => false;
public override bool CanWrite => false;
public override long Length => length;
public override long Position
{
get => BytesRead;
set => throw new NotSupportedException();
}
public override int Read(byte[] buffer, int offset, int count)
{
if (BytesRead >= length)
{
return 0;
}
int toRead = (int)Math.Min(count, length - BytesRead);
Array.Clear(buffer, offset, toRead);
BytesRead += toRead;
return toRead;
}
public override void Flush()
{
}
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}
/// <summary>
/// A readable stream that returns headers-worth of data instantly and then drips forever —
/// the case that <see cref="HttpCompletionOption.ResponseHeadersRead" /> leaves outside
/// <see cref="HttpClient.Timeout" />.
/// </summary>
internal sealed class SlowDripStream : Stream
{
public override bool CanRead => true;
public override bool CanSeek => false;
public override bool CanWrite => false;
public override long Length => throw new NotSupportedException();
public override long Position
{
get => 0;
set => throw new NotSupportedException();
}
public override async ValueTask<int> ReadAsync(
Memory<byte> buffer,
CancellationToken cancellationToken = default)
{
await Task.Delay(Timeout.Infinite, cancellationToken);
throw new UnreachableException();
}
public override int Read(byte[] buffer, int offset, int count) =>
ReadAsync(buffer.AsMemory(offset, count), CancellationToken.None).AsTask().GetAwaiter().GetResult();
public override void Flush()
{
}
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}
internal sealed class ThrowingHttpMessageHandler(Exception exception) : HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken) =>
Task.FromException<HttpResponseMessage>(exception);
}
/// <summary>
/// A handler that never completes until the request is cancelled — stands in for a host that
/// accepts the connection and then hangs.
/// </summary>
internal sealed class HangingHttpMessageHandler : HttpMessageHandler
{
protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
await Task.Delay(Timeout.Infinite, cancellationToken);
throw new UnreachableException();
}
}
/// <summary>
/// A readable stream that yields <paramref name="length" /> bytes but only ever a little at a
/// time, so a size cap has to be enforced during the copy rather than from Content-Length.
/// </summary>
internal sealed class ChunkedZeroStream(long length, int chunkSize = 4096) : Stream
{
private long _position;
public override bool CanRead => true;
public override bool CanSeek => false;
public override bool CanWrite => false;
public override long Length => length;
public override long Position
{
get => _position;
set => throw new NotSupportedException();
}
public override int Read(byte[] buffer, int offset, int count)
{
if (_position >= length)
{
return 0;
}
int toRead = (int)Math.Min(Math.Min(count, chunkSize), length - _position);
Array.Clear(buffer, offset, toRead);
_position += toRead;
return toRead;
}
public override void Flush()
{
}
public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
public override void SetLength(long value) => throw new NotSupportedException();
public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();
}
@@ -0,0 +1,83 @@
using ErsatzTV.Core.Images;
using ErsatzTV.Core.Interfaces.Images;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats;
using Image = SixLabors.ImageSharp.Image;
namespace ErsatzTV.Infrastructure.Images;
public class RemoteImageValidator : IRemoteImageValidator
{
public async Task Validate(Stream stream, Uri uri, CancellationToken cancellationToken)
{
using Image _ = await DecodeAndValidate(stream, uri, cancellationToken);
}
/// <summary>
/// Decodes a remote image only after the header says decoding it is affordable.
/// </summary>
/// <remarks>
/// The fetcher's byte cap does NOT bound this: a decompression bomb is small on the wire and
/// huge in memory. A 4 KB PNG can declare 30000x30000 (~3.6 GB), and a 60 KiB GIF can
/// declare 2500x2500 across 600 frames (~14 GiB). The budget is therefore on the PRODUCT of
/// dimensions and frames, read from the header before the decoder allocates.
/// Local images are deliberately not checked — they are files an operator put on disk, not
/// bytes an arbitrary host returned. (ersatztv#511)
/// </remarks>
public static async Task<Image> DecodeAndValidate(Stream stream, Uri uri, CancellationToken cancellationToken)
{
if (!stream.CanSeek)
{
// Identify consumes the stream, so the decode below needs to rewind it. Fail with the
// real reason rather than letting Position throw NotSupportedException, which the
// caller's blanket catch would report as a generic initialization failure.
throw new InvalidOperationException(
$"Remote image {uri} was returned on a non-seekable stream; IRemoteImageFetcher must "
+ "return a fully buffered, seekable stream");
}
// MaxFrames = 1 on the IDENTIFY is not a limit, it is a workaround: a default Identify
// throws InvalidImageContentException on most APNGs — including files ImageSharp's own
// PngEncoder wrote, which Image.Load then reads back perfectly (measured: 13 of 16 shapes).
// Without this, adding the header pre-pass would silently disable every animated-PNG logo
// that worked before this change. Only Width/Height are read below, and those stay correct.
ImageInfo info = await Image.IdentifyAsync(
new DecoderOptions { MaxFrames = 1 },
stream,
cancellationToken);
// DIMENSIONS from the header are trustworthy; the FRAME COUNT is not, and is deliberately
// not used as a budget input. Measured on ImageSharp 3.1.12: an APNG reports
// FrameMetadataCollection.Count == 0 while the decoder happily produces 600 frames, so a
// header-derived frame budget is enforced on a number the decoder does not honor — a
// 134 KiB file decodes to ~36 GiB. (Second adversarial re-review; ersatztv#511.)
RemoteImageDecodeBudget.EnsureDimensionsAffordable(info.Width, info.Height, uri);
int affordableFrames = RemoteImageDecodeBudget.AffordableFrames(info.Width, info.Height);
stream.Position = 0;
// MaxFrames is enforced BY THE DECODER, so it holds whatever the header claimed — measured
// as honored by every animated decoder here (APNG, GIF, WebP, TIFF). Ask for two more than
// the budget allows so that an animation exactly AT the limit still decodes in full, while
// anything over it is present in the decoded image for the post-decode check below to
// reject. Slop is at most two frames: MaxFrames = N yields N frames for GIF/WebP/TIFF but
// N-1 for APNG, so the exact count varies by format and only the upper bound matters.
var decoderOptions = new DecoderOptions { MaxFrames = (uint)(affordableFrames + 2) };
Image image = await Image.LoadAsync(decoderOptions, stream, cancellationToken);
try
{
// re-verify against REALITY rather than against the header. this is the check that
// actually holds; everything above it only avoids decoding when we can tell in advance.
RemoteImageDecodeBudget.EnsureDecodeAffordable(image.Width, image.Height, image.Frames.Count, uri);
return image;
}
catch
{
image.Dispose();
throw;
}
}
}
@@ -0,0 +1,35 @@
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Streaming;
using LanguageExt;
namespace ErsatzTV.Infrastructure.Images;
public class RemoteLogoCacher(
IRemoteImageFetcher fetcher,
IRemoteImageValidator validator,
IImageCache imageCache) : IRemoteLogoCacher
{
public async Task<Either<BaseError, string>> CacheFromUrl(Uri uri, CancellationToken cancellationToken)
{
try
{
await using Stream stream = await fetcher.Fetch(uri, cancellationToken);
// validate by decoding under the budget (throws if unsafe); we cache the raw bytes
await validator.Validate(stream, uri, cancellationToken);
stream.Position = 0;
return await imageCache.SaveArtworkToCache(stream, ArtworkKind.Logo);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
return BaseError.New($"Could not download logo from {uri}: {ex.Message}");
}
}
}
@@ -16,6 +16,7 @@ public class GraphicsEngine(
ITempFilePool tempFilePool,
IConfigElementRepository configElementRepository,
ILocalStatisticsProvider localStatisticsProvider,
IRemoteImageFetcher remoteImageFetcher,
ILogger<GraphicsEngine> logger)
: IGraphicsEngine
{
@@ -33,7 +34,10 @@ public class GraphicsEngine(
switch (element)
{
case WatermarkElementContext watermarkElementContext:
var watermark = new WatermarkElement(watermarkElementContext.Options, logger);
var watermark = new WatermarkElement(
watermarkElementContext.Options,
remoteImageFetcher,
logger);
if (watermark.IsValid)
{
elements.Add(watermark);
@@ -42,7 +46,8 @@ public class GraphicsEngine(
break;
case ImageElementContext imageElementContext:
elements.Add(new ImageElement(imageElementContext.ImageElement, logger));
elements.Add(
new ImageElement(imageElementContext.ImageElement, remoteImageFetcher, logger));
break;
case TextElementDataContext textElementContext:
@@ -63,23 +68,23 @@ public class GraphicsEngine(
break;
case SubtitleElementDataContext subtitleElementContext:
{
var variables = context.TemplateVariables.ToDictionary();
foreach (KeyValuePair<string, string> variable in subtitleElementContext.Variables)
{
variables.Add(variable.Key, variable.Value);
var variables = context.TemplateVariables.ToDictionary();
foreach (KeyValuePair<string, string> variable in subtitleElementContext.Variables)
{
variables.Add(variable.Key, variable.Value);
}
var subtitleElement = new SubtitleElement(
templateFunctions,
tempFilePool,
subtitleElementContext.SubtitleElement,
variables,
logger);
elements.Add(subtitleElement);
break;
}
var subtitleElement = new SubtitleElement(
templateFunctions,
tempFilePool,
subtitleElementContext.SubtitleElement,
variables,
logger);
elements.Add(subtitleElement);
break;
}
}
}
@@ -6,7 +6,10 @@ using SkiaSharp;
namespace ErsatzTV.Infrastructure.Streaming.Graphics;
public class ImageElement(ImageGraphicsElement imageGraphicsElement, ILogger logger) : ImageElementBase
public class ImageElement(
ImageGraphicsElement imageGraphicsElement,
IRemoteImageFetcher remoteImageFetcher,
ILogger logger) : ImageElementBase(remoteImageFetcher)
{
private Option<Expression> _maybeOpacityExpression;
private float _opacity;
@@ -1,6 +1,9 @@
using System.Runtime.InteropServices;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Images;
using ErsatzTV.Core.Interfaces.Streaming;
using ErsatzTV.FFmpeg.State;
using ErsatzTV.Infrastructure.Images;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats;
using SixLabors.ImageSharp.Formats.Gif;
@@ -14,8 +17,18 @@ using Image = SixLabors.ImageSharp.Image;
namespace ErsatzTV.Infrastructure.Streaming.Graphics;
public abstract class ImageElementBase : GraphicsElement, IDisposable
public abstract class ImageElementBase(IRemoteImageFetcher remoteImageFetcher) : GraphicsElement, IDisposable
{
/// <summary>
/// Ceiling on total pixels RETAINED after scaling — frames x scaled width x scaled height.
/// Independent of the source decode budget in <see cref="RemoteImageDecodeBudget" />: a
/// 100x100 source is trivial to decode but, at 600
/// frames scaled to 1920x1080, retains ~5 GB of <see cref="SKBitmap" />. At 4 bytes per
/// pixel this bounds retention at ~800 MB, which still allows ~96 full-frame 1080p frames
/// (~3s at 30fps) or 600 frames of a 577x577 logo.
/// </summary>
internal const long MaxRemoteScaledPixels = 200_000_000;
private readonly List<double> _frameDelays = [];
private readonly List<SKBitmap> _scaledFrames = [];
private double _animatedDurationSeconds;
@@ -49,9 +62,8 @@ public abstract class ImageElementBase : GraphicsElement, IDisposable
if (isRemoteUri)
{
using var client = new HttpClient();
await using Stream imageStream = await client.GetStreamAsync(uriResult, cancellationToken);
_sourceImage = await Image.LoadAsync(imageStream, cancellationToken);
await using Stream imageStream = await remoteImageFetcher.Fetch(uriResult, cancellationToken);
_sourceImage = await DecodeRemoteImage(imageStream, uriResult, cancellationToken);
}
else
{
@@ -67,6 +79,11 @@ public abstract class ImageElementBase : GraphicsElement, IDisposable
scaledHeight = (int)(scaledWidth * aspectRatio);
}
if (isRemoteUri)
{
EnsureScaledFramesAffordable(_sourceImage.Frames.Count, scaledWidth, scaledHeight, uriResult);
}
(int horizontalMargin, int verticalMargin) = placeWithinSourceContent
? SourceContentMargins(
squarePixelFrameSize,
@@ -103,6 +120,36 @@ public abstract class ImageElementBase : GraphicsElement, IDisposable
}
}
/// <summary>
/// Decodes a remote image only after the header says decoding it is affordable.
/// </summary>
/// <remarks>
/// The fetcher's byte cap does NOT bound this: a decompression bomb is small on the wire and
/// huge in memory. A 4 KB PNG can declare 30000x30000 (~3.6 GB), and a 60 KiB GIF can
/// declare 2500x2500 across 600 frames (~14 GiB). The budget is therefore on the PRODUCT of
/// dimensions and frames, read from the header before the decoder allocates.
/// Local images are deliberately not checked — they are files an operator put on disk, not
/// bytes an arbitrary host returned. (ersatztv#511)
/// </remarks>
internal static async Task<Image> DecodeRemoteImage(Stream stream, Uri uri, CancellationToken cancellationToken)
=> await RemoteImageValidator.DecodeAndValidate(stream, uri, cancellationToken);
/// <summary>
/// Bounds what is RETAINED after scaling. Separate from the source budget because the two
/// are independent: a cheap-to-decode 100x100 source scaled to 1920x1080 across 600 frames
/// retains ~5 GB. Only applied to remote images, matching the rest of this guard.
/// </summary>
internal static void EnsureScaledFramesAffordable(int frameCount, int scaledWidth, int scaledHeight, Uri uri)
{
long retainedPixels = (long)Math.Max(frameCount, 1) * scaledWidth * scaledHeight;
if (retainedPixels > MaxRemoteScaledPixels)
{
throw new InvalidOperationException(
$"Remote image {uri} scales to {frameCount} frames of {scaledWidth}x{scaledHeight} "
+ $"({retainedPixels} pixels), over the {MaxRemoteScaledPixels} pixel limit");
}
}
protected static SKBitmap ToSkiaBitmap(Image image)
{
using Image<Rgba32> rgbaImage = image.CloneAs<Rgba32>();
@@ -17,7 +17,8 @@ public class WatermarkElement : ImageElementBase
private Option<Expression> _maybeOpacityExpression;
private float _opacity;
public WatermarkElement(WatermarkOptions watermarkOptions, ILogger logger)
public WatermarkElement(WatermarkOptions watermarkOptions, IRemoteImageFetcher remoteImageFetcher, ILogger logger)
: base(remoteImageFetcher)
{
_logger = logger;
// TODO: better model coming in here?
@@ -0,0 +1,157 @@
using System.Buffers;
using ErsatzTV.Core.Interfaces.Streaming;
using Microsoft.Extensions.Logging;
using Microsoft.IO;
namespace ErsatzTV.Infrastructure.Streaming;
/// <summary>
/// Fetches a remote graphics-engine image over HTTP with a bounded timeout and a bounded size.
/// </summary>
/// <remarks>
/// This runs during graphics-engine element initialization, which happens inside stream startup
/// while ffmpeg waits on the pipe — so an unbounded fetch stalls the tune. Every failure here is
/// surfaced as an exception and the calling element disables itself. (ersatztv#511)
/// </remarks>
public class HttpRemoteImageFetcher : IRemoteImageFetcher
{
/// <summary>Named <see cref="HttpClient" /> configured with the redirect cap in Startup.</summary>
public const string HttpClientName = "RemoteImage";
/// <summary>
/// Covers the whole exchange — connect, headers AND body — because the body read happens
/// outside <see cref="HttpClient.Timeout" /> under <see cref="HttpCompletionOption.ResponseHeadersRead" />,
/// so a slow-drip host would otherwise hang forever. (docs/decisions.md, #289)
/// </summary>
internal static readonly TimeSpan DefaultFetchTimeout = TimeSpan.FromSeconds(10);
/// <summary>
/// Wire-size cap. A channel logo or overlay is orders of magnitude smaller than this.
/// This bounds transfer and buffering only — it is NOT decode-bomb protection, since a bomb
/// is by definition small on the wire. That check lives in
/// <c>ImageElementBase.DecodeRemoteImage</c>, which reads the declared dimensions and frame
/// count from the header before the decoder allocates.
/// </summary>
internal const long MaxImageBytes = 10 * 1024 * 1024;
private readonly TimeSpan _fetchTimeout;
private readonly IHttpClientFactory _httpClientFactory;
private readonly ILogger<HttpRemoteImageFetcher> _logger;
private readonly RecyclableMemoryStreamManager _memoryStreamManager;
public HttpRemoteImageFetcher(
IHttpClientFactory httpClientFactory,
RecyclableMemoryStreamManager memoryStreamManager,
ILogger<HttpRemoteImageFetcher> logger)
: this(httpClientFactory, memoryStreamManager, logger, DefaultFetchTimeout)
{
}
// the timeout is only parameterized so a test can prove the deadline fires without waiting the
// real ten seconds; production always goes through the constructor above.
internal HttpRemoteImageFetcher(
IHttpClientFactory httpClientFactory,
RecyclableMemoryStreamManager memoryStreamManager,
ILogger<HttpRemoteImageFetcher> logger,
TimeSpan fetchTimeout)
{
_httpClientFactory = httpClientFactory;
_memoryStreamManager = memoryStreamManager;
_logger = logger;
_fetchTimeout = fetchTimeout;
}
public async Task<Stream> Fetch(Uri uri, CancellationToken cancellationToken)
{
using var timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeoutCts.CancelAfter(_fetchTimeout);
try
{
using HttpClient client = _httpClientFactory.CreateClient(HttpClientName);
// set here, not only in the DI registration, so the deadline cannot silently widen to
// HttpClient's 100s default if that registration is ever dropped or reordered. the
// factory hands back a fresh wrapper each call, so mutating it is safe.
client.Timeout = Timeout.InfiniteTimeSpan;
using HttpResponseMessage response = await client.GetAsync(
uri,
HttpCompletionOption.ResponseHeadersRead,
timeoutCts.Token);
response.EnsureSuccessStatusCode();
string mediaType = response.Content.Headers.ContentType?.MediaType;
if (!IsAcceptableMediaType(mediaType))
{
throw new InvalidOperationException(
$"Remote image {uri} returned content type '{mediaType}', which is not an image");
}
// the advertised length is a cheap early reject; it is not trusted, because it can be
// absent or a lie. the copy below is what actually enforces the cap.
long? contentLength = response.Content.Headers.ContentLength;
if (contentLength > MaxImageBytes)
{
throw new InvalidOperationException(
$"Remote image {uri} advertises {contentLength} bytes, over the {MaxImageBytes} byte limit");
}
await using Stream source = await response.Content.ReadAsStreamAsync(timeoutCts.Token);
return await CopyCapped(source, uri, timeoutCts.Token);
}
catch (OperationCanceledException canceled) when (!cancellationToken.IsCancellationRequested)
{
// OUR deadline fired, not the caller's. re-thrown as a TimeoutException so the element's
// warning names the real cause; caller cancellation (shutdown / client disconnect) still
// propagates as OperationCanceledException.
_logger.LogWarning(
"Timed out after {Seconds}s fetching remote image {Uri}",
_fetchTimeout.TotalSeconds,
uri);
throw new TimeoutException($"Timed out fetching remote image {uri}", canceled);
}
}
// a missing content type is allowed (some hosts omit it) and octet-stream is allowed (a common
// default for statically served files). anything else that is positively NOT an image -- an html
// error page, say -- is rejected before it reaches the decoder.
private static bool IsAcceptableMediaType(string mediaType) =>
string.IsNullOrWhiteSpace(mediaType)
|| mediaType.StartsWith("image/", StringComparison.OrdinalIgnoreCase)
|| mediaType.Equals("application/octet-stream", StringComparison.OrdinalIgnoreCase);
private async Task<Stream> CopyCapped(Stream source, Uri uri, CancellationToken cancellationToken)
{
MemoryStream buffer = _memoryStreamManager.GetStream(nameof(HttpRemoteImageFetcher));
byte[] chunk = ArrayPool<byte>.Shared.Rent(81920);
try
{
int read;
while ((read = await source.ReadAsync(chunk.AsMemory(), cancellationToken)) > 0)
{
if (buffer.Length + read > MaxImageBytes)
{
throw new InvalidOperationException(
$"Remote image {uri} exceeds the {MaxImageBytes} byte limit");
}
await buffer.WriteAsync(chunk.AsMemory(0, read), cancellationToken);
}
}
catch
{
await buffer.DisposeAsync();
throw;
}
finally
{
ArrayPool<byte>.Shared.Return(chunk);
}
buffer.Position = 0;
return buffer;
}
}
@@ -0,0 +1,87 @@
using System.Text;
using ErsatzTV.Mcp;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Mcp.Tests;
[TestFixture]
public class BoundedLineReaderTests
{
[Test]
public async Task ReadLineAsync_Should_Return_Line_Without_Trailing_Newline()
{
using var reader = new StringReader("hello world\n");
BoundedLineReader.Line line = await BoundedLineReader.ReadLineAsync(reader, 1024);
line.EndOfStream.ShouldBeFalse();
line.Overflowed.ShouldBeFalse();
line.Text.ShouldBe("hello world");
}
[Test]
public async Task ReadLineAsync_Should_Return_Line_Of_Exactly_Cap_Length_Intact()
{
// The cap is the inclusive max: a line of exactly `cap` chars is returned, not overflowed.
using var reader = new StringReader("abcdefgh\n");
BoundedLineReader.Line line = await BoundedLineReader.ReadLineAsync(reader, 8);
line.Overflowed.ShouldBeFalse();
line.Text.ShouldBe("abcdefgh");
}
[Test]
public async Task ReadLineAsync_Should_Strip_Carriage_Return_In_Crlf()
{
using var reader = new StringReader("hello\r\n");
BoundedLineReader.Line line = await BoundedLineReader.ReadLineAsync(reader, 1024);
line.Text.ShouldBe("hello");
}
[Test]
public async Task ReadLineAsync_Should_Signal_End_Of_Stream()
{
using var reader = new StringReader(string.Empty);
BoundedLineReader.Line line = await BoundedLineReader.ReadLineAsync(reader, 1024);
line.EndOfStream.ShouldBeTrue();
}
[Test]
public async Task ReadLineAsync_Should_Overflow_And_Not_Buffer_Oversized_Line()
{
// A line far longer than the cap must be reported overflowed with no buffered text —
// the memory-exhaustion guard.
string oversized = new string('x', 10_000) + "\n";
using var reader = new StringReader(oversized);
BoundedLineReader.Line line = await BoundedLineReader.ReadLineAsync(reader, 16);
line.EndOfStream.ShouldBeFalse();
line.Overflowed.ShouldBeTrue();
line.Text.ShouldBeEmpty();
}
[Test]
public async Task ReadLineAsync_Should_Keep_Subsequent_Lines_Aligned_After_Overflow()
{
// After draining an oversized line, the next line must still be read intact.
var content = new StringBuilder()
.Append(new string('x', 100)).Append('\n')
.Append("good\n")
.ToString();
using var reader = new StringReader(content);
BoundedLineReader.Line first = await BoundedLineReader.ReadLineAsync(reader, 16);
BoundedLineReader.Line second = await BoundedLineReader.ReadLineAsync(reader, 16);
first.Overflowed.ShouldBeTrue();
second.Overflowed.ShouldBeFalse();
second.Text.ShouldBe("good");
}
}
@@ -0,0 +1,24 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="NUnit" />
<PackageReference Include="NUnit.Analyzers">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="NUnit3TestAdapter" />
<PackageReference Include="Shouldly" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\ErsatzTV.Mcp\ErsatzTV.Mcp.csproj" />
</ItemGroup>
</Project>
@@ -0,0 +1,485 @@
using System.Net;
using System.Net.Http.Headers;
using System.Text.Json;
using ErsatzTV.Mcp;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Mcp.Tests;
[TestFixture]
public class ErsatzTvApiClientTests
{
private static ToolDefinition GetChannel() => new(
"ersatztv_get_channel",
"Get channel",
HttpMethod.Get,
"/api/v1/channels/{id}",
ToolInputSchemas.Object(new SchemaProperty("id", "integer", "Channel id", Required: true)));
[Test]
public async Task CallToolAsync_Should_Substitute_Path_Parameters_And_Send_Api_Key()
{
CapturingHandler handler = new("""{"id":12,"name":"Kids"}""");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost:8409/"), "secret"));
ToolCallResult result = await client.CallToolAsync(
GetChannel(),
JsonDocument.Parse("""{"id":12}""").RootElement,
CancellationToken.None);
result.IsError.ShouldBeFalse();
result.Text.ShouldBe("""{"id":12,"name":"Kids"}""");
handler.RequestUri.ShouldBe(new Uri("http://localhost:8409/api/v1/channels/12"));
handler.ApiKey.ShouldBe("secret");
}
[Test]
public async Task CallToolAsync_Should_Url_Encode_Path_Parameters()
{
CapturingHandler handler = new("""{"id":1}""");
var client = new ErsatzTvApiClient(new HttpClient(handler), new ErsatzTvApiClientOptions(new Uri("http://localhost"), null));
await client.CallToolAsync(
new ToolDefinition(
"ersatztv_get_resolution_by_name",
"Get resolution",
HttpMethod.Get,
"/api/v1/ffmpeg/resolution/by-name/{name}",
ToolInputSchemas.Object(new SchemaProperty("name", "string", "Resolution name", Required: true))),
JsonDocument.Parse("""{"name":"1920 x 1080"}""").RootElement,
CancellationToken.None);
handler.RequestUri.ShouldBe(new Uri("http://localhost/api/v1/ffmpeg/resolution/by-name/1920%20x%201080"));
}
[Test]
public async Task CallToolAsync_Should_Return_Error_Result_For_Non_Success_Status()
{
CapturingHandler handler = new("""{"status":404,"title":"Resource not found"}""", HttpStatusCode.NotFound);
var client = new ErsatzTvApiClient(new HttpClient(handler), new ErsatzTvApiClientOptions(new Uri("http://localhost"), null));
ToolCallResult result = await client.CallToolAsync(
GetChannel(),
JsonDocument.Parse("""{"id":404}""").RootElement,
CancellationToken.None);
result.IsError.ShouldBeTrue();
result.Text.ShouldContain("404");
result.Text.ShouldContain("Resource not found");
}
[Test]
public async Task CallToolAsync_Should_Refuse_Non_Get_Tool_When_Read_Only()
{
CapturingHandler handler = new("{}");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), null));
ToolCallResult result = await client.CallToolAsync(
new ToolDefinition(
"ersatztv_delete_channel",
"Delete channel",
HttpMethod.Delete,
"/api/v1/channels/{id}",
ToolInputSchemas.Object(new SchemaProperty("id", "integer", "Channel id", Required: true))),
JsonDocument.Parse("""{"id":12}""").RootElement,
CancellationToken.None);
result.IsError.ShouldBeTrue();
result.Text.ShouldContain("read-only");
// The request must never reach the API.
handler.RequestUri.ShouldBeNull();
}
[Test]
public async Task CallToolAsync_Should_Allow_Non_Get_Tool_When_Writes_Enabled()
{
CapturingHandler handler = new("", HttpStatusCode.NoContent);
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), null, AllowWrites: true));
ToolCallResult result = await client.CallToolAsync(
new ToolDefinition(
"ersatztv_delete_channel",
"Delete channel",
HttpMethod.Delete,
"/api/v1/channels/{id}",
ToolInputSchemas.Object(new SchemaProperty("id", "integer", "Channel id", Required: true))),
JsonDocument.Parse("""{"id":12}""").RootElement,
CancellationToken.None);
result.IsError.ShouldBeFalse();
handler.RequestUri.ShouldBe(new Uri("http://localhost/api/v1/channels/12"));
}
[Test]
public async Task CallToolAsync_Should_Serialize_Non_Path_Args_As_Json_Body_For_Post()
{
CapturingHandler handler = new("""{"id":9,"name":"Kids"}""", HttpStatusCode.Created);
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k", AllowWrites: true));
await client.CallToolAsync(
new ToolDefinition(
"ersatztv_create_collection",
"Create collection",
HttpMethod.Post,
"/api/v1/collections",
ToolInputSchemas.Object(new SchemaProperty("name", "string", "Name", Required: true))),
JsonDocument.Parse("""{"name":"Kids"}""").RootElement,
CancellationToken.None);
handler.RequestUri.ShouldBe(new Uri("http://localhost/api/v1/collections"));
handler.ContentType.ShouldBe("application/json");
handler.Body.ShouldBe("""{"name":"Kids"}""");
}
[Test]
public async Task CallToolAsync_Should_Keep_Path_Args_Out_Of_The_Body_And_Preserve_Array_Values()
{
CapturingHandler handler = new("", HttpStatusCode.NoContent);
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k", AllowWrites: true));
await client.CallToolAsync(
new ToolDefinition(
"ersatztv_add_collection_items",
"Add items",
HttpMethod.Post,
"/api/v1/collections/{id}/items",
ToolInputSchemas.Object(
new SchemaProperty("id", "integer", "Collection id", Required: true),
new SchemaProperty("artistIds", "array", "Artist ids", Required: false, ItemType: "integer"))),
JsonDocument.Parse("""{"id":20,"artistIds":[1,2,3]}""").RootElement,
CancellationToken.None);
handler.RequestUri.ShouldBe(new Uri("http://localhost/api/v1/collections/20/items"));
// The path id must not leak into the body; array values are preserved verbatim.
handler.Body.ShouldBe("""{"artistIds":[1,2,3]}""");
}
[Test]
public async Task CallToolAsync_Should_Send_IfMatch_Header_And_Keep_It_Out_Of_The_Body()
{
CapturingHandler handler = new("[]");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k", AllowWrites: true));
await client.CallToolAsync(
new ToolDefinition(
"ersatztv_update_collection_custom_order",
"Reorder",
HttpMethod.Put,
"/api/v1/collections/{id}/custom-order",
ToolInputSchemas.Object(
new SchemaProperty("id", "integer", "Collection id", Required: true),
new SchemaProperty("mediaItemIds", "array", "Order", Required: true, ItemType: "integer"),
new SchemaProperty("ifMatch", "string", "ETag", Required: false))),
JsonDocument.Parse("""{"id":20,"mediaItemIds":[3,1,2],"ifMatch":"\"5\""}""").RootElement,
CancellationToken.None);
handler.IfMatch.ShouldBe("\"5\"");
// ifMatch is a header, not a body field; the path id is also excluded.
handler.Body.ShouldBe("""{"mediaItemIds":[3,1,2]}""");
}
[Test]
public async Task CallToolAsync_Should_Pass_Through_IfMatch_Wildcard_Force_Write()
{
CapturingHandler handler = new("[]");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k", AllowWrites: true));
await client.CallToolAsync(
new ToolDefinition(
"ersatztv_update_collection_custom_order",
"Reorder",
HttpMethod.Put,
"/api/v1/collections/{id}/custom-order",
ToolInputSchemas.Object(
new SchemaProperty("id", "integer", "Collection id", Required: true),
new SchemaProperty("mediaItemIds", "array", "Order", Required: true, ItemType: "integer"),
new SchemaProperty("ifMatch", "string", "ETag", Required: false))),
JsonDocument.Parse("""{"id":20,"mediaItemIds":[1],"ifMatch":"*"}""").RootElement,
CancellationToken.None);
handler.IfMatch.ShouldBe("*");
}
[Test]
public async Task CallToolAsync_Should_Reject_IfMatch_With_Control_Characters()
{
CapturingHandler handler = new("[]");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k", AllowWrites: true));
// A CR/LF in ifMatch would smuggle a second header via TryAddWithoutValidation — must be
// rejected before the request is sent (SocketsHttpHandler does not strip it).
await Should.ThrowAsync<ArgumentException>(async () => await client.CallToolAsync(
new ToolDefinition(
"ersatztv_update_collection_custom_order",
"Reorder",
HttpMethod.Put,
"/api/v1/collections/{id}/custom-order",
ToolInputSchemas.Object(
new SchemaProperty("id", "integer", "Collection id", Required: true),
new SchemaProperty("mediaItemIds", "array", "Order", Required: true, ItemType: "integer"),
new SchemaProperty("ifMatch", "string", "ETag", Required: false))),
JsonDocument.Parse("{\"id\":20,\"mediaItemIds\":[1],\"ifMatch\":\"\\\"5\\\"\\r\\nX-Evil: 1\"}").RootElement,
CancellationToken.None));
handler.RequestUri.ShouldBeNull();
}
[Test]
public async Task CallToolAsync_Should_Serialize_Explicit_Null_Body_Field()
{
CapturingHandler handler = new("{}");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k", AllowWrites: true));
// An explicit null on an optional field is preserved in the body (clears a nullable API field).
await client.CallToolAsync(
new ToolDefinition(
"ersatztv_update_playout",
"Update playout",
HttpMethod.Put,
"/api/v1/playouts/{id}",
ToolInputSchemas.Object(
new SchemaProperty("id", "integer", "Playout id", Required: true),
new SchemaProperty("dailyRebuildTime", "string", "Daily rebuild time; null clears", Required: false))),
JsonDocument.Parse("""{"id":1,"dailyRebuildTime":null}""").RootElement,
CancellationToken.None);
handler.Body.ShouldBe("""{"dailyRebuildTime":null}""");
}
[Test]
public async Task CallToolAsync_Should_Append_Declared_Query_Parameters_On_Get()
{
CapturingHandler handler = new("[]");
var client = new ErsatzTvApiClient(new HttpClient(handler), new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k"));
await client.CallToolAsync(
new ToolDefinition(
"ersatztv_search_all_items",
"Search",
HttpMethod.Get,
"/api/v1/search/all-items",
ToolInputSchemas.Object(
new SchemaProperty("query", "string", "Query", Required: false),
new SchemaProperty("pageNum", "integer", "Page", Required: false)),
new HashSet<string>(StringComparer.Ordinal) { "query", "pageNum" }),
JsonDocument.Parse("""{"query":"genre:jazz","pageNum":2}""").RootElement,
CancellationToken.None);
handler.RequestUri!.PathAndQuery.ShouldBe("/api/v1/search/all-items?query=genre%3Ajazz&pageNum=2");
}
[Test]
public async Task CallToolAsync_Should_Put_Query_Params_In_Query_Not_Body_For_Post()
{
CapturingHandler handler = new("", HttpStatusCode.Accepted);
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k", AllowWrites: true));
await client.CallToolAsync(
new ToolDefinition(
"ersatztv_scan_library",
"Scan",
HttpMethod.Post,
"/api/v1/libraries/{id}/scan",
ToolInputSchemas.Object(
new SchemaProperty("id", "integer", "Library id", Required: true),
new SchemaProperty("deep", "boolean", "Deep", Required: false)),
new HashSet<string>(StringComparer.Ordinal) { "deep" }),
JsonDocument.Parse("""{"id":3,"deep":true}""").RootElement,
CancellationToken.None);
handler.RequestUri!.PathAndQuery.ShouldBe("/api/v1/libraries/3/scan?deep=true");
// deep is a query param; there is no JSON body.
handler.Body.ShouldBeNull();
}
[Test]
public async Task CallToolAsync_Should_Surface_Response_ETag()
{
CapturingHandler handler = new("""{"items":[]}""", HttpStatusCode.OK, etag: "\"3\"");
var client = new ErsatzTvApiClient(new HttpClient(handler), new ErsatzTvApiClientOptions(new Uri("http://localhost"), "k"));
ToolCallResult result = await client.CallToolAsync(
new ToolDefinition(
"ersatztv_get_collection_items",
"Items",
HttpMethod.Get,
"/api/v1/collections/{id}/items",
ToolInputSchemas.Object(new SchemaProperty("id", "integer", "Collection id", Required: true))),
JsonDocument.Parse("""{"id":20}""").RootElement,
CancellationToken.None);
result.IsError.ShouldBeFalse();
result.Text.ShouldContain("[etag: \"3\"]");
}
[Test]
public async Task CallToolAsync_Should_Truncate_Oversized_Response()
{
CapturingHandler handler = new(new string('x', 500));
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), null, MaxResponseBytes: 16));
ToolCallResult result = await client.CallToolAsync(
new ToolDefinition("ersatztv_list_channels", "List", HttpMethod.Get, "/api/v1/channels", ToolInputSchemas.Empty),
JsonDocument.Parse("{}").RootElement,
CancellationToken.None);
result.Text.ShouldStartWith(new string('x', 16));
result.Text.ShouldContain("truncated");
result.Text.Length.ShouldBeLessThan(500);
}
[Test]
public async Task CallToolAsync_Should_Preserve_Reverse_Proxy_Path_Prefix()
{
CapturingHandler handler = new("""{"id":12}""");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://host/etv/"), null));
await client.CallToolAsync(
GetChannel(),
JsonDocument.Parse("""{"id":12}""").RootElement,
CancellationToken.None);
handler.RequestUri.ShouldBe(new Uri("http://host/etv/api/v1/channels/12"));
}
[Test]
public async Task CallToolAsync_Should_Reject_Unknown_Argument()
{
CapturingHandler handler = new("{}");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), null));
await Should.ThrowAsync<ArgumentException>(async () => await client.CallToolAsync(
GetChannel(),
JsonDocument.Parse("""{"id":12,"evil":"drop"}""").RootElement,
CancellationToken.None));
handler.RequestUri.ShouldBeNull();
}
[Test]
public async Task CallToolAsync_Should_Reject_Dot_Segment_Path_Parameter()
{
CapturingHandler handler = new("{}");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), null));
// ".." would canonicalize the URL onto a different route — must be rejected pre-flight.
await Should.ThrowAsync<ArgumentException>(async () => await client.CallToolAsync(
new ToolDefinition(
"ersatztv_get_resolution_by_name",
"Get resolution",
HttpMethod.Get,
"/api/v1/ffmpeg/resolution/by-name/{name}",
ToolInputSchemas.Object(new SchemaProperty("name", "string", "Resolution name", Required: true))),
JsonDocument.Parse("""{"name":".."}""").RootElement,
CancellationToken.None));
handler.RequestUri.ShouldBeNull();
}
[Test]
public async Task CallToolAsync_Should_Fall_Back_To_Default_Cap_On_Overflowing_Configured_Cap()
{
CapturingHandler handler = new("""{"ok":true}""");
// int.MaxValue would overflow `cap + 1` to a negative array length; the client must
// clamp to the default instead of crashing.
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), null, MaxResponseBytes: int.MaxValue));
ToolCallResult result = await client.CallToolAsync(
new ToolDefinition("ersatztv_list_channels", "List", HttpMethod.Get, "/api/v1/channels", ToolInputSchemas.Empty),
JsonDocument.Parse("{}").RootElement,
CancellationToken.None);
result.IsError.ShouldBeFalse();
result.Text.ShouldBe("""{"ok":true}""");
}
[Test]
public async Task CallToolAsync_Should_Not_Emit_Replacement_Char_When_Truncating_Mid_Codepoint()
{
// "ab😀" — the emoji is a 4-byte sequence starting at byte index 2; a 4-byte cap cuts it
// mid-sequence. The truncated text must end cleanly, not with a U+FFFD replacement char.
CapturingHandler handler = new("ab\U0001F600");
var client = new ErsatzTvApiClient(
new HttpClient(handler),
new ErsatzTvApiClientOptions(new Uri("http://localhost"), null, MaxResponseBytes: 4));
ToolCallResult result = await client.CallToolAsync(
new ToolDefinition("ersatztv_list_channels", "List", HttpMethod.Get, "/api/v1/channels", ToolInputSchemas.Empty),
JsonDocument.Parse("{}").RootElement,
CancellationToken.None);
result.Text.ShouldStartWith("ab");
result.Text.ShouldNotContain("");
result.Text.ShouldContain("truncated");
}
private sealed class CapturingHandler(string response, HttpStatusCode statusCode = HttpStatusCode.OK, string? etag = null)
: HttpMessageHandler
{
public Uri? RequestUri { get; private set; }
public string? ApiKey { get; private set; }
public string? Body { get; private set; }
public string? ContentType { get; private set; }
public string? IfMatch { get; private set; }
protected override Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
RequestUri = request.RequestUri;
ApiKey = request.Headers.TryGetValues("X-Api-Key", out IEnumerable<string>? values)
? values.Single()
: null;
IfMatch = request.Headers.TryGetValues("If-Match", out IEnumerable<string>? ifMatch)
? ifMatch.Single()
: null;
if (request.Content is not null)
{
Body = request.Content.ReadAsStringAsync(cancellationToken).GetAwaiter().GetResult();
ContentType = request.Content.Headers.ContentType?.MediaType;
}
var message = new HttpResponseMessage(statusCode)
{
Content = new StringContent(response)
};
if (etag is not null)
{
message.Headers.ETag = new EntityTagHeaderValue(etag);
}
return Task.FromResult(message);
}
}
}
+149
View File
@@ -0,0 +1,149 @@
using System.Text.Json;
using ErsatzTV.Mcp;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Mcp.Tests;
[TestFixture]
public class McpServerTests
{
[Test]
public async Task HandleAsync_Should_Return_Server_Capabilities_For_Initialize()
{
using JsonDocument response = await HandleAsync("""{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}""");
JsonElement result = response.RootElement.GetProperty("result");
result.GetProperty("protocolVersion").GetString().ShouldBe("2024-11-05");
result.GetProperty("serverInfo").GetProperty("name").GetString().ShouldBe("ersatztv-mcp");
result.GetProperty("capabilities").TryGetProperty("tools", out _).ShouldBeTrue();
}
[Test]
public async Task HandleAsync_Should_List_Tools()
{
using JsonDocument response = await HandleAsync("""{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}""");
string[] toolNames = response.RootElement
.GetProperty("result")
.GetProperty("tools")
.EnumerateArray()
.Select(t => t.GetProperty("name").GetString())
.OfType<string>()
.ToArray();
toolNames.ShouldContain("ersatztv_list_channels");
toolNames.ShouldContain("ersatztv_get_version");
}
[Test]
public async Task HandleAsync_Should_Call_Tool_And_Return_Text_Content()
{
using JsonDocument response = await HandleAsync(
"""{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"ersatztv_get_version","arguments":{}}}""",
new StubToolExecutor("""{"apiVersion":3,"appVersion":"develop"}"""));
JsonElement content = response.RootElement.GetProperty("result").GetProperty("content").EnumerateArray().Single();
content.GetProperty("type").GetString().ShouldBe("text");
content.GetProperty("text").GetString().ShouldBe("""{"apiVersion":3,"appVersion":"develop"}""");
}
[Test]
public async Task HandleAsync_Should_Return_Method_Not_Found_For_Unknown_Tool()
{
using JsonDocument response = await HandleAsync(
"""{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"missing","arguments":{}}}""");
response.RootElement.GetProperty("error").GetProperty("code").GetInt32().ShouldBe(-32602);
string message = response.RootElement.GetProperty("error").GetProperty("message").GetString().ShouldNotBeNull();
message.ShouldContain("Unknown tool");
}
[Test]
public async Task HandleAsync_Should_Return_Parse_Error_For_Malformed_Json()
{
McpServer server = new(new StubToolExecutor("{}"));
// A malformed line must be answered with a JSON-RPC parse error, never crash the loop.
string? response = await server.HandleAsync("{ this is not json", CancellationToken.None);
using JsonDocument document = JsonDocument.Parse(response.ShouldNotBeNull());
document.RootElement.GetProperty("error").GetProperty("code").GetInt32().ShouldBe(-32700);
document.RootElement.GetProperty("id").ValueKind.ShouldBe(JsonValueKind.Null);
}
[Test]
public async Task HandleAsync_Should_Return_Invalid_Request_For_Missing_Method()
{
McpServer server = new(new StubToolExecutor("{}"));
string? response = await server.HandleAsync("""{"jsonrpc":"2.0","id":7}""", CancellationToken.None);
using JsonDocument document = JsonDocument.Parse(response.ShouldNotBeNull());
document.RootElement.GetProperty("error").GetProperty("code").GetInt32().ShouldBe(-32600);
document.RootElement.GetProperty("id").GetInt32().ShouldBe(7);
}
[Test]
public async Task HandleAsync_Should_Return_Invalid_Request_For_Non_Object_Request()
{
McpServer server = new(new StubToolExecutor("{}"));
string? response = await server.HandleAsync("5", CancellationToken.None);
using JsonDocument document = JsonDocument.Parse(response.ShouldNotBeNull());
document.RootElement.GetProperty("error").GetProperty("code").GetInt32().ShouldBe(-32600);
}
[Test]
public async Task HandleAsync_Should_Ignore_Notifications()
{
McpServer server = new(new StubToolExecutor("{}"));
string? response = await server.HandleAsync("""{"jsonrpc":"2.0","method":"notifications/initialized","params":{}}""", CancellationToken.None);
response.ShouldBeNull();
}
[Test]
public async Task HandleAsync_Should_Return_Internal_Error_When_Executor_Throws_Transport_Error()
{
// A network/transport failure must still yield a JSON-RPC error for the id, not escape
// HandleAsync (which would leave a compliant client hanging).
McpServer server = new(new ThrowingToolExecutor(new HttpRequestException("connection refused")));
string? response = await server.HandleAsync(
"""{"jsonrpc":"2.0","id":9,"method":"tools/call","params":{"name":"ersatztv_get_version","arguments":{}}}""",
CancellationToken.None);
using JsonDocument document = JsonDocument.Parse(response.ShouldNotBeNull());
document.RootElement.GetProperty("error").GetProperty("code").GetInt32().ShouldBe(-32603);
document.RootElement.GetProperty("id").GetInt32().ShouldBe(9);
}
private static async Task<JsonDocument> HandleAsync(string request, IToolExecutor? executor = null)
{
McpServer server = new(executor ?? new StubToolExecutor("{}"));
string? response = await server.HandleAsync(request, CancellationToken.None);
response.ShouldNotBeNull();
return JsonDocument.Parse(response);
}
private sealed class StubToolExecutor(string response) : IToolExecutor
{
public Task<ToolCallResult> CallToolAsync(
ToolDefinition tool,
JsonElement arguments,
CancellationToken cancellationToken) =>
Task.FromResult(new ToolCallResult(false, response));
}
private sealed class ThrowingToolExecutor(Exception exception) : IToolExecutor
{
public Task<ToolCallResult> CallToolAsync(
ToolDefinition tool,
JsonElement arguments,
CancellationToken cancellationToken) =>
throw exception;
}
}
@@ -0,0 +1,101 @@
using System.Text.Json;
using ErsatzTV.Mcp;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Mcp.Tests;
[TestFixture]
public class ToolArgumentValidatorTests
{
private static ToolDefinition IdTool() => new(
"ersatztv_get_channel",
"Get channel",
HttpMethod.Get,
"/api/v1/channels/{id}",
ToolInputSchemas.Object(new SchemaProperty("id", "integer", "Channel id", Required: true)));
private static ToolDefinition ArrayTool() => new(
"ersatztv_add_collection_items",
"Add items",
HttpMethod.Post,
"/api/v1/collections/{id}/items",
ToolInputSchemas.Object(
new SchemaProperty("id", "integer", "Collection id", Required: true),
new SchemaProperty("artistIds", "array", "Artist ids", Required: false, ItemType: "integer")));
[Test]
public void Validate_Should_Accept_Well_Formed_Arguments()
{
Should.NotThrow(() =>
ToolArgumentValidator.Validate(IdTool(), JsonDocument.Parse("""{"id":12}""").RootElement));
}
[Test]
public void Validate_Should_Accept_Empty_Arguments_For_No_Param_Tool()
{
ToolDefinition tool = new("ersatztv_get_version", "Version", HttpMethod.Get, "/api/v1/version", ToolInputSchemas.Empty);
Should.NotThrow(() =>
ToolArgumentValidator.Validate(tool, JsonDocument.Parse("{}").RootElement));
}
[Test]
public void Validate_Should_Reject_Unknown_Argument()
{
Should.Throw<ArgumentException>(() =>
ToolArgumentValidator.Validate(IdTool(), JsonDocument.Parse("""{"id":12,"extra":1}""").RootElement));
}
[Test]
public void Validate_Should_Reject_Missing_Required_Argument()
{
Should.Throw<ArgumentException>(() =>
ToolArgumentValidator.Validate(IdTool(), JsonDocument.Parse("{}").RootElement));
}
[Test]
public void Validate_Should_Reject_Wrong_Type()
{
Should.Throw<ArgumentException>(() =>
ToolArgumentValidator.Validate(IdTool(), JsonDocument.Parse("""{"id":"twelve"}""").RootElement));
}
[Test]
public void Validate_Should_Reject_Non_Object_Arguments()
{
Should.Throw<ArgumentException>(() =>
ToolArgumentValidator.Validate(IdTool(), JsonDocument.Parse("[]").RootElement));
}
[Test]
public void Validate_Should_Accept_Array_Argument()
{
Should.NotThrow(() =>
ToolArgumentValidator.Validate(ArrayTool(), JsonDocument.Parse("""{"id":1,"artistIds":[3,4,5]}""").RootElement));
}
[Test]
public void Validate_Should_Reject_Non_Array_For_Array_Argument()
{
Should.Throw<ArgumentException>(() =>
ToolArgumentValidator.Validate(ArrayTool(), JsonDocument.Parse("""{"id":1,"artistIds":7}""").RootElement));
}
[Test]
public void Validate_Should_Accept_Explicit_Null_For_Optional_Property()
{
ToolDefinition tool = new(
"ersatztv_update_playout",
"Update playout",
HttpMethod.Put,
"/api/v1/playouts/{id}",
ToolInputSchemas.Object(
new SchemaProperty("id", "integer", "Playout id", Required: true),
new SchemaProperty("dailyRebuildTime", "string", "null clears", Required: false)));
// An explicit JSON null clears a nullable API field; the validator must not reject it on type.
Should.NotThrow(() =>
ToolArgumentValidator.Validate(tool, JsonDocument.Parse("""{"id":1,"dailyRebuildTime":null}""").RootElement));
}
}
+214
View File
@@ -0,0 +1,214 @@
using System.Text.Json;
using System.Text.RegularExpressions;
using ErsatzTV.Mcp;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Mcp.Tests;
[TestFixture]
public class ToolCatalogTests
{
[Test]
public void All_Should_Expose_Read_Tools_For_The_Six_Families_And_Discovery()
{
string[] names = ToolCatalog.All.Select(t => t.Name).ToArray();
names.ShouldContain("ersatztv_list_channels");
names.ShouldContain("ersatztv_get_channel");
names.ShouldContain("ersatztv_list_collections");
names.ShouldContain("ersatztv_get_collection");
names.ShouldContain("ersatztv_get_collection_items");
names.ShouldContain("ersatztv_list_smart_collections");
names.ShouldContain("ersatztv_list_schedules");
names.ShouldContain("ersatztv_get_schedule_items");
names.ShouldContain("ersatztv_list_playouts");
names.ShouldContain("ersatztv_get_playout");
names.ShouldContain("ersatztv_list_ffmpeg_profiles");
names.ShouldContain("ersatztv_get_version");
names.ShouldContain("ersatztv_list_media_sources");
// Discovery reads for populating collections (#487).
names.ShouldContain("ersatztv_search_all_items");
names.ShouldContain("ersatztv_search_artists");
}
[Test]
public void All_Should_Expose_Cautious_Write_Tools()
{
string[] names = ToolCatalog.All.Select(t => t.Name).ToArray();
names.ShouldContain("ersatztv_create_collection");
names.ShouldContain("ersatztv_update_collection");
names.ShouldContain("ersatztv_delete_collection");
names.ShouldContain("ersatztv_add_collection_items");
names.ShouldContain("ersatztv_remove_collection_item");
names.ShouldContain("ersatztv_update_collection_custom_order");
names.ShouldContain("ersatztv_create_smart_collection");
names.ShouldContain("ersatztv_create_schedule");
names.ShouldContain("ersatztv_create_playout");
names.ShouldContain("ersatztv_delete_playout");
names.ShouldContain("ersatztv_create_channel");
names.ShouldContain("ersatztv_update_channel");
names.ShouldContain("ersatztv_delete_channel");
names.ShouldContain("ersatztv_enable_jellyfin_library_sync");
names.ShouldContain("ersatztv_scan_library");
}
[Test]
public void Create_And_Update_Channel_Should_Share_The_Body_Fields_And_Require_Essentials()
{
ToolDefinition create = ToolCatalog.Find("ersatztv_create_channel").ShouldNotBeNull();
ToolDefinition update = ToolCatalog.Find("ersatztv_update_channel").ShouldNotBeNull();
create.HttpMethod.ShouldBe(HttpMethod.Post);
update.HttpMethod.ShouldBe(HttpMethod.Put);
update.PathTemplate.ShouldBe("/api/v1/channels/{id}");
JsonElement createProps = create.InputSchema.RootElement.GetProperty("properties");
createProps.TryGetProperty("name", out _).ShouldBeTrue();
createProps.TryGetProperty("ffmpegProfileId", out _).ShouldBeTrue();
createProps.TryGetProperty("streamingMode", out _).ShouldBeTrue();
var createRequired = create.InputSchema.RootElement.GetProperty("required")
.EnumerateArray().Select(e => e.GetString()).ToHashSet();
createRequired.ShouldContain("name");
createRequired.ShouldContain("number");
createRequired.ShouldContain("ffmpegProfileId");
// Enums must NOT be forced required (they have server-side defaults).
createRequired.ShouldNotContain("streamingMode");
// Update carries the same body fields plus the route id.
update.InputSchema.RootElement.GetProperty("properties").TryGetProperty("id", out _).ShouldBeTrue();
update.InputSchema.RootElement.GetProperty("properties").TryGetProperty("showInEpg", out _).ShouldBeTrue();
}
[Test]
public void All_Should_Not_Expose_Deferred_Redesign_Workflow_Tools()
{
string[] names = ToolCatalog.All.Select(t => t.Name).ToArray();
names.ShouldNotContain("ersatztv_create_channel_from_lineup");
names.ShouldNotContain("ersatztv_list_channel_templates");
names.ShouldNotContain("ersatztv_browse_library");
names.ShouldNotContain("ersatztv_upload_channel_logo");
names.ShouldNotContain("ersatztv_resume_playback");
}
[Test]
public void All_Should_Not_Expose_Deferred_Large_Dto_Writes()
{
// Deferred as too-large for a cautious v0.1 (documented in docs/mcp.md): the ~40-field
// replace-list writes.
string[] names = ToolCatalog.All.Select(t => t.Name).ToArray();
names.ShouldNotContain("ersatztv_replace_schedule_items");
names.ShouldNotContain("ersatztv_replace_playout_templates");
}
[Test]
public void Every_Tool_Path_Should_Be_Versioned()
{
foreach (ToolDefinition tool in ToolCatalog.All)
{
tool.PathTemplate.ShouldStartWith("/api/v1/", customMessage: tool.Name);
}
}
[Test]
public void Tool_Names_Should_Be_Unique()
{
ToolCatalog.All
.GroupBy(t => t.Name, StringComparer.Ordinal)
.Where(g => g.Count() > 1)
.Select(g => g.Key)
.ShouldBeEmpty();
}
[Test]
public void Every_Path_Parameter_Should_Be_A_Required_Declared_Property()
{
foreach (ToolDefinition tool in ToolCatalog.All)
{
JsonElement schema = tool.InputSchema.RootElement;
bool hasProps = schema.TryGetProperty("properties", out JsonElement properties);
var required = schema.TryGetProperty("required", out JsonElement req)
? req.EnumerateArray().Select(e => e.GetString()).ToHashSet()
: new HashSet<string?>();
foreach (Match match in Regex.Matches(tool.PathTemplate, @"\{([^}]+)\}"))
{
string name = match.Groups[1].Value;
hasProps.ShouldBeTrue($"{tool.Name}: path param {name} needs a properties block");
properties.TryGetProperty(name, out _).ShouldBeTrue($"{tool.Name}: path param {name} not declared");
required.ShouldContain(name, $"{tool.Name}: path param {name} must be required");
}
}
}
[Test]
public void Every_Query_Parameter_Should_Be_A_Declared_Property()
{
foreach (ToolDefinition tool in ToolCatalog.All.Where(t => t.QueryParameters is { Count: > 0 }))
{
JsonElement properties = tool.InputSchema.RootElement.GetProperty("properties");
foreach (string name in tool.QueryParameters!)
{
properties.TryGetProperty(name, out _).ShouldBeTrue($"{tool.Name}: query param {name} not declared");
}
}
}
[Test]
public void Every_Non_Empty_Schema_Should_Forbid_Additional_Properties()
{
foreach (ToolDefinition tool in ToolCatalog.All)
{
JsonElement schema = tool.InputSchema.RootElement;
schema.GetProperty("additionalProperties").ValueKind.ShouldBe(JsonValueKind.False, tool.Name);
}
}
[Test]
public void Get_Channel_Tool_Should_Have_OpenApi_Aligned_Path_And_Id_Input()
{
ToolDefinition tool = ToolCatalog.Find("ersatztv_get_channel").ShouldNotBeNull();
tool.HttpMethod.ShouldBe(HttpMethod.Get);
tool.PathTemplate.ShouldBe("/api/v1/channels/{id}");
tool.InputSchema.RootElement.GetProperty("required").EnumerateArray().Single().GetString().ShouldBe("id");
tool.InputSchema.RootElement.GetProperty("properties").TryGetProperty("id", out _).ShouldBeTrue();
}
[Test]
public void Add_Collection_Items_Tool_Should_Post_With_Array_Buckets()
{
ToolDefinition tool = ToolCatalog.Find("ersatztv_add_collection_items").ShouldNotBeNull();
tool.HttpMethod.ShouldBe(HttpMethod.Post);
tool.PathTemplate.ShouldBe("/api/v1/collections/{id}/items");
JsonElement artistIds = tool.InputSchema.RootElement.GetProperty("properties").GetProperty("artistIds");
artistIds.GetProperty("type").GetString().ShouldBe("array");
artistIds.GetProperty("items").GetProperty("type").GetString().ShouldBe("integer");
}
[Test]
public void Custom_Order_Tool_Should_Declare_IfMatch_Header_Argument()
{
ToolDefinition tool = ToolCatalog.Find("ersatztv_update_collection_custom_order").ShouldNotBeNull();
tool.HttpMethod.ShouldBe(HttpMethod.Put);
tool.InputSchema.RootElement.GetProperty("properties").TryGetProperty("ifMatch", out _).ShouldBeTrue();
// ifMatch is a header, not a query parameter.
(tool.QueryParameters ?? new HashSet<string>()).ShouldNotContain("ifMatch");
}
[Test]
public void Scan_Library_Tool_Should_Register_Deep_As_A_Query_Parameter()
{
ToolDefinition tool = ToolCatalog.Find("ersatztv_scan_library").ShouldNotBeNull();
tool.HttpMethod.ShouldBe(HttpMethod.Post);
tool.QueryParameters.ShouldNotBeNull();
tool.QueryParameters!.ShouldContain("deep");
}
}
+56
View File
@@ -0,0 +1,56 @@
namespace ErsatzTV.Mcp;
/// <summary>
/// Reads newline-delimited lines from a <see cref="TextReader"/> with a hard character cap, so a
/// hostile client cannot exhaust memory by sending an enormous line with no newline. A line longer
/// than the cap is drained (not buffered) and reported as overflowed rather than returned.
/// </summary>
public static class BoundedLineReader
{
public readonly record struct Line(bool EndOfStream, bool Overflowed, string Text);
public const int DefaultMaxChars = 1024 * 1024;
public static async Task<Line> ReadLineAsync(TextReader reader, int maxChars = DefaultMaxChars)
{
int cap = maxChars > 0 ? maxChars : DefaultMaxChars;
var builder = new System.Text.StringBuilder();
var buffer = new char[1];
bool sawAny = false;
bool overflowed = false;
while (await reader.ReadAsync(buffer, 0, 1) == 1)
{
sawAny = true;
char c = buffer[0];
if (c == '\n')
{
return new Line(false, overflowed, overflowed ? string.Empty : builder.ToString());
}
if (c == '\r')
{
continue;
}
if (overflowed || builder.Length >= cap)
{
// Past the cap: stop buffering and free what we have, but keep draining to the
// newline so the next line stays aligned.
overflowed = true;
builder.Clear();
continue;
}
builder.Append(c);
}
if (!sawAny)
{
return new Line(true, false, string.Empty);
}
// Final line with no trailing newline.
return new Line(false, overflowed, overflowed ? string.Empty : builder.ToString());
}
}
+10
View File
@@ -0,0 +1,10 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
</Project>
+264
View File
@@ -0,0 +1,264 @@
using System.Text;
using System.Text.Json;
using System.Text.RegularExpressions;
namespace ErsatzTV.Mcp;
public sealed partial class ErsatzTvApiClient(HttpClient httpClient, ErsatzTvApiClientOptions options) : IToolExecutor
{
// Reserved argument name: carried as the RFC 7232 If-Match request header (never the path/body/query).
// A write tool declares it as an optional string so a caller can round-trip an ETag from a prior read.
public const string IfMatchArgument = "ifMatch";
private static readonly IReadOnlySet<string> EmptyNameSet = new HashSet<string>(StringComparer.Ordinal);
public async Task<ToolCallResult> CallToolAsync(
ToolDefinition tool,
JsonElement arguments,
CancellationToken cancellationToken)
{
// Runtime backstop for the read-only posture: even if a catalog entry is wrong, a non-GET
// tool cannot execute unless writes are explicitly enabled. This is the seam behind the
// cautious-write tools (#58) — they run only when the operator opts in via ERSATZTV_ALLOW_WRITES.
if (!options.AllowWrites && tool.HttpMethod != HttpMethod.Get)
{
return new ToolCallResult(
true,
$"Refused: tool '{tool.Name}' uses HTTP {tool.HttpMethod.Method}, but this MCP server is "
+ "read-only. Set ERSATZTV_ALLOW_WRITES=true to enable write/operational tools.");
}
ToolArgumentValidator.Validate(tool, arguments);
HashSet<string> pathParameters = PathParameterNames(tool.PathTemplate);
IReadOnlySet<string> queryParameters = tool.QueryParameters ?? EmptyNameSet;
string path = BuildPath(tool.PathTemplate, arguments, pathParameters);
// Each declared argument routes to exactly one place: path {param}, an explicit query
// parameter, the reserved If-Match header, or (write verbs only) the JSON request body.
// additionalProperties:false in the schema means only declared args ever arrive here.
bool hasBody = tool.HttpMethod == HttpMethod.Post
|| tool.HttpMethod == HttpMethod.Put
|| tool.HttpMethod == HttpMethod.Patch;
var queryArgs = arguments.EnumerateObject()
.Where(p => queryParameters.Contains(p.Name))
.ToList();
var bodyArgs = arguments.EnumerateObject()
.Where(p => !pathParameters.Contains(p.Name)
&& !queryParameters.Contains(p.Name)
&& !string.Equals(p.Name, IfMatchArgument, StringComparison.Ordinal))
.ToList();
using var request = new HttpRequestMessage(
tool.HttpMethod,
CombineUri(options.BaseUrl, AppendQuery(path, queryArgs)));
if (!string.IsNullOrWhiteSpace(options.ApiKey))
{
request.Headers.Add("X-Api-Key", options.ApiKey);
}
if (arguments.TryGetProperty(IfMatchArgument, out JsonElement ifMatch)
&& ifMatch.ValueKind == JsonValueKind.String
&& ifMatch.GetString() is { Length: > 0 } ifMatchValue)
{
// SECURITY: reject control characters (CR/LF above all). TryAddWithoutValidation writes
// the value to the wire verbatim — SocketsHttpHandler does NOT strip CR/LF — so a value
// like "5"\r\nX-Evil: 1 would smuggle extra headers onto a request that carries the
// machine X-Api-Key. The arg is model-controlled, so this must be guarded here; the
// server still parses the RFC 7232 grammar (quoted tag / list / "*") and returns 400/412.
// TryAddWithoutValidation (not Add) is still required so a valid quoted opaque tag passes
// HttpClient's otherwise-stricter parsing.
if (ifMatchValue.Any(char.IsControl))
{
throw new ArgumentException("Invalid 'ifMatch' value: control characters are not allowed.");
}
request.Headers.TryAddWithoutValidation("If-Match", ifMatchValue);
}
if (hasBody && bodyArgs.Count > 0)
{
request.Content = new StringContent(SerializeBody(bodyArgs), Encoding.UTF8, "application/json");
}
// ResponseHeadersRead streams the body so we can cap it without buffering the whole thing —
// but that moves the body read outside HttpClient.Timeout, so a per-request timeout token
// must cover the entire operation (headers + body) or a slow-drip upstream would hang the
// single-threaded session.
using var timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeoutCts.CancelAfter(options.EffectiveRequestTimeout);
CancellationToken token = timeoutCts.Token;
using HttpResponseMessage response = await httpClient.SendAsync(
request,
HttpCompletionOption.ResponseHeadersRead,
token);
string body = await ReadCappedBodyAsync(response.Content, token);
// Surface the aggregate ETag (versioned roots emit it on GET and on a successful replace PUT)
// so an agent can round-trip it as `ifMatch` on a subsequent write. Header-only by contract.
string etagSuffix = response.Headers.ETag is { } etag ? $"\n[etag: {etag}]" : string.Empty;
if (response.IsSuccessStatusCode)
{
return new ToolCallResult(false, body + etagSuffix);
}
string message = $"{(int)response.StatusCode} {response.ReasonPhrase}: {body}";
return new ToolCallResult(true, message + etagSuffix);
}
private static string SerializeBody(IReadOnlyList<JsonProperty> payload)
{
using var stream = new MemoryStream();
using (var writer = new Utf8JsonWriter(stream))
{
writer.WriteStartObject();
foreach (JsonProperty property in payload)
{
property.WriteTo(writer);
}
writer.WriteEndObject();
}
return Encoding.UTF8.GetString(stream.ToArray());
}
private static string AppendQuery(string path, IReadOnlyList<JsonProperty> payload)
{
if (payload.Count == 0)
{
return path;
}
var query = new StringBuilder(path);
char separator = '?';
foreach (JsonProperty property in payload)
{
string value = property.Value.ValueKind == JsonValueKind.String
? property.Value.GetString() ?? string.Empty
: property.Value.GetRawText();
query.Append(separator)
.Append(Uri.EscapeDataString(property.Name))
.Append('=')
.Append(Uri.EscapeDataString(value));
separator = '&';
}
return query.ToString();
}
// Read at most MaxResponseBytes from the response, truncating gracefully with a marker rather
// than buffering an unbounded body into memory / the model's context.
private async Task<string> ReadCappedBodyAsync(HttpContent content, CancellationToken cancellationToken)
{
int cap = options.MaxResponseBytes is > 0 and <= ErsatzTvApiClientOptions.MaxAllowedResponseBytes
? options.MaxResponseBytes
: ErsatzTvApiClientOptions.DefaultMaxResponseBytes;
await using Stream stream = await content.ReadAsStreamAsync(cancellationToken);
// One extra byte lets us detect (but not keep) overflow past the cap.
byte[] buffer = new byte[cap + 1];
int total = 0;
int read;
while (total < buffer.Length
&& (read = await stream.ReadAsync(buffer.AsMemory(total, buffer.Length - total), cancellationToken)) > 0)
{
total += read;
}
bool truncated = total > cap;
int length = truncated ? TrimToUtf8Boundary(buffer, cap) : total;
string text = Encoding.UTF8.GetString(buffer, 0, length);
return truncated
? text + $"\n…[truncated: response exceeded {cap} bytes]"
: text;
}
// When cutting at a fixed byte cap, back off any incomplete trailing UTF-8 sequence so the
// decoded text ends on a complete code point instead of a U+FFFD replacement char.
private static int TrimToUtf8Boundary(byte[] buffer, int length)
{
int i = length;
while (i > 0 && (buffer[i - 1] & 0b1100_0000) == 0b1000_0000)
{
i--; // step back over UTF-8 continuation bytes (10xxxxxx)
}
if (i == 0)
{
return length; // all continuation bytes (not valid UTF-8) — leave the cut as-is
}
byte lead = buffer[i - 1];
int expected = lead switch
{
< 0x80 => 1,
>= 0xF0 => 4,
>= 0xE0 => 3,
>= 0xC0 => 2,
_ => 1 // stray continuation byte as "lead"; leave the cut as-is
};
// Keep the sequence if it is complete within the cap; otherwise drop the incomplete lead.
return length - (i - 1) >= expected ? length : i - 1;
}
private static Uri CombineUri(Uri baseUrl, string relativeUri)
{
// relativeUri is a root-relative "/api/..." path (optionally with a query). new Uri(baseUrl,
// "/api/...") would discard any path prefix on baseUrl (e.g. a reverse-proxy mount like
// http://host/etv/), so combine on the base's full path instead to preserve the prefix.
string prefix = baseUrl.GetLeftPart(UriPartial.Path).TrimEnd('/');
return new Uri(prefix + relativeUri);
}
private static HashSet<string> PathParameterNames(string pathTemplate)
{
var names = new HashSet<string>(StringComparer.Ordinal);
foreach (Match match in PathParameterRegex().Matches(pathTemplate))
{
names.Add(match.Groups[1].Value);
}
return names;
}
private static string BuildPath(string pathTemplate, JsonElement arguments, HashSet<string> pathParameters)
{
string path = pathTemplate;
foreach (JsonProperty property in arguments.EnumerateObject())
{
if (!pathParameters.Contains(property.Name))
{
continue;
}
string value = property.Value.ValueKind == JsonValueKind.String
? property.Value.GetString() ?? string.Empty
: property.Value.GetRawText();
// EscapeDataString escapes '/', but bare "." / ".." survive and would collapse the URL
// onto a different route during Uri canonicalization — reject them. This assumes each
// {param} is its own path segment (true for every current template).
if (value is "." or "..")
{
throw new ArgumentException($"Invalid value for argument '{property.Name}': '{value}'.");
}
path = path.Replace("{" + property.Name + "}", Uri.EscapeDataString(value), StringComparison.Ordinal);
}
Match unresolved = PathParameterRegex().Match(path);
if (unresolved.Success)
{
throw new ArgumentException($"Missing required argument '{unresolved.Groups[1].Value}'");
}
return path;
}
[GeneratedRegex(@"\{([^}]+)\}")]
private static partial Regex PathParameterRegex();
}
+171
View File
@@ -0,0 +1,171 @@
using System.Text.Json;
namespace ErsatzTV.Mcp;
public sealed class McpServer(IToolExecutor toolExecutor)
{
private static readonly JsonSerializerOptions JsonOptions = new(JsonSerializerDefaults.Web);
// Process-lifetime document so NullId stays valid; used as the JSON-RPC id for
// parse errors / requests with no usable id.
private static readonly JsonDocument NullIdDocument = JsonDocument.Parse("null");
private static readonly JsonElement NullId = NullIdDocument.RootElement;
// Process-lifetime empty-arguments document, reused for tool calls that omit "arguments" — so a
// no-arg call doesn't leak a pooled JsonDocument per invocation.
private static readonly JsonDocument EmptyArgsDocument = JsonDocument.Parse("{}");
private static readonly JsonElement EmptyArgs = EmptyArgsDocument.RootElement;
public async Task<string?> HandleAsync(string requestJson, CancellationToken cancellationToken)
{
JsonDocument request;
try
{
request = JsonDocument.Parse(requestJson);
}
catch (JsonException)
{
// A malformed line must never crash the session loop (JSON-RPC parse error, id null).
return SerializeError(NullId, -32700, "Parse error: invalid JSON.");
}
using (request)
{
JsonElement root = request.RootElement;
JsonElement id = NullId;
bool hasId = false;
if (root.ValueKind == JsonValueKind.Object && root.TryGetProperty("id", out JsonElement idValue))
{
id = idValue;
hasId = true;
}
try
{
if (root.ValueKind != JsonValueKind.Object)
{
throw new JsonRpcException(-32600, "Invalid Request: expected a JSON-RPC object.");
}
if (!root.TryGetProperty("method", out JsonElement methodElement)
|| methodElement.ValueKind != JsonValueKind.String)
{
throw new JsonRpcException(-32600, "Invalid Request: missing or non-string 'method'.");
}
// No id ⇒ notification ⇒ no response.
if (!hasId)
{
return null;
}
string? method = methodElement.GetString();
object result = method switch
{
"initialize" => InitializeResult(),
"tools/list" => ToolsListResult(),
"tools/call" => await CallToolAsync(RequireParams(root), cancellationToken),
_ => throw new JsonRpcException(-32601, $"Method not found: {method}")
};
return SerializeResponse(id, result);
}
catch (JsonRpcException ex)
{
return SerializeError(id, ex.Code, ex.Message);
}
catch (Exception ex) when (ex is ArgumentException or KeyNotFoundException or InvalidOperationException)
{
return hasId ? SerializeError(id, -32602, ex.Message) : null;
}
catch (Exception ex)
{
// Transport/timeout/unexpected failures (HttpRequestException, a fired request
// timeout, etc.) must still return a JSON-RPC error for the id — otherwise a
// compliant client blocks forever awaiting a response that never comes.
return hasId ? SerializeError(id, -32603, $"Internal error: {ex.Message}") : null;
}
}
}
private static JsonElement RequireParams(JsonElement root) =>
root.TryGetProperty("params", out JsonElement parameters)
? parameters
: throw new JsonRpcException(-32602, "Invalid params: missing 'params'.");
private static object InitializeResult() => new
{
protocolVersion = "2024-11-05",
capabilities = new
{
tools = new { }
},
serverInfo = new
{
name = "ersatztv-mcp",
version = "0.1.0"
}
};
private static object ToolsListResult() => new
{
tools = ToolCatalog.All.Select(t => new
{
name = t.Name,
description = t.Description,
inputSchema = t.InputSchema.RootElement
})
};
private async Task<object> CallToolAsync(JsonElement parameters, CancellationToken cancellationToken)
{
string name = parameters.GetProperty("name").GetString() ?? throw new ArgumentException("Tool name is required");
ToolDefinition tool = ToolCatalog.Find(name) ?? throw new ArgumentException($"Unknown tool: {name}");
JsonElement arguments = parameters.TryGetProperty("arguments", out JsonElement args)
? args
: EmptyArgs;
ToolCallResult result = await toolExecutor.CallToolAsync(tool, arguments, cancellationToken);
return new
{
content = new[]
{
new
{
type = "text",
text = result.Text
}
},
isError = result.IsError
};
}
private static string SerializeResponse(JsonElement id, object result) =>
JsonSerializer.Serialize(
new
{
jsonrpc = "2.0",
id = id.Clone(),
result
},
JsonOptions);
private static string SerializeError(JsonElement id, int code, string message) =>
JsonSerializer.Serialize(
new
{
jsonrpc = "2.0",
id = id.Clone(),
error = new
{
code,
message
}
},
JsonOptions);
private sealed class JsonRpcException(int code, string message) : Exception(message)
{
public int Code { get; } = code;
}
}
+87
View File
@@ -0,0 +1,87 @@
namespace ErsatzTV.Mcp;
public static class Program
{
public static async Task Main()
{
string baseUrl = Environment.GetEnvironmentVariable("ERSATZTV_URL") ?? "http://localhost:8409";
string? apiKey = Environment.GetEnvironmentVariable("ERSATZTV_API_KEY");
bool allowWrites = ParseBool(Environment.GetEnvironmentVariable("ERSATZTV_ALLOW_WRITES"));
int maxResponseBytes = ParseInt(
Environment.GetEnvironmentVariable("ERSATZTV_MAX_RESPONSE_BYTES"),
fallback: ErsatzTvApiClientOptions.DefaultMaxResponseBytes,
min: 1024,
max: ErsatzTvApiClientOptions.MaxAllowedResponseBytes);
int timeoutSeconds = ParseInt(
Environment.GetEnvironmentVariable("ERSATZTV_REQUEST_TIMEOUT_SECONDS"),
fallback: 30,
min: 1,
max: 3600);
var requestTimeout = TimeSpan.FromSeconds(timeoutSeconds);
// The per-request timeout is enforced via a CancellationToken inside the client (it must
// cover the streamed body read too), so leave HttpClient's own timeout off to avoid a
// second, header-only timer racing it.
using var httpClient = new HttpClient { Timeout = Timeout.InfiniteTimeSpan };
var apiClient = new ErsatzTvApiClient(
httpClient,
new ErsatzTvApiClientOptions(new Uri(baseUrl), apiKey, allowWrites, maxResponseBytes, requestTimeout));
var server = new McpServer(apiClient);
while (true)
{
BoundedLineReader.Line line = await BoundedLineReader.ReadLineAsync(Console.In);
if (line.EndOfStream)
{
break;
}
if (line.Overflowed)
{
await Console.Error.WriteLineAsync("[ersatztv-mcp] dropped oversized request line.");
continue;
}
if (string.IsNullOrWhiteSpace(line.Text))
{
continue;
}
try
{
string? response = await server.HandleAsync(line.Text, CancellationToken.None);
if (response is not null)
{
await Console.Out.WriteLineAsync(response);
await Console.Out.FlushAsync();
}
}
catch (Exception ex)
{
// Last-resort guard: a single failed request must never terminate the session.
await Console.Error.WriteLineAsync($"[ersatztv-mcp] error handling request: {ex.Message}");
}
}
}
private static bool ParseBool(string? value)
{
if (value is null)
{
return false;
}
if (bool.TryParse(value, out bool parsed))
{
return parsed;
}
string trimmed = value.Trim();
return trimmed is "1"
|| string.Equals(trimmed, "yes", StringComparison.OrdinalIgnoreCase)
|| string.Equals(trimmed, "on", StringComparison.OrdinalIgnoreCase);
}
private static int ParseInt(string? value, int fallback, int min, int max) =>
int.TryParse(value, out int parsed) ? Math.Clamp(parsed, min, max) : fallback;
}
+87
View File
@@ -0,0 +1,87 @@
using System.Text.Json;
namespace ErsatzTV.Mcp;
/// <summary>
/// Lightweight validation of caller-supplied tool arguments against a tool's declared
/// <c>InputSchema</c>. Not a full JSON-Schema implementation — it enforces the shapes the
/// catalog actually emits (typed properties, a required list, <c>additionalProperties:false</c>)
/// so unknown/malformed arguments are rejected before an API request is built.
/// Throws <see cref="ArgumentException"/> (mapped to JSON-RPC -32602 by the server).
/// </summary>
public static class ToolArgumentValidator
{
public static void Validate(ToolDefinition tool, JsonElement arguments)
{
if (arguments.ValueKind != JsonValueKind.Object)
{
throw new ArgumentException($"Arguments for tool '{tool.Name}' must be a JSON object.");
}
JsonElement schema = tool.InputSchema.RootElement;
JsonElement properties = schema.TryGetProperty("properties", out JsonElement props)
? props
: default;
bool additionalAllowed = !schema.TryGetProperty("additionalProperties", out JsonElement additional)
|| additional.ValueKind != JsonValueKind.False;
foreach (JsonProperty arg in arguments.EnumerateObject())
{
if (properties.ValueKind != JsonValueKind.Object
|| !properties.TryGetProperty(arg.Name, out JsonElement propertySchema))
{
if (!additionalAllowed)
{
throw new ArgumentException($"Unknown argument '{arg.Name}' for tool '{tool.Name}'.");
}
continue;
}
string? type = propertySchema.TryGetProperty("type", out JsonElement typeElement)
? typeElement.GetString()
: null;
if (!MatchesType(type, arg.Value))
{
throw new ArgumentException(
$"Argument '{arg.Name}' for tool '{tool.Name}' must be of type '{type}'.");
}
}
if (schema.TryGetProperty("required", out JsonElement required)
&& required.ValueKind == JsonValueKind.Array)
{
foreach (JsonElement name in required.EnumerateArray())
{
string? propertyName = name.GetString();
if (propertyName is not null && !arguments.TryGetProperty(propertyName, out _))
{
throw new ArgumentException(
$"Missing required argument '{propertyName}' for tool '{tool.Name}'.");
}
}
}
}
private static bool MatchesType(string? type, JsonElement value)
{
// JSON null is accepted for any declared property: it clears an optional/nullable API field
// (e.g. UpdatePlayoutDetailsRequest.DailyRebuildTime). Required-presence is checked separately;
// the server remains the authority on required-non-null (a null there returns 422).
if (value.ValueKind == JsonValueKind.Null)
{
return true;
}
return type switch
{
"integer" => value.ValueKind == JsonValueKind.Number && value.TryGetInt64(out _),
"number" => value.ValueKind == JsonValueKind.Number,
"string" => value.ValueKind == JsonValueKind.String,
"boolean" => value.ValueKind is JsonValueKind.True or JsonValueKind.False,
"array" => value.ValueKind == JsonValueKind.Array,
"object" => value.ValueKind == JsonValueKind.Object,
_ => true
};
}
}
+316
View File
@@ -0,0 +1,316 @@
namespace ErsatzTV.Mcp;
/// <summary>
/// The explicit, narrow set of tools this MCP server exposes over the frozen ErsatzTV
/// <c>/api/v1</c> surface. Read tools are always available; write tools execute only when the
/// operator sets <c>ERSATZTV_ALLOW_WRITES=true</c> (enforced at runtime in <see cref="ErsatzTvApiClient"/>).
///
/// Deliberately deferred from this v0.1 (documented in docs/mcp.md): the ~40-field schedule-item
/// and playout alternate-schedule/template replace-list writes, and the redesign-aware workflow
/// tools (#63#68).
/// </summary>
public static class ToolCatalog
{
public static IReadOnlyList<ToolDefinition> All { get; } =
[
// ---- Reads ----
Get("ersatztv_list_channels", "List channels.", "/api/v1/channels"),
Get("ersatztv_get_channel", "Get a channel by id.", "/api/v1/channels/{id}", IdPath("Channel id.")),
Get("ersatztv_list_collections", "List collections.", "/api/v1/collections"),
Get("ersatztv_get_collection", "Get a collection by id.", "/api/v1/collections/{id}", IdPath("Collection id.")),
Get(
"ersatztv_get_collection_items",
"Get the items in a manual collection (paged). Emits the collection ETag for optimistic-concurrency reorder.",
"/api/v1/collections/{id}/items",
[IdPath("Collection id.")],
Page()),
Get("ersatztv_list_smart_collections", "List smart collections.", "/api/v1/smart-collections"),
Get("ersatztv_get_smart_collection", "Get a smart collection by id.", "/api/v1/smart-collections/{id}", IdPath("Smart collection id.")),
Get("ersatztv_list_schedules", "List schedules.", "/api/v1/schedules"),
Get("ersatztv_get_schedule", "Get a schedule by id.", "/api/v1/schedules/{id}", IdPath("Schedule id.")),
Get("ersatztv_get_schedule_items", "Get a schedule's items. Emits the schedule ETag.", "/api/v1/schedules/{id}/items", IdPath("Schedule id.")),
Get("ersatztv_list_playouts", "List playouts.", "/api/v1/playouts"),
Get("ersatztv_get_playout", "Get a playout by id.", "/api/v1/playouts/{id}", IdPath("Playout id.")),
Get("ersatztv_get_playout_items", "Get upcoming items (and unscheduled gaps) for a playout.", "/api/v1/playouts/{id}/items", IdPath("Playout id.")),
Get("ersatztv_list_ffmpeg_profiles", "List FFmpeg profiles.", "/api/v1/ffmpeg/profiles"),
Get("ersatztv_get_ffmpeg_profile", "Get an FFmpeg profile by id.", "/api/v1/ffmpeg/profiles/{id}", IdPath("FFmpeg profile id.")),
Get(
"ersatztv_get_resolution_by_name",
"Get an FFmpeg resolution by name.",
"/api/v1/ffmpeg/resolution/by-name/{name}",
Str("name", "Resolution name.", required: true, arg: In.Path)),
Get("ersatztv_list_sessions", "List active HLS sessions.", "/api/v1/sessions"),
Get("ersatztv_get_version", "Get API and app version.", "/api/v1/version"),
Get("ersatztv_list_media_sources", "Get all media sources with their libraries.", "/api/v1/media-sources"),
Get("ersatztv_get_jellyfin_libraries", "Get a Jellyfin source's libraries.", "/api/v1/media-sources/jellyfin/{id}/libraries", IdPath("Jellyfin media source id.")),
Get("ersatztv_list_local_libraries", "Get all local libraries.", "/api/v1/libraries/local"),
Get("ersatztv_get_library_scan_status", "Get active library scan status.", "/api/v1/libraries/scan-status"),
Get(
"ersatztv_search",
"Search library items across all media kinds (paged, hydrated results).",
"/api/v1/search",
[],
[Str("query", "Lucene query string.", arg: In.Query), .. Page()]),
Get(
"ersatztv_search_all_items",
"Search library items across all media kinds and return raw id lists — use to discover media ids to add to a collection.",
"/api/v1/search/all-items",
[],
[Str("query", "Lucene query string.", arg: In.Query), .. Page()]),
Get(
"ersatztv_search_artists",
"Search artists by name; returns matching artist ids.",
"/api/v1/search/artists",
[],
[Str("query", "Artist name query.", arg: In.Query)]),
// ---- Writes (require ERSATZTV_ALLOW_WRITES=true) ----
Post(
"ersatztv_create_collection",
"Create an empty manual collection.",
"/api/v1/collections",
Str("name", "Collection name.", required: true)),
Put(
"ersatztv_update_collection",
"Rename a collection and/or toggle its custom playback order.",
"/api/v1/collections/{id}",
IdPath("Collection id."),
Str("name", "Collection name.", required: true),
Bool("useCustomPlaybackOrder", "Whether the collection uses a custom playback order.")),
Delete("ersatztv_delete_collection", "Delete a collection.", "/api/v1/collections/{id}", IdPath("Collection id.")),
Post(
"ersatztv_add_collection_items",
"Add media items to a manual collection. Send only the buckets you need; re-adding an already-present item is an idempotent no-op. All referenced ids must exist or the whole batch is rejected (422).",
"/api/v1/collections/{id}/items",
IdPath("Collection id."),
IntArray("movieIds", "Movie ids to add."),
IntArray("showIds", "Show ids to add."),
IntArray("seasonIds", "Season ids to add."),
IntArray("episodeIds", "Episode ids to add."),
IntArray("artistIds", "Artist ids to add."),
IntArray("musicVideoIds", "Music video ids to add."),
IntArray("otherVideoIds", "Other-video ids to add."),
IntArray("songIds", "Song ids to add."),
IntArray("imageIds", "Image ids to add."),
IntArray("remoteStreamIds", "Remote-stream ids to add.")),
Delete(
"ersatztv_remove_collection_item",
"Remove a single media item from a collection.",
"/api/v1/collections/{id}/items/{mediaItemId}",
IdPath("Collection id."),
Int("mediaItemId", "Media item id to remove.", required: true, arg: In.Path)),
Put(
"ersatztv_update_collection_custom_order",
"Replace a collection's custom playback order with the given media-item id order. Honors If-Match (read the ETag from ersatztv_get_collection_items first); omit ifMatch to force-write.",
"/api/v1/collections/{id}/custom-order",
IdPath("Collection id."),
IntArray("mediaItemIds", "Media item ids in the desired playback order.", required: true),
IfMatch),
Post(
"ersatztv_create_smart_collection",
"Create a smart collection backed by a search query.",
"/api/v1/smart-collections",
Str("name", "Smart collection name.", required: true),
Str("query", "Lucene query defining membership.", required: true)),
Put(
"ersatztv_update_smart_collection",
"Update a smart collection's name and query.",
"/api/v1/smart-collections/{id}",
IdPath("Smart collection id."),
Str("name", "Smart collection name.", required: true),
Str("query", "Lucene query defining membership.", required: true)),
Delete("ersatztv_delete_smart_collection", "Delete a smart collection.", "/api/v1/smart-collections/{id}", IdPath("Smart collection id.")),
Post(
"ersatztv_create_schedule",
"Create a program schedule.",
"/api/v1/schedules",
[Str("name", "Schedule name.", required: true), .. ScheduleFlags()]),
Put(
"ersatztv_update_schedule",
"Update a program schedule's settings.",
"/api/v1/schedules/{id}",
[IdPath("Schedule id."), Str("name", "Schedule name.", required: true), .. ScheduleFlags()]),
Delete("ersatztv_delete_schedule", "Delete a program schedule.", "/api/v1/schedules/{id}", IdPath("Schedule id.")),
Post(
"ersatztv_create_playout",
"Create a playout for a channel. Classic requires programScheduleId; Sequential/Scripted/ExternalJson require scheduleFile.",
"/api/v1/playouts",
Int("channelId", "Channel id.", required: true),
Str("scheduleKind", "One of: Classic, Block, Sequential, Scripted, ExternalJson.", required: true),
Int("programScheduleId", "Program schedule id (Classic only)."),
Str("scheduleFile", "Schedule file path (Sequential/Scripted/ExternalJson only).")),
Put(
"ersatztv_update_playout",
"Update a playout's daily rebuild time and/or schedule file. dailyRebuildTime is always applied (null clears the daily reset).",
"/api/v1/playouts/{id}",
IdPath("Playout id."),
Str("dailyRebuildTime", "Daily rebuild time as an ISO 8601 duration/timespan (e.g. \"04:00:00\"); null clears it."),
Str("scheduleFile", "Schedule file path (Sequential/Scripted/ExternalJson only); omit to leave unchanged.")),
Delete("ersatztv_delete_playout", "Delete a playout.", "/api/v1/playouts/{id}", IdPath("Playout id.")),
Post(
"ersatztv_create_channel",
"Create a channel. Enum fields take the enum name; GET an existing channel (ersatztv_get_channel) to see valid values and sensible defaults before creating.",
"/api/v1/channels",
ChannelFields()),
Put(
"ersatztv_update_channel",
"Update a channel. Send the full desired state; enum fields take the enum name (GET the channel first to copy current values).",
"/api/v1/channels/{id}",
[IdPath("Channel id."), .. ChannelFields()]),
Post(
"ersatztv_reset_channel_playout",
"Queue a rebuild of a channel's playout (202 Accepted; 409 if a build is already running).",
"/api/v1/channels/{id}/playout/reset",
[IdPath("Channel id.")],
[Str("mode", "Optional playout build mode; omit for the default. GET a playout to see valid values.", arg: In.Query)]),
Delete("ersatztv_delete_channel", "Delete a channel.", "/api/v1/channels/{id}", IdPath("Channel id.")),
Put(
"ersatztv_enable_jellyfin_library_sync",
"Replace a Jellyfin source's library sync preferences. The body must be the COMPLETE set of the source's libraries; enabling shouldSyncItems also enqueues a scan. A row absent from the request is left untouched.",
"/api/v1/media-sources/jellyfin/{id}/libraries",
IdPath("Jellyfin media source id."),
ObjArray("libraries", "Complete set of the source's libraries; each element { id: number, shouldSyncItems: boolean }.", required: true)),
Post(
"ersatztv_refresh_jellyfin_libraries",
"Refresh the list of a Jellyfin source's libraries from the server (202 Accepted).",
"/api/v1/media-sources/jellyfin/{id}/refresh-libraries",
IdPath("Jellyfin media source id.")),
Post(
"ersatztv_scan_jellyfin_collections",
"Scan a Jellyfin source's collections (202 Accepted; 409 if already scanning).",
"/api/v1/media-sources/jellyfin/{id}/scan-collections",
[IdPath("Jellyfin media source id.")],
[Bool("deep", "Whether to perform a deep scan.", arg: In.Query)]),
Post(
"ersatztv_scan_library",
"Trigger a scan of a library (202 queued; 404 not found; 409 already scanning; 422 sync disabled).",
"/api/v1/libraries/{id}/scan",
[IdPath("Library id.")],
[Bool("deep", "Whether to perform a deep scan.", arg: In.Query)])
];
public static ToolDefinition? Find(string name) =>
All.FirstOrDefault(t => string.Equals(t.Name, name, StringComparison.Ordinal));
// ---- Argument location + builders ----
private enum In
{
Path,
Query,
Body,
Header
}
private sealed record Arg(string Name, string Type, string Description, bool Required, In Location, string? ItemType = null);
// A property (not a static field): the `All` initializer runs before a static field declared
// later would be assigned, which would pass a null Arg here. A property is evaluated on access.
private static Arg IfMatch => new(
ErsatzTvApiClient.IfMatchArgument,
"string",
"Optional RFC 7232 ETag from a prior read (e.g. \"3\") for optimistic concurrency; omit to force-write.",
Required: false,
In.Header);
private static Arg IdPath(string description) => new("id", "integer", description, Required: true, In.Path);
private static Arg Str(string name, string description, bool required = false, In arg = In.Body) =>
new(name, "string", description, required, arg);
private static Arg Int(string name, string description, bool required = false, In arg = In.Body) =>
new(name, "integer", description, required, arg);
private static Arg Bool(string name, string description, In arg = In.Body) =>
new(name, "boolean", description, Required: false, arg);
private static Arg IntArray(string name, string description, bool required = false) =>
new(name, "array", description, required, In.Body, ItemType: "integer");
private static Arg ObjArray(string name, string description, bool required = false) =>
new(name, "array", description, required, In.Body, ItemType: "object");
private static Arg[] Page() =>
[
Int("pageNum", "1-based page number (optional).", arg: In.Query),
Int("pageSize", "Page size (optional).", arg: In.Query)
];
// The channel create/update body (CreateChannelRequest / UpdateChannelRequest — the id comes from
// the route on update). Only name/number/ffmpegProfileId are marked required; the rest have
// server-side defaults. Enum fields are typed "string" (the enum name) — the API validates them.
private static Arg[] ChannelFields() =>
[
Str("name", "Channel name.", required: true),
Str("number", "Channel number (e.g. \"5\" or \"5.1\").", required: true),
Str("group", "Channel group."),
Str("categories", "Comma-separated categories."),
Int("ffmpegProfileId", "FFmpeg profile id.", required: true),
new Arg("slugSeconds", "number", "Optional slug/pad seconds.", Required: false, In.Body),
new Arg("logo", "object", "Channel logo as an ArtworkContentTypeModel object.", Required: false, In.Body),
Str("streamSelectorMode", "ChannelStreamSelectorMode enum name."),
Str("streamSelector", "Stream selector value (when streamSelectorMode uses one)."),
Str("preferredAudioLanguageCode", "Preferred audio language code."),
Str("preferredAudioTitle", "Preferred audio title."),
Str("playoutSource", "ChannelPlayoutSource enum name."),
Str("playoutMode", "ChannelPlayoutMode enum name."),
Int("mirrorSourceChannelId", "Channel id to mirror (when playoutSource is a mirror)."),
Str("playoutOffset", "Playout offset as a timespan (e.g. \"01:00:00\")."),
Str("streamingMode", "StreamingMode enum name."),
Int("watermarkId", "Watermark id."),
Int("fallbackFillerId", "Fallback filler id."),
Str("preferredSubtitleLanguageCode", "Preferred subtitle language code."),
Str("subtitleMode", "ChannelSubtitleMode enum name."),
Str("musicVideoCreditsMode", "ChannelMusicVideoCreditsMode enum name."),
Str("musicVideoCreditsTemplate", "Music video credits template name."),
Str("songVideoMode", "ChannelSongVideoMode enum name."),
Str("transcodeMode", "ChannelTranscodeMode enum name."),
Str("idleBehavior", "ChannelIdleBehavior enum name."),
Bool("isEnabled", "Whether the channel is enabled."),
Bool("showInEpg", "Whether the channel appears in the EPG/guide.")
];
private static Arg[] ScheduleFlags() =>
[
Bool("keepMultiPartEpisodesTogether", "Keep multi-part episodes together."),
Bool("treatCollectionsAsShows", "Treat collections as shows."),
Bool("shuffleScheduleItems", "Shuffle schedule items."),
Bool("randomStartPoint", "Use a random start point."),
Str("fixedStartTimeBehavior", "Fixed start-time behavior (enum name; GET a schedule to see valid values).")
];
// ---- Tool factories ----
private static ToolDefinition Get(string name, string description, string path, params Arg[] args) =>
Tool(name, description, HttpMethod.Get, path, args);
private static ToolDefinition Get(string name, string description, string path, Arg[] pathArgs, Arg[] otherArgs) =>
Tool(name, description, HttpMethod.Get, path, [.. pathArgs, .. otherArgs]);
private static ToolDefinition Post(string name, string description, string path, params Arg[] args) =>
Tool(name, description, HttpMethod.Post, path, args);
private static ToolDefinition Post(string name, string description, string path, Arg[] pathArgs, Arg[] otherArgs) =>
Tool(name, description, HttpMethod.Post, path, [.. pathArgs, .. otherArgs]);
private static ToolDefinition Put(string name, string description, string path, params Arg[] args) =>
Tool(name, description, HttpMethod.Put, path, args);
private static ToolDefinition Delete(string name, string description, string path, params Arg[] args) =>
Tool(name, description, HttpMethod.Delete, path, args);
private static ToolDefinition Tool(string name, string description, HttpMethod method, string path, Arg[] args)
{
System.Text.Json.JsonDocument schema = args.Length == 0
? ToolInputSchemas.Empty
: ToolInputSchemas.Object(
args.Select(a => new SchemaProperty(a.Name, a.Type, a.Description, a.Required, a.ItemType)).ToArray());
var queryParameters = args
.Where(a => a.Location == In.Query)
.Select(a => a.Name)
.ToHashSet(StringComparer.Ordinal);
return new ToolDefinition(name, description, method, path, schema, queryParameters);
}
}
+41
View File
@@ -0,0 +1,41 @@
using System.Text.Json;
namespace ErsatzTV.Mcp;
public sealed record ToolDefinition(
string Name,
string Description,
HttpMethod HttpMethod,
string PathTemplate,
JsonDocument InputSchema,
IReadOnlySet<string>? QueryParameters = null);
public sealed record ToolCallResult(bool IsError, string Text);
public sealed record ErsatzTvApiClientOptions(
Uri BaseUrl,
string? ApiKey,
bool AllowWrites = false,
int MaxResponseBytes = ErsatzTvApiClientOptions.DefaultMaxResponseBytes,
TimeSpan RequestTimeout = default)
{
// Cap the response body buffered back to the model so a large/hostile API
// response cannot exhaust memory or flood the context window.
public const int DefaultMaxResponseBytes = 1024 * 1024;
// Hard ceiling so a hostile/typo'd cap can't request a huge (or overflowing) allocation.
public const int MaxAllowedResponseBytes = 64 * 1024 * 1024;
public static readonly TimeSpan DefaultRequestTimeout = TimeSpan.FromSeconds(30);
// The per-request timeout, covering headers *and* body (see ErsatzTvApiClient).
public TimeSpan EffectiveRequestTimeout => RequestTimeout > TimeSpan.Zero ? RequestTimeout : DefaultRequestTimeout;
}
public interface IToolExecutor
{
Task<ToolCallResult> CallToolAsync(
ToolDefinition tool,
JsonElement arguments,
CancellationToken cancellationToken);
}
+60
View File
@@ -0,0 +1,60 @@
using System.Text.Json;
namespace ErsatzTV.Mcp;
/// <summary>
/// A single tool input-schema property. <paramref name="ItemType"/> is only consulted when
/// <paramref name="Type"/> is <c>"array"</c> (it becomes the array's <c>items.type</c>).
/// </summary>
public sealed record SchemaProperty(
string Name,
string Type,
string Description,
bool Required,
string? ItemType = null);
public static class ToolInputSchemas
{
public static JsonDocument Empty { get; } = JsonDocument.Parse(
"""
{"type":"object","properties":{},"additionalProperties":false}
""");
public static JsonDocument Object(params SchemaProperty[] properties)
{
using var stream = new MemoryStream();
using (var writer = new Utf8JsonWriter(stream))
{
writer.WriteStartObject();
writer.WriteString("type", "object");
writer.WriteStartObject("properties");
foreach (SchemaProperty property in properties)
{
writer.WriteStartObject(property.Name);
writer.WriteString("type", property.Type);
if (property is { Type: "array", ItemType: { } itemType })
{
writer.WriteStartObject("items");
writer.WriteString("type", itemType);
writer.WriteEndObject();
}
writer.WriteString("description", property.Description);
writer.WriteEndObject();
}
writer.WriteEndObject();
writer.WriteStartArray("required");
foreach (SchemaProperty property in properties.Where(p => p.Required))
{
writer.WriteStringValue(property.Name);
}
writer.WriteEndArray();
writer.WriteBoolean("additionalProperties", false);
writer.WriteEndObject();
}
return JsonDocument.Parse(stream.ToArray());
}
}
@@ -1,12 +1,17 @@
using System.Buffers.Binary;
using ErsatzTV.Application.Artworks;
using ErsatzTV.Core;
using ErsatzTV.Core.Api.Artwork;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Infrastructure.Images;
using LanguageExt;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Png;
using SixLabors.ImageSharp.PixelFormats;
using static LanguageExt.Prelude;
namespace ErsatzTV.Tests.Application.Artworks;
@@ -14,10 +19,11 @@ namespace ErsatzTV.Tests.Application.Artworks;
[TestFixture]
public class UploadArtworkHandlerTests
{
// A minimal valid 1x1 PNG. The handler derives the content type from bytes like these, never
// from a client-declared value (issue #283), so the tests exercise the real sniffer.
private static readonly byte[] PngBytes = Convert.FromBase64String(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+M8AAAMCAoAB/QwAAAAASUVORK5CYII=");
// A minimal, genuinely decodable PNG. The handler derives the content type from bytes like
// these, never from a client-declared value (issue #283), so the tests exercise the real
// sniffer; and it now also decode-budget-validates (issue #525), so these bytes must survive a
// full decode, not just a magic-byte sniff.
private static readonly byte[] PngBytes = EncodePng(1, 1);
private IImageCache _imageCache = null!;
private UploadArtworkHandler _handler = null!;
@@ -26,7 +32,8 @@ public class UploadArtworkHandlerTests
public void SetUp()
{
_imageCache = Substitute.For<IImageCache>();
_handler = new UploadArtworkHandler(_imageCache);
// Use the REAL validator so the decode-budget check is exercised end to end, not stubbed.
_handler = new UploadArtworkHandler(_imageCache, new RemoteImageValidator());
}
[Test]
@@ -101,6 +108,94 @@ public class UploadArtworkHandlerTests
LeftOf(result).Value.ShouldBe("disk full");
}
[Test]
public async Task Should_Reject_An_Upload_That_Busts_The_Decode_Budget()
{
// A tiny PNG header declaring a 30000x30000 canvas: a decompression bomb, small on the wire
// and huge in memory. The content-type sniff passes (it is a real PNG), so only the
// decode-budget check can stop it entering the cache.
await using MemoryStream bomb = PngHeaderDeclaring(30000, 30000);
Either<BaseError, ArtworkUploadResponseModel> result =
await _handler.Handle(new UploadArtwork(bomb, ArtworkKind.Logo), CancellationToken.None);
result.IsLeft.ShouldBeTrue();
LeftOf(result).Value.ShouldContain("pixel limit");
await _imageCache.DidNotReceive().SaveArtworkToCache(Arg.Any<Stream>(), Arg.Any<ArtworkKind>());
}
[Test]
public async Task Should_Accept_A_Normal_Upload()
{
_imageCache.SaveArtworkToCache(Arg.Any<Stream>(), ArtworkKind.Logo)
.Returns(Right<BaseError, string>("ok789"));
await using MemoryStream png = await RealPng(64, 64);
Either<BaseError, ArtworkUploadResponseModel> result =
await _handler.Handle(new UploadArtwork(png, ArtworkKind.Logo), CancellationToken.None);
result.IsRight.ShouldBeTrue();
}
/// <summary>A real, decodable PNG as a stream.</summary>
private static async Task<MemoryStream> RealPng(int width, int height)
{
using var image = new Image<Rgba32>(width, height);
var stream = new MemoryStream();
await image.SaveAsync(stream, new PngEncoder());
stream.Position = 0;
return stream;
}
/// <summary>A real, decodable PNG as bytes.</summary>
private static byte[] EncodePng(int width, int height)
{
using var image = new Image<Rgba32>(width, height);
using var stream = new MemoryStream();
image.Save(stream, new PngEncoder());
return stream.ToArray();
}
/// <summary>
/// A structurally complete PNG whose IHDR is patched to declare <paramref name="width" /> x
/// <paramref name="height" />: a decompression bomb. It starts from a real 1x1 PNG so the
/// upload's content-type sniff (SkiaSharp <c>SKCodec.Create</c>, which needs a full codec,
/// not a lone IHDR) still recognizes it as image/png; the huge declared dimensions are read
/// from the header by the validator's Identify and rejected by the pixel budget before any
/// pixels are decoded (the patched IDAT never has to be valid at that size).
/// </summary>
private static MemoryStream PngHeaderDeclaring(int width, int height)
{
// A real PNG: [8-byte signature][IHDR: 4 len + 4 "IHDR" + 13 data + 4 CRC] then IDAT/IEND.
byte[] bytes = EncodePng(1, 1);
// IHDR data begins at offset 16 (8 signature + 4 length + 4 "IHDR"); width then height.
BinaryPrimitives.WriteInt32BigEndian(bytes.AsSpan(16), width);
BinaryPrimitives.WriteInt32BigEndian(bytes.AsSpan(20), height);
// Recompute the IHDR CRC over "IHDR" + the 13 data bytes (offset 12, length 17).
uint crc = Crc32(bytes.AsSpan(12, 17));
BinaryPrimitives.WriteUInt32BigEndian(bytes.AsSpan(29), crc);
return new MemoryStream(bytes) { Position = 0 };
}
// PNG chunk CRC-32 (IEEE, reflected). hand-rolled because the repo does not reference
// System.IO.Hashing, and ImageSharp validates the CRC of critical chunks like IHDR.
private static uint Crc32(ReadOnlySpan<byte> data)
{
uint crc = 0xFFFFFFFF;
foreach (byte b in data)
{
crc ^= b;
for (var i = 0; i < 8; i++)
{
crc = (crc & 1) != 0 ? (crc >> 1) ^ 0xEDB88320 : crc >> 1;
}
}
return crc ^ 0xFFFFFFFF;
}
private static TR RightOf<TR>(Either<BaseError, TR> either) =>
either.Match(Right: v => v, Left: e => throw new AssertionException($"Expected Right, got Left: {e.Value}"));
@@ -9,6 +9,7 @@ using ErsatzTV.Core.Api.LibraryBrowse;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Domain.Filler;
using ErsatzTV.Core.Errors;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Core.Scheduling;
using ErsatzTV.Infrastructure.Data;
@@ -19,6 +20,7 @@ using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using static LanguageExt.Prelude;
using DomainChannel = ErsatzTV.Core.Domain.Channel;
using DomainPlaylistItem = ErsatzTV.Core.Domain.PlaylistItem;
@@ -30,6 +32,7 @@ public class CreateChannelFromLineupHandlerTests
private Channel<IBackgroundServiceRequest> _background = null!;
private InMemoryTvContext _db = null!;
private ISearchTargets _searchTargets = null!;
private IRemoteLogoCacher _remoteLogoCacher = null!;
[SetUp]
public async Task SetUp()
@@ -37,6 +40,7 @@ public class CreateChannelFromLineupHandlerTests
_background = System.Threading.Channels.Channel.CreateUnbounded<IBackgroundServiceRequest>();
_db = await InMemoryTvContext.CreateAsync();
_searchTargets = Substitute.For<ISearchTargets>();
_remoteLogoCacher = Substitute.For<IRemoteLogoCacher>();
}
[TearDown]
@@ -610,6 +614,71 @@ public class CreateChannelFromLineupHandlerTests
LeftOf(result).Value.ShouldContain("External logo url is invalid");
}
[Test]
public async Task Should_Download_External_Url_Logo_And_Store_Cache_Name()
{
await SeedTemplateDependencies();
await SeedTemplate();
await SeedMovie(42);
_remoteLogoCacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Right<BaseError, string>("cachedhash"));
var logo = new ArtworkContentTypeModel("https://example.com/logo.png", string.Empty);
Either<BaseError, CreateChannelFromLineupResponseModel> result =
await MakeHandler().Handle(MakeRequest(logo: logo), CancellationToken.None);
result.IsRight.ShouldBeTrue();
await using TvContext context = _db.CreateContext();
Artwork artwork = (await context.Channels.Include(c => c.Artwork).SingleAsync())
.Artwork.Single(a => a.ArtworkKind == ArtworkKind.Logo);
artwork.Path.ShouldBe("cachedhash");
artwork.IsExternalUrl().ShouldBeFalse();
}
[Test]
public async Task Should_Fail_The_Create_When_The_Logo_Download_Fails()
{
await SeedTemplateDependencies();
await SeedTemplate();
await SeedMovie(42);
_remoteLogoCacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Left<BaseError, string>(BaseError.New("Could not download logo: host timed out")));
var logo = new ArtworkContentTypeModel("https://example.com/logo.png", string.Empty);
Either<BaseError, CreateChannelFromLineupResponseModel> result =
await MakeHandler().Handle(MakeRequest(logo: logo), CancellationToken.None);
LeftOf(result).Value.ShouldContain("Could not download logo");
// nothing is persisted when the download fails (resolution runs before PersistAndDispatch)
await using TvContext context = _db.CreateContext();
(await context.Channels.CountAsync()).ShouldBe(0);
(await context.Playouts.CountAsync()).ShouldBe(0);
}
[Test]
public async Task Should_Not_Call_The_Cacher_For_An_Uploaded_Logo_Path()
{
await SeedTemplateDependencies();
await SeedTemplate();
await SeedMovie(42);
var logo = new ArtworkContentTypeModel("iptv/logos/deadbeef", string.Empty);
Either<BaseError, CreateChannelFromLineupResponseModel> result =
await MakeHandler().Handle(MakeRequest(logo: logo), CancellationToken.None);
result.IsRight.ShouldBeTrue();
await _remoteLogoCacher.DidNotReceive().CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>());
await using TvContext context = _db.CreateContext();
Artwork artwork = (await context.Channels.Include(c => c.Artwork).SingleAsync())
.Artwork.Single(a => a.ArtworkKind == ArtworkKind.Logo);
artwork.Path.ShouldBe("deadbeef");
}
[Test]
public async Task Should_Return_NotFound_When_Lineup_Target_Is_Missing()
{
@@ -638,6 +707,7 @@ public class CreateChannelFromLineupHandlerTests
_background.Writer,
_db.Factory,
_searchTargets,
_remoteLogoCacher,
NullLogger<CreateChannelFromLineupHandler>.Instance);
private async Task SeedTemplateDependencies()
@@ -1,19 +1,22 @@
using ErsatzTV.Application.Channels;
using ErsatzTV.Core;
using LanguageExt;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Errors;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using static LanguageExt.Prelude;
namespace ErsatzTV.Tests.Application.Channels;
[TestFixture]
public class CreateChannelHandlerTests : ChannelHandlerTestBase
{
private CreateChannelHandler MakeHandler() => new(Worker, Db.Factory, SearchTargets);
private CreateChannelHandler MakeHandler() => new(Worker, Db.Factory, SearchTargets, RemoteLogoCacher);
[Test]
public async Task Should_Create_Channel_When_Valid()
@@ -99,6 +102,61 @@ public class CreateChannelHandlerTests : ChannelHandlerTestBase
error.Value.ShouldContain("FFmpegProfile");
}
[Test]
public async Task Should_Download_External_Url_Logo_And_Store_Cache_Name()
{
await SeedFFmpegProfile();
RemoteLogoCacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Right<BaseError, string>("cachedhash"));
Either<BaseError, CreateChannelResult> result = await MakeHandler().Handle(
MakeCreate(number: "20", logoPath: "https://example.com/logo.png"),
CancellationToken.None);
result.IsRight.ShouldBeTrue();
await using TvContext db = Db.CreateContext();
Artwork logo = db.Channels.Include(c => c.Artwork).Single(c => c.Number == "20")
.Artwork.Single(a => a.ArtworkKind == ArtworkKind.Logo);
logo.Path.ShouldBe("cachedhash");
logo.IsExternalUrl().ShouldBeFalse();
}
[Test]
public async Task Should_Fail_The_Create_When_The_Logo_Download_Fails()
{
await SeedFFmpegProfile();
RemoteLogoCacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Left<BaseError, string>(BaseError.New("Could not download logo: host timed out")));
Either<BaseError, CreateChannelResult> result = await MakeHandler().Handle(
MakeCreate(number: "21", logoPath: "https://example.com/logo.png"),
CancellationToken.None);
result.IsLeft.ShouldBeTrue();
LeftOf(result).Value.ShouldContain("Could not download logo");
await using TvContext db = Db.CreateContext();
(await db.Channels.AnyAsync(c => c.Number == "21")).ShouldBeFalse();
}
[Test]
public async Task Should_Not_Call_The_Cacher_For_An_Uploaded_Logo_Path()
{
await SeedFFmpegProfile();
Either<BaseError, CreateChannelResult> result = await MakeHandler().Handle(
MakeCreate(number: "22", logoPath: "iptv/logos/deadbeef"),
CancellationToken.None);
result.IsRight.ShouldBeTrue();
await RemoteLogoCacher.DidNotReceive().CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>());
await using TvContext db = Db.CreateContext();
Artwork logo = db.Channels.Include(c => c.Artwork).Single(c => c.Number == "22")
.Artwork.Single(a => a.ArtworkKind == ArtworkKind.Logo);
logo.Path.ShouldBe("deadbeef");
}
private static BaseError LeftOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => e, Right: _ => throw new AssertionException("Expected a Left result"));
}
@@ -6,15 +6,17 @@ using ErsatzTV.Core.Errors;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Tests.Support;
using Microsoft.EntityFrameworkCore;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using static LanguageExt.Prelude;
namespace ErsatzTV.Tests.Application.Channels;
[TestFixture]
public class UpdateChannelHandlerTests : ChannelHandlerTestBase
{
private UpdateChannelHandler MakeHandler() => new(Worker, Db.Factory, SearchTargets);
private UpdateChannelHandler MakeHandler() => new(Worker, Db.Factory, SearchTargets, RemoteLogoCacher);
[Test]
public async Task Should_Return_NotFoundError_When_Channel_Missing()
@@ -115,6 +117,24 @@ public class UpdateChannelHandlerTests : ChannelHandlerTestBase
error.Value.ShouldContain("FFmpegProfile");
}
// the applicative validation accumulates every failure; the 400 body must carry all of them,
// not just the first (regression guard for the #525 handler refactor — errors.Join, not .Head).
[Test]
public async Task Should_Report_All_Validation_Errors_Not_Just_The_First()
{
await SeedFFmpegProfile();
await SeedChannel(1, "5");
Either<BaseError, ChannelViewModel> result =
await MakeHandler().Handle(
MakeUpdate(1, number: "5", group: "", ffmpegProfileId: 999),
CancellationToken.None);
BaseError error = LeftOf(result);
error.Value.ShouldContain("group");
error.Value.ShouldContain("FFmpegProfile");
}
[Test]
public async Task Should_Reject_Mirror_Transition_When_Channel_Has_Playout()
{
@@ -186,6 +206,53 @@ public class UpdateChannelHandlerTests : ChannelHandlerTestBase
channel.PlayoutOffset.ShouldBeNull();
}
[Test]
public async Task Should_Download_External_Url_Logo_And_Store_Cache_Name()
{
await SeedFFmpegProfile();
Channel channel = await SeedChannel(1, "5");
RemoteLogoCacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Right<BaseError, string>("cachedhash"));
Either<BaseError, ChannelViewModel> result = await MakeHandler().Handle(
MakeUpdate(channel.Id, number: "5", logoPath: "https://example.com/logo.png"),
CancellationToken.None);
result.IsRight.ShouldBeTrue();
await using TvContext db = Db.CreateContext();
Artwork logo = db.Channels.Include(c => c.Artwork).Single(c => c.Id == channel.Id)
.Artwork.Single(a => a.ArtworkKind == ArtworkKind.Logo);
logo.Path.ShouldBe("cachedhash");
logo.IsExternalUrl().ShouldBeFalse();
}
[Test]
public async Task Should_Fail_The_Save_When_The_Logo_Download_Fails()
{
await SeedFFmpegProfile();
Channel channel = await SeedChannel(1, "5");
RemoteLogoCacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Left<BaseError, string>(BaseError.New("Could not download logo: host timed out")));
Either<BaseError, ChannelViewModel> result = await MakeHandler().Handle(
MakeUpdate(channel.Id, number: "5", logoPath: "https://example.com/logo.png"),
CancellationToken.None);
result.IsLeft.ShouldBeTrue();
LeftOf(result).Value.ShouldContain("Could not download logo");
}
[Test]
public async Task Should_Not_Call_The_Cacher_For_An_Uploaded_Logo_Path()
{
await SeedFFmpegProfile();
Channel channel = await SeedChannel(1, "5");
await MakeHandler().Handle(
MakeUpdate(channel.Id, number: "5", logoPath: "iptv/logos/deadbeef"),
CancellationToken.None);
await RemoteLogoCacher.DidNotReceive().CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>());
}
private static BaseError LeftOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => e, Right: _ => throw new AssertionException("Expected a Left result"));
}
@@ -134,6 +134,68 @@ public class FFmpegProfileHandlerTests
persisted.QsvPreferNativeDecoder.ShouldBe(false);
}
// ersatztv#529: a stored 0 reached ffmpeg as hwupload=extra_hw_frames=0, leaving the QSV pool no
// headroom; FFmpegState floors it at render time, and these pin that the stored row converges too
// so the profile never keeps displaying a value the pipeline would override.
[TestCase(0, 64)]
[TestCase(-8, 64)]
[TestCase(63, 64)]
[TestCase(64, 64)]
[TestCase(128, 128)]
public async Task Create_Should_Floor_QsvExtraHardwareFrames(int configured, int expected)
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result = await handler.Handle(
MakeCreate(1, qsvExtraHardwareFrames: configured),
CancellationToken.None);
CreateFFmpegProfileResult created = RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.QsvExtraHardwareFrames.ShouldBe(expected);
}
[TestCase(0, 64)]
[TestCase(-8, 64)]
[TestCase(128, 128)]
public async Task Update_Should_Floor_QsvExtraHardwareFrames(int configured, int expected)
{
await SeedProfile(1);
await SeedResolution(1);
var handler = new UpdateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, UpdateFFmpegProfileResult> result = await handler.Handle(
MakeUpdate(1, qsvExtraHardwareFrames: configured),
CancellationToken.None);
RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(1);
persisted.QsvExtraHardwareFrames.ShouldBe(expected);
}
// null means "unconfigured" and FFmpegState already resolves it to the same 64; it must stay
// null rather than being silently materialized into a stored value
[Test]
public async Task Create_Should_Leave_Null_QsvExtraHardwareFrames_Null()
{
await SeedResolution(1);
var handler = new CreateFFmpegProfileHandler(_db.Factory, _searchTargets);
Either<BaseError, CreateFFmpegProfileResult> result =
await handler.Handle(MakeCreate(1), CancellationToken.None);
CreateFFmpegProfileResult created = RightOf(result);
await using TvContext context = _db.CreateContext();
FFmpegProfile persisted = await context.FFmpegProfiles.FindAsync(created.FFmpegProfileId);
persisted.QsvExtraHardwareFrames.ShouldBeNull();
}
private static TR RightOf<TR>(Either<BaseError, TR> either) =>
either.Match(Left: e => throw new AssertionException($"Expected a Right result, got {e}"), Right: r => r);
@@ -184,7 +246,10 @@ public class FFmpegProfileHandlerTests
await context.SaveChangesAsync();
}
private static CreateFFmpegProfile MakeCreate(int resolutionId, bool qsvPreferNativeDecoder = true) =>
private static CreateFFmpegProfile MakeCreate(
int resolutionId,
bool qsvPreferNativeDecoder = true,
int? qsvExtraHardwareFrames = null) =>
new(
"Default",
1,
@@ -194,7 +259,7 @@ public class FFmpegProfileHandlerTests
"drm",
VaapiDriver.Default,
"/dev/dri/renderD128",
null,
qsvExtraHardwareFrames,
resolutionId,
ScalingBehavior.ScaleAndPad,
FilterMode.Software,
@@ -221,7 +286,8 @@ public class FFmpegProfileHandlerTests
private static UpdateFFmpegProfile MakeUpdate(
int id,
int resolutionId = 1,
bool qsvPreferNativeDecoder = true) =>
bool qsvPreferNativeDecoder = true,
int? qsvExtraHardwareFrames = null) =>
new(
id,
"Default",
@@ -232,7 +298,7 @@ public class FFmpegProfileHandlerTests
"drm",
VaapiDriver.Default,
"/dev/dri/renderD128",
null,
qsvExtraHardwareFrames,
resolutionId,
ScalingBehavior.ScaleAndPad,
FilterMode.Software,
@@ -21,7 +21,7 @@ public class ChannelLifecycleIntegrationTests : ChannelHandlerTestBase
{
await SeedFFmpegProfile();
var createHandler = new CreateChannelHandler(Worker, Db.Factory, SearchTargets);
var createHandler = new CreateChannelHandler(Worker, Db.Factory, SearchTargets, RemoteLogoCacher);
Either<BaseError, CreateChannelResult> created =
await createHandler.Handle(MakeCreate(number: "42", name: "Integration"), CancellationToken.None);
@@ -0,0 +1,114 @@
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Infrastructure.Data;
using ErsatzTV.Services.RunOnce;
using ErsatzTV.Tests.Support;
using LanguageExt;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Tests.Services;
[TestFixture]
public class ExternalLogoMigratorTests
{
private InMemoryTvContext _db = null!;
private IRemoteLogoCacher _cacher = null!;
[SetUp]
public async Task SetUp()
{
_db = await InMemoryTvContext.CreateAsync();
_cacher = Substitute.For<IRemoteLogoCacher>();
}
[TearDown]
public async Task TearDown() => await _db.DisposeAsync();
[Test]
public async Task Should_Convert_Url_Logo_Row_To_Cache_Name()
{
await Seed(new Artwork
{
Path = "https://example.com/logo.png",
ArtworkKind = ArtworkKind.Logo
});
_cacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Either<BaseError, string>.Right("hash"));
await using (TvContext db = _db.CreateContext())
{
await ExternalLogoMigratorService.MigrateAsync(
db,
_cacher,
NullLogger.Instance,
CancellationToken.None);
}
await using TvContext verify = _db.CreateContext();
Artwork row = await verify.Artwork.SingleAsync();
row.Path.ShouldBe("hash");
row.IsExternalUrl().ShouldBeFalse();
}
[Test]
public async Task Should_Leave_Row_Unchanged_When_Cacher_Fails()
{
const string Url = "https://example.com/logo.png";
await Seed(new Artwork
{
Path = Url,
ArtworkKind = ArtworkKind.Logo
});
_cacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Either<BaseError, string>.Left(BaseError.New("boom")));
await using (TvContext db = _db.CreateContext())
{
await ExternalLogoMigratorService.MigrateAsync(
db,
_cacher,
NullLogger.Instance,
CancellationToken.None);
}
await using TvContext verify = _db.CreateContext();
Artwork row = await verify.Artwork.SingleAsync();
row.Path.ShouldBe(Url);
row.IsExternalUrl().ShouldBeTrue();
}
[Test]
public async Task Should_Not_Call_Cacher_For_Already_Migrated_Rows()
{
await Seed(new Artwork
{
Path = "some-bare-hash",
ArtworkKind = ArtworkKind.Logo
});
await using (TvContext db = _db.CreateContext())
{
await ExternalLogoMigratorService.MigrateAsync(
db,
_cacher,
NullLogger.Instance,
CancellationToken.None);
}
await _cacher.DidNotReceive().CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>());
}
private async Task Seed(Artwork artwork)
{
await using TvContext db = _db.CreateContext();
await db.Artwork.AddAsync(artwork);
await db.SaveChangesAsync();
}
}
@@ -3,6 +3,7 @@ using ErsatzTV.Application;
using ErsatzTV.Application.Artworks;
using ErsatzTV.Application.Channels;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Search;
using ErsatzTV.Infrastructure.Data;
using NSubstitute;
@@ -16,6 +17,7 @@ public abstract class ChannelHandlerTestBase
protected InMemoryTvContext Db = null!;
protected ChannelWriter<IBackgroundServiceRequest> Worker = null!;
protected ISearchTargets SearchTargets = null!;
protected IRemoteLogoCacher RemoteLogoCacher = null!;
[SetUp]
public async Task BaseSetUp()
@@ -23,6 +25,7 @@ public abstract class ChannelHandlerTestBase
Db = await InMemoryTvContext.CreateAsync();
Worker = System.Threading.Channels.Channel.CreateUnbounded<IBackgroundServiceRequest>().Writer;
SearchTargets = Substitute.For<ISearchTargets>();
RemoteLogoCacher = Substitute.For<IRemoteLogoCacher>();
}
[TearDown]
@@ -35,7 +38,7 @@ public abstract class ChannelHandlerTestBase
await context.SaveChangesAsync();
}
protected async Task SeedChannel(
protected async Task<DomainChannel> SeedChannel(
int id,
string number,
string name = "Test",
@@ -43,25 +46,26 @@ public abstract class ChannelHandlerTestBase
string group = "ErsatzTV")
{
await using TvContext context = Db.CreateContext();
context.Channels.Add(
new DomainChannel(Guid.NewGuid())
{
Id = id,
Number = number,
Name = name,
Group = group,
Categories = string.Empty,
FFmpegProfileId = ffmpegProfileId,
StreamSelector = string.Empty,
PreferredAudioLanguageCode = string.Empty,
PreferredAudioTitle = string.Empty,
PreferredSubtitleLanguageCode = string.Empty,
MusicVideoCreditsTemplate = string.Empty,
StreamingMode = StreamingMode.TransportStreamHybrid,
PlayoutSource = ChannelPlayoutSource.Generated,
PlayoutMode = ChannelPlayoutMode.Continuous
});
var channel = new DomainChannel(Guid.NewGuid())
{
Id = id,
Number = number,
Name = name,
Group = group,
Categories = string.Empty,
FFmpegProfileId = ffmpegProfileId,
StreamSelector = string.Empty,
PreferredAudioLanguageCode = string.Empty,
PreferredAudioTitle = string.Empty,
PreferredSubtitleLanguageCode = string.Empty,
MusicVideoCreditsTemplate = string.Empty,
StreamingMode = StreamingMode.TransportStreamHybrid,
PlayoutSource = ChannelPlayoutSource.Generated,
PlayoutMode = ChannelPlayoutMode.Continuous
};
context.Channels.Add(channel);
await context.SaveChangesAsync();
return channel;
}
protected async Task SeedPlayout(int id, int channelId)
+40
View File
@@ -32,6 +32,10 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ErsatzTV.Tests", "ErsatzTV.
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ErsatzTV.Architecture.Tests", "ErsatzTV.Architecture.Tests\ErsatzTV.Architecture.Tests.csproj", "{49123AF1-38FE-4546-9DBD-8B7F1B4CFC7F}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ErsatzTV.Mcp", "ErsatzTV.Mcp\ErsatzTV.Mcp.csproj", "{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ErsatzTV.Mcp.Tests", "ErsatzTV.Mcp.Tests\ErsatzTV.Mcp.Tests.csproj", "{65F2FAF2-705F-4837-A92B-BB66182B163E}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
@@ -315,6 +319,42 @@ Global
{49123AF1-38FE-4546-9DBD-8B7F1B4CFC7F}.Debug No Sync|x64.Build.0 = Debug|Any CPU
{49123AF1-38FE-4546-9DBD-8B7F1B4CFC7F}.Debug No Sync|x86.ActiveCfg = Debug|Any CPU
{49123AF1-38FE-4546-9DBD-8B7F1B4CFC7F}.Debug No Sync|x86.Build.0 = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug|Any CPU.Build.0 = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug|x64.ActiveCfg = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug|x64.Build.0 = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug|x86.ActiveCfg = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug|x86.Build.0 = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Release|Any CPU.ActiveCfg = Release|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Release|Any CPU.Build.0 = Release|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Release|x64.ActiveCfg = Release|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Release|x64.Build.0 = Release|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Release|x86.ActiveCfg = Release|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Release|x86.Build.0 = Release|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug No Sync|Any CPU.ActiveCfg = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug No Sync|Any CPU.Build.0 = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug No Sync|x64.ActiveCfg = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug No Sync|x64.Build.0 = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug No Sync|x86.ActiveCfg = Debug|Any CPU
{2D0F528F-1260-45E5-BA0E-2BAFBCAE6AE2}.Debug No Sync|x86.Build.0 = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug|x64.ActiveCfg = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug|x64.Build.0 = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug|x86.ActiveCfg = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug|x86.Build.0 = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Release|Any CPU.ActiveCfg = Release|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Release|Any CPU.Build.0 = Release|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Release|x64.ActiveCfg = Release|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Release|x64.Build.0 = Release|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Release|x86.ActiveCfg = Release|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Release|x86.Build.0 = Release|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug No Sync|Any CPU.ActiveCfg = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug No Sync|Any CPU.Build.0 = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug No Sync|x64.ActiveCfg = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug No Sync|x64.Build.0 = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug No Sync|x86.ActiveCfg = Debug|Any CPU
{65F2FAF2-705F-4837-A92B-BB66182B163E}.Debug No Sync|x86.Build.0 = Debug|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
@@ -0,0 +1,90 @@
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Infrastructure.Data;
using LanguageExt;
using Microsoft.EntityFrameworkCore;
namespace ErsatzTV.Services.RunOnce;
/// <summary>
/// One-time startup migration that downloads existing external-URL channel logos into the image
/// cache. Before ersatztv#525 a channel logo could be stored as a raw http(s) URL in
/// <see cref="Artwork.Path" />; the render path used to fetch it live. Now that URLs are cached on
/// save, these legacy rows are converted here. A download failure leaves the row untouched and logs
/// a warning naming the URL — re-saving the channel fixes it. Idempotent: a converted row's Path is
/// a bare cache name, so a second run selects nothing.
/// </summary>
public class ExternalLogoMigratorService(
IServiceScopeFactory serviceScopeFactory,
ILogger<ExternalLogoMigratorService> logger,
SystemStartup systemStartup)
: BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await Task.Yield();
await systemStartup.WaitForDatabase(stoppingToken);
if (stoppingToken.IsCancellationRequested)
{
return;
}
logger.LogInformation("Migrating external URL channel logos to the image cache");
try
{
using IServiceScope scope = serviceScopeFactory.CreateScope();
await using TvContext dbContext = scope.ServiceProvider.GetRequiredService<TvContext>();
IRemoteLogoCacher cacher = scope.ServiceProvider.GetRequiredService<IRemoteLogoCacher>();
await MigrateAsync(dbContext, cacher, logger, stoppingToken);
logger.LogInformation("Done migrating external URL channel logos to the image cache");
}
catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested)
{
// shutdown mid-migration — the single trailing SaveChangesAsync never ran, so no partial
// persist; the next boot retries idempotently.
}
catch (Exception ex)
{
// this is a run-once BackgroundService: an escaping exception trips the default
// StopHost behavior and kills the app. A logo migration must never do that — the fetch
// races (e.g. a channel deleted mid-run -> DbUpdateConcurrencyException) are transient
// and self-heal on the next boot. Log and let the host keep serving.
logger.LogError(ex, "Failed migrating external URL channel logos to the image cache; will retry next start");
}
}
internal static async Task MigrateAsync(
TvContext db,
IRemoteLogoCacher cacher,
ILogger logger,
CancellationToken cancellationToken)
{
// IsExternalUrl is a C# predicate EF cannot translate, so load logo artwork then filter in memory.
List<Artwork> logos = await db.Artwork
.Where(a => a.ArtworkKind == ArtworkKind.Logo)
.ToListAsync(cancellationToken);
foreach (Artwork artwork in logos.Where(a => a.IsExternalUrl()))
{
string oldUrl = artwork.Path;
Either<BaseError, string> result = await cacher.CacheFromUrl(new Uri(oldUrl), cancellationToken);
result.Match(
name =>
{
artwork.Path = name;
artwork.DateUpdated = DateTime.UtcNow;
},
error => logger.LogWarning(
"Could not download existing channel logo {Url}; leaving it. Re-save the channel to fix. ({Error})",
oldUrl,
error.Value));
}
await db.SaveChangesAsync(cancellationToken);
}
}
+22
View File
@@ -686,6 +686,21 @@ public class Startup
services.AddHttpClient("RefitCustomClient").AddHttpMessageHandler<SlowApiHandler>();
// graphics-engine remote images (channel-logo watermarks and image elements). the client
// timeout is INFINITE on purpose: under ResponseHeadersRead the body read falls outside
// HttpClient.Timeout, so HttpRemoteImageFetcher owns the deadline with a linked CTS that
// covers headers AND body. the fetcher re-asserts that on the client it gets, so losing
// this line widens nothing; what is ONLY configurable here is the redirect cap, since it
// lives on the handler. redirects stay enabled (logo hosts and CDNs use them) but are
// capped well below the default 50 hops. (ersatztv#511)
services.AddHttpClient(HttpRemoteImageFetcher.HttpClientName)
.ConfigureHttpClient(c => c.Timeout = Timeout.InfiniteTimeSpan)
.ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler
{
AllowAutoRedirect = true,
MaxAutomaticRedirections = 3
});
services.Configure<TraktConfiguration>(Configuration.GetSection("Trakt"));
services.AddResponseCompression(options => { options.EnableForHttps = true; });
@@ -1085,6 +1100,9 @@ public class Startup
services.AddScoped<ILocalStatisticsProvider, LocalStatisticsProvider>();
services.AddScoped<IExternalJsonPlayoutItemProvider, ExternalJsonPlayoutItemProvider>();
services.AddScoped<IRemoteStreamProber, HttpRemoteStreamProber>();
services.AddScoped<IRemoteImageFetcher, HttpRemoteImageFetcher>();
services.AddScoped<IRemoteImageValidator, RemoteImageValidator>();
services.AddScoped<IRemoteLogoCacher, RemoteLogoCacher>();
services.AddScoped<IPlayoutBuilder, PlayoutBuilder>();
services.AddScoped<IBlockPlayoutBuilder, BlockPlayoutBuilder>();
services.AddScoped<IBlockPlayoutPreviewBuilder, BlockPlayoutPreviewBuilder>();
@@ -1159,6 +1177,10 @@ public class Startup
// BackgroundService, so registration order alone does not guarantee the schema exists.
services.AddHostedService<LocalAdminSeedService>();
services.AddHostedService<DatabaseCleanerService>();
// One-time migration of existing external-URL channel logos into the image cache (ersatztv#525).
// It awaits SystemStartup.WaitForDatabase itself, so the schema is guaranteed; registration
// order relative to the other hosted services is not load-bearing (it touches only logo artwork).
services.AddHostedService<ExternalLogoMigratorService>();
services.AddHostedService<LoadLoggingLevelService>();
services.AddHostedService<CacheCleanerService>();
services.AddHostedService<ResourceExtractorService>();
+79 -28
View File
@@ -1,36 +1,85 @@
# docs/ — reading order
# docs/ — task-signal map
Purpose: index of `docs/` so a fresh contributor/agent knows what to read and in what order.
**Update this doc in the same PR that adds, removes, or retitles a doc below.**
Purpose: route a fresh contributor/agent to the minimal set of docs for the task at hand, instead of
a mandatory front-to-back read. **Update this doc in the same PR that adds, removes, or retitles a
doc below, or that changes which sections a task signal points to.**
Read in this order at session start:
## Start here, always
1. **`CLAUDE.md`** (repo root) — project intro: architecture, layout, dev commands, conventions.
2. **`docs/contributing.md`** — established code patterns (CQRS/MediatR, LanguageExt, the ChicoryTV
SPA, EF Core dual-provider migrations, FFmpeg pipeline, analyzers, testing). Read before
any non-trivial change.
3. **`docs/domain-model.md`** — what the app IS: entity glossary, channel→playout→schedule/block
concept map, where each concept is edited in the SPA.
4. **`docs/api-conventions.md`** — checklist for adding/changing a `/api/*` endpoint (controllers,
DTOs, error mapping, auth, OpenAPI regen, tests).
5. **`docs/spa-conventions.md`** — playbook for adding a screen to the ChicoryTV React SPA.
6. **`docs/e2e-local.md`** (+ `scripts/e2e-local.sh`) — how to run a live local instance for manual
or Playwright-MCP verification.
7. **`docs/testing.md`** — testing map: what each `*.Tests` project / `web` suite covers,
golden-file nets, the timezone-independence rule, how to run subsets, the per-PR verification
gate.
8. **`docs/blazor-route-parity.md`** — historical record of the completed #91 phase (b) cutover:
the Blazor Server UI is removed and every legacy route now 302-redirects to its SPA equivalent
(or falls through to the catch-all → `/app`). Read it for the full legacy→SPA route inventory.
9. **`docs/decisions.md`** — append-only "why" log. Check here before challenging an existing
convention. Start from its **Index**, which links the four topic files under `docs/decisions/`
(large same-topic clusters) and lists the remaining in-file entries.
10. **`docs/ci-cd.md`** — build/test/release pipeline, versioning, dependency management.
- **`CLAUDE.md`** (repo root) — project intro: architecture, layout, dev commands, conventions,
Task Completion Protocol.
- **`docs/contributing.md`** — established code patterns (CQRS/MediatR, LanguageExt, the ChicoryTV
SPA, EF Core dual-provider migrations, FFmpeg pipeline, analyzers, testing). Read before any
non-trivial change.
Also present in `docs/`:
## Task signal → minimal sections
| Signal | Read |
| --- | --- |
| Session startup / "what's next" (no issue named) | `docs/handoffs/chicorytv-issue-queue.md` (standing kickoff — two concurrent tracks: orientation ‖ `scripts/select-queue.sh 5`) |
| Named-issue pickup | Skip queue selection; go straight to focused retrieval — see "Knowledge retrieval" below, then the issue body |
| Adding/changing a `/api/*` endpoint | `docs/api-conventions.md` checklist + `docs/endpoint-index.md` |
| Adding a ChicoryTV SPA screen | `docs/spa-conventions.md` |
| Scheduling / playout engine work | `docs/domain-model.md` + decisions catalog rows keyed `sched.*` (`docs/decisions/README.md`) |
| Concurrency / optimistic-locking work | `docs/api-conventions.md` §7a/b/c + `docs/decisions/optimistic-concurrency.md` |
| Auth / security-surface work | `docs/decisions/api-auth-security.md` |
| CI / release pipeline work | `docs/ci-cd.md` + `docs/decisions/release-ci-governance.md` |
| Live local run / Playwright-MCP verification | `docs/e2e-local.md` + `scripts/e2e-local.sh` |
| What does a test suite cover | `docs/testing.md` |
| Legacy Blazor route lookup | `docs/blazor-route-parity.md` (historical #91 phase (b) inventory) |
| "Why do we do X this way" / challenging a convention | **Catalog-first**: `docs/decisions/README.md` (active rows) → `docs/decisions.md` + `docs/decisions/*.md` for full rationale. `docs/decisions/archive/` only for "what did the rule used to be." |
## Knowledge retrieval (MemPalace + catalog + Gitea)
These four rules are the seam agreed with server-management#642 (the Gitea→MemPalace exporter).
They apply whether the question comes up via MemPalace, a grep, or a stale comment:
1. **Current conventions/decisions → catalog-first.** Start at `docs/decisions/README.md`; discover
via the `ErsatzTV-Decisions` wing (active) / `ErsatzTV-Decisions-Archive` (superseded/retired).
**Resolve by topic/key, never by chasing a file path.**
2. **Issue history → evidence, not authority.** The `Gitea-ErsatzTV` wing is historical narrative
that may be stale; it never overrides current Markdown.
3. **The breadcrumb rule (the crux behavior change).** A file path named inside a *historical issue
comment* (e.g. "grep `docs/decisions.md` 2026-07-17", "see …") is a **breadcrumb, not a live
pointer.** Find the current rule via the catalog / active wing **by concept**; do not treat the
named path as current. (Why it's safe: still-current → in the active wing, breadcrumb resolves;
superseded → the active wing returns the *successor* and a literal follow lands on a record that
announces its own `status: superseded`; retired → the active wing returns nothing, which is
itself the signal. The validator-enforced move-to-`archive/` is what prevents the catastrophic
"superseded rule read as current" case.)
4. **Fallback when MemPalace is stale/down:** `docs/decisions/README.md` catalog, then
`` rg '^`key: <dotted.key>`' docs/decisions/ ``. MemPalace is never authority nor sole fallback.
MemPalace is candidate discovery only — every passage is verified against its cited Markdown/Gitea
source before use. Never derive live queue state from MemPalace, #237, or historical comments; queue
state is live Gitea state, retrieved via `scripts/select-queue.sh` (see
`docs/handoffs/chicorytv-issue-queue.md`). Full retrieval contract (altitude/precedence, staleness
bounds, what's mined per issue): `docs/handoffs/chicorytv-issue-queue.md` → "Knowledge retrieval".
## Also present in `docs/`
- **`docs/domain-model.md`** — what the app IS: entity glossary, channel→playout→schedule/block
concept map, where each concept is edited in the SPA.
- **`docs/api-conventions.md`** — checklist for adding/changing a `/api/*` endpoint (controllers,
DTOs, error mapping, auth, OpenAPI regen, tests).
- **`docs/spa-conventions.md`** — playbook for adding a screen to the ChicoryTV React SPA.
- **`docs/e2e-local.md`** (+ `scripts/e2e-local.sh`) — how to run a live local instance for manual
or Playwright-MCP verification.
- **`docs/testing.md`** — testing map: what each `*.Tests` project / `web` suite covers,
golden-file nets, the timezone-independence rule, how to run subsets, the per-PR verification
gate.
- **`docs/blazor-route-parity.md`** — historical record of the completed #91 phase (b) cutover:
the Blazor Server UI is removed and every legacy route now 302-redirects to its SPA equivalent
(or falls through to the catch-all → `/app`). Read it for the full legacy→SPA route inventory.
- **`docs/decisions.md`** + **`docs/decisions/*.md`** — active decision records (lifecycle schema:
key/status/since/supersedes/superseded-by). **Generated active view**: `docs/decisions/README.md`
(catalog / task router) — start there. Superseded/retired records live in
`docs/decisions/archive/` and are read only for history, never for "what is the current rule."
- **`docs/ci-cd.md`** — build/test/release pipeline, versioning, dependency management.
- **`docs/rest-api.md`** — REST API design doc for ersatztv#2 (goals, conventions, per-slice plan).
Largely superseded day-to-day by `docs/api-conventions.md`; read this for the original rationale.
- **`docs/mcp.md`** — the `ErsatzTV.Mcp` stdio JSON-RPC MCP server (#58): how it wraps `/api/v1` as
read + cautious-write tools, its config/env vars, auth, security posture, and the tool catalog.
- **`docs/channels.md`** — Channel entity field reference.
- **`docs/m3u-xmltv.md`** — M3U/XMLTV generation overview (`ChannelPlaylist`, `GetChannelGuideHandler`).
- **`docs/fork-strategy.md`** — divergence policy vs upstream ErsatzTV.
@@ -39,6 +88,8 @@ Also present in `docs/`:
OpenAPI tag). Do not edit by hand; regenerated by `scripts/generate-endpoint-index.py` /
`scripts/update-openapi.sh`.
- **`docs/handoffs/chicorytv-issue-queue.md`** — static session kickoff prompt + workflow lore.
Queue state (goal, arc, what's in flight/next) lives in the pinned Gitea tracker ersatztv#237
— read that, not this file, for current state (protocol: decisions.md 2026-07-11).
Queue state is **live Gitea state**, retrieved each session via `scripts/select-queue.sh` — see
that file's standing kickoff for the two concurrent tracks (orientation ‖ selection). ersatztv#237
is a closed, archival historical tracker (superseded by `startup.parallel-orientation` in
`docs/decisions.md`) — not a live pointer.
- **`docs/handoffs/rest-api.md`** — original handoff prompt for kicking off the REST API work (#2).
+20 -6
View File
@@ -282,12 +282,26 @@ mappers) for new API DTOs — those still return the old Blazor-convention relat
the domain/VM directly and root the path yourself, following the PR #181 pattern.
Channel **logos** live under a different route than posters/thumbnails: an uploaded logo roots to
`/iptv/logos/{file}` (served by `IptvController`), and an external logo is an absolute URL passed
through unchanged. Browse-surface DTOs (`ChannelResponseModel` list, `ChannelGuideChannelResponseModel`
guide) get this rooted `Logo` URL from the single `Channels.Mapper.GetLogoUrl` helper (#464), which
returns `null` when the channel has no logo so the SPA falls back to its generated initials icon. The
raw un-rooted `{path, contentType}` form is still used only by the channel **editor** DTO
(`ChannelDetailResponseModel.Logo`), which round-trips it back on save.
`/iptv/logos/{file}` (served by `IptvController`). An **external logo URL is no longer stored as a URL**
since #525, `PUT`/`POST /api/v1/channels…` downloads it, decode-validates it, and caches it at save
time, so `Artwork.Path` holds a content-hash name and the browse/guide DTOs emit an `/iptv/logos/…`
URL exactly as for an uploaded logo. Browse-surface DTOs (`ChannelResponseModel` list,
`ChannelGuideChannelResponseModel` guide) get this rooted `Logo` URL from the single
`Channels.Mapper.GetLogoUrl` helper (#464), which returns `null` when the channel has no logo so the
SPA falls back to its generated initials icon. The raw un-rooted `{path, contentType}` form is still
used only by the channel **editor** DTO (`ChannelDetailResponseModel.Logo`), which round-trips it back
on save.
**New logo-download rejections (#525).** `PUT /api/v1/channels/{id}`, the two channel-create
endpoints, and `POST /api/v1/artwork/uploads` now reject a logo that cannot be used. The failure is a
`BaseError`, so it surfaces as this API's standard **422 `ValidationProblemDetails`** (via
`ToErrorResult()`), **not** a 400 — a 400 here still means model-binding/validation-attribute failure.
Rejected cases: an external URL that is unreachable, times out (>10s), is oversized (>10 MiB), is not
an image, or is a decode bomb (over 50 MP total pixels or 600 frames); an upload gets the same
decode-budget check. The `detail` names the reason (e.g. *"Could not download logo from … : Connection
refused"*, *"Remote image … returned content type 'text/html'"*, *"Image cannot be used: … pixel
limit"*). Response **shapes are unchanged** — only the error set — so the OpenAPI models did not change.
(Verified by local live-E2E: good URL → cached `/iptv/logos/<hash>`; unreachable/non-image → 422.)
`GET /api/v1/watermarks` returns picker-grade rows that carry `imageSource` alongside `id`/`name`
(#67), so a client can find the seeded logo-driven `Channel Bug` preset without matching its
+13 -4
View File
@@ -164,10 +164,19 @@ the system channel templates it creates, so the library-to-lineup builder (which
are left alone, so builder-created and auto-tuned channels there inherit whatever the template
already specifies.
**Limitation:** a logo set via **External logo URL** cannot drive the bug. `WatermarkSelector`
resolves it to the URL and then `File.Exists`-checks it, which is never true, so the watermark is
silently dropped the URL wins for the guide listing but disables the on-screen bug. Tracked as
**#502**; the editor does not offer a bug preview in that case.
**An external logo URL drives the bug too — it is downloaded and cached at save time (#525).** When
you save a channel whose logo is an **External logo URL**, the URL is fetched, decode-validated, and
stored in the image cache under a content-hash name — after which it is byte-identical to an
uploaded logo. So `Artwork.Path` never holds a URL: the on-screen bug renders, the editor previews
it, and M3U/XMLTV emit the cached `/iptv/logos/…` URL like any uploaded logo. A URL that is dead,
slow (>10s), oversized (>10 MiB), a non-image, or a decode bomb (over 50 MP total or 600 frames)
**fails the save with a specific 422** in the editor — you see it immediately, rather than a silent
render-time drop at 3am (the pre-#502/#511 behavior). To change the remote image, re-enter the URL;
there is no refresh button by design. Existing channels that still hold a raw URL are converted by a
one-time startup migration; one that fails to download is left alone (a warning names it) and renders
with no bug until you re-save it. Historical context (the old `File.Exists`-on-a-URL drop, and the
bounded render-time fetch that preceded caching) is in `docs/decisions.md` under
`graphics.channel-logo-caching`, #502 and #511.
Note: `ChannelLogoGenerator.GenerateChannelLogoUrl()` hardcodes `localhost` for watermark logo
fetching — see issue #1 for details.
+110 -41
View File
@@ -38,12 +38,30 @@ Upstream's final release was **`v26.3.0`** (archived). Our line continues from t
| `v26.9.0` | Configurable advertised IPTV base URL for M3U/XMLTV (#340) + SPA shell/routing + playouts modularization (#247/#245); on-air/Plex/library-path fixes (#99/#345/#371); coverage + functional-E2E CI (#15/#299). |
| `v26.10.0` | Auto-Tune channel workflow (#69) + weighted content distribution (#70); scheduling refactors, health-check remediation UX (#164), HLS cold-start instrumentation (#350), security hardening (#293/#376/#308). |
| `v26.11.0` | **QSV profiles decode via VA-API**`QsvPreferNativeDecoder`, default **on**, fixes ~50% channel cold-start failures on Intel (#498); unified logo/on-screen bug via a shared watermark preset (#67). Media-scanner resilience: Jellyfin mixed-content libraries (#489), music-video scan correctness (#488/#494/#497), remote-stream probing before ffmpeg (#473/#480); weighted-distribution SPA (#404). **First release deployed to `jazz`** (server-management#633). |
| `v26.12.0` | **`ErsatzTV.Mcp` MCP server** — read + cautious-write over `/api/v1`, `ERSATZTV_ALLOW_WRITES`-gated (#58). **External channel-logo URLs download + cache at save time** (#525), with the on-screen bug now rendered for external-URL logos (#502). HLS cold-start hardening: burst-read the first segments so start isn't `-readrate`-bound (#350) and floor QSV extra hardware frames so an unthrottled read can't exhaust the pool (#529); remote graphics-engine image fetches bounded — timeout, size cap, decode cap, redirects, pooling (#511). Decision-lifecycle tooling + parallel-orientation startup rewrite (#520/#521); CI `docker build` lane rebalance (#508). |
**Before cutting a release — consolidate `docs/decisions.md`.** The log is append-only between releases
(ersatztv#303 H9), so a release boundary is where superseded entries get pruned/merged and the Index
refreshed. Fold and drop any entry marked `> **Superseded …**`, then commit with `[decisions-edit]` in
the message (the append-only guard blocks history edits otherwise). Mark-and-keep during the arc,
consolidate here — or sooner if the `decisions-guard` job's 1800-line consolidation-floor warning fires.
**Before cutting a release — sweep `docs/decisions.md` + `docs/decisions/`** (ersatztv#521, supersedes
the ersatztv#303 H9 append-only ritual). Supersession/retirement is now a same-PR act (add the new
active record, relocate the predecessor to `docs/decisions/archive/` with reciprocal
`supersedes`/`superseded-by` links), not a release-boundary batch job — most of the old "consolidate"
step is now continuous. The release boundary is instead where you:
1. Run `PYTHONPATH=. python3 scripts/decisions_validate.py` — confirms lifecycle metadata is
well-formed and every `supersedes`/`superseded-by` link resolves both ways.
2. Confirm every record already classified `superseded`/`retired` actually lives under
`docs/decisions/archive/` (the validator fails this, but eyeball it at the boundary too).
3. Regenerate the active catalog: `PYTHONPATH=. python3 scripts/build_decisions_catalog.py` and
commit any drift.
4. Check the aggregate active-corpus budget (`decisions_validate.py --budget <n>`, default **4800**
lines across `docs/decisions.md` + topic files + the catalog — replaces the old single-file
1800-line floor). **Re-baselined 2026-07-21 (#520)**: the corpus is now fully migrated at
~4366 lines; 4800 gives headroom so the warning fires on real future growth, not on the expected
post-migration size. Going over budget is a **non-blocking warning** (`::warning::` to stderr,
not a validator error) — a ratchet/reminder to extract a new topic file or archive more history,
not a release gate.
5. Report the remaining `legacy-unmigrated` count (the validator prints it as a `::notice::`) so the
backlog is visible, even though it isn't required to hit zero before a release.
A genuine rationale-prose rewrite still needs `[decisions-edit]` in the commit message (see the
`decisions.md` header) — routine lifecycle metadata writes above do not.
**Cutting a release:** keep build and promotion as two explicit phases (#335):
@@ -52,11 +70,20 @@ consolidate here — or sooner if the `decisions-guard` job's 1800-line consolid
2. Wait for tag CI to build `:prod` + the immutable `:<version>` + `:<sha>` images. Run
`scripts/security-scan.sh` on jazz against **the immutable `:<version>` image**, not a
moving tag, and triage every ZAP/semgrep finding.
3. Only after the candidate passes, manually `DeployStack media-servers` and observe its pre-deploy
3. Only after the candidate passes, manually `DeployStack jazz-media` and observe its pre-deploy
output. Prod's compose deliberately follows floating `:prod` (Timothy's 2026-07-11 decision), so
no CI push or pin bump is needed. Global Auto Update (`auto_update: true` on `media-servers`) is
the daily fallback, not the pre-scan promotion mechanism; do not cut a tag close enough to its
03:00 run that an unscanned digest could be promoted first.
no CI push or pin bump is needed.
> **The Komodo stack is `jazz-media`, not `media-servers`** (verified live 2026-07-20 during the
> v26.11.0 cut). The *compose project* is still `media-servers` — which is what the container labels
> show — but the **Komodo stack name** changed with the move to jazz. A stack named `media-servers`
> still exists on bumblebee and is `unhealthy` (the stopped migration leftovers), so
> `DeployStack media-servers` silently targets the **dead** stack. Confirm with
> `/read ListStacks` before deploying.
>
> **There is no Global Auto Update fallback anymore**: `jazz-media` has `auto_update: false`
> (`poll_for_updates: true` only), so nothing promotes `:prod` on a timer — promotion is manual,
> full stop. The old "don't cut a tag near the 03:00 run" caveat no longer applies.
server-management#585 source-confirmed that Global Auto Update invokes the **same** `DeployStack`
execution as a manual promotion, and extended the #553 pre-deploy hook to detect a floating-tag
@@ -101,16 +128,30 @@ its superseded run, and image builds still serialize within their own ref. Do NO
(History: originally one global group serializing ALL runs for the single runner —
with three runners that starved the queue; changed 2026-07-11, server-management#574.)
Three runners serve the fork (server-management#570/#574): `ci-runner` (VM 127 pve4,
`ubuntu-latest`, 2 slots), `bumblebee-runner` (bumblebee, `ubuntu-latest`, 2 slots,
jobs capped `--cpus=4 --memory=10g` so CI can't starve prod media playback), and
`small-runner` (bumblebee, label **`small`**, 4 slots) — the small-jobs lane. The
`build` and `docs-reminder` jobs use `runs-on: small`: Gitea dispatches a job as a
runner task even when its `if` skips it, and those skip-tasks used to wait behind
long builds (observed 31 min) stalling every PR run.
**Four** runners serve the fork (server-management#570/#574/#639):
**Lane assignment (ersatztv#390).** The `ubuntu-latest` lane has **4 slots** (2 + 2) and the
`small` lane has 4. A 2026-07-17 audit of the Actions API found the `ubuntu-latest` lane
| Runner | Host | Label | Slots | Per-job cap |
|---|---|---|---|---|
| `ci-runner` | VM 127 (pve4) — no prod workload | `ubuntu-latest` | 4 | `--cpus=4 --memory=10g` |
| `bumblebee-runner` | bumblebee — prod media | `ubuntu-latest` | 2 | `--cpus=4 --memory=10g --cpu-shares=256` |
| `small-runner` | bumblebee — prod media | **`small`** | 2 | `--cpus=1 --memory=1g --cpu-shares=256` |
| `jazz-small-runner` | jazz — prod media (#633) | **`small`** | 2 | `--cpus=1 --memory=1g --cpu-shares=128` |
The `small` lane exists because Gitea dispatches a job as a runner task **even when its
`if` skips it**, and those skip-tasks used to wait behind long builds (observed 31 min),
stalling every PR run. `--cpu-shares` below the default 1024 is what makes a runner on a
prod media host acceptable: under contention CI loses to the transcoders (ersatztv 1536 /
jellyfin), which are the reason those hosts exist.
**`small` is git-only, and that is load-bearing (server-management#639).** Everything in
the lane is a checkout plus a `git diff`: `decisions-guard`, `ci-image-pin`,
`docs-reminder`. Nothing there runs a compiler or a `docker build`, which is why the lane
can be capped at 1 GiB per job. Route a heavy job here and it will OOM — give it
`ubuntu-latest`, or its own label on `ci-runner`, the only host with no prod workload.
**Lane assignment (ersatztv#390).** *Slot counts below are as-of 2026-07-17; the table above is
current.* At the time, the `ubuntu-latest` lane had **4 slots** (2 + 2) and the
`small` lane 4. A 2026-07-17 audit of the Actions API found the `ubuntu-latest` lane
saturated and the `small` lane idle — **queue wait exceeded every job's runtime**:
| Job | Runtime | Queue wait | Lane |
@@ -143,6 +184,26 @@ gone, so it is no longer a reason to keep the lane large. `api-docs` on an API-t
Queue wait is still a dominant cost and capacity is server-management's boundary — tracked in
**server-management#604**. The redundant triple-build behind those runtimes is **ersatztv#398**.
**The other failure mode: setup-phase starvation (server-management#639, 2026-07-20).** The
table above measures *queue wait* — time before a job is dispatched. A saturated lane also
produces a second, much more confusing symptom: a job that *is* dispatched, sits `in_progress`
for >10 minutes, writes **no log file at all** (`OpenLogs … .log.zst: file does not exist`),
and then fails — wedged in act's job-**setup** phase, before Checkout. Same-config siblings
that started 90s earlier finished in seconds; a concurrent job's log showed a normally-fast
compile taking a 7-minute gap between projects. This is the origin of the "`decisions.md` is a
known flake, just rerun it" folklore: the rerun succeeds only because it lands after load
clears, so the guard's logic gets blamed for a capacity problem.
The fix was **not** more capacity for its own sake. `small` was stuck at one slot because it
still held two heavy jobs — `docker-build.yml`'s image build and `ci-image.yml`'s toolchain
buildx (the latter reads as lightweight because it is "docker-only", but it is the heaviest
thing that ran in the lane) — and their 10 GiB requirement set the lane's per-job cap, which
on a 25 GiB host permits exactly one slot. Moving both to `ubuntu-latest` made the lane
genuinely tiny, so it could widen to **4 slots across two hosts while committing less RAM to
CI than the single slot did**. `docker-build.yml`'s `build` does not re-create #574's
skip-task queueing, because `needs: [test, migrations]` means it cannot be dispatched until
the lane it would queue behind has already drained.
### CI build memory: no persistent compiler servers (ersatztv#406)
Roslyn's `VBCSCompiler` is a **persistent** compiler server — it outlives the `dotnet build` that
@@ -468,7 +529,7 @@ matrix. A false skip could ship an under-validated image, so every ambiguous cas
*fast-forward-equivalent* merge: main did not advance since the PR's last green run **and** the PR
head was not rebased at merge time. Two routine patterns defeat it in this repo: (1) under parallel
merges main usually advances; and (2) — the bigger one — the standard workflow **rebases a PR
before merging** to resolve the append-only `docs/decisions.md` conflict (see MEMORY: the
before merging** to resolve the `docs/decisions.md` lifecycle conflict (see MEMORY: the
"decisions.md conflict treadmill"), which mints a new head SHA whose tree was never itself
CI-validated, so the tree-match check correctly declines. So the skip is a genuine but *occasional*
win (clean, up-to-date, un-rebased merges in quiet periods) — correct-but-conservative by
@@ -487,20 +548,22 @@ thus no `actions/cache`), so it can't hit the cache-save hangs seen on the VM-12
(domain-model, spa-conventions) — those stay on the author. (The API contract is mechanized by the
blocking `api-docs` job, and `docs/decisions.md` by the blocking `decisions-guard` job below.)
### `decisions-guard` job (blocking, PR-only)
### `decisions-guard` job (`decisions lifecycle`, blocking, PR-only)
Enforces the `docs/decisions.md` append-only convention (ersatztv#303 H9): fails a PR whose
merge-base diff **deletes or modifies** any existing line of that file, unless a commit in the range
carries the `[decisions-edit]` token (for a factual fix or a documented supersession — see the
`decisions.md` header). Pure insertions (a normal new entry: TOC line + appended block) pass. The job
also emits a **non-blocking** consolidation nudge once the file exceeds **1800 lines** (the read-cost
floor — one default agent Read caps at 2000 lines), so append-only can't outgrow what agents read. It runs
the same `.claude/hooks/decisions-guard.sh` the Husky `commit-msg` hook uses, so the *detection logic*
is shared and can't drift. Granularity differs, deliberately: Husky checks **each commit** (`staged`
mode, that commit's own message must carry the token); CI checks the **PR-wide** net diff (`range` mode,
accepts the token in *any* commit of the range). The local hook is therefore the stricter, primary
gate; CI is the backstop for direct pushes or bypassed hooks. Like `docs-reminder`, it's a
seconds-long `git diff` with no dotnet/node setup (`runs-on: small`).
Enforces decision-record lifecycle invariants (ersatztv#521, supersedes the ersatztv#303 H9
append-only mechanic): well-formed 5-field metadata, exactly one `active` record per `key`,
reciprocal `supersedes`/`superseded-by` links, no record vanishing from the active set without an
archive copy, no rationale-prose rewrite without the `[decisions-edit]` token in the commit range,
and the generated active catalog (`docs/decisions/README.md`) in sync with source. Two steps:
`scripts/decisions_validate.py --base origin/<base> --head HEAD` (the merge-base diff checks, which
need a base/head range — CI-only) and `scripts/build_decisions_catalog.py --check` (catalog drift).
The **same validator** backs the Husky `pre-commit` hook (`.claude/hooks/decisions-guard.sh`, no
base/head there — structural checks only, over the working tree), so local and CI enforcement can't
drift on the rules that don't need a range. `python3` isn't guaranteed on the bare `small` lane, so
the job adds `actions/setup-python@v5` before invoking it; that install is lightweight (no
compiler/docker build), so it doesn't violate the "small is git-only" lane rule. Like
`docs-reminder`, otherwise a seconds-long `git diff` + parse with no dotnet/node setup
(`runs-on: small`).
## CI toolchain image (`docker/ci/Dockerfile`, `.gitea/workflows/ci-image.yml`)
@@ -533,7 +596,9 @@ ersatztv#299 seeded-media/scanner E2E follow-ups will need.
`ci-image.yml` triggers on pushes touching `docker/ci/**`, `workflow_dispatch`, and a weekly Monday
05:00 UTC cron (base-image security updates; Gitea registers `schedule` only from `main`). It runs on
the `small` lane and, like `docker-build.yml`, needs BuildKit's inline `http = true` for the HTTP
`ubuntu-latest` — it was on `small` until server-management#639, where "docker-only" was found to be
a poor proxy for "small": this is a full buildx of the .NET toolchain image, the heaviest job in that
lane. Like `docker-build.yml`, it needs BuildKit's inline `http = true` for the HTTP
registry. Renovate tracks the Dockerfile's image pins (`dockerfile` manager, see `renovate.json`).
**Three container-specific gotchas** — worth knowing if you add a job or a step:
@@ -866,20 +931,24 @@ entirely in `web/`), the wiring is:
files (~6-7s wall in practice, dominated by the workspace load); (c) **H3** (ersatztv#303) —
refuses a staged **root-level `*.png`** (`git diff --cached --name-only | grep -E '^[^/]+\.png$'`),
belt-and-suspenders with the `.gitignore` screenshot rule so a forced `git add -f` still can't land
a review/debug screenshot at the repo root. Nested `*.png` (real assets) pass.
a review/debug screenshot at the repo root. Nested `*.png` (real assets) pass; (d) **decision
lifecycle validator** (ersatztv#521, supersedes the ersatztv#303 H9 append-only mechanic) — runs
`.claude/hooks/decisions-guard.sh` (no args; a fail-open shim around
`scripts/decisions_validate.py`), the structural checks over the working tree (metadata
well-formedness, one active record per key, reciprocal links). It has no base/head here, so the
body-diff/no-vanish checks it also knows about are skipped locally and only run in the CI
`decisions lifecycle` job, which has a PR base to diff against.
2. **`pre-push`** — CI-parity gate: `cd web && npm run check:api && npm run lint && npm run
typecheck && npm run build`. `check:api` guards generated-OpenAPI drift
(`ErsatzTV/wwwroot/openapi/v1.json` → `web/src/api/generated/v1.d.ts`); the full
lint/typecheck/build catch a staged change that breaks an *unstaged* file (lint-staged
only sees staged files). Any failure blocks the push.
3. **`commit-msg`** — (a) enforces the CLAUDE.md protocol: the message must carry a
3. **`commit-msg`** — enforces the CLAUDE.md protocol: the message must carry a
`Co-Authored-By:` trailer, else the commit is rejected (merge commits are exempt, detected
via `git rev-parse --verify MERGE_HEAD`); (b) **H9** (ersatztv#303) — runs
`.claude/hooks/decisions-guard.sh staged "$1"`, which blocks the commit if it deletes/modifies
an existing line of `docs/decisions.md` unless the message carries the `[decisions-edit]` token.
Append-only enforcement; see `docs/decisions.md` header for the supersession/consolidation rules.
The same script backs the blocking **`decisions-guard`** CI job (`range` mode over the PR's
merge-base diff) so local and CI enforcement can't drift.
via `git rev-parse --verify MERGE_HEAD`). The decision-lifecycle check lives in `pre-commit`
(above), not here — `[decisions-edit]` is read from the commit message, but only by the CI
`decisions lifecycle` job's body-diff step (`range` mode over the PR's merge-base diff), which is
the only place a base/head range exists to diff against.
- **Worktree/subdir gotcha**: git exports `GIT_DIR` (and friends) while running hooks. In a
worktree or any subdir, an explicit `GIT_DIR` makes nested `git` commands mislocate the
+879 -37
View File
File diff suppressed because it is too large Load Diff
+120
View File
@@ -0,0 +1,120 @@
<!-- GENERATED by scripts/build_decisions_catalog.py — do not edit by hand. -->
# Active decisions — catalog / task router
The compact current view of settled decisions. Each row is an **active** record; follow
the link for rationale. Superseded/retired history lives in `archive/`. Regenerated by
`scripts/build_decisions_catalog.py`.
| Key | Current rule | Since | Record |
| --- | --- | --- | --- |
| `api.artwork-rooted-urls` | API response DTOs return artwork as rooted, directly-usable URLs (plus passthrough for absolute/proxy URLs), never relative Blazor-convention paths. | 2026-07-07 | [link](../decisions.md#2026-07-07--api-artwork-contract-rooted-urls-produced-server-side) |
| `api.async-op-contract` | Queue-triggering `/api/*` endpoints normalize onto one contract — 202 Accepted (queued), 404 (missing entity), 409 (lock held), 422 (domain precondition) — with Trakt as the reference implementation; playout list/detail GETs also carry an `isLocked` observability flag as the HTTP-observable substitute for a live push channel. | 2026-07-11 | [link](../decisions.md#2026-07-11--async-op-api-contract-normalization--playout-build-observability--f9-scan-endpoints-235) |
| `api.channel-health-signal` | Channel health rides `ChannelResponseModel`/`ChannelListItem` DTOs as a raw `int PlayoutCount` fact (free — `GetAll` already `Include`s `Playouts`), not a new endpoint, not `/channels/state` (runtime-liveness cadence), and not a derived `ChannelHealth` enum (would freeze policy before the #383/#384 auto-tune status taxonomy lands). | 2026-07-17 | [link](../decisions.md#2026-07-17--channel-health-on-the-api--the-raw-playoutcount-fact-on-the-list-dto-not-a-derived-status-enum-72) |
| `api.decode-by-id` | Endpoints that decode/expand opaque stored state accept a database row id and resolve it server-side rather than round-tripping client-supplied serialized state. | 2026-07-07 | [link](../decisions.md#2026-07-07--decode-style-endpoints-take-a-row-id-and-look-up-server-side) |
| `api.healthcheck-remediation-dto` | Health-check remediation is server-declared `{Kind, Target}` metadata on an additive DTO field; the SPA renders/acts on it, it doesn't derive labels itself. | 2026-07-17 | [link](../decisions.md#2026-07-17--health-check-remediation-is-server-declared-kind-target-on-an-additive-dto-the-spa-acts-on-it-164) |
| `api.healthcheck-ttl-cache` | Health-check results are held in a 30s TTL cache inside `HealthCheckService`; a non-forced `GET /api/v1/health` returns the cached list, and `?refresh=true` (or a forced internal caller) bypasses it to run fresh. | 2026-07-19 | [link](../decisions.md#2026-07-19--health-check-results-are-ttl-cached-refreshtrue-forces-a-fresh-run-431) |
| `api.logs-sort-params` | `GET /api/logs` takes allow-listed `sortField` (`timestamp`\|`level`) and `sortDirection` (`asc`\|`desc`) query params, normalized (not rejected) on an unrecognized value. | 2026-07-11 | [link](../decisions.md#2026-07-11--logs-column-sorting-allow-listed-sortfieldsortdirection-on-get-apilogs) |
| `api.mediatr-passthrough` | The REST API is thin controllers over existing MediatR handlers, with no new service/business-logic layer. | 2026-06 | [link](../decisions.md#2026-06--rest-api-wraps-existing-mediatr-handlers-11-no-service-layer) |
| `api.openapi-mirrors-runtime` | The generated OpenAPI spec is made to match the runtime Newtonsoft wire contract (via `NewtonsoftSchemaNamingTransformer`), not the reverse. | 2026-07-09 | [link](../decisions.md#2026-07-09--openapi-spec-mirrors-the-runtime-newtonsoft-serializer-198) |
| `api.parentid-drillin` | Media drill-in (season/episode/artist/music-video) is served by an optional `parentId` query param on library-browse, not dedicated per-kind child-listing endpoints. | 2026-07-07 | [link](../decisions.md#2026-07-07--seasonepisodemusic-video-drill-in-via-parentid-not-new-child-listing-endpoints) |
| `api.playout-build-lock-409` | Every id-keyed playout/channel mutation endpoint checks `IEntityLocker.IsPlayoutLocked(id)` and returns 409 Conflict while a build is in-flight, mirroring Blazor's disabled-buttons behavior; reset-all stays 202 and silently skips locked playouts. | 2026-07-10 | [link](../decisions.md#2026-07-10--playout-api-mutations-return-409-while-the-build-lock-is-held-215) |
| `api.postcommit-cancellation-none` | Once a mutation commits, its entire compensating side effect (enqueues, publishes, reindexes, cache refresh, and any post-commit lookup gating one of those) runs on `CancellationToken.None` so a late client disconnect can't half-abort an already-committed change. | 2026-07-11 | [link](../decisions.md#2026-07-11--post-commit-side-effects-run-on-cancellationtokennone-generalized-from-251-to-254) |
| `api.put-replace-index-order` | PUT-replace-the-whole-list endpoints derive each item's `Index` from its request-array position, never a client-supplied field; alternate-schedule/playout-template rows are evaluated in `Index` order with the least-conditional row placed last as the catch-all default. | 2026-07 | [link](../decisions.md#2026-07--put-replace-list-endpoints-derive-index-from-array-order-alternate-schedules-last-row--catch-all-default) |
| `api.response-dtos` | New REST response DTOs live in `ErsatzTV.Core/Api/<Domain>/*ResponseModel.cs` with a file-scoped `#nullable enable` pragma; controllers never expose Application VM types directly. | 2026-07 | [link](../decisions.md#2026-07--response-dtos-live-in-ersatztvcoreapi-file-scoped-nullable-enable) |
| `api.schedule-item-flat-dto` | Schedule-item GET/POST/PUT use a flat, non-polymorphic `ScheduleItemResponseModel` (every subtype field promoted to a nullable top-level member) instead of the polymorphic Application VM hierarchy, with mutation field names matching `ScheduleItemRequest` 1:1 for a lossless round-trip. | 2026-07-10 | [link](../decisions.md#2026-07-10--schedule-item-get-returns-a-flat-non-polymorphic-dto-scheduleitemresponsemodel) |
| `api.scheduling-hardening` | Create/Replace handlers guard against null/whitespace `name` (`IsNullOrWhiteSpace`, not just `Length`) to prevent NRE-500s, template-item overlap validation compares by index (not record value-equality) to catch exact-duplicate items, and unreachable 404 `ProducesResponseType` attributes on create-only actions are trimmed. | 2026-07-13 | [link](../decisions.md#2026-07-13--scheduling-api-hardening-null-name-500s-duplicate-template-items-unreachable-404-172) |
| `api.search-allitems-paging` | `GET /api/v1/search/all-items` is paginated (capped page size, `Totals` field) to bound DoS exposure; the SPA add-all flow pages to completeness instead of relying on an unbounded response. | 2026-07-18 | [link](../decisions.md#2026-07-18--search-all-items-is-paged-to-cap-dos-exposure-spa-add-all-pages-to-completeness-293) |
| `api.search-paging-cap` | Search stays capped at 100 items per media kind; an overflowing kind's "See all" reuses library-browse paging instead of adding new API surface. | 2026-07-11 | [link](../decisions.md#2026-07-11--trash-see-all-reuses-library-browse-paging-search-stays-capped-per-kind-213) |
| `api.versioning-v1` | The entire `/api` surface is versioned to `/api/v1` uniformly (no unversioned corner); legacy unversioned callers are rewritten in-pipeline (not redirected) with Deprecation/Link/Sunset headers, and post-freeze `/api/v1` is additive-only — a breaking change requires `/api/v2`. | 2026-07-13 | [link](../decisions.md#2026-07-13--api-versioning-the-whole-api-surface-is-mounted-at-apiv1-additive-only-after-freeze-286) |
| `blazor.rollback-tag` | The commit immediately preceding the Blazor-removal merge is tagged `blazor-final` (not a `v*` tag, so it doesn't trigger a prod release build) as the documented rollback/restore path. | 2026-07-11 | [link](../decisions.md#2026-07-11--pre-removal-blazor-rollback-tag-blazor-final-205) |
| `blazor.ui-removed` | The legacy Blazor Server UI (`Pages/`, `Shared/`, `ViewModels/`, `Validators/`, MudBlazor + 8 other packages, Blazor Startup wiring) is fully deleted now that the SPA has parity; the legacy `MapWhen` branch is kept only for controllers/docs/OpenAPI/`LegacyUiRedirects`, and the catch-all fallback 302s any unmatched non-api/artwork/docs/openapi path to `/app`. | 2026-07-11 | [link](../decisions.md#2026-07-11--blazor-server-ui-removed-91-phase-b) |
| `ci.build-once-rejected` | CI build-once (a shared compile artifact across jobs) was implemented, measured, and rejected for a 40-85% wall-clock regression; keep the #420 cross-run tree-identity skip instead. | 2026-07-18 | [link](../decisions.md#2026-07-18--ci-build-once-was-measured-and-rejected-keep-the-420-tree-skip) |
| `ci.docs-only-detect-shallow-safe` | The docs-only detect script must diff against `FETCH_HEAD` (always resolves after `git fetch`, even shallow) using a two-dot tree diff — not `origin/<base>` with three-dot — because a `fetch-depth: 1` shallow clone has no remote-tracking ref and no merge-base, which silently fails the original detect into `docs_only=false` (full matrix, no functional error). A CI-behavior change must be verified by measuring the effect (job durations), not just a green check. | 2026-07-17 | [link](../decisions.md#2026-07-17--docs-only-detect-must-be-shallow-checkout-safe-fetch_head--two-dot-not-originmain--three-dot-416-follow-up) |
| `ci.docs-only-skip-steps` | A docs-only change must still run every required job (`test`, `migrations`) so their commit-status contexts always report; each heavy job runs `scripts/ci-detect-docs-only.sh` first and gates its real STEPS on `if: steps.detect.outputs.docs_only != 'true'`, never `if:`-skips the whole job (an `if:`-skipped job reports `skipped`, not `success`, which branch protection may never unblock on). Detection biases toward running more on any doubt. | 2026-07-17 | [link](../decisions.md#2026-07-17--docs-only-ci-skip-gates-steps-in-always-running-required-jobs-never-if-skips-them-416) |
| `ci.format-gate-folder-mode` | The blocking `format` CI job (and matching pre-commit hook) runs `dotnet format whitespace . --folder --include <files>` instead of loading the full MSBuild/Roslyn solution, cutting the gate from ~480s to ~0.5s with unchanged whitespace/charset coverage. | 2026-07-19 | [link](../decisions.md#2026-07-19--the-format-gate-runs-dotnet-format-whitespace----folder-not-the-full-solution-format-469) |
| `ci.functional-e2e-harness` | The `functional-e2e` CI job boots the PR's own code from source via `dotnet run` (`scripts/e2e-local.sh`) and runs curl-only, deterministic assertions (`scripts/e2e-functional.sh`) as an advisory (non-blocking) job, not a `build` dependency or required check. | 2026-07-16 | [link](../decisions.md#2026-07-16--functional-e2e-ci-harness-advisory-curl-contract-job-over-an-app-booted-from-source-299) |
| `ci.peak-anon-measurement` | The `test` job's headline memory figure is a sampled high-water mark of cgroup `anon`, produced by `scripts/ci-peak-anon.sh`; `memory.peak` and the end-of-job `anon`/`file` split are kept only as a cache-inflated reference. | 2026-07-19 | [link](../decisions.md#2026-07-19--ci-test-job-reports-a-sampled-true-peak-anon-not-cache-inflated-memorypeak-412) |
| `ci.root-screenshot-guard` | The Husky `pre-commit` hook refuses a staged root-level `*.png` (belt-and-suspenders with the `.gitignore` rule); nested `*.png` real assets are unaffected. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--root-screenshot-guard-pre-commit-refuses-root-level-png-303-h3) |
| `ci.runner-placement` | No persistent Roslyn compiler server survives a CI build (`UseSharedCompilation=false` etc., runner env + Dockerfile `ENV`); every `services:` container gets its own explicit `--memory`/`--memory-swap`/`--cpus` cap (it does not inherit the job container's). | 2026-07-17 | [link](../decisions.md#2026-07-17--no-persistent-compiler-servers-in-ci-every-services-container-gets-an-explicit-cap-390s-small-lane-move-reversed-406) |
| `ci.small-lane-git-only` | `runs-on: small` is defined by what a job does (git-only), not its usual runtime; the two `docker build` jobs (docker-build.yml, ci-image.yml) move to `ubuntu-latest` because their worst-case memory, not median runtime, was pinning the small lane's per-slot cap. | 2026-07-20 | [link](../decisions.md#2026-07-20--runs-on-small-means-git-only-the-two-docker-build-jobs-move-to-ubuntu-latest-server-management639) |
| `concurrency.diff-scalar-fanout` | The frozen Block optimistic-concurrency recipe (api-conventions §7a) fans out to Collection/Playout×2/MultiCollection/RerunCollection, keeping a guard-returned `PreconditionFailedError` out of any handler's generic `catch(Exception)`→422 mapping, and preserving each aggregate's existing `SaveChangesAsync() > 0` gate semantics under the new unconditional `Version++`. | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--253-pr3-diff--scalar-concurrency-fan-out-collection--playout2--multicollection--reruncollection) |
| `concurrency.etag-rotation-completion` | Every handler that mutates a versioned root's editor-visible config state must bump `Version` (rotating the ETag) with no per-aggregate carve-outs, short-circuiting on a genuine no-op before the bump so idempotent re-submits don't fire spurious rebuild fan-out; `SaveChangesForcingVersion` rebases the retry (stored + pending delta), never adopts the stored token verbatim. | 2026-07-12 | [link](optimistic-concurrency.md#2026-07-12--cross-editor-etag-rotation-completed-for-collectionplayout-config-siblings-269) |
| `concurrency.force-write-non-ifmatch` | Any handler that leaves a versioned root `Modified` or `Deleted` but takes no `If-Match` (deletes, item add/remove bumpers, scalar-config writers) must save through `ConcurrencyExtensions.SaveChangesForcingVersion` — force-write past a concurrent `Version` bump rather than throw an unhandled `DbUpdateConcurrencyException` (500). | 2026-07-12 | [link](optimistic-concurrency.md#2026-07-12-269--non-if-match-root-writers-force-write-past-a-concurrent-version-bump) |
| `concurrency.idempotent-concurrent-add` | A concurrent duplicate `Add*ToCollection` that loses the race on the composite-key unique constraint is treated as an idempotent no-op (skip the reindex/rebuild fan-out), not a 500 — detected via a provider-specific `TvContext.IsUniqueConstraintViolation` delegate defaulting to "no". | 2026-07-18 | [link](optimistic-concurrency.md#2026-07-18--concurrent-same-item-add-is-idempotent-not-a-500-catch-the-unique-violation-per-provider-308) |
| `concurrency.ifmatch-rfc7232` | `ConcurrencyHeaders.ParseIfMatch` is a real RFC 7232 entity-tag/list parser: a syntactically-valid tag that doesn't strong-match (weak/empty/non-canonical/out-of-range/list) returns 412, and only a genuine grammar violation returns 400. | 2026-07-12 | [link](optimistic-concurrency.md#2026-07-12--if-match-evaluates-per-rfc-7232-valid-but-non-matching--412-only-grammar-violations--400-265) |
| `concurrency.replace-all-contract` | Replace-all aggregate PUTs carry a uniform plain `int Version` concurrency token (EF `.IsConcurrencyToken()`), checked pre-save and enforced by the EF UPDATE guard, returning 412 (not 409) on a stale `If-Match`. | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--optimistic-concurrency-contract-for-replace-all-puts-253-pr1-infra--block-reference) |
| `concurrency.schedule-item-child-identity` | `PUT /api/schedules/{id}/items` reconciles by an optional round-tripped child `Id` (null/absent/0 ⇒ new item), never by array position, so fill-group/shuffle state follows the logical item across reorders; an unknown or duplicate id is rejected 422 (checked after the §7a `CheckVersion`, so 412 precedes 422). | 2026-07-11 | [link](optimistic-concurrency.md#2026-07-11--stable-child-identity-for-schedule-item-replace-259-split-from-252253) |
| `docs.convention-docs-session-start` | Docs-first, not source-first: conventions (api-conventions, spa-conventions, e2e-local, blazor-route-parity, domain-model, decisions, README) are read from docs, not reverse-engineered from code, via `docs/README.md`'s task-signal map — only the sections it points to for the task at hand, not the whole set. Each doc is updated in the same PR that changes what it documents, replacing deferred/follow-up doc updates. | 2026-07-07 | [link](../decisions.md#2026-07-07--convention-docs-read-at-session-start-updated-in-pr) |
| `docs.decision-lifecycle` | every decision `##` record (active or archived) carries a 5-field metadata block (`key`, `status`, `since`, `supersedes`, `superseded-by`) checked by `scripts/decisions_validate.py`; a record is never deleted or line-edited to reverse a call — it is moved to `docs/decisions/archive/` with `status: superseded`/`retired` and a reciprocal `superseded-by`/`supersedes` key pair to its replacement. | 2026-07-21 | [link](../decisions.md#2026-07-21--decision-records-carry-a-lifecycle-schema-validated-by-a-script-append-only-by-diff-is-retired-521) |
| `ffmpeg.external-logo-graphics-engine` | External-URL channel logos pass through to the graphics engine like any other watermark source; `WatermarkSelector` must never gate them on `File.Exists` (always false for a URL) and never route them through the ffmpeg-native overlay shortcut. | 2026-07-20 | [link](../decisions.md#2026-07-20--external-url-channel-logos-pass-through-to-the-graphics-engine-never-fileexists-gated-never-ffmpeg-native-502) |
| `ffmpeg.hls-cold-start-burst` | HLS cold-start latency is fixed with a bounded `-readrate_initial_burst` (gated on FFmpeg ≥6.1 capability detection), not by raising `work_ahead_limit`, which would remove the concurrency guarantee it exists for. | 2026-07-20 | [link](../decisions.md#2026-07-20--hls-cold-start-is-fixed-with--readrate_initial_burst-not-by-raising-the-work-ahead-limit-350) |
| `ffmpeg.qsv-decode-encode-split` | QSV decode is decoupled from QSV encode via a single `FFmpegProfile.QsvPreferNativeDecoder` bool (default ON, Linux-only), so a QSV encode profile can decode with the more tolerant native VA-API decoder instead of the QSV decoder, mirroring Jellyfin's hybrid decode/encode toggle instead of a general decode-family enum. | 2026-07-20 | [link](../decisions.md#2026-07-20-498--qsv-decode-is-split-from-qsv-encode-via-a-single-qsvprefernativedecoder-bool) |
| `ffmpeg.qsv-extra-hw-frames-floor` | a QSV upload never emits `extra_hw_frames` below `FFmpegState.MinimumQsvExtraHardwareFrames` (64); a stored `0` or negative value is treated as "no pool configured" rather than honored literally, because with no headroom any unthrottled read exhausts the pool and the transcode writes nothing at all. | 2026-07-21 | [link](../decisions.md#2026-07-21--qsv-hardware-frame-headroom-is-a-floor-not-an-operator-preference-529) |
| `ffmpeg.remote-image-fetcher-bounded` | remote graphics-engine images are fetched through `IRemoteImageFetcher` with a pooled `HttpClientFactory` client, a body-covering deadline, a wire-transfer size cap, and a decoder-enforced `DecoderOptions.MaxFrames` bound re-verified post-decode — never cached, re-fetched per element init. | 2026-07-20 | [link](../decisions.md#2026-07-20--remote-graphics-engine-images-are-fetched-through-a-bounded-pooled-iremoteimagefetcher-re-fetched-per-element-init-not-cached-511) |
| `graphics.channel-logo-caching` | An external `http(s)` channel-logo URL is fetched, decode-budget-validated, and stored in the image cache under a content-hash name at SAVE time — becoming byte-identical to an uploaded logo — so the render path never fetches a logo over HTTP; a bad URL fails the save with a 422 (BaseError → ValidationProblemDetails). | 2026-07-21 | [link](../decisions.md#2026-07-21--external-channel-logo-urls-are-downloaded-and-cached-at-save-time-the-render-path-never-fetches-a-logo-525) |
| `iptv.base-url` | An optional advertised base URL (`iptv.base_url`) is resolved centrally via a pure Core helper (`AdvertisedBaseUrl`) inside the two IPTV generation handlers (M3U + XMLTV); unset/malformed values fall back byte-identical to the request-derived host, and it's a new `iptv` settings group distinct from `ETV_BASE_URL` and out of scope for HDHomeRun. | 2026-07-16 | [link](../decisions.md#2026-07-16--optional-advertised-iptv-base-url-iptvbase_url-resolved-centrally-in-the-two-generators-340) |
| `iptv.logo-drives-bug-preset` | One uploaded channel logo drives both the listing logo and the on-screen bug via a shared, seeded `ChannelLogo`-sourced watermark preset (`Channel Bug`), not new per-channel schema. | 2026-07-20 | [link](../decisions.md#2026-07-20--one-logo-drives-the-bug-via-a-shared-channellogo-preset-not-new-schema-67) |
| `locking.entitylocker-atomic-flags` | `EntityLocker` uses `Interlocked.CompareExchange`-guarded atomic flags plus a documented single-owner-release discipline (no owner tokens/leases); `Unlock*` on an already-unlocked slot returns `false` and logs a Warning rather than throwing. | 2026-07-11 | [link](../decisions.md#2026-07-11--entitylocker-atomic-flags--single-owner-release-discipline-no-owner-tokens-231) |
| `mcp.server-foundation` | `ErsatzTV.Mcp` is a fresh stdio JSON-RPC server wrapping frozen `/api/v1` with explicit narrow per-endpoint tools, read-only-by-default enforced at runtime (`ERSATZTV_ALLOW_WRITES`), machine-key auth, and opt-in `If-Match`. | 2026-07-20 | [link](../decisions.md#2026-07-20--mcp-server-ersatztvmcp-built-fresh-over-frozen-apiv1-read--cautious-writes-58) |
| `media.lastscan-null-boundary` | A never-scanned `LastScan` surfaces as `null` at the API/MCP boundary, not the `0001-01-01` MinValue sentinel — enforced by an ongoing read-boundary coercion plus a one-time data migration cleanup. | 2026-07-18 | [link](../decisions.md#2026-07-18--never-scanned-lastscan-surfaces-as-null-at-the-api-boundary-not-the-0001-01-01-minvalue-sentinel-409) |
| `media.remote-stream-probe` | `ValidatePlayoutItemPath` probes the Plex/Jellyfin/Emby remote-stream URL via `IRemoteStreamProber` before returning it; only a redirected 404 fails closed (`PlayoutItemNotAvailableFromMediaServer`), everything else fails open, and there is no toggle. | 2026-07-19 | [link](../decisions.md#2026-07-19--media-server-remote-stream-urls-are-probed-before-use-a-redirected-404-fails-closed-everything-else-fails-open-no-toggle-473) |
| `media.remote-stream-probe-externaljson` | External-JSON playout channels' `StreamRemotely` now probes the remote-stream URL through the same `IRemoteStreamProber` seam as the generated-playout path, closing the #473 scope gap for a channel kind with no DB `PlayoutItem` rows. | 2026-07-20 | [link](../decisions.md#2026-07-20--external-json-playout-channels-now-probe-the-remote-stream-url-too-closing-the-473-scope-gap-480) |
| `media.source-mgmt-write-api` | Media-source management (local/Plex/Jellyfin/Emby) is a REST write API + SPA under `/app/libraries/*`, wrapping existing MediatR commands 1:1 with no new commands or DB migration; connection GETs never leak a stored `apiKey`, and each PUT-replace family's identity contract is documented per-family (not assumed uniform). | 2026-07-11 | [link](../decisions.md#2026-07-11--media-source-management-rest-write-api--spa-202) |
| `release.api-contract-ci-gate` | A PR touching `ErsatzTV/Controllers/Api/**` or `ErsatzTV.Core/Api/**` must ship regenerated OpenAPI artifacts (`v1.json`, `v1.d.ts`, `endpoint-index.md`) in the same diff, enforced by a blocking `api-docs` CI job that regenerates-and-diffs against a fresh build. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--blocking-ci-gate-for-api-contract-artifacts-303-h4h5) |
| `release.done-when-merge-consent` | A PR may merge only when its linked issue's `## Done-when` checklist is fully ticked and the PR's CI is green, enforced by a PreToolUse hook on the Gitea merge tool (deny/allow/ask) plus a pre-push backstop for direct pushes to main. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--merge-consent-derived-from-state-via-a--done-when-issue-checklist-303-h6) |
| `release.format-as-you-touch-rebase` | A blocking `format` CI job runs `dotnet format --verify-no-changes` scoped only to the PR's changed `.cs` files (never the legacy BOM backlog), and a PR branch must be kept current by rebasing on `origin/main` (never merging main in), enforced by `.husky/pre-push``prepush-rebase-check.sh`. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--formatting-as-you-touch-enforced-rebase-not-merge-for-pr-branches-311-h11--format-ci) |
| `release.live-e2e-required` | A PR that changes an API write-path handler must include a live-E2E pass (driving the real endpoint/screen and confirming the round-trip through a subsequent read), not only unit/characterization tests, and must state whether live-E2E ran or wasn't required. | 2026-07-12 | [link](../decisions.md#2026-07-12--live-e2e-is-a-required-step-for-api-write-path-handler-changes-303) |
| `release.merge-consent-autogrant` | When Done-when boxes are ticked, CI is green, and a fresh positive Review-verdict references head, the merge-consent hook emits `permissionDecision: allow` to actually suppress the redundant mechanical prompt — the derived state IS the consent, no separate conversational confirmation on that path. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--merge-consent-gate-auto-grants-when-satisfied-no-redundant-prompt-state-is-the-consent-314) |
| `release.migration-rehearsal-prodcopy` | Before promoting a migration-bearing release, rehearse the new image's migrations against a throwaway copy of the latest prod backup (`scripts/migration-smoke.sh`), gating PASS on the migrator's completion log line rather than HTTP readiness alone. | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--release-path-rehearses-migrations-on-a-prod-db-copy-before-promoting-315) |
| `release.prepush-clean-worktree-guard` | A fail-open pre-push hook blocks a push when any file in the branch's diff vs `origin/main` also has uncommitted working-tree or index changes, since a stale-index commit (e.g. `git reset --soft` + `git add` over an edited-but-unstaged fix) can silently push, CI-test, and get reviewed a different tree than the one on disk. Scope is precise to pushed-diff files; escape hatch `ETV_ALLOW_DIRTY_PUSH=1`. | 2026-07-17 | [link](../decisions.md#2026-07-17--pre-push-guard-dont-push-a-file-whose-working-tree-copy-is-uncommitted-h13-416-session) |
| `release.promotion-floating-prod` | Prod tracks the floating `:prod` image reference; a tag build's immutable `:<version>` image is scanned first, then promotion happens via a separate manual `DeployStack`, with daily auto-update only as a fallback — tag with enough runway before 03:00 to avoid an unscanned promotion. | 2026-07-13 | [link](release-ci-governance.md#2026-07-13--release-promotion-floating-prod-exact-image-scan-before-manual-deploy-335) |
| `release.review-verdict-gate` | A PR may not merge until a `Review-verdict: <MERGEABLE\|APPROVED\|BLOCKED\|NOT-MERGEABLE> @ <head-sha>` comment references the PR's current head sha (short-sha prefix match, line-start marker only, negative wins over positive on the same head); folds into the H6 merge-consent hook as condition (c). | 2026-07-12 | [link](release-ci-governance.md#2026-07-12--review-verdict-merge-gate-latest-commit-must-be-reviewed-303-h10) |
| `scan.collections-scan-status` | `GET /api/v1/media-sources/collections-scan-status` reports a family-global (not per-source), boolean-only active-scan set read from `IEntityLocker`; the SPA reconciles authoritatively against it (with a grace-tick helper) instead of a fixed client-side timeout. | 2026-07-12 | [link](../decisions.md#2026-07-12--external-collections-scans-get-an-authoritative-status-surface-271-the-spa-timeout-is-retired) |
| `scan.getoraddfolder-db-lookup` | `ILibraryRepository.GetOrAddFolder` resolves the existing folder via a DB query on `(LibraryPathId, Path)`, not the caller's `LibraryPath.LibraryFolders` in-memory navigation, since that navigation is only eager-loaded on the local scan path and is null on remote (Jellyfin) callers. | 2026-07-20 | [link](../decisions.md#2026-07-20--ilibraryrepositorygetoraddfolder-resolves-the-folder-from-the-db-not-the-callers-librarypathlibraryfolders-navigation-488) |
| `scan.jellyfin-mixed-content-library` | A Jellyfin library whose collection type is `mixed` (or absent) maps to one ErsatzTV library of `LibraryMediaKind.Mixed`, scanned by running the movie/television/music-video scanners in sequence against that single library — a library is a place, not a media kind. | 2026-07-20 | [link](../decisions.md#2026-07-20-489--jellyfin-mixed-content-libraries-map-to-one-library-holding-many-kinds) |
| `scan.musicvideo-reconciliation` | `JellyfinMusicVideoLibraryScanner` reconciles removed music videos by a library-scoped local-path diff plus hard delete (`TrashMissingMusicVideos`), not the server-itemId soft-trash pattern the other media-server scanners use, because music videos carry no server identity. | 2026-07-20 | [link](../decisions.md#2026-07-20--jellyfinmusicvideolibraryscanner-reconciles-by-library-scoped-path-diff--hard-delete-not-server-itemid-soft-trash-494) |
| `scan.zero-item-fetch-guard` | A media-server library sweep refuses to flag missing items when a successful fetch returns zero incoming items against a non-empty existing set (`MediaServerReconciliationGuard.ShouldFlagMissing`), rather than treating an ambiguous empty result as a full-library deletion. | 2026-07-19 | [link](../decisions.md#2026-07-19--a-media-server-library-sweep-refuses-to-flag-when-a-successful-fetch-returns-zero-items-rather-than-nuking-the-whole-library-477) |
| `sched.auto-tune-foundation` | Auto-tune preview enumeration uses EF distinct+count queries for exact counts, while each created channel is persisted as a live SmartCollection; coexistence with existing channels/numbers is additive-only, never mutating. | 2026-07-16 | [link](../decisions.md#2026-07-16--auto-tuning-enumerates-via-ef-persists-via-smartcollection-additive-coexistence-69) |
| `sched.autotune-detailpanel-members` | The Auto-Tune DetailPanel's per-channel content-source list is a live `ISearchIndex.Search` roll-up through the server-owned `AutoTuneAxisMap.GenerateQuery`, not an EF distinct+count query, so the preview matches exactly what the built channel's SmartCollection will contain. | 2026-07-17 | [link](../decisions.md#2026-07-17--auto-tune-detailpanel-member-list--live-search-index-roll-up-not-ef-enumeration-384) |
| `sched.autotune-per-channel-overrides` | Auto-Tune per-channel overrides reuse the Channel Builder's advanced-options DTO verbatim; per-source weights and bug-colour logo are deferred to #425. | 2026-07-17 | [link](../decisions.md#2026-07-17--auto-tune-per-channel-overrides-reuse-the-channel-builder-advanced-options-dto-weights--bug-colour-logo-split-out-to-425-385) |
| `sched.autotune-per-source-weights` | Auto-Tune per-source rotation weights and query corrections are supplied at bulk-create time via #70's MultiCollection/SmartCollection machinery, not a post-hoc PUT. | 2026-07-18 | [link](../decisions.md#2026-07-18--auto-tune-per-source-weights-ride-70s-multicollection-machinery-created-at-tune-time-not-a-post-hoc-put-425) |
| `sched.clock-padding-existing` | Clock-boundary padding already exists via `FillerPreset`'s `FillerMode.Pad` (Classic) and `pad_to_next`/`pad_until` (Sequential/YAML); #77 is closed as verified+documented, not built new, with a one-click per-channel toggle deferred behind the #388 design-system epic. | 2026-07-17 | [link](../decisions.md#2026-07-17--clock-boundary-schedule-padding-already-exists-fillermodepad-77-verified-convenience-toggle-deferred) |
| `sched.playbackorder-support-matrix` | Every build-time dispatch site logs a loud (non-fatal) warning on an unsupported `PlaybackOrder`, and a declared `PlaybackOrderSupport` matrix + partition tripwire test makes adding a new order safe by construction. | 2026-07-18 | [link](../decisions.md#2026-07-18--unsupported-playbackorder-is-loud-at-build-time-a-declared-support-matrix-and-tripwire-test-make-new-orders-safe-by-construction-403) |
| `sched.reshuffle-scoped-reset` | `POST /api/v1/playouts/{id}/reshuffle` runs `ErasePlayoutHistory` (reseeds `Playout.Seed` + clears anchors/rerun-history) then enqueues a scoped `Reset` build, so reshuffle always reseeds — even for the non-Classic kinds `Reset` alone wouldn't reseed; `Playout.Seed` is surfaced on list/detail DTOs as visible confirmation. | 2026-07-16 | [link](../decisions.md#2026-07-16--per-playout-reshuffle--scoped-reset-build-seed-surfaced-71) |
| `sched.seasonal-scheduling-existing` | Seasonal/date-conditional scheduling already ships first-class via `IAlternateScheduleItem` (Classic `ProgramScheduleAlternate`, Block `PlayoutTemplate`) evaluated by `AlternateScheduleSelector.GetScheduleForDate` (first match in `Index` order, catch-all last); #73 is closed as already-implemented with a docs-only "seasonal/holiday" recipe added, not new code. | 2026-07-17 | [link](../decisions.md#2026-07-17--seasonal--date-conditional-scheduling-already-exists-alternate-schedules--playout-templates-73-closed-as-implemented) |
| `sched.shuffle-source-builder` | Shuffle-source construction moves to a static, DI-free `ShuffleSourceBuilder` (a shared seam, not a service) so Classic and Playlist stop cross-engine reaching into `PlayoutBuilder` statics; a unified Classic+Playlist enumerator factory is explicitly rejected as a god-factory. Block/Scripted/YAML duplication is left alone, deferred to a follow-up gated on #381. | 2026-07-17 | [link](../decisions.md#2026-07-17--shuffle-source-construction-extracted-to-shufflesourcebuilder-per-family-seam-not-a-god-factory-380) |
| `sched.weighted-shuffle` | Fair-share/weighted airtime distribution ships as one new `PlaybackOrder.WeightedShuffle = 9` order (equal weights = fair-share), not a retrofit of `ShuffleInOrder` (which only anti-clumps, since its padding spacers emit nothing) and not a separate orthogonal "distribution" setting; weights live on `MultiCollectionItem`/`MultiCollectionSmartItem` (DB default 1, dual-provider migration), bounded at write (1..1000) and clamped again in the enumerator, and the write path rejects `WeightedShuffle` at every dispatch site that doesn't handle it rather than let it silently degrade to unweighted random. | 2026-07-17 | [link](../decisions.md#2026-07-17--weighted--fair-share-distribution-is-a-new-weightedshuffle-order-shuffleinorder-is-anti-clumping-not-fair-share-70) |
| `sched.weightedshuffle-editor` | WeightedShuffle per-source weights are edited on the multi-collection editor (property of the MultiCollection), while the WeightedShuffle order itself is offered only on classic MultiCollection schedule items; fair-share is a "reset weights to 1" action, not a stored mode. | 2026-07-19 | [link](../decisions.md#2026-07-19--weightedshuffle-spa-weights-edited-on-the-multi-collection-order-offered-only-on-classic-multicollection-schedule-items-fair-share-is-a-reset-not-a-mode-404) |
| `security.artwork-content-type-sniff` | Artwork content type is always derived from the stored bytes (never the client-declared value or a `?contentType=` query param) at both upload and serve, clamped to an image allow-list, closing the unauthenticated stored-XSS chain; Kestrel `MaxRequestBodySize` bounds upload DoS. | 2026-07-12 | [link](api-auth-security.md#2026-07-12--artwork-content-type-is-sniffed-never-reflected-283-s4s9-stored-xss) |
| `security.baseline-response-headers` | `SecurityHeadersMiddleware`, registered first in the pipeline, sets `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, and `Referrer-Policy: strict-origin-when-cross-origin` on every response (CSP/HSTS deliberately deferred); API-key comparison is constant-time and playout pagination is clamped. | 2026-07-11 | [link](api-auth-security.md#2026-07-11--baseline-security-response-headers--phase-0-api-hardening-197-pr-279) |
| `security.blazor-removal-auth-posture` | Removing the Blazor UI's OIDC-challenged surface exposes nothing a user couldn't already reach via the already-open `/app` SPA (open since phase (a)); real SPA/API authentication is deliberately deferred to #197, and the removal PR must preserve `ConditionalIptvAuthorizeFilter`, `ApiKeyAuthorizationFilter`, and `JwtHelper` access_token support. | 2026-07-11 | [link](api-auth-security.md#2026-07-11--blazor-removal-auth-posture-no-new-exposure-beyond-phase-a-real-auth-deferred-to-197-206) |
| `security.contract-freeze-honesty` | The OpenAPI doc's declared security/401 scheme is generated from the same `ApiKeyAuthorizationFilter.EndpointRequiresKey` predicate the runtime enforces (so declared auth can't drift from enforced auth), every `/api/*` action returns a ResponseModel (no raw Application VMs), and Channel REST resources are keyed by immutable `Id`, never mutable `Number`. | 2026-07-12 | [link](api-auth-security.md#2026-07-12-197-bundle-c--contract-freeze-honesty) |
| `security.corp-same-origin` | `SecurityHeadersMiddleware` sends `Cross-Origin-Resource-Policy: same-origin` on every response including `/docs`/`/openapi`, blocking cross-origin `no-cors` embedding without affecting allowed CORS-mode fetches or server-side Jellyfin `/iptv/*` requests. | 2026-07-13 | [link](api-auth-security.md#2026-07-13--cross-origin-resource-policy-same-origin-on-every-response-330) |
| `security.csp-permissions-policy` | `SecurityHeadersMiddleware` sends an enforcing (not report-only) `Content-Security-Policy` (no `unsafe-inline`/`unsafe-eval`; the one inline theme-bootstrap script allow-listed by hash) and a deny-all `Permissions-Policy` on the SPA/`/api`/`/artwork`/`/iptv`; `/docs` and `/openapi` keep only the baseline headers, excluded from CSP because Scalar needs inline bootstrap. | 2026-07-12 | [link](api-auth-security.md#2026-07-12--enforcing-csp--permissions-policy-on-the-host-319-zap-baseline) |
| `security.fail-closed-api-auth` | Every mutating `/api` request requires `X-Api-Key` (no open mode); reads are gated by `Api:RequireKeyForReads` (default true) OR `[RequiresApiKey]` on sensitive controllers; CORS is an exact-origin allowlist (`ApiCors`); `ForwardedHeaders` trust stays configurable but defaults to trust-all-with-warning. | 2026-07-12 | [link](api-auth-security.md#2026-07-12--fail-closed-api-auth--sensitive-read-tier--corsforwardedheaders-lockdown-197-bundle-a-pr-292) |
| `security.session-auth-dual-credential` | `ApiAuthorizationFilter` accepts a request when a valid `X-Api-Key` matches OR the principal is an authenticated session (cookie `ctv-session`, `HttpOnly`/`SameSite=Lax`); session-authenticated mutations require the presence-only `X-CSRF` header or are rejected 403. This narrows the OIDC-inert sub-claim of `security.blazor-removal-auth-posture` (#206) — the rest of that record's auth-surface enumeration still holds. | 2026-07-12 | [link](api-auth-security.md#2026-07-12--browser-spa-session-auth-api-accepts-session-or-machine-key-295-pr1-server-only) |
| `security.session-cutover-postify` | The browser SPA authenticates cookie-only (no more `X-Api-Key` from `web/`); the machine key is repurposed to external/MCP-only via `GET /api/auth/machine-key`; every side-effecting GET/HEAD under `/api` is converted to POST so the existing CSRF gate covers it (standing rule: never add a side-effecting GET/HEAD under `/api`). | 2026-07-12 | [link](api-auth-security.md#2026-07-12--295-pr2-spa-session-cutover--301-side-effecting-get-post-ification) |
| `spa.add-to-layer` | All add-to-collection/playlist/schedule affordances share one component layer at `web/src/media/addTo/`; multi-select is an explicit screen-level toggle, and the per-card menu offers schedule only for the server-validated kinds. | 2026-07-10 | [link](../decisions.md#2026-07-10--shared-add-to-layer-lives-in-websrcmediaaddto-select-mode-is-an-explicit-toggle) |
| `spa.app-shell-extraction` | `App.tsx` is only the composition root over `web/src/app/routes.tsx` (stable route-object identity), `app/AppShell.tsx` (shell chrome), and `app/ScreenContent.tsx` (exhaustive screen dispatch); primary actions are one explicit `PrimaryActionProvider` registration per screen, replacing the old global `ctv:primary-action` window event. | 2026-07-15 | [link](spa-modularization.md#2026-07-15--app-shellrouting-extraction--explicit-primary-action-ownership-247) |
| `spa.autotune-detailpanel-slideover` | The Auto-Tune DetailPanel SPA is a reusable `SlideOver` primitive sharing `useOverlayBehavior` with `Dialog`, plus a shared advanced-options model extracted from ChannelBuilder; decorative panes without backend support are dropped. | 2026-07-18 | [link](../decisions.md#2026-07-18--auto-tune-detailpanel-spa-reusable-slideover--shared-advanced-options-model-decorative-panes-dropped-to-match-the-backend-386) |
| `spa.channel-editor-create-logo` | Bare-channel create is a "New blank channel" action on the channels list (reusing Blazor's add-mode defaults) that navigates into the full editor, and an external logo URL always wins over an uploaded logo, matching `ChannelEditViewModel` precedence. | 2026-07-11 | [link](../decisions.md#2026-07-11--channel-editor-bare-create-entry-point--external-logo-mutual-exclusion-212) |
| `spa.channel-renumber-prompt` | Channel renumbering uses a sequential `prompt()`-driven "Renumber" action instead of drag-to-reorder. | 2026-07-09 | [link](../decisions.md#2026-07-09--channel-numbers-prompt-driven-sequential-renumber-instead-of-drag-to-reorder) |
| `spa.channels-screen-extraction` | The Channels domain is a single-file zero-prop screen (`web/src/screens/ChannelsScreen.tsx`) with a colocated test file and no sibling helper directory, since its pure logic is too small (~30 lines) to justify a separate business-rule layer like Schedules' `itemRules.ts`. | 2026-07-11 | [link](spa-modularization.md#2026-07-11--channels-screen-extraction-244-single-file-screen-no-sibling-helper-dir-epic-243-phase-1) |
| `spa.collection-custom-order-ui` | Collection custom ordering uses per-row Move up/Move down buttons (not drag) and is offered for any manual collection with custom order enabled, not just movies-only. | 2026-07-09 | [link](../decisions.md#2026-07-09--collection-custom-order-move-updown-buttons-any-kind-collections) |
| `spa.datetime-local-input` | The channel-mode date/time input uses a native `<input type="datetime-local">` instead of free-text Chronic natural-language parsing. | 2026-07-09 | [link](../decisions.md#2026-07-09--datetime-local-instead-of-chronic-natural-language-start-parsing) |
| `spa.deco-templates-table` | The deco-templates editor also renders its day/deco assignment as a table, extending (not replacing) the templates-editor-table convention. | 2026-07-09 | [link](../decisions.md#2026-07-09--table-not-calendar-convention-also-covers-the-deco-templates-editor) |
| `spa.download-sample-gate` | The SPA disables both Download Media Sample and Download Results while a troubleshooting session is starting/running (Blazor only gated Download Results). | 2026-07-09 | [link](../decisions.md#2026-07-09--spa-gates-download-media-sample-while-a-session-is-active) |
| `spa.legacy-redirect-matcher` | `LegacyUiRedirects.TryGetRedirect` is a two-tier matcher — an exact `OrdinalIgnoreCase` `Map` (Tier 1) then an ordered segment-template pattern list (Tier 2, first-match-wins) — collision-free by construction, with a guard invariant that no rule may prefix-match `/api`, `/artwork`, `/docs`, `/openapi`, `/iptv`, `/app`, or `/media/sources`. | 2026-07-11 | [link](../decisions.md#2026-07-11--legacyspa-redirect-matcher-exact-map--ordered-segment-template-patterns-204) |
| `spa.logs-page-size-local` | The Logs page rows-per-page preference is stored in `window.localStorage` (`ctv-logs-page-size`), not a server `ConfigElement`. | 2026-07-11 | [link](../decisions.md#2026-07-11--logs-page-size-is-a-client-local-preference-not-a-server-configelement) |
| `spa.playback-troubleshoot-poll` | The playback-troubleshooting screen reports FFmpeg completion by polling `GET /api/troubleshoot/playback/status` (~2s) rather than a server push channel. | 2026-07-09 | [link](../decisions.md#2026-07-09--playback-troubleshooting-completion-feedback-poll-status-no-push-channel) |
| `spa.playout-reset-button` | The SPA keeps a single Reset action (server picks the default build mode) and drops Blazor's separate "Schedule reset" button since its capability already exists via the playout's Edit-details flow. | 2026-07-09 | [link](../decisions.md#2026-07-09--per-playout-schedule-reset-button-dropped-reset-uses-the-server-default-build-mode) |
| `spa.playouts-screen-extraction` | The Playouts domain (including its unguarded `PlayoutsRouteScreen` route wrapper with local pathname/popstate state) moved as one unit into `web/src/screens/PlayoutsScreen.tsx`, keeping its screen-specific sub-path route ownership colocated with the base screen; a pure structural move with no API/route/CSS/behavior change. | 2026-07-14 | [link](spa-modularization.md#2026-07-14--playouts-screen-extraction-245-screen-owned-route-wrapper-epic-243-phase-2) |
| `spa.schedules-editor-draft-save` | The schedules SPA editor mutates a local draft and flushes one explicit Save (`PUT /api/schedules/{id}/items`) instead of instant-persisting each action; Copy deep-copies all source references (fixing a Blazor omission); the shuffled-schedule GET's `EnforceProperties` lossy normalization is preserved and mirrored in the SPA's option lists. | 2026-07-11 | [link](../decisions.md#2026-07-11--schedules-spa-editor-draftexplicit-save-over-instant-persist-copy-includes-multismartrerun-shuffled-get-normalization-preserved) |
| `spa.sidebar-collapsible-accordions` | The shell sidebar's collapse + nav-group-accordion state persists under two hyphenated `ctv-sidebar-*` localStorage keys (matching the repo's `ctv-` convention, not the prototype's dotted names); labeled groups default-collapsed. | 2026-07-18 | [link](../decisions.md#2026-07-18--collapsible-sidebar--nav-group-accordions-two-ctv-sidebar--localstorage-keys-labeled-groups-default-collapsed-396) |
| `spa.smartcollection-rule-builder` | The SmartCollection visual rule builder compiles to/from a closed subset of the Lucene grammar over the existing stored query string — no new AST, one level of group nesting. | 2026-07-18 | [link](../decisions.md#2026-07-18--smartcollection-rule-builder-compile-only-closed-subset-no-stored-ast-one-level-nesting-176) |
| `spa.spa-rebuild-decision` | The UI is a full React SPA (ChicoryTV) rebuild over the REST API, not a Blazor Server reskin. | 2026-06 | [link](../decisions.md#2026-06--ui-rebuild-is-a-react-spa-chicorytv-on-the-rest-api-not-a-blazor-reskin) |
| `spa.templates-editor-table` | The SPA templates editor renders day/block assignment as a table, not Blazor's drag-and-drop calendar grid — an accepted, deliberate parity deviation. | 2026-07 | [link](../decisions.md#2026-07--templates-editor-in-the-spa-is-a-table-not-blazors-drag-calendar) |
| `spa.topbar-primary-action` | The TopBar's primary-action "+" button renders only when the active route declares a non-empty `primaryAction`, is wired (via a shared `usePrimaryAction` hook) only on single-unambiguous-create-flow list screens, and is dropped everywhere else rather than left as a dead/no-op button. | 2026-07-12 | [link](../decisions.md#2026-07-12--topbar-primary-action-button-wire-creates-drop-the-rest-238) |
| `spa.yaml-validator-textarea` | The YAML playout validator takes pasted YAML via a `<textarea>`, not a server-side file path, since the SPA has no filesystem access. | 2026-07-09 | [link](../decisions.md#2026-07-09--yaml-playout-validator-paste-textarea-instead-of-a-server-file-path) |
| `startup.parallel-orientation` | A fresh session runs two concurrent tracks at startup — Orientation (`AGENTS.md`/`CLAUDE.md``docs/README.md` task-signal map → the active decisions catalog `docs/decisions/README.md`) and, only when no issue is named, Selection (`scripts/select-queue.sh N`, deterministic live-Gitea ranking). A named issue skips Selection entirely. ersatztv#237, the closed pickup tracker this replaces, is reduced to a single archival breadcrumb and MUST NOT be read for live state. | 2026-07-21 | [link](../decisions.md#2026-07-21--parallel-orientation--selection-is-the-startup-protocol-237-retired-520) |
+36
View File
@@ -26,6 +26,10 @@ contract-freeze), #206 (Blazor-removal auth posture), #283 (artwork content-type
---
## 2026-07-11 — Blazor removal auth posture: no new exposure beyond phase (a); real auth deferred to #197 (#206)
`key: security.blazor-removal-auth-posture` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
**Rule:** Removing the Blazor UI's OIDC-challenged surface exposes nothing a user couldn't already reach via the already-open `/app` SPA (open since phase (a)); real SPA/API authentication is deliberately deferred to #197, and the removal PR must preserve `ConditionalIptvAuthorizeFilter`, `ApiKeyAuthorizationFilter`, and `JwtHelper` access_token support.
**Signals:** Blazor removal, auth posture sign-off, OIDC attachment point · paths: `ErsatzTV/Startup.cs`, `ErsatzTV/Pages` · issues: #206, #91, #197
**Mechanics:** `ErsatzTV/Startup.cs` (Razor Pages/OIDC registration); #91 phase (b) removal PR
Sign-off for the #91 phase (b) removal-gate item #206 ("deleting the last challenged Blazor page leaves
only the open SPA"). The actual authorization wiring in `ErsatzTV/Startup.cs` + `ErsatzTV/Pages` was
@@ -68,6 +72,10 @@ pieces above; `MapControllers()` + `/docs` (Scalar), currently co-hosted in the
survive the surgical reduction.
## 2026-07-11 — Baseline security response headers + Phase-0 API hardening (#197, PR #279)
`key: security.baseline-response-headers` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
**Rule:** `SecurityHeadersMiddleware`, registered first in the pipeline, sets `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, and `Referrer-Policy: strict-origin-when-cross-origin` on every response (CSP/HSTS deliberately deferred); API-key comparison is constant-time and playout pagination is clamped.
**Signals:** security headers, nosniff, Phase-0 hardening, constant-time comparison · paths: `ErsatzTV/Middleware/SecurityHeadersMiddleware` · issues: #197, #279, #283
**Mechanics:** `ErsatzTV/Middleware/SecurityHeadersMiddleware`
Phase-0 of the #197 remediation — the posture-**independent** safe subset, shipped ahead of the
fail-closed/CORS/versioning posture work tracked in #280#289.
@@ -95,6 +103,10 @@ the OpenAPI security scheme) is decomposed into #280#289 with the phased road
append their own decisions here as they land.
## 2026-07-12 — Artwork content-type is sniffed, never reflected (#283, S4/S9 stored XSS)
`key: security.artwork-content-type-sniff` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** Artwork content type is always derived from the stored bytes (never the client-declared value or a `?contentType=` query param) at both upload and serve, clamped to an image allow-list, closing the unauthenticated stored-XSS chain; Kestrel `MaxRequestBodySize` bounds upload DoS.
**Signals:** stored XSS, content-type sniffing, artwork upload/serve · paths: `ErsatzTV.Core/Images/ImageContentTypes.DetectContentType`, `GetCachedImagePathHandler` · issues: #283, S4, S9
**Mechanics:** `ErsatzTV.Core/Images/ImageContentTypes`
The artwork upload/serve path trusted client-supplied content types at both ends, giving a stored-XSS
chain on **unauthenticated** GET sinks: upload `<script>` bytes declared `image/png`
@@ -131,6 +143,10 @@ duplicated in `UploadArtworkHandler`). Both serve sinks are `[ApiExplorerSetting
none of this changes the OpenAPI document.
## 2026-07-12 — Fail-closed API auth + sensitive-read tier + CORS/ForwardedHeaders lockdown (#197 Bundle A, PR #292)
`key: security.fail-closed-api-auth` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** Every mutating `/api` request requires `X-Api-Key` (no open mode); reads are gated by `Api:RequireKeyForReads` (default true) OR `[RequiresApiKey]` on sensitive controllers; CORS is an exact-origin allowlist (`ApiCors`); `ForwardedHeaders` trust stays configurable but defaults to trust-all-with-warning.
**Signals:** fail-closed auth, sensitive-read tier, CORS allowlist, single API key · paths: `IApiKeyProvider`, `ErsatzTV/Services/ApiKeyProvider.cs`, `FileSystemLayout.ApiKeyPath` · issues: #197, #280, #281, #282, #284, #285
**Mechanics:** `docs/api-conventions.md` §5; `ApiControllerSecurityTests`
Phase-1 of the #197 remediation — the auth posture that must land before any remote exposure.
Owner decisions (confirmed this session): **single API key** (no read/write split), and
@@ -176,6 +192,10 @@ versioning — remains #286/#287/#288. Phase-3 follow-ups: #265, #269, #172 rema
paging, per-key rate limiting.
## 2026-07-12 (#197 Bundle C — contract-freeze honesty)
`key: security.contract-freeze-honesty` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** The OpenAPI doc's declared security/401 scheme is generated from the same `ApiKeyAuthorizationFilter.EndpointRequiresKey` predicate the runtime enforces (so declared auth can't drift from enforced auth), every `/api/*` action returns a ResponseModel (no raw Application VMs), and Channel REST resources are keyed by immutable `Id`, never mutable `Number`.
**Signals:** OpenAPI contract honesty, ResponseModel wrapping, Id vs Number key · paths: `OpenApiContractHonestyTests`, `ErsatzTV.Core/Api` · issues: #197, #287, #288
**Mechanics:** `OpenApiContractHonestyTests`; `docs/api-conventions.md`
**#287 — OpenAPI contract honesty by construction.** The "v1" document now emits the `ApiKey` security
scheme plus per-operation `security`/`401` derived from the *same*
@@ -207,6 +227,10 @@ number-based lookup endpoint may be added additively later; `UniqueId` (Guid) st
contract absent a federation requirement.
## 2026-07-12 — Browser SPA session auth: `/api` accepts session OR machine key (#295 PR1, server-only)
`key: security.session-auth-dual-credential` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** `ApiAuthorizationFilter` accepts a request when a valid `X-Api-Key` matches OR the principal is an authenticated session (cookie `ctv-session`, `HttpOnly`/`SameSite=Lax`); session-authenticated mutations require the presence-only `X-CSRF` header or are rejected 403. This narrows the OIDC-inert sub-claim of `security.blazor-removal-auth-posture` (#206) — the rest of that record's auth-surface enumeration still holds.
**Signals:** session auth, dual credential, CSRF, OIDC revival, local admin · paths: `ApiAuthorizationFilter`, `ErsatzTV/Startup.cs` · issues: #295, #206, #197
**Mechanics:** `docs/spa-conventions.md` §5e (PR2); `RootWriterForceVersionTests`-adjacent auth tests
Implements the ratified #295 design (Fable [PLAN-MODE] pass, issue comment 9548). Supersedes the #206
"OIDC wiring stays inert until #197" note: the retained OIDC service registration is now **revived**, and a
@@ -306,6 +330,10 @@ is an additional accepted credential the doc needn't express. `AuthController` i
same release, then a manual Authelia round-trip checklist before the prod pin bump.
## 2026-07-12 — #295 PR2: SPA session cutover + #301 side-effecting-GET POST-ification
`key: security.session-cutover-postify` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** The browser SPA authenticates cookie-only (no more `X-Api-Key` from `web/`); the machine key is repurposed to external/MCP-only via `GET /api/auth/machine-key`; every side-effecting GET/HEAD under `/api` is converted to POST so the existing CSRF gate covers it (standing rule: never add a side-effecting GET/HEAD under `/api`).
**Signals:** SPA cookie-only cutover, CSRF POST-ification, AuthGate boot flow · paths: `web/src/api/client.ts`, `web/src/AuthGate.tsx` · issues: #295, #301, #197
**Mechanics:** `docs/spa-conventions.md` §5e; `docs/api-conventions.md` §9; `docs/e2e-local.md`
PR1 shipped the server side (previous entry): `/api` accepts a session cookie OR the machine `X-Api-Key`, with
`X-CSRF` required on session-authenticated mutations. **PR2 is the SPA cutover** — the browser now authenticates
@@ -346,6 +374,10 @@ logout is a future nicety. Docs: `spa-conventions.md §5e` (SPA seams), `api-con
(browser setup/login flow). Refs #295 #301 #197.
## 2026-07-12 — Enforcing CSP + Permissions-Policy on the host (#319, ZAP baseline)
`key: security.csp-permissions-policy` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** `SecurityHeadersMiddleware` sends an enforcing (not report-only) `Content-Security-Policy` (no `unsafe-inline`/`unsafe-eval`; the one inline theme-bootstrap script allow-listed by hash) and a deny-all `Permissions-Policy` on the SPA/`/api`/`/artwork`/`/iptv`; `/docs` and `/openapi` keep only the baseline headers, excluded from CSP because Scalar needs inline bootstrap.
**Signals:** enforcing CSP, script-src hash allowlist, Permissions-Policy · paths: `ErsatzTV/Middleware/SecurityHeadersMiddleware`, `SecurityHeadersMiddlewareTests` · issues: #319, #314, #197, #279
**Mechanics:** `SecurityHeadersMiddlewareTests.Csp_Script_Hash_Should_Match_The_Spa_Index`
Completes the CSP that the #279 baseline-headers entry deferred ("CSP must be validated against the ChicoryTV
SPA's inline assets"). Surfaced by the #314 out-of-ecosystem ZAP baseline (missing CSP/Permissions-Policy WARNs);
@@ -375,6 +407,10 @@ camera/microphone/geolocation/payment/usb) and an **enforcing** `Content-Securit
absent on `/docs`/`/openapi`). HSTS remains out (proxy/TLS decision). Refs #319 #314 #197.
## 2026-07-13 — Cross-origin resource policy: `same-origin` on every response (#330)
`key: security.corp-same-origin` · `status: active` · `since: 2026-07-13` · `supersedes: none` · `superseded-by: none`
**Rule:** `SecurityHeadersMiddleware` sends `Cross-Origin-Resource-Policy: same-origin` on every response including `/docs`/`/openapi`, blocking cross-origin `no-cors` embedding without affecting allowed CORS-mode fetches or server-side Jellyfin `/iptv/*` requests.
**Signals:** CORP, same-origin, cross-origin embedding · paths: `ErsatzTV/Middleware/SecurityHeadersMiddleware` · issues: #330, #319, #314
**Mechanics:** `ErsatzTV/Middleware/SecurityHeadersMiddleware`
The authenticated #314 ZAP scan found that ErsatzTV's baseline response posture omitted
`Cross-Origin-Resource-Policy`. `SecurityHeadersMiddleware` now sends
+22
View File
@@ -0,0 +1,22 @@
# Archive — superseded / retired decision records
This directory holds decision records whose `status` is `superseded` or `retired`. They are kept
verbatim (rationale prose untouched — see `docs/decisions.md` header and `scripts/decisions_lib.py`)
for history: *why we changed our mind* is the point, never silently rewritten.
They are **out of the active startup path**: `scripts/decisions_lib.py active_files()` /
`all_active_records()` do not glob this directory, `docs/decisions/README.md` (the active catalog)
never lists a record from here, and an agent doing task-router discovery should not need to read
this directory to find the *current* rule — follow a record's `superseded-by` key to the active
successor instead.
The lifecycle validator (`scripts/decisions_validate.py`) still enforces invariants here:
- a `superseded`/`retired` record MUST live under this directory, never in an active file;
- an `active` record MUST NOT live under this directory;
- `supersedes`/`superseded-by` keys must resolve reciprocally to a record in the active set OR here;
- a record moved here must not have its rationale prose changed in the same commit (unless the
commit message carries the `[decisions-edit]` token, reserved for genuine rationale edits).
One file per topic cluster (e.g. `ci.md`, `release-ci-governance.md`), mirroring the active
`docs/decisions/*.md` topic-file split. See `docs/decisions/migration-map.md` for the legacy
heading → key → status → location mapping produced during the #521 lifecycle migration.
@@ -0,0 +1,43 @@
# Archived — release/CI/merge-governance records superseded by the #521 lifecycle schema
Records relocated verbatim (rationale prose unchanged) from `docs/decisions/release-ci-governance.md`
during the #521 decision-lifecycle migration. See `docs/decisions/archive/README.md` for what this
directory is and `docs/decisions/migration-map.md` for the full mapping.
- **`docs.append-only-guard`** (below) — the `docs/decisions.md`-is-append-only-by-construction half
of #303 H9/H3. Superseded by `docs.decision-lifecycle` (active, in `docs/decisions.md`) — the
lifecycle validator (`scripts/decisions_validate.py`) that this very PR introduces replaces
line-level append-only enforcement with record-level lifecycle checks. The companion H3
root-screenshot guard was split out (#521) into its own still-active record,
`ci.root-screenshot-guard`, in `docs/decisions/release-ci-governance.md` — it is not covered by
this supersession and was never archived.
---
## 2026-07-12 — decisions.md is append-only, enforced; root-screenshot guard (#303 H9/H3)
`key: docs.append-only-guard` · `status: superseded` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: docs.decision-lifecycle@2026-07-21`
**Rule:** (superseded) `docs/decisions.md` is append-only, enforced by hook + CI.
**Signals:** decisions-guard hook · paths: `.claude/hooks/decisions-guard.sh`, `.husky/commit-msg` · issues: #303 (H9)
**Mechanics:** superseded by `scripts/decisions_validate.py` (ersatztv#521); see `docs/decisions.md``docs.decision-lifecycle`
**This log is append-only by construction, not just by convention.** A commit or PR that deletes or
modifies an existing line of `docs/decisions.md` is blocked — by the Husky `commit-msg` hook
(`.claude/hooks/decisions-guard.sh staged`) locally and the blocking `decisions-guard` CI job (same
script, `range` mode) on PRs. Shared detection, deliberately different granularity: the Husky hook
gates **each commit** (its own message must carry the token); CI gates the **PR-wide** net diff
(token in any commit of the range suffices), so the local hook is the stricter primary gate and CI the
push/bypass backstop. Insertions anywhere are always allowed, so a normal new entry (TOC line
near the top + a block appended at the bottom, both pure insertions) passes untouched. Detection is
`git diff --numstat` deleted-count > 0, which is robust to markdown `-` list markers (a byte-level `-`
prefix would false-match). The block is lifted only by the literal **`[decisions-edit]`** token in the
commit message, reserved for two cases: fixing a factual error, and superseding a reversed decision
(add the new entry, prepend a `> **Superseded …**` banner to the old one, tag its Index line
`(superseded)` — keep the old rationale, never silently rewrite). **Consolidation** of superseded
entries is a release-checklist step (`docs/ci-cd.md` → Versioning & releases), backstopped by a
non-blocking 1800-line **size floor** in the `decisions-guard` job (the read-cost point past which the
log no longer fits one default agent Read), so append-only doesn't accrete contradictory *or
unreadably-large* history between releases (Timothy's call, 2026-07-12: mark-and-keep on reversal,
consolidate at each milestone, size-floor backstop).
The companion H3 root-screenshot guard was split out during the #521 migration to the active record
`ci.root-screenshot-guard` in `docs/decisions/release-ci-governance.md`.
+40
View File
@@ -0,0 +1,40 @@
# Archive — startup / kickoff protocol
Superseded/retired records for the session-startup / queue-selection protocol. See
`docs/decisions/archive/README.md` for the archive's general rules (rationale kept verbatim,
never in the active read-path). Active successor: `startup.parallel-orientation` in
`docs/decisions.md`.
---
## 2026-07-11 — Queue state lives in the pinned Gitea tracker (#237), not in the handoff file
`key: docs.queue-state-gitea-tracker` · `status: superseded` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: startup.parallel-orientation@2026-07-21`
**Rule:** (superseded) Volatile session/queue state lives in pinned Gitea tracker issue #237 (goal + arc in the body, append-only session-comment log, in-progress/review labels), not in a wholesale-rewritten handoff file; the handoff file keeps only the static kickoff prompt and append-only lore.
**Signals:** last-writer-wins race, claim/`in-progress` label, triage verdict · paths: `docs/handoffs/chicorytv-issue-queue.md` · issues: #237, #520
**Mechanics:** superseded by `scripts/select-queue.sh` (ersatztv#520); see `docs/decisions.md``startup.parallel-orientation`
With multiple sessions/agents working the repo in parallel, the old protocol — every session
wholesale-rewrites `docs/handoffs/chicorytv-issue-queue.md` on main (session state + queue +
next-session prompt) — became a last-writer-wins race. New protocol: **volatile queue state
moved to Gitea**, which is concurrency-safe by construction. Pinned tracker issue **#237**
holds the goal + ordered arc in its body (edited rarely, only on arc changes, re-read before
edit) and an append-only session-comment log (fixed template: Closed / Filed / Triage /
Arc change / Recommended next). Milestone `Blazor removal (#91 phase b)` + the `review` and
`in-progress` labels are the machine-queryable view. Sessions **claim** an issue before working
it (`in-progress` label + claim comment; the tiny read→claim race window is accepted, later
claimant backs off; stale claims — no commits/comments ~48h — may be taken over with a comment).
Every new issue gets an explicit end-of-session triage verdict — gate-blocker (milestone + arc
slot) or backlog (label only) — so review findings adjust the queue only through that step and
the arc doesn't drift. The handoff file keeps only the **static kickoff prompt** and the
**append-only Lessons lore** (per-session prompts are gone; task context lives in issue bodies).
**Why superseded (#520, 2026-07-21):** the arc completed and #237 closed (2026-07-13); a closed
tracker cannot serve as live queue state, and continuing to read it as such was a live regression
risk (an agent skimming an old comment or this very record could re-treat #237's prose as current).
`scripts/select-queue.sh` (2026-07-19) already replaced the mechanical parts of this rule with a
deterministic, live-Gitea-only query — this record's job was really "queue state lives in Gitea,
not in the handoff file," and that half is still true; what's superseded is the *specific store*
(#237's body/comments) now that maintenance/backlog mode has no arc to narrate. See
`startup.parallel-orientation` for the replacement: two concurrent session-start tracks
(orientation via the docs/decisions catalog + `docs/README.md` map, and selection via the script),
with #237 reduced to a single archival breadcrumb.
+62
View File
@@ -0,0 +1,62 @@
# Decision migration map (legacy heading → active record / archive)
The full corpus (`docs/decisions.md` and the `docs/decisions/*.md` topic files) was **fully migrated
to the lifecycle schema on 2026-07-21 (#520/#521)**: `PYTHONPATH=. python3 scripts/decisions_validate.py`
reports 0 `legacy-unmigrated` headings, and the catalog (`docs/decisions/README.md`, built by
`scripts/build_decisions_catalog.py`) lists 108 active records. Every active decision heading now
carries a metadata block (`key:`/`status:`/`since:`/`supersedes:`/`superseded-by:`). Most headings were
migrated **in place** — the heading text is unchanged, only the metadata block was added — so their key
is discoverable via the catalog or `rg '^\`key:\`' docs/decisions/`, not via this file. This table
records only the **non-trivial** mappings produced during the migration: splits, supersessions, and
prose-only reversal notes. It is not, and was never meant to be, a 1:1 index of all 108 records.
| Legacy heading | Key | Status | Location |
| --- | --- | --- | --- |
| 2026-07-17 — No persistent compiler servers in CI; every `services:` container gets an explicit cap; #390's small-lane move reversed (#406) | `ci.runner-placement` | active | `docs/decisions.md` |
| 2026-07-19 — CI `test` job reports a sampled true peak-anon, not cache-inflated `memory.peak` (#412) | `ci.peak-anon-measurement` | active | `docs/decisions.md` |
| 2026-07-12 — decisions.md is append-only, enforced; root-screenshot guard (#303 H9/H3) — heading kept identical to the pre-split original (only the body/metadata were split) so the lifecycle validator's relocation check matches it | `docs.append-only-guard` | superseded (`superseded-by: docs.decision-lifecycle@2026-07-21`) | `docs/decisions/archive/release-ci-governance.md` |
| 2026-07-12 — Root-screenshot guard: pre-commit refuses root-level `*.png` (#303 H3) — split from the original H9/H3 bundle | `ci.root-screenshot-guard` | active | `docs/decisions/release-ci-governance.md` |
| 2026-07-21 — Decision records carry a lifecycle schema, validated by a script; append-only-by-diff is retired (#521) | `docs.decision-lifecycle` | active (`supersedes: docs.append-only-guard@2026-07-12`) | `docs/decisions.md` |
| 2026-07-11 — Queue state lives in the pinned Gitea tracker (#237), not in the handoff file | `docs.queue-state-gitea-tracker` | superseded (`superseded-by: startup.parallel-orientation@2026-07-21`) | `docs/decisions/archive/startup.md` |
| 2026-07-21 — Parallel orientation + selection is the startup protocol; #237 retired (#520) | `startup.parallel-orientation` | active (`supersedes: docs.queue-state-gitea-tracker@2026-07-11`) | `docs/decisions.md` |
## Migration conventions
A legacy `##` entry that bundles an active sub-decision with a superseded/retired one is SPLIT during
migration — the superseded half moves to `archive/`, the active half becomes its own `status: active`
record in the active files. Never relocate an active decision to `archive/`. First applied to the
#303 H9/H3 bundle (below): H9 (docs.decisions.md append-only) is genuinely superseded and archived as
`docs.append-only-guard`; H3 (the root-screenshot pre-commit guard) is independent and still active, so
it was split out into its own active record, `ci.root-screenshot-guard`, in
`docs/decisions/release-ci-governance.md` rather than being carried into archive with its superseded
sibling.
## Notes on records considered but NOT given a standalone entry
- **#390 (small-lane move for `api-docs`/`format`).** Investigated whether #390's reversal (described
in #406's prose, point 3: "#390's `small`-lane move ... reversed") ever existed as its own `##`
record. It did not — grep for `#390` across `docs/decisions.md` and `docs/decisions/*.md` finds it
only inside #406's own body text (and one other unrelated mention of #390's apt-ffmpeg estimate,
also inside #406). #390 was **prose-only**: no separate heading, so nothing to mark `superseded` or
move to archive. `ci.runner-placement` (#406) therefore carries `supersedes: none`; the reversal is
recorded only in its `**Signals:**` line (`#390 (prose-reversed, no standalone record)`) and in this
note.
- **#411 (the earlier `memory.peak`-headline measurement).** Investigated the "#411 obsolescence" the
brief references. #411 is mentioned twice in `docs/decisions.md`, both inside #412's own body
("the `test` job's memory instrument (added in #411) now reports ..." and "the older #411 probe
... is superseded"). Like #390, #411 never had its own `##` heading — its "decision" was
narrated retroactively inside #406 (the "Measurement is now continuous" paragraph, which describes
the instrument #411 actually shipped) and explicitly marked obsolete inside #412's own prose. Since
there is no standalone #411 record, there is nothing to move to `archive/`; `ci.peak-anon-measurement`
(#412) — the record that supersedes #411's approach — carries `supersedes: none` and documents the
prose reversal in its `**Signals:**` line. The governed surface (CI memory measurement) still exists
and is very much active, so this is a straightforward **supersession of an unnamed predecessor**, not
a retirement — there's no case to make for `retired` here since the surface it measures is live.
## Still open
- **DONE**: `docs.queue-state-gitea-tracker` (#237) — superseded by `startup.parallel-orientation`
(2026-07-21, #520), archived at `docs/decisions/archive/startup.md`. See the table above.
- Nothing else is open from this migration — the corpus is fully migrated (0 legacy-unmigrated). Future
new decisions are simply added directly in the lifecycle schema; there is no further migration pass
to track here.
+28
View File
@@ -22,6 +22,10 @@ cross-editor ETag rotation). Refs #197.
---
## 2026-07-11 — Optimistic-concurrency contract for replace-all PUTs (#253 PR1: infra + Block reference)
`key: concurrency.replace-all-contract` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
**Rule:** Replace-all aggregate PUTs carry a uniform plain `int Version` concurrency token (EF `.IsConcurrencyToken()`), checked pre-save and enforced by the EF UPDATE guard, returning 412 (not 409) on a stale `If-Match`.
**Signals:** ETag, If-Match, Version token, 412 Precondition Failed · paths: `api-conventions.md` §7a, `ConcurrencyHeaders`, `ApiResults.ToErrorResult` · issues: #253, #197, #265, #259
**Mechanics:** `docs/api-conventions.md` §7a; `SaveChangesWithConcurrencyGuard`
Replace-all aggregate PUTs had **no** optimistic concurrency — a stale second tab silently overwrote a
fresher edit (200, no signal) across ~10 aggregate surfaces. PR1 lands the shared contract on the Block
@@ -55,6 +59,10 @@ the mechanics live in `api-conventions.md` §7a. Decisions frozen here:
"valid-but-non-matching → 412" refinement is deferred to #197 (**#265**).
## 2026-07-11 — #253 PR3: Diff + Scalar concurrency fan-out (Collection / Playout×2 / MultiCollection / RerunCollection)
`key: concurrency.diff-scalar-fanout` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
**Rule:** The frozen Block optimistic-concurrency recipe (api-conventions §7a) fans out to Collection/Playout×2/MultiCollection/RerunCollection, keeping a guard-returned `PreconditionFailedError` out of any handler's generic `catch(Exception)`→422 mapping, and preserving each aggregate's existing `SaveChangesAsync() > 0` gate semantics under the new unconditional `Version++`.
**Signals:** concurrency fan-out, PreconditionFailedError, SaveChangesWithConcurrencyGuard · paths: `api-conventions.md` §7a · issues: #253, #269, #232, #197
**Mechanics:** `RootWriterForceVersionTests`-adjacent handler tests; `api-conventions.md` §7a
**Context.** PR3 of the #253 optimistic-concurrency arc fans the frozen Block recipe (api-conventions §7a)
across the five Diff/Scalar aggregates. Three judgment calls beyond the mechanical copy:
@@ -97,6 +105,10 @@ force-write on conflict: adopt the stored token, retry, never revert the concurr
above is re-scoped to the DELETE handlers + repository `Add*` writers only (→ #269).
## 2026-07-11 — Stable child identity for schedule-item replace (#259, split from #252/#253)
`key: concurrency.schedule-item-child-identity` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
**Rule:** `PUT /api/schedules/{id}/items` reconciles by an optional round-tripped child `Id` (null/absent/0 ⇒ new item), never by array position, so fill-group/shuffle state follows the logical item across reorders; an unknown or duplicate id is rejected 422 (checked after the §7a `CheckVersion`, so 412 precedes 422).
**Signals:** stable child identity, schedule-item replace, id-based reconcile · paths: `api-conventions.md` §7c · issues: #259, #252, #253, #197
**Mechanics:** `docs/api-conventions.md` §7c
`PUT /api/schedules/{id}/items` now reconciles by an optional round-tripped child id, not by array
position, so an item's persisted fill-group/shuffle state (`PlayoutScheduleItemFillGroupIndex`, FK
@@ -121,6 +133,10 @@ whatever previously held its new slot. Contract + rules in **api-conventions §7
now 422s).
## 2026-07-12 (#269 — non-If-Match root writers force-write past a concurrent Version bump)
`key: concurrency.force-write-non-ifmatch` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** Any handler that leaves a versioned root `Modified` or `Deleted` but takes no `If-Match` (deletes, item add/remove bumpers, scalar-config writers) must save through `ConcurrencyExtensions.SaveChangesForcingVersion` — force-write past a concurrent `Version` bump rather than throw an unhandled `DbUpdateConcurrencyException` (500).
**Signals:** force-write, non-If-Match writers, DbUpdateConcurrencyException · paths: `ConcurrencyExtensions.SaveChangesForcingVersion` · issues: #269, #253, #302, #197
**Mechanics:** `RootWriterForceVersionTests`; `docs/api-conventions.md` §7a
**Routing the aggregate delete handlers + `UpdateProgramScheduleHandler` through `SaveChangesForcingVersion`.**
Once #253 made each replace-all root's `Version` an `IsConcurrencyToken`, EF started guarding *every*
@@ -169,6 +185,10 @@ bump *through the handler* via a pre-tracked context (`RootWriterForceVersionTes
negative control proving the plain-save path throws.
## 2026-07-12 — Cross-editor ETag rotation completed for Collection/Playout config siblings (#269)
`key: concurrency.etag-rotation-completion` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** Every handler that mutates a versioned root's editor-visible config state must bump `Version` (rotating the ETag) with no per-aggregate carve-outs, short-circuiting on a genuine no-op before the bump so idempotent re-submits don't fire spurious rebuild fan-out; `SaveChangesForcingVersion` rebases the retry (stored + pending delta), never adopts the stored token verbatim.
**Signals:** ETag rotation, no-op idempotence, SaveChangesForcingVersion rebase · paths: `CollectionEtagRotationTests`, `PlayoutScheduleFileEtagRotationTests` · issues: #269, #253, #197, #308
**Mechanics:** `docs/api-conventions.md` §7a; `CollectionEtagRotationTests`, `PlayoutScheduleFileEtagRotationTests`
The #253 optimistic-concurrency contract (§7a) had a documented tail: the non-If-Match config-sibling
writers of a versioned root mutated editor-visible state **without** bumping `Version`, so editing through
@@ -223,6 +243,10 @@ refinement (valid-but-non-matching/weak/list → 412 not 400) is a **separate**
parser + `CheckVersion`, not the handler saves). Refs #253 #269 #197 · `api-conventions.md` §7a.
## 2026-07-12 — If-Match evaluates per RFC 7232: valid-but-non-matching → 412, only grammar violations → 400 (#265)
`key: concurrency.ifmatch-rfc7232` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** `ConcurrencyHeaders.ParseIfMatch` is a real RFC 7232 entity-tag/list parser: a syntactically-valid tag that doesn't strong-match (weak/empty/non-canonical/out-of-range/list) returns 412, and only a genuine grammar violation returns 400.
**Signals:** RFC 7232, If-Match parsing, strong-tag matching · paths: `ConcurrencyHeaders`, `IfMatchCondition`, `VersionedAggregateExtensions.CheckVersion` · issues: #265, #253, #197
**Mechanics:** `docs/api-conventions.md` §7a
Closing the last #253 concurrency-contract piece. `ConcurrencyHeaders.ParseIfMatch` previously classified
**any** non-canonical/weak/list `If-Match` value as `Malformed → 400` (a deliberate fail-safe: reject rather
@@ -257,6 +281,10 @@ the change only makes a hand-written/tooling `If-Match` get the RFC-correct stat
§7a. Refs #265 #253 #197.
## 2026-07-18 — Concurrent same-item add is idempotent, not a 500: catch the unique-violation per provider (#308)
`key: concurrency.idempotent-concurrent-add` · `status: active` · `since: 2026-07-18` · `supersedes: none` · `superseded-by: none`
**Rule:** A concurrent duplicate `Add*ToCollection` that loses the race on the composite-key unique constraint is treated as an idempotent no-op (skip the reindex/rebuild fan-out), not a 500 — detected via a provider-specific `TvContext.IsUniqueConstraintViolation` delegate defaulting to "no".
**Signals:** idempotent add, unique-constraint violation, provider error classifier · paths: `TvContext.IsUniqueConstraintViolation`, `SqliteErrorClassifier`, `MySqlErrorClassifier` · issues: #308, #269, #253
**Mechanics:** `docs/api-conventions.md` §7a ("Idempotent insert under concurrency"); `ConcurrencyExtensions.TrySaveChangesForcingVersion`
**Decision.** The `Add*ToCollection` family's membership pre-check (#269) is not atomic with the insert, so two
*concurrent* adds of the same item both observe it absent and both stage the `CollectionItem` composite key; the
+37 -23
View File
@@ -7,8 +7,8 @@ auto-grant fix, migration rehearsal on a prod-DB copy, and release promotion. Ra
relocated from the append-only `docs/decisions.md` at the v26.9.0 consolidation; operational
detail cross-links to `docs/ci-cd.md` and CLAUDE.md → Task Completion Protocol.
Issue trail: #303 (H4/H5 api-docs gate, H6 Done-when, H9/H3 append-only + root-png, H10
review-verdict), #311 (H11 formatting/rebase), #314 (merge-gate auto-grant), #315 (migration
Issue trail: #303 (H4/H5 api-docs gate, H6 Done-when, H9 append-only (archived), H3 root-png
(active), H10 review-verdict), #311 (H11 formatting/rebase), #314 (merge-gate auto-grant), #315 (migration
rehearsal), #335 (release promotion). The whole hook program's throughline: make each process
rule a derivation/hook, not prose to remember (#303 methodology review).
@@ -16,7 +16,8 @@ rule a derivation/hook, not prose to remember (#303 methodology review).
- [2026-07-12 — Blocking CI gate for API-contract artifacts (#303 H4/H5)](#2026-07-12--blocking-ci-gate-for-api-contract-artifacts-303-h4h5)
- [2026-07-12 — Merge-consent derived from state via a `## Done-when` issue checklist (#303 H6)](#2026-07-12--merge-consent-derived-from-state-via-a--done-when-issue-checklist-303-h6)
- [2026-07-12 — decisions.md is append-only, enforced; root-screenshot guard (#303 H9/H3)](#2026-07-12--decisionsmd-is-append-only-enforced-root-screenshot-guard-303-h9h3)
- **(archived 2026-07-21)** 2026-07-12 — decisions.md is append-only, enforced; root-screenshot guard (#303 H9/H3)`docs.append-only-guard` is `status: superseded` by `docs.decision-lifecycle` (see `docs/decisions.md`); relocated to [`archive/release-ci-governance.md`](archive/release-ci-governance.md#2026-07-12--decisionsmd-is-append-only-enforced-root-screenshot-guard-303-h9h3) (heading kept identical to the pre-split original so the lifecycle validator's relocation check matches it by heading). The bundled companion root-screenshot guard (H3) was split out into its own active record, below.
- [2026-07-12 — Root-screenshot guard: pre-commit refuses root-level `*.png` (#303 H3)](#2026-07-12--root-screenshot-guard-pre-commit-refuses-root-level-png-303-h3)
- [2026-07-12 — Review-verdict merge-gate: latest commit must be reviewed (#303 H10)](#2026-07-12--review-verdict-merge-gate-latest-commit-must-be-reviewed-303-h10)
- [2026-07-12 — Formatting-as-you-touch, enforced; rebase-not-merge for PR branches (#311 H11 + format CI)](#2026-07-12--formatting-as-you-touch-enforced-rebase-not-merge-for-pr-branches-311-h11--format-ci)
- [2026-07-12 — Merge-consent gate auto-grants when satisfied (no redundant prompt); state IS the consent (#314)](#2026-07-12--merge-consent-gate-auto-grants-when-satisfied-no-redundant-prompt-state-is-the-consent-314)
@@ -26,6 +27,10 @@ rule a derivation/hook, not prose to remember (#303 methodology review).
---
## 2026-07-12 — Blocking CI gate for API-contract artifacts (#303 H4/H5)
`key: release.api-contract-ci-gate` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** A PR touching `ErsatzTV/Controllers/Api/**` or `ErsatzTV.Core/Api/**` must ship regenerated OpenAPI artifacts (`v1.json`, `v1.d.ts`, `endpoint-index.md`) in the same diff, enforced by a blocking `api-docs` CI job that regenerates-and-diffs against a fresh build.
**Signals:** OpenAPI drift gate, blocking CI, api-docs job · paths: `ErsatzTV/wwwroot/openapi/v1.json`, `web/src/api/generated/v1.d.ts`, `docs/endpoint-index.md` · issues: #303 (H4/H5)
**Mechanics:** `docs/api-conventions.md` §5; `scripts/update-openapi.sh`; `.gitea/workflows` `api-docs` job
**A PR whose diff touches `ErsatzTV/Controllers/Api/**` or `ErsatzTV.Core/Api/**` must ship the
regenerated OpenAPI artifacts in that same diff, enforced by a blocking `api-docs` CI job.** It rebuilds
@@ -41,6 +46,10 @@ local-only: `update-openapi.sh` runs `dotnet-getdocument` against the already-bu
immune (no `bin/` on a fresh checkout).
## 2026-07-12 — Merge-consent derived from state via a `## Done-when` issue checklist (#303 H6)
`key: release.done-when-merge-consent` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** A PR may merge only when its linked issue's `## Done-when` checklist is fully ticked and the PR's CI is green, enforced by a PreToolUse hook on the Gitea merge tool (deny/allow/ask) plus a pre-push backstop for direct pushes to main.
**Signals:** Done-when checklist, merge consent, state-derived gate · paths: `.claude/settings.json`, `.husky/pre-push` · issues: #303 (H6)
**Mechanics:** `pretooluse-merge-consent.sh`; `prepush-donewhen.sh`; CLAUDE.md → Task Completion Protocol
**An issue's `## Done-when` checklist (in the issue body) is the machine-readable source of truth for whether
its PR may merge; consent is *derived*, not asserted.** Rationale: DONE/OPEN status used to live in
@@ -62,26 +71,11 @@ issues adopt `## Done-when`, the merge hook simply *asks* rather than auto-allow
Completion Protocol. (H6 lives with H1/H2/H8 in `.claude/settings.json`; H7 worktree-owner guard is its
sibling Wave-2 hook.)
## 2026-07-12 — decisions.md is append-only, enforced; root-screenshot guard (#303 H9/H3)
**This log is append-only by construction, not just by convention.** A commit or PR that deletes or
modifies an existing line of `docs/decisions.md` is blocked — by the Husky `commit-msg` hook
(`.claude/hooks/decisions-guard.sh staged`) locally and the blocking `decisions-guard` CI job (same
script, `range` mode) on PRs. Shared detection, deliberately different granularity: the Husky hook
gates **each commit** (its own message must carry the token); CI gates the **PR-wide** net diff
(token in any commit of the range suffices), so the local hook is the stricter primary gate and CI the
push/bypass backstop. Insertions anywhere are always allowed, so a normal new entry (TOC line
near the top + a block appended at the bottom, both pure insertions) passes untouched. Detection is
`git diff --numstat` deleted-count > 0, which is robust to markdown `-` list markers (a byte-level `-`
prefix would false-match). The block is lifted only by the literal **`[decisions-edit]`** token in the
commit message, reserved for two cases: fixing a factual error, and superseding a reversed decision
(add the new entry, prepend a `> **Superseded …**` banner to the old one, tag its Index line
`(superseded)` — keep the old rationale, never silently rewrite). **Consolidation** of superseded
entries is a release-checklist step (`docs/ci-cd.md` → Versioning & releases), backstopped by a
non-blocking 1800-line **size floor** in the `decisions-guard` job (the read-cost point past which the
log no longer fits one default agent Read), so append-only doesn't accrete contradictory *or
unreadably-large* history between releases (Timothy's call, 2026-07-12: mark-and-keep on reversal,
consolidate at each milestone, size-floor backstop).
## 2026-07-12 — Root-screenshot guard: pre-commit refuses root-level *.png (#303 H3)
`key: ci.root-screenshot-guard` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** The Husky `pre-commit` hook refuses a staged root-level `*.png` (belt-and-suspenders with the `.gitignore` rule); nested `*.png` real assets are unaffected.
**Signals:** git hooks, screenshot guard, root png · paths: `.husky/pre-commit`, `.gitignore` · issues: #303 (H3)
**Mechanics:** `.husky/pre-commit`
Companion guard **H3**: the Husky `pre-commit` hook refuses a staged **root-level `*.png`** (a
review/debug screenshot dropped at the repo root) — belt-and-suspenders with the `.gitignore` rule, so
@@ -89,6 +83,10 @@ a forced `git add -f` still can't land one. Nested `*.png` (real assets) are una
both: the methodology review (#303) — make the process rules derivations/hooks, not prose to remember.
## 2026-07-12 — Review-verdict merge-gate: latest commit must be reviewed (#303 H10)
`key: release.review-verdict-gate` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** A PR may not merge until a `Review-verdict: <MERGEABLE|APPROVED|BLOCKED|NOT-MERGEABLE> @ <head-sha>` comment references the PR's current head sha (short-sha prefix match, line-start marker only, negative wins over positive on the same head); folds into the H6 merge-consent hook as condition (c).
**Signals:** review-verdict, head-sha match, stale-review prevention · paths: `.claude/settings.json` · issues: #303 (H10), #242
**Mechanics:** `pretooluse-merge-consent.sh`; CLAUDE.md → Task Completion Protocol (H10 convention)
**A PR may not merge until a `Review-verdict:` comment on it references the PR's CURRENT head sha** —
so the *latest* commit is proven-reviewed, not a stale earlier diff. This mechanizes the ersatztv#242
@@ -128,6 +126,10 @@ real merge path; docs-only PRs remain exempt via H6's file-set exemption). Ratio
Wave-1/2/3 hook set: make the process rule a derivation/hook, not prose to remember (#303).
## 2026-07-12 — Formatting-as-you-touch, enforced; rebase-not-merge for PR branches (#311 H11 + format CI)
`key: release.format-as-you-touch-rebase` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** A blocking `format` CI job runs `dotnet format --verify-no-changes` scoped only to the PR's changed `.cs` files (never the legacy BOM backlog), and a PR branch must be kept current by rebasing on `origin/main` (never merging main in), enforced by `.husky/pre-push``prepush-rebase-check.sh`.
**Signals:** format-as-you-touch, rebase not merge, BOM backlog · paths: `.husky/pre-push`, `.claude/hooks/prepush-rebase-check.sh` · issues: #311 (H11), #309, #310, #269, #312
**Mechanics:** `docs/contributing.md` §7; `.claude/hooks/prepush-rebase-check.sh`; `npm run check:api`
Two coupled process decisions, prompted when a stale docs branch *merged main in*, dragged ~17
legacy-BOM `.cs` files it never touched into the merge commit, and the pre-commit `dotnet format`
@@ -159,6 +161,10 @@ Rationale, as with the whole hook program: make the process rule a derivation/ho
remember (#303 methodology review). Tracked: #311; sibling #312 (H12 issue-qualification audit).
## 2026-07-12 — Merge-consent gate auto-grants when satisfied (no redundant prompt); state IS the consent (#314)
`key: release.merge-consent-autogrant` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** When Done-when boxes are ticked, CI is green, and a fresh positive Review-verdict references head, the merge-consent hook emits `permissionDecision: allow` to actually suppress the redundant mechanical prompt — the derived state IS the consent, no separate conversational confirmation on that path.
**Signals:** auto-grant, permissionDecision allow, merge consent · paths: `.claude/settings.json` · issues: #314, #303, #317
**Mechanics:** `pretooluse-merge-consent.sh`; CLAUDE.md → Task Completion Protocol
Completes the #303 H6/H10 intent — *derive merge-consent from state* — which the original hook only
half-delivered. The rule the user set: **merge permission is auto-granted for the session when the
@@ -192,6 +198,10 @@ warranted only when the gate **asks** (state not derivable). This supersedes the
consent in-conversation per session" phrasing in the kickoff HARD CONSTRAINTS (updated in the same PR).
## 2026-07-12 — Release path rehearses migrations on a prod-DB copy before promoting (#315)
`key: release.migration-rehearsal-prodcopy` · `status: active` · `since: 2026-07-12` · `supersedes: none` · `superseded-by: none`
**Rule:** Before promoting a migration-bearing release, rehearse the new image's migrations against a throwaway copy of the latest prod backup (`scripts/migration-smoke.sh`), gating PASS on the migrator's completion log line rather than HTTP readiness alone.
**Signals:** migration rehearsal, prod-copy smoke test, DatabaseMigratorService · paths: `scripts/migration-smoke.sh` · issues: #315
**Mechanics:** `docs/ci-cd.md` → Migration-on-prod-copy smoke; `scripts/migration-smoke.sh`
The CI `migrations` job proves a migration is well-formed against a **fresh, empty** DB (model-drift +
apply-to-fresh, per provider). That is necessary but not sufficient: it never exercises the migration —
@@ -211,6 +221,10 @@ Rationale: data-plane rigor — catch a bad migration on a disposable copy, not
See `docs/ci-cd.md` → Migration-on-prod-copy smoke. Cross-repo wiring tracked in server-management.
## 2026-07-13 — Release promotion: floating `:prod`, exact-image scan before manual deploy (#335)
`key: release.promotion-floating-prod` · `status: active` · `since: 2026-07-13` · `supersedes: none` · `superseded-by: none`
**Rule:** Prod tracks the floating `:prod` image reference; a tag build's immutable `:<version>` image is scanned first, then promotion happens via a separate manual `DeployStack`, with daily auto-update only as a fallback — tag with enough runway before 03:00 to avoid an unscanned promotion.
**Signals:** floating prod tag, exact-image scan, manual promotion · paths: Komodo `DeployStack` · issues: #335, server-management#585, server-management#589
**Mechanics:** `docs/ci-cd.md` → Versioning & releases
Prod keeps the floating `:prod` image reference; PR #191's workflow-driven immutable pin bump is
closed as superseded. server-management#585 proved that automatic and manual promotions share
+192
View File
@@ -0,0 +1,192 @@
# Retrieval-eval question bank (decision-lifecycle corpus, #521)
This bank measures **correct active-record selection + citation**, not mere semantic proximity.
The migrated decision corpus (`docs/decisions.md` + `docs/decisions/*.md`, catalog at
`docs/decisions/README.md`, archive at `docs/decisions/archive/`) intentionally keeps a superseded
or retired record's prose *verbatim* next to a live `superseded-by`/`supersedes` pointer. A retrieval
approach that matches on wording alone can walk straight past that pointer into the archived record
and return a decision this project no longer follows. For each question below, the only correct
answer is the cited active `key` (status: `active`, in a file `active_files()` globs) — landing on
an archived/superseded key, or missing a key that already answers the question (and thus proposing a
reimplementation), is scored a **miss**, regardless of how relevant the returned text reads.
This bank is run as the **Task 10 cold-agent retrieval sim**: a fresh agent is given the catalog
(`docs/decisions/README.md`) plus the active `docs/decisions/*.md` wing and asked each question with
no other context, then graded on whether it names the expected `key` and cites the right file.
Verification: every `key` cited below was confirmed present and `status: active` via
`PYTHONPATH=. python3 -c "import scripts.decisions_lib as dl; [print(r.key, r.status) for f in dl.active_files() for r in dl.parse_file(f) if r.key]"`
against this worktree's corpus on 2026-07-21.
---
## 1. Paraphrased task → decision discovery
**Q1.** "I'm adding a new REST endpoint for a feature that touches three tables — do I need to stand
up a service/business-logic layer, or can the controller call MediatR directly?"
- **Expected key:** `api.mediatr-passthrough`
- **File:** `docs/decisions.md`
- **Why:** The rule is literally "thin controllers over existing MediatR handlers, no new
service/business-logic layer" — answer is NO, don't add one. A naive search might instead surface
`mcp.server-foundation` (also MediatR-adjacent, wrong layer) or nothing at all if it only matches
on "REST endpoint" rather than the layering question.
**Q2.** "Should the playout-build-finished notification go out over a websocket/SignalR push channel
so the SPA doesn't have to poll?"
- **Expected key:** `api.async-op-contract`
- **File:** `docs/decisions.md`
- **Why:** The active rule explicitly rejects a live push channel in favor of an `isLocked`
HTTP-observable flag on list/detail GETs as the substitute. A naive search keying on "poll" alone
might return `spa.playback-troubleshoot-poll` (a different, narrower polling decision for the
troubleshoot screen) instead of the general async-op contract that actually governs new
queue-triggering endpoints.
## 2. Exact code/path lookup
**Q3.** "Where do new API response DTOs live, and what nullable pragma convention do they follow?"
- **Expected key:** `api.response-dtos`
- **File:** `docs/decisions.md`
- **Why:** Direct hit — rule gives the exact path pattern
(`ErsatzTV.Core/Api/<Domain>/*ResponseModel.cs`) and the file-scoped `#nullable enable` convention.
No plausible superseded alternative exists for this key.
**Q4.** "Where does the `EntityLocker` implementation live, and how does it avoid a torn
check-then-set race on its lock flags?"
- **Expected key:** `locking.entitylocker-atomic-flags`
- **File:** `docs/decisions.md`
- **Why:** Rule + Mechanics cite `ErsatzTV.Infrastructure/Locking/EntityLocker.cs` and the
`Interlocked.CompareExchange`-guarded atomic-flag fix directly; no other record touches this file.
## 3. Active-vs-superseded (selecting the archived record here is a FAILURE)
**Q5** *(verbatim, coordinated with server-management#642).* "Is #390's small-lane CI move current?"
- **Expected answer:** **No** — superseded by `ci.runner-placement`.
- **Expected key:** `ci.runner-placement`
- **File:** `docs/decisions.md`
- **Why:** #390 moved `docker build` jobs onto the `small` runner lane to dodge queue time; #406
reversed that (worst-case memory, not runtime, was the real constraint) and the durable rule now
lives under `ci.runner-placement` (and its sibling `ci.small-lane-git-only`, which defines
`small` by job *kind*, not usual runtime). #390 itself was never given its own `##` record (it's
prose-only, referenced inside `ci.runner-placement`'s own Signals line) — answering as if #390's
move still holds, or citing #390 as a standalone current decision, is the failure mode this
question targets.
**Q6.** "Is `docs/decisions.md` still kept append-only by a line-deletion-diff CI/hook guard?"
- **Expected answer:** **No** — that mechanism is superseded.
- **Expected key:** `docs.decision-lifecycle`
- **File:** `docs/decisions.md`
- **Superseded record a naive search might return instead:** `docs.append-only-guard`
(`docs/decisions/archive/release-ci-governance.md`, `status: superseded`,
`superseded-by: docs.decision-lifecycle@2026-07-21`)
- **Why:** `docs.append-only-guard`'s prose (numstat-deleted-count hook + CI job) still reads as a
perfectly good, on-topic answer to "is decisions.md append-only" — that's exactly the trap: the
*mechanism* it describes was replaced by the lifecycle-schema validator
(`scripts/decisions_validate.py`) that this migration introduced. Selecting the archived record
instead of following its `superseded-by` pointer to `docs.decision-lifecycle` is the scored
failure.
**Q6b.** "What is the current source of live queue state at session start?"
- **Expected answer:** Two concurrent tracks — Orientation (docs/README.md task-signal map → the
active decisions catalog) and, only when no issue is named, Selection
(`scripts/select-queue.sh N`, a deterministic query over live Gitea state). Selecting the archived
`docs.queue-state-gitea-tracker` ("queue state lives in pinned tracker issue #237") is a **FAILURE**
#237 closed 2026-07-13 and is now a single archival breadcrumb that MUST NOT be read for live
state.
- **Expected key:** `startup.parallel-orientation`
- **File:** `docs/decisions.md`
- **Superseded record a naive search might return instead:** `docs.queue-state-gitea-tracker`
(`docs/decisions/archive/startup.md`, `status: superseded`,
`superseded-by: startup.parallel-orientation@2026-07-21`)
- **Why:** `docs.queue-state-gitea-tracker`'s prose ("Pinned tracker issue #237 holds the goal +
ordered arc") still reads as a plausible, on-topic answer — that's the trap: the arc completed and
#237 closed, so `scripts/select-queue.sh` (2026-07-19) replaced the *mechanical* selection logic
with live Gitea queries, and #520 formalized the two-track protocol that retires #237 entirely.
Following `docs.queue-state-gitea-tracker`'s own `superseded-by` pointer to
`startup.parallel-orientation` (2026-07-21) rather than citing the archived record as current is
the scored behavior.
## 4. Retired feature
No `status: retired` record exists yet anywhere in the migrated corpus (confirmed via
`grep -rn "status: retired" docs/decisions.md docs/decisions/*.md docs/decisions/archive/*.md`
zero hits). Using the one available superseded record as the closest analog instead:
**Q7.** "Has the Husky `commit-msg` decisions-guard hook (the one that blocks any line-deletion diff
to `decisions.md`) been removed now that decisions carry a lifecycle schema?"
- **Expected answer:** Superseded, not simply "removed" — the append-only-by-diff mechanism is
replaced by `scripts/decisions_validate.py`'s lifecycle-field checks; the sibling H3
root-screenshot guard from the same original record was split out and is **still active** today.
- **Expected key:** `docs.decision-lifecycle` (supersedes `docs.append-only-guard`); sibling active
key `ci.root-screenshot-guard` (`docs/decisions/release-ci-governance.md`) must NOT be reported as
superseded — it's a distinct, still-live record split from the same legacy heading.
- **Why:** Tests that an agent doesn't over-generalize "the old #303 H9/H3 heading was archived" into
wrongly retiring the H3 half too — the migration-map explicitly documents the split.
## 5. Rationale / rejected-alternative
**Q8.** "Why doesn't `EntityLocker` use owner tokens or lease objects instead of plain unlock calls?"
- **Expected key:** `locking.entitylocker-atomic-flags`
- **File:** `docs/decisions.md`
- **Why:** The record's body explicitly documents the single-owner-release discipline as the chosen
design and that `Unlock*` on an already-unlocked slot returns `false` + logs a Warning rather than
throwing — the rejected alternative (owner tokens/leases) is named in the Rule line itself.
**Q9.** "Why did the project reject a CI build-once shared-compile-artifact approach?"
- **Expected key:** `ci.build-once-rejected`
- **File:** `docs/decisions.md`
- **Why:** Rule states it plainly: measured and rejected for a 40-85% wall-clock regression, keeping
the #420 cross-run tree-identity skip instead. A search for "CI build speed" alone might surface
`ci.docs-only-skip-steps` or `ci.peak-anon-measurement` (both real, both wrong for "why was
build-once rejected").
## 6. Convention already implemented — do NOT reimplement
**Q10.** "We need seasonal/holiday scheduling (different programming around Christmas, say) — should
I design a new date-conditional scheduling feature?"
- **Expected key:** `sched.seasonal-scheduling-existing`
- **File:** `docs/decisions.md`
- **Why:** Already ships via `IAlternateScheduleItem` (Classic `ProgramScheduleAlternate`, Block
`PlayoutTemplate`) evaluated by `AlternateScheduleSelector.GetScheduleForDate`; #73 was closed as
already-implemented with a docs-only recipe added. Building a new feature here duplicates existing,
shipped functionality — the correct answer is "use alternate schedules," not a design doc.
**Q11.** "Should I add a way to pad content out to the next clock boundary (e.g. keep a channel's
7:00 PM start exact) per channel?"
- **Expected key:** `sched.clock-padding-existing`
- **File:** `docs/decisions.md`
- **Why:** Already exists via `FillerPreset`'s `FillerMode.Pad` (Classic) and
`pad_to_next`/`pad_until` (Sequential/YAML); #77 was closed as verified+documented, not built new.
Only the one-click per-channel UI toggle is deferred (behind the #388 design-system epic) — the
underlying capability is not missing.
**Q12.** "Do we need to build a fair-share / weighted rotation mode for multi-collections, or does
`ShuffleInOrder` already cover that?"
- **Expected key:** `sched.weighted-shuffle`
- **File:** `docs/decisions.md`
- **Why:** This one cuts the other way on purpose — `ShuffleInOrder` only anti-clumps (its padding
spacers emit nothing), so it does NOT already cover fair-share; the correct answer is that a new
`PlaybackOrder.WeightedShuffle = 9` order was deliberately added rather than retrofitting
`ShuffleInOrder`. Tests that "already exists, don't rebuild" isn't over-applied to a
superficially-similar existing feature that actually doesn't do the job.
---
## Class coverage summary
| # | Class | Key(s) |
| - | ----- | ---- |
| Q1 | paraphrased-discovery | `api.mediatr-passthrough` |
| Q2 | paraphrased-discovery | `api.async-op-contract` |
| Q3 | exact-lookup | `api.response-dtos` |
| Q4 | exact-lookup | `locking.entitylocker-atomic-flags` |
| Q5 | active-vs-superseded (verbatim #390) | `ci.runner-placement` |
| Q6 | active-vs-superseded | `docs.decision-lifecycle` (vs archived `docs.append-only-guard`) |
| Q6b | active-vs-superseded | `startup.parallel-orientation` (vs archived `docs.queue-state-gitea-tracker`) |
| Q7 | retired-feature (no `retired` record exists; superseded used as analog) | `docs.decision-lifecycle` / `ci.root-screenshot-guard` (must stay active) |
| Q8 | rationale/rejected-alternative | `locking.entitylocker-atomic-flags` |
| Q9 | rationale/rejected-alternative | `ci.build-once-rejected` |
| Q10 | convention-already-implemented | `sched.seasonal-scheduling-existing` |
| Q11 | convention-already-implemented | `sched.clock-padding-existing` |
| Q12 | convention-already-implemented (negative control) | `sched.weighted-shuffle` |
13 scored questions (Q1Q12 plus Q6b) across all six required classes.
+12
View File
@@ -18,6 +18,10 @@ Issue trail: epic #243 — phase 1 #244 (Channels), phase 2 #245 (Playouts), pha
---
## 2026-07-11 — Channels screen extraction (#244): single-file screen, no sibling helper dir (epic #243 phase 1)
`key: spa.channels-screen-extraction` · `status: active` · `since: 2026-07-11` · `supersedes: none` · `superseded-by: none`
**Rule:** The Channels domain is a single-file zero-prop screen (`web/src/screens/ChannelsScreen.tsx`) with a colocated test file and no sibling helper directory, since its pure logic is too small (~30 lines) to justify a separate business-rule layer like Schedules' `itemRules.ts`.
**Signals:** screen extraction, single-file screen, no helper dir · paths: `web/src/screens/ChannelsScreen.tsx` · issues: #244, #243, #238
**Mechanics:** `docs/spa-conventions.md` §6
First bounded extraction under the App.tsx modularization epic (#243): the Channels domain moved
verbatim out of `web/src/App.tsx` into `web/src/screens/ChannelsScreen.tsx` (zero-prop, self-sufficient,
@@ -44,6 +48,10 @@ dispatch is #238's owned bug and out of scope for a behavior-preserving extracti
is deferred to #247 (epic phase 4).
## 2026-07-14 — Playouts screen extraction (#245): screen-owned route wrapper (epic #243 phase 2)
`key: spa.playouts-screen-extraction` · `status: active` · `since: 2026-07-14` · `supersedes: none` · `superseded-by: none`
**Rule:** The Playouts domain (including its unguarded `PlayoutsRouteScreen` route wrapper with local pathname/popstate state) moved as one unit into `web/src/screens/PlayoutsScreen.tsx`, keeping its screen-specific sub-path route ownership colocated with the base screen; a pure structural move with no API/route/CSS/behavior change.
**Signals:** screen extraction, route wrapper ownership, sub-path routing · paths: `web/src/screens/PlayoutsScreen.tsx` · issues: #245, #243
**Mechanics:** `docs/spa-conventions.md` §2
Second bounded extraction under the App.tsx modularization epic (#243): the Playouts domain moved from
`web/src/App.tsx` into `web/src/screens/PlayoutsScreen.tsx`, including its loading/error/empty states,
@@ -64,6 +72,10 @@ zero-playout Add Playout affordance, lock/409 handling, refresh/poll ownership,
dialog flows. Refs #245 #243.
## 2026-07-15 — App shell/routing extraction + explicit primary-action ownership (#247)
`key: spa.app-shell-extraction` · `status: active` · `since: 2026-07-15` · `supersedes: none` · `superseded-by: none`
**Rule:** `App.tsx` is only the composition root over `web/src/app/routes.tsx` (stable route-object identity), `app/AppShell.tsx` (shell chrome), and `app/ScreenContent.tsx` (exhaustive screen dispatch); primary actions are one explicit `PrimaryActionProvider` registration per screen, replacing the old global `ctv:primary-action` window event.
**Signals:** app shell extraction, route identity, primary-action ownership · paths: `web/src/app/routes.tsx`, `web/src/app/AppShell.tsx`, `web/src/app/ScreenContent.tsx` · issues: #247, #243, #238, #230
**Mechanics:** `docs/spa-conventions.md` §10; `App.test.tsx`
Final phase of the App.tsx modularization epic (#243). `web/src/App.tsx` is now only the composition
root: it owns `activeRoute`, `currentPathRef`, the `navigate`/`popstate` pair that consults the dirty
+198 -186
View File
@@ -1,22 +1,21 @@
# ChicoryTV issue-queue handoff (client-neutral kickoff + workflow lore)
> **PROTOCOL CHANGED 2026-07-11** (decisions.md entry of same date). Queue state lives in the
> **pinned Gitea tracker [ersatztv#237](http://192.168.1.95:3000/timothy/ersatztv/issues/237)**,
> not in this file. Do **NOT** write session state, queue order, or next-session prompts here —
> sessions ending under the old protocol should append their session comment to #237 instead.
> This file holds only stable operating guidance: the standing kickoff prompt and the workflow lore.
> Historical per-session state: `git log` of this file (last state-bearing revision: 8b77d5e7).
Static kickoff prompt + workflow lore, pruned (not appended) as protocols change — git history has
the rest. **Queue state is live Gitea state**, resolved fresh every session by
`scripts/select-queue.sh`; never written here as a snapshot, never read from stale prose or a
closed tracker.
<!-- archival:237 --> ersatztv#237 was the pinned pickup tracker through 2026-07-21; it is CLOSED
and now purely historical (superseded by `startup.parallel-orientation` in `docs/decisions.md`; full
history in `docs/decisions/archive/startup.md`). Do not read it for current queue state.
---
## Current phase — maintenance / backlog mode (terminal transition, 2026-07-13)
The ChicoryTV rewrite + go-live arc is **COMPLETE**: the pinned arc tracker
[ersatztv#237](http://192.168.1.95:3000/timothy/ersatztv/issues/237) is **CLOSED** and all six
numbered arc items shipped. The project is now in **maintenance / backlog mode**. This is a
*terminal* fact (the arc will not re-open), so it belongs in this standing file — unlike any
"current frontier issue", which must never be written here (read it live; see the queue-drift
lesson at the end of this file).
The ChicoryTV rewrite + go-live arc is **COMPLETE** and closed out. The project is now in
**maintenance / backlog mode**. This is a *terminal* fact (the arc will not re-open), so it belongs
in this standing file — unlike any "current frontier issue", which must never be written here (read
it live; see the queue-drift lesson in the archived section below).
So a fresh session stops re-discovering this every time:
- The **arc / gate / open-`review` tiers are normally EMPTY now.** An empty arc frontier is the
@@ -100,43 +99,44 @@ already-collected bounded evidence for explicitly tool-free synthesis or report
silently expanding orchestrator reconnaissance. Global Codex hook enforcement is tracked in
`timothy/server-management#592`; the wider Claude-hook port is tracked in `timothy/server-management#593`.
**When the user has not named an issue, queue selection is mechanical fast/small work, never orchestrator
work.** FIRST run the deterministic selector — `ETV_GITEA_BASICAUTH=user:pass scripts/select-queue.sh [N]`
— which does the parts a cheap model kept getting wrong IN CODE: it excludes `in-progress`/`parked`/PRs,
resolves `GET /issues/{n}/dependencies` on every candidate (dropping any with an OPEN blocker), tiers by
LOCAL `.milestone.state`/`review`/`priority:` filters (never the server `?milestones=` name filter, which
no-ops on `:`/`+`), and orders by (tier, priority, issue#). **Its DEPS/tiering/ordering are deterministic —
trust them, do not re-derive them by hand.** It also raises two JUDGMENT flags it deliberately does not
decide — `CLAIM?` (a claim can precede the label) and `UMBRELLA?` (an epic whose children are the real
pickups); the orchestrator (or a cheap model) resolves ONLY those by reading the flagged issue's
comments/body. The script does not rank the arc tier (arc order is #237 prose, currently CLOSED) — read
#237 if an arc re-opens. If the script is unavailable (no creds / Gitea down), fall back to the cheap-model
path below. Dispatch exactly one selector on the cheapest suitable model at `low` effort. Give it #237's
body and last ~6 comments, then have it query live candidates as a cascade in this exact tier order:
**arc → OPEN issues assigned to OPEN milestones → open `review` → unmilestoned/unreviewed
`priority: high``priority: medium``priority: low`**. Every candidate row represents an issue
(or eligible reviewer audit): milestone records provide tier metadata and are NEVER pickup candidates.
An OPEN milestone with zero eligible OPEN issues contributes zero candidates. Deduplicate candidates,
apply the live eligibility exclusions at each tier (including automatic exclusion of `parked` issues and any issue with an open `blocked-by` dependency),
and stop as soon as five eligible issues have been accumulated (or all tiers are exhausted). It returns
only that ranked top-five shortlist, with one-line rationales and live-state evidence; a small result
packet is the desired behavior, but an empty or undersized higher tier must fall through to the next tier.
The orchestrator receives only that compact packet, then performs a focused live recheck of the winner
before claiming. If the cheap worker lacks a repository-scoped issue-list tool, it must immediately return
the literal result `TOOL_LIMITATION`; it must not rank issue IDs mentioned in tracker prose, turn milestone
records into candidates, or infer that any tier is empty. The orchestrator then performs only the mechanical
tier queries (repository + state + milestone/label), passes at most the bounded raw ISSUE rows back to the
cheap worker, and leaves all filtering/ranking to that worker. Never substitute an owner-wide/global search
or treat that tool limitation as an empty tier.
Do not load implementation docs or issue bodies into the selector. If the client cannot route a
cheaper subagent, run the selector in a separate low-cost session before starting or resuming the orchestrator
and pass in its packet. **Do not fall back to inline sorting or an equally expensive selector.** If no cheaper
route or session is available, pause and request the selector packet rather than consuming orchestrator tokens
on queue ranking. If the user names an issue, skip selection and only verify that issue's live claimability;
this explicit user choice is the sole path by which a `parked` issue may be worked.
## Two concurrent tracks at session start
FIRST read `AGENTS.md` and `CLAUDE.md` when present, then `docs/README.md`, the convention docs it
indexes, and the Lessons below. Apply both client instruction files; where they differ, follow the
A fresh session runs **two independent tracks** — neither blocks the other, and both are cheap/mechanical,
never orchestrator-tier work:
- **Track A — Orientation** (always, regardless of whether an issue is named): read `AGENTS.md` and
`CLAUDE.md` when present, then `docs/README.md`'s task-signal map, and the active decisions catalog
`docs/decisions/README.md`. This is what replaces re-deriving conventions from source or from a
pinned tracker's prose — see "Knowledge retrieval" below for the full catalog-first / breadcrumb-rule
contract.
- **Track B — Selection** (only when the user has NOT named an issue): run the deterministic selector,
`ETV_GITEA_BASICAUTH=user:pass scripts/select-queue.sh [N]`. It does the mechanical parts a cheap
model used to get wrong IN CODE: excludes `in-progress`/`parked`/PRs, resolves
`GET /issues/{n}/dependencies` on every candidate (dropping any with an OPEN blocker), tiers by
LOCAL `.milestone.state`/`review`/`priority:` filters (never the server `?milestones=` name filter,
which no-ops on `:`/`+`), and orders by (tier, priority, issue#). **Its DEPS/tiering/ordering are
deterministic — trust them, do not re-derive them by hand** (see the archived section below for why
that used to be necessary and isn't anymore). It also raises two JUDGMENT flags it deliberately does
not decide — `CLAIM?` (a claim can precede the label) and `UMBRELLA?` (an epic whose children are the
real pickups) — resolve ONLY those by reading the flagged issue's comments/body. The script does not
rank an arc tier (there is no active arc right now — maintenance/backlog mode, see above); if a new
arc/milestone is ever established, add it as a script tier rather than reverting to prose-derived
ranking. If the script is unavailable (no creds / Gitea down), it fails open (prints a notice, exits
0) — in that case fall back to a single low-cost model session querying live tiers directly
(OPEN-milestone → `review``priority: high/medium/low`), never to inline sorting by the
orchestrator.
**If the user names an issue, skip Track B entirely** and go straight to focused retrieval (below)
plus a live claimability check on that issue; this explicit user choice is the sole path by which a
`parked` issue may be worked.
Dispatch exactly one selector on the cheapest suitable model at `low` effort when the client can route
a cheaper subagent; otherwise run it in a separate low-cost session before starting or resuming the
orchestrator and pass in its packet. **Do not fall back to inline sorting or an equally expensive
selector.** If no cheaper route or session is available, pause and request the selector packet rather
than consuming orchestrator tokens on queue ranking.
FIRST read `AGENTS.md` and `CLAUDE.md` when present, then `docs/README.md` and the sections its
task-signal map points to for your task, plus the Lessons below. Apply both client instruction files; where they differ, follow the
stricter safety/completion requirement unless a higher-priority instruction resolves the conflict.
FRONTIER ESCALATION — these moments go to the strongest available reasoning model. In Claude Code,
@@ -154,75 +154,101 @@ the client cannot route the escalation. Do not block merely because one brand-sp
Everything else (claiming, worktrees, dispatching implementers, CI monitoring, protocol
bookkeeping, routine merges of green reviewed PRs with user consent) stays at your level.
## Knowledge retrieval (MemPalace + catalog + Gitea — the #642 seam)
**MemPalace = candidate discovery only**, retrieved at a bounded `k`; every passage **verified
against its cited Markdown/Gitea source** before use. **Default wing = `ErsatzTV-Decisions`** (active
`docs/decisions.md` + `docs/decisions/*.md` + catalog). History wings `ErsatzTV-Decisions-Archive`
(`docs/decisions/archive/**`) and `Gitea-ErsatzTV` (issues/comments incl. closing records) are touched
**only** when the question is explicitly "what did the rule *used to be*." "What is the current rule
for X" never touches the history wings.
The four load-bearing orientation bullets — carry these **VERBATIM** wherever this contract is
referenced (`docs/README.md` too):
1. **Current conventions/decisions → catalog-first.** Start at `docs/decisions/README.md`; discover
via the `ErsatzTV-Decisions` wing (active) / `ErsatzTV-Decisions-Archive` (superseded/retired).
**Resolve by topic/key, never by chasing a file path.**
2. **Issue history → evidence, not authority.** The `Gitea-ErsatzTV` wing is historical narrative
that may be stale; it never overrides current Markdown.
3. **The breadcrumb rule (the crux behavior change).** A file path named inside a *historical issue
comment* (e.g. "grep `docs/decisions.md` 2026-07-17", "see …") is a **breadcrumb, not a live
pointer.** Find the current rule via the catalog / active wing **by concept**; do not treat the
named path as current. (Why it's safe: still-current → in the active wing, breadcrumb resolves;
superseded → the active wing returns the *successor* and a literal follow lands on a record that
announces its own `status: superseded`; retired → the active wing returns nothing, which is itself
the signal. The validator-enforced move-to-`archive/` is what prevents the catastrophic "superseded
rule read as current" case.)
4. **Fallback when MemPalace is stale/down:** `docs/decisions/README.md` catalog, then
`` rg '^`key: <dotted.key>`' docs/decisions/ ``. MemPalace is never authority nor sole fallback.
**Graceful degradation (calibration, not a correctness cliff):** even an under-oriented agent that
literally greps `docs/decisions.md` post-#521 gets valid-but-*incomplete* results (misses topic files
+ archived records), never *wrong* ones — that file holds only active records. So this is a
completeness/latency risk during rollout, not a correctness cliff.
**Altitude / precedence:** `docs/decisions.md` (+ topic files) is **normative/current** ("the rule is
X") — the `ErsatzTV-Decisions` wing, authority for "what is the current rule." Gitea issues are
**evidentiary/historical** ("we worked X on date D; spec, discussion, outcome") — the `Gitea-ErsatzTV`
wing, authority for "how was this handled / what was the context." A decision record points to its
issue; the issue carries the provenance the record compresses. Same fact, two altitudes — canonical
Markdown wins for "current."
**Issues in MemPalace are for DISCOVERY, not queue state.** Encounter a problem → search MemPalace →
find relevant docs AND relevant open issue(s): this **avoids filing duplicates** and **surfaces
bundles** (related issues to pick up together). But a MemPalace issue hit is a *candidate*:
open/closed/claimed/blocked/priority flip constantly and MemPalace lags (seconds1h, up to a week if
the webhook is down), so a hit may be a since-closed issue shown open, or miss a just-filed one.
**Always re-confirm live state in Gitea (`scripts/select-queue.sh`) before acting** — never treat a
MemPalace issue hit as current queue state.
**What to mine per issue:** open issue → the BODY (dedup + bundle-clustering; no closing record yet).
Closed issue → body + the `## Closing record` (how it was resolved). Ephemeral comments (claim /
progress / `Review-verdict:`) → skip or de-weight both ways — process exhaust, not the knowledge
store. The `## Closing record` is the one structured per-issue summary meant for retrieval + export.
**Staleness bounds** (so agents know when to distrust a hit): webhook re-mine in seconds; hourly
reconcile; weekly full sweep catches file moves/deletes. Worst case before a supersession takes effect
in retrieval: ~1h, or ~1 week if the webhook is down and only the move happened. When in doubt, fall
back to exact search (bullet 4 above).
**Never** derive live queue state from MemPalace, #237, or historical comments.
Then work the queue:
1. Unless the user named an issue, dispatch or obtain the mandatory low-cost selector packet. The
**selector**, not the orchestrator, reads the pinned tracker **ersatztv#237** — body = goal + ordered
arc + session protocol — and its **last ~6 session comments** (newest-first; the full comment
payload is large, so stop at ~6).
**SOURCE OF TRUTH = live Gitea issue state, NEVER the prose.** The arc body carries ORDER + goal
only; a session comment's "Recommended next" is a forward *guess* written before the next session
acted. Both go stale the instant an item closes (especially under parallel sessions narrating each
other's work as "the gate that unblocks X"). So derive the candidate set from LIVE state, not from
any inline marker or a prior comment's "next": the **current gate = the lowest-numbered OPEN arc
item in #237's arc list**; its open children are the gate cluster (query them by the `review`
label). Cross-check every arc / "recommended" item's real open/closed state (issue **and**
milestone) before trusting it — do NOT hardcode which issue is the frontier; read it. Candidate
discovery is a bounded cascade, not a full inventory: query **arc → OPEN issues assigned to OPEN
milestones → open `review` → unmilestoned/unreviewed `priority: high``priority: medium`
`priority: low`**, carrying eligible unique results forward until the shortlist contains five issues
or all tiers are exhausted. Milestone records are tier metadata, NEVER pickup candidates; an OPEN
milestone with zero eligible OPEN issues contributes zero candidates.
This priority-label fall-through is mandatory: an empty arc/milestone/review pool is never evidence
that the queue is empty. Each priority tier queries all OPEN `timothy/ersatztv` issues carrying that
label, not only IDs mentioned in #237 or recent comments, and excludes pull requests. Treat an
umbrella/epic as a container rather than a pickup when #237 names its eligible children. In the
review tier, ALSO list
open `ersatztv`-labeled issues in **timothy/adversarial-reviewer** — unclaimed audits there are
pickup candidates too (read-only, parallel-safe; see the tracker's "Pending adversarial reviews"
section). Reviewer audits are claimed by comment: a claim remains active until a later comment
explicitly releases or abandons it, and a posted audit/review deliverable is completed work even
when its issue stays open for implementer replies. If the Gitea MCP is down, hit the REST API
directly (use credentials supplied by the environment or your global client instructions; never
print them):
`curl -u <user>:<pass> http://192.168.1.95:3000/api/v1/repos/timothy/ersatztv/issues/237`.
**The authoritative pickup + ranking protocol lives in #237's "Session protocol" section — this is a summary; if the two ever disagree, #237 wins.**
2. The selector returns the highest-ranked eligible candidate plus up to four fallbacks. Apply this
strict tier order: **(1)** open arc items in #237 arc order; **(2)** OPEN issues assigned to an OPEN
milestone;
**(3)** OPEN `review`-labeled issues and eligible reviewer-repo audits; **(4)** remaining
unmilestoned/unreviewed `priority: high` issues; **(5)** `priority: medium`; **(6)** `priority: low`.
Within milestone and review tiers, order by `priority: high` > `medium` > `low`; for otherwise equal
candidates preserve #237's explicit eligible order, then use lowest issue number. At every tier exclude
pull requests, CLOSED issues, `in-progress` claims, `parked` issues, issues blocked by a CLOSED milestone,
**any issue with an open `blocked-by` dependency** (`GET /issues/{n}/dependencies`; Gitea auto-clears the
block when the blocker closes — do not pick an issue ahead of its open blocker), containers whose children
are the pickups, and reviewer audits that are already claimed or have a posted deliverable. Note the
2026-07-17 convention: `parked` (no concrete plans) issues are **closed**, not left parked, so `parked`
should rarely appear; if it does, treat it as excluded and surface it to the user.
Query the next tier only while fewer than five eligible unique candidates have been collected; do not
enumerate the remainder after the shortlist is full.
The orchestrator then makes one focused Gitea read to confirm the proposed winner is still OPEN,
unclaimed, and not blocked by a closed milestone; if it changed, check the next supplied fallback.
**An empty arc frontier is not a stopping condition.** If the selector returns any eligible
candidate, claim its top-ranked winner; do not ask the user to choose merely because candidates
belong to different workstreams.
Do not reread the full tracker or comments for selection. If the prose says "recommended next / now
unblocked" but the issue (or its milestone) is already CLOSED, it is done — skip it and fix the
stale line in your session comment. Prose lags live state; live state wins; tier + `priority:` labels
decide order, not the prose. Never invent a fix-size, recency, or perceived-relevance tiebreaker.
(This mirrors #237's Session-protocol ranking — #237 is canonical.)
1. Run the two tracks above (orientation always; selection unless an issue is named).
2. If Track B ran: apply its ranked shortlist. The orchestrator makes one focused live recheck of the
winner (still OPEN, unclaimed, not blocked) before claiming; if it changed, check the next fallback.
**An empty backlog is not a stopping condition** — if the selector returns any eligible candidate,
claim its top-ranked winner; do not ask the user to choose merely because candidates belong to
different workstreams. Never invent a fix-size, recency, or perceived-relevance tiebreaker.
3. **Claim it**: add the `in-progress` label + a "claiming" comment on the issue(s);
reviewer-repo audits are claimed by comment only. Treat that claim as live until a later comment
explicitly releases or abandons it, and exclude audits with a posted deliverable even while the
issue remains open for implementer replies.
4. Read the issue bodies (they carry the task context/evidence) and work the item under the
HARD CONSTRAINTS below.
5. Finish by following the session-end protocol in #237: run the **H12 qualification audit**
5. Finish the session: run the **H12 qualification audit**
(`ETV_GITEA_BASICAUTH=user:pass scripts/issue-qualification-audit.sh`) and add a `priority:`
label to anything it lists (every issue you filed this session included); then ONE session
comment on the tracker (template in the tracker body, incl. triage verdicts for any new issues),
remove your `in-progress` labels, and complete the per-issue Task Completion Protocol from the
applicable `AGENTS.md` / `CLAUDE.md` instructions (including the `done` workflow when required).
label to anything it lists (every issue you filed this session included); post a `## Closing
record` (template below) on each issue you closed or substantially progressed; remove your
`in-progress` labels; and complete the per-issue Task Completion Protocol from the applicable
`AGENTS.md` / `CLAUDE.md` instructions (including the `done` workflow when required).
## Closing record (session-end / issue-close artifact)
Post this as the structured closing comment on any issue you close (or substantially progress) — it
is both the human-readable summary and the per-issue unit MemPalace mines for retrieval:
```markdown
## Closing record
**Outcome:** <what shipped / what didn't; PR link>
**Root cause:** <for bug fixes/incidents — why the problem existed, or "unknown, see follow-up #N">
**Decisions/conventions changed:** <keys added/superseded in docs/decisions.md, or "none">
**Reusable knowledge:** <a fact/gotcha worth surfacing to a future session or MemPalace search>
**Verification:** <tests run, live-E2E, CI status>
**Deferred:** <anything explicitly punted, with a follow-up issue link, or "none">
**Docs updated:** <which docs/*.md files changed in this PR, or "none required and why">
```
HARD CONSTRAINTS:
- Work in worktrees off `origin/main` — ALWAYS, including for a one-line docs commit. `/Users/timothy/ersatztv`
@@ -249,7 +275,7 @@ HARD CONSTRAINTS:
must supersede a live run, SAY SO explicitly instead of leaving it burning. (`dispatch_workflow` is
a different route and still works for re-triggering a **main** run.) Corrects the older "superseded
runs drain on their own" note below: they do finish, but they hold a slot while doing it.
- **A lone `decisions.md append-only` red is a KNOWN INFRA FLAKE — do NOTHING** (operator-stated,
- **A lone `decisions lifecycle` red is a KNOWN INFRA FLAKE — do NOTHING** (operator-stated,
2026-07-19). When it is the **only** red: do not investigate it, and do **not** create a new run or
commit to clear it — no rebase, no `--amend`, no no-op push. **The operator reruns that single job
from the Gitea UI.** Report it as a known flake and carry on; only if *other* jobs are red too does
@@ -315,58 +341,19 @@ HARD CONSTRAINTS:
> (+ `api-conventions.md` §7a/b/c for the concurrency/side-effect mechanics, `spa-conventions.md`,
> `e2e-local.md`). Do NOT re-record a code/API/SPA decision here — it will duplicate and drift.
> Pruned 2026-07-12: issue-specific engineering bullets now covered in those docs, and dead
> Blazor-parity process notes, were removed (git history has them).
> Blazor-parity process notes, were removed (git history has them). Pruned 2026-07-21: pre-script
> selector lore and the #237-specific queue-drift narrative moved to the archived section below —
> the mechanism they describe no longer exists.
- READ docs/README.md → the convention docs FIRST; point recon/implementer agents at specific
doc sections. Only recon the task-specific delta.
- **The mechanical selection checks now live in `scripts/select-queue.sh` — run it instead of
re-deriving them by hand (2026-07-19).** The three bullets below are the *evidence* for why the
script exists: a cheap model repeatedly reported blocked issues as `deps:clear`, mis-tiered on the
no-op `?milestones=` filter, and ranked by issue number instead of priority-within-tier — so the
lore grew to "re-derive the whole contested tier every session," which is exactly the tax the
script pays down. The script does the deterministic work in code (dependency exclusion via
- **The mechanical selection checks live in `scripts/select-queue.sh` — run it instead of
re-deriving them by hand.** It does the deterministic work in code (dependency exclusion via
`GET /dependencies`, LOCAL `.milestone.state`/`review`/`priority:` tiering, (tier, priority, issue#)
ordering, `in-progress`/`parked`/PR exclusion) and only *flags* `CLAIM?`/`UMBRELLA?` for judgment.
**Trust its DEPS/tiering/ordering; do the focused recheck only on the flagged rows.** Keep the
evidence bullets below (they explain the failure modes the script encodes), but the *prescription*
is now "run the script," not "hand-re-derive." If you extend the tiers/filters, update the script
first, then these bullets.
- **Gitea milestone-name issue filter silently NO-OPS on names with special chars (2026-07-17,
#77 selection)**: `GET /issues?milestones=<name>` returns the WHOLE open-issue list — not a
filtered set — when the milestone title contains `:` or `+` (e.g. `Scheduling: refactor +
distribution`), because the raw name doesn't round-trip through Gitea's filter. A selector that
trusts that response will mis-tier issues (it made #72 look like a milestone-10 member when it's
unmilestoned, and hid #77's true sibling set). **Robust recipe: fetch all open issues once and
filter LOCALLY on each issue's `.milestone.title`** (`?state=open&type=issues&limit=50`, then
`[i for i in issues if (i['milestone'] or {}).get('title')==NAME]`) — do not rely on the
server-side `milestones=` name filter. If a count looks suspiciously like "all open issues," that
filter silently failed; re-derive membership locally. (Dependencies API is fine: `POST
/issues/{n}/dependencies` with `{"owner","repo","index"}` sets blocked-by; the bare `{index}`
form 201s but may not attach — verify with the GET.)
- **Confirmed again + widened (2026-07-17, #72/#164 selection): treat EVERY mechanical claim in the
selector packet as unverified — not just deps and priority.** That run's dependency data was
correct, yet it still ranked **#383 first while missing that #384 was `in-progress`** (claimed 40
minutes earlier) **and that #383 is that bundle's umbrella** (its body: "Follow-up arc to #69" with
#384#386 as deferred sub-work). Claiming its #1 pick would have collided head-on with a live
parallel session. The packet does not reliably see: `blocked-by`, priority-within-tier,
**`in-progress` claim state**, or umbrella-vs-child. Before claiming, check the candidate's **last
comments** for a live claim (a claim stays live until explicitly released) and read its **body** for
umbrella framing. An empty-ish tier is normal in maintenance mode — three of the four open
milestone issues that day were legitimately blocked or claimed, which is the expected steady state,
not a reason to relax the checks.
- **The cheap selector's dependency + priority data is NOT trustworthy — re-derive the whole contested tier,
not just the winner (2026-07-17, #73 selection)**: the selector packet reported **#385/#386 as `deps:clear`
when both were blocked** (#385 by open #70; #386 by open #385+#384) — it never called
`GET /issues/{n}/dependencies` — and **missed #383 entirely** (the only `priority: medium` in tier 1),
ranking by issue number and ignoring the priority-within-tier rule. Either error alone produces a bad
pickup: working a blocked issue, or skipping the top-priority one. The kickoff's "orchestrator makes one
focused recheck of the **winner**" is **insufficient** whenever a tier has >1 candidate — a wrong winner is
invisible to a winner-only recheck. **Recipe: fetch all open issues once, filter locally on
`.milestone.title`/labels (the `?milestones=` filter no-ops on `:`/`+` names — see the bullet above; both
current open milestones have them), then run `GET /issues/{n}/dependencies` yourself on every candidate you'd
plausibly claim.** That's ~3 cheap calls and it is the only thing standing between the packet and a wasted
session. Keep delegating selection (it's still not orchestrator work) — just verify its two mechanical
outputs (deps, priority order) rather than its ranking prose.
**Trust its DEPS/tiering/ordering; do the focused recheck only on the flagged rows.** If you extend
the tiers/filters, update the script first, then these bullets. (See the archived section below for
the pre-script failure modes that motivated writing it, if you need the history.)
- **Codex cheap-worker launch (tested 2026-07-14)**: the native `spawn_agent` interface currently
has no model/effort selector, so it provides parallelism but not cost savings. For bounded,
tool-bearing selector/recon work, launch a separate worker with
@@ -437,7 +424,7 @@ HARD CONSTRAINTS:
refs/heads/v4: …` is act refreshing its `/root/.cache/act` action cache and is followed by `Cloned …`
— it is NOISE, not a cause. Grep for the failure marker, not for the word "error".
- **`cancelled` is NOT `failure` — never read a cancel as a CI verdict** (2026-07-17, operator-surfaced).
The operator cancels runs by hand (they're the only one who can — see the batching constraint), and a
The operator cancels runs by hand (they're the only one who can — see the batching constraint above), and a
run-level `conclusion: cancelled` means **no verdict**, not a pass and not a fail. Two traps: (1) a
run whose overall state is `failure` may hold a *genuine* job failure that happened **before** the
cancel — check job-level `conclusion` + timestamps, don't attribute it to the cancel; (2) a
@@ -467,16 +454,15 @@ HARD CONSTRAINTS:
(tiny ffmpeg testsrc MKVs + `LibraryPath` SQL rows + scan). Live-E2E is a **stated requirement**
for write-path handler changes — see that doc's "When live-E2E is required" + the decisions.md
entry; it's the only net for the lazy-enumeration / reload-through-read-path 500 class (#229).
- **Parallel sessions (2026-07-11 protocol)**: claim before working (`in-progress` label — the
tiny read→claim race window is accepted; later claimant backs off). Claiming prevents
duplicate pickup, NOT overlapping code changes — check the tracker's dependency notes
("#234 after #231", "coordinate with #215") before touching shared surfaces. Editing THIS
lore: prune covered/stale bullets too (not append-only — git keeps history), `git pull --rebase`
before commit. Two runners (ci-runner VM 127 + bumblebee-runner, 4 slots); **no cancel route exists
on this Gitea version — verified 1.25.4: REST and MCP `cancel_run` both 404** (see the batching
constraint above). Superseded runs do finish on their own, but they hold a runner slot while doing
it, so with 4 parallel sessions they are a real cost — batch pushes rather than relying on them to
drain.
- **Parallel sessions**: claim before working (`in-progress` label — the tiny read→claim race window
is accepted; later claimant backs off). Claiming prevents duplicate pickup, NOT overlapping code
changes — check the issue's dependency notes ("#234 after #231", "coordinate with #215") before
touching shared surfaces. Editing THIS lore: prune covered/stale bullets too (not append-only — git
keeps history), `git pull --rebase` before commit. Two runners (ci-runner VM 127 + bumblebee-runner,
4 slots); **no cancel route exists on this Gitea version — verified 1.25.4: REST and MCP
`cancel_run` both 404** (see the batching constraint above). Superseded runs do finish on their own,
but they hold a runner slot while doing it, so with 4 parallel sessions they are a real cost — batch
pushes rather than relying on them to drain.
- **Two sessions touching one machine**: a branch may be checked out in ANOTHER session's
worktree — never commit/merge inside a worktree you didn't create. To land a merge on such a
branch without touching their checkout: plumbing merge (`git read-tree -m base ours theirs`
@@ -510,7 +496,7 @@ HARD CONSTRAINTS:
them plausibly-but-wrong and `npm run check:api` is the guard. Escape hatch for a deliberate
non-rebased push: `ETV_SKIP_REBASE_CHECK=1 git push`.
- **H12 issue-qualification audit** (ersatztv#312, `scripts/issue-qualification-audit.sh`): a
session-end check that lists OPEN issues missing a `priority:` label — the #237 ranking keys off
session-end check that lists OPEN issues missing a `priority:` label — the queue tiering keys off
`priority:`/gate labels, so an unlabeled issue is invisible to it. "Fully qualified" = has a
`priority: {high,medium,low}` label (that signals triage ran; gate-vs-backlog is then derivable
from the `review` label / milestone, and a milestone is NOT required — backlog is unmilestoned).
@@ -582,20 +568,46 @@ HARD CONSTRAINTS:
`CancellationTokenSource.CancelAfter(timeout)` into BOTH `SendAsync` and the stream reads +
`HttpClient.Timeout = InfiniteTimeSpan`; catch transport/timeout exceptions and turn them into a response.
(Not yet in decisions.md — #289 landed on PR#76, not main.)
- **Queue-drift root cause + standing rule (2026-07-12, user-surfaced)**: pickups repeatedly re-picked
already-done work — #91b was framed "recommended next / now unblocked" for two sessions *after* it had
merged (2026-07-11), and the #251/#252 priority-pickups sat listed "open" after closing — requiring a
reactive body correction (tracker comment 16:00). ROOT CAUSE: DONE/OPEN status was read from **prose** (the
arc body's inline "DONE" markers + each session comment's "Recommended next"), which is append-only and
hand-edited, so it lags real issue state — worst under parallel sessions that narrate each other's merges as
"the gate that unblocks X" (#271 got cast as the last gate for an already-merged #91b). It is a *structural*
bug, not a stale-writer bug: any status embedded in prose will drift. STANDING FIX (don't just re-patch the
body next time): **live Gitea state is the ONLY source of truth for status.** The arc body carries order+goal;
a comment's "Recommended next" is a *candidate* that MUST be re-verified OPEN (issue **and** milestone) at
pickup. When prose disagrees with live state, live state wins — correct the prose in your session comment,
never propagate it. STRUCTURAL CURE (2026-07-12 review, Fable): #237's arc no longer carries inline `DONE`
markers — closed items move to a "Done (history)" section, so status lives ONLY in live Gitea state and can't
drift; and the gate/frontier is defined **structurally** (lowest-numbered open arc item), never hardcoded to
an issue number. Beware: the *first* pass at this fix re-planted the very bug by hardcoding "#197 cluster" /
"#91b milestone CLOSED" into the kickoff — if you name today's frontier issue in this standing file, you are
writing the next drift. Gate cluster = open `review`-labeled issues serving the current open arc item.
---
## Archived — pre-script selector history (do not follow)
> This section is history, not instruction. It explains why `scripts/select-queue.sh` exists and
> why the closed #237 tracker used to be the queue-state store; none of it is a live protocol. The
> current protocol is "Two concurrent tracks at session start" above.
- **(Pre-2026-07-19) Before the selector script existed**, a cheap model was dispatched each session
to rank the backlog by reading tracker prose directly, and the orchestrator was told to RE-DERIVE
its mechanical claims because the model kept getting them wrong: it reported blocked issues as
`deps:clear` (never called `/dependencies`), mis-tiered issues when the server-side `?milestones=`
filter silently no-ops on `:`/`+` names, ranked by issue number and ignored priority-within-tier,
and missed live `in-progress` claims. `scripts/select-queue.sh` was written to do the mechanical
parts deterministically in code instead — the evidence bullets below are kept for context, not
because you need to re-derive anything by hand anymore.
- **Gitea milestone-name issue filter silently NO-OPS on names with special chars (2026-07-17,
#77 selection)**: `GET /issues?milestones=<name>` returns the WHOLE open-issue list — not a
filtered set — when the milestone title contains `:` or `+` (e.g. `Scheduling: refactor +
distribution`), because the raw name doesn't round-trip through Gitea's filter. A selector that
trusts that response will mis-tier issues (it made #72 look like a milestone-10 member when it's
unmilestoned, and hid #77's true sibling set). Robust recipe (now encoded in the script): fetch
all open issues once and filter LOCALLY on each issue's `.milestone.title`.
- **Confirmed again + widened (2026-07-17, #72/#164 selection): the packet does not reliably see
`blocked-by`, priority-within-tier, `in-progress` claim state, or umbrella-vs-child** — that run's
dependency data was correct, yet it still ranked #383 first while missing that #384 was
`in-progress` (claimed 40 minutes earlier) and that #383 is that bundle's umbrella. The script's
`CLAIM?`/`UMBRELLA?` flags now surface exactly this instead of relying on a full manual re-check.
- **The cheap selector's dependency + priority data was NOT trustworthy — the fix was to make the
script compute them, not to re-derive the whole contested tier by hand every session** (2026-07-17,
#73 selection): the old packet reported #385/#386 as `deps:clear` when both were blocked, and
missed #383 entirely (the only `priority: medium` in tier 1). The script now runs
`GET /issues/{n}/dependencies` on every surviving candidate and tiers by priority-within-tier
deterministically, closing this class.
- **Queue-drift root cause (2026-07-12, user-surfaced, historical)**: pickups repeatedly re-picked
already-done work because DONE/OPEN status was read from **prose** (the old tracker's arc body
inline markers + session comments' "Recommended next"), which lags real issue state — worst under
parallel sessions narrating each other's merges. STANDING FIX (still the rule, just no longer tied
to any specific tracker): **live Gitea state is the ONLY source of truth for status**; a comment's
"Recommended next" is a candidate that MUST be re-verified OPEN before pickup, never propagated
when stale. Do not name today's frontier issue in this standing file — that's how the drift
started the first time.
+176
View File
@@ -0,0 +1,176 @@
# ErsatzTV MCP Server
`ErsatzTV.Mcp` is a stdio JSON-RPC [MCP](https://modelcontextprotocol.io) server that wraps the
frozen ErsatzTV/ChicoryTV `/api/v1` REST surface as explicit, narrow tools for AI agents. It exposes
**read** tools by default and **cautious-write** tools behind an opt-in (issue #58).
It maps each tool to an OpenAPI-backed endpoint in `ErsatzTV/wwwroot/openapi/v1.json`. It does **not**
scrape the web UI and does **not** read or write SQLite directly.
> This is a fresh build against the versioned `/api/v1` contract (mounted by #286), superseding the
> read-only v0 foundation in the closed PR #76. The security baseline below is carried forward from
> PR #76 / #289 verbatim.
## Running
```bash
dotnet build ErsatzTV.Mcp/ErsatzTV.Mcp.csproj
```
Configure an MCP client to start the server over stdio:
```bash
dotnet run --project /path/to/ersatztv/ErsatzTV.Mcp/ErsatzTV.Mcp.csproj
```
Environment variables:
| Variable | Default | Purpose |
|---|---:|---|
| `ERSATZTV_URL` | `http://localhost:8409` | Base URL for the ErsatzTV API. A reverse-proxy path prefix (e.g. `https://host/etv/`) is preserved. |
| `ERSATZTV_API_KEY` | unset | Sent as `X-Api-Key` on every request. Effectively **required** (see Authentication). |
| `ERSATZTV_ALLOW_WRITES` | `false` | Write posture. While `false`, the executor refuses any non-GET tool before it reaches the API. Set `true` to enable the write tools below. |
| `ERSATZTV_MAX_RESPONSE_BYTES` | `1048576` | Cap on the API response body buffered back to the model; larger responses are truncated with a marker. |
| `ERSATZTV_REQUEST_TIMEOUT_SECONDS` | `30` | Per-request HTTP timeout (covers headers **and** the streamed body). |
## Authentication
ErsatzTV's `/api` surface is gated by a fail-closed session-or-key filter (`api-conventions.md` §9).
The MCP server is a **machine client**, so it authenticates with **`X-Api-Key`** on every request:
- **Every write** (POST/PUT/PATCH/DELETE) requires the key. There is no "open" write mode.
- **Reads** require the key too under the default `Api:RequireKeyForReads=true`.
- Key-authed requests are **CSRF-exempt** (the `X-CSRF` header the browser session path needs does not
apply to the machine key), so the MCP server sends no CSRF header.
So **`ERSATZTV_API_KEY` is effectively required**; without it tool calls return `401`. The key is the
server machine key — surfaced read-only by the SPA's machine-key screen
(`GET /api/v1/auth/machine-key`) or persisted at `/config/api.key`.
## Security posture
- **Read-only by default, runtime-enforced.** Even if a catalog entry were wrong, the executor refuses
any non-GET tool unless `ERSATZTV_ALLOW_WRITES=true` — a single bad entry cannot mutate or delete.
- **Malformed input never crashes the session.** Invalid JSON → JSON-RPC `-32700` (id `null`); a
malformed request object → `-32600`; a bad tool call → `-32602`; a transport/timeout failure →
`-32603` for the id (a compliant client never hangs). The `Program.Main` read loop also catches any
unexpected per-line error.
- **Bounded input and output.** A hostile client cannot exhaust memory with a giant unterminated line
(`BoundedLineReader` caps + drains it), and API bodies are read up to `ERSATZTV_MAX_RESPONSE_BYTES`
and truncated (on a UTF-8 code-point boundary). Every request has a timeout covering headers **and**
the streamed body.
- **Arguments are validated** against each tool's declared `InputSchema` (required present, no unknown
args — `additionalProperties:false` — basic types) before any request is built. Path params reject
`.`/`..` so a value can't canonicalize onto a different route.
- **Tool results are untrusted data.** Response bodies (media titles, file paths, error text) can be
attacker-influenced and are returned to the model verbatim. Treat all tool output as data, never as
instructions; the consuming agent's system prompt should frame it as such. This is the standard
prompt-injection caveat for any tool that surfaces external content.
## How tools map to the API
Each declared argument routes to exactly one place:
- **path** — a `{param}` in the path template (URL-encoded; `.`/`..` rejected).
- **query** — an argument listed in the tool's query-parameter set (URL-encoded onto the query string,
for any verb).
- **`ifMatch`** — the reserved header argument, carried as the RFC 7232 `If-Match` request header (see
Optimistic concurrency). A value containing control characters (CR/LF) is rejected before the request
is sent, so it cannot smuggle additional headers onto the API-key-bearing request.
- **body** — for write verbs (POST/PUT/PATCH), every remaining argument is serialized as the JSON
request body (`application/json`).
When a response carries an `ETag` header (versioned aggregates emit it on GET and on a successful
replace PUT), the tool result appends a `\n[etag: "N"]` marker so an agent can round-trip it as
`ifMatch` on a subsequent write.
### Optimistic concurrency (`api-conventions.md` §7a)
Only the **replace-all aggregate PUTs** honor `If-Match` — here that is
`ersatztv_update_collection_custom_order`. Read the ETag from the matching GET
(`ersatztv_get_collection_items`), pass it back as `ifMatch` (e.g. `"3"`); a stale tag → `412`, a
grammar violation → `400`, `"*"` or omitting it force-writes. All other writes ignore `If-Match` and
force-write, so no ETag handshake is needed for them.
## Read tools
| Tool | API route |
|---|---|
| `ersatztv_list_channels` | `GET /api/v1/channels` |
| `ersatztv_get_channel` | `GET /api/v1/channels/{id}` |
| `ersatztv_list_collections` | `GET /api/v1/collections` |
| `ersatztv_get_collection` | `GET /api/v1/collections/{id}` |
| `ersatztv_get_collection_items` | `GET /api/v1/collections/{id}/items` (paged; emits ETag) |
| `ersatztv_list_smart_collections` | `GET /api/v1/smart-collections` |
| `ersatztv_get_smart_collection` | `GET /api/v1/smart-collections/{id}` |
| `ersatztv_list_schedules` | `GET /api/v1/schedules` |
| `ersatztv_get_schedule` | `GET /api/v1/schedules/{id}` |
| `ersatztv_get_schedule_items` | `GET /api/v1/schedules/{id}/items` (emits ETag) |
| `ersatztv_list_playouts` | `GET /api/v1/playouts` |
| `ersatztv_get_playout` | `GET /api/v1/playouts/{id}` |
| `ersatztv_get_playout_items` | `GET /api/v1/playouts/{id}/items` |
| `ersatztv_list_ffmpeg_profiles` | `GET /api/v1/ffmpeg/profiles` |
| `ersatztv_get_ffmpeg_profile` | `GET /api/v1/ffmpeg/profiles/{id}` |
| `ersatztv_get_resolution_by_name` | `GET /api/v1/ffmpeg/resolution/by-name/{name}` |
| `ersatztv_list_sessions` | `GET /api/v1/sessions` |
| `ersatztv_get_version` | `GET /api/v1/version` |
| `ersatztv_list_media_sources` | `GET /api/v1/media-sources` |
| `ersatztv_get_jellyfin_libraries` | `GET /api/v1/media-sources/jellyfin/{id}/libraries` |
| `ersatztv_list_local_libraries` | `GET /api/v1/libraries/local` |
| `ersatztv_get_library_scan_status` | `GET /api/v1/libraries/scan-status` |
| `ersatztv_search` | `GET /api/v1/search` |
| `ersatztv_search_all_items` | `GET /api/v1/search/all-items` (raw id lists) |
| `ersatztv_search_artists` | `GET /api/v1/search/artists` |
## Write tools (require `ERSATZTV_ALLOW_WRITES=true`)
| Tool | API route |
|---|---|
| `ersatztv_create_collection` | `POST /api/v1/collections` |
| `ersatztv_update_collection` | `PUT /api/v1/collections/{id}` |
| `ersatztv_delete_collection` | `DELETE /api/v1/collections/{id}` |
| `ersatztv_add_collection_items` | `POST /api/v1/collections/{id}/items` (idempotent; existence-checked) |
| `ersatztv_remove_collection_item` | `DELETE /api/v1/collections/{id}/items/{mediaItemId}` |
| `ersatztv_update_collection_custom_order` | `PUT /api/v1/collections/{id}/custom-order` (honors `If-Match`) |
| `ersatztv_create_smart_collection` | `POST /api/v1/smart-collections` |
| `ersatztv_update_smart_collection` | `PUT /api/v1/smart-collections/{id}` |
| `ersatztv_delete_smart_collection` | `DELETE /api/v1/smart-collections/{id}` |
| `ersatztv_create_schedule` | `POST /api/v1/schedules` |
| `ersatztv_update_schedule` | `PUT /api/v1/schedules/{id}` |
| `ersatztv_delete_schedule` | `DELETE /api/v1/schedules/{id}` |
| `ersatztv_create_playout` | `POST /api/v1/playouts` |
| `ersatztv_update_playout` | `PUT /api/v1/playouts/{id}` |
| `ersatztv_delete_playout` | `DELETE /api/v1/playouts/{id}` |
| `ersatztv_create_channel` | `POST /api/v1/channels` |
| `ersatztv_update_channel` | `PUT /api/v1/channels/{id}` |
| `ersatztv_reset_channel_playout` | `POST /api/v1/channels/{id}/playout/reset` |
| `ersatztv_delete_channel` | `DELETE /api/v1/channels/{id}` |
| `ersatztv_enable_jellyfin_library_sync` | `PUT /api/v1/media-sources/jellyfin/{id}/libraries` |
| `ersatztv_refresh_jellyfin_libraries` | `POST /api/v1/media-sources/jellyfin/{id}/refresh-libraries` |
| `ersatztv_scan_jellyfin_collections` | `POST /api/v1/media-sources/jellyfin/{id}/scan-collections` |
| `ersatztv_scan_library` | `POST /api/v1/libraries/{id}/scan` |
### Populating a collection (the #487 acceptance case)
`ersatztv_add_collection_items` funnels every media kind through one endpoint — send only the id
buckets you need (`artistIds`, `musicVideoIds`, `songIds`, `movieIds`, …). Discover ids with
`ersatztv_search_all_items` (returns raw id lists for a Lucene query) or `ersatztv_search_artists`.
Re-adding an already-present item is an **idempotent no-op** (no duplicate rows, still `204`); if any
referenced id does not exist the whole batch is rejected (`422`). So the flow is: search → add ids →
re-run to confirm idempotence.
## Deferred
Channel create/update (`ersatztv_create_channel` / `ersatztv_update_channel`) wrap a 28-field DTO with
nine enum fields. Only `name`/`number`/`ffmpegProfileId` are required; the rest have server-side
defaults, and the enum fields take the enum **name** (the API validates them). Discover an existing
channel's shape and current enum values with `ersatztv_get_channel` before creating/updating.
Deliberately **not** exposed in this cautious first write pass:
- **The replace-list writes with large item DTOs** — schedule items (`PUT .../schedules/{id}/items`,
~40 fields per item) and playout alternate-schedules/templates. The simple
`update_collection_custom_order` replace is exposed as the `If-Match` exemplar.
- **Redesign-aware workflow tools** — create-channel-from-lineup (#63), Channel Templates (#64),
library browse/artwork (#65), image/logo/watermark (#66/#67), resume/bookmark (#68). These should
wrap the composite backend endpoints once those contracts exist, not recreate workflows in MCP.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,899 @@
# External Channel-Logo Download-On-Save Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Turn an external channel-logo URL into a download-on-save input method: on save the URL is fetched, decode-validated, and cached under a content-hash name so it becomes byte-identical to an uploaded logo; the render path never fetches a logo again.
**Architecture:** A pure arithmetic budget (`RemoteImageDecodeBudget`, Core) is shared by the render path and the new save path. An Infrastructure decoder (`RemoteImageValidator`) performs the ImageSharp identify/decode/validate step; a Core interface `IRemoteLogoCacher` (Infrastructure impl) composes fetch → validate → cache and returns the cache name or a `BaseError`. The three channel handlers call it; `UploadArtworkHandler` reuses the validator; a startup `BackgroundService` migrates existing URL rows.
**Tech Stack:** C#/.NET 10, MediatR CQRS, LanguageExt (`Either`/`Validation`/`Option`), EF Core (SQLite + MySql), SixLabors.ImageSharp 3.1.12, NUnit + Shouldly + NSubstitute, ChicoryTV React SPA (Vite + TS).
## Global Constraints
- **Layering (enforced by `ErsatzTV.Architecture.Tests`):** `Core` may depend on `FFmpeg` only — no EF, no Infrastructure, no ImageSharp-in-a-way-that-breaks-purity. `Application` may depend on `Core` + `Infrastructure` abstractions. Put interfaces in `Core`, implementations in `Infrastructure`, DI wiring in `ErsatzTV/Startup.cs`.
- **NUnit + Shouldly + NSubstitute only.** Never xUnit. Handler tests extend `ChannelHandlerTestBase` (`ErsatzTV.Tests/Support/ChannelHandlerTestBase.cs`) using `InMemoryTvContext`.
- **Decode budgets (verbatim from #511, do not change the numbers):** `MaxRemoteDecodedPixels = 50_000_000`; `MaxRemoteFrames = 600`. Decode bound must be imposed on the DECODER (`DecoderOptions.MaxFrames`) and re-verified against the decoded image — header frame counts lie (APNG reports 0).
- **Fix formatting as you touch it:** run `dotnet format ErsatzTV.sln --include <changed .cs>` under `bash -c` before committing; no UTF-8 BOM on any touched `.cs` (`head -c3 | xxd -p` must not be `efbbbf`). `charset=utf-8` in `.editorconfig`.
- **Dual-provider migrations:** any `TvContext` model change needs `scripts/add-migration.sh <Name>` (SQLite + MySql). This plan adds **no** schema change (reuses `Artwork.Path`), so no migration is expected — if you find you need one, stop and reconsider.
- **Docs-in-same-PR:** update `docs/decisions.md`, `docs/channels.md`, `docs/api-conventions.md` in the implementation PR (Task 9). Regenerate OpenAPI (`./scripts/update-openapi.sh` + `npm run generate:api`) only if a response shape changes — this plan changes only error status/messages, not shapes, so likely just the endpoint prose.
- **Central Package Management:** no `Version=` on `<PackageReference>`; versions live in `Directory.Packages.props`.
- **Content-hash name, not GUID:** reuse `IImageCache.SaveArtworkToCache` (MD5-of-bytes). No new naming scheme.
- **Fixes #525.**
---
### Task 1: Extract the pure decode budget into `RemoteImageDecodeBudget` (Core)
Lift the pure arithmetic budget out of `ImageElementBase` so both the render path and the save path share one implementation. No behavior change — this is a move + delegate.
**Files:**
- Create: `ErsatzTV.Core/Images/RemoteImageDecodeBudget.cs`
- Modify: `ErsatzTV.Infrastructure/Streaming/Graphics/Image/ImageElementBase.cs` (delete the moved members, delegate to the new class)
- Create: `ErsatzTV.Core.Tests/Images/RemoteImageDecodeBudgetTests.cs`
- Move (into the test above): the budget-arithmetic cases from `ErsatzTV.Infrastructure.Tests/Streaming/Graphics/RemoteImageDecodeLimitTests.cs` (keep the ImageSharp-decode tests where they are)
**Interfaces:**
- Produces:
- `RemoteImageDecodeBudget.MaxRemoteDecodedPixels` (`const long = 50_000_000`)
- `RemoteImageDecodeBudget.MaxRemoteFrames` (`const int = 600`)
- `static void EnsureDimensionsAffordable(int width, int height, Uri uri)`
- `static int AffordableFrames(int width, int height)`
- `static void EnsureDecodeAffordable(int width, int height, int frameCount, Uri uri)`
- Consumes: nothing (pure).
- [ ] **Step 1: Write the failing test**
Create `ErsatzTV.Core.Tests/Images/RemoteImageDecodeBudgetTests.cs`:
```csharp
using ErsatzTV.Core.Images;
using NUnit.Framework;
using Shouldly;
namespace ErsatzTV.Core.Tests.Images;
[TestFixture]
public class RemoteImageDecodeBudgetTests
{
private static readonly Uri Uri = new("https://example.com/logo.png");
// the product is the real bound: 2500x2500 x600 is affordable on each axis alone but not together
[Test]
public void Should_Reject_Dimensions_And_Frames_Affordable_Alone_But_Not_Together()
{
((long)2500 * 2500).ShouldBeLessThanOrEqualTo(RemoteImageDecodeBudget.MaxRemoteDecodedPixels);
600.ShouldBeLessThanOrEqualTo(RemoteImageDecodeBudget.MaxRemoteFrames);
InvalidOperationException ex = Should.Throw<InvalidOperationException>(
() => RemoteImageDecodeBudget.EnsureDecodeAffordable(2500, 2500, 600, Uri));
ex.Message.ShouldContain("pixel limit");
}
[Test]
public void Should_Reject_Too_Many_Frames_Even_When_Each_Is_Tiny() =>
Should.Throw<InvalidOperationException>(
() => RemoteImageDecodeBudget.EnsureDecodeAffordable(8, 8, RemoteImageDecodeBudget.MaxRemoteFrames + 1, Uri))
.Message.ShouldContain("frame limit");
[Test]
public void Should_Reject_A_Single_Oversized_Frame() =>
Should.Throw<InvalidOperationException>(
() => RemoteImageDecodeBudget.EnsureDimensionsAffordable(30000, 30000, Uri))
.Message.ShouldContain("pixel limit");
[Test]
public void Should_Allow_A_Single_Large_Still_Within_Budget() =>
Should.NotThrow(() => RemoteImageDecodeBudget.EnsureDecodeAffordable(7680, 4320, 1, Uri));
[Test]
public void Should_Charge_At_Least_One_Frame_When_Header_Reports_None() =>
Should.Throw<InvalidOperationException>(
() => RemoteImageDecodeBudget.EnsureDecodeAffordable(30000, 30000, 0, Uri));
[Test]
public void Should_Afford_Fewer_Frames_As_Frames_Get_Larger()
{
RemoteImageDecodeBudget.AffordableFrames(8, 8).ShouldBe(RemoteImageDecodeBudget.MaxRemoteFrames);
RemoteImageDecodeBudget.AffordableFrames(1000, 1000).ShouldBe(50);
RemoteImageDecodeBudget.AffordableFrames(7000, 7000).ShouldBe(1);
}
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `dotnet test ErsatzTV.Core.Tests/ErsatzTV.Core.Tests.csproj --filter "FullyQualifiedName~RemoteImageDecodeBudget"`
Expected: FAIL — `RemoteImageDecodeBudget` does not exist.
- [ ] **Step 3: Create `RemoteImageDecodeBudget`**
Create `ErsatzTV.Core/Images/RemoteImageDecodeBudget.cs` (bodies copied verbatim from `ImageElementBase`, only the type moved):
```csharp
namespace ErsatzTV.Core.Images;
/// <summary>
/// The decode-budget policy for a remote image, as pure arithmetic so it can be enforced both at
/// render time (graphics engine) and at save time (logo download) without materializing
/// multi-gigabyte images. Extracted from ImageElementBase for reuse. (ersatztv#525, from #511.)
/// </summary>
public static class RemoteImageDecodeBudget
{
/// <summary>
/// Ceiling on TOTAL decoded pixels — width x height x frames, as one product. Checking
/// dimensions and frame count independently does not bound the decode: a 60 KiB 2500x2500 x600
/// GIF passes both a 50 MP dimension check and a 600 frame check and costs ~14 GiB.
/// </summary>
public const long MaxRemoteDecodedPixels = 50_000_000;
/// <summary>Frame ceiling, a cheap legible guard against absurd counts of tiny frames.</summary>
public const int MaxRemoteFrames = 600;
public static void EnsureDimensionsAffordable(int width, int height, Uri uri)
{
long pixels = (long)width * height;
if (pixels > MaxRemoteDecodedPixels)
{
throw new InvalidOperationException(
$"Remote image {uri} is {width}x{height} ({pixels} pixels), over the "
+ $"{MaxRemoteDecodedPixels} pixel limit");
}
}
public static int AffordableFrames(int width, int height)
{
long perFrame = Math.Max((long)width * height, 1);
return (int)Math.Clamp(MaxRemoteDecodedPixels / perFrame, 1, MaxRemoteFrames);
}
public static void EnsureDecodeAffordable(int width, int height, int frameCount, Uri uri)
{
int frames = Math.Max(frameCount, 1);
if (frames > MaxRemoteFrames)
{
throw new InvalidOperationException(
$"Remote image {uri} has {frames} frames, over the {MaxRemoteFrames} frame limit");
}
long totalPixels = (long)width * height * frames;
if (totalPixels > MaxRemoteDecodedPixels)
{
throw new InvalidOperationException(
$"Remote image {uri} decodes to {width}x{height} x{frames} frames "
+ $"({totalPixels} pixels), over the {MaxRemoteDecodedPixels} pixel limit");
}
}
}
```
- [ ] **Step 4: Delegate from `ImageElementBase`**
In `ErsatzTV.Infrastructure/Streaming/Graphics/Image/ImageElementBase.cs`: delete the `MaxRemoteDecodedPixels`, `MaxRemoteFrames`, `EnsureDimensionsAffordable`, `AffordableFrames`, `EnsureDecodeAffordable` members. Keep `MaxRemoteScaledPixels` + `EnsureScaledFramesAffordable` (retention budget — render-only). Add `using ErsatzTV.Core.Images;` and update the three call sites inside `DecodeRemoteImage`:
```csharp
RemoteImageDecodeBudget.EnsureDimensionsAffordable(info.Width, info.Height, uri);
int affordableFrames = RemoteImageDecodeBudget.AffordableFrames(info.Width, info.Height);
// ... after decode:
RemoteImageDecodeBudget.EnsureDecodeAffordable(image.Width, image.Height, image.Frames.Count, uri);
```
Delete the now-duplicated arithmetic tests from `RemoteImageDecodeLimitTests.cs` (the `EnsureDecodeAffordable`/`AffordableFrames`/`EnsureDimensionsAffordable` cases moved to Task 1's test). KEEP its ImageSharp-decode tests (`DecodeRemoteImage`, APNG regression, CRC-crafted PNG) — those move to Task 2.
- [ ] **Step 5: Run tests to verify they pass**
Run: `dotnet test ErsatzTV.Core.Tests/ErsatzTV.Core.Tests.csproj --filter "FullyQualifiedName~RemoteImageDecodeBudget"` → PASS
Run: `dotnet build ErsatzTV.sln``Build succeeded`, 0 warnings (warnings are errors).
- [ ] **Step 6: Commit**
```bash
bash -c 'dotnet format ErsatzTV.sln --no-restore --include ErsatzTV.Core/Images/RemoteImageDecodeBudget.cs ErsatzTV.Infrastructure/Streaming/Graphics/Image/ImageElementBase.cs ErsatzTV.Core.Tests/Images/RemoteImageDecodeBudgetTests.cs ErsatzTV.Infrastructure.Tests/Streaming/Graphics/RemoteImageDecodeLimitTests.cs'
git add -A && git commit -m "refactor(525): extract RemoteImageDecodeBudget from ImageElementBase"
```
---
### Task 2: `RemoteImageValidator` (Infrastructure) — decode + budget-validate a stream
Extract the ImageSharp identify/decode/validate step so both the render path and the save path share it. It returns the decoded `Image` (render needs it; save disposes it). This is the `DecodeRemoteImage` logic relocated behind an interface.
**Files:**
- Create: `ErsatzTV.Core/Interfaces/Images/IRemoteImageValidator.cs`
- Create: `ErsatzTV.Infrastructure/Images/RemoteImageValidator.cs`
- Modify: `ErsatzTV.Infrastructure/Streaming/Graphics/Image/ImageElementBase.cs` (delegate `DecodeRemoteImage` to the validator; it is constructed with `IRemoteImageFetcher` today — add `IRemoteImageValidator` alongside)
- Move: the ImageSharp-decode tests from `RemoteImageDecodeLimitTests.cs``ErsatzTV.Infrastructure.Tests/Images/RemoteImageValidatorTests.cs`
- Modify: `ErsatzTV.Infrastructure/Streaming/Graphics/GraphicsEngine.cs`, `Image/ImageElement.cs`, `Image/WatermarkElement.cs` (thread the validator through, same pattern as `IRemoteImageFetcher`)
**RESOLVED (was a VERIFY item): `ErsatzTV.Core` does NOT reference SixLabors.ImageSharp** (it has SkiaSharp only). So the Core interface must NOT return an ImageSharp `Image`. Final shape:
- Core interface `IRemoteImageValidator.Validate(Stream, Uri, CancellationToken) : Task` — throws on an invalid image (budget violation or corrupt stream), completes on valid. No ImageSharp type crosses Core. This is all the save/upload paths need.
- The render path keeps returning the decoded `Image`, but via a **static** method on the Infra `RemoteImageValidator` (`ImageElementBase` calls it directly — no interface, no DI threading through `GraphicsEngine`). This is a simplification vs. the original draft: no new constructor param on the graphics elements.
**Interfaces:**
- Produces:
- Core: `IRemoteImageValidator.Validate(Stream stream, Uri uri, CancellationToken) : Task` (throws `InvalidOperationException` on a budget violation / ImageSharp exception on a corrupt stream; returns on success)
- Infra static: `RemoteImageValidator.DecodeAndValidate(Stream stream, Uri uri, CancellationToken) : Task<Image>` (SixLabors `Image`; same throws; caller owns + disposes the returned `Image`) — used by `ImageElementBase` and internally by `Validate`
- Consumes: `RemoteImageDecodeBudget` (Task 1).
- [ ] **Step 1: Write the failing test** — move the existing decode tests and retarget them
Create `ErsatzTV.Infrastructure.Tests/Images/RemoteImageValidatorTests.cs` by moving the `DecodeRemoteImage` tests out of `RemoteImageDecodeLimitTests.cs` and calling the validator instead. Key cases (bodies come from the existing tests — reuse the crafted-PNG + APNG helpers verbatim):
```csharp
using ErsatzTV.Infrastructure.Images;
using NUnit.Framework;
using Shouldly;
using SixLabors.ImageSharp;
using Image = SixLabors.ImageSharp.Image;
namespace ErsatzTV.Infrastructure.Tests.Images;
[TestFixture]
public class RemoteImageValidatorTests
{
private static readonly Uri Uri = new("https://example.com/logo.png");
// decode cases exercise the static method (used by the render path)
[Test]
public async Task Should_Decode_A_Normal_Image()
{
await using MemoryStream stream = await RealPng(64, 32);
using Image image = await RemoteImageValidator.DecodeAndValidate(stream, Uri, CancellationToken.None);
image.Width.ShouldBe(64);
image.Height.ShouldBe(32);
}
[Test]
public async Task Should_Reject_A_Decompression_Bomb_By_Declared_Dimensions()
{
await using MemoryStream stream = PngHeaderDeclaring(30000, 30000);
InvalidOperationException ex = await Should.ThrowAsync<InvalidOperationException>(
() => RemoteImageValidator.DecodeAndValidate(stream, Uri, CancellationToken.None));
ex.Message.ShouldContain("pixel limit");
}
[Test]
public async Task Should_Reject_An_Apng_Whose_Header_Under_Reports_Its_Frames()
{
await using MemoryStream stream = Apng(64, 64, RemoteImageDecodeBudget.MaxRemoteFrames + 100);
InvalidOperationException ex = await Should.ThrowAsync<InvalidOperationException>(
() => RemoteImageValidator.DecodeAndValidate(stream, Uri, CancellationToken.None));
ex.Message.ShouldContain("frame limit");
}
[Test]
public async Task Should_Decode_An_Apng_That_A_Default_Identify_Cannot_Read()
{
await using MemoryStream stream = Apng(288, 288, 60);
stream.Position = 0;
await Should.ThrowAsync<Exception>(() => Image.IdentifyAsync(stream));
stream.Position = 0;
using Image image = await RemoteImageValidator.DecodeAndValidate(stream, Uri, CancellationToken.None);
image.Frames.Count.ShouldBe(60);
}
// the Core interface Validate() is the save/upload contract: throws on invalid, returns on valid,
// never surfaces an ImageSharp type
[Test]
public async Task Validate_Returns_On_A_Good_Image()
{
IRemoteImageValidator validator = new RemoteImageValidator();
await using MemoryStream stream = await RealPng(64, 32);
await Should.NotThrowAsync(() => validator.Validate(stream, Uri, CancellationToken.None));
}
[Test]
public async Task Validate_Throws_On_A_Bomb()
{
IRemoteImageValidator validator = new RemoteImageValidator();
await using MemoryStream stream = PngHeaderDeclaring(30000, 30000);
await Should.ThrowAsync<InvalidOperationException>(
() => validator.Validate(stream, Uri, CancellationToken.None));
}
// (move RealPng / PngHeaderDeclaring / Apng / Crc32 helpers here verbatim from RemoteImageDecodeLimitTests)
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `dotnet test ErsatzTV.Infrastructure.Tests/ErsatzTV.Infrastructure.Tests.csproj --filter "FullyQualifiedName~RemoteImageValidator"`
Expected: FAIL — `RemoteImageValidator` / `IRemoteImageValidator` do not exist.
- [ ] **Step 3: Create the interface and implementation**
`ErsatzTV.Core/Interfaces/Images/IRemoteImageValidator.cs` (NO ImageSharp — Core does not reference it):
```csharp
namespace ErsatzTV.Core.Interfaces.Images;
/// <summary>
/// Validates that a stream is a decodable image within the decode budget, throwing if not.
/// Used by the logo save path and the artwork upload path (neither needs the decoded pixels,
/// only "is this safe to cache"). The graphics engine uses the static
/// RemoteImageValidator.DecodeAndValidate instead, which returns the Image it composites.
/// (ersatztv#525)
/// </summary>
public interface IRemoteImageValidator
{
Task Validate(Stream stream, Uri uri, CancellationToken cancellationToken);
}
```
`ErsatzTV.Infrastructure/Images/RemoteImageValidator.cs` — move the body of `ImageElementBase.DecodeRemoteImage` here verbatim into the **static** `DecodeAndValidate` (the `!CanSeek` guard, the `MaxFrames = 1` Identify workaround, `RemoteImageDecodeBudget.*` calls, the `MaxFrames = affordable + 2` decode, the post-decode re-verify + dispose-on-throw). `Validate` wraps it and disposes:
```csharp
using ErsatzTV.Core.Images;
using ErsatzTV.Core.Interfaces.Images;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats;
namespace ErsatzTV.Infrastructure.Images;
public class RemoteImageValidator : IRemoteImageValidator
{
public async Task Validate(Stream stream, Uri uri, CancellationToken cancellationToken)
{
using Image _ = await DecodeAndValidate(stream, uri, cancellationToken);
}
public static async Task<Image> DecodeAndValidate(Stream stream, Uri uri, CancellationToken cancellationToken)
{
// <verbatim body of ImageElementBase.DecodeRemoteImage, RemoteImageDecodeBudget.* for the
// three budget calls; see that method for the exact code and comments>
}
}
```
- [ ] **Step 4: Delegate `DecodeRemoteImage` to the static method**
`ImageElementBase.DecodeRemoteImage` body becomes `return await RemoteImageValidator.DecodeAndValidate(stream, uri, cancellationToken);` (add `using ErsatzTV.Infrastructure.Images;`). **No** constructor change, **no** `GraphicsEngine` threading — the render path calls the static method directly (as it already calls `Image.LoadAsync` statically today). The existing `RemoteImageDecodeLimitTests` decode tests either move to `RemoteImageValidatorTests` (Step 1) or keep calling `ImageElementBase.DecodeRemoteImage` (which now delegates) — either is fine; do not duplicate. Register the interface for the save/upload paths in `Startup.cs`: `services.AddScoped<IRemoteImageValidator, RemoteImageValidator>();`.
- [ ] **Step 5: Run tests to verify they pass**
Run: `dotnet test ErsatzTV.Infrastructure.Tests/ErsatzTV.Infrastructure.Tests.csproj --filter "FullyQualifiedName~Streaming|FullyQualifiedName~Images"` → PASS
Run: `dotnet build ErsatzTV.sln``Build succeeded`.
- [ ] **Step 6: Commit**
```bash
bash -c 'dotnet format ErsatzTV.sln --no-restore --include <all touched .cs>'
git add -A && git commit -m "refactor(525): extract RemoteImageValidator; render path delegates to it"
```
---
### Task 3: `IRemoteLogoCacher` — fetch + validate + cache a URL to a cache name
The save-path primitive: given a URL, fetch (hardened, #511), validate (Task 2), and cache the original bytes (`IImageCache`), returning the content-hash name or a `BaseError`. This is what the handlers call.
**Files:**
- Create: `ErsatzTV.Core/Interfaces/Images/IRemoteLogoCacher.cs`
- Create: `ErsatzTV.Infrastructure/Images/RemoteLogoCacher.cs`
- Create: `ErsatzTV.Infrastructure.Tests/Images/RemoteLogoCacherTests.cs`
- Modify: `ErsatzTV/Startup.cs` (register)
**Interfaces:**
- Produces: `IRemoteLogoCacher.CacheFromUrl(Uri uri, CancellationToken) : Task<Either<BaseError, string>>` — Right = bare cache file name (as `IImageCache.SaveArtworkToCache` returns), Left = a `BaseError` whose message names the failure (timeout / status / not-image / over-size / over-budget / cache write).
- Consumes: `IRemoteImageFetcher.Fetch` (Task from #511), `IRemoteImageValidator.DecodeAndValidate` (Task 2), `IImageCache.SaveArtworkToCache`.
- [ ] **Step 1: Write the failing test**
Create `ErsatzTV.Infrastructure.Tests/Images/RemoteLogoCacherTests.cs`:
```csharp
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Streaming;
using ErsatzTV.Infrastructure.Images;
using LanguageExt;
using NSubstitute;
using NUnit.Framework;
using Shouldly;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Png;
using SixLabors.ImageSharp.PixelFormats;
using Image = SixLabors.ImageSharp.Image;
namespace ErsatzTV.Infrastructure.Tests.Images;
[TestFixture]
public class RemoteLogoCacherTests
{
private static readonly Uri Uri = new("https://example.com/logo.png");
[Test]
public async Task Should_Fetch_Validate_And_Cache_Returning_The_Name()
{
MemoryStream png = await RealPng(64, 64);
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Uri, Arg.Any<CancellationToken>()).Returns(png);
var validator = Substitute.For<IRemoteImageValidator>();
validator.Validate(png, Uri, Arg.Any<CancellationToken>()).Returns(Task.CompletedTask);
var cache = Substitute.For<IImageCache>();
cache.SaveArtworkToCache(Arg.Any<Stream>(), ArtworkKind.Logo).Returns(Right<BaseError, string>("abc123"));
var cacher = new RemoteLogoCacher(fetcher, validator, cache);
Either<BaseError, string> result = await cacher.CacheFromUrl(Uri, CancellationToken.None);
result.IsRight.ShouldBeTrue();
result.IfRight(name => name.ShouldBe("abc123"));
}
[Test]
public async Task Should_Return_Left_When_The_Fetch_Throws()
{
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Uri, Arg.Any<CancellationToken>()).Returns<Stream>(_ => throw new TimeoutException("timed out"));
var cacher = new RemoteLogoCacher(fetcher, Substitute.For<IRemoteImageValidator>(), Substitute.For<IImageCache>());
Either<BaseError, string> result = await cacher.CacheFromUrl(Uri, CancellationToken.None);
result.IsLeft.ShouldBeTrue();
result.IfLeft(e => e.Value.ShouldContain("timed out"));
}
[Test]
public async Task Should_Return_Left_When_Validation_Rejects_A_Bomb()
{
MemoryStream png = await RealPng(64, 64);
var fetcher = Substitute.For<IRemoteImageFetcher>();
fetcher.Fetch(Uri, Arg.Any<CancellationToken>()).Returns(png);
var validator = Substitute.For<IRemoteImageValidator>();
validator.Validate(png, Uri, Arg.Any<CancellationToken>())
.Returns<Task>(_ => throw new InvalidOperationException("over the 50000000 pixel limit"));
var cacher = new RemoteLogoCacher(fetcher, validator, Substitute.For<IImageCache>());
Either<BaseError, string> result = await cacher.CacheFromUrl(Uri, CancellationToken.None);
result.IsLeft.ShouldBeTrue();
result.IfLeft(e => e.Value.ShouldContain("pixel limit"));
}
private static async Task<MemoryStream> RealPng(int w, int h)
{
using var img = new Image<Rgba32>(w, h);
var ms = new MemoryStream();
await img.SaveAsync(ms, new PngEncoder());
ms.Position = 0;
return ms;
}
}
```
- [ ] **Step 2: Run test to verify it fails**
Run: `dotnet test ErsatzTV.Infrastructure.Tests/ErsatzTV.Infrastructure.Tests.csproj --filter "FullyQualifiedName~RemoteLogoCacher"`
Expected: FAIL — `RemoteLogoCacher` / `IRemoteLogoCacher` do not exist.
- [ ] **Step 3: Create the interface and implementation**
`ErsatzTV.Core/Interfaces/Images/IRemoteLogoCacher.cs`:
```csharp
using ErsatzTV.Core;
using LanguageExt;
namespace ErsatzTV.Core.Interfaces.Images;
/// <summary>
/// Fetches an external logo URL, validates it against the decode budget, and stores it in the
/// image cache — turning a URL into a cache name so it is thereafter identical to an uploaded
/// logo. Errors are returned, not thrown, so a save handler can surface a 400. (ersatztv#525)
/// </summary>
public interface IRemoteLogoCacher
{
Task<Either<BaseError, string>> CacheFromUrl(Uri uri, CancellationToken cancellationToken);
}
```
`ErsatzTV.Infrastructure/Images/RemoteLogoCacher.cs`:
```csharp
using ErsatzTV.Core;
using ErsatzTV.Core.Domain;
using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Streaming;
using LanguageExt;
namespace ErsatzTV.Infrastructure.Images;
public class RemoteLogoCacher(
IRemoteImageFetcher fetcher,
IRemoteImageValidator validator,
IImageCache imageCache) : IRemoteLogoCacher
{
public async Task<Either<BaseError, string>> CacheFromUrl(Uri uri, CancellationToken cancellationToken)
{
try
{
await using Stream stream = await fetcher.Fetch(uri, cancellationToken);
// validate by decoding under the budget (throws if unsafe); we cache the raw bytes
await validator.Validate(stream, uri, cancellationToken);
stream.Position = 0;
return await imageCache.SaveArtworkToCache(stream, ArtworkKind.Logo);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw;
}
catch (Exception ex)
{
return BaseError.New($"Could not download logo from {uri}: {ex.Message}");
}
}
}
```
Note: `IRemoteImageFetcher.Fetch` returns a seekable, fully-buffered stream at position 0 (its contract), so `stream.Position = 0` after validation rewinds it for the cache write.
Register in `Startup.cs`: `services.AddScoped<IRemoteLogoCacher, RemoteLogoCacher>();`.
- [ ] **Step 4: Run tests to verify they pass**
Run: `dotnet test ErsatzTV.Infrastructure.Tests/ErsatzTV.Infrastructure.Tests.csproj --filter "FullyQualifiedName~RemoteLogoCacher"` → PASS
- [ ] **Step 5: Commit**
```bash
bash -c 'dotnet format ErsatzTV.sln --no-restore --include <touched .cs>'
git add -A && git commit -m "feat(525): add RemoteLogoCacher (fetch + validate + cache a logo URL)"
```
---
### Task 4: `UpdateChannelHandler` downloads a URL logo on save
Route an incoming external-URL logo through `IRemoteLogoCacher` before it reaches `Artwork.Path`, so a saved channel never stores a URL. A cacher failure fails the save.
**Files:**
- Modify: `ErsatzTV.Application/Channels/Commands/UpdateChannelHandler.cs` (inject `IRemoteLogoCacher`; resolve URL → cache name inside `ApplyUpdateRequest`; surface failure)
- Modify: `ErsatzTV.Tests/Application/Channels/UpdateChannelHandlerTests.cs`
- Modify: `ErsatzTV.Tests/Support/ChannelHandlerTestBase.cs` (add a substituted `IRemoteLogoCacher`)
**Interfaces:**
- Consumes: `IRemoteLogoCacher.CacheFromUrl` (Task 3).
- Produces: on an external-URL logo, `Artwork.Path` holds the cache name (not the URL); a cacher `Left` becomes a `Left<BaseError, ChannelViewModel>` from `Handle`.
Design note on error flow: `ApplyUpdateRequest` currently returns `Task<ChannelViewModel>` and is invoked via `validation.Apply(...)`. The download can fail, so it must be able to produce a `Left`. Change the logo resolution to happen in `Handle` *before* `ApplyUpdateRequest` (so the `Either` composes cleanly), OR change `ApplyUpdateRequest` to return `Task<Either<BaseError, ChannelViewModel>>` and `Bind` it. The plan uses the first (resolve-before-apply) to keep `ApplyUpdateRequest` synchronous-shaped.
- [ ] **Step 1: Write the failing tests**
Add to `UpdateChannelHandlerTests.cs`:
```csharp
[Test]
public async Task Should_Download_External_Url_Logo_And_Store_Cache_Name()
{
Channel channel = await SeedChannel(number: "5");
RemoteLogoCacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Right<BaseError, string>("cachedhash"));
Either<BaseError, ChannelViewModel> result = await MakeHandler().Handle(
MakeUpdate(channel.Id, number: "5", logoPath: "https://example.com/logo.png"),
CancellationToken.None);
result.IsRight.ShouldBeTrue();
await using TvContext db = Db.CreateContext();
Artwork logo = db.Channels.Include(c => c.Artwork).Single(c => c.Id == channel.Id)
.Artwork.Single(a => a.ArtworkKind == ArtworkKind.Logo);
logo.Path.ShouldBe("cachedhash");
logo.IsExternalUrl().ShouldBeFalse();
}
[Test]
public async Task Should_Fail_The_Save_When_The_Logo_Download_Fails()
{
Channel channel = await SeedChannel(number: "5");
RemoteLogoCacher.CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>())
.Returns(Left<BaseError, string>(BaseError.New("Could not download logo: host timed out")));
Either<BaseError, ChannelViewModel> result = await MakeHandler().Handle(
MakeUpdate(channel.Id, number: "5", logoPath: "https://example.com/logo.png"),
CancellationToken.None);
result.IsLeft.ShouldBeTrue();
LeftOf(result).Value.ShouldContain("Could not download logo");
}
[Test]
public async Task Should_Not_Call_The_Cacher_For_An_Uploaded_Logo_Path()
{
Channel channel = await SeedChannel(number: "5");
await MakeHandler().Handle(
MakeUpdate(channel.Id, number: "5", logoPath: "iptv/logos/deadbeef"),
CancellationToken.None);
await RemoteLogoCacher.DidNotReceive().CacheFromUrl(Arg.Any<Uri>(), Arg.Any<CancellationToken>());
}
```
(Add `RemoteLogoCacher` to `ChannelHandlerTestBase` as `protected IRemoteLogoCacher RemoteLogoCacher = Substitute.For<IRemoteLogoCacher>();` set in `BaseSetUp`, and to `MakeHandler()`/`MakeUpdate` a `logoPath` parameter. If `SeedChannel` doesn't exist, use the fixture's existing channel-seeding helper — check the file.)
- [ ] **Step 2: Run tests to verify they fail**
Run: `dotnet test ErsatzTV.Tests/ErsatzTV.Tests.csproj --filter "FullyQualifiedName~UpdateChannelHandlerTests"`
Expected: FAIL — handler does not download; `RemoteLogoCacher` not a ctor param.
- [ ] **Step 3: Implement**
`UpdateChannelHandler`: add `IRemoteLogoCacher remoteLogoCacher` to the primary constructor. In `Handle`, after validation passes and before `ApplyUpdateRequest`, if `request.Logo?.Path` is an external URL, call `remoteLogoCacher.CacheFromUrl`; on `Left` return it; on `Right` replace `request.Logo.Path` with the returned cache name (wrap the request or pass the resolved path into `ApplyUpdateRequest`). Then `ApplyUpdateRequest` stores the (now non-URL) path exactly as today — its existing `iptv/logos/` strip is a no-op for a bare cache name.
Concretely, change the `Handle` continuation:
```csharp
return await maybeChannel.Match(
Some: async channel =>
{
Validation<BaseError, Channel> validation = await Validate(dbContext, request, channel, cancellationToken);
return await validation.Match(
Succ: async c =>
{
Either<BaseError, string> resolvedLogo = await ResolveLogoPath(request, cancellationToken);
return await resolvedLogo.Match(
Right: async logoPath => Right<BaseError, ChannelViewModel>(
await ApplyUpdateRequest(dbContext, c, request, logoPath, cancellationToken)),
Left: e => Task.FromResult(Left<BaseError, ChannelViewModel>(e)));
},
Fail: errors => Task.FromResult(Left<BaseError, ChannelViewModel>(errors.Head)));
},
None: () => Task.FromResult(Left<BaseError, ChannelViewModel>(
new NotFoundError($"Channel {request.ChannelId} does not exist."))));
```
where `ResolveLogoPath` returns `Right(string.Empty)`/`Right(originalPath)` for empty/non-URL and `remoteLogoCacher.CacheFromUrl(...)` for a URL, and `ApplyUpdateRequest` takes the resolved `logoPath` instead of reading `update.Logo.Path`. (Keep `ContentType` handling as-is; a downloaded logo's content type can be left null — the serve route sniffs it, per #283.)
- [ ] **Step 4: Run tests to verify they pass**
Run: `dotnet test ErsatzTV.Tests/ErsatzTV.Tests.csproj --filter "FullyQualifiedName~UpdateChannelHandlerTests"` → PASS
- [ ] **Step 5: Commit**
```bash
bash -c 'dotnet format ErsatzTV.sln --no-restore --include <touched .cs>'
git add -A && git commit -m "feat(525): download external-url logo on channel update"
```
---
### Task 5: `CreateChannelHandler` + `CreateChannelFromLineupHandler` download on create
Same treatment for the two create paths, so a channel can never be created with a URL in `Artwork.Path`.
**Files:**
- Modify: `ErsatzTV.Application/Channels/Commands/CreateChannelHandler.cs`
- Modify: `ErsatzTV.Application/Channels/Commands/CreateChannelFromLineupHandler.cs`
- Modify/Create: the corresponding `*HandlerTests` in `ErsatzTV.Tests/Application/Channels/`
**Interfaces:**
- Consumes: `IRemoteLogoCacher.CacheFromUrl`.
- [ ] **Step 1: Write the failing test** — mirror Task 4's download + fail cases for `CreateChannelHandler` (URL → cache name; cacher `Left` → save fails). Use that fixture's create helpers.
- [ ] **Step 2: Run to verify fail.**
Run: `dotnet test ErsatzTV.Tests/ErsatzTV.Tests.csproj --filter "FullyQualifiedName~CreateChannelHandlerTests"` → FAIL
- [ ] **Step 3: Implement** — inject `IRemoteLogoCacher` into both handlers; resolve a URL logo → cache name before persisting `Artwork.Path`, propagating a `Left` as the handler result. `CreateChannelFromLineupHandler` (`:360-362`) builds logo artwork from the lineup — only channels whose lineup logo is a URL need the download; a lineup that already references a local/cached path is unchanged.
- [ ] **Step 4: Run to verify pass.** → PASS
- [ ] **Step 5: Commit**
```bash
git add -A && git commit -m "feat(525): download external-url logo on channel create + create-from-lineup"
```
---
### Task 6: Apply the decode budget to `UploadArtworkHandler`
Close the pre-existing gap: a direct upload is not budget-checked, and once URL logos become uploads that inconsistency is created by this feature. One rule: anything entering the logo cache is budget-checked.
**Files:**
- Modify: `ErsatzTV.Application/Artworks/Commands/UploadArtworkHandler.cs` (validate the buffered bytes via `IRemoteImageValidator` before `SaveArtworkToCache`)
- Modify: `ErsatzTV.Tests/Application/Artworks/UploadArtworkHandlerTests.cs` (create if absent)
**Interfaces:**
- Consumes: `IRemoteImageValidator.DecodeAndValidate` (Task 2).
- [ ] **Step 1: Write the failing test**
```csharp
[Test]
public async Task Should_Reject_An_Upload_That_Busts_The_Decode_Budget()
{
// craft a tiny PNG header declaring 30000x30000 (reuse PngHeaderDeclaring helper)
await using MemoryStream bomb = PngHeaderDeclaring(30000, 30000);
var handler = new UploadArtworkHandler(ImageCache, new RemoteImageValidator());
Either<BaseError, ArtworkUploadResponseModel> result =
await handler.Handle(new UploadArtwork(bomb, ArtworkKind.Logo), CancellationToken.None);
result.IsLeft.ShouldBeTrue();
result.IfLeft(e => e.Value.ShouldContain("pixel limit"));
}
[Test]
public async Task Should_Accept_A_Normal_Upload()
{
await using MemoryStream png = await RealPng(64, 64);
var handler = new UploadArtworkHandler(ImageCache, new RemoteImageValidator());
Either<BaseError, ArtworkUploadResponseModel> result =
await handler.Handle(new UploadArtwork(png, ArtworkKind.Logo), CancellationToken.None);
result.IsRight.ShouldBeTrue();
}
```
- [ ] **Step 2: Run to verify fail.**
Run: `dotnet test ErsatzTV.Tests/ErsatzTV.Tests.csproj --filter "FullyQualifiedName~UploadArtworkHandlerTests"` → FAIL (validator not a ctor param; bomb currently accepted)
- [ ] **Step 3: Implement** — add `IRemoteImageValidator validator` to `UploadArtworkHandler`'s constructor. After the content-type sniff and before `SaveArtworkToCache`, decode-validate the bytes:
```csharp
using (var probe = new MemoryStream(bytes, writable: false))
{
try
{
await validator.Validate(probe, new Uri("upload://artwork"), cancellationToken);
}
catch (Exception ex)
{
return BaseError.New($"Image cannot be used: {ex.Message}");
}
}
```
(`upload://artwork` is a synthetic Uri for the message text only.)
- [ ] **Step 4: Run to verify pass.** → PASS. Also run the full `Artworks` + `Channels` test folders.
- [ ] **Step 5: Commit**
```bash
git add -A && git commit -m "feat(525): budget-check direct artwork uploads (close the upload gap)"
```
---
### Task 7: `WatermarkSelector` stops treating a URL logo as renderable
After migration, a logo path is a URL only for a row that failed migration. Such a row must degrade to "no bug" with a warning, never fetch.
**Files:**
- Modify: `ErsatzTV.Core/FFmpeg/WatermarkSelector.cs` (`ChannelLogoWatermarkOptions`, `:301-325`)
- Modify: `ErsatzTV.Core.Tests/FFmpeg/WatermarkSelectorChannelLogoTests.cs`
**Interfaces:**
- Produces: for an external-URL logo path, `ChannelLogoWatermarkOptions` returns `None` and logs a warning (was: returned the URL as `ImagePath` for render-time fetch, added in #502).
- [ ] **Step 1: Write the failing test** — extend `WatermarkSelectorChannelLogoTests`: a channel whose logo `Artwork.Path` is `https://example.com/logo.png` yields `None` (no watermark), and the existing cached-local-path case still renders. Assert the URL case does NOT produce a `WatermarkOptions` with the URL as `ImagePath`.
- [ ] **Step 2: Run to verify fail.**
Run: `dotnet test ErsatzTV.Core.Tests/ErsatzTV.Core.Tests.csproj --filter "FullyQualifiedName~WatermarkSelectorChannelLogo"` → FAIL (URL still passed through)
- [ ] **Step 3: Implement** — in `ChannelLogoWatermarkOptions`, replace the `if (Artwork.IsExternalUrl(logoArtwork.Path)) return new WatermarkOptions(watermark, logoArtwork.Path, None);` branch with:
```csharp
if (Artwork.IsExternalUrl(logoArtwork.Path))
{
// As of #525 an external-URL logo is downloaded and cached at save time, so a URL here
// means a row that failed migration. Do not fetch at render time; degrade to no bug.
logger.LogWarning(
"Channel logo for channel {Channel} is still an un-downloaded URL {Url}; re-save the "
+ "channel to download it. Rendering without an on-screen bug.",
channel.Number,
logoArtwork.Path);
return None;
}
```
(Confirm `logger` and `channel` are in scope in that method; the recon shows `logger` is injected and `channel` is the parameter.)
- [ ] **Step 4: Run to verify pass.** → PASS. Also run `ChannelPlaylistGoldenTests` + `ChannelGuideGoldenTests` (M3U/XMLTV still emit the raw URL for a not-yet-migrated row — those consumers are unchanged; goldens should be green).
- [ ] **Step 5: Commit**
```bash
git add -A && git commit -m "feat(525): render path no longer fetches a URL logo; degrades to no bug"
```
---
### Task 8: One-time startup migration of existing URL logo rows
Convert `Artwork` rows whose `Path` is an `http(s)` URL and kind `Logo` into cached rows. Failures leave the row + warn. Idempotent.
**Files:**
- Create: `ErsatzTV/Services/RunOnce/ExternalLogoMigratorService.cs`
- Modify: `ErsatzTV/Startup.cs` (register in the run-once block)
- Create: `ErsatzTV.Tests/Services/ExternalLogoMigratorTests.cs` (test the migration method against `InMemoryTvContext`)
**Interfaces:**
- Consumes: `IRemoteLogoCacher.CacheFromUrl`, `TvContext`, `SystemStartup.WaitForDatabase`.
- [ ] **Step 1: Write the failing test** — extract the migration body into an internal static/instance method `MigrateAsync(TvContext db, IRemoteLogoCacher cacher, ILogger, CancellationToken)` so it is testable without hosting. Tests:
- a row with a URL path is converted to the cache name (cacher returns `Right`), `IsExternalUrl()` false afterward;
- a row whose cacher returns `Left` is left unchanged (still the URL) and a warning is logged (assert via a substituted `ILogger` `Received` or just that the path is unchanged);
- a second run over already-migrated rows calls the cacher zero times (idempotent — only URL rows are selected).
- [ ] **Step 2: Run to verify fail.**
Run: `dotnet test ErsatzTV.Tests/ErsatzTV.Tests.csproj --filter "FullyQualifiedName~ExternalLogoMigrator"` → FAIL (type absent)
- [ ] **Step 3: Implement** — mirror `DatabaseCleanerService` (primary-ctor `IServiceScopeFactory` + `ILogger<>` + `SystemStartup`; `Task.Yield()`; `await systemStartup.WaitForDatabase`; scope → `TvContext`; resolve `IRemoteLogoCacher` from the scope). Selection: EF-side filter is awkward (`IsExternalUrl` is C#), so load logo artwork and filter in memory: `db.Artwork.Where(a => a.ArtworkKind == ArtworkKind.Logo)``.Where(a => a.IsExternalUrl())`. For each: `CacheFromUrl(new Uri(a.Path))` → on `Right` set `a.Path = name; a.DateUpdated = DateTime.UtcNow;` on `Left` log a warning naming the row/channel; `SaveChangesAsync` once at the end. Register after `DatabaseMigratorService` / `DatabaseCleanerService` so the schema exists.
- [ ] **Step 4: Run to verify pass.** → PASS
- [ ] **Step 5: Commit**
```bash
git add -A && git commit -m "feat(525): startup migration converts existing URL logo rows to cache"
```
---
### Task 9: SPA — preview a saved logo, drop the stale copy, inline error on rejected save
**Files:**
- Modify: `web/src/screens/ChannelEditScreen.tsx` (remove `&& !externalUrlLogo` preview suppression; delete the "cannot drive the bug" help text; surface the save 400 inline on the URL field; simplify the mutual-exclusion now that a URL never survives a save)
- Modify: `web/src/screens/ChannelEditScreen.test.tsx`
- Modify: `docs/spa-conventions.md` only if a documented screen convention changes (likely not)
**Interfaces:**
- Consumes: the channel `PUT` now returns a normal cached logo on success and a `400` with a specific message on a bad URL.
- [ ] **Step 1: Write the failing test** — in `ChannelEditScreen.test.tsx`:
- after a successful save of a channel whose logo was an external URL, the logo preview renders (the `&& !externalUrlLogo` suppression is gone);
- a save that returns a `400` "Could not download logo…" shows that message inline near the URL field and does not navigate away;
- the removed help text ("cannot be used as the on-screen bug") is absent.
- [ ] **Step 2: Run to verify fail.**
Run: `cd web && npx vitest run src/screens/ChannelEditScreen.test.tsx` → FAIL
- [ ] **Step 3: Implement** — delete the `externalUrlLogo` branch in the "Use logo as on-screen bug" help (`:797-803`), remove the `&& !externalUrlLogo` guard on the preview (`:817`), and render the save error (from the existing `ApiError` handling) beside the External-logo-URL `Input`. Keep the URL field as an input that, on a successful save, is cleared and the cached logo shown (hydration already treats an external URL specially at `:136`/`:164` — since a saved logo is no longer external, that path naturally stops triggering).
- [ ] **Step 4: Run to verify pass.**
Run: `cd web && npm run typecheck && npx vitest run src/screens/ChannelEditScreen.test.tsx` → PASS
- [ ] **Step 5: Commit**
```bash
git add -A && git commit -m "feat(525): SPA previews saved logos, drops stale external-URL copy"
```
---
### Task 10: Docs + final gate
**Files:**
- Modify: `docs/decisions.md` (new entry — see below)
- Modify: `docs/channels.md` (replace the "External logo URLs drive the bug… fetched at render time" text with the download-on-save behavior + the save-time failure)
- Modify: `docs/api-conventions.md` (note `PUT /api/v1/channels/{id}` and `POST /api/v1/artwork/uploads` can now `400` on a bad/oversized/over-budget logo)
- Modify: `docs/README.md` only if a doc is added/retitled (no)
- [ ] **Step 1: `docs/decisions.md` entry** (append at EOF + index line). Must state: external logo URLs are downloaded and cached at save time (content-hash name, identical to an upload); this **supersedes the #511 "not cached, re-fetched per element init" paragraph** and **narrows #502's "external artwork passes through"** to the client-facing consumers (M3U/XMLTV/SPA still emit whatever `Artwork.Path` resolves to — now a cache URL, not the external URL); the decode budget is shared (`RemoteImageDecodeBudget`) and now also guards direct uploads; the render path no longer fetches a logo (a leftover URL row degrades to no bug + warning); migration is a startup task, failures left intact; no refresh button by design (re-add the URL).
- [ ] **Step 2: `docs/channels.md`** — rewrite the external-logo paragraph to the new behavior.
- [ ] **Step 3: OpenAPI** — response shapes are unchanged (still `ChannelViewModel` / `ArtworkUploadResponseModel`), only error status/messages differ, so `v1.json` likely does not change. Run `./scripts/update-openapi.sh` and `git diff --exit-code docs/v1.json`; commit only if it actually changed.
- [ ] **Step 4: Full local gate** (BEFORE any push):
```bash
dotnet build ErsatzTV.sln # Build succeeded, 0 warnings
dotnet test ErsatzTV.sln # all green
cd web && npm run typecheck && npm run test && cd ..
# BOM + format on the touched set:
for f in $(git diff --name-only origin/main...HEAD -- '*.cs'); do head -c3 "$f" | xxd -p | grep -q '^efbbbf' && echo "BOM: $f"; done
bash -c 'mapfile -t files < <(git diff --name-only --diff-filter=ACM origin/main...HEAD -- "*.cs"); dotnet format whitespace . --folder --verify-no-changes --include "${files[@]}"'
```
- [ ] **Step 5: Cold adversarial review** over the whole diff (mandatory here — this touches API write-path handlers and a data migration; see the review skip rubric). Fold fixes in, then push and open the PR (arm the CI monitor at open). Live-E2E the write path (`scripts/e2e-local.sh`): create a channel with an external-URL logo, confirm it downloads + previews + the M3U emits an `/iptv/logos/` URL; a deliberately-bad URL is rejected in the editor.
- [ ] **Step 6: Commit + PR**
```bash
git add -A && git commit -m "docs(525): record download-on-save; supersede #511 not-cached note"
git push -u origin feat/525-external-logo-download-on-save
```
---
## Self-Review
**Spec coverage:** save path (Tasks 4/5) ✓; content-hash naming (reuses `SaveArtworkToCache`) ✓; downstream no-change (verified — nothing in M3U/XMLTV/mapper touched) ✓; render path stops fetching (Task 7) ✓; decode validation shared + uploads folded in (Tasks 1/2/3/6) ✓; migration startup task, failures intact (Task 8) ✓; synchronous save + 400 (Task 4 + Task 9) ✓; preview works, stale copy gone (Task 9) ✓; docs incl. superseding #511 + narrowing #502 (Task 10) ✓; `IRemoteImageFetcher` namespace unchanged (respected — not touched) ✓.
**Placeholder scan:** the two `<verbatim body …>` markers in Task 2 point at an exact existing method (`ImageElementBase.DecodeRemoteImage`, quoted in the spec's source recon) to move unchanged — not new logic to invent. One explicit VERIFY (does `ErsatzTV.Core` reference ImageSharp) with a stated fallback, because the answer changes the interface signature and must be checked in-repo rather than guessed.
**Type consistency:** `IRemoteLogoCacher.CacheFromUrl → Task<Either<BaseError,string>>` (Task 3) is what Tasks 4/5/8 consume; the Core `IRemoteImageValidator.Validate → Task` (Task 2, throws-on-invalid, no ImageSharp type) is what Tasks 3/6 consume; the Infra static `RemoteImageValidator.DecodeAndValidate → Task<Image>` (Task 2) is what `ImageElementBase` delegates to; `RemoteImageDecodeBudget` static members (Task 1) are consumed by Task 2. Names match across tasks. **Layering note resolved:** Core does not reference ImageSharp, so the Core interface returns `Task`, not `Image`.
@@ -0,0 +1,217 @@
# Decision-lifecycle + retrieval-efficient startup — design (ersatztv #520 + #521)
Date: 2026-07-21
Issues: [#521](http://192.168.1.95:3000/timothy/ersatztv/issues/521) (decision knowledge lifecycle),
[#520](http://192.168.1.95:3000/timothy/ersatztv/issues/520) (retire #237 from startup; parallel orientation + selection).
Both originate from a bounded knowledge-retrieval audit in `timothy/adversarial-reviewer`.
## Goal
Two coupled problems from the same audit:
1. **#521** — `docs/decisions.md` (~2,468 lines; ~3,469 incl. topic files) is optimized for *append
safety*, not *current-state retrieval*. Status lives in prose, supersession is inferred at
release time, and fresh agents are told to read the corpus broadly before the task is known.
2. **#520** — Startup still tells agents (and the selector fallback) to read the closed arc tracker
#237 and its stale forward-looking comments, and preloads the whole doc corpus before the task is
known. `scripts/select-queue.sh` already does live mechanical selection correctly.
The fix is one arc: **give decisions a stable identity + explicit lifecycle + a compact generated
active view (#521), then point startup at that compact view and run orientation ‖ selection in
parallel (#520).** Keep Git-backed Markdown authoritative; MemPalace stays candidate-discovery only.
## Non-goals
- Making MemPalace the source of truth; storing live queue state in decision docs.
- Reopening #237 or creating a new manual session-log issue / Markdown ledger.
- Implementing the MemPalace/Gitea exporter (`server-management#642`).
- Changing `select-queue.sh`'s dependency/tier/priority algorithm (only its comments/output text),
unless a test exposes a bug.
## Delegation model (agreed)
- **Judgment (orchestrator / escalated):** schema, validator, catalog generator, kickoff + docs
rewrite, and reconciling the prose-only reversals (#390#406; #411 measurement obsolescence).
- **Mechanical (balanced/cheap subagents, fanned out by topic batch):** reformatting existing
entries into the keyed schema. Each subagent gets exact schema rules + the batch's entries + a
worktree/branch of its own; results merge back. No two committing agents share a worktree.
## Guard rework (the crux)
The current append-only guard blocks **any** modified/deleted line in `decisions.md` unless the
commit carries `[decisions-edit]` (`.claude/hooks/decisions-guard.sh`, wired to Husky `commit-msg`
and the CI `decisions-guard` job). That line-level mechanic is **incompatible** with a lifecycle
model where migration reformats every entry, supersession *edits* a predecessor's `superseded-by:`
line, and retiring *moves* a record to archive.
**Decision:** replace the line-level mechanic with a **lifecycle validator** that enforces the same
*spirit* — rationale is never silently rewritten or deleted; every history touch is deliberate and
reviewable. The `[decisions-edit]` token is **kept narrow, not retired** (revised after Fable review):
routine lifecycle *metadata* writes (add record, set `superseded-by`, relocate to archive, regen
catalog) are token-free and proven well-formed by the validator; a change to a record's *rationale
prose* still requires `[decisions-edit]`, enforced by the validator's **body-diff**. Retiring the
token entirely would let an agent silently rewrite rationale (or archive a falsified body) and pass —
the exact case the old guard existed for.
Spirit preserved by these validator invariants (see Validator below): ≤1 active record per key; no
record block may vanish (if it leaves the active set it must reappear under `docs/decisions/archive/`);
**a surviving record's rationale prose can't change, and an archived copy must body-match its pre-move
active version, without `[decisions-edit]` in the commit range**; supersession links are reciprocal
and resolvable; status only moves active → superseded/retired. The no-vanish/body-diff checks run in
CI over `merge-base(base,head)…head` across `decisions.md` **and** every topic file.
**Delivery: two PRs, one arc** (revised after Fable review). PR1 = machinery (parser, validator,
catalog generator, guard swap, header rewrite, exemplar + append-only→lifecycle supersession); PR2 =
full corpus migration + #520 startup rewrite + kickoff guard + retrieval-eval. Matches #521's
"bounded topic batches, avoid a single conflict-heavy rewrite" and keeps each diff's conflict surface
small on the treadmill-prone `decisions.md`.
## Information model (#521)
Each decision record carries a compact, **visible**, deterministically-parseable metadata block
immediately under its `##` heading (front-loaded so MemPalace indexes key/status/rule first):
```
## 2026-07-17 — No persistent compiler servers in CI … (#406)
`key: ci.runner-placement` · `status: active` · `since: 2026-07-17` · `supersedes: none` · `superseded-by: none`
**Rule:** Every CI `services:` container gets an explicit CPU/mem cap; no shared persistent compiler daemon.
**Signals:** ci, runners, services, memory-cap · paths: .gitea/workflows/docker-build.yml · issues: #390 #406
**Mechanics:** docs/ci-cd.md → CI lanes
<existing rationale prose, incl. rejected alternatives — unchanged>
```
Fields (issue-required):
- `key` — stable subject identity, dotted lowercase (`^[a-z0-9]+(\.[a-z0-9-]+)+$`), e.g.
`ci.runner-placement`. A superseding record reuses the predecessor's key.
- `status``active` | `superseded` | `retired` | `legacy-unmigrated` (the last is a transitional
marker; see Migration). "Relocated"/"consolidated" are maintenance actions, not statuses.
- `since` — effective date (YYYY-MM-DD) or version.
- `supersedes` / `superseded-by` — predecessor/successor key+date, or `none`.
- `**Rule:**` — one-line current rule, before the long rationale.
- `**Signals:**` — concepts, `paths:`, `issues:` for task-driven retrieval.
- `**Mechanics:**` — link(s) to the current convention/source/test anchors.
- Rationale prose (incl. rejected alternatives) follows, unchanged from today where migrated.
Rejected alternatives stay *inside* their accepted record as rationale; they are never active
records.
## Document architecture (#521)
- `docs/decisions.md` — stays the chronological record store for in-file (non-topic) decisions, now
in schema form. Header rewritten: append-only-log framing → lifecycle framing.
- `docs/decisions/*.md` — existing topic files, entries migrated to schema form.
- **`docs/decisions/README.md` — NEW generated active catalog / task router.** A compact table
(key · one-line rule · status · record link · signals) built by
`scripts/build-decisions-catalog.py` from all `active` records across `decisions.md` + topic files.
Regenerated like `endpoint-index.md`; drift fails CI. This is the compact "active view" startup
points at.
- **`docs/decisions/archive/` — NEW.** `superseded`/`retired` records relocated here, out of the
default startup path, with forward/back links preserved. Not scanned by the catalog generator.
## Validator (#521)
`scripts/decisions-validate.py`, run locally (a Husky hook replacing the append-only commit-msg
check) and in the CI `decisions-guard` job (renamed → lifecycle guard). Checks:
1. Required metadata present + well-formed on every structured record.
2. Valid `status` enum; valid `key` format.
3. **≤1 `active` record per key.**
4. Reciprocal, resolvable `supersedes`/`superseded-by` links (successor exists; predecessor points
back).
5. **No-vanish (diff-aware):** any record block removed from the active files (`decisions.md` +
topic files) between base and HEAD must be present under `docs/decisions/archive/`. Fail-open on
missing refs, like the current guard.
6. Active-catalog completeness + freshness (regenerate to a temp file, diff — like `check:api`).
7. Archive/active placement consistency (no `active` record in archive; no `superseded`/`retired`
in the active set once migration of its key is complete).
8. **Aggregate active-corpus budget** — sum of lines across `decisions.md` + topic files + catalog
(excluding archive). Warn over a threshold (start at the current 1800 applied to the aggregate;
tune during migration). Report the `legacy-unmigrated` remainder count; it must be visible and
trend to zero.
9. Best-effort broken-link check on `Mechanics:`/archive links where practical.
Fail-open on tooling trouble, matching the current guard's philosophy.
## Startup / kickoff rewrite (#520)
- **`docs/handoffs/chicorytv-issue-queue.md`** — retire #237 from normal startup. Two concurrent
tracks when no issue is named: **orientation** (`AGENTS.md`, `CLAUDE.md`, compact `docs/README.md`
map + the active catalog) ‖ **selection** (`scripts/select-queue.sh 5`). After both: resolve only
`CLAIM?`/`UMBRELLA?` flags + the winner, recheck live state, claim, then build a focused
task-specific knowledge packet. Named-issue path skips selection → focused retrieval directly.
Archive the contradictory *pre-script* selector lore (the "re-derive the whole contested tier by
hand" bullets) into a clearly-labeled historical section. #237 appears only as labeled archival
history.
- **`docs/README.md`** — mandatory 110 reading order → compact **task-signal → minimal sections**
authority/task map, plus a pointer to `docs/decisions/README.md` as the decision active view.
- **`CLAUDE.md`** — docs-first guidance requires the map + relevant sections, not the whole corpus.
Task-completion protocol gains the structured `## Closing record` template.
- **`scripts/select-queue.sh`** — comments/output: active tiers lead with open
milestones/review/priorities; drop "read #237" phrasing.
- **Structured closing record** — documented template (Outcome / Root cause / Decisions-conventions
changed / Reusable knowledge / Verification / Deferred-follow-ups / Docs updated) as the future
per-issue history record. No second manual ledger.
### Regression check (#520 "cannot regress" box)
`scripts/check-kickoff-guard.sh` (shares the lifecycle CI job): fails if active kickoff/README/
CLAUDE text reintroduces "#237 is the queue / source of truth / read #237 for current state"
patterns. Allow-list the explicitly-archival mentions.
## Migration plan (#521 — full, via subagents)
1. Land schema rules + validator + catalog generator + archive skeleton first (validator tolerant of
`legacy-unmigrated`).
2. Inventory legacy entries by stable subject (not just chronology); assign keys.
3. Reconcile known prose-only reversals/retirements first (#390#406 supersession; #411
obsolescence) — orchestrator/escalated, not a cheap subagent.
4. Fan out mechanical reformatting in bounded topic batches (each subagent: exact schema + its
entries + its own branch). Merge back.
5. Move superseded/retired records to `docs/decisions/archive/` with links.
6. Regenerate the catalog; flip aggregate budget check on; drive `legacy-unmigrated` to zero.
7. Preserve an auditable mapping from every legacy heading to its active record or archive location
(a migration map committed alongside).
Safety valve: if mechanical migration balloons, land complete machinery + a partial migration with a
visible remainder count + a tracked follow-up issue (issue done-when explicitly allows a bounded
remainder with no ambiguous active rules).
## Retrieval evaluation (#521)
A bounded question bank (committed, e.g. `docs/decisions/retrieval-eval.md` or a test fixture)
covering: paraphrased task→decision discovery; exact code/path lookup; active-vs-superseded; retired
features; rationale/rejected-alternatives; and at least one "convention already implemented"
question that must prevent reimplementation. Success = correct **active-record selection + citation**,
not merely a semantically-related passage. Executed as the cold-agent step below (deterministic
grading where the answer is a specific key/citation).
## Verification
- **Deterministic:** validator + regression check + catalog freshness all green locally before push.
- **Behavioral (cold-agent sim, #520 done-when):** a fresh subagent handed only the new
kickoff/README produces the startup flow — orientation ‖ selector, resolve flags, recheck+claim
winner, focused retrieval — and **never reads #237**. Second cold agent runs a sample of the
retrieval question bank against the new active view.
- **Independent adversarial review** of the whole diff (mandatory: >150 lines, touches CI) before
push; cross-model if available, else a cold review-only agent.
- Docs updated in the same PR (README index, decisions header, ci-cd release ritual, CLAUDE.md).
## CI / release-ritual changes
- Rename/extend the `decisions-guard` job → run `decisions-validate.py` + `check-kickoff-guard.sh` +
catalog freshness.
- Release ritual (`docs/ci-cd.md` → Versioning & releases): change "discover superseded entries" →
"validate lifecycle metadata + reciprocal links; archive already-classified history; refresh the
active catalog; enforce the aggregate budget; report unresolved legacy records."
- Remove `[decisions-edit]` from the Husky `commit-msg` hook and the CI job; update docs/memory that
reference the token.
## Risks
- **Conflict surface:** `decisions.md` conflicts on nearly every CI cycle here; this arc rewrites it
wholesale. Mitigation: single PR, rebase-then-`merge_when_checks_succeed`, land fast.
- **Regenerated-artifact merges:** on rebase conflicts in the generated catalog, regenerate (never
hand-merge), same discipline as `v1.json`.
- **Memory/lore references** to `[decisions-edit]` and "#237 is the queue" are spread across
MEMORY.md and the handoff lore; sweep by subject.
@@ -0,0 +1,220 @@
# External channel-logo URLs become download-on-save
**Date:** 2026-07-21
**Status:** design, awaiting approval
**Relates to:** #502 (external URL logos reach the graphics engine), #511 / PR #518 (bounded
render-time fetch), #1 (generated-initials `localhost` URL), #510 (deco path)
## Problem
A channel logo set as an **external URL** is stored raw in `Artwork.Path` and passed through to every
consumer. The render path therefore has to fetch it over HTTP *during stream startup*, once per
playout item, while ffmpeg waits on the pipe. #511 bounded that fetch (10s deadline, 10 MiB wire cap,
3 redirects, decode budgets) but did not remove it.
Bounding the fetch treats the symptom. The fetch itself is the problem:
- **Failure is invisible and late.** A dead, slow, oversized or non-image URL surfaces as a log line
at render time. The operator who typed the URL is long gone.
- **No preview.** The editor cannot show the bug for an external URL, so the operator cannot tell
whether it will work until a stream runs.
- **Repeated work.** The same image is re-fetched on every playout item transition.
- **Third-party dependency inside stream startup.** A logo host having a bad day degrades tuning.
## Goal
**An external logo URL becomes an input method, not a storage format.** Entering a URL downloads the
image once, at save time, into the existing artwork cache — after which it is indistinguishable from
an uploaded logo. Nothing downstream knows the logo ever came from a URL.
To refresh a changed image, the operator re-enters the URL. There is no refresh button and no
staleness tracking; that is a deliberate simplification, not an oversight.
## Non-goals
- **No refresh button, no TTL, no ETag/Last-Modified tracking.** Re-add the URL.
- **No change to `ImageGraphicsElement`** (operator-authored YAML `image:`), which may still point at
a URL and still fetches at render time through the hardened `IRemoteImageFetcher`. Removing that is
an unrelated feature removal.
- **No change to the generated-initials fallback** (#1) or the deco path (#510).
- **No new artwork storage mechanism.** Reuses `IImageCache` exactly as the upload path does.
## Design
### Save path
All three handlers that persist a channel logo share one code path today and will share the new one:
| Handler | Current logo logic |
|---|---|
| `UpdateChannelHandler.ApplyUpdateRequest` | `UpdateChannelHandler.cs:79-123` |
| `CreateChannelHandler` | `CreateChannelHandler.cs:63-65` |
| `CreateChannelFromLineupHandler` | `CreateChannelFromLineupHandler.cs:360-362` |
New behavior when the incoming logo path is an absolute `http(s)` URL:
1. Fetch it with **`IRemoteImageFetcher`** — the primitive #511 already built and hardened (bounded
deadline covering headers and body, 10 MiB wire cap, 3 redirects, content-type check, pooled
client).
2. **Validate the decode budgets** against the downloaded bytes (see *Decode validation* below).
3. `IImageCache.SaveArtworkToCache(stream, ArtworkKind.Logo)` → an opaque content-hash name.
4. Store that name in `Artwork.Path`, stamp `DateAdded`/`DateUpdated`, exactly as the upload path does.
Any failure **rejects the save** with a validation error naming the cause. The channel is not
persisted and the field stays editable.
### Naming: content hash, not GUID
The request was "a random name/guid". This design uses the **existing content hash** that
`SaveArtworkToCache` already returns (MD5 of the bytes, stored as `{hash}` with the file at
`{LogoCacheFolder}/{hash[..2]}/{hash}`).
Rationale — it satisfies the intent (opaque, generated, not the URL) while being *strictly better*
than a GUID here:
- It is byte-for-byte the same mechanism as an uploaded logo, so there is one storage convention
rather than two.
- Re-adding an **unchanged** URL is a natural no-op (same bytes → same hash → same file).
- Re-adding a **changed** URL naturally produces a new name, which is exactly the refresh semantic.
A GUID would deviate from the established convention for no benefit, which the deviation policy in
`docs/contributing.md` §10 asks us not to do.
### Downstream consumers: no code change
Because `Artwork.Path` now holds a cache name, every consumer already does the right thing:
| Consumer | Result |
|---|---|
| M3U (`ChannelPlaylist.cs:63-70`) | `{scheme}://{host}{baseUrl}/iptv/logos/{hash}.jpg` |
| XMLTV (`RefreshChannelListHandler.cs:85-95`, `_channel.sbntxt:29-35`) | `{RequestBase}/iptv/logos/{hash}.jpg` |
| SPA/API mapper (`Channels/Mapper.cs:129-166`) | `iptv/logos/{hash}` |
| Render (`WatermarkSelector.cs:301-325`) | resolves via `imageCache.GetPathForImage`, existence-checked |
This is the intended outcome: clients stop depending on the third-party host, and the
`IsExternalUrl` branches in those consumers become unreachable *for channel logos*. Those branches
are **left in place**`Artwork` is shared with other artwork kinds and with rows that failed
migration.
### Render path
`WatermarkSelector.ChannelLogoWatermarkOptions` stops treating a URL as renderable. For a logo path
that is still a URL (only possible for a row that failed migration), it logs a warning naming the
channel and returns `None` — no fetch, no bug, stream unaffected.
`ImageElementBase.LoadImage` keeps its remote branch for `ImageGraphicsElement`. #511's fetch and
decode budgets stay exactly as merged.
### Decode validation (important)
Today `ImageElementBase` exempts **local** images from the decode budgets, on the reasoning that a
local file is something an operator put on disk rather than bytes an arbitrary host returned. This
design invalidates that reasoning for logos: a downloaded URL *becomes* a local file, so without a
check at save time the decode bomb simply relocates from the render path to the cache.
Therefore the save path must validate before caching:
- Reuse #511's budgets — dimensions, `width × height × frames ≤ 50 MP`, `≤ 600` frames — enforced the
same way (`DecoderOptions.MaxFrames` + post-decode re-verification against the decoded image,
because header frame counts lie).
- To share them, extract the budget helpers currently on `ImageElementBase`
(`EnsureDimensionsAffordable`, `AffordableFrames`, `EnsureDecodeAffordable`) into a single reusable
component. Proposed: `ErsatzTV.Core/Images/RemoteImageDecodeBudget.cs`, with `ImageElementBase` and
the save path both calling it. The retention budget (`EnsureScaledFramesAffordable`) stays in
`ImageElementBase` — it depends on render-time scale and has no meaning at save time.
**Uploads are budget-checked too (decided: fold in).** Direct **uploads** (`UploadArtworkHandler`)
are not budget-checked today. Once URL logos become uploads, they inherit that gap on any subsequent
re-upload — an inconsistency this change would *create* (same bytes, same cache, enforcement
depending only on arrival path). The `RemoteImageDecodeBudget` component is being built regardless, so
`UploadArtworkHandler` calls it too. One consistent rule: **anything entering the logo cache is
budget-checked, however it arrived.** A budget failure returns a `400` from the upload endpoint the
same way it does from the channel save. (Risk is admin-only, like #511's SSRF stance, but the failure
mode — cache succeeds, render OOMs concurrent streams later — is exactly the fail-late pattern this
redesign exists to kill, so it is closed here rather than deferred.)
### Migration of existing rows
A one-time migration walks `Artwork` rows whose `Path` is an absolute `http(s)` URL and whose kind is
`Logo`:
- fetch through the same hardened fetcher → validate → `SaveArtworkToCache` → rewrite `Path`, bump
`DateUpdated`;
- **on failure, leave the row untouched** and log a warning naming the channel and the reason, so the
operator gets an actionable list rather than silent breakage.
Run as a **startup task**, not an EF migration: it performs network I/O and must be resilient and
restartable, which does not belong in a schema migration (and would have to be written twice for
SQLite and MySql). It follows the existing precedent of `LocalFolderScanner.RefreshArtwork`
(`LocalFolderScanner.cs:132-200`), which already does fetch → `*ArtworkToCache` → persist.
Idempotent by construction: after a successful pass the row's `Path` is no longer a URL, so it is not
selected again.
### Editor UX
- Save is **synchronous**: the `PUT` performs the download and returns `400` with a specific message
on failure (e.g. *"Could not download logo: host did not respond within 10s"*, *"Logo is 41 MB;
the limit is 10 MB"*, *"URL returned text/html, not an image"*). Worst case latency is the fetch
deadline.
- On success the response carries a normal cached logo, so **the preview works with no special
casing** — the `&& !externalUrlLogo` suppression at `ChannelEditScreen.tsx:817` is deleted, as is
the help text claiming external URLs cannot drive the bug.
- The external-URL field is an *input*: after a successful save it clears and the uploaded-logo
preview shows the cached image. The existing mutual-exclusion logic
(`ChannelEditScreen.tsx:119-123`) is simplified accordingly — the two fields can no longer disagree
because only one storage form now exists.
- Help text states that changing the remote image requires re-entering the URL.
## Error handling
| Case | Behavior |
|---|---|
| Host unreachable / times out | Save rejected, message names the timeout |
| Non-2xx | Save rejected, message names the status |
| Not an image content type | Save rejected |
| Over the wire cap | Save rejected, message names actual vs limit |
| Over a decode budget | Save rejected, message names dimensions/frames vs limit |
| Cache write fails | Save rejected, `BaseError` surfaced |
| Migration failure | Row untouched, warning logged, channel keeps rendering without a bug |
## Testing
- **Handler tests** (`ErsatzTV.Tests`): URL → fetch → cache → `Artwork.Path` is the hash; each failure
mode rejects the save and persists nothing; a non-URL path is unchanged; re-adding identical bytes
is a no-op.
- **Decode budget tests**: move/extend the existing `RemoteImageDecodeLimitTests`, keeping the APNG
regression coverage (a default `Identify` throws on most APNGs; header frame counts lie).
- **Migration tests**: URL row is converted; failing row is left intact and warned about; a second run
is a no-op.
- **`WatermarkSelector` tests**: extend `WatermarkSelectorChannelLogoTests` — a cached path renders; a
leftover URL path returns `None` with a warning and never fetches.
- **SPA tests**: `ChannelEditScreen.test.tsx` — preview renders after a URL save; the removed
suppression is not reintroduced; error surfaces inline on a rejected save.
- **Golden nets**: `ChannelPlaylistGoldenTests` / `ChannelGuideGoldenTests` should be *unchanged* for
uploaded logos, and a channel whose logo came from a URL should now emit an `/iptv/logos/` URL.
## Docs to update in the same PR
- `docs/decisions.md` — new entry; explicitly supersedes the "not cached, re-fetched per element
init" paragraph of the #511 entry and narrows #502's "external artwork passes through" to the
client-facing consumers it still describes.
- `docs/channels.md` — replace the stale limitation text (this supersedes PR #522, which should be
closed unmerged).
- `docs/api-conventions.md``PUT /api/v1/channels/{id}` can now fail on logo download; note the new
400 cases. Regenerate `v1.json` + `endpoint-index.md` if any response shape changes.
## Out of scope / follow-ups
- `ArtworkController.RedirectArtwork` (`ArtworkController.cs:37-57`) builds `"/iptv/logos/" + Path`
unconditionally, producing a malformed redirect when `Path` is a URL. Pre-existing, unrelated to
this change, and largely mooted by it for logos — **file separately**.
- #1 (generated-initials `localhost`) and #510 (deco path) remain untouched.
## Resolved decisions
1. **`UploadArtworkHandler` decode validation is folded into this PR**, not deferred — see *Decode
validation*. The component exists either way and the inconsistency is created by this change.
2. **`IRemoteImageFetcher` stays in `Core/Interfaces/Streaming/`.** It is still used by the streaming
path (YAML image elements), and a namespace move is churn against `git blame` for weak
naming-accuracy benefit. Trivial standalone rename if ever wanted.
View File
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env python3
"""Generate docs/decisions/README.md — the compact ACTIVE decision catalog / task router.
Do not edit docs/decisions/README.md by hand; regenerate with this script. The decisions-validate
CI job checks it is in sync.
"""
from __future__ import annotations
import sys
from pathlib import Path
import scripts.decisions_lib as dl
OUTPUT = dl.TOPIC_DIR / "README.md"
BANNER = "<!-- GENERATED by scripts/build_decisions_catalog.py — do not edit by hand. -->"
def _rel(rec) -> str:
"""Link from docs/decisions/README.md to the record's heading."""
src = Path(rec.source).name
anchor = _anchor(rec.heading)
if src == "decisions.md":
return f"../decisions.md#{anchor}"
return f"{src}#{anchor}"
def _anchor(heading: str) -> str:
# Gitea slugger: lowercase, keep alphanumerics + underscore, spaces/hyphens → '-' each (one
# hyphen PER such char — Gitea does NOT collapse runs of '-'), drop other punctuation. No
# collapsing, no trimming. Keeping '_' matters — headings like "…(`iptv.base_url`)…" anchor to
# …iptvbase_url…. A " — " (space em-dash space) heading therefore anchors with "--", e.g.
# "2026-07-19 — Foo" → "2026-07-19--foo". VERIFIED against this repo's own working Index
# anchors — do not "fix" this back to a collapsed form without re-checking against Gitea.
out = []
for ch in heading.lower():
if ch.isalnum() or ch == "_":
out.append(ch)
elif ch in " -":
out.append("-")
return "".join(out)
def render_catalog(records) -> str:
active = sorted(
[r for r in records if r.status == "active" and r.key],
key=lambda r: r.key,
)
lines = [
BANNER,
"",
"# Active decisions — catalog / task router",
"",
"The compact current view of settled decisions. Each row is an **active** record; follow",
"the link for rationale. Superseded/retired history lives in `archive/`. Regenerated by",
"`scripts/build_decisions_catalog.py`.",
"",
"| Key | Current rule | Since | Record |",
"| --- | --- | --- | --- |",
]
for r in active:
rule = (r.rule or "").replace("|", "\\|")
lines.append(f"| `{r.key}` | {rule} | {r.since or ''} | [link]({_rel(r)}) |")
lines.append("")
return "\n".join(lines)
def main(argv=None) -> int:
check = "--check" in (argv or sys.argv[1:])
want = render_catalog(dl.all_active_records())
have = OUTPUT.read_text(encoding="utf-8") if OUTPUT.exists() else ""
if check:
if want.strip() != have.strip():
print(
"build_decisions_catalog: docs/decisions/README.md is stale — regenerate.",
file=sys.stderr,
)
return 1
print("build_decisions_catalog: catalog up to date")
return 0
OUTPUT.write_text(want.rstrip("\n") + "\n", encoding="utf-8")
print(f"build_decisions_catalog: wrote {OUTPUT}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
# ersatztv#520 — block re-treating the CLOSED arc tracker #237 as live queue state / source of truth.
# Scans active startup docs for forbidden phrasings. A hit is exempt when the line itself contains
# "archiv" OR the line is inside an archival section (a heading whose text contains "archiv", up to
# the next heading of the same or higher level). The "archiv" escape is an anti-footgun convenience,
# not an anti-adversary control.
set -euo pipefail
FILES=(
"docs/handoffs/chicorytv-issue-queue.md"
"docs/README.md"
"CLAUDE.md"
"scripts/select-queue.sh"
)
# Case-insensitive patterns that assert #237 is live/authoritative.
PATTERNS='read #237|#237.?s (body|arc|comments)|tracker #237|queue state lives in|read that, not this file'
rc=0
for f in "${FILES[@]}"; do
[ -f "$f" ] || continue
archive_level=0 # 0 = not in an archival section; else the heading level that opened it
lineno=0
while IFS= read -r line; do
lineno=$((lineno + 1))
# Track archival-section state via Markdown ATX headings.
if [[ "$line" =~ ^(#{1,6})[[:space:]] ]]; then
level=${#BASH_REMATCH[1]}
if printf '%s' "$line" | grep -qi 'archiv'; then
archive_level=$level
elif [ "$archive_level" -ne 0 ] && [ "$level" -le "$archive_level" ]; then
archive_level=0 # a same-or-higher heading closes the archival section
fi
continue
fi
[ "$archive_level" -ne 0 ] && continue # inside archival section → exempt
printf '%s' "$line" | grep -qi 'archiv' && continue # per-line escape
if printf '%s' "$line" | grep -qiE "$PATTERNS"; then
echo "kickoff-guard: $f:$lineno reintroduces #237-as-live-state: ${line}" >&2
rc=1
fi
done < "$f"
done
[ "$rc" -eq 0 ] && echo "kickoff-guard: OK"
exit "$rc"
+126
View File
@@ -0,0 +1,126 @@
#!/usr/bin/env python3
"""Parse ErsatzTV decision records from docs/decisions.md and docs/decisions/*.md.
A decision record is a Markdown H2 section. A *migrated* record carries a visible metadata
block as its first non-blank content:
## 2026-07-17 — Title … (#406)
`key: ci.runner-placement` · `status: active` · `since: 2026-07-17` · `supersedes: none` · `superseded-by: none`
**Rule:** one-line current rule.
**Signals:** concept · paths: a/b.yml · issues: #406
**Mechanics:** docs/ci-cd.md CI lanes
<rationale prose >
An H2 with no metadata line is treated as status `legacy-unmigrated` (migration target).
"""
from __future__ import annotations
import re
from dataclasses import dataclass
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
DECISIONS_MD = REPO_ROOT / "docs" / "decisions.md"
TOPIC_DIR = REPO_ROOT / "docs" / "decisions"
ARCHIVE_DIR = TOPIC_DIR / "archive"
STATUSES = {"active", "superseded", "retired", "legacy-unmigrated"}
KEY_RE = re.compile(r"^[a-z0-9]+(\.[a-z0-9-]+)+$")
_HEADING_RE = re.compile(r"^##\s+(.*\S)\s*$")
_META_FIELD_RE = re.compile(r"`([a-z-]+):\s*([^`]*)`")
@dataclass
class Record:
heading: str
source: Path
lineno: int
key: str | None = None
status: str = "legacy-unmigrated"
since: str | None = None
supersedes: str | None = None
superseded_by: str | None = None
rule: str | None = None
signals: str | None = None
mechanics: str | None = None
body: str = ""
def _parse_meta_line(line: str) -> dict[str, str]:
return {m.group(1): m.group(2).strip() for m in _META_FIELD_RE.finditer(line)}
def parse_text(text: str, source: Path) -> list[Record]:
lines = text.splitlines()
records: list[Record] = []
i = 0
while i < len(lines):
m = _HEADING_RE.match(lines[i])
if not m:
i += 1
continue
rec = Record(heading=m.group(1), source=source, lineno=i + 1)
j = i + 1
body_lines: list[str] = []
while j < len(lines) and not _HEADING_RE.match(lines[j]):
body_lines.append(lines[j])
j += 1
for k, bl in enumerate(body_lines):
if not bl.strip():
continue
meta = _parse_meta_line(bl)
if "key" in meta and "status" in meta:
rec.key = meta.get("key") or None
rec.status = meta.get("status") or "legacy-unmigrated"
rec.since = meta.get("since") or None
rec.supersedes = meta.get("supersedes") or None
rec.superseded_by = meta.get("superseded-by") or None
# The metadata block is contiguous: scan only until the first blank line, so a
# bolded **Rule:** appearing later inside rationale prose can't overwrite the real one.
for bl2 in body_lines[k + 1 :]:
if not bl2.strip():
break
if bl2.startswith("**Rule:**"):
rec.rule = bl2[len("**Rule:**") :].strip()
elif bl2.startswith("**Signals:**"):
rec.signals = bl2[len("**Signals:**") :].strip()
elif bl2.startswith("**Mechanics:**"):
rec.mechanics = bl2[len("**Mechanics:**") :].strip()
break
rec.body = "\n".join(body_lines).strip()
records.append(rec)
i = j
return records
def parse_file(path: Path) -> list[Record]:
return parse_text(path.read_text(encoding="utf-8"), path)
def metadata_line_count(rec: Record) -> int:
"""Count how many lines in `rec.body` look like a metadata line (a line starting with
`` `key: `` after stripping). A well-formed record has exactly 1; more indicates a
duplicate/stacked metadata block left behind by a botched migration."""
count = 0
for line in rec.body.splitlines():
if line.strip().startswith("`key:"):
count += 1
return count
_NON_DECISION_FILES = {"README.md", "migration-map.md", "retrieval-eval.md"}
def active_files() -> list[Path]:
files = [DECISIONS_MD]
files += sorted(p for p in TOPIC_DIR.glob("*.md") if p.name not in _NON_DECISION_FILES)
return files
def all_active_records() -> list[Record]:
recs: list[Record] = []
for f in active_files():
if f.exists():
recs += parse_file(f)
return recs
+334
View File
@@ -0,0 +1,334 @@
#!/usr/bin/env python3
"""Validate ErsatzTV decision-record lifecycle invariants.
Replaces the line-level append-only guard (ersatztv#303 H9) with lifecycle checks that preserve its
spirit rationale is never silently rewritten or deleted; every history touch is deliberate and
reviewable. `[decisions-edit]` is kept ONLY for rationale-prose edits (see body-diff below); routine
lifecycle metadata writes are token-free. Fail-open on tooling trouble (missing refs, git errors,
parse issues), matching the old guard.
"""
from __future__ import annotations
import argparse
import subprocess
import sys
from pathlib import Path
import scripts.decisions_lib as dl # noqa: E402 (run with PYTHONPATH=. or as module)
SKIP_HEADINGS = {"Index", "Active catalog", "Contents"}
REQUIRED_META = ("key", "status", "since", "supersedes", "superseded_by")
EDIT_TOKEN = "[decisions-edit]" # noqa: S105 (a commit-message marker, not a credential)
def _key_of(ref: str | None) -> str | None:
if not ref or ref == "none":
return None
return ref.split("@", 1)[0].strip()
def validate(
records,
*,
archive_keys,
catalog_ok,
budget_ok,
removed,
rewritten,
archive_records=None,
demoted=(),
) -> list[str]:
errs: list[str] = []
archive_records = archive_records or []
decision_recs = [r for r in records if r.heading not in SKIP_HEADINGS and r.status != "legacy-unmigrated"]
active_by_key: dict[str, int] = {}
known_keys = set(archive_keys)
for r in decision_recs:
if r.status not in dl.STATUSES:
errs.append(f"{r.heading}: invalid status {r.status!r}")
if not r.key or not dl.KEY_RE.match(r.key):
errs.append(f"{r.heading}: bad key format {r.key!r}")
continue
known_keys.add(r.key)
for f in REQUIRED_META:
if getattr(r, f) in (None, ""):
errs.append(f"{r.heading}: missing required metadata {f}")
if r.status == "active":
active_by_key[r.key] = active_by_key.get(r.key, 0) + 1
if r.status in ("superseded", "retired"):
errs.append(
f"{r.heading}: status {r.status} but still in the active set — relocate to docs/decisions/archive/"
)
n_meta = dl.metadata_line_count(r)
if n_meta > 1:
errs.append(f"{r.heading}: {n_meta} metadata blocks found (expected 1) — duplicate metadata block")
for r in archive_records:
if r.status == "active":
errs.append(f"{r.heading}: active record must not live under docs/decisions/archive/")
for key, n in active_by_key.items():
if n > 1:
errs.append(f"key {key!r}: more than one active record ({n})")
# by_key map across BOTH wings, for reciprocity checks (existence-only checks still use
# known_keys, which additionally includes keys the caller only knows about via archive_keys).
by_key: dict[str, dl.Record] = {}
for r in decision_recs + list(archive_records):
if r.key:
by_key[r.key] = r
# reciprocal supersession (successor/predecessor may live in active OR archive). Runs over the
# union of active + archive records so a twice-reversed decision (an archived record whose
# superseded-by points to ANOTHER archived record) is checked from both sides too.
for r in decision_recs:
if r.status == "active" and r.superseded_by not in (None, "", "none"):
errs.append(f"{r.heading}: active record cannot already be superseded (superseded-by set)")
for r in decision_recs + list(archive_records):
sk = _key_of(r.superseded_by)
if sk:
if sk not in known_keys:
errs.append(f"{r.heading}: superseded-by points to unknown key {sk!r}")
else:
b = by_key.get(sk)
if b is not None and r.key and _key_of(b.supersedes) != r.key:
errs.append(f"{r.heading}: superseded-by {sk} but {sk} does not point back (supersedes)")
pk = _key_of(r.supersedes)
if pk:
if pk not in known_keys:
errs.append(f"{r.heading}: supersedes points to unknown key {pk!r}")
else:
a = by_key.get(pk)
if a is not None and r.key and _key_of(a.superseded_by) != r.key:
errs.append(f"{r.heading}: supersedes {pk} but {pk} does not point back (superseded-by)")
for h in removed:
errs.append(f"record removed from the active set without an archive copy: {h!r}")
for h in rewritten:
errs.append(f"rationale prose of {h!r} changed without the {EDIT_TOKEN} token")
for h in demoted:
errs.append(f"{h}: migrated record demoted to legacy-unmigrated (metadata block removed)")
if not catalog_ok:
errs.append("docs/decisions/README.md active catalog is stale — run build_decisions_catalog.py")
if not budget_ok:
# non-blocking: reported as a warning by the caller (main()), never added here.
pass
return errs
# ---- git helpers (all fail-open: return neutral values on any error) ----
def _run(args: list[str]) -> str | None:
try:
r = subprocess.run(args, capture_output=True, text=True, timeout=30)
except (OSError, subprocess.SubprocessError):
return None
return r.stdout if r.returncode == 0 else None
def _merge_base(base: str, head: str) -> str | None:
out = _run(["git", "merge-base", base, head])
return out.strip() if out else None
def _active_paths_at(ref: str) -> list[str]:
"""Active decision files at a ref: decisions.md + docs/decisions/*.md minus README/archive/migration-map."""
out = _run(["git", "ls-tree", "-r", "--name-only", ref, "docs/decisions.md", "docs/decisions/"])
if out is None:
return []
paths = []
for p in out.splitlines():
if not p.endswith(".md"):
continue
if p.startswith("docs/decisions/archive/") or Path(p).name in dl._NON_DECISION_FILES:
continue
paths.append(p)
return paths
def _records_at(ref: str, paths: list[str]):
"""{heading: Record} across the given paths at a ref."""
by_heading = {}
for p in paths:
blob = _run(["git", "show", f"{ref}:{p}"])
if blob is None:
continue
for rec in dl.parse_text(blob, Path(p)):
if rec.heading not in SKIP_HEADINGS:
by_heading[rec.heading] = rec
return by_heading
def _archive_records_at(ref: str):
out = _run(["git", "ls-tree", "-r", "--name-only", ref, "docs/decisions/archive/"])
paths = [p for p in (out or "").splitlines() if p.endswith(".md")]
by_heading = {}
for p in paths:
blob = _run(["git", "show", f"{ref}:{p}"])
if blob is None:
continue
for rec in dl.parse_text(blob, Path(p)):
by_heading[rec.heading] = rec
return by_heading
def _rationale(rec) -> str:
"""Record body with the contiguous top metadata block stripped, whitespace-normalized.
Only the block from the first non-blank line (when it is the `key:` meta line) up to the first
following blank line is metadata. A **Rule:**/**Signals:**/**Mechanics:**/`key:` line appearing
later in rationale prose is prose, not metadata (mirrors decisions_lib's contiguous-block rule).
"""
lines = rec.body.splitlines()
start = 0
while start < len(lines) and not lines[start].strip():
start += 1
if start < len(lines) and lines[start].strip().startswith("`key:"):
end = start + 1
while end < len(lines) and lines[end].strip():
end += 1
rest = lines[:start] + lines[end:]
else:
rest = lines
return "\n".join(s for s in (ln.strip() for ln in rest) if s)
def _diff_findings(base: str, head: str) -> tuple[list[str], list[str], list[str]]:
"""(removed, rewritten, demoted) between merge-base(base,head) and head. Fail-open → ([], [], [])."""
mb = _merge_base(base, head)
if not mb:
print(
f"::warning::decisions-validate: could not resolve merge-base({base},{head}); no-vanish/body-diff skipped",
file=sys.stderr,
)
return [], [], []
if EDIT_TOKEN.lower() in (_run(["git", "log", "--format=%B", f"{mb}..{head}"]) or "").lower():
token = True
else:
token = False
base_active = _records_at(mb, _active_paths_at(mb))
head_active = _records_at(head, _active_paths_at(head))
head_archive = _archive_records_at(head)
base_archive = _archive_records_at(mb)
gone = set(base_active) - set(head_active)
removed = sorted(h for h in gone if h not in head_archive)
# an archive record present at base must not vanish entirely (neither wing has it at head)
archive_gone = set(base_archive) - set(head_archive)
removed += sorted(h for h in archive_gone if h not in head_active)
rewritten: list[str] = []
if not token:
# surviving records whose rationale prose changed
for h in set(base_active) & set(head_active):
if _rationale(base_active[h]) != _rationale(head_active[h]):
rewritten.append(h)
# archived records must body-match their base active copy (no laundering rewrites via archive)
for h in gone & set(head_archive):
if _rationale(base_active[h]) != _rationale(head_archive[h]):
rewritten.append(h)
# archive records that survive in the archive wing: rationale must not be rewritten either
for h in set(base_archive) & set(head_archive):
if _rationale(base_archive[h]) != _rationale(head_archive[h]):
rewritten.append(h)
# migrated (had a key) at base, demoted to legacy-unmigrated (no key, or key changed) at head,
# same heading, still present in the active set.
demoted: list[str] = []
for h in set(base_active) & set(head_active):
base_key = base_active[h].key
head_key = head_active[h].key
if base_key and base_key != head_key:
demoted.append(h)
return sorted(set(removed)), sorted(set(rewritten)), sorted(demoted)
def _archive_keys() -> set[str]:
keys: set[str] = set()
if dl.ARCHIVE_DIR.exists():
for f in dl.ARCHIVE_DIR.glob("*.md"):
for r in dl.parse_file(f):
if r.key:
keys.add(r.key)
return keys
def _budget_total() -> int:
total = 0
for f in dl.active_files():
if f.exists():
total += len(f.read_text(encoding="utf-8").splitlines())
cat = dl.TOPIC_DIR / "README.md"
if cat.exists():
total += len(cat.read_text(encoding="utf-8").splitlines())
return total
def _budget_ok(limit: int) -> bool:
return _budget_total() <= limit
def _catalog_ok() -> bool:
try:
import scripts.build_decisions_catalog as bc # pyright: ignore[reportMissingImports]
except Exception:
return True # fail-open; the CI job also runs an independent --check step
want = bc.render_catalog(dl.all_active_records())
cat = dl.TOPIC_DIR / "README.md"
have = cat.read_text(encoding="utf-8") if cat.exists() else ""
return want.strip() == have.strip()
def main(argv=None) -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--base")
ap.add_argument("--head")
ap.add_argument("--budget", type=int, default=4800) # aggregate; re-baselined post-#521 migration
args = ap.parse_args(argv)
records = dl.all_active_records()
archive_records = []
if dl.ARCHIVE_DIR.exists():
for f in dl.ARCHIVE_DIR.glob("*.md"):
archive_records += dl.parse_file(f)
removed, rewritten, demoted = _diff_findings(args.base, args.head) if args.base and args.head else ([], [], [])
budget_ok = _budget_ok(args.budget)
errs = validate(
records,
archive_keys=_archive_keys(),
catalog_ok=_catalog_ok(),
budget_ok=budget_ok,
removed=removed,
rewritten=rewritten,
archive_records=archive_records,
demoted=demoted,
)
if not budget_ok:
print(
f"::warning::decisions-validate: aggregate active-corpus is {_budget_total()} lines "
f"(budget {args.budget}) — schedule a consolidation",
file=sys.stderr,
)
unmigrated = [r for r in records if r.status == "legacy-unmigrated" and r.heading not in SKIP_HEADINGS]
if unmigrated:
print(f"::notice::{len(unmigrated)} legacy-unmigrated decision record(s) remain (must trend to 0).")
if errs:
for e in errs:
print(f"decisions-validate: {e}", file=sys.stderr)
return 1
print("decisions-validate: OK")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+6 -5
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env bash
# scripts/select-queue.sh — deterministic queue selector for the #237 pickup protocol.
# scripts/select-queue.sh — deterministic queue selector (live Gitea state, never a tracker issue).
#
# WHY THIS EXISTS. The kickoff dispatches a cheap model to rank the backlog, and the lore then makes
# the orchestrator RE-DERIVE the selector's mechanical claims because a small model kept getting them
@@ -25,8 +25,9 @@
# whose most-recent comments look like a live claim so the orchestrator reads them before claiming.
# - UMBRELLA? — an epic/umbrella whose children are the real pickups reads like a normal issue; the
# script flags bodies that open with umbrella framing so the orchestrator treats it as a container.
# - ARC tier — arc order lives in #237's prose (currently CLOSED / arc complete), not in any label,
# so this script deliberately does NOT rank an arc tier. If an arc ever re-opens, read #237.
# - ARC tier — there is no active arc right now (maintenance/backlog mode; arc complete), so this
# script deliberately does not rank an arc tier. If a new arc/milestone is ever established, add
# it as a tier here — never fall back to hand-ranking from a tracker's prose.
#
# Usage:
# ETV_GITEA_BASICAUTH=user:pass scripts/select-queue.sh [N]
@@ -94,7 +95,7 @@ rows=$(
| (($L | index("review")) != null) as $isReview
| ((.milestone != null) and (.milestone.state == "open")) as $inOpenMs
# tier: 2=open-milestone, 3=review, 4/5/6=priority high/medium/low (unmilestoned/unreviewed).
# (Tier 1 = arc; not label-derivable, handled by the orchestrator via #237 prose.)
# (Tier 1 = arc; there is no active arc right now — see the header comment above.)
| (if $inOpenMs then 2
elif $isReview then 3
elif $prank==0 then 4
@@ -162,4 +163,4 @@ done <<< "$rows"
echo
echo "Ranked by (tier, priority, issue#). DEPS/tiering/ordering are deterministic — trust them."
echo "Resolve CLAIM?/UMBRELLA? flags before claiming (read the issue's comments/body). Arc tier: read #237."
echo "Resolve CLAIM?/UMBRELLA? flags before claiming (read the issue's comments/body). No active arc tier right now."
View File
+15
View File
@@ -0,0 +1,15 @@
# Decisions
## Index
- [x](#y)
## 2026-07-17 — Migrated example (#406)
`key: ci.runner-placement` · `status: active` · `since: 2026-07-17` · `supersedes: none` · `superseded-by: none`
**Rule:** Every CI services container gets an explicit cap.
**Signals:** ci, runners · paths: .gitea/workflows/docker-build.yml · issues: #390 #406
**Mechanics:** docs/ci-cd.md → CI lanes
Rationale prose here.
## 2026-07-11 — Legacy unmigrated example (#231)
Some prose with no metadata line at all.
+51
View File
@@ -0,0 +1,51 @@
from pathlib import Path
import scripts.build_decisions_catalog as bc
import scripts.decisions_lib as dl
def test_catalog_lists_only_active_sorted_by_key():
recs = [
dl.Record(
heading="H2",
source=Path("x"),
lineno=1,
key="z.a",
status="active",
rule="Zeta rule",
),
dl.Record(
heading="H1",
source=Path("x"),
lineno=1,
key="a.b",
status="active",
rule="Alpha rule",
),
dl.Record(
heading="Old",
source=Path("x"),
lineno=1,
key="a.b",
status="superseded",
rule="old",
),
]
out = bc.render_catalog(recs)
assert "a.b" in out and "z.a" in out
assert out.index("a.b") < out.index("z.a") # sorted
assert "Alpha rule" in out and "Zeta rule" in out
assert "old" not in out # superseded excluded
assert "GENERATED" in out # do-not-edit banner
def test_anchor_matches_gitea_double_hyphen_slug():
# Ground truth: Gitea does NOT collapse hyphen runs. " — " (space, em dash, space) becomes
# "--" in the anchor (one hyphen per space/dash char), never collapsed to a single "-".
a = bc._anchor(
"2026-07-19 — CI `test` job reports a sampled true peak-anon, not cache-inflated `memory.peak` (#412)"
)
assert a == ("2026-07-19--ci-test-job-reports-a-sampled-true-peak-anon-not-cache-inflated-memorypeak-412")
b = bc._anchor("2026-07-16 — Optional advertised IPTV base URL (`iptv.base_url`)…")
assert "iptvbase_url" in b
+34
View File
@@ -0,0 +1,34 @@
from pathlib import Path
import scripts.decisions_lib as dl
FIX = Path(__file__).parent / "fixtures" / "sample_decisions.md"
def test_parses_migrated_record():
recs = dl.parse_file(FIX)
migrated = [r for r in recs if r.key == "ci.runner-placement"]
assert len(migrated) == 1
r = migrated[0]
assert r.status == "active"
assert r.since == "2026-07-17"
assert r.supersedes == "none"
assert r.superseded_by == "none"
assert r.rule == "Every CI services container gets an explicit cap."
assert r.signals is not None
assert "issues: #390 #406" in r.signals
assert r.mechanics is not None
assert r.mechanics.startswith("docs/ci-cd.md")
def test_legacy_record_is_unmigrated():
recs = dl.parse_file(FIX)
legacy = [r for r in recs if r.heading.endswith("(#231)")]
assert len(legacy) == 1
assert legacy[0].key is None
assert legacy[0].status == "legacy-unmigrated"
def test_index_section_parses_as_heading():
recs = dl.parse_file(FIX)
assert any(r.heading == "Index" for r in recs)
+398
View File
@@ -0,0 +1,398 @@
import subprocess
from pathlib import Path
from typing import Any
import scripts.decisions_lib as dl
import scripts.decisions_validate as dv
def _rec(**kw: Any) -> dl.Record:
base: dict[str, Any] = dict(
heading="H",
source=Path("x"),
lineno=1,
status="active",
since="2026-01-01",
supersedes="none",
superseded_by="none",
)
base.update(kw)
return dl.Record(**base)
def _v(recs: list[dl.Record], **kw: Any) -> list[str]:
args: dict[str, Any] = dict(
archive_keys=set(),
catalog_ok=True,
budget_ok=True,
removed=[],
rewritten=[],
archive_records=[],
demoted=[],
)
args.update(kw)
return dv.validate(recs, **args)
def test_two_active_same_key_fails():
assert any("more than one active" in e for e in _v([_rec(key="a.b"), _rec(key="a.b")]))
def test_bad_key_format_fails():
assert any("key format" in e for e in _v([_rec(key="BadKey")]))
def test_dangling_superseded_by_fails():
recs = [_rec(key="a.b", status="superseded", superseded_by="a.c@2026-07-01")]
assert any("superseded-by" in e and "a.c" in e for e in _v(recs))
def test_superseded_by_resolves_to_archive_key_passes():
# successor lives in archive → known via archive_keys, no dangling error
recs = [_rec(key="a.b", status="superseded", superseded_by="a.c@2026-07-01")]
assert not any("superseded-by" in e for e in _v(recs, archive_keys={"a.c"}))
def test_reciprocal_superseded_by_with_no_back_link_fails():
# B exists (has a record) but its `supersedes` does not point back to A → dangling back-link.
a = _rec(heading="A", key="a.b", status="superseded", superseded_by="a.c@2026-07-01")
b = _rec(heading="B", key="a.c", status="active", supersedes="none")
assert any("does not point back" in e and "a.c" in e for e in _v([a], archive_records=[b], archive_keys={"a.c"}))
def test_reciprocal_supersession_correct_pair_passes():
# active supersedes archived; archived is superseded-by active → reciprocal, no errors.
active = _rec(heading="Active", key="a.b", status="active", supersedes="a.c@2026-01-01")
archived = _rec(heading="Archived", key="a.c", status="superseded", superseded_by="a.b@2026-07-01")
errs = _v([active], archive_records=[archived], archive_keys={"a.c"})
assert errs == []
def test_archive_to_archive_reciprocal_pair_passes():
# A twice-reversed decision: both records live in the archive wing, pointing at each other.
a = _rec(heading="A", key="a.b", status="superseded", superseded_by="a.c@2026-07-01")
b = _rec(heading="B", key="a.c", status="retired", supersedes="a.b@2026-07-01")
errs = _v([], archive_records=[a, b], archive_keys={"a.b", "a.c"})
assert errs == []
def test_archive_to_archive_broken_back_link_fails():
# A.superseded-by=B but B.supersedes=none → the back-link is broken, and neither side is active
# (so this could previously only be caught by iterating archive_records, not decision_recs).
a = _rec(heading="A", key="a.b", status="superseded", superseded_by="a.c@2026-07-01")
b = _rec(heading="B", key="a.c", status="retired", supersedes="none")
errs = _v([], archive_records=[a, b], archive_keys={"a.b", "a.c"})
assert any("does not point back" in e and "a.c" in e for e in errs)
def test_active_record_with_superseded_by_fails():
recs = [_rec(key="a.b", status="active", superseded_by="a.c@2026-07-01")]
assert any("active record cannot already be superseded" in e for e in _v(recs, archive_keys={"a.c"}))
def test_removed_active_not_in_archive_fails():
assert any("removed from the active set" in e for e in _v([], removed=["2026-01-01 — Gone (#9)"]))
def test_rewritten_rationale_without_token_fails():
assert any(
"rationale" in e and "decisions-edit" in e
for e in _v([_rec(key="a.b")], rewritten=["2026-01-01 — Reworded (#9)"])
)
def test_clean_corpus_passes():
assert _v([_rec(key="a.b"), _rec(key="c.d")]) == []
def test_superseded_record_in_active_fails():
recs = [_rec(key="a.b", status="superseded", superseded_by="a.c@2026-01-01")]
assert any("relocate to docs/decisions/archive/" in e for e in _v(recs, archive_keys={"a.c"}))
def test_active_record_in_archive_fails():
assert any(
"must not live under docs/decisions/archive/" in e
for e in _v([], archive_records=[_rec(key="a.b", status="active")])
)
def test_missing_since_fails():
rec = dl.Record(
heading="H",
source=Path("x"),
lineno=1,
key="a.b",
status="active",
since=None,
supersedes="none",
superseded_by="none",
)
assert any("missing required metadata since" in e for e in _v([rec]))
def test_demoted_heading_fails():
assert any(
"demoted to legacy-unmigrated" in e for e in _v([_rec(key="a.b")], demoted=["2026-01-01 — Some decision (#1)"])
)
def test_duplicate_metadata_block_fails():
# A migration bug left records with two stacked metadata blocks; the parser only reads the
# first, so only an explicit body scan (metadata_line_count) can catch the leftover second block.
body = (
"`key: a.b` · `status: active` · `since: 2026-01-01` · `supersedes: none` · `superseded-by: none`\n"
"**Rule:** first block rule.\n"
"\n"
"`key: a.b` · `status: active` · `since: 2026-01-01` · `supersedes: none` · `superseded-by: none`\n"
"**Rule:** second block rule.\n"
"\n"
"Rationale prose goes here."
)
rec = _rec(key="a.b", body=body)
assert dl.metadata_line_count(rec) == 2
assert any("duplicate metadata block" in e and "2 metadata blocks" in e for e in _v([rec]))
def test_single_metadata_block_passes():
body = (
"`key: a.b` · `status: active` · `since: 2026-01-01` · `supersedes: none` · `superseded-by: none`\n"
"**Rule:** the only rule.\n"
"\n"
"Rationale prose goes here."
)
rec = _rec(key="a.b", body=body)
assert dl.metadata_line_count(rec) == 1
assert not any("duplicate metadata block" in e for e in _v([rec]))
def test_contents_heading_is_skipped():
# docs/decisions/*.md topic files use "## Contents" as their index heading (the analog of
# decisions.md's "## Index") — it carries no metadata and must not be miscounted as a
# legacy-unmigrated record, same as "Index" already isn't.
assert "Contents" in dv.SKIP_HEADINGS
rec = dl.Record(
heading="Contents",
source=Path("x"),
lineno=1,
status="legacy-unmigrated",
since=None,
supersedes=None,
superseded_by=None,
)
# validate() itself excludes SKIP_HEADINGS from decision_recs, so a bare Contents record
# produces no errors (it isn't checked for required metadata, bad status, etc.)
assert _v([rec]) == []
# mirrors main()'s legacy-unmigrated count filter
unmigrated = [r for r in [rec] if r.status == "legacy-unmigrated" and r.heading not in dv.SKIP_HEADINGS]
assert unmigrated == []
def _git(cwd: Path, *args: str) -> None:
subprocess.run(["git", *args], cwd=cwd, check=True, capture_output=True, text=True)
def _write_decisions(path: Path, *, status: str, since: str, rationale: str) -> None:
path.write_text(
"## 2026-01-01 — Some decision (#1)\n"
f"`key: a.b` · `status: {status}` · `since: {since}` · `supersedes: none` · `superseded-by: none`\n"
"**Rule:** one-line current rule.\n"
"**Signals:** concept · paths: a/b.py · issues: #1\n"
"**Mechanics:** docs/foo.md\n"
"\n"
f"{rationale}\n",
encoding="utf-8",
)
def test_diff_engine_detects_rationale_rewrite_without_token(tmp_path, monkeypatch):
repo = tmp_path / "repo"
(repo / "docs").mkdir(parents=True)
_git(tmp_path, "init", str(repo))
_git(repo, "config", "user.email", "t@example.com")
_git(repo, "config", "user.name", "T")
_write_decisions(repo / "docs" / "decisions.md", status="active", since="2026-01-01", rationale="Original prose.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "base")
monkeypatch.chdir(repo)
# (a) rewrite rationale prose only, no token
_write_decisions(
repo / "docs" / "decisions.md",
status="active",
since="2026-01-01",
rationale="**Rule:** this looks like metadata but is prose appended later.",
)
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "rewrite rationale")
removed, rewritten, *_rest = dv._diff_findings("HEAD~1", "HEAD")
assert removed == []
assert "2026-01-01 — Some decision (#1)" in rewritten
def test_diff_engine_allows_rewrite_with_token(tmp_path, monkeypatch):
repo = tmp_path / "repo"
(repo / "docs").mkdir(parents=True)
_git(tmp_path, "init", str(repo))
_git(repo, "config", "user.email", "t@example.com")
_git(repo, "config", "user.name", "T")
_write_decisions(repo / "docs" / "decisions.md", status="active", since="2026-01-01", rationale="Original prose.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "base")
monkeypatch.chdir(repo)
_write_decisions(repo / "docs" / "decisions.md", status="active", since="2026-01-01", rationale="Reworded prose.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "rewrite rationale [decisions-edit]")
removed, rewritten, *_rest = dv._diff_findings("HEAD~1", "HEAD")
assert removed == []
assert rewritten == []
def test_diff_engine_detects_appended_smuggled_rationale(tmp_path, monkeypatch):
repo = tmp_path / "repo"
(repo / "docs").mkdir(parents=True)
_git(tmp_path, "init", str(repo))
_git(repo, "config", "user.email", "t@example.com")
_git(repo, "config", "user.name", "T")
_write_decisions(repo / "docs" / "decisions.md", status="active", since="2026-01-01", rationale="Original prose.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "base")
monkeypatch.chdir(repo)
# append-shape exploit: original prose line is PRESERVED, a smuggled **Rule:** paragraph is
# appended after a blank line, no [decisions-edit] token. The old buggy _rationale() strips
# ANY `**Rule:**`/`**Signals:**`/`**Mechanics:**`/`` `key:`` line wherever it appears, so it
# silently strips the appended line too → base==head → bypass succeeds. The bounded strip only
# removes the contiguous top metadata block, so the appended paragraph survives as prose →
# base!=head → flagged.
_write_decisions(
repo / "docs" / "decisions.md",
status="active",
since="2026-01-01",
rationale="Original prose.\n\n**Rule:** smuggled rewrite that changes the actual meaning.",
)
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "append smuggled rewrite")
removed, rewritten, *_rest = dv._diff_findings("HEAD~1", "HEAD")
assert removed == []
assert "2026-01-01 — Some decision (#1)" in rewritten
def test_diff_engine_metadata_only_edit_is_free(tmp_path, monkeypatch):
repo = tmp_path / "repo"
(repo / "docs").mkdir(parents=True)
_git(tmp_path, "init", str(repo))
_git(repo, "config", "user.email", "t@example.com")
_git(repo, "config", "user.name", "T")
_write_decisions(repo / "docs" / "decisions.md", status="active", since="2026-01-01", rationale="Original prose.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "base")
monkeypatch.chdir(repo)
# metadata-only edit: status/since change, prose unchanged, no token
_write_decisions(
repo / "docs" / "decisions.md", status="superseded", since="2026-02-01", rationale="Original prose."
)
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "metadata edit")
removed, rewritten, *_rest = dv._diff_findings("HEAD~1", "HEAD")
assert removed == []
assert rewritten == []
def _write_archive(path: Path, *, rationale: str) -> None:
path.write_text(
"## 2026-01-01 — Some archived decision (#2)\n"
"`key: a.z` · `status: superseded` · `since: 2026-01-01` · `supersedes: none` · "
"`superseded-by: a.b@2026-07-01`\n"
"**Rule:** one-line historical rule.\n"
"**Signals:** concept · paths: a/z.py · issues: #2\n"
"**Mechanics:** docs/foo.md\n"
"\n"
f"{rationale}\n",
encoding="utf-8",
)
def test_diff_engine_detects_archive_rationale_rewrite_without_token(tmp_path, monkeypatch):
repo = tmp_path / "repo"
(repo / "docs" / "decisions" / "archive").mkdir(parents=True)
_git(tmp_path, "init", str(repo))
_git(repo, "config", "user.email", "t@example.com")
_git(repo, "config", "user.name", "T")
archive_path = repo / "docs" / "decisions" / "archive" / "x.md"
_write_archive(archive_path, rationale="Original archived prose.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "base")
monkeypatch.chdir(repo)
_write_archive(archive_path, rationale="Rewritten archived prose, no token.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "rewrite archived rationale")
_removed, rewritten, *_rest = dv._diff_findings("HEAD~1", "HEAD")
assert "2026-01-01 — Some archived decision (#2)" in rewritten
def test_diff_engine_detects_archive_record_removed(tmp_path, monkeypatch):
repo = tmp_path / "repo"
(repo / "docs" / "decisions" / "archive").mkdir(parents=True)
_git(tmp_path, "init", str(repo))
_git(repo, "config", "user.email", "t@example.com")
_git(repo, "config", "user.name", "T")
archive_path = repo / "docs" / "decisions" / "archive" / "x.md"
_write_archive(archive_path, rationale="Original archived prose.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "base")
monkeypatch.chdir(repo)
archive_path.unlink()
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "delete archived record")
removed, _rewritten, *_rest = dv._diff_findings("HEAD~1", "HEAD")
assert "2026-01-01 — Some archived decision (#2)" in removed
def test_diff_engine_demoted_migrated_to_legacy_unmigrated(tmp_path, monkeypatch):
repo = tmp_path / "repo"
(repo / "docs").mkdir(parents=True)
_git(tmp_path, "init", str(repo))
_git(repo, "config", "user.email", "t@example.com")
_git(repo, "config", "user.name", "T")
decisions_path = repo / "docs" / "decisions.md"
_write_decisions(decisions_path, status="active", since="2026-01-01", rationale="Original prose.")
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "base")
monkeypatch.chdir(repo)
# same heading, metadata block stripped: prose survives, migration silently reverted.
decisions_path.write_text(
"## 2026-01-01 — Some decision (#1)\n\nOriginal prose.\n",
encoding="utf-8",
)
_git(repo, "add", "-A")
_git(repo, "commit", "-m", "demote to legacy-unmigrated")
_removed, _rewritten, demoted = dv._diff_findings("HEAD~1", "HEAD")
assert "2026-01-01 — Some decision (#1)" in demoted

Some files were not shown because too many files have changed in this diff Show More