Per-channel fault detection beyond "no playout" (empty schedule, broken source) #415

Closed
opened 2026-07-17 16:32:47 +02:00 by timothy · 3 comments
Owner

Deferred from #72 (channel health / status UX) — the rest of scope item (b). #72 shipped only the no-playout fault; the channels list badge and filter are deliberately named "No playout" rather than "Problems" precisely because these classes are uncomputed (a "Problems 0" reading on a lineup holding a dead channel is a false all-clear — worse than no affordance). See docs/decisions.md 2026-07-17.

1. Empty schedule behind an existing playout

EmptyScheduleHealthCheck (ErsatzTV.Infrastructure/Health/Checks/) already runs the "schedule has playouts but zero items" query and says such schedules "WILL cause errors" — but it selects ProgramSchedule names, never channel ids.

The trap that deferred it: it only understands Classic ProgramSchedule playouts. Block, Sequential, Scripted and ExternalJson channels have no ProgramSchedule at all (docs/domain-model.md → entity chain), so a badge driven off that query would be silently absent or wrong for four of the five schedule kinds — the #71 "verify a shared primitive covers ALL variants" class of bug.

Needs a per-kind notion of "empty" first (what does an empty Block template / empty YAML mean?), then a per-channel projection.

2. Broken / missing source

FileNotFoundHealthCheck / UnavailableHealthCheck count media items server-wide; nothing maps them back to a channel. Attribution means walking media item → collection → schedule/block → playout → channel. That's a project, not a field — and needs a cost story (#72's PlayoutCount was free because GetAll already included the data).

Suggested approach

Once a fault taxonomy genuinely exists, revisit naming the SPA filter "Problems" (see willNeverPlay in ChannelsScreen.tsx — its comment explicitly asks the next person not to broaden the label until this lands).

Done-when

  • A per-channel "empty schedule" signal that is correct for all five PlayoutScheduleKind values (or explicitly scoped, with the unsupported kinds reporting unknown rather than healthy) — resolved via built-timeline detection (kind-agnostic by construction; on-demand/unbuilt → unknown)
  • A decision on broken-source attribution: implement, or record why not — implemented via the built-timeline PlayoutItem → MediaItem.State seam (exact attribution)
  • Cost measured — no N+1 on a large lineup — one bounded GROUP BY PlayoutId aggregate; integration test + live-E2E confirmed single query
  • SPA filter renamed only if the taxonomy justifies it; willNeverPlay's comment updated — renamed to "Problems"; willNeverPlay → hasProblems, comment rewritten
  • docs/decisions.md + docs/domain-model.md updated — api.channel-health-signal superseded by api.channel-health-object; domain-model + api/spa conventions updated
  • adversarial review passed — 6 per-task reviews + a frontier whole-branch review (caught + fixed a mirror-of-on-demand false-Problems bug), re-reviewed clean

Implemented in PR #581.

Deferred from **#72** (channel health / status UX) — the rest of scope item (b). #72 shipped only the **no-playout** fault; the channels list badge and filter are deliberately named "No playout" rather than "Problems" precisely because these classes are uncomputed (a "Problems 0" reading on a lineup holding a dead channel is a false all-clear — worse than no affordance). See `docs/decisions.md` 2026-07-17. ## 1. Empty schedule behind an existing playout `EmptyScheduleHealthCheck` (`ErsatzTV.Infrastructure/Health/Checks/`) already runs the "schedule has playouts but zero items" query and says such schedules "WILL cause errors" — but it selects `ProgramSchedule` **names**, never channel ids. **The trap that deferred it:** it only understands **Classic** `ProgramSchedule` playouts. **Block, Sequential, Scripted and ExternalJson** channels have no `ProgramSchedule` at all (`docs/domain-model.md` → entity chain), so a badge driven off that query would be **silently absent or wrong for four of the five schedule kinds** — the #71 "verify a shared primitive covers ALL variants" class of bug. Needs a **per-kind notion of "empty"** first (what does an empty Block template / empty YAML mean?), then a per-channel projection. ## 2. Broken / missing source `FileNotFoundHealthCheck` / `UnavailableHealthCheck` count media items **server-wide**; nothing maps them back to a channel. Attribution means walking media item → collection → schedule/block → playout → channel. That's a project, not a field — and needs a cost story (#72's `PlayoutCount` was free because `GetAll` already included the data). ## Suggested approach Once a fault taxonomy genuinely exists, revisit naming the SPA filter "Problems" (see `willNeverPlay` in `ChannelsScreen.tsx` — its comment explicitly asks the next person not to broaden the label until this lands). ## Done-when - [x] A per-channel "empty schedule" signal that is correct for **all five** `PlayoutScheduleKind` values (or explicitly scoped, with the unsupported kinds reporting *unknown* rather than *healthy*) — resolved via built-timeline detection (kind-agnostic by construction; on-demand/unbuilt → `unknown`) - [x] A decision on broken-source attribution: implement, or record why not — **implemented** via the built-timeline `PlayoutItem → MediaItem.State` seam (exact attribution) - [x] Cost measured — no N+1 on a large lineup — one bounded `GROUP BY PlayoutId` aggregate; integration test + live-E2E confirmed single query - [x] SPA filter renamed only if the taxonomy justifies it; `willNeverPlay`'s comment updated — renamed to "Problems"; `willNeverPlay` → `hasProblems`, comment rewritten - [x] `docs/decisions.md` + `docs/domain-model.md` updated — `api.channel-health-signal` superseded by `api.channel-health-object`; domain-model + api/spa conventions updated - [x] adversarial review passed — 6 per-task reviews + a frontier whole-branch review (caught + fixed a mirror-of-on-demand false-`Problems` bug), re-reviewed clean Implemented in PR #581.
timothy added the enhancementapipriority: lowfrontend labels 2026-07-17 16:32:47 +02:00
Author
Owner

🔗 Session bundle — channel health / status UX (deferred from #72): #414, #415

The two scope items deferred from #72 (channel health/status UX): #414 (distinguish auto-tuned from user-created channels — needs a real Origin signal) and #415 (per-channel fault detection beyond 'no playout'). Both extend the same channels-list health surface; see docs/decisions.md 2026-07-17.

When you claim any member, check this cluster for co-workable siblings and sweep the disjoint set in one session (per docs/handoffs/chicorytv-issue-queue.md → bundles).

🔗 **Session bundle — channel health / status UX (deferred from #72):** #414, #415 The two scope items deferred from #72 (channel health/status UX): #414 (distinguish auto-tuned from user-created channels — needs a real Origin signal) and #415 (per-channel fault detection beyond 'no playout'). Both extend the same channels-list health surface; see docs/decisions.md 2026-07-17. When you claim any member, check this cluster for co-workable siblings and sweep the disjoint set in one session (per docs/handoffs/chicorytv-issue-queue.md → bundles).
timothy added the in-progress label 2026-07-23 20:16:43 +02:00
Author
Owner

Claiming (Claude Code, Opus 4.8 orchestrator). Bundle sibling #414 has merged (#575), so this is the last open member of the channel-health bundle deferred from #72. This is design-first ("a project, not a field") — starting with scope/design on the per-kind "empty schedule" taxonomy and the broken-source-attribution go/no-go before any code.

Claiming (Claude Code, Opus 4.8 orchestrator). Bundle sibling #414 has merged (#575), so this is the last open member of the channel-health bundle deferred from #72. This is design-first ("a project, not a field") — starting with scope/design on the per-kind "empty schedule" taxonomy and the broken-source-attribution go/no-go before any code.
Author
Owner

Closing record

Outcome: Shipped in PR #581 (squash-merged). ChannelResponseModel/ChannelDetailResponseModel now carry a server-derived health object { status: Healthy|Problems|Unknown, faults[], playoutCount, brokenSourceItemCount }; the SPA channels list gained a single "Problems" rollup filter with per-fault badges (No playout/Never built/Build failed/Empty schedule/Broken source).
Root cause: N/A (feature). The pre-existing gap: only PlayoutCount==0 ("No playout") was computed per-channel; empty-schedule and broken-source faults were real but unattributed.
Decisions/conventions changed: api.channel-health-signal (#72) → superseded by new active api.channel-health-object. Chose built-timeline detection (Approach A) over per-kind config introspection: every fault derives from Playout.BuildStatus + the materialized PlayoutItem timeline (Finish >= now, MediaItem.State), which is kind-agnostic by construction (all 5 PlayoutScheduleKind covered — the #71 trap can't bite) and gives exact broken-source attribution with no media→collection→channel join. Server owns the rollup so SPA + MCP read one verdict.
Reusable knowledge: (1) The assessable gate must key on the owning channel's PlayoutMode, not the surface channel's — a mirror channel is forced Continuous, so a mirror of an on-demand source would otherwise false-flag Problems (absence signals) on the source's drained playouts while the source reports Unknown. Presence signals (BuildFailed/BrokenSource) apply in all modes; absence signals (NeverBuilt/EmptyUpcoming) suppress for on-demand. (2) A supersession's archived record must keep its rationale prose verbatim — the CI decisions lifecycle job runs the validator in diff mode (--base/--head) and flags an appended "Why superseded" paragraph as a prose edit needing [decisions-edit]; the bare validator misses this.
Verification: 1867/1867 ErsatzTV.Tests (per-kind guard, real-SQLite repo aggregate, mirror-of-on-demand regression tests); 24 SPA Vitest; live-E2E (health serializes on list+detail, no N+1); 6 per-task reviews + frontier whole-branch review (caught+fixed the mirror bug); full CI green on head 4f028120.
Deferred: A (PlayoutId, Finish) index on PlayoutItem if the aggregate is ever measured hot on a large lineup (not needed now). No follow-up issue filed.
Docs updated: docs/decisions.md (+ docs/decisions/archive/api.md), docs/domain-model.md, docs/api-conventions.md, docs/spa-conventions.md.

## Closing record **Outcome:** Shipped in PR #581 (squash-merged). `ChannelResponseModel`/`ChannelDetailResponseModel` now carry a server-derived `health` object `{ status: Healthy|Problems|Unknown, faults[], playoutCount, brokenSourceItemCount }`; the SPA channels list gained a single "Problems" rollup filter with per-fault badges (`No playout`/`Never built`/`Build failed`/`Empty schedule`/`Broken source`). **Root cause:** N/A (feature). The pre-existing gap: only `PlayoutCount==0` ("No playout") was computed per-channel; empty-schedule and broken-source faults were real but unattributed. **Decisions/conventions changed:** `api.channel-health-signal` (#72) → **superseded** by new active `api.channel-health-object`. Chose **built-timeline detection** (Approach A) over per-kind config introspection: every fault derives from `Playout.BuildStatus` + the materialized `PlayoutItem` timeline (`Finish >= now`, `MediaItem.State`), which is kind-agnostic by construction (all 5 `PlayoutScheduleKind` covered — the #71 trap can't bite) and gives exact broken-source attribution with no media→collection→channel join. Server owns the rollup so SPA + MCP read one verdict. **Reusable knowledge:** (1) The **assessable gate** must key on the *owning* channel's `PlayoutMode`, not the surface channel's — a mirror channel is forced `Continuous`, so a mirror of an on-demand source would otherwise false-flag `Problems` (absence signals) on the source's drained playouts while the source reports `Unknown`. Presence signals (`BuildFailed`/`BrokenSource`) apply in all modes; absence signals (`NeverBuilt`/`EmptyUpcoming`) suppress for on-demand. (2) A supersession's archived record must keep its **rationale prose verbatim** — the CI `decisions lifecycle` job runs the validator in *diff* mode (`--base/--head`) and flags an appended "Why superseded" paragraph as a prose edit needing `[decisions-edit]`; the bare validator misses this. **Verification:** 1867/1867 `ErsatzTV.Tests` (per-kind guard, real-SQLite repo aggregate, mirror-of-on-demand regression tests); 24 SPA Vitest; live-E2E (health serializes on list+detail, no N+1); 6 per-task reviews + frontier whole-branch review (caught+fixed the mirror bug); full CI green on head 4f028120. **Deferred:** A `(PlayoutId, Finish)` index on `PlayoutItem` if the aggregate is ever measured hot on a large lineup (not needed now). No follow-up issue filed. **Docs updated:** `docs/decisions.md` (+ `docs/decisions/archive/api.md`), `docs/domain-model.md`, `docs/api-conventions.md`, `docs/spa-conventions.md`.
timothy removed the in-progress label 2026-07-23 22:45:54 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: timothy/ersatztv#415