# Per-channel fault detection beyond "no playout" (#415)

**Status:** approved design (2026-07-23)
**Issue:** ersatztv#415 — deferred from #72 scope item (b); bundle sibling #414 merged (#575).
**Supersedes:** the temporary "raw fact only, no derived health enum" stance of `api.channel-health-signal` (#72), which was explicitly deferred pending the auto-tune status taxonomy (#383, now closed).

## Problem

The channels list surfaces exactly one per-channel fault today: **no playout** (`PlayoutCount == 0`,
#72). Three other real fault classes are invisible:

- **Empty schedule behind an existing playout.** The only existing detector,
  `EmptyScheduleHealthCheck`, is Classic-only (keyed on `ProgramSchedule.Name`, no channel id) and is
  silently absent/wrong for the other four `PlayoutScheduleKind` values (Block, Sequential, Scripted,
  ExternalJson) — the #71 "shared primitive must cover ALL variants" trap.
- **Broken / missing source.** `FileNotFoundHealthCheck` / `UnavailableHealthCheck` are server-wide
  `MediaItem` roll-ups with zero channel attribution.
- **Build failures / never-built playouts** are not surfaced per channel at all.

The `willNeverPlay` comment in `ChannelsScreen.tsx` deliberately keeps the badge/filter named
"No playout" (not "Problems") until a real taxonomy exists, because a "Problems 0" reading over a
lineup that holds a dead channel is a false all-clear — worse than no affordance.

## Chosen approach — built-timeline observation (Approach A)

Derive **every** per-channel fault from what the scheduler has already materialized, rather than
introspecting per-kind configuration:

- `Playout.BuildStatus` (`{ LastBuild, Success, Message }`) — never-built / build-failed signals.
- `Playout.Items` (`PlayoutItem`, the built timeline) — each item carries `MediaItemId`/`MediaItem`,
  so "empty upcoming" and "broken source" both fall out of one seam.
- `Playout.OnDemandCheckpoint` — distinguishes a legitimately-idle on-demand playout from a fault.

**Why this and not per-kind config introspection (Approach B):** the built timeline is kind-agnostic,
so all five `PlayoutScheduleKind` values are covered *by construction* — the #71 trap cannot bite. It
also yields **exact** broken-source attribution (`PlayoutItem → MediaItem.State`) with no fragile
media→collection→schedule→channel join and no query-time YAML/JSON parsing. Scripted, which has no
schedule entity to introspect, needs no special case.

**Cost/staleness note:** `MediaItem.State` flips on **scan**, not on build, which rules out a
build-time snapshot (it would go stale after a scan). Detection is therefore **read-time**.

### Trade-off accepted

The badge means *"this channel won't actually play something good going forward,"* not *"its config
is empty."* A config error in an unbuilt or far-future portion of a schedule surfaces only once built.
This is the honest reading for an operator and is why unbuilt/on-demand-idle playouts report
`unknown` rather than a false `healthy`.

## Fault taxonomy

Evaluated per playout, rolled up per channel. "Upcoming" = `PlayoutItem.Finish >= now`.

| Fault | Definition | Source |
|---|---|---|
| `noPlayout` | channel has 0 playouts (mirror-aware, via existing `GetPlayoutsCount`) | #72 fact, absorbed |
| `neverBuilt` | assessable playout with `BuildStatus.LastBuild == null` (see assessable gate below) | `BuildStatus`, `OnDemandCheckpoint` |
| `buildFailed` | last build `Success == false` (`Message` retained for detail) | `BuildStatus` |
| `emptyUpcoming` | built OK but **0 upcoming `PlayoutItem`s** | `Playout.Items` window |
| `brokenSource` | ≥1 upcoming `PlayoutItem` with `MediaItem.State ∈ {FileNotFound, Unavailable}` | `PlayoutItem → MediaItem.State` |

### The "assessable" gate (applied first, per playout)

Distinguish **absence** signals (a fault inferred from *missing* content) from **presence** signals
(a fault proven by content that *is* there and is bad):

- **Absence signals — `neverBuilt`, `emptyUpcoming`** — are suppressed for an **on-demand channel**
  (`PlayoutMode == OnDemand`), because an on-demand playout legitimately has no fresh build and drains
  its built timeline between tune-ins. Suppressing them is what prevents a false-positive storm across
  on-demand channels. A suppressed absence signal contributes `unknown`, never a false `problems`.
- **Presence signals — `buildFailed`, `brokenSource`** — stay live in **all** modes: a failed build or
  an upcoming item that points at a `FileNotFound`/`Unavailable` `MediaItem` is a real problem whether
  the channel is continuous or on-demand (these only fire when the bad thing actually exists, so they
  cannot false-positive on idleness).

Operationally: `isOnDemand = channel.PlayoutMode == ChannelPlayoutMode.OnDemand`. For a continuous
channel every fault applies. For an on-demand channel, `neverBuilt`/`emptyUpcoming` are dropped;
`buildFailed`/`brokenSource` still apply.

### Rollup → `status`

- `problems` — any fault fired on any assessable contributing playout.
- `healthy` — has ≥1 assessable playout, all built OK, non-empty upcoming, no broken upcoming items.
- `unknown` — nothing assessable to judge (e.g. only idle on-demand playouts). Never a false
  `healthy` and never a false `problems`.

Multi-playout / mirror channels: `status = problems` if *any* contributing playout has a problem;
otherwise `healthy` if *any* contributing playout is assessable-and-clean; otherwise `unknown`.
`faults` = the union across contributing playouts. Mirror channels evaluate the mirror source's
playouts, consistent with `GetPlayoutsCount`.

### Accepted judgment defaults

1. **Window = `Finish >= now`** (current + future built items) — no arbitrary N-hour horizon.
   Index-friendly via the existing `PlayoutItem (Start, Finish)` index; the scheduler builds ahead, so
   a healthy channel always has upcoming items.
2. **On-demand idle → `unknown`, not a fault.**
3. **Successfully-built playout whose entire timeline is now in the past → `emptyUpcoming`.** Honest
   but potentially noisy (a stale build pending its next rebuild reads as empty). Shipped as a fault;
   limitation documented.

## API design

Fold a server-derived `health` object onto the channel resource so **SPA and MCP read one
authoritative verdict** — machine clients get health in the same `list channels` / `get channel` call
they already make, with no second endpoint to correlate by id. Additive to the frozen `/api/v1`.

```jsonc
"health": {
  "status": "problems",                 // healthy | problems | unknown
  "faults": ["emptyUpcoming", "brokenSource"],
  "playoutCount": 2,                     // existing #72 fact, retained
  "brokenSourceItemCount": 7            // underlying raw count for machine triage
}
```

- Added to `ChannelResponseModel` (list) and `ChannelDetailResponseModel` (detail) via
  `Mapper.ProjectToResponseModel` / `ProjectToDetailResponseModel`.
- `PlayoutCount` remains for backward compatibility (additive-only freeze); the SPA and MCP migrate to
  `health`.
- OpenAPI regenerated (`./scripts/update-openapi.sh`), then `npm run generate:api` for `v1.d.ts`.

## Server implementation

- `ChannelRepository.GetAll`: add `.ThenInclude(p => p.BuildStatus)` on `Playouts` (and on
  `MirrorSourceChannel.Playouts`).
- **One bounded aggregate query** (not an N+1): `GROUP BY PlayoutId` over `PlayoutItem` where
  `Finish >= now`, projecting `(totalUpcoming, brokenUpcoming)` where `brokenUpcoming` counts
  `MediaItem.State ∈ {FileNotFound, Unavailable}`. Returned as a `Dictionary<int, (int total, int broken)>`
  keyed by playout id.
- `Mapper.GetHealth(Channel channel, IReadOnlyDictionary<int,(int,int)> upcoming)` derives the object;
  `GetAllChannelsHandler` and the detail handler pass the aggregate in.
- Cost measured on a realistic lineup per Done-when; add a `(PlayoutId, Finish)` index only if the
  aggregate shows up as hot (noted as a tuning follow-up, not a blocker).

## SPA implementation (`web/src/screens/ChannelsScreen.tsx`)

- Replace the `noplayout` filter with a single `problems` filter reading `health.status === 'problems'`;
  count = channels whose status is `problems`.
- Per-row badge text names the specific fault (No playout / Never built / Build failed / Empty /
  Broken source), preserving the existing "spell out the fault" convention.
- Rewrite the `willNeverPlay` predicate/comment: it becomes `hasProblems(channel)`; the comment's
  "do not broaden the label until the taxonomy exists" line is replaced with a pointer to this design +
  the decision record, since the taxonomy now exists.
- Coerce nullable Core-DTO fields at the hook boundary per `core-dtos-generate-nullable-in-spa`.

## Testing

- **Unit — derivation:** one test per fault class, **plus one per `PlayoutScheduleKind`** proving the
  seam is kind-agnostic (the #71 guard: an empty Block/Sequential/Scripted/ExternalJson playout flags
  `emptyUpcoming` exactly like Classic). On-demand-idle → `unknown`; multi-playout/mirror rollup.
- **Unit/integration — aggregate query:** the `GROUP BY` returns correct `(total, broken)`; no N+1
  (single query for the whole list).
- **SPA:** `ChannelsScreen.test.tsx` — `problems` filter + count, per-fault badge text, `unknown`
  renders no false badge.
- **Live-E2E** (`scripts/e2e-local.sh`, fresh config dir): build a channel → `healthy`; induce an
  empty upcoming window and a `FileNotFound` item → `problems` with the right faults + count. Curl the
  channel endpoints; do not drive a browser tab.

## Docs to update (same PR)

- `docs/decisions.md` — supersede `api.channel-health-signal`: record the derived `health` object +
  the built-timeline approach + the three accepted defaults.
- `docs/domain-model.md` — health derivation from the built timeline; update the `PlayoutCount` row.
- `docs/api-conventions.md` — the `health` object shape on the channel DTOs.
- `docs/spa-conventions.md` — the `problems` filter + per-fault badge convention.

## Out of scope

- Config-time / pre-build emptiness detection (Approach B) — explicitly not built.
- A dedicated `(PlayoutId, Finish)` index unless the cost measurement demands it.
- Any change to the server-wide health checks (`EmptyScheduleHealthCheck` etc.) — left as-is; this
  work is purely additive on the channel resource.

## Done-when (from #415)

- [ ] Per-channel "empty schedule" signal correct for all 5 `PlayoutScheduleKind` values (unbuilt/
      on-demand report `unknown`, never `healthy`).
- [ ] Broken-source attribution implemented (built-timeline seam; decision recorded here).
- [ ] Cost measured — no N+1 on a large lineup.
- [ ] SPA filter renamed to "Problems"; `willNeverPlay` comment/predicate updated.
- [ ] `docs/decisions.md` + `docs/domain-model.md` (+ api/spa conventions) updated.
- [ ] Adversarial review passed.
