Files
ersatztv/docs/decisions/records/sched/weighted-shuffle.md
T
timothy fba5233caf
PR Gates / CI image pin matches docker/ci (pull_request) Successful in 11s
PR Gates / Docs update reminder (pull_request) Successful in 16s
PR Gates / decisions lifecycle (pull_request) Failing after 23s
Build ErsatzTV Image / Formatting (changed .cs conform to .editorconfig) (pull_request) Successful in 1m17s
Build ErsatzTV Image / API docs in sync (OpenAPI + endpoint index) (pull_request) Successful in 1m29s
Build ErsatzTV Image / Build & test (.NET) (pull_request) Successful in 8m5s
Build ErsatzTV Image / EF migration integrity (SQLite + MySql) (pull_request) Successful in 16m5s
Build ErsatzTV Image / Functional E2E (curl + UI contracts) (pull_request) Successful in 17m6s
Build ErsatzTV Image / Build & push image (amd64) (pull_request) Has been skipped
feat(610): split the decision corpus into one YAML-frontmatter file per record
168 records -> docs/decisions/records/<area>/<topic>.md (163 active, 23 dirs) and
docs/decisions/archive/<area>/<topic>.md (5 archived). The filename IS the key,
so one-active-record-per-key becomes a filesystem property rather than a
validator check, and supersession becomes a `git mv`.

WHY: the monolith was a concurrency problem before an aesthetic one. A
3,900-line append target made parallel sessions collide -- PR #605 and PR #614
both hit append-vs-append conflicts during routine rebases, and hand-resolving
those inside the corpus is exactly the operation the rationale-rewrite guard
exists to police.

HOW IT IS VERIFIED: a ~170-file diff cannot be meaningfully read, so correctness
does not rest on reading it. The parser was taught BOTH formats first, so the
body-diff guard parses the old form at the merge-base and the new form at head --
the migration validates itself, no bypass. The proof is a field-level equivalence
harness: 168 records before and after, zero lost, zero gained, zero field
mismatches, zero rationale bodies differing. Reviewers should scrutinise the
harness; it is the actual evidence.

What measuring caught that reading would not have:

- ~500 lines sit OUTSIDE any record -- decisions.md's lifecycle schema and each
  topic file's preamble, mostly the only copy. Source files are kept and
  stripped, never deleted. They also cannot be filed per-area: topic files hold
  several areas and 4 of 23 areas span several files.
- Archive discovery was a non-recursive glob; after the split it found ZERO
  archived records, surfacing as four bogus "supersedes points to unknown key"
  errors rather than an obvious failure.
- ~32 live docs point into the corpus BY DATE, which the split dangles. Each
  stripped file now ends with a generated "Records formerly in this file" index,
  which also rescues the identical breadcrumbs in old issue comments.
- decisions.md's "In this file:" list was 97 same-file anchor bullets that the
  split makes WRONG, not merely stale. Dropped; the generated index replaces
  them with links that resolve.

The equivalence harness now runs against a checked-in FIXTURE, not the live
corpus. The earlier version migrated the real tree, which made it a one-shot:
the moment the migration landed there was nothing left to move and the tests
failed for reasons unrelated to the code. A fixture keeps them testing the
SCRIPT rather than the repo's current state.

Keys preserved verbatim, warts included: `sched` (12) and `scheduling` (1) remain
two directories for one concept. Renaming a key is not a move -- it changes
identity, breaks the equivalence proof, and invalidates MemPalace's per-key
drawers. Taxonomy normalisation is separate work.

refs #610
2026-07-25 19:45:09 +02:00

11 KiB
Raw Blame History

key, title, status, since, supersedes, superseded-by, rule, signals, mechanics
key title status since supersedes superseded-by rule signals mechanics
sched.weighted-shuffle 2026-07-17 — Weighted / fair-share distribution is a new `WeightedShuffle` order; `ShuffleInOrder` is anti-clumping, not fair-share (#70) active 2026-07-17 none none Fair-share/weighted airtime distribution ships as one new `PlaybackOrder.WeightedShuffle = 9` order (equal weights = fair-share), not a retrofit of `ShuffleInOrder` (which only anti-clumps, since its padding spacers emit nothing) and not a separate orthogonal "distribution" setting; weights live on `MultiCollectionItem`/`MultiCollectionSmartItem` (DB default 1, dual-provider migration), bounded at write (1..1000) and clamped again in the enumerator, and the write path rejects `WeightedShuffle` at every dispatch site that doesn't handle it rather than let it silently degrade to unweighted random. WeightedShuffle, fair-share, ShuffleInOrder anti-clumping, MultiCollectionItem.Weight, write-path validation · paths: `WeightedShuffleCollectionEnumerator`, `MultiCollectionGrouper`, `ShuffleSourceBuilder.GetCollectionItemsForShuffleInOrder`, `AddMultiCollectionItemWeight` migration · issues: #70, #380, #403, #385, #386 `CollectionWithItems.Weight`; `ReplacePlaylistItems`/`CreateChannelFromLineup` write-path gates; `EffectiveWeight` clamp

PlaybackOrder.WeightedShuffle = 9 ships the two behaviors #70 asked for — "air Show A 70% / Show B 30%" and "no single show dominates" — as one order, because fair-share is the equal-weights case of weighted.

  • ShuffleInOrder does NOT already do fair-share, despite looking like it. Its balanced shuffle (keyj) pads every source to the longest with Option<MediaItem>.None spacers — but spacers emit nothing (ShuffleInOrderCollectionEnumerator.cs, result.AddRange(maybeItem) over an Option). One cycle therefore plays every item exactly once, so a 200-episode show still takes 10× the airtime of a 20-episode one. What it buys is anti-clumping: the small show is spread evenly instead of arriving in bursts, and each show plays chronologically. That is a different product from "airs as often". Recorded because the padding reads as equalization and this was misread once during #70's own design pass — the distinction is the entire justification for the issue.
  • Plain Shuffle is already implicitly weighted by collection size (FisherYates over flattened items), which is why "just use Shuffle" isn't fair-share either.
  • One new enum value, not two, and not a separate setting. Fair-share = WeightedShuffle with weights left at their default of 1, so the UI's "fair-share toggle" needs no second mechanism. A separate non-enum "distribution" setting was rejected: it would add an orthogonal axis every PlaybackOrder dispatch site must also consult, multiplying the blast radius below. Retrofitting weights onto ShuffleInOrder was rejected: it would silently change shipped users' output.
  • Weights live on MultiCollectionItem AND MultiCollectionSmartItem (int Weight, DB default 1, dual-provider migration AddMultiCollectionItemWeight). The mirror is mandatory — omitting the smart item un-weights smart-collection members silently. The DB default must be 1, not 0: a 0 backfill would hand every pre-existing member to the rotation carrying a weight that means nothing on a share-of-airtime scale. (The enumerator clamps such rows to the floor so they rotate fair-share rather than vanish — but the backfill should be right at the source.) Rejected carriers: PlaylistItem (sequential engine; its Count/PlayAll already mean "how much of this source", and drain-N-consecutively A A A B is a different product from smooth interleave A A B A); CollectionItem (wrong grain — per media item, and the largest table).
  • Applied on the ShuffleInOrder-shaped path only, because that is where source identity survives. MultiCollectionGrouper collapses sources into GroupedMediaItem + .Distinct(), so by the time the Shuffle path has its list, which source an item came from is gone. CollectionWithItems gains int Weight = 1; WeightedShuffleCollectionEnumerator consumes ShuffleSourceBuilder.GetCollectionItemsForShuffleInOrder unchanged — the schedule-entity-free entry point #380 reserved for this issue (no signature change, no god-factory, engines stay separate).
  • Stateless. Smooth weighted round-robin (acc += weight; richest wins; pays the total) is a pure function of (Seed, Index), so it restores by replay like its siblings and needs no per-source counters (CollectionEnumeratorState has nowhere to put them). A rotation is sized so the source needing the most picks works through all its items at its share; smaller sources loop inside it — that looping is what makes equal weights mean equal airtime. Weight ratios are data, not state: a Reset build re-randomizes Playout.Seed (fresh within-source shuffle and start point) but the ratios hold identically.
  • Ties break to the earliest source in list order (strict >), keeping the sequence deterministic.
  • ScheduleAsGroup is not read by this order, unlike ShuffleInOrder, which merges every non-ScheduleAsGroup collection into ONE source. Here each CollectionWithItems is its own weighted source, which is the entire point — per-source weights are meaningless if sources are merged. Recorded because it means a persisted per-item flag quietly has no effect under this order.
  • Within-source order is shuffled, not chronological (custom-ordered collections are still honored). Chronological inner order would make every rotation byte-identical, since source selection is deterministic — the reseed on wrap would change nothing and the avoid-an-immediate-repeat retry could never succeed. "Weighted in order" would be a separate order if wanted.
  • Cross-engine exposure is closed by write-path validation, not by touching the silent fallbacks. An unhandled PlaybackOrder fails silently at five of six dispatch sites: Classic substitutes RandomizedMediaCollectionEnumerator (PlayoutBuilder's default:, // TODO: handle this error case differently?); PlaylistEnumerator has no default: arm, so the enumerator stays null and the item is dropped from the playlist (and null is legitimate there for SeasonEpisode with Count == 0, so nothing flags it); BlockPlayoutBuilder filters block items against an allow-list and continues past the rest; YAML/Scripted return Option.None, which their callers read as "no content". Only MultiCollectionGroup throws. A weighted order degrading to unweighted random is the worst case, because the output is supposed to look arbitrary. So the write path rejects WeightedShuffle everywhere it isn't handled — if it can't be persisted where it isn't handled, the silent sites never see it — and YAML/Scripted (which address orders by name, so Enum.Parse accepts it regardless) log a warning when an order falls through unhandled. Making those pre-existing fallbacks loud is a real but separate defect class: #403, an explicit non-goal here.
    • Exactly two writers persist a CALLER-SUPPLIED PlaylistItem.PlaybackOrder, and the second is not obvious. ReplacePlaylistItems is the expected one; CreateChannelFromLineup is the other — a lineup of 2+ entries is stored as a Playlist, and its own guard only covered MultiCollection entries, so a multi-entry lineup of plain collections slipped WeightedShuffle straight through to PlaylistEnumerator's null-drop. Both are gated. The full writer set, since "persisting writer" alone is the wrong axis: Add*ToPlaylist and TraktCommandBase do persist the field but hardcode it (Shuffle/Chronological), so no caller value reaches them; ReplaceBlockItems writes the different field BlockItem.PlaybackOrder (also gated); and PreviewPlaylistPlayout, PreviewBlockPlayout and Engine/PlaylistHelper build in memory without persisting. This bullet has now been wrong three times, each time in the same shape: it recorded the perimeter as complete when the gate covered only the writers already in hand (missing CreateChannelFromLineup); the first correction miscounted by conflating PlaylistItem with BlockItem; the second said "two persisting writers" when the true predicate is two writers that persist a caller-supplied order. The failure is always enumerating from a list instead of re-deriving from a grep — which is also how the "0-weight is filtered out" claims survived their own correction. Before trusting any "this order can't reach that engine" claim, grep every writer of each field separately and classify each as persists-caller-value / persists-hardcoded / in-memory. The non-obvious composite handler is the one that gets missed.
    • Weight is bounded at the write path (MultiCollectionItemWeight, 1..1000) and clamped again in the enumerator (EffectiveWeight). They do different jobs, which is why both stay: EF's HasDefaultValue(1) substitutes 1 for a 0 on INSERT (0 reads as "not set") but an UPDATE writes the 0 through, so create and update disagreed on the same input — the gate makes them agree and refuses values that mean nothing on a share-of-airtime scale (0, negative, or absurdly large). The clamp is what makes the rotation arithmetic safe for rows that predate the gate: it bounds every weight before the sum, so Sum(weights) cannot overflow regardless of what is stored. Historical note: an earlier revision of this PR filtered Weight > 0 instead of clamping, which silently deleted a 0-weight source from the channel; the clamp replaced it, and any comment still describing that filter is stale.
  • Weight is returned on the multi-collection GET, not just accepted on write. The update replaces the item list, so a client that reads, edits a name, and writes back would silently reset every weight to the default if the read didn't carry it. Edits ride the existing Version token (If-Match/412, §7a).
  • Scope: Classic engine only; no SPA (the schedule-editor weight UI is gated on #388). Plain single/smart collections (the GroupIntoFakeCollections path) carry no persisted weights — fair-share applies there by construction, and per-source weighting requires a multi collection. Auto-tune's per-member weights (#385/#386) need their own carrier: an auto-tuned channel is backed by one live SmartCollection, so its "sources" are members of a single collection, and a PlaylistItem cannot even reference a search query. Nothing here forecloses it — the enumerator reads weights off CollectionWithItems source-agnostically.