ersatztv/docs/decisions/records/ffmpeg/readrate-catchup-sparse-streams.md

key, title, status, since, supersedes, superseded-by, rule, signals, mechanics

key	title	status	since	supersedes	superseded-by	rule	signals	mechanics
ffmpeg.readrate-catchup-sparse-streams	2026-08-04 — a realtime input gets `-readrate_catchup`, because `-readrate` paces off its furthest-behind stream (#726)	active	2026-08-04	none	none	a realtime video/audio input also gets `-readrate_catchup` (6.0) when the binary supports it — but NOT a still-image input (mirroring the #350 exclusion) and NOT a concat input, which keep at most bare `-readrate` (a still image's video input takes none at all). Reason: `-readrate` paces the whole input off its furthest-behind stream, so a sparse stream sharing that input (an embedded PGS/DVD bitmap subtitle feeding the overlay) otherwise pins output at ~0.53x realtime. Catchup is a ceiling that applies only WHILE an input is behind, never a target, so it does not let a caught-up input race ahead.	readrate, readrate_catchup, sparse stream, bitmap subtitle, PGS, DVD subtitle, dvdsub, pgssub, overlay burn-in, Live TV buffering/stalling, "Resumed reading at pts N with rate R after a lag of Ns" · paths: `PipelineBuilderBase.SetRealtimeInput`, `ReadrateInputOption`, `FFmpegKnownOption` · issues: #726, #350, #529	`PipelineBuilderBase.CatchupReadRate` (6.0); `ReadrateInputOption` catchup arg; `FFmpegKnownOption.ReadrateCatchup` capability gate

• -readrate throttles an input, not a stream, and it paces off whichever stream is furthest behind. An embedded bitmap subtitle is read through the same -i as the video — SubtitleInputFile carries the video's path and ComplexFilter resolves it to a stream specifier on that input, and CommandGenerator never emits a second -i for it. Being sparse, the subtitle stream falls further behind every second and drags the video down with it. FFmpeg says so itself at -loglevel warning: [sist#0:3/dvd_subtitle] Resumed reading at pts 10.400 with rate 6.000 after a lag of 0.922s, repeating with the lag growing 0.9→3.8 s while pts stays pinned (no new packet).

• Measured on prod (QSV, -threads 1, dvd_subtitle→overlay), 45 s steady-state window after a 6 s settle:

  variant	throughput
  -readrate 1.05 (baseline)	0.533x (×3 runs)
  + -readrate_catchup 2.0	0.711x
  + -readrate_catchup 6.0	1.067x (×2 runs)
  + -readrate_catchup 20.0	1.067x
  no subtitle overlay (control)	1.067x

  A live client consumes at 1.0x, so 0.53x drains its buffer until it stalls — the reported symptom.

• 20.0 measuring the same as 6.0 is why 6.0 was chosen — above the catch-up point the value is not a throughput dial, so there is nothing to buy by going higher. It is not evidence about allocation: that is a steady-state throughput number, not a count of frames in flight.

• Why this does not reopen ffmpeg.qsv-extra-hw-frames-floor (#529). Not because catchup is brief (a permanently GPU-bound channel lags forever, so 6x is a standing licence), and not because read rate is allocation-irrelevant — #529 measured that it is not (at extra_hw_frames=0, 1.05 without a burst exits 0 while 1.05+burst hits ENOMEM). Read rate changes how fast frames enter the graph, not how deep its queues are, and #529 showed that only bites when the pool has no headroom. The 64-frame floor now guarantees headroom, so the load-bearing measurement is row 5 of that truth table — no -readrate at all with 64 frames → 14 segments, exit 0 — and a 6x ceiling is strictly less aggressive than no throttle. Reinforcing it, -readrate_initial_burst 8 has read flat out at the start of every playout item since #350, so an unbounded read here is not new. A 240 s QSV soak (64 frames, 60 segment boundaries) adds 1.043x sustained with zero Cannot allocate memory — but it stayed largely caught-up, so it corroborates rather than proves; the argument above is what carries the decision.

• Not QSV-specific: reproduces on libx264 too (0.533x → 1.067x), as expected for an input-pacing option upstream of any encoder or filter choice.

• Raising the base -readrate is not an alternative, and was measured: 2.0→0.62x, 3.0→0.80x, 4.0→0.80x, 6.0→0.89x. It asymptotes below realtime, because the rate ceiling was never the binding constraint. Recorded so it is not re-proposed.

• Catchup does NOT subsume the #350 burst; they fix orthogonal metrics. Measured time-to-first-segment: -readrate alone 3.71 s, +burst 0.72 s, +catchup alone 3.65 s, both 0.67 s. Catchup buys nothing at cold start (no accumulated lag at t=0 to recover) and the burst buys nothing for throughput (the 0.533x baseline already had it), so removing the burst on the theory that catchup replaces it would regress tune-in ~5x.

• Applied to realtime video/audio inputs generally, not only subtitle pipelines — it is inert unless an input is behind, and any sparse stream can trigger this, so gating it on "has a bitmap subtitle" would fix the site instead of the boundary. Two deliberate exclusions, both test-pinned: ConcatInputFile (reads already-written segments at a flat 1.0, nothing sparse to lag on) and still images, mirroring #350 — their video input takes no readrate at all, so catchup would reach only the separate audio input and break the pacing symmetry #350 declined to break. An image-based subtitle always rides the video path, so that shape cannot starve this way anyway.

• Capability-gated via FFmpegKnownOption.HasOption, the same fail-safe posture as -readrate_initial_burst: detection parses ffmpeg -h long, so a binary without the option silently keeps today's behavior instead of failing to start.

Accepted residual: the affected population is items carrying an embedded bitmap subtitle matching the channel's subtitle mode — 3,182 of 24,646 media versions (12.9%) on prod. It is a property of the item, not the channel, which is why the stall presented as random: a channel plays one episode fine and stalls on the next.