-readrate throttles an INPUT and paces it off whichever of its streams is
furthest behind. An embedded bitmap subtitle (PGS/DVD) is read through the same
-i as the video -- SubtitleInputFile carries the video's path and ComplexFilter
resolves it to a stream specifier on that input, so CommandGenerator never emits
a second -i for it. Being sparse, that subtitle stream falls further behind every
second and drags video throughput down with it: measured 0.53x realtime against
the 1.0x a live client consumes at, which drains the client buffer until it
stalls. FFmpeg names the culprit itself at -loglevel warning:
[sist#0:3/dvd_subtitle] Resumed reading at pts 10.400 with rate 6.000
after a lag of 0.922s (then 1.24, 1.56, ... 3.80)
Add -readrate_catchup (6.0) to realtime video/audio inputs, capability-gated
through FFmpegKnownOption.HasOption exactly as -readrate_initial_burst is, so a
binary without it silently keeps today's behavior instead of failing to start.
The option first shipped in ffmpeg 8.0, which is NEWER than 7.1 -- hence runtime
detection rather than a version assumption. Still images and concat inputs are
excluded, mirroring #350.
Measured on prod (QSV, -threads 1, dvd_subtitle -> overlay), 45s steady-state
window after a 6s settle, replaying the captured production command line:
baseline 1.05 0.533x (x3 runs)
+ catchup 2.0 0.711x
+ catchup 6.0 1.067x (x2 runs)
+ catchup 20.0 1.067x
no subtitle overlay 1.067x (control)
Baseline reproduces the reported 0.53x and the control the reported 1.07x, so
the harness is validated on both sides. Reproduces on software libx264 too
(0.533x -> 1.067x), as expected for an input-pacing option. Raising the base
-readrate is not an alternative and was measured: 2.0 -> 0.62x, 3.0 -> 0.80x,
4.0 -> 0.80x, 6.0 -> 0.89x -- it asymptotes below realtime because the rate
ceiling was never the binding constraint.
On #529 (readrate was incidentally bounding QSV hardware-frame allocation): the
20.0-vs-6.0 row is why 6.0 was chosen, NOT evidence about allocation -- it is a
steady-state throughput number, not a count of frames in flight. Nor is the bound
safe because read rate is allocation-irrelevant: #529 measured that it is not (at
extra_hw_frames=0, 1.05 without a burst exits 0 while 1.05+burst hits ENOMEM).
Read rate changes how fast frames enter the graph, not how deep its queues are,
and #529's failure only appeared with NO pool headroom. The 64-frame floor now
guarantees headroom, so the load-bearing measurement is row 5 of that truth
table -- no -readrate at all with 64 frames -> 14 segments, exit 0 -- and a 6x
ceiling is strictly less aggressive than no throttle. Reinforcing it,
-readrate_initial_burst 8 has read flat out at the start of every playout item
since #350, so an unbounded read here is not new. A 240s QSV soak at
QsvExtraHardwareFrames=64 across 60 segment boundaries corroborates: 1.043x
sustained, zero "Cannot allocate memory" / "Could not open encoder", RSS 166MB
vs 156MB at baseline -- corroborates rather than demonstrates, since it stayed
largely caught up.
Catchup does NOT subsume the #350 burst; measured time-to-first-segment:
-readrate alone 3.71s, +burst 0.72s, +catchup alone 3.65s, both 0.67s. They fix
orthogonal metrics.
The regression test is built on a BITMAP subtitle deliberately -- a text subtitle
is fetched by the libass filter outside the demuxer, so the same assertions would
pass vacuously while the bug is fully present. It asserts on the "[0:0][0:2]overlay"
label, which is the mechanism: subtitle stream 2 resolving onto the video's input.
Every new test was mutation-checked, each producing exactly its own expected red.
Fixes #726
Decisions-Edit: yes
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
6.2 KiB
key, title, status, since, supersedes, superseded-by, rule, signals, mechanics
| key | title | status | since | supersedes | superseded-by | rule | signals | mechanics |
|---|---|---|---|---|---|---|---|---|
| ffmpeg.hls-cold-start-burst | 2026-07-20 — HLS cold start is fixed with `-readrate_initial_burst`, not by raising the work-ahead limit (#350) | active | 2026-07-20 | none | none | HLS cold-start latency is fixed with a bounded `-readrate_initial_burst` (gated on FFmpeg ≥6.1 capability detection), not by raising `work_ahead_limit`, which would remove the concurrency guarantee it exists for. | HLS cold start, readrate, work_ahead_limit, HlsSessionWorker, FFmpegKnownOption capability gate · paths: `HlsSessionWorker`, `SetRealtimeInput`, `FFmpegPlaybackSettingsCalculator`, `FFmpegKnownOption`/`HasOption` · issues: #350 | `SetRealtimeInput` readrate-burst option; `FFmpegKnownOption.HasOption` version-capability gate |
Correction (2026-07-21, #529): the claim below that the burst is "bounded" is true only in seconds of input — it is not a bound on memory or hardware surfaces.
-readratewas also incidentally bounding how fast decoded frames enter the filter graph, and removing that on a QSV pipeline whose profile storesqsvExtraHardwareFrames: 0exhausts the upload pool: the graph fails with-12 (Cannot allocate memory),h264_qsvnever opens, and zero segments are written. The burst is not the root cause (a work-ahead start takes no-readrateat all and was already failing the same way in production), but it removed the throttle on every realtime session and so made the failure near-deterministic. Seeffmpeg.qsv-extra-hw-frames-floor; the decision recorded here still stands, with that floor in place.
Correction (2026-07-21, #536): the bullet below stating that "every concurrent tune-in falls back to the throttled path" described the intent, not the behaviour. The slot check and its increment straddled an
await, so N simultaneous tune-ins all read0 < limitand all started unthrottled; the "one winner, two throttled" observation held only because those tunes were effectively staggered. The measurement and the conclusions drawn from it stand — slot availability was the variable behind the bimodality — but the guarantee itself was not enforced untilffmpeg.work-ahead-slot-atomic.
-readratethrottles from the first read, so it sets a floor on time-to-first-segment. The realtime playback path pins input reading to 1.05× wall clock so a channel behaves like live TV. SinceOutputFormatHls.SegmentSecondsis 4 and the segmenter serves the playlist only once the first segment exists, the playlist cannot appear sooner than ~4/1.05 ≈ 3.8 s. Measured on real media: time-to-first-playlist 5369/5344 ms with-readrate 1.05, 648/649 ms with an initial burst.- The cold-start bimodality was never about the media.
HlsSessionWorkergrants an unthrottledSeekAndWorkAheadstart only while_workAheadCount < ffmpeg.segmenter.work_ahead_limit(prod: 1); every concurrent tune-in falls back to the throttled path. Three concurrent tunes on prod: the one that won the slot reachedfirstGopin 866 ms, the other two in 3845 ms and 6357 ms. This is why the earlier rounds found no correlation with subtitle burn-in, GOP length, or source file — the variable was slot availability, and the same channel could differ 7.2× between tunes. - Two earlier hypotheses are falsified, not deferred. Accurate-seek decode-discard (the issue's
ranked #1 driver) costs 30–100 ms on real prod media, and capping
-probesize/-analyzedurationbuys 20–50 ms. Neither can account for seconds. Recorded here so they are not re-proposed. - Burst rather than a bigger work-ahead budget. Raising
work_ahead_limitwould fix latency by deleting the guarantee that limit exists for — it caps how many unthrottled transcodes N viewers can start at once. The burst is bounded (SegmentSeconds * 2= 8 s of input, enough for the first segments at the defaultInitialSegmentCountof 1), after which live pacing resumes. An operator who raisesInitialSegmentCountabove 2 gets less of the benefit; that is a deliberate trade. - The burst is per ffmpeg process — i.e. per playout item — not per session.
SetRealtimeInputruns on every pipeline build andHlsSessionWorkerspawns a process per item, so each item boundary bursts too; this is not only the session's cold start. Two consequences, both accepted: item transitions get the same head start (a benefit), and on a channel whose items are shorter than the burst every item transcodes unthrottled, so the instantaneous-concurrency guarantee thatwork_ahead_limitprovides is weaker than before — weaker, not absent, becauseHlsSessionWorker'stranscodedBuffer <= 1mingate still stops the loop at a 60 s buffer, leaving average CPU unchanged. Making the burst strictly cold-start-only would mean plumbing a "first process of this session" flag throughFFmpegState; that complexity was not judged worth a bounded peak. - Still images are excluded. Their video input is paced by the realtime filter and takes no
readrate at all, so a burst would only run the audio input ahead of the video for songs and offline
filler, with no cold-start gain to show for it. (
-readrate_catchupmirrors this exclusion for the same reason —ffmpeg.readrate-catchup-sparse-streams, #726.) - Non-HLS realtime outputs (
TransportStream, HLS-Direct) burst too, sinceFFmpegPlaybackSettingsCalculatormakes them unconditionally realtime. That is untested by the benchmark, which was segmenter-only; it is kept because the same first-read throttle delays those clients identically, and the outerWrapSegmenter/Concatprocesses still pace atreadrate 1.0with no burst. - Gated on runtime capability, not on a parsed version.
-readrate_initial_burstneeds FFmpeg ≥ 6.1, andFFmpegKnownOption/HasOptionalready existed for exactly this (itsAllOptionslist had simply been empty). Detection parsesffmpeg -h long, so an older binary silently keeps today's behavior instead of failing to start — the same fail-safe posture as the other capability gates, and cheaper to reason about than the version-string parsing inNvidiaHardwareCapabilities.