ersatztv/docs/decisions/records/ffmpeg/qsv-hdr-tonemap-opencl.md

key, title, status, since, supersedes, superseded-by, rule, signals, mechanics

key	title	status	since	supersedes	superseded-by	rule	signals	mechanics
ffmpeg.qsv-hdr-tonemap-opencl	2026-07-26 — the QSV pipeline tonemaps HDR through OpenCL, never vpp_qsv (#505)	active	2026-07-26	none	none	the QSV pipeline never emits `vpp_qsv=tonemap=1`, which is a SILENT no-op on pre-Gen11 Intel graphics; HDR is tonemapped on the GPU via `hwupload=derive_device=vaapi` → `scale_vaapi` → `hwmap=derive_device=opencl` → `tonemap_opencl` when a VA-API device exists, the frames are still in software, and `tonemap_opencl` is available, and by the software `TonemapFilter` otherwise. The scale runs BEFORE the tonemap, and any hardware filter on the path forces the output to be re-tagged bt709.	QSV, HDR, tonemap, vpp_qsv, tonemap_opencl, hwmap, derive_device, smpte2084, bt2020, washed out HDR, silent no-op, Gen9.5/Gen11, iHD · paths: `QsvPipelineBuilder`, `TonemapOpenClQsvFilter`, `HardwareUploadVaapiFilter`, `ScaleVaapiFilter`, `TonemapFilter` · issues: #505, #498, #523, #529	`QsvPipelineBuilder.UseOpenClTonemap`; `QsvPipelineBuilder.SetScaleVaapiForTonemap`; `TonemapOpenClQsvFilter`; `HardwareUploadVaapiFilter(setFormat, deriveDevice)`; the `usesVppQsv` predicate in `SetPixelFormat`

• vpp_qsv=tonemap=1 does not tonemap — it returns the frame untouched, with no warning. Measured on the deployed FFmpeg 8.1.2 / iHD 25.1.4 / UHD 630 (i7-10700K, Gen9.5) against real HDR HEVC Main10 (bt2020nc/bt2020/smpte2084): a graph ending in vpp_qsv=tonemap=1 produced a frame byte-identical (same md5) to the same graph with no tonemap step at all. True with the explicit hevc_qsv decoder, on genuine QSV video-memory surfaces, with the filter before the scale, after the scale, and with format=nv12. The output still carried bt2020/smpte2084 tags. QSV VPP tonemapping requires Gen11+; pre-Gen11 iHD ignores the option silently. Reference luma for the same frame: software tonemap YAVG=26.6, no tonemap YAVG=44.3, vpp_qsv 44.3.

• So #505's premise was inverted, and the branch it wanted to extend was already broken. The issue asked to route the #498 native-decode path through TonemapQsvFilter to save CPU; doing so would have shipped untonemapped HDR. Worse, the pre-existing DecoderHardwareAccelerationMode == Qsv branch already did exactly that — so any operator who flipped QsvPreferNativeDecoder off, which is precisely the escape hatch #498 and #523 recommend, got washed-out HDR. Prod was not affected (native-decode is the default, and it took the working software branch). TonemapQsvFilter is deleted rather than left in place: a filter that silently does nothing is worse than no filter, because it looks like coverage.

• OpenCL is the working GPU route, and it is genuinely faster — but only if the scale runs first. 300 frames, 3840x1608 HDR HEVC → 1280x720 h264_qsv, two reproducible rounds:

  arm	user CPU	total CPU	wall (12.5s of content)	tonemapped
  software zscale/tonemap (what #505 wanted to replace)	33.2s	35.6s	10.25s	yes
  vpp_qsv=tonemap=1	—	—	—	no (no-op)
  OpenCL, tonemap at full size then scale	13.4s	21.3s	15.5s	yes
  OpenCL, scale_vaapi first then tonemap	9.9s	14.1s	9.25s	yes
  no tonemap at all (floor)	13.1s	15.6s	9.2s	no

  Scale-first cuts total CPU ~60% versus the software tonemap and lands at the no-tonemap wall-clock floor. Tonemapping at full size instead is slower than the software path it replaces (15.5s for 12.5s of content — below realtime), which is why the ordering is a correctness-adjacent requirement and not a micro-optimization. This is what forces the scale decision and the tonemap decision to be made together, up front, in UseOpenClTonemap.

• A QSV surface is a dead end: it maps to neither OpenCL nor VA-API. hwmap=derive_device=opencl from QSV fails ("Media sharing must be enabled on context creation"), and hwmap=derive_device=vaapi from QSV fails with -38 (function not implemented). So the OpenCL route is only reachable while frames are still in software, which is why the gate excludes the QSV decoder (-hwaccel_output_format qsv) and ShouldDeinterlace (deinterlace_qsv uploads first). Both fall back to the software tonemap — slower, but correct, which is the whole point. The upload must also say derive_device=vaapi explicitly: the QSV pipeline sets -filter_hw_device hw (the QSV device), so a bare hwupload would land on a QSV surface and strand the frames.

• Tonemapping the pixels is only half the job; the stream has to stop claiming it is HDR. The first end-to-end run on the Intel host was correctly tonemapped (YAVG=26.39, transfer bt709) and still tagged color_primaries=bt2020/color_space=bt2020nc, inviting the player to convert it a second time. Cause: SetPixelFormat's usesVppQsv predicate — which is really "did a hardware filter strip the color info" — listed only the QSV filters, and this path replaces them with ScaleVaapiFilter/TonemapOpenClQsvFilter. Both are now in the predicate. The tell was reachable only from ffprobe on the real output; exit code 0 and a correct luma average both looked clean.

• Re-tagging follows the TONEMAP, not the normalization preference. The colorspace filter was originally reached only when desiredState.ColorsAreBt709 (the profile's NormalizeColors) was on, so a profile with normalization off got tonemapped pixels still tagged bt2020 — the same double-conversion bug as above, just for a different operator setting. The guard is now tonemapped || (ColorsAreBt709 && …). Deliberately not fixed by hoisting usesVppQsv out of the guard: a scale-only hardware chain on non-HDR content should still respect the preference. Converting the pixels to SDR is what obliges the stream to stop announcing HDR; nothing else does.

• A new scale filter has to be declared to every consumer that asks "was the video scaled". Swapping ScaleQsvFilter for ScaleVaapiFilter silently broke image-subtitle burn-in: the subtitle canvas is resized only when the video chain contains a recognized scale filter, and that predicate listed the QSV ones only. A 4K HDR source with PGS subtitles scaled the video to 720p and left the subtitle at source size. ScaleVaapiFilter is now in the predicate (as it already was in VaapiPipelineBuilder). Generalizable: replacing a filter means grepping for every is <OldFilter> type test, not just its construction site.

• Anamorphic sources stay on the software tonemap. ScaleQsvFilter is handed the SAR that VideoStream calculates (with a fallback for a missing or 0:0 SAR); ScaleVaapiFilter instead multiplies by ffmpeg's runtime sar, which is a different value when the decoded frame leaves SAR unspecified. Rather than ship an anamorphic HDR graph nobody has run, IsAnamorphic is excluded from the gate — which leaves those sources exactly where they were before this change, so it costs them nothing. Revisit only with a real anamorphic HDR sample on the Intel host, asserting dimensions/SAR/DAR rather than just exit status.

Accepted residual: HardwareUploadVaapiFilter's deriveDevice is an optional bool defaulting to false, which is behavior-preserving for all four existing VA-API call sites but is a trap for a future QSV one: copying the familiar new HardwareUploadVaapiFilter(true) yields a bare hwupload, which lands on the QSV device (-filter_hw_device hw) and makes the OpenCL mapping unreachable. A named factory or an explicit device-target enum would be safer; it was not done here because it would push this diff into the VA-API pipeline for no behavior change. The XML comment on the parameter is the mitigation.

Accepted residual: on Gen11+ hardware, where vpp_qsv=tonemap=1 presumably does work, we now use OpenCL instead. That is deliberate — we have no capability probe that can tell the two apart (FFmpeg reports no error either way, which is the entire problem), and OpenCL is validated here and is the route Jellyfin uses. The gate is therefore a reachability test (VA-API device present, frames in software, tonemap_opencl compiled in), never a hardware-generation guess. If a Gen11+ box is ever available to measure, compare the two there before adding a generation check — do not add one on inference.