fix(vision-mixer): use gpu-aware videoconvert in CPU pipeline - #534
Merged
Conversation
The CPU compositor path hardcoded plain videoconvert for DSK, overlay and video-input chains. When an upstream source produces GPU memory (e.g. nvh264dec fed from efpsrt_input), plain videoconvert cannot negotiate against the CPU compositor backend and the pipeline fails to link. Switch all three chains to gpu::video_convert_mode().element_name(), matching the pattern already used by the videoenc and videoformat blocks — autovideoconvert on hosts with working CUDA-GL interop, plain videoconvert elsewhere. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…name The openapi.json snapshot drifted on main because two previous PRs updated code that affects the generated spec but did not regenerate the committed snapshot: - #523 bumped the workspace to 0.4.12 (info.version field) - #533 renamed the client-log tag from [WHIP-CLIENT] to [CLIENT] and rewrote the doc-comment for /api/client-log This commit just runs the regeneration the snapshot test asks for. No code change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
srperens
force-pushed
the
fix/vision-mixer-gpu-videoconvert
branch
from
April 30, 2026 14:10
654c7ec to
418a6dd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The CPU compositor path in the vision mixer hardcoded plain
videoconvertfor the DSK, overlay, and video-input chains. When an upstream source produces GPU memory (e.g.nvh264decfed fromefpsrt_input), plainvideoconvertcannot negotiate against the CPU compositor backend and the pipeline fails to link.This PR switches all three chains in
build_cpu_pipelinetogpu::video_convert_mode().element_name(), matching the pattern already used by the videoenc, videoformat, decklink and ndi blocks —autovideoconverton hosts with working CUDA-GL interop, plainvideoconverteverywhere else.The selection is computed once at startup in
backend/src/gpu.rs:42based on WSL detection, GL renderer probing, and CUDA capability checks.Why this is safe
video_convert_mode()returnsSoftware, which maps back to plainvideoconvert.videoformat.rs:107,decklink.rs:55,255,ndi.rs:209already use the exact same helper. Vision mixer was the only hold-out."videoconvert"by string, so the rename does not break the existingvision_mixer/tests.rsassertions.build_cpu_pipeline, line 581+). The GPU pipeline (build_gpu_pipeline, line 242+) is untouched.autovideoconvert(a bin) does not expose the same properties as plainvideoconvertdoes not matter.Risks worth watching
autovideoconvertcorrectly insertscudadownloadwhen downstream is the CPU compositor (it is designed to, but worth a quick eyeball during the first run).autovideoconvertis a bin and adds a small amount of probing at element creation; negligible for typical input counts.Test plan
cargo build --release --features efp,nvidiacleannvh264decupstream (e.g. viaefpsrt_input): same flow, verify the pipeline links and runs without errorsvision_mixer/tests.rsstill passes🤖 Generated with Claude Code