Skip to content

fix(vision-mixer): use gpu-aware videoconvert in CPU pipeline - #534

Merged
srperens merged 2 commits into
mainfrom
fix/vision-mixer-gpu-videoconvert
Apr 30, 2026
Merged

fix(vision-mixer): use gpu-aware videoconvert in CPU pipeline#534
srperens merged 2 commits into
mainfrom
fix/vision-mixer-gpu-videoconvert

Conversation

@srperens

Copy link
Copy Markdown
Collaborator

Summary

The CPU compositor path in the vision mixer hardcoded plain videoconvert for the DSK, overlay, and video-input chains. When an upstream source produces GPU memory (e.g. nvh264dec fed from efpsrt_input), plain videoconvert cannot negotiate against the CPU compositor backend and the pipeline fails to link.

This PR switches all three chains in build_cpu_pipeline to gpu::video_convert_mode().element_name(), matching the pattern already used by the videoenc, videoformat, decklink and ndi blocks — autovideoconvert on hosts with working CUDA-GL interop, plain videoconvert everywhere else.

The selection is computed once at startup in backend/src/gpu.rs:42 based on WSL detection, GL renderer probing, and CUDA capability checks.

Why this is safe

  • CPU-only hosts behave identicallyvideo_convert_mode() returns Software, which maps back to plain videoconvert.
  • Consistent with existing conventionvideoformat.rs:107, decklink.rs:55,255, ndi.rs:209 already use the exact same helper. Vision mixer was the only hold-out.
  • No tests reference "videoconvert" by string, so the rename does not break the existing vision_mixer/tests.rs assertions.
  • Only the CPU pipeline branch is touched (build_cpu_pipeline, line 581+). The GPU pipeline (build_gpu_pipeline, line 242+) is untouched.
  • No code sets properties on these specific converter elements after construction, so the fact that autovideoconvert (a bin) does not expose the same properties as plain videoconvert does not matter.

Risks worth watching

  • On a NVIDIA + working-interop host, sanity-check that autovideoconvert correctly inserts cudadownload when downstream is the CPU compositor (it is designed to, but worth a quick eyeball during the first run).
  • autovideoconvert is a bin and adds a small amount of probing at element creation; negligible for typical input counts.

Test plan

  • cargo build --release --features efp,nvidia clean
  • On a CPU-only host: build a vision-mixer flow with multiple inputs, start it, verify no CAPS negotiation errors in the log
  • On a NVIDIA host with nvh264dec upstream (e.g. via efpsrt_input): same flow, verify the pipeline links and runs without errors
  • Verify existing vision_mixer/tests.rs still passes

🤖 Generated with Claude Code

Per Enstedt and others added 2 commits April 30, 2026 16:05
The CPU compositor path hardcoded plain videoconvert for DSK, overlay
and video-input chains. When an upstream source produces GPU memory
(e.g. nvh264dec fed from efpsrt_input), plain videoconvert cannot
negotiate against the CPU compositor backend and the pipeline fails to
link.

Switch all three chains to gpu::video_convert_mode().element_name(),
matching the pattern already used by the videoenc and videoformat
blocks — autovideoconvert on hosts with working CUDA-GL interop, plain
videoconvert elsewhere.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…name

The openapi.json snapshot drifted on main because two previous PRs
updated code that affects the generated spec but did not regenerate
the committed snapshot:

- #523 bumped the workspace to 0.4.12 (info.version field)
- #533 renamed the client-log tag from [WHIP-CLIENT] to [CLIENT] and
  rewrote the doc-comment for /api/client-log

This commit just runs the regeneration the snapshot test asks for.
No code change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@srperens
srperens force-pushed the fix/vision-mixer-gpu-videoconvert branch from 654c7ec to 418a6dd Compare April 30, 2026 14:10
@srperens
srperens merged commit 52dcf7f into main Apr 30, 2026
7 checks passed
@srperens
srperens deleted the fix/vision-mixer-gpu-videoconvert branch April 30, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant