Webkit support?

[boraturant]

As far as I am aware, Webkit currently only supports video on their Webcodecs support, not audio. Are you planning any work around on this? Also https://omniclip.app doesn't work on safari.

[zenkyuv]

I was aware that there was AudioDecoder of Webcodecs API, but I was okay with the speed of re-encoding audio using ffmpeg so I didnt bother using AudioDecoder for added complexity -- but if AudioDecoder is fully supported in most browsers then I will think about replacing ffmpeg with it, best if you could give me some idea what you need AudioDecoder for, do you need some tool to easily re-encode audio? Im aware that it doesnt work on safari, but I dont think I want to do some fallbacks to eg. ffmpeg

[boraturant]

I am working on an open social media platform. I need a video encoding solution similar to Instagram/Tiktok that works on web(mobile/desktop). Encode video(with text, image, video overlays) with webcodecs on client and send the aggregated total EncodedVideo/AudioChunks to server FFMPEG to mux on server. I can't send/trust ffmpeg on client for muxing on mobile devices.

New AI capabilities on web client(wasm, webgpu) are also super interesting to explore/add.

The above solution may be designed to work on server(puppeteer) for server-side rendering- for advanced ai/gpu capabilities.

Snapchat uses a native solution for their filters on native mobile apps and export the same functionality to web with wasm. This is a nice approach, but webcodecs and new emerging client ai webgpu capabilities will work better/efficient/expandable on mobile/web.

What is your view on these?

[zenkyuv]

If you are working on open social media which relies on videos then i would suggest using webcodecs API as this is the way to go, sure its not the easiest to work with, i will do something about it in sometime like a tool to easily decode and encode is my priority to do as it seems like what mostly people need. Indeed there are some emerging opportunities with the new capabilities on the web and seems like video processing was one of those bottlenecks here so its definitely something to take a closer look at

[cy920820]

I found during testing on Safari that the image after frame extraction was not displayed properly, and the console reported an error @zenkyuv

[zenkyuv]

I found during testing on Safari that the image after frame extraction was not displayed properly, and the console reported an error @zenkyuv

Hmm i think i saw that myself but dont exacly know how to reproduce, I will try to fix it, thanks for pointing out