Feature Request: Add SenseVoice/FunASR as ASR engine option

[LauraGPT]

Note

License and capability clarification (2026-07-14): FunASR is a toolkit, not a single checkpoint. The FunASR and SenseVoice repository source code is MIT; model weights follow each model card. SenseVoiceSmall supports Chinese, Cantonese, English, Japanese, and Korean, and its weights use the linked FunASR Model Open Source License Agreement. Fun-ASR-Nano-2512 is Apache-2.0. Language coverage, punctuation, and performance depend on the selected model and runtime configuration.

Feature Request

SoniTranslate handles video dubbing with speech recognition and translation. SenseVoice/Paraformer from FunASR would be a strong ASR engine option.

Why

• 170x real-time on GPU — non-autoregressive, much faster than Whisper for long video processing
• Built-in speaker diarization (cam++) — essential for multi-speaker dubbing
• Built-in punctuation — auto-adds punctuation for better subtitle quality
• 50+ languages with automatic language detection
• Chinese/Japanese/Korean accuracy — outperforms Whisper on CJK benchmarks

Quick Integration

from funasr import AutoModel

model = AutoModel(
    model="iic/SenseVoiceSmall",
    vad_model="fsmn-vad",
    spk_model="cam++",
)
result = model.generate(input="video_audio.wav")
# Returns text with timestamps and speaker labels

Happy to help with integration!

[artistlevid-alt]

Can you do MiniMax AI Voices please?

[artistlevid-alt]

And please fix SoniTranslate in HuggingFace

[Roy-Bertoli]

Hi, how do you install it in sonitraslate, or is it not possible?

[LauraGPT]

Hi @Roy-Bertoli, it is not installable inside SoniTranslate as a built-in ASR engine yet. This issue is a feature request / integration proposal, so the SoniTranslate UI does not currently expose a FunASR or SenseVoice option.

For now there are two practical paths:

1. Use FunASR/SenseVoice outside SoniTranslate, then import/use the transcript/subtitles in your workflow if SoniTranslate accepts an existing subtitle/transcript file.

python -m pip install -U funasr modelscope torch torchaudio

Minimal Python smoke test:

from funasr import AutoModel

model = AutoModel(
    model="iic/SenseVoiceSmall",
    vad_model="fsmn-vad",
    punc_model="ct-punc",
    disable_update=True,
)

result = model.generate(input="audio.wav", language="auto")
print(result)

2. Native SoniTranslate support needs a PR that adds a new ASR backend next to the existing engines. The wrapper would call AutoModel.generate(...), then map FunASR/SenseVoice output into the transcript segment format SoniTranslate expects. If speaker diarization is needed for dubbing, the backend can later enable spk_model="cam++" and map speaker labels too.

So short answer: not possible as a one-click install today, but it is feasible as an integration. I can help sketch or contribute the backend if the maintainer confirms where new ASR engines should plug into the current code path.

[Roy-Bertoli]

OK thank you