What this fork changes

• WebSocket server (voxmlx-serve) — OpenAI Realtime API-compatible endpoint for streaming transcription
• Memory management — caps MLX's Metal buffer cache at 4 GB (unbounded by default, was hitting 20+ GB), materializes lazy arrays after concat, and clears GPU memory on disconnect
• DFT matrix caching — precomputes STFT components instead of rebuilding per log_mel_spectrogram_step() call
• Encoder KV window alignment — incremental encoder cache now uses the model's configured encoder.sliding_window (750 for Voxtral Mini Realtime), which fixes long-stream semantic drift (see upstream issue #7 and PR #4)

Managed launchers can run voxmlx-serve --parent-pid PID so the server exits when the launcher process dies. After the model loads it prints VOXMLX_READY port=PORT; use GET /health as the readiness probe because the socket binds inside uvicorn.run.

---

voxmlx

Realtime speech-to-text with Voxtral Mini Realtime in MLX.

Install

pip install voxmlx

Usage

voxmlx

Transcribe audio from a file or stream from the microphone in real-time.

Stream from microphone:

voxmlx

Transcribe a file:

voxmlx --audio audio.flac

Options:

Flag	Description	Default
--audio	Path to audio file (omit to stream from mic)	None
--model	Model path or HuggingFace model ID	mlx-community/Voxtral-Mini-4B-Realtime-6bit
--temp	Sampling temperature (0 = greedy)	0.0

voxmlx-convert

Convert Voxtral weights to voxmlx/MLX format with optional quantization.

Basic conversion:

voxmlx-convert --mlx-path voxtral-mlx

4-bit quantized conversion:

voxmlx-convert -q --mlx-path voxtral-mlx-4bit

Convert and upload to HuggingFace:

voxmlx-convert -q --mlx-path voxtral-mlx-4bit --upload-repo username/voxtral-mlx-4bit

Options:

Flag	Description	Default
--hf-path	HuggingFace model ID or local path	mistralai/Voxtral-Mini-4B-Realtime-2602
--mlx-path	Output directory	mlx_model
-q, --quantize	Quantize the model	Off
--group-size	Quantization group size	64
--bits	Bits per weight	4
--dtype	Cast weights (float16, bfloat16, float32)	None
--upload-repo	HuggingFace repo to upload converted model	None

Python API

from voxmlx import transcribe

text = transcribe("audio.flac")
print(text)