Audio8-ASR-Infinite-mlx

Edge0/Audio8-ASR-Infinite in the layout mlx-audio-swift reads, at bfloat16. Made for OpenMeet.

Chinese and English only. That is the upstream model's own language list, not a limitation of this conversion.

The upstream model is published for Photon, which this is not. Same weights, different names.

What was changed

Nothing retrained or pruned. The weights are the upstream ones renamed, and the 8 semantic_vad_heads tensors are dropped — they steer Photon's own endpointing and transcription never reads them. 938 source tensors, 930 kept.

The audio tower needed no reimplementation: this checkpoint's audio_config says model_type: voxtral_realtime_encoder and means it, down to the bias pattern (query, value and output carry one; key does not) and the projector, which is the encoder's own downsampleAndProject at the same 10240-wide input (1280 × 8). Only the Qwen2 decoder, the frame-length embedding and the decode loop are new.

How it was checked

The inference loop does not ship with the model — upstream publishes it for Photon, and the HF module has forward and nothing that drives it. It was reconstructed and validated in Python first, and that reference is what every stage was checked against: mel, conv stem, encoder output, t_cond, and the decoded tokens.

Transcribes word for word against that reference, at about 1.1× real time on a 19-second clip:

The quarterly revenue forecast assumes a 17% increase in enterprise subscriptions, driven largely by the new authentication features we shipped in March. Sarah raised a concern about churn in the mid-market segment, and I think she is right that we should model a pessimistic scenario before committing to the hiring plan.

Requires a patched mlx-audio-swift

VoxtralRealtimeEncoderAttention upstream applies interleaved rotary embedding; transformers applies split-half (rotate_half) there, and this checkpoint was trained for split-half. With the interleaved rotation the model loads, runs at full speed, and returns fluent-looking nonsense. The fix is in selcukkubur/mlx-audio-swift.

Licence and credit

Apache-2.0, inherited. Credit to Edge0 for Audio8-ASR-Infinite, and to Mistral for the Voxtral Realtime audio architecture it builds on. This repository is a format conversion and claims nothing else.