--- license: other license_name: netease-model-use-license-agreement license_link: https://raw.githubusercontent.com/netease-youdao/Confucius4-R2T2/refs/heads/master/MODEL_LICENSE base_model: - netease-youdao/Confucius4-R2T2 tags: - gguf - llama.cpp - asr - speech-recognition - streaming - real-time - low-latency - multilingual - quantized pipeline_tag: automatic-speech-recognition --- # Confucius4-R2T2 · GGUF GGUF builds of [netease-youdao/Confucius4-R2T2](https://huggingface.co/netease-youdao/Confucius4-R2T2), a streaming ASR model trained on top of Qwen3-ASR-1.7B, so you can run it on [llama.cpp](https://github.com/ggml-org/llama.cpp) — CPU, CUDA and Metal all work. **中文版：[README.zh-CN.md](README.zh-CN.md)** > The Q4_K_M build matches the official PyTorch bf16 reference, takes a third of the > space, and runs straight on llama.cpp. --- ## Files | File | Size | What it is | |---|---|---| | `Confucius4-R2T2-Q4_K_M.gguf` | 1.03 GiB | LLM Decoder | | `mmproj-Confucius4-R2T2-Q4_K_M.gguf` | 211 MiB | Audio Encoder | | `Confucius4-R2T2-BF16.gguf` | 3.21 GiB | LLM Decoder, BF16 (reference) | | `mmproj-Confucius4-R2T2-F16.gguf` | 612 MiB | Audio Encoder, F16 | --- ## Accuracy ### True streaming results R2T2 is a streaming model, and streaming is what it is actually for. On zh + en 569: | Implementation | Weights | Mode | CER | |---|---|---|---| | Official PyTorch | bf16 | offline | 2.03% | | Official PyTorch | bf16 | streaming | 2.65% | | [EdgeSpeak](https://edgespeak.com/) engine | Q4_K_M GGUF | offline | 2.04% | | [EdgeSpeak](https://edgespeak.com/) engine | Q4_K_M GGUF | streaming | **2.19%** | > **llama.cpp cannot do streaming today.** Both `llama-mtmd-cli` and `llama-server` take > the whole clip in one shot; there is no streaming ASR path. The two streaming rows above > use the same Q4_K_M pair, running on the EdgeSpeak on-device engine. ### Offline results > Running these GGUF files on llama.cpp means offline, non-streaming decoding. All public datasets, no training-set overlap. Greedy decoding throughout, no sampling. Lower CER is better. | Dataset | Clips | Official PyTorch bf16 (reference) | BF16 GGUF | Q4_K_M | Q4_K_M · [EdgeSpeak](https://edgespeak.com/) engine | |---|---|---|---|---|------| | `seedtts_testset` (Chinese) | 269 | 1.42% | 1.29% | 1.63% | 1.58% | | `seedtts_testset` (English) | 186 | 1.74% | 1.84% | 1.64% | 1.54% | | `voxpopuli-cleaned-aa` (English) | 106 | 3.28% | 3.31% | 3.24% | 3.18% | | `librispeech-long` (English) | 8 | 2.77% | 3.08% | 2.77% | 2.46% | | **zh + en total** | **569** | **2.03%** | **2.01%** | **2.12%** | **2.04%** | --- ## Quick start ### llama-mtmd-cli ```bash llama-mtmd-cli \ -m Confucius4-R2T2-Q4_K_M.gguf \ --mmproj mmproj-Confucius4-R2T2-Q4_K_M.gguf \ --audio your.wav \ -p " " --temp 0 -n 256 -ngl 99 \ --jinja # ← REQUIRED. See below. ``` ### llama-server (OpenAI-compatible) ```bash llama-server \ -m Confucius4-R2T2-Q4_K_M.gguf \ --mmproj mmproj-Confucius4-R2T2-Q4_K_M.gguf \ -ngl 99 --temp 0 ``` ```bash curl http://localhost:8080/v1/audio/transcriptions \ -F file=@your.wav -F model=whisper-1 ``` --- ## Three things that will bite you ### 1. `llama-mtmd-cli` needs `--jinja`. Without it you silently get a different prompt. The prompt this model was trained on carries an **unconditional `system` block**, even when it is empty: ``` <|im_start|>system <|im_end|> <|im_start|>user <|audio_start|><|audio_pad|><|audio_end|><|im_end|> <|im_start|>assistant ``` The embedded Jinja template in these GGUF files produces exactly that. But `llama.cpp` has two rendering paths, and the legacy one **does not interpret Jinja at all** — it pattern-matches the template string (`<|im_start|>` → ChatML) and then runs a hardcoded C++ renderer that emits a `system` turn only if the caller supplied one. `llama-mtmd-cli` defaults to that legacy path; `--jinja` switches it to the real one. - `llama-mtmd-cli` — **defaults to legacy. Pass `--jinja`.** (Or set `LLAMA_ARG_JINJA=1`.) - `llama-server` — defaults to Jinja **on**. Don't pass `--no-jinja`. There is no metadata-only workaround: no builtin `llama.cpp` template family emits an unconditional `system` block, so editing the template string cannot fix the legacy path. The failure is silent — you just get slightly worse output. ### 2. `-p ""` is not "no prompt". An empty `-p` puts `llama-mtmd-cli` into interactive mode. Use `-p " "` for batch runs. (This leaves one stray space before `<|im_end|>`; there is no way to avoid it from the CLI, and its measured effect is negligible.) ### 3. Pad the tail with silence, or you lose the last few characters. R2T2 uses a **Longest Stable Prefix** paradigm: it emits a `|` to mark "this much is safe to commit, I need more audio before I continue". If your audio ends mid-sentence — which it does, because recordings stop the moment speech stops — the model never gets the evidence it needs to commit the final characters, and the transcript ends in a bare `|`. **Stripping the `|` does not recover the text.** The missing words were never generated. What fixes it is appending silence: ```bash ffmpeg -i in.wav -af apad=pad_dur=0.3 -ac 1 -ar 16000 -c:a pcm_s16le padded.wav ``` Measured on 22 clips that reproduce the truncation: | Tail silence | Still truncated | |---|---| | 0 s | 22 / 22 | | 0.1 s | 2 / 22 | | 0.15 – 0.25 s | 1 / 22 | | **0.3 s** | **0 / 22** | Before / after on one of them: ``` 0 s : I still say that this John Doe person is a| 0.3 s : I still say that this John Doe person is a myth. ``` The official streaming client pads 0.5 s before signalling end-of-stream. 0.3 s was the first fully clean point in our sweep; use 0.5 s if you want margin. --- ## License These are quantized derivatives. The original model's license applies unchanged: **NetEase Model Use License Agreement** — Base model: [Qwen/Qwen3-ASR-1.7B](https://huggingface.co/Qwen/Qwen3-ASR-1.7B). ## Credits Model by [NetEase Youdao](https://github.com/netease-youdao/Confucius4-R2T2). GGUF conversion, calibration and evaluation by us.