Post
150
DuplexJev: a System One model for speech š§
š Oct 22: now hears Japanese. 32B-Para v1.4: Japanese error 32.7 ā 11.6% (FLEURS), Korean 31.8 ā 7.6%, Vietnamese 41.7 ā 14.6%. Notes: https://github.com/adventists-ai/duplexjev/blob/main/research/2026-10-japanese.md
Jev made "System One" decision models the hot idea of the season: state in, typed probabilistic decisions out, many questions in one pass. Voice agents need it most, yet a text decision model only sees a transcript, and the transcript is where tone disappears.
DuplexJev takes raw audio as its state. A frozen ASR encoder feeds a frozen LLM through a small connector; every question is a choice question read as one token, nothing decoded.
ā” Fast: 10 decisions in 92 ms on one H200 (transcribe + LLM: 1,978 ms), ~170 decisions/s per GPU, under $0.50 per million decisions (4B). A new decision is one line of text.
š Hears what the transcript drops: #1 in paralinguistic understanding. LISTEN (EACL 2026) + StressTest + MSPB, 1,035 questions, none from our training corpora, same for every model:
DuplexJev-32B-Para v1.3: 50.2
Gemini 3.8 Flash: 47.8
Qwen3-Omni-30B: 45.0
Gemini 3.1 Pro: 41.0
GPT-audio-1.5: 40.9
Gemini 3.8 Live: 40.0
GPT-realtime-2.1: 35.6
Prosody (MSPB): 77.3 vs 71.9 next best. (Split suggested by @dipankarsarkar , thank you!)
How it learned to listen: speech LLMs trained by transcript distillation imitate a teacher that never heard a voice (gender ~55%, emotion ~28%). Training the answer token on the correct answer instead takes both to ~90%.
Paper: Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcript (2610.02638)
Models: adventists-ai/duplexjev-6acc69e95e4fc69acd09adaf
Demo: BitKnight/DuplexJev
Code + every model's raw answers: https://github.com/adventists-ai/duplexjev
(Independent project by Adventists AI, Zhejiang University and University of Virginia; not affiliated with TypeSafe.)
š Oct 22: now hears Japanese. 32B-Para v1.4: Japanese error 32.7 ā 11.6% (FLEURS), Korean 31.8 ā 7.6%, Vietnamese 41.7 ā 14.6%. Notes: https://github.com/adventists-ai/duplexjev/blob/main/research/2026-10-japanese.md
Jev made "System One" decision models the hot idea of the season: state in, typed probabilistic decisions out, many questions in one pass. Voice agents need it most, yet a text decision model only sees a transcript, and the transcript is where tone disappears.
DuplexJev takes raw audio as its state. A frozen ASR encoder feeds a frozen LLM through a small connector; every question is a choice question read as one token, nothing decoded.
ā” Fast: 10 decisions in 92 ms on one H200 (transcribe + LLM: 1,978 ms), ~170 decisions/s per GPU, under $0.50 per million decisions (4B). A new decision is one line of text.
š Hears what the transcript drops: #1 in paralinguistic understanding. LISTEN (EACL 2026) + StressTest + MSPB, 1,035 questions, none from our training corpora, same for every model:
DuplexJev-32B-Para v1.3: 50.2
Gemini 3.8 Flash: 47.8
Qwen3-Omni-30B: 45.0
Gemini 3.1 Pro: 41.0
GPT-audio-1.5: 40.9
Gemini 3.8 Live: 40.0
GPT-realtime-2.1: 35.6
Prosody (MSPB): 77.3 vs 71.9 next best. (Split suggested by @dipankarsarkar , thank you!)
How it learned to listen: speech LLMs trained by transcript distillation imitate a teacher that never heard a voice (gender ~55%, emotion ~28%). Training the answer token on the correct answer instead takes both to ~90%.
Paper: Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcript (2610.02638)
Models: adventists-ai/duplexjev-6acc69e95e4fc69acd09adaf
Demo: BitKnight/DuplexJev
Code + every model's raw answers: https://github.com/adventists-ai/duplexjev
(Independent project by Adventists AI, Zhejiang University and University of Virginia; not affiliated with TypeSafe.)
