VAmoS Pro Bench
Leaderboard
| Voice agent | Task completion | Latency | Turns | Barge-in/call | Cost | Type | Model | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | ------- | ----- | ------------- | -------- | ---------------- | ------------------------------------------------------------- | | grok voice") | 44.7% | 2.64s | 16 | 1.07 | \~$0.210 | speech‑to‑speech | Grok Voice Think Fast 2.0 | | gemini 3.8 live") | 40.1% | 1.66s | 12 | 0.13 | \~$0.193 | speech‑to‑speech | Gemini 3.8 Live | | gpt-live 1") | 39.2% | 1.62s | 17 | 1.05 | \~$0.192 | speech‑to‑speech | GPT-Live 1 | | openai realtime 2.1") | 37.9% | 2.12s | 14 | 0.44 | \~$0.633 | speech‑to‑speech | GPT Realtime 2.1 | | elevenlabs") | 36.4% | 2.18s | 12 | 0.04 | \~$0.272 | cascade | ElevenLabs ASR → gpt-4.1-mini → eleven\_flash\_v2 | | openai realtime") | 33.6% | 2.32s | 14 | 0.35 | \~$0.607 | speech‑to‑speech | GPT Realtime 2 | | livekit") | 29.4% | 2.96s | 12 | 0.04 | \~$0.077 | cascade | Deepgram nova-3 → gpt-4.1-mini → ElevenLabs eleven\_flash\_v2 | | gemini 3.1 live") | 25.6% | 2.66s | 12 | 0.10 | \~$0.199 | speech‑to‑speech | Gemini 3.1 Flash Live | | mistral") | 22.7% | 3.75s | 12 | 0.08 | \~$0.279 | cascade | Voxtral realtime → mistral-medium-2604 → Voxtral TTS | | gradium") | 21.3% | 2.42s | 14 | 1.18 | \~$0.120 | cascade | Gradium STT → gpt-4.1-mini → Gradium TTS | | vapi") | 20.5% | 2.08s | 20 | 0.71 | \~$0.271 | cascade | Deepgram → gpt-4.1-mini → ElevenLabs | | deepgram") | 19.8% | 2.04s | 13 | 0.34 | \~$0.193 | cascade | nova-3 → gpt-4.1-mini → aura-2 | | hugging face") | 19.0% | 5.28s | 12 | 9.86 | \~$0.053 | cascade | Whisper large-v3 → gpt-oss-120b → Kokoro-82M | | pipecat") | 17.3% | 1.96s | 18 | 7.94 | \~$0.109 | cascade | Deepgram nova-3 → gpt-4.1-mini → ElevenLabs eleven\_flash\_v2 |
Cost vs. completed tasks
10% 20% 30% 40% 50% $0.05 $0.10 $0.20 $0.50 Mean cost per call (USD, log scale) · Y: task completion (% of calls) Pareto frontier grok voice: 44.7% completion, \~$0.210 per callgrok voice: 44.7% completion, \~$0.210 per call gemini 3.8 live: 40.1% completion, \~$0.193 per call3.8 Livegemini 3.8 live: 40.1% completion, \~$0.193 per call gpt-live 1: 39.2% completion, \~$0.192 per callGPT-Livegpt-live 1: 39.2% completion, \~$0.192 per call openai realtime 2.1: 37.9% completion, \~$0.633 per callRealtime 2.1openai realtime 2.1: 37.9% completion, \~$0.633 per call elevenlabs: 36.4% completion, \~$0.272 per callelevenlabs: 36.4% completion, \~$0.272 per call openai realtime: 33.6% completion, \~$0.607 per callRealtime 2openai realtime: 33.6% completion, \~$0.607 per call livekit: 29.4% completion, \~$0.077 per calllivekit: 29.4% completion, \~$0.077 per call gemini 3.1 live: 25.6% completion, \~$0.199 per call3.1 Livegemini 3.1 live: 25.6% completion, \~$0.199 per call mistral: 22.7% completion, \~$0.279 per callmistral: 22.7% completion, \~$0.279 per call gradium: 21.3% completion, \~$0.120 per callgradium: 21.3% completion, \~$0.120 per call vapi: 20.5% completion, \~$0.271 per callvapi: 20.5% completion, \~$0.271 per call deepgram: 19.8% completion, \~$0.193 per calldeepgram: 19.8% completion, \~$0.193 per call hugging face: 19.0% completion, \~$0.053 per callhugging face: 19.0% completion, \~$0.053 per call pipecat: 17.3% completion, \~$0.109 per callpipecat: 17.3% completion, \~$0.109 per call grok voice: 44.7% completion, \~$0.210 per call grok voice: 44.7% completion, \~$0.210 per callgrok voice gemini 3.8 live: 40.1% completion, \~$0.193 per call gemini 3.8 live: 40.1% completion, \~$0.193 per callgemini 3.8 live gpt-live 1: 39.2% completion, \~$0.192 per call gpt-live 1: 39.2% completion, \~$0.192 per callgpt-live 1 openai realtime 2.1: 37.9% completion, \~$0.633 per call openai realtime 2.1: 37.9% completion, \~$0.633 per callopenai realtime 2.1 elevenlabs: 36.4% completion, \~$0.272 per call elevenlabs: 36.4% completion, \~$0.272 per callelevenlabs openai realtime: 33.6% completion, \~$0.607 per call openai realtime: 33.6% completion, \~$0.607 per callopenai realtime livekit: 29.4% completion, \~$0.077 per call livekit: 29.4% completion, \~$0.077 per calllivekit gemini 3.1 live: 25.6% completion, \~$0.199 per call gemini 3.1 live: 25.6% completion, \~$0.199 per callgemini 3.1 live mistral: 22.7% completion, \~$0.279 per call mistral: 22.7% completion, \~$0.279 per callmistral gradium: 21.3% completion, \~$0.120 per call gradium: 21.3% completion, \~$0.120 per callgradium vapi: 20.5% completion, \~$0.271 per call vapi: 20.5% completion, \~$0.271 per callvapi deepgram: 19.8% completion, \~$0.193 per call deepgram: 19.8% completion, \~$0.193 per calldeepgram hugging face: 19.0% completion, \~$0.053 per call hugging face: 19.0% completion, \~$0.053 per callhugging face pipecat: 17.3% completion, \~$0.109 per call pipecat: 17.3% completion, \~$0.109 per callpipecat
Build this around your own systems
This board is one exam, on one utility’s policies. Veris builds the same thing around yours: your workflows, your systems, your rules, and the calls your agents actually get. Simulated callers, twinned services, and a verdict from the world the agent leaves behind rather than the transcript it produced.
Running a voice stack of your own? We’ll put it through the same 100 tasks and publish the result. Contact us at hello@veris.ai