Leaderboard — τ-bench
NEWτ³-bench is here: τ-knowledge evaluates agents on knowledge-intensive tasks, and τ-voice benchmarks real-time voice agents.
🏦 τ³-Banking🎙️ τ³-Voice📝 τ²-bench
τ³-Banking Leaderboard
Text agents resolving banking customer-service tasks over a \~700-document knowledge base. Published as τ-knowledge.
StandardCustom
ⓘ
| Rank | Model | Retrieval | Reasoning | User Sim | | | | | ---- | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | -------- | ------- | ----- | - | | #1 | Qwen 3.8 Max | Jul 19, 2026 | 🔍 alltools | xhigh | gpt-5.2 | 55.2% | ▶ | | #2 | Claude Opus 5 | Jul 24, 2026 | 🔍 alltools | max | gpt-5.2 | 48.7% | ▶ | | #3 | Grok 4.5 | Jul 16, 2026 | 🔍 alltools | high | gpt-5.2 | 47.9% | ▶ | | #4 | GPT-5.6-sol | Jul 9, 2026 | 🔍 alltools | xhigh | gpt-5.2 | 46.9% | ▶ | | #5 | GPT-5.5 | Apr 23, 2026 | 🔍 alltools | xhigh | gpt-5.2 | 44.6% | ▶ | | #6 | Muse Spark 1.1 | Jul 9, 2026 | 🔍 alltools | xhigh | gpt-5.2 | 40.5% | ▶ | | #7 | Claude Opus 4.7 | Apr 16, 2026 | 🔍 alltools | max | gpt-5.2 | 40.2% | ▶ | | #8 | Claude Opus 4.8 | May 28, 2026 | 🔍 alltools | max | gpt-5.2 | 39.7% | ▶ | | #9 | Claude Fable 5 | Jun 9, 2026 | 🔍 alltools | max | gpt-5.2 | 39.7% | ▶ | | #10 | GPT-5.4 | Mar 5, 2026 | 🔍 alltools | xhigh | gpt-5.2 | 39.4% | ▶ | | #11 | Kimi K3 | Jul 16, 2026 | 🔍 alltools | max | gpt-5.2 | 37.1% | ▶ | | #12 | GLM-5.2 | Jun 16, 2026 | 🔍 alltools | xhigh | gpt-5.2 | 37.1% | ▶ | | #13 | GPT-5.2 | Dec 11, 2025 | 🔍 alltools | high | gpt-5.2 | 32.2% | ▶ | | #14 | Gemini 3 Flash | Dec 17, 2025 | 🔍 Terminal | high | gpt-5.2 | 27.3% | ▶ | | #15 | Claude Opus 4.6 | Feb 5, 2026 | 🔍 alltools | max | gpt-5.2 | 27.3% | ▶ | | #16 | Gemini 3.1 Pro Preview | Feb 19, 2026 | 🔍 alltools | high | gpt-5.2 | 26.0% | ▶ | | #17 | Claude Sonnet 4.5 | Sep 29, 2025 | 🔍 Terminal | enabled | gpt-5.2 | 25.3% | ▶ | | #18 | Inkling | Jul 15, 2026 | 🔍 alltools | max | gpt-5.2 | 25.0% | ▶ | | #19 | Claude Opus 4.5 | Nov 24, 2025 | 🔍 alltools | high | gpt-5.2 | 24.7% | ▶ | | #20 | Gemini 3 Pro | Nov 18, 2025 | 🔍 Terminal | high | gpt-5.2 | 18.0% | ▶ | | #21 | Grok 4.2 | Mar 10, 2026") | 🔍 alltools | high | gpt-5.2 | 18.0% | ▶ | | #22 | Grok 4 fast | Sep 19, 2025 | 🔍 alltools | high | gpt-5.2 | 15.7% | ▶ | | #23 | Gemini 2.5 Pro | Jun 17, 2025 | 🔍 alltools | high | gpt-5.2 | 13.7% | ▶ | | #24 | Grok 4.1 fast | Nov 19, 2025 | 🔍 alltools | high | gpt-5.2 | 13.1% | ▶ | | #25 | GPT-5.2 | Dec 11, 2025 | 🔍 qwen\_embeddings | none | gpt-5.2 | 12.6% | ▶ | | #26 | GLM-5 | Feb 11, 2026 | 🔍 text-emb-3-large | enabled | gpt-5.2 | 9.8% | ▶ | | #27 | Qwen3.5-397B-A17B | Feb 16, 2026 | 🔍 text-emb-3-large | enabled | gpt-5.2 | 9.8% | ▶ |
τ³-Banking Banking pass@1 by release date
Pass@1 on τ³-Banking · banking · plotted at each model's public release date. 26 models.
Label every model
0%10%20%30%40%50%60%70%80%90%100%BANKING PASS@1Jun 2025Aug 2025Oct 2025Dec 2025Feb 2026Apr 2026Jun 2026Aug 2026Gemini 2.5 Pro (Google) · 2025-06-17 · 13.7% bankingGrok 4 fast (xAI) · 2025-09-19 · 15.7% bankingClaude Sonnet 4.5 (Anthropic) · 2025-09-29 · 25.3% bankingGemini 3 Pro (Google) · 2025-11-18 · 18.0% bankingGrok 4.1 fast (xAI) · 2025-11-19 · 13.1% bankingClaude Opus 4.5 (Anthropic) · 2025-11-24 · 24.7% bankingGPT-5.2 (OpenAI) · 2025-12-11 · 32.2% bankingGemini 3 Flash (Google) · 2025-12-17 · 27.3% bankingClaude Opus 4.6 (Anthropic) · 2026-02-05 · 27.3% banking🧠GLM-5 (Zhipu AI) · 2026-02-11 · 9.8% bankingQwen3.5-397B-A17B (Alibaba Cloud) · 2026-02-16 · 9.8% bankingGemini 3.1 Pro Preview (Google) · 2026-02-19 · 26.0% bankingGPT-5.4 (OpenAI) · 2026-03-05 · 39.4% bankingGrok 4.2 (xAI) · 2026-03-10 · 18.0% bankingClaude Opus 4.7 (Anthropic) · 2026-04-16 · 40.2% bankingGPT-5.5 (OpenAI) · 2026-04-23 · 44.6% bankingClaude Opus 4.8 (Anthropic) · 2026-05-28 · 39.7% bankingClaude Fable 5 (Anthropic) · 2026-06-09 · 39.7% bankingZGLM-5.2 (Z.ai) · 2026-06-16 · 37.1% bankingGPT-5.6-sol (OpenAI) · 2026-07-09 · 46.9% bankingMMuse Spark 1.1 (Meta) · 2026-07-09 · 40.5% bankingTInkling (Thinking Machines Lab) · 2026-07-15 · 25.0% banking🚀Kimi K3 (Moonshot AI) · 2026-07-16 · 37.1% bankingGrok 4.5 (xAI) · 2026-07-16 · 47.9% bankingQwen 3.8 Max (Qwen) · 2026-07-19 · 55.2% bankingClaude Opus 5 (Anthropic) · 2026-07-24 · 48.7% bankingGemini 2.5 Pro13.7%Grok 4 fast15.7%Claude Sonnet 4.525.3%GPT-5.232.2%GPT-5.439.4%Claude Opus 4.740.2%GPT-5.544.6%GPT-5.6-sol46.9%Grok 4.547.9%Qwen 3.8 Max55.2%
Google
xAI
Anthropic
OpenAI🧠Zhipu AI
Alibaba CloudZZ.aiMMetaTThinking Machines Lab🚀Moonshot AIQwenFrontier (best so far)
Markers show each model's banking pass@1 plotted at its public release date. Hover any marker for details. The dashed line tracks the running best. Submissions without a published `model_release.release_date` fall back to the evaluation date.
Submit Your Results
Have new results to share? Submit your model evaluation results by creating a pull request to add your JSON submission file. See our submission guidelines for the required format and process.