PERFORMANCELlama 3.3 70B → Scaleway: throughput ↑ 53%6hPERFORMANCEGLM-5.2 → Scaleway: TTFT ↓ 30%6hPERFORMANCEMistral Medium 3.5 → Scaleway: throughput ↑ 22%6hPERFORMANCEQwen3-235B → Scaleway: throughput ↑ 67%6hPERFORMANCEGPT-5.6 Sol → OpenAI: TTFT ↓ 22%6hPERFORMANCEMistral Large 3 → Mistral AI: TTFT ↓ 21%6hPERFORMANCEGPT-5.6 Luna → OpenAI: throughput ↑ 21%6hPERFORMANCEDeepSeek V4 Flash → Scaleway: throughput ↑ 147%6hPERFORMANCEDeepSeek V4 Flash → Scaleway: TTFT ↓ 71%6hPERFORMANCEGPT-OSS 120B → Scaleway: TTFT ↓ 46%6hPERFORMANCEGPT-OSS 120B → Together AI: TTFT ↓ 40%6hPERFORMANCEKimi K3 → Together AI: TTFT ↓ 20%6hPERFORMANCEDeepSeek V4 Flash → Scaleway: throughput ↑ 36%30hPERFORMANCEDeepSeek V4 Flash → Scaleway: TTFT ↓ 19%30hPERFORMANCEQwen3-235B → Scaleway: throughput ↑ 19%2dPERFORMANCEDeepSeek V4 Flash → Scaleway: throughput ↑ 58%2dPERFORMANCEDeepSeek V4 Flash → Scaleway: TTFT ↑ 21%2dPERFORMANCEGPT-5.6 Sol → OpenAI: throughput ↑ 16%2dPERFORMANCEGPT-OSS 120B → Together AI: TTFT ↑ 22%2dPERFORMANCEKimi K3 → Together AI: throughput ↑ 62%2dPERFORMANCEKimi K3 → Together AI: TTFT ↓ 42%2dPERFORMANCEQwen3-235B → Scaleway: throughput ↑ 30%3dPERFORMANCEDeepSeek V4 Flash → Scaleway: TTFT ↓ 56%3dPERFORMANCELlama 3.3 70B → Scaleway: throughput ↑ 34%3dPERFORMANCELlama 3.3 70B → Scaleway: TTFT ↓ 26%3dPERFORMANCEKimi K3 → Together AI: throughput ↑ 128%3dPERFORMANCEGPT-OSS 120B → Together AI: throughput ↑ 29%3dPERFORMANCEGPT-OSS 120B → Together AI: TTFT ↓ 21%3dPERFORMANCEGPT-OSS 20B → Groq: TTFT ↑ 66%3dPERFORMANCEGLM-5.2 → Together AI: throughput ↑ 47%3dPERFORMANCEMistral Medium 3.5 → Scaleway: TTFT ↑ 18%4dPERFORMANCEGLM-5.2 → Scaleway: throughput ↓ 27%4dPERFORMANCEGLM-5.2 → Scaleway: TTFT ↑ 49%4dPERFORMANCEDeepSeek V4 Flash → Scaleway: throughput ↓ 71%4dPERFORMANCEDeepSeek V4 Flash → Scaleway: TTFT ↑ 190%4dPERFORMANCEGPT-5.6 Luna → OpenAI: throughput ↓ 23%4dPERFORMANCELlama 3.3 70B → Scaleway: TTFT ↑ 60%4dPERFORMANCEGLM-5.2 → Together AI: TTFT ↓ 25%4dPERFORMANCEGPT-OSS 120B → Together AI: throughput ↓ 16%4dPERFORMANCEGPT-OSS 120B → Together AI: TTFT ↓ 17%4d

Best inference providers for AI agents

Benchmarked for tool reliability, structured output, multi-step execution, latency and production reliability.

Provider ranking for Agents

#ProviderΔ30d
01CerebrasBEST OVERALL● MEASUREDUS · Serverless · 1 model · 1 region100.0%100.0%100.0%2513650.5s1789P83↓ 1.0
02MistralBEST EUROPEANEU RESIDENT● MEASUREDFR · Serverless · 2 models · 1 region100.0%100.0%91.7%38610874.3s112P50—
03GroqBEST VALUE● MEASUREDUS · Serverless · 2 models · 1 region100.0%100.0%100.0%2834180.9s691P100↑ 0.3
04Together● MEASUREDUS · Serverless · 3 models · 1 region100.0%100.0%100.0%30924263.3s174P33↑ 16.5
05OpenAISOLE SERVER● MEASUREDUS · Serverless · 2 models · 1 region100.0%100.0%100.0%102123594.7s108P17—
06ScalewayEU RESIDENTEU OWNED● MEASUREDFR · Serverless · 6 models · 1 region100.0%100.0%100.0%1836903.9s112P67↑ 2.5
6 of 18 providers ranked · · Methodology →
01Cerebras99.0
Best overallUS · Serverless · 1 model · 1 region
TTFT251 ms
Rel.100.0%
Tok/s1789
$ Out$0.75
Tools100.0%
Δ30d-1.0
View dossier
02Mistral99.0
Best EuropeanFR · Serverless · 2 models · 1 region
TTFT386 ms
Rel.100.0%
Tok/s112
$ Out$4.50
Tools100.0%
Δ30d—
View dossier
03Groq97.6
Best valueUS · Serverless · 2 models · 1 region
TTFT283 ms
Rel.100.0%
Tok/s691
$ Out$0.45
Tools100.0%
Δ30d+0.3
View dossier
04Together95.8
US · Serverless · 3 models · 1 region
TTFT309 ms
Rel.100.0%
Tok/s174
$ Out$4.40
Tools100.0%
Δ30d+16.5
View dossier
05OpenAI94.6
US · Serverless · 2 models · 1 region
TTFT1021 ms
Rel.100.0%
Tok/s108
$ Out$15.60
Tools100.0%
Δ30d—
View dossier
06Scaleway93.2
FR · Serverless · 6 models · 1 region
TTFT183 ms
Rel.100.0%
Tok/s112
$ Out$1.81
Tools100.0%
Δ30d+2.5
View dossier
6 of 18 providers ranked · · Methodology →
Best inference providers for AI agents · InferenceBench