BENCHMARKFirst benchmarks recorded: MiniMax M3 → Cortecs → ?3hBENCHMARKFirst benchmarks recorded: Kimi K3 → Cortecs → ?3hBENCHMARKFirst benchmarks recorded: DeepSeek V4 Pro → Cortecs → ?4hBENCHMARKFirst benchmarks recorded: GLM-4.7 → Cortecs → ?4hBENCHMARKFirst benchmarks recorded: GLM-5.2 → Cortecs → ?4h
ENGINEERING STUDY

Same Model, Different Infrastructure: How Large Is the Gap?

Quantifying the spread between providers serving identical models.

Aug 15, 2026103,921 benchmark runs13 providers8 modelsMethodology v0.10-devDEV DATA
Key findings
  • TTFT for Qwen3-235B spans 148–587 ms across 12 providers — a 4.0× spread.
  • Throughput spans 74–2100 tok/s: a 28.4× difference on identical weights.
  • Output price spans $0.31–$1.20/M — and price does not predict performance.

The ranges

Every metric below is observed on the same logical model under the same workload. The only variable is infrastructure.

TTFT148 ms587 ms
Throughput74 tok/s2100 tok/s
Output price$0.31/M$1.20/M
Reliability96.6%99.3%
Qwen3-235B across providers — per-metric ranges · DEV DATA

Reliability diverges too

Latency differences are expected between infrastructures; capability differences are not. Yet tool-call success — a correctness metric — varies by provider on identical weights, which points at serving stacks (structured-output modes, sampling defaults, streaming implementations), not at the model.

Fireworks99.2OpenRouter99.0Cerebras98.8Nebius98.1Groq97.5Together96.9Nextbit96.7Scaleway96.6DeepInfra95.7Mistral95.7SiliconFlow95.2OVHcloud94.7
Tool-call success by provider · DEV DATA
What this means for engineers
  • Benchmark the combination, not the model. A model choice validated on one provider does not transfer.
  • Correctness metrics (tools, structured output) belong in provider evaluations alongside latency and price.
Dataset
103,921 runs30-day rolling window · Aug 2026snapshot 2026-08-15-v1Methodology v0.10-devDEV DATA

Published figures are pinned to this snapshot and are not recalculated retrospectively. View the current benchmark →

Cite this researchInferenceBench (2026). Same Model, Different Infrastructure: How Large Is the Gap?. InferenceBench Research.
← All research
Related research
ENGINEERING STUDY

P95 Matters: The Hidden Tail Latency of AI Providers

Tail analysis across the provider pool: p95/p50 ratios, who degrades under the tail, and why medians alone are insufficient for interactive workloads.

Aug 15, 2026ENGINEERING STUDY

Tool Calling Is an Infrastructure Problem Too

Tool-call success varies across providers serving the same weights — evidence that serving stacks, not just models, decide agent reliability.

Aug 15, 2026ENGINEERING STUDY

Structured Output Reliability Across Providers

JSON validity and schema compliance measured across the provider pool: who enforces structure, and how large the gap is on identical models.

Aug 15, 2026