PERFORMANCEGLM-5.2 → Scaleway: TTFT ↑ 20%25mPERFORMANCEDeepSeek V4 Flash → Scaleway: TTFT ↑ 35%25mPERFORMANCEMistral Medium 3.5 → Scaleway: TTFT ↓ 21%25mPERFORMANCEDeepSeek V4 Flash → Scaleway: throughput ↓ 20%25mPERFORMANCEGPT-OSS 120B → Scaleway: throughput ↑ 31%25mPERFORMANCEGPT-OSS 120B → Scaleway: TTFT ↓ 24%25mPERFORMANCEGPT-OSS 120B → Together AI: throughput ↑ 20%25mPERFORMANCEKimi K3 via OpenRouter: TTFT ↓ 49%25mPERFORMANCEGLM-5.2 via OpenRouter: reliability recovered 96.8% → 100.0%25mPERFORMANCEKimi K3 via Cortecs: throughput ↑ 184%25mPERFORMANCEKimi K3 via Cortecs: TTFT ↓ 42%25mPERFORMANCEGLM-5.2 via OpenRouter: throughput ↓ 49%25mPERFORMANCEGLM-5.2 via OpenRouter: TTFT ↑ 121%25mPERFORMANCEMiniMax M3 via OpenRouter: reliability recovered 96.8% → 100.0%25mPERFORMANCEDeepSeek V4 Pro via Cortecs: throughput ↑ 27%25mPERFORMANCEGPT-OSS 120B via OpenRouter: throughput ↓ 22%25mPERFORMANCEGLM-4.7 via OpenRouter: TTFT ↓ 78%25mPERFORMANCEGPT-OSS 20B → Groq: TTFT ↑ 24%25mPERFORMANCEKimi K3 → Together AI: TTFT ↓ 34%25mPERFORMANCEDeepSeek V4 Flash via OpenRouter: throughput ↓ 17%25mPERFORMANCELlama 3.3 70B via Cortecs: TTFT ↓ 51%25mPERFORMANCEQwen3 235B via OpenRouter: throughput ↑ 37%25mPERFORMANCEQwen3 235B via OpenRouter: TTFT ↓ 29%25mPERFORMANCEDeepSeek V4 Flash → Together AI: throughput ↑ 19%25mPERFORMANCEGLM-5.3 via OpenRouter: throughput ↓ 17%25mPERFORMANCEGPT-OSS 20B via Cortecs: throughput ↑ 28%25mPERFORMANCELlama 3.3 70B → Together AI: TTFT ↓ 33%25mPRICE~moonshotai/kimi-latest cached_input_per_mtok decreased from 0.80 to 0.2926mPRICE~moonshotai/kimi-latest output_per_mtok decreased from 13.00 to 11.3626mPRICE~moonshotai/kimi-latest input_per_mtok decreased from 0.99 to 0.7526mPRICE~deepseek/deepseek-v4-flash-latest cached_input_per_mtok decreased from 0.0077 to 0.001126mPRICE~deepseek/deepseek-v4-flash-latest output_per_mtok decreased from 1.28 to 1.0426mPRICE~deepseek/deepseek-v4-flash-latest input_per_mtok decreased from 0.0077 to 0.003826mPRICEz-ai/glm-5.1 cached_input_per_mtok increased from 0.18 to 0.2626mPRICEz-ai/glm-5.1 output_per_mtok increased from 3.03 to 4.4026mPRICEz-ai/glm-5.1 input_per_mtok increased from 0.96 to 1.4026mPRICEtencent/hy4-preview cached_input_per_mtok increased from 0.038 to 0.04226mPRICEtencent/hy4-preview input_per_mtok increased from 0.75 to 0.8326mPRICEtencent/hy3 cached_input_per_mtok increased from 0.021 to 0.03326mPRICEtencent/hy3 output_per_mtok increased from 0.33 to 0.5326m

Models

Models are inputs to the benchmark — pick one to see who runs it best. 15 benchmarked, 471 observed across the registry.

Benchmarked models

ModelFamilyContextProvidersBest providerBest TTFTBest $/M out
GLM-5.3GLM1M1OpenRouter 79.8412 ms$4.40
DeepSeek V4 ProDeepSeek1M2OpenRouter 80.2480 ms$0.42
DeepSeek V4 FlashDeepSeek1.3M4Cortecs 94.4389 ms$0.056
Kimi K3Kimi1M3Cortecs 73.7602 ms$13.50
GPT-5.6 LunaGPT1M1OpenAI 78.2946 ms$1.20
GPT-5.6 SolGPT1M1OpenAI 66.21134 ms$30.00
GLM-5.2GLM1M4Together 82.5271 ms$3.33
MiniMax M3MiniMax1M3Cortecs 84.4517 ms$1.20
Mistral Medium 3.5Mistral256k4Mistral 80.9164 ms$7.50
GLM-4.7GLM200k1OpenRouter 59.92259 ms$2.20
Mistral Large 3Mistral256k3Mistral 82.1389 ms$1.50
GPT-OSS 120BGPT-OSS128k6OpenRouter 93.3179 ms$0.17
GPT-OSS 20BGPT-OSS128k3Cortecs 97.5175 ms$0.09
Qwen3-235BQwen128k3OpenRouter 86.2162 ms$0.35
Llama 3.3 70BLlama128k4Scaleway 89.9144 ms$0.32

Observed across the registry

Live discovery data
gpt-oss-120bserved by 2 providers · ● BENCHMARKEDgpt-5.6-lunaserved by 2 providers · ● BENCHMARKEDgpt-5.6-solserved by 2 providers · ● BENCHMARKEDopenai/gpt-oss-120bserved by 2 providers · ● BENCHMARKEDdeepseek-v4-flash-0731served by 1 provider · ● BENCHMARKEDdeepseek-v4-proserved by 1 provider · ● BENCHMARKEDdeepseek-v4-pro-0813served by 1 provider · ● BENCHMARKEDglm-4.7-flashserved by 1 provider · ● BENCHMARKEDglm-5.2served by 1 provider · ● BENCHMARKEDglm-5.3served by 1 provider · ● BENCHMARKEDglm-5.3-flashserved by 1 provider · ● BENCHMARKEDgpt-oss-20bserved by 1 provider · ● BENCHMARKEDkimi-k3served by 1 provider · ● BENCHMARKEDllama-3.3-70b-instructserved by 1 provider · ● BENCHMARKEDminimax-m3served by 1 provider · ● BENCHMARKEDqwen3-235b-a22b-instruct-2507served by 1 provider · ● BENCHMARKEDopenai/gpt-oss-20bserved by 1 provider · ● BENCHMARKEDmistral-medium-3-5served by 1 provider · ● BENCHMARKEDdeepseek-ai/DeepSeek-V4-Flash-0731served by 1 provider · ● BENCHMARKEDdeepseek-ai/DeepSeek-V4-Proserved by 1 provider · ● BENCHMARKEDdeepseek-ai/DeepSeek-V4-Pro-0813served by 1 provider · ● BENCHMARKEDMiniMaxAI/MiniMax-M3served by 1 provider · ● BENCHMARKEDmoonshotai/Kimi-K3served by 1 provider · ● BENCHMARKEDQwen/Qwen3-235B-A22B-Instruct-2507served by 1 provider · ● BENCHMARKEDzai-org/GLM-5.2served by 1 provider · ● BENCHMARKEDzai-org/GLM-5.3served by 1 provider · ● BENCHMARKEDzai-org/GLM-5.3-Flashserved by 1 provider · ● BENCHMARKED~deepseek/deepseek-v4-flash-latestserved by 1 provider · ● BENCHMARKEDdeepseek/deepseek-v4-flashserved by 1 provider · ● BENCHMARKEDdeepseek/deepseek-v4-flash-0731served by 1 provider · ● BENCHMARKEDdeepseek/deepseek-v4-flash-vision-expserved by 1 provider · ● BENCHMARKEDdeepseek/deepseek-v4-proserved by 1 provider · ● BENCHMARKEDdeepseek/deepseek-v4-pro-0813served by 1 provider · ● BENCHMARKED
claude-opus-5served by 2 providers · not yet benchmarked
claude-sonnet-5served by 2 providers · not yet benchmarked
codestral-2508served by 2 providers · not yet benchmarked