Correct "no violations" response

Test: Codex Red Herring (False Positive Detection)

Avg. Score
55.9%
Scenarios
8

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1GPT-5.4 Nano (Reasoning, Low)97.5%$0.00083.9s73%
2Nemotron 3 Super98.8%$0.00001.3m78%
3Inception Mercury 295.0%$0.00273.8s56%
4Z.AI GLM 5 Turbo95.0%$0.007116.0s56%
5GPT-5.4 Mini (Reasoning, Low)93.8%$0.00344.0s52%
6o4 Mini95.0%$0.01425.0s56%
7ByteDance Seed 1.6 Flash92.5%$0.00089.1s47%
8GPT-5.4 Nano (Reasoning)92.5%$0.001711.4s47%
9o4 Mini High96.3%$0.02752.5s62%
10Z.AI GLM 5.2 (Reasoning, High)92.5%$0.008125.8s47%
11Grok 4.5 (Reasoning, Low)92.5%$0.01217.0s47%
12GPT-5.193.8%$0.02526.1s52%
13GPT-5.6 Luna (Reasoning)90.0%$0.00484.7s40%
14GPT-5 Mini91.3%$0.005937.8s43%
15GPT-4.185.6%$0.00481.2s38%
16Gemini 2.5 Flash Lite (Reasoning)88.8%$0.002316.6s37%
17GPT-5 Nano90.0%$0.00351.1m40%
18Grok 4.3 (Reasoning)90.0%$0.01450.4s40%
19Grok 4.5 (Reasoning, High)92.5%$0.0311.0m47%
20Ministral 8B74.4%$0.00074.4s33%
21MiniMax M2.786.3%$0.004734.6s31%
22Z.AI GLM 591.3%$0.0171.4m43%
23Ministral 3 8B80.6%$0.001317.9s30%
24GPT-5.4 Mini (Reasoning)85.0%$0.009010.8s29%
25DeepSeek V4 Flash (Reasoning)85.0%$0.000930.8s29%
26Gemma 4 26B (Reasoning)91.3%$0.00272.6m43%
27Gemini 2.5 Flash (Reasoning)82.5%$0.008514.2s24%
28Qwen 3.5 Plus (2026-04-20)90.0%$0.0201.9m40%
29Z.AI GLM 5.191.3%$0.0242.1m43%
30ByteDance Seed 1.681.3%$0.004332.7s22%
31Claude Sonnet 4.673.8%$0.03317.4s39%
32GPT-5.281.3%$0.01314.5s22%
33GPT-4.1 Nano68.1%$0.00054.6s24%
34Xiaomi MIMO v2.581.3%$0.007530.3s22%
35Claude Opus 4.681.9%$0.04913.2s38%
36Grok 4.20 (Reasoning)83.8%$0.01747.9s26%
37GPT-OSS 120B81.3%$0.001256.4s22%
38MiniMax M383.8%$0.00481.4m26%
39Aion 2.083.8%$0.00961.3m26%
40Mistral Small 4 (Reasoning)77.5%$0.002522.6s16%
41Claude Opus 4.7 (Reasoning)90.0%$0.07513.1s40%
42GPT-5.4 Nano66.3%$0.00051.5s18%
43Z.AI GLM 4.5 Air76.9%$0.002743.1s18%
44Xiaomi MIMO v2.5 Pro81.3%$0.0151.1m22%
45Ministral 3B43.1%$0.000412.4s33%
46MiniMax M2.573.1%$0.003225.9s12%
47Claude Haiku 4.551.2%$0.00804.2s24%
48GPT-5.4 (Reasoning, Low)71.3%$0.0128.8s9%
49GPT-5.6 Sol68.8%$0.00911.7s7%
50GPT-4.1 Mini51.2%$0.00186.9s18%
51Mistral Medium 3.153.1%$0.00324.7s17%
52Nemotron 3 Nano83.8%$0.00272.9m26%
53Arcee AI: Trinity Mini67.5%$0.000926.3s6%
54Z.AI GLM 4.561.3%$0.003825.0s12%
55Aion 3.0 Mini78.8%$0.0112.0m21%
56Gemma 4 26B61.9%$0.001239.3s13%
57GPT-5.6 Luna63.7%$0.00201.2s4%
58Gemma 4 31B63.7%$0.000911.7s5%
59DeepSeek V4 Flash35.0%$0.000512.2s27%
60Cydonia 24B V4.158.1%$0.00148.7s8%
61Ministral 3 3B37.5%$0.001331.1s28%
62Z.AI GLM 4.668.1%$0.01559.0s14%
63Claude Opus 4.6 (Reasoning)92.5%$0.1201.0m47%
64Qwen 2.5 72B45.6%$0.000911.3s15%
65Ministral 3 14B50.6%$0.001929.2s15%
66Qwen 3.6 27B83.8%$0.0332.3m26%
67Claude Opus 4.762.5%$0.0493.9s22%
68GPT-5.6 Terra (Reasoning)61.3%$0.00935.0s3%
69Qwen 3.6 Flash68.8%$0.01644.4s7%
70Mistral Small 428.1%$0.00127.4s23%
71Qwen 3 32B60.6%$0.001437.7s3%
72Gemma 4 31B (Reasoning)78.8%$0.00253.1m18%
73Hermes 3 70B42.5%$0.001812.1s9%
74DeepSeek V4 Pro36.3%$0.006942.0s22%
75Qwen3.7 Max78.8%$0.0541.3m18%
76GPT-5.4 (Reasoning)63.7%$0.03231.9s4%
77Z.AI GLM 4.7 Flash70.0%$0.00392.5m8%
78GPT-4o Mini (temp=1)32.5%$0.00085.6s7%
79Claude Sonnet 532.5%$0.0206.0s15%
80Cohere Command R+ (Aug. 2024)50.0%$0.0177.4s0%
81Qwen 3.6 35B62.5%$0.0171.2m3%
82Claude Sonnet 5 (Reasoning)75.0%$0.06957.0s13%
83Aion 3.079.4%$0.0582.4m20%
84Mistral Large 336.9%$0.004011.7s0%
85Claude Sonnet 5 (Reasoning, Low)72.5%$0.06955.4s11%
86Claude Opus 4.540.0%$0.0364.4s9%
87ByteDance Seed 2.0 Mini68.1%$0.00333.1m7%
88GPT-4o, Aug. 6th (temp=1)32.5%$0.00942.4s0%
89GPT-5.6 Terra28.7%$0.00451.4s0%
90Hermes 3 405B30.6%$0.00595.2s0%
91Gemini 3.5 Flash (Reasoning)66.3%$0.06729.5s5%
92Gemini 2.5 Flash Lite25.0%$0.00104.0s0%
93GPT-567.5%$0.0481.5m6%
94Mistral Small 3.2 24B26.3%$0.001011.9s0%
95Mistral Large 235.0%$0.01611.4s0%
96Grok 4.326.9%$0.00774.4s0%
97GPT-5.5 (Reasoning, Low)38.8%$0.02512.0s0%
98Claude Sonnet 435.0%$0.0224.5s0%
99GPT-5.527.5%$0.0132.9s0%
100Llama 3.1 70B25.0%$0.003116.4s0%
101GPT-5.6 Sol (Reasoning)35.0%$0.0269.1s0%
102Gemini 3.5 Flash (Reasoning, Minimal)23.8%$0.0112.2s0%
103Gemini 3 Flash (Preview, Reasoning)44.4%$0.02849.2s0%
104Writer: Palmyra X522.5%$0.008513.8s0%
105GPT-5.4 Mini15.0%$0.00201.2s0%
106GPT-4o Mini (temp=0)16.9%$0.000914.5s0%
107Qwen3 235B A22B Instruct 250721.3%$0.001132.0s0%
108Gemini 3.1 Flash Lite13.1%$0.00173.2s0%
109Gemini 3.1 Flash Lite (Preview)11.9%$0.00171.6s0%
110WizardLM 2 8x22b26.9%$0.004155.3s0%
111Gemini 3.1 Flash Lite (Reasoning)11.3%$0.00151.9s0%
112DeepSeek V4 Pro (Reasoning)65.0%$0.0173.4m5%
113Z.AI GLM 4.755.0%$0.0232.2m1%
114ByteDance Seed 2.0 Lite25.0%$0.00671.0m0%
115Grok 4.2012.5%$0.008211.7s0%
116Claude Sonnet 4.520.6%$0.0245.9s0%
117DeepSeek V3.112.5%$0.001936.6s0%
118DeepSeek V3.213.8%$0.001842.8s0%
119MoonshotAI: Kimi K2.552.5%$0.0172.6m0%
120DeepSeek-V2 Chat5.0%$0.00238.3s0%
121DeepSeek V3 (2024-12-26)4.4%$0.00236.6s0%
122GPT-4o, Aug. 6th (temp=0)8.8%$0.0112.6s0%
123DeepSeek V3 (2025-03-24)5.6%$0.001515.7s0%
124GPT-5.5 (Reasoning)38.8%$0.05124.1s0%
125Gemini 3 Flash (Preview)2.5%$0.00343.2s0%
126Qwen 3.5 Flash34.4%$0.00712.0m0%
127Gemini 2.5 Flash0.6%$0.00211.5s0%
128GPT-5.42.5%$0.00483.1s0%
129Gemma 3 12B1.3%$0.000511.7s0%
130Gemma 3 27B0.0%$0.000712.6s0%
131Gemma 3 4B0.0%$0.000315.0s0%
132Claude Opus 4.8 (Reasoning, Low)78.8%$0.15052.5s18%
133Claude Opus 4.8 (Reasoning)80.0%$0.15856.0s20%
134Gemini 2.5 Pro46.3%$0.07352.1s0%
135Mistral NeMO8.8%$0.00181.6m0%
136Qwen 3.5 Plus (2026-02-15)18.8%$0.0212.0m0%
137Qwen3.6 Max Preview67.5%$0.0723.8m6%
138Claude Sonnet 4.6 (Reasoning)76.3%$0.1452.2m15%
139Qwen 3.5 35B35.0%$0.0442.4m0%
140Qwen 3.5 9B42.5%$0.00384.7m0%
141Gemini 3.1 Pro (Preview)57.5%$0.1201.5m1%
142MoonshotAI: Kimi K2.656.3%$0.0475.2m1%
143Claude Opus 418.1%$0.1097.1s0%
144Qwen 3.5 27B50.6%$0.0564.8m2%
145Qwen 3.5 397B A17B31.3%$0.0285.4m0%
146Qwen 3.5 122B49.4%$0.1356.4m0%
55.90%

Individual Scenarios

basic entries

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100100100.0%
Aion 2.0100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 4.61001001001001001001001001005095.0%
Ministral 3 8B1001001001001001001001001005095.0%
GPT-4.1 Nano1001001001001001001001001005095.0%
GPT-5100100100100100100100100100090.0%
Grok 4.3 (Reasoning)100100100100100100100100100090.0%
GPT-5.2100100100100100100100100100090.0%
Xiaomi MIMO v2.5100100100100100100100100100090.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100090.0%
Nemotron 3 Super100100100100100100100100100090.0%
Mistral Small 4 (Reasoning)100100100100100100100100100090.0%
Nemotron 3 Nano100100100100100100100100100090.0%
ByteDance Seed 1.6 Flash100100100100100100100100100090.0%
Claude Sonnet 4.6 (Reasoning)1001001001001001001001000080.0%
Z.AI GLM 51001001001001001001001000080.0%
o4 Mini High1001001001001001001001000080.0%
o4 Mini1001001001001001001001000080.0%
GPT-OSS 120B1001001001001001001001000080.0%
Gemini 2.5 Flash (Reasoning)1001001001001001001001000080.0%
Aion 3.0 Mini1001001001001001001001000080.0%
GPT-5.4 Mini (Reasoning, Low)1001001001001001001001000080.0%
Cydonia 24B V4.1100100100100100100505050075.0%
Grok 4.5 (Reasoning, Low)10010010010010010010000070.0%
MoonshotAI: Kimi K2.510010010010010010010000070.0%
GPT-5.6 Luna (Reasoning)10010010010010010010000070.0%
Qwen 3.5 Plus (2026-04-20)10010010010010010010000070.0%
DeepSeek V4 Flash (Reasoning)10010010010010010010000070.0%
Z.AI GLM 4.7 Flash10010010010010010010000070.0%
Z.AI GLM 5 Turbo100100100100100100000060.0%
MoonshotAI: Kimi K2.6100100100100100100000060.0%
Claude Sonnet 4.6100100100100505050500060.0%
Ministral 8B10010010050505050500055.0%
GPT-5.4 (Reasoning)1001001001001000000050.0%
Z.AI GLM 5.11001001001001000000050.0%
Z.AI GLM 5.2 (Reasoning, High)1001001001001000000050.0%
Aion 3.01001001001001000000050.0%
Qwen 3.6 27B1001001001001000000050.0%
GPT-4.11001005050505050500050.0%
MiniMax M2.51001001001001000000050.0%
MiniMax M2.71001001001001000000050.0%
Xiaomi MIMO v2.5 Pro1001001001001000000050.0%
Z.AI GLM 4.5 Air1001001001001000000050.0%
GPT-5.4 Nano (Reasoning)1001001001001000000050.0%
Qwen 3 32B1001001001001000000050.0%
GPT-5.4 Nano1001001001005050000050.0%
Claude Opus 4.7505050505050505050045.0%
Z.AI GLM 4.6100100100100500000045.0%
GPT-5.5 (Reasoning, Low)10010010010000000040.0%
Grok 4.20 (Reasoning)10010010010000000040.0%
ByteDance Seed 1.610010010010000000040.0%
Claude Sonnet 550505050505050500040.0%
DeepSeek V4 Flash50505050505050500040.0%
Writer: Palmyra X550505050505050500040.0%
Qwen 2.5 72B50505050505050500040.0%
Arcee AI: Trinity Mini10010010010000000040.0%
Ministral 3B50505050505050500040.0%
Claude Opus 4.55050505050505000035.0%
Qwen3 235B A22B Instruct 25075050505050505000035.0%
Mistral Small 45050505050505000035.0%
Ministral 3 14B5050505050505000035.0%
GPT-5.6 Sol (Reasoning)100100100000000030.0%
GPT-5.5 (Reasoning)100100100000000030.0%
Claude Opus 4.7 (Reasoning)100100100000000030.0%
GPT-5 Mini100100100000000030.0%
GPT-5.4 (Reasoning, Low)100100100000000030.0%
Claude Sonnet 5 (Reasoning)100100100000000030.0%
Gemma 4 26B (Reasoning)100100100000000030.0%
Qwen 3.6 35B100100100000000030.0%
Claude Sonnet 4.5505050505050000030.0%
Gemini 2.5 Flash Lite100100100000000030.0%
WizardLM 2 8x22b505050505050000030.0%
Ministral 3 3B505050505050000030.0%
Mistral NeMO100100100000000030.0%
Gemini 3.1 Pro (Preview)1001000000000020.0%
GPT-5.6 Terra (Reasoning)1001000000000020.0%
Claude Sonnet 5 (Reasoning, Low)1001000000000020.0%
Gemma 4 31B (Reasoning)1001000000000020.0%
Claude Sonnet 45050505000000020.0%
DeepSeek V4 Pro5050505000000020.0%
GPT-4o, Aug. 6th (temp=1)5050505000000020.0%
Hermes 3 405B1001000000000020.0%
GPT-4o Mini (temp=1)5050505000000020.0%
Mistral Medium 3.15050505000000020.0%
Claude Opus 4505050000000015.0%
Z.AI GLM 4.5100500000000015.0%
Gemma 4 26B505050000000015.0%
Claude Haiku 4.5505050000000015.0%
DeepSeek V3.1505050000000015.0%
Llama 3.1 70B100500000000015.0%
Hermes 3 70B100500000000015.0%
Gemini 3.5 Flash (Reasoning)10000000000010.0%
Qwen 3.6 Flash10000000000010.0%
Z.AI GLM 4.710000000000010.0%
Gemma 4 31B50500000000010.0%
DeepSeek V3.250500000000010.0%
GPT-4.1 Mini50500000000010.0%
Grok 4.2050500000000010.0%
GPT-5.4 Mini10000000000010.0%
Grok 4.310000000000010.0%
Cohere Command R+ (Aug. 2024)10000000000010.0%
Qwen 3.5 122B500000000005.0%
Qwen 3.5 27B500000000005.0%
Gemini 3 Flash (Preview, Reasoning)500000000005.0%
Gemini 3.1 Flash Lite (Reasoning)500000000005.0%
ByteDance Seed 2.0 Mini500000000005.0%
Gemini 3 Flash (Preview)500000000005.0%
Gemini 3.1 Flash Lite500000000005.0%
DeepSeek V3 (2024-12-26)500000000005.0%
Qwen3.7 Max00000000000.0%
Qwen3.6 Max Preview00000000000.0%
Claude Opus 4.8 (Reasoning)00000000000.0%
Claude Opus 4.8 (Reasoning, Low)00000000000.0%
Qwen 3.5 397B A17B00000000000.0%
GPT-5.6 Sol00000000000.0%
MiniMax M300000000000.0%
GPT-5.4 Mini (Reasoning)00000000000.0%
GPT-5.500000000000.0%
Gemini 2.5 Pro00000000000.0%
GPT-5.6 Terra00000000000.0%
Qwen 3.5 35B00000000000.0%
Qwen 3.5 Plus (2026-02-15)00000000000.0%
Gemini 3.5 Flash (Reasoning, Minimal)00000000000.0%
Qwen 3.5 Flash00000000000.0%
Gemini 3.1 Flash Lite (Preview)00000000000.0%
GPT-5.6 Luna00000000000.0%
GPT-5.400000000000.0%
Mistral Large 300000000000.0%
ByteDance Seed 2.0 Lite00000000000.0%
DeepSeek-V2 Chat00000000000.0%
Qwen 3.5 9B00000000000.0%
GPT-4o, Aug. 6th (temp=0)00000000000.0%
Mistral Large 200000000000.0%
Gemini 2.5 Flash00000000000.0%
DeepSeek V3 (2025-03-24)00000000000.0%
Mistral Small 3.2 24B00000000000.0%
GPT-4o Mini (temp=0)00000000000.0%
Gemma 3 12B00000000000.0%
Gemma 3 27B00000000000.0%
Gemma 3 4B00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Qwen3.7 Max100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
Aion 3.0 Mini100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Claude Sonnet 4.61001001001001001001001001005095.0%
Mistral Medium 3.11001001001001001001001001005095.0%
GPT-4.1 Nano1001001001001001001001001005095.0%
Qwen 3.6 Flash100100100100100100100100100090.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100090.0%
Aion 2.0100100100100100100100100100090.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100090.0%
Z.AI GLM 4.5 Air100100100100100100100100100090.0%
Ministral 3 8B100100100100100100100100100090.0%
Aion 3.010010010010010010010010050085.0%
Claude Sonnet 5 (Reasoning, Low)1001001001001001001001000080.0%
Gemini 2.5 Pro1001001001001001001001000080.0%
Qwen 3.6 35B1001001001001001001001000080.0%
MiniMax M2.51001001001001001001001000080.0%
Gemini 3.1 Flash Lite1001001001001001001001000080.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100500075.0%
Ministral 8B100100100100100505050505075.0%
Gemini 3.5 Flash (Reasoning)10010010010010010010000070.0%
Gemini 2.5 Flash Lite10010010010010010010000070.0%
Mistral Small 3.2 24B10010010010010010010000070.0%
Hermes 3 70B10010010010010010050500070.0%
Gemini 3.1 Flash Lite (Preview)1001001001001001005000065.0%
Z.AI GLM 4.51001001001001005050500065.0%
GPT-5.4 Nano1001001001001001005000065.0%
Claude Opus 4.8 (Reasoning)100100100100100100000060.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100000060.0%
Mistral Small 4 (Reasoning)100100100100100100000060.0%
Qwen 3 32B100100100100100100000060.0%
Cydonia 24B V4.1100100100100100100000060.0%
Qwen 3.5 27B10010010010010050000055.0%
Z.AI GLM 4.610010010010050505000055.0%
Grok 4.2010010010010050505000055.0%
Claude Opus 4.75050505050505050505050.0%
Claude Haiku 4.55050505050505050505050.0%
Z.AI GLM 4.7 Flash1001001001001000000050.0%
Claude Sonnet 4.5505050505050505050045.0%
DeepSeek V4 Flash505050505050505050045.0%
GPT-4.1 Mini505050505050505050045.0%
Writer: Palmyra X5505050505050505050045.0%
Ministral 3 3B505050505050505050045.0%
Ministral 3B505050505050505050045.0%
Gemini 3.1 Pro (Preview)10010010010000000040.0%
Qwen 3.5 397B A17B10010010010000000040.0%
DeepSeek V4 Pro50505050505050500040.0%
Arcee AI: Trinity Mini10010010010000000040.0%
Claude Opus 4.65050505050505000035.0%
Claude Sonnet 45050505050505000035.0%
Qwen3 235B A22B Instruct 25075050505050505000035.0%
GPT-4o Mini (temp=1)1005050505050000035.0%
Mistral Small 41005050505050000035.0%
Qwen 3.5 122B100100100000000030.0%
Claude Sonnet 5505050505050000030.0%
Claude Opus 4505050505050000030.0%
Qwen 3.5 35B100100100000000030.0%
Gemini 3.5 Flash (Reasoning, Minimal)100505050500000030.0%
Gemma 4 26B100505050500000030.0%
Qwen 3.5 9B100100100000000030.0%
WizardLM 2 8x22b100505050500000030.0%
Qwen 3.5 Plus (2026-02-15)50505050500000025.0%
DeepSeek V3.110050505000000025.0%
GPT-4o, Aug. 6th (temp=1)10050505000000025.0%
Qwen 2.5 72B10050505000000025.0%
DeepSeek V3.25050505000000020.0%
Mistral NeMO1001000000000020.0%
DeepSeek V3 (2025-03-24)505050000000015.0%
Grok 4.3100500000000015.0%
Ministral 3 14B505050000000015.0%
Gemini 3 Flash (Preview, Reasoning)10000000000010.0%
Claude Opus 4.550500000000010.0%
Qwen 3.5 Flash10000000000010.0%
Hermes 3 405B10000000000010.0%
GPT-5.4 Mini10000000000010.0%
Cohere Command R+ (Aug. 2024)10000000000010.0%
Gemini 3 Flash (Preview)500000000005.0%
DeepSeek-V2 Chat500000000005.0%
DeepSeek V3 (2024-12-26)500000000005.0%
Mistral Large 2500000000005.0%
Llama 3.1 70B500000000005.0%
GPT-5.500000000000.0%
GPT-5.400000000000.0%
Mistral Large 300000000000.0%
ByteDance Seed 2.0 Lite00000000000.0%
GPT-4o, Aug. 6th (temp=0)00000000000.0%
Gemini 2.5 Flash00000000000.0%
GPT-4o Mini (temp=0)00000000000.0%
Gemma 3 12B00000000000.0%
Gemma 3 27B00000000000.0%
Gemma 3 4B00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Qwen3.7 Max100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen3.6 Max Preview100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 8B1001001001001001001001001005095.0%
Grok 4.3 (Reasoning)100100100100100100100100100090.0%
GPT-5.1100100100100100100100100100090.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100090.0%
Gemma 4 31B (Reasoning)100100100100100100100100100090.0%
Qwen 3.6 Flash100100100100100100100100100090.0%
Aion 2.0100100100100100100100100100090.0%
MiniMax M2.7100100100100100100100100100090.0%
ByteDance Seed 2.0 Mini100100100100100100100100100090.0%
Gemma 4 26B100100100100100100100100100090.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100090.0%
Z.AI GLM 4.5 Air100100100100100100100100505090.0%
Mistral Small 4 (Reasoning)100100100100100100100100100090.0%
Qwen 3 32B100100100100100100100100100090.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100100100100090.0%
Aion 3.01001001001001001001001000080.0%
Gemini 2.5 Pro1001001001001001001001000080.0%
DeepSeek V4 Flash (Reasoning)1001001001001001001001000080.0%
Qwen 3.6 35B1001001001001001001001000080.0%
Xiaomi MIMO v2.5 Pro1001001001001001001001000080.0%
GPT-5.4 Mini (Reasoning, Low)1001001001001001001001000080.0%
Z.AI GLM 4.7 Flash1001001001001001001001000080.0%
Nemotron 3 Nano1001001001001001001001000080.0%
Arcee AI: Trinity Mini1001001001001001001001000080.0%
Claude Opus 4.7100100100100100505050505075.0%
Claude Sonnet 4100100100100100100505050075.0%
GPT-5.4 Nano100100100100100100505050075.0%
Ministral 8B100100100100100100505050075.0%
GPT-5.6 Luna (Reasoning)10010010010010010010000070.0%
Z.AI GLM 510010010010010010010000070.0%
GPT-5.210010010010010010010000070.0%
Xiaomi MIMO v2.510010010010010010010000070.0%
Aion 3.0 Mini10010010010010010010000070.0%
Hermes 3 405B10010010010010010010000070.0%
GPT-5 Nano10010010010010010010000070.0%
Hermes 3 70B1001001001001005050500065.0%
Z.AI GLM 4.6100100100100100505000060.0%
MiniMax M2.5100100100100100100000060.0%
GPT-OSS 120B100100100100100100000060.0%
Cohere Command R+ (Aug. 2024)100100100100100100000060.0%
Claude Haiku 4.510050505050505050505055.0%
Qwen 3.5 9B1001001001001000000050.0%
DeepSeek V4 Pro1001005050505050500050.0%
Mistral Medium 3.15050505050505050505050.0%
Cydonia 24B V4.11001001001005050000050.0%
Ministral 3 3B5050505050505050505050.0%
Ministral 3B5050505050505050505050.0%
Claude Sonnet 4.6505050505050505050045.0%
GPT-4.1 Mini505050505050505050045.0%
GPT-4o, Aug. 6th (temp=1)100100100505050000045.0%
Qwen 2.5 72B100505050505050500045.0%
MoonshotAI: Kimi K2.610010010010000000040.0%
GPT-510010010010000000040.0%
Qwen 3.5 397B A17B10010010010000000040.0%
MoonshotAI: Kimi K2.510010010010000000040.0%
Qwen 3.5 27B10010010050500000040.0%
Gemini 3 Flash (Preview, Reasoning)10010010010000000040.0%
DeepSeek V4 Pro (Reasoning)10010010010000000040.0%
Z.AI GLM 4.710010010010000000040.0%
Z.AI GLM 4.510010010050500000040.0%
DeepSeek V4 Flash50505050505050500040.0%
Ministral 3 14B50505050505050500040.0%
GPT-4.1 Nano10010010010000000040.0%
WizardLM 2 8x22b1005050505050000035.0%
GPT-5.5 (Reasoning)100100100000000030.0%
Qwen 3.5 122B100100100000000030.0%
Qwen 3.5 Plus (2026-02-15)505050505050000030.0%
Writer: Palmyra X5100100505000000030.0%
Mistral Large 350505050500000025.0%
Mistral Large 250505050500000025.0%
Grok 4.310010050000000025.0%
Mistral Small 450505050500000025.0%
Qwen 3.5 35B1001000000000020.0%
Llama 3.1 70B1005050000000020.0%
Mistral NeMO1005050000000020.0%
DeepSeek V3.1505050000000015.0%
GPT-4o Mini (temp=0)505050000000015.0%
GPT-5.4 (Reasoning)10000000000010.0%
GPT-5.4 (Reasoning, Low)10000000000010.0%
Claude Opus 4.510000000000010.0%
GPT-5.6 Terra10000000000010.0%
Claude Sonnet 4.510000000000010.0%
Qwen 3.5 Flash10000000000010.0%
Gemini 3.1 Flash Lite (Preview)50500000000010.0%
ByteDance Seed 2.0 Lite10000000000010.0%
DeepSeek V3.250500000000010.0%
GPT-5.4 Mini10000000000010.0%
Qwen3 235B A22B Instruct 250710000000000010.0%
GPT-4o Mini (temp=1)50500000000010.0%
Claude Sonnet 5500000000005.0%
Claude Opus 4500000000005.0%
Gemini 3.1 Flash Lite (Reasoning)500000000005.0%
Gemini 3.1 Flash Lite500000000005.0%
DeepSeek V3 (2024-12-26)500000000005.0%
GPT-4o, Aug. 6th (temp=0)500000000005.0%
Gemini 2.5 Flash500000000005.0%
Mistral Small 3.2 24B500000000005.0%
Gemma 3 12B500000000005.0%
GPT-5.6 Sol (Reasoning)00000000000.0%
GPT-5.5 (Reasoning, Low)00000000000.0%
GPT-5.6 Terra (Reasoning)00000000000.0%
Gemini 3.5 Flash (Reasoning, Minimal)00000000000.0%
Gemini 3 Flash (Preview)00000000000.0%
GPT-5.400000000000.0%
DeepSeek-V2 Chat00000000000.0%
Grok 4.2000000000000.0%
DeepSeek V3 (2025-03-24)00000000000.0%
Gemini 2.5 Flash Lite00000000000.0%
Gemma 3 27B00000000000.0%
Gemma 3 4B00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Qwen3.7 Max100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen3.6 Max Preview100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Gemma 4 26B100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Z.AI GLM 4.61001001001001001001001001005095.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100090.0%
GPT-5.1100100100100100100100100100090.0%
Z.AI GLM 5100100100100100100100100100090.0%
Qwen 3.6 Flash100100100100100100100100100090.0%
Aion 3.0100100100100100100100100100090.0%
o4 Mini High100100100100100100100100100090.0%
MiniMax M2.5100100100100100100100100100090.0%
ByteDance Seed 2.0 Mini100100100100100100100100100090.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100090.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100090.0%
Nemotron 3 Nano100100100100100100100100100090.0%
Claude Sonnet 4.6 (Reasoning)1001001001001001001001000080.0%
Grok 4.3 (Reasoning)1001001001001001001001000080.0%
Grok 4.5 (Reasoning, Low)1001001001001001001001000080.0%
Claude Sonnet 5 (Reasoning)1001001001001001001001000080.0%
Qwen 3.5 27B1001001001001001001001000080.0%
GPT-OSS 120B1001001001001001001001000080.0%
GPT-5 Nano1001001001001001001001000080.0%
Arcee AI: Trinity Mini1001001001001001001001000080.0%
Z.AI GLM 4.5100100100100100100100500075.0%
GPT-5.4 Nano100100100100100100100500075.0%
GPT-4.1 Nano100100100100100505050505075.0%
GPT-5.210010010010010010010000070.0%
Qwen 3.6 27B10010010010010010010000070.0%
DeepSeek V4 Flash (Reasoning)10010010010010010010000070.0%
Qwen 3.6 35B10010010010010010010000070.0%
Aion 2.010010010010010010010000070.0%
Qwen 3 32B10010010010010010010000070.0%
GPT-4.1 Mini1001001001005050505050065.0%
Qwen 3.5 122B100100100100100100000060.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100000060.0%
Xiaomi MIMO v2.5 Pro100100100100100100000060.0%
Claude Haiku 4.5100100505050505050505060.0%
Llama 3.1 70B100100100100100505000060.0%
Qwen 2.5 72B100100100100505050500060.0%
Cohere Command R+ (Aug. 2024)100100100100100100000060.0%
Aion 3.0 Mini10010010010010050000055.0%
MoonshotAI: Kimi K2.61001001001001000000050.0%
Qwen 3.5 397B A17B1001001001001000000050.0%
GPT-5.4 (Reasoning, Low)1001001001001000000050.0%
Gemini 3 Flash (Preview, Reasoning)1001001001001000000050.0%
Gemini 2.5 Pro1001001001001000000050.0%
Xiaomi MIMO v2.51001001001001000000050.0%
Cydonia 24B V4.11001001005050505000050.0%
Ministral 3 14B5050505050505050505050.0%
Hermes 3 70B1001001005050505000050.0%
Claude Sonnet 4.6505050505050505050045.0%
Claude Sonnet 4100100100100500000045.0%
Ministral 3 3B505050505050505050045.0%
Ministral 8B505050505050505050045.0%
Ministral 3B505050505050505050045.0%
Mistral Large 350505050505050500040.0%
Qwen 3.5 9B10010010010000000040.0%
Hermes 3 405B10010010010000000040.0%
Mistral Medium 3.150505050505050500040.0%
GPT-4.11001005050500000035.0%
Ministral 3 8B5050505050505000035.0%
GPT-5.5 (Reasoning, Low)100100100000000030.0%
Z.AI GLM 4.7100100100000000030.0%
DeepSeek V4 Pro505050505050000030.0%
DeepSeek V4 Flash505050505050000030.0%
Mistral Large 2505050505050000030.0%
GPT-4o, Aug. 6th (temp=1)100505050500000030.0%
Claude Sonnet 4.510010050000000025.0%
DeepSeek V3.250505050500000025.0%
Grok 4.310010050000000025.0%
GPT-4o Mini (temp=1)10050505000000025.0%
GPT-51001000000000020.0%
GPT-5.6 Terra1001000000000020.0%
DeepSeek-V2 Chat1001000000000020.0%
GPT-5.4 Mini1001000000000020.0%
GPT-4o Mini (temp=0)5050505000000020.0%
Mistral Small 45050505000000020.0%
DeepSeek V3 (2024-12-26)100500000000015.0%
WizardLM 2 8x22b505050000000015.0%
GPT-5.4 (Reasoning)10000000000010.0%
GPT-5.5 (Reasoning)10000000000010.0%
DeepSeek V4 Pro (Reasoning)10000000000010.0%
Qwen 3.5 35B10000000000010.0%
Qwen 3.5 Plus (2026-02-15)50500000000010.0%
GPT-5.410000000000010.0%
Grok 4.2010000000000010.0%
Writer: Palmyra X550500000000010.0%
Gemini 3.1 Flash Lite500000000005.0%
DeepSeek V3.1500000000005.0%
Qwen3 235B A22B Instruct 2507500000000005.0%
GPT-5.6 Sol (Reasoning)00000000000.0%
GPT-5.6 Terra (Reasoning)00000000000.0%
MoonshotAI: Kimi K2.500000000000.0%
Claude Opus 4.500000000000.0%
Claude Sonnet 500000000000.0%
Claude Opus 400000000000.0%
Gemini 3.1 Flash Lite (Reasoning)00000000000.0%
Qwen 3.5 Flash00000000000.0%
Gemini 3 Flash (Preview)00000000000.0%
Gemini 3.1 Flash Lite (Preview)00000000000.0%
ByteDance Seed 2.0 Lite00000000000.0%
GPT-4o, Aug. 6th (temp=0)00000000000.0%
Gemini 2.5 Flash00000000000.0%
DeepSeek V3 (2025-03-24)00000000000.0%
Gemini 2.5 Flash Lite00000000000.0%
Mistral Small 3.2 24B00000000000.0%
Gemma 3 12B00000000000.0%
Gemma 3 27B00000000000.0%
Gemma 3 4B00000000000.0%
Mistral NeMO00000000000.0%

detailed entries

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen3.6 Max Preview100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
Aion 2.0100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Ministral 8B100100100100100100100100100100100.0%
Ministral 3 14B1001001001001001001001001005095.0%
Qwen3.7 Max100100100100100100100100100090.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100090.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100090.0%
GPT-5.6 Terra (Reasoning)100100100100100100100100100090.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100090.0%
MoonshotAI: Kimi K2.5100100100100100100100100100090.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100090.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100090.0%
GPT-5.2100100100100100100100100100090.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100090.0%
MiniMax M2.7100100100100100100100100100090.0%
Aion 3.0 Mini100100100100100100100100100090.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100090.0%
GPT-5 Nano100100100100100100100100100090.0%
Arcee AI: Trinity Mini100100100100100100100100100090.0%
GPT-5.6 Luna (Reasoning)1001001001001001001001000080.0%
Qwen 3.5 122B1001001001001001001001000080.0%
Gemma 4 31B (Reasoning)1001001001001001001001000080.0%
ByteDance Seed 1.61001001001001001001001000080.0%
Qwen 3.6 Flash1001001001001001001001000080.0%
MiniMax M2.51001001001001001001001000080.0%
Qwen 3.5 Flash1001001001001001001001000080.0%
Gemini 2.5 Flash (Reasoning)1001001001001001001001000080.0%
Inception Mercury 21001001001001001001001000080.0%
Nemotron 3 Nano1001001001001001001001000080.0%
ByteDance Seed 1.6 Flash1001001001001001001001000080.0%
GPT-5.5 (Reasoning, Low)10010010010010010010000070.0%
Qwen 3.6 35B10010010010010010010000070.0%
Z.AI GLM 4.510010010010010010050500070.0%
GPT-5.4 Nano10010010010010010010000070.0%
Qwen 2.5 72B1001001001001005050500065.0%
Gemini 3.1 Pro (Preview)100100100100100100000060.0%
GPT-5.4 (Reasoning)100100100100100100000060.0%
GPT-5.6 Sol100100100100100100000060.0%
Claude Sonnet 4.6100100505050505050505060.0%
Z.AI GLM 4.7100100100100100100000060.0%
Qwen 3.5 35B100100100100100100000060.0%
Z.AI GLM 4.610010010010010050000055.0%
Qwen 3 32B10010010010010050000055.0%
GPT-5.6 Sol (Reasoning)1001001001001000000050.0%
Claude Sonnet 4.51001005050505050500050.0%
Claude Haiku 4.55050505050505050505050.0%
Mistral Small 4 (Reasoning)1001001001001000000050.0%
Mistral Medium 3.15050505050505050505050.0%
GPT-4.1 Nano1001001001005050000050.0%
Cohere Command R+ (Aug. 2024)1001001001001000000050.0%
GPT-4.1 Mini505050505050505050045.0%
GPT-5.5 (Reasoning)10010010010000000040.0%
Qwen 3.5 397B A17B10010010010000000040.0%
GPT-OSS 120B10010010010000000040.0%
DeepSeek V4 Pro50505050505050500040.0%
Z.AI GLM 4.5 Air10010010010000000040.0%
Gemini 2.5 Flash Lite10010010050500000040.0%
Qwen3 235B A22B Instruct 250710050505050505000040.0%
Cydonia 24B V4.110050505050505000040.0%
Claude Opus 4.75050505050505000035.0%
Claude Sonnet 55050505050505000035.0%
DeepSeek V4 Flash5050505050505000035.0%
Ministral 3B5050505050505000035.0%
Qwen 3.5 9B100100505000000030.0%
GPT-4o, Aug. 6th (temp=1)100100505000000030.0%
Writer: Palmyra X5505050505050000030.0%
Mistral Small 3.2 24B100100100000000030.0%
WizardLM 2 8x22b100100505000000030.0%
Ministral 3 3B505050505050000030.0%
DeepSeek V3.150505050500000025.0%
GPT-4o, Aug. 6th (temp=0)50505050500000025.0%
Grok 4.310010050000000025.0%
Claude Opus 45050505000000020.0%
DeepSeek V3.25050505000000020.0%
GPT-5.4 Mini1001000000000020.0%
Mistral Small 45050505000000020.0%
Claude Opus 4.5505050000000015.0%
GPT-4o Mini (temp=1)505050000000015.0%
Hermes 3 70B100500000000015.0%
GPT-5.510000000000010.0%
Gemini 3 Flash (Preview)50500000000010.0%
Gemma 4 26B50500000000010.0%
DeepSeek-V2 Chat50500000000010.0%
Z.AI GLM 4.7 Flash10000000000010.0%
Qwen 3.5 27B500000000005.0%
Claude Sonnet 4500000000005.0%
Gemini 3.1 Flash Lite (Preview)500000000005.0%
Mistral Large 3500000000005.0%
Mistral Large 2500000000005.0%
Grok 4.20500000000005.0%
Hermes 3 405B500000000005.0%
Llama 3.1 70B500000000005.0%
Gemini 2.5 Pro00000000000.0%
GPT-5.6 Terra00000000000.0%
Qwen 3.5 Plus (2026-02-15)00000000000.0%
Gemini 3.1 Flash Lite (Reasoning)00000000000.0%
Gemini 3.5 Flash (Reasoning, Minimal)00000000000.0%
Gemma 4 31B00000000000.0%
Gemini 3.1 Flash Lite00000000000.0%
GPT-5.6 Luna00000000000.0%
GPT-5.400000000000.0%
DeepSeek V3 (2024-12-26)00000000000.0%
Gemini 2.5 Flash00000000000.0%
DeepSeek V3 (2025-03-24)00000000000.0%
GPT-4o Mini (temp=0)00000000000.0%
Gemma 3 12B00000000000.0%
Gemma 3 27B00000000000.0%
Gemma 3 4B00000000000.0%
Mistral NeMO00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100090.0%
GPT-5.5 (Reasoning)100100100100100100100100100090.0%
GPT-5100100100100100100100100100090.0%
Grok 4.3 (Reasoning)100100100100100100100100100090.0%
Grok 4.20 (Reasoning)100100100100100100100100100090.0%
MiniMax M3100100100100100100100100100090.0%
Qwen 3.6 Flash100100100100100100100100100090.0%
Qwen 3.6 27B100100100100100100100100100090.0%
Qwen 3.6 35B100100100100100100100100100090.0%
o4 Mini100100100100100100100100100090.0%
ByteDance Seed 2.0 Mini100100100100100100100100100090.0%
GPT-OSS 120B100100100100100100100100100090.0%
Z.AI GLM 4.7 Flash100100100100100100100100100090.0%
GPT-5 Nano100100100100100100100100100090.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100090.0%
Claude Sonnet 4.610010010010010010010010050085.0%
Claude Sonnet 5 (Reasoning, Low)1001001001001001001001000080.0%
DeepSeek V4 Flash (Reasoning)1001001001001001001001000080.0%
Qwen 3.5 9B1001001001001001001001000080.0%
Xiaomi MIMO v2.51001001001001001001001000080.0%
Z.AI GLM 4.5 Air1001001001001001001001000080.0%
Qwen 3 32B1001001001001001001001000080.0%
Nemotron 3 Nano1001001001001001001001000080.0%
Z.AI GLM 4.5100100100100100100505050075.0%
Aion 3.0 Mini100100100100100100100500075.0%
GPT-4.1 Nano100100100100100100100500075.0%
GPT-5.5 (Reasoning, Low)10010010010010010010000070.0%
Claude Sonnet 5 (Reasoning)10010010010010010010000070.0%
DeepSeek V4 Pro (Reasoning)10010010010010010010000070.0%
MiniMax M2.710010010010010010010000070.0%
Xiaomi MIMO v2.5 Pro10010010010010010010000070.0%
Qwen 3.5 Flash10010010010010010010000070.0%
Gemini 2.5 Flash Lite (Reasoning)10010010010010010010000070.0%
Mistral Small 4 (Reasoning)10010010010010010010000070.0%
Z.AI GLM 4.61001001001001001005000065.0%
MiniMax M2.51001001001001001005000065.0%
Gemma 4 26B1001001001005050505050065.0%
Mistral Medium 3.11001001001005050505050065.0%
Qwen 3.5 27B100100100100100100000060.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100000060.0%
Qwen 3.5 35B100100100100100100000060.0%
GPT-4o, Aug. 6th (temp=1)100100100100100505000060.0%
Hermes 3 70B100100100505050505050060.0%
Cohere Command R+ (Aug. 2024)100100100100100100000060.0%
Cydonia 24B V4.110010010010050505000055.0%
Qwen3.7 Max1001001001001000000050.0%
Claude Opus 4.55050505050505050505050.0%
GPT-5.21001001001001000000050.0%
Z.AI GLM 4.71001001001001000000050.0%
Claude Haiku 4.55050505050505050505050.0%
Ministral 3B5050505050505050505050.0%
Claude Opus 4.7505050505050505050045.0%
GPT-4.1 Mini505050505050505050045.0%
Ministral 8B100100100505050000045.0%
Claude Opus 4.650505050505050500040.0%
GPT-5.6 Sol10010010010000000040.0%
Gemma 4 31B (Reasoning)10010010010000000040.0%
Aion 2.010010010010000000040.0%
Mistral Small 410050505050505000040.0%
Qwen 2.5 72B10010050505050000040.0%
Claude Sonnet 55050505050505000035.0%
Qwen 3.5 Plus (2026-02-15)5050505050505000035.0%
DeepSeek V4 Pro5050505050505000035.0%
WizardLM 2 8x22b5050505050505000035.0%
Qwen3.6 Max Preview100100100000000030.0%
Qwen 3.5 122B100100100000000030.0%
Gemini 3 Flash (Preview, Reasoning)100100100000000030.0%
ByteDance Seed 1.6100100100000000030.0%
Claude Sonnet 4100505050500000030.0%
Claude Opus 4505050505050000030.0%
GPT-5.4 Nano100505050500000030.0%
Ministral 3 8B100100505000000030.0%
Mistral Large 350505050500000025.0%
DeepSeek V4 Flash50505050500000025.0%
Ministral 3 3B50505050500000025.0%
Gemini 2.5 Pro1001000000000020.0%
Gemini 2.5 Flash (Reasoning)1001000000000020.0%
Hermes 3 405B1001000000000020.0%
Gemini 2.5 Flash Lite1001000000000020.0%
Grok 4.31005050000000020.0%
Llama 3.1 70B1001000000000020.0%
Arcee AI: Trinity Mini1001000000000020.0%
Mistral Large 2505050000000015.0%
Mistral Small 3.2 24B100500000000015.0%
Ministral 3 14B505050000000015.0%
MoonshotAI: Kimi K2.610000000000010.0%
MoonshotAI: Kimi K2.510000000000010.0%
GPT-5.510000000000010.0%
Gemini 3.1 Flash Lite50500000000010.0%
GPT-5.410000000000010.0%
DeepSeek V3.150500000000010.0%
DeepSeek V3.250500000000010.0%
Writer: Palmyra X550500000000010.0%
Qwen3 235B A22B Instruct 250750500000000010.0%
Claude Sonnet 4.5500000000005.0%
Gemini 3.1 Flash Lite (Reasoning)500000000005.0%
Gemini 3.1 Flash Lite (Preview)500000000005.0%
DeepSeek-V2 Chat500000000005.0%
DeepSeek V3 (2024-12-26)500000000005.0%
Qwen 3.5 397B A17B00000000000.0%
Gemini 3.5 Flash (Reasoning, Minimal)00000000000.0%
Gemini 3 Flash (Preview)00000000000.0%
Gemma 4 31B00000000000.0%
ByteDance Seed 2.0 Lite00000000000.0%
GPT-4o, Aug. 6th (temp=0)00000000000.0%
Grok 4.2000000000000.0%
Gemini 2.5 Flash00000000000.0%
GPT-5.4 Mini00000000000.0%
DeepSeek V3 (2025-03-24)00000000000.0%
Gemma 3 12B00000000000.0%
Gemma 3 27B00000000000.0%
Gemma 3 4B00000000000.0%
Mistral NeMO00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Qwen3.7 Max100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Ministral 8B100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)1001001001001001001001001005095.0%
Qwen3.6 Max Preview100100100100100100100100100090.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100090.0%
Grok 4.3 (Reasoning)100100100100100100100100100090.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100090.0%
Z.AI GLM 5100100100100100100100100100090.0%
Gemini 2.5 Pro100100100100100100100100100090.0%
Z.AI GLM 4.6100100100100100100100100505090.0%
MiniMax M2.5100100100100100100100100100090.0%
Aion 2.0100100100100100100100100100090.0%
Z.AI GLM 4.5100100100100100100100100100090.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100090.0%
Z.AI GLM 4.7 Flash100100100100100100100100100090.0%
Mistral Small 4 (Reasoning)100100100100100100100100100090.0%
ByteDance Seed 1.6 Flash100100100100100100100100100090.0%
Gemma 4 26B10010010010010010010010050085.0%
Z.AI GLM 4.5 Air10010010010010010010010050085.0%
Grok 4.5 (Reasoning, High)1001001001001001001001000080.0%
Z.AI GLM 5.11001001001001001001001000080.0%
GPT-5.6 Terra (Reasoning)1001001001001001001001000080.0%
Claude Opus 4.8 (Reasoning, Low)1001001001001001001001000080.0%
GPT-5.11001001001001001001001000080.0%
MiniMax M31001001001001001001001000080.0%
Qwen 3.5 122B1001001001001001001001000080.0%
Gemini 3 Flash (Preview, Reasoning)1001001001001001001001000080.0%
DeepSeek V4 Pro (Reasoning)1001001001001001001001000080.0%
Qwen 3.6 27B1001001001001001001001000080.0%
Aion 3.0 Mini1001001001001001001001000080.0%
Gemini 2.5 Flash Lite (Reasoning)1001001001001001001001000080.0%
GPT-5.4 Nano1001001001001001005050505080.0%
MoonshotAI: Kimi K2.610010010010010010010000070.0%
Z.AI GLM 4.710010010010010010010000070.0%
Qwen 3.6 35B10010010010010010010000070.0%
Qwen 3.5 35B10010010010010010010000070.0%
ByteDance Seed 2.0 Lite10010010010010010010000070.0%
Qwen 3.5 9B10010010010010010010000070.0%
Xiaomi MIMO v2.510010010010010010010000070.0%
Nemotron 3 Nano10010010010010010010000070.0%
Cohere Command R+ (Aug. 2024)10010010010010010010000070.0%
Qwen 3.5 Flash1001001001001001005000065.0%
Grok 4.31001001001001001005000065.0%
Claude Opus 4.6 (Reasoning)100100100100100100000060.0%
GPT-5100100100100100100000060.0%
MoonshotAI: Kimi K2.5100100100100100100000060.0%
GPT-5.6 Sol100100100100100100000060.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100000060.0%
Qwen 3.5 27B100100100100100100000060.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100000060.0%
ByteDance Seed 2.0 Mini100100100100100100000060.0%
GPT-4.1 Mini100100100100505050500060.0%
Mistral Medium 3.1100100100100505050500060.0%
Cydonia 24B V4.110010010010050505000055.0%
Ministral 3 14B10050505050505050505055.0%
Grok 4.20 (Reasoning)1001001001001000000050.0%
Claude Opus 4.75050505050505050505050.0%
Aion 3.01001001001001000000050.0%
DeepSeek V4 Pro1001001005050505000050.0%
Qwen 2.5 72B1001005050505050500050.0%
GPT-4.1 Nano1001005050505050500050.0%
Gemini 3.5 Flash (Reasoning)10010010010000000040.0%
Qwen 3.5 397B A17B10010010010000000040.0%
Claude Sonnet 5 (Reasoning)10010010010000000040.0%
Qwen 3.6 Flash10010010010000000040.0%
GPT-5.4 Mini10010010010000000040.0%
Qwen 3 32B10010010010000000040.0%
Claude Haiku 4.55050505050505000035.0%
DeepSeek V4 Flash5050505050505000035.0%
WizardLM 2 8x22b1001005050500000035.0%
Ministral 3 3B5050505050505000035.0%
Ministral 3B5050505050505000035.0%
Gemini 3.1 Pro (Preview)100100100000000030.0%
Claude Sonnet 4100100100000000030.0%
Gemini 2.5 Flash Lite100100100000000030.0%
Mistral Small 3.2 24B100100505000000030.0%
Claude Opus 410050505000000025.0%
Qwen 3.5 Plus (2026-02-15)50505050500000025.0%
Qwen3 235B A22B Instruct 250710010050000000025.0%
Mistral Small 450505050500000025.0%
Hermes 3 70B50505050500000025.0%
Claude Sonnet 4.6 (Reasoning)1001000000000020.0%
DeepSeek V3 (2025-03-24)1001000000000020.0%
GPT-4o Mini (temp=1)5050505000000020.0%
Llama 3.1 70B1005050000000020.0%
Claude Sonnet 5505050000000015.0%
Writer: Palmyra X5505050000000015.0%
GPT-5.6 Luna10000000000010.0%
Hermes 3 405B10000000000010.0%
Gemini 3.1 Flash Lite (Preview)500000000005.0%
DeepSeek V3.1500000000005.0%
DeepSeek V3.2500000000005.0%
Gemma 3 12B500000000005.0%
GPT-5.6 Sol (Reasoning)00000000000.0%
GPT-5.5 (Reasoning)00000000000.0%
GPT-5.5 (Reasoning, Low)00000000000.0%
GPT-5.500000000000.0%
GPT-5.6 Terra00000000000.0%
Claude Sonnet 4.500000000000.0%
Gemini 3.1 Flash Lite (Reasoning)00000000000.0%
Gemini 3 Flash (Preview)00000000000.0%
Gemini 3.1 Flash Lite00000000000.0%
GPT-5.400000000000.0%
DeepSeek-V2 Chat00000000000.0%
DeepSeek V3 (2024-12-26)00000000000.0%
GPT-4o, Aug. 6th (temp=0)00000000000.0%
GPT-4o, Aug. 6th (temp=1)00000000000.0%
Grok 4.2000000000000.0%
Gemini 2.5 Flash00000000000.0%
GPT-4o Mini (temp=0)00000000000.0%
Gemma 3 27B00000000000.0%
Gemma 3 4B00000000000.0%
Mistral NeMO00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Z.AI GLM 5.1100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Claude Sonnet 5100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Gemma 4 26B100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100100100.0%
Ministral 3 14B100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Ministral 8B100100100100100100100100100100100.0%
Claude Haiku 4.51001001001001001001001001005095.0%
GPT-4.1 Mini1001001001001001001001001005095.0%
GPT-5.4 Nano (Reasoning, Low)1001001001001001001001001005095.0%
Qwen3.7 Max100100100100100100100100100090.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100090.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100090.0%
Grok 4.20 (Reasoning)100100100100100100100100100090.0%
GPT-5.6 Sol100100100100100100100100100090.0%
GPT-5.1100100100100100100100100100090.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100090.0%
Aion 2.0100100100100100100100100100090.0%
o4 Mini100100100100100100100100100090.0%
MiniMax M2.7100100100100100100100100100090.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100090.0%
Xiaomi MIMO v2.5100100100100100100100100100090.0%
GPT-5 Nano100100100100100100100100100090.0%
Arcee AI: Trinity Mini100100100100100100100100100090.0%
Claude Opus 4.610010010010010010010010050085.0%
GPT-5.4 Nano10010010010010010010050505085.0%
Claude Opus 4.6 (Reasoning)1001001001001001001001000080.0%
GPT-5.4 (Reasoning)1001001001001001001001000080.0%
Claude Opus 4.8 (Reasoning)1001001001001001001001000080.0%
Grok 4.3 (Reasoning)1001001001001001001001000080.0%
Claude Sonnet 5 (Reasoning)1001001001001001001001000080.0%
Qwen 3.5 122B1001001001001001001001000080.0%
Claude Sonnet 5 (Reasoning, Low)1001001001001001001001000080.0%
GPT-5.21001001001001001001001000080.0%
Aion 3.01001001001001001001001000080.0%
Qwen 3.6 27B1001001001001001001001000080.0%
DeepSeek V4 Flash (Reasoning)1001001001001001001001000080.0%
Z.AI GLM 4.71001001001001001001001000080.0%
Z.AI GLM 4.61001001001001001001001000080.0%
Aion 3.0 Mini1001001001001001001005050080.0%
Inception Mercury 21001001001001001001001000080.0%
Z.AI GLM 4.5 Air1001001001001001001001000080.0%
Nemotron 3 Nano1001001001001001001001000080.0%
Cydonia 24B V4.11001001001001001001001000080.0%
ByteDance Seed 1.6 Flash1001001001001001001001000080.0%
Cohere Command R+ (Aug. 2024)1001001001001001001001000080.0%
MiniMax M2.510010010010010010010000070.0%
Z.AI GLM 4.7 Flash10010010010010010010000070.0%
Hermes 3 405B10010010010010010010000070.0%
Mistral Small 4 (Reasoning)10010010010010010010000070.0%
GPT-4.1 Nano1001001005050505050505065.0%
Grok 4.5 (Reasoning, High)100100100100100100000060.0%
Qwen 3.6 Flash100100100100100100000060.0%
Z.AI GLM 4.5100100100100100505000060.0%
Mistral Small 3.2 24B100100100100100100000060.0%
Llama 3.1 70B10010010010050505000055.0%
MoonshotAI: Kimi K2.51001001001001000000050.0%
Gemini 3 Flash (Preview, Reasoning)1001001001001000000050.0%
Gemini 2.5 Pro1001001001001000000050.0%
GPT-4o, Aug. 6th (temp=1)1001001001005050000050.0%
Mistral Medium 3.1100100100505050000045.0%
Ministral 3B505050505050505050045.0%
Claude Sonnet 4.6 (Reasoning)10010010010000000040.0%
GPT-510010010010000000040.0%
Qwen 3.5 397B A17B10010010010000000040.0%
DeepSeek V4 Pro (Reasoning)10010010010000000040.0%
Claude Sonnet 410010010010000000040.0%
Qwen 3.5 Flash10010010010000000040.0%
Qwen 3.5 9B10010010010000000040.0%
GPT-4o, Aug. 6th (temp=0)10010010010000000040.0%
Qwen 3 32B10010010010000000040.0%
Qwen 2.5 72B10050505050505000040.0%
Hermes 3 70B10010010050500000040.0%
Ministral 3 3B50505050505050500040.0%
GPT-4o Mini (temp=1)5050505050505000035.0%
Qwen 3.5 35B100100100000000030.0%
DeepSeek V4 Flash505050505050000030.0%
Grok 4.3100100100000000030.0%
Qwen 3.5 Plus (2026-02-15)50505050500000025.0%
DeepSeek V4 Pro50505050500000025.0%
Mistral Small 450505050500000025.0%
Gemini 3.1 Pro (Preview)1001000000000020.0%
Qwen3.6 Max Preview1001000000000020.0%
Gemini 3.5 Flash (Reasoning)1001000000000020.0%
MoonshotAI: Kimi K2.61001000000000020.0%
Claude Opus 41001000000000020.0%
ByteDance Seed 2.0 Lite1001000000000020.0%
Grok 4.201001000000000020.0%
GPT-5.5 (Reasoning)10000000000010.0%
Qwen 3.6 35B10000000000010.0%
ByteDance Seed 2.0 Mini10000000000010.0%
DeepSeek V3.250500000000010.0%
GPT-5.4 Mini10000000000010.0%
DeepSeek V3 (2025-03-24)10000000000010.0%
Gemini 2.5 Flash Lite10000000000010.0%
Qwen3 235B A22B Instruct 250750500000000010.0%
Gemini 3.1 Flash Lite (Preview)500000000005.0%
WizardLM 2 8x22b500000000005.0%
GPT-5.6 Sol (Reasoning)00000000000.0%
GPT-5.5 (Reasoning, Low)00000000000.0%
GPT-5.500000000000.0%
GPT-5.6 Terra00000000000.0%
Claude Sonnet 4.500000000000.0%
Gemini 3.1 Flash Lite (Reasoning)00000000000.0%
Gemini 3.5 Flash (Reasoning, Minimal)00000000000.0%
Gemini 3 Flash (Preview)00000000000.0%
Gemini 3.1 Flash Lite00000000000.0%
GPT-5.400000000000.0%
DeepSeek-V2 Chat00000000000.0%
DeepSeek V3 (2024-12-26)00000000000.0%
DeepSeek V3.100000000000.0%
Gemini 2.5 Flash00000000000.0%
Writer: Palmyra X500000000000.0%
GPT-4o Mini (temp=0)00000000000.0%
Gemma 3 12B00000000000.0%
Gemma 3 27B00000000000.0%
Gemma 3 4B00000000000.0%
Mistral NeMO00000000000.0%