Structural validity

Test: Codex Violation Detection

Avg. Score
97.8%
Scenarios
8

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Gemini 3.1 Flash Lite (Preview)100.0%$0.00192.2s100%
2Gemini 3.1 Flash Lite100.0%$0.00183.2s100%
3Ministral 3B99.9%$0.00022.9s99%
4GPT-6 Luna100.0%$0.00046.0s99%
5Ministral 3 3B99.9%$0.00053.3s99%
6Gemini 2.5 Flash100.0%$0.00252.8s100%
7GPT-4.1 Mini100.0%$0.00155.8s100%
8Arcee AI: Trinity Mini100.0%$0.000410.3s100%
9Mistral Small 3.2 24B100.0%$0.000610.4s100%
10Claude Haiku 5.5 (Reasoning, Medium)100.0%$0.00158.6s100%
11Claude Haiku 5.5 (Adaptive)100.0%$0.00178.4s100%
12Inception Mercury 2100.0%$0.00304.6s100%
13Gemini 3 Flash (Preview)100.0%$0.00314.5s100%
14Claude Haiku 5.5 (Reasoning, Low)100.0%$0.00168.9s100%
15ByteDance Seed 1.6 Flash100.0%$0.000912.0s100%
16Gemma 3 27B100.0%$0.000513.3s100%
17DeepSeek V4 Flash99.9%$0.00039.3s99%
18Z.AI GLM 5.3 Flash (Reasoning, Low)100.0%$0.000416.9s100%
19Mistral Large 3100.0%$0.003010.2s100%
20GPT-5.6 Luna100.0%$0.00483.9s99%
21DeepSeek V3.2100.0%$0.001317.2s100%
22DeepSeek V3 (2024-12-26)100.0%$0.001915.6s100%
23GPT-6 Luna (Reasoning, Medium)100.0%$0.001018.7s100%
24Grok 4.20100.0%$0.00536.4s100%
25Mistral Small 4 (Reasoning)100.0%$0.002016.3s100%
26GPT-5.4 Mini (Reasoning, Low)100.0%$0.00556.7s100%
27Qwen 2.5 72B99.9%$0.000813.9s98%
28Qwen 3 235B A22B Instruct 2507100.0%$0.000721.1s100%
29Gemini 2.5 Flash Lite (Reasoning)100.0%$0.002017.6s100%
30Gemini 3.5 Flash Lite (Reasoning)99.7%$0.00292.0s96%
31Z.AI GLM 4.5100.0%$0.002618.5s100%
32GPT-6 Luna (Reasoning, High)100.0%$0.001422.5s100%
33GPT-6 Sol100.0%$0.00746.5s100%
34Hermes 3 405B100.0%$0.004417.2s100%
35Writer: Palmyra X5100.0%$0.006212.1s100%
36GPT-5.4 Mini99.7%$0.00333.1s94%
37MiniMax M2.599.9%$0.002025.5s99%
38Gemini 3.5 Flash (Reasoning, Minimal)100.0%$0.0113.6s100%
39Gemini 2.5 Flash (Reasoning)100.0%$0.007912.5s100%
40MiniMax M2.7100.0%$0.002229.4s100%
41Gemini 3.6 Flash (Reasoning, Minimal)100.0%$0.0113.3s100%
42Qwen 3.7 Flash (Reasoning)100.0%$0.000933.3s100%
43GPT-5.6 Terra100.0%$0.0113.8s100%
44Mistral Medium 3.5100.0%$0.0114.5s100%
45Mistral Medium 3.5 (Reasoning)100.0%$0.0114.6s100%
46Z.AI GLM 5 Turbo100.0%$0.007217.1s100%
47GPT-5.6 Luna (Reasoning, Medium)100.0%$0.0109.3s100%
48GPT-6.1 Sol (Reasoning, Medium)100.0%$0.009113.4s100%
49Xiaomi MIMO v2.5100.0%$0.005821.8s100%
50Gemma 4 31B100.0%$0.000937.9s100%
51Gemini 3.5 Flash Lite (Reasoning, Minimal)99.4%$0.00302.1s92%
52Muse Spark 1.1 (Reasoning, Minimal)100.0%$0.01110.5s100%
53DeepSeek V4 Flash (Reasoning)100.0%$0.001040.3s100%
54Mistral Large 2100.0%$0.0128.8s100%
55DeepSeek V4.1 Flash (Reasoning, High)100.0%$0.004630.8s100%
56GPT-5.4100.0%$0.0138.8s100%
57Xiaomi MIMO v2.6 Pro100.0%$0.002340.3s100%
58GPT-4o, Aug. 6th (temp=0)100.0%$0.0155.0s100%
59Gemini 3.7 Flash (Reasoning, Medium)100.0%$0.0139.9s99%
60GPT-6 Sol (Reasoning, Medium)100.0%$0.01313.9s100%
61Gemini 3 Flash (Preview, Reasoning)100.0%$0.01118.0s100%
62Z.AI GLM 5.2 (Reasoning, High)100.0%$0.010027.9s100%
63Muse Glimmer 30B (Reasoning, Medium)100.0%$0.003649.9s100%
64Muse Spark 1.1 (Reasoning, Medium)100.0%$0.01616.1s100%
65Xiaomi MIMO v2.5 Pro100.0%$0.009137.0s100%
66GPT-5.4 (Reasoning, Low)100.0%$0.02013.5s100%
67GPT-5.6 Terra (Reasoning, Medium)100.0%$0.02110.3s100%
68Grok 4.5 (Reasoning, Low)100.0%$0.01329.6s99%
69Z.AI GLM 4.7 Flash100.0%$0.00181.1m100%
70Gemini 3.8 Flash (Reasoning, Medium)100.0%$0.02013.0s100%
71Claude Sonnet 5.5 (Reasoning)100.0%$0.0236.5s100%
72Claude Sonnet 4100.0%$0.0239.0s100%
73Thinking Machines Inkling Small (Reasoning, High)100.0%$0.006655.7s100%
74Muse Spark 1.2 (Reasoning, Medium)100.0%$0.02116.6s100%
75Claude Sonnet 4.5100.0%$0.0248.9s100%
76GPT-5.6 Sol100.0%$0.0266.4s100%
77Claude Sonnet 4.6100.0%$0.0249.9s100%
78ByteDance Seed 1.6100.0%$0.00671.0m100%
79o4 Mini100.0%$0.01928.1s100%
80GPT-5 Mini99.9%$0.009251.7s99%
81Gemini 3.1 Flash Lite (Reasoning)98.9%$0.00183.7s80%
82Ministral 3 14B98.9%$0.00106.3s80%
83ByteDance Seed 2.0 Lite100.0%$0.00671.1m100%
84GPT-5.4 Mini (Reasoning)100.0%$0.02028.6s100%
85Z.AI GLM 4.7100.0%$0.00911.0m100%
86Gemma 3 12B98.7%$0.000312.0s80%
87Mistral Small 498.8%$0.00084.3s78%
88Qwen 3.6 35B100.0%$0.01351.0s100%
89Grok 4.20 (Reasoning)100.0%$0.01545.4s100%
90Mistral Medium 3.198.9%$0.00297.7s80%
91DeepSeek-V2 Chat98.9%$0.002013.1s80%
92Z.AI GLM 5100.0%$0.01356.4s100%
93GPT-5.5100.0%$0.0307.7s100%
94GPT-5.2100.0%$0.02424.2s100%
95MiniMax M3100.0%$0.00461.4m100%
96GPT-4o Mini (temp=1)98.3%$0.00067.4s77%
97Z.AI GLM 4.699.9%$0.00491.3m99%
98GPT-OSS 120B100.0%$0.00201.5m100%
99Mistral Large 4.098.9%$0.002816.9s80%
100Cydonia 24B V4.198.6%$0.001014.4s78%
101Qwen 3.5 35B100.0%$0.01754.8s100%
102Kimi K3 (Reasoning, Low)100.0%$0.02436.5s100%
103Muse Spark 1.3 (Reasoning, Medium)100.0%$0.02243.0s100%
104GPT-4o, Aug. 6th (temp=1)98.9%$0.0113.6s80%
105Qwen 3.5 Flash99.4%$0.00381.0m89%
106Xiaomi MIMO v2.6 Flash98.8%$0.000639.8s80%
107Qwen 3 32B98.2%$0.001027.8s77%
108Qwen 3.5 Plus (2026-02-15)98.9%$0.004134.2s80%
109Z.AI GLM 4.5 Air98.9%$0.002539.8s80%
110Gemini 2.5 Pro100.0%$0.03523.2s100%
111Claude Opus 4.6100.0%$0.04010.2s100%
112Claude Opus 4.5100.0%$0.0419.7s100%
113Ministral 3 8B97.5%$0.00074.8s69%
114Gemini 3.6 Flash (Reasoning)100.0%$0.03916.1s100%
115Qwen 3.5 Plus (2026-04-20)100.0%$0.0151.4m100%
116GPT-5.5 (Reasoning, Low)100.0%$0.04014.2s100%
117Grok 4.5 (Reasoning, High)99.9%$0.02746.7s99%
118Kimi K2.5100.0%$0.0151.5m100%
119Claude Haiku 5.5 (Reasoning, Max)100.0%$0.0131.6m100%
120Qwen 3.6 Flash98.8%$0.01129.9s80%
121DeepSeek V3 (2025-03-24)97.7%$0.001522.9s72%
122GPT-4.197.7%$0.008110.0s72%
123DeepSeek V4 Pro97.8%$0.003125.6s72%
124Qwen 3.5 122B100.0%$0.0261.2m100%
125Z.AI GLM 5.3 Flash (Reasoning, Max)100.0%$0.00192.3m100%
126Z.AI GLM 5.1100.0%$0.0171.6m100%
127o4 Mini High100.0%$0.03351.3s100%
128Qwen 3.6 27B100.0%$0.0221.4m100%
129Grok 4.3 (Reasoning)99.8%$0.0191.4m97%
130Gemini 3.5 Flash (Reasoning)100.0%$0.04518.1s100%
131Qwen 3.8 27B (Reasoning, XHigh)100.0%$0.0251.4m100%
132Thinking Machines Inkling98.9%$0.00771.0m80%
133Claude Opus 5.5 (Reasoning)100.0%$0.05112.7s100%
134GPT-5.1100.0%$0.03752.8s100%
135Laguna XS 2.196.4%$0.000519.0s65%
136Qwen 3.5 27B100.0%$0.0211.7m100%
137GPT-5.4 (Reasoning)100.0%$0.04243.1s100%
138Mistral Large 4.0 (Reasoning)100.0%$0.0171.9m100%
139Mistral NeMO96.0%$0.000713.3s62%
140GPT-5.6 Sol (Reasoning, Medium)100.0%$0.05219.9s100%
141ByteDance Seed 2.0 Mini100.0%$0.00292.7m100%
142Qwen 3.8 Flash (Reasoning)100.0%$0.00602.6m100%
143Gemma 4 31B (Reasoning)100.0%$0.00172.8m100%
144DeepSeek V4 Flash 0731 (Reasoning, High)98.8%$0.00221.5m78%
145Claude Haiku 4.595.5%$0.00785.8s61%
146Gemma 3 4B93.5%$0.000212.5s59%
147Qwen 3.5 9B99.4%$0.00202.4m89%
148Grok 4.394.6%$0.00616.1s59%
149Qwen 3.7 Max100.0%$0.0451.1m100%
150 Kimi K3 (Reasoning, High)100.0%$0.0441.3m100%
151Claude Opus 4.7 (Reasoning)100.0%$0.06711.4s100%
152GPT-4o Mini (temp=0)94.4%$0.000625.0s56%
153Gemma 4 26B (Reasoning)98.8%$0.00222.0m78%
154Aion 2.097.6%$0.00841.3m70%
155DeepSeek V3.194.3%$0.001431.1s56%
156Nemotron 3 Nano100.0%$0.00313.5m100%
157Grok 4.6 (Reasoning, High)100.0%$0.0441.7m100%
158Claude Opus 4.796.9%$0.0527.6s83%
159Ministral 8B91.9%$0.00055.6s47%
160GPT-5.5 (Reasoning, Medium)100.0%$0.07130.3s100%
161Claude Opus 598.6%$0.05210.1s80%
162Nemotron 3 Super97.6%$0.00002.3m72%
163Llama 3.1 70B92.3%$0.002124.3s49%
164Gemini 3.1 Pro (Preview)100.0%$0.06852.3s100%
165Claude Opus 4.6 (Reasoning)100.0%$0.07632.0s100%
166Aion 3.5 Mini (Reasoning, High)98.8%$0.0102.3m78%
167Hermes 3 70B92.2%$0.001329.2s47%
168Claude Sonnet 5 (Reasoning, Low)97.1%$0.04226.3s71%
169Laguna S 2.194.4%$0.00091.4m56%
170Aion 3.098.8%$0.0351.4m78%
171DeepSeek V4 Pro 0813 (Reasoning, High)98.8%$0.0261.9m78%
172Claude Sonnet 4.6 (Reasoning)100.0%$0.07651.3s100%
173GPT-5 Nano96.5%$0.00491.9m64%
174GPT-5100.0%$0.0611.6m100%
175Gemini 2.5 Flash Lite88.8%$0.00052.3s39%
176Claude Sonnet 591.0%$0.0219.7s52%
177Cohere Command R+ (Aug. 2024)91.5%$0.01410.0s45%
178Gemma 4 26B90.0%$0.000625.0s40%
179Qwen 3.6 Max Preview100.0%$0.0502.5m100%
180Hy4 Preview (Reasoning, High)100.0%$0.0363.2m100%
181Claude Opus 5 (Reasoning, Low)98.5%$0.07016.6s80%
182Claude Opus 4.8 (Reasoning, Low)98.4%$0.08020.4s88%
183Grok 4.7 (Reasoning, High)100.0%$0.0572.3m100%
184Claude Sonnet 5 (Reasoning)95.0%$0.04327.5s61%
185Z.AI GLM 5.3 (Reasoning, Max)100.0%$0.0453.1m100%
186Aion 3.5 (Reasoning, High)95.0%$0.0231.2m56%
187Qwen 3.5 397B A17B98.9%$0.0262.9m80%
188Claude Opus 4.8 (Reasoning)97.2%$0.08221.0s84%
189Claude Opus 5 (Reasoning)97.5%$0.07016.7s72%
190Claude Opus 4100.0%$0.11615.8s100%
191DeepSeek V4 Pro (Reasoning)96.4%$0.0142.8m63%
192GPT-5.4 Nano (Reasoning)81.3%$0.003516.6s23%
193Thinking Machines Inkling (Reasoning)100.0%$0.0275.6m100%
194Kimi K2.698.9%$0.0384.1m80%
195Qwen 3.8 Max (Reasoning, XHigh)100.0%$0.0734.5m100%
196WizardLM 2 8x22b61.5%$0.003615.7s11%
197GPT-5.4 Nano53.0%$0.00082.9s4%
198GPT-4.1 Nano42.8%$0.00043.9s16%
199GPT-5.4 Nano (Reasoning, Low)52.1%$0.00136.1s1%
200Aion 3.0 Mini51.0%$0.00911.9m1%
97.80%

Individual Scenarios

matrix

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100100100 – – – – – 100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Kimi K2.5100100100100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-6 Sol100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Claude Opus 5100100100100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Qwen 3.6 Flash100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100 – – – – – – – – 100.0%
Z.AI GLM 4.6100100100100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100100100100100100.0%
Claude Opus 4100100100100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100100100100.0%
Gemma 4 26B100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Medium 3.5100100100100100100100100100100100.0%
GPT-5.4100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100100100100.0%
Qwen 3.5 9B100100100100100100100100100100100.0%
GPT-6 Luna100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Claude Haiku 4.5100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
DeepSeek V4 Pro100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100100100100.0%
Grok 4.20100100100100100100100100100100100.0%
Hermes 3 405B100100100100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100100100100.0%
GPT-5.4 Mini100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
Qwen 3 32B100100100100100100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100100100100100100.0%
Mistral Medium 3.1100100100100100100100100100100100.0%
Gemma 3 27B100100100100100100100100100100100.0%
Mistral Small 4100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
Qwen 2.5 72B100100100100100100100100100100100.0%
Cydonia 24B V4.1100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Ministral 3 3B100100100100100100100100100100100.0%
Mistral NeMO100100100100100100100100100100100.0%
Ministral 8B100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
Mistral Small 3.2 24B1001001001001001001001001009999.9%
Writer: Palmyra X51001001001001001001001001009999.9%
Qwen 3.5 Flash100100100100100100100100999999.8%
Gemini 3.7 Flash (Reasoning, Medium)1001001001001001001001001009799.7%
Grok 4.5 (Reasoning, Low)1001001001001001001001001009799.7%
Gemini 3.5 Flash Lite (Reasoning)1001001001001001001001001009699.6%
WizardLM 2 8x22b1001001001001001001001001009699.6%
MiniMax M2.51001001001001001001001001009699.6%
Grok 4.5 (Reasoning, High)1001001001001001001001001009599.5%
DeepSeek V4 Flash100100100100100100100100989499.2%
Ministral 3B1001001001001001009999989499.0%
Gemma 3 12B100100100100100100100100949398.7%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100976996.6%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100757595.0%
GPT-4o Mini (temp=1)1001001001001001001001001005095.0%
Aion 2.01001001001001001001001001001091.0%
Qwen 3.5 Plus (2026-02-15)1001001001001001001001001001091.0%
Mistral Large 4.01001001001001001001001001001091.0%
DeepSeek V3 (2025-03-24)1001001001001001001001001001091.0%
Laguna XS 2.11001001001001001001001001001091.0%
Ministral 3 14B1001001001001001001001001001091.0%
Thinking Machines Inkling100100100100100100100100981090.8%
DeepSeek V3.110010010010010010010099961090.5%
Claude Opus 4.8 (Reasoning)1001001001001001007575757590.0%
Aion 3.5 (Reasoning, High)100100100100100100100100100090.0%
Claude Opus 4.71001001001001001007575757590.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100100100100090.0%
Hermes 3 70B100100100100100100100100100090.0%
Claude Sonnet 51001001001001001009892901089.0%
Cohere Command R+ (Aug. 2024)100100100100100100100100101082.0%
Grok 4.310010010010010010010098101081.8%
DeepSeek V4 Pro (Reasoning)10010010010010010010010010081.0%
GPT-5.4 Nano (Reasoning)1001001001001001001001000080.0%
Llama 3.1 70B100100100100100100951010071.5%
GPT-5.4 Nano (Reasoning, Low)100100100100100100000060.0%
Gemma 3 4B10010010010050505010101058.0%
Gemini 2.5 Flash Lite1001001001009869000056.7%
GPT-4o Mini (temp=0)100100100100100101010101055.0%
Laguna S 2.1100100100100100101010101055.0%
Aion 3.0 Mini1001001001001000000050.0%
GPT-4.1 Nano1001005050402810100038.8%
GPT-5.4 Nano505050505010000026.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100100 – – – – – – 100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Kimi K2.5100100100100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Qwen 3.6 Flash100100100100100100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100100100100100 – – – – 100.0%
Z.AI GLM 4.6100100100100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4100100100100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
MiniMax M2.5100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
Aion 2.0100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
Qwen 3.5 Flash100100100100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 4.0100100100100100100100100100100100.0%
Mistral Medium 3.5100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
GPT-5.4100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100100100100.0%
Qwen 3.5 9B100100100100100100100100100100100.0%
GPT-6 Luna100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100100100100.0%
DeepSeek V4 Pro100100100100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100100100100.0%
Grok 4.20100100100100100100100100100100100.0%
Hermes 3 405B100100100100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100100100100.0%
GPT-5.4 Mini100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100100100.0%
Laguna S 2.1100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100100100100100100.0%
Grok 4.3100100100100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100100100100.0%
Mistral Medium 3.1100100100100100100100100100100100.0%
Gemma 3 12B100100100100100100100100100100100.0%
Gemma 3 27B100100100100100100100100100100100.0%
Mistral Small 4100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 14B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Ministral 3 3B100100100100100100100100100100100.0%
Mistral NeMO100100100100100100100100100100100.0%
Ministral 8B100100100100100100100100100100100.0%
Ministral 3B100100100100100100100100100100100.0%
GPT-5.6 Luna1001001001001001001001001009999.9%
ByteDance Seed 2.0 Mini1001001001001001001001001009999.9%
Xiaomi MIMO v2.51001001001001001001001001009999.9%
Qwen 3 32B1001001001001001001001001009799.7%
GPT-5 Mini1001001001001001001001001009599.5%
Qwen 2.5 72B1001001001001001001001001009299.2%
Gemini 2.5 Flash Lite100100100100100100100100959599.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100949498.9%
Cydonia 24B V4.1100100100100100100100100939298.5%
Claude Opus 5 (Reasoning)10010010010010010010094949498.3%
Claude Sonnet 510010010010010010010094949498.3%
Claude Opus 51001001001001001009494949497.8%
Claude Opus 4.8 (Reasoning)1001001001001001001001001007597.5%
GPT-4o, Aug. 6th (temp=1)1001001001001001001001001001091.0%
Z.AI GLM 4.5 Air1001001001001001001001001001091.0%
DeepSeek V3 (2025-03-24)1001001001001001001001001001091.0%
GPT-4o Mini (temp=1)1001001001001001001001001001091.0%
Laguna XS 2.11001001001001001001001001001091.0%
Xiaomi MIMO v2.6 Flash100100100100100100100100941090.4%
Aion 3.5 (Reasoning, High)100100100100100100100100100090.0%
Ministral 3 8B100100100100100100100100100090.0%
Cohere Command R+ (Aug. 2024)100100100100100100100100100090.0%
Claude Sonnet 5 (Reasoning)10010010010010010010094941089.9%
Gemma 3 4B10010010010010010010099505089.9%
Llama 3.1 70B1001001001001001001009695089.1%
Hermes 3 70B10010010010010010010010077087.7%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100101082.0%
Claude Haiku 4.5100100100100100100100100101082.0%
DeepSeek V3.1100100100100100100100100101082.0%
GPT-5 Nano10010010010010010010010010081.0%
Aion 3.0 Mini1001001001001000000050.0%
GPT-5.4 Nano (Reasoning, Low)10010010010000000040.0%
GPT-4.1 Nano1005050505040301010039.0%
GPT-5.4 Nano10010050505010000036.0%
Gemma 4 26B100100100000000030.0%
WizardLM 2 8x22b101010100000004.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100100100 – – – – – 100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100100100100100100.0%
Aion 3.5 (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Kimi K2.5100100100100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Qwen 3.6 Flash100100100100100100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100100100 – – – – – – 100.0%
Z.AI GLM 4.6100100100100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100100100100100100.0%
Claude Opus 4100100100100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
MiniMax M2.5100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
Aion 2.0100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
Qwen 3.5 Flash100100100100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100100100100.0%
Gemma 4 26B100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 4.0100100100100100100100100100100100.0%
Mistral Medium 3.5100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
GPT-5.4100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100100100100.0%
Qwen 3.5 9B100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
GPT-6 Luna100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Claude Haiku 4.5100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100100100100.0%
Grok 4.20100100100100100100100100100100100.0%
Hermes 3 405B100100100100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
Laguna S 2.1100100100100100100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
Qwen 3 32B100100100100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100100100100100100.0%
Grok 4.3100100100100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100100100100.0%
Gemma 3 12B100100100100100100100100100100100.0%
Gemma 3 27B100100100100100100100100100100100.0%
Laguna XS 2.1100100100100100100100100100100100.0%
Mistral Small 4100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100100100100.0%
Qwen 2.5 72B100100100100100100100100100100100.0%
Cydonia 24B V4.1100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 14B100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Gemma 3 4B100100100100100100100100100100100.0%
Ministral 3B100100100100100100100100100100100.0%
Ministral 3 3B1001001001001001001001001009899.8%
Llama 3.1 70B1001001001001001001001001009499.4%
Nemotron 3 Super1001001001001001001001001009099.0%
GPT-5.4 Mini1001001001001001001001001007597.5%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100947596.9%
Ministral 8B1001001001001001001001001005095.0%
Claude Opus 51001001001001001001001001001091.0%
DeepSeek V3.11001001001001001001001001001091.0%
DeepSeek V4 Pro1001001001001001001001001001091.0%
Mistral Medium 3.11001001001001001001001001001091.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100901090.0%
Gemma 4 26B (Reasoning)100100100100100100100100100090.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100090.0%
Hermes 3 70B100100100100100100100100100090.0%
Claude Opus 5 (Reasoning)100100100100100100100100101082.0%
Mistral NeMO1001001001001001001001000080.0%
GPT-5.4 Nano10010010010010010010000070.0%
Claude Sonnet 5100100987575757575101069.3%
WizardLM 2 8x22b10010010010063505050101063.3%
Cohere Command R+ (Aug. 2024)100100100100100100000060.0%
Aion 3.0 Mini1001001001001010000042.0%
GPT-5.4 Nano (Reasoning, Low)10010010010000000040.0%
GPT-4.1 Nano10010050505047000039.7%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100 – – – – – – – – 100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100100100100.0%
Aion 3.5 (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Kimi K2.5100100100100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-6 Sol100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Claude Opus 5100100100100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Qwen 3.6 Flash100100100100100100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100100 – – – – – – – 100.0%
Z.AI GLM 4.6100100100100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100100100100100100.0%
Claude Sonnet 5100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100100100100100100.0%
Claude Opus 4100100100100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
MiniMax M2.5100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
Qwen 3.5 Flash100100100100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100100100100.0%
Gemma 4 26B100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 4.0100100100100100100100100100100100.0%
Mistral Medium 3.5100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
GPT-5.4100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100100100100.0%
Qwen 3.5 9B100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
GPT-6 Luna100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Claude Haiku 4.5100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100100100100.0%
DeepSeek V3.1100100100100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100100100100.0%
Grok 4.20100100100100100100100100100100100.0%
Hermes 3 405B100100100100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100100100100.0%
GPT-5.4 Mini100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
Laguna S 2.1100100100100100100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
Qwen 3 32B100100100100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100100100100100100.0%
Grok 4.3100100100100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100100100100.0%
Llama 3.1 70B100100100100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100100100100.0%
Mistral Medium 3.1100100100100100100100100100100100.0%
Gemma 3 12B100100100100100100100100100100100.0%
Gemma 3 27B100100100100100100100100100100100.0%
Laguna XS 2.1100100100100100100100100100100100.0%
Mistral Small 4100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100100100100.0%
Qwen 2.5 72B100100100100100100100100100100100.0%
Cydonia 24B V4.1100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 14B100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Cohere Command R+ (Aug. 2024)100100100100100100100100100100100.0%
Gemma 3 4B100100100100100100100100100100100.0%
Ministral 3 3B100100100100100100100100100100100.0%
Mistral NeMO100100100100100100100100100100100.0%
Ministral 8B100100100100100100100100100100100.0%
Ministral 3B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)1001001001001001001001001001091.0%
DeepSeek V4 Pro1001001001001001001001001001091.0%
Aion 2.0100100100100100100100100100090.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100090.0%
Hermes 3 70B100100100100100100100100100090.0%
GPT-5.4 Nano1001001001001001001001000080.0%
Aion 3.0 Mini1001001001001001001001010072.0%
GPT-4.1 Nano1001001007550505050505067.5%
GPT-5.4 Nano (Reasoning, Low)1001001001001000000050.0%
WizardLM 2 8x22b100505050101010100029.0%

tiers

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100100 – – – – – – 100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Kimi K2.5100100100100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-6 Sol100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Claude Opus 5100100100100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Qwen 3.6 Flash100100100100100100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100 – – – – – – – – 100.0%
Z.AI GLM 4.6100100100100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100100100100100100.0%
Claude Sonnet 5100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100100100100100100.0%
Claude Opus 4100100100100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
MiniMax M2.5100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
Aion 2.0100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
Qwen 3.5 Flash100100100100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100100100100.0%
Gemma 4 26B100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 4.0100100100100100100100100100100100.0%
Mistral Medium 3.5100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
GPT-5.4100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100100100100.0%
Qwen 3.5 9B100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
GPT-6 Luna100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Claude Haiku 4.5100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100100100100.0%
DeepSeek V4 Pro100100100100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100100100100.0%
Grok 4.20100100100100100100100100100100100.0%
Hermes 3 405B100100100100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100100100100.0%
GPT-5.4 Mini100100100100100100100100100100100.0%
Laguna S 2.1100100100100100100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
Qwen 3 32B100100100100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100100100100.0%
Mistral Medium 3.1100100100100100100100100100100100.0%
Gemma 3 12B100100100100100100100100100100100.0%
Gemma 3 27B100100100100100100100100100100100.0%
Laguna XS 2.1100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100100100100.0%
Qwen 2.5 72B100100100100100100100100100100100.0%
Cydonia 24B V4.1100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 14B100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Cohere Command R+ (Aug. 2024)100100100100100100100100100100100.0%
Gemma 3 4B100100100100100100100100100100100.0%
Ministral 3 3B100100100100100100100100100100100.0%
Ministral 8B100100100100100100100100100100100.0%
Ministral 3B100100100100100100100100100100100.0%
Llama 3.1 70B1001001001001001001001001009699.6%
Mistral NeMO1001001001001001001001001008398.3%
Grok 4.31001001001001001001001001005095.0%
Claude Sonnet 5 (Reasoning)1001001001001001001001001001091.0%
DeepSeek V3.11001001001001001001001001001091.0%
GPT-5 Nano1001001001001001001001001001091.0%
Aion 3.5 (Reasoning, High)100100100100100100100100100090.0%
Mistral Small 4100100100100100100100100100090.0%
Hermes 3 70B100100100100100100100100100090.0%
GPT-5.4 Nano (Reasoning)10010010010010010010010090089.0%
GPT-5.4 Nano1001001001001001005000065.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100000060.0%
GPT-4.1 Nano100100505050505000045.0%
Aion 3.0 Mini10010010010000000040.0%
WizardLM 2 8x22b10101010101000006.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100100 – – – – – – 100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Kimi K2.5100100100100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-6 Sol100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Claude Opus 5100100100100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100100100100 – – – – – 100.0%
Z.AI GLM 4.6100100100100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100100100100100100.0%
Claude Sonnet 5100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100100100100100100.0%
Claude Opus 4100100100100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
MiniMax M2.5100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
Aion 2.0100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
Qwen 3.5 Flash100100100100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100100100100.0%
Gemma 4 26B100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 4.0100100100100100100100100100100100.0%
Mistral Medium 3.5100100100100100100100100100100100.0%
GPT-5.4100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100100100100.0%
Qwen 3.5 9B100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
GPT-6 Luna100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Claude Haiku 4.5100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100100100100.0%
DeepSeek V3.1100100100100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100100100100.0%
DeepSeek V4 Pro100100100100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100100100100.0%
Grok 4.20100100100100100100100100100100100.0%
Hermes 3 405B100100100100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100100100100.0%
GPT-5.4 Mini100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
Laguna S 2.1100100100100100100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
Qwen 3 32B100100100100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100100100100.0%
Mistral Medium 3.1100100100100100100100100100100100.0%
Gemma 3 12B100100100100100100100100100100100.0%
Gemma 3 27B100100100100100100100100100100100.0%
Laguna XS 2.1100100100100100100100100100100100.0%
Mistral Small 4100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100100100100.0%
Qwen 2.5 72B100100100100100100100100100100100.0%
Cydonia 24B V4.1100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 14B100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Hermes 3 70B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Cohere Command R+ (Aug. 2024)100100100100100100100100100100100.0%
Gemma 3 4B100100100100100100100100100100100.0%
Mistral NeMO100100100100100100100100100100100.0%
Ministral 8B100100100100100100100100100100100.0%
Ministral 3B100100100100100100100100100100100.0%
Ministral 3 3B1001001001001001001001001009499.4%
Grok 4.3 (Reasoning)1001001001001001001001001008898.8%
Gemini 3.5 Flash Lite (Reasoning, Minimal)1001001001001001001001001008398.3%
Gemini 3.5 Flash Lite (Reasoning)1001001001001001001001001008398.3%
Grok 4.31001001001001001001001001001091.0%
Qwen 3.6 Flash100100100100100100100100961090.6%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100090.0%
Aion 3.5 (Reasoning, High)100100100100100100100100100090.0%
Aion 3.0100100100100100100100100100090.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100090.0%
Llama 3.1 70B100100100100100100100100100090.0%
WizardLM 2 8x22b100100100100100100100100100090.0%
GPT-5.4 Nano (Reasoning, Low)10010010010050101000047.0%
Aion 3.0 Mini10010010010010101000043.0%
GPT-5.4 Nano10010010050100000036.0%
GPT-4.1 Nano10050505000000025.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100 – – – – – – – 100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
Kimi K2.6100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100100100100100100.0%
Aion 3.5 (Reasoning, High)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Kimi K2.5100100100100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Claude Opus 5100100100100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Qwen 3.6 Flash100100100100100100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100100 – – – – – – – 100.0%
Claude Sonnet 4100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100100100100100100.0%
Claude Opus 4100100100100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100100100100.0%
MiniMax M2.5100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
Aion 2.0100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100100100100.0%
Gemma 4 26B100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 4.0100100100100100100100100100100100.0%
Mistral Medium 3.5100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
GPT-5.4100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
GPT-6 Luna100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100100100100.0%
DeepSeek V3.1100100100100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100100100100.0%
DeepSeek V4 Pro100100100100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100100100100.0%
Grok 4.20100100100100100100100100100100100.0%
Hermes 3 405B100100100100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100100100100.0%
GPT-5.4 Mini100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
Laguna S 2.1100100100100100100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100100100100100100.0%
Grok 4.3100100100100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100100100100.0%
Llama 3.1 70B100100100100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100100100100.0%
Mistral Medium 3.1100100100100100100100100100100100.0%
Gemma 3 27B100100100100100100100100100100100.0%
Laguna XS 2.1100100100100100100100100100100100.0%
Mistral Small 4100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100100100100.0%
Qwen 2.5 72B100100100100100100100100100100100.0%
WizardLM 2 8x22b100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 14B100100100100100100100100100100100.0%
Ministral 3 8B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Cohere Command R+ (Aug. 2024)100100100100100100100100100100100.0%
Ministral 3 3B100100100100100100100100100100100.0%
Mistral NeMO100100100100100100100100100100100.0%
Ministral 3B100100100100100100100100100100100.0%
Gemma 3 4B1001001001001001001001001009799.7%
Z.AI GLM 4.61001001001001001001001001009599.5%
Claude Opus 5 (Reasoning)1001001001001001001001001009499.4%
Claude Opus 5 (Reasoning, Low)100100100100100100100100949498.9%
Claude Sonnet 5 (Reasoning, Low)1001001001001001009595959598.0%
Claude Opus 4.8 (Reasoning, Low)1001001001001001001001001007597.5%
Qwen 3.5 Flash1001001001001001001001001005095.0%
Qwen 3.5 9B1001001001001001001001001005095.0%
Qwen 3 32B1001001001001001001001001005095.0%
Qwen 3.5 397B A17B1001001001001001001001001001091.0%
DeepSeek-V2 Chat1001001001001001001001001001091.0%
Claude Haiku 4.51001001001001001001001001001091.0%
Gemma 3 12B1001001001001001001001001001091.0%
Cydonia 24B V4.1100100100100100100100100100090.0%
Hermes 3 70B100100100100100100100100100090.0%
Ministral 8B100100100100100100100100100090.0%
Nemotron 3 Super100100100100100100100100101082.0%
GPT-4.110010010010010010010093101081.3%
Gemini 2.5 Flash Lite1001001001001001001001000080.0%
Claude Sonnet 5 (Reasoning)1001001009595959595101079.5%
Claude Sonnet 5100100100100100949310101071.7%
Aion 3.0 Mini100100100100100100100100071.0%
GPT-5.4 Nano (Reasoning)100100100100100100000060.0%
GPT-5.4 Nano (Reasoning, Low)1001001001001000000050.0%
GPT-5.4 Nano1001001001001000000050.0%
GPT-4.1 Nano1001005050505010100042.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100100100 – – – – – 100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100100100100.0%
Aion 3.5 (Reasoning, High)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Kimi K2.5100100100100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
MiniMax M3100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100100100100.0%
GPT-6 Sol100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Claude Opus 5100100100100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Qwen 3.6 Flash100100100100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
Aion 3.0100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100 – – – – – – – – – 100.0%
Z.AI GLM 4.6100100100100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100100100100100100.0%
Claude Opus 4100100100100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100100100100.0%
GPT-4.1100100100100100100100100100100100.0%
MiniMax M2.5100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
Aion 2.0100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100100100100100100.0%
Qwen 3.5 Flash100100100100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100100100100.0%
Mistral Large 4.0100100100100100100100100100100100.0%
Mistral Medium 3.5100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
GPT-5.4100100100100100100100100100100100.0%
Mistral Large 3100100100100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100100100100.0%
Qwen 3.5 9B100100100100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100100100100.0%
DeepSeek V3.1100100100100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100100100100.0%
DeepSeek V4 Pro100100100100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Mistral Large 2100100100100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100100100100.0%
Grok 4.20100100100100100100100100100100100.0%
Hermes 3 405B100100100100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100100100100.0%
GPT-5.4 Mini100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
Laguna S 2.1100100100100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100100100100.0%
Mistral Medium 3.1100100100100100100100100100100100.0%
Gemma 3 12B100100100100100100100100100100100.0%
Gemma 3 27B100100100100100100100100100100100.0%
Mistral Small 4100100100100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100100100100.0%
Qwen 2.5 72B100100100100100100100100100100100.0%
Cydonia 24B V4.1100100100100100100100100100100100.0%
WizardLM 2 8x22b100100100100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100100100100.0%
Ministral 3 14B100100100100100100100100100100100.0%
Hermes 3 70B100100100100100100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100100100100100100.0%
Cohere Command R+ (Aug. 2024)100100100100100100100100100100100.0%
Gemma 3 4B100100100100100100100100100100100.0%
Ministral 3 3B100100100100100100100100100100100.0%
Ministral 3B100100100100100100100100100100100.0%
GPT-5.6 Luna1001001001001001001001001009899.8%
GPT-6 Luna1001001001001001001001001009899.8%
DeepSeek V3 (2025-03-24)1001001001001001001001001009899.8%
Claude Sonnet 51001001001001001001001001009599.5%
Claude Opus 4.8 (Reasoning, Low)10010010010010010010098757594.8%
Kimi K2.61001001001001001001001001001091.0%
Claude Haiku 4.51001001001001001001001001001091.0%
Qwen 3 32B1001001001001001001001001001091.0%
Claude Opus 4.8 (Reasoning)1001001001001001007575757590.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100090.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100100090.0%
Gemma 4 26B100100100100100100100100100090.0%
Ministral 3 8B100100100100100100100100100090.0%
Mistral NeMO100100100100100100100100100090.0%
Laguna XS 2.11001001001001001001009897089.4%
Llama 3.1 70B10010010010010010010010091089.1%
Grok 4.3100100100100100100100100801089.0%
Claude Opus 4.710010010010075757575757585.0%
Gemini 2.5 Flash Lite10010010010010010098480074.6%
GPT-5.4 Nano (Reasoning, Low)10010010010010010010000070.0%
GPT-5.4 Nano1001001001001001001000061.0%
GPT-5.4 Nano (Reasoning)10010010010010010000051.0%
Ministral 8B1001001001001000000050.0%
GPT-4.1 Nano100505050505050500045.0%
Aion 3.0 Mini10010010010000000040.0%