Past progressive (was/were + -ing) overuse

Test: Bad Writing Habits

Avg. Score
87.8%
Scenarios
18

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1GPT-4.1 Mini99.9%$0.002719.0s99%
2Inception Mercury 299.7%$0.00327.0s96%
3GPT-5.6 Luna100.0%$0.01515.8s100%
4GPT-5.6 Luna (Reasoning)100.0%$0.01718.0s100%
5o4 Mini99.8%$0.01525.7s97%
6GPT-5.4 Nano99.2%$0.005726.3s93%
7GPT-4o, Aug. 6th (temp=1)99.7%$0.01824.4s95%
8GPT-5.4 Nano (Reasoning, Low)99.0%$0.005520.6s90%
9o4 Mini High100.0%$0.02547.2s100%
10GPT-4.199.8%$0.01844.7s96%
11GPT-5 Mini99.7%$0.010057.4s96%
12GPT-5.4 Mini (Reasoning, Low)98.2%$0.01516.8s88%
13Grok 4.5 (Reasoning, Low)99.9%$0.0181.1m97%
14GPT-5.6 Terra (Reasoning)100.0%$0.04435.2s100%
15GPT-5 Nano99.6%$0.00421.4m95%
16GPT-5.6 Terra100.0%$0.04636.9s100%
17GPT-5.4 Mini (Reasoning)98.6%$0.02228.1s88%
18Grok 4.397.6%$0.006930.5s80%
19GPT-4o Mini (temp=1)97.7%$0.001234.8s77%
20GPT-4o Mini (temp=0)97.9%$0.001234.8s76%
21Grok 4.5 (Reasoning, High)99.9%$0.0301.6m98%
22GPT-OSS 120B99.3%$0.00151.8m90%
23Nemotron 3 Nano98.1%$0.00101.1m81%
24GPT-5.4 Nano (Reasoning)96.8%$0.006124.5s76%
25Nemotron 3 Super98.5%$0.00001.4m83%
26GPT-5.4100.0%$0.0491.4m100%
27Grok 4.3 (Reasoning)99.9%$0.0212.3m99%
28Grok 4.20 (Reasoning)98.4%$0.0181.5m88%
29Gemini 3.5 Flash Lite (Reasoning)94.8%$0.00356.6s67%
30GPT-5.2100.0%$0.0561.5m100%
31GPT-4o, Aug. 6th (temp=0)97.0%$0.02322.7s73%
32GPT-5.4 Mini96.2%$0.01516.8s68%
33GPT-5.1100.0%$0.0541.8m99%
34Mistral Small 4 (Reasoning)92.9%$0.002230.2s66%
35ByteDance Seed 1.6 Flash93.9%$0.001327.3s61%
36DeepSeek V3 (2025-03-24)94.5%$0.001439.4s63%
37Qwen 3.5 Plus (2026-02-15)94.4%$0.006031.5s62%
38Gemini 3.5 Flash Lite (Reasoning, Minimal)92.9%$0.00356.3s58%
39GPT-5.4 (Reasoning, Low)99.0%$0.0551.4m87%
40Writer: Palmyra X594.3%$0.01122.0s62%
41Qwen3 235B A22B Instruct 250794.8%$0.001159.2s65%
42Grok 4.2094.3%$0.009345.7s66%
43GPT-5.6 Sol100.0%$0.1051.2m100%
44Qwen 2.5 72B91.6%$0.001036.7s62%
45Qwen 3.6 Flash93.6%$0.01041.4s63%
46GPT-5.6 Sol (Reasoning)100.0%$0.1081.2m100%
47Hermes 3 405B93.3%$0.003253.2s62%
48DeepSeek-V2 Chat92.9%$0.002153.3s61%
49Gemini 2.5 Flash (Reasoning)92.8%$0.01121.5s57%
50Gemini 2.5 Flash Lite91.3%$0.00099.5s52%
51Gemini 2.5 Flash92.0%$0.005210.6s53%
52Qwen 3.6 35B94.0%$0.00831.0m62%
53GPT-5100.0%$0.0652.8m100%
54Arcee AI: Trinity Mini90.5%$0.00039.2s51%
55DeepSeek V3 (2024-12-26)92.6%$0.002154.6s58%
56Hermes 3 70B93.9%$0.00101.2m58%
57Z.AI GLM 4.5 Air92.8%$0.002958.2s57%
58Z.AI GLM 4.591.5%$0.005142.1s56%
59Thinking Machines Inkling94.3%$0.00751.5m61%
60Mistral Small 490.1%$0.001418.2s49%
61Cohere Command R+ (Aug. 2024)93.1%$0.02052.5s58%
62Claude Sonnet 4.592.9%$0.03538.1s60%
63Gemini 2.5 Flash Lite (Reasoning)89.7%$0.002830.8s49%
64Claude Opus 4.8 (Reasoning, Low)94.7%$0.07141.9s69%
65Mistral Medium 3.188.6%$0.004836.5s50%
66Gemini 3.6 Flash (Reasoning, Minimal)89.5%$0.01714.3s47%
67Mistral NeMO86.3%$0.000510.1s45%
68GPT-5.4 (Reasoning)99.4%$0.0892.6m93%
69GPT-5.5100.0%$0.1391.7m100%
70Qwen 3.6 27B95.4%$0.0252.3m67%
71DeepSeek V4 Flash86.5%$0.000631.6s48%
72GPT-5.5 (Reasoning, Low)100.0%$0.1391.8m100%
73GPT-5.5 (Reasoning)100.0%$0.1421.8m100%
74Qwen 3 32B89.1%$0.001554.6s45%
75MoonshotAI: Kimi K2.595.7%$0.0193.2m73%
76Cydonia 24B V4.187.1%$0.001444.8s46%
77ByteDance Seed 2.0 Lite92.8%$0.0122.2m59%
78ByteDance Seed 1.694.0%$0.0132.5m60%
79Claude Sonnet 489.6%$0.03243.7s52%
80Muse Spark 1.1 (Reasoning, Medium)87.6%$0.01523.9s44%
81GPT-4.1 Nano84.8%$0.000713.3s39%
82Mistral Large 385.0%$0.003330.3s42%
83Gemini 3.6 Flash (Reasoning)90.8%$0.04432.5s50%
84Aion 3.0 Mini87.6%$0.00531.2m48%
85Mistral Large 285.7%$0.01329.4s45%
86Gemma 3 4B82.4%$0.000220.0s41%
87Qwen 3.5 9B88.8%$0.00111.4m43%
88Muse Spark 1.1 (Reasoning, Minimal)85.3%$0.01219.2s39%
89Llama 3.1 70B84.5%$0.001529.4s38%
90WizardLM 2 8x22b89.1%$0.00261.8m48%
91Qwen 3.5 Flash85.9%$0.002547.5s39%
92Qwen 3.5 Plus (2026-04-20)91.7%$0.0171.8m48%
93Qwen3.6 Max Preview96.7%$0.0503.5m76%
94Gemini 3.5 Flash (Reasoning, Minimal)84.1%$0.01812.0s37%
95Gemma 3 12B83.0%$0.000441.3s36%
96Thinking Machines Inkling (Reasoning)98.1%$0.0255.1m82%
97Aion 2.084.8%$0.00641.3m43%
98Claude Sonnet 5 (Reasoning, Low)85.0%$0.03138.4s43%
99DeepSeek V4 Flash (Reasoning)81.7%$0.000731.1s34%
100MoonshotAI: Kimi K3 (Reasoning, High)90.2%$0.0441.8m55%
101Claude Opus 4.8 (Reasoning)89.0%$0.07141.7s53%
102Qwen 3.5 397B A17B91.3%$0.0143.0m56%
103Xiaomi MIMO v2.5 Pro84.0%$0.008553.5s36%
104MiniMax M2.783.8%$0.00401.1m38%
105Claude Sonnet 583.8%$0.02733.5s39%
106Laguna XS 2.179.2%$0.000314.4s29%
107Z.AI GLM 5 Turbo79.1%$0.008133.2s34%
108Aion 3.083.5%$0.0241.0m38%
109Gemini 3.1 Flash Lite (Preview)75.4%$0.00308.4s31%
110Laguna S 2.177.0%$0.000522.1s30%
111Qwen 3.5 122B85.5%$0.0251.1m34%
112Ministral 3 3B77.7%$0.000511.1s24%
113Z.AI GLM 5.183.6%$0.0141.5m38%
114Gemini 3.1 Flash Lite75.0%$0.003012.1s28%
115DeepSeek V3.281.5%$0.00141.9m39%
116Z.AI GLM 5.2 (Reasoning, High)80.3%$0.0111.0m32%
117Z.AI GLM 581.3%$0.00841.2m31%
118MoonshotAI: Kimi K2.699.4%$0.0586.5m93%
119DeepSeek V4 Pro (Reasoning)88.4%$0.0153.1m45%
120Claude Opus 4.584.5%$0.07053.4s45%
121Gemini 3.1 Pro (Preview)92.7%$0.1071.8m57%
122Gemini 3.1 Flash Lite (Reasoning)74.8%$0.003011.9s22%
123Claude Sonnet 5 (Reasoning)78.9%$0.03038.9s32%
124Qwen 3.5 35B78.9%$0.0181.0m31%
125Xiaomi MIMO v2.574.4%$0.005431.8s26%
126DeepSeek V4 Pro76.6%$0.00481.3m32%
127ByteDance Seed 2.0 Mini90.9%$0.00454.9m56%
128Claude Haiku 4.573.6%$0.01121.6s26%
129Gemini 3.5 Flash (Reasoning)84.9%$0.07137.6s35%
130Claude Opus 4.685.4%$0.0781.2m44%
131Gemma 3 27B74.1%$0.000652.6s25%
132Claude Opus 5 (Reasoning, Low)86.4%$0.0971.0m46%
133Claude Opus 5 (Reasoning)85.2%$0.0981.0m48%
134Gemini 2.5 Pro75.2%$0.03636.2s34%
135Ministral 8B69.4%$0.000410.4s23%
136Gemini 3 Flash (Preview, Reasoning)72.7%$0.01230.1s26%
137MoonshotAI: Kimi K3 (Reasoning, Low)80.2%$0.0281.1m27%
138Ministral 3 14B69.3%$0.000711.7s21%
139MiniMax M2.573.6%$0.00341.3m27%
140Qwen 3.5 27B80.2%$0.0201.6m26%
141Claude Opus 4.6 (Reasoning)84.8%$0.0881.4m39%
142Ministral 3 8B68.2%$0.000819.6s18%
143Gemma 4 26B71.2%$0.000955.1s21%
144Ministral 3B66.8%$0.00018.1s16%
145Gemma 4 31B (Reasoning)76.6%$0.00142.2m24%
146Claude Opus 578.8%$0.08250.1s35%
147Gemma 4 31B72.3%$0.00101.6m21%
148Qwen3.7 Max85.3%$0.0682.3m34%
149Gemma 4 26B (Reasoning)73.3%$0.00132.0m22%
150DeepSeek V3.169.5%$0.00201.8m24%
151Gemini 3 Flash (Preview)63.0%$0.007819.6s16%
152MiniMax M375.6%$0.00603.1m29%
153Claude Sonnet 4.667.1%$0.03139.3s18%
154Claude Opus 491.7%$0.2091.4m55%
155Z.AI GLM 4.661.6%$0.006551.5s14%
156Claude Sonnet 4.6 (Reasoning)69.4%$0.0601.2m20%
157Claude Opus 4.766.2%$0.06930.4s19%
158Z.AI GLM 4.760.9%$0.0101.4m12%
159Mistral Small 3.2 24B80.5%$0.00695.7m30%
160Claude Opus 4.7 (Reasoning)62.2%$0.07632.0s12%
161Z.AI GLM 4.7 Flash51.6%$0.00171.2m8%
87.82%

Individual Scenarios

Detailed Writing Rules

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 5100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.2100100100100100100.0%
GPT-5.5100100100100100100.0%
Aion 3.0100100100100100100.0%
o4 Mini High100100100100100100.0%
Gemini 2.5 Pro100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Opus 4100100100100100100.0%
GPT-4.1100100100100100100.0%
MiniMax M2.5100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
Gemma 4 31B100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
Claude Haiku 4.5100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
DeepSeek V4 Pro100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Nemotron 3 Super100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100.0%
Writer: Palmyra X5100100100100100100.0%
Qwen3 235B A22B Instruct 2507100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
Llama 3.1 70B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Gemma 3 12B100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
Cydonia 24B V4.1100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Mistral NeMO100100100100100100.0%
ByteDance Seed 2.0 Lite1001001001009899.7%
Qwen 3.5 35B1001001001009899.6%
Mistral Small 41001001001009899.6%
MoonshotAI: Kimi K3 (Reasoning, High)1001001001009799.5%
MiniMax M2.71001001001009699.3%
Xiaomi MIMO v2.5 Pro1001001001009699.3%
Claude Opus 4.8 (Reasoning)1001001001009599.0%
DeepSeek V3.11001001001009498.8%
Mistral Medium 3.11001001001009498.8%
Qwen 3.5 9B1001001001009398.7%
Aion 2.01001001001008997.8%
Qwen 3.6 27B1001001001008697.2%
Gemma 4 31B (Reasoning)1001001001008396.6%
Claude Sonnet 51001001001008296.4%
ByteDance Seed 1.6 Flash1001001001008296.4%
Claude Opus 5 (Reasoning, Low)1001001001008096.0%
GPT-4.1 Nano1001001001007895.6%
DeepSeek V4 Pro (Reasoning)1001001001007895.5%
Claude Sonnet 41001001001007394.5%
Cohere Command R+ (Aug. 2024)1001001001007194.2%
Claude Sonnet 4.61001001001007194.2%
Claude Sonnet 4.6 (Reasoning)1001001001007094.0%
Gemini 2.5 Flash (Reasoning)1001001001007094.0%
Mistral Large 21001001001007094.0%
Qwen 3.5 27B1001001001006993.7%
Gemma 3 27B100100100867792.7%
Gemini 3.5 Flash (Reasoning, Minimal)1001001001006392.6%
Qwen 3.6 Flash1001001001006192.2%
Claude Sonnet 4.5100100100827992.2%
Mistral Large 3100100100797891.3%
DeepSeek V3.21001001001005090.0%
Claude Sonnet 5 (Reasoning)1001001001005089.9%
Z.AI GLM 4.61001001001004889.6%
Qwen 2.5 72B100100100747389.4%
Qwen 3.5 Flash1001001001004589.0%
Gemma 3 4B1001001001004589.0%
Hermes 3 405B1001001001003386.7%
MiniMax M310010081816685.5%
Gemini 3 Flash (Preview, Reasoning)1001001001002785.5%
WizardLM 2 8x22b1001001001001282.4%
Gemini 3.1 Flash Lite (Preview)100100100100681.2%
Arcee AI: Trinity Mini100100100100080.0%
Laguna S 2.110010010073275.0%
Z.AI GLM 4.7100100100443174.9%
Mistral Small 3.2 24B10010086444274.5%
Qwen 3 32B10010010068073.6%
Xiaomi MIMO v2.51001009276073.6%
Laguna XS 2.110010010056071.2%
Gemini 3.1 Flash Lite (Reasoning)10010084492070.6%
Ministral 3B10010010050070.0%
Ministral 3 8B10010010028867.1%
Gemini 3.1 Flash Lite10010059412164.2%
Gemma 4 26B1001008136063.3%
Ministral 3 14B1001001002060.4%
Gemma 4 26B (Reasoning)100100570051.4%
Z.AI GLM 4.7 Flash100100295046.8%
Gemini 3 Flash (Preview)93842120043.5%
Ministral 3 3B1008900037.8%
Ministral 8B6352505034.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
GPT-5100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
GPT-5.2100100100100100100.0%
GPT-5.5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
o4 Mini High100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
GPT-4.1100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Qwen 3.5 Flash100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Nemotron 3 Super100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
Grok 4.20100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100.0%
Qwen 3 32B100100100100100100.0%
Qwen3 235B A22B Instruct 2507100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Mistral Medium 3.1100100100100100100.0%
Gemma 3 12B100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Mistral Small 410010010010010099.9%
Claude Opus 4.6 (Reasoning)1001001001009899.6%
Z.AI GLM 51001001001009899.6%
ByteDance Seed 2.0 Lite1001001001009599.1%
Z.AI GLM 5.11001001001009598.9%
Claude Sonnet 51001001001009498.8%
Qwen 3.6 35B1001001001009398.7%
DeepSeek V3 (2024-12-26)1001001001009398.6%
Claude Opus 5 (Reasoning, Low)100100100999498.5%
Claude Opus 41001001001009198.1%
Cohere Command R+ (Aug. 2024)1001001001008897.6%
MiniMax M2.71001001001008697.3%
Claude Opus 4.8 (Reasoning, Low)1001001001008697.1%
Claude Opus 4.61001001001008596.9%
ByteDance Seed 1.61001001001008296.5%
Mistral Large 3100100100938996.4%
Gemini 2.5 Flash Lite (Reasoning)100100100938996.4%
GPT-OSS 120B1001001001008095.9%
Claude Opus 4.5100100100958395.7%
Z.AI GLM 4.5 Air1001001001007895.6%
GPT-4o, Aug. 6th (temp=1)1001001001007795.5%
Claude Sonnet 4.6 (Reasoning)1001001001007795.3%
MoonshotAI: Kimi K3 (Reasoning, Low)1001001001007695.2%
Aion 3.01001001001007394.7%
GPT-4o Mini (temp=1)1001001001007294.5%
DeepSeek V4 Pro (Reasoning)100100100947694.0%
DeepSeek V3 (2025-03-24)1001001001006993.9%
GPT-5 Nano10010099898093.5%
DeepSeek V4 Flash1001001001006192.2%
Ministral 3 3B100100100946792.2%
ByteDance Seed 2.0 Mini1001001001006091.9%
Mistral Small 3.2 24B1001001001005991.8%
Qwen 3.5 35B100100100965890.8%
Z.AI GLM 4.5100100100806989.9%
Claude Opus 5 (Reasoning)100100100806789.2%
Claude Opus 4.8 (Reasoning)1001001001004188.3%
Ministral 3B100100100894887.5%
Gemini 3 Flash (Preview, Reasoning)1001001001003787.3%
Claude Opus 5100100100834986.4%
Gemini 3.5 Flash (Reasoning)1001001001003186.2%
Claude Sonnet 4.61001001001002584.9%
Claude Sonnet 5 (Reasoning, Low)10010082786184.3%
Z.AI GLM 5.2 (Reasoning, High)100100100645483.6%
Laguna XS 2.11001001001001883.6%
GPT-4o, Aug. 6th (temp=0)1001001001001783.5%
Z.AI GLM 5 Turbo100100100912583.1%
Aion 2.010010097615382.1%
Cydonia 24B V4.1100100100634481.3%
MiniMax M3100100100100581.0%
Gemini 2.5 Pro10010092594980.0%
Gemma 4 26B (Reasoning)100100100100080.0%
Claude Sonnet 4.5100100100100080.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100080.0%
Ministral 3 14B100100100100080.0%
GPT-4.1 Nano100100100643479.6%
Arcee AI: Trinity Mini100100100554179.3%
Llama 3.1 70B10010086544877.7%
Mistral Large 210010010088077.6%
Gemma 4 31B (Reasoning)10010080683877.3%
Claude Sonnet 5 (Reasoning)10010010080075.9%
Gemma 3 4B10010092562374.2%
DeepSeek V4 Flash (Reasoning)10010010071074.1%
WizardLM 2 8x22b100100100482173.8%
Qwen 2.5 72B10010069662572.2%
Gemini 3.5 Flash (Reasoning, Minimal)10010010056071.2%
DeepSeek V4 Pro10010073612070.7%
DeepSeek V3.21001009748068.9%
Xiaomi MIMO v2.510010010037868.9%
Hermes 3 405B10010010037067.5%
MiniMax M2.510010064343266.1%
Gemini 3.1 Flash Lite1001006664065.9%
Writer: Palmyra X51001008041064.3%
Claude Sonnet 41001007246063.6%
Laguna S 2.110010010015063.0%
Gemini 3 Flash (Preview)1001006933060.4%
Z.AI GLM 4.71001001000060.0%
Gemini 2.5 Flash Lite100100800056.0%
Mistral NeMO1006664262155.3%
Claude Opus 4.7 (Reasoning)100826131054.9%
Z.AI GLM 4.6100746630054.1%
Gemma 4 26B10010032121050.9%
Ministral 8B10080710050.1%
Claude Opus 4.7100884913049.9%
Gemma 3 27B73736914045.8%
DeepSeek V3.110068330040.2%
Gemma 4 31B10010000040.0%
Claude Haiku 4.510088101039.5%
Gemini 3.1 Flash Lite (Preview)804623221437.2%
Gemini 3.1 Flash Lite (Reasoning)1007900035.8%
Ministral 3 8B10040284034.4%
Z.AI GLM 4.7 Flash10037290033.3%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.2100100100100100100.0%
GPT-5.5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
o4 Mini High100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
Z.AI GLM 4.6100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Opus 4100100100100100100.0%
GPT-4.1100100100100100100.0%
MiniMax M2.5100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
Mistral Large 3100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100.0%
Qwen 3 32B100100100100100100.0%
Writer: Palmyra X5100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Qwen 2.5 72B100100100100100100.0%
Cydonia 24B V4.1100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
WizardLM 2 8x22b100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100.0%
GPT-4.1 Nano100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Gemma 3 4B100100100100100100.0%
Ministral 3 3B100100100100100100.0%
Mistral NeMO100100100100100100.0%
Mistral Large 21001001001009899.6%
Aion 2.01001001001009598.9%
Z.AI GLM 4.5100100100979698.5%
Hermes 3 405B1001001001009298.5%
Laguna S 2.11001001001009298.5%
Qwen 3.5 35B1001001001009098.0%
Gemini 2.5 Flash Lite (Reasoning)1001001001008997.8%
Gemma 4 31B1001001001008897.6%
Xiaomi MIMO v2.5 Pro1001001001008797.4%
Mistral Medium 3.11001001001008797.4%
DeepSeek V4 Flash1001001001008697.2%
Ministral 3B1001001001008496.8%
Gemini 2.5 Flash1001001001008096.0%
Qwen 3.6 35B1001001001007995.8%
Claude Opus 4.8 (Reasoning)1001001001007895.5%
Mistral Small 41001001001007795.3%
Ministral 3 14B1001001001006793.5%
Gemma 3 12B100100100976592.3%
Qwen 3.5 Flash1001001001006192.1%
Thinking Machines Inkling100100100976291.9%
Claude Sonnet 5 (Reasoning, Low)10010098976191.4%
Z.AI GLM 51001001001005390.7%
Z.AI GLM 5.2 (Reasoning, High)1001001001005190.1%
Gemma 4 26B (Reasoning)100100100896089.7%
Aion 3.01001001001004589.0%
Aion 3.0 Mini1001001001004589.0%
Gemini 3.1 Flash Lite (Preview)1001001001004488.8%
DeepSeek V4 Pro100100100915188.3%
DeepSeek V3.2100100100855688.2%
Gemini 3 Flash (Preview, Reasoning)100100100964588.2%
Nemotron 3 Super100100100974087.4%
Mistral Small 4 (Reasoning)1001001001003687.2%
Gemma 3 27B10010084757386.3%
Laguna XS 2.11001001001002985.8%
Claude Opus 5 (Reasoning)100100100863784.7%
Qwen3 235B A22B Instruct 25071001001001002184.3%
Gemini 3.1 Flash Lite (Reasoning)1001001001001983.7%
MiniMax M2.71001001001001583.0%
Claude Sonnet 4.510010092774582.9%
Claude Opus 5100100100971682.5%
Gemini 2.5 Pro10010096803682.4%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100080.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100080.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100080.0%
Llama 3.1 70B100100100100080.0%
Cohere Command R+ (Aug. 2024)100100100100080.0%
Arcee AI: Trinity Mini10010010093078.7%
Claude Sonnet 5 (Reasoning)10010010089077.8%
MiniMax M3100100100532275.0%
Gemini 3 Flash (Preview)1001008969071.5%
Xiaomi MIMO v2.5100100100411270.6%
Ministral 3 8B10010072502870.2%
Claude Haiku 4.510010010049069.9%
Z.AI GLM 5 Turbo100878675069.6%
Gemma 4 26B100998657068.4%
Claude Opus 5 (Reasoning, Low)10010088291766.9%
DeepSeek V4 Flash (Reasoning)10010010029065.8%
Claude Sonnet 510010079262165.1%
Ministral 8B1001009132064.7%
Gemini 3.1 Flash Lite10010010010061.9%
Claude Sonnet 4.6 (Reasoning)1001008029061.7%
Claude Opus 4.71001001000060.0%
DeepSeek V3.11001007015056.9%
Z.AI GLM 4.795824720048.7%
Z.AI GLM 4.7 Flash10071480043.7%
Claude Opus 4.7 (Reasoning)908600035.2%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
GPT-5.2100100100100100100.0%
GPT-5.5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
o4 Mini High100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Nemotron 3 Super100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Writer: Palmyra X5100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
Mistral Medium 3.1100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
WizardLM 2 8x22b100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)10010010010010099.9%
Xiaomi MIMO v2.5 Pro1001001001009799.4%
Claude Opus 4.61001001001009699.3%
Claude Opus 4.7 (Reasoning)1001001001009599.0%
MiniMax M31001001001009298.5%
Llama 3.1 70B1001001001009298.5%
Cohere Command R+ (Aug. 2024)1001001001009298.5%
Muse Spark 1.1 (Reasoning, Minimal)1001001001009097.9%
Qwen3.7 Max1001001001008897.6%
Qwen 3 32B100100100968496.1%
Gemma 4 31B (Reasoning)1001001001007795.3%
MoonshotAI: Kimi K3 (Reasoning, High)1001001001007294.4%
ByteDance Seed 1.61001001001006893.7%
DeepSeek V4 Flash (Reasoning)1001001001006893.6%
Qwen 3.6 27B1001001001006693.2%
Claude Opus 4.8 (Reasoning, Low)1001001001006693.2%
Mistral Large 3100100100868093.0%
Gemini 2.5 Flash Lite1001001001006492.8%
Claude Opus 51001001001005891.6%
Z.AI GLM 4.6100100100827691.5%
Gemma 3 4B100100100807691.1%
Gemini 3.5 Flash Lite (Reasoning, Minimal)1001001001005090.0%
Qwen 3.5 Flash100100100757489.8%
Qwen 3.5 35B1001001001004989.8%
Z.AI GLM 5.11001001001004789.5%
Gemini 2.5 Flash (Reasoning)1001001001004789.4%
DeepSeek V4 Flash100100100895588.8%
Claude Opus 4.8 (Reasoning)10010097944687.6%
Claude Opus 41001001001003486.8%
Mistral Large 210010091727186.8%
Claude Sonnet 4.51001001001003186.1%
Mistral NeMO1001001001002985.8%
Claude Sonnet 4.6 (Reasoning)1001001001002885.6%
Gemini 2.5 Flash1001001001002384.6%
Gemini 3 Flash (Preview, Reasoning)100100100814184.6%
Qwen 3.5 27B1001001001002084.0%
DeepSeek V3.21001001001001983.7%
MoonshotAI: Kimi K3 (Reasoning, Low)1001001001001783.5%
Qwen 2.5 72B100100100793783.2%
Aion 2.01001001001001482.7%
Gemma 3 12B100100100100881.5%
Mistral Small 4 (Reasoning)10010090813681.4%
MiniMax M2.7100100100100781.4%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100581.0%
ByteDance Seed 2.0 Lite100100100100480.8%
Gemini 3.1 Flash Lite (Reasoning)100100100574680.7%
Claude Opus 4.7100100100712980.0%
ByteDance Seed 1.6 Flash100100100100080.0%
Ministral 3 3B100100100100080.0%
DeepSeek V3.1100100100802079.9%
GPT-4o Mini (temp=0)1001009694478.8%
Gemini 3.5 Flash (Reasoning, Minimal)10010010092078.5%
DeepSeek V3 (2024-12-26)10010010091078.1%
Gemini 3.1 Flash Lite10010010085077.0%
Claude Haiku 4.5100100100671776.7%
Laguna S 2.1100100100513276.6%
Mistral Small 410010010079075.8%
Aion 3.010010010071074.2%
Arcee AI: Trinity Mini1001009271072.6%
DeepSeek V4 Pro10010073474172.4%
Z.AI GLM 5 Turbo10010010060072.0%
Mistral Small 3.2 24B10010010053070.6%
Gemma 4 31B10010010052070.4%
Gemma 4 26B (Reasoning)10010010046069.2%
MiniMax M2.510010010040068.1%
Claude Opus 4.51001008648568.0%
Ministral 3B10010010039067.9%
Gemini 3.1 Flash Lite (Preview)100100100151565.9%
Qwen3 235B A22B Instruct 25071001009229064.3%
Claude Sonnet 4.61009454511162.1%
Z.AI GLM 51001001000060.0%
Gemini 2.5 Flash Lite (Reasoning)1001001000060.0%
Gemma 3 27B1001001000060.0%
Laguna XS 2.11001001000060.0%
GPT-4.1 Nano100100735055.8%
Gemma 4 26B100827218054.3%
Cydonia 24B V4.1100814739053.4%
Ministral 3 14B10088640050.4%
Gemini 2.5 Pro1001002611047.4%
Gemini 3 Flash (Preview)100684619046.6%
Ministral 8B10083330043.3%
Ministral 3 8B10062290038.2%
Z.AI GLM 4.71004700029.4%
Xiaomi MIMO v2.51001300022.5%
Z.AI GLM 4.7 Flash622100016.7%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.2100100100100100100.0%
GPT-5.5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
o4 Mini High100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Opus 4100100100100100100.0%
Qwen 3.5 35B100100100100100100.0%
GPT-4.1100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Mistral Large 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
Laguna S 2.1100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100.0%
Qwen 3 32B100100100100100100.0%
Writer: Palmyra X5100100100100100100.0%
Qwen3 235B A22B Instruct 2507100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
Qwen 2.5 72B100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
WizardLM 2 8x22b100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Ministral 3 3B100100100100100100.0%
ByteDance Seed 2.0 Lite1001001001009999.8%
MoonshotAI: Kimi K2.61001001001009899.6%
Nemotron 3 Super1001001001009799.3%
Z.AI GLM 4.71001001001009699.3%
Qwen 3.6 27B1001001001009498.8%
Claude Sonnet 4.51001001001009398.7%
Z.AI GLM 5.11001001001009298.3%
Gemini 3.6 Flash (Reasoning, Minimal)1001001001009298.3%
Claude Sonnet 51001001001008997.8%
Laguna XS 2.11001001001008997.8%
Qwen 3.5 Flash1001001001008897.6%
Gemini 3.5 Flash (Reasoning, Minimal)1001001001008897.6%
Z.AI GLM 4.51001001001008797.4%
Z.AI GLM 4.5 Air1001001001008697.2%
Cohere Command R+ (Aug. 2024)1001001001008496.8%
DeepSeek V3.21001001001008496.7%
Gemini 3.5 Flash (Reasoning)1001001001007895.6%
Claude Opus 4.5100100100968095.2%
Claude Sonnet 4.6 (Reasoning)1001001001007595.0%
Claude Haiku 4.5100100100908494.8%
Z.AI GLM 51001001001007394.7%
Arcee AI: Trinity Mini1001001001007194.1%
Gemma 3 27B1001001001007094.0%
Hermes 3 405B1001001001006793.3%
Claude Opus 5 (Reasoning)1001001001006693.2%
Gemini 3.1 Flash Lite (Preview)100100100848092.9%
DeepSeek V4 Pro (Reasoning)1001001001006092.1%
Gemini 2.5 Flash (Reasoning)1001001001005490.9%
DeepSeek V4 Pro100100100777790.8%
Claude Sonnet 5 (Reasoning, Low)1001001001005390.7%
Cydonia 24B V4.11001001001005390.6%
Aion 3.0 Mini1001001001005290.4%
Gemma 3 4B100100100757590.1%
Gemini 3 Flash (Preview, Reasoning)1001001001005090.0%
Gemma 4 31B1001001001004889.7%
Aion 3.01001001001004789.3%
Ministral 8B100100100915388.9%
DeepSeek V4 Flash1001001001004388.6%
ByteDance Seed 1.6 Flash100100100934888.1%
Gemini 2.5 Flash Lite1001001001003486.9%
GPT-4.1 Nano1001001001003086.0%
Mistral Large 3100100100913885.9%
Gemini 3 Flash (Preview)1001001001002985.8%
Mistral Medium 3.1100100100983185.8%
Claude Sonnet 5 (Reasoning)1001001001002084.0%
Claude Opus 4.8 (Reasoning)100100100962183.5%
Ministral 3 14B1001001001001583.0%
Aion 2.0100100100833082.6%
MiniMax M3100100100813182.4%
Claude Opus 4.7 (Reasoning)100100100585181.8%
Mistral NeMO100100100931581.6%
Ministral 3 8B1008884736281.4%
Gemini 2.5 Pro100100100644080.8%
Claude Sonnet 4.6100100100100080.0%
Gemma 4 26B100100100100080.0%
Mistral Small 4100100100100080.0%
Gemini 3.1 Flash Lite100100100702178.2%
MiniMax M2.5100100100652678.2%
Gemma 3 12B10010010086478.0%
Xiaomi MIMO v2.510010072655377.9%
Ministral 3B1001008579072.8%
Z.AI GLM 4.7 Flash10010087531170.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100282270.0%
Gemini 2.5 Flash Lite (Reasoning)10010086312668.7%
MoonshotAI: Kimi K3 (Reasoning, Low)10010010037067.5%
Z.AI GLM 5 Turbo10010010031667.3%
Claude Opus 4.