Missing dialogue indicators (quotation marks)

Test: Bad Writing Habits

Avg. Score
97.8%
Scenarios
18

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Ministral 3B100.0%$0.00018.1s100%
2Inception Mercury 2100.0%$0.00327.0s100%
3GPT-4.1 Nano100.0%$0.000713.3s100%
4Mistral Small 4100.0%$0.001418.2s100%
5GPT-5.4 Nano (Reasoning, Low)100.0%$0.005520.6s100%
6GPT-4.1 Mini99.9%$0.002719.0s99%
7Ministral 3 8B99.9%$0.000819.6s98%
8Gemini 3 Flash (Preview)100.0%$0.007819.6s100%
9GPT-5.4 Nano (Reasoning)100.0%$0.006124.5s100%
10GPT-5.4 Nano100.0%$0.005726.3s100%
11Ministral 3 3B99.7%$0.000511.1s95%
12DeepSeek V4 Flash99.8%$0.000631.6s98%
13Claude Haiku 4.5100.0%$0.01121.6s100%
14DeepSeek V3 (2025-03-24)100.0%$0.001439.4s99%
15Writer: Palmyra X5100.0%$0.01122.0s100%
16Mistral Medium 3.1100.0%$0.004836.5s100%
17GPT-5.4 Mini (Reasoning, Low)100.0%$0.01516.8s100%
18GPT-5.4 Mini100.0%$0.01516.8s100%
19Laguna S 2.199.7%$0.000522.1s96%
20Grok 4.3100.0%$0.006930.5s99%
21Gemini 3.5 Flash (Reasoning, Minimal)100.0%$0.01812.0s100%
22Gemini 3.6 Flash (Reasoning, Minimal)100.0%$0.01714.3s100%
23Mistral Large 399.9%$0.003330.3s98%
24ByteDance Seed 1.6 Flash99.7%$0.001327.3s97%
25Muse Glimmer 30B (Reasoning, Medium)100.0%$0.003843.4s100%
26Ministral 8B99.4%$0.000410.4s93%
27Gemma 3 4B99.4%$0.000220.0s94%
28Gemini 3 Flash (Preview, Reasoning)100.0%$0.01230.1s100%
29Mistral Small 4 (Reasoning)99.7%$0.002230.2s96%
30Gemma 4 26B100.0%$0.000955.1s100%
31DeepSeek V3 (2024-12-26)100.0%$0.002154.6s100%
32GPT-5.6 Luna (Reasoning)99.9%$0.01718.0s99%
33Ministral 3 14B99.4%$0.000711.7s91%
34Laguna XS 2.199.4%$0.000314.4s91%
35Nemotron 3 Nano100.0%$0.00101.1m100%
36Grok 4.2099.9%$0.009345.7s99%
37Qwen 3 235B A22B Instruct 250799.9%$0.001159.2s99%
38Xiaomi MIMO v2.599.7%$0.005431.8s95%
39Gemini 2.5 Flash (Reasoning)99.7%$0.01121.5s95%
40GPT-5.6 Luna99.7%$0.01515.8s95%
41Cydonia 24B V4.199.6%$0.001444.8s95%
42Z.AI GLM 5 Turbo99.6%$0.008133.2s95%
43Z.AI GLM 5.3 Flash (Reasoning, Low)99.9%$0.00051.1m98%
44GPT-4o Mini (temp=0)99.6%$0.001234.8s92%
45Qwen 2.5 72B99.6%$0.001036.7s92%
46GPT-4o, Aug. 6th (temp=0)99.8%$0.02322.7s98%
47Z.AI GLM 4.5 Air99.8%$0.002958.2s97%
48DeepSeek V4.1 Flash (Reasoning, High)99.6%$0.002935.4s92%
49Claude Sonnet 5100.0%$0.02733.5s100%
50Mistral Large 299.6%$0.01329.4s94%
51DeepSeek V4 Pro100.0%$0.00481.3m99%
52Z.AI GLM 4.599.5%$0.005142.1s93%
53MiniMax M2.599.9%$0.00341.3m98%
54Muse Spark 1.1 (Reasoning, Medium)99.6%$0.01523.9s92%
55Z.AI GLM 4.7 Flash99.7%$0.00171.2m96%
56DeepSeek V4 Flash (Reasoning)99.4%$0.000731.1s88%
57Grok 4.5 (Reasoning, Low)100.0%$0.0181.1m100%
58Claude Sonnet 4.6100.0%$0.03139.3s100%
59Z.AI GLM 4.7100.0%$0.0101.4m100%
60Z.AI GLM 599.8%$0.00841.2m97%
61GPT-OSS 120B100.0%$0.00151.8m99%
62Arcee AI: Trinity Mini98.7%$0.00039.2s83%
63Thinking Machines Inkling99.8%$0.00751.5m98%
64WizardLM 2 8x22b100.0%$0.00261.8m100%
65Hermes 3 405B99.3%$0.003253.2s91%
66Mistral NeMO98.6%$0.000510.1s83%
67Thinking Machines Inkling Small (Reasoning, High)100.0%$0.00881.7m100%
68MiniMax M2.799.4%$0.00401.1m93%
69Z.AI GLM 5.2 (Reasoning, High)99.5%$0.0111.0m94%
70Claude Sonnet 499.9%$0.03243.7s98%
71Gemini 2.5 Flash Lite97.9%$0.00099.5s83%
72GPT-4o Mini (temp=1)98.9%$0.001234.8s86%
73Gemma 4 26B (Reasoning)100.0%$0.00132.0m100%
74DeepSeek V3.199.9%$0.00201.8m98%
75DeepSeek-V2 Chat99.4%$0.002153.3s88%
76Z.AI GLM 5.199.9%$0.0141.5m99%
77Gemini 2.5 Flash98.5%$0.005210.6s83%
78GPT-4.199.2%$0.01844.7s93%
79GPT-5 Mini99.3%$0.010057.4s92%
80Claude Sonnet 4.599.8%$0.03538.1s96%
81Claude Sonnet 5 (Reasoning, Low)99.6%$0.03138.4s96%
82Qwen 3 32B99.1%$0.001554.6s87%
83Hermes 3 70B98.9%$0.00101.2m91%
84GPT-5.6 Terra99.9%$0.04636.9s99%
85Aion 2.099.3%$0.00641.3m92%
86Kimi K3 (Reasoning, Low)99.8%$0.0281.1m97%
87Xiaomi MIMO v2.5 Pro99.3%$0.008553.5s88%
88GPT-5.4 Mini (Reasoning)99.2%$0.02228.1s88%
89GPT-5.6 Terra (Reasoning)99.7%$0.04435.2s96%
90Llama 3.1 70B98.9%$0.001529.4s79%
91Gemma 3 12B98.5%$0.000441.3s82%
92Cohere Command R+ (Aug. 2024)98.9%$0.02052.5s90%
93Gemini 2.5 Pro99.4%$0.03636.2s91%
94Qwen 3.5 Plus (2026-02-15)98.9%$0.006031.5s79%
95Muse Spark 1.1 (Reasoning, Minimal)98.9%$0.01219.2s79%
96Aion 3.0 Mini98.8%$0.00531.2m87%
97Aion 3.099.3%$0.0241.0m91%
98Muse Spark 1.2 (Reasoning, Medium)99.0%$0.01838.3s84%
99Gemini 3.6 Flash (Reasoning)99.6%$0.04432.5s92%
100Gemma 4 31B99.4%$0.00101.6m88%
101GPT-5 Nano99.1%$0.00421.4m87%
102Gemma 3 27B98.2%$0.000652.6s82%
103Claude Sonnet 5 (Reasoning)99.2%$0.03038.9s88%
104ByteDance Seed 1.6100.0%$0.0132.5m99%
105o4 Mini98.6%$0.01525.7s79%
106Grok 4.3 (Reasoning)100.0%$0.0212.3m100%
107DeepSeek V3.299.0%$0.00141.9m89%
108Grok 4.5 (Reasoning, High)99.7%$0.0301.6m95%
109GPT-4o, Aug. 6th (temp=1)98.0%$0.01824.4s81%
110Claude Opus 4.8 (Reasoning)100.0%$0.07141.7s100%
111Claude Opus 4.8 (Reasoning, Low)100.0%$0.07141.9s100%
112 Kimi K3 (Reasoning, High)100.0%$0.0441.8m99%
113Nemotron 3 Super98.9%$0.00001.4m79%
114MiniMax M399.9%$0.00603.1m99%
115Qwen 3.5 9B98.5%$0.00111.4m78%
116GPT-5.499.6%$0.0491.4m96%
117DeepSeek V4 Pro 0813 (Reasoning, High)99.2%$0.0251.8m91%
118Gemini 3.5 Flash (Reasoning)99.7%$0.07137.6s95%
119Z.AI GLM 4.697.2%$0.006551.5s77%
120GPT-5.1100.0%$0.0541.8m100%
121Claude Opus 5100.0%$0.08250.1s100%
122Claude Opus 4.799.6%$0.06930.4s92%
123Claude Opus 4.599.8%$0.07053.4s96%
124GPT-5.299.9%$0.0561.5m98%
125Gemini 3.7 Flash (Reasoning, Medium)97.8%$0.01218.2s71%
126ByteDance Seed 2.0 Lite98.8%$0.0122.2m90%
127GPT-5.4 (Reasoning, Low)99.6%$0.0551.4m96%
128Kimi K2.5100.0%$0.0193.2m100%
129Grok 4.20 (Reasoning)98.1%$0.0181.5m84%
130Claude Opus 4.7 (Reasoning)99.5%$0.07632.0s92%
131DeepSeek V4 Flash 0731 (Reasoning, High)99.1%$0.00222.6m87%
132Claude Opus 4.699.9%$0.0781.2m98%
133Claude Sonnet 4.6 (Reasoning)99.4%$0.0601.2m91%
134Gemma 4 31B (Reasoning)98.9%$0.00142.2m79%
135DeepSeek V4 Pro (Reasoning)99.3%$0.0153.1m93%
136Claude Opus 5 (Reasoning, Low)100.0%$0.0971.0m99%
137Claude Opus 5 (Reasoning)100.0%$0.0981.0m99%
138Gemini 2.5 Flash Lite (Reasoning)95.2%$0.002830.8s68%
139Claude Opus 4.6 (Reasoning)99.9%$0.0881.4m99%
140Grok 4.6 (Reasoning, High)99.6%$0.0422.5m92%
141o4 Mini High97.1%$0.02547.2s69%
142GPT-5.6 Sol99.6%$0.1051.2m92%
143Gemini 3.8 Flash (Reasoning, Medium)94.9%$0.01418.0s57%
144GPT-5.4 (Reasoning)100.0%$0.0892.6m99%
145GPT-5.6 Sol (Reasoning)99.3%$0.1081.2m88%
146Gemini 3.5 Flash Lite (Reasoning)92.9%$0.00356.6s51%
147Thinking Machines Inkling (Reasoning)99.8%$0.0255.1m96%
148Qwen 3.8 Flash (Reasoning)97.4%$0.00643.1m70%
149GPT-5.5 (Reasoning, Low)99.8%$0.1391.8m97%
150Qwen 3.6 27B96.5%$0.0252.3m66%
151GPT-5.599.7%$0.1391.7m95%
152GPT-5.5 (Reasoning)99.5%$0.1421.8m95%
153ByteDance Seed 2.0 Mini98.1%$0.00454.9m78%
154Hy4 Preview (Reasoning, High)98.4%$0.0404.4m82%
155Z.AI GLM 5.3 Flash (Reasoning, Max)99.0%$0.00316.5m89%
156Gemini 3.1 Flash Lite (Preview)87.6%$0.00308.4s41%
157Mistral Small 3.2 24B98.0%$0.00695.7m76%
158Qwen 3.8 27B (Reasoning, XHigh)98.2%$0.0404.7m76%
159Qwen 3.5 Plus (2026-04-20)92.2%$0.0171.8m49%
160Claude Opus 4100.0%$0.2091.4m100%
161Qwen 3.8 Max (Reasoning, XHigh)100.0%$0.0835.9m100%
162Gemini 3.1 Flash Lite85.6%$0.003012.1s38%
163Qwen 3.5 122B90.5%$0.0251.1m42%
164Gemini 3.5 Flash Lite (Reasoning, Minimal)86.4%$0.00356.3s33%
165Muse Spark 1.3 (Reasoning, Medium)90.4%$0.0251.8m42%
166Qwen 3.5 27B89.1%$0.0201.6m39%
167Kimi K2.698.8%$0.0586.5m84%
168Qwen 3.5 Flash83.5%$0.002547.5s29%
169Gemini 3.1 Flash Lite (Reasoning)81.4%$0.003011.9s28%
170Z.AI GLM 5.3 (Reasoning, Max)98.8%$0.0667.3m85%
171GPT-590.9%$0.0652.8m45%
172Qwen 3.6 Flash80.1%$0.01041.4s24%
173Gemini 3.1 Pro (Preview)90.0%$0.1071.8m40%
174Qwen 3.5 35B80.2%$0.0181.0m24%
175Qwen 3.6 35B76.2%$0.00831.0m19%
176Qwen 3.7 Flash (Reasoning)74.9%$0.001055.2s17%
177Qwen 3.7 Max84.4%$0.0682.3m28%
178Qwen 3.5 397B A17B79.2%$0.0143.0m20%
179Qwen 3.6 Max Preview72.2%$0.0503.5m11%
97.83%

Individual Scenarios

Detailed Writing Rules

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100.0%
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100.0%
Qwen 3.7 Max100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100.0%
Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
MiniMax M3100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 5100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.2100100100100100100.0%
GPT-5.5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100.0%
Aion 3.0100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Z.AI GLM 4.6100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
Claude Opus 4100100100100100100.0%
GPT-4.1100100100100100100.0%
MiniMax M2.5100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Aion 2.0100100100100100100.0%
o4 Mini100100100100100100.0%
MiniMax M2.7100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
Mistral Large 3100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
Claude Haiku 4.5100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
DeepSeek V4 Pro100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Nemotron 3 Super100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
Laguna S 2.1100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100.0%
Qwen 3 32B100100100100100100.0%
Writer: Palmyra X5100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
Llama 3.1 70B100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Mistral Medium 3.1100100100100100100.0%
Gemma 3 12B100100100100100100.0%
Gemma 3 27B100100100100100100.0%
Laguna XS 2.1100100100100100100.0%
Mistral Small 4100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
Qwen 2.5 72B100100100100100100.0%
Cydonia 24B V4.1100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
WizardLM 2 8x22b100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100.0%
Ministral 3 14B100100100100100100.0%
Ministral 3 8B100100100100100100.0%
GPT-4.1 Nano100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Cohere Command R+ (Aug. 2024)100100100100100100.0%
Gemma 3 4B100100100100100100.0%
Ministral 3 3B100100100100100100.0%
Mistral NeMO100100100100100100.0%
Ministral 8B100100100100100100.0%
Ministral 3B100100100100100100.0%
Claude Sonnet 4.51001001001009799.4%
Grok 4.201001001001009799.4%
Grok 4.31001001001009799.4%
DeepSeek V4 Pro (Reasoning)1001001001008997.9%
Mistral Large 21001001001008997.9%
Gemini 2.5 Pro1001001001008396.7%
Claude Sonnet 4.6 (Reasoning)1001001001008396.7%
Gemini 2.5 Flash Lite (Reasoning)100100100978396.1%
GPT-51001001001005791.4%
Hy4 Preview (Reasoning, High)1001001001004288.3%
Arcee AI: Trinity Mini1001001001004288.3%
Qwen 3.6 Max Preview100100100100080.0%
Qwen 3.5 27B100100100100080.0%
o4 Mini High100100100100080.0%
ByteDance Seed 2.0 Mini100100100752580.0%
Qwen 3.5 Flash100100100100080.0%
Qwen 3.6 35B1001001000060.0%
Qwen 3.5 35B10010000040.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100.0%
Qwen 3.7 Max100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100.0%
Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
MiniMax M3100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Claude Opus 5100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.2100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100.0%
Aion 3.0100100100100100100.0%
o4 Mini High100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100.0%
Gemini 2.5 Pro100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
Claude Opus 4100100100100100100.0%
MiniMax M2.5100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Aion 2.0100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
Qwen 3.5 Flash100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
Mistral Large 3100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
Claude Haiku 4.5100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100.0%
DeepSeek V4 Pro100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Nemotron 3 Super100100100100100100.0%
Mistral Large 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
Laguna S 2.1100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100.0%
Qwen 3 32B100100100100100100.0%
Writer: Palmyra X5100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
Llama 3.1 70B100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Mistral Medium 3.1100100100100100100.0%
Gemma 3 12B100100100100100100.0%
Laguna XS 2.1100100100100100100.0%
Mistral Small 4100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
Qwen 2.5 72B100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
WizardLM 2 8x22b100100100100100100.0%
Ministral 3 14B100100100100100100.0%
Ministral 3 8B100100100100100100.0%
GPT-4.1 Nano100100100100100100.0%
Ministral 3 3B100100100100100100.0%
Mistral NeMO100100100100100100.0%
Ministral 8B100100100100100100.0%
Ministral 3B100100100100100100.0%
Claude Opus 5 (Reasoning, Low)1001001001009799.4%
Claude Opus 4.51001001001009799.4%
Z.AI GLM 5.3 Flash (Reasoning, Low)1001001001009799.4%
Gemini 3.1 Flash Lite (Preview)1001001001009799.4%
Cydonia 24B V4.11001001001009799.4%
ByteDance Seed 1.6 Flash1001001001009799.4%
Z.AI GLM 5.3 (Reasoning, Max)1001001001009498.8%
Claude Opus 4.61001001001009498.8%
GPT-5.6 Terra (Reasoning)1001001001008997.9%
GPT-5.4 (Reasoning, Low)1001001001008997.9%
Claude Sonnet 5 (Reasoning)1001001001008997.9%
Cohere Command R+ (Aug. 2024)1001001001008997.9%
GPT-5.5 (Reasoning, Low)1001001001008797.3%
GPT-4o, Aug. 6th (temp=0)100100100978997.3%
GPT-5.5 (Reasoning)1001001001008396.7%
GPT-4.11001001001008396.7%
MiniMax M2.71001001001008396.7%
GPT-5.51001001001007595.0%
GPT-5.6 Luna1001001001007595.0%
Xiaomi MIMO v2.51001001001007595.0%
DeepSeek V3.21001001001007595.0%
Hermes 3 70B1001001001007595.0%
Gemma 3 4B100100100898394.5%
Claude Sonnet 5 (Reasoning, Low)10010094898393.3%
Gemma 3 27B100100100897592.9%
Grok 4.6 (Reasoning, High)1001001001006392.5%
Grok 4.20 (Reasoning)1001001001006392.5%
Aion 3.0 Mini1001001001006392.5%
Gemini 2.5 Flash Lite1001001001006392.5%
GPT-5.6 Sol (Reasoning)1001001001004288.3%
Arcee AI: Trinity Mini1001001001004288.3%
Gemini 2.5 Flash100100100834285.0%
Z.AI GLM 4.6100100100754283.3%
Gemini 2.5 Flash Lite (Reasoning)100100100754283.3%
GPT-5100100100100080.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100080.0%
Qwen 3.5 35B100100100100080.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100.0%
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100.0%
Qwen 3.7 Max100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100.0%
Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
MiniMax M3100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 5100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.2100100100100100100.0%
GPT-5.5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100.0%
Aion 3.0100100100100100100.0%
o4 Mini High100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100.0%
Gemini 2.5 Pro100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Z.AI GLM 4.6100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
Claude Opus 4100100100100100100.0%
GPT-4.1100100100100100100.0%
MiniMax M2.5100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Aion 2.0100100100100100100.0%
o4 Mini100100100100100100.0%
MiniMax M2.7100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
Mistral Large 3100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
Claude Haiku 4.5100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
DeepSeek V4 Pro100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Mistral Large 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100.0%
Qwen 3 32B100100100100100100.0%
Writer: Palmyra X5100100100100100100.0%
Qwen 3 235B A22B Instruct 2507100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
Llama 3.1 70B100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Mistral Medium 3.1100100100100100100.0%
Gemma 3 12B100100100100100100.0%
Gemma 3 27B100100100100100100.0%
Laguna XS 2.1100100100100100100.0%
Mistral Small 4100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
Qwen 2.5 72B100100100100100100.0%
Cydonia 24B V4.1100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
WizardLM 2 8x22b100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100.0%
Ministral 3 14B100100100100100100.0%
Ministral 3 8B100100100100100100.0%
GPT-4.1 Nano100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100.0%
Cohere Command R+ (Aug. 2024)100100100100100100.0%
Gemma 3 4B100100100100100100.0%
Ministral 3 3B100100100100100100.0%
Mistral NeMO100100100100100100.0%
Ministral 8B100100100100100100.0%
Ministral 3B100100100100100100.0%
GPT-5.5 (Reasoning)1001001001009799.4%
Grok 4.201001001001009799.4%
Laguna S 2.11001001001008997.9%
GPT-5 Nano1001001001004589.0%
Qwen 3.5 27B100100100100080.0%
Qwen 3.5 35B100100100100080.0%
Nemotron 3 Super100100100100080.0%
Qwen 3.5 Flash10010010089077.9%
Qwen 3.6 Max Preview1001001000060.0%
GPT-51001001000060.0%
Qwen 3.7 Flash (Reasoning)1001001000060.0%