Parse dialogue

Test: Language Writing

Avg. Score
89.9%
Scenarios
5

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1GPT-4o Mini (temp=0)100.0%$0.00034.8s100%
2GPT-4o Mini (temp=1)100.0%$0.00035.6s100%
3Inception Mercury 299.6%$0.00061.4s96%
4Gemini 3 Flash (Preview)100.0%$0.00205.6s100%
5GPT-5.4 Mini (Reasoning, Low)99.9%$0.00223.5s99%
6DeepSeek-V2 Chat100.0%$0.000116.1s100%
7GPT-4.1 Mini99.3%$0.00043.4s93%
8Gemini 3.6 Flash (Reasoning, Minimal)100.0%$0.00505.1s100%
9GPT-4o, Aug. 6th (temp=0)100.0%$0.00526.1s100%
10Muse Spark 1.1 (Reasoning, Minimal)100.0%$0.00478.0s100%
11Z.AI GLM 4.599.7%$0.001314.5s97%
12Gemini 3.5 Flash (Reasoning, Minimal)100.0%$0.00735.3s100%
13Z.AI GLM 5 Turbo99.8%$0.003714.7s98%
14GPT-5.6 Luna98.7%$0.00525.1s95%
15Hermes 3 405B99.1%$0.000021.0s94%
16Gemma 4 31B100.0%$0.000332.1s100%
17GPT-4o, Aug. 6th (temp=1)99.4%$0.00566.5s94%
18GPT-5.4 Mini97.5%$0.00202.5s87%
19Muse Spark 1.1 (Reasoning, Medium)100.0%$0.007312.8s100%
20GPT-OSS 120B99.4%$0.000328.0s96%
21Gemini 3.1 Flash Lite (Reasoning)98.4%$0.00115.3s84%
22o4 Mini100.0%$0.007116.7s100%
23Nemotron 3 Super97.8%$0.000021.7s91%
24GPT-5.6 Luna (Reasoning)98.3%$0.00788.0s96%
25GPT-5.4 Nano (Reasoning)98.0%$0.00166.1s83%
26Claude Sonnet 4.6100.0%$0.01013.7s100%
27Gemini 2.5 Flash97.5%$0.00266.0s83%
28DeepSeek V3.198.7%$0.000722.8s88%
29Gemini 3.1 Flash Lite96.8%$0.00116.2s78%
30GPT-5.6 Terra (Reasoning)99.4%$0.01210.4s98%
31Gemini 3.5 Flash Lite (Reasoning)95.4%$0.00224.4s79%
32MoonshotAI: Kimi K3 (Reasoning, Low)99.8%$0.009722.6s98%
33Nemotron 3 Nano95.3%$0.000210.8s77%
34GPT-4.197.8%$0.00416.8s79%
35Z.AI GLM 5.2 (Reasoning, High)99.6%$0.007234.7s98%
36Z.AI GLM 5.198.1%$0.004630.6s92%
37MiniMax M2.597.1%$0.001321.5s81%
38Gemini 3.5 Flash Lite (Reasoning, Minimal)94.4%$0.00214.1s75%
39Gemini 2.5 Flash (Reasoning)97.1%$0.006713.1s86%
40Gemini 3.1 Flash Lite (Preview)95.0%$0.00113.7s72%
41Grok 4.394.5%$0.00093.8s72%
42GPT-5.6 Terra98.7%$0.01410.9s96%
43Laguna S 2.195.8%$0.000113.7s73%
44GPT-5.4 Mini (Reasoning)96.2%$0.00467.3s77%
45GPT-4.1 Nano92.9%$0.00014.0s69%
46Aion 3.0 Mini97.4%$0.002337.9s87%
47GPT-5.4 Nano (Reasoning, Low)93.7%$0.00187.0s69%
48GPT-5.6 Sol99.2%$0.01911.2s97%
49Gemini 3 Flash (Preview, Reasoning)94.9%$0.005111.8s74%
50Claude Opus 4.599.3%$0.01915.7s96%
51Grok 4.2092.7%$0.002110.9s69%
52MoonshotAI: Kimi K3 (Reasoning, High)100.0%$0.01636.2s100%
53GPT-5 Nano99.4%$0.00281.2m96%
54Z.AI GLM 4.5 Air95.1%$0.001230.1s73%
55GPT-5.498.0%$0.01319.0s85%
56GPT-5 Mini98.0%$0.006637.2s82%
57o4 Mini High99.5%$0.01536.6s97%
58Gemma 4 26B94.0%$0.000336.3s74%
59Thinking Machines Inkling93.0%$0.002018.7s67%
60Claude Sonnet 5 (Reasoning)98.1%$0.01419.0s84%
61Laguna XS 2.193.9%$0.000213.4s58%
62Claude Haiku 4.588.7%$0.00377.8s67%
63Qwen 3.6 35B97.1%$0.005938.3s79%
64GPT-5.4 (Reasoning, Low)96.6%$0.01216.1s77%
65Claude Sonnet 492.6%$0.009211.1s71%
66GPT-5.4 Nano91.6%$0.00176.6s55%
67Aion 3.093.7%$0.008427.0s76%
68Claude Sonnet 596.0%$0.01217.0s73%
69Gemma 3 27B89.4%$0.000327.9s64%
70Grok 4.3 (Reasoning)100.0%$0.0121.2m100%
71Gemma 3 12B90.2%$0.000233.9s66%
72Qwen 3.5 Plus (2026-02-15)90.2%$0.001927.2s65%
73Gemma 4 26B (Reasoning)95.4%$0.000858.3s75%
74GPT-5.6 Sol (Reasoning)99.9%$0.02917.7s99%
75Claude Sonnet 5 (Reasoning, Low)96.1%$0.01520.4s78%
76Grok 4.5 (Reasoning, High)99.6%$0.01847.7s96%
77GPT-5.297.4%$0.01626.0s82%
78Grok 4.20 (Reasoning)98.2%$0.009749.7s83%
79Llama 3.1 70B85.4%$0.000720.5s62%
80Gemini 2.5 Flash Lite85.5%$0.00055.1s53%
81Gemini 3.6 Flash (Reasoning)96.4%$0.01713.1s76%
82Hermes 3 70B88.3%$0.000316.6s54%
83Aion 2.092.3%$0.003041.5s69%
84Qwen 3.6 Flash93.7%$0.007730.0s69%
85Mistral Large 384.0%$0.001417.3s59%
86DeepSeek V4 Flash (Reasoning)89.5%$0.000220.8s51%
87Xiaomi MIMO v2.587.7%$0.002314.2s51%
88Grok 4.5 (Reasoning, Low)94.0%$0.006024.5s57%
89GPT-5.5 (Reasoning)99.4%$0.03322.8s98%
90Claude Sonnet 4.589.8%$0.01113.2s62%
91Gemma 4 31B (Reasoning)92.6%$0.00051.1m70%
92DeepSeek V4 Flash87.0%$0.000212.4s42%
93GPT-5.4 (Reasoning)94.8%$0.01730.6s74%
94Qwen 3.5 Flash88.9%$0.001942.8s60%
95ByteDance Seed 1.6 Flash82.5%$0.000614.6s48%
96ByteDance Seed 1.691.3%$0.004050.4s65%
97Claude Opus 4.799.6%$0.03818.6s98%
98Xiaomi MIMO v2.5 Pro90.4%$0.003622.3s46%
99GPT-5.5 (Reasoning, Low)98.5%$0.03523.8s96%
100GPT-5.598.3%$0.03626.1s96%
101Qwen 3.5 122B90.0%$0.01335.0s65%
102Claude Opus 599.5%$0.03926.4s97%
103Gemini 3.5 Flash (Reasoning)93.8%$0.02614.8s72%
104Qwen 3.5 27B91.0%$0.009648.7s63%
105Gemini 2.5 Flash Lite (Reasoning)83.7%$0.001923.9s41%
106Claude Opus 4.8 (Reasoning)97.8%$0.03220.8s80%
107Claude Opus 4.8 (Reasoning, Low)97.6%$0.03220.8s79%
108MoonshotAI: Kimi K2.594.2%$0.00821.2m68%
109MiniMax M389.4%$0.00221.3m65%
110Qwen 3.5 Plus (2026-04-20)94.3%$0.0111.1m69%
111DeepSeek V4 Pro (Reasoning)92.0%$0.00291.3m63%
112Arcee AI: Trinity Mini81.2%$0.000215.9s32%
113Claude Sonnet 4.6 (Reasoning)95.2%$0.02531.1s72%
114ByteDance Seed 2.0 Lite93.6%$0.00691.3m67%
115Z.AI GLM 4.693.2%$0.00441.2m60%
116Gemini 2.5 Pro95.1%$0.02422.5s61%
117GPT-5.197.3%$0.02752.6s81%
118Z.AI GLM 589.1%$0.00591.3m63%
119DeepSeek V3.290.0%$0.00041.1m45%
120Gemma 3 4B74.6%$0.000113.4s29%
121Qwen 3.5 35B83.9%$0.01033.8s49%
122Z.AI GLM 4.7 Flash80.3%$0.001252.0s45%
123Qwen 3 32B79.2%$0.000635.4s33%
124Cohere Command R+ (Aug. 2024)73.2%$0.006213.4s38%
125Claude Opus 4.7 (Reasoning)97.5%$0.04319.7s81%
126Ministral 3B59.5%$0.00002.9s30%
127Mistral Small 3.2 24B70.5%$0.000311.0s20%
128Qwen 3.5 9B86.4%$0.00111.6m51%
129Mistral Small 4 (Reasoning)71.1%$0.001115.3s21%
130DeepSeek V3 (2025-03-24)72.8%$0.000514.6s16%
131Claude Opus 5 (Reasoning)98.7%$0.05335.0s96%
132Claude Opus 5 (Reasoning, Low)98.9%$0.05334.0s97%
133Z.AI GLM 4.790.9%$0.00541.9m64%
134DeepSeek V3 (2024-12-26)75.8%$0.000719.4s15%
135Gemini 3.1 Pro (Preview)94.8%$0.03740.3s71%
136MiniMax M2.769.6%$0.002128.4s26%
137Qwen 2.5 72B67.9%$0.000419.8s20%
138Mistral NeMO66.6%$0.00014.3s10%
139Claude Opus 4.692.3%$0.03634.3s67%
140DeepSeek V4 Pro75.6%$0.002035.4s22%
141ByteDance Seed 2.0 Mini90.2%$0.00232.5m66%
142Mistral Large 270.4%$0.004218.6s16%
143Qwen 3.6 27B88.0%$0.0171.5m60%
144Claude Opus 4.6 (Reasoning)92.2%$0.04038.1s65%
145WizardLM 2 8x22b61.1%$0.000815.6s12%
146Ministral 8B52.8%$0.00016.6s14%
147Mistral Small 448.9%$0.00046.4s17%
148Qwen 3.5 397B A17B90.0%$0.0152.0m60%
149GPT-598.0%$0.0441.3m81%
150Mistral Medium 3.149.0%$0.002016.7s17%
151Qwen3.7 Max94.1%$0.0361.3m68%
152Qwen3.6 Max Preview100.0%$0.0362.5m100%
153Ministral 3 8B47.9%$0.00026.1s3%
154Claude Opus 486.0%$0.04824.7s61%
155Cydonia 24B V4.150.0%$0.000623.3s5%
156Ministral 3 3B36.2%$0.00012.6s0%
157Qwen3 235B A22B Instruct 250746.7%$0.000422.5s0%
158Thinking Machines Inkling (Reasoning)91.2%$0.0152.9m63%
159MoonshotAI: Kimi K2.693.5%$0.0192.9m65%
160Writer: Palmyra X543.2%$0.005113.6s0%
161Ministral 3 14B10.0%$0.00039.0s0%
89.89%

Individual Scenarios

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
o4 Mini High100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Z.AI GLM 4.6100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
MiniMax M2.5100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Aion 2.0100100100100100100.0%
o4 Mini100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
Gemma 4 26B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
GPT-5.4100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Inception Mercury 2100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
Laguna S 2.1100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100.0%
GPT-5.6 Luna (Reasoning)1001001001009999.8%
GPT-5.21001001001009999.8%
GPT-5.6 Luna1001001001009999.7%
GPT-5.4 Nano (Reasoning, Low)1001001001009899.7%
GPT-5.5 (Reasoning, Low)1001001001009899.6%
GPT-5.6 Sol (Reasoning)1001001001009899.6%
GPT-5.5 (Reasoning)1001001001009899.5%
Claude Opus 51001001001009799.4%
Aion 3.01001001001009699.3%
Z.AI GLM 5.11001001001009699.1%
Claude Sonnet 4.6 (Reasoning)1001001001009699.1%
Gemini 2.5 Pro1001001001009599.1%
Z.AI GLM 5.2 (Reasoning, High)1001001001009599.0%
GPT-5.4 (Reasoning)100100100989799.0%
MoonshotAI: Kimi K3 (Reasoning, Low)1001001001009599.0%
Xiaomi MIMO v2.51001001001009598.9%
GPT-5 Mini1001001001009498.9%
Laguna XS 2.11001001001009498.9%
Claude Sonnet 5 (Reasoning)1001001001009498.8%
Qwen 3.5 Flash1001001001009498.8%
Gemini 2.5 Flash1001001001009498.8%
GPT-5.6 Terra (Reasoning)10010099989798.7%
GPT-5 Nano1001001001009398.6%
Gemma 3 27B1001001001009398.6%
GPT-5.51009999989798.5%
Claude Opus 5 (Reasoning, Low)100100100979698.5%
GPT-OSS 120B1001001001009298.5%
DeepSeek V3.21001001001009298.5%
GPT-5.6 Terra10010099979698.4%
Gemini 2.5 Flash Lite (Reasoning)1001001001009298.3%
Gemma 3 12B1001001001009298.3%
Claude Opus 5 (Reasoning)10010098979698.3%
Grok 4.5 (Reasoning, High)1001001001009198.2%
Claude Opus 4.7 (Reasoning)1001001001009098.1%
Gemini 3.5 Flash Lite (Reasoning)1001001001008997.8%
Qwen 3.6 35B100100100949397.5%
Nemotron 3 Nano1001001001008697.1%
Nemotron 3 Super1001001001008596.9%
Claude Sonnet 4100100100939196.8%
Z.AI GLM 4.7 Flash100100100949096.8%
GPT-4.1 Mini1001001001008296.4%
ByteDance Seed 1.6 Flash100100100919096.2%
Gemma 3 4B1001001001008096.0%
GPT-5.4 Mini1001001001007995.8%
DeepSeek V4 Pro1001001001007595.0%
Qwen 3.5 Plus (2026-02-15)1009494939394.8%
Aion 3.0 Mini100100100888694.6%
Mistral Small 4 (Reasoning)100100100928194.6%
Claude Opus 41009494928993.9%
Gemini 2.5 Flash Lite10010093928393.8%
Claude Haiku 4.5100100100917793.6%
Llama 3.1 70B10010092918593.6%
GPT-4.1 Nano100100100907392.5%
Z.AI GLM 51009490888691.7%
Qwen 3.5 35B100100100965990.9%
Qwen 3.5 9B1009695935687.9%
MiniMax M3100100100716787.6%
ByteDance Seed 2.0 Mini100100100864085.1%
Mistral Large 31009280766783.1%
DeepSeek V4 Flash (Reasoning)100100100100080.0%
Mistral Large 2100100100100080.0%
DeepSeek V3 (2024-12-26)10010010094078.8%
Qwen 3 32B10010010089077.8%
Writer: Palmyra X510010010086077.1%
Cydonia 24B V4.110010010069073.8%
Mistral Medium 3.11007567505068.3%
Ministral 8B1001007367067.9%
Cohere Command R+ (Aug. 2024)927065635067.8%
Ministral 3B1001007150064.3%
MiniMax M2.7100956150061.2%
Mistral NeMO1001001000060.0%
Qwen 2.5 72B10095850056.0%
WizardLM 2 8x22b100100710054.3%
Ministral 3 8B100100500050.0%
Mistral Small 48571600043.1%
Mistral Small 3.2 24B1005000030.0%
Qwen3 235B A22B Instruct 2507100000020.0%
Ministral 3 14B000000.0%
Ministral 3 3B000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
MiniMax M3100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Claude Opus 5100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
Laguna S 2.1100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100.0%
GPT-5.41001001001009999.8%
GPT-5.4 (Reasoning)1001001001009899.7%
GPT-5.6 Luna1001001001009899.6%
GPT-5.4 Mini (Reasoning, Low)1001001001009799.5%
Claude Opus 5 (Reasoning, Low)1001001001009799.4%
GPT-5.4 Nano100100100989899.3%
GPT-5.6 Sol100100100989899.3%
GPT-5.5 (Reasoning)10010099999899.2%
Claude Opus 4.71001001001009599.0%
Claude Sonnet 4.6 (Reasoning)1001001001009598.9%
Claude Opus 4.51001001001009598.9%
GPT-5.6 Terra (Reasoning)100100100979798.9%
GPT-5.6 Terra10010099989898.9%
Gemini 3 Flash (Preview, Reasoning)1001001001009498.8%
o4 Mini High1001001001009498.8%
Gemini 2.5 Pro1001001001009498.8%
Qwen 3.5 Flash1001001001009498.8%
Aion 3.0 Mini1001001001009498.8%
GPT-5.51009999989898.7%
MiniMax M2.51001001001009398.6%
Mistral Large 31001001001009398.6%
Xiaomi MIMO v2.5 Pro100100100979698.5%
GPT-5.4 Nano (Reasoning, Low)100100100989598.5%
Qwen 3.6 Flash1001001001009298.5%
Claude Opus 5 (Reasoning)100100100979498.3%
Nemotron 3 Super1001001001009298.3%
DeepSeek V3.21001001001009298.3%
GPT-5.5 (Reasoning, Low)10010098979798.2%
Aion 2.01001001001009198.2%
ByteDance Seed 2.0 Mini1001001001009198.2%
Gemini 3.5 Flash Lite (Reasoning)1001001001009198.2%
Hermes 3 405B1001001001009198.2%
GPT-5.21009998979698.1%
GPT-5.11001001001009097.9%
DeepSeek V3 (2025-03-24)1001001001008997.8%
Gemini 2.5 Flash Lite1001001001008997.8%
Grok 4.31001001001008997.8%
GPT-5.6 Luna (Reasoning)1009998969697.7%
Claude Opus 4100100100959397.7%
Thinking Machines Inkling1001001001008897.5%
Claude Sonnet 4100100100939397.1%
Qwen 3.5 9B100100100949197.0%
Claude Sonnet 4.510010095949396.5%
Xiaomi MIMO v2.51001001001008096.1%
Gemma 3 4B100100100928695.6%
Z.AI GLM 510010095958895.6%
Gemma 4 26B1001001001007895.6%
Gemma 3 27B100100100928395.1%
Gemma 3 12B10010091919094.4%
Llama 3.1 70B100100100838092.7%
DeepSeek V4 Pro100100100837992.4%
Claude Haiku 4.510010094887992.2%
ByteDance Seed 1.6 Flash100100100808092.0%
GPT-4.1 Nano1001001001006092.0%
Z.AI GLM 4.5 Air1001001001005891.7%
Z.AI GLM 4.7 Flash100100100886289.8%
Aion 3.010010095875888.0%
Mistral Small 410010080736383.2%
Qwen 3.5 35B100100100100080.0%
DeepSeek V3 (2024-12-26)100100100100080.0%
Qwen 3 32B100100100100080.0%
Mistral Small 3.2 24B100100100100080.0%
Laguna XS 2.1100100100100080.0%
Gemini 2.5 Flash Lite (Reasoning)10010010090078.0%
Z.AI GLM 4.61001009594077.8%
DeepSeek V4 Flash1001009292076.8%
MiniMax M2.71001009286075.5%
Hermes 3 70B10010010060072.0%
Mistral Small 4 (Reasoning)1001009853070.2%
Cohere Command R+ (Aug. 2024)867570625068.5%
Mistral Large 21001001000060.0%
Mistral Medium 3.11001005050060.0%
Ministral 3 3B100100900058.0%
Qwen3 235B A22B Instruct 2507100100670053.3%
Ministral 8B10092690052.2%
WizardLM 2 8x22b10088630050.0%
Ministral 3B67645858049.3%
Mistral NeMO100100330046.7%
Qwen 2.5 72B10010000040.0%
Cydonia 24B V4.11008600037.1%
Writer: Palmyra X5100000020.0%
Ministral 3 8B68000013.7%
Ministral 3 14B000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
Claude Opus 5100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
o4 Mini High100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
o4 Mini100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
Laguna S 2.1100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
GPT-5.5 (Reasoning, Low)1001001001009999.7%
GPT-5.51001001001009899.6%
GPT-5.6 Luna (Reasoning)1001001001009899.6%
GPT-5.6 Terra100100100999899.4%
Claude Opus 5 (Reasoning)1001001001009699.3%
Claude Opus 4.51001001001009599.0%
Aion 3.0 Mini1001001001009498.7%
GPT-5.6 Sol100100100989398.2%
Hermes 3 405B1001001001008897.5%
Z.AI GLM 5.11001001001008697.1%
GPT-4o, Aug. 6th (temp=1)1001001001008697.1%
Nemotron 3 Super100100100908895.5%
Gemini 2.5 Flash (Reasoning)1001001001006793.3%
Aion 3.0100100100966993.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)1001001001006392.5%
Z.AI GLM 4.6100100100956792.3%
GPT-5.41001001001006192.2%
Gemini 3.1 Flash Lite (Reasoning)1001001001006092.0%
Claude Sonnet 5 (Reasoning)1001001001005891.7%
GPT-5.21001001001005691.2%
GPT-5.4 Nano (Reasoning)1001001001005691.2%
Grok 4.20 (Reasoning)1001001001005691.1%
GPT-5 Mini1001001001005591.0%
Laguna XS 2.11001001001005490.8%
Claude Opus 4.7 (Reasoning)1001001001005390.5%
GPT-51001001001005390.5%
GPT-5.11001001001005190.3%
Claude Opus 4.8 (Reasoning)1001001001005090.0%
Grok 4.5 (Reasoning, Low)1001001001005090.0%
Gemini 2.5 Flash100100100925890.0%
GPT-4.1 Nano1001001001005090.0%
Claude Opus 4.8 (Reasoning, Low)1001001001004789.5%
Qwen 3.6 35B1001001001004689.2%
MiniMax M2.5100100100935389.2%
GPT-4.11001001001004589.1%
Hermes 3 70B100100100716086.3%
Gemini 3.1 Flash Lite100100100606084.0%
Z.AI GLM 4.5 Air100100100675283.8%
Claude Sonnet 5 (Reasoning, Low)100100100625683.6%
GPT-5.4 (Reasoning, Low)100100100595783.2%
GPT-5.4 Mini (Reasoning)100100100605582.9%
Claude Haiku 4.5100100100605382.5%
Gemini 3.6 Flash (Reasoning)100100100565582.2%
Gemini 3.5 Flash Lite (Reasoning)10010071676781.0%
GPT-5.4 Nano10010098555080.7%
Gemini 2.5 Pro100100100100080.0%
Claude Sonnet 5100100100505080.0%
Xiaomi MIMO v2.5 Pro100100100100080.0%
DeepSeek V4 Flash100100100505080.0%
Mistral NeMO100100100100080.0%
Claude Sonnet 4.6 (Reasoning)100100100504779.5%
Mistral Small 4 (Reasoning)10010091505078.2%
Claude Sonnet 4939292575377.7%
Gemma 4 26B (Reasoning)10010067605877.0%
GPT-5.4 (Reasoning)10010065585776.0%
Gemini 3 Flash (Preview, Reasoning)10010063585775.6%
Gemini 3.1 Flash Lite (Preview)10010067575074.8%
Gemma 4 26B1008870605674.6%
Grok 4.310010063575374.6%
Aion 2.010010063605074.5%
Gemini 3.1 Pro (Preview)10010067535074.0%
MiniMax M310010056545372.7%
Qwen 3.5 Plus (2026-04-20)10010057505071.4%
Z.AI GLM 4.7 Flash10010057505071.4%
GPT-5.4 Nano (Reasoning, Low)1009955534971.1%
MoonshotAI: Kimi K2.510010054544771.0%
Gemini 2.5 Flash Lite937864606070.9%
Qwen 3.6 Flash10010055505070.9%
Qwen3.7 Max10010052505070.5%
ByteDance Seed 2.0 Mini1006767605770.1%
Qwen 3.5 Plus (2026-02-15)10010054504670.0%
ByteDance Seed 2.0 Lite10010057504370.0%
Thinking Machines Inkling10010052504669.7%
Gemini 3.5 Flash (Reasoning)1006462625869.1%
DeepSeek V4 Flash (Reasoning)10010050504569.1%
Cohere Command R+ (Aug. 2024)1001008855068.4%
MoonshotAI: Kimi K2.610010050454367.7%
Gemma 3 27B1008750505067.3%
Llama 3.1 70B1006057565465.3%
Gemma 3 12B1008155533865.2%
Grok 4.20936762555065.1%
Qwen 3.5 35B1006258535265.0%
Gemini 2.5 Flash Lite (Reasoning)1006756505064.4%
Claude Opus 4.61005955545264.0%
Xiaomi MIMO v2.5956359505063.2%
Gemma 4 31B (Reasoning)676464645863.2%
Claude Opus 4.6 (Reasoning)1005552525262.0%
Arcee AI: Trinity Mini1001005550060.9%
Qwen 3 32B786355545460.5%
DeepSeek V4 Pro (Reasoning)1005450504760.1%
Thinking Machines Inkling (Reasoning)1005050504759.3%
Qwen 2.5 72B1005348464658.6%
Z.AI GLM 51005050474458.3%
WizardLM 2 8x22b100646357056.8%
Qwen 3.5 9B100765850056.8%
ByteDance Seed 1.6636055555056.3%
Qwen 3.5 122B575656555555.8%
Qwen 3.5 27B705653504755.3%
MiniMax M2.7685555504754.8%
Z.AI GLM 4.7585554545354.6%
Claude Opus 4785550474254.3%
Mistral Large 3605750505053.4%
DeepSeek V3.2100100670053.3%
Ministral 3B605350505052.7%
Claude Sonnet 4.5625350504752.4%
DeepSeek V3 (2025-03-24)100100560051.1%
Qwen 3.6 27B575250484750.9%
Qwen 3.5 397B A17B535050504750.1%
Qwen 3.5 Flash535250474449.4%
ByteDance Seed 1.6 Flash58565453044.2%
Mistral Small 3.2 24B59555050042.7%
Mistral Large 210056550042.2%
Gemma 3 4B58505045040.8%
DeepSeek V3 (2024-12-26)10010000040.0%
Qwen3 235B A22B Instruct 250710010000040.0%
Ministral 3 8B10010000040.0%
Cydonia 24B V4.11009300038.6%
Mistral Small 46057560034.7%
Ministral 8B6456500034.0%
DeepSeek V4 Pro1006000032.0%
Ministral 3 3B675600024.4%
Mistral Medium 3.1675000023.3%
Writer: Palmyra X5000000.0%
Ministral 3 14B000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
o4 Mini High100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Z.AI GLM 4.6100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Aion 2.0100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 Flash100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Gemma 4 26B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Mistral Large 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Laguna XS 2.1100100100100100100.0%
GPT-5.6 Sol (Reasoning)1001001001009999.7%
GPT-5.4 (Reasoning, Low)1001001001009899.7%
GPT-5.4 Nano (Reasoning)1001001001009899.6%
GPT-5.6 Terra (Reasoning)1001001001009899.5%
Claude Opus 5 (Reasoning)1001001001009899.5%
GPT-51001001001009899.5%
GPT-5.210010099999899.1%
GPT-5.1100100100999799.1%
Claude Sonnet 4.6 (Reasoning)1001001001009599.1%
Claude Opus 51001001001009599.1%
Z.AI GLM 5 Turbo1001001001009599.0%
Gemini 2.5 Pro1001001001009598.9%
Qwen 3.6 35B1001001001009498.9%
GPT-5.6 Sol100100100989698.9%
GPT-5.4100100100999598.8%
Qwen 3.5 9B1001001001009498.8%
GPT-5.6 Terra10010099989798.7%
Z.AI GLM 5.11001001001009398.7%
Claude Opus 4.8 (Reasoning, Low)1001001001009398.7%
MiniMax M31001001001009398.7%
Claude Opus 5 (Reasoning, Low)100100100979798.6%
Thinking Machines Inkling (Reasoning)1001001001009398.6%
Claude Opus 4.51001001001009398.6%
Qwen 3.5 122B100100100969698.5%
MiniMax M2.5100100100979698.5%
DeepSeek V4 Flash (Reasoning)1001001001009298.5%
Grok 4.201001001001009298.5%
Z.AI GLM 4.51001001001009298.3%
GPT-OSS 120B1001001001009298.3%
Nemotron 3 Super1001001001009298.3%
GPT-5.5 (Reasoning)1009998989698.2%
DeepSeek V4 Flash1001001001009198.2%
Inception Mercury 21001001001009198.2%
ByteDance Seed 2.0 Lite1001001001009098.0%
ByteDance Seed 2.0 Mini1001001001008997.8%
Gemini 2.5 Flash Lite (Reasoning)1001001001008997.8%
Gemma 3 12B1001001001008997.8%
Thinking Machines Inkling1001001001008997.8%
GPT-5.5 (Reasoning, Low)999897979797.5%
MiniMax M2.710010098969497.5%
Claude Sonnet 4100100100949297.2%
ByteDance Seed 1.6 Flash100100100939296.9%
GPT-5.6 Luna (Reasoning)989797979696.9%
GPT-5.5989897979596.9%
Mistral Large 3100100100938395.2%
GPT-5.6 Luna989696939195.0%
Gemini 2.5 Flash (Reasoning)10010095918994.9%
Xiaomi MIMO v2.5 Pro1001001001007494.9%
Aion 3.0 Mini1001001001007394.7%
Aion 3.01001001001007094.1%
Gemini 2.5 Flash Lite10010092908693.5%
Claude Haiku 4.510010094937893.0%
Qwen 3.5 35B100100100956792.3%
Claude Opus 4949392928591.4%
GPT-4.1 Nano1001001001005090.0%
Gemma 3 27B939392868589.9%
Hermes 3 70B100100100915889.8%
Qwen 2.5 72B100100100945389.4%
Qwen 3.6 27B10010094886489.1%
Llama 3.1 70B10010093836788.6%
Nemotron 3 Nano100100100835086.7%
Gemini 3.5 Flash Lite (Reasoning, Minimal)10010092676083.7%
Cohere Command R+ (Aug. 2024)10010078706081.6%
Grok 4.5 (Reasoning, Low)100100100100080.0%
DeepSeek V3 (2024-12-26)100100100100080.0%
Laguna S 2.1100100100505080.0%
Qwen 3 32B100100100100080.0%
Mistral NeMO100100100100080.0%
GPT-5.4 Nano10010010097079.5%
DeepSeek V4 Pro10010010093078.6%
Z.AI GLM 4.7 Flash10010095504377.6%
Ministral 3 8B10010010086077.1%
Mistral Small 4 (Reasoning)10010010073074.5%
Ministral 3B1008664575672.5%
Gemma 3 4B1001008573071.5%
WizardLM 2 8x22b10010010056071.1%
Ministral 8B100916454061.8%
Cydonia 24B V4.1100946350061.4%
Writer: Palmyra X51001001000060.0%
Qwen3 235B A22B Instruct 25071001001000060.0%
Arcee AI: Trinity Mini1001001000060.0%
DeepSeek V3 (2025-03-24)100100890057.8%
Mistral Small 48581710047.3%
Ministral 3 3B1009300038.6%
Mistral Medium 3.15050500030.0%
Ministral 3 14B1005000030.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
o4 Mini100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Nemotron 3 Super100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Laguna XS 2.1100100100100100100.0%
Thinking Machines Inkling100100100100100100.0%
GPT-4.1 Nano100100100100100100.0%
GPT-5.6 Sol1001001001009899.6%
GPT-5.4 Nano (Reasoning, Low)100100100999899.4%
GPT-5.4 (Reasoning)1001001001009799.3%
GPT-5.4 Nano (Reasoning)100100100989899.2%
Claude Opus 4.71001001001009699.2%
MiniMax M2.51001001001009699.2%
Claude Opus 4.6 (Reasoning)1001001001009699.2%
GPT-5.4100100100999799.2%
Claude Sonnet 4.6 (Reasoning)1001001001009699.1%
Z.AI GLM 5.2 (Reasoning, High)1001001001009699.1%
Laguna S 2.11001001001009599.1%
GPT-5.1100100100989799.1%
Claude Opus 4.7 (Reasoning)1001001001009599.0%
GPT-5.6 Luna10010099999899.0%
Qwen 3.6 Flash1001001001009598.9%
Claude Opus 51001001001009498.9%
Gemini 2.5 Pro1001001001009498.9%
Claude Opus 4.8 (Reasoning)1001001001009498.8%
o4 Mini High1001001001009498.8%
GPT-5.4 Nano10010099989798.8%
GPT-5.21009999999798.7%
Gemini 2.5 Flash1001001001009398.6%
GPT-5.6 Terra10010098979698.3%
Claude Opus 5 (Reasoning)100100100969598.3%
GPT-5.4 Mini (Reasoning)1001001001009298.3%
Thinking Machines Inkling (Reasoning)1001001001009198.2%
GPT-5 Nano1001001001009198.2%
Claude Opus 5 (Reasoning, Low)10010098979598.1%
Qwen 3 32B1001001001008997.8%
GPT-5.5999997979797.7%
GPT-5.6 Luna (Reasoning)999999979697.7%
Qwen 3.5 Flash1001001001008897.5%
Gemini 2.5 Flash (Reasoning)100100100959297.4%
GPT-5.5 (Reasoning, Low)999897979697.4%
Claude Opus 4.610010096959597.3%
Claude Sonnet 5 (Reasoning, Low)100100100939297.0%
Gemma 3 27B10010094949396.2%
Gemini 3.5 Flash Lite (Reasoning, Minimal)1001001001008096.0%
Z.AI GLM 4.61009696959395.9%
Qwen 3.5 122B10010094949095.8%
Z.AI GLM 5.1100100100908995.8%
Qwen 2.5 72B100100100898995.6%
Gemma 3 12B1001001001007695.3%
Claude Sonnet 410010093928694.2%
Aion 3.01001001001007194.1%
DeepSeek V3.11001001001006893.7%
Hermes 3 70B100100100888093.5%
Claude Opus 41009393928692.7%
Nemotron 3 Nano100100100887592.5%
GPT-5.4 Mini10010096828091.7%
Qwen 3.5 35B1001001001005791.3%
Qwen 3.5 9B10010094907391.3%
Mistral Large 310010093906789.9%
Aion 2.010010095945689.0%
MiniMax M3100100100855688.2%
Llama 3.1 70B10010086757386.7%
Qwen 3.5 Plus (2026-02-15)100100100755686.3%
Arcee AI: Trinity Mini100100100755085.0%
ByteDance Seed 1.6 Flash949389895083.0%
Claude Haiku 4.510010082656382.0%
Xiaomi MIMO v2.5100100100100080.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100080.0%
DeepSeek V4 Pro100100100100080.0%
DeepSeek V4 Flash100100100100080.0%
Cohere Command R+ (Aug. 2024)10010090555379.6%
Xiaomi MIMO v2.5 Pro10010010092078.5%
WizardLM 2 8x22b10010010067073.3%
Gemini 2.5 Flash Lite100918780071.5%
Mistral Large 210010010050070.0%
Gemma 3 4B1001008362069.0%
Mistral NeMO1001009240066.3%
Z.AI GLM 4.7 Flash1001006764066.1%
Mistral Medium 3.11006750505063.3%
Qwen3 235B A22B Instruct 25071001001000060.0%
Ministral 3 3B1001001000060.0%
MiniMax M2.7100100950058.9%
Ministral 3 8B100100940058.8%
Ministral 3B865755504658.7%
Writer: Palmyra X5100100930058.7%
DeepSeek V3 (2025-03-24)100100880057.5%
Ministral 8B10082590048.2%
Cydonia 24B V4.11009500038.9%
Mistral Small 4 (Reasoning)1008900037.8%
Mistral Small 46460580036.4%
Ministral 3 14B100000020.0%