Parse dialogue

Test: Language Writing

Avg. Score
88.7%
Scenarios
5

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1GPT-4o Mini (temp=0)100.0%$0.00034.8s100%
2GPT-4o Mini (temp=1)100.0%$0.00035.6s100%
3Inception Mercury 299.6%$0.00061.4s96%
4Gemini 3 Flash (Preview)100.0%$0.00205.6s100%
5GPT-5.4 Mini (Reasoning, Low)99.9%$0.00223.5s99%
6DeepSeek-V2 Chat100.0%$0.000116.1s100%
7GPT-4.1 Mini99.3%$0.00043.4s93%
8GPT-4o, Aug. 6th (temp=0)100.0%$0.00526.1s100%
9Z.AI GLM 4.599.7%$0.001314.5s97%
10Gemini 3.5 Flash (Reasoning, Minimal)100.0%$0.00735.3s100%
11Z.AI GLM 5 Turbo99.8%$0.003714.7s98%
12Hermes 3 405B99.1%$0.000021.0s94%
13Gemma 4 31B100.0%$0.000332.1s100%
14GPT-5.4 Mini97.5%$0.00202.5s87%
15GPT-4o, Aug. 6th (temp=1)99.4%$0.00566.5s94%
16GPT-OSS 120B99.4%$0.000328.0s96%
17Gemini 3.1 Flash Lite (Reasoning)98.4%$0.00115.3s84%
18Nemotron 3 Super97.8%$0.000021.7s91%
19o4 Mini100.0%$0.007116.7s100%
20GPT-5.4 Nano (Reasoning)98.0%$0.00166.1s83%
21Gemini 2.5 Flash97.5%$0.00266.0s83%
22DeepSeek V3.198.7%$0.000722.8s88%
23Claude Sonnet 4.6100.0%$0.01013.7s100%
24Gemini 3.1 Flash Lite96.8%$0.00116.2s78%
25Nemotron 3 Nano95.3%$0.000210.8s77%
26GPT-4.197.8%$0.00416.8s79%
27MiniMax M2.597.1%$0.001321.5s81%
28Gemini 3.1 Flash Lite (Preview)95.0%$0.00113.7s72%
29Z.AI GLM 5.198.1%$0.004630.6s92%
30Grok 4.394.5%$0.00093.8s72%
31Z.AI GLM 5.2 (Reasoning, High)99.6%$0.007234.7s98%
32Gemini 2.5 Flash (Reasoning)97.1%$0.006713.1s86%
33GPT-5.4 Mini (Reasoning)96.2%$0.00467.3s77%
34GPT-4.1 Nano92.9%$0.00014.0s69%
35Aion 3.0 Mini97.4%$0.002337.9s87%
36GPT-5.4 Nano (Reasoning, Low)93.7%$0.00187.0s69%
37Gemini 3 Flash (Preview, Reasoning)94.9%$0.005111.8s74%
38Grok 4.2092.7%$0.002110.9s69%
39GPT-5 Nano99.4%$0.00281.2m96%
40Claude Opus 4.599.3%$0.01915.7s96%
41Z.AI GLM 4.5 Air95.1%$0.001230.1s73%
42Gemma 4 26B94.0%$0.000336.3s74%
43GPT-5.498.0%$0.01319.0s85%
44GPT-5 Mini98.0%$0.006637.2s82%
45o4 Mini High99.5%$0.01536.6s97%
46Claude Haiku 4.588.7%$0.00377.8s67%
47Qwen 3.6 35B97.1%$0.005938.3s79%
48Claude Sonnet 5 (Reasoning)98.1%$0.01419.0s84%
49GPT-5.4 (Reasoning, Low)96.6%$0.01216.1s77%
50GPT-5.4 Nano91.6%$0.00176.6s55%
51Claude Sonnet 492.6%$0.009211.1s71%
52Gemma 3 27B89.4%$0.000327.9s64%
53Aion 3.093.7%$0.008427.0s76%
54Gemma 3 12B90.2%$0.000233.9s66%
55Claude Sonnet 596.0%$0.01217.0s73%
56Qwen 3.5 Plus (2026-02-15)90.2%$0.001927.2s65%
57Gemma 4 26B (Reasoning)95.4%$0.000858.3s75%
58Grok 4.3 (Reasoning)100.0%$0.0121.2m100%
59Llama 3.1 70B85.4%$0.000720.5s62%
60Gemini 2.5 Flash Lite85.5%$0.00055.1s53%
61Grok 4.20 (Reasoning)98.2%$0.009749.7s83%
62Hermes 3 70B88.3%$0.000316.6s54%
63Claude Sonnet 5 (Reasoning, Low)96.1%$0.01520.4s78%
64GPT-5.297.4%$0.01626.0s82%
65Grok 4.5 (Reasoning, High)99.6%$0.01847.7s96%
66Aion 2.092.3%$0.003041.5s69%
67Mistral Large 384.0%$0.001417.3s59%
68Qwen 3.6 Flash93.7%$0.007730.0s69%
69DeepSeek V4 Flash (Reasoning)89.5%$0.000220.8s51%
70Xiaomi MIMO v2.587.7%$0.002314.2s51%
71Grok 4.5 (Reasoning, Low)94.0%$0.006024.5s57%
72Gemma 4 31B (Reasoning)92.6%$0.00051.1m70%
73DeepSeek V4 Flash87.0%$0.000212.4s42%
74Claude Sonnet 4.589.8%$0.01113.2s62%
75Qwen 3.5 Flash88.9%$0.001942.8s60%
76ByteDance Seed 1.6 Flash82.5%$0.000614.6s48%
77ByteDance Seed 1.691.3%$0.004050.4s65%
78GPT-5.4 (Reasoning)94.8%$0.01730.6s74%
79GPT-5.5 (Reasoning)99.4%$0.03322.8s98%
80Xiaomi MIMO v2.5 Pro90.4%$0.003622.3s46%
81Claude Opus 4.799.6%$0.03818.6s98%
82Qwen 3.5 122B90.0%$0.01335.0s65%
83GPT-5.5 (Reasoning, Low)98.5%$0.03523.8s96%
84Qwen 3.5 27B91.0%$0.009648.7s63%
85Gemini 2.5 Flash Lite (Reasoning)83.7%$0.001923.9s41%
86GPT-5.598.3%$0.03626.1s96%
87Gemini 3.5 Flash (Reasoning)93.8%$0.02614.8s72%
88MiniMax M389.4%$0.00221.3m65%
89MoonshotAI: Kimi K2.594.2%$0.00821.2m68%
90Arcee AI: Trinity Mini81.2%$0.000215.9s32%
91DeepSeek V4 Pro (Reasoning)92.0%$0.00291.3m63%
92Qwen 3.5 Plus (2026-04-20)94.3%$0.0111.1m69%
93Z.AI GLM 4.693.2%$0.00441.2m60%
94ByteDance Seed 2.0 Lite93.6%$0.00691.3m67%
95Claude Opus 4.8 (Reasoning)97.8%$0.03220.8s80%
96Claude Opus 4.8 (Reasoning, Low)97.6%$0.03220.8s79%
97Claude Sonnet 4.6 (Reasoning)95.2%$0.02531.1s72%
98Z.AI GLM 589.1%$0.00591.3m63%
99DeepSeek V3.290.0%$0.00041.1m45%
100Gemma 3 4B74.6%$0.000113.4s29%
101Gemini 2.5 Pro95.1%$0.02422.5s61%
102GPT-5.197.3%$0.02752.6s81%
103Qwen 3.5 35B83.9%$0.01033.8s49%
104Z.AI GLM 4.7 Flash80.3%$0.001252.0s45%
105Qwen 3 32B79.2%$0.000635.4s33%
106Cohere Command R+ (Aug. 2024)73.2%$0.006213.4s38%
107Ministral 3B59.5%$0.00002.9s30%
108Mistral Small 3.2 24B70.5%$0.000311.0s20%
109Claude Opus 4.7 (Reasoning)97.5%$0.04319.7s81%
110Qwen 3.5 9B86.4%$0.00111.6m51%
111Mistral Small 4 (Reasoning)71.1%$0.001115.3s21%
112DeepSeek V3 (2025-03-24)72.8%$0.000514.6s16%
113DeepSeek V3 (2024-12-26)75.8%$0.000719.4s15%
114Z.AI GLM 4.790.9%$0.00541.9m64%
115Qwen 2.5 72B67.9%$0.000419.8s20%
116MiniMax M2.769.6%$0.002128.4s26%
117Mistral NeMO66.6%$0.00014.3s10%
118DeepSeek V4 Pro75.6%$0.002035.4s22%
119Gemini 3.1 Pro (Preview)94.8%$0.03740.3s71%
120ByteDance Seed 2.0 Mini90.2%$0.00232.5m66%
121Claude Opus 4.692.3%$0.03634.3s67%
122Mistral Large 270.4%$0.004218.6s16%
123WizardLM 2 8x22b61.1%$0.000815.6s12%
124Qwen 3.6 27B88.0%$0.0171.5m60%
125Ministral 8B52.8%$0.00016.6s14%
126Mistral Small 448.9%$0.00046.4s17%
127Claude Opus 4.6 (Reasoning)92.2%$0.04038.1s65%
128Qwen 3.5 397B A17B90.0%$0.0152.0m60%
129Mistral Medium 3.149.0%$0.002016.7s17%
130GPT-598.0%$0.0441.3m81%
131Ministral 3 8B47.9%$0.00026.1s3%
132Qwen3.7 Max94.1%$0.0361.3m68%
133Qwen3.6 Max Preview100.0%$0.0362.5m100%
134Cydonia 24B V4.150.0%$0.000623.3s5%
135Claude Opus 486.0%$0.04824.7s61%
136Ministral 3 3B36.2%$0.00012.6s0%
137Qwen3 235B A22B Instruct 250746.7%$0.000422.5s0%
138Writer: Palmyra X543.2%$0.005113.6s0%
139MoonshotAI: Kimi K2.693.5%$0.0192.9m65%
140Ministral 3 14B10.0%$0.00039.0s0%
88.72%

Individual Scenarios

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
GPT-5.1100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
o4 Mini High100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Z.AI GLM 4.6100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
MiniMax M2.5100100100100100100.0%
Aion 2.0100100100100100100.0%
o4 Mini100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-5.4100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Inception Mercury 2100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
GPT-5.4 Nano100100100100100100.0%
Hermes 3 70B100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100.0%
GPT-5.21001001001009999.8%
GPT-5.4 Nano (Reasoning, Low)1001001001009899.7%
GPT-5.5 (Reasoning, Low)1001001001009899.6%
GPT-5.5 (Reasoning)1001001001009899.5%
Aion 3.01001001001009699.3%
Z.AI GLM 5.11001001001009699.1%
Claude Sonnet 4.6 (Reasoning)1001001001009699.1%
Gemini 2.5 Pro1001001001009599.1%
Z.AI GLM 5.2 (Reasoning, High)1001001001009599.0%
GPT-5.4 (Reasoning)100100100989799.0%
Xiaomi MIMO v2.51001001001009598.9%
GPT-5 Mini1001001001009498.9%
Claude Sonnet 5 (Reasoning)1001001001009498.8%
Qwen 3.5 Flash1001001001009498.8%
Gemini 2.5 Flash1001001001009498.8%
GPT-5 Nano1001001001009398.6%
Gemma 3 27B1001001001009398.6%
GPT-5.51009999989798.5%
GPT-OSS 120B1001001001009298.5%
DeepSeek V3.21001001001009298.5%
Gemini 2.5 Flash Lite (Reasoning)1001001001009298.3%
Gemma 3 12B1001001001009298.3%
Grok 4.5 (Reasoning, High)1001001001009198.2%
Claude Opus 4.7 (Reasoning)1001001001009098.1%
Qwen 3.6 35B100100100949397.5%
Nemotron 3 Nano1001001001008697.1%
Nemotron 3 Super1001001001008596.9%
Claude Sonnet 4100100100939196.8%
Z.AI GLM 4.7 Flash100100100949096.8%
GPT-4.1 Mini1001001001008296.4%
ByteDance Seed 1.6 Flash100100100919096.2%
Gemma 3 4B1001001001008096.0%
GPT-5.4 Mini1001001001007995.8%
DeepSeek V4 Pro1001001001007595.0%
Qwen 3.5 Plus (2026-02-15)1009494939394.8%
Aion 3.0 Mini100100100888694.6%
Mistral Small 4 (Reasoning)100100100928194.6%
Claude Opus 41009494928993.9%
Gemini 2.5 Flash Lite10010093928393.8%
Claude Haiku 4.5100100100917793.6%
Llama 3.1 70B10010092918593.6%
GPT-4.1 Nano100100100907392.5%
Z.AI GLM 51009490888691.7%
Qwen 3.5 35B100100100965990.9%
Qwen 3.5 9B1009695935687.9%
MiniMax M3100100100716787.6%
ByteDance Seed 2.0 Mini100100100864085.1%
Mistral Large 31009280766783.1%
DeepSeek V4 Flash (Reasoning)100100100100080.0%
Mistral Large 2100100100100080.0%
DeepSeek V3 (2024-12-26)10010010094078.8%
Qwen 3 32B10010010089077.8%
Writer: Palmyra X510010010086077.1%
Cydonia 24B V4.110010010069073.8%
Mistral Medium 3.11007567505068.3%
Ministral 8B1001007367067.9%
Cohere Command R+ (Aug. 2024)927065635067.8%
Ministral 3B1001007150064.3%
MiniMax M2.7100956150061.2%
Mistral NeMO1001001000060.0%
Qwen 2.5 72B10095850056.0%
WizardLM 2 8x22b100100710054.3%
Ministral 3 8B100100500050.0%
Mistral Small 48571600043.1%
Mistral Small 3.2 24B1005000030.0%
Qwen3 235B A22B Instruct 2507100000020.0%
Ministral 3 14B000000.0%
Ministral 3 3B000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
MiniMax M3100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
Arcee AI: Trinity Mini100100100100100100.0%
GPT-5.41001001001009999.8%
GPT-5.4 (Reasoning)1001001001009899.7%
GPT-5.4 Mini (Reasoning, Low)1001001001009799.5%
GPT-5.4 Nano100100100989899.3%
GPT-5.5 (Reasoning)10010099999899.2%
Claude Opus 4.71001001001009599.0%
Claude Sonnet 4.6 (Reasoning)1001001001009598.9%
Claude Opus 4.51001001001009598.9%
Gemini 3 Flash (Preview, Reasoning)1001001001009498.8%
o4 Mini High1001001001009498.8%
Gemini 2.5 Pro1001001001009498.8%
Qwen 3.5 Flash1001001001009498.8%
Aion 3.0 Mini1001001001009498.8%
GPT-5.51009999989898.7%
MiniMax M2.51001001001009398.6%
Mistral Large 31001001001009398.6%
Xiaomi MIMO v2.5 Pro100100100979698.5%
GPT-5.4 Nano (Reasoning, Low)100100100989598.5%
Qwen 3.6 Flash1001001001009298.5%
DeepSeek V3.21001001001009298.3%
Nemotron 3 Super1001001001009298.3%
GPT-5.5 (Reasoning, Low)10010098979798.2%
Aion 2.01001001001009198.2%
ByteDance Seed 2.0 Mini1001001001009198.2%
Hermes 3 405B1001001001009198.2%
GPT-5.21009998979698.1%
GPT-5.11001001001009097.9%
DeepSeek V3 (2025-03-24)1001001001008997.8%
Gemini 2.5 Flash Lite1001001001008997.8%
Grok 4.31001001001008997.8%
Claude Opus 4100100100959397.7%
Claude Sonnet 4100100100939397.1%
Qwen 3.5 9B100100100949197.0%
Claude Sonnet 4.510010095949396.5%
Xiaomi MIMO v2.51001001001008096.1%
Gemma 3 4B100100100928695.6%
Z.AI GLM 510010095958895.6%
Gemma 4 26B1001001001007895.6%
Gemma 3 27B100100100928395.1%
Gemma 3 12B10010091919094.4%
Llama 3.1 70B100100100838092.7%
DeepSeek V4 Pro100100100837992.4%
Claude Haiku 4.510010094887992.2%
ByteDance Seed 1.6 Flash100100100808092.0%
GPT-4.1 Nano1001001001006092.0%
Z.AI GLM 4.5 Air1001001001005891.7%
Z.AI GLM 4.7 Flash100100100886289.8%
Aion 3.010010095875888.0%
Mistral Small 410010080736383.2%
Qwen 3.5 35B100100100100080.0%
DeepSeek V3 (2024-12-26)100100100100080.0%
Qwen 3 32B100100100100080.0%
Mistral Small 3.2 24B100100100100080.0%
Gemini 2.5 Flash Lite (Reasoning)10010010090078.0%
Z.AI GLM 4.61001009594077.8%
DeepSeek V4 Flash1001009292076.8%
MiniMax M2.71001009286075.5%
Hermes 3 70B10010010060072.0%
Mistral Small 4 (Reasoning)1001009853070.2%
Cohere Command R+ (Aug. 2024)867570625068.5%
Mistral Large 21001001000060.0%
Mistral Medium 3.11001005050060.0%
Ministral 3 3B100100900058.0%
Qwen3 235B A22B Instruct 2507100100670053.3%
Ministral 8B10092690052.2%
WizardLM 2 8x22b10088630050.0%
Ministral 3B67645858049.3%
Mistral NeMO100100330046.7%
Qwen 2.5 72B10010000040.0%
Cydonia 24B V4.11008600037.1%
Writer: Palmyra X5100000020.0%
Ministral 3 8B68000013.7%
Ministral 3 14B000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Grok 4.5 (Reasoning, High)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
o4 Mini High100100100100100100.0%
o4 Mini100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
Nemotron 3 Nano100100100100100100.0%
GPT-5.5 (Reasoning, Low)1001001001009999.7%
GPT-5.51001001001009899.6%
Claude Opus 4.51001001001009599.0%
Aion 3.0 Mini1001001001009498.7%
Hermes 3 405B1001001001008897.5%
Z.AI GLM 5.11001001001008697.1%
GPT-4o, Aug. 6th (temp=1)1001001001008697.1%
Nemotron 3 Super100100100908895.5%
Gemini 2.5 Flash (Reasoning)1001001001006793.3%
Aion 3.0100100100966993.0%
Z.AI GLM 4.6100100100956792.3%
GPT-5.41001001001006192.2%
Gemini 3.1 Flash Lite (Reasoning)1001001001006092.0%
Claude Sonnet 5 (Reasoning)1001001001005891.7%
GPT-5.21001001001005691.2%
GPT-5.4 Nano (Reasoning)1001001001005691.2%
Grok 4.20 (Reasoning)1001001001005691.1%
GPT-5 Mini1001001001005591.0%
Claude Opus 4.7 (Reasoning)1001001001005390.5%
GPT-51001001001005390.5%
GPT-5.11001001001005190.3%
Claude Opus 4.8 (Reasoning)1001001001005090.0%
Grok 4.5 (Reasoning, Low)1001001001005090.0%
Gemini 2.5 Flash100100100925890.0%
GPT-4.1 Nano1001001001005090.0%
Claude Opus 4.8 (Reasoning, Low)1001001001004789.5%
Qwen 3.6 35B1001001001004689.2%
MiniMax M2.5100100100935389.2%
GPT-4.11001001001004589.1%
Hermes 3 70B100100100716086.3%
Gemini 3.1 Flash Lite100100100606084.0%
Z.AI GLM 4.5 Air100100100675283.8%
Claude Sonnet 5 (Reasoning, Low)100100100625683.6%
GPT-5.4 (Reasoning, Low)100100100595783.2%
GPT-5.4 Mini (Reasoning)100100100605582.9%
Claude Haiku 4.5100100100605382.5%
GPT-5.4 Nano10010098555080.7%
Gemini 2.5 Pro100100100100080.0%
Claude Sonnet 5100100100505080.0%
Xiaomi MIMO v2.5 Pro100100100100080.0%
DeepSeek V4 Flash100100100505080.0%
Mistral NeMO100100100100080.0%
Claude Sonnet 4.6 (Reasoning)100100100504779.5%
Mistral Small 4 (Reasoning)10010091505078.2%
Claude Sonnet 4939292575377.7%
Gemma 4 26B (Reasoning)10010067605877.0%
GPT-5.4 (Reasoning)10010065585776.0%
Gemini 3 Flash (Preview, Reasoning)10010063585775.6%
Gemini 3.1 Flash Lite (Preview)10010067575074.8%
Gemma 4 26B1008870605674.6%
Grok 4.310010063575374.6%
Aion 2.010010063605074.5%
Gemini 3.1 Pro (Preview)10010067535074.0%
MiniMax M310010056545372.7%
Qwen 3.5 Plus (2026-04-20)10010057505071.4%
Z.AI GLM 4.7 Flash10010057505071.4%
GPT-5.4 Nano (Reasoning, Low)1009955534971.1%
MoonshotAI: Kimi K2.510010054544771.0%
Gemini 2.5 Flash Lite937864606070.9%
Qwen 3.6 Flash10010055505070.9%
Qwen3.7 Max10010052505070.5%
ByteDance Seed 2.0 Mini1006767605770.1%
Qwen 3.5 Plus (2026-02-15)10010054504670.0%
ByteDance Seed 2.0 Lite10010057504370.0%
Gemini 3.5 Flash (Reasoning)1006462625869.1%
DeepSeek V4 Flash (Reasoning)10010050504569.1%
Cohere Command R+ (Aug. 2024)1001008855068.4%
MoonshotAI: Kimi K2.610010050454367.7%
Gemma 3 27B1008750505067.3%
Llama 3.1 70B1006057565465.3%
Gemma 3 12B1008155533865.2%
Grok 4.20936762555065.1%
Qwen 3.5 35B1006258535265.0%
Gemini 2.5 Flash Lite (Reasoning)1006756505064.4%
Claude Opus 4.61005955545264.0%
Xiaomi MIMO v2.5956359505063.2%
Gemma 4 31B (Reasoning)676464645863.2%
Claude Opus 4.6 (Reasoning)1005552525262.0%
Arcee AI: Trinity Mini1001005550060.9%
Qwen 3 32B786355545460.5%
DeepSeek V4 Pro (Reasoning)1005450504760.1%
Qwen 2.5 72B1005348464658.6%
Z.AI GLM 51005050474458.3%
WizardLM 2 8x22b100646357056.8%
Qwen 3.5 9B100765850056.8%
ByteDance Seed 1.6636055555056.3%
Qwen 3.5 122B575656555555.8%
Qwen 3.5 27B705653504755.3%
MiniMax M2.7685555504754.8%
Z.AI GLM 4.7585554545354.6%
Claude Opus 4785550474254.3%
Mistral Large 3605750505053.4%
DeepSeek V3.2100100670053.3%
Ministral 3B605350505052.7%
Claude Sonnet 4.5625350504752.4%
DeepSeek V3 (2025-03-24)100100560051.1%
Qwen 3.6 27B575250484750.9%
Qwen 3.5 397B A17B535050504750.1%
Qwen 3.5 Flash535250474449.4%
ByteDance Seed 1.6 Flash58565453044.2%
Mistral Small 3.2 24B59555050042.7%
Mistral Large 210056550042.2%
Gemma 3 4B58505045040.8%
DeepSeek V3 (2024-12-26)10010000040.0%
Qwen3 235B A22B Instruct 250710010000040.0%
Ministral 3 8B10010000040.0%
Cydonia 24B V4.11009300038.6%
Mistral Small 46057560034.7%
Ministral 8B6456500034.0%
DeepSeek V4 Pro1006000032.0%
Ministral 3 3B675600024.4%
Mistral Medium 3.1675000023.3%
Writer: Palmyra X5000000.0%
Ministral 3 14B000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
o4 Mini High100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Z.AI GLM 4.6100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
Aion 2.0100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 Flash100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Gemma 4 26B100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Mistral Large 2100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
GPT-5.4 Mini100100100100100100.0%
GPT-5 Nano100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
GPT-5.4 (Reasoning, Low)1001001001009899.7%
GPT-5.4 Nano (Reasoning)1001001001009899.6%
GPT-51001001001009899.5%
GPT-5.210010099999899.1%
GPT-5.1100100100999799.1%
Claude Sonnet 4.6 (Reasoning)1001001001009599.1%
Z.AI GLM 5 Turbo1001001001009599.0%
Gemini 2.5 Pro1001001001009598.9%
Qwen 3.6 35B1001001001009498.9%
GPT-5.4100100100999598.8%
Qwen 3.5 9B1001001001009498.8%
Z.AI GLM 5.11001001001009398.7%
Claude Opus 4.8 (Reasoning, Low)1001001001009398.7%
MiniMax M31001001001009398.7%
Claude Opus 4.51001001001009398.6%
Qwen 3.5 122B100100100969698.5%
MiniMax M2.5100100100979698.5%
DeepSeek V4 Flash (Reasoning)1001001001009298.5%
Grok 4.201001001001009298.5%
Z.AI GLM 4.51001001001009298.3%
GPT-OSS 120B1001001001009298.3%
Nemotron 3 Super1001001001009298.3%
GPT-5.5 (Reasoning)1009998989698.2%
DeepSeek V4 Flash1001001001009198.2%
Inception Mercury 21001001001009198.2%
ByteDance Seed 2.0 Lite1001001001009098.0%
ByteDance Seed 2.0 Mini1001001001008997.8%
Gemini 2.5 Flash Lite (Reasoning)1001001001008997.8%
Gemma 3 12B1001001001008997.8%
GPT-5.5 (Reasoning, Low)999897979797.5%
MiniMax M2.710010098969497.5%
Claude Sonnet 4100100100949297.2%
ByteDance Seed 1.6 Flash100100100939296.9%
GPT-5.5989897979596.9%
Mistral Large 3100100100938395.2%
Gemini 2.5 Flash (Reasoning)10010095918994.9%
Xiaomi MIMO v2.5 Pro1001001001007494.9%
Aion 3.0 Mini1001001001007394.7%
Aion 3.01001001001007094.1%
Gemini 2.5 Flash Lite10010092908693.5%
Claude Haiku 4.510010094937893.0%
Qwen 3.5 35B100100100956792.3%
Claude Opus 4949392928591.4%
GPT-4.1 Nano1001001001005090.0%
Gemma 3 27B939392868589.9%
Hermes 3 70B100100100915889.8%
Qwen 2.5 72B100100100945389.4%
Qwen 3.6 27B10010094886489.1%
Llama 3.1 70B10010093836788.6%
Nemotron 3 Nano100100100835086.7%
Cohere Command R+ (Aug. 2024)10010078706081.6%
Grok 4.5 (Reasoning, Low)100100100100080.0%
DeepSeek V3 (2024-12-26)100100100100080.0%
Qwen 3 32B100100100100080.0%
Mistral NeMO100100100100080.0%
GPT-5.4 Nano10010010097079.5%
DeepSeek V4 Pro10010010093078.6%
Z.AI GLM 4.7 Flash10010095504377.6%
Ministral 3 8B10010010086077.1%
Mistral Small 4 (Reasoning)10010010073074.5%
Ministral 3B1008664575672.5%
Gemma 3 4B1001008573071.5%
WizardLM 2 8x22b10010010056071.1%
Ministral 8B100916454061.8%
Cydonia 24B V4.1100946350061.4%
Writer: Palmyra X51001001000060.0%
Qwen3 235B A22B Instruct 25071001001000060.0%
Arcee AI: Trinity Mini1001001000060.0%
DeepSeek V3 (2025-03-24)100100890057.8%
Mistral Small 48581710047.3%
Ministral 3 3B1009300038.6%
Mistral Medium 3.15050500030.0%
Ministral 3 14B1005000030.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
GPT-4.1100100100100100100.0%
o4 Mini100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Nemotron 3 Super100100100100100100.0%
GPT-4.1 Mini100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100.0%
Grok 4.20100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100.0%
Grok 4.3100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100.0%
GPT-4.1 Nano100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100999899.4%
GPT-5.4 (Reasoning)1001001001009799.3%
GPT-5.4 Nano (Reasoning)100100100989899.2%
Claude Opus 4.71001001001009699.2%
MiniMax M2.51001001001009699.2%
Claude Opus 4.6 (Reasoning)1001001001009699.2%
GPT-5.4100100100999799.2%
Claude Sonnet 4.6 (Reasoning)1001001001009699.1%
Z.AI GLM 5.2 (Reasoning, High)1001001001009699.1%
GPT-5.1100100100989799.1%
Claude Opus 4.7 (Reasoning)1001001001009599.0%
Qwen 3.6 Flash1001001001009598.9%
Gemini 2.5 Pro1001001001009498.9%
Claude Opus 4.8 (Reasoning)1001001001009498.8%
o4 Mini High1001001001009498.8%
GPT-5.4 Nano10010099989798.8%
GPT-5.21009999999798.7%
Gemini 2.5 Flash1001001001009398.6%
GPT-5.4 Mini (Reasoning)1001001001009298.3%
GPT-5 Nano1001001001009198.2%
Qwen 3 32B1001001001008997.8%
GPT-5.5999997979797.7%
Qwen 3.5 Flash1001001001008897.5%
Gemini 2.5 Flash (Reasoning)100100100959297.4%
GPT-5.5 (Reasoning, Low)999897979697.4%
Claude Opus 4.610010096959597.3%
Claude Sonnet 5 (Reasoning, Low)100100100939297.0%
Gemma 3 27B10010094949396.2%
Z.AI GLM 4.61009696959395.9%
Qwen 3.5 122B10010094949095.8%
Z.AI GLM 5.1100100100908995.8%
Qwen 2.5 72B100100100898995.6%
Gemma 3 12B1001001001007695.3%
Claude Sonnet 410010093928694.2%
Aion 3.01001001001007194.1%
DeepSeek V3.11001001001006893.7%
Hermes 3 70B100100100888093.5%
Claude Opus 41009393928692.7%
Nemotron 3 Nano100100100887592.5%
GPT-5.4 Mini10010096828091.7%
Qwen 3.5 35B1001001001005791.3%
Qwen 3.5 9B10010094907391.3%
Mistral Large 310010093906789.9%
Aion 2.010010095945689.0%
MiniMax M3100100100855688.2%
Llama 3.1 70B10010086757386.7%
Qwen 3.5 Plus (2026-02-15)100100100755686.3%
Arcee AI: Trinity Mini100100100755085.0%
ByteDance Seed 1.6 Flash949389895083.0%
Claude Haiku 4.510010082656382.0%
Xiaomi MIMO v2.5100100100100080.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100080.0%
DeepSeek V4 Pro100100100100080.0%
DeepSeek V4 Flash100100100100080.0%
Cohere Command R+ (Aug. 2024)10010090555379.6%
Xiaomi MIMO v2.5 Pro10010010092078.5%
WizardLM 2 8x22b10010010067073.3%
Gemini 2.5 Flash Lite100918780071.5%
Mistral Large 210010010050070.0%
Gemma 3 4B1001008362069.0%
Mistral NeMO1001009240066.3%
Z.AI GLM 4.7 Flash1001006764066.1%
Mistral Medium 3.11006750505063.3%
Qwen3 235B A22B Instruct 25071001001000060.0%
Ministral 3 3B1001001000060.0%
MiniMax M2.7100100950058.9%
Ministral 3 8B100100940058.8%
Ministral 3B865755504658.7%
Writer: Palmyra X5100100930058.7%
DeepSeek V3 (2025-03-24)100100880057.5%
Ministral 8B10082590048.2%
Cydonia 24B V4.11009500038.9%
Mistral Small 4 (Reasoning)1008900037.8%
Mistral Small 46460580036.4%
Ministral 3 14B100000020.0%