Matches word count

Test: Dialogue tags

Avg. Score
49.1%
Scenarios
6

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1GPT-6.1 Sol (Reasoning, Medium)100.0%$0.01738.4s100%
2Muse Spark 1.2 (Reasoning, Medium)99.3%$0.02426.7s94%
3Claude Opus 4.8 (Reasoning)96.9%$0.02116.2s75%
4Claude Opus 4.8 (Reasoning, Low)96.1%$0.02115.6s74%
5DeepSeek V4.1 Flash (Reasoning, High)93.7%$0.005652.5s57%
6GPT-5 Mini95.2%$0.01158.5s58%
7Qwen 3.8 Flash (Reasoning)98.3%$0.00682.7m74%
8Muse Spark 1.3 (Reasoning, Medium)96.7%$0.0281.4m64%
9Muse Spark 1.1 (Reasoning, Medium)91.3%$0.01925.7s48%
10Claude Opus 4.686.8%$0.01415.5s44%
11Gemini 3.1 Flash Lite (Preview)78.4%$0.00072.9s36%
12Gemini 3.1 Flash Lite (Reasoning)79.8%$0.00074.2s33%
13GPT-4o Mini (temp=0)77.7%$0.00038.5s34%
14Gemini 3.8 Flash (Reasoning, Medium)88.0%$0.03224.7s43%
15Inception Mercury 281.6%$0.00426.8s28%
16Claude Sonnet 577.3%$0.007713.5s36%
17Qwen 3.8 Max (Reasoning, XHigh)100.0%$0.0643.8m100%
18Claude Sonnet 4.670.9%$0.007912.8s33%
19Claude Sonnet 5 (Reasoning)74.6%$0.008013.4s29%
20Claude Opus 4.774.7%$0.02012.0s36%
21Hy4 Preview (Reasoning, High)99.0%$0.0464.2m86%
22Claude Haiku 5.5 (Reasoning, Max)95.1%$0.0253.2m57%
23GPT-6 Sol (Reasoning, Medium)77.5%$0.01222.7s26%
24GPT-5.6 Sol (Reasoning, Medium)92.9%$0.07632.5s52%
25Gemini 3.1 Flash Lite70.4%$0.00075.2s18%
26Claude Opus 4.7 (Reasoning)73.0%$0.02011.3s29%
27Claude Opus 5.5 (Reasoning)84.1%$0.04526.0s37%
28Muse Spark 1.1 (Reasoning, Minimal)76.5%$0.01015.7s19%
29Claude Opus 5 (Reasoning, Low)76.1%$0.02317.2s26%
30GPT-4o, Aug. 6th (temp=0)68.9%$0.00536.3s18%
31Claude Opus 573.9%$0.02016.2s24%
32Claude Opus 5 (Reasoning)73.0%$0.02418.0s27%
33Z.AI GLM 5 Turbo87.5%$0.0351.4m36%
34Z.AI GLM 5.3 Flash (Reasoning, Low)66.6%$0.000124.3s17%
35Claude Sonnet 5 (Reasoning, Low)64.5%$0.008114.0s20%
36Z.AI GLM 5.3 Flash (Reasoning, Max)87.1%$0.00263.6m43%
37Qwen 3.8 27B (Reasoning, XHigh)93.3%$0.0313.3m50%
38GPT-590.1%$0.0531.5m40%
39MiniMax M2.588.9%$0.0173.1m42%
40Claude Opus 4.567.4%$0.01414.3s19%
41GPT-6 Luna (Reasoning, Medium)66.9%$0.001124.9s13%
42GPT-4o, Aug. 6th (temp=1)64.6%$0.00546.3s12%
43GPT-6 Luna (Reasoning, High)71.9%$0.002448.4s12%
44Gemini 3 Flash (Preview)59.8%$0.00155.0s12%
45MiniMax M2.795.0%$0.0234.7m57%
46Nemotron 3 Super80.6%$0.00002.8m29%
47o4 Mini High88.5%$0.0501.9m40%
48GPT-6 Sol60.1%$0.005112.3s14%
49GPT-4.161.4%$0.00447.8s9%
50Grok 4.2056.4%$0.00139.3s12%
51Qwen 3.7 Max93.1%$0.0742.5m54%
52GPT-4o Mini (temp=1)59.1%$0.00038.4s7%
53Claude Opus 4.6 (Reasoning)84.2%$0.08042.3s39%
54Gemini 3.7 Flash (Reasoning, Medium)67.9%$0.02021.5s11%
55Xiaomi MIMO v2.6 Pro66.5%$0.00441.3m16%
56o4 Mini72.7%$0.0261.1m18%
57Gemini 3.6 Flash (Reasoning)78.5%$0.05931.6s24%
58Kimi K3 (Reasoning, Low)55.3%$0.01026.0s12%
59Gemini 3.5 Flash (Reasoning, Minimal)50.1%$0.00413.7s6%
60DeepSeek V4 Flash51.0%$0.000214.6s6%
61DeepSeek V4 Pro 0813 (Reasoning, High)71.9%$0.0251.4m14%
62Aion 3.5 Mini (Reasoning, High)73.0%$0.0102.5m18%
63GPT-5.4 Nano (Reasoning)54.7%$0.004221.2s1%
64Gemini 3.1 Pro (Preview)98.7%$0.1572.1m80%
65Z.AI GLM 5.2 (Reasoning, High)71.7%$0.0301.6m18%
66Gemini 3 Flash (Preview, Reasoning)58.0%$0.01933.7s7%
67Gemini 2.5 Flash Lite41.3%$0.00022.6s5%
68Muse Glimmer 30B (Reasoning, Medium)49.9%$0.002432.0s4%
69GPT-5.6 Terra (Reasoning, Medium)60.9%$0.02616.6s4%
70GPT-OSS 120B63.7%$0.00151.9m8%
71Claude Opus 459.0%$0.04425.5s18%
72 Kimi K3 (Reasoning, High)57.8%$0.02045.3s9%
73Mistral Medium 3.144.0%$0.001213.1s3%
74GPT-5.6 Luna (Reasoning, Medium)50.7%$0.01212.3s2%
75Aion 3.5 (Reasoning, High)49.6%$0.008830.8s6%
76Llama 3.1 70B40.6%$0.00055.2s3%
77Qwen 3.7 Flash (Reasoning)48.8%$0.001258.1s6%
78GPT-4.1 Mini40.2%$0.00096.9s1%
79Gemini 3.6 Flash (Reasoning, Minimal)41.1%$0.00354.2s1%
80Claude Haiku 4.539.2%$0.00266.6s2%
81GPT-5.4 Mini (Reasoning)45.3%$0.00779.4s0%
82Qwen 3.6 Flash53.3%$0.01549.1s5%
83GPT-5 Nano59.7%$0.00401.6m4%
84GPT-4.1 Nano37.3%$0.00026.0s0%
85DeepSeek V3 (2024-12-26)37.2%$0.000619.3s3%
86Z.AI GLM 5.3 (Reasoning, Max)85.4%$0.0553.9m41%
87Gemini 3.5 Flash (Reasoning)90.4%$0.13756.9s45%
88GPT-5.6 Sol43.4%$0.01813.5s5%
89Hermes 3 405B35.7%$0.000026.5s3%
90Claude Sonnet 5.5 (Reasoning)43.7%$0.02015.8s4%
91GPT-5.4 Nano (Reasoning, Low)34.5%$0.002010.4s0%
92Laguna XS 2.132.4%$0.00017.6s0%
93Xiaomi MIMO v2.534.3%$0.002212.5s0%
94Mistral Small 431.2%$0.00035.7s1%
95GPT-5.4 Mini (Reasoning, Low)32.9%$0.00324.6s0%
96GPT-5.547.2%$0.02119.2s0%
97Mistral Large 332.6%$0.000912.1s0%
98Qwen 3.6 35B52.8%$0.0131.3m3%
99DeepSeek-V2 Chat36.0%$0.000229.9s0%
100DeepSeek V3 (2025-03-24)33.3%$0.000617.8s0%
101Gemini 2.5 Flash30.2%$0.00154.0s0%
102GPT-5.4 Mini30.5%$0.00294.0s0%
103GPT-5.253.3%$0.02738.3s0%
104GPT-5.4 (Reasoning)53.9%$0.03039.6s1%
105DeepSeek V4 Flash (Reasoning)34.4%$0.000241.4s2%
106GPT-5.156.5%$0.03052.9s2%
107DeepSeek V3.133.2%$0.000628.3s0%
108GPT-5.4 (Reasoning, Low)42.5%$0.01723.5s0%
109Grok 4.7 (Reasoning, High)51.8%$0.02357.4s2%
110Qwen 2.5 72B27.2%$0.000315.6s0%
111DeepSeek V3.234.3%$0.000547.8s0%
112Claude Haiku 5.5 (Adaptive)24.4%$0.00099.8s0%
113Hermes 3 70B25.9%$0.000220.8s0%
114Qwen 3.5 27B57.5%$0.0251.9m6%
115Gemma 4 26B23.8%$0.000216.6s1%
116Qwen 3.5 Plus (2026-04-20)56.9%$0.0202.2m7%
117Xiaomi MIMO v2.6 Flash35.1%$0.00031.1m0%
118DeepSeek V4 Flash 0731 (Reasoning, High)41.9%$0.00121.5m1%
119Xiaomi MIMO v2.5 Pro26.2%$0.003220.8s0%
120Grok 4.3 (Reasoning)73.5%$0.0373.2m16%
121GPT-5.5 (Reasoning, Low)46.0%$0.03324.6s0%
122Mistral Small 3.2 24B21.1%$0.00029.0s0%
123Qwen 3.5 Flash41.7%$0.00501.4m0%
124Aion 2.025.7%$0.001624.7s0%
125Mistral Large 4.025.0%$0.001422.5s0%
126Gemma 3 12B21.3%$0.000111.9s0%
127Gemini 3.5 Flash Lite (Reasoning, Minimal)19.7%$0.00102.6s0%
128Gemma 4 31B24.2%$0.000224.3s0%
129Gemini 2.5 Flash Lite (Reasoning)25.2%$0.002224.1s0%
130GPT-5.6 Terra26.5%$0.0109.6s0%
131Grok 4.5 (Reasoning, Low)33.0%$0.01134.2s0%
132Ministral 8B18.6%$0.00004.0s0%
133Cydonia 24B V4.120.6%$0.000312.0s0%
134Ministral 3 3B17.2%$0.00012.0s0%
135Qwen 3.5 Plus (2026-02-15)22.7%$0.001322.1s0%
136Arcee AI: Trinity Mini16.2%$0.00015.6s0%
137Z.AI GLM 5.177.8%$0.0483.9m24%
138ByteDance Seed 2.0 Lite26.5%$0.003642.6s0%
139GPT-5.424.1%$0.009618.7s0%
140Qwen 3.5 35B46.6%$0.0241.3m1%
141Claude Sonnet 4.520.8%$0.008011.8s0%
142Ministral 3B13.5%$0.00002.5s0%
143GPT-5.4 Nano14.6%$0.00115.2s0%
144Ministral 3 8B13.5%$0.00013.3s0%
145Ministral 3 14B13.8%$0.00026.4s0%
146Mistral Small 4 (Reasoning)18.6%$0.001922.1s0%
147GPT-5.5 (Reasoning, Medium)49.5%$0.04929.5s0%
148Aion 3.036.5%$0.0171.0m0%
149Gemini 3.5 Flash Lite (Reasoning)12.4%$0.00102.5s0%
150Claude Haiku 5.5 (Reasoning, Low)12.7%$0.00056.3s0%
151Z.AI GLM 4.5 Air15.3%$0.000717.8s0%
152Qwen 3.5 122B47.9%$0.0331.3m2%
153Qwen 3 235B A22B Instruct 250714.4%$0.000318.7s0%
154Cohere Command R+ (Aug. 2024)15.7%$0.005412.4s0%
155Nemotron 3 Nano70.4%$0.00424.8m11%
156Writer: Palmyra X514.3%$0.003812.6s0%
157Laguna S 2.112.0%$0.000114.4s0%
158ByteDance Seed 1.6 Flash11.7%$0.000612.2s0%
159Z.AI GLM 4.511.1%$0.00098.9s0%
160Claude Sonnet 415.1%$0.007711.4s0%
161GPT-5.6 Luna10.3%$0.00344.8s0%
162DeepSeek V4 Pro14.2%$0.000928.7s0%
163Grok 4.20 (Reasoning)48.6%$0.0272.0m4%
164Grok 4.37.9%$0.00117.2s0%
165Qwen 3 32B10.8%$0.000521.3s0%
166Mistral Medium 3.5 (Reasoning)8.6%$0.00354.7s0%
167Z.AI GLM 4.740.6%$0.00692.4m4%
168Thinking Machines Inkling16.7%$0.002242.5s0%
169Mistral NeMO6.5%$0.00016.6s0%
170Claude Sonnet 4.6 (Reasoning)71.3%$0.1171.3m27%
171Gemini 2.5 Pro28.2%$0.02925.0s0%
172Gemma 3 4B5.3%$0.00005.5s0%
173Claude Haiku 5.5 (Reasoning, Medium)5.3%$0.00045.8s0%
174Mistral Medium 3.56.8%$0.00344.5s0%
175GPT-6 Luna5.6%$0.00038.4s0%
176ByteDance Seed 1.627.5%$0.00731.4m0%
177Qwen 3.6 27B49.1%$0.0292.1m3%
178Grok 4.6 (Reasoning, High)59.2%$0.0452.3m5%
179MiniMax M357.5%$0.0113.7m6%
180Gemma 3 27B5.2%$0.000115.7s0%
181Z.AI GLM 4.7 Flash18.3%$0.00141.1m0%
182Aion 3.0 Mini30.6%$0.00571.9m0%
183Gemma 4 26B (Reasoning)50.9%$0.00353.8m4%
184Mistral Large 22.6%$0.003113.0s0%
185Gemma 4 31B (Reasoning)52.4%$0.00244.2m7%
186Mistral Large 4.0 (Reasoning)60.3%$0.0313.7m10%
187Z.AI GLM 528.2%$0.0101.8m0%
188Gemini 2.5 Flash (Reasoning)12.4%$0.01529.3s0%
189WizardLM 2 8x22b0.0%$0.000722.8s0%
190Grok 4.5 (Reasoning, High)59.1%$0.0592.5m6%
191Kimi K2.549.8%$0.0243.2m4%
192Thinking Machines Inkling Small (Reasoning, High)27.4%$0.00872.2m0%
193Z.AI GLM 4.611.5%$0.00441.3m0%
194Qwen 3.5 9B25.8%$0.00172.5m0%
195DeepSeek V4 Pro (Reasoning)48.3%$0.0193.7m3%
196Qwen 3.6 Max Preview75.1%$0.0724.4m17%
197ByteDance Seed 2.0 Mini25.8%$0.00293.3m0%
198Qwen 3.5 397B A17B62.6%$0.0394.9m8%
199Thinking Machines Inkling (Reasoning)30.9%$0.0304.1m0%
200Kimi K2.671.5%$0.0547.0m15%
49.07%

Individual Scenarios

dialogue-200

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100 – – – – – – – 100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
Kimi K2.6100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Inception Mercury 2100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Claude Opus 5.5 (Reasoning)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100100100100100100.0%
o4 Mini High100100100100100100100100100100100.0%
o4 Mini100100100100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)1001001001001001001001001009999.9%
GPT-5.4 Mini (Reasoning)1001001001001001001001001009999.9%
Gemini 3.1 Flash Lite (Reasoning)1001001001001001001001001009999.9%
Gemini 3.1 Flash Lite (Preview)1001001001001001001001001009999.8%
Mistral Large 4.0 (Reasoning)10010010010010010010099999999.7%
Gemini 3.1 Flash Lite1001001001001001001001001009899.7%
GPT-6 Sol (Reasoning, Medium)1001001001001001001001001009699.6%
Claude Opus 4.8 (Reasoning)1001001001001001001001001009699.5%
Claude Opus 4.510010010010010010010099989899.4%
Claude Opus 4.8 (Reasoning, Low)1001001001001001009999989899.4%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100989699.4%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100100100999499.3%
Kimi K2.510010010010010010010099999499.1%
Z.AI GLM 5.11001001001001001001001001009099.0%
Gemini 3.6 Flash (Reasoning, Minimal)10010010010010010010099989499.0%
Qwen 3.6 27B100100100100100100100100999099.0%
Claude Opus 4.6100100100100100999999999098.7%
GPT-4o Mini (temp=0)10010010010010010010099989098.6%
GPT-4o Mini (temp=1)100100100100100100100100998698.4%
MiniMax M2.51001001001001001001001001008198.1%
Claude Opus 4.7 (Reasoning)10010010010099999999988698.0%
GPT-4.1100100100100100999999968697.9%
Claude Opus 4.71001001009999999998969097.9%
GPT-4o, Aug. 6th (temp=1)10010010010099999998949097.8%
Claude Opus 5 (Reasoning, Low)10010010010099989696969497.8%
Gemma 4 31B (Reasoning)100100100100100100100100968197.7%
Gemini 3 Flash (Preview)10010010010099999898949097.7%
Claude Sonnet 4.6100100999999999998948697.2%
Qwen 3.5 Plus (2026-04-20)10010010010099999999947596.5%
Qwen 3.6 Flash1001001001001001009998966896.0%
GPT-5.5100100100100100999998867595.8%
Grok 4.6 (Reasoning, High)10010010010010010010099906895.7%
GPT-4o, Aug. 6th (temp=0)10010010010010010010099986095.6%
Grok 4.20 (Reasoning)1001001001001001009990866894.3%
Gemma 4 31B10010010010099999996945294.0%
Qwen 3.5 122B10010010010010010010099944393.6%
Qwen 3.6 35B100100100100100100100100904393.3%
Xiaomi MIMO v2.6 Pro1001001001001001001001001002792.7%
Qwen 3.7 Flash (Reasoning)1001001001001001009690756092.2%
GPT-5.4 (Reasoning, Low)10010010010010010010098863591.9%
Qwen 3.5 27B100100100100100100100100992091.9%
Claude Sonnet 5 (Reasoning, Low)100100999494909090866891.2%
Claude Opus 5 (Reasoning)100100999998989694943591.1%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100090.0%
Nemotron 3 Nano100100100100100100100100100090.0%
GPT-OSS 120B100100100100100100100100100090.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100100100100090.0%
GPT-5.6 Sol9998949490908681818189.5%
Claude Sonnet 510010010010099999494862089.1%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100999975087.4%
Claude Sonnet 5 (Reasoning)1001001009998968675754387.3%
Kimi K3 (Reasoning, Low)100100100100100999975751085.7%
Claude Opus 4100100100100100999981681085.6%
Claude Opus 5100100100100100998675752085.6%
Llama 3.1 70B100100989898949081682785.4%
Claude Sonnet 4.510010010010096907568606085.0%
GPT-4.1 Nano1001001009996868668605284.8%
Gemma 4 26B (Reasoning)100100100100100100948168484.6%
Muse Glimmer 30B (Reasoning, Medium)10010010010010098989652084.3%
Claude Haiku 4.51001001009998969486601084.2%
GPT-5.5 (Reasoning, Low)100100100100100100100100202084.0%
GPT-6 Sol10010010010098948675601482.7%
MiniMax M310010010010010010010010014682.0%
Qwen 3.5 35B100100100100100100100966080.2%
Qwen 3.5 Flash100100100100100100100990080.0%
DeepSeek V4 Flash 0731 (Reasoning, High)10010010010010010010060141078.4%
GPT-5.4 Nano (Reasoning, Low)100100100100100100100430074.4%
Qwen 3.5 9B1001001001001009999430074.1%
GPT-5.4 Mini (Reasoning, Low)10010099989486686035074.0%
GPT-4.1 Mini10010099999986814327073.6%
Gemini 3.5 Flash (Reasoning, Minimal)100100100999468685243072.4%
Aion 3.5 (Reasoning, High)1009696948686606043072.2%
DeepSeek V4 Flash (Reasoning)10010096868175756035070.9%
GPT-5.6 Luna (Reasoning, Medium)1001001001001001009064470.4%
Z.AI GLM 4.71001001001009675685210070.1%
Claude Sonnet 41009996909086684320670.0%
Thinking Machines Inkling Small (Reasoning, High)1001009999989490141069.5%
DeepSeek V3 (2025-03-24)100999894907575600069.2%
DeepSeek V4 Flash100999999948668430068.9%
DeepSeek V4 Pro (Reasoning)1001001001001001008600068.6%
Aion 3.0 Mini1001001009994818110065.7%
Claude Haiku 5.5 (Reasoning, Low)100900 – – – – – – – 63.5%
Qwen 3.5 Plus (2026-02-15)1001009690867552141061.5%
Aion 3.01001001009490754340060.6%
Claude Haiku 5.5 (Adaptive)9999999694811464059.2%
DeepSeek V3.1100989690686052201158.7%
Gemini 2.5 Flash100969490754335272056.3%
Gemma 4 26B94907568686060356255.9%
Thinking Machines Inkling (Reasoning)100100989860434366055.5%
Gemini 2.5 Pro99969690754327140054.2%
Grok 4.201001001009981352040053.8%
Hermes 3 405B100999081756814100053.8%
GPT-5.6 Terra98989896434335201053.2%
Gemma 3 12B100949490683514146652.1%
DeepSeek V3.21001001009675272000051.9%
Ministral 3 3B10010099989610000050.2%
Gemini 2.5 Flash Lite100999999940000049.1%
GPT-5.4 Mini10094818175351420048.3%
Gemini 3.5 Flash Lite (Reasoning)999486816052110047.5%
DeepSeek V3 (2024-12-26)10010099942727600045.3%
Z.AI GLM 4.59494907552271400044.6%
Mistral Medium 3.11009694863514600043.1%
ByteDance Seed 2.0 Mini99965252433535140042.6%
Qwen 2.5 72B10010090863510100042.2%
Xiaomi MIMO v2.6 Flash100100100100202000042.2%
Grok 4.5 (Reasoning, Low)1001009975431000041.9%
Hermes 3 70B100999686276200041.7%
Gemini 2.5 Flash Lite (Reasoning)1001009494204000041.2%
DeepSeek-V2 Chat9996944335351000041.2%
Laguna XS 2.11001001009962200040.9%
GPT-5.49490686035272062240.5%
ByteDance Seed 2.0 Lite100999081206200039.9%
Z.AI GLM 5100988686271000039.9%
Mistral Large 310099989600000039.2%
Xiaomi MIMO v2.51009694602710000038.7%
Cydonia 24B V4.19999998600000038.4%
Xiaomi MIMO v2.5 Pro100100987500000037.3%
Z.AI GLM 4.5 Air100946052270000033.4%
DeepSeek V4 Pro10096685264400032.9%
Mistral Large 4.010096814342000032.7%
Mistral Small 3.2 24B9986603514141000031.9%
ByteDance Seed 1.6100100752760000030.8%
Mistral Small 4100100752000000029.6%
Mistral Small 4 (Reasoning)10099681000000027.8%
Ministral 3 8B1009868000000026.6%
GPT-5.4 Nano1009452000000024.6%
Gemma 3 4B1009827642000023.6%
Ministral 3B94524320200000022.9%
Aion 2.01008143400000022.8%
Thinking Machines Inkling10052521000000021.3%
Claude Haiku 5.5 (Reasoning, Medium)1008114640000020.5%
Z.AI GLM 4.7 Flash1006827000000019.5%
Mistral Medium 3.598904200000019.4%
Gemma 3 27B868114600000018.9%
Gemini 2.5 Flash (Reasoning)946027000000018.1%
Z.AI GLM 4.6945227000000017.3%
Mistral NeMO96520000000014.8%
ByteDance Seed 1.6 Flash86600000000014.7%
Ministral 8B99270000000012.6%
Mistral Medium 3.5 (Reasoning)75434000000012.3%
Gemini 3.5 Flash Lite (Reasoning, Minimal)81352210000012.2%
Qwen 3 32B96200000000011.6%
Writer: Palmyra X586272000000011.6%
Laguna S 2.1602720100000010.9%
Qwen 3 235B A22B Instruct 2507860000000008.7%
Ministral 3 14B4310000000005.3%
Mistral Large 22720110000004.9%
Cohere Command R+ (Aug. 2024)274000000003.1%
GPT-6 Luna20000000000.2%
GPT-5.6 Luna00000000000.0%
Grok 4.300000000000.0%
WizardLM 2 8x22b00000000000.0%
Arcee AI: Trinity Mini00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100100 – – – – – – 100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
GPT-5.1100100100100100100100100100100100.0%
MiniMax M2.5100100100100100100100100100100100.0%
GPT-5 Nano100100100100100100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)1001001001001001001001001009999.9%
Claude Opus 4.8 (Reasoning)10010010010010010010010010010099.9%
Grok 4.3 (Reasoning)1001001001001001001001001009999.9%
Claude Opus 4.8 (Reasoning, Low)1001001001001001001001001009999.9%
GPT-5.4 Mini (Reasoning)1001001001001001001001001009999.9%
MiniMax M2.7100100100100100100100100999999.8%
Gemini 3.1 Flash Lite (Reasoning)1001001001001001001001001009999.8%
GPT-5.4 Nano (Reasoning)100100100100100100100100999999.8%
Z.AI GLM 5.3 (Reasoning, Max)1001001001001001001001001009899.8%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100999999.8%
Nemotron 3 Super1001001001001001001001001009899.8%
Claude Opus 4.610010010010010010010099999999.6%
o4 Mini High1001001001001001001001001009699.6%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100999699.4%
Gemini 3.1 Flash Lite100100100100100100100100999699.4%
GPT-OSS 120B1001001001001001001001001009499.4%
Grok 4.6 (Reasoning, High)100100100100100100100100989699.3%
Claude Opus 4.6 (Reasoning)10010010010010010010099999699.3%
Qwen 3.6 35B10010010010010010010099989699.2%
Claude Opus 5 (Reasoning, Low)1001001001001001009999989699.1%
Inception Mercury 21001001001001001001001001009099.0%
Claude Opus 5.5 (Reasoning)1001001001001001001001001009099.0%
GPT-4o Mini (temp=1)10010010010010010010099989098.7%
Claude Opus 5100100100100100999999969498.6%
o4 Mini100100100100100100100100998698.5%
GPT-5.5 (Reasoning, Low)10010010010010010010099969098.5%
GPT-5.6 Terra (Reasoning, Medium)1001001001001001001001001008198.1%
Z.AI GLM 5.3 Flash (Reasoning, Max)1001001001001001009998988698.0%
Claude Sonnet 4.6 (Reasoning)100100100100100999898948697.5%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100997597.4%
Qwen 3.5 397B A17B10010010010010010010099997597.2%
Z.AI GLM 5.11001001001001001001001001006896.8%
Qwen 3.5 35B10010010010010010010099907596.5%
Qwen 3.5 27B1001001001001001009998907596.2%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100906895.8%
Grok 4.201001001001001001009999866895.3%
GPT-4o, Aug. 6th (temp=0)1001001001001001009998906094.7%
Claude Opus 4.59998989696949494908694.4%
Claude Opus 4.7 (Reasoning)10099999998989490867593.8%
Grok 4.5 (Reasoning, High)100100100100100100100100993593.4%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100864393.0%
Gemma 4 31B (Reasoning)100100100100100100100100754391.8%
Claude Opus 5 (Reasoning)100100100100100999998942791.7%
GPT-4.1100100100100100999999991090.5%
Claude Sonnet 51001001009999999981686090.4%
Nemotron 3 Nano100100100100100100100100100090.0%
Claude Sonnet 5 (Reasoning)10010010010099999896941489.9%
Aion 3.5 Mini (Reasoning, High)10010010010010010010010099089.9%
GPT-5.510010010010099989898862089.8%
MiniMax M31001001001001001001009996089.5%
Aion 3.5 (Reasoning, High)1001001001009999999894489.1%
Gemma 4 26B (Reasoning)10010010010010010010010090089.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)1001001001009999999894088.7%
GPT-5.4 (Reasoning, Low)100100100100100100100100523588.7%
GPT-4o Mini (temp=0)1001001009996908686686088.6%
Qwen 3.6 Flash10010010010010096969694088.2%
Xiaomi MIMO v2.6 Pro1001001001001001009990682087.7%
DeepSeek V4 Pro (Reasoning)10010010010010010010086682087.5%
GPT-5.4 Nano (Reasoning, Low)100100100100100100989081087.0%
Claude Opus 4.710099989090868681756086.7%
DeepSeek-V2 Chat100100100100100100948681086.1%
Claude Sonnet 4.69996949090908686755286.0%
Grok 4.20 (Reasoning)1001001009999969675751085.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100271083.7%
Claude Sonnet 5.5 (Reasoning)100100100100100969086352783.5%
GPT-4.1 Nano10010099999696908660082.8%
Kimi K2.6100100100100100100987543281.8%
Kimi K2.5100100100999490908160081.5%
Kimi K3 (Reasoning, High)1001001001001001007560431479.3%
DeepSeek V4 Flash1009999989896946043679.2%
Claude Sonnet 5 (Reasoning, Low)10099989694818168432778.7%
Gemini 3 Flash (Preview)9999999898909068201477.6%
Grok 4.5 (Reasoning, Low)1001009999969090860076.1%
Gemini 3.6 Flash (Reasoning, Minimal)1001001009999866868201075.0%
GPT-4.1 Mini999898949086818120074.8%
Z.AI GLM 4.71001001001009996962714073.2%
Mistral Medium 3.11001001001001009996144171.3%
GPT-6 Sol10010099969090684320671.3%
GPT-5.4 Mini1009994868181756820671.1%
Qwen 3.7 Flash (Reasoning)10010098909075605243070.9%
Grok 4.7 (Reasoning, High)10010010010010010010000070.0%
Mistral Large 4.0 (Reasoning)1001001001009990602714069.1%
GPT-5.6 Sol999896908181753527168.4%
DeepSeek V3.210098948675686843351468.1%
DeepSeek V3 (2024-12-26)10010090868181524327066.2%
Qwen 3.5 Flash100100999996906060065.2%
Mistral Large 3100999894816860520065.2%
Llama 3.1 70B1009898868675523510464.3%
ByteDance Seed 1.6100100100100100984300064.1%
GPT-5.4 Mini (Reasoning, Low)10010010096866868106163.5%
Aion 3.01001001001001005243350063.0%
Kimi K3 (Reasoning, Low)100100967575686020201462.9%
GPT-5.6 Terra100989081816860430062.2%
Qwen 3.6 27B10010010010098685200061.8%
Thinking Machines Inkling (Reasoning)1001009081757560350061.7%
Gemini 3.5 Flash (Reasoning, Minimal)10099989486865220061.7%
DeepSeek V3 (2025-03-24)100100100907560432720061.7%
Claude Haiku 5.5 (Adaptive)10099997568686027101061.6%
Hermes 3 405B948181757568524343261.5%
Laguna XS 2.110010010099757543146061.3%
Xiaomi MIMO v2.51001001001001005235140060.1%
Qwen 3.5 122B10010010098984327276059.9%
DeepSeek V4 Flash 0731 (Reasoning, High)10010010010010086100058.7%
Claude Opus 499908681757543142056.7%
Mistral Small 410099999896601000056.2%
ByteDance Seed 2.0 Mini10010099967568000053.8%
Gemini 2.5 Pro86818175685252350053.2%
Claude Haiku 4.590909081684320206051.1%
GPT-5.4989486818168000051.0%
Gemma 4 31B909086525252432014250.3%
Qwen 2.5 72B100999998522014102049.4%
DeepSeek V3.1100100998168202020049.1%
Gemini 3.5 Flash Lite (Reasoning, Minimal)99948681432727200047.8%
Z.AI GLM 4.6999694866820600047.0%
Thinking Machines Inkling Small (Reasoning, High)10010090906027000046.9%
Mistral Small 3.2 24B10010099686014000044.1%
Gemini 2.5 Flash Lite1001009875680000044.1%
GPT-5.4 Nano1009894686014400043.8%
Mistral Large 4.0100100100961010000041.5%
ByteDance Seed 2.0 Lite10010010010000000040.0%
Z.AI GLM 4.7 Flash10010010060270000038.7%
DeepSeek V4 Flash (Reasoning)10099814327141010037.6%
Aion 3.0 Mini9890908100000036.0%
Writer: Palmyra X599989052104000035.3%
Gemma 4 26B9086524335271060035.1%
Qwen 3 235B A22B Instruct 250710099756800000034.3%
Qwen 3.5 Plus (2026-02-15)1008168352720000033.2%
Hermes 3 70B10010043352720200032.8%
Gemini 2.5 Flash Lite (Reasoning)10099604340000030.7%
DeepSeek V4 Pro9896862060000030.6%
Xiaomi MIMO v2.6 Flash100100100220000030.4%
Z.AI GLM 596946043101000030.4%
Aion 2.09994862000000029.9%
Gemini 2.5 Flash (Reasoning)1009875000000027.3%
Ministral 3B9686682000000027.1%
Z.AI GLM 4.5 Air9868602700000025.4%
ByteDance Seed 1.6 Flash10010052000000025.2%
Qwen 3.5 9B10068433500000024.6%
Gemini 2.5 Flash1001001410106000024.0%
Cydonia 24B V4.198752714104400023.1%
Ministral 8B96944000000019.4%
Ministral 3 14B94900000000018.4%
Qwen 3 32B90684000000016.2%
Mistral Small 4 (Reasoning)964310100000015.0%
Claude Sonnet 4.552272020141110013.7%
Mistral Medium 3.5 (Reasoning)75600000000013.6%
Xiaomi MIMO v2.5 Pro98271000000012.6%
Gemini 3.5 Flash Lite (Reasoning)6020201000000011.0%
Arcee AI: Trinity Mini10000000000010.0%
Cohere Command R+ (Aug. 2024)68201010000009.9%
Thinking Machines Inkling8114000000009.6%
Z.AI GLM 4.552202021000009.5%
Gemma 3 12B816000000008.9%
Laguna S 2.15235000000008.7%
Claude Sonnet 443141062000007.6%
Mistral NeMO750000000007.5%
Mistral Medium 3.5524000000005.6%
Grok 4.3522000000005.4%
Ministral 3 3B14141062000004.7%
Gemma 3 27B200000000002.0%
GPT-5.6 Luna60000000000.7%
GPT-6 Luna41000000000.5%
Ministral 3 8B20000000000.2%
Claude Haiku 5.5 (Reasoning, Low)0 – – – – – – – – – 0.0%
Mistral Large 200000000000.0%
Gemma 3 4B00000000000.0%
Claude Haiku 5.5 (Reasoning, Medium)00000000000.0%
WizardLM 2 8x22b00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-5.2100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100100100100100100.0%
MiniMax M2.7100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100100100 – – – – – 100.0%
o4 Mini High10010010010010010010010010010099.9%
Muse Spark 1.1 (Reasoning, Medium)1001001001001001001001001009999.9%
GPT-5.6 Sol (Reasoning, Medium)1001001001001001001001001009999.9%
Gemini 3.8 Flash (Reasoning, Medium)10010010010010010010099999999.7%
GPT-4o, Aug. 6th (temp=0)100100100100100999999999699.2%
MiniMax M2.5100100100100100100100100999099.0%
Claude Opus 51001001001001001009999999499.0%
Claude Opus 4.8 (Reasoning)10010010010010010010099969498.8%
GPT-5.6 Terra (Reasoning, Medium)1001001001001001001001001008698.6%
GPT-5.4 Nano (Reasoning)1001001001001001001001001008698.6%
Inception Mercury 21001001001001001001001001008698.6%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100998698.6%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100100100998698.5%
Kimi K2.61001001001001001009998949498.4%
Qwen 3.5 27B100100100100100100100100949098.4%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100999998949498.2%
Gemini 3.1 Flash Lite (Preview)100100100100100999996949498.2%
GPT-5.51001001009999999896949097.4%
Gemini 3 Flash (Preview, Reasoning)1001001009999999896908696.8%
Gemini 3.5 Flash (Reasoning)1001001001001001001001001006896.8%
GPT-5.5 (Reasoning, Medium)100100100100100100100100996896.7%
Claude Opus 5 (Reasoning, Low)10010010010010010010096967596.7%
Claude Opus 5.5 (Reasoning)1001001001001001009996967596.6%
GPT-4o Mini (temp=0)10010010010099999898966895.7%
Qwen 3.5 397B A17B100100100100100100100100817595.6%
Gemini 3.1 Flash Lite (Reasoning)10010010010098989896868195.6%
GPT-6 Luna (Reasoning, High)100100100100100100100100906095.0%
Grok 4.3 (Reasoning)100100100100100100100100757595.0%
Mistral Large 4.0 (Reasoning)100100100100100999999757594.7%
GPT-4o Mini (temp=1)1001001001001001009998965294.4%
Claude Opus 5 (Reasoning)100100100100100999890866894.1%
GPT-5.5 (Reasoning, Low)10010010010099999490866893.7%
GPT-5.4 (Reasoning)100100100100100100100100904393.4%
GPT-4o, Aug. 6th (temp=1)100100999998949086818192.9%
Gemini 3.7 Flash (Reasoning, Medium)10010010010099989896964392.9%
Aion 3.5 Mini (Reasoning, High)100100100100100999494816092.8%
Claude Opus 4.610010010010099989090906092.8%
Gemini 3.6 Flash (Reasoning)1001001001001001009999863592.0%
Claude Opus 4.79999999696909090866891.6%
GPT-4.1100100999999999986815291.4%
GPT-5100100100100100100100100100490.4%
GPT-5 Nano100100100100100100100100100090.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100090.0%
Nemotron 3 Nano100100100100100100100100100090.0%
Grok 4.6 (Reasoning, High)10010010010010010010010099089.8%
GPT-OSS 120B1001001001001001001009999089.8%
Z.AI GLM 5.3 Flash (Reasoning, Max)1001001001001001001009999089.8%
Grok 4.5 (Reasoning, High)10010010010010010010010098089.7%
Claude Opus 4.6 (Reasoning)1001001009996969075756089.2%
GPT-6 Sol (Reasoning, Medium)1001001001001001009996752089.1%
Xiaomi MIMO v2.6 Pro10010010010099999890752788.8%
Qwen 3.6 Max Preview100100100100100100989090087.8%
GPT-5.110010010010010010010010068687.4%
Gemini 3.5 Flash (Reasoning, Minimal)10010010010098989686752087.3%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100999875087.1%
Qwen 3.5 Flash10010010010010098969481086.8%
GPT-6 Luna (Reasoning, Medium)100100100100100100988681086.5%
Qwen 3.7 Flash (Reasoning)1001001001001001009981681486.2%
Gemini 3.1 Flash Lite1001001001009996908686185.9%
Claude Sonnet 510010010010099986868684384.4%
Z.AI GLM 5.1100100100100100999486352784.1%
Z.AI GLM 5.2 (Reasoning, High)9998969694948681524383.9%
Gemma 4 26B (Reasoning)100100100999696948643281.6%
Qwen 3.6 27B10010010010010099998627081.2%
Qwen 3.6 35B10010010010010010010010010181.0%
Nemotron 3 Super100100100100100999060431480.7%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100999998816060179.9%
Kimi K3 (Reasoning, Low)10010099969086817568079.6%
Qwen 3.5 122B100100100100100100100900079.0%
Kimi K3 (Reasoning, High)1001001009999948160201076.3%
Claude Sonnet 5 (Reasoning)10010010098969490814076.3%
Claude Opus 4.7 (Reasoning)9994949490868160521076.0%
Mistral Large 4.09999969486866843434375.9%
Z.AI GLM 5 Turbo100100100100999981750075.4%
DeepSeek V4 Pro 0813 (Reasoning, High)1001001001009994904320074.7%
GPT-5.4 (Reasoning, Low)10010010010010075685243674.5%
GPT-6 Sol1001001009986756852352774.2%
DeepSeek V4 Pro (Reasoning)100100100969090756020073.3%
Aion 3.5 (Reasoning, High)10010099999490813527072.6%
GPT-5.4 Mini (Reasoning)100100100100100100604314071.8%
Qwen 3.5 35B100100100100999481274070.5%
Kimi K2.5100100100999696683510070.3%
Qwen 3.5 Plus (2026-04-20)1001009998968686350070.0%
Grok 4.2010096909090816835272069.9%
Claude Opus 4.5999896908686862720469.3%
MiniMax M310010010098907575430068.2%
DeepSeek V4 Flash 0731 (Reasoning, High)10010010096868681206067.6%
Claude Opus 4969490868675686010667.2%
o4 Mini10010010010010068522720266.9%
Grok 4.20 (Reasoning)1001009999909052276166.6%
Grok 4.5 (Reasoning, Low)10010010010010068434310066.4%
Llama 3.1 70B1009998948168524314065.0%
Gemini 3 Flash (Preview)9996868175757535141064.7%
GPT-5.4 Mini1009494908675682010063.7%
Muse Glimmer 30B (Reasoning, Medium)100100999986866060063.7%
Qwen 3.6 Flash100999999998627200063.1%
DeepSeek V3.2100100989686757500063.0%
Claude Sonnet 5 (Reasoning, Low)10099989481752720201462.8%
Xiaomi MIMO v2.5 Pro100999986816052350061.3%
DeepSeek V4 Flash100999896969420100061.2%
GPT-5.4 Mini (Reasoning, Low)100100996860605227201460.2%
Ministral 8B1001001009690753520059.9%
DeepSeek V3.1100999894865235350059.8%
Mistral Small 4 (Reasoning)1001001001009896000059.3%
Grok 4.7 (Reasoning, High)1001001001009990000058.9%
Z.AI GLM 4.71001009681686043350058.4%
Gemini 3.6 Flash (Reasoning, Minimal)9998948181755221058.3%
GPT-5.6 Sol10098968681753500057.1%
Claude Sonnet 4.69868686860525235353557.1%
GPT-4.1 Nano100999486814327274056.2%
Thinking Machines Inkling (Reasoning)98969475755235271055.3%
Gemma 4 31B (Reasoning)100998181755243142054.8%
ByteDance Seed 1.69999968675681400053.8%
Hermes 3 405B1009081816852272014053.4%
Mistral Medium 3.110096946060605264053.2%
GPT-5.4100948675525252200053.1%
Hermes 3 70B1001009681523535274053.0%
Thinking Machines Inkling999894949052000052.6%
GPT-4.1 Mini100100989881351020052.3%
Laguna XS 2.110010010096944000049.4%
Xiaomi MIMO v2.5999998866043200048.8%
Thinking Machines Inkling Small (Reasoning, High)100100100100810000048.1%
Claude Sonnet 4.6 (Reasoning)999481524343272020048.1%
Z.AI GLM 51009999906014421047.0%
DeepSeek-V2 Chat1009081756852000046.7%
GPT-5.4 Nano (Reasoning, Low)100100100100524200045.7%
DeepSeek V3 (2025-03-24)10010010094602000045.6%
DeepSeek V4 Flash (Reasoning)1009896605243000044.9%
Qwen 2.5 72B10010010099432100044.5%
DeepSeek V3 (2024-12-26)10010099755210600044.3%
Mistral Small 41009994812720664243.9%
GPT-5.6 Terra10099685252431462043.6%
Gemini 2.5 Flash Lite90756852434343200043.6%
Qwen 3.5 Plus (2026-02-15)908175605252220041.5%
Claude Haiku 4.5999475754310640040.6%
Gemini 2.5 Flash10099989460000039.6%
Gemma 4 26B100865243272020144236.9%
Ministral 3 14B1009086202020221034.2%
Mistral Small 3.2 24B96948152106000033.9%
Claude Sonnet 5.5 (Reasoning)9668433535271420032.1%
Z.AI GLM 4.5 Air100100862744000032.1%
Aion 3.010096523562200029.3%
ByteDance Seed 2.0 Lite9990353566420027.8%
Qwen 3 235B A22B Instruct 250786605235351000027.0%
Z.AI GLM 4.7 Flash1009075400000026.9%
Mistral Large 39952525210000025.6%
Ministral 3 3B9694271020000022.8%
DeepSeek V4 Pro8175431410000021.6%
ByteDance Seed 1.6 Flash1008627000000021.3%
Laguna S 2.1100946000000020.1%
GPT-5.4 Nano1008110000000019.2%
Gemini 3.5 Flash Lite (Reasoning, Minimal)99816000000018.7%
Writer: Palmyra X5996810410000018.2%
ByteDance Seed 2.0 Mini757527400000018.1%
Qwen 3 32B99810000000018.1%
Qwen 3.5 9B100604200000016.6%
Cydonia 24B V4.1100604000000016.4%
GPT-6 Luna86521000000013.9%
Gemini 2.5 Pro606010600000013.6%
Ministral 3B75600000000013.5%
Gemini 3.5 Flash Lite (Reasoning)81521000000013.5%
Gemini 2.5 Flash Lite (Reasoning)99351000000013.5%
Xiaomi MIMO v2.6 Flash942014400000013.2%
Aion 3.0 Mini68526000000012.6%
Z.AI GLM 4.590274000000012.1%
Claude Haiku 5.5 (Reasoning, Medium)86206000000011.3%
Aion 2.0100101000000011.0%
Ministral 3 8B10020000000010.2%
Claude Sonnet 4.55227640000008.9%
Grok 4.3812000000008.4%
Mistral Medium 3.5 (Reasoning)750000000007.6%
Gemma 3 12B3514400000005.3%
Z.AI GLM 4.6432000000004.6%
GPT-5.6 Luna354200000004.1%
Claude Sonnet 4206110000003.0%
Cohere Command R+ (Aug. 2024)270000000002.7%
Gemma 4 31B62110000001.0%
Gemini 2.5 Flash (Reasoning)64000000001.0%
Mistral Medium 3.560000000000.6%
Mistral Large 260000000000.6%
Arcee AI: Trinity Mini60000000000.6%
Claude Haiku 5.5 (Adaptive)11000000000.2%
Gemma 3 27B00000000000.0%
Claude Haiku 5.5 (Reasoning, Low)0000 – – – – – – 0.0%
WizardLM 2 8x22b00000000000.0%
Mistral NeMO00000000000.0%
Gemma 3 4B00000000000.0%

dialogue-500

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.7 Max100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100100100100.0%
MiniMax M2.71001001001001001001001001009999.9%
Hy4 Preview (Reasoning, High)1001001001001001001001001009899.8%
Qwen 3.8 Flash (Reasoning)1001001001001001001001001009899.8%
Claude Haiku 5.5 (Reasoning, Max)10098 – – – – – – – – 98.8%
Muse Spark 1.2 (Reasoning, Medium)1001001001001001001001001008198.1%
Z.AI GLM 5.3 Flash (Reasoning, Max)1001001001001001009994864392.3%
Gemini 3.1 Pro (Preview)1001001001001001001001001002092.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100962091.6%
o4 Mini High100100100100100100100100100090.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100999894689.6%
Claude Opus 4.8 (Reasoning)10010010010010010010099861089.5%
GPT-6 Sol (Reasoning, Medium)1001001009999989690605289.4%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100989490604388.5%
Claude Sonnet 4.610099999694949081686088.1%
MiniMax M2.510010010010010096969490087.6%
Claude Sonnet 4.6 (Reasoning)100100100100100999690353585.5%
DeepSeek V4.1 Flash (Reasoning, High)1001001009999989086431482.9%
GPT-5 Mini10010010010010010010010014081.4%
Qwen 3.8 27B (Reasoning, XHigh)1001001001001001001001000080.0%
GPT-51001001001001001001001000080.0%
DeepSeek V4 Pro 0813 (Reasoning, High)1001001001001001001001000080.0%
Z.AI GLM 5.3 (Reasoning, Max)1001001001009999996035079.3%
Z.AI GLM 5.110010010010010010096904079.0%
Claude Opus 4.51009998968681686860175.7%
Gemini 3.6 Flash (Reasoning)100100100100969694680075.4%
Claude Opus 5.5 (Reasoning)10010010010099986060141074.1%
Claude Sonnet 5 (Reasoning, Low)1009998969675756020472.3%
Inception Mercury 2100100100100999860350069.2%
Claude Sonnet 5 (Reasoning)10010090868181605227067.9%
Gemini 3 Flash (Preview)100100999998988140067.8%
Nemotron 3 Super100100100100100987500067.3%
Claude Sonnet 51001009999816868350065.0%
Z.AI GLM 5.2 (Reasoning, High)1001001009999964300063.7%
Grok 4.3 (Reasoning)10010010010010010020140063.4%
o4 Mini100100100100908135206063.3%
Qwen 3.6 Max Preview1001001009994815240062.9%
Claude Opus 4.71009994907568433514462.3%
Aion 3.5 Mini (Reasoning, High)100100969686865200061.7%
GPT-4o, Aug. 6th (temp=0)99868686868175100061.1%
GPT-4o Mini (temp=0)10010090756860523527060.8%
Nemotron 3 Nano100100100100100100000060.0%
Claude Opus 4.6 (Reasoning)100100999996861000059.1%
Xiaomi MIMO v2.6 Pro100989486866027200057.2%
Gemini 3.1 Flash Lite (Preview)99999686755235272057.1%
Gemini 3.7 Flash (Reasoning, Medium)100100100948681000056.2%
Claude Opus 4.6100100100998668600056.0%
Claude Opus 4.7 (Reasoning)1001009681815227200055.8%
Gemini 3.1 Flash Lite (Reasoning)100999694904314106055.3%
Claude Opus 5100989086684335100053.1%
Claude Opus 494949081755220140052.1%
GPT-6 Sol10010010099900000049.0%
GPT-6 Luna (Reasoning, High)100100100100680000046.9%
Claude Opus 5 (Reasoning)90868181602720141046.2%
Qwen 3.5 397B A17B100999894520000044.2%
Mistral Medium 3.110010010098350000043.3%
Grok 4.201009986812727622043.2%
Gemma 3 12B100998675600000042.1%
Muse Spark 1.1 (Reasoning, Minimal)100100100100200000042.0%
Grok 4.5 (Reasoning, High)1001009060600000041.1%
MiniMax M310010010010061000040.7%
Z.AI GLM 5.3 Flash (Reasoning, Low)10099603535352760039.8%
Kimi K3 (Reasoning, Low)100908686206200039.1%
Gemini 3.5 Flash (Reasoning, Minimal)1009875523520440038.7%
Kimi K2.61001001008600000038.6%
DeepSeek V4 Flash10099998610000038.6%
Qwen 3.6 Flash100100988600000038.4%
Hermes 3 405B100908168202000036.2%
Qwen 3.5 27B100100995200000035.1%
Gemini 2.5 Flash Lite10099945260000035.1%
Grok 4.6 (Reasoning, High)99988643100000033.6%
GPT-5 Nano1001001003500000033.5%
DeepSeek V3 (2024-12-26)9998944300000033.4%
Arcee AI: Trinity Mini9996686800000033.2%
Aion 3.010090686010000032.0%
Mistral Large 386756843430000031.6%
Claude Opus 5 (Reasoning, Low)100686843274200031.2%
Gemini 3.1 Flash Lite9890684364000031.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100000030.1%
Claude Sonnet 5.5 (Reasoning)9990684300000030.1%
Kimi K3 (Reasoning, High)10099682741100030.0%
Grok 4.7 (Reasoning, High)100100100000000030.0%
Claude Haiku 4.510094524360000029.5%
Z.AI GLM 510090525200000029.4%
GPT-6 Luna (Reasoning, Medium)9890752700000029.0%
Kimi K2.51009694000000029.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)1009990000000029.0%
Qwen 3.6 27B9999434300000028.4%
Aion 3.5 (Reasoning, High)1009075600000027.2%
DeepSeek V4 Flash 0731 (Reasoning, High)9494681000000026.5%
Gemma 4 31B (Reasoning)1008175600000026.3%
DeepSeek V4 Flash (Reasoning)968175000000025.3%
Ministral 3 8B10010052000000025.2%
ByteDance Seed 2.0 Lite10098272040000024.9%
Mistral Small 4907568600000024.0%
Z.AI GLM 4.7 Flash1008652000000023.8%
GPT-OSS 120B947568000000023.7%
Cohere Command R+ (Aug. 2024)969443000000023.3%
Gemini 3 Flash (Preview, Reasoning)10075431400000023.3%
GPT-5.6 Luna (Reasoning, Medium)1006860000000022.8%
ByteDance Seed 2.0 Mini10098141000000022.2%
Qwen 3.7 Flash (Reasoning)1008635000000022.2%
Gemini 2.5 Pro1007543000000021.8%
Xiaomi MIMO v2.6 Flash989620210000021.7%
Mistral Large 4.0 (Reasoning)10010010200000021.2%
GPT-5.1100990000000019.9%
Qwen 3.5 122B100990000000019.9%
Gemma 4 26B (Reasoning)10043431000000019.6%
Gemini 2.5 Flash Lite (Reasoning)100940000000019.4%
Mistral Medium 3.5 (Reasoning)90900000000018.1%
Gemini 2.5 Flash906820000000017.9%
Z.AI GLM 4.752434320141000017.5%
Qwen 3.5 Flash686035000000016.4%
Xiaomi MIMO v2.5 Pro99604000000016.3%
Aion 3.0 Mini98600000000015.8%
Xiaomi MIMO v2.5943527000000015.6%
GPT-4o, Aug. 6th (temp=1)994310400000015.6%
GPT-4o Mini (temp=1)816010000000015.2%
Mistral Medium 3.5100430000000014.3%
Qwen 3.5 Plus (2026-04-20)755210600000014.3%
Laguna S 2.1100350000000013.5%
Laguna XS 2.199350000000013.4%
Thinking Machines Inkling99271000000012.8%
DeepSeek-V2 Chat99204200000012.5%
GPT-5.6 Sol525220000000012.4%
GPT-4.196270000000012.3%
DeepSeek V4 Pro (Reasoning)96206000000012.2%
Gemma 4 26B60600000000012.1%
Qwen 3.6 35B99200000000011.9%
GPT-5.6 Luna68434000000011.5%
Ministral 3 3B99140000000011.4%
Grok 4.310066000000011.2%
Claude Sonnet 4.5811410200000010.7%
Gemma 3 27B9960000000010.5%
GPT-5.4 (Reasoning)10000000000010.0%
DeepSeek V3.210000000000010.0%
Qwen 3 32B10000000000010.0%
Qwen 2.5 72B9460000000010.0%
Qwen 3.5 35B990000000009.9%
GPT-4.1 Mini990000000009.9%
Ministral 8B990000000009.9%
GPT-5.2980000000009.8%
Muse Glimmer 30B (Reasoning, Medium)6827000000009.5%
Gemini 2.5 Flash (Reasoning)43272000000009.1%
Grok 4.20 (Reasoning)900000000009.0%
DeepSeek V3 (2025-03-24)861000000008.7%
Qwen 3 235B A22B Instruct 2507860000000008.7%
Hermes 3 70B860000000008.6%
Ministral 3B860000000008.6%
Aion 2.0754210000008.2%
Gemma 3 4B754000000007.9%
Claude Sonnet 4750000000007.5%
Gemini 3.6 Flash (Reasoning, Minimal)750000000007.5%
Llama 3.1 70B6010000000007.0%
Ministral 3 14B520000000005.2%
Mistral Small 3.2 24B430000000004.3%
ByteDance Seed 1.6352000000003.7%
Mistral Large 2200000000002.0%
Gemini 3.5 Flash Lite (Reasoning)106200000001.8%
DeepSeek V3.1140000000001.5%
Claude Haiku 5.5 (Adaptive)41000000000.5%
ByteDance Seed 1.6 Flash40000000000.4%
Gemma 4 31B00000000000.1%
Z.AI GLM 4.600000000000.0%
Z.AI GLM 4.500000000000.0%
Grok 4.5 (Reasoning, Low)00000000000.0%
Mistral NeMO00000000000.0%
Cydonia 24B V4.100000000000.0%
Z.AI GLM 4.5 Air00000000000.0%
Mistral Large 4.000000000000.0%
Writer: Palmyra X500000000000.0%
Qwen 3.5 9B00000000000.0%
DeepSeek V4 Pro00000000000.0%
WizardLM 2 8x22b00000000000.0%
GPT-6 Luna00000000000.0%
Qwen 3.5 Plus (2026-02-15)00000000000.0%
Thinking Machines Inkling (Reasoning)00000000000.0%
GPT-4.1 Nano00000000000.0%
Mistral Small 4 (Reasoning)00000000000.0%
GPT-5.6 Terra00000000000.0%
Claude Haiku 5.5 (Reasoning, Medium)00000000000.0%
Thinking Machines Inkling Small (Reasoning, High)00000000000.0%
Claude Haiku 5.5 (Reasoning, Low)000 – – – – – – – 0.0%
GPT-5.500000000000.0%
GPT-5.5 (Reasoning, Medium)00000000000.0%
GPT-5.5 (Reasoning, Low)00000000000.0%
GPT-5.4 (Reasoning, Low)00000000000.0%
GPT-5.4 Mini (Reasoning)00000000000.0%
GPT-5.400000000000.0%
GPT-5.4 Mini (Reasoning, Low)00000000000.0%
GPT-5.4 Mini00000000000.0%
GPT-5.4 Nano (Reasoning)00000000000.0%
GPT-5.4 Nano (Reasoning, Low)00000000000.0%
GPT-5.4 Nano00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-5100100100100100100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Max)100100100 – – – – – – – 100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
GPT-5 Mini100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Hy4 Preview (Reasoning, High)1001001001001001001001001009899.8%
Z.AI GLM 5 Turbo1001001001001001001001001009699.6%
Gemini 3.5 Flash (Reasoning)100100100100100100100100969098.6%
Muse Spark 1.1 (Reasoning, Minimal)10010010010010010010099998197.9%
Claude Opus 4.61001001001001001009996966895.8%
Claude Opus 4.8 (Reasoning, Low)100100100100100999994818195.4%
Claude Opus 4.8 (Reasoning)10010010010099999996757594.2%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100090.0%
MiniMax M2.7100100100100100100100100100090.0%
DeepSeek V4.1 Flash (Reasoning, High)1001001001001001001009999089.9%
Muse Spark 1.1 (Reasoning, Medium)10010010010010010010010010081.0%
o4 Mini100100100100100100100962079.8%
Qwen 3.7 Max100100100100100100100901079.2%
Z.AI GLM 5.110010010010010094909014179.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100999994947520478.4%
Claude Opus 5.5 (Reasoning)1009999989494947527077.9%
Claude Sonnet 4.6 (Reasoning)10099989090756052432072.8%
Claude Sonnet 5 (Reasoning)100100100999490753535072.8%
Claude Opus 4.6 (Reasoning)100100100100999894350072.6%
o4 Mini High100100100100100100100140071.4%
MiniMax M2.510010010010010010010060070.6%
Gemini 3.6 Flash (Reasoning)1001001001001009075350070.0%
Kimi K2.61001001001001009875270070.0%
Claude Opus 4.7100100998168606052433570.0%
GPT-6 Luna (Reasoning, High)1001001001001001009600069.6%
Z.AI GLM 5.2 (Reasoning, High)100100100100999068350069.2%
Claude Opus 5 (Reasoning, Low)1001001009999988100067.7%
Nemotron 3 Super10010099999875602010066.1%
Claude Opus 5 (Reasoning)100999994868168276066.0%
Claude Opus 4.7 (Reasoning)100999490907575202164.7%
Gemini 3.1 Flash Lite (Preview)989890686860605243063.8%
Aion 3.5 Mini (Reasoning, High)1001001009896944360063.6%
Claude Sonnet 4.6969075757575684335063.4%
Inception Mercury 210010010010096943560063.1%
GPT-OSS 120B10010010010098685260062.4%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100400060.3%
Qwen 3.6 Max Preview10010010010010094600060.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)10099969494605200059.4%
GPT-4o Mini (temp=0)10099969481812740058.2%
Xiaomi MIMO v2.6 Flash10010010010090751400057.9%
GPT-6 Sol100999986816820100056.4%
Claude Sonnet 51001009681686035106055.6%
Claude Opus 51009996868681600055.5%
Gemini 3.7 Flash (Reasoning, Medium)100100100999843000054.0%
Nemotron 3 Nano1001001001007560000053.5%
Gemini 3.1 Flash Lite (Reasoning)99989686814320100053.4%
GPT-4.110010096948643440052.6%
GPT-6 Sol (Reasoning, Medium)1009999908643100052.0%
Gemini 3.1 Flash Lite10010010075683527100051.5%
Claude Sonnet 5 (Reasoning, Low)10010099867552000051.3%
Z.AI GLM 5.3 Flash (Reasoning, Max)1009494868152000050.7%
Claude Opus 49990686868523500048.1%
Qwen 3.5 Plus (2026-04-20)1001009999680000046.6%
GPT-6 Luna (Reasoning, Medium)100100100686027620046.4%
Kimi K3 (Reasoning, High)10010010090524000044.7%
Gemma 4 31B (Reasoning)100998686604000043.6%
Xiaomi MIMO v2.599988675681000042.8%
Gemini 2.5 Flash Lite1009486686010600042.4%
Xiaomi MIMO v2.6 Pro1001009960524000041.4%
MiniMax M310010010010000000040.0%
Arcee AI: Trinity Mini10010075683514000039.2%
GPT-5.6 Terra (Reasoning, Medium)1001001009000000039.0%
Aion 2.01009996602010000038.5%
Mistral Large 4.0 (Reasoning)1001009060140000036.5%
Qwen 3.5 397B A17B10094866800000034.8%
GPT-4o, Aug. 6th (temp=1)100908152201000034.5%
Claude Opus 4.5989486431010220034.5%
Gemini 2.5 Flash10098816000000033.9%
Aion 3.5 (Reasoning, High)9996905210000033.8%
Grok 4.3 (Reasoning)10099993500000033.3%
Grok 4.20999999400000030.1%
GPT-5.1100100100000000030.0%
GPT-5.4 Nano (Reasoning)100100100000000030.0%
Qwen 3.5 9B10010098000000029.7%
DeepSeek V4 Flash100864335270000029.2%
DeepSeek V4 Pro (Reasoning)10010086000000028.6%
Kimi K3 (Reasoning, Low)1006052431410200028.1%
Gemma 4 26B (Reasoning)100100602000000028.0%
GPT-4o Mini (temp=1)1008681640000027.7%
Cydonia 24B V4.11009975000000027.4%
Claude Haiku 4.59996681000000027.4%
Grok 4.20 (Reasoning)999675000000027.1%
Qwen 3.6 35B10010068000000026.8%
Cohere Command R+ (Aug. 2024)969075200000026.4%
DeepSeek V3.1998675000000026.1%
Mistral Medium 3.110010060000000026.1%
Claude Haiku 5.5 (Adaptive)868181000000024.9%
GPT-5 Nano10010043000000024.4%
Qwen 3.6 27B10010043000000024.3%
Grok 4.5 (Reasoning, High)1009935000000023.4%
Qwen 3.5 35B1008143000000022.5%
Grok 4.7 (Reasoning, High)1008143000000022.5%
Gemini 3 Flash (Preview, Reasoning)908143000000021.6%
Gemini 2.5 Pro1009810000000020.7%
GPT-5.4 (Reasoning)1001000000000020.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100990000000019.9%
DeepSeek V4 Flash (Reasoning)9060271400000019.3%
Z.AI GLM 4.7100866000000019.2%
GPT-5.6 Sol948610000000019.0%
Gemini 2.5 Flash (Reasoning)756843000000018.7%
Qwen 3.7 Flash (Reasoning)100860000000018.6%
Hermes 3 70B686843000000018.0%
Aion 3.0 Mini1005227000000017.9%
Claude Sonnet 5.5 (Reasoning)905210100000015.3%
DeepSeek V3 (2024-12-26)8135271000000015.3%
Mistral Small 4816010000000015.1%
Kimi K2.5993514000000014.8%
Aion 3.0100430000000014.3%
Qwen 3.5 27B962720000000014.3%
GPT-5.6 Luna99430000000014.3%
Writer: Palmyra X586520000000013.8%
Llama 3.1 70B98350000000013.3%
Thinking Machines Inkling (Reasoning)86430000000013.0%
GPT-5.6 Luna (Reasoning, Medium)100270000000012.7%
DeepSeek V3.290350000000012.6%
Gemma 3 12B100140000000011.4%
GPT-4.1 Mini86270000000011.4%
Gemini 3.5 Flash (Reasoning, Minimal)81270000000010.9%
Mistral Large 310062000000010.8%
Gemini 3.5 Flash Lite (Reasoning, Minimal)10040000000010.3%
Laguna S 2.168350000000010.3%
DeepSeek V3 (2025-03-24)9940000000010.3%
Grok 4.6 (Reasoning, High)10000000000010.0%
GPT-5.210000000000010.0%
Grok 4.3962000000009.8%
Ministral 3 3B980000000009.8%
Mistral Small 4 (Reasoning)944000000009.7%
Xiaomi MIMO v2.5 Pro961000000009.7%
DeepSeek-V2 Chat902000000009.3%
Hermes 3 405B900000000009.0%
Gemini 3 Flash (Preview)7514000000009.0%
Qwen 3.5 122B861000000008.7%
Qwen 3 32B860000000008.6%
ByteDance Seed 1.6 Flash860000000008.6%
Laguna XS 2.1814000000008.5%
Qwen 3.6 Flash60141000000008.4%
ByteDance Seed 2.0 Mini811000000008.2%
Ministral 3 8B810000000008.1%
Gemini 2.5 Flash Lite (Reasoning)604100000006.5%
Claude Sonnet 4.5602000000006.2%
ByteDance Seed 1.6526000000005.8%
Gemini 3.6 Flash (Reasoning, Minimal)524000000005.6%
Thinking Machines Inkling270000000002.7%
Grok 4.5 (Reasoning, Low)270000000002.7%
Claude Sonnet 4270000000002.7%
Mistral Small 3.2 24B270000000002.7%
Z.AI GLM 51410000000002.4%
Qwen 3.5 Flash200000000002.0%
ByteDance Seed 2.0 Lite140000000001.4%
Z.AI GLM 4.7 Flash100000000001.0%
Mistral Large 2100000000001.0%
Mistral Medium 3.560000000000.7%
Z.AI GLM 4.520000000000.2%
GPT-4o, Aug. 6th (temp=0)10000000000.1%
Muse Glimmer 30B (Reasoning, Medium)10000000000.1%
GPT-6 Luna00000000000.0%
Mistral NeMO00000000000.0%
Thinking Machines Inkling Small (Reasoning, High)00000000000.0%
Gemma 4 26B00000000000.0%
Qwen 3 235B A22B Instruct 250700000000000.0%
Z.AI GLM 4.5 Air00000000000.0%
Ministral 3B00000000000.0%
Ministral 3 14B00000000000.0%
Qwen 2.5 72B00000000000.0%
Gemma 4 31B00000000000.0%
Gemini 3.5 Flash Lite (Reasoning)00000000000.0%
WizardLM 2 8x22b00000000000.0%
Mistral Medium 3.5 (Reasoning)00000000000.0%
Qwen 3.5 Plus (2026-02-15)00000000000.0%
Ministral 8B00000000000.0%
GPT-5.4 Mini00000000000.0%
Gemma 3 4B00000000000.0%
GPT-4.1 Nano00000000000.0%
DeepSeek V4 Pro00000000000.0%
Mistral Large 4.000000000000.0%
Gemma 3 27B00000000000.0%
Claude Haiku 5.5 (Reasoning, Low)00 – – – – – – – – 0.0%
GPT-5.4 (Reasoning, Low)00000000000.0%
Claude Haiku 5.5 (Reasoning, Medium)00000000000.0%
GPT-5.6 Terra00000000000.0%
GPT-5.5 (Reasoning, Medium)00000000000.0%
GPT-5.400000000000.0%
GPT-5.5 (Reasoning, Low)00000000000.0%
GPT-5.4 Mini (Reasoning)00000000000.0%
GPT-5.500000000000.0%
Z.AI GLM 4.600000000000.0%
GPT-5.4 Mini (Reasoning, Low)00000000000.0%
GPT-5.4 Nano (Reasoning, Low)00000000000.0%
GPT-5.4 Nano00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)10010010010010010010099999999.6%
Muse Spark 1.2 (Reasoning, Medium)10010010010010010010098948697.7%
Hy4 Preview (Reasoning, High)100100100100100100100100994394.3%
Claude Opus 4.8 (Reasoning, Low)10010010010099999886817593.8%
Z.AI GLM 5.3 Flash (Reasoning, Max)10010010010099949490816091.7%
GPT-5 Mini100100100100100100100100100090.0%
DeepSeek V4.1 Flash (Reasoning, High)10010010010010010010010090489.4%
Claude Opus 4.6 (Reasoning)100100999696948668684385.1%
MiniMax M2.710010010010010010099992080.0%
Muse Spark 1.3 (Reasoning, Medium)1001001001001001001001000080.0%
Qwen 3.7 Max100100100100100100816843079.3%
Claude Sonnet 51001001001009990866843279.0%
MiniMax M2.5100100100100999996816078.1%
Claude Opus 4.6100100999996868168272077.7%
Gemini 3.1 Flash Lite (Reasoning)999998908681757543475.1%
Claude Haiku 5.5 (Reasoning, Max)1001001000 – – – – – – 75.0%
o4 Mini High1001001001001009968350070.3%
GPT-510010010010010010010000070.0%
Nemotron 3 Super100100100100100999610069.5%
Z.AI GLM 5.3 (Reasoning, Max)10010099969660524335068.1%
Muse Spark 1.1 (Reasoning, Medium)10010010099987560350066.7%
GPT-5.6 Sol (Reasoning, Medium)100100100100100996000065.9%
GPT-4o Mini (temp=0)1001009490907568270064.5%
Claude Opus 5 (Reasoning, Low)10098989086605235141064.3%
GPT-4o, Aug. 6th (temp=0)10010098966043434343062.7%
Inception Mercury 210010010010010094200059.6%
Claude Opus 5.5 (Reasoning)1001001009894681020057.1%
Gemini 3.1 Flash Lite989696968668610054.7%
Claude Sonnet 5 (Reasoning)100100908668602700053.2%
Gemini 3.1 Flash Lite (Preview)1001009460524343272052.2%
Claude Opus 51009490908143664051.6%
Z.AI GLM 5 Turbo10010010099992000050.0%
Claude Opus 4.7 (Reasoning)100988660524335201049.6%
Grok 4.3 (Reasoning)1001001001008610000049.5%
Claude Opus 5 (Reasoning)99989681433527102049.1%
Muse Glimmer 30B (Reasoning, Medium)10010098866835000048.7%
Gemini 3.5 Flash (Reasoning)10010099986010210047.0%
GPT-4o, Aug. 6th (temp=1)100100100966010400046.9%
Grok 4.20989694865235000046.1%
Xiaomi MIMO v2.6 Flash1009999756020000045.4%
Z.AI GLM 5.3 Flash (Reasoning, Low)1001009475750000044.4%
Claude Opus 49996904343352741044.0%
Aion 2.010010010068606000043.4%
Gemini 3 Flash (Preview)1001008181524000041.9%
Mistral Large 4.0 (Reasoning)10010099434320400040.9%
Qwen 3.6 Max Preview100100999940000040.1%
Aion 3.5 Mini (Reasoning, High)10010010090100000040.0%
Gemini 3.8 Flash (Reasoning, Medium)10010086602720600040.0%
Kimi K2.6100100989642000039.9%
Claude Opus 4.7100907552202014146639.9%
Gemini 2.5 Flash Lite (Reasoning)99969486202000039.7%
GPT-6 Luna (Reasoning, Medium)100989675270000039.6%
Nemotron 3 Nano1001001008160000038.7%
Kimi K3 (Reasoning, Low)10096946842000036.3%
Aion 3.0 Mini10094946800000035.5%
GPT-6 Sol (Reasoning, Medium)999681431414000034.8%
Gemini 3.6 Flash (Reasoning)10099993500000033.4%
Claude Sonnet 4.6100524343352714144033.3%
Gemini 2.5 Flash Lite10099904300000033.3%
Xiaomi MIMO v2.6 Pro10099941421100031.1%
GPT-5.6 Luna10099981400000031.1%
Claude Sonnet 5 (Reasoning, Low)94945243270000031.0%
Claude Opus 4.590909014144410030.9%
Gemini 3.5 Flash (Reasoning, Minimal)100100100000000030.0%
Grok 4.7 (Reasoning, High)1009996000000029.5%
Cohere Command R+ (Aug. 2024)10099523541000029.0%
DeepSeek V4 Flash988143352010000028.7%
Z.AI GLM 5.11009881610000028.6%
o4 Mini10010075200000027.7%
Mistral Medium 3.1999875100000027.3%
GPT-6 Sol9968525200000027.1%
Grok 4.6 (Reasoning, High)1009968100000026.8%
Qwen 3.5 122B1009868000000026.6%
Qwen 3.6 Flash1008168640000025.9%
ByteDance Seed 2.0 Lite988668000000025.2%
MiniMax M31009935660000024.6%
Claude Sonnet 4.6 (Reasoning)814343352014000023.8%
Gemini 3 Flash (Preview, Reasoning)1009935000000023.4%
GPT-4.1999635400000023.3%
Muse Spark 1.1 (Reasoning, Minimal)999935000000023.3%
Mistral Large 31009827420000023.0%
Laguna XS 2.11008127000000020.9%
GPT-4o Mini (temp=1)989810000000020.5%
DeepSeek-V2 Chat1001002000000020.2%
Z.AI GLM 59075201400000020.0%
GPT-6 Luna (Reasoning, High)1001000000000020.0%
Xiaomi MIMO v2.5 Pro1001000000000020.0%
Aion 3.0866843100000019.9%
Ministral 3 14B99990000000019.8%
DeepSeek V4 Pro (Reasoning)1008610000000019.6%
GPT-4.1 Mini99960000000019.5%
GPT-6 Luna99940000000019.2%
Cydonia 24B V4.11007510100000018.6%
DeepSeek V3 (2024-12-26)756043600000018.5%
Mistral Small 4946027200000018.4%
Qwen 3.5 Plus (2026-04-20)986020000000017.8%
Qwen 2.5 72B994327200000017.1%
GPT-OSS 120B96750000000017.1%
Mistral NeMO100680000000016.8%
Kimi K3 (Reasoning, High)1005214000000016.6%
DeepSeek V4 Pro 0813 (Reasoning, High)1003527000000016.2%
Arcee AI: Trinity Mini100430000000014.3%
GPT-5.6 Sol81600000000014.2%
Z.AI GLM 5.2 (Reasoning, High)90430000000013.4%
Grok 4.386356000000012.7%
Grok 4.5 (Reasoning, Low)94140000000010.8%
Ministral 3 8B68276400000010.6%
GPT-5 Nano10010000000010.1%
Ministral 8B10000000000010.0%
Mistral Small 3.2 24B980000000009.8%
Qwen 3.5 9B960000000009.6%
ByteDance Seed 2.0 Mini960000000009.6%
Gemini 2.5 Flash940000000009.4%
Grok 4.20 (Reasoning)902100000009.4%
Qwen 3.5 27B861000000008.7%
Llama 3.1 70B860000000008.6%
Ministral 3B860000000008.6%
DeepSeek V4 Flash (Reasoning)6814100000008.3%
Gemma 3 12B811000000008.2%
Laguna S 2.16814000000008.2%
Qwen 3 235B A22B Instruct 2507752000000007.7%
Grok 4.5 (Reasoning, High)680000000006.8%
Mistral Large 2680000000006.8%
Writer: Palmyra X5680000000006.8%
ByteDance Seed 1.6680000000006.8%
Gemini 2.5 Pro3520200000005.7%
Z.AI GLM 4.735101000000005.5%
Gemini 3.7 Flash (Reasoning, Medium)431100000004.6%
Kimi K2.5430000000004.4%
Ministral 3 3B430000000004.3%
DeepSeek V3 (2025-03-24)430000000004.3%
Qwen 3.6 35B430000000004.3%
DeepSeek V3.1430000000004.3%
Qwen 3.5 397B A17B350000000003.5%
Gemma 4 26B270000000002.7%
Gemma 4 26B (Reasoning)270000000002.7%
Aion 3.5 (Reasoning, High)202200000002.5%
Qwen 3.7 Flash (Reasoning)204000000002.4%
Claude Haiku 4.5202100000002.3%
Claude Sonnet 5.5 (Reasoning)140000000001.4%
GPT-5.1140000000001.4%
Hermes 3 70B64200000001.2%
Thinking Machines Inkling64000000001.0%
Gemini 3.6 Flash (Reasoning, Minimal)100000000001.0%
Z.AI GLM 4.5 Air61000000000.7%
Gemini 3.5 Flash Lite (Reasoning)60000000000.6%
Hermes 3 405B40000000000.4%
Xiaomi MIMO v2.520000000000.2%
Qwen 3.5 Plus (2026-02-15)20000000000.2%
DeepSeek V3.210000000000.1%
Gemma 4 31B (Reasoning)00000000000.0%
Qwen 3 32B00000000000.0%
DeepSeek V4 Flash 0731 (Reasoning, High)00000000000.0%
GPT-5.5 (Reasoning, Medium)00000000000.0%
Claude Sonnet 4.500000000000.0%
Qwen 3.6 27B00000000000.0%
GPT-4.1 Nano00000000000.0%
Claude Haiku 5.5 (Adaptive)00000000000.0%
GPT-5.6 Luna (Reasoning, Medium)00000000000.0%
Mistral Medium 3.500000000000.0%
Thinking Machines Inkling (Reasoning)00000000000.0%
Z.AI GLM 4.500000000000.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)00000000000.0%
Gemini 2.5 Flash (Reasoning)00000000000.0%
Claude Sonnet 400000000000.0%
Qwen 3.5 35B00000000000.0%
Mistral Small 4 (Reasoning)00000000000.0%
Gemma 3 27B00000000000.0%
Z.AI GLM 4.7 Flash00000000000.0%
Gemma 4 31B00000000000.0%
ByteDance Seed 1.6 Flash00000000000.0%
Z.AI GLM 4.600000000000.0%
GPT-5.6 Terra (Reasoning, Medium)00000000000.0%
GPT-5.4 Mini00000000000.0%
Claude Haiku 5.5 (Reasoning, Low)00 – – – – – – – – 0.0%
Qwen 3.5 Flash00000000000.0%
WizardLM 2 8x22b00000000000.0%
Claude Haiku 5.5 (Reasoning, Medium)00000000000.0%
GPT-5.5 (Reasoning, Low)00000000000.0%
DeepSeek V4 Pro00000000000.0%
GPT-5.4 Mini (Reasoning, Low)00000000000.0%
GPT-5.6 Terra00000000000.0%
Mistral Medium 3.5 (Reasoning)00000000000.0%
Mistral Large 4.000000000000.0%
Gemma 3 4B00000000000.0%
GPT-5.4 (Reasoning)00000000000.0%
GPT-5.4 (Reasoning, Low)00000000000.0%
GPT-5.4 Mini (Reasoning)00000000000.0%
GPT-5.200000000000.0%
GPT-5.500000000000.0%
Thinking Machines Inkling Small (Reasoning, High)00000000000.0%
GPT-5.400000000000.0%
GPT-5.4 Nano (Reasoning)00000000000.0%
GPT-5.4 Nano (Reasoning, Low)00000000000.0%
GPT-5.4 Nano00000000000.0%