Isolated character handling

Test: Relationship tree

Avg. Score
85.5%
Scenarios
2

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Gemini 3.1 Flash Lite100.0%$0.00313.3s100%
2Gemini 3.1 Flash Lite (Reasoning)100.0%$0.00333.2s100%
3Gemini 2.5 Flash Lite100.0%$0.00234.8s100%
4Gemini 3.1 Flash Lite (Preview)100.0%$0.00463.2s100%
5Gemini 3.5 Flash Lite (Reasoning, Minimal)100.0%$0.00573.2s100%
6Gemini 3.5 Flash Lite (Reasoning)100.0%$0.00623.1s100%
7Gemini 3 Flash (Preview)100.0%$0.00877.1s100%
8Inception Mercury 2100.0%$0.009612.8s100%
9Claude Haiku 5.5 (Reasoning, Low)100.0%$0.005019.9s100%
10DeepSeek V4 Pro100.0%$0.008317.3s100%
11Gemma 4 26B100.0%$0.001827.2s100%
12Claude Haiku 5.5 (Adaptive)100.0%$0.006429.8s100%
13Gemini 3.5 Flash (Reasoning, Minimal)100.0%$0.0246.3s100%
14DeepSeek V3.2100.0%$0.004633.9s100%
15Claude Haiku 4.5100.0%$0.0248.2s100%
16Z.AI GLM 5.3 Flash (Reasoning, Low)100.0%$0.001242.5s100%
17Gemini 3.6 Flash (Reasoning, Minimal)100.0%$0.0297.1s100%
18Mistral Medium 3.5 (Reasoning)100.0%$0.0328.6s100%
19DeepSeek V4 Flash100.0%$0.002051.4s100%
20GPT-5.6 Luna (Reasoning, Medium)100.0%$0.02718.6s100%
21GPT-5.6 Terra100.0%$0.0349.4s100%
22Muse Spark 1.1 (Reasoning, Minimal)100.0%$0.03018.2s100%
23DeepSeek V4.1 Flash 1017 (Reasoning, High)100.0%$0.005658.2s100%
24GPT-6 Sol100.0%$0.03521.1s100%
25Gemini 2.5 Flash (Reasoning)100.0%$0.02337.3s100%
26Gemini 2.5 Flash Lite (Reasoning)100.0%$0.00731.0m100%
27Xiaomi MIMO v2.5100.0%$0.00271.1m100%
28Gemini 4 Argon (Reasoning, Medium)100.0%$0.03919.1s100%
29Muse Glimmer 30B (Reasoning, Medium)100.0%$0.00621.1m100%
30Qwen 3.6 Flash100.0%$0.01551.7s100%
31GPT-6 Luna (Reasoning, Medium)100.0%$0.00351.1m100%
32Gemini 3.7 Flash (Reasoning, Medium)100.0%$0.03623.4s100%
33Gemini 3 Flash (Preview, Reasoning)100.0%$0.03040.3s100%
34Thinking Machines Inkling Small (Reasoning, High)100.0%$0.0131.1m100%
35ByteDance Seed 1.6100.0%$0.0141.2m100%
36Muse Spark 1.2 (Reasoning, Medium)100.0%$0.04629.8s100%
37Muse Spark 1.1 (Reasoning, Medium)100.0%$0.04235.1s100%
38GPT-6 Luna (Reasoning, High)100.0%$0.00551.4m100%
39Grok 4.5 (Reasoning, Low)100.0%$0.04049.7s100%
40DeepSeek V4.1 Flash (Reasoning, High)100.0%$0.0141.5m100%
41GPT-5.6 Terra (Reasoning, Medium)100.0%$0.06024.2s100%
42GPT-6.1 Sol (Reasoning, Medium)100.0%$0.04250.1s100%
43GPT-6 Sol (Reasoning, Medium)100.0%$0.05146.3s100%
44Claude Sonnet 4100.0%$0.07217.5s100%
45Gemma 4 31B100.0%$0.00281.9m100%
46Claude Sonnet 4.5100.0%$0.07718.6s100%
47Xiaomi MIMO v2.6 Pro100.0%$0.00712.0m100%
48Gemini 3.8 Flash (Reasoning, Medium)100.0%$0.06742.1s100%
49Claude Sonnet 4.6100.0%$0.08123.3s100%
50GPT-5.2100.0%$0.06250.7s100%
51GPT-OSS 120B100.0%$0.00402.2m100%
52Qwen 3.5 35B100.0%$0.0152.0m100%
53GPT-5.6 Sol100.0%$0.09618.1s100%
54Kimi K3 (Reasoning, Low)100.0%$0.0591.2m100%
55o4 Mini High100.0%$0.0591.3m100%
56Z.AI GLM 5.2 (Reasoning, High)100.0%$0.0391.8m100%
57DeepSeek V4 Pro (Reasoning)100.0%$0.0202.3m100%
58Claude Sonnet 5.5 (Reasoning)100.0%$0.10128.5s100%
59Gemini 3.6 Flash (Reasoning)100.0%$0.09641.4s100%
60Muse Wonder (Reasoning, Medium)100.0%$0.11620.5s100%
61GPT-5.5100.0%$0.10537.1s100%
62Muse Spark 1.3 (Reasoning, Medium)100.0%$0.0551.9m100%
63Gemma 4 26B (Reasoning)100.0%$0.00413.1m100%
64Gemini 2.5 Pro100.0%$0.09958.2s100%
65Claude Opus 4.5100.0%$0.13119.8s100%
66Gemini 3.5 Flash (Reasoning)100.0%$0.11349.4s100%
67DeepSeek V4 Flash (Reasoning)100.0%$0.00413.5m100%
68Qwen 3.5 Plus (2026-04-20)100.0%$0.0233.1m100%
69Grok 4.5 (Reasoning, High)100.0%$0.0802.0m100%
70ByteDance Seed 2.0 Lite100.0%$0.0143.6m100%
71Gemini 2.5 Flash94.8%$0.00756.1s69%
72Claude Opus 4.6100.0%$0.15431.4s100%
73Z.AI GLM 4.7100.0%$0.0253.7m100%
74GPT-5.4 Mini (Reasoning, Low)94.8%$0.01414.4s69%
75Qwen 3.5 27B100.0%$0.0293.7m100%
76GPT-5100.0%$0.0922.3m100%
77Qwen 3.5 Plus (2026-02-15)100.0%$0.0213.9m100%
78Z.AI GLM 5 Turbo100.0%$0.0443.6m100%
79Qwen 3.7 Max100.0%$0.0453.5m100%
80Claude Opus 4.7100.0%$0.18224.3s100%
81Claude Opus 4.7 (Reasoning)100.0%$0.18425.0s100%
82Qwen 3.5 122B100.0%$0.0354.0m100%
83Claude Opus 5100.0%$0.19526.4s100%
84DeepSeek V4 Pro 0813 (Reasoning, High)100.0%$0.0603.5m100%
85Mistral Large 4.0 (Reasoning)100.0%$0.0453.9m100%
86Qwen 3.8 Flash (Reasoning)100.0%$0.0154.9m100%
87GPT-5.6 Sol (Reasoning, Medium)100.0%$0.18659.0s100%
88Claude Opus 5.5 (Reasoning)100.0%$0.20340.6s100%
89Gemini 3.1 Pro (Preview)100.0%$0.1721.5m100%
90 Kimi K3 (Reasoning, High)100.0%$0.1023.1m100%
91GPT-5.4 Mini (Reasoning)100.0%$0.1172.8m100%
92Thinking Machines Inkling (Reasoning)100.0%$0.0604.1m100%
93GPT-5.4 (Reasoning, Low)94.8%$0.05335.5s69%
94Claude Sonnet 594.8%$0.06817.1s69%
95Qwen 3.6 Max Preview100.0%$0.0843.7m100%
96Claude Sonnet 5 (Reasoning)100.0%$0.1721.7m100%
97Claude Sonnet 5 (Reasoning, Low)100.0%$0.1751.7m100%
98Claude Haiku 5.5 (Reasoning, Medium)89.6%$0.004616.7s59%
99DeepSeek V4 Flash 0731 (Reasoning, High)100.0%$0.00735.6m100%
100GPT-5.493.2%$0.03317.4s59%
101Hy4 Preview (Reasoning, High)100.0%$0.0564.7m100%
102Grok 4.6 (Reasoning, High)100.0%$0.1043.7m100%
103Mistral Small 3.2 24B92.0%$0.001713.8s52%
104Kimi K2.5100.0%$0.0315.4m100%
105Qwen 3.8 27B (Reasoning, XHigh)100.0%$0.0534.9m100%
106DeepSeek V3 (2024-12-26)89.6%$0.004540.2s59%
107Claude Opus 5 (Reasoning, Low)100.0%$0.24646.5s100%
108Claude Opus 5 (Reasoning)100.0%$0.24747.3s100%
109DeepSeek-V2 Chat92.0%$0.004934.0s52%
110GPT-5.4 (Reasoning)100.0%$0.1752.6m100%
111Kimi K2.6100.0%$0.0825.0m100%
112Xiaomi MIMO v2.5 Pro93.2%$0.00761.9m59%
113Grok 4.390.6%$0.00947.8s44%
114Qwen 3.6 27B94.8%$0.0342.7m69%
115Xiaomi MIMO v2.6 Flash100.0%$0.00787.0m100%
116Z.AI GLM 4.694.8%$0.0183.0m69%
117GPT-5 Mini93.2%$0.0192.0m59%
118Qwen 3.5 Flash94.8%$0.00403.8m69%
119Llama 3.1 70B86.8%$0.006434.0s46%
120GPT-5.5 (Reasoning, Medium)100.0%$0.2512.0m100%
121Grok 4.3 (Reasoning)90.0%$0.01313.9s40%
122Qwen 3.5 9B100.0%$0.00407.8m100%
123Mistral Large 4.082.8%$0.008030.1s47%
124Z.AI GLM 5.3 Flash (Reasoning, Max)100.0%$0.00628.0m100%
125DeepSeek V3.190.0%$0.005552.0s40%
126Ember-1 (Reasoning, Max)100.0%$0.2113.4m100%
127Grok 4.7 (Reasoning, High)100.0%$0.1315.4m100%
128GPT-5 Nano88.0%$0.00802.2m51%
129Qwen 3.6 35B90.0%$0.0141.1m40%
130o4 Mini90.0%$0.03745.7s40%
131Ministral 3 8B80.0%$0.002217.0s37%
132GPT-5.6 Luna79.5%$0.0128.7s37%
133Z.AI GLM 5.1100.0%$0.0817.3m100%
134Grok 4.20 (Reasoning)81.6%$0.02740.0s42%
135Thinking Machines Inkling80.3%$0.01843.6s36%
136Qwen 3.5 397B A17B88.0%$0.0452.6m51%
137Mistral Medium 3.582.6%$0.0338.3s30%
138MiniMax M3100.0%$0.0369.3m100%
139GPT-5.5 (Reasoning, Low)90.0%$0.09742.6s40%
140Z.AI GLM 4.582.0%$0.007548.1s27%
141Claude Opus 4.8 (Reasoning, Low)100.0%$0.3831.9m100%
142Claude Opus 4.8 (Reasoning)100.0%$0.3811.9m100%
143Claude Opus 4.6 (Reasoning)100.0%$0.3562.6m100%
144Claude Opus 4100.0%$0.3682.4m100%
145GPT-4.175.8%$0.0248.5s25%
146GPT-6 Luna70.1%$0.001212.1s25%
147Ministral 8B74.4%$0.001615.6s21%
148MiniMax M2.580.0%$0.003645.0s20%
149GPT-5.4 Nano (Reasoning)76.1%$0.008841.7s24%
150Mistral Medium 3.174.6%$0.01021.2s22%
151Qwen 3.7 Flash (Reasoning)80.0%$0.001756.4s20%
152Mistral Small 4 (Reasoning)67.4%$0.005124.7s17%
153Z.AI GLM 590.0%$0.0374.6m40%
154GPT-5.164.0%$0.02517.7s19%
155Gemma 4 31B (Reasoning)90.0%$0.00435.8m40%
156Qwen 3.8 Max (Reasoning, XHigh)100.0%$0.1609.6m100%
157Hermes 3 405B66.1%$0.0151.0m13%
158GPT-4o Mini (temp=1)62.6%$0.002633.4s8%
159WizardLM 2 8x22b68.6%$0.00982.3m19%
160GPT-4o, Aug. 6th (temp=0)59.9%$0.0328.4s12%
161MiniMax M2.758.0%$0.006321.4s8%
162GPT-5.4 Nano (Reasoning, Low)54.4%$0.003813.8s9%
163Z.AI GLM 4.5 Air73.8%$0.00713.2m19%
164Z.AI GLM 5.3 (Reasoning, Max)100.0%$0.18710.3m100%
165GPT-4.1 Nano54.6%$0.00096.9s5%
166Gemma 3 27B53.9%$0.001627.5s9%
167Nemotron 3 Nano68.0%$0.00392.8m17%
168Mistral Large 258.2%$0.03720.5s10%
169Mistral Small 450.9%$0.00307.4s6%
170Aion 3.5 (Reasoning, High)90.0%$0.0836.1m40%
171GPT-4o, Aug. 6th (temp=1)55.1%$0.0299.1s6%
172Cydonia 24B V4.156.1%$0.00401.0m6%
173Mistral Large 353.1%$0.009117.5s3%
174Laguna XS 2.151.1%$0.001241.3s6%
175DeepSeek V3 (2025-03-24)54.8%$0.003750.3s4%
176Z.AI GLM 4.7 Flash61.9%$0.00502.8m15%
177Aion 3.5 Mini (Reasoning, High)90.0%$0.0367.7m40%
178Aion 3.0 Mini80.0%$0.0304.9m20%
179GPT-4.1 Mini47.9%$0.004626.0s3%
180Claude Sonnet 4.6 (Reasoning)100.0%$0.4485.9m100%
181Aion 2.070.0%$0.0243.0m8%
182Writer: Palmyra X547.9%$0.01618.4s3%
183GPT-4o Mini (temp=0)45.4%$0.001920.7s2%
184ByteDance Seed 1.6 Flash43.9%$0.002034.2s5%
185Qwen 2.5 72B52.6%$0.00561.2m3%
186Hermes 3 70B43.8%$0.004542.7s7%
187Qwen 3 235B A22B Instruct 250732.7%$0.00171.0m13%
188Aion 3.080.0%$0.0745.8m20%
189Nemotron 3 Super70.0%$0.00005.5m8%
190Laguna S 2.141.1%$0.00062.0m3%
191Grok 4.2027.7%$0.0207.5s2%
192ByteDance Seed 2.0 Mini81.3%$0.00628.7m25%
193Cohere Command R+ (Aug. 2024)33.4%$0.05756.5s5%
194Ministral 3 14B18.3%$0.003125.4s3%
195GPT-5.4 Mini11.8%$0.00877.9s5%
196Ministral 3B8.4%$0.00079.8s6%
197Ministral 3 3B9.2%$0.00079.9s5%
198GPT-5.4 Nano6.6%$0.002611.2s6%
199Mistral NeMO3.8%$0.002415.4s6%
200Gemma 3 12B10.6%$0.001430.4s0%
201Qwen 3 32B32.6%$0.00422.9m0%
202Gemma 3 4B5.7%$0.000727.5s3%
203Arcee AI: Trinity Mini13.5%$0.00371.9m0%
85.53%

Individual Scenarios

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100.0%
Ember-1 (Reasoning, Max)100100100100100100.0%
Muse Wonder (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100.0%
Qwen 3.7 Max100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
DeepSeek V4.1 Flash 1017 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Gemini 4 Argon (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100.0%
Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100.0%
Aion 3.5 Mini (Reasoning, High)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
Aion 3.5 (Reasoning, High)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100.0%
GPT-5100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100.0%
MiniMax M3100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
GPT-6 Sol100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Claude Opus 5100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.2100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100.0%
GPT-5.5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100.0%
Aion 3.0100100100100100100.0%
o4 Mini High100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100.0%
Gemini 2.5 Pro100100100100100100.0%
Qwen 3.6 27B100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Z.AI GLM 4.6100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100.0%
Claude Opus 4100100100100100100.0%
Qwen 3.5 35B100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 Flash100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100100100100100.0%
Mistral Medium 3.5100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
DeepSeek-V2 Chat100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
Claude Haiku 4.5100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
DeepSeek V4 Pro100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100.0%
Llama 3.1 70B100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
Mistral Medium 3.1100100100100100100.0%
GPT-5 Mini1001001001003286.3%
Qwen 3.5 397B A17B1001001001003286.3%
GPT-4.11001001001003286.3%
Xiaomi MIMO v2.5 Pro1001001001003286.3%
Mistral Large 4.01001001001003286.3%
GPT-5.41001001001003286.3%
GPT-4o, Aug. 6th (temp=1)1001001001003286.3%
GPT-5 Nano1001001001003286.3%
Nemotron 3 Nano1001001001003286.3%
Ministral 3 8B1001001001003286.3%
Hermes 3 405B100100100100681.3%
Grok 4.3100100100100681.3%
GPT-4o Mini (temp=1)100100100100681.3%
GPT-4o Mini (temp=0)100100100100681.3%
Thinking Machines Inkling100100100100681.3%
GPT-4.1 Nano100100100100681.3%
Grok 4.3 (Reasoning)100100100100080.0%
MiniMax M2.5100100100100080.0%
o4 Mini100100100100080.0%
Z.AI GLM 4.5100100100100080.0%
DeepSeek V3.1100100100100080.0%
Nemotron 3 Super100100100100080.0%
Z.AI GLM 4.5 Air10010010032667.6%
Z.AI GLM 4.7 Flash10010010032066.3%
Gemma 3 27B10010010032066.3%
Laguna XS 2.110010010032066.3%
WizardLM 2 8x22b10010010032066.3%
ByteDance Seed 2.0 Mini1001001006662.5%
GPT-5.4 Nano (Reasoning)1001001006662.5%
Mistral Small 41001001006662.5%
Mistral Small 4 (Reasoning)1001001006061.3%
Qwen 2.5 72B1001001006061.3%
DeepSeek V3 (2025-03-24)1001001000060.0%
GPT-5.110010032323259.0%
GPT-5.6 Luna10010032323259.0%
GPT-5.4 Nano (Reasoning, Low)1001003232653.9%
ByteDance Seed 1.6 Flash1001003232052.7%
Hermes 3 70B1001003232052.7%
Laguna S 2.1100100326648.8%
Ministral 8B100100326648.8%
MiniMax M2.7100100320046.3%
Grok 4.2010010066643.8%
Cydonia 24B V4.110010066042.5%
GPT-6 Luna100323232640.2%
Aion 2.010010000040.0%
Cohere Command R+ (Aug. 2024)1003260027.6%
GPT-4.1 Mini100666625.0%
Writer: Palmyra X5100666625.0%
Qwen 3 32B100600021.3%
Mistral Large 2323266616.4%
Qwen 3 235B A22B Instruct 2507323266616.4%
Gemma 3 4B32666611.3%
Arcee AI: Trinity Mini3200006.4%
Mistral Large 3666666.3%
GPT-5.4 Mini666666.3%
GPT-5.4 Nano666666.3%
Ministral 3 14B666666.3%
Mistral NeMO666666.3%
Ministral 3 3B666605.1%
Ministral 3B666605.1%
Gemma 3 12B600001.3%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100.0%
Ember-1 (Reasoning, Max)100100100100100100.0%
Muse Wonder (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100.0%
Hy4 Preview (Reasoning, High)100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100.0%
Qwen 3.7 Max100100100100100100.0%
DeepSeek V4.1 Flash (Reasoning, High)100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
DeepSeek V4.1 Flash 1017 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100.0%
GPT-6 Luna (Reasoning, High)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Gemini 4 Argon (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
Kimi K3 (Reasoning, High)100100100100100100.0%
Qwen 3.8 27B (Reasoning, XHigh)100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100.0%
Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.6 Terra (Reasoning, Medium)100100100100100100.0%
GPT-6 Luna (Reasoning, Medium)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
Mistral Large 4.0 (Reasoning)100100100100100100.0%
Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
Xiaomi MIMO v2.6 Pro100100100100100100.0%
GPT-5100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
GPT-5 Mini100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
Kimi K2.5100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning, Medium)100100100100100100.0%
MiniMax M3100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Qwen 3.5 122B100100100100100100.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
GPT-6 Sol100100100100100100.0%
Qwen 3.5 27B100100100100100100.0%
Claude Opus 5100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.2100100100100100100.0%
Claude Haiku 5.5 (Adaptive)100100100100100100.0%
GPT-5.5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100.0%
o4 Mini High100100100100100100.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100100.0%
Gemini 2.5 Pro100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100.0%
Z.AI GLM 4.7100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Thinking Machines Inkling Small (Reasoning, High)100100100100100100.0%
Xiaomi MIMO v2.6 Flash100100100100100100.0%
Claude Opus 4100100100100100100.0%
Qwen 3.5 35B100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Aion 2.0100100100100100100.0%
o4 Mini100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
ByteDance Seed 2.0 Mini100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Mistral Medium 3.5 (Reasoning)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Claude Haiku 5.5 (Reasoning, Low)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
GPT-5.4100100100100100100.0%
Mistral Large 3100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100.0%
GPT-6 Luna100100100100100100.0%
Claude Haiku 4.5100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100.0%
DeepSeek V3.1100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
DeepSeek V4 Pro100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Inception Mercury 2100100100100100100.0%
Mistral Large 2100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100.0%
Grok 4.3100100100100100100.0%
Ministral 8B100100100100100100.0%
Qwen 3.5 397B A17B1001001001004889.6%
GPT-5.4 (Reasoning, Low)1001001001004889.6%
Qwen 3.6 27B1001001001004889.6%
Z.AI GLM 4.61001001001004889.6%
Claude Sonnet 51001001001004889.6%
Qwen 3.5 Flash1001001001004889.6%
GPT-5.4 Mini (Reasoning, Low)1001001001004889.6%
Gemini 2.5 Flash1001001001004889.6%
GPT-5 Nano1001001001004889.6%
GPT-5.4 Nano (Reasoning)1001001001004889.6%
Z.AI GLM 4.51001001001002084.0%
DeepSeek-V2 Chat1001001001002084.0%
Mistral Small 3.2 24B1001001001002084.0%
Z.AI GLM 4.5 Air100100100100080.0%
GPT-5.5 (Reasoning, Low)100100100100080.0%
Aion 3.5 Mini (Reasoning, High)100100100100080.0%
Aion 3.5 (Reasoning, High)100100100100080.0%
Gemma 4 31B (Reasoning)100100100100080.0%
Z.AI GLM 5100100100100080.0%
Qwen 3.6 35B100100100100080.0%
MiniMax M2.5100100100100080.0%
Claude Haiku 5.5 (Reasoning, Medium)100100100484879.3%
Mistral Large 4.0100100100484879.3%
DeepSeek V3 (2024-12-26)100100100484879.3%
Thinking Machines Inkling100100100484879.3%
Llama 3.1 70B100100100482073.6%
Mistral Small 4 (Reasoning)100100100482073.6%
Ministral 3 8B100100100482073.6%
GPT-4.1 Mini10010010048670.9%
Writer: Palmyra X510010010048670.9%
WizardLM 2 8x22b10010010048670.9%
MiniMax M2.710010010048069.6%
Cydonia 24B V4.110010010048069.6%
GPT-5.110010048484868.9%
GPT-4.110010010020665.2%
Mistral Medium 3.510010010020665.2%
Grok 4.20 (Reasoning)10010048482063.2%
Aion 3.01001001000060.0%
Qwen 3.7 Flash (Reasoning)1001001000060.0%
Aion 3.0 Mini1001001000060.0%
Nemotron 3 Super1001001000060.0%
Z.AI GLM 4.7 Flash10010048202057.5%
GPT-5.4 Nano (Reasoning, Low)1001004820654.8%
Hermes 3 405B100100486050.9%
DeepSeek V3 (2025-03-24)100100480049.6%
Nemotron 3 Nano100100480049.6%
Mistral Medium 3.11001002020649.2%
Qwen 3 235B A22B Instruct 2507100484848048.9%
Qwen 2.5 72B100100200044.0%
Qwen 3 32B100100200044.0%
GPT-4o Mini (temp=1)100100200044.0%
Gemma 3 27B1004820202041.5%
Mistral Small 410048480039.3%
Cohere Command R+ (Aug. 2024)10048480039.3%
Laguna XS 2.1100482012035.9%
Hermes 3 70B10048206034.8%
Laguna S 2.11004866633.4%
Ministral 3 14B10020206630.4%
GPT-4.1 Nano10020200027.9%
GPT-4o, Aug. 6th (temp=1)1002000024.0%
Arcee AI: Trinity Mini484860020.5%
Gemma 3 12B100000020.0%
GPT-4o, Aug. 6th (temp=0)202020202019.8%
GPT-5.4 Mini482066617.3%
Ministral 3 3B48666013.4%
Grok 4.20202066611.7%
Ministral 3B202066611.7%
GPT-4o Mini (temp=0)4800009.6%
GPT-5.4 Nano2066116.9%
Mistral NeMO600001.3%
ByteDance Seed 1.6 Flash0 – – – – 0.1%
Gemma 3 4B000000.1%