Relationship type accuracy

Test: Relationship tree

Avg. Score
78.7%
Scenarios
2

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Gemma 3 12B100.0%$0.001430.4s100%
2GPT-6.1 Sol (Reasoning, Medium)100.0%$0.04250.1s100%
3GPT-6 Sol97.5%$0.03521.1s87%
4GPT-5.4 Nano (Reasoning)95.0%$0.008841.7s86%
5GPT-4o, Aug. 6th (temp=0)94.8%$0.0328.4s86%
6GPT-6 Sol (Reasoning, Medium)98.3%$0.05146.3s90%
7Gemini 2.5 Flash92.3%$0.00756.1s83%
8GPT-5.194.6%$0.02517.7s85%
9Z.AI GLM 4.594.0%$0.007548.1s84%
10Gemini 4 Argon (Reasoning, Medium)94.8%$0.03919.1s84%
11Grok 4.2093.0%$0.0207.5s81%
12GPT-5.4 (Reasoning, Low)95.2%$0.05335.5s84%
13GPT-5 Mini95.7%$0.0192.0m86%
14o4 Mini94.3%$0.03745.7s80%
15Gemini 3.7 Flash (Reasoning, Medium)91.5%$0.03623.4s78%
16Claude Opus 4.596.0%$0.13119.8s89%
17Grok 4.20 (Reasoning)91.6%$0.02740.0s77%
18GPT-5.294.2%$0.06250.7s82%
19GPT-6 Luna (Reasoning, High)91.6%$0.00551.4m78%
20ByteDance Seed 1.690.6%$0.0141.2m76%
21o4 Mini High93.3%$0.0591.3m81%
22Muse Spark 1.1 (Reasoning, Minimal)88.0%$0.03018.2s73%
23DeepSeek V3.188.1%$0.005552.0s69%
24DeepSeek V4.1 Flash 1017 (Reasoning, High)86.2%$0.005658.2s72%
25Mistral Large 386.9%$0.009117.5s67%
26Gemini 2.5 Flash (Reasoning)88.7%$0.02337.3s69%
27GPT-5.6 Terra87.0%$0.0349.4s69%
28GPT-5.4 (Reasoning)99.5%$0.1752.6m98%
29Gemini 3 Flash (Preview, Reasoning)87.1%$0.03040.3s71%
30GPT-6 Luna (Reasoning, Medium)87.4%$0.00351.1m69%
31Mistral Large 289.4%$0.03720.5s66%
32Grok 4.5 (Reasoning, High)93.1%$0.0802.0m83%
33DeepSeek V4.1 Flash (Reasoning, High)85.2%$0.0141.5m75%
34GPT-594.0%$0.0922.3m84%
35GPT-5.6 Terra (Reasoning, Medium)88.3%$0.06024.2s70%
36Claude Opus 4.694.5%$0.15431.4s80%
37Muse Spark 1.1 (Reasoning, Medium)86.8%$0.04235.1s70%
38Gemini 3.8 Flash (Reasoning, Medium)88.7%$0.06742.1s73%
39DeepSeek V3.285.1%$0.004633.9s64%
40Claude Sonnet 5.5 (Reasoning)84.8%$0.10128.5s81%
41GPT-5.6 Sol (Reasoning, Medium)96.5%$0.18659.0s86%
42Z.AI GLM 5.3 Flash (Reasoning, Low)78.7%$0.001242.5s69%
43GPT-5.6 Luna (Reasoning, Medium)83.6%$0.02718.6s65%
44Mistral Medium 3.183.3%$0.01021.2s62%
45Muse Spark 1.2 (Reasoning, Medium)83.1%$0.04629.8s70%
46GPT-5.483.6%$0.03317.4s65%
47Gemini 3 Flash (Preview)75.2%$0.00877.1s68%
48Mistral Large 4.0 (Reasoning)94.3%$0.0453.9m82%
49Muse Spark 1.3 (Reasoning, Medium)83.3%$0.0551.9m81%
50Inception Mercury 278.0%$0.009612.8s65%
51Gemini 3.6 Flash (Reasoning)87.6%$0.09641.4s73%
52GPT-5.6 Sol87.6%$0.09618.1s70%
53Xiaomi MIMO v2.5 Pro87.1%$0.00761.9m67%
54Gemini 3.6 Flash (Reasoning, Minimal)82.7%$0.0297.1s62%
55Grok 4.5 (Reasoning, Low)85.4%$0.04049.7s65%
56DeepSeek V3 (2025-03-24)85.7%$0.003750.3s59%
57Claude Haiku 5.5 (Adaptive)76.5%$0.006429.8s67%
58Muse Wonder (Reasoning, Medium)84.8%$0.11620.5s75%
59Gemini 2.5 Pro88.5%$0.09958.2s72%
60GPT-4.182.0%$0.0248.5s59%
61Aion 2.090.0%$0.0243.0m73%
62GPT-5.5 (Reasoning, Low)88.0%$0.09742.6s70%
63Gemini 2.5 Flash Lite (Reasoning)83.3%$0.00731.0m61%
64Xiaomi MIMO v2.583.8%$0.00271.1m61%
65Claude Opus 591.1%$0.19526.4s81%
66GPT-5.580.7%$0.10537.1s78%
67GPT-4o, Aug. 6th (temp=1)82.9%$0.0299.1s58%
68DeepSeek V4 Flash 0731 (Reasoning, High)94.4%$0.00735.6m84%
69Qwen 3.7 Flash (Reasoning)82.3%$0.001756.4s60%
70Ministral 3 14B80.8%$0.003125.4s57%
71DeepSeek V4 Flash (Reasoning)88.9%$0.00413.5m71%
72Xiaomi MIMO v2.6 Pro85.0%$0.00712.0m64%
73MiniMax M2.578.0%$0.003645.0s61%
74Kimi K3 (Reasoning, Low)79.6%$0.0591.2m72%
75Claude Sonnet 4.676.7%$0.08123.3s72%
76Grok 4.6 (Reasoning, High)93.4%$0.1043.7m82%
77Gemini 3.5 Flash (Reasoning, Minimal)81.2%$0.0246.3s54%
78Gemini 3.5 Flash Lite (Reasoning, Minimal)80.0%$0.00573.2s52%
79Thinking Machines Inkling (Reasoning)91.8%$0.0604.1m80%
80Claude Haiku 5.5 (Reasoning, Low)71.2%$0.005019.9s63%
81ByteDance Seed 1.6 Flash81.7%$0.002034.2s51%
82Z.AI GLM 4.787.2%$0.0253.7m73%
83Gemini 3.5 Flash (Reasoning)84.7%$0.11349.4s69%
84Z.AI GLM 5.2 (Reasoning, High)80.8%$0.0391.8m67%
85Mistral Medium 3.578.2%$0.0338.3s55%
86Z.AI GLM 5 Turbo88.7%$0.0443.6m72%
87Grok 4.3 (Reasoning)76.6%$0.01313.9s53%
88Grok 4.7 (Reasoning, High)98.2%$0.1315.4m90%
89Claude Opus 5 (Reasoning)92.6%$0.24747.3s81%
90Gemini 3.1 Flash Lite (Preview)66.0%$0.00463.2s62%
91Mistral Medium 3.5 (Reasoning)77.1%$0.0328.6s54%
92Qwen 3.6 Flash78.5%$0.01551.7s54%
93Xiaomi MIMO v2.6 Flash93.9%$0.00787.0m83%
94Mistral Large 4.069.9%$0.008030.1s59%
95GPT-5.4 Mini (Reasoning)89.4%$0.1172.8m74%
96Claude Opus 5.5 (Reasoning)89.5%$0.20340.6s72%
97Claude Opus 4.783.9%$0.18224.3s73%
98DeepSeek V4 Pro (Reasoning)82.0%$0.0202.3m61%
99Gemini 3.1 Pro (Preview)87.8%$0.1721.5m75%
100Kimi K2.692.1%$0.0825.0m81%
101Qwen 3.7 Max88.5%$0.0453.5m67%
102Qwen 3.8 Flash (Reasoning)88.1%$0.0154.9m71%
103DeepSeek V4 Pro 0813 (Reasoning, High)87.3%$0.0603.5m68%
104Muse Glimmer 30B (Reasoning, Medium)69.0%$0.00621.1m60%
105ByteDance Seed 2.0 Lite83.5%$0.0143.6m64%
106Claude Opus 5 (Reasoning, Low)90.2%$0.24646.5s75%
107Claude Opus 4.7 (Reasoning)83.0%$0.18425.0s69%
108Aion 3.0 Mini88.3%$0.0304.9m70%
109Thinking Machines Inkling Small (Reasoning, High)71.7%$0.0131.1m57%
110Thinking Machines Inkling66.4%$0.01843.6s61%
111GPT-6 Luna72.1%$0.001212.1s47%
112 Kimi K3 (Reasoning, High)80.5%$0.1023.1m78%
113Nemotron 3 Nano81.9%$0.00392.8m56%
114Hy4 Preview (Reasoning, High)88.4%$0.0564.7m73%
115Qwen 3.5 397B A17B78.2%$0.0452.6m65%
116Qwen 3.5 27B83.9%$0.0293.7m63%
117Gemini 3.1 Flash Lite (Reasoning)67.1%$0.00333.2s50%
118GPT-OSS 120B79.1%$0.00402.2m53%
119Claude Sonnet 4.577.6%$0.07718.6s53%
120Gemma 4 31B68.3%$0.00281.9m62%
121Qwen 3.5 Plus (2026-02-15)81.1%$0.0213.9m65%
122Z.AI GLM 4.676.9%$0.0183.0m62%
123Claude Haiku 5.5 (Reasoning, Medium)64.9%$0.004616.7s52%
124Nemotron 3 Super86.1%$0.00005.5m66%
125Cohere Command R+ (Aug. 2024)79.2%$0.05756.5s48%
126DeepSeek V4 Pro65.0%$0.008317.3s51%
127GPT-5.4 Mini (Reasoning, Low)64.1%$0.01414.4s52%
128Qwen 3.6 27B77.6%$0.0342.7m59%
129Claude Sonnet 5 (Reasoning, Low)79.5%$0.1751.7m74%
130Qwen 3 32B74.7%$0.00422.9m57%
131Ministral 8B66.6%$0.001615.6s46%
132Gemma 4 26B59.1%$0.001827.2s56%
133Gemini 3.1 Flash Lite64.6%$0.00313.3s47%
134Qwen 3.5 Plus (2026-04-20)81.5%$0.0233.1m54%
135Kimi K2.586.0%$0.0315.4m68%
136Qwen 3.8 27B (Reasoning, XHigh)86.9%$0.0534.9m67%
137GPT-5.4 Nano (Reasoning, Low)67.9%$0.003813.8s43%
138Gemini 3.5 Flash Lite (Reasoning)70.5%$0.00623.1s39%
139Qwen 3.6 Max Preview82.9%$0.0843.7m67%
140MiniMax M2.763.2%$0.006321.4s49%
141Qwen 3.6 35B70.0%$0.0141.1m49%
142WizardLM 2 8x22b75.9%$0.00982.3m49%
143GPT-5.4 Mini58.0%$0.00877.9s51%
144Qwen 3.5 Flash74.9%$0.00403.8m58%
145GPT-5.5 (Reasoning, Medium)88.6%$0.2512.0m72%
146Writer: Palmyra X569.1%$0.01618.4s40%
147Z.AI GLM 581.6%$0.0374.6m62%
148Claude Sonnet 562.8%$0.06817.1s55%
149Claude Sonnet 5 (Reasoning)81.6%$0.1721.7m63%
150Z.AI GLM 5.3 Flash (Reasoning, Max)89.2%$0.00628.0m72%
151Ember-1 (Reasoning, Max)89.6%$0.2113.4m71%
152Qwen 3.5 122B76.3%$0.0354.0m59%
153Aion 3.086.9%$0.0745.8m67%
154Qwen 3 235B A22B Instruct 250769.5%$0.00171.0m37%
155Claude Opus 4.6 (Reasoning)94.1%$0.3562.6m83%
156Z.AI GLM 4.7 Flash74.3%$0.00502.8m44%
157Aion 3.5 (Reasoning, High)88.2%$0.0836.1m67%
158Qwen 3.5 35B65.4%$0.0152.0m49%
159Gemini 2.5 Flash Lite64.7%$0.00234.8s32%
160Claude Opus 4.8 (Reasoning, Low)92.6%$0.3831.9m82%
161MiniMax M391.1%$0.0369.3m79%
162Mistral Small 3.2 24B53.8%$0.001713.8s45%
163GPT-5 Nano69.2%$0.00802.2m44%
164GPT-4.1 Nano57.8%$0.00096.9s40%
165GPT-4o Mini (temp=1)59.7%$0.002633.4s41%
166GPT-4.1 Mini55.2%$0.004626.0s45%
167Z.AI GLM 5.182.2%$0.0817.3m80%
168DeepSeek V3 (2024-12-26)64.8%$0.004540.2s35%
169DeepSeek-V2 Chat64.1%$0.004934.0s35%
170Claude Sonnet 462.2%$0.07217.5s47%
171Mistral Small 465.4%$0.00307.4s29%
172Claude Opus 4.8 (Reasoning)91.1%$0.3811.9m80%
173Laguna XS 2.155.0%$0.001241.3s45%
174Aion 3.5 Mini (Reasoning, High)84.5%$0.0367.7m72%
175DeepSeek V4 Flash61.4%$0.002051.4s38%
176ByteDance Seed 2.0 Mini87.6%$0.00628.7m70%
177GPT-5.6 Luna56.3%$0.0128.7s40%
178Grok 4.355.6%$0.00947.8s39%
179Mistral Small 4 (Reasoning)64.2%$0.005124.7s29%
180GPT-4o Mini (temp=0)51.6%$0.001920.7s41%
181Gemma 4 26B (Reasoning)65.9%$0.00413.1m43%
182Gemma 4 31B (Reasoning)73.9%$0.00435.8m54%
183Ministral 3 8B51.7%$0.002217.0s35%
184Cydonia 24B V4.159.8%$0.00401.0m30%
185Llama 3.1 70B47.5%$0.006434.0s41%
186Qwen 3.8 Max (Reasoning, XHigh)94.2%$0.1609.6m83%
187GPT-5.4 Nano46.3%$0.002611.2s37%
188Gemma 3 4B52.3%$0.000727.5s30%
189Claude Haiku 4.547.2%$0.0248.2s35%
190Ministral 3B44.7%$0.00079.8s33%
191Laguna S 2.148.5%$0.00062.0m39%
192Hermes 3 405B57.4%$0.0151.0m24%
193Ministral 3 3B40.8%$0.00079.9s32%
194Z.AI GLM 4.5 Air57.5%$0.00713.2m38%
195Qwen 2.5 72B44.8%$0.00561.2m36%
196Hermes 3 70B50.3%$0.004542.7s20%
197Gemma 3 27B42.7%$0.001627.5s23%
198Arcee AI: Trinity Mini52.8%$0.00371.9m23%
199Qwen 3.5 9B64.4%$0.00407.8m53%
200Z.AI GLM 5.3 (Reasoning, Max)85.6%$0.18710.3m75%
201Mistral NeMO30.3%$0.002415.4s21%
202Claude Sonnet 4.6 (Reasoning)85.2%$0.4485.9m73%
203Claude Opus 465.2%$0.3682.4m49%
78.73%

Individual Scenarios

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-6.1 Sol (Reasoning, Medium)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
o4 Mini100100100100100100.0%
Mistral Large 2100100100100100100.0%
Gemma 3 12B100100100100100100.0%
Claude Opus 4.51001001001009098.1%
GPT-5.4 Nano (Reasoning)1001001001008597.0%
GPT-5.11001001001008496.9%
GPT-6 Sol (Reasoning, Medium)1001001001008396.5%
Grok 4.7 (Reasoning, High)1001001001008396.5%
GPT-5 Mini1001001001008096.1%
Mistral Large 4.0 (Reasoning)1001001001008095.9%
GPT-6 Sol1001001001008095.9%
Z.AI GLM 4.5100100100938695.8%
Gemini 3.5 Flash (Reasoning, Minimal)100100100958095.0%
o4 Mini High100100100957694.1%
GPT-4o, Aug. 6th (temp=0)100100100898194.0%
Gemini 2.5 Flash (Reasoning)1001001001006593.1%
GPT-5.6 Sol (Reasoning, Medium)100100100838393.0%
Gemini 2.5 Flash10010093908192.8%
Claude Opus 4.6100100100937192.8%
Xiaomi MIMO v2.6 Flash100100100838092.5%
Grok 4.20100100100887492.4%
ByteDance Seed 1.6100100100887392.3%
Mistral Large 3100100100907192.2%
MiniMax M3100100100818092.1%
GPT-51009898838292.0%
GPT-5.4 (Reasoning, Low)100100100817992.0%
GPT-5.4 Mini (Reasoning)100100100817991.9%
Claude Opus 5 (Reasoning)100100100837791.9%
GPT-5.2100100100857491.8%
Mistral Medium 3.1100100100876690.5%
Grok 4.20 (Reasoning)100100100757590.0%
Claude Opus 4.6 (Reasoning)10010083838389.5%
Gemini 3.7 Flash (Reasoning, Medium)10010083838389.5%
Grok 4.5 (Reasoning, High)10010083838389.5%
Gemini 4 Argon (Reasoning, Medium)10010083838389.5%
Grok 4.6 (Reasoning, High)10010083838389.5%
Claude Opus 510010083838389.5%
Gemini 2.5 Pro10010083838389.5%
DeepSeek V4 Flash 0731 (Reasoning, High)10010083838389.5%
Kimi K2.610010083838289.4%
Qwen 3.8 Max (Reasoning, XHigh)10010083838189.2%
Gemini 3.5 Flash (Reasoning)10010083838089.0%
Gemini 3 Flash (Preview, Reasoning)10010083838089.0%
ByteDance Seed 2.0 Mini100100100727288.9%
Xiaomi MIMO v2.5 Pro100100100776688.6%
Aion 3.5 (Reasoning, High)10010083807988.2%
Thinking Machines Inkling (Reasoning)10010083817788.1%
Aion 2.0100100100726888.1%
GPT-5.6 Terra (Reasoning, Medium)1009382817986.8%
ByteDance Seed 1.6 Flash100100100904486.7%
Z.AI GLM 4.7 Flash10010095756386.7%
Gemini 2.5 Flash Lite (Reasoning)100100100726086.5%
Xiaomi MIMO v2.510010083767286.1%
Muse Spark 1.2 (Reasoning, Medium)1008383838386.0%
Qwen 3.8 Flash (Reasoning)1008383838386.0%
GPT-5.5 (Reasoning, Low)1008383838386.0%
Claude Opus 5 (Reasoning, Low)1008383838386.0%
Ember-1 (Reasoning, Max)1008383838386.0%
GPT-6 Luna (Reasoning, High)1008383838286.0%
Muse Spark 1.1 (Reasoning, Minimal)1008383838185.7%
Muse Spark 1.1 (Reasoning, Medium)1008383838085.5%
Aion 3.01008383838085.5%
DeepSeek V3.110010087756585.4%
Grok 4.5 (Reasoning, Low)1008383828085.4%
Claude Opus 4.8 (Reasoning, Low)1008383818085.1%
Claude Opus 4.8 (Reasoning)1008383837784.9%
Z.AI GLM 5 Turbo1008381817984.7%
Gemini 3.1 Pro (Preview)1008383837684.7%
Gemini 3.6 Flash (Reasoning, Minimal)10010083736784.7%
Qwen 3.7 Max1008383817684.3%
DeepSeek V4 Pro 0813 (Reasoning, High)1008383807684.1%
GPT-5.6 Luna (Reasoning, Medium)1008382797784.1%
Nemotron 3 Nano10010090666383.7%
Z.AI GLM 4.71008682807183.6%
DeepSeek V4 Flash (Reasoning)10010083756083.5%
Qwen 3.8 27B (Reasoning, XHigh)1008382827183.4%
Nemotron 3 Super1008578777783.3%
Claude Opus 5.5 (Reasoning)838383838382.6%
Gemini 3.8 Flash (Reasoning, Medium)838383838382.6%
Z.AI GLM 5.3 (Reasoning, Max)838383838382.6%
Gemini 3.6 Flash (Reasoning)838383838382.6%
Muse Spark 1.3 (Reasoning, Medium)838383838382.6%
GPT-5.5 (Reasoning, Medium)838383838382.6%
Claude Sonnet 4.6 (Reasoning)838383838382.6%
Claude Sonnet 5.5 (Reasoning)838383838382.6%
Claude Opus 4.7 (Reasoning)838383838382.6%
GPT-5.5838383838382.6%
Qwen 3.5 27B1008177767682.0%
DeepSeek V4.1 Flash (Reasoning, High)838383838082.0%
Z.AI GLM 5.1838383838082.0%
Muse Wonder (Reasoning, Medium)838383818181.8%
Kimi K3 (Reasoning, High)838383818181.8%
GPT-5.6 Sol838383837981.8%
Aion 3.0 Mini1008077777481.7%
Ministral 3 14B1009692744681.7%
GPT-4.110010085675581.5%
Aion 3.5 Mini (Reasoning, High)838383808081.4%
Hy4 Preview (Reasoning, High)868383807681.3%
DeepSeek V4.1 Flash 1017 (Reasoning, High)838383837681.2%
Z.AI GLM 5.3 Flash (Reasoning, Max)838383837681.2%
Gemini 3.5 Flash Lite (Reasoning, Minimal)10010078656180.8%
GPT-6 Luna (Reasoning, Medium)838381807780.6%
Gemini 3.5 Flash Lite (Reasoning)10010072686180.1%
Kimi K2.51008383686679.9%
Qwen 3.6 Max Preview838080807779.8%
Qwen 3.5 Flash1007976737179.8%
Claude Opus 4.7838383826879.5%
DeepSeek V4 Pro (Reasoning)1008076726979.4%
GPT-4o, Aug. 6th (temp=1)1009188803779.0%
Gemini 3 Flash (Preview)808079787878.8%
Qwen 3.5 397B A17B838079797478.8%
Kimi K3 (Reasoning, Low)878383727078.7%
DeepSeek V3.210010072645678.6%
Claude Sonnet 4.6837979797478.5%
GPT-5.4797878787878.2%
ByteDance Seed 2.0 Lite1008380765278.1%
Qwen 3.5 122B907776747277.8%
Qwen 3.5 Plus (2026-02-15)818079747477.5%
Xiaomi MIMO v2.6 Pro807877767677.2%
GPT-5.6 Terra807979787077.1%
Cohere Command R+ (Aug. 2024)100100100424176.6%
Claude Sonnet 5 (Reasoning, Low)838176727276.4%
Qwen 3.5 Plus (2026-04-20)1007472716476.4%
Qwen 3 32B1008178626176.2%
Claude Sonnet 5 (Reasoning)837776737275.9%
GPT-5 Nano948983575575.7%
Qwen 3.6 27B858277716475.7%
Z.AI GLM 5.3 Flash (Reasoning, Low)807776747175.6%
Z.AI GLM 5.2 (Reasoning, High)797979766675.5%
Thinking Machines Inkling Small (Reasoning, High)797776757075.4%
Mistral Small 4100100100512575.3%
Qwen 3.6 Flash1007670696175.0%
Mistral Large 4.0797978716975.0%
Inception Mercury 2787875717174.6%
Z.AI GLM 5838380666174.4%
GPT-OSS 120B1007767676074.3%
Claude Haiku 5.5 (Adaptive)827777726274.2%
WizardLM 2 8x22b1009568654274.1%
MiniMax M2.5887674726074.0%
Grok 4.3 (Reasoning)10010058565373.5%
Muse Glimmer 30B (Reasoning, Medium)777776736573.5%
DeepSeek V3 (2025-03-24)10010058584772.7%
Claude Haiku 5.5 (Reasoning, Low)767271707071.8%
Z.AI GLM 4.6808078685371.7%
Qwen 3.7 Flash (Reasoning)817272715971.0%
DeepSeek V4 Pro827671675470.2%
Gemma 4 31B (Reasoning)767272706069.9%
Qwen 3.5 9B807469665669.0%
Gemini 3.1 Flash Lite (Preview)727067676768.9%
Thinking Machines Inkling727169696368.7%
Qwen 3.5 35B757266666468.6%
Gemma 4 31B747270646168.2%
Claude Sonnet 5746968666167.6%
Gemini 3.1 Flash Lite (Reasoning)707070705867.5%
Gemma 4 26B (Reasoning)827666585266.9%
GPT-4.1 Nano848258565366.5%
Gemini 2.5 Flash Lite10010071332565.7%
DeepSeek V4 Flash896665575065.3%
GPT-5.4 Mini (Reasoning, Low)836765614464.0%
Z.AI GLM 4.5 Air797569564264.0%
Claude Sonnet 4766666664463.6%
GPT-5.4 Nano (Reasoning, Low)1006559553963.5%
Mistral Small 4 (Reasoning)10010050422463.1%
Claude Haiku 5.5 (Reasoning, Medium)746659595662.9%
Qwen 3.6 35B666661616062.9%
Mistral Medium 3.5726161616162.9%
Ministral 8B777668504262.7%
Claude Opus 4666666654862.3%
Qwen 3 235B A22B Instruct 25071006356504262.2%
Cydonia 24B V4.11006967383662.1%
DeepSeek V3 (2024-12-26)1005852524661.9%
Gemini 3.1 Flash Lite736761584961.7%
Claude Sonnet 4.5767666454561.6%
DeepSeek-V2 Chat867552484661.6%
Mistral Medium 3.5 (Reasoning)706161615360.9%
MiniMax M2.7726363535260.7%
Grok 4.3916358503559.2%
GPT-6 Luna716556525059.0%
Laguna XS 2.1725655555057.6%
Gemma 4 26B605858565657.6%
Gemma 3 4B857156532357.5%
GPT-5.4 Mini606059505055.7%
GPT-4o Mini (temp=1)815749484355.6%
Gemma 3 27B1005043404054.6%
GPT-4.1 Mini746649483554.2%
Mistral Small 3.2 24B695954513653.8%
Arcee AI: Trinity Mini1005347362552.1%
Ministral 3 8B685941403649.0%
Laguna S 2.1655352522048.3%
GPT-5.6 Luna605445424248.3%
Writer: Palmyra X5585249463648.2%
Claude Haiku 4.5716038383448.2%
Hermes 3 405B1004338322848.0%
Llama 3.1 70B565450423747.9%
GPT-5.4 Nano626147342445.4%
GPT-4o Mini (temp=0)744837363245.3%
Ministral 3 3B634040322640.0%
Qwen 2.5 72B484637363239.6%
Mistral NeMO484846252237.8%
Hermes 3 70B434242282736.4%
Ministral 3B484336292636.2%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-6.1 Sol (Reasoning, Medium)100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100.0%
Gemini 4 Argon (Reasoning, Medium)100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
Gemma 3 12B100100100100100100.0%
Grok 4.7 (Reasoning, High)1001001001009999.8%
Qwen 3.8 Max (Reasoning, XHigh)100100100999799.2%
DeepSeek V4 Flash 0731 (Reasoning, High)10010099999899.2%
GPT-6 Sol999999999999.0%
GPT-5.4 (Reasoning)100100100979798.9%
Claude Opus 4.6 (Reasoning)100100100979698.7%
DeepSeek V3 (2025-03-24)1001001001009398.7%
GPT-5.4 (Reasoning, Low)100100100979698.5%
Claude Opus 4.8 (Reasoning)1009797979797.3%
Grok 4.6 (Reasoning, High)1009796969697.3%
GPT-6 Luna (Reasoning, High)100100100998797.2%
Z.AI GLM 5.3 Flash (Reasoning, Max)10010097979297.1%
GPT-5.6 Terra1009998949496.9%
GPT-5.2989797969696.7%
Grok 4.5 (Reasoning, High)979796969696.6%
Claude Opus 5.5 (Reasoning)999999998896.5%
Claude Opus 4.61009797959396.2%
GPT-51009895959296.1%
GPT-4o, Aug. 6th (temp=0)1009797939295.7%
Thinking Machines Inkling (Reasoning)1009695959195.5%
Hy4 Preview (Reasoning, High)1009797978795.4%
Xiaomi MIMO v2.6 Flash1009997978495.4%
GPT-5 Mini979795949395.3%
Gemini 3.8 Flash (Reasoning, Medium)10010099888894.9%
Aion 3.0 Mini10010096968294.8%
Kimi K2.61009999968094.8%
GPT-5.5 (Reasoning, Medium)10010099888894.7%
Claude Opus 5 (Reasoning, Low)10010095938494.4%
DeepSeek V4 Flash (Reasoning)999796958594.3%
GPT-6 Luna (Reasoning, Medium)100100100868494.1%
Claude Opus 4.5969594949193.8%
Qwen 3.7 Flash (Reasoning)10010097898293.6%
Mistral Medium 3.5969695918993.6%
Claude Sonnet 4.5949494939393.5%
Grok 4.20989895928593.5%
GPT-5.6 Sol1009796938193.5%
Gemini 3.7 Flash (Reasoning, Medium)1009797888593.5%
Claude Opus 5 (Reasoning)1009797878693.3%
Grok 4.20 (Reasoning)959494929193.2%
Mistral Medium 3.5 (Reasoning)969494929193.2%
Ember-1 (Reasoning, Max)1009997858493.1%
GPT-5.4 Nano (Reasoning)1009696888593.0%
Qwen 3.7 Max100100100828192.7%
Z.AI GLM 5 Turbo1009696888392.7%
Xiaomi MIMO v2.6 Pro100100100828192.7%
Claude Opus 5969694948392.7%
Gemini 3.6 Flash (Reasoning)979696908492.7%
Mistral Large 4.0 (Reasoning)969595938392.6%
o4 Mini High989591918892.6%
GPT-5.1989392928692.3%
Z.AI GLM 4.5989493918592.2%
Kimi K2.5969595957992.1%
Aion 2.0999994947491.9%
Gemini 2.5 Flash939392929091.9%
DeepSeek V3.2969393898891.7%
DeepSeek V4.1 Flash 1017 (Reasoning, High)1009788858591.1%
Gemini 3.1 Pro (Preview)979291918390.9%
Z.AI GLM 4.7959494908190.8%
DeepSeek V3.11009190868690.8%
DeepSeek V4 Pro 0813 (Reasoning, High)10010088838290.5%
Muse Spark 1.1 (Reasoning, Minimal)1009493927390.4%
Qwen 3.8 27B (Reasoning, XHigh)999995857390.3%
Qwen 3.8 Flash (Reasoning)1009987848190.2%
MiniMax M3969695838090.1%
GPT-5.5 (Reasoning, Low)10010084848289.9%
Writer: Palmyra X5979289898389.9%
GPT-5.6 Terra (Reasoning, Medium)1009987828189.8%
ByteDance Seed 1.6969292848188.9%
GPT-5.4949494937088.9%
ByteDance Seed 2.0 Lite939190908088.9%
Nemotron 3 Super10010087797888.8%
Z.AI GLM 51009992827188.7%
Z.AI GLM 5.3 (Reasoning, Max)1008886868488.7%
o4 Mini969592926888.5%
DeepSeek V4.1 Flash (Reasoning, High)978888878288.4%
Aion 3.5 (Reasoning, High)10010084807888.3%
Aion 3.010010089876688.3%
Claude Opus 4.7948988868488.2%
Muse Spark 1.1 (Reasoning, Medium)959494817688.1%
Claude Sonnet 4.6 (Reasoning)969684828087.8%
Muse Wonder (Reasoning, Medium)968888868287.8%
Aion 3.5 Mini (Reasoning, High)1009385837787.6%
Gemini 2.5 Pro969590797787.5%
Claude Sonnet 5 (Reasoning)10010082807587.3%
Claude Sonnet 5.5 (Reasoning)888887878687.0%
GPT-5.4 Mini (Reasoning)969586847486.9%
GPT-4o, Aug. 6th (temp=1)979390807386.7%
Qwen 3.5 Plus (2026-04-20)999893747186.7%
ByteDance Seed 2.0 Mini969089837386.3%
Qwen 3.6 Max Preview979393757286.0%
Z.AI GLM 5.2 (Reasoning, High)1008483828086.0%
Qwen 3.5 27B999685777185.8%
Xiaomi MIMO v2.5 Pro989488826685.5%
Grok 4.5 (Reasoning, Low)979694746785.5%
GPT-6 Luna919190886685.2%
Gemini 3 Flash (Preview, Reasoning)949290787385.1%
Qwen 3.5 Plus (2026-02-15)949493746984.7%
DeepSeek V4 Pro (Reasoning)1009380767484.5%
Gemini 2.5 Flash (Reasoning)939390796784.2%
Muse Spark 1.3 (Reasoning, Medium)868484838284.0%
GPT-OSS 120B959592706683.9%
Claude Opus 4.7 (Reasoning)999084806583.4%
GPT-5.6 Luna (Reasoning, Medium)969578737383.1%
Claude Sonnet 5 (Reasoning, Low)858484827882.6%
GPT-4.1929290835582.4%
Z.AI GLM 5.1848483818182.4%
Qwen 3.6 Flash959291745882.1%
Z.AI GLM 4.6928878767682.0%
Cohere Command R+ (Aug. 2024)100100100585181.9%
MiniMax M2.5929086865581.9%
Z.AI GLM 5.3 Flash (Reasoning, Low)888879787681.8%
Mistral Large 3959393686181.7%
Xiaomi MIMO v2.51009593605981.6%
Inception Mercury 2978382816581.4%
Gemini 3.6 Flash (Reasoning, Minimal)959289705980.8%
Gemini 3.5 Flash (Reasoning)858378787880.4%
Kimi K3 (Reasoning, Low)868180807680.4%
Nemotron 3 Nano10010083635580.2%
Muse Spark 1.2 (Reasoning, Medium)957877757580.1%
Gemini 2.5 Flash Lite (Reasoning)929183696780.0%
Ministral 3 14B979489625880.0%
Grok 4.3 (Reasoning)898883825679.7%
Qwen 3.6 27B989476656479.5%
Kimi K3 (Reasoning, High)828079787779.2%
Gemini 3.5 Flash Lite (Reasoning, Minimal)1009780625779.2%
Claude Haiku 5.5 (Adaptive)848482786778.9%
GPT-5.5838080787478.9%
Mistral Large 2939291615878.8%
Gemma 4 31B (Reasoning)1009170666377.9%
WizardLM 2 8x22b948986645577.7%
Qwen 3.5 397B A17B997874736477.5%
Qwen 3.6 35B1009167666377.2%
Qwen 3 235B A22B Instruct 25071009177605676.7%
Mistral Medium 3.1848375746576.1%
Claude Sonnet 4.6787776747175.0%
Qwen 3.5 122B1007767656474.7%
Qwen 3 32B878280615673.3%
GPT-5.4 Nano (Reasoning, Low)897769646272.3%
Gemini 3 Flash (Preview)837570666471.5%
Claude Haiku 5.5 (Reasoning, Low)807770646270.6%
Ministral 8B918881514170.4%
Qwen 3.5 Flash807474685570.1%
Gemma 4 31B756966666568.4%
Claude Opus 4987062585368.2%
Thinking Machines Inkling Small (Reasoning, High)917068614968.0%
DeepSeek V3 (2024-12-26)888763534667.7%
Gemini 3.1 Flash Lite976564585567.6%
Gemini 3.5 Flash (Reasoning, Minimal)837269585567.3%
Claude Haiku 5.5 (Reasoning, Medium)767574555466.9%
Hermes 3 405B1007956534666.8%
Gemini 3.1 Flash Lite (Reasoning)986059595866.6%
DeepSeek-V2 Chat928856534466.6%
MiniMax M2.7797962604865.6%
Mistral Small 4 (Reasoning)976359565165.2%
Mistral Large 4.0666664646464.8%
Gemma 4 26B (Reasoning)1007572443364.8%
GPT-5.6 Luna916662564864.4%
Muse Glimmer 30B (Reasoning, Medium)706863635964.4%
Hermes 3 70B10010054422564.2%
GPT-5.4 Mini (Reasoning, Low)717063615664.1%
Thinking Machines Inkling706564635864.0%
GPT-4o Mini (temp=1)817860603963.7%
Gemini 2.5 Flash Lite878170503163.7%
Gemini 3.1 Flash Lite (Preview)686563606063.0%
GPT-5 Nano827157535162.7%
Qwen 3.5 35B807371622562.2%
Z.AI GLM 4.7 Flash846962583662.0%
Gemini 3.5 Flash Lite (Reasoning)955753504960.9%
Claude Sonnet 4886059494860.8%
Gemma 4 26B636361605760.7%
GPT-5.4 Mini746259555260.3%
DeepSeek V4 Pro746463594059.9%
Qwen 3.5 9B716158575259.8%
Claude Sonnet 5676057555058.0%
GPT-4o Mini (temp=0)655956555457.8%
Cydonia 24B V4.11006652422857.5%
DeepSeek V4 Flash905551464557.5%
ByteDance Seed 1.6 Flash57 – – – – 56.6%
GPT-4.1 Mini595958565156.3%
Mistral Small 4676552504455.5%
Ministral 3 8B815049474654.4%
Mistral Small 3.2 24B585754544553.8%
Arcee AI: Trinity Mini1005648432353.6%
Ministral 3B696051473953.2%
Laguna XS 2.1705453444152.4%
Grok 4.3625954533352.1%
Z.AI GLM 4.5 Air69655855951.1%
Qwen 2.5 72B635352424049.9%
GPT-4.1 Nano585046464549.2%
Laguna S 2.1585048474048.6%
Llama 3.1 70B565353522347.2%
Gemma 3 4B786556261047.2%
GPT-5.4 Nano514947464247.1%
Claude Haiku 4.5535245443846.2%
Ministral 3 3B484743353541.6%
Gemma 3 27B383433272230.8%
Mistral NeMO353118161422.8%