Relationship category recall

Test: Relationship tree

Avg. Score
37.2%
Scenarios
2

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Gemini 4 Argon (Reasoning, Medium)100.0%$0.03919.1s100%
2GPT-6.1 Sol (Reasoning, Medium)100.0%$0.04250.1s100%
3GPT-6 Sol (Reasoning, Medium)99.5%$0.05146.3s97%
4Gemini 3.7 Flash (Reasoning, Medium)97.8%$0.03623.4s91%
5Gemini 3.8 Flash (Reasoning, Medium)99.7%$0.06742.1s98%
6GPT-6 Luna (Reasoning, High)91.4%$0.00551.4m77%
7Claude Opus 5.5 (Reasoning)100.0%$0.20340.6s100%
8GPT-5.6 Sol (Reasoning, Medium)99.5%$0.18659.0s97%
9Claude Sonnet 5.5 (Reasoning)90.7%$0.10128.5s75%
10GPT-6 Sol85.0%$0.03521.1s56%
11Gemini 3.6 Flash (Reasoning)87.1%$0.09641.4s66%
12DeepSeek V4.1 Flash (Reasoning, High)83.4%$0.0141.5m54%
13Grok 4.5 (Reasoning, High)88.4%$0.0802.0m68%
14Grok 4.6 (Reasoning, High)95.3%$0.1043.7m81%
15Muse Wonder (Reasoning, Medium)83.3%$0.11620.5s60%
16GPT-5.4 (Reasoning)94.3%$0.1752.6m80%
17GPT-6 Luna (Reasoning, Medium)73.6%$0.00351.1m47%
18Grok 4.7 (Reasoning, High)97.9%$0.1315.4m89%
19Ember-1 (Reasoning, Max)95.2%$0.2113.4m82%
20GPT-5.6 Terra (Reasoning, Medium)71.1%$0.06024.2s43%
21Muse Spark 1.3 (Reasoning, Medium)78.8%$0.0551.9m47%
22DeepSeek V4.1 Flash 1017 (Reasoning, High)70.9%$0.005658.2s34%
23DeepSeek V4 Flash 0731 (Reasoning, High)87.4%$0.00735.6m63%
24GPT-5.5 (Reasoning, Medium)90.0%$0.2512.0m72%
25GPT-5.6 Sol71.0%$0.09618.1s34%
26Muse Spark 1.1 (Reasoning, Medium)67.0%$0.04235.1s27%
27Qwen 3.8 Flash (Reasoning)81.4%$0.0154.9m47%
28GPT-5.570.8%$0.10537.1s31%
29Kimi K2.680.3%$0.0825.0m56%
30GPT-5.5 (Reasoning, Low)70.3%$0.09742.6s27%
31Kimi K3 (Reasoning, Low)65.0%$0.0591.2m29%
32Muse Spark 1.2 (Reasoning, Medium)64.2%$0.04629.8s19%
33Grok 4.5 (Reasoning, Low)61.4%$0.04049.7s23%
34Claude Haiku 5.5 (Adaptive)51.8%$0.006429.8s22%
35Claude Opus 4.672.3%$0.15431.4s33%
36Xiaomi MIMO v2.6 Pro59.0%$0.00712.0m26%
37GPT-5.4 (Reasoning, Low)47.0%$0.05335.5s33%
38Qwen 3.8 27B (Reasoning, XHigh)72.5%$0.0534.9m46%
39Claude Opus 4.6 (Reasoning)89.6%$0.3562.6m72%
40GPT-5.6 Luna (Reasoning, Medium)45.6%$0.02718.6s19%
41Gemini 3 Flash (Preview)34.9%$0.00877.1s23%
42Mistral Large 4.0 (Reasoning)66.8%$0.0453.9m34%
43Z.AI GLM 5.2 (Reasoning, High)43.1%$0.0391.8m34%
44Gemini 2.5 Pro63.0%$0.09958.2s19%
45 Kimi K3 (Reasoning, High)68.4%$0.1023.1m34%
46Claude Haiku 5.5 (Reasoning, Low)32.4%$0.005019.9s20%
47Claude Sonnet 5 (Reasoning)66.5%$0.1721.7m34%
48GPT-5.6 Terra35.9%$0.0349.4s19%
49Gemini 3.5 Flash (Reasoning)58.2%$0.11349.4s20%
50Claude Opus 4.763.8%$0.18224.3s25%
51Claude Sonnet 4.643.9%$0.08123.3s22%
52Xiaomi MIMO v2.6 Flash73.1%$0.00787.0m41%
53Claude Opus 4.7 (Reasoning)67.2%$0.18425.0s21%
54Gemini 3.6 Flash (Reasoning, Minimal)45.0%$0.0297.1s6%
55DeepSeek V4 Flash (Reasoning)54.5%$0.00413.5m23%
56Gemini 3 Flash (Preview, Reasoning)47.3%$0.03040.3s8%
57GPT-5.429.2%$0.03317.4s23%
58Claude Opus 4.8 (Reasoning)84.0%$0.3811.9m61%
59Muse Spark 1.1 (Reasoning, Minimal)46.7%$0.03018.2s4%
60Claude Sonnet 5 (Reasoning, Low)65.2%$0.1751.7m31%
61GPT-5.240.9%$0.06250.7s21%
62Claude Haiku 5.5 (Reasoning, Medium)29.5%$0.004616.7s13%
63Kimi K2.566.3%$0.0315.4m31%
64GPT-5.132.7%$0.02517.7s13%
65Xiaomi MIMO v2.533.8%$0.00271.1m15%
66GPT-5.4 Nano (Reasoning)24.7%$0.008841.7s21%
67Z.AI GLM 5 Turbo53.5%$0.0443.6m27%
68Z.AI GLM 4.639.3%$0.0183.0m29%
69Mistral Large 4.026.3%$0.008030.1s15%
70Grok 4.20 (Reasoning)26.5%$0.02740.0s20%
71Hy4 Preview (Reasoning, High)63.9%$0.0564.7m27%
72DeepSeek V4 Pro 0813 (Reasoning, High)61.4%$0.0603.5m18%
73Gemini 3.1 Pro (Preview)66.3%$0.1721.5m15%
74Z.AI GLM 5.3 Flash (Reasoning, Low)22.6%$0.001242.5s13%
75ByteDance Seed 1.625.4%$0.0141.2m18%
76Claude Sonnet 4.525.0%$0.07718.6s23%
77GPT-5.4 Mini (Reasoning)51.1%$0.1172.8m29%
78Qwen 3.7 Max55.8%$0.0453.5m15%
79Thinking Machines Inkling Small (Reasoning, High)24.5%$0.0131.1m15%
80GPT-6 Luna17.7%$0.001212.1s9%
81Inception Mercury 218.1%$0.009612.8s10%
82MiniMax M2.517.9%$0.003645.0s14%
83Mistral Medium 3.116.2%$0.01021.2s13%
84o4 Mini20.7%$0.03745.7s18%
85MiniMax M2.717.2%$0.006321.4s11%
86Qwen 3.5 397B A17B35.0%$0.0452.6m23%
87GPT-556.5%$0.0922.3m9%
88Claude Sonnet 523.0%$0.06817.1s17%
89DeepSeek V4 Pro (Reasoning)36.9%$0.0202.3m12%
90Claude Opus 5 (Reasoning, Low)68.4%$0.24646.5s16%
91Claude Opus 4.536.6%$0.13119.8s17%
92Gemini 3.5 Flash (Reasoning, Minimal)24.1%$0.0246.3s4%
93Ministral 8B14.1%$0.001615.6s10%
94Gemma 4 26B13.1%$0.001827.2s13%
95Qwen 3.7 Flash (Reasoning)19.3%$0.001756.4s11%
96Gemini 3.1 Flash Lite (Reasoning)11.9%$0.00333.2s10%
97Ministral 3 8B14.2%$0.002217.0s10%
98Z.AI GLM 5.173.7%$0.0817.3m36%
99Z.AI GLM 5.3 Flash (Reasoning, Max)69.7%$0.00628.0m30%
100Gemini 3.1 Flash Lite (Preview)11.7%$0.00463.2s10%
101Xiaomi MIMO v2.5 Pro26.5%$0.00761.9m14%
102Gemini 2.5 Flash14.3%$0.00756.1s8%
103Grok 4.313.4%$0.00947.8s9%
104Muse Glimmer 30B (Reasoning, Medium)18.5%$0.00621.1m12%
105DeepSeek V3.214.0%$0.004633.9s11%
106Gemini 3.1 Flash Lite12.0%$0.00313.3s8%
107Ministral 3 14B12.8%$0.003125.4s10%
108Z.AI GLM 4.740.9%$0.0253.7m19%
109DeepSeek V4 Pro13.1%$0.008317.3s10%
110Qwen 3.6 Flash17.4%$0.01551.7s12%
111Claude Opus 553.4%$0.19526.4s12%
112GPT-4.113.4%$0.0248.5s11%
113Mistral Large 312.0%$0.009117.5s10%
114o4 Mini High27.4%$0.0591.3m15%
115GPT-5 Mini27.1%$0.0192.0m13%
116GPT-5.4 Mini (Reasoning, Low)11.4%$0.01414.4s10%
117GPT-5.6 Luna10.3%$0.0128.7s9%
118Mistral Medium 3.5 (Reasoning)12.6%$0.0328.6s11%
119Qwen 3.6 35B15.1%$0.0141.1m14%
120DeepSeek V4 Flash15.5%$0.002051.4s8%
121Gemini 3.5 Flash Lite (Reasoning)15.1%$0.00623.1s2%
122Mistral Medium 3.513.5%$0.0338.3s10%
123Qwen 3.5 35B19.6%$0.0152.0m17%
124Z.AI GLM 4.513.3%$0.007548.1s10%
125Thinking Machines Inkling14.5%$0.01843.6s9%
126GPT-5.4 Mini9.1%$0.00877.9s6%
127Mistral Large 213.3%$0.03720.5s10%
128GPT-4o, Aug. 6th (temp=0)15.2%$0.0328.4s5%
129Grok 4.208.6%$0.0207.5s9%
130Qwen 3.5 27B32.1%$0.0293.7m22%
131MiniMax M365.5%$0.0369.3m44%
132GPT-5.4 Nano7.4%$0.002611.2s6%
133Ministral 3B7.5%$0.00079.8s4%
134Ministral 3 3B6.1%$0.00079.9s5%
135Mistral Small 3.2 24B6.6%$0.001713.8s5%
136Qwen 3.5 Plus (2026-02-15)33.7%$0.0213.9m17%
137DeepSeek V3 (2024-12-26)7.6%$0.004540.2s8%
138Qwen 3.6 Max Preview43.8%$0.0843.7m19%
139Gemini 3.5 Flash Lite (Reasoning, Minimal)7.1%$0.00573.2s2%
140Thinking Machines Inkling (Reasoning)44.4%$0.0604.1m17%
141Claude Sonnet 417.7%$0.07217.5s9%
142GPT-4.1 Mini7.0%$0.004626.0s6%
143GPT-5.4 Nano (Reasoning, Low)5.6%$0.003813.8s4%
144DeepSeek-V2 Chat7.5%$0.004934.0s6%
145Grok 4.3 (Reasoning)5.5%$0.01313.9s6%
146Laguna XS 2.17.6%$0.001241.3s6%
147DeepSeek V3.111.2%$0.005552.0s5%
148Qwen 3.5 Plus (2026-04-20)22.3%$0.0233.1m19%
149Claude Haiku 4.57.6%$0.0248.2s5%
150GPT-OSS 120B17.1%$0.00402.2m11%
151Mistral Small 4 (Reasoning)5.9%$0.005124.7s5%
152Writer: Palmyra X55.8%$0.01618.4s7%
153Qwen 3.6 27B26.3%$0.0342.7m13%
154Gemma 4 31B14.1%$0.00281.9m10%
155Gemini 2.5 Flash (Reasoning)10.7%$0.02337.3s4%
156Gemini 2.5 Flash Lite (Reasoning)10.3%$0.00731.0m5%
157Aion 2.024.4%$0.0243.0m14%
158Z.AI GLM 543.2%$0.0374.6m13%
159Mistral NeMO4.3%$0.002415.4s1%
160Llama 3.1 70B5.6%$0.006434.0s4%
161Aion 3.0 Mini39.7%$0.0304.9m15%
162GPT-4o Mini (temp=0)3.1%$0.001920.7s2%
163GPT-4o, Aug. 6th (temp=1)4.3%$0.0299.1s4%
164Mistral Small 41.8%$0.00307.4s0%
165GPT-4.1 Nano0.6%$0.00096.9s0%
166Laguna S 2.111.6%$0.00062.0m7%
167Gemini 2.5 Flash Lite0.7%$0.00234.8s0%
168Claude Opus 4.8 (Reasoning, Low)69.3%$0.3831.9m33%
169Claude Opus 5 (Reasoning)49.9%$0.24747.3s12%
170Qwen 3.5 Flash22.4%$0.00403.8m14%
171Qwen 3.8 Max (Reasoning, XHigh)78.2%$0.1609.6m49%
172Z.AI GLM 5.3 (Reasoning, Max)84.7%$0.18710.3m56%
173ByteDance Seed 1.6 Flash3.0%$0.002034.2s1%
174Qwen 2.5 72B5.0%$0.00561.2m5%
175Aion 3.058.1%$0.0745.8m12%
176Qwen 3 235B A22B Instruct 25074.3%$0.00171.0m4%
177GPT-4o Mini (temp=1)2.1%$0.002633.4s1%
178Cydonia 24B V4.15.4%$0.00401.0m3%
179Gemma 3 27B0.8%$0.001627.5s0%
180Hermes 3 70B2.4%$0.004542.7s1%
181Gemma 3 4B0.3%$0.000727.5s0%
182Aion 3.5 Mini (Reasoning, High)58.4%$0.0367.7m21%
183Gemma 3 12B0.0%$0.001430.4s0%
184Hermes 3 405B4.0%$0.0151.0m2%
185DeepSeek V3 (2025-03-24)1.8%$0.003750.3s0%
186WizardLM 2 8x22b9.8%$0.00982.3m6%
187ByteDance Seed 2.0 Lite25.0%$0.0143.6m4%
188Nemotron 3 Super27.6%$0.00005.5m15%
189Gemma 4 26B (Reasoning)11.6%$0.00413.1m6%
190Z.AI GLM 4.7 Flash7.3%$0.00502.8m7%
191GPT-5 Nano4.8%$0.00802.2m3%
192Qwen 3.5 122B17.9%$0.0354.0m12%
193Z.AI GLM 4.5 Air8.5%$0.00713.2m6%
194Arcee AI: Trinity Mini0.7%$0.00371.9m0%
195Qwen 3 32B5.3%$0.00422.9m4%
196Aion 3.5 (Reasoning, High)44.5%$0.0836.1m14%
197Cohere Command R+ (Aug. 2024)1.0%$0.05756.5s0%
198Gemma 4 31B (Reasoning)25.2%$0.00435.8m7%
199Nemotron 3 Nano0.9%$0.00392.8m0%
200Claude Sonnet 4.6 (Reasoning)75.9%$0.4485.9m43%
201ByteDance Seed 2.0 Mini20.8%$0.00628.7m20%
202Qwen 3.5 9B8.5%$0.00407.8m4%
203Claude Opus 420.0%$0.3682.4m16%
37.22%

Individual Scenarios

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100.0%
Ember-1 (Reasoning, Max)100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Gemini 4 Argon (Reasoning, Medium)100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
GPT-5.5100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100.0%
GPT-51001001001009498.8%
GPT-6 Luna (Reasoning, High)1001001001008196.3%
Claude Opus 4.7100100100848493.7%
Kimi K2.6100100100887793.1%
DeepSeek V4.1 Flash (Reasoning, High)1001001001005991.7%
Muse Spark 1.1 (Reasoning, Medium)1001001001005991.7%
Z.AI GLM 5.11001001001005991.7%
GPT-6 Sol1001001001005991.7%
Aion 3.01001001001005991.7%
Gemini 2.5 Pro1001001001005991.7%
Muse Wonder (Reasoning, Medium)100100100777790.9%
Kimi K3 (Reasoning, High)100100100777790.9%
Claude Opus 4.61001001001005390.6%
GPT-5.6 Sol1001001001004889.6%
Grok 4.5 (Reasoning, Low)100100100815988.0%
Qwen 3.8 Max (Reasoning, XHigh)100100100775987.2%
Z.AI GLM 5.3 Flash (Reasoning, Max)1001001001003286.3%
DeepSeek V4.1 Flash 1017 (Reasoning, High)1001001001003286.3%
Qwen 3.8 27B (Reasoning, XHigh)10010088814983.6%
Claude Sonnet 5 (Reasoning)10010081775983.5%
Aion 3.5 Mini (Reasoning, High)100100100595983.4%
GPT-5.6 Terra (Reasoning, Medium)10010084774882.0%
Qwen 3.7 Max100100100773281.8%
Mistral Large 4.0 (Reasoning)10010084595980.3%
Hy4 Preview (Reasoning, High)100100100593278.1%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100593278.1%
Claude Sonnet 5 (Reasoning, Low)10010071595977.6%
Kimi K2.5100100100444477.4%
Kimi K3 (Reasoning, Low)100100100592476.5%
GPT-6 Luna (Reasoning, Medium)10010077594475.9%
Muse Spark 1.1 (Reasoning, Minimal)10010010077175.6%
Gemini 3.5 Flash (Reasoning)10010010059472.5%
Gemini 3 Flash (Preview, Reasoning)10010010059472.5%
Z.AI GLM 5 Turbo1007171555369.8%
Xiaomi MIMO v2.6 Pro10010059454469.4%
Claude Opus 4.8 (Reasoning, Low)1001007759467.9%
MiniMax M3847771594867.8%
Claude Haiku 5.5 (Adaptive)1008471443967.6%
Qwen 3.6 Max Preview1005959594463.9%
GPT-5.4 Mini (Reasoning)947759533262.9%
Thinking Machines Inkling (Reasoning)1007759443262.2%
Aion 3.5 (Reasoning, High)1001005948462.1%
Z.AI GLM 510010059321861.7%
Claude Opus 51001001004461.5%
Gemini 3.6 Flash (Reasoning, Minimal)1001001004461.5%
Xiaomi MIMO v2.6 Flash1005959553260.7%
Claude Sonnet 4.61005948484860.7%
GPT-5.6 Luna (Reasoning, Medium)1007753441858.3%
GPT-5.4 (Reasoning, Low)686848464655.4%
Aion 3.0 Mini100725939454.8%
GPT-5.2846848412954.2%
Z.AI GLM 4.6595959464252.7%
GPT-5.6 Terra844846423851.7%
Z.AI GLM 4.792595939751.1%
Gemini 3 Flash (Preview)595948454551.0%
GPT-5.1845648302448.7%
Claude Opus 4.51004832323248.6%
DeepSeek V4 Pro (Reasoning)1005932222247.0%
Qwen 3.5 Plus (2026-02-15)715948282846.8%
DeepSeek V4 Flash (Reasoning)1004232302946.6%
Qwen 3.5 397B A17B594848462745.6%
Claude Haiku 5.5 (Reasoning, Low)534848443245.0%
Xiaomi MIMO v2.51003832322244.7%
Claude Opus 5 (Reasoning)100100104443.5%
o4 Mini High645334323242.9%
Gemini 3.5 Flash (Reasoning, Minimal)1005932131042.6%
Claude Haiku 5.5 (Reasoning, Medium)775732261842.0%
Gemma 4 31B (Reasoning)64594832541.6%
Z.AI GLM 5.2 (Reasoning, High)484848323241.6%
ByteDance Seed 2.0 Lite100593211541.3%
Nemotron 3 Super77474434040.3%
GPT-5.4463838383839.4%
GPT-5 Mini724832222239.4%
Thinking Machines Inkling Small (Reasoning, High)554435332037.4%
Qwen 3.6 27B88392723837.1%
ByteDance Seed 1.6454340242034.5%
Qwen 3.5 27B68393232034.1%
Grok 4.20 (Reasoning)493333292233.2%
Mistral Large 4.0484240191432.6%
Claude Sonnet 4.5323232323231.6%
Qwen 3.5 Flash513632201731.1%
Claude Sonnet 5463229242230.7%
Inception Mercury 2483824221729.7%
Xiaomi MIMO v2.5 Pro59442711729.4%
Muse Glimmer 30B (Reasoning, Medium)373230242028.7%
Qwen 3.7 Flash (Reasoning)71222220828.7%
Gemini 3.5 Flash Lite (Reasoning)10022106027.8%
Z.AI GLM 5.3 Flash (Reasoning, Low)592720191027.0%
MiniMax M2.7383222221525.8%
Qwen 3.5 35B323229271025.8%
Qwen 3.6 Flash383227171525.7%
Qwen 3.5 122B363228221025.7%
Claude Sonnet 432323232125.6%
o4 Mini353222221525.4%
MiniMax M2.5323024221925.3%
GPT-OSS 120B373022221425.2%
Qwen 3.5 Plus (2026-04-20)402725231025.0%
ByteDance Seed 2.0 Mini322422222224.5%
GPT-4o, Aug. 6th (temp=0)55322411124.4%
GPT-5.4 Nano (Reasoning)392424221224.4%
Claude Opus 432323224224.3%
Qwen 3.6 35B322222222023.7%
DeepSeek V4 Flash382424201123.3%
GPT-6 Luna473214111022.8%
Gemini 2.5 Flash322920141121.0%
Gemma 4 26B (Reasoning)35321914420.6%
Mistral Large 2272417171720.1%
Thinking Machines Inkling45171514919.6%
Aion 2.0554400019.6%
Grok 4.3382017121119.5%
DeepSeek V3.132242417019.2%
Ministral 3 8B30242311718.9%
Gemma 4 31B272519121018.6%
Mistral Large 3241717171718.0%
Gemini 3.1 Flash Lite (Reasoning)191919191517.9%
Ministral 3 14B222119131217.6%
Gemma 4 26B191917171717.5%
Gemini 3.1 Flash Lite241919151017.4%
Gemini 3.1 Flash Lite (Preview)221915151517.3%
DeepSeek V4 Pro32201515417.2%
DeepSeek V3.224221714816.8%
Z.AI GLM 4.5222115111116.1%
Laguna S 2.121201918416.0%
GPT-4.124191212915.1%
Ministral 8B28171410715.1%
GPT-5.4 Mini (Reasoning, Low)20171616114.3%
GPT-5.4 Mini17171711813.6%
Qwen 3.5 9B21181412213.3%
Mistral Medium 3.12421117213.1%
Mistral Medium 3.5221010101012.5%
Laguna XS 2.11716108711.7%
Grok 4.2018121111711.7%
Mistral Medium 3.5 (Reasoning)21101010711.6%
Ministral 3B211899111.5%
DeepSeek-V2 Chat2411107411.2%
Claude Haiku 4.51717126511.2%
WizardLM 2 8x22b242433211.1%
Z.AI GLM 4.5 Air291175010.3%
Z.AI GLM 4.7 Flash171198710.3%
DeepSeek V3 (2024-12-26)12111110710.2%
Gemini 3.5 Flash Lite (Reasoning, Minimal)381020010.2%
GPT-5.6 Luna1710107710.0%
GPT-4.1 Mini121110979.7%
Mistral Small 3.2 24B1411101049.5%
Gemini 2.5 Flash (Reasoning)3254229.1%
Gemini 2.5 Flash Lite (Reasoning)2275448.5%
Llama 3.1 70B17108538.3%
Ministral 3 3B121010738.3%
Writer: Palmyra X512108848.2%
Qwen 2.5 72B1396668.0%
Mistral Small 4 (Reasoning)121110707.9%
Mistral NeMO22123207.9%
GPT-5.4 Nano1976527.9%
Qwen 3 32B11106637.1%
GPT-5 Nano1398416.9%
Cydonia 24B V4.12073206.5%
Qwen 3 235B A22B Instruct 25071086536.0%
Grok 4.3 (Reasoning)977605.8%
GPT-4o Mini (temp=0)875425.3%
GPT-5.4 Nano (Reasoning, Low)1453225.1%
GPT-4o, Aug. 6th (temp=1)1076115.1%
Hermes 3 405B1163204.6%
GPT-4o Mini (temp=1)554323.8%
Hermes 3 70B1132213.7%
DeepSeek V3 (2025-03-24)774003.6%
ByteDance Seed 1.6 Flash1411103.5%
Cohere Command R+ (Aug. 2024)730001.9%
Nemotron 3 Nano422001.7%
Mistral Small 4800001.5%
Gemma 3 27B222201.4%
Arcee AI: Trinity Mini511001.4%
GPT-4.1 Nano221001.1%
Gemini 2.5 Flash Lite110000.4%
Gemma 3 4B110000.3%
Gemma 3 12B000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100.0%
Gemini 4 Argon (Reasoning, Medium)100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)1001001001009799.4%
GPT-6 Sol (Reasoning, Medium)1001001001009599.0%
GPT-5.6 Sol (Reasoning, Medium)1001001001009598.9%
Grok 4.7 (Reasoning, High)100100100978195.7%
Gemini 3.7 Flash (Reasoning, Medium)100100100898995.6%
Grok 4.6 (Reasoning, High)100100100817290.7%
Ember-1 (Reasoning, Max)10010097797690.4%
GPT-5.4 (Reasoning)10010091817188.7%
GPT-6 Luna (Reasoning, High)959387817686.5%
Xiaomi MIMO v2.6 Flash10010083776885.5%
Claude Sonnet 5.5 (Reasoning)979076737081.3%
GPT-5.5 (Reasoning, Medium)818181817480.1%
Claude Opus 4.6 (Reasoning)847979797479.1%
GPT-6 Sol979793663778.2%
Grok 4.5 (Reasoning, High)797979747276.7%
Muse Wonder (Reasoning, Medium)818177776275.6%
DeepSeek V4.1 Flash (Reasoning, High)1008181743875.0%
DeepSeek V4 Flash 0731 (Reasoning, High)1009564595774.8%
Gemini 3.6 Flash (Reasoning)817974686874.2%
GPT-6 Luna (Reasoning, Medium)898270625471.4%
Claude Opus 4.8 (Reasoning, Low)887068666270.7%
Z.AI GLM 5.3 (Reasoning, Max)887777733269.5%
Qwen 3.8 Max (Reasoning, XHigh)908177514769.3%
Claude Opus 4.8 (Reasoning)906662626068.0%
Kimi K2.6896766605667.6%
MiniMax M3927458484363.1%
Qwen 3.8 Flash (Reasoning)907970551962.7%
DeepSeek V4 Flash (Reasoning)907858483862.4%
Qwen 3.8 27B (Reasoning, XHigh)807255524761.3%
GPT-5.6 Terra (Reasoning, Medium)856854504460.2%
Muse Spark 1.3 (Reasoning, Medium)756559543657.7%
Claude Opus 5 (Reasoning)89785751756.3%
Z.AI GLM 5.1706261503655.7%
DeepSeek V4.1 Flash 1017 (Reasoning, High)817940393855.4%
Kimi K2.5717056502955.2%
Claude Opus 4.6706059473454.0%
Kimi K3 (Reasoning, Low)686157414053.5%
Mistral Large 4.0 (Reasoning)796243434053.3%
Z.AI GLM 5.3 Flash (Reasoning, Max)777763262253.0%
Claude Sonnet 5 (Reasoning, Low)926447342752.8%
GPT-5.6 Sol817243362852.3%
Claude Sonnet 4.6 (Reasoning)775049424251.9%
Hy4 Preview (Reasoning, High)715343414149.8%
Claude Sonnet 5 (Reasoning)676460292749.5%
Xiaomi MIMO v2.6 Pro727038343048.7%
Kimi K3 (Reasoning, High)545147403845.9%
Claude Opus 583636115545.4%
DeepSeek V4 Pro 0813 (Reasoning, High)953932292844.6%
Z.AI GLM 5.2 (Reasoning, High)644341413544.6%
Gemini 3.5 Flash (Reasoning)545149392744.0%
Muse Spark 1.1 (Reasoning, Medium)674236333342.3%
GPT-5.5564545402241.6%
GPT-5.5 (Reasoning, Low)74544133040.6%
GPT-5.4 Mini (Reasoning)56545126939.2%
GPT-5.4 (Reasoning, Low)605331252438.6%
Z.AI GLM 5 Turbo66653520037.2%
Claude Opus 5 (Reasoning, Low)858275436.7%
Claude Haiku 5.5 (Adaptive)534136311936.1%
Grok 4.5 (Reasoning, Low)573529292334.8%
Claude Opus 4.7 (Reasoning)66612910734.5%
Gemini 2.5 Pro64463425334.3%
Claude Opus 4.7604424212033.9%
Aion 3.5 Mini (Reasoning, High)54453731033.3%
GPT-5.6 Luna (Reasoning, Medium)523131262432.8%
Gemini 3.1 Pro (Preview)797055432.7%
Z.AI GLM 4.7353231292530.6%
Qwen 3.5 27B373431252230.0%
Qwen 3.7 Max6155340029.9%
Aion 2.0433824201929.1%
Gemini 3.6 Flash (Reasoning, Minimal)62541310428.6%
Muse Spark 1.2 (Reasoning, Medium)473926151528.3%
GPT-5.2383325241827.6%
Claude Sonnet 4.6323130222027.1%
Aion 3.5 (Reasoning, High)42363027026.9%
DeepSeek V4 Pro (Reasoning)59292019926.8%
Thinking Machines Inkling (Reasoning)363125212026.6%
Z.AI GLM 4.6432522211825.9%
GPT-5.4 Nano (Reasoning)332726231725.1%
Z.AI GLM 545422116024.8%
Claude Opus 4.5362423211824.6%
Aion 3.0 Mini4942320024.6%
Aion 3.07827170024.5%
Qwen 3.5 397B A17B382724221124.4%
Qwen 3.6 Max Preview292624221723.7%
Xiaomi MIMO v2.5 Pro5826198723.7%
Xiaomi MIMO v2.55330206622.9%
Gemini 3 Flash (Preview, Reasoning)392723111122.2%
Qwen 3.5 Plus (2026-02-15)252420181620.7%
GPT-5.6 Terra252220181620.2%
Mistral Large 4.0241919191920.1%
Claude Haiku 5.5 (Reasoning, Low)252220171619.8%
Grok 4.20 (Reasoning)29232315919.7%
Qwen 3.5 Plus (2026-04-20)292219141419.6%
Mistral Medium 3.1282620111119.3%
GPT-5.4202019181718.9%
Gemini 3 Flash (Preview)252318161118.8%
Claude Sonnet 4.5232019181218.4%
Z.AI GLM 5.3 Flash (Reasoning, Low)44241010218.2%
Muse Spark 1.1 (Reasoning, Minimal)232017151417.7%
ByteDance Seed 2.0 Mini22211716917.0%
Claude Haiku 5.5 (Reasoning, Medium)242117121216.9%
GPT-5.1242315111116.8%
ByteDance Seed 1.625231616116.3%
o4 Mini25251411416.0%
Claude Opus 420201713915.7%
Qwen 3.6 27B25231311515.4%
Claude Sonnet 5201913131215.3%
Nemotron 3 Super3226170014.9%
GPT-5 Mini191814131014.8%
Mistral Medium 3.5181717111014.5%
GPT-5201514111014.2%
Qwen 3.5 Flash23151412513.8%
Mistral Medium 3.5 (Reasoning)161514121213.7%
Qwen 3.5 35B2618149013.4%
Ministral 8B29121110313.1%
GPT-6 Luna191311101012.6%
Gemini 2.5 Flash (Reasoning)39887012.3%
Gemini 2.5 Flash Lite (Reasoning)321874012.1%
o4 Mini High16151210711.9%
GPT-4.11816117711.7%
Thinking Machines Inkling Small (Reasoning, High)2415115411.5%
DeepSeek V3.21712109811.1%
GPT-5.6 Luna1412109810.6%
MiniMax M2.51410109910.5%
Z.AI GLM 4.5161099810.4%
Qwen 3.5 122B18121210010.2%
Qwen 3.7 Flash (Reasoning)28157009.9%
Claude Sonnet 41499999.9%
Gemma 4 31B1399989.6%
Ministral 3 8B13139859.5%
Thinking Machines Inkling121110869.3%
Qwen 3.6 Flash1998549.2%
GPT-OSS 120B13108859.0%
DeepSeek V4 Pro1988649.0%
ByteDance Seed 2.0 Lite141110908.8%
Gemma 4 31B (Reasoning)13129908.8%
Gemma 4 26B11109778.7%
WizardLM 2 8x22b171010618.6%
MiniMax M2.712108768.6%
GPT-5.4 Mini (Reasoning, Low)131110548.5%
Muse Glimmer 30B (Reasoning, Medium)1188778.2%
Ministral 3 14B11108658.0%
DeepSeek V4 Flash1097767.7%
Gemini 2.5 Flash987777.6%
Grok 4.31387727.4%
Laguna S 2.11087667.1%
GPT-5.4 Nano1276636.9%
Z.AI GLM 4.5 Air1098706.7%
Mistral Large 2777766.6%
Gemini 3.1 Flash Lite877556.6%
Qwen 3.6 35B1598006.6%
Inception Mercury 21277336.4%
Gemini 3.1 Flash Lite (Preview)776556.1%
GPT-4o, Aug. 6th (temp=0)966546.1%
GPT-5.4 Nano (Reasoning, Low)877546.1%
Mistral Large 3766656.0%
Gemini 3.1 Flash Lite (Reasoning)866545.9%
Gemini 3.5 Flash (Reasoning, Minimal)988115.5%
Grok 4.201196005.4%
Grok 4.3 (Reasoning)885315.2%
DeepSeek V3 (2024-12-26)1174204.9%
GPT-5.4 Mini755424.7%
Cydonia 24B V4.11173104.3%
Z.AI GLM 4.7 Flash865204.3%
GPT-4.1 Mini654424.2%
Claude Haiku 4.5654334.1%
Gemini 3.5 Flash Lite (Reasoning, Minimal)963204.1%
Ministral 3 3B1232213.9%
Mistral Small 4 (Reasoning)844403.9%
DeepSeek-V2 Chat763213.9%
Qwen 3.5 9B554323.8%
Mistral Small 3.2 24B544323.7%
Ministral 3B554413.6%
GPT-4o, Aug. 6th (temp=1)554323.6%
Hermes 3 405B1430003.5%
Laguna XS 2.1653213.5%
Writer: Palmyra X5772103.4%
Qwen 3 32B1043003.4%
DeepSeek V3.1752113.2%
Llama 3.1 70B543202.8%
GPT-5 Nano433212.6%
Qwen 3 235B A22B Instruct 2507831002.6%
Gemma 4 26B (Reasoning)1300002.6%
Gemini 3.5 Flash Lite (Reasoning)533112.5%
Mistral Small 4631102.1%
Qwen 2.5 72B522002.0%
Hermes 3 70B500001.0%
Gemini 2.5 Flash Lite400000.9%
GPT-4o Mini (temp=0)111100.8%
Mistral NeMO210000.7%
ByteDance Seed 1.6 Flash0 – – – – 0.4%
GPT-4o Mini (temp=1)110000.4%
Gemma 3 27B000000.3%
GPT-4.1 Nano000000.2%
Gemma 3 4B100000.2%
Cohere Command R+ (Aug. 2024)000000.1%
DeepSeek V3 (2025-03-24)000000.1%
Arcee AI: Trinity Mini000000.1%
Nemotron 3 Nano000000.0%
Gemma 3 12B000000.0%