Relationship recall

Test: Relationship tree

Avg. Score
32.6%
Scenarios
2

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Gemini 4 Argon (Reasoning, Medium)100.0%$0.03919.1s100%
2GPT-6.1 Sol (Reasoning, Medium)100.0%$0.04250.1s100%
3GPT-6 Sol (Reasoning, Medium)99.6%$0.05146.3s98%
4Gemini 3.7 Flash (Reasoning, Medium)97.9%$0.03623.4s92%
5Gemini 3.8 Flash (Reasoning, Medium)99.3%$0.06742.1s96%
6GPT-6 Luna (Reasoning, High)92.7%$0.00551.4m80%
7Claude Opus 5.5 (Reasoning)100.0%$0.20340.6s100%
8GPT-5.6 Sol (Reasoning, Medium)98.6%$0.18659.0s92%
9Claude Sonnet 5.5 (Reasoning)88.2%$0.10128.5s67%
10DeepSeek V4.1 Flash (Reasoning, High)85.7%$0.0141.5m58%
11GPT-6 Sol81.7%$0.03521.1s52%
12Grok 4.6 (Reasoning, High)96.4%$0.1043.7m85%
13Grok 4.5 (Reasoning, High)88.7%$0.0802.0m69%
14GPT-5.4 (Reasoning)96.3%$0.1752.6m87%
15Gemini 3.6 Flash (Reasoning)84.3%$0.09641.4s59%
16Grok 4.7 (Reasoning, High)98.0%$0.1315.4m91%
17Muse Wonder (Reasoning, Medium)79.9%$0.11620.5s52%
18DeepSeek V4.1 Flash 1017 (Reasoning, High)73.6%$0.005658.2s39%
19Ember-1 (Reasoning, Max)94.5%$0.2113.4m81%
20GPT-5.5 (Reasoning, Medium)91.5%$0.2512.0m75%
21DeepSeek V4 Flash 0731 (Reasoning, High)86.7%$0.00735.6m60%
22GPT-6 Luna (Reasoning, Medium)63.7%$0.00351.1m33%
23Qwen 3.8 Flash (Reasoning)81.4%$0.0154.9m46%
24GPT-5.6 Sol68.7%$0.09618.1s29%
25GPT-5.6 Terra (Reasoning, Medium)60.9%$0.06024.2s26%
26Muse Spark 1.3 (Reasoning, Medium)69.9%$0.0551.9m29%
27GPT-5.5 (Reasoning, Low)65.0%$0.09742.6s20%
28Grok 4.5 (Reasoning, Low)57.5%$0.04049.7s15%
29Muse Spark 1.1 (Reasoning, Medium)57.5%$0.04235.1s13%
30Claude Opus 4.6 (Reasoning)88.7%$0.3562.6m69%
31Xiaomi MIMO v2.6 Pro52.3%$0.00712.0m21%
32Claude Haiku 5.5 (Adaptive)42.3%$0.006429.8s15%
33GPT-5.4 (Reasoning, Low)42.3%$0.05335.5s26%
34GPT-5.562.8%$0.10537.1s16%
35Muse Spark 1.2 (Reasoning, Medium)56.1%$0.04629.8s7%
36Kimi K3 (Reasoning, Low)54.7%$0.0591.2m17%
37Mistral Large 4.0 (Reasoning)63.5%$0.0453.9m31%
38Claude Opus 4.664.2%$0.15431.4s21%
39Kimi K2.672.6%$0.0825.0m39%
40Gemini 3 Flash (Preview)28.3%$0.00877.1s19%
41Gemini 2.5 Pro58.0%$0.09958.2s14%
42GPT-5.6 Terra31.6%$0.0349.4s15%
43GPT-5.6 Luna (Reasoning, Medium)34.1%$0.02718.6s11%
44Qwen 3.8 27B (Reasoning, XHigh)64.8%$0.0534.9m30%
45Gemini 3.6 Flash (Reasoning, Minimal)39.1%$0.0297.1s3%
46DeepSeek V4 Flash (Reasoning)49.8%$0.00413.5m19%
47Z.AI GLM 5.2 (Reasoning, High)33.8%$0.0391.8m25%
48Hy4 Preview (Reasoning, High)64.9%$0.0564.7m26%
49Claude Haiku 5.5 (Reasoning, Low)24.0%$0.005019.9s13%
50GPT-5.425.4%$0.03317.4s17%
51Xiaomi MIMO v2.6 Flash68.7%$0.00787.0m34%
52Gemini 3 Flash (Preview, Reasoning)41.5%$0.03040.3s4%
53GPT-5.129.3%$0.02517.7s11%
54Claude Opus 4.7 (Reasoning)61.7%$0.18425.0s15%
55Muse Spark 1.1 (Reasoning, Minimal)39.4%$0.03018.2s1%
56Claude Sonnet 5 (Reasoning, Low)59.2%$0.1751.7m26%
57Claude Sonnet 4.636.4%$0.08123.3s15%
58GPT-5.4 Nano (Reasoning)19.5%$0.008841.7s17%
59Gemini 3.5 Flash (Reasoning)49.7%$0.11349.4s10%
60Gemini 3.1 Pro (Preview)65.3%$0.1721.5m14%
61GPT-5.235.0%$0.06250.7s13%
62Claude Opus 4.756.1%$0.18224.3s14%
63Kimi K2.560.5%$0.0315.4m24%
64Xiaomi MIMO v2.526.5%$0.00271.1m10%
65Claude Sonnet 5 (Reasoning)58.1%$0.1721.7m21%
66Claude Haiku 5.5 (Reasoning, Medium)21.2%$0.004616.7s6%
67Z.AI GLM 5 Turbo47.0%$0.0443.6m21%
68DeepSeek V4 Pro 0813 (Reasoning, High)56.5%$0.0603.5m13%
69Z.AI GLM 4.633.1%$0.0183.0m22%
70Z.AI GLM 5.3 Flash (Reasoning, Low)17.1%$0.001242.5s11%
71Mistral Large 4.018.9%$0.008030.1s8%
72Inception Mercury 214.7%$0.009612.8s8%
73 Kimi K3 (Reasoning, High)55.2%$0.1023.1m16%
74Claude Opus 5 (Reasoning, Low)65.1%$0.24646.5s15%
75ByteDance Seed 1.618.7%$0.0141.2m13%
76Qwen 3.7 Max50.8%$0.0453.5m10%
77Gemini 3.5 Flash (Reasoning, Minimal)20.7%$0.0246.3s2%
78Claude Opus 4.8 (Reasoning)76.4%$0.3811.9m43%
79Grok 4.20 (Reasoning)16.9%$0.02740.0s11%
80Claude Sonnet 4.518.4%$0.07718.6s16%
81GPT-6 Luna11.4%$0.001212.1s5%
82DeepSeek V4 Pro (Reasoning)31.7%$0.0202.3m9%
83GPT-551.8%$0.0922.3m4%
84Claude Opus 549.0%$0.19526.4s11%
85Claude Opus 4.530.3%$0.13119.8s13%
86Gemini 3.5 Flash Lite (Reasoning)12.7%$0.00623.1s0%
87MiniMax M2.511.6%$0.003645.0s7%
88Muse Glimmer 30B (Reasoning, Medium)13.7%$0.00621.1m9%
89Gemini 2.5 Flash9.3%$0.00756.1s4%
90MiniMax M2.711.0%$0.006321.4s4%
91Mistral Large 38.2%$0.009117.5s6%
92Mistral Medium 3.19.4%$0.01021.2s6%
93Gemini 3.1 Flash Lite (Reasoning)6.0%$0.00333.2s5%
94Qwen 3.5 397B A17B28.3%$0.0452.6m16%
95Thinking Machines Inkling Small (Reasoning, High)16.4%$0.0131.1m7%
96Grok 4.38.4%$0.00947.8s4%
97Gemini 3.1 Flash Lite (Preview)5.8%$0.00463.2s4%
98Ministral 3 8B7.6%$0.002217.0s4%
99Z.AI GLM 5.3 Flash (Reasoning, Max)64.7%$0.00628.0m22%
100Ministral 8B6.4%$0.001615.6s4%
101Qwen 3.7 Flash (Reasoning)13.2%$0.001756.4s4%
102Claude Sonnet 516.0%$0.06817.1s10%
103DeepSeek V3.28.7%$0.004633.9s5%
104Gemma 4 26B6.5%$0.001827.2s5%
105Gemini 3.1 Flash Lite5.9%$0.00313.3s2%
106DeepSeek V4 Pro7.4%$0.008317.3s4%
107o4 Mini High21.0%$0.0591.3m11%
108Qwen 3.6 Flash11.4%$0.01551.7s7%
109DeepSeek V4 Flash11.8%$0.002051.4s4%
110GPT-5.4 Mini (Reasoning, Low)6.8%$0.01414.4s5%
111GPT-5.4 Mini6.1%$0.00877.9s3%
112GPT-5 Mini20.3%$0.0192.0m9%
113Ministral 3 14B6.9%$0.003125.4s4%
114MiniMax M359.9%$0.0369.3m44%
115Z.AI GLM 5.168.0%$0.0817.3m26%
116Xiaomi MIMO v2.5 Pro20.6%$0.00761.9m5%
117Z.AI GLM 4.58.0%$0.007548.1s6%
118o4 Mini12.0%$0.03745.7s8%
119Ministral 3 3B3.5%$0.00079.9s3%
120Grok 4.205.9%$0.0207.5s4%
121GPT-4.17.0%$0.0248.5s4%
122Ministral 3B4.1%$0.00079.8s2%
123Mistral Large 29.2%$0.03720.5s6%
124Qwen 3.6 35B10.4%$0.0141.1m8%
125GPT-5.6 Luna4.5%$0.0128.7s3%
126Gemini 3.5 Flash Lite (Reasoning, Minimal)4.8%$0.00573.2s1%
127GPT-5.4 Mini (Reasoning)38.2%$0.1172.8m18%
128GPT-5.4 Nano3.1%$0.002611.2s2%
129Thinking Machines Inkling9.7%$0.01843.6s3%
130Mistral Small 3.2 24B3.5%$0.001713.8s1%
131GPT-4o, Aug. 6th (temp=0)9.3%$0.0328.4s1%
132GPT-4.1 Mini3.5%$0.004626.0s3%
133GPT-5.4 Nano (Reasoning, Low)2.4%$0.003813.8s2%
134Laguna XS 2.14.8%$0.001241.3s3%
135Mistral Medium 3.55.5%$0.0338.3s4%
136GPT-4o Mini (temp=0)2.6%$0.001920.7s1%
137DeepSeek V3.19.1%$0.005552.0s1%
138Writer: Palmyra X53.9%$0.01618.4s3%
139Claude Haiku 4.54.8%$0.0248.2s2%
140Z.AI GLM 4.731.8%$0.0253.7m9%
141Mistral NeMO2.2%$0.002415.4s0%
142GPT-4.1 Nano0.2%$0.00096.9s0%
143Gemini 2.5 Flash Lite0.2%$0.00234.8s0%
144DeepSeek-V2 Chat4.3%$0.004934.0s1%
145Mistral Medium 3.5 (Reasoning)4.0%$0.0328.6s3%
146Z.AI GLM 540.4%$0.0374.6m11%
147Mistral Small 40.4%$0.00307.4s0%
148DeepSeek V3 (2024-12-26)3.3%$0.004540.2s3%
149Mistral Small 4 (Reasoning)2.0%$0.005124.7s1%
150Grok 4.3 (Reasoning)1.5%$0.01313.9s1%
151Qwen 3.5 35B13.1%$0.0152.0m8%
152Qwen 3.5 27B25.1%$0.0293.7m15%
153GPT-OSS 120B13.4%$0.00402.2m7%
154Claude Sonnet 412.5%$0.07217.5s4%
155Llama 3.1 70B3.1%$0.006434.0s1%
156Gemini 2.5 Flash Lite (Reasoning)5.6%$0.00731.0m2%
157GPT-4o Mini (temp=1)1.4%$0.002633.4s1%
158Gemini 2.5 Flash (Reasoning)6.1%$0.02337.3s1%
159ByteDance Seed 1.6 Flash1.4%$0.002034.2s0%
160Gemma 3 4B0.1%$0.000727.5s0%
161Gemma 3 27B0.2%$0.001627.5s0%
162Gemma 3 12B0.0%$0.001430.4s0%
163GPT-4o, Aug. 6th (temp=1)1.6%$0.0299.1s1%
164Qwen 3 235B A22B Instruct 25073.1%$0.00171.0m2%
165Thinking Machines Inkling (Reasoning)36.5%$0.0604.1m11%
166Aion 3.0 Mini35.2%$0.0304.9m13%
167Hermes 3 70B1.2%$0.004542.7s0%
168Cydonia 24B V4.12.6%$0.00401.0m1%
169Aion 2.017.7%$0.0243.0m10%
170Qwen 3.5 Plus (2026-02-15)24.9%$0.0213.9m10%
171DeepSeek V3 (2025-03-24)0.8%$0.003750.3s0%
172Claude Opus 5 (Reasoning)43.4%$0.24747.3s11%
173Aion 3.054.8%$0.0745.8m9%
174Gemma 4 31B6.8%$0.00281.9m3%
175Qwen 3.6 Max Preview34.7%$0.0843.7m10%
176Qwen 2.5 72B1.7%$0.00561.2m1%
177Laguna S 2.16.4%$0.00062.0m3%
178Qwen 3.6 27B17.7%$0.0342.7m5%
179Hermes 3 405B1.8%$0.0151.0m1%
180Qwen 3.5 Flash16.3%$0.00403.8m10%
181Qwen 3.8 Max (Reasoning, XHigh)74.9%$0.1609.6m42%
182Z.AI GLM 5.3 (Reasoning, Max)80.6%$0.18710.3m49%
183Aion 3.5 Mini (Reasoning, High)54.1%$0.0367.7m16%
184Qwen 3.5 Plus (2026-04-20)11.8%$0.0233.1m8%
185ByteDance Seed 2.0 Lite20.3%$0.0143.6m1%
186WizardLM 2 8x22b6.2%$0.00982.3m2%
187Claude Opus 4.8 (Reasoning, Low)61.4%$0.3831.9m25%
188Nemotron 3 Super23.0%$0.00005.5m13%
189Arcee AI: Trinity Mini0.5%$0.00371.9m0%
190Gemma 4 26B (Reasoning)8.1%$0.00413.1m3%
191Z.AI GLM 4.7 Flash3.9%$0.00502.8m4%
192GPT-5 Nano1.9%$0.00802.2m1%
193Cohere Command R+ (Aug. 2024)0.4%$0.05756.5s0%
194Qwen 3 32B2.0%$0.00422.9m2%
195Z.AI GLM 4.5 Air4.8%$0.00713.2m2%
196Nemotron 3 Nano0.4%$0.00392.8m0%
197Qwen 3.5 122B12.1%$0.0354.0m6%
198Aion 3.5 (Reasoning, High)39.5%$0.0836.1m8%
199Gemma 4 31B (Reasoning)19.5%$0.00435.8m2%
200Claude Sonnet 4.6 (Reasoning)70.7%$0.4485.9m32%
201ByteDance Seed 2.0 Mini12.7%$0.00628.7m11%
202Qwen 3.5 9B4.6%$0.00407.8m1%
203Claude Opus 413.6%$0.3682.4m7%
32.56%

Individual Scenarios

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100.0%
Ember-1 (Reasoning, Max)100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100.0%
GPT-5.6 Sol (Reasoning, Medium)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100.0%
Qwen 3.8 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Gemini 4 Argon (Reasoning, Medium)100100100100100100.0%
GPT-5.5 (Reasoning, Medium)100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100.0%
Claude Sonnet 5.5 (Reasoning)100100100100100100.0%
Grok 4.6 (Reasoning, High)100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)100100100100100100.0%
Grok 4.7 (Reasoning, High)100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
GPT-5.5100100100100100100.0%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100100100100.0%
GPT-6 Luna (Reasoning, High)1001001001009198.2%
GPT-51001001001009198.2%
Claude Opus 4.7100100100838393.1%
DeepSeek V4.1 Flash (Reasoning, High)1001001001005591.0%
Muse Spark 1.1 (Reasoning, Medium)1001001001005591.0%
Z.AI GLM 5.11001001001005591.0%
GPT-6 Sol1001001001005591.0%
Aion 3.01001001001005591.0%
Gemini 2.5 Pro1001001001005591.0%
Kimi K2.6100100100836890.1%
Claude Opus 4.61001001001004989.8%
Grok 4.5 (Reasoning, Low)100100100915589.2%
GPT-5.6 Sol1001001001004488.8%
Muse Wonder (Reasoning, Medium)100100100686887.1%
Kimi K3 (Reasoning, High)100100100686887.1%
Z.AI GLM 5.3 Flash (Reasoning, Max)1001001001002785.4%
DeepSeek V4.1 Flash 1017 (Reasoning, High)1001001001002785.4%
Qwen 3.8 27B (Reasoning, XHigh)10010091834984.6%
Qwen 3.8 Max (Reasoning, XHigh)100100100685584.5%
Claude Sonnet 5 (Reasoning)10010091685582.7%
Aion 3.5 Mini (Reasoning, High)100100100555581.9%
Qwen 3.7 Max100100100682778.9%
GPT-5.6 Terra (Reasoning, Medium)10010083684478.8%
Mistral Large 4.0 (Reasoning)10010083555578.5%
Hy4 Preview (Reasoning, High)100100100552776.4%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100552776.4%
Claude Sonnet 5 (Reasoning, Low)10010068555575.5%
Kimi K3 (Reasoning, Low)100100100552175.1%
Muse Spark 1.1 (Reasoning, Minimal)10010010068073.6%
Kimi K2.5100100100352772.3%
GPT-6 Luna (Reasoning, Medium)10010068553571.4%
Gemini 3.5 Flash (Reasoning)10010010055171.3%
Gemini 3 Flash (Preview, Reasoning)10010010055171.3%
Xiaomi MIMO v2.6 Pro10010055393565.7%
Z.AI GLM 5 Turbo1006868494465.7%
Claude Opus 4.8 (Reasoning, Low)1001006855164.8%
MiniMax M3836868554463.3%
Claude Haiku 5.5 (Adaptive)1008361353562.6%
Claude Opus 51001001001160.6%
Gemini 3.6 Flash (Reasoning, Minimal)1001001001160.6%
Aion 3.5 (Reasoning, High)1001005544160.0%
Qwen 3.6 Max Preview1005555553559.8%
Z.AI GLM 510010055271659.5%
GPT-5.4 (Reasoning, Low)757549494458.4%
Claude Sonnet 4.61005544444457.3%
Xiaomi MIMO v2.6 Flash1005555492757.2%
Thinking Machines Inkling (Reasoning)1006855352456.2%
GPT-5.4 Mini (Reasoning)916844442153.4%
GPT-5.2837544352452.0%
GPT-5.6 Terra834944443951.7%
GPT-5.6 Luna (Reasoning, Medium)1006844351151.5%
Aion 3.0 Mini100615535150.4%
Z.AI GLM 4.6555555443949.5%
GPT-5.1836144312448.4%
Z.AI GLM 4.791555535648.2%
Gemini 3 Flash (Preview)555544393946.3%
Claude Opus 4.51004427272745.0%
DeepSeek V4 Flash (Reasoning)1004427242143.1%
DeepSeek V4 Pro (Reasoning)1005527161642.6%
Claude Opus 5 (Reasoning)10010081142.2%
Xiaomi MIMO v2.51003927271641.7%
Qwen 3.5 Plus (2026-02-15)685544212141.6%
Qwen 3.5 397B A17B554444442141.4%
GPT-5.4493939393941.1%
Gemini 3.5 Flash (Reasoning, Minimal)100552711840.3%
Claude Haiku 5.5 (Reasoning, Low)494444352739.7%
ByteDance Seed 2.0 Lite10055277438.5%
Z.AI GLM 5.2 (Reasoning, High)444444272737.1%
o4 Mini High554431272736.7%
Gemma 4 31B (Reasoning)55554427236.6%
Claude Haiku 5.5 (Reasoning, Medium)754927181035.7%
Nemotron 3 Super68393531034.4%
Qwen 3.5 27B75352727032.7%
GPT-5 Mini614427161632.6%
Mistral Large 4.0444439131030.0%
ByteDance Seed 1.6443931181829.9%
Qwen 3.6 27B83311813529.9%
Thinking Machines Inkling Small (Reasoning, High)493527241129.2%
Claude Sonnet 4.5272727272727.0%
Claude Sonnet 5442724211626.2%
Grok 4.20 (Reasoning)442424241626.1%
Inception Mercury 2393924131325.7%
Qwen 3.5 Flash443127131325.7%
Gemini 3.5 Flash Lite (Reasoning)1001664025.0%
Muse Glimmer 30B (Reasoning, Medium)313127181824.9%
Xiaomi MIMO v2.5 Pro4935247323.5%
Qwen 3.7 Flash (Reasoning)68161613523.4%
Z.AI GLM 5.3 Flash (Reasoning, Low)55211813823.0%
GPT-OSS 120B313121211022.5%
DeepSeek V4 Flash39242416721.8%
Claude Sonnet 427272727121.7%
GPT-5.4 Nano (Reasoning)35242416821.2%
Claude Opus 427272724121.1%
Qwen 3.5 35B27272421620.8%
MiniMax M2.735271616719.9%
Qwen 3.5 122B31272116619.9%
Qwen 3.6 Flash312721111019.9%
MiniMax M2.5272421161119.7%
ByteDance Seed 2.0 Mini272416161619.5%
Qwen 3.6 35B272116161318.4%
GPT-4o, Aug. 6th (temp=0)4921135017.6%
DeepSeek V3.127242413017.6%
o4 Mini27211616817.4%
Qwen 3.5 Plus (2026-04-20)31211613617.2%
Gemini 2.5 Flash27241810717.0%
GPT-6 Luna3924107617.0%
Mistral Large 2242113131316.9%
Aion 2.0493500016.8%
Thinking Machines Inkling39161310516.5%
Grok 4.33118138715.4%
Gemma 4 26B (Reasoning)2727117214.9%
Mistral Large 3211313131314.8%
DeepSeek V3.224161310613.6%
Ministral 3 8B2116166512.4%
Ministral 3 14B16161111611.9%
DeepSeek V4 Pro271388211.8%
Gemma 4 31B2116116411.4%
GPT-4.1241188511.2%
Gemma 4 26B131111101011.1%
GPT-5.4 Mini13131310611.1%
Z.AI GLM 4.5181687711.1%
Gemini 3.1 Flash Lite (Reasoning)11111111810.8%
Laguna S 2.118111111110.7%
Mistral Medium 3.1241873110.6%
Gemini 3.1 Flash Lite1811118410.6%
GPT-5.4 Mini (Reasoning, Low)1813118110.3%
Gemini 3.1 Flash Lite (Preview)161188810.3%
WizardLM 2 8x22b242421110.3%
Grok 4.2021117659.9%
Ministral 8B18117649.2%
Gemini 3.5 Flash Lite (Reasoning, Minimal)3941008.8%
Laguna XS 2.116118438.4%
Qwen 3.5 9B16108718.4%
Claude Haiku 4.513138228.0%
DeepSeek-V2 Chat2476217.9%
Z.AI GLM 4.5 Air24103207.8%
Ministral 3B1686607.1%
Gemini 2.5 Flash (Reasoning)2722116.5%
Writer: Palmyra X51186616.4%
Mistral Small 3.2 24B1087616.3%
Mistral Medium 3.51644446.2%
GPT-4.1 Mini876556.0%
Z.AI GLM 4.7 Flash1066545.9%
GPT-5.6 Luna1364335.8%
Llama 3.1 70B1384215.7%
DeepSeek V3 (2024-12-26)776635.6%
Gemini 2.5 Flash Lite (Reasoning)1643225.5%
Ministral 3 3B877415.4%
Qwen 3 235B A22B Instruct 2507886215.1%
Mistral Medium 3.5 (Reasoning)1144425.0%
GPT-4o Mini (temp=0)1055324.9%
GPT-5.4 Nano1143214.4%
Mistral NeMO1361004.1%
Cydonia 24B V4.11322103.7%
Mistral Small 4 (Reasoning)644303.3%
GPT-5 Nano654203.3%
Qwen 3 32B643213.2%
Qwen 2.5 72B632223.1%
GPT-5.4 Nano (Reasoning, Low)1021103.0%
GPT-4o Mini (temp=1)532212.6%
Hermes 3 405B722102.5%
GPT-4o, Aug. 6th (temp=1)632002.4%
Hermes 3 70B811102.1%
Grok 4.3 (Reasoning)332202.1%
ByteDance Seed 1.6 Flash710001.6%
DeepSeek V3 (2025-03-24)332001.6%
Arcee AI: Trinity Mini500001.0%
Cohere Command R+ (Aug. 2024)220000.8%
Nemotron 3 Nano111000.7%
Mistral Small 4200000.5%
Gemma 3 27B100000.3%
GPT-4.1 Nano000000.3%
Gemini 2.5 Flash Lite100000.2%
Gemma 3 4B000000.1%
Gemma 3 12B000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 5.5 (Reasoning)100100100100100100.0%
GPT-6.1 Sol (Reasoning, Medium)100100100100100100.0%
Gemini 4 Argon (Reasoning, Medium)100100100100100100.0%
GPT-6 Sol (Reasoning, Medium)1001001001009699.3%
Gemini 3.8 Flash (Reasoning, Medium)1001001001009398.6%
GPT-5.6 Sol (Reasoning, Medium)1001001001008697.3%
Grok 4.7 (Reasoning, High)100100100938695.9%
Gemini 3.7 Flash (Reasoning, Medium)100100100909095.9%
Grok 4.6 (Reasoning, High)100100100867792.7%
GPT-5.4 (Reasoning)10010096868092.6%
Ember-1 (Reasoning, Max)10010093807289.0%
GPT-6 Luna (Reasoning, High)939086868087.2%
GPT-5.5 (Reasoning, Medium)868686866982.9%
DeepSeek V4.1 Flash (Reasoning, High)1008686834680.4%
Xiaomi MIMO v2.6 Flash10010074666180.3%
Claude Opus 4.6 (Reasoning)808080776977.3%
Grok 4.5 (Reasoning, High)808080776977.3%
Claude Sonnet 5.5 (Reasoning)939372665976.5%
DeepSeek V4 Flash 0731 (Reasoning, High)1009669564673.4%
Muse Wonder (Reasoning, Medium)868674744272.7%
GPT-6 Sol939383504472.5%
Gemini 3.6 Flash (Reasoning)868069545468.7%
Qwen 3.8 Max (Reasoning, XHigh)938674383565.3%
Qwen 3.8 Flash (Reasoning)938066591662.8%
DeepSeek V4.1 Flash 1017 (Reasoning, High)868050484661.9%
Z.AI GLM 5.3 (Reasoning, Max)867461543061.2%
Claude Opus 4.8 (Reasoning, Low)865954504258.1%
MiniMax M3806969333256.5%
DeepSeek V4 Flash (Reasoning)937748442156.5%
GPT-6 Luna (Reasoning, Medium)745959523656.0%
Kimi K2.6905652403855.1%
Hy4 Preview (Reasoning, High)745454503553.3%
Claude Opus 4.8 (Reasoning)935042423852.8%
Kimi K2.5726159322048.6%
GPT-5.6 Sol866440322148.5%
Mistral Large 4.0 (Reasoning)746438363048.5%
Z.AI GLM 5.1594642404045.1%
Qwen 3.8 27B (Reasoning, XHigh)696446252245.0%
Claude Opus 5 (Reasoning)74564838744.6%
Z.AI GLM 5.3 Flash (Reasoning, Max)747444171144.0%
GPT-5.6 Terra (Reasoning, Medium)803838362243.0%
Claude Sonnet 5 (Reasoning, Low)804640301943.0%
Claude Sonnet 4.6 (Reasoning)744432292941.4%
Muse Spark 1.3 (Reasoning, Medium)524836362739.9%
Xiaomi MIMO v2.6 Pro645929222138.9%
Claude Opus 4.6594038362038.6%
Claude Opus 561545017537.4%
DeepSeek V4 Pro 0813 (Reasoning, High)862924242136.7%
Kimi K3 (Reasoning, Low)544033271734.3%
Claude Sonnet 5 (Reasoning)464238241933.6%
Gemini 3.1 Pro (Preview)806443230.6%
Z.AI GLM 5.2 (Reasoning, High)463526242230.4%
Claude Opus 5 (Reasoning, Low)726693230.3%
GPT-5.5 (Reasoning, Low)69322921030.1%
Z.AI GLM 5 Turbo6148267028.4%
Gemini 3.5 Flash (Reasoning)442727241928.2%
Aion 3.5 Mini (Reasoning, High)46352625026.2%
GPT-5.4 (Reasoning, Low)443820151426.1%
Grok 4.5 (Reasoning, Low)462622201525.9%
GPT-5.5353532161125.6%
Gemini 2.5 Pro56262616125.0%
Muse Spark 1.1 (Reasoning, Medium)422918171424.0%
Claude Opus 4.7 (Reasoning)5032229423.3%
Kimi K3 (Reasoning, High)292524211823.3%
GPT-5.4 Mini (Reasoning)40322418223.0%
Qwen 3.7 Max6126260022.6%
Claude Haiku 5.5 (Adaptive)353019151122.0%
Z.AI GLM 5504684021.4%
DeepSeek V4 Pro (Reasoning)5629116220.8%
Aion 3.0 Mini4429270019.9%
Claude Opus 4.733301310919.1%
Aion 3.5 (Reasoning, High)29242419019.0%
Aion 2.0322216131118.7%
Aion 3.0642550018.6%
GPT-5.227201915918.1%
GPT-5.4 Nano (Reasoning)25211917717.8%
Gemini 3.6 Flash (Reasoning, Minimal)4227116317.7%
Xiaomi MIMO v2.5 Pro661083217.6%
Qwen 3.5 27B292116111117.5%
Thinking Machines Inkling (Reasoning)27221413816.8%
Z.AI GLM 4.632211511516.8%
GPT-5.6 Luna (Reasoning, Medium)25241412916.7%
Claude Opus 4.52718149915.6%
Claude Sonnet 4.620191713815.5%
Z.AI GLM 4.721181711915.4%
Qwen 3.5 397B A17B25241311415.2%
Muse Spark 1.2 (Reasoning, Medium)2413118712.3%
Gemini 3 Flash (Preview, Reasoning)331183311.8%
Nemotron 3 Super2221140011.6%
GPT-5.6 Terra15131110811.4%
Xiaomi MIMO v2.5351092111.3%
Z.AI GLM 5.3 Flash (Reasoning, Low)261865111.2%
Gemini 3 Flash (Preview)211575310.3%
GPT-5.1201287410.1%
Claude Sonnet 4.5131111869.8%
GPT-5.412119979.8%
Qwen 3.6 Max Preview1698879.5%
Qwen 3.5 Plus (2026-02-15)14106558.3%
Mistral Medium 3.12177328.2%
Claude Haiku 5.5 (Reasoning, Low)12108658.2%
GPT-5 Mini1399558.0%
Mistral Large 4.01377767.7%
Grok 4.20 (Reasoning)12119427.7%
ByteDance Seed 1.619105407.5%
Qwen 3.5 Flash1775416.9%
Claude Haiku 5.5 (Reasoning, Medium)1385436.6%
o4 Mini1096436.5%
Qwen 3.5 Plus (2026-04-20)1366526.3%
Claude Opus 4977526.1%
ByteDance Seed 2.0 Mini1095326.0%
Claude Sonnet 51086335.8%
Gemini 2.5 Flash Lite (Reasoning)1973105.8%
GPT-6 Luna1165535.8%
Gemini 2.5 Flash (Reasoning)2422205.7%
Qwen 3.6 27B1085325.6%
GPT-51354325.5%
Qwen 3.5 35B1465205.4%
o4 Mini High876425.4%
Muse Spark 1.1 (Reasoning, Minimal)655545.1%
Z.AI GLM 4.5974324.9%
Mistral Medium 3.5766324.8%
Qwen 3.5 122B954404.3%
GPT-OSS 120B754324.3%
DeepSeek V3.2653323.8%
Ministral 8B843313.7%
Thinking Machines Inkling Small (Reasoning, High)863113.7%
Inception Mercury 2961113.6%
MiniMax M2.5643223.6%
Claude Sonnet 4822223.3%
GPT-5.4 Mini (Reasoning, Low)653213.2%
GPT-5.6 Luna543223.2%
Mistral Medium 3.5 (Reasoning)333333.1%
Qwen 3.6 Flash822213.0%
Thinking Machines Inkling443313.0%
Qwen 3.7 Flash (Reasoning)851002.9%
DeepSeek V4 Pro1011112.9%
GPT-4.1443222.8%
Ministral 3 8B543212.8%
Muse Glimmer 30B (Reasoning, Medium)522222.6%
Gemma 4 31B (Reasoning)432202.4%
Qwen 3.6 35B533002.3%
Gemma 4 31B322222.2%
WizardLM 2 8x22b432102.1%
ByteDance Seed 2.0 Lite333202.1%
MiniMax M2.7432112.1%
Laguna S 2.1332112.0%
Gemma 4 26B222222.0%
Grok 4.20432001.9%
Ministral 3 14B332111.9%
Z.AI GLM 4.7 Flash522001.9%
GPT-5.4 Nano322211.8%
DeepSeek V4 Flash322211.8%
GPT-5.4 Nano (Reasoning, Low)322111.8%
Z.AI GLM 4.5 Air421101.7%
Ministral 3 3B421101.6%
Gemini 2.5 Flash222111.6%
Claude Haiku 4.5222111.6%
Mistral Large 3222111.5%
Mistral Large 2221111.5%
Grok 4.3321101.4%
Cydonia 24B V4.1421101.4%
Writer: Palmyra X5331001.4%
Gemini 3.1 Flash Lite221111.3%
Gemini 3.1 Flash Lite (Preview)211111.3%
Gemma 4 26B (Reasoning)600001.3%
Laguna XS 2.1321001.2%
Gemini 3.1 Flash Lite (Reasoning)211111.2%
Hermes 3 405B510001.1%
Ministral 3B211101.1%
Qwen 3 235B A22B Instruct 2507411001.1%
GPT-4o, Aug. 6th (temp=0)211111.1%
Gemini 3.5 Flash (Reasoning, Minimal)221001.1%
GPT-5.4 Mini211101.0%
GPT-4.1 Mini211101.0%
DeepSeek V3 (2024-12-26)211001.0%
Grok 4.3 (Reasoning)211100.9%
Qwen 3 32B311000.9%
GPT-4o, Aug. 6th (temp=1)111100.9%
Qwen 3.5 9B111100.9%
Gemini 3.5 Flash Lite (Reasoning, Minimal)211000.8%
Mistral Small 4 (Reasoning)111100.8%
DeepSeek-V2 Chat211000.7%
Mistral Small 3.2 24B111000.6%
DeepSeek V3.1110000.6%
GPT-5 Nano111000.5%
Llama 3.1 70B110000.5%
Mistral Small 4110000.4%
GPT-4o Mini (temp=0)110000.4%
Gemini 3.5 Flash Lite (Reasoning)110000.4%
Hermes 3 70B200000.4%
Qwen 2.5 72B100000.4%
Mistral NeMO100000.3%
Gemini 2.5 Flash Lite100000.2%
GPT-4o Mini (temp=1)000000.1%
ByteDance Seed 1.6 Flash0 – – – – 0.1%
Gemma 3 4B000000.1%
Gemma 3 27B000000.0%
Cohere Command R+ (Aug. 2024)000000.0%
GPT-4.1 Nano000000.0%
Arcee AI: Trinity Mini000000.0%
DeepSeek V3 (2025-03-24)000000.0%
Nemotron 3 Nano000000.0%
Gemma 3 12B000000.0%