Recall

Test: Codex Extraction

Avg. Score
95.1%
Scenarios
4

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Gemini 3.1 Flash Lite (Preview)99.7%$0.00172.0s98%
2Gemini 3.5 Flash Lite (Reasoning, Minimal)99.3%$0.00251.5s97%
3Gemini 3.1 Flash Lite (Reasoning)98.9%$0.00182.0s97%
4Gemini 3 Flash (Preview)99.7%$0.00273.9s97%
5Qwen 3.5 Plus (2026-02-15)100.0%$0.003010.6s100%
6Gemini 3.1 Flash Lite99.1%$0.00175.1s97%
7Gemini 3.6 Flash (Reasoning, Minimal)100.0%$0.0102.9s100%
8GPT-5.6 Luna99.0%$0.00433.6s96%
9Mistral Medium 3.198.5%$0.00265.8s96%
10Ministral 3 8B97.3%$0.00063.3s94%
11GPT-5.6 Luna (Reasoning)99.7%$0.00787.2s98%
12GPT-5.4 Mini97.6%$0.00312.2s94%
13Gemini 3.5 Flash (Reasoning, Minimal)99.7%$0.0103.1s98%
14Gemma 4 31B99.8%$0.000825.9s98%
15Gemini 3.5 Flash Lite (Reasoning)97.3%$0.00241.5s91%
16DeepSeek V4 Pro98.3%$0.002116.0s96%
17Z.AI GLM 4.598.4%$0.002816.8s96%
18Z.AI GLM 5.2 (Reasoning, High)100.0%$0.007121.2s100%
19Muse Spark 1.1 (Reasoning, Minimal)98.9%$0.00897.1s96%
20GPT-5.6 Terra98.5%$0.0123.1s96%
21GPT-5.4 Mini (Reasoning, Low)97.2%$0.00423.7s92%
22Ministral 3 14B96.7%$0.00096.0s90%
23Gemini 2.5 Flash Lite95.1%$0.00051.9s88%
24Laguna S 2.198.5%$0.000524.7s94%
25Mistral Large 396.5%$0.00278.2s91%
26MiniMax M2.798.2%$0.002221.7s95%
27Ministral 8B95.1%$0.00043.8s88%
28Claude Haiku 4.597.5%$0.00734.5s92%
29Gemma 3 27B96.8%$0.000514.5s91%
30Xiaomi MIMO v2.597.7%$0.003413.4s91%
31DeepSeek V4 Flash (Reasoning)99.4%$0.000937.0s97%
32GPT-4.197.0%$0.00734.7s91%
33Muse Spark 1.1 (Reasoning, Medium)99.3%$0.01313.1s97%
34Gemini 2.5 Flash Lite (Reasoning)97.0%$0.002115.6s91%
35GPT-5.6 Terra (Reasoning)99.2%$0.0167.3s96%
36MoonshotAI: Kimi K3 (Reasoning, Low)99.7%$0.01415.9s98%
37GPT-5.497.5%$0.0116.4s93%
38Grok 4.2096.2%$0.00484.9s89%
39Claude Sonnet 4.5100.0%$0.0226.6s100%
40Mistral Small 4 (Reasoning)96.1%$0.001915.6s91%
41Claude Sonnet 4100.0%$0.0227.9s100%
42Mistral Small 3.2 24B94.3%$0.00054.4s86%
43GPT-5.6 Sol100.0%$0.0235.2s100%
44Grok 4.5 (Reasoning, Low)98.5%$0.01013.1s94%
45GPT-5.4 Nano (Reasoning)95.9%$0.002312.2s89%
46Thinking Machines Inkling97.2%$0.005412.3s90%
47GPT-5.4 (Reasoning, Low)99.3%$0.01710.4s97%
48Gemini 3 Flash (Preview, Reasoning)99.1%$0.009622.2s96%
49Z.AI GLM 5 Turbo97.7%$0.006816.0s92%
50Claude Sonnet 4.699.8%$0.0227.4s99%
51Claude Sonnet 598.8%$0.0198.6s95%
52Qwen 3.6 Flash98.5%$0.009630.2s96%
53GPT-4o, Aug. 6th (temp=0)95.6%$0.01004.0s87%
54MiniMax M398.6%$0.002340.5s93%
55Mistral Large 296.3%$0.0118.3s89%
56DeepSeek V3.297.6%$0.001136.7s91%
57Grok 4.20 (Reasoning)99.8%$0.01237.2s98%
58DeepSeek V3 (2025-03-24)96.4%$0.001427.0s87%
59GPT-5.599.2%$0.0276.0s96%
60Z.AI GLM 4.698.1%$0.005738.8s93%
61DeepSeek-V2 Chat94.3%$0.001914.8s83%
62GPT-4o, Aug. 6th (temp=1)94.4%$0.00923.8s85%
63Claude Sonnet 5 (Reasoning, Low)99.0%$0.02413.0s96%
64Inception Mercury 292.5%$0.00223.5s80%
65GPT-5.4 Mini (Reasoning)99.0%$0.01725.6s95%
66o4 Mini97.7%$0.01421.5s92%
67Qwen 3.6 35B97.9%$0.007237.6s93%
68Claude Sonnet 5 (Reasoning)99.1%$0.02513.6s96%
69MiniMax M2.595.3%$0.002330.1s87%
70Gemini 2.5 Flash95.8%$0.00232.5s74%
71GPT-5.297.6%$0.01816.8s92%
72Xiaomi MIMO v2.5 Pro96.3%$0.004818.7s82%
73Qwen 2.5 72B91.5%$0.000811.0s81%
74Laguna XS 2.190.9%$0.000311.1s81%
75Z.AI GLM 599.2%$0.009151.4s97%
76Ministral 3B90.7%$0.00021.8s76%
77Mistral Small 490.7%$0.00083.2s77%
78DeepSeek V3 (2024-12-26)92.8%$0.001713.5s80%
79WizardLM 2 8x22b94.7%$0.002631.3s86%
80GPT-5.4 Nano (Reasoning, Low)90.0%$0.00113.7s78%
81Hermes 3 405B93.0%$0.004017.8s83%
82GPT-4.1 Mini90.0%$0.00156.6s78%
83GPT-5 Mini97.3%$0.007045.5s92%
84Qwen 3.5 Flash96.9%$0.003149.5s91%
85Claude Opus 4.599.3%$0.0377.7s97%
86MoonshotAI: Kimi K3 (Reasoning, High)99.7%$0.02336.0s97%
87Claude Opus 4.699.0%$0.0378.1s97%
88Qwen 3.5 35B98.4%$0.01547.3s95%
89Qwen 3 32B91.8%$0.001037.9s86%
90Gemini 3.6 Flash (Reasoning)99.8%$0.03717.3s99%
91GPT-5.4 Nano87.7%$0.00113.8s75%
92Z.AI GLM 5.1100.0%$0.0151.1m100%
93Writer: Palmyra X588.9%$0.00527.5s78%
94Nemotron 3 Super96.1%$0.00001.0m90%
95GPT-5.5 (Reasoning, Low)99.0%$0.03813.3s97%
96Gemini 3.5 Flash (Reasoning)99.8%$0.04016.4s98%
97Claude Opus 4.8 (Reasoning, Low)100.0%$0.0478.3s100%
98Claude Opus 4.7 (Reasoning)99.8%$0.0465.9s99%
99Claude Opus 4.8 (Reasoning)100.0%$0.0478.3s100%
100Grok 4.5 (Reasoning, High)99.3%$0.02445.2s97%
101Claude Opus 4.799.7%$0.0476.1s98%
102DeepSeek V4 Flash94.7%$0.00037.7s62%
103o4 Mini High98.4%$0.02540.5s95%
104Gemini 2.5 Pro98.4%$0.03422.8s95%
105Aion 2.098.5%$0.00801.2m93%
106Z.AI GLM 4.5 Air92.5%$0.002340.1s80%
107Ministral 3 3B85.2%$0.00041.8s69%
108ByteDance Seed 2.0 Lite98.6%$0.00781.4m95%
109GPT-5.6 Sol (Reasoning)99.5%$0.04517.0s97%
110Claude Opus 599.1%$0.0489.7s97%
111Aion 3.099.4%$0.02458.8s97%
112GPT-OSS 120B93.9%$0.001157.1s83%
113Claude Opus 5 (Reasoning)98.8%$0.0489.8s97%
114GPT-5 Nano97.2%$0.00431.3m91%
115Claude Opus 5 (Reasoning, Low)98.8%$0.0489.6s96%
116Grok 4.391.6%$0.00564.3s64%
117ByteDance Seed 1.697.2%$0.00731.3m91%
118Qwen 3.5 9B97.0%$0.00131.5m91%
119Qwen 3.5 Plus (2026-04-20)99.1%$0.0151.4m97%
120Arcee AI: Trinity Mini79.7%$0.00035.8s69%
121GPT-5.197.9%$0.03543.6s94%
122Gemini 2.5 Flash (Reasoning)90.4%$0.008211.8s65%
123Z.AI GLM 4.7 Flash93.2%$0.00191.2m83%
124Claude Opus 4.6 (Reasoning)99.5%$0.05521.4s98%
125Gemma 4 31B (Reasoning)99.8%$0.00162.2m99%
126Gemma 3 4B79.2%$0.00026.7s66%
127Grok 4.3 (Reasoning)97.9%$0.0181.3m93%
128Qwen3 235B A22B Instruct 250787.5%$0.000719.3s62%
129ByteDance Seed 1.6 Flash86.4%$0.001139.3s72%
130Z.AI GLM 4.798.3%$0.00981.7m93%
131Hermes 3 70B81.2%$0.001215.8s67%
132GPT-4o Mini (temp=0)76.6%$0.00057.9s68%
133GPT-5.4 (Reasoning)99.8%$0.04453.6s99%
134Qwen3.7 Max99.9%$0.0411.1m99%
135Cydonia 24B V4.187.8%$0.001012.2s54%
136Claude Sonnet 4.6 (Reasoning)99.0%$0.05236.0s96%
137DeepSeek V3.191.3%$0.001226.1s55%
138GPT-5.5 (Reasoning)98.2%$0.05625.8s95%
139Qwen 3.5 397B A17B97.6%$0.0121.8m93%
140GPT-4o Mini (temp=1)75.3%$0.00068.0s62%
141Cohere Command R+ (Aug. 2024)81.8%$0.01417.9s64%
142Llama 3.1 70B81.1%$0.001616.0s52%
143DeepSeek V4 Pro (Reasoning)98.3%$0.00932.3m93%
144GPT-599.4%$0.0491.3m98%
145Gemma 3 12B78.3%$0.000314.1s47%
146MoonshotAI: Kimi K2.598.1%$0.0132.4m93%
147Nemotron 3 Nano84.3%$0.00131.6m75%
148GPT-4.1 Nano68.6%$0.00032.8s50%
149MoonshotAI: Kimi K2.699.5%$0.0262.4m97%
150Qwen 3.6 27B92.3%$0.0181.2m61%
151Gemma 4 26B83.8%$0.000618.7s29%
152Claude Opus 4100.0%$0.11013.5s100%
153ByteDance Seed 2.0 Mini97.4%$0.00343.4m93%
154Thinking Machines Inkling (Reasoning)99.8%$0.0273.0m98%
155Qwen3.6 Max Preview99.1%$0.0452.4m94%
156Gemma 4 26B (Reasoning)94.2%$0.00232.6m56%
157Gemini 3.1 Pro (Preview)95.2%$0.0651.0m67%
158Qwen 3.5 27B89.2%$0.0211.6m40%
159Aion 3.0 Mini74.0%$0.00771.8m18%
160Qwen 3.5 122B98.0%$0.0793.6m95%
161Mistral NeMO23.9%$0.00061.4s0%
95.14%

Individual Scenarios

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 5100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
ByteDance Seed 1.6100100100100100100.0%
GPT-5.5100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
Aion 3.0100100100100100100.0%
GPT-5.6 Terra100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
Claude Opus 4100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100.0%
Mistral Large 3100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100.0%
Aion 3.0 Mini100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
Claude Haiku 4.5100100100100100100.0%
DeepSeek V3.2100100100100100100.0%
Mistral Large 2100100100100100100.0%
Grok 4.20100100100100100100.0%
Laguna S 2.1100100100100100100.0%
Mistral Medium 3.1100100100100100100.0%
Qwen3.7 Max1001001001009899.6%
GPT-5.6 Terra (Reasoning)1001001001009899.6%
GPT-5 Mini1001001001009899.6%
MoonshotAI: Kimi K2.51001001001009899.6%
Qwen 3.5 27B1001001001009899.6%
Qwen 3.5 Plus (2026-04-20)1001001001009899.6%
DeepSeek V4 Flash (Reasoning)1001001001009899.6%
Gemini 2.5 Flash Lite (Reasoning)1001001001009899.6%
Z.AI GLM 4.61001001001009899.5%
MiniMax M2.71001001001009899.5%
Qwen 3.5 397B A17B100100100989899.2%
Qwen 3.5 122B100100100989899.2%
Gemini 2.5 Pro100100100989899.2%
Z.AI GLM 4.5 Air1001001001009699.2%
Thinking Machines Inkling1001001001009699.2%
Gemini 3.1 Flash Lite (Reasoning)1001001001009599.1%
Gemini 3.1 Flash Lite100100100989899.1%
DeepSeek V3 (2025-03-24)100100100989899.1%
GPT-5.21001001001009498.8%
ByteDance Seed 2.0 Mini10010098989898.8%
Z.AI GLM 4.7100100100989698.8%
MiniMax M31001001001009398.7%
Qwen 3.5 9B100100100989598.7%
GPT-5 Nano1001001001009398.7%
GPT-5.6 Luna1001001001009398.6%
GPT-5.4 (Reasoning, Low)100100100989498.5%
GPT-5.110010098989698.5%
Aion 2.0100100100989498.4%
Z.AI GLM 5 Turbo100100100969698.3%
Qwen 3.5 Flash10010098989598.3%
Z.AI GLM 4.5100100100969598.3%
o4 Mini High100100100959598.1%
Ministral 3 8B100100100959598.1%
GPT-51009898989698.1%
GPT-5.41009898989698.1%
Gemini 2.5 Flash (Reasoning)1009898989698.1%
Qwen 3.5 35B10010096969697.7%
Qwen 3.6 Flash10010098969397.5%
Ministral 3 14B10010098969397.5%
Gemma 3 27B989898989597.5%
o4 Mini10010095959597.2%
DeepSeek V4 Pro10010095959597.2%
GPT-5.4 Mini10010095959396.8%
Ministral 3 3B1009898959396.8%
Mistral Small 4 (Reasoning)1009896949496.5%
GPT-4.1989896969496.5%
Gemini 2.5 Flash Lite1009895959396.4%
Grok 4.31009898959196.3%
Ministral 3B100100100938896.3%
GPT-5.4 Nano (Reasoning, Low)10010095939296.2%
Z.AI GLM 4.7 Flash10010095949196.1%
Mistral Small 41009895959196.0%
MiniMax M2.51009595959395.9%
Qwen 3.6 27B10010098938795.6%
Qwen 3.6 35B10010096948795.5%
GPT-5.4 Nano (Reasoning)989696949495.4%
Ministral 8B989595959194.9%
DeepSeek-V2 Chat1009893939194.9%
Nemotron 3 Super989693939394.7%
GPT-4o, Aug. 6th (temp=0)1009393939394.4%
Cydonia 24B V4.11009695918994.3%
Laguna XS 2.1989695938894.1%
WizardLM 2 8x22b10010095957994.0%
GPT-5.4 Nano1009694938693.8%
GPT-4.1 Mini1009593918893.6%
DeepSeek V3 (2024-12-26)1009593918893.5%
Mistral Small 3.2 24B959595919193.5%
Qwen3 235B A22B Instruct 2507959593938893.0%
GPT-4o, Aug. 6th (temp=1)959393919192.6%
Qwen 2.5 72B989593888892.6%
Gemma 3 12B959391918991.9%
ByteDance Seed 1.6 Flash969490888690.9%
GPT-OSS 120B939191888689.8%
Hermes 3 405B959591848489.8%
Cohere Command R+ (Aug. 2024)969191868489.6%
Writer: Palmyra X5959593917088.8%
Qwen 3 32B939186868688.5%
Gemini 2.5 Flash1001001001004087.9%
Inception Mercury 2919186848286.7%
Gemma 3 4B888686858285.5%
Llama 3.1 70B989681797084.9%
Gemini 3.1 Pro (Preview)10010098982383.9%
DeepSeek V4 Flash1001001001001482.8%
Nemotron 3 Nano848484827982.4%
GPT-4o Mini (temp=0)868479797480.5%
Arcee AI: Trinity Mini868679777280.1%
DeepSeek V3.1100100100100080.0%
Hermes 3 70B888179777279.5%
GPT-4o Mini (temp=1)817979676574.5%
GPT-4.1 Nano857775685872.8%
Gemma 4 26B100100790055.8%
Mistral NeMO000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
GPT-5.1100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Z.AI GLM 5100100100100100100.0%
Qwen 3.6 Flash100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100.0%
o4 Mini High100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Opus 4100100100100100100.0%
Qwen 3.5 35B100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Qwen 3.5 9B100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)1001001001009799.3%
MoonshotAI: Kimi K2.61001001001009799.3%
Muse Spark 1.1 (Reasoning, Minimal)1001001001009799.3%
Qwen 3.5 397B A17B1001001001009799.3%
Grok 4.20 (Reasoning)1001001001009799.3%
Thinking Machines Inkling (Reasoning)1001001001009799.3%
Qwen 3.5 27B1001001001009799.3%
Aion 3.01001001001009799.3%
o4 Mini1001001001009799.3%
Gemma 4 31B1001001001009799.3%
GPT-5.6 Luna1001001001009799.3%
Gemini 3.5 Flash Lite (Reasoning, Minimal)1001001001009799.3%
DeepSeek V4 Pro1001001001009799.3%
Inception Mercury 21001001001009799.3%
Laguna S 2.11001001001009799.3%
Gemma 3 27B1001001001009799.3%
GPT-5.5 (Reasoning, Low)1001001001009699.3%
Claude Opus 5 (Reasoning, Low)1001001001009699.3%
Qwen 3.5 Plus (2026-04-20)1001001001009699.3%
GPT-5.21001001001009699.3%
Gemini 2.5 Pro1001001001009699.3%
GPT-4.11001001001009699.3%
Gemma 4 26B1001001001009699.3%
GPT-5.41001001001009699.3%
Gemini 3.1 Pro (Preview)100100100979798.6%
Claude Sonnet 4.6 (Reasoning)100100100979798.6%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100979798.6%
GPT-5.6 Terra100100100979798.6%
Qwen 3.6 35B100100100979798.6%
Gemini 3.1 Flash Lite (Preview)100100100979798.6%
ByteDance Seed 2.0 Lite100100100979798.6%
Gemini 2.5 Flash100100100979798.6%
DeepSeek V4 Flash (Reasoning)100100100979698.6%
Claude Opus 5 (Reasoning)100100100969698.6%
GPT-5.4 (Reasoning, Low)100100100969698.6%
Claude Opus 5100100100969698.6%
GPT-5.51001001001009398.6%
Mistral Small 4 (Reasoning)100100100969698.6%
Grok 4.5 (Reasoning, High)10010097979797.9%
Claude Sonnet 5 (Reasoning)10010097979797.9%
Z.AI GLM 4.510010097979797.9%
Gemini 3.5 Flash Lite (Reasoning)10010097979797.9%
Claude Opus 4.6 (Reasoning)10010097979797.9%
Claude Opus 4.610010097979797.9%
Qwen 3.5 122B1001001001009097.9%
GPT-5.4 Mini (Reasoning)1001001001009097.9%
Z.AI GLM 4.71001001001009097.9%
GPT-5.4 Mini1001001001009097.9%
GPT-5.4 Nano (Reasoning)10010097979697.9%
GPT-5 Mini10010097969697.9%
GPT-5.6 Sol (Reasoning)10010096969697.9%
Claude Opus 4.51009797979797.2%
ByteDance Seed 2.0 Mini1009797979797.2%
Gemini 3.1 Flash Lite1009797979797.2%
Mistral Large 31009797979797.2%
Nemotron 3 Super100100100979097.2%
MoonshotAI: Kimi K2.51001001001008697.2%
Aion 2.01001001001008697.2%
Thinking Machines Inkling1009797979697.2%
Ministral 3 14B10010097969397.2%
GPT-5.5 (Reasoning)1009696969697.1%
GPT-5.6 Terra (Reasoning)10010096969397.1%
Qwen3.6 Max Preview100100100978696.6%
MiniMax M3100100100978696.6%
Z.AI GLM 4.6100100100978696.6%
Gemini 3.1 Flash Lite (Reasoning)979797979796.6%
Claude Haiku 4.5979797979796.6%
DeepSeek V4 Flash100100100978696.6%
GPT-5 Nano979797979796.6%
Grok 4.5 (Reasoning, Low)10010097978695.9%
Claude Sonnet 5 (Reasoning, Low)979797979395.9%
Mistral Large 2100100100909095.9%
GPT-4o, Aug. 6th (temp=1)100100100909095.9%
Mistral Medium 3.1979797979395.9%
Ministral 3 8B10010093939395.9%
Z.AI GLM 4.7 Flash10010093939395.8%
Gemini 2.5 Flash Lite10010097968695.8%
Qwen 3.5 Flash1009797978995.8%
Claude Sonnet 5979797939395.2%
MiniMax M2.71009793939395.2%
ByteDance Seed 1.61009797968695.1%
GPT-5.4 Nano979796939395.1%
Ministral 8B979796939395.1%
WizardLM 2 8x22b1009797909094.5%
DeepSeek V3.110010097908694.5%
DeepSeek V3.210010093909094.5%
Qwen 3.6 27B10010096908694.5%
Gemini 2.5 Flash (Reasoning)969696938994.3%
GPT-OSS 120B100100100868393.8%
GPT-4o, Aug. 6th (temp=0)1009797908693.8%
Grok 4.3 (Reasoning)979797908693.1%
Z.AI GLM 5 Turbo1009797868693.1%
DeepSeek V4 Pro (Reasoning)979797908693.1%
Xiaomi MIMO v2.5 Pro939393939393.1%
Xiaomi MIMO v2.51009797868693.1%
Mistral Small 41009393909093.1%
GPT-5.4 Nano (Reasoning, Low)979696908693.0%
Qwen 3 32B979793938392.4%
MiniMax M2.5979793868691.7%
Cydonia 24B V4.1979393938391.7%
Grok 4.2010010086868691.7%
Mistral Small 3.2 24B939090909090.3%
Z.AI GLM 4.5 Air1009790838390.3%
ByteDance Seed 1.6 Flash1009090868389.6%
Laguna XS 2.1969086868688.9%
Hermes 3 405B939390838388.3%
DeepSeek V3 (2025-03-24)1009086837987.6%
DeepSeek-V2 Chat909090907286.2%
Qwen 2.5 72B938686837985.5%
Llama 3.1 70B868686867984.8%
GPT-4.1 Mini938383837984.1%
Writer: Palmyra X5938383797983.4%
DeepSeek V3 (2024-12-26)909083767282.1%
Nemotron 3 Nano908379797681.4%
Cohere Command R+ (Aug. 2024)938676696978.5%
Hermes 3 70B838379727277.9%
Ministral 3B867976767277.9%
Gemma 3 12B96969393075.8%
Gemma 3 4B767676767275.0%
Grok 4.3909086862174.5%
Arcee AI: Trinity Mini797672727274.5%
GPT-4o Mini (temp=0)767676696973.1%
Ministral 3 3B797272726973.1%
GPT-4.1 Nano868669665973.1%
Qwen3 235B A22B Instruct 2507938383792171.7%
GPT-4o Mini (temp=1)727269696669.7%
Aion 3.0 Mini90901010040.0%
Mistral NeMO000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
Claude Opus 4100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
Claude Haiku 4.5100100100100100100.0%
DeepSeek V3 (2025-03-24)100100100100100100.0%
Mistral Small 3.2 24B100100100100100100.0%
Mistral Medium 3.1100100100100100100.0%
Gemini 3.6 Flash (Reasoning)1001001001009799.3%
Gemini 3.1 Pro (Preview)1001001001009799.3%
GPT-5.4 (Reasoning)1001001001009799.3%
Z.AI GLM 5 Turbo1001001001009799.3%
Claude Opus 4.7 (Reasoning)1001001001009799.3%
GPT-51001001001009799.3%
Grok 4.3 (Reasoning)1001001001009799.3%
Claude Sonnet 4.61001001001009799.3%
MiniMax M31001001001009799.3%
Qwen 3.5 Plus (2026-04-20)1001001001009799.3%
Gemma 4 31B (Reasoning)1001001001009799.3%
Z.AI GLM 51001001001009799.3%
GPT-5.51001001001009799.3%
Aion 3.01001001001009799.3%
DeepSeek V4 Flash (Reasoning)1001001001009799.3%
Z.AI GLM 4.71001001001009799.3%
Aion 2.01001001001009799.3%
DeepSeek V4 Flash1001001001009799.3%
Ministral 3 14B1001001001009799.3%
MoonshotAI: Kimi K3 (Reasoning, High)1001001001009398.7%
MoonshotAI: Kimi K2.61001001001009398.7%
GPT-5.5 (Reasoning, Low)100100100979798.7%
GPT-5.6 Luna (Reasoning)100100100979798.7%
Claude Sonnet 5 (Reasoning)1001001001009398.7%
Qwen 3.5 122B100100100979798.7%
Claude Opus 4.7100100100979798.7%
GPT-5.2100100100979798.7%
Gemini 3.5 Flash (Reasoning, Minimal)100100100979798.7%
Gemini 3 Flash (Preview)1001001001009398.7%
ByteDance Seed 2.0 Lite100100100979798.7%
DeepSeek-V2 Chat100100100979798.7%
Xiaomi MIMO v2.51001001001009398.7%
DeepSeek V3.21001001001009398.7%
Gemma 3 27B1001001001009398.7%
Claude Opus 4.610010097979798.0%
Qwen 3.5 27B10010097979798.0%
Claude Opus 510010097979798.0%
GPT-5.4 Mini (Reasoning)10010097979798.0%
Gemini 2.5 Pro1001001001009098.0%
MiniMax M2.7100100100979398.0%
GPT-5.6 Luna10010097979798.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100979398.0%
GPT-5.4100100100979398.0%
Nemotron 3 Super100100100979398.0%
GPT-5.5 (Reasoning)1009797979797.3%
MoonshotAI: Kimi K2.510010097979397.3%
Gemini 3 Flash (Preview, Reasoning)100100100939397.3%
Qwen 3.6 Flash1009797979797.3%
o4 Mini High1009797979797.3%
Qwen 3.6 27B10010097979397.3%
GPT-5.6 Terra1009797979797.3%
Qwen 3.6 35B100100100939397.3%
Z.AI GLM 4.610010097979397.3%
ByteDance Seed 2.0 Mini100100100939397.3%
Grok 4.31009797979797.3%
MiniMax M2.5100100100979097.3%
Z.AI GLM 4.51009797979797.3%
Qwen 3.5 35B10010097979096.7%
DeepSeek V3 (2024-12-26)100100100978796.7%
DeepSeek V4 Pro1009797979396.7%
Gemini 2.5 Flash10010097939396.7%
Laguna S 2.1100100100939096.7%
Claude Opus 5 (Reasoning)979797979796.7%
Gemma 4 26B (Reasoning)100100100939096.7%
Gemini 3.5 Flash Lite (Reasoning)100100100939096.7%
GPT-5.4 Mini1009797979396.7%
Thinking Machines Inkling10010097978796.0%
Claude Opus 5 (Reasoning, Low)979797979396.0%
GPT-5.1979797979396.0%
GPT-4.110010097939096.0%
o4 Mini979797979396.0%
Grok 4.20979797979396.0%
GPT-5 Mini1009793939395.3%
ByteDance Seed 1.61009797939095.3%
GPT-5.4 Mini (Reasoning, Low)1009793939395.3%
DeepSeek V3.11009797939095.3%
GPT-5 Nano979797939395.3%
Ministral 3 8B1009797939095.3%
Qwen3 235B A22B Instruct 2507979797939094.7%
Gemini 2.5 Flash Lite (Reasoning)1009797908794.0%
GPT-4o, Aug. 6th (temp=0)979393939394.0%
Mistral Large 2979393939394.0%
GPT-4o, Aug. 6th (temp=1)1009793909094.0%
Hermes 3 405B979797938794.0%
Gemini 2.5 Flash Lite979797938794.0%
Qwen 3 32B979793939094.0%
WizardLM 2 8x22b979797938794.0%
Mistral Small 4 (Reasoning)979793939094.0%
Qwen 3.5 Flash979793909093.3%
Mistral Large 3939393939393.3%
Z.AI GLM 4.5 Air979793909093.3%
Cydonia 24B V4.11009393909093.3%
Qwen 3.5 397B A17B979790909092.7%
Writer: Palmyra X5979793908792.7%
Laguna XS 2.1979793908792.7%
GPT-OSS 120B979393908792.0%
Qwen 3.5 9B979793908392.0%
GPT-4.1 Mini979393908792.0%
GPT-5.4 Nano (Reasoning)979393908792.0%
Xiaomi MIMO v2.5 Pro1001001001006092.0%
Ministral 8B979390908791.3%
Z.AI GLM 4.7 Flash979393908391.3%
Mistral Small 4979393878390.7%
Qwen 2.5 72B979390878790.7%
Ministral 3B979393878390.7%
Inception Mercury 21009387878089.3%
Gemma 3 12B939087878788.7%
Ministral 3 3B939087878388.0%
Gemini 2.5 Flash (Reasoning)100100100874386.0%
GPT-4o Mini (temp=1)908787808084.7%
Nemotron 3 Nano908787807784.0%
Arcee AI: Trinity Mini938783837384.0%
GPT-4o Mini (temp=0)878080808081.3%
GPT-5.4 Nano (Reasoning, Low)908380777380.7%
Gemma 4 26B100100100100080.0%
ByteDance Seed 1.6 Flash938077735776.0%
GPT-5.4 Nano838077706775.3%
Hermes 3 70B878377676074.7%
Gemma 3 4B807773736373.3%
Cohere Command R+ (Aug. 2024)937777674371.3%
Llama 3.1 70B87838070064.0%
GPT-4.1 Nano777773672062.7%
Aion 3.0 Mini10097970058.7%
Mistral NeMO7373700043.3%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100.0%
Qwen3.7 Max100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100.0%
Z.AI GLM 5.1100100100100100100.0%
Qwen3.6 Max Preview100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100.0%
Claude Opus 4.6100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100.0%
GPT-5100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100.0%
GPT-5.6 Sol100100100100100100.0%
Claude Sonnet 4.6100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100.0%
MiniMax M3100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100.0%
Claude Opus 5100100100100100100.0%
Claude Opus 4.7100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100.0%
Claude Opus 4.5100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100.0%
Qwen 3.6 35B100100100100100100.0%
Claude Sonnet 4100100100100100100.0%
Claude Sonnet 4.5100100100100100100.0%
Claude Sonnet 5100100100100100100.0%
Claude Opus 4100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100.0%
MiniMax M2.7100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100.0%
Qwen 3.5 Flash100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100.0%
Gemma 4 31B100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100.0%
GPT-5.6 Luna100100100100100100.0%
Z.AI GLM 4.5100100100100100100.0%
Gemma 4 26B100100100100100100.0%
GPT-OSS 120B100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100.0%
DeepSeek V4 Pro100100100100100100.0%
DeepSeek V4 Flash100100100100100100.0%
Hermes 3 405B100100100100100100.0%
Gemini 2.5 Flash100100100100100100.0%
Ministral 3 8B100100100100100100.0%
Grok 4.5 (Reasoning, High)1001001001009599.1%
Gemini 3.1 Pro (Preview)1001001001009599.1%
Qwen 3.5 397B A17B1001001001009599.1%
Grok 4.3 (Reasoning)1001001001009599.1%
Gemini 3 Flash (Preview, Reasoning)1001001001009599.1%
GPT-5.51001001001009599.1%
Qwen 3.6 Flash1001001001009599.1%
Aion 3.01001001001009599.1%
Z.AI GLM 4.61001001001009599.1%
Qwen 3.5 35B1001001001009599.1%
Aion 2.01001001001009599.1%
Xiaomi MIMO v2.51001001001009599.1%
DeepSeek V3 (2024-12-26)1001001001009599.1%
GPT-5.4 Mini1001001001009599.1%
DeepSeek V3 (2025-03-24)1001001001009599.1%
Ministral 8B1001001001009599.1%
GPT-5.5 (Reasoning)100100100959598.2%
GPT-5.5 (Reasoning, Low)100100100959598.2%
Grok 4.5 (Reasoning, Low)100100100959598.2%
MoonshotAI: Kimi K2.5100100100959598.2%
Qwen 3.5 Plus (2026-04-20)100100100959598.2%
ByteDance Seed 1.6100100100959598.2%
GPT-5.6 Terra100100100959598.2%
o4 Mini100100100959598.2%
GPT-5.4 Nano (Reasoning)1001001001009198.2%
Laguna S 2.1100100100959598.2%
Grok 4.3100100100959598.2%
Mistral Medium 3.1100100100959598.2%
o4 Mini High1001001001009098.1%
GPT-5 Nano1001001001009098.1%
Ministral 3B1001001001009098.1%
Muse Spark 1.1 (Reasoning, Medium)10010095959597.3%
Claude Sonnet 4.6 (Reasoning)10010095959597.3%
GPT-5.110010095959597.3%
Z.AI GLM 510010095959597.3%
Gemini 2.5 Pro10010095959597.3%
ByteDance Seed 2.0 Lite100100100959197.3%
DeepSeek-V2 Chat10010095959597.3%
Aion 3.0 Mini100100100959197.3%
Grok 4.20100100100959197.3%
Qwen 2.5 72B100100100959197.3%
Qwen 3.5 9B100100100959197.2%
DeepSeek V3.21001001001008697.2%
Z.AI GLM 4.71001001001008697.2%
Muse Spark 1.1 (Reasoning, Minimal)1009595959596.4%
GPT-5 Mini1009595959596.4%
Qwen 3.5 122B1009595959596.4%
GPT-4.11009595959596.4%
ByteDance Seed 2.0 Mini10010095959196.4%
Thinking Machines Inkling10010095959196.4%
MiniMax M2.510010095959196.3%
WizardLM 2 8x22b10010095959196.3%
Mistral Large 3959595959595.5%
DeepSeek V3.1959595959595.5%
Mistral Large 2959595959595.5%
Mistral Small 4 (Reasoning)959595959595.4%
GPT-4o, Aug. 6th (temp=1)100100100908695.2%
Gemini 3.5 Flash Lite (Reasoning)100100100868694.5%
GPT-5.4959595959194.5%
Nemotron 3 Super959595959194.5%
Inception Mercury 21009595919194.5%
Gemini 2.5 Flash Lite (Reasoning)10010095918694.5%
Gemini 2.5 Flash Lite1009595909094.3%
GPT-5.210010091918693.6%
Claude Haiku 4.5959595958693.6%
GPT-5.4 Mini (Reasoning, Low)10010091918693.5%
Mistral Small 3.2 24B1009591909093.5%
Ministral 3 14B959591919192.7%
Hermes 3 70B1009191919092.5%
Qwen 3 32B959591909092.5%
Gemma 3 27B959591918691.8%
Writer: Palmyra X5919191919190.7%
Qwen3 235B A22B Instruct 2507919191909090.6%
Llama 3.1 70B959190908690.6%
GPT-4.1 Mini10010090867690.5%
GPT-5.4 Nano (Reasoning, Low)959591868290.0%
Z.AI GLM 4.7 Flash1009586868189.7%
Nemotron 3 Nano959090868689.5%
ByteDance Seed 1.6 Flash959591867789.0%
Cohere Command R+ (Aug. 2024)959190867687.8%
Laguna XS 2.1959190867687.7%
Z.AI GLM 4.5 Air919190867787.1%
GPT-5.4 Nano959182828286.4%
Mistral Small 4959191766283.2%
Gemini 2.5 Flash (Reasoning)10010091824383.1%
Gemma 3 4B908686767683.0%
Ministral 3 3B908181818182.9%
Qwen 3.6 27B100100100951481.9%
Arcee AI: Trinity Mini908181767280.3%
Gemma 4 26B (Reasoning)100100100100080.0%
GPT-4o Mini (temp=1)767272727272.6%
Cydonia 24B V4.191919086071.7%
GPT-4o Mini (temp=0)727272727271.6%
GPT-4.1 Nano817667574865.8%
Qwen 3.5 27B1001001000060.0%
Gemma 3 12B685954544956.7%
Mistral NeMO9090810052.4%