Accuracy

Test: Codex Extraction

Avg. Score
84.1%
Scenarios
4

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Z.AI GLM 5.3 Flash (Reasoning, Low)92.1%$0.00038.6s86%
2Gemini 3 Flash (Preview)90.9%$0.00273.9s85%
3DeepSeek V4.1 Flash (Reasoning, High)92.9%$0.002517.5s88%
4Gemini 3.7 Flash (Reasoning, Medium)94.1%$0.01210.2s88%
5Z.AI GLM 5 Turbo92.2%$0.006816.0s87%
6Z.AI GLM 5.2 (Reasoning, High)93.3%$0.007121.2s87%
7Xiaomi MIMO v2.590.7%$0.003413.4s84%
8Gemini 3.6 Flash (Reasoning, Minimal)91.0%$0.0102.9s84%
9Gemini 3.5 Flash (Reasoning, Minimal)91.8%$0.0103.1s82%
10Qwen 3.5 Plus (2026-02-15)90.4%$0.003010.6s81%
11Muse Spark 1.1 (Reasoning, Medium)92.3%$0.01313.1s86%
12Gemini 3 Flash (Preview, Reasoning)93.5%$0.009622.2s86%
13Muse Spark 1.2 (Reasoning, Medium)93.8%$0.01617.4s89%
14Grok 4.5 (Reasoning, Low)91.2%$0.01013.1s85%
15DeepSeek V4 Flash87.1%$0.00037.7s80%
16Gemini 3.8 Flash (Reasoning, Medium)92.9%$0.01510.2s86%
17Kimi K3 (Reasoning, Low)92.4%$0.01415.9s88%
18Mistral Medium 3.187.8%$0.00265.8s76%
19Claude Haiku 4.586.8%$0.00734.5s80%
20Muse Spark 1.1 (Reasoning, Minimal)89.5%$0.00897.1s78%
21Mistral Large 387.8%$0.00278.2s76%
22Grok 4.2086.4%$0.00484.9s77%
23Xiaomi MIMO v2.5 Pro89.4%$0.004818.7s80%
24MiniMax M391.4%$0.002340.5s84%
25Claude Sonnet 4.691.4%$0.0227.4s86%
26Thinking Machines Inkling86.8%$0.005412.3s79%
27DeepSeek-V2 Chat86.5%$0.001914.8s76%
28DeepSeek V4 Flash (Reasoning)89.7%$0.000937.0s81%
29Claude Sonnet 590.5%$0.0198.6s82%
30Gemini 2.5 Flash82.8%$0.00232.5s75%
31Z.AI GLM 4.586.9%$0.002816.8s77%
32Mistral Large 288.0%$0.0118.3s78%
33Gemini 2.5 Flash (Reasoning)86.6%$0.008211.8s79%
34Claude Opus 4.695.1%$0.0378.1s90%
35DeepSeek V4 Pro84.7%$0.002116.0s76%
36DeepSeek V3 (2024-12-26)84.0%$0.001713.5s76%
37Grok 4.20 (Reasoning)91.5%$0.01237.2s86%
38Claude Opus 4.594.9%$0.0377.7s89%
39Claude Sonnet 5 (Reasoning, Low)91.4%$0.02413.0s85%
40Claude Sonnet 5 (Reasoning)91.2%$0.02513.6s85%
41Gemma 4 31B85.5%$0.000825.9s75%
42Qwen 3.7 Flash (Reasoning)85.4%$0.000934.5s78%
43GPT-5.6 Luna81.3%$0.00433.6s72%
44MiniMax M2.785.8%$0.002221.7s73%
45Gemini 2.5 Flash Lite (Reasoning)84.3%$0.002115.6s72%
46Gemini 3.1 Flash Lite (Reasoning)79.8%$0.00182.0s71%
47DeepSeek V3.286.4%$0.001136.7s76%
48Qwen 3.5 Flash88.5%$0.003149.5s81%
49Z.AI GLM 4.688.3%$0.005738.8s78%
50Writer: Palmyra X580.9%$0.00527.5s74%
51Z.AI GLM 591.1%$0.009151.4s83%
52Mistral Small 3.2 24B78.9%$0.00054.4s70%
53Inception Mercury 280.2%$0.00223.5s70%
54Muse Spark 1.3 (Reasoning, Medium)92.6%$0.01949.7s88%
55 Kimi K3 (Reasoning, High)92.5%$0.02336.0s86%
56GPT-5.6 Terra84.1%$0.0123.1s71%
57Ministral 3 14B80.7%$0.00096.0s68%
58GPT-5.4 Mini (Reasoning)88.3%$0.01725.6s80%
59Gemini 3.1 Flash Lite (Preview)78.5%$0.00172.0s69%
60Grok 4.5 (Reasoning, High)93.8%$0.02445.2s88%
61Laguna S 2.183.0%$0.000524.7s72%
62MiniMax M2.584.9%$0.002330.1s73%
63Claude Sonnet 4.587.9%$0.0226.6s75%
64GPT-5.6 Terra (Reasoning)85.3%$0.0167.3s73%
65Ministral 3 8B79.7%$0.00063.3s65%
66Nemotron 3 Super89.5%$0.00001.0m78%
67Aion 2.092.6%$0.00801.2m86%
68DeepSeek V3 (2025-03-24)84.7%$0.001427.0s70%
69GPT-5 Mini86.9%$0.007045.5s80%
70GPT-5.6 Sol86.3%$0.0235.2s77%
71Qwen 3 235B A22B Instruct 250780.2%$0.000719.3s71%
72Grok 4.383.5%$0.00564.3s64%
73Hermes 3 405B83.4%$0.004017.8s69%
74Gemini 2.5 Pro92.4%$0.03422.8s85%
75Ministral 8B77.5%$0.00043.8s67%
76Gemini 3.1 Flash Lite78.0%$0.00175.1s68%
77GPT-5.6 Luna (Reasoning)80.5%$0.00787.2s70%
78GPT-4o, Aug. 6th (temp=0)77.9%$0.01004.0s74%
79Claude Sonnet 486.7%$0.0227.9s74%
80Gemini 3.5 Flash (Reasoning)93.4%$0.04016.4s85%
81Gemini 2.5 Flash Lite77.6%$0.00051.9s65%
82GPT-5.4 Nano (Reasoning)79.8%$0.002312.2s68%
83Gemini 3.6 Flash (Reasoning)91.7%$0.03717.3s83%
84Z.AI GLM 5.192.6%$0.0151.1m84%
85Claude Opus 4.8 (Reasoning)93.8%$0.0478.3s84%
86GPT-5.4 (Reasoning, Low)83.1%$0.01710.4s73%
87Qwen 2.5 72B77.2%$0.000811.0s67%
88GPT-5.585.8%$0.0276.0s76%
89Claude Opus 4.8 (Reasoning, Low)93.7%$0.0478.3s84%
90WizardLM 2 8x22b82.5%$0.002631.3s70%
91GPT-5.4 Mini (Reasoning, Low)76.5%$0.00423.7s67%
92Qwen 3.5 35B87.5%$0.01547.3s80%
93Qwen 3.6 Flash81.8%$0.009630.2s75%
94GPT-5.4 Mini75.4%$0.00312.2s65%
95GPT-5.479.7%$0.0116.4s67%
96GPT-5.283.0%$0.01816.8s74%
97o4 Mini83.8%$0.01421.5s72%
98o4 Mini High89.0%$0.02540.5s81%
99Aion 3.092.2%$0.02458.8s84%
100Claude Opus 5 (Reasoning)92.1%$0.0489.8s82%
101GPT-5.192.1%$0.03543.6s86%
102GPT-4.175.6%$0.00734.7s66%
103Claude Opus 5 (Reasoning, Low)92.0%$0.0489.6s82%
104Claude Opus 4.6 (Reasoning)94.5%$0.05521.4s89%
105Qwen 3.6 35B81.3%$0.007237.6s72%
106GPT-4o, Aug. 6th (temp=1)74.4%$0.00923.8s66%
107Claude Opus 4.789.5%$0.0476.1s80%
108Laguna XS 2.173.0%$0.000311.1s63%
109GPT-5.6 Sol (Reasoning)89.9%$0.04517.0s82%
110Claude Opus 4.7 (Reasoning)89.1%$0.0465.9s79%
111DeepSeek V3.185.4%$0.001226.1s54%
112GPT-4.1 Mini73.5%$0.00156.6s60%
113Claude Opus 591.2%$0.0489.7s79%
114Z.AI GLM 4.5 Air79.4%$0.002340.1s67%
115Gemma 3 4B70.1%$0.00026.7s63%
116Hermes 3 70B76.6%$0.001215.8s59%
117GPT-OSS 120B81.1%$0.001157.1s71%
118Z.AI GLM 5.3 Flash (Reasoning, Max)93.9%$0.00152.3m90%
119Grok 4.3 (Reasoning)89.5%$0.0181.3m85%
120GPT-5.5 (Reasoning, Low)86.1%$0.03813.3s76%
121Arcee AI: Trinity Mini69.4%$0.00035.8s61%
122Qwen 3 32B75.9%$0.001037.9s68%
123Mistral Small 469.0%$0.00083.2s60%
124Thinking Machines Inkling Small (Reasoning, High)85.8%$0.00741.1m70%
125ByteDance Seed 2.0 Lite87.1%$0.00781.4m77%
126Gemini 3.5 Flash Lite (Reasoning, Minimal)73.8%$0.00251.5s53%
127Claude Sonnet 4.6 (Reasoning)91.8%$0.05236.0s86%
128Z.AI GLM 4.790.0%$0.00981.7m81%
129Mistral Small 4 (Reasoning)72.6%$0.001915.6s59%
130Qwen 3.5 Plus (2026-04-20)88.0%$0.0151.4m79%
131Ministral 3B65.4%$0.00021.8s59%
132Qwen 3.8 Flash (Reasoning)88.9%$0.00451.9m80%
133Qwen 3.5 397B A17B91.3%$0.0121.8m82%
134Cydonia 24B V4.174.6%$0.001012.2s50%
135Gemma 3 27B67.8%$0.000514.5s58%
136Ministral 3 3B63.6%$0.00041.8s57%
137GPT-4o Mini (temp=0)66.1%$0.00057.9s57%
138GPT-4o Mini (temp=1)66.7%$0.00068.0s56%
139GPT-5.4 Nano64.7%$0.00113.8s56%
140DeepSeek V4 Pro 0813 (Reasoning, High)91.2%$0.0281.6m86%
141ByteDance Seed 1.6 Flash72.1%$0.001139.3s61%
142Llama 3.1 70B71.9%$0.001616.0s51%
143ByteDance Seed 1.681.1%$0.00731.3m70%
144GPT-5.4 Nano (Reasoning, Low)65.0%$0.00113.7s53%
145Nemotron 3 Nano82.0%$0.00131.6m72%
146GPT-594.0%$0.0491.3m89%
147Qwen 3.6 27B83.5%$0.0181.2m72%
148Qwen 3.5 9B82.0%$0.00131.5m68%
149Gemma 4 31B (Reasoning)87.6%$0.00162.2m78%
150Gemini 3.5 Flash Lite (Reasoning)69.5%$0.00241.5s45%
151Qwen 3.7 Max89.0%$0.0411.1m79%
152GPT-5.5 (Reasoning)86.8%$0.05625.8s78%
153Grok 4.6 (Reasoning, High)93.8%$0.0411.7m90%
154DeepSeek V4 Pro (Reasoning)90.0%$0.00932.3m82%
155GPT-5.4 (Reasoning)87.0%$0.04453.6s77%
156DeepSeek V4 Flash 0731 (Reasoning, High)91.6%$0.00232.8m84%
157Muse Glimmer 30B (Reasoning, Medium)75.7%$0.002521.5s39%
158Gemma 3 12B63.0%$0.000314.1s46%
159Kimi K2.588.9%$0.0132.4m81%
160Cohere Command R+ (Aug. 2024)66.1%$0.01417.9s51%
161GPT-5 Nano71.8%$0.00431.3m59%
162Gemma 4 26B68.1%$0.000618.7s33%
163Z.AI GLM 5.3 (Reasoning, Max)96.2%$0.0362.8m94%
164Kimi K2.687.9%$0.0262.4m83%
165Gemini 3.1 Pro (Preview)88.3%$0.0651.0m78%
166Z.AI GLM 4.7 Flash69.1%$0.00191.2m52%
167GPT-4.1 Nano51.8%$0.00032.8s42%
168Claude Opus 491.4%$0.11013.5s84%
169Qwen 3.8 27B (Reasoning, XHigh)90.0%$0.0252.9m83%
170Qwen 3.6 Max Preview90.7%$0.0452.4m84%
171ByteDance Seed 2.0 Mini84.4%$0.00343.4m77%
172Hy4 Preview (Reasoning, High)94.1%$0.0343.4m89%
173Thinking Machines Inkling (Reasoning)88.4%$0.0273.0m78%
174Gemma 4 26B (Reasoning)81.7%$0.00232.6m52%
175Qwen 3.5 27B81.4%$0.0211.6m41%
176Qwen 3.8 Max (Reasoning, XHigh)93.3%$0.0513.4m88%
177Aion 3.0 Mini68.2%$0.00771.8m18%
178Qwen 3.5 122B90.4%$0.0793.6m83%
179Mistral NeMO22.4%$0.00061.4s0%
84.11%

Individual Scenarios

Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 4.8 (Reasoning)989797979696.8%
Claude Opus 4.8 (Reasoning, Low)979696969395.6%
Muse Spark 1.2 (Reasoning, Medium)979696959395.5%
Z.AI GLM 5.3 (Reasoning, Max)979696949395.2%
Gemini 3.5 Flash (Reasoning)989494949394.7%
Kimi K3 (Reasoning, High)989795948994.6%
Gemini 3 Flash (Preview, Reasoning)979695939294.3%
Gemini 3.8 Flash (Reasoning, Medium)959594949394.3%
Aion 2.0989493929093.6%
Gemini 3.7 Flash (Reasoning, Medium)959494929293.6%
Muse Spark 1.3 (Reasoning, Medium)959393939393.4%
Z.AI GLM 5.3 Flash (Reasoning, Low)969595948793.3%
Claude Opus 4.5959393939393.1%
Z.AI GLM 5.2 (Reasoning, High)999692928592.9%
Gemini 3.6 Flash (Reasoning)979392929092.8%
Z.AI GLM 5.3 Flash (Reasoning, Max)959393929092.7%
Muse Spark 1.1 (Reasoning, Medium)949392919192.3%
Claude Opus 4.6939392929292.2%
Aion 3.0 Mini979392928692.1%
Claude Opus 5 (Reasoning, Low)939292929292.0%
Claude Opus 5 (Reasoning)939292929292.0%
Hy4 Preview (Reasoning, High)979391908992.0%
Grok 4.6 (Reasoning, High)949492909091.9%
Claude Opus 5929292929291.8%
Claude Opus 4929292929191.7%
Qwen 3.6 Max Preview969292908991.7%
Claude Opus 4.6 (Reasoning)969191918991.5%
Qwen 3.8 Max (Reasoning, XHigh)959392908891.5%
Nemotron 3 Super969291898991.4%
Gemini 3 Flash (Preview)949391908991.3%
o4 Mini High949490898891.2%
GPT-5939391908891.1%
GPT-5.4 (Reasoning)949290908990.9%
Z.AI GLM 5939391888890.5%
Grok 4.5 (Reasoning, High)959288888790.2%
DeepSeek V4 Flash 0731 (Reasoning, High)949390908490.2%
Claude Opus 4.7919190898889.9%
Z.AI GLM 5.1939290898589.8%
DeepSeek V4.1 Flash (Reasoning, High)949089888889.8%
Kimi K3 (Reasoning, Low)929289898689.8%
Claude Sonnet 4.6 (Reasoning)929089898889.7%
Qwen 3.7 Max959291908089.7%
Claude Opus 4.7 (Reasoning)919090898989.7%
Grok 4.3 (Reasoning)959290858489.5%
Gemini 2.5 Pro929189888689.4%
Xiaomi MIMO v2.5929189888789.3%
Gemini 3.5 Flash (Reasoning, Minimal)929189888589.2%
Gemini 3.6 Flash (Reasoning, Minimal)919089888889.2%
Gemini 3.1 Pro (Preview)929190898288.9%
Aion 3.0959087868588.6%
Grok 4.20 (Reasoning)948988878588.5%
DeepSeek V4 Pro 0813 (Reasoning, High)928887878688.1%
Z.AI GLM 5 Turbo898988878787.9%
Qwen 3.5 27B928988868487.8%
DeepSeek V4 Flash (Reasoning)898988888587.8%
Claude Sonnet 5 (Reasoning)929086858487.6%
GPT-5.4 Mini (Reasoning)898888888687.6%
Qwen 3.5 Plus (2026-02-15)898888888687.4%
GPT-5.1898887878587.3%
Gemma 4 26B (Reasoning)919188848287.3%
Claude Sonnet 4.6908987858587.2%
GPT-5.5 (Reasoning)898886868686.9%
Claude Sonnet 5918988878086.9%
DeepSeek-V2 Chat919087868186.9%
DeepSeek V4 Pro (Reasoning)928886858286.8%
Mistral Medium 3.1878787868586.7%
Qwen 3.5 122B949187818086.7%
Qwen 3.5 Plus (2026-04-20)928888877986.7%
Qwen 3.5 35B929285838286.7%
Gemini 2.5 Flash (Reasoning)908987858386.6%
Qwen 3.8 27B (Reasoning, XHigh)918785858586.6%
GPT-5.5888886868686.4%
Grok 4.5 (Reasoning, Low)878786868586.3%
Xiaomi MIMO v2.5 Pro928886848286.3%
DeepSeek V4 Flash929191877086.1%
GPT-5.4 (Reasoning, Low)888787878186.1%
Qwen 3.8 Flash (Reasoning)898786858486.0%
Kimi K2.5918887867886.0%
Muse Spark 1.1 (Reasoning, Minimal)938887847785.9%
Nemotron 3 Nano949187847385.9%
Z.AI GLM 4.7919185838085.8%
GPT-5.6 Terra (Reasoning)908784848485.8%
Claude Sonnet 5 (Reasoning, Low)888786858385.7%
GPT-5.5 (Reasoning, Low)878685858585.6%
Grok 4.20898685848385.5%
Kimi K2.6898784848385.5%
Qwen 3.5 397B A17B898685848385.4%
Z.AI GLM 4.5888685858285.4%
GPT-5.6 Sol878686858485.3%
Qwen 3.5 Flash898986828085.3%
GPT-5.2888785848185.1%
Thinking Machines Inkling878585848485.1%
Gemini 2.5 Flash888885838285.1%
MiniMax M3928584848085.0%
GPT-5.6 Luna878585848484.8%
GPT-5.6 Sol (Reasoning)878585848384.8%
Claude Haiku 4.5888584848384.8%
GPT-5.6 Luna (Reasoning)878685828284.2%
Gemini 2.5 Flash Lite (Reasoning)928881807984.2%
Mistral Large 2868685847984.1%
o4 Mini938987826884.1%
Z.AI GLM 4.6878786818084.0%
Thinking Machines Inkling (Reasoning)868584838284.0%
Inception Mercury 2898785817783.9%
Gemma 4 31B868484848383.9%
GPT-5 Mini898383838083.7%
Claude Sonnet 4918885787683.6%
DeepSeek V3.2898483817983.2%
Grok 4.3908682807783.0%
Mistral Large 3878580808082.7%
GPT-5.4888382827982.6%
Hermes 3 405B888583807782.6%
Qwen 3.7 Flash (Reasoning)908685767582.6%
Qwen 3.6 27B898683817382.3%
GPT-5.6 Terra838383818182.3%
GPT-OSS 120B958580767582.0%
GPT-5.4 Mini (Reasoning, Low)868382807981.8%
WizardLM 2 8x22b888584807181.6%
Gemma 4 31B (Reasoning)838282827881.5%
Mistral Small 3.2 24B858483817481.3%
GPT-5.4 Nano (Reasoning)888481767580.8%
Claude Sonnet 4.5828181807880.5%
Qwen 3.6 Flash868280787680.4%
ByteDance Seed 2.0 Lite858483777380.2%
MiniMax M2.7848280787580.0%
DeepSeek V3 (2024-12-26)848278777779.8%
Qwen 3.6 35B848180797579.7%
GPT-4o, Aug. 6th (temp=0)818181817679.6%
Hermes 3 70B858382796879.2%
ByteDance Seed 2.0 Mini838278787579.1%
DeepSeek V4 Pro878377757078.6%
Gemini 3.1 Flash Lite (Reasoning)818080757578.4%
Thinking Machines Inkling Small (Reasoning, High)878685825178.2%
Laguna S 2.1937674747278.0%
Qwen 3 235B A22B Instruct 2507787878777677.4%
Writer: Palmyra X5847776757577.3%
Qwen 3.5 9B868583805277.2%
MiniMax M2.5848280756577.1%
Ministral 3 14B797777767476.7%
ByteDance Seed 1.6837774747276.0%
Qwen 3 32B828279696976.0%
Gemini 3.1 Flash Lite (Preview)787877747476.0%
DeepSeek V3 (2025-03-24)838374706775.3%
Gemini 3.1 Flash Lite827874727075.2%
Llama 3.1 70B797874737074.6%
Cydonia 24B V4.1858174696574.6%
GPT-5.4 Mini807773717174.3%
GPT-4.1 Mini797777706874.1%
GPT-4.1767474746973.3%
GPT-4o, Aug. 6th (temp=1)777674716973.3%
Z.AI GLM 4.5 Air787876686773.3%
Gemini 2.5 Flash Lite787670696571.7%
DeepSeek V3.193938684071.3%
GPT-4o Mini (temp=1)757474706371.2%
Mistral Small 4 (Reasoning)797870696071.0%
Ministral 8B737272696871.0%
Ministral 3 8B737371696870.8%
GPT-5 Nano797877724770.3%
Mistral Small 4767271706170.1%
GPT-4o Mini (temp=0)757270696470.1%
Qwen 2.5 72B767272666570.0%
Cohere Command R+ (Aug. 2024)857267646269.9%
Gemma 3 4B737271676569.6%
Arcee AI: Trinity Mini757169676569.3%
ByteDance Seed 1.6 Flash787770634567.0%
GPT-5.4 Nano (Reasoning, Low)757064625765.8%
GPT-5.4 Nano737170694465.4%
Laguna XS 2.1756965625064.3%
Ministral 3B696563625963.7%
Gemma 3 12B686767645363.7%
Muse Glimmer 30B (Reasoning, Medium)85787877063.6%
Ministral 3 3B666361616162.2%
Z.AI GLM 4.7 Flash806160594260.5%
Gemma 3 27B686758584759.4%
Gemini 3.5 Flash Lite (Reasoning, Minimal)935249474657.5%
Gemini 3.5 Flash Lite (Reasoning)525048474748.8%
Gemma 4 26B8484750048.8%
GPT-4.1 Nano584846393244.6%
Mistral NeMO000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 4.6999999999998.6%
Claude Opus 4.8 (Reasoning)999999979597.6%
Claude Opus 4.8 (Reasoning, Low)999999969697.6%
Claude Opus 4.6 (Reasoning)999999979397.3%
Claude Opus 4.5999996969697.1%
Claude Opus 5 (Reasoning, Low)969696969696.4%
Claude Opus 5 (Reasoning)969696969696.4%
Z.AI GLM 5.3 (Reasoning, Max)979797959496.1%
Claude Opus 5969696969495.9%
Claude Opus 4.71009695929295.1%
Z.AI GLM 5.3 Flash (Reasoning, Max)979695949395.0%
GPT-5969595959594.9%
Claude Opus 4.7 (Reasoning)969595959294.5%
Kimi K3 (Reasoning, Low)969594949394.4%
Hy4 Preview (Reasoning, High)969694939294.3%
DeepSeek V4.1 Flash (Reasoning, High)999494939294.2%
Qwen 3.8 Max (Reasoning, XHigh)969695939194.0%
Claude Sonnet 5979494939093.5%
GPT-5.1969595919193.5%
Grok 4.5 (Reasoning, High)979393929193.3%
Gemini 3.1 Pro (Preview)969695918993.3%
MiniMax M3979692919093.2%
Grok 4.6 (Reasoning, High)959492929192.8%
Gemini 3.7 Flash (Reasoning, Medium)969292929292.6%
GPT-5.6 Sol (Reasoning)949493918992.3%
Z.AI GLM 5 Turbo969292919092.1%
Claude Sonnet 4.6929292929192.1%
Grok 4.20 (Reasoning)949392919092.0%
Kimi K3 (Reasoning, High)949492928991.9%
Gemini 3.6 Flash (Reasoning)939392918991.7%
Gemini 3 Flash (Preview, Reasoning)969392918691.7%
Gemini 3.8 Flash (Reasoning, Medium)929292929291.7%
Claude Sonnet 5 (Reasoning, Low)939292929091.6%
Aion 2.0959292908991.5%
Claude Sonnet 5 (Reasoning)979492898491.4%
Z.AI GLM 4.7989490888691.3%
Qwen 3.5 Flash969392898691.3%
Claude Sonnet 4.6 (Reasoning)929291919091.2%
GPT-5.6 Sol939291908990.9%
Gemini 3.5 Flash Lite (Reasoning)949291908790.9%
Mistral Large 2949191908890.9%
Gemini 3.5 Flash (Reasoning)979391878590.8%
Mistral Large 3929292928690.8%
Grok 4.5 (Reasoning, Low)929291908890.7%
Qwen 3.8 27B (Reasoning, XHigh)979290888790.6%
Qwen 3.5 27B959490898590.6%
GPT-5.4 (Reasoning)929190908990.4%
Qwen 3.5 122B939191908790.3%
Qwen 3.8 Flash (Reasoning)939291908590.3%
Xiaomi MIMO v2.5939291918590.1%
DeepSeek V4 Flash 0731 (Reasoning, High)979493848290.1%
Gemini 2.5 Pro939290908590.1%
Z.AI GLM 4.6969492848389.8%
Z.AI GLM 5.3 Flash (Reasoning, Low)939389878789.7%
DeepSeek V4 Pro 0813 (Reasoning, High)959190898489.6%
Qwen 3.5 397B A17B969291878289.5%
DeepSeek V3.1909089898989.4%
DeepSeek V4 Flash (Reasoning)939190878789.3%
Gemma 4 31B969387868589.2%
GPT-5.5 (Reasoning, Low)929188888789.2%
DeepSeek V4 Pro (Reasoning)928989888789.1%
Muse Spark 1.2 (Reasoning, Medium)909089888889.1%
Claude Opus 4959190868489.0%
Muse Spark 1.3 (Reasoning, Medium)938989888789.0%
GPT-5.6 Terra929188878788.8%
GPT-5.5 (Reasoning)929291858388.7%
Gemini 3.5 Flash Lite (Reasoning, Minimal)909088888788.7%
Z.AI GLM 5.2 (Reasoning, High)898988888888.6%
Qwen 3.7 Max898989888688.5%
Grok 4.3 (Reasoning)919090898288.5%
DeepSeek V4 Flash949289858288.4%
DeepSeek-V2 Chat919191888188.4%
DeepSeek V3.2919189878488.3%
Thinking Machines Inkling (Reasoning)958988868388.1%
Qwen 3.6 Max Preview939289867988.1%
Gemini 3.1 Flash Lite (Reasoning)918989888388.0%
GPT-5 Mini929188858487.9%
Xiaomi MIMO v2.5 Pro939385858487.9%
Aion 3.0938987868587.9%
Kimi K2.6918989888087.7%
Gemma 4 31B (Reasoning)958985858487.7%
Gemini 3.6 Flash (Reasoning, Minimal)948686868687.7%
ByteDance Seed 2.0 Mini898988888387.5%
Qwen 3.5 Plus (2026-02-15)919088858487.5%
Thinking Machines Inkling908987868487.4%
Gemini 3 Flash (Preview)918986868587.3%
Gemini 3.5 Flash (Reasoning, Minimal)909089887987.2%
Qwen 3.5 35B918887868587.2%
Muse Spark 1.1 (Reasoning, Medium)908988868287.2%
Z.AI GLM 5.1918988848387.0%
ByteDance Seed 2.0 Lite918988838386.8%
DeepSeek V3 (2025-03-24)888787868586.6%
GPT-5.5918787868286.5%
GPT-5.6 Terra (Reasoning)908986848486.4%
Qwen 3.5 Plus (2026-04-20)888786868486.2%
Thinking Machines Inkling Small (Reasoning, High)908984848486.2%
MiniMax M2.7898787848385.9%
Kimi K2.5898989837985.8%
Gemini 3.1 Flash Lite898886848185.7%
Gemini 2.5 Flash (Reasoning)898785858285.6%
DeepSeek V3 (2024-12-26)918988827885.6%
Qwen 3.7 Flash (Reasoning)918785857985.5%
Grok 4.20878685858585.5%
MiniMax M2.5898786848085.3%
GPT-5.4 Mini (Reasoning)908888818085.3%
o4 Mini High908887827985.3%
Claude Haiku 4.5868684848485.2%
Claude Sonnet 4.5868685858385.1%
Qwen 3.5 9B938888867085.1%
GPT-OSS 120B878785848284.9%
Muse Spark 1.1 (Reasoning, Minimal)898585857984.8%
Gemini 3.1 Flash Lite (Preview)888786848084.8%
Z.AI GLM 4.5938583827984.5%
Gemini 2.5 Flash Lite (Reasoning)908884827684.0%
Z.AI GLM 5878584828183.7%
GPT-5.2898582818083.6%
WizardLM 2 8x22b888883817683.2%
GPT-5.4858484818183.1%
Nemotron 3 Super868682818083.0%
GPT-5.4 (Reasoning, Low)858584837882.9%
GPT-5 Nano888682817882.9%
Nemotron 3 Nano908581817782.7%
Z.AI GLM 4.5 Air928680797582.5%
DeepSeek V4 Pro868583797982.2%
Qwen 3.6 35B868483827682.2%
Mistral Small 3.2 24B848484817782.2%
ByteDance Seed 1.6868381807982.0%
Mistral Medium 3.1848482808081.9%
Writer: Palmyra X5858482797881.7%
Ministral 3 8B828282827981.6%
Cydonia 24B V4.1898583816981.3%
Claude Sonnet 4868181807881.2%
Ministral 3 14B848282817781.1%
Qwen 3.6 Flash848481797881.1%
Gemini 2.5 Flash848482797680.8%
Laguna S 2.1838380797880.5%
Qwen 2.5 72B828281807680.3%
Qwen 3 235B A22B Instruct 2507908482806480.2%
Ministral 8B828181797780.1%
GPT-5.6 Luna (Reasoning)848482797180.0%
GPT-4o, Aug. 6th (temp=0)838080807780.0%
Qwen 3.6 27B888583786680.0%
o4 Mini878583757079.9%
GPT-5.6 Luna828280797779.9%
Inception Mercury 2917977767579.6%
Gemini 2.5 Flash Lite878281806779.3%
GPT-4.1 Mini867979777679.2%
Gemma 4 26B (Reasoning)898986784978.1%
GPT-4o, Aug. 6th (temp=1)848280796578.0%
Hermes 3 405B827977777478.0%
GPT-4.1818080767277.9%
Gemma 4 26B787877777777.7%
GPT-5.4 Mini (Reasoning, Low)828076767377.7%
Llama 3.1 70B808080796677.0%
GPT-5.4 Mini797979757076.6%
Qwen 3 32B857976756876.6%
Hermes 3 70B817774747275.4%
Z.AI GLM 4.7 Flash828170686673.4%
Mistral Small 4 (Reasoning)777673716873.1%
Laguna XS 2.1827574686572.8%
GPT-5.4 Nano (Reasoning)838272636372.5%
GPT-4o Mini (temp=1)837669676571.8%
Grok 4.3868379733571.2%
ByteDance Seed 1.6 Flash827867676271.1%
Gemma 3 4B737272706871.1%
Arcee AI: Trinity Mini787469686169.8%
Mistral Small 4817068685768.8%
GPT-4o Mini (temp=0)737067626066.3%
Cohere Command R+ (Aug. 2024)726665635864.8%
Muse Glimmer 30B (Reasoning, Medium)82817974063.1%
GPT-5.4 Nano726461615863.1%
Ministral 3B666363626062.8%
Gemma 3 27B727062575362.7%
Ministral 3 3B636362616161.6%
GPT-5.4 Nano (Reasoning, Low)716664554860.9%
GPT-4.1 Nano665756545357.3%
Gemma 3 12B71716765054.9%
Aion 3.0 Mini928500035.3%
Mistral NeMO000000.0%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Claude Opus 5999998989898.3%
Gemini 3.5 Flash (Reasoning, Minimal)999998969597.7%
Claude Opus 5 (Reasoning, Low)989898989897.7%
Claude Opus 5 (Reasoning)989898989897.7%
Z.AI GLM 5.3 (Reasoning, Max)989898979697.2%
Grok 4.5 (Reasoning, High)989898939295.8%
Z.AI GLM 5.1989797949395.7%
Gemini 3.5 Flash (Reasoning)989796939295.0%
Z.AI GLM 5.2 (Reasoning, High)979595949395.0%
Claude Opus 4.6 (Reasoning)969595959594.9%
Gemini 3.6 Flash (Reasoning, Minimal)969595949494.7%
Claude Opus 4.8 (Reasoning, Low)969595949294.6%
Z.AI GLM 5 Turbo969594949394.5%
Grok 4.6 (Reasoning, High)969694939294.3%
Claude Opus 4.6959595959394.3%
Gemini 3.1 Pro (Preview)969594949294.3%
Gemini 3 Flash (Preview, Reasoning)989594939294.3%
Aion 3.0979695938994.3%
Claude Sonnet 5959594939394.3%
Z.AI GLM 5.3 Flash (Reasoning, Max)969595939294.2%
Muse Spark 1.2 (Reasoning, Medium)969595949094.1%
Claude Opus 4.7 (Reasoning)959595939294.1%
Hy4 Preview (Reasoning, High)989694929094.0%
Grok 4.5 (Reasoning, Low)979593929293.9%
Claude Opus 4.8 (Reasoning)959494939393.9%
Z.AI GLM 5969595929193.9%
Muse Spark 1.1 (Reasoning, Medium)959493939393.8%
GPT-5.6 Terra (Reasoning)959595949093.8%
Gemini 3.6 Flash (Reasoning)989593919193.6%
Muse Spark 1.3 (Reasoning, Medium)979593929093.6%
Gemini 3 Flash (Preview)959593939193.6%
Claude Opus 4.7959493939293.4%
Claude Sonnet 5 (Reasoning, Low)969493929193.3%
Gemini 3.7 Flash (Reasoning, Medium)1009695918593.2%
GPT-5.5959593939093.2%
GPT-5.6 Terra949493939293.1%
DeepSeek V4 Pro 0813 (Reasoning, High)949493939093.0%
Qwen 3.5 397B A17B959593928992.9%
Claude Opus 4.5949494948892.9%
Gemini 3.8 Flash (Reasoning, Medium)969393938992.7%
GPT-5949393929292.7%
GPT-5.5 (Reasoning, Low)969392919192.7%
Qwen 3.7 Max979494918792.6%
DeepSeek V4 Pro (Reasoning)979393918892.4%
DeepSeek V4.1 Flash (Reasoning, High)979592908892.4%
Gemini 2.5 Pro949393919092.2%
Qwen 3.8 Max (Reasoning, XHigh)939392929292.2%
MiniMax M3939392929092.1%
GPT-5.6 Sol949292929092.1%
GPT-5.1959292928892.0%
Kimi K3 (Reasoning, High)959492908891.8%
GPT-5.4 (Reasoning)969292908991.8%
Gemma 4 31B949292919091.7%
Kimi K3 (Reasoning, Low)949391908991.7%
Claude Sonnet 5 (Reasoning)949392908991.4%
Xiaomi MIMO v2.5 Pro979392898791.4%
Claude Sonnet 4.6939191919191.4%
GPT-5.5 (Reasoning)939392908991.4%
GPT-5.6 Sol (Reasoning)949290909091.3%
Qwen 3.8 Flash (Reasoning)939391908991.2%
Kimi K2.5959390898991.2%
GPT-5.4 (Reasoning, Low)959190908990.9%
Grok 4.20 (Reasoning)939292918690.8%
Claude Sonnet 4.6 (Reasoning)929191909090.7%
GPT-5 Mini939090908990.6%
DeepSeek V4 Flash 0731 (Reasoning, High)969290878790.5%
Muse Spark 1.1 (Reasoning, Minimal)949390888890.4%
Aion 2.0929191918690.3%
Qwen 3.5 122B939290908689.9%
Claude Haiku 4.5929090908889.7%
Gemma 4 26B (Reasoning)939191898489.5%
Qwen 3.5 27B929190898689.5%
DeepSeek V4 Pro939089888789.4%
Qwen 3.8 27B (Reasoning, XHigh)959089888589.4%
Grok 4.3 (Reasoning)919190888689.4%
Gemma 4 31B (Reasoning)919089898789.3%
Z.AI GLM 4.6949290878389.3%
Z.AI GLM 5.3 Flash (Reasoning, Low)939190878689.3%
ByteDance Seed 2.0 Lite939290868589.2%
o4 Mini High929088888789.0%
Qwen 3.5 Plus (2026-02-15)918989888788.9%
Z.AI GLM 4.7959189858488.8%
Qwen 3.6 Max Preview949089878488.8%
GPT-5.4928888888688.5%
Claude Sonnet 4.5939087878588.4%
Claude Opus 4929088868688.4%
GPT-5.6 Luna928989888388.2%
Xiaomi MIMO v2.5939189858187.9%
GPT-5.2929288878087.8%
Gemini 2.5 Flash908989888387.7%
Grok 4.3908987868587.6%
Qwen 3.7 Flash (Reasoning)888887878787.5%
GPT-5.6 Luna (Reasoning)919087868387.4%
Kimi K2.6908887858587.1%
DeepSeek V3.1898888858487.0%
GPT-5.4 Mini (Reasoning)908787878487.0%
MiniMax M2.5928786868587.0%
Thinking Machines Inkling (Reasoning)919186838286.6%
Gemini 3.5 Flash Lite (Reasoning)908888878186.6%
Grok 4.20928985838386.5%
o4 Mini908888838286.2%
Claude Sonnet 4918686858386.2%
Nemotron 3 Super918984848386.1%
Qwen 3.5 Flash938787828185.8%
Gemini 2.5 Flash (Reasoning)898884848385.7%
DeepSeek V4 Flash (Reasoning)888885848385.7%
Writer: Palmyra X5898987818085.4%
Qwen 3.5 Plus (2026-04-20)938685847885.3%
Qwen 3.5 35B878784848385.1%
DeepSeek V3.2878785848185.1%
Mistral Medium 3.1878684848485.0%
Laguna S 2.1898885828185.0%
Thinking Machines Inkling Small (Reasoning, High)878685848485.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)878686857984.7%
Qwen 3.6 27B898585828084.2%
DeepSeek V4 Flash868584838183.8%
Z.AI GLM 4.5878584837983.4%
Qwen 3 235B A22B Instruct 2507888787807383.0%
Thinking Machines Inkling858583828083.0%
Muse Glimmer 30B (Reasoning, Medium)888584807782.9%
Qwen 3.5 9B888785787582.7%
MiniMax M2.7928381807782.6%
ByteDance Seed 2.0 Mini858583817982.5%
Mistral Large 2878281818182.4%
Qwen 3.6 35B908380797882.1%
Qwen 3.6 Flash878583807581.7%
DeepSeek V3 (2024-12-26)898878777681.7%
Mistral Large 3838181818181.6%
GPT-4.1868483817181.1%
DeepSeek V3 (2025-03-24)848279797880.5%
DeepSeek-V2 Chat898883716879.9%
Gemini 3.1 Flash Lite (Preview)858178777679.5%
Gemini 3.1 Flash Lite (Reasoning)838080787679.3%
ByteDance Seed 1.6 Flash838381767479.3%
ByteDance Seed 1.6888077767278.6%
Hermes 3 405B827979777778.5%
Z.AI GLM 4.5 Air858280767078.4%
Nemotron 3 Nano808079777578.2%
GPT-4o, Aug. 6th (temp=0)787878787878.1%
GPT-4.1 Mini868178727177.5%
GPT-5.4 Nano (Reasoning)868178756777.3%
GPT-4o, Aug. 6th (temp=1)817877767577.3%
Gemini 3.1 Flash Lite857676757477.0%
Gemini 2.5 Flash Lite (Reasoning)807979766976.9%
Laguna XS 2.1848176717076.2%
WizardLM 2 8x22b867975736675.8%
Mistral Small 3.2 24B777777757075.3%
Inception Mercury 2828077746375.3%
GPT-5.4 Mini (Reasoning, Low)787676757175.2%
GPT-4o Mini (temp=0)767676767175.0%
GPT-OSS 120B797575737174.7%
GPT-5.4 Mini867474706874.3%
Ministral 3 8B797675727074.3%
Qwen 2.5 72B777574747174.2%
Gemma 3 27B797971717074.1%
Ministral 3 14B777575727274.1%
Ministral 8B777372717072.8%
Cydonia 24B V4.1827774676472.8%
GPT-5 Nano817973676372.4%
Qwen 3 32B767372716771.9%
Gemini 2.5 Flash Lite757471706971.8%
Mistral Small 4 (Reasoning)807872666371.7%
Hermes 3 70B817373686271.6%
Mistral Small 4737271676369.4%
GPT-4o Mini (temp=1)767169676469.4%
Z.AI GLM 4.7 Flash847573694268.5%
Ministral 3 3B747371645968.1%
Ministral 3B747070655667.3%
Arcee AI: Trinity Mini777169615867.0%
Gemma 3 12B676762616063.6%
Gemma 3 4B756361595562.4%
GPT-5.4 Nano (Reasoning, Low)696964585262.3%
Gemma 4 26B81797575062.0%
GPT-5.4 Nano696865594861.8%
Cohere Command R+ (Aug. 2024)696560585361.0%
Llama 3.1 70B80796963058.1%
Aion 3.0 Mini8786860051.8%
GPT-4.1 Nano535351484550.1%
Mistral NeMO8272720045.3%
Model # 1 # 2 # 3 # 4 # 5 Avg ▼
Aion 3.0989898989798.2%
Z.AI GLM 5.11009898979798.0%
Gemini 2.5 Pro989898989697.9%
Qwen 3.5 Plus (2026-02-15)989898989697.9%
Claude Sonnet 4.5989898969697.5%
Mistral Medium 3.1989897979697.5%
GPT-5989797979797.3%
Qwen 3.5 397B A17B989898979497.3%
Nemotron 3 Super989897979697.2%
Gemini 3.7 Flash (Reasoning, Medium)979797979797.2%
Claude Opus 4.5989898949496.7%
Muse Spark 1.1 (Reasoning, Minimal)989897969496.7%
Z.AI GLM 5.2 (Reasoning, High)989796969696.6%
DeepSeek V3 (2025-03-24)999797959496.5%
Claude Opus 4989898949396.5%
Hy4 Preview (Reasoning, High)989897969296.3%
Muse Spark 1.2 (Reasoning, Medium)989797969396.3%
Z.AI GLM 5.3 (Reasoning, Max)989797969296.3%
Mistral Large 3989895959596.3%
Z.AI GLM 5979797959496.2%
Muse Spark 1.1 (Reasoning, Medium)989696969496.0%
Grok 4.6 (Reasoning, High)979797959496.0%
Z.AI GLM 5.3 Flash (Reasoning, Low)969696969696.0%
Claude Sonnet 4969696969696.0%
DeepSeek V4 Flash (Reasoning)989696959495.9%
Grok 4.5 (Reasoning, High)969696969595.8%
DeepSeek V4 Flash 0731 (Reasoning, High)989796959295.8%
Claude Sonnet 4.6 (Reasoning)989696949495.7%
Qwen 3.8 Max (Reasoning, XHigh)989796959295.7%
GPT-5.1989696959295.6%
Claude Opus 4.6959595959595.3%
Xiaomi MIMO v2.5989696949295.3%
DeepSeek V4.1 Flash (Reasoning, High)979695959395.2%
MiniMax M3989695949295.1%
Claude Sonnet 5 (Reasoning, Low)989695949294.9%
MiniMax M2.7989896948894.9%
Aion 2.0989897919094.9%
Claude Sonnet 4.6969696939394.8%
Thinking Machines Inkling (Reasoning)979797929194.8%
Qwen 3.5 122B989694929294.7%
Grok 4.20 (Reasoning)989892929294.6%
Hermes 3 405B959595949394.6%
Mistral Large 2989595929294.6%
Qwen 3.6 Max Preview949494949494.4%
Claude Sonnet 5 (Reasoning)989694929294.4%
Muse Spark 1.3 (Reasoning, Medium)969494949394.3%
Z.AI GLM 4.5959595959294.3%
Z.AI GLM 5 Turbo969695939294.2%
DeepSeek V4 Pro 0813 (Reasoning, High)969694939294.2%
Claude Opus 4.6 (Reasoning)949494949494.1%
Grok 4.5 (Reasoning, Low)989594929294.1%
DeepSeek V3.1979796918993.9%
Gemini 3 Flash (Preview, Reasoning)989797977993.9%
Z.AI GLM 4.7989494939093.9%
Qwen 3.5 Plus (2026-04-20)979795919093.9%
Kimi K3 (Reasoning, Low)969694929293.8%
Z.AI GLM 5.3 Flash (Reasoning, Max)979595929093.8%
Thinking Machines Inkling Small (Reasoning, High)979694948793.7%
Aion 3.0 Mini989896928393.5%
GPT-5.4 Mini (Reasoning)989594938793.5%
Qwen 3.8 27B (Reasoning, XHigh)979494919093.4%
Muse Glimmer 30B (Reasoning, Medium)989794938393.2%
Gemini 3.5 Flash (Reasoning, Minimal)979696928593.2%
Gemini 3.5 Flash (Reasoning)989797957893.2%
Gemini 3.8 Flash (Reasoning, Medium)979797947892.8%
Gemini 3.6 Flash (Reasoning, Minimal)989392898992.6%
Kimi K2.5969492909092.4%
Ministral 3 8B949494919092.3%
Grok 4.3979493898792.3%
Gemini 2.5 Flash Lite (Reasoning)969493928792.3%
ByteDance Seed 2.0 Lite969494908692.0%
Kimi K3 (Reasoning, High)979694947891.9%
Gemma 4 31B (Reasoning)989894927791.9%
Xiaomi MIMO v2.5 Pro1009696927591.9%
Thinking Machines Inkling969291909091.8%
DeepSeek V4 Pro (Reasoning)969595947991.8%
Qwen 3.5 Flash979290908891.5%
Gemini 3 Flash (Preview)949492928591.5%
GPT-5.6 Sol (Reasoning)969594937891.3%
Kimi K2.6939291919091.3%
Qwen 3.5 35B959292928691.2%
Ministral 3 14B959292888891.0%
DeepSeek-V2 Chat969292908490.9%
Grok 4.3 (Reasoning)939191918890.7%
o4 Mini High979594907890.7%
DeepSeek V4 Flash929190898990.2%
MiniMax M2.5969695897590.2%
Z.AI GLM 4.6969689858490.1%
WizardLM 2 8x22b979593877589.4%
DeepSeek V3.2989592897088.9%
DeepSeek V3 (2024-12-26)929090878688.9%
DeepSeek V4 Pro929089898488.8%
Gemini 3.6 Flash (Reasoning)979793787888.7%
GPT-5.4 Nano (Reasoning)919188888688.7%
Laguna S 2.1949292858088.5%
Gemini 2.5 Flash (Reasoning)989190887588.4%
ByteDance Seed 2.0 Mini948988878588.4%
Grok 4.20969688877388.2%
Qwen 3.8 Flash (Reasoning)949492906988.1%
ByteDance Seed 1.6989286818187.7%
Claude Haiku 4.5949090907487.7%
Gemini 2.5 Flash Lite928887868587.5%
Qwen 3.6 27B949390907087.5%
Claude Sonnet 5969494777787.5%
Claude Opus 4.8 (Reasoning, Low)989879797987.0%
Claude Opus 4.8 (Reasoning)989879797987.0%
Ministral 8B958783838286.1%
Qwen 3.7 Flash (Reasoning)959087807785.9%
GPT-5 Mini939292797385.6%
Qwen 3.7 Max979678787785.3%
o4 Mini969493726984.8%
Qwen 2.5 72B908783818184.4%
Qwen 3.6 Flash928783807883.9%
Gemma 4 26B868684828283.8%
Z.AI GLM 4.5 Air939083787383.5%
Qwen 3.5 9B939385806482.8%
GPT-OSS 120B888781807982.8%
Claude Opus 5 (Reasoning)987979777782.2%
Inception Mercury 2888681787682.0%
Claude Opus 5 (Reasoning, Low)987977777781.7%
Qwen 3.6 35B908887776481.4%
Nemotron 3 Nano898280797781.3%
GPT-5.5 (Reasoning)957877767580.1%
Hermes 3 70B1009188734880.0%
Qwen 3 235B A22B Instruct 2507818181797779.9%
Claude Opus 4.7797979797979.4%
Writer: Palmyra X5818179797579.3%
Claude Opus 5797979797778.9%
Qwen 3 32B848180767478.9%
Laguna XS 2.1818080787678.8%
Claude Opus 4.7 (Reasoning)797977777777.9%
Llama 3.1 70B828178777077.9%
Gemini 2.5 Flash787878777777.7%
Gemma 3 4B817876767577.3%
GPT-5.5787878767677.2%
Gemma 4 31B797976767577.1%
GPT-5.5 (Reasoning, Low)797877767577.1%
Gemini 3.1 Pro (Preview)787878757576.9%
Mistral Small 3.2 24B898174726876.8%
GPT-5.6 Sol787877767576.6%
GPT-5.4 Mini928171706876.6%
GPT-5.2777675757375.3%
GPT-5.6 Terra (Reasoning)797675737375.2%
Gemma 3 27B797776727275.0%
GPT-5.4 (Reasoning)767675747474.9%
Mistral Small 4 (Reasoning)908881664774.3%
Gemini 3.1 Flash Lite787773737074.2%
GPT-4o, Aug. 6th (temp=0)747474747273.8%
Z.AI GLM 4.7 Flash927870686173.7%
Gemini 3.1 Flash Lite (Preview)777776716773.5%
Gemini 3.1 Flash Lite (Reasoning)777773707073.5%
GPT-5.4 (Reasoning, Low)757473727072.6%
GPT-5.6 Terra757373727072.3%
GPT-5.6 Luna737372717172.1%
Gemma 4 26B (Reasoning)92929085071.8%
Arcee AI: Trinity Mini797471706371.5%
GPT-5.4 Mini (Reasoning, Low)857373685771.4%
ByteDance Seed 1.6 Flash817670676271.2%
GPT-5.4 Nano (Reasoning, Low)818073635971.2%
GPT-5.6 Luna (Reasoning)737372676670.2%
GPT-4.1777472656270.0%
Gemma 3 12B797571715469.9%
Cydonia 24B V4.191908781069.8%
GPT-4o, Aug. 6th (temp=1)817065656468.9%
Cohere Command R+ (Aug. 2024)888171673668.6%
GPT-5.4 Nano777370636068.5%
Ministral 3B727068676368.0%
Mistral Small 4847065635567.6%
GPT-5.4676564646264.5%
Gemini 3.5 Flash Lite (Reasoning, Minimal)777770593864.3%
GPT-4.1 Mini777370633463.1%
Ministral 3 3B656463625862.6%
GPT-5 Nano787058564761.7%
Qwen 3.5 27B9896940057.7%
GPT-4.1 Nano796653413755.1%
GPT-4o Mini (temp=1)645454505054.5%
GPT-4o Mini (temp=0)545454545052.9%
Gemini 3.5 Flash Lite (Reasoning)736840383851.7%
Mistral NeMO8180610044.3%