Accuracy (recall)

Test: Codex Violation Detection

Avg. Score
80.7%
Scenarios
8

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1GPT-6 Sol99.0%$0.00746.5s95%
2GPT-5.6 Terra96.6%$0.0113.8s90%
3Xiaomi MIMO v2.6 Pro97.2%$0.002340.3s92%
4Gemini 3.7 Flash (Reasoning, Medium)97.3%$0.0139.9s91%
5DeepSeek V4.1 Flash (Reasoning, High)97.1%$0.004630.8s89%
6GPT-6.1 Sol (Reasoning, Medium)95.8%$0.009113.4s88%
7Gemini 3.8 Flash (Reasoning, Medium)98.6%$0.02013.0s94%
8Gemini 2.5 Flash (Reasoning)94.0%$0.007912.5s86%
9GPT-6 Sol (Reasoning, Medium)95.3%$0.01313.9s89%
10GPT-5.6 Sol98.4%$0.0266.4s94%
11Grok 4.5 (Reasoning, Low)97.7%$0.01329.6s91%
12Z.AI GLM 5 Turbo94.8%$0.007217.1s85%
13Qwen 3.7 Flash (Reasoning)93.9%$0.000933.3s85%
14GPT-5.495.9%$0.0138.8s85%
15Muse Spark 1.1 (Reasoning, Minimal)94.5%$0.01110.5s86%
16Z.AI GLM 5.3 Flash (Reasoning, Low)92.5%$0.000416.9s79%
17Gemini 3 Flash (Preview, Reasoning)94.7%$0.01118.0s86%
18Muse Spark 1.1 (Reasoning, Medium)95.2%$0.01616.1s88%
19DeepSeek V4 Flash (Reasoning)94.4%$0.001040.3s82%
20Muse Spark 1.2 (Reasoning, Medium)96.0%$0.02116.6s88%
21GPT-6 Luna (Reasoning, Medium)91.7%$0.001018.7s76%
22GPT-5.596.7%$0.0307.7s90%
23Grok 4.20 (Reasoning)95.8%$0.01545.4s90%
24Gemma 4 31B92.5%$0.000937.9s78%
25Z.AI GLM 5.2 (Reasoning, High)93.3%$0.010027.9s80%
26Kimi K3 (Reasoning, Low)97.3%$0.02436.5s90%
27GPT-6 Luna (Reasoning, High)91.0%$0.001422.5s73%
28Grok 4.5 (Reasoning, High)98.0%$0.02746.7s94%
29Muse Spark 1.3 (Reasoning, Medium)96.6%$0.02243.0s90%
30GPT-5.295.7%$0.02424.2s87%
31Xiaomi MIMO v2.6 Flash94.0%$0.000639.8s73%
32Xiaomi MIMO v2.5 Pro92.5%$0.009137.0s80%
33Gemini 3.5 Flash (Reasoning, Minimal)90.7%$0.0113.6s73%
34GPT-6 Luna87.5%$0.00046.0s68%
35Gemini 3 Flash (Preview)88.8%$0.00314.5s69%
36Claude Sonnet 5.5 (Reasoning)92.4%$0.0236.5s82%
37Claude Opus 4.598.0%$0.0419.7s91%
38Gemini 3.6 Flash (Reasoning, Minimal)90.4%$0.0113.3s73%
39Gemini 2.5 Pro97.4%$0.03523.2s91%
40GPT-5.6 Terra (Reasoning, Medium)93.1%$0.02110.3s80%
41Muse Glimmer 30B (Reasoning, Medium)92.1%$0.003649.9s77%
42GPT-5.4 (Reasoning, Low)92.4%$0.02013.5s79%
43Claude Sonnet 4.593.6%$0.0248.9s80%
44Gemini 3.5 Flash Lite (Reasoning, Minimal)88.9%$0.00302.1s65%
45Gemini 3.6 Flash (Reasoning)96.9%$0.03916.1s90%
46GPT-5.6 Luna (Reasoning, Medium)89.7%$0.0109.3s72%
47Qwen 3.5 Flash92.8%$0.00381.0m77%
48Claude Opus 4.697.3%$0.04010.2s88%
49Gemini 2.5 Flash82.5%$0.00252.8s69%
50MiniMax M393.2%$0.00461.4m82%
51Claude Opus 5.5 (Reasoning)98.8%$0.05112.7s94%
52Qwen 3.6 Flash92.4%$0.01129.9s72%
53Z.AI GLM 5.3 Flash (Reasoning, Max)97.7%$0.00192.3m91%
54Claude Haiku 5.5 (Reasoning, Medium)85.9%$0.00158.6s65%
55GPT-5.5 (Reasoning, Low)95.5%$0.04014.2s88%
56Claude Haiku 5.5 (Adaptive)85.7%$0.00178.4s64%
57Claude Haiku 5.5 (Reasoning, Max)95.1%$0.0131.5m87%
58ByteDance Seed 1.691.9%$0.00671.0m77%
59Qwen 3.5 35B93.3%$0.01754.8s81%
60Qwen 3.6 35B92.1%$0.01351.0s78%
61Z.AI GLM 592.1%$0.01356.4s79%
62Xiaomi MIMO v2.589.2%$0.005821.8s65%
63Z.AI GLM 4.791.3%$0.00911.0m77%
64GPT-5 Mini91.2%$0.009251.7s74%
65Z.AI GLM 5.195.5%$0.0171.6m88%
66Claude Sonnet 490.3%$0.0239.0s72%
67Inception Mercury 282.2%$0.00304.6s62%
68Qwen 3.5 Plus (2026-04-20)93.4%$0.0151.4m83%
69o4 Mini88.9%$0.01928.1s74%
70Gemini 3.5 Flash (Reasoning)94.8%$0.04518.1s86%
71DeepSeek V4 Flash 0731 (Reasoning, High)93.7%$0.00221.5m73%
72ByteDance Seed 2.0 Lite89.8%$0.00671.1m73%
73Gemini 3.1 Flash Lite81.1%$0.00183.2s59%
74Qwen 3.5 27B95.6%$0.0211.7m89%
75GPT-5.4 Mini (Reasoning, Low)83.0%$0.00556.7s61%
76Gemini 3.1 Flash Lite (Preview)81.9%$0.00192.2s57%
77Mistral Large 4.0 (Reasoning)96.2%$0.0171.9m88%
78Grok 4.3 (Reasoning)94.3%$0.0191.4m81%
79Gemini 2.5 Flash Lite (Reasoning)81.1%$0.002017.6s62%
80Claude Haiku 5.5 (Reasoning, Low)82.5%$0.00158.6s57%
81 Kimi K3 (Reasoning, High)98.6%$0.0441.3m94%
82Thinking Machines Inkling Small (Reasoning, High)90.4%$0.006655.7s65%
83Gemini 3.5 Flash Lite (Reasoning)84.7%$0.00292.0s52%
84Kimi K2.592.9%$0.0151.5m79%
85GPT-5.6 Sol (Reasoning, Medium)95.4%$0.05219.9s86%
86Qwen 3.5 122B92.7%$0.0261.2m81%
87Qwen 3.8 Flash (Reasoning)95.0%$0.00602.6m86%
88GPT-5.4 Mini (Reasoning)87.9%$0.02028.6s68%
89Claude Sonnet 4.682.9%$0.0249.9s70%
90Gemini 3.1 Flash Lite (Reasoning)79.3%$0.00183.7s52%
91Gemma 4 31B (Reasoning)94.4%$0.00172.8m85%
92Mistral Large 4.079.4%$0.002816.9s56%
93DeepSeek V4 Flash78.1%$0.00039.3s53%
94MiniMax M2.779.1%$0.002229.4s59%
95Mistral Large 375.5%$0.003010.2s58%
96Aion 2.091.7%$0.00841.3m64%
97Grok 4.6 (Reasoning, High)98.1%$0.0441.7m94%
98Qwen 3.5 Plus (2026-02-15)85.0%$0.004134.2s55%
99Qwen 3.7 Max96.3%$0.0451.1m85%
100GPT-5.6 Luna78.9%$0.00483.9s52%
101GPT-5.192.7%$0.03752.8s79%
102o4 Mini High90.7%$0.03351.3s76%
103Qwen 3.8 27B (Reasoning, XHigh)94.3%$0.0251.4m75%
104DeepSeek V4 Pro81.7%$0.003125.6s53%
105Aion 3.5 Mini (Reasoning, High)95.9%$0.0102.3m76%
106Mistral Medium 3.5 (Reasoning)79.6%$0.0114.6s52%
107Mistral Large 275.9%$0.0128.8s57%
108Grok 4.2077.0%$0.00536.4s50%
109Gemini 3.1 Pro (Preview)98.3%$0.06852.3s93%
110Mistral Medium 3.578.4%$0.0114.5s52%
111Gemma 4 26B (Reasoning)90.0%$0.00222.0m65%
112Claude Opus 4.788.3%$0.0527.6s76%
113GPT-5.4 (Reasoning)91.7%$0.04243.1s75%
114Z.AI GLM 4.686.1%$0.00491.3m59%
115MiniMax M2.574.6%$0.002025.5s52%
116Claude Opus 4.7 (Reasoning)92.8%$0.06711.4s82%
117Mistral Medium 3.175.1%$0.00297.7s46%
118Claude Opus 4.6 (Reasoning)97.3%$0.07632.0s90%
119GPT-5.5 (Reasoning, Medium)95.6%$0.07130.3s88%
120Claude Opus 593.7%$0.05210.1s68%
121Aion 3.094.1%$0.0351.4m74%
122ByteDance Seed 2.0 Mini90.8%$0.00292.7m73%
123DeepSeek V3.275.0%$0.001317.2s45%
124Z.AI GLM 4.575.6%$0.002618.5s45%
125GPT-OSS 120B81.0%$0.00201.5m59%
126GPT-4.177.7%$0.008110.0s44%
127Mistral Small 4 (Reasoning)70.9%$0.002016.3s47%
128Claude Sonnet 5 (Reasoning, Low)90.6%$0.04226.3s63%
129Qwen 3.6 27B89.5%$0.0221.4m63%
130ByteDance Seed 1.6 Flash67.9%$0.000912.0s46%
131Z.AI GLM 4.5 Air77.1%$0.002539.8s46%
132Claude Opus 4.8 (Reasoning)95.8%$0.08221.0s87%
133Aion 3.5 (Reasoning, High)93.6%$0.0231.2m57%
134Laguna XS 2.172.9%$0.000519.0s41%
135Claude Opus 4.8 (Reasoning, Low)94.7%$0.08020.4s85%
136DeepSeek V4 Pro 0813 (Reasoning, High)92.0%$0.0261.9m69%
137DeepSeek-V2 Chat71.1%$0.002013.1s41%
138DeepSeek V3 (2024-12-26)69.9%$0.001915.6s41%
139GPT-4.1 Mini63.7%$0.00155.8s43%
140Claude Haiku 4.566.8%$0.00785.8s45%
141Qwen 3 235B A22B Instruct 250767.3%$0.000721.1s43%
142Gemma 4 26B75.0%$0.000625.0s35%
143Z.AI GLM 5.3 (Reasoning, Max)98.4%$0.0453.1m94%
144Writer: Palmyra X567.2%$0.006212.1s44%
145Qwen 3.6 Max Preview96.4%$0.0502.5m90%
146Claude Opus 5 (Reasoning, Low)94.7%$0.07016.6s70%
147Claude Sonnet 4.6 (Reasoning)94.9%$0.07651.3s84%
148Grok 4.7 (Reasoning, High)96.2%$0.0572.3m90%
149Claude Sonnet 5 (Reasoning)88.2%$0.04327.5s54%
150Nemotron 3 Super83.1%$0.00002.3m55%
151Qwen 3.5 9B84.7%$0.00202.4m55%
152Qwen 3.5 397B A17B93.7%$0.0262.9m75%
153Grok 4.364.3%$0.00616.1s39%
154GPT-593.3%$0.0611.6m81%
155Hy4 Preview (Reasoning, High)94.7%$0.0363.2m85%
156Claude Opus 5 (Reasoning)94.4%$0.07016.7s63%
157DeepSeek V3 (2025-03-24)68.9%$0.001522.9s34%
158Z.AI GLM 4.7 Flash68.2%$0.00181.1m45%
159DeepSeek V4 Pro (Reasoning)91.5%$0.0142.8m60%
160GPT-4o, Aug. 6th (temp=1)65.4%$0.0113.6s36%
161GPT-4o, Aug. 6th (temp=0)67.9%$0.0155.0s36%
162DeepSeek V3.166.1%$0.001431.1s33%
163Qwen 3 32B64.1%$0.001027.8s31%
164Laguna S 2.175.8%$0.00091.4m35%
165GPT-5.4 Nano (Reasoning)67.5%$0.003516.6s23%
166Mistral Small 3.2 24B53.8%$0.000610.4s32%
167Gemma 3 27B53.2%$0.000513.3s33%
168Claude Sonnet 570.8%$0.0219.7s31%
169GPT-5.4 Mini54.5%$0.00333.1s29%
170GPT-5 Nano67.2%$0.00491.9m42%
171Gemini 2.5 Flash Lite47.9%$0.00052.3s24%
172Thinking Machines Inkling63.0%$0.00771.0m31%
173Ministral 3 14B46.4%$0.00106.3s25%
174Hermes 3 405B56.2%$0.004417.2s20%
175Kimi K2.694.7%$0.0384.1m75%
176Llama 3.1 70B51.2%$0.002124.3s18%
177Ministral 3 8B39.9%$0.00074.8s20%
178Qwen 2.5 72B42.3%$0.000813.9s18%
179Thinking Machines Inkling (Reasoning)93.5%$0.0275.6m80%
180Ministral 8B34.7%$0.00055.6s18%
181Claude Opus 484.4%$0.11615.8s65%
182Cydonia 24B V4.135.4%$0.001014.4s18%
183Arcee AI: Trinity Mini33.3%$0.000410.3s18%
184Mistral Small 430.7%$0.00084.3s18%
185Qwen 3.8 Max (Reasoning, XHigh)96.6%$0.0734.5m88%
186Ministral 3 3B28.6%$0.00053.3s16%
187Gemma 3 12B35.9%$0.000312.0s11%
188GPT-4o Mini (temp=0)31.3%$0.000625.0s19%
189GPT-4o Mini (temp=1)29.4%$0.00067.4s13%
190Ministral 3B25.0%$0.00022.9s14%
191Nemotron 3 Nano64.5%$0.00313.5m37%
192Cohere Command R+ (Aug. 2024)32.3%$0.01410.0s15%
193Hermes 3 70B28.5%$0.001329.2s14%
194GPT-5.4 Nano (Reasoning, Low)32.9%$0.00136.1s2%
195GPT-5.4 Nano21.3%$0.00082.9s4%
196Mistral NeMO20.6%$0.000713.3s7%
197WizardLM 2 8x22b17.1%$0.003615.7s0%
198Aion 3.0 Mini44.4%$0.00911.9m0%
199GPT-4.1 Nano2.6%$0.00043.9s0%
200Gemma 3 4B3.2%$0.000212.5s0%
80.67%

Individual Scenarios

matrix

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100989794949297.6%
Aion 3.5 Mini (Reasoning, High)10010010010098979797958897.3%
Z.AI GLM 5.3 (Reasoning, Max)10097979797979797979497.0%
Grok 4.6 (Reasoning, High)100100979797979797959196.8%
Hy4 Preview (Reasoning, High)100100989797959595949296.5%
Muse Spark 1.2 (Reasoning, Medium)100100989797979594929296.4%
Qwen 3.7 Max9898989897959595929296.2%
GPT-6 Sol9898989797979794929296.2%
GPT-5.6 Sol9898989795959494949295.8%
Kimi K3 (Reasoning, High)100100979797959592928895.5%
Grok 4.5 (Reasoning, High)10098989895959595898695.3%
Qwen 3.8 Max (Reasoning, XHigh)100100979794949491919194.8%
Muse Spark 1.3 (Reasoning, Medium)9797979595949494929194.7%
Xiaomi MIMO v2.6 Pro10098959595959594928394.5%
Qwen 3.6 Max Preview9797979494949494919194.2%
Claude Opus 5.5 (Reasoning)9797949494949494929194.1%
Claude Sonnet 4.6 (Reasoning)9795959494949492928893.6%
Grok 4.20 (Reasoning)10097979494929291918693.5%
GPT-5.59797959494949292898993.5%
Claude Opus 5 (Reasoning, Low)9494949494949494919193.3%
Qwen 3.5 397B A17B100100979594948989888593.2%
Claude Opus 5 (Reasoning)9494949494949494918993.2%
GPT-5.5 (Reasoning, Medium)9797949494929289898992.9%
Gemini 3.8 Flash (Reasoning, Medium)9494949494949491918892.7%
DeepSeek V4.1 Flash (Reasoning, High)9897979792929289868592.7%
Kimi K3 (Reasoning, Low)10098979797959186838092.6%
GPT-5.5 (Reasoning, Low)9595959592929292918392.6%
GPT-5.49794949292929191919192.6%
Claude Haiku 5.5 (Reasoning, Max)10095959494929189888692.6%
Qwen 3.8 Flash (Reasoning)9797979494929191918292.6%
Aion 3.0100100979594949289887492.4%
Mistral Large 4.0 (Reasoning)100100949491918989888592.1%
Xiaomi MIMO v2.6 Flash10098979492918989888292.1%
Gemini 3.6 Flash (Reasoning)9797979494949289838292.0%
Claude Sonnet 5 (Reasoning, Low)10097949191919189898692.0%
GPT-6.1 Sol (Reasoning, Medium)9292929292919191898991.4%
Kimi K2.610098959494898888867991.2%
GPT-5.29797949292918988868591.2%
Gemini 2.5 Pro10095929292898888888691.2%
Thinking Machines Inkling Small (Reasoning, High)9794929291919189888691.2%
Grok 4.7 (Reasoning, High)9594949291918989888691.1%
Claude Opus 4.8 (Reasoning)9494949291919191888591.1%
Gemini 3.7 Flash (Reasoning, Medium)9494949494919189858390.9%
Qwen 3.5 27B9492929291919191888690.9%
Thinking Machines Inkling (Reasoning)9797979492898986858290.9%
DeepSeek V4 Flash (Reasoning)9595949292929189858290.9%
GPT-59494929291918888888890.6%
GPT-6 Sol (Reasoning, Medium)9492929291918989888690.6%
Muse Spark 1.1 (Reasoning, Medium)9494919191919191888390.5%
Claude Sonnet 5 (Reasoning)9595949289898888888390.3%
Claude Opus 4.8 (Reasoning, Low)9494919191898988888890.3%
GPT-5.6 Sol (Reasoning, Medium)9292928989898989898690.0%
Gemini 3.1 Pro (Preview)9494919191898888888690.0%
Claude Opus 4.6 (Reasoning)9492919191919188868590.0%
GPT-5.4 (Reasoning, Low)9494929291898888868389.8%
Z.AI GLM 5.19595929191918985858289.7%
Muse Spark 1.1 (Reasoning, Minimal)9794949191888888828289.4%
Grok 4.5 (Reasoning, Low)9595949292918883837789.2%
GPT-5.6 Terra9492919189888888858589.1%
GPT-5.4 (Reasoning)9494929191918685858289.1%
Z.AI GLM 5.3 Flash (Reasoning, Low)9794929189898686858089.1%
Claude Opus 4.59292919189898888858388.9%
Z.AI GLM 5.2 (Reasoning, High)10095898988888686858088.8%
MiniMax M39794918989898686867988.8%
GPT-5.6 Luna (Reasoning, Medium)9189898989898888868588.5%
Grok 4.3 (Reasoning)10091888888888888858088.3%
Claude Opus 4.7 (Reasoning)9191918988888886858388.0%
GPT-5.6 Terra (Reasoning, Medium)9292928989888885838088.0%
Z.AI GLM 5 Turbo9292918989868683838387.7%
Claude Sonnet 5.5 (Reasoning)9191918986868685858587.6%
Claude Opus 59491918989888585838087.6%
Nemotron 3 Super9491898989888683838087.4%
Claude Opus 4.79291918989868583838087.1%
Aion 3.5 (Reasoning, High)1009797979797979494087.0%
GPT-5 Mini9191888886868585858587.0%
DeepSeek V4 Pro 0813 (Reasoning, High)9797949488888280767186.7%
GPT-6 Luna (Reasoning, High)9189898888868683828286.5%
Gemini 3.5 Flash (Reasoning)9291918886868585827986.5%
Qwen 3.8 27B (Reasoning, XHigh)1009897979795949289386.4%
Gemini 2.5 Flash (Reasoning)9189898988858585857786.4%
ByteDance Seed 1.69491918988858382807986.2%
Z.AI GLM 4.69492928888868382777786.1%
Qwen 3.6 35B9797919185858280777485.9%
Muse Glimmer 30B (Reasoning, Medium)9191888685858583828285.8%
Qwen 3.5 Plus (2026-04-20)9791898988858582777485.8%
o4 Mini High9491918885838282827985.6%
GPT-6 Luna (Reasoning, Medium)9488888685838282828285.2%
GPT-5.19491888585858282808085.2%
Qwen 3.6 Flash8988888888858580797684.5%
Gemma 4 31B (Reasoning)9291898988888379746884.2%
Qwen 3.7 Flash (Reasoning)9191888585858280797684.1%
Qwen 3.5 122B8988868683828280797983.5%
Gemini 3 Flash (Preview, Reasoning)8988858383828282827483.0%
Z.AI GLM 59785858382828279797783.0%
Qwen 3.5 35B9291898585837979767183.0%
DeepSeek V4 Flash 0731 (Reasoning, High)959494949292918986082.9%
Aion 2.01009794929189888888082.7%
Claude Opus 4.68585858583838080808082.7%
Claude Haiku 5.5 (Adaptive)9488858583838080767182.6%
Xiaomi MIMO v2.5 Pro9288868583838079797082.6%
Qwen 3.6 27B9291858585837776767382.3%
Z.AI GLM 4.79488888383827979747382.3%
Gemma 4 31B8686858382828080797982.3%
Claude Opus 49289858382827977777081.7%
Qwen 3.5 Flash9491868582827977766281.4%
Kimi K2.59186868682828080736581.2%
Claude Sonnet 4.68682828280777776767479.2%
Gemma 4 26B (Reasoning)8885808079797776747479.2%
Claude Sonnet 48382808080797977737178.5%
Gemini 3 Flash (Preview)8582828079767676747478.3%
Claude Sonnet 4.58583828079777777717078.2%
Gemini 3.6 Flash (Reasoning, Minimal)8580807979777777747378.2%
Gemini 3.5 Flash (Reasoning, Minimal)8582828079797774717077.9%
o4 Mini8886828279767673706777.7%
Claude Haiku 5.5 (Reasoning, Medium)8686868279777676645977.1%
GPT-5.4 Mini (Reasoning)8382828080747373717076.8%
Claude Haiku 5.5 (Reasoning, Low)8880797777767674736576.5%
ByteDance Seed 2.0 Lite8585838079777668656276.1%
ByteDance Seed 2.0 Mini8382767474747473736474.7%
Qwen 3.5 9B8582797777767371626274.4%
GPT-6 Luna8579797474747167625872.3%
Xiaomi MIMO v2.5898585827979767070071.4%
DeepSeek V4 Pro (Reasoning)97959492888582760070.9%
Gemini 2.5 Flash7977767674737064615970.8%
GPT-5.6 Luna8979767168676565626270.5%
Inception Mercury 27977747171707070615870.0%
Gemini 2.5 Flash Lite (Reasoning)7977767373686767625569.5%
GPT-5.4 Mini (Reasoning, Low)7777717070686764625968.5%
DeepSeek V4 Pro7776717168686764565367.1%
Gemini 3.5 Flash Lite (Reasoning, Minimal)7170706868676764595966.2%
GPT-OSS 120B7676737067676261565365.9%
Qwen 3.5 Plus (2026-02-15)928583827977763938065.2%
Mistral Large 4.0807976767171686562064.8%
Mistral Large 37373706868646161565264.4%
Gemini 3.5 Flash Lite (Reasoning)7067676765656262615864.2%
GPT-4.17774706765646258564563.8%
Gemma 4 26B7067676565646459585663.3%
MiniMax M2.77771686464645956554862.6%
Gemini 3.1 Flash Lite7067626262585858564860.0%
DeepSeek V4 Flash7973676764645955383259.5%
Gemini 3.1 Flash Lite (Preview)6765656259595656535259.4%
Mistral Large 26767676159585656534758.9%
Gemini 3.1 Flash Lite (Reasoning)6565626161616159533257.9%
GPT-5.4 Nano (Reasoning)82807674706864590057.3%
DeepSeek V3.29465595952474747443955.3%
MiniMax M2.57371646153535042413654.4%
Z.AI GLM 4.57667555353504545453652.6%
Claude Sonnet 5767673717068671411052.4%
Z.AI GLM 4.5 Air807168686756523223352.0%
Grok 4.207361555352504747393551.1%
Claude Haiku 4.56758505050504847474250.9%
DeepSeek-V2 Chat6158565350504744393349.1%
Mistral Medium 3.16256535252484848472148.8%
DeepSeek V3 (2024-12-26)6764595858564835241748.5%
Mistral Medium 3.5 (Reasoning)6458565048444241393848.0%
GPT-5 Nano5858565250484544353548.0%
Qwen 3 235B A22B Instruct 25075855555555444442363347.6%
DeepSeek V3.1716764625848413624047.1%
Laguna S 2.198919191890000046.1%
Mistral Medium 3.55655535345454439383045.9%
Aion 3.0 Mini98979582800000045.3%
Mistral Small 4 (Reasoning)5350504747444441392944.4%
Writer: Palmyra X5565653474444413938342.1%
Laguna XS 2.1625656524545413329042.0%
Grok 4.373616155504742210040.9%
Z.AI GLM 4.7 Flash5347454241383333322939.4%
DeepSeek V3 (2025-03-24)686248474533323023038.9%
ByteDance Seed 1.6 Flash5941393938383532292337.3%
Qwen 3 32B4441393938322618171230.6%
GPT-4.1 Mini5344333330292621201430.3%
Nemotron 3 Nano393836323029292714327.7%
GPT-5.4 Nano (Reasoning, Low)594847413932000026.7%
GPT-5.4 Mini3832322929292318151425.8%
Mistral Small 3.2 24B44393935242118189024.8%
GPT-4o, Aug. 6th (temp=0)44352929292927230024.4%
GPT-4o, Aug. 6th (temp=1)3532292724212015141423.0%
Thinking Machines Inkling62474133320000021.5%
Ministral 3 8B353029242121201814321.5%
Ministral 3 14B3232272424211460018.0%
Qwen 2.5 72B27262017151515129816.4%
Ministral 8B333224232312222215.3%
Gemma 3 27B3626151514121199014.7%
Cydonia 24B V4.136181715129983012.7%
Mistral Small 4292018121211920011.2%
Gemini 2.5 Flash Lite3230261850000011.1%
Hermes 3 405B2718111188665210.0%
Hermes 3 70B14111188852006.4%
Arcee AI: Trinity Mini159665332004.8%
Cohere Command R+ (Aug. 2024)1212965300004.7%
Gemma 3 12B119865000003.8%
GPT-5.4 Nano239330000003.8%
GPT-4o Mini (temp=1)96652000002.7%
Llama 3.1 70B126532000002.7%
WizardLM 2 8x22b96622000002.4%
Mistral NeMO110000000001.1%
GPT-4o Mini (temp=0)80000000000.8%
Ministral 3 3B53000000000.8%
GPT-4.1 Nano30000000000.3%
Ministral 3B30000000000.3%
Gemma 3 4B20000000000.2%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
GPT-6 Sol1001001001001001001001001009799.7%
GPT-5.6 Sol100100100100100100100100979799.5%
Gemini 2.5 Pro1001001001001001001001001009299.2%
Gemini 3.1 Pro (Preview)1001001001001001009797979798.9%
Claude Opus 5.5 (Reasoning)1001001001001001009797959598.4%
DeepSeek V4.1 Flash (Reasoning, High)1001001001001001009797959298.2%
GPT-5.410010010010097979797979297.9%
Qwen 3.8 Max (Reasoning, XHigh)1001001001001001009797929297.9%
Kimi K3 (Reasoning, High)10010010010097979797959597.9%
Claude Opus 4.610010010010097979797979297.9%
Kimi K3 (Reasoning, Low)10010010010010010010095929297.9%
Aion 3.010010010010097979797929297.4%
Z.AI GLM 5.3 (Reasoning, Max)10097979797979797979297.1%
GPT-5.5 (Reasoning, Low)100100100100100959595959297.1%
Z.AI GLM 5.3 Flash (Reasoning, Max)10097979797979797979297.1%
Grok 4.7 (Reasoning, High)9797979797979797979597.1%
Grok 4.5 (Reasoning, High)100100979797979795959296.8%
Gemini 3.7 Flash (Reasoning, Medium)10010010010095959595959596.8%
Grok 4.5 (Reasoning, Low)100100100100100979592929296.8%
Z.AI GLM 5 Turbo100100100100100979792928796.6%
Grok 4.6 (Reasoning, High)9797979797979797959296.6%
Kimi K2.6100100979797979595959296.6%
Z.AI GLM 5.11001001009795959595959296.3%
Xiaomi MIMO v2.6 Pro10010010010097979795898796.3%
Kimi K2.51001001009797959592929296.1%
Claude Sonnet 4.6 (Reasoning)10097979797979792929296.1%
Qwen 3.7 Max1001001009595959595929295.8%
Mistral Large 4.0 (Reasoning)1001001009595959595929295.8%
Claude Sonnet 4.610010010010095959595928795.8%
Gemini 3.6 Flash (Reasoning)10095959595959595959595.3%
MiniMax M31001001009595959592928995.3%
Z.AI GLM 5.3 Flash (Reasoning, Low)10010010010095929292928795.0%
GPT-5.6 Terra9797979795959595958795.0%
Gemini 3.5 Flash (Reasoning)10095959595959595929294.7%
DeepSeek V4 Pro (Reasoning)10097979595959595928794.7%
Qwen 3.8 Flash (Reasoning)10097979797959592898494.5%
GPT-5.5 (Reasoning, Medium)9797959595959592929294.5%
DeepSeek V4 Flash 0731 (Reasoning, High)1001001009595959589878794.2%
Aion 3.5 Mini (Reasoning, High)10097979795929289898993.9%
Qwen 3.8 27B (Reasoning, XHigh)1001001009595928989898993.9%
DeepSeek V4 Flash (Reasoning)1001001009797959289848493.9%
Claude Sonnet 410010010010092929292878493.9%
Muse Spark 1.3 (Reasoning, Medium)9595959595959595898993.7%
GPT-6.1 Sol (Reasoning, Medium)9595959595959292929293.7%
Qwen 3.6 Max Preview10095959595929292928993.7%
Muse Spark 1.1 (Reasoning, Minimal)10095959595959595898293.4%
Hy4 Preview (Reasoning, High)9797979592929292928793.4%
Muse Spark 1.2 (Reasoning, Medium)10095959595929292898993.4%
Claude Opus 4.8 (Reasoning)10095959595959589898793.4%
Qwen 3.6 35B1001001009595959589897693.4%
Z.AI GLM 5.2 (Reasoning, High)1001001009595929289878293.2%
Grok 4.3 (Reasoning)10095959595959589898493.2%
GPT-5.6 Sol (Reasoning, Medium)9797959592929292898792.9%
GPT-5.29797959595928989898792.6%
Qwen 3.5 35B10097979592928989878792.6%
Claude Opus 4.6 (Reasoning)100100959595928787878792.4%
Claude Sonnet 5.5 (Reasoning)10095959292929289898792.4%
GPT-6 Sol (Reasoning, Medium)9595959592929289898792.1%
Claude Opus 4.8 (Reasoning, Low)9595959595959589878292.1%
Thinking Machines Inkling (Reasoning)9595959595928989898792.1%
Qwen 3.6 Flash9595959595958989878792.1%
Claude Haiku 5.5 (Reasoning, Max)9595959592929289898491.8%
Gemini 3 Flash (Preview, Reasoning)10097959592928987878491.8%
ByteDance Seed 1.610095959592898989898491.8%
Qwen 3.7 Flash (Reasoning)100100959292928989878291.8%
Xiaomi MIMO v2.5 Pro100100959292929289848291.8%
GPT-5.4 (Reasoning)9595959292929289878791.6%
GPT-5.6 Terra (Reasoning, Medium)9592929292929292928291.3%
Muse Spark 1.1 (Reasoning, Medium)9595959595928987878291.1%
Grok 4.20 (Reasoning)9595959589898989898491.1%
Qwen 3.5 Flash10095929289898989878791.1%
Gemma 4 31B (Reasoning)10097959595959284797690.8%
Grok 4.209797979797898989827190.8%
Claude Opus 4.79292929292929292927990.8%
Qwen 3.5 122B10095929292898987878290.5%
Gemini 2.5 Flash (Reasoning)10095928989898989878490.5%
Qwen 3.5 27B9292929289898989898790.3%
Qwen 3.5 Plus (2026-04-20)10095898989898987878490.0%
GPT-59595928989898989878289.7%
Gemma 4 31B9292929292929284848489.7%
Qwen 3.5 397B A17B9292929292929284848489.7%
Claude Opus 4.7 (Reasoning)9595958989898989828289.5%
Claude Opus 5 (Reasoning, Low)100100100100100100100100474789.5%
GPT-5.4 (Reasoning, Low)9595898989898989848489.5%
Qwen 3.6 27B10095959592898989846389.2%
Gemini 3.5 Flash (Reasoning, Minimal)9292929292928784848489.2%
GPT-6 Luna (Reasoning, Medium)9292928989898987877988.7%
GPT-5.6 Luna (Reasoning, Medium)9592898989878787848488.4%
GPT-5.4 Mini (Reasoning)9595898989898484848488.4%
Aion 2.09595898989898987847688.4%
DeepSeek V4 Pro 0813 (Reasoning, High)10010010010097979566636187.9%
Z.AI GLM 4.710092929289878484847487.9%
Z.AI GLM 5100100959289828279797987.6%
Xiaomi MIMO v2.59595898989878484828287.6%
Aion 3.5 (Reasoning, High)10010010010010095959292087.4%
GPT-5.19292928989898482828287.4%
Muse Glimmer 30B (Reasoning, Medium)8989898989898984847987.4%
Gemini 3.6 Flash (Reasoning, Minimal)9292929292848484847687.4%
Gemini 3 Flash (Preview)9292929292848484847687.4%
GPT-6 Luna9795929289878282797687.1%
GPT-6 Luna (Reasoning, High)9292898787878784847986.8%
o4 Mini High9595878787878484828286.8%
Z.AI GLM 4.69792928989898784747186.6%
ByteDance Seed 2.0 Lite9589898989848282828286.3%
Gemini 3.5 Flash Lite (Reasoning)9292928787848484827986.3%
GPT-4.1100100959292848476716686.1%
Laguna S 2.1100100959592929266666386.1%
Gemini 3.5 Flash Lite (Reasoning, Minimal)9289898989848484797685.8%
o4 Mini9589898787878282797485.0%
Nemotron 3 Super8989898484848484827985.0%
Claude Opus 5 (Reasoning)10010010010010010010047474784.2%
DeepSeek V4 Flash9592929287877979746684.2%
Qwen 3.5 9B9595898984848276747484.2%
Gemma 4 26B (Reasoning)9592878484847979797683.9%
Gemini 3.1 Flash Lite (Preview)9789828282828282827983.7%
ByteDance Seed 2.0 Mini9595928482828279766883.4%
Mistral Medium 3.5 (Reasoning)8987878782828282827683.4%
Claude Haiku 5.5 (Reasoning, Low)898989828282