Precision

Test: Codex Violation Detection

Avg. Score
88.5%
Scenarios
8

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Gemini 3.7 Flash (Reasoning, Medium)99.8%$0.0139.9s98%
2Qwen 3.7 Flash (Reasoning)98.3%$0.000933.3s92%
3Gemini 2.5 Flash Lite (Reasoning)96.8%$0.002017.6s90%
4Gemini 3.8 Flash (Reasoning, Medium)100.0%$0.02013.0s100%
5Z.AI GLM 5.3 Flash (Reasoning, Low)96.2%$0.000416.9s88%
6Gemma 4 31B97.4%$0.000937.9s93%
7Z.AI GLM 5 Turbo97.4%$0.007217.1s91%
8Inception Mercury 295.0%$0.00304.6s87%
9GPT-5.6 Terra96.6%$0.0113.8s91%
10Gemini 2.5 Flash (Reasoning)96.9%$0.007912.5s89%
11ByteDance Seed 1.6 Flash94.1%$0.000912.0s85%
12GPT-5.4 Mini (Reasoning, Low)95.2%$0.00556.7s85%
13MiniMax M2.795.5%$0.002229.4s88%
14DeepSeek V4.1 Flash (Reasoning, High)97.3%$0.004630.8s88%
15Gemini 3 Flash (Preview, Reasoning)96.7%$0.01118.0s91%
16DeepSeek V4 Flash (Reasoning)95.9%$0.001040.3s89%
17Gemini 3 Flash (Preview)93.9%$0.00314.5s83%
18Muse Spark 1.1 (Reasoning, Minimal)96.5%$0.01110.5s88%
19Gemini 3.6 Flash (Reasoning, Minimal)94.6%$0.0113.3s87%
20Gemini 3.5 Flash (Reasoning, Minimal)94.8%$0.0113.6s87%
21Grok 4.5 (Reasoning, Low)98.0%$0.01329.6s92%
22GPT-5.495.7%$0.0138.8s88%
23Z.AI GLM 5.2 (Reasoning, High)97.3%$0.010027.9s89%
24Gemini 3.5 Flash Lite (Reasoning, Minimal)93.5%$0.00302.1s79%
25Muse Spark 1.1 (Reasoning, Medium)96.9%$0.01616.1s89%
26MiniMax M2.593.5%$0.002025.5s84%
27ByteDance Seed 2.0 Lite98.1%$0.00671.1m93%
28GPT-5.6 Sol97.6%$0.0266.4s92%
29Muse Glimmer 30B (Reasoning, Medium)96.1%$0.003649.9s87%
30GPT-5.6 Luna (Reasoning)94.1%$0.0109.3s83%
31Grok 4.20 (Reasoning)98.5%$0.01545.4s92%
32Gemini 2.5 Flash91.4%$0.00252.8s77%
33Qwen 3.5 Flash97.1%$0.00381.0m88%
34Xiaomi MIMO v2.5 Pro95.4%$0.009137.0s87%
35o4 Mini97.4%$0.01928.1s90%
36ByteDance Seed 1.696.8%$0.00671.0m91%
37Xiaomi MIMO v2.594.7%$0.005821.8s80%
38Mistral Small 4 (Reasoning)92.2%$0.002016.3s78%
39GPT-5.6 Terra (Reasoning)95.3%$0.02110.3s88%
40Claude Opus 4.599.8%$0.0419.7s98%
41Claude Sonnet 4.596.3%$0.0248.9s88%
42Kimi K3 (Reasoning, Low)98.3%$0.02436.5s93%
43Claude Sonnet 495.6%$0.0239.0s88%
44Z.AI GLM 597.9%$0.01356.4s90%
45Muse Spark 1.3 (Reasoning, Medium)98.6%$0.02243.0s93%
46Muse Spark 1.2 (Reasoning, Medium)96.0%$0.02116.6s87%
47MiniMax M396.9%$0.00461.4m90%
48Z.AI GLM 4.796.2%$0.00911.0m89%
49Gemini 3.6 Flash (Reasoning)99.7%$0.03916.1s96%
50GPT-5.4 (Reasoning, Low)94.9%$0.02013.5s84%
51Gemini 3.1 Flash Lite (Preview)89.2%$0.00192.2s73%
52Z.AI GLM 4.7 Flash94.0%$0.00181.1m86%
53Qwen 3.6 35B96.9%$0.01351.0s88%
54GPT-5.4 Mini (Reasoning)96.0%$0.02028.6s87%
55Qwen 3.5 35B97.5%$0.01754.8s91%
56GPT-5.596.2%$0.0307.7s89%
57GPT-5 Mini94.8%$0.009251.7s86%
58GPT-5.295.7%$0.02424.2s89%
59Qwen 3.6 Flash96.7%$0.01129.9s77%
60Gemini 2.5 Pro98.4%$0.03523.2s93%
61Claude Opus 4.698.5%$0.04010.2s94%
62Mistral Large 387.5%$0.003010.2s74%
63Gemini 3.1 Flash Lite87.8%$0.00183.2s71%
64Qwen 3.5 Plus (2026-04-20)98.5%$0.0151.4m93%
65Grok 4.5 (Reasoning, High)97.4%$0.02746.7s92%
66DeepSeek V3 (2024-12-26)89.6%$0.001915.6s70%
67DeepSeek V4 Flash87.3%$0.00039.3s68%
68Gemini 3.5 Flash (Reasoning)98.9%$0.04518.1s94%
69Grok 4.3 (Reasoning)98.4%$0.0191.4m90%
70DeepSeek-V2 Chat89.5%$0.002013.1s66%
71GPT-5.5 (Reasoning, Low)96.3%$0.04014.2s90%
72GPT-OSS 120B94.3%$0.00201.5m83%
73Gemini 3.5 Flash Lite (Reasoning)90.4%$0.00292.0s61%
74Z.AI GLM 4.588.7%$0.002618.5s68%
75Z.AI GLM 4.5 Air92.0%$0.002539.8s70%
76Z.AI GLM 5.3 Flash (Reasoning, Max)97.7%$0.00192.3m91%
77Qwen 3 32B89.4%$0.001027.8s67%
78Thinking Machines Inkling Small (Reasoning, High)93.4%$0.006655.7s75%
79Z.AI GLM 5.197.6%$0.0171.6m90%
80GPT-4.1 Mini84.4%$0.00155.8s66%
81Mistral Large 287.9%$0.0128.8s72%
82Kimi K2.595.9%$0.0151.5m88%
83Gemini 3.1 Flash Lite (Reasoning)87.0%$0.00183.7s62%
84Claude Sonnet 4.691.5%$0.0249.9s77%
85DeepSeek V3.286.6%$0.001317.2s66%
86o4 Mini High97.1%$0.03351.3s90%
87Mistral Medium 3.185.4%$0.00297.7s64%
88DeepSeek V4 Flash 0731 (Reasoning, High)95.5%$0.00221.5m75%
89GPT-5.6 Luna84.9%$0.00483.9s64%
90Qwen 3.5 122B96.2%$0.0261.2m88%
91GPT-5.197.3%$0.03752.8s90%
92Qwen 3.6 27B97.4%$0.0221.4m86%
93Qwen 3.5 27B97.2%$0.0211.7m91%
94ByteDance Seed 2.0 Mini98.0%$0.00292.7m92%
95Qwen 3.5 Plus (2026-02-15)89.3%$0.004134.2s66%
96Qwen 3.8 27B (Reasoning, XHigh)96.7%$0.0251.4m89%
97Qwen 3 235B A22B Instruct 250783.6%$0.000721.1s65%
98Z.AI GLM 4.691.9%$0.00491.3m76%
99GPT-4.189.1%$0.008110.0s61%
100Writer: Palmyra X584.3%$0.006212.1s66%
101GPT-4o, Aug. 6th (temp=1)87.6%$0.0113.6s62%
102Laguna XS 2.188.8%$0.000519.0s56%
103Gemma 4 31B (Reasoning)97.6%$0.00172.8m92%
104Gemma 4 26B (Reasoning)96.6%$0.00222.0m77%
105Grok 4.2083.4%$0.00536.4s62%
106Aion 2.095.9%$0.00841.3m68%
107GPT-5.6 Sol (Reasoning)95.9%$0.05219.9s89%
108 Kimi K3 (Reasoning, High)98.9%$0.0441.3m95%
109Qwen 3.7 Max98.3%$0.0451.1m92%
110DeepSeek V4 Pro86.1%$0.003125.6s60%
111DeepSeek V3 (2025-03-24)86.8%$0.001522.9s56%
112Qwen 3.8 Flash (Reasoning)95.7%$0.00602.6m88%
113GPT-4o, Aug. 6th (temp=0)87.3%$0.0155.0s60%
114Gemma 3 27B78.6%$0.000513.3s60%
115GPT-5.4 (Reasoning)94.5%$0.04243.1s84%
116Claude Opus 4.7 (Reasoning)98.0%$0.06711.4s91%
117Qwen 3.5 9B95.5%$0.00202.4m76%
118Claude Opus 597.0%$0.05210.1s77%
119Claude Opus 4.792.9%$0.0527.6s80%
120Hermes 3 405B82.2%$0.004417.2s53%
121Gemini 3.1 Pro (Preview)99.3%$0.06852.3s96%
122Grok 4.6 (Reasoning, High)97.5%$0.0441.7m91%
123Claude Opus 4.6 (Reasoning)99.2%$0.07632.0s96%
124Arcee AI: Trinity Mini80.9%$0.000410.3s47%
125Claude Haiku 4.581.8%$0.00785.8s49%
126Claude Sonnet 5 (Reasoning, Low)94.7%$0.04226.3s68%
127Aion 3.095.9%$0.0351.4m76%
128GPT-5.5 (Reasoning)96.8%$0.07130.3s90%
129Claude Opus 4.8 (Reasoning)98.4%$0.08221.0s92%
130Claude Opus 4.8 (Reasoning, Low)98.1%$0.08020.4s90%
131Nemotron 3 Super90.9%$0.00002.3m65%
132Grok 4.378.2%$0.00616.1s47%
133GPT-5.4 Mini75.5%$0.00333.1s47%
134DeepSeek V4 Pro 0813 (Reasoning, High)93.7%$0.0261.9m76%
135Mistral Small 3.2 24B72.3%$0.000610.4s49%
136Gemma 4 26B83.7%$0.000625.0s38%
137Claude Opus 5 (Reasoning, Low)98.1%$0.07016.6s77%
138Qwen 2.5 72B78.0%$0.000813.9s41%
139DeepSeek V3.179.4%$0.001431.1s44%
140Laguna S 2.186.8%$0.00091.4m49%
141Nemotron 3 Nano93.5%$0.00313.5m81%
142Qwen 3.6 Max Preview98.9%$0.0502.5m94%
143GPT-5 Nano88.5%$0.00491.9m57%
144Claude Sonnet 5 (Reasoning)92.3%$0.04327.5s57%
145Claude Sonnet 4.6 (Reasoning)95.9%$0.07651.3s87%
146GPT-596.2%$0.0611.6m87%
147Claude Opus 5 (Reasoning)97.5%$0.07016.7s69%
148Claude Sonnet 583.6%$0.0219.7s41%
149Z.AI GLM 5.3 (Reasoning, Max)98.6%$0.0453.1m94%
150Ministral 3 14B67.1%$0.00106.3s41%
151Qwen 3.5 397B A17B95.5%$0.0262.9m76%
152DeepSeek V4 Pro (Reasoning)94.0%$0.0142.8m62%
153GPT-4o Mini (temp=1)67.1%$0.00067.4s34%
154Llama 3.1 70B72.2%$0.002124.3s32%
155Hy4 Preview (Reasoning, High)94.1%$0.0363.2m83%
156GPT-5.4 Nano (Reasoning)77.0%$0.003516.6s23%
157GPT-4o Mini (temp=0)69.1%$0.000625.0s32%
158Gemini 2.5 Flash Lite64.3%$0.00052.3s31%
159Gemma 3 12B62.3%$0.000312.0s34%
160Ministral 3 8B58.4%$0.00074.8s36%
161Mistral Small 456.5%$0.00084.3s35%
162Cydonia 24B V4.159.1%$0.001014.4s34%
163Thinking Machines Inkling68.2%$0.00771.0m40%
164Ministral 3 3B56.1%$0.00053.3s29%
165Hermes 3 70B60.7%$0.001329.2s31%
166Ministral 8B52.4%$0.00055.6s31%
167Ministral 3B53.5%$0.00022.9s28%
168Kimi K2.695.5%$0.0384.1m76%
169Claude Opus 490.4%$0.11615.8s77%
170Cohere Command R+ (Aug. 2024)60.3%$0.01410.0s24%
171Thinking Machines Inkling (Reasoning)95.2%$0.0275.6m87%
172Mistral NeMO44.0%$0.000713.3s21%
173Qwen 3.8 Max (Reasoning, XHigh)96.9%$0.0734.5m87%
174GPT-5.4 Nano35.5%$0.00082.9s13%
175GPT-5.4 Nano (Reasoning, Low)43.0%$0.00136.1s4%
176WizardLM 2 8x22b32.2%$0.003615.7s6%
177Gemma 3 4B20.8%$0.000212.5s14%
178GPT-4.1 Nano20.4%$0.00043.9s0%
179Aion 3.0 Mini46.5%$0.00911.9m0%
88.46%

Individual Scenarios

matrix

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Qwen 3.8 Max (Reasoning, XHigh)100100100100100100100100100100100.0%
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Z.AI GLM 5.3 (Reasoning, Max)100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100100100100.0%
Grok 4.6 (Reasoning, High)1001001001001001001001001009799.7%
Claude Opus 4.8 (Reasoning)1001001001001001001001001009799.7%
Claude Opus 5 (Reasoning)1001001001001001001001001009799.7%
o4 Mini1001001001001001001001001009799.7%
o4 Mini High1001001001001001001001001009799.7%
Grok 4.3 (Reasoning)1001001001001001001001001009699.6%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100979799.4%
GPT-5100100100100100100100100979799.4%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100979799.4%
Gemini 3.1 Pro (Preview)100100100100100100100100979799.3%
Muse Spark 1.1 (Reasoning, Minimal)1001001001001001001001001009399.3%
Z.AI GLM 5100100100100100100100100979699.3%
Muse Spark 1.3 (Reasoning, Medium)10010010010010010010097979799.1%
Grok 4.20 (Reasoning)10010010010010010010097979799.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100979499.0%
Qwen 3.8 Flash (Reasoning)100100100100100100100100979399.0%
Gemini 3.6 Flash (Reasoning)10010010010010010010097979799.0%
Claude Opus 4.7 (Reasoning)10010010010010010010097979799.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100969398.9%
Muse Spark 1.2 (Reasoning, Medium)1001001001001001009797979798.8%
Kimi K3 (Reasoning, High)1001001001001001009797979798.8%
Qwen 3.8 27B (Reasoning, XHigh)1001001001001001009797979798.8%
Claude Opus 4.6 (Reasoning)10010010010010010010097979498.7%
Thinking Machines Inkling Small (Reasoning, High)1001001001001001009797979798.7%
Gemini 3.7 Flash (Reasoning, Medium)10010010010010010010097979498.7%
Claude Sonnet 5 (Reasoning, Low)1001001001001001009797979498.4%
GPT-5.5 (Reasoning)100100100100100979797979798.4%
Claude Opus 4.5100100100100100979797979798.4%
Claude Sonnet 4.6 (Reasoning)100100100100100979797979498.1%
GPT-5.2100100100100100979797979498.1%
GPT-5 Mini1001001001001001009797949498.1%
Muse Glimmer 30B (Reasoning, Medium)1001001001001001009797949498.0%
GPT-5.110010010010010010010096949098.0%
Qwen 3.6 Flash1001001001001001009796949498.0%
Gemini 2.5 Flash (Reasoning)100100100100100979797969498.0%
Gemini 2.5 Flash Lite (Reasoning)1001001001001001009696959298.0%
Hy4 Preview (Reasoning, High)10010010010097979797979497.9%
Qwen 3.6 27B1001001001001001009796939197.7%
Z.AI GLM 4.7100100100100100979796939397.5%
Claude Sonnet 5 (Reasoning)1001001009797979797979497.5%
Gemma 4 26B (Reasoning)10010010010096969696969397.4%
Qwen 3.5 Flash1001001001001001009796938897.4%
GPT-5.4 (Reasoning)100100100100100979794949397.4%
ByteDance Seed 2.0 Mini10010010010097969696969397.4%
MiniMax M31001001009797979797979397.3%
Qwen 3.5 Plus (2026-04-20)10010010010097979696949397.3%
Qwen 3.5 397B A17B10010010010097979794949497.2%
Qwen 3.6 35B10010010010010010010096908497.1%
Gemini 3.5 Flash (Reasoning)10010010010097979794949397.1%
Nemotron 3 Super100100979797979797969497.0%
ByteDance Seed 2.0 Lite1001001009796969696959396.9%
Gemini 2.5 Pro1001001009797979797949196.9%
Z.AI GLM 5.11001001009797979794949396.9%
Thinking Machines Inkling (Reasoning)10010010010097979794939196.9%
GPT-5.6 Sol (Reasoning)9797979797979797979796.8%
Z.AI GLM 5.2 (Reasoning, High)100100979797979796949496.8%
Qwen 3.7 Max9797979797979797979496.7%
Grok 4.5 (Reasoning, High)10097979797979797979196.7%
Inception Mercury 210010010010096969696919196.7%
Kimi K3 (Reasoning, Low)10010010010097979694919196.6%
Qwen 3.5 35B10010010010097979793938996.6%
Claude Opus 4.7100100979797979796949196.4%
DeepSeek V4.1 Flash (Reasoning, High)100100979797979794949196.3%
Xiaomi MIMO v2.5 Pro100100979797979694939396.3%
DeepSeek V4 Flash (Reasoning)10097979797979794949396.2%
GPT-5.6 Luna (Reasoning)10097979797979794949496.2%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100979797979794949096.2%
GPT-5.5 (Reasoning, Low)9797979797979797949196.0%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100949393908996.0%
Qwen 3.5 27B10097979797949494949495.7%
Z.AI GLM 4.610097979796969494939195.5%
Aion 3.010010010010097979794947695.5%
Gemma 4 31B (Reasoning)10097979797969494938995.3%
Z.AI GLM 5 Turbo9797979797979794919195.3%
Qwen 3.5 122B10097979796949393939195.1%
Gemma 4 31B9797979696949393939394.9%
GPT-OSS 120B100100100100100959387868694.7%
GPT-5.6 Terra (Reasoning)9797979797949494919094.6%
MiniMax M2.7100100969695959292898994.4%
GPT-5.6 Terra9797949494949494949494.4%
Claude Opus 510097979494949494919094.3%
GPT-5.6 Sol9797979494949492929194.3%
GPT-5.4 (Reasoning, Low)10097979494949491919194.2%
Grok 4.5 (Reasoning, Low)10097979797949491908694.2%
Claude Sonnet 4.610097969693939393909094.2%
Qwen 3.5 9B1001001009695939389888594.0%
Kimi K2.610097979494949491918893.9%
GPT-5.4 Mini (Reasoning, Low)10096969695959292908693.9%
Claude Sonnet 49796969693939393908993.7%
ByteDance Seed 1.610097949494949391888592.9%
Gemini 3 Flash (Preview, Reasoning)9797949493939391908892.9%
GPT-5.59494949494929191919192.8%
GPT-5.49494949494919191898992.3%
GPT-5.4 Mini (Reasoning)10097969393909089878692.1%
Kimi K2.510097979693919088877891.7%
MiniMax M2.51001001009695948885837591.7%
Claude Opus 4.69494949491919090909091.7%
Gemini 3 Flash (Preview)9493939393939090908791.6%
GPT-5 Nano1001001009591898684818190.7%
Mistral Small 4 (Reasoning)9494949494939188867890.7%
Mistral Large 310095939292908989858290.6%
Z.AI GLM 4.7 Flash10095949393928886867990.5%
GPT-4.19695959292919090867590.1%
Mistral Large 210095959291878786847889.5%
Nemotron 3 Nano10010010010093929187676589.4%
Gemini 3.1 Flash Lite (Preview)9595929088888888888189.2%
Gemini 2.5 Flash9693939390888786857989.0%
Aion 2.0100100100100100100979794088.7%
Gemini 3.5 Flash (Reasoning, Minimal)9393939390888584848188.6%
Claude Sonnet 4.59493919090888685858488.5%
Gemma 4 26B9292929288888786848188.3%
DeepSeek V3 (2024-12-26)1001001009595918981715688.0%
Xiaomi MIMO v2.5100100979493939392922387.7%
Claude Opus 49793918888868583818187.3%
DeepSeek V4 Flash 0731 (Reasoning, High)100100100979797979491087.3%
ByteDance Seed 1.6 Flash10092928988888382797586.7%
DeepSeek-V2 Chat10095959486857979777686.6%
Gemini 3.6 Flash (Reasoning, Minimal)9088888885858585848286.0%
Gemini 3.1 Flash Lite (Reasoning)9692888885858585817585.9%
Gemini 3.1 Flash Lite9292888887848482827985.8%
Z.AI GLM 4.5 Air100100969695928989543885.0%
Gemini 3.5 Flash Lite (Reasoning)8888888686868683797984.8%
Gemini 3.5 Flash Lite (Reasoning, Minimal)8989868685838181817783.9%
GPT-5.6 Luna9188838281797877777481.2%
Mistral Medium 3.18987838282808077767380.9%
Qwen 3 32B9493939288787773704780.4%
DeepSeek V4 Pro9287858181797979756580.3%
DeepSeek V3.210085838181797872696879.7%
Claude Haiku 4.58686867979787876747479.6%
Qwen 3 235B A22B Instruct 25079190908380777372676278.6%
Claude Sonnet 5939389898786868380078.6%
Z.AI GLM 4.510090818076757574706378.5%
GPT-4.1 Mini10095947977767369625878.2%
Laguna XS 2.1959591908987817676078.1%
DeepSeek V4 Pro (Reasoning)10010010097979494870076.9%
Grok 4.209085797878787471686476.5%
Qwen 3.5 Plus (2026-02-15)979493918888858247076.3%
Qwen 2.5 72B10090867575757371675076.2%
DeepSeek V3.11008886858483827971075.8%
DeepSeek V4 Flash8382827776767372676475.3%
DeepSeek V3 (2025-03-24)969586858281807568074.8%
Writer: Palmyra X58785848076757271685074.8%
GPT-4o, Aug. 6th (temp=1)9183818080757371585374.6%
GPT-5.4 Nano (Reasoning)96969393929088860073.3%
GPT-5.4 Mini7575717167615958575664.9%
GPT-4o, Aug. 6th (temp=0)92847979797975732264.3%
Hermes 3 405B10083806360575745444063.0%
Arcee AI: Trinity Mini1001001006050505050302861.8%
Grok 4.388858279736664480058.5%
Gemma 3 27B7877755754535050453357.2%
Hermes 3 70B808075675750504414051.8%
Ministral 3 8B7158575552504847423551.5%
Ministral 8B6357565050505047464551.5%
Mistral Small 3.2 24B6867645755524644392151.2%
Cydonia 24B V4.17560565452474341371748.1%
Mistral Small 49156525045444236333148.0%
Laguna S 2.1100979494910000047.6%
Aion 3.0 Mini100979790880000047.2%
GPT-5.4 Nano (Reasoning, Low)888583797659000047.0%
Ministral 3 14B575757575350443826043.9%
Gemma 3 12B7567604540333329292543.5%
GPT-4o Mini (temp=1)676056503833333020038.6%
Thinking Machines Inkling7068636148282000035.6%
Cohere Command R+ (Aug. 2024)56545045434227250034.1%
WizardLM 2 8x22b5641393635302925211732.9%
Ministral 3B3633333030302827211828.8%
Gemini 2.5 Flash Lite565251473827000027.1%
Llama 3.1 70B545050403829200026.1%
Ministral 3 3B3735312624222220131324.3%
GPT-5.4 Nano50505036270000021.3%
Mistral NeMO552524171411995517.4%
Gemma 3 4B3529202013131100014.1%
GPT-4.1 Nano1002112400000013.7%
GPT-4o Mini (temp=0)5033222060000013.2%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100100100100.0%
Claude Opus 5100100100100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
GPT-5.5100100100100100100100100100100100.0%
Muse Glimmer 30B (Reasoning, Medium)100100100100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)1001001001001001001001001009599.5%
ByteDance Seed 1.61001001001001001001001001009599.5%
Gemini 2.5 Pro1001001001001001001001001009599.5%
Claude Opus 4.8 (Reasoning)1001001001001001001001001009499.4%
Muse Spark 1.1 (Reasoning, Minimal)1001001001001001001001001009499.4%
Nemotron 3 Super1001001001001001001001001009499.4%
Qwen 3.6 35B1001001001001001001001001009499.4%
GPT-5.6 Sol100100100100100100100100959599.0%
Z.AI GLM 5.1100100100100100100100100959599.0%
Qwen 3.7 Max100100100100100100100100959598.9%
Gemini 3.5 Flash (Reasoning)100100100100100100100100959598.9%
Kimi K3 (Reasoning, Low)100100100100100100100100959598.9%
Grok 4.3 (Reasoning)1001001001001001001001001008998.9%
MiniMax M3100100100100100100100100959598.9%
Claude Sonnet 4.6100100100100100100100100959498.9%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100949498.9%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100949498.9%
Claude Opus 4.7 (Reasoning)100100100100100100100100949498.8%
DeepSeek V4.1 Flash (Reasoning, High)10010010010010010010095959598.5%
Gemini 3.1 Pro (Preview)1001001001001001009595959598.0%
Kimi K3 (Reasoning, High)1001001001001001009595959598.0%
Thinking Machines Inkling (Reasoning)10010010010010010010095949098.0%
Qwen 3.8 Max (Reasoning, XHigh)1001001001001001009595959597.9%
Grok 4.5 (Reasoning, Low)1001001001001001009595959597.9%
Qwen 3.6 Max Preview1001001001001001009595959597.9%
Qwen 3.6 Flash10010010010010010010094949097.9%
DeepSeek V4 Flash 0731 (Reasoning, High)10010010010010010010094949097.9%
Aion 2.010010010010010010010094949097.8%
Z.AI GLM 5.2 (Reasoning, High)1001001001001001009595949497.8%
Muse Spark 1.1 (Reasoning, Medium)1001001001001001009594949497.8%
ByteDance Seed 2.0 Lite1001001001001001009494949497.6%
Qwen 3.5 9B10010010010010010010094948897.6%
Inception Mercury 210010010010010010010094948897.5%
Nemotron 3 Nano10010010010010010010093929097.5%
Qwen 3.8 27B (Reasoning, XHigh)10010010010010010010095909097.5%
GPT-5.5 (Reasoning)100100100100100959595959597.4%
Z.AI GLM 5 Turbo100100100100100959595959497.4%
Z.AI GLM 5.3 Flash (Reasoning, Low)100100100100100959595959497.3%
GPT-51001001001001001009594949097.3%
Qwen 3.7 Flash (Reasoning)100100100100100959595949497.3%
Claude Opus 4.6 (Reasoning)100100100100100959494949497.3%
GPT-OSS 120B1001001001001001009494948997.1%
Grok 4.5 (Reasoning, High)10010010010095959595959597.0%
Claude Opus 4.610010010010095959595959597.0%
DeepSeek V4 Pro (Reasoning)100100100100100959595949097.0%
Aion 3.010010010010095959595959596.9%
Qwen 3.5 35B10010010010095959595949496.8%
Qwen 3.5 Flash100100100100100959594949096.8%
o4 Mini High10010010010094949494949496.6%
Kimi K2.610010010010095959595959096.5%
Kimi K2.510010010010095959595959096.5%
Muse Spark 1.2 (Reasoning, Medium)100100100100100959595909096.4%
Z.AI GLM 51001001001001001009594908596.4%
Claude Sonnet 410010010010095959595948996.3%
Xiaomi MIMO v2.5 Pro10010010010095959595948996.3%
Qwen 3.6 27B10010010010010010010095897896.2%
Gemma 4 31B (Reasoning)100100100100100959594898996.2%
DeepSeek V4 Pro 0813 (Reasoning, High)100100100100100959593928796.2%
GPT-5.410010010010095959595958696.1%
GPT-4.110010010010095959493938995.9%
ByteDance Seed 2.0 Mini10010010010095949494948895.8%
GPT-5.4 (Reasoning)1001001009595959594949095.8%
GPT-5.6 Luna (Reasoning)10010010010095949494898995.7%
MiniMax M2.510010010010094949493928895.5%
Z.AI GLM 5.3 (Reasoning, Max)10095959595959595959595.5%
Z.AI GLM 5.3 Flash (Reasoning, Max)10095959595959595959595.5%
Claude Sonnet 4.6 (Reasoning)10095959595959595959595.4%
Qwen 3.8 Flash (Reasoning)1001001009595959595908995.4%
DeepSeek V4 Flash (Reasoning)10010010010095959590898995.4%
GPT-5.6 Sol (Reasoning)100100959595959595949095.3%
Z.AI GLM 4.7 Flash1001001001001001009392888095.3%
Hy4 Preview (Reasoning, High)10095959595959595959495.3%
Gemini 3 Flash (Preview, Reasoning)1001001009595959494908995.3%
GPT-5.6 Terra (Reasoning)10095959595959595959495.2%
Gemma 4 26B (Reasoning)10010010010095949489898995.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100949489888495.0%
o4 Mini1001001009494949494898894.9%
Gemini 2.5 Flash (Reasoning)10010010010095949090908994.9%
MiniMax M2.7100100100100100949392868394.8%
Z.AI GLM 4.71001001009595959490898894.6%
GPT-5.1100100959595949494908994.6%
Grok 4.6 (Reasoning, High)9595959595959595959094.5%
Xiaomi MIMO v2.5100100100100100949090868594.5%
Mistral Small 4 (Reasoning)1001001001001001009486858094.4%
GPT-5.4 Nano (Reasoning)1001001009494949393888794.3%
Claude Opus 4.79595959595959595958994.2%
Gemini 2.5 Flash Lite (Reasoning)1001001009494949489888794.0%
Laguna S 2.110010010010095959593827893.7%
GPT-5 Mini100100949494949090898993.6%
Qwen 3.5 397B A17B9595959595959589898993.2%
Gemma 4 31B9595959595959589898993.2%
Gemini 3.5 Flash (Reasoning, Minimal)9595959595959489898993.1%
Qwen 3.5 122B100100959595949090868693.0%
GPT-5.4 (Reasoning, Low)1001001009090909090898992.9%
ByteDance Seed 1.6 Flash1001001009392929291868192.7%
Z.AI GLM 4.610095959594909089888392.0%
GPT-5.6 Terra9595959590909090908691.8%
Thinking Machines Inkling Small (Reasoning, High)100100100100100959494943991.7%
Gemini 3.6 Flash (Reasoning, Minimal)9595959595898989898491.6%
Gemini 3 Flash (Preview)9595959595898989898491.6%
Z.AI GLM 4.59494949494948988878791.5%
Qwen 3.5 27B9595959590909090908691.5%
Grok 4.209595959595909090858391.3%
DeepSeek V4 Flash10095959594898988868291.3%
Grok 4.31001001009494938887817691.3%
GPT-5.29595959090909090908691.2%
Qwen 3 32B10010010010094929187756990.8%
Gemini 3.5 Flash Lite (Reasoning)9595959489898989868590.7%
GPT-4o, Aug. 6th (temp=0)10092929292928686868590.1%
Claude Opus 410095948989898989837989.6%
Gemini 3.5 Flash Lite (Reasoning, Minimal)9590909090898989898489.6%
Claude Sonnet 510094898989898988838289.4%
DeepSeek-V2 Chat100100949488888787817689.4%
DeepSeek V4 Pro10089898989888888868689.3%
Gemini 3.1 Flash Lite (Reasoning)10095898989898484838088.4%
Claude Sonnet 5 (Reasoning)1001001001001001001009488088.3%
Qwen 3.5 Plus (2026-02-15)9595898989898989797888.2%
DeepSeek V3 (2024-12-26)100100948888878382817888.0%
Gemini 2.5 Flash9595949090858583837988.0%
Writer: Palmyra X510093908989868582807987.4%
Arcee AI: Trinity Mini100100100100100888367676086.4%
Gemini 3.1 Flash Lite (Preview)9590858585858585858186.1%
Mistral Medium 3.19488888888888482797885.8%
Mistral Large 38888888887878383828185.4%
GPT-4o, Aug. 6th (temp=1)1001001001009392929086085.3%
Mistral Large 29489898783828282827884.9%
Gemini 3.1 Flash Lite9089898985848080807984.5%
DeepSeek V3.210094898884838381785083.2%
Laguna XS 2.1100100100949389888179082.4%
GPT-5.6 Luna9586858481808079797582.4%
Qwen 3 235B A22B Instruct 25079489898784828079765982.0%
Gemma 3 27B9089838280808080807581.9%
Z.AI GLM 4.5 Air10010094949393888567081.4%
Hermes 3 405B10091908882797070696079.8%
DeepSeek V3 (2025-03-24)10010088888887868276079.5%
Thinking Machines Inkling8585858484777674737079.3%
Claude Sonnet 5 (Reasoning, Low)100100100100959489880076.6%
GPT-4o Mini (temp=0)7878787878787878786076.0%
Mistral Small 3.2 24B8881807878767268676775.5%
GPT-5 Nano100949494939392880074.9%
GPT-4.1 Mini8080787776757574656474.4%
Ministral 3B9190898583757563424273.4%
Qwen 2.5 72B10089888078757150383370.1%
Claude Haiku 4.594898989878383810069.6%
Gemini 2.5 Flash Lite8479747170686762595769.0%
Gemma 3 12B8075737371707067644568.8%
Ministral 3 8B857777757573737171067.7%
Ministral 3 3B1009189898675564738667.6%
Cydonia 24B V4.18678767569676764504467.5%
Ministral 8B7977716765646363635966.9%
DeepSeek V3.194948985838372650066.6%
Ministral 3 14B7171696764636260535062.9%
GPT-5.4 Mini9491898885838200061.3%
Mistral Small 48886676260575653532660.6%
Cohere Command R+ (Aug. 2024)1008883786457504033059.3%
Hermes 3 70B1007573736756555036058.4%
Llama 3.1 70B888378755654504738056.7%
GPT-4o Mini (temp=1)837163605756504329051.1%
Aion 3.0 Mini100959490890000046.9%
GPT-5.4 Nano (Reasoning, Low)1001001009300000039.3%
GPT-5.4 Nano85807877500000036.9%
Gemma 4 26B898989000000026.7%
Mistral NeMO555031302218171513725.7%
GPT-4.1 Nano5020151180000010.5%
Gemma 3 4B201310108700006.8%
WizardLM 2 8x22b00000000000.0%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 # 8 # 9 # 10 Avg ▼
Gemini 3.8 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 3.7 Flash (Reasoning, Medium)100100100100100100100100100100100.0%
Muse Spark 1.2 (Reasoning, Medium)100100100100100100100100100100100.0%
Muse Spark 1.3 (Reasoning, Medium)100100100100100100100100100100100.0%
Qwen 3.6 Max Preview100100100100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100100100100.0%
Qwen 3.7 Flash (Reasoning)100100100100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100100100100.0%
Qwen 3.5 Flash100100100100100100100100100100100.0%
Gemma 4 31B100100100100100100100100100100100.0%
Claude Haiku 4.5100100100100100100100100100100100.0%
Gemini 3.1 Pro (Preview)1001001001001001001001001009199.1%
Muse Spark 1.1 (Reasoning, Medium)1001001001001001001001001009199.1%
Gemini 3.5 Flash (Reasoning)1001001001001001001001001009099.0%
Claude Opus 4.61001001001001001001001001009099.0%
Claude Opus 4.8 (Reasoning, Low)1001001001001001001001001009099.0%
Muse Spark 1.1 (Reasoning, Minimal)1001001001001001001001001009099.0%
Qwen 3.5 397B A17B1001001001001001001001001009099.0%
GPT-5.6 Sol1001001001001001001001001009099.0%
Claude Sonnet 5 (Reasoning, Low)1001001001001001001001001008998.9%
Qwen 3.5 Plus (2026-04-20)1001001001001001001001001008998.9%
Qwen 3.5 122B1001001001001001001001001008898.8%
Grok 4.5 (Reasoning, Low)100100100100100100100100919198.2%
Z.AI GLM 5.3 Flash (Reasoning, Max)100100100100100100100100919098.1%
Claude Opus 4.6 (Reasoning)100100100100100100100100909098.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100909098.0%
Claude Opus 4.8 (Reasoning)1001001001001001001001001008098.0%
Claude Sonnet 5 (Reasoning)1001001001001001001001001008098.0%
Qwen 3.6 27B100100100100100100100100908997.9%
Aion 2.0100100100100100100100100908997.9%
Qwen 3.6 Flash100100100100100100100100898997.8%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100898997.8%
ByteDance Seed 2.0 Mini100100100100100100100100898897.6%
Z.AI GLM 5.3 (Reasoning, Max)10010010010010010010091919197.3%
Kimi K3 (Reasoning, High)10010010010010010010091919097.2%
Gemini 3 Flash (Preview, Reasoning)10010010010010010010091909097.1%
DeepSeek V4 Pro (Reasoning)100100100100100100100100918097.1%
GPT-5.1100100100100100100100100908097.0%
o4 Mini10010010010010010010090909097.0%
Kimi K3 (Reasoning, Low)10010010010010010010091908997.0%
Z.AI GLM 5.110010010010010010010090908996.9%
Z.AI GLM 5100100100100100100100100898096.9%
Gemini 3.6 Flash (Reasoning, Minimal)10010010010010010010089898996.7%
Z.AI GLM 4.5 Air100100100100100100100100897896.7%
ByteDance Seed 1.610010010010010010010089898996.7%
Gemma 4 31B (Reasoning)10010010010010010010090908296.2%
Grok 4.3 (Reasoning)10010010010010010010091918096.2%
GPT-5.6 Terra1001001001001001009090909096.0%
Qwen 3.6 35B10010010010010010010089898095.8%
Qwen 3.7 Max100100100100100100100100787895.6%
Grok 4.5 (Reasoning, High)100100100100100919191919195.5%
Grok 4.6 (Reasoning, High)100100100100100919191919195.5%
GPT-OSS 120B1001001001001001008989898895.4%
MiniMax M2.51001001001001001008989888895.3%
Qwen 3.8 Flash (Reasoning)100100100100100919191909095.3%
MiniMax M2.71001001001001001008989898695.2%
o4 Mini High1001001001001001009190908095.1%
Z.AI GLM 5 Turbo1001001001001001008989898094.7%
Xiaomi MIMO v2.5100100100100100909089898994.7%
Grok 4.20 (Reasoning)10010010010091919191919194.5%
Qwen 3.5 35B1001001001001001008989888094.5%
ByteDance Seed 2.0 Lite100100100100100908989898894.4%
DeepSeek V4 Flash (Reasoning)100100100100100919090898294.2%
Laguna S 2.11001001001001001008989867894.1%
Claude Sonnet 4.6 (Reasoning)10010010010010010010080808094.0%
Qwen 3.5 Plus (2026-02-15)10010010010090909090909094.0%
GPT-4.1 Mini1001001001001001008888887593.8%
DeepSeek V4.1 Flash (Reasoning, High)10010010010010010010090806793.7%
Qwen 2.5 72B1001001001001001008686838093.5%
MiniMax M310010010010090898989898993.4%
Gemma 4 26B10010010010089898989898993.3%
Aion 3.01001001001001001009189807393.3%
GPT-5.51001001009090909090909093.0%
Z.AI GLM 4.7 Flash100100100100100888886868393.0%
DeepSeek-V2 Chat1001001001001001008683837592.7%
GPT-5 Mini100100919191919191919092.6%
DeepSeek V4 Flash 0731 (Reasoning, High)10010010010091919191907392.6%
Writer: Palmyra X5100100100100100868686838392.4%
Gemini 2.5 Flash (Reasoning)100100100100100918383828292.1%
Claude Sonnet 4.6100100909090909090909092.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100909090909090909092.0%
GPT-5.210091919191919191919191.8%
GPT-510010010010091919090837391.8%
Thinking Machines Inkling (Reasoning)10010010010091919090837391.8%
Z.AI GLM 4.710010010010090898989808091.7%
Muse Glimmer 30B (Reasoning, Medium)100100919191919191918091.6%
Kimi K2.5100100100100100908882787891.5%
Gemini 2.5 Flash Lite (Reasoning)1001001008989898888868691.3%
Nemotron 3 Nano10010010010088868686838391.1%
GPT-5.6 Sol (Reasoning)9191919191919191919190.9%
Qwen 3.8 27B (Reasoning, XHigh)10091919191909090908090.4%
Kimi K2.61001001009191909080808090.2%
DeepSeek V4 Pro 0813 (Reasoning, High)10091919191919190907590.0%
Claude Opus 5100100100100100100100100100090.0%
GPT-4o, Aug. 6th (temp=0)100100888888888888888689.8%
ByteDance Seed 1.6 Flash1001001008988888686807889.3%
GPT-4o, Aug. 6th (temp=1)1001001008888868686867589.3%
Gemini 3 Flash (Preview)10089898989898989898089.1%
Gemma 4 26B (Reasoning)10010010010010010010010090089.0%
GPT-5.410010010010090808080808089.0%
Thinking Machines Inkling Small (Reasoning, High)9191919191919190907388.9%
Xiaomi MIMO v2.5 Pro100100919190908383808088.8%
Claude Sonnet 4.5100100909090898980808088.8%
Gemini 3.5 Flash Lite (Reasoning)9090909090908989898088.7%
Z.AI GLM 4.6100100909090898989826788.5%
DeepSeek V3 (2025-03-24)1001001008686868383837588.2%
Z.AI GLM 5.2 (Reasoning, High)1001001009089898078787888.1%
GPT-5.6 Terra (Reasoning)10091919191919083807388.1%
DeepSeek V3 (2024-12-26)100100888686868683838388.0%
Inception Mercury 28989898989898989898088.0%
GPT-5.5 (Reasoning, Low)9191919191918983828087.9%
DeepSeek V4 Flash100100908989898280808087.8%
GPT-5.6 Luna100100898989898888787087.8%
Qwen 3 235B A22B Instruct 25071001001008888888683757187.8%
Grok 4.310010010010088888680706787.7%
GPT-5.5 (Reasoning)9191919191918382828287.4%
Gemini 2.5 Flash1001001008989828080807087.0%
Claude Opus 410089898989888878787886.4%
Hy4 Preview (Reasoning, High)9191919191838383827786.3%
Qwen 3 32B100100888888868683836386.3%
Z.AI GLM 5.3 Flash (Reasoning, Low)10090908989888080787886.1%
Claude Opus 5 (Reasoning, Low)100100100100100100909080086.0%
Z.AI GLM 4.510089898989898880787086.0%
Mistral Large 210089888888888878757585.4%
Laguna XS 2.1100100898886867878757585.3%
GPT-4.18988888888888886787585.2%
Mistral Small 4 (Reasoning)100100898888888680706485.1%
Claude Sonnet 410090909080808080808085.0%
GPT-5.4 Mini (Reasoning)10089898988808080807584.9%
GPT-5.4 Mini (Reasoning, Low)10089898988888680707084.7%
Qwen 3.5 9B10010010010010089898873083.8%
Qwen 3.8 Max (Reasoning, XHigh)9191909083838277757383.5%
Llama 3.1 70B8886868686868680787583.5%
Mistral Large 39089888888787878787883.0%
GPT-5 Nano10089888886868378676783.0%
Mistral Small 3.2 24B8989898886868075757382.8%
Gemini 2.5 Flash Lite10089868686867875716381.8%
Ministral 3 14B100100898988888878505081.8%
GPT-5.6 Luna (Reasoning)9090828280808080807081.4%
Gemini 3.1 Flash Lite (Preview)8980808080808080808080.9%
Gemini 3.1 Flash Lite8980808080808080808080.9%
DeepSeek V3.28888888686868371676380.4%
Claude Opus 5 (Reasoning)1001001001001001001001000080.0%
Cydonia 24B V4.110089888080787878676079.6%
Gemini 3.1 Flash Lite (Reasoning)8080808080808080807379.3%
GPT-5.4 (Reasoning, Low)10089808080737373737379.3%
GPT-5.4 Nano (Reasoning)1009089898988888078078.9%
Gemma 3 27B10088868680757067676778.4%
Mistral Medium 3.1898989898989898080078.2%
Nemotron 3 Super10091838080808067625878.1%
GPT-5.4 (Reasoning)9090828073737373737377.8%
GPT-4o Mini (temp=0)8075757575757575757575.5%
Hermes 3 405B8886837571717167676774.6%
Grok 4.208282808080736967676774.6%
DeepSeek V3.11008686868375717171073.0%
GPT-5.4 Mini8883837875707070585072.5%
Arcee AI: Trinity Mini100100838067606060575071.7%
GPT-4o Mini (temp=1)8080808080757560505071.0%
DeepSeek V4 Pro909086808080706762070.4%
Claude Sonnet 590908989898080800068.7%
Thinking Machines Inkling8073737367676767624266.8%
Hermes 3 70B1008380757167635636063.1%
Ministral 3 3B7567676360606056565061.2%
Gemma 3 12B7563636357575750504457.8%
Ministral 3B8656565656545044423853.5%
Mistral Small 47163635650504545443051.7%
Ministral 3 8B6462585854504746403551.4%
Cohere Command R+ (Aug. 2024)1008380757167000047.6%
Ministral 8B735853535050474619045.0%
Mistral NeMO75555454505033330040.4%
Aion 3.0 Mini100100918900000038.0%
Gemma 3 4B5747433837353329272637.2%
GPT-5.4 Nano (Reasoning, Low)8989887800000034.3%
GPT-5.4 Nano555044444342000027.8%
GPT-4.1 Nano252318120000007.7%
WizardLM 2 8x22b200000000002.0%