Name replacement accuracy

Test: Text Replacement

Avg. Score
97.2%
Scenarios
14

Overall Performance

Rank ▲ Model Score Avg. Cost Avg. Time Stability
1Gemini 3.1 Flash Lite (Reasoning)99.4%$0.00092.6s98%
2Gemini 3.1 Flash Lite (Preview)99.3%$0.00091.7s97%
3Gemini 3.1 Flash Lite99.4%$0.00092.8s97%
4Claude Haiku 4.599.7%$0.00342.8s98%
5DeepSeek V4 Flash98.9%$0.00027.6s96%
6Gemini 3.6 Flash (Reasoning, Minimal)99.5%$0.00472.2s98%
7Gemma 4 26B99.6%$0.000215.4s98%
8GPT-4.1 Mini98.9%$0.00106.6s95%
9Qwen 3.5 Plus (2026-02-15)99.2%$0.00156.7s95%
10Mistral Large 398.7%$0.00117.4s96%
11GPT-5.6 Luna99.0%$0.00362.6s95%
12GPT-5.6 Luna (Reasoning)99.2%$0.00464.4s97%
13Gemini 3 Flash (Preview)98.4%$0.00183.2s94%
14GPT-4.199.4%$0.00524.1s97%
15Inception Mercury 298.3%$0.00162.2s92%
16Qwen 2.5 72B98.4%$0.000310.3s94%
17GPT-4o Mini (temp=1)98.2%$0.00049.2s94%
18Gemini 2.5 Flash Lite98.1%$0.00031.6s88%
19Grok 4.2098.2%$0.00194.2s93%
20GPT-4o Mini (temp=0)98.0%$0.00049.2s93%
21Gemini 3.5 Flash (Reasoning, Minimal)98.7%$0.00542.5s95%
22Muse Spark 1.1 (Reasoning, Minimal)99.6%$0.00648.1s98%
23Mistral Large 298.8%$0.00427.2s96%
24DeepSeek V4 Pro99.4%$0.001221.5s98%
25Gemma 3 27B98.6%$0.000216.0s94%
26GPT-4o, Aug. 6th (temp=0)99.0%$0.00642.6s94%
27GPT-5.6 Terra99.6%$0.00893.2s97%
28Gemma 4 31B99.7%$0.000329.4s99%
29Gemini 2.5 Flash (Reasoning)99.1%$0.005410.7s97%
30DeepSeek-V2 Chat98.4%$0.000816.5s94%
31Claude Sonnet 499.8%$0.0105.7s99%
32Thinking Machines Inkling98.9%$0.002516.9s95%
33GPT-5.6 Terra (Reasoning)99.5%$0.0104.1s98%
34Claude Sonnet 4.599.6%$0.0104.6s98%
35GPT-5.4 Nano (Reasoning)96.9%$0.00115.1s90%
36GPT-5.499.2%$0.00895.4s97%
37GPT-5.4 (Reasoning, Low)99.4%$0.00945.6s97%
38GPT-4o, Aug. 6th (temp=1)98.7%$0.00642.6s92%
39Z.AI GLM 4.599.7%$0.003126.1s98%
40Gemini 2.5 Flash98.7%$0.00142.1s82%
41Gemini 3.5 Flash Lite (Reasoning, Minimal)97.5%$0.00141.4s84%
42Z.AI GLM 5.2 (Reasoning, High)99.6%$0.006418.8s98%
43Muse Spark 1.1 (Reasoning, Medium)99.5%$0.009311.0s98%
44Claude Sonnet 4.699.2%$0.0104.4s96%
45Mistral Medium 3.196.2%$0.00126.0s89%
46GPT-5.4 Nano95.9%$0.00073.1s87%
47Grok 4.20 (Reasoning)99.6%$0.006421.1s98%
48GPT-5.298.8%$0.00876.0s94%
49Cydonia 24B V4.197.3%$0.000411.9s88%
50DeepSeek V4 Flash (Reasoning)99.5%$0.000536.8s98%
51Gemini 3.5 Flash Lite (Reasoning)97.2%$0.00141.4s82%
52Z.AI GLM 5 Turbo99.7%$0.008318.8s98%
53Laguna XS 2.196.9%$0.000210.6s86%
54Mistral Small 3.2 24B97.7%$0.00024.7s80%
55GPT-5.4 Mini (Reasoning)98.4%$0.00537.2s88%
56Claude Sonnet 598.7%$0.00928.3s95%
57Claude Sonnet 5 (Reasoning, Low)99.0%$0.0119.5s96%
58Claude Sonnet 5 (Reasoning)99.0%$0.0119.7s97%
59Gemma 3 12B97.3%$0.00018.6s82%
60Llama 3.1 70B98.0%$0.000523.9s91%
61Gemini 3 Flash (Preview, Reasoning)99.5%$0.009717.2s98%
62Qwen 3.6 35B99.4%$0.005529.6s98%
63GPT-5.4 Nano (Reasoning, Low)95.3%$0.00073.7s83%
64GPT-5.4 Mini96.1%$0.00272.1s82%
65Grok 4.5 (Reasoning, Low)99.7%$0.009624.2s98%
66GPT-5.199.5%$0.01313.7s98%
67GPT-5.599.7%$0.0184.7s99%
68GPT-5 Mini99.3%$0.005233.1s97%
69GPT-5.4 Mini (Reasoning, Low)96.1%$0.00293.3s82%
70Xiaomi MIMO v2.598.2%$0.003313.1s83%
71Grok 4.396.3%$0.00204.5s80%
72Claude Opus 4.699.5%$0.0175.6s98%
73DeepSeek V3.298.9%$0.000444.5s96%
74Claude Opus 4.599.3%$0.0175.2s97%
75GPT-5.6 Sol99.5%$0.0184.3s97%
76Z.AI GLM 4.5 Air98.3%$0.001530.2s89%
77DeepSeek V3 (2024-12-26)97.6%$0.000715.0s80%
78Z.AI GLM 4.699.7%$0.004542.4s98%
79ByteDance Seed 2.0 Lite99.7%$0.004044.8s98%
80GPT-OSS 120B98.9%$0.000848.6s97%
81ByteDance Seed 1.699.4%$0.004043.2s98%
82GPT-5.4 (Reasoning)99.2%$0.01613.3s97%
83Grok 4.3 (Reasoning)99.3%$0.007935.9s97%
84GPT-5.5 (Reasoning, Low)99.7%$0.0216.7s99%
85Aion 2.099.5%$0.003848.1s97%
86Mistral Small 495.2%$0.00043.2s72%
87Grok 4.5 (Reasoning, High)99.7%$0.01523.7s98%
88ByteDance Seed 1.6 Flash95.5%$0.000712.2s76%
89GPT-5 Nano98.8%$0.002350.9s96%
90Qwen3 235B A22B Instruct 250795.6%$0.000314.8s75%
91Qwen 3.5 Flash98.7%$0.002748.6s93%
92GPT-5.6 Sol (Reasoning)99.5%$0.0227.0s97%
93MoonshotAI: Kimi K3 (Reasoning, Low)99.5%$0.01527.0s98%
94Claude Opus 599.5%$0.0237.9s98%
95Hermes 3 405B97.3%$0.001222.3s74%
96Laguna S 2.196.2%$0.000213.8s69%
97Ministral 3 14B93.4%$0.00023.9s70%
98Xiaomi MIMO v2.5 Pro97.4%$0.003515.5s72%
99Gemini 3.6 Flash (Reasoning)99.6%$0.02411.5s98%
100Claude Opus 4.798.5%$0.0234.3s95%
101Qwen 3.6 Flash98.2%$0.007421.9s80%
102Gemini 2.5 Flash Lite (Reasoning)96.3%$0.001816.2s71%
103Claude Opus 4.8 (Reasoning, Low)98.9%$0.0237.3s94%
104Claude Opus 4.7 (Reasoning)98.7%$0.0254.7s95%
105Writer: Palmyra X594.3%$0.003411.1s73%
106Claude Opus 4.8 (Reasoning)98.7%$0.0247.7s93%
107GPT-5.5 (Reasoning)99.4%$0.0279.8s97%
108Qwen 3.5 35B99.0%$0.01343.1s96%
109Z.AI GLM 599.5%$0.008360.0s97%
110Mistral Small 4 (Reasoning)94.1%$0.001714.8s69%
111Gemini 2.5 Pro99.6%$0.02719.7s98%
112Gemini 3.5 Flash (Reasoning)99.6%$0.03012.5s98%
113Qwen 3 32B95.3%$0.000832.0s72%
114o4 Mini97.7%$0.01321.3s80%
115MiniMax M398.7%$0.003156.9s84%
116Claude Opus 4.6 (Reasoning)99.6%$0.03012.5s98%
117o4 Mini High99.1%$0.02133.0s96%
118WizardLM 2 8x22b96.2%$0.000737.2s70%
119MoonshotAI: Kimi K3 (Reasoning, High)99.5%$0.02239.0s98%
120MiniMax M2.597.7%$0.001859.0s80%
121Qwen 3.5 Plus (2026-04-20)99.4%$0.0111.2m98%
122Z.AI GLM 5.199.5%$0.0131.1m98%
123Claude Sonnet 4.6 (Reasoning)99.6%$0.03219.3s98%
124Claude Opus 5 (Reasoning)98.2%$0.02910.5s90%
125Qwen 3.5 27B99.5%$0.0151.1m98%
126Claude Opus 5 (Reasoning, Low)98.0%$0.03010.5s90%
127Z.AI GLM 4.799.7%$0.00701.5m97%
128Arcee AI: Trinity Mini88.5%$0.00026.5s58%
129GPT-4.1 Nano89.9%$0.00033.6s52%
130Nemotron 3 Super95.2%$0.000051.1s66%
131DeepSeek V4 Pro (Reasoning)99.7%$0.00641.7m98%
132Aion 3.098.7%$0.01649.1s82%
133GPT-599.8%$0.02944.3s99%
134Gemma 4 31B (Reasoning)99.4%$0.00111.9m98%
135Gemma 4 26B (Reasoning)99.5%$0.00131.9m98%
136Qwen 3.5 122B99.6%$0.02559.3s98%
137DeepSeek V3 (2025-03-24)92.5%$0.000635.4s58%
138ByteDance Seed 2.0 Mini97.6%$0.00171.7m91%
139Qwen 3.6 27B97.6%$0.01458.6s81%
140Ministral 3 8B85.7%$0.00023.3s52%
141Mistral NeMO85.5%$0.00022.5s51%
142MiniMax M2.796.7%$0.00581.2m74%
143Ministral 8B84.8%$0.00013.3s50%
144Qwen3.7 Max99.6%$0.03155.7s98%
145Claude Opus 499.8%$0.0517.7s98%
146Z.AI GLM 4.7 Flash94.0%$0.00151.1m68%
147Aion 3.0 Mini95.7%$0.00411.1m64%
148Qwen 3.5 9B96.8%$0.00121.8m78%
149MoonshotAI: Kimi K2.599.6%$0.0112.1m98%
150Thinking Machines Inkling (Reasoning)99.4%$0.0132.0m97%
151Qwen 3.5 397B A17B99.4%$0.00892.2m97%
152DeepSeek V3.188.5%$0.000634.3s46%
153MoonshotAI: Kimi K2.699.6%$0.0181.9m98%
154Ministral 3 3B78.9%$0.00012.0s43%
155Ministral 3B78.4%$0.00002.0s39%
156Gemini 3.1 Pro (Preview)98.8%$0.04542.6s87%
157Qwen3.6 Max Preview99.7%$0.0331.8m99%
158Gemma 3 4B73.9%$0.00016.0s24%
159Nemotron 3 Nano83.4%$0.00162.0m46%
160Cohere Command R+ (Aug. 2024)71.4%$0.006727.2s23%
161Hermes 3 70B67.7%$0.00152.0m10%
97.18%

Individual Scenarios

Generic Prompt

Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100.0%
Qwen3.7 Max100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100.0%
GPT-5.4 (Reasoning)100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100.0%
Qwen3.6 Max Preview100100100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100100100.0%
Claude Sonnet 4.6 (Reasoning)100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100100100.0%
Claude Opus 4.7 (Reasoning)100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100100100.0%
Claude Opus 4.8 (Reasoning)100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)100100100100100100100100.0%
Claude Opus 5 (Reasoning, Low)100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100.0%
GPT-5100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100.0%
GPT-5 Mini100100100100100100100100.0%
Claude Opus 5 (Reasoning)100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100.0%
Grok 4.3 (Reasoning)100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100.0%
GPT-5.4 (Reasoning, Low)100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100100100.0%
Thinking Machines Inkling (Reasoning)100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100.0%
GPT-5.1100100100100100100100100.0%
Claude Sonnet 4.6100100100100100100100100.0%
GPT-5.6 Luna (Reasoning)100100100100100100100100.0%
MiniMax M3100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100.0%
Claude Opus 5100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100.0%
Claude Opus 4.7100100100100100100100100.0%
GPT-5.4 Mini (Reasoning)100100100100100100100100.0%
Qwen 3.5 Plus (2026-04-20)100100100100100100100100.0%
Gemma 4 31B (Reasoning)100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100.0%
ByteDance Seed 1.6100100100100100100100100.0%
GPT-5.2100100100100100100100100.0%
GPT-5.5100100100100100100100100.0%
Qwen 3.6 Flash100100100100100100100100.0%
DeepSeek V4 Pro (Reasoning)100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100.0%
Aion 3.0100100100100100100100100.0%
o4 Mini High100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100.0%
GPT-5.6 Terra100100100100100100100100.0%
DeepSeek V4 Flash (Reasoning)100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100.0%
Z.AI GLM 4.6100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100.0%
Claude Sonnet 5100100100100100100100100.0%
Claude Opus 4100100100100100100100100.0%
Qwen 3.5 35B100100100100100100100100.0%
GPT-4.1100100100100100100100100.0%
MiniMax M2.5100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100.0%
Aion 2.0100100100100100100100100.0%
o4 Mini100100100100100100100100.0%
MiniMax M2.7100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100.0%
Xiaomi MIMO v2.5 Pro100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100.0%
Qwen 3.5 Flash100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100.0%
Gemma 4 31B100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100.0%
Gemma 4 26B100100100100100100100100.0%
GPT-OSS 120B100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100100100.0%
GPT-5.4100100100100100100100100.0%
Mistral Large 3100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100.0%
Qwen 3.5 9B100100100100100100100100.0%
Xiaomi MIMO v2.5100100100100100100100100.0%
GPT-5.4 Mini (Reasoning, Low)100100100100100100100100.0%
Claude Haiku 4.5100100100100100100100100.0%
Gemini 2.5 Flash Lite (Reasoning)100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100.0%
DeepSeek V3.2100100100100100100100100.0%
Z.AI GLM 4.7 Flash100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100.0%
DeepSeek V4 Pro100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100.0%
Inception Mercury 2100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100.0%
Mistral Large 2100100100100100100100100.0%
GPT-4.1 Mini100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100.0%
Grok 4.20100100100100100100100100.0%
Hermes 3 405B100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100.0%
GPT-5.4 Mini100100100100100100100100.0%
GPT-5 Nano100100100100100100100100.0%
GPT-5.4 Nano (Reasoning)100100100100100100100100.0%
Laguna S 2.1100100100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100100100.0%
Mistral Small 4 (Reasoning)100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100.0%
Qwen3 235B A22B Instruct 2507100100100100100100100100.0%
Grok 4.3100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100.0%
GPT-5.4 Nano (Reasoning, Low)100100100100100100100100.0%
Llama 3.1 70B100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100.0%
Mistral Medium 3.1100100100100100100100100.0%
Gemma 3 12B100100100100100100100100.0%
Gemma 3 27B100100100100100100100100.0%
Laguna XS 2.1100100100100100100100100.0%
Mistral Small 4100100100100100100100100.0%
Nemotron 3 Nano100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100.0%
Qwen 2.5 72B100100100100100100100100.0%
Cydonia 24B V4.1100100100100100100100100.0%
GPT-5.4 Nano100100100100100100100100.0%
WizardLM 2 8x22b100100100100100100100100.0%
ByteDance Seed 1.6 Flash100100100100100100100100.0%
Ministral 3 14B100100100100100100100100.0%
Ministral 3 8B100100100100100100100100.0%
GPT-4.1 Nano100100100100100100100100.0%
Hermes 3 70B100100100100100100100100.0%
Gemma 3 4B100100100100100100100100.0%
Ministral 8B100100100100100100100100.0%
ByteDance Seed 2.0 Mini1001001001001001009298.8%
DeepSeek V3 (2025-03-24)1001001001001001007596.4%
Qwen 3 32B1001001001001001007596.4%
Ministral 3 3B9292929292929291.7%
Ministral 3B100100929292925889.3%
Mistral NeMO100100100100100100886.9%
DeepSeek V3.110010010010010092885.7%
Aion 3.0 Mini100100100100100100085.7%
Arcee AI: Trinity Mini10010092929283079.8%
Cohere Command R+ (Aug. 2024)1001008000029.8%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 Avg ▼
Thinking Machines Inkling (Reasoning)100100100100100100100100.0%
GPT-5.6 Luna100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100989899.5%
GPT-5.510010010010098989899.2%
Z.AI GLM 4.510010010010098989899.2%
GPT-5.21001001009898989698.7%
GPT-5.5 (Reasoning)100100989898989898.7%
Grok 4.5 (Reasoning, Low)100100989898989898.7%
GPT-5.4 (Reasoning, Low)1001001009898989698.7%
GPT-5.6 Luna (Reasoning)100100989898989898.7%
ByteDance Seed 1.6100100989898989898.7%
GPT-5 Nano1001001009898989698.7%
Grok 4.3 (Reasoning)1001001009898989598.5%
Claude Opus 41001001009898969698.5%
GPT-5.5 (Reasoning, Low)10098989898989898.5%
Qwen 3.5 Plus (2026-04-20)100100989898989698.5%
Z.AI GLM 4.710098989898989898.5%
GPT-510098989898989698.2%
GPT-5.4 Mini10098989898989698.2%
GPT-5.4 (Reasoning)10098989898989698.2%
GPT-5.6 Terra10098989898989698.2%
Claude Opus 4.6 (Reasoning)9898989898989898.2%
Qwen3.6 Max Preview9898989898989898.2%
Claude Opus 4.7 (Reasoning)9898989898989898.2%
Claude Opus 4.8 (Reasoning)9898989898989898.2%
Claude Opus 4.69898989898989898.2%
GPT-5.4 Mini (Reasoning)9898989898989898.2%
Gemma 4 31B (Reasoning)9898989898989898.2%
Claude Sonnet 49898989898989898.2%
Gemma 4 31B9898989898989898.2%
Grok 4.5 (Reasoning, High)10098989898969698.0%
Grok 4.20 (Reasoning)9898989898989698.0%
Qwen 3.5 122B9898989898989698.0%
Qwen 3.5 27B9898989898989698.0%
Z.AI GLM 4.69898989898989698.0%
Gemini 3.5 Flash Lite (Reasoning)9898989898989698.0%
Gemini 2.5 Flash Lite (Reasoning)1001001009898969398.0%
GPT-5.6 Terra (Reasoning)9898989898989698.0%
Thinking Machines Inkling100100989898969397.7%
DeepSeek V4 Pro (Reasoning)100100989896969597.7%
MiniMax M2.59898989898969697.7%
Aion 2.09898989898969697.7%
Claude Haiku 4.510098989896969697.7%
Muse Spark 1.1 (Reasoning, Medium)10098989696969697.5%
Claude Sonnet 4.6 (Reasoning)10098989696969697.5%
GPT-5.19898989898969597.5%
Gemini 2.5 Pro9898989896969697.5%
GPT-5.49898989896969697.5%
GPT-5.6 Sol (Reasoning)9898989896969697.5%
DeepSeek V4 Flash (Reasoning)10098989696969697.5%
Gemini 3.5 Flash Lite (Reasoning, Minimal)9898989896969697.5%
Z.AI GLM 5.2 (Reasoning, High)9898989696969697.2%
Gemini 3.5 Flash (Reasoning)9898989696969697.2%
MoonshotAI: Kimi K2.510098969696969697.2%
GPT-5.6 Sol9898989696969697.2%
DeepSeek V4 Pro9898989696969697.2%
Muse Spark 1.1 (Reasoning, Minimal)10098969696969697.2%
Claude Opus 4.79898989896969597.2%
Qwen 3.6 35B9898989696969697.2%
Claude Sonnet 4.59898989696969697.2%
MoonshotAI: Kimi K3 (Reasoning, High)9898969696969697.0%
MiniMax M39898989696969597.0%
Claude Opus 59898989898959397.0%
GPT-OSS 120B9898989696969597.0%
DeepSeek V3.19898969696969697.0%
Claude Opus 4.8 (Reasoning, Low)9898989898968996.7%
Qwen3.7 Max9896969696969696.7%
MoonshotAI: Kimi K3 (Reasoning, Low)9896969696969696.7%
Xiaomi MIMO v2.5 Pro9898969696969596.7%
Qwen 3.5 Flash9896969696969696.7%
Gemma 4 26B9896969696969696.7%
GPT-4.1 Mini9898989696969396.7%
Hermes 3 405B9896969696969696.7%
Z.AI GLM 4.5 Air9896969696969696.7%
Gemma 4 26B (Reasoning)9898969696969596.7%
Qwen 3.5 35B9898969696969596.7%
Gemini 3.6 Flash (Reasoning)9696969696969696.5%
Gemini 3.1 Pro (Preview)9696969696969696.5%
Z.AI GLM 5.19696969696969696.5%
MoonshotAI: Kimi K2.69696969696969696.5%
Qwen 3.5 397B A17B9696969696969696.5%
Claude Sonnet 4.69696969696969696.5%
Claude Sonnet 5 (Reasoning)9696969696969696.5%
Claude Opus 4.59696969696969696.5%
Qwen 3.6 Flash9896969696969596.5%
Claude Sonnet 59696969696969696.5%
Gemini 3.6 Flash (Reasoning, Minimal)9696969696969696.5%
MiniMax M2.79898969696969396.5%
Qwen 3.5 Plus (2026-02-15)9696969696969696.5%
ByteDance Seed 2.0 Lite9696969696969696.5%
GPT-5.4 Mini (Reasoning, Low)9898969696959596.5%
DeepSeek V3.29696969696969696.5%
GPT-4o, Aug. 6th (temp=0)9696969696969696.5%
GPT-4o Mini (temp=1)9696969696969696.5%
GPT-4o Mini (temp=0)9696969696969696.5%
Hermes 3 70B9696969696969596.2%
GPT-5 Mini9898969696959396.2%
Claude Sonnet 5 (Reasoning, Low)9696969696969596.2%
Gemini 3 Flash (Preview, Reasoning)9896969696969396.2%
Gemini 3.1 Flash Lite (Reasoning)9696969696969596.2%
GPT-5.4 Nano (Reasoning, Low)9696969696969596.2%
GPT-5.4 Nano9696969696969596.2%
Z.AI GLM 59696969696969396.0%
Xiaomi MIMO v2.510096969696959196.0%
Gemma 3 27B9896969695959596.0%
GPT-4.19696969695959595.7%
Gemini 2.5 Flash Lite9696969695959595.7%
Laguna XS 2.110096969595959395.7%
Qwen 2.5 72B9696969595959595.5%
Gemini 3.1 Flash Lite9696969695959395.5%
Z.AI GLM 4.7 Flash9896969595959395.5%
GPT-5.4 Nano (Reasoning)9696969696959195.5%
Gemini 3 Flash (Preview)9696969595959395.2%
Gemini 3.5 Flash (Reasoning, Minimal)9896959595939395.0%
WizardLM 2 8x22b10098969693918995.0%
DeepSeek V4 Flash9696969696958895.0%
Claude Opus 5 (Reasoning, Low)9695959595959394.7%
Gemini 3.1 Flash Lite (Preview)9595959595959594.7%
Gemini 2.5 Flash (Reasoning)9696959595939394.7%
GPT-4o, Aug. 6th (temp=1)9896959595919194.5%
Cydonia 24B V4.19695959595939394.5%
Claude Opus 5 (Reasoning)9695959595939194.2%
o4 Mini High9896959595918994.2%
Mistral Small 3.2 24B9595939393939393.5%
GPT-4.1 Nano9695959393918993.2%
Mistral Large 39393939393939393.0%
Mistral Large 29393939393939393.0%
Inception Mercury 210098989696956392.5%
DeepSeek V3 (2024-12-26)9393939393918992.2%
o4 Mini9595939391898892.0%
DeepSeek-V2 Chat9393939191919192.0%
DeepSeek V3 (2025-03-24)9393919191918991.5%
Qwen 3.5 9B9898989696965391.0%
Mistral Medium 3.19191888888888888.7%
ByteDance Seed 2.0 Mini9593938986827587.7%
Grok 4.209189898888867987.2%
Grok 4.3100100989898674987.2%
Qwen 3 32B9693939184776886.2%
Llama 3.1 70B9389848484828185.5%
Gemini 2.5 Flash9898989898951185.2%
Aion 3.09898989696951184.7%
Qwen 3.6 27B10098989898722684.5%
Mistral Small 4 (Reasoning)9896939393892383.7%
Gemma 3 12B9696959595951183.2%
ByteDance Seed 1.6 Flash9393888481812677.9%
Aion 3.0 Mini1009896969653077.2%
Mistral Small 49593939389403076.2%
Nemotron 3 Super1001001009896181174.7%
Nemotron 3 Nano9898969389141271.7%
Ministral 3 3B8984707067635871.7%
Mistral NeMO7775727070686771.4%
Qwen3 235B A22B Instruct 25078281796765605870.2%
Ministral 3B8989796358494767.9%
Writer: Palmyra X57467656161605863.7%
Ministral 3 14B6363636361585661.2%
Arcee AI: Trinity Mini96827979777460.7%
Laguna S 2.19696969311111159.1%
Ministral 8B5351474444444246.4%
Ministral 3 8B5449464444424045.6%
Cohere Command R+ (Aug. 2024)891612420017.5%
Gemma 3 4B1111111111111110.5%
Model # 1 # 2 # 3 # 4 # 5 # 6 # 7 Avg ▼
GPT-5.6 Sol (Reasoning)100100100100100100100100.0%
Claude Opus 4.6 (Reasoning)100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning)100100100100100100100100.0%
Qwen3.7 Max100100100100100100100100.0%
Grok 4.5 (Reasoning, High)100100100100100100100100.0%
Gemini 3.1 Pro (Preview)100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Medium)100100100100100100100100.0%
Z.AI GLM 5.1100100100100100100100100.0%
Qwen3.6 Max Preview100100100100100100100100.0%
GPT-5.5 (Reasoning)100100100100100100100100.0%
Z.AI GLM 5.2 (Reasoning, High)100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning)100100100100100100100100.0%
Z.AI GLM 5 Turbo100100100100100100100100.0%
MoonshotAI: Kimi K2.6100100100100100100100100.0%
GPT-5.5 (Reasoning, Low)100100100100100100100100.0%
GPT-5.6 Terra (Reasoning)100100100100100100100100.0%
Claude Opus 4.6100100100100100100100100.0%
Claude Opus 4.8 (Reasoning, Low)100100100100100100100100.0%
GPT-5100100100100100100100100.0%
Muse Spark 1.1 (Reasoning, Minimal)100100100100100100100100.0%
Qwen 3.5 397B A17B100100100100100100100100.0%
Grok 4.5 (Reasoning, Low)100100100100100100100100.0%
Grok 4.20 (Reasoning)100100100100100100100100.0%
MoonshotAI: Kimi K2.5100100100100100100100100.0%
GPT-5.6 Sol100100100100100100100100.0%
GPT-5.1100100100100100100100100.0%
MiniMax M3100100100100100100100100.0%
Claude Sonnet 5 (Reasoning)100100100100100100100100.0%
Qwen 3.5 122B100100100100100100100100.0%
Claude Sonnet 5 (Reasoning, Low)100100100100100100100100.0%
Qwen 3.5 27B100100100100100100100100.0%
Gemini 3 Flash (Preview, Reasoning)100100100100100100100100.0%
Claude Opus 4.5100100100100100100100100.0%
Z.AI GLM 5100100100100100100100100.0%
GPT-5.2100100100100100100100100.0%
GPT-5.5100100100100100100100100.0%
Gemma 4 26B (Reasoning)100100100100100100100100.0%
Aion 3.0100100100100100100100100.0%
Gemini 2.5 Pro100100100100100100100100.0%
Qwen 3.6 27B100100100100100100100100.0%
Z.AI GLM 4.7100100100100100100100100.0%
Qwen 3.6 35B100100100100100100100100.0%
Z.AI GLM 4.6100100100100100100100100.0%
Claude Sonnet 4100100100100100100100100.0%
Claude Sonnet 4.5100100100100100100100100.0%
Claude Sonnet 5100100100100100100100100.0%
Claude Opus 4100100100100100100100100.0%
Gemini 3.6 Flash (Reasoning, Minimal)100100100100100100100100.0%
Qwen 3.5 Plus (2026-02-15)100100100100100100100100.0%
Gemini 3.1 Flash Lite (Reasoning)100100100100100100100100.0%
Gemini 3.5 Flash (Reasoning, Minimal)100100100100100100100100.0%
Gemini 3 Flash (Preview)100100100100100100100100.0%
Gemini 3.1 Flash Lite (Preview)100100100100100100100100.0%
Gemma 4 31B100100100100100100100100.0%
Gemini 3.1 Flash Lite100100100100100100100100.0%
Z.AI GLM 4.5100100100100100100100100.0%
Gemma 4 26B100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning, Minimal)100100100100100100100100.0%
Gemini 3.5 Flash Lite (Reasoning)100100100100100100100100.0%
Mistral Large 3100100100100100100100100.0%
ByteDance Seed 2.0 Lite100100100100100100100100.0%
Gemini 2.5 Flash (Reasoning)100100100100100100100100.0%
DeepSeek-V2 Chat100100100100100100100100.0%
Aion 3.0 Mini100100100100100100100100.0%
Claude Haiku 4.5100100100100100100100100.0%
DeepSeek V3 (2024-12-26)100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=0)100100100100100100100100.0%
DeepSeek V4 Flash100100100100100100100100.0%
Nemotron 3 Super100100100100100100100100.0%
Mistral Large 2100100100100100100100100.0%
GPT-4o, Aug. 6th (temp=1)100100100100100100100100.0%
Hermes 3 405B100100100100100100100100.0%
Z.AI GLM 4.5 Air100100100100100100100100.0%
Gemini 2.5 Flash100100100100100100100100.0%
Gemini 2.5 Flash Lite100100100100100100100100.0%
Writer: Palmyra X5100100100100100100100100.0%
Qwen3 235B A22B Instruct 2507100100100100100100100100.0%
Grok 4.3100100100100100100100100.0%
GPT-4o Mini (temp=1)100100100100100100100100.0%
Mistral Small 3.2 24B100100100100100100100100.0%
GPT-4o Mini (temp=0)100100100100100100100100.0%
Thinking Machines Inkling100100100100100100100100.0%
WizardLM 2 8x22b100100100100100100100100.0%
Hermes 3 70B100100100100100100100100.0%
MoonshotAI: Kimi K3 (Reasoning, Low)1001001001001001009899.7%
Grok 4.3 (Reasoning)1001001001001001009899.7%
ByteDance Seed 1.61001001001001001009899.7%
Qwen 3.6 Flash1001001001001001009899.7%
o4 Mini High1001001001001001009899.7%
DeepSeek V4 Flash (Reasoning)1001001001001001009899.7%
GPT-4.11001001001001001009899.7%
MiniMax M2.71001001001001001009899.7%
GPT-OSS 120B1001001001001001009899.7%
Xiaomi MIMO v2.51001001001001001009899.7%
DeepSeek V3.21001001001001001009899.7%
Z.AI GLM 4.7 Flash1001001001001001009899.7%
DeepSeek V3 (2025-03-24)1001001001001001009899.7%
Llama 3.1 70B1001001001001001009899.7%
ByteDance Seed 1.6 Flash1001001001001001009899.7%
MoonshotAI: Kimi K3 (Reasoning, High)100100100100100989899.3%
Claude Opus 4.8 (Reasoning)100100100100100989899.3%
Claude Sonnet 4.6100100100100100989899.3%
GPT-5.4 Mini (Reasoning)100100100100100989899.3%
Qwen 3.5 Plus (2026-04-20)100100100100100989899.3%
DeepSeek V4 Pro (Reasoning)1001001001001001009599.3%
Qwen 3.5 35B1001001001001001009599.3%
Aion 2.0100100100100100989899.3%
o4 Mini100100100100100989899.3%
DeepSeek V3.11001001001001001009599.3%
Claude Opus 5 (Reasoning, Low)10010010010098989899.0%
GPT-5.4 (Reasoning, Low)10010010010098989899.0%
Thinking Machines Inkling (Reasoning)100100100100100989599.0%
Claude Opus 4.710010010010098989899.0%
ByteDance Seed 2.0 Mini10010010010098989899.0%
GPT-5.410010010010098989899.0%
GPT-5 Nano10010010010098989899.0%
Laguna XS 2.110010010010098989899.0%
GPT-5 Mini10010010010098989598.7%
Claude Opus 51001001009898989898.7%
Xiaomi MIMO v2.5 Pro10010010010098989598.7%
Qwen 3.5 Flash100100100100100959598.7%
Inception Mercury 21001001009898989898.7%
Gemma 3 27B10010010010098989598.7%
Qwen 2.5 72B1001001009898989898.7%
Cydonia 24B V4.1100100100100100989398.7%
GPT-5.4 (Reasoning)1001001009898989598.3%
Claude Sonnet 4.6 (Reasoning)100100989898989898.3%
Claude Opus 5 (Reasoning)100100989898989898.3%
Gemini 2.5 Flash Lite (Reasoning)100100989898989898.3%
DeepSeek V4 Pro100100989898989898.3%
GPT-5.6 Terra1001001001001001008698.0%
Grok 4.2010098989898989898.0%
Mistral Small 410098989898989898.0%
Claude Opus 4.7 (Reasoning)9898989898989897.7%
Gemma 4 31B (Reasoning)10098989898989597.7%
GPT-5.6 Luna (Reasoning)100100989895959597.3%
Laguna S 2.19898989898989597.3%
GPT-4.1 Mini9898989898959597.0%
Mistral Small 4 (Reasoning)100100989895958696.0%
Mistral Medium 3.19595959595959595.3%
Mistral NeMO9895959595939395.0%
GPT-5.4 Nano (Reasoning)9895959593939394.7%
GPT-5.4 Nano (Reasoning, Low)9895959593939194.4%
Ministral 3 14B9595939393939393.7%
Gemma 3 12B9893939393939393.7%
GPT-5.6 Luna9893939393938893.0%
GPT-5.4 Nano9595959593888893.0%
GPT-4.1 Nano9393939191919191.7%
Arcee AI: Trinity Mini10095959393936090.0%
Qwen 3.5 9B100100989898983389.0%
Ministral 3 8B9393939388887288.7%
Ministral 8B9593939388726786.0%
MiniMax M2.5100100100989898285.0%
Qwen 3 32B9898989898121273.1%
Nemotron 3 Nano10095959547442872.1%
GPT-5.4 Mini (Reasoning, Low)7470676765605165.1%
GPT-5.4 Mini7065656563636364.8%
Gemma 3 4B6735353333302837.2%
Ministral 3B3737373737282634.2%
Cohere Command R+ (Aug. 2024)6037373328161432.2%
Ministral 3 3B3737333330282832.2%