Precision

Test: Codex Extraction

Avg. Score

95.7%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Gemini 3.1 Flash Lite (Preview)	100.0%	$0.0017	2.0s	100%
2	Ministral 3 8B	99.5%	$0.0006	3.3s	97%
3	Gemini 3 Flash (Preview)	99.7%	$0.0027	3.9s	97%
4	Qwen 3.5 Plus (2026-02-15)	100.0%	$0.0030	10.6s	100%
5	GPT-4.1 Mini	99.5%	$0.0015	6.6s	97%
6	LFM2 24B	99.5%	$0.0002	12.5s	97%
7	GPT-4o, Aug. 6th (temp=1)	100.0%	$0.0092	3.8s	100%
8	Gemini 2.5 Flash Lite	98.0%	$0.0005	1.9s	94%
9	GPT-4o, Aug. 6th (temp=0)	100.0%	$0.0100	4.0s	100%
10	Ministral 8B	98.2%	$0.0004	3.8s	94%
11	Mistral Medium 3.1	99.2%	$0.0026	5.8s	95%
12	Grok 4.1 Fast	100.0%	$0.0017	22.1s	100%
13	Gemini 2.5 Flash	98.5%	$0.0023	2.5s	93%
14	Ministral 3B	98.1%	$0.0002	1.8s	91%
15	Mistral Small 3.2 24B	98.3%	$0.0005	4.4s	92%
16	Mistral Small Creative	97.7%	$0.0006	3.9s	92%
17	Z.AI GLM 4.5	99.6%	$0.0028	16.8s	97%
18	Grok 4 Fast	98.2%	$0.0012	8.7s	94%
19	Grok 4.20 (Beta)	98.1%	$0.0049	2.0s	94%
20	DeepSeek V3 (2024-12-26)	98.8%	$0.0017	13.5s	94%
21	Hermes 3 405B	99.6%	$0.0040	17.8s	97%
22	Ministral 3 3B	96.4%	$0.0004	1.8s	90%
23	Z.AI GLM 5 Turbo	99.6%	$0.0068	16.0s	98%
24	Qwen 2.5 72B	98.1%	$0.0008	11.0s	92%
25	GPT-5.4 Mini	97.3%	$0.0031	2.2s	91%
26	DeepSeek-V2 Chat	98.4%	$0.0019	14.8s	94%
27	GPT-4o Mini (temp=1)	97.1%	$0.0006	8.0s	91%
28	GPT-4o Mini (temp=0)	97.2%	$0.0005	7.9s	90%
29	DeepSeek V3 (2025-03-24)	99.6%	$0.0014	27.0s	96%
30	MiniMax M2.7	99.4%	$0.0022	21.7s	95%
31	Claude 3 Haiku	97.6%	$0.0017	5.5s	89%
32	Writer: Palmyra X5	97.0%	$0.0052	7.5s	92%
33	Inception Mercury 2	95.4%	$0.0022	3.5s	89%
34	Qwen3 235B A22B Instruct 2507	97.7%	$0.0007	19.3s	92%
35	Hermes 3 70B	96.9%	$0.0012	15.8s	91%
36	Claude Haiku 4.5	97.5%	$0.0073	4.5s	91%
37	GPT-5.4 Mini (Reasoning, Low)	95.9%	$0.0042	3.7s	88%
38	Llama 3.1 Nemotron 70B	97.3%	$0.0050	17.3s	93%
39	Claude 3.7 Sonnet	100.0%	$0.021	7.8s	100%
40	Claude Sonnet 4.5	100.0%	$0.022	6.6s	100%
41	Claude Sonnet 4	100.0%	$0.022	7.9s	100%
42	Arcee AI: Trinity Mini	94.7%	$0.0003	5.8s	87%
43	Gemini 2.5 Flash Lite (Reasoning)	96.9%	$0.0021	15.6s	89%
44	Grok 4.20 (Beta, Reasoning)	100.0%	$0.020	12.7s	100%
45	MiniMax M2.5	98.3%	$0.0023	30.1s	94%
46	GPT-4o, May 13th (temp=0)	100.0%	$0.025	4.3s	100%
47	Gemma 3 27B	96.0%	$0.0005	14.5s	87%
48	Gemini 3 Flash (Preview, Reasoning)	98.9%	$0.0096	22.2s	95%
49	Stealth: Healer Alpha	96.2%	$0.0000	24.6s	90%
50	Claude Sonnet 4.6	99.7%	$0.022	7.4s	97%
51	Mistral Large 3	95.0%	$0.0027	8.2s	86%
52	DeepSeek V3.2	98.8%	$0.0011	36.7s	92%
53	GPT-5.4 (Reasoning, Low)	98.6%	$0.017	10.4s	94%
54	GPT-5.4 Nano (Reasoning)	95.1%	$0.0023	12.2s	87%
55	GPT-4.1	95.4%	$0.0073	4.7s	87%
56	WizardLM 2 8x22b	97.4%	$0.0026	31.3s	92%
57	Z.AI GLM 4.6	99.3%	$0.0057	38.8s	95%
58	o4 Mini	99.2%	$0.014	21.5s	95%
59	Ministral 3 14B	94.2%	$0.0009	6.0s	82%
60	GPT-5.4	95.4%	$0.011	6.4s	88%
61	Qwen 3 32B	97.8%	$0.0010	37.9s	90%
62	GPT-5.4 Mini (Reasoning)	99.4%	$0.017	25.6s	96%
63	Stealth: Hunter Alpha	96.5%	$0.0000	36.7s	90%
64	Mistral Small 4	92.9%	$0.0008	3.2s	79%
65	Mistral Large 2	95.0%	$0.011	8.3s	86%
66	Mistral Small 4 (Reasoning)	93.8%	$0.0019	15.6s	82%
67	GPT-4o, May 13th (temp=1)	97.3%	$0.025	4.0s	92%
68	Claude Opus 4.5	100.0%	$0.037	7.7s	100%
69	Z.AI GLM 5	98.8%	$0.0091	51.4s	94%
70	o4 Mini High	100.0%	$0.025	40.5s	100%
71	Qwen 3.5 Flash	96.4%	$0.0031	49.5s	89%
72	GPT-5.2	95.9%	$0.018	16.8s	87%
73	Gemma 3 4B	88.1%	$0.0002	6.7s	76%
74	Claude Opus 4.6	99.1%	$0.037	8.1s	96%
75	Nemotron 3 Super	96.5%	$0.0000	1.0m	90%
76	Grok 4	100.0%	$0.031	37.2s	100%
77	Claude 3.5 Sonnet	100.0%	$0.043	13.9s	100%
78	Inception Mercury	88.6%	$0.0006	3.9s	73%
79	GPT-5 Mini	95.9%	$0.0070	45.5s	88%
80	Nemotron 3 Nano	99.8%	$0.0013	1.6m	98%
81	Qwen 3.5 35B	97.0%	$0.015	47.3s	91%
82	Aion 2.0	98.8%	$0.0080	1.2m	94%
83	ByteDance Seed 1.6	97.9%	$0.0073	1.3m	93%
84	Gemini 2.5 Pro	97.4%	$0.034	22.8s	92%
85	ByteDance Seed 2.0 Lite	98.8%	$0.0078	1.4m	93%
86	Gemini 2.5 Flash (Reasoning)	90.8%	$0.0082	11.8s	71%
87	Z.AI GLM 4.7 Flash	95.5%	$0.0019	1.2m	87%
88	GPT-4.1 Nano	86.8%	$0.0003	2.8s	63%
89	GPT-5.4 Nano (Reasoning, Low)	83.4%	$0.0011	3.7s	68%
90	GPT-5 Nano	96.4%	$0.0043	1.3m	86%
91	Claude Opus 4.6 (Reasoning)	100.0%	$0.055	21.4s	100%
92	Cohere Command R+ (Aug. 2024)	91.3%	$0.014	17.9s	71%
93	Z.AI GLM 4.7	99.1%	$0.0098	1.7m	95%
94	Qwen 3.5 9B	95.1%	$0.0013	1.5m	87%
95	GPT-5.4 Nano	82.2%	$0.0011	3.8s	64%
96	Llama 3.1 70B	91.7%	$0.0016	16.0s	56%
97	Gemini 3 Pro (Preview)	100.0%	$0.055	35.6s	100%
98	DeepSeek V3.1	92.6%	$0.0012	26.1s	57%
99	GPT-5.1	96.4%	$0.035	43.6s	90%
100	GPT-5.4 (Reasoning)	99.7%	$0.044	53.6s	97%
101	Claude Sonnet 4.6 (Reasoning)	98.8%	$0.052	36.0s	94%
102	ByteDance Seed 1.6 Flash	86.3%	$0.0011	39.3s	66%
103	Qwen 3.5 397B A17B	97.6%	$0.012	1.8m	94%
104	Mistral Large	89.6%	$0.011	7.6s	53%
105	Llama 3.1 8B	84.5%	$0.0001	10.0s	51%
106	Arcee AI: Trinity Large (Preview)	89.3%	$0.0000	20.5s	40%
107	GPT-5	98.8%	$0.049	1.3m	96%
108	Gemma 3 12B	79.7%	$0.0003	14.1s	47%
109	MoonshotAI: Kimi K2.5	98.3%	$0.013	2.4m	94%
110	Gemini 3.1 Pro (Preview)	98.9%	$0.065	1.0m	95%
111	Claude Opus 4	100.0%	$0.110	13.5s	100%
112	ByteDance Seed 2.0 Mini	97.8%	$0.0034	3.4m	92%
113	Qwen 3.5 27B	88.9%	$0.021	1.6m	41%
114	Rocinante 12B	60.4%	$0.0013	25.6s	9%
115	Qwen 3.5 122B	97.0%	$0.079	3.6m	93%
116	Mistral NeMO	28.2%	$0.0006	1.4s	0%
95.73%

Individual Scenarios

▼

Long: The Spire of Echoes (Dense)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	96	99.3%
MoonshotAI: Kimi K2.5	100	100	100	100	96	99.3%
Qwen 3.5 27B	100	100	100	100	96	99.3%
Grok 4 Fast	100	100	100	100	96	99.3%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	96	99.3%
Gemini 2.5 Flash Lite	100	100	100	100	96	99.3%
Mistral Small Creative	100	100	100	100	96	99.3%
Claude 3 Haiku	100	100	100	100	96	99.3%
GPT-4.1 Mini	100	100	100	100	96	99.2%
GPT-5.4 Mini	100	100	100	100	96	99.2%
Ministral 3 8B	100	100	100	100	96	99.2%
Qwen 3 32B	100	100	100	100	95	99.1%
Arcee AI: Trinity Mini	100	100	100	100	95	99.1%
Nemotron 3 Nano	100	100	100	100	95	99.0%
GPT-4o Mini (temp=1)	100	100	100	100	95	99.0%
Z.AI GLM 4.5	100	100	100	100	93	98.6%
Gemini 3.1 Pro (Preview)	100	100	100	96	96	98.5%
Qwen 3.5 397B A17B	100	100	100	96	96	98.5%
Qwen 3.5 122B	100	100	100	96	96	98.5%
Gemini 2.5 Pro	100	100	100	96	96	98.5%
Qwen 3.5 Flash	100	100	100	96	96	98.5%
Z.AI GLM 4.7	100	100	100	96	96	98.5%
Ministral 3 3B	100	100	100	96	96	98.5%
Z.AI GLM 5 Turbo	100	100	100	96	96	98.5%
MiniMax M2.5	100	100	100	96	96	98.4%
Inception Mercury 2	100	100	100	100	91	98.3%
GPT-5.2	100	100	100	100	90	97.9%
Aion 2.0	100	100	100	100	90	97.9%
LFM2 24B	100	100	100	95	95	97.9%
Qwen 3.5 9B	100	100	100	96	93	97.8%
Ministral 3B	100	100	100	96	93	97.8%
ByteDance Seed 2.0 Mini	100	100	96	96	96	97.8%
Gemma 3 27B	100	100	96	96	96	97.8%
Stealth: Hunter Alpha	100	100	96	96	96	97.7%
GPT-4o, May 13th (temp=1)	100	100	100	100	88	97.7%
Mistral Small 4	100	100	100	96	92	97.7%
GPT-5.4 (Reasoning, Low)	100	100	100	96	90	97.2%
Llama 3.1 70B	100	100	96	95	94	97.1%
GPT-5.1	100	100	96	96	93	97.1%
Z.AI GLM 4.7 Flash	100	100	100	96	89	97.1%
Llama 3.1 8B	100	100	95	95	95	97.0%
Stealth: Healer Alpha	100	100	96	96	93	97.0%
Llama 3.1 Nemotron 70B	100	96	96	96	96	96.9%
Gemma 3 12B	100	100	96	96	92	96.9%
Nemotron 3 Super	100	96	96	96	96	96.8%
GPT-5	100	96	96	96	93	96.3%
Gemini 2.5 Flash (Reasoning)	100	96	96	96	93	96.3%
GPT-5.4	100	96	96	96	93	96.3%
Qwen 3.5 35B	100	100	93	93	93	95.7%
Ministral 3 14B	100	100	96	93	90	95.7%
Gemma 3 4B	100	95	95	95	91	95.5%
Cohere Command R+ (Aug. 2024)	100	100	96	92	90	95.5%
GPT-5.4 Nano (Reasoning)	96	96	96	93	93	94.8%
GPT-4.1	96	96	93	93	90	93.6%
GPT-5 Nano	100	100	100	96	69	93.0%
GPT-4.1 Nano	100	95	90	90	83	91.7%
Mistral Small 4 (Reasoning)	100	93	90	90	84	91.2%
Inception Mercury	100	95	84	81	81	88.5%
GPT-5.4 Nano (Reasoning, Low)	92	90	90	89	76	87.4%
ByteDance Seed 1.6 Flash	93	93	84	83	81	86.7%
GPT-5.4 Nano	93	89	87	84	73	84.9%
DeepSeek V3.1	100	100	100	100	0	80.0%
Mistral Large	100	100	100	100	0	80.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	0	80.0%
Rocinante 12B	100	100	92	0	0	58.3%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Medium: The Hollow (Inferred)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Mini	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
Nemotron 3 Super	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Inception Mercury 2	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Qwen 3 32B	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Nemotron 3 Nano	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	93	98.7%
GPT-5.2	100	100	100	100	93	98.7%
GPT-4.1	100	100	100	100	93	98.7%
Gemini 2.5 Pro	100	100	100	100	93	98.7%
GPT-5.4	100	100	100	100	93	98.7%
GPT-5.4 Nano (Reasoning)	100	100	100	100	93	98.7%
Mistral Large	100	100	100	100	93	98.7%
Mistral Small Creative	100	100	100	100	93	98.7%
Qwen 3.5 122B	100	100	100	100	93	98.6%
Z.AI GLM 4.7 Flash	100	100	100	100	93	98.6%
Grok 4.20 (Beta)	100	100	100	100	93	98.6%
Mistral Small 4	100	100	100	100	93	98.6%
Stealth: Healer Alpha	100	100	100	100	92	98.5%
ByteDance Seed 1.6 Flash	100	100	100	100	92	98.5%
Ministral 3 3B	100	100	100	100	90	98.0%
GPT-5 Mini	100	100	100	93	93	97.3%
GPT-5.4 (Reasoning, Low)	100	100	100	93	93	97.3%
Gemini 2.5 Flash Lite	100	100	100	93	93	97.2%
GPT-5.4 Nano	100	100	100	93	93	97.2%
Ministral 3 14B	100	100	100	93	93	97.2%
Ministral 8B	100	100	100	93	93	97.2%
Qwen 3.5 Flash	100	100	100	100	82	96.5%
GPT-4.1 Nano	100	100	100	100	81	96.3%
Arcee AI: Trinity Mini	100	100	100	91	90	96.2%
Mistral Small 4 (Reasoning)	100	100	100	93	88	96.2%
Inception Mercury	100	100	90	87	85	92.3%
GPT-5.4 Nano (Reasoning, Low)	100	93	93	92	81	92.0%
Cohere Command R+ (Aug. 2024)	93	92	92	91	85	90.5%
Gemini 2.5 Flash (Reasoning)	93	93	93	88	82	90.0%
Llama 3.1 8B	100	91	86	79	76	86.3%
Gemma 3 4B	91	91	85	83	79	85.7%
Arcee AI: Trinity Large (Preview)	100	100	100	100	0	80.0%
Gemma 3 12B	93	93	93	93	0	74.5%
Rocinante 12B	100	85	83	78	0	69.1%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Medium: Through the Thornveil (Scattered)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Mini	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Nemotron 3 Nano	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	94	98.8%
GPT-5.4 (Reasoning)	100	100	100	100	94	98.8%
GPT-5	100	100	100	100	94	98.8%
Claude Sonnet 4.6	100	100	100	100	94	98.8%
Aion 2.0	100	100	100	100	94	98.8%
DeepSeek-V2 Chat	100	100	100	100	94	98.8%
Nemotron 3 Super	100	100	100	100	94	98.8%
Z.AI GLM 4.6	100	100	100	100	93	98.7%
Stealth: Healer Alpha	100	100	100	100	93	98.7%
Gemini 3 Flash (Preview)	100	100	100	100	93	98.7%
GPT-4.1 Mini	100	100	100	100	93	98.7%
DeepSeek V3.1	100	100	100	100	93	98.7%
DeepSeek V3.2	100	100	100	100	93	98.7%
Arcee AI: Trinity Large (Preview)	100	100	100	100	93	98.7%
Ministral 3 8B	100	100	100	100	93	98.7%
Grok 4 Fast	100	100	100	100	93	98.6%
DeepSeek V3 (2024-12-26)	100	100	100	100	93	98.6%
Hermes 3 405B	100	100	100	100	93	98.6%
GPT-4o Mini (temp=1)	100	100	100	100	91	98.2%
MiniMax M2.7	100	100	100	100	88	97.6%
Qwen 3.5 122B	100	100	100	94	94	97.5%
GPT-5.4 Mini (Reasoning)	100	100	100	94	94	97.5%
GPT-5.2	100	100	100	94	94	97.5%
Gemini 2.5 Pro	100	100	100	100	88	97.5%
Mistral Small Creative	100	100	100	94	94	97.5%
Ministral 3 14B	100	100	100	94	94	97.5%
MoonshotAI: Kimi K2.5	100	100	100	94	93	97.4%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	93	93	97.3%
Grok 4.20 (Beta)	100	100	100	93	93	97.3%
Gemini 2.5 Flash Lite	100	100	100	94	93	97.3%
Writer: Palmyra X5	100	100	100	93	93	97.2%
Qwen 2.5 72B	100	100	100	93	92	97.0%
Ministral 8B	100	100	100	93	92	97.0%
Arcee AI: Trinity Mini	100	100	100	93	91	96.8%
ByteDance Seed 1.6	100	100	100	94	88	96.4%
GPT-5.4	100	100	100	94	88	96.4%
Claude Opus 4.6	100	100	94	94	94	96.3%
Qwen 3.5 27B	100	100	94	94	94	96.3%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	94	88	96.3%
Ministral 3B	100	100	100	100	81	96.3%
GPT-4.1	100	100	100	93	88	96.2%
Qwen3 235B A22B Instruct 2507	100	100	100	93	88	96.2%
Qwen 3 32B	100	100	100	100	79	95.8%
WizardLM 2 8x22b	100	100	93	93	88	94.9%
Stealth: Hunter Alpha	100	94	93	93	93	94.8%
Gemini 2.5 Flash	100	100	94	88	88	94.0%
Qwen 3.5 35B	100	100	94	94	82	94.0%
Llama 3.1 Nemotron 70B	100	93	92	92	92	94.0%
GPT-5 Mini	100	94	93	93	88	93.7%
GPT-5.1	94	94	94	94	93	93.7%
Qwen 3.5 397B A17B	94	94	93	93	93	93.5%
Inception Mercury 2	100	93	92	92	88	93.0%
Hermes 3 70B	100	92	92	91	90	93.0%
Mistral Small 4 (Reasoning)	100	100	88	88	88	92.8%
GPT-5 Nano	94	94	94	93	88	92.6%
Z.AI GLM 4.7 Flash	94	93	93	93	88	92.2%
GPT-5.4 Mini	100	94	94	88	83	91.8%
GPT-5.4 Mini (Reasoning, Low)	100	94	88	88	88	91.7%
Qwen 3.5 Flash	94	94	93	88	83	90.5%
GPT-5.4 Nano (Reasoning)	94	93	88	88	88	90.1%
Mistral Small 4	100	93	93	87	74	89.3%
Qwen 3.5 9B	94	94	93	83	82	89.3%
Ministral 3 3B	93	93	87	87	87	89.2%
Gemma 3 12B	93	88	88	88	88	88.8%
Mistral Large 3	88	88	88	88	88	88.2%
Mistral Large 2	88	88	88	88	88	88.2%
Mistral Large	88	88	88	88	88	88.2%
Inception Mercury	100	93	93	87	67	87.9%
Cohere Command R+ (Aug. 2024)	100	100	100	86	47	86.5%
Gemini 2.5 Flash (Reasoning)	100	100	100	79	47	85.2%
Gemma 3 4B	86	80	80	79	73	79.5%
Llama 3.1 70B	100	93	92	91	0	75.1%
Rocinante 12B	100	100	87	85	0	74.3%
ByteDance Seed 1.6 Flash	100	74	68	68	52	72.5%
GPT-5.4 Nano (Reasoning, Low)	83	72	71	67	65	71.8%
Llama 3.1 8B	100	100	80	69	0	69.8%
GPT-5.4 Nano	81	72	71	61	58	68.7%
GPT-4.1 Nano	83	72	70	67	41	66.6%
Mistral NeMO	100	90	83	0	0	54.7%

▼

Short: The Rusty Lantern (Explicit)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Nemotron 3 Nano	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
LFM2 24B	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	92	98.3%
Qwen 3.5 397B A17B	100	100	100	100	92	98.3%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	92	98.3%
Aion 2.0	100	100	100	100	92	98.3%
Z.AI GLM 4.6	100	100	100	100	92	98.3%
Qwen 3.5 35B	100	100	100	100	92	98.3%
GPT-5.4 Mini	100	100	100	100	92	98.3%
DeepSeek V3 (2025-03-24)	100	100	100	100	92	98.3%
Gemini 2.5 Flash Lite	100	100	100	100	92	98.3%
Llama 3.1 Nemotron 70B	100	100	100	100	92	98.3%
Arcee AI: Trinity Large (Preview)	100	100	100	100	92	98.3%
Ministral 8B	100	100	100	100	92	98.3%
Ministral 3B	100	100	100	100	92	98.3%
Z.AI GLM 4.7	100	100	100	100	90	98.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	85	96.9%
DeepSeek V3.2	100	100	100	100	83	96.7%
MoonshotAI: Kimi K2.5	100	100	100	92	92	96.7%
ByteDance Seed 1.6	100	100	100	92	92	96.7%
o4 Mini	100	100	100	92	92	96.7%
Grok 4.20 (Beta)	100	100	100	92	92	96.7%
DeepSeek V3 (2024-12-26)	100	100	100	92	92	96.7%
Mistral Medium 3.1	100	100	100	92	92	96.7%
Qwen 3 32B	100	100	100	92	91	96.5%
ByteDance Seed 2.0 Lite	100	100	100	92	85	95.3%
Qwen 2.5 72B	100	100	100	92	85	95.3%
Mistral Small Creative	100	100	100	92	85	95.3%
Claude Sonnet 4.6 (Reasoning)	100	100	92	92	92	95.0%
GPT-5.1	100	100	92	92	92	95.0%
Z.AI GLM 5	100	100	92	92	92	95.0%
Gemini 2.5 Pro	100	100	92	92	92	95.0%
Grok 4 Fast	100	100	92	92	92	95.0%
DeepSeek-V2 Chat	100	100	92	92	92	95.0%
Mistral Small 4 (Reasoning)	100	100	92	92	92	95.0%
MiniMax M2.5	100	100	92	92	91	94.8%
WizardLM 2 8x22b	100	100	92	92	91	94.8%
Qwen3 235B A22B Instruct 2507	100	100	91	91	91	94.5%
Hermes 3 70B	100	100	91	91	91	94.5%
Llama 3.1 70B	100	100	92	91	90	94.5%
Z.AI GLM 4.7 Flash	100	100	92	90	90	94.3%
ByteDance Seed 2.0 Mini	100	100	92	92	85	93.6%
Qwen 3.5 9B	100	100	92	91	85	93.4%
GPT-5 Mini	100	92	92	92	92	93.3%
Qwen 3.5 122B	100	92	92	92	92	93.3%
GPT-4.1	100	92	92	92	92	93.3%
Stealth: Hunter Alpha	100	92	92	92	92	93.3%
Mistral Small 3.2 24B	100	100	92	90	85	93.3%
GPT-4.1 Nano	100	100	100	89	75	92.8%
Cohere Command R+ (Aug. 2024)	100	92	91	91	90	92.7%
GPT-5.4 Mini (Reasoning, Low)	100	100	91	85	85	92.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	92	85	83	91.9%
Mistral Large 3	92	92	92	92	92	91.7%
GPT-4o, May 13th (temp=1)	92	92	92	92	92	91.7%
Mistral Large 2	92	92	92	92	92	91.7%
DeepSeek V3.1	92	92	92	92	92	91.7%
Mistral Large	92	92	92	92	92	91.7%
Gemini 2.5 Flash (Reasoning)	100	100	100	85	73	91.6%
Gemma 3 4B	100	90	90	89	89	91.6%
GPT-4o Mini (temp=1)	100	89	89	89	89	91.1%
Claude 3 Haiku	100	100	92	85	79	91.0%
Writer: Palmyra X5	91	91	91	91	91	90.9%
Stealth: Healer Alpha	100	92	92	85	85	90.5%
Inception Mercury 2	100	92	92	85	85	90.5%
Nemotron 3 Super	92	92	92	92	85	90.3%
GPT-5.4	92	92	92	92	85	90.3%
Claude Haiku 4.5	92	92	92	92	83	90.0%
GPT-5.2	100	100	85	85	79	89.6%
GPT-4o Mini (temp=0)	89	89	89	89	89	88.9%
ByteDance Seed 1.6 Flash	100	100	91	79	69	87.6%
Arcee AI: Trinity Mini	100	89	83	82	80	86.8%
Gemma 3 27B	92	92	85	85	79	86.2%
Ministral 3 14B	92	92	85	85	79	86.2%
Mistral Small 4	92	89	85	85	80	86.0%
Inception Mercury	100	92	90	75	71	85.6%
Llama 3.1 8B	90	90	82	82	80	84.7%
GPT-5.4 Nano (Reasoning, Low)	92	85	85	79	73	82.6%
GPT-5.4 Nano	85	85	73	73	73	77.8%
Qwen 3.5 27B	100	100	100	0	0	60.0%
Gemma 3 12B	67	60	56	56	56	58.8%
Mistral NeMO	100	100	90	0	0	58.0%
Rocinante 12B	100	100	0	0	0	40.0%

Precision

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Long: The Spire of Echoes (Dense)

Medium: The Hollow (Inferred)

Medium: Through the Thornveil (Scattered)

Short: The Rusty Lantern (Explicit)