Structural validity

Test: Codex Extraction

Avg. Score

96.8%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Inception Mercury	99.9%	$0.0006	3.9s	99%
2	Inception Mercury 2	100.0%	$0.0022	3.5s	100%
3	Gemini 3.1 Flash Lite (Preview)	99.9%	$0.0017	2.0s	99%
4	Gemini 2.5 Flash	99.8%	$0.0023	2.5s	99%
5	Ministral 3 8B	99.6%	$0.0006	3.3s	98%
6	Mistral Small 3.2 24B	99.5%	$0.0005	4.4s	98%
7	Grok 4.20 (Beta)	100.0%	$0.0049	2.0s	100%
8	GPT-5.4 Mini (Reasoning, Low)	100.0%	$0.0042	3.7s	100%
9	Ministral 3 3B	99.1%	$0.0004	1.8s	97%
10	GPT-5.4 Mini	99.6%	$0.0031	2.2s	98%
11	GPT-4.1 Mini	99.6%	$0.0015	6.6s	98%
12	Grok 4 Fast	99.7%	$0.0012	8.7s	98%
13	DeepSeek V3 (2024-12-26)	100.0%	$0.0017	13.5s	100%
14	Ministral 3B	98.3%	$0.0002	1.8s	95%
15	Claude Haiku 4.5	100.0%	$0.0073	4.5s	100%
16	DeepSeek-V2 Chat	99.9%	$0.0019	14.8s	99%
17	Qwen 3.5 Plus (2026-02-15)	99.8%	$0.0030	10.6s	99%
18	Gemma 3 27B	99.4%	$0.0005	14.5s	97%
19	GPT-4o, Aug. 6th (temp=1)	99.8%	$0.0092	3.8s	99%
20	Stealth: Healer Alpha	100.0%	$0.0000	24.6s	100%
21	Grok 4.1 Fast	100.0%	$0.0017	22.1s	100%
22	Mistral Medium 3.1	98.6%	$0.0026	5.8s	95%
23	GPT-4o Mini (temp=1)	98.0%	$0.0006	8.0s	95%
24	GPT-4o, Aug. 6th (temp=0)	99.7%	$0.0100	4.0s	99%
25	Gemini 2.5 Flash (Reasoning)	100.0%	$0.0082	11.8s	100%
26	MiniMax M2.7	99.9%	$0.0022	21.7s	99%
27	GPT-4o Mini (temp=0)	98.3%	$0.0005	7.9s	93%
28	Gemini 3 Flash (Preview)	98.2%	$0.0027	3.9s	93%
29	Mistral Large 3	98.3%	$0.0027	8.2s	95%
30	DeepSeek V3 (2025-03-24)	100.0%	$0.0014	27.0s	100%
31	Z.AI GLM 4.5	99.5%	$0.0028	16.8s	96%
32	Mistral Small 4 (Reasoning)	98.7%	$0.0019	15.6s	95%
33	Claude 3 Haiku	97.3%	$0.0017	5.5s	92%
34	Z.AI GLM 5 Turbo	99.8%	$0.0068	16.0s	98%
35	GPT-5.4 Nano	98.1%	$0.0011	3.8s	89%
36	Hermes 3 405B	98.9%	$0.0040	17.8s	97%
37	Stealth: Hunter Alpha	100.0%	$0.0000	36.7s	100%
38	MiniMax M2.5	99.7%	$0.0023	30.1s	99%
39	Qwen 2.5 72B	97.0%	$0.0008	11.0s	92%
40	WizardLM 2 8x22b	99.9%	$0.0026	31.3s	99%
41	DeepSeek V3.2	100.0%	$0.0011	36.7s	100%
42	Mistral Small 4	96.4%	$0.0008	3.2s	89%
43	Mistral Large 2	98.9%	$0.011	8.3s	96%
44	Gemma 3 4B	95.9%	$0.0002	6.7s	90%
45	Gemini 3 Flash (Preview, Reasoning)	100.0%	$0.0096	22.2s	100%
46	Arcee AI: Trinity Mini	95.8%	$0.0003	5.8s	90%
47	Gemini 2.5 Flash Lite (Reasoning)	97.7%	$0.0021	15.6s	92%
48	GPT-5.4	98.2%	$0.011	6.4s	95%
49	GPT-5.4 (Reasoning, Low)	99.8%	$0.017	10.4s	98%
50	o4 Mini	100.0%	$0.014	21.5s	100%
51	Claude Sonnet 4.5	100.0%	$0.022	6.6s	100%
52	Writer: Palmyra X5	96.7%	$0.0052	7.5s	90%
53	Z.AI GLM 4.6	100.0%	$0.0057	38.8s	100%
54	Claude 3.7 Sonnet	100.0%	$0.021	7.8s	100%
55	Claude Sonnet 4	100.0%	$0.022	7.9s	100%
56	Claude Sonnet 4.6	100.0%	$0.022	7.4s	100%
57	GPT-5.4 Nano (Reasoning, Low)	96.6%	$0.0011	3.7s	83%
58	Qwen 3 32B	98.9%	$0.0010	37.9s	96%
59	GPT-5.2	100.0%	$0.018	16.8s	100%
60	Grok 4.20 (Beta, Reasoning)	100.0%	$0.020	12.7s	100%
61	Ministral 8B	96.0%	$0.0004	3.8s	83%
62	GPT-4.1 Nano	94.1%	$0.0003	2.8s	85%
63	Hermes 3 70B	97.1%	$0.0012	15.8s	87%
64	Qwen 3.5 Flash	100.0%	$0.0031	49.5s	100%
65	GPT-4.1	96.6%	$0.0073	4.7s	87%
66	GPT-5.4 Mini (Reasoning)	100.0%	$0.017	25.6s	100%
67	Gemini 2.5 Flash Lite	94.2%	$0.0005	1.9s	82%
68	Mistral Small Creative	94.3%	$0.0006	3.9s	83%
69	GPT-5 Mini	100.0%	$0.0070	45.5s	100%
70	Qwen3 235B A22B Instruct 2507	96.2%	$0.0007	19.3s	85%
71	ByteDance Seed 1.6 Flash	97.3%	$0.0011	39.3s	93%
72	GPT-4o, May 13th (temp=0)	99.5%	$0.025	4.3s	95%
73	Cohere Command R+ (Aug. 2024)	97.7%	$0.014	17.9s	94%
74	GPT-5.4 Nano (Reasoning)	97.3%	$0.0023	12.2s	81%
75	Nemotron 3 Super	99.8%	$0.0000	1.0m	99%
76	Z.AI GLM 5	100.0%	$0.0091	51.4s	100%
77	GPT-4o, May 13th (temp=1)	98.4%	$0.025	4.0s	94%
78	Qwen 3.5 35B	99.9%	$0.015	47.3s	99%
79	Claude Opus 4.5	100.0%	$0.037	7.7s	100%
80	Claude Opus 4.6	100.0%	$0.037	8.1s	100%
81	o4 Mini High	100.0%	$0.025	40.5s	100%
82	Gemini 2.5 Pro	100.0%	$0.034	22.8s	100%
83	Aion 2.0	100.0%	$0.0080	1.2m	100%
84	Ministral 3 14B	90.7%	$0.0009	6.0s	75%
85	ByteDance Seed 1.6	100.0%	$0.0073	1.3m	100%
86	Z.AI GLM 4.7 Flash	98.7%	$0.0019	1.2m	94%
87	Claude 3.5 Sonnet	100.0%	$0.043	13.9s	100%
88	GPT-5 Nano	99.5%	$0.0043	1.3m	96%
89	Grok 4	100.0%	$0.031	37.2s	100%
90	Nemotron 3 Nano	99.9%	$0.0013	1.6m	99%
91	ByteDance Seed 2.0 Lite	99.6%	$0.0078	1.4m	98%
92	GPT-5.1	100.0%	$0.035	43.6s	100%
93	Z.AI GLM 4.7	100.0%	$0.0098	1.7m	100%
94	DeepSeek V3.1	95.5%	$0.0012	26.1s	61%
95	Claude Opus 4.6 (Reasoning)	100.0%	$0.055	21.4s	100%
96	Gemma 3 12B	89.8%	$0.0003	14.1s	59%
97	Qwen 3.5 397B A17B	100.0%	$0.012	1.8m	100%
98	Llama 3.1 70B	90.6%	$0.0016	16.0s	59%
99	Claude Sonnet 4.6 (Reasoning)	100.0%	$0.052	36.0s	100%
100	GPT-5.4 (Reasoning)	100.0%	$0.044	53.6s	100%
101	Gemini 3 Pro (Preview)	100.0%	$0.055	35.6s	100%
102	Mistral Large	93.4%	$0.011	7.6s	56%
103	Qwen 3.5 9B	95.6%	$0.0013	1.5m	79%
104	Arcee AI: Trinity Large (Preview)	89.5%	$0.0000	20.5s	46%
105	GPT-5	100.0%	$0.049	1.3m	100%
106	MoonshotAI: Kimi K2.5	100.0%	$0.013	2.4m	100%
107	Llama 3.1 Nemotron 70B	81.0%	$0.0050	17.3s	55%
108	Llama 3.1 8B	78.9%	$0.0001	10.0s	48%
109	Gemini 3.1 Pro (Preview)	99.5%	$0.065	1.0m	96%
110	Qwen 3.5 27B	95.0%	$0.021	1.6m	70%
111	ByteDance Seed 2.0 Mini	98.2%	$0.0034	3.4m	95%
112	Claude Opus 4	100.0%	$0.110	13.5s	100%
113	Rocinante 12B	51.3%	$0.0013	25.6s	16%
114	Qwen 3.5 122B	100.0%	$0.079	3.6m	100%
115	LFM2 24B	43.2%	$0.0002	12.5s	9%
116	Mistral NeMO	30.0%	$0.0006	1.4s	0%
96.78%

Individual Scenarios

▼

Long: The Spire of Echoes (Dense)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
GPT-4.1	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
Nemotron 3 Super	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
Inception Mercury 2	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Nemotron 3 Nano	100	100	100	100	100	100.0%
GPT-5.4 Nano	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	99	99.8%
Grok 4	100	100	100	100	99	99.8%
Z.AI GLM 4.5	100	100	100	100	99	99.8%
Claude 3.7 Sonnet	100	100	100	100	99	99.8%
DeepSeek V3.2	100	100	100	100	99	99.8%
DeepSeek V3 (2025-03-24)	100	100	100	100	99	99.8%
Mistral Small 4 (Reasoning)	100	100	100	100	98	99.7%
Gemma 3 27B	100	100	100	100	98	99.6%
MiniMax M2.7	100	100	100	100	98	99.6%
Gemini 2.5 Flash	100	100	100	100	98	99.6%
DeepSeek-V2 Chat	100	100	100	99	99	99.6%
MiniMax M2.5	100	100	100	100	98	99.6%
Inception Mercury	100	100	100	99	99	99.6%
GPT-4o, Aug. 6th (temp=1)	100	100	100	99	99	99.6%
WizardLM 2 8x22b	100	100	100	100	97	99.4%
Qwen 3 32B	100	100	100	99	98	99.3%
Gemini 2.5 Flash Lite	100	100	99	99	98	99.2%
Mistral Small 3.2 24B	100	99	99	99	99	99.2%
Ministral 3 3B	100	100	99	98	98	99.0%
GPT-4o, Aug. 6th (temp=0)	99	99	99	99	99	98.9%
Qwen 2.5 72B	100	100	99	99	95	98.5%
Ministral 3 8B	100	100	98	98	96	98.4%
Ministral 8B	100	99	99	98	96	98.3%
GPT-4o Mini (temp=1)	100	99	99	97	96	98.1%
Mistral Large 2	99	99	99	99	94	98.1%
Gemini 3.1 Pro (Preview)	100	100	100	100	90	98.0%
LFM2 24B	100	100	99	97	94	97.9%
Mistral Small 4	100	100	98	97	94	97.9%
Z.AI GLM 4.7 Flash	100	100	100	98	91	97.8%
Ministral 3B	100	98	98	97	96	97.8%
Hermes 3 405B	99	99	99	97	95	97.7%
Llama 3.1 70B	100	99	99	96	94	97.5%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	99	88	97.5%
Gemma 3 12B	100	100	99	99	89	97.5%
GPT-4o Mini (temp=0)	100	100	96	96	94	97.4%
Writer: Palmyra X5	100	98	98	96	95	97.4%
Mistral Small Creative	99	96	96	96	96	96.8%
ByteDance Seed 2.0 Mini	99	98	96	95	94	96.6%
Llama 3.1 Nemotron 70B	98	98	97	95	94	96.3%
Claude 3 Haiku	100	98	96	95	92	96.3%
ByteDance Seed 1.6 Flash	99	99	96	94	93	96.2%
Cohere Command R+ (Aug. 2024)	98	98	97	96	93	96.2%
Arcee AI: Trinity Mini	99	99	95	95	92	95.9%
Qwen3 235B A22B Instruct 2507	96	96	96	96	95	95.6%
Hermes 3 70B	100	99	99	98	83	95.6%
Gemma 3 4B	100	96	96	95	90	95.4%
Mistral Large 3	98	98	93	93	93	95.2%
Ministral 3 14B	99	96	95	92	92	94.9%
GPT-4.1 Nano	100	98	98	92	80	93.6%
Qwen 3.5 9B	100	100	100	99	53	90.4%
DeepSeek V3.1	100	100	100	100	10	82.0%
Arcee AI: Trinity Large (Preview)	99	99	99	87	10	78.8%
Mistral Large	99	98	93	93	0	76.7%
Llama 3.1 8B	94	91	88	84	25	76.5%
Rocinante 12B	96	89	79	0	0	52.8%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Medium: The Hollow (Inferred)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Nemotron 3 Super	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
Inception Mercury 2	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
GPT-5.4 Mini	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Inception Mercury	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Nemotron 3 Nano	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	98	99.6%
GPT-4o, May 13th (temp=1)	100	100	100	100	98	99.6%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	98	99.6%
Qwen 3.5 35B	100	100	100	100	98	99.6%
Gemini 2.5 Flash	100	100	100	100	98	99.6%
Claude 3 Haiku	100	100	100	100	98	99.6%
GPT-4.1 Mini	100	100	100	100	98	99.5%
Ministral 3 3B	100	100	100	100	98	99.5%
GPT-5.4 (Reasoning, Low)	100	100	100	100	96	99.3%
GPT-5.4 Nano (Reasoning)	100	100	100	100	96	99.3%
Mistral Large 2	100	100	100	98	98	99.2%
MiniMax M2.5	100	100	100	98	98	99.2%
Z.AI GLM 4.7 Flash	100	100	100	100	96	99.2%
Qwen 3 32B	100	100	100	100	96	99.2%
GPT-5.4 Nano	100	100	100	100	96	99.2%
Ministral 3B	100	100	100	98	98	99.1%
Hermes 3 70B	100	100	100	98	98	99.0%
Hermes 3 405B	100	100	100	98	96	98.7%
Mistral Small 3.2 24B	100	100	98	98	98	98.7%
Mistral Large	100	98	98	98	98	98.5%
Cohere Command R+ (Aug. 2024)	100	100	98	98	96	98.5%
GPT-4o Mini (temp=1)	100	100	100	97	94	98.3%
Mistral Large 3	98	98	98	98	98	98.1%
GPT-4.1	100	100	100	96	93	98.0%
Mistral Small 4 (Reasoning)	100	100	100	97	93	97.9%
Gemini 2.5 Flash Lite (Reasoning)	100	98	98	96	96	97.9%
GPT-5.4	100	100	96	96	93	97.1%
GPT-4o Mini (temp=0)	100	100	100	100	85	97.0%
Qwen 2.5 72B	100	100	96	95	93	97.0%
ByteDance Seed 2.0 Mini	100	98	96	96	94	97.0%
Writer: Palmyra X5	100	100	96	95	93	96.9%
Gemini 2.5 Flash Lite	100	98	98	96	90	96.5%
Qwen 3.5 9B	100	100	96	96	89	96.4%
ByteDance Seed 1.6 Flash	100	98	96	94	92	96.1%
GPT-5.4 Nano (Reasoning, Low)	100	100	97	94	90	96.1%
Mistral Small Creative	97	96	96	94	91	94.9%
Mistral Medium 3.1	96	94	94	94	94	94.5%
Mistral Small 4	100	100	96	91	83	94.2%
Arcee AI: Trinity Mini	98	97	93	92	90	93.8%
GPT-4.1 Nano	98	94	93	93	89	93.4%
Qwen3 235B A22B Instruct 2507	100	100	96	95	75	93.3%
Gemini 3 Flash (Preview)	100	93	93	89	89	92.9%
Gemma 3 4B	96	93	91	91	90	92.3%
Llama 3.1 70B	98	94	92	90	86	91.9%
Ministral 3 14B	92	91	89	89	88	90.0%
Llama 3.1 8B	93	88	88	82	75	85.0%
Arcee AI: Trinity Large (Preview)	100	98	98	94	10	80.0%
Gemma 3 12B	90	89	88	88	10	73.0%
Llama 3.1 Nemotron 70B	91	84	79	77	31	72.5%
Rocinante 12B	85	83	79	69	10	65.3%
LFM2 24B	25	25	25	25	25	25.0%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Medium: Through the Thornveil (Scattered)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Qwen 3.5 27B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Mini	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
GPT-5.4	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
Inception Mercury 2	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Inception Mercury	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	98	99.6%
Nemotron 3 Nano	100	100	100	100	98	99.6%
GPT-5.4 Mini	100	100	100	100	97	99.4%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	97	99.3%
Nemotron 3 Super	100	100	100	100	97	99.3%
GPT-4o Mini (temp=1)	100	100	100	100	95	99.0%
Arcee AI: Trinity Large (Preview)	100	100	100	98	97	99.0%
Ministral 3 3B	100	100	100	98	97	99.0%
GPT-4.1 Mini	100	100	100	98	97	98.9%
GPT-4.1	100	100	100	100	93	98.7%
Arcee AI: Trinity Mini	100	100	100	100	92	98.3%
Mistral Large 2	100	100	100	97	94	98.2%
Mistral Large	100	100	100	97	94	98.2%
Ministral 3B	100	100	98	97	96	98.2%
Qwen 3 32B	100	100	100	98	93	98.2%
ByteDance Seed 1.6 Flash	100	100	100	96	95	98.2%
Z.AI GLM 4.5	100	100	100	100	90	98.0%
Cohere Command R+ (Aug. 2024)	100	100	98	98	95	98.0%
Gemma 3 27B	100	100	98	97	95	97.9%
GPT-4o, May 13th (temp=0)	100	100	100	100	89	97.9%
Z.AI GLM 4.7 Flash	100	100	100	100	89	97.9%
Mistral Small 4 (Reasoning)	100	100	97	94	94	97.0%
Qwen 3.5 9B	100	100	100	94	91	96.9%
Gemma 3 4B	98	97	96	96	95	96.6%
Mistral Small 4	100	100	100	98	84	96.5%
GPT-4o, May 13th (temp=1)	100	98	96	96	89	96.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	93	86	95.8%
Qwen3 235B A22B Instruct 2507	100	100	96	93	89	95.7%
GPT-5.4 Nano	100	100	100	100	76	95.2%
Hermes 3 70B	100	100	98	98	78	94.7%
Claude 3 Haiku	100	98	96	90	88	94.2%
Qwen 2.5 72B	98	96	93	92	88	93.5%
Gemma 3 12B	97	94	94	91	90	93.1%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	64	92.8%
Writer: Palmyra X5	96	96	95	90	86	92.5%
Gemini 2.5 Flash Lite	100	100	98	93	71	92.5%
GPT-4.1 Nano	96	93	91	89	88	91.3%
GPT-5.4 Nano (Reasoning)	100	100	100	94	56	90.0%
Mistral Small Creative	100	90	81	78	78	85.5%
Ministral 3 14B	88	86	85	70	65	78.7%
Llama 3.1 70B	95	94	93	91	10	76.7%
Llama 3.1 Nemotron 70B	77	73	69	67	50	67.3%
Llama 3.1 8B	88	81	70	69	10	63.4%
Mistral NeMO	100	100	100	0	0	60.0%
Rocinante 12B	92	83	60	50	0	57.1%
LFM2 24B	25	25	25	25	25	25.0%

▼

Short: The Rusty Lantern (Explicit)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Qwen 3.5 122B	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
GPT-5.2	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
MiniMax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
Gemini 2.5 Pro	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Stealth: Hunter Alpha	100	100	100	100	100	100.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Stealth: Healer Alpha	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Z.AI GLM 4.7 Flash	100	100	100	100	100	100.0%
Nemotron 3 Super	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Grok 4.20 (Beta)	100	100	100	100	100	100.0%
Inception Mercury 2	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
Mistral Small 4 (Reasoning)	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
GPT-5.4 Nano (Reasoning)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Mistral Large	100	100	100	100	100	100.0%
Qwen3 235B A22B Instruct 2507	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Inception Mercury	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Nemotron 3 Nano	100	100	100	100	100	100.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	98	99.6%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	98	99.5%
Hermes 3 405B	100	100	100	100	98	99.5%
Gemma 3 4B	100	100	100	100	98	99.5%
Claude 3 Haiku	100	100	100	100	96	99.3%
ByteDance Seed 2.0 Mini	100	100	100	100	96	99.2%
GPT-5.4 Mini	100	100	100	100	96	99.2%
Ministral 3 14B	100	100	100	98	98	99.2%
Z.AI GLM 5 Turbo	100	100	100	100	95	99.1%
Qwen 2.5 72B	100	100	100	100	95	99.1%
Hermes 3 70B	100	100	100	98	98	99.1%
Qwen 3 32B	100	100	100	100	95	99.0%
GPT-4o Mini (temp=0)	100	100	100	100	94	98.9%
Qwen 3.5 9B	100	100	100	100	94	98.8%
Ministral 3 3B	100	100	100	100	94	98.8%
Grok 4 Fast	100	100	98	98	98	98.7%
ByteDance Seed 1.6 Flash	100	100	98	98	98	98.7%
ByteDance Seed 2.0 Lite	100	100	98	98	98	98.6%
Cohere Command R+ (Aug. 2024)	100	100	100	96	95	98.3%
Ministral 3B	100	100	100	96	95	98.3%
GPT-5 Nano	100	100	100	100	91	98.2%
GPT-4.1 Nano	100	100	100	96	94	98.1%
GPT-5.4 Nano	100	100	100	97	93	98.0%
GPT-4o, May 13th (temp=1)	100	100	100	96	94	97.9%
GPT-5.4 Nano (Reasoning, Low)	100	100	100	100	88	97.7%
Mistral Small 4	100	100	100	100	85	96.9%
GPT-4o Mini (temp=1)	100	100	94	94	94	96.7%
Llama 3.1 70B	100	98	97	95	92	96.4%
GPT-5.4	100	96	96	96	92	95.8%
Gemma 3 12B	97	97	96	96	92	95.4%
Arcee AI: Trinity Mini	100	100	97	90	89	95.2%
Llama 3.1 8B	98	95	91	85	85	90.7%
GPT-4.1	100	100	91	83	75	89.8%
Gemini 2.5 Flash Lite	100	93	90	85	75	88.6%
Llama 3.1 Nemotron 70B	95	93	92	82	77	87.9%
Ministral 8B	100	100	79	77	73	85.8%
Qwen 3.5 27B	100	100	100	50	50	80.0%
Mistral NeMO	100	100	100	0	0	60.0%
Rocinante 12B	83	67	0	0	0	30.0%
LFM2 24B	25	25	25	25	25	25.0%

Structural validity

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Long: The Spire of Echoes (Dense)

Medium: The Hollow (Inferred)

Medium: Through the Thornveil (Scattered)

Short: The Rusty Lantern (Explicit)