Long: The Spire of Echoes (Dense)

Codex Extraction

Evaluates a model's ability to extract structured codex entries (characters, locations, objects, lore) from prose passages and return them as well-formed XML.

Tooling Reasoning

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
Gemini 3 Pro (Preview)	99%
Gemini 3.5 Flash (Reasoning)	99%
Gemini 3 Flash (Preview, Reasoning)	99%
Claude Opus 4.5	98%
Claude Opus 4.6	98%
Claude Opus 4	98%
Qwen3.6 Max Preview	98%
Claude Opus 4.6 (Reasoning)	98%
Gemini 3 Flash (Preview)	98%
GPT-5.4 (Reasoning)	98%
Z.AI GLM 5	98%
Claude Opus 4.7	97%
Aion 2.0	97%
Z.AI GLM 5.1	97%
Claude Sonnet 4.6 (Reasoning)	97%
Claude Opus 4.7 (Reasoning)	97%
Grok 4	97%
Grok 4.3 (Reasoning)	97%
o4 Mini High	97%
Gemini 3.5 Flash (Reasoning, Minimal)	97%

	Score	Cost	Time
DeepSeek V4 Flash	91%	$0.0005	11.2s
Gemini 3 Flash (Preview)	98%	$0.0033	5.4s
Grok 4 Fast	96%	$0.0014	9.6s
Mistral Medium 3.1	97%	$0.0035	7.5s
Mistral Small Creative	95%	$0.0007	5.8s
Gemini 2.5 Flash	93%	$0.0033	3.6s
Xiaomi MIMO v2.5	97%	$0.0042	16.1s
Qwen 3.5 Plus (2026-02-15)	97%	$0.0041	14.9s
Grok 4.20 (Beta)	96%	$0.0072	2.8s
Gemini 3.1 Flash Lite (Preview)	94%	$0.0022	2.3s
Gemini 3.5 Flash (Reasoning, Minimal)	97%	$0.012	4.3s
GPT-5.4 Mini (Reasoning, Low)	95%	$0.0060	4.5s
Gemini 3.1 Flash Lite (Reasoning)	94%	$0.0023	2.7s
Mistral Small 3.2 24B	93%	$0.0007	5.7s
Grok 4.20	96%	$0.0064	7.8s
Gemini 2.5 Flash Lite (Reasoning)	95%	$0.0025	15.5s
Grok 4.1 Fast	95%	$0.0018	17.9s
DeepSeek V3.1	78%	$0.0016	38.0s
Z.AI GLM 4.5	96%	$0.0056	30.8s
Gemini 3 Flash (Preview, Reasoning)	99%	$0.011	19.1s

	Score	Cost	Speed	Stability
Gemini 3 Flash (Preview)	98%	$0.0033	5.4s	97%
Mistral Medium 3.1	97%	$0.0035	7.5s	96%
Mistral Small Creative	95%	$0.0007	5.8s	94%
Grok 4 Fast	96%	$0.0014	9.6s	94%
Qwen 3.5 Plus (2026-02-15)	97%	$0.0041	14.9s	96%
Xiaomi MIMO v2.5	97%	$0.0042	16.1s	96%
Grok 4.20	96%	$0.0064	7.8s	95%
Gemini 3.1 Flash Lite (Preview)	94%	$0.0022	2.3s	93%
Grok 4.20 (Beta)	96%	$0.0072	2.8s	95%
Gemini 3.1 Flash Lite (Reasoning)	94%	$0.0023	2.7s	93%
GPT-5.4 Mini (Reasoning, Low)	95%	$0.0060	4.5s	94%
Gemini 3.5 Flash (Reasoning, Minimal)	97%	$0.012	4.3s	96%
Gemini 3 Flash (Preview, Reasoning)	99%	$0.011	19.1s	98%
Claude Haiku 4.5	96%	$0.0096	6.6s	95%
Mistral Small 3.2 24B	93%	$0.0007	5.7s	91%
Gemma 4 31B	96%	$0.0009	34.4s	95%
DeepSeek-V2 Chat	95%	$0.0022	20.5s	93%
Gemini 3.1 Flash Lite	94%	$0.0021	4.0s	91%
Grok 4.1 Fast	95%	$0.0018	17.9s	92%
Xiaomi MIMO v2.5 Pro	97%	$0.0065	23.3s	95%

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	Total
124	Gemini 3 Pro (Preview)	$0.061	38.1s	98%	99	99	99	99	99	99%
111	Gemini 3.5 Flash (Reasoning)	$0.052	22.0s	98%	99	99	99	99	98	99%
13	Gemini 3 Flash (Preview, Reasoning)	$0.011	19.1s	98%	99	99	99	98	98	99%
99	Claude Opus 4.5	$0.049	11.8s	98%	99	98	98	98	98	98%
102	Claude Opus 4.6	$0.050	11.3s	98%	98	98	98	98	98	98%
144	Claude Opus 4	$0.147	20.0s	98%	98	98	98	98	98	98%
142	Qwen3.6 Max Preview	$0.055	3.0m	97%	99	98	98	97	97	98%
133	Claude Opus 4.6 (Reasoning)	$0.078	26.6s	97%	99	98	98	98	97	98%
1	Gemini 3 Flash (Preview)	$0.0033	5.4s	97%	99	98	98	97	97	98%
135	GPT-5.4 (Reasoning)	$0.063	1.2m	97%	98	98	98	98	97	98%
48	Z.AI GLM 5	$0.0095	51.0s	97%	98	98	98	97	97	98%
118	Claude Opus 4.7	$0.062	9.0s	97%	98	98	98	97	97	97%
95	Aion 2.0	$0.010	1.6m	96%	98	98	98	98	96	97%
103	Z.AI GLM 5.1	$0.019	1.4m	96%	98	98	97	97	96	97%
136	Claude Sonnet 4.6 (Reasoning)	$0.077	53.4s	97%	98	98	97	97	97	97%
116	Claude Opus 4.7 (Reasoning)	$0.061	8.8s	97%	98	98	98	97	97	97%
98	Grok 4	$0.035	38.5s	97%	98	98	98	98	96	97%
117	Grok 4.3 (Reasoning)	$0.023	1.7m	95%	99	98	98	96	96	97%
105	o4 Mini High	$0.032	54.1s	96%	98	98	97	97	96	97%
12	Gemini 3.5 Flash (Reasoning, Minimal)	$0.012	4.3s	96%	98	98	97	97	96	97%
131	Qwen3.7 Max	$0.051	1.3m	95%	98	98	98	97	95	97%
6	Xiaomi MIMO v2.5	$0.0042	16.1s	96%	98	98	97	97	96	97%
55	Grok 4.20 (Reasoning)	$0.015	39.0s	96%	98	97	97	97	96	97%
87	GPT-5.4 Mini (Reasoning)	$0.027	37.0s	97%	97	97	97	97	97	97%
5	Qwen 3.5 Plus (2026-02-15)	$0.0041	14.9s	96%	97	97	97	97	96	97%
121	Gemma 4 26B (Reasoning)	$0.0023	2.8m	95%	98	98	97	96	95	97%
66	Claude Sonnet 4.6	$0.029	10.6s	96%	98	97	97	96	96	97%
104	Gemini 2.5 Pro	$0.041	27.4s	95%	98	97	97	96	95	97%
41	Stealth: Hunter Alpha	$0.0000	50.4s	93%	99	99	96	96	94	97%
139	GPT-5.5 (Reasoning)	$0.088	37.5s	96%	97	97	97	97	96	97%
109	DeepSeek V4 Pro (Reasoning)	$0.0047	2.1m	95%	98	97	97	96	95	97%
2	Mistral Medium 3.1	$0.0035	7.5s	96%	97	97	97	97	96	97%
106	Qwen 3.5 27B	$0.018	1.5m	95%	98	97	97	96	96	97%
33	DeepSeek V4 Flash (Reasoning)	$0.0012	50.2s	96%	97	97	97	96	96	97%
93	GPT-5.5	$0.040	8.6s	96%	97	97	96	96	96	97%
20	Xiaomi MIMO v2.5 Pro	$0.0065	23.3s	95%	98	97	97	96	95	97%
60	Grok 4.20 (Beta, Reasoning)	$0.023	13.8s	94%	98	97	97	96	94	97%
112	Qwen 3.5 Plus (2026-04-20)	$0.017	1.6m	95%	98	97	97	96	95	96%
113	GPT-5.5 (Reasoning, Low)	$0.050	15.6s	96%	97	97	96	96	96	96%
7	Grok 4.20	$0.0064	7.8s	95%	97	97	96	96	96	96%
140	GPT-5	$0.065	1.6m	95%	97	97	97	96	95	96%
137	MoonshotAI: Kimi K2.6	$0.033	2.6m	95%	97	97	96	96	96	96%
134	MoonshotAI: Kimi K2.5	$0.015	2.9m	94%	98	97	96	95	95	96%
14	Claude Haiku 4.5	$0.0096	6.6s	95%	97	96	96	96	96	96%
24	Z.AI GLM 5 Turbo	$0.0093	17.7s	95%	97	97	97	95	95	96%
150	Qwen 3.5 122B	$0.131	6.4m	95%	97	97	96	95	95	96%
67	Claude 3.7 Sonnet	$0.027	10.8s	95%	97	96	96	96	96	96%
4	Grok 4 Fast	$0.0014	9.6s	94%	97	97	96	96	94	96%
16	Gemma 4 31B	$0.0009	34.4s	95%	96	96	96	96	96	96%
82	Claude Sonnet 4	$0.029	11.1s	93%	98	97	96	95	94	96%
59	Z.AI GLM 4.6	$0.0050	59.2s	95%	97	96	96	95	95	96%
128	Qwen 3.5 397B A17B	$0.0096	2.5m	94%	97	97	96	95	95	96%
101	Z.AI GLM 4.7	$0.012	1.3m	92%	98	98	96	95	93	96%
35	DeepSeek V3.2	$0.0015	44.1s	94%	97	96	96	95	95	96%
74	Nemotron 3 Super	$0.0000	1.3m	93%	97	96	95	95	95	96%
130	GPT-5.1	$0.051	57.9s	94%	97	96	95	95	95	96%
69	GPT-5 Mini	$0.0086	55.7s	94%	97	96	96	95	94	96%
9	Grok 4.20 (Beta)	$0.0072	2.8s	95%	96	96	96	95	95	96%
40	Mistral Large 2	$0.014	12.4s	94%	96	96	96	96	93	96%
58	Qwen 3.5 Flash	$0.0034	55.0s	93%	97	96	95	95	94	96%
34	Z.AI GLM 4.5	$0.0056	30.8s	94%	97	96	96	96	94	96%
76	GPT-5.2	$0.023	20.4s	93%	97	96	96	95	93	95%
11	GPT-5.4 Mini (Reasoning, Low)	$0.0060	4.5s	94%	96	96	95	95	95	95%
77	GPT-5.4 (Reasoning, Low)	$0.024	14.5s	93%	97	97	97	96	91	95%
3	Mistral Small Creative	$0.0007	5.8s	94%	97	96	95	95	94	95%
123	Gemma 4 31B (Reasoning)	$0.0017	2.7m	95%	96	96	96	95	94	95%
17	DeepSeek-V2 Chat	$0.0022	20.5s	93%	97	96	96	94	93	95%
71	o4 Mini	$0.018	26.2s	92%	97	97	96	94	92	95%
32	Gemini 2.5 Flash (Reasoning)	$0.0099	14.2s	94%	96	96	96	94	94	95%
19	Grok 4.1 Fast	$0.0018	17.9s	92%	97	97	95	94	93	95%
25	Stealth: Healer Alpha	$0.0000	27.4s	92%	97	97	94	94	94	95%
22	Gemini 2.5 Flash Lite (Reasoning)	$0.0025	15.5s	91%	98	97	95	94	92	95%
78	Claude Sonnet 4.5	$0.029	9.0s	95%	96	95	95	95	95	95%
110	ByteDance Seed 2.0 Lite	$0.0089	1.7m	93%	96	96	96	94	93	95%
126	Claude 3.5 Sonnet	$0.056	19.0s	93%	96	96	96	94	93	95%
83	Qwen 3.5 35B	$0.015	43.7s	93%	96	96	95	95	93	95%
27	Grok 4.3	$0.0071	6.0s	91%	98	96	95	93	93	95%
29	MiniMax M2.7	$0.0024	26.2s	93%	96	95	95	95	94	95%
26	Mistral Large 3	$0.0036	11.9s	91%	96	96	93	93	93	94%
10	Gemini 3.1 Flash Lite (Reasoning)	$0.0023	2.7s	93%	95	95	94	94	94	94%
53	GPT-5.4	$0.015	9.8s	91%	97	94	94	93	93	94%
115	ByteDance Seed 1.6	$0.0095	1.7m	92%	96	94	94	93	93	94%
8	Gemini 3.1 Flash Lite (Preview)	$0.0022	2.3s	93%	95	94	94	93	93	94%
36	DeepSeek V4 Pro	$0.0030	18.7s	90%	97	95	93	93	91	94%
63	WizardLM 2 8x22b	$0.0031	40.4s	90%	97	95	95	93	89	94%
84	Qwen 3.6 Flash	$0.011	36.7s	89%	97	96	93	92	91	94%
18	Gemini 3.1 Flash Lite	$0.0021	4.0s	91%	95	94	93	93	92	94%
45	DeepSeek V3 (2025-03-24)	$0.0027	23.7s	90%	96	95	93	92	92	94%
92	GPT-4o, May 13th (temp=0)	$0.032	6.4s	93%	94	94	94	93	93	93%
107	Qwen 3.6 27B	$0.015	56.2s	87%	97	96	93	92	88	93%
15	Mistral Small 3.2 24B	$0.0007	5.7s	91%	95	94	93	93	92	93%
39	DeepSeek V3 (2024-12-26)	$0.0021	17.5s	89%	96	94	93	92	92	93%
46	GPT-4o, Aug. 6th (temp=0)	$0.013	6.6s	93%	94	93	93	93	93	93%
47	Gemini 2.5 Flash	$0.0033	3.6s	85%	97	96	96	95	82	93%
138	ByteDance Seed 2.0 Mini	$0.0039	3.7m	92%	94	94	94	93	91	93%
85	GPT-OSS 120B	$0.0010	1.0m	89%	95	94	92	92	91	93%
65	MiniMax M2.5	$0.0028	42.0s	90%	95	94	92	92	91	93%
43	GPT-5.4 Nano (Reasoning)	$0.0033	18.3s	91%	95	94	93	92	90	93%
62	Z.AI GLM 4.5 Air	$0.0025	34.5s	89%	95	94	92	92	91	93%
21	GPT-5.4 Mini	$0.0042	2.7s	92%	93	93	93	92	92	93%
51	Hermes 3 405B	$0.0049	22.5s	90%	94	93	93	92	90	93%
91	Qwen 3.6 35B	$0.0087	47.3s	89%	95	94	93	91	89	92%
143	Gemini 3.1 Pro (Preview)	$0.085	1.4m	79%	98	97	97	96	74	92%
28	Inception Mercury 2	$0.0027	4.2s	91%	93	93	92	92	91	92%
129	Nemotron 3 Nano	$0.0019	2.3m	87%	94	94	92	92	88	92%
42	GPT-4.1 Mini	$0.0018	10.0s	88%	94	92	91	91	90	92%
23	Gemini 2.5 Flash Lite	$0.0005	2.4s	90%	93	92	91	91	91	92%
30	Ministral 3 8B	$0.0008	4.9s	90%	93	92	92	91	90	92%
52	Qwen3 235B A22B Instruct 2507	$0.0009	31.4s	91%	92	92	92	91	90	92%
96	DeepSeek V4 Flash	$0.0005	11.2s	74%	98	98	98	97	67	91%
61	GPT-4o, Aug. 6th (temp=1)	$0.012	6.5s	90%	93	92	91	91	91	91%
37	Ministral 3 14B	$0.0010	9.9s	89%	93	92	92	90	89	91%
64	Llama 3.1 Nemotron 70B	$0.0060	23.2s	89%	93	92	92	90	88	91%
31	Ministral 8B	$0.0005	5.3s	90%	92	92	91	91	90	91%
125	Qwen 3.5 9B	$0.0014	1.5m	80%	96	95	94	94	76	91%
108	GPT-4o, May 13th (temp=1)	$0.031	6.6s	87%	93	93	92	91	86	91%
57	Writer: Palmyra X5	$0.0071	10.6s	89%	92	91	91	91	89	91%
56	GPT-4.1	$0.0087	6.5s	89%	92	91	91	91	90	91%
80	Qwen 3 32B	$0.0010	39.2s	88%	93	92	91	90	88	91%
122	GPT-5 Nano	$0.0053	1.6m	85%	95	94	93	87	85	90%
38	Mistral Small 4	$0.0010	4.4s	88%	92	92	91	89	88	90%
54	Qwen 2.5 72B	$0.0012	16.5s	88%	92	91	90	90	88	90%
49	Claude 3 Haiku	$0.0024	8.8s	88%	92	90	90	90	89	90%
68	Mistral Small 4 (Reasoning)	$0.0025	19.4s	87%	92	90	90	89	87	90%
44	Ministral 3 3B	$0.0005	2.7s	88%	90	90	89	88	88	89%
79	Cydonia 24B V4.1	$0.0013	18.7s	84%	93	92	91	88	82	89%
50	Ministral 3B	$0.0002	2.9s	86%	92	89	89	88	87	89%
90	Hermes 3 70B	$0.0015	21.1s	83%	92	90	90	90	81	89%
73	Gemma 3 27B	$0.0005	15.8s	84%	91	90	88	88	85	89%
86	Llama 3.1 70B	$0.0019	21.9s	84%	92	90	90	86	84	89%
127	Z.AI GLM 4.7 Flash	$0.0025	1.3m	80%	95	89	87	86	83	88%
114	Cohere Command R+ (Aug. 2024)	$0.018	23.1s	84%	90	90	87	86	86	88%
89	Gemma 3 12B	$0.0003	21.5s	83%	90	89	89	89	81	88%
75	GPT-5.4 Nano (Reasoning, Low)	$0.0014	5.4s	83%	91	88	88	87	83	87%
70	GPT-4o Mini (temp=0)	$0.0007	11.6s	85%	88	88	87	86	85	87%
81	Gemma 3 4B	$0.0002	9.7s	83%	89	87	86	86	84	87%
72	Arcee AI: Trinity Mini	$0.0003	6.4s	84%	87	87	86	86	85	86%
94	GPT-5.4 Nano	$0.0015	4.8s	79%	90	89	88	87	76	86%
88	GPT-4o Mini (temp=1)	$0.0007	10.9s	83%	88	87	87	85	82	86%
119	ByteDance Seed 1.6 Flash	$0.0014	54.8s	79%	92	85	84	83	83	85%
97	LFM2 24B	$0.0002	14.4s	80%	87	87	86	86	78	85%
100	Inception Mercury	$0.0008	4.8s	78%	88	86	83	81	81	84%
132	Skyfall 36B V2	$0.0023	21.1s	65%	90	87	85	77	58	79%
145	DeepSeek V3.1	$0.0016	38.0s	23%	98	98	97	96	3	78%
147	Mistral Large	$0.014	9.4s	23%	97	95	94	93	0	76%
120	GPT-4.1 Nano	$0.0003	3.3s	73%	79	76	76	75	72	76%
141	Llama 3.1 8B	$0.0002	11.1s	49%	85	83	82	81	32	73%
146	Arcee AI: Trinity Large (Preview)	$0.0000	33.2s	27%	92	89	89	87	3	72%
148	Gemma 4 26B	$0.0008	16.3s	10%	96	96	86	3	3	57%
149	Rocinante 12B	$0.0012	7.4s	13%	88	86	80	0	0	51%
151	Mistral NeMO	$0.0006	500ms	0%	0	0	0	0	0	0%
92.04%

Median	Evaluator	Top 3	Flop 3
83.0%	Accuracy	95Gemini 3 Pro (Preview) 95Gemini 3.5 Flash (Reasoning) 94Gemini 3 Flash (Preview, Reasoning)	0Mistral NeMO 43Rocinante 12B 45GPT-4.1 Nano
99.3%	Precision	100Grok 4.3 100DeepSeek-V2 Chat 100Gemini 3 Flash (Preview, Reasoning)	0Mistral NeMO 58Rocinante 12B 60Gemma 4 26B
98.3%	Recall	100ByteDance Seed 1.6 100Grok 4.3 (Reasoning) 100Grok 4	0Mistral NeMO 49Rocinante 12B 56Gemma 4 26B
100.0%	Structural validity	100Qwen3.6 Max Preview 100Nemotron 3 Nano 100o4 Mini	0Mistral NeMO 53Rocinante 12B 62Gemma 4 26B

Codex Extraction

Long: The Spire of Echoes (Dense)

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

	Score	Consistency	Stability
Gemini 3 Pro (Preview)	99%	100%	98%
Gemini 3.5 Flash (Reasoning)	99%	99%	98%
Claude Opus 4	98%	100%	98%
Gemini 3 Flash (Preview, Reasoning)	99%	99%	98%
Claude Opus 4.5	98%	100%	98%
Claude Opus 4.6	98%	100%	98%
Claude Opus 4.7	97%	100%	97%
Claude Opus 4.7 (Reasoning)	97%	100%	97%
Z.AI GLM 5	98%	99%	97%
Claude Sonnet 4.6 (Reasoning)	97%	99%	97%
Qwen3.6 Max Preview	98%	99%	97%
GPT-5.4 (Reasoning)	98%	99%	97%
Gemini 3 Flash (Preview)	98%	99%	97%
Grok 4	97%	99%	97%
Claude Opus 4.6 (Reasoning)	98%	99%	97%
GPT-5.4 Mini (Reasoning)	97%	100%	97%
Mistral Medium 3.1	97%	100%	96%
Qwen 3.5 Plus (2026-02-15)	97%	99%	96%
Gemini 3.5 Flash (Reasoning, Minimal)	97%	99%	96%
Z.AI GLM 5.1	97%	99%	96%