Short: The Rusty Lantern (Explicit)

Codex Extraction

Evaluates a model's ability to extract structured codex entries (characters, locations, objects, lore) from prose passages and return them as well-formed XML.

Tooling Reasoning

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

	Score
Z.AI GLM 5.1	99%
Qwen 3.5 Plus (2026-02-15)	99%
Grok 4	99%
Claude Sonnet 4.5	99%
GPT-5	99%
Claude Opus 4.5	99%
Claude Opus 4	99%
Claude Sonnet 4	99%
Grok 4.1 Fast	99%
DeepSeek V4 Flash (Reasoning)	99%
Claude Opus 4.6	99%
Grok 4.20 (Beta, Reasoning)	99%
MiniMax M2.7	99%
Claude Sonnet 4.6	99%
Qwen 3.5 397B A17B	99%
Grok 4.20 (Reasoning)	99%
Qwen3.6 Max Preview	99%
Z.AI GLM 4.5	99%
Hermes 3 405B	99%
Claude Opus 4.6 (Reasoning)	99%

	Score	Cost	Time
Mistral Small Creative	97%	$0.0004	2.7s
Ministral 3 8B	98%	$0.0005	2.6s
DeepSeek V4 Flash	98%	$0.0002	5.6s
Mistral Medium 3.1	98%	$0.0021	6.5s
Z.AI GLM 4.5	99%	$0.0013	8.1s
Qwen 3.5 Plus (2026-02-15)	99%	$0.0024	7.9s
Gemini 3 Flash (Preview)	98%	$0.0022	3.1s
DeepSeek V4 Pro	97%	$0.0005	10.9s
Xiaomi MIMO v2.5	98%	$0.0032	12.3s
DeepSeek V3 (2024-12-26)	96%	$0.0016	11.9s
Gemma 4 26B	96%	$0.0004	12.8s
MiniMax M2.7	99%	$0.0016	22.0s
DeepSeek V4 Flash (Reasoning)	99%	$0.0007	24.8s
Grok 4.1 Fast	99%	$0.0015	19.0s
GPT-5.4 Nano (Reasoning)	96%	$0.0017	7.3s
Grok 4 Fast	96%	$0.0010	7.4s
Qwen 2.5 72B	94%	$0.0007	8.5s
DeepSeek V3 (2025-03-24)	98%	$0.0008	28.8s
Xiaomi MIMO v2.5 Pro	98%	$0.0035	13.8s
Arcee AI: Trinity Large (Preview)	95%	$0.0000	14.5s

	Score	Cost	Speed	Stability
Ministral 3 8B	98%	$0.0005	2.6s	98%
Qwen 3.5 Plus (2026-02-15)	99%	$0.0024	7.9s	99%
Z.AI GLM 4.5	99%	$0.0013	8.1s	98%
DeepSeek V4 Flash	98%	$0.0002	5.6s	97%
Gemini 3 Flash (Preview)	98%	$0.0022	3.1s	96%
Mistral Small Creative	97%	$0.0004	2.7s	94%
Mistral Medium 3.1	98%	$0.0021	6.5s	96%
DeepSeek V4 Pro	97%	$0.0005	10.9s	96%
Grok 4.1 Fast	99%	$0.0015	19.0s	98%
Grok 4.20 (Beta)	97%	$0.0044	1.6s	95%
DeepSeek V4 Flash (Reasoning)	99%	$0.0007	24.8s	98%
Hermes 3 405B	99%	$0.0034	16.8s	98%
Xiaomi MIMO v2.5	98%	$0.0032	12.3s	96%
Gemini 2.5 Flash	94%	$0.0018	2.0s	94%
Mistral Large 3	96%	$0.0022	6.3s	95%
MiniMax M2.7	99%	$0.0016	22.0s	97%
Gemini 3.5 Flash (Reasoning, Minimal)	98%	$0.0088	2.5s	97%
Z.AI GLM 5 Turbo	98%	$0.0053	12.9s	97%
Gemma 4 26B	96%	$0.0004	12.8s	95%
Grok 4.3	97%	$0.0047	3.4s	94%

Rank	Model	Avg. Cost	Avg. Time	Stability	# 1	# 2	# 3	# 4	# 5	Total
95	Z.AI GLM 5.1	$0.014	1.1m	99%	100	100	100	99	99	99%
2	Qwen 3.5 Plus (2026-02-15)	$0.0024	7.9s	99%	100	100	100	100	99	99%
91	Grok 4	$0.025	32.0s	99%	100	100	100	99	99	99%
34	Claude Sonnet 4.5	$0.018	5.3s	99%	100	100	100	99	99	99%
136	GPT-5	$0.043	1.2m	99%	100	99	99	99	99	99%
77	Claude Opus 4.5	$0.029	5.4s	99%	100	100	100	99	99	99%
143	Claude Opus 4	$0.087	8.8s	98%	100	100	100	99	98	99%
42	Claude Sonnet 4	$0.017	6.3s	99%	99	99	99	99	99	99%
9	Grok 4.1 Fast	$0.0015	19.0s	98%	100	99	99	99	98	99%
11	DeepSeek V4 Flash (Reasoning)	$0.0007	24.8s	98%	100	99	99	99	99	99%
81	Claude Opus 4.6	$0.029	6.3s	99%	99	99	99	99	99	99%
53	Grok 4.20 (Beta, Reasoning)	$0.018	12.4s	98%	100	99	99	99	98	99%
16	MiniMax M2.7	$0.0016	22.0s	97%	100	100	99	98	97	99%
44	Claude Sonnet 4.6	$0.018	5.4s	98%	99	99	99	98	98	99%
127	Qwen 3.5 397B A17B	$0.013	1.7m	97%	100	100	99	99	96	99%
60	Grok 4.20 (Reasoning)	$0.011	33.1s	96%	100	100	98	98	98	99%
140	Qwen3.6 Max Preview	$0.036	1.9m	99%	99	99	99	99	99	99%
3	Z.AI GLM 4.5	$0.0013	8.1s	98%	99	99	99	99	98	99%
12	Hermes 3 405B	$0.0034	16.8s	98%	99	99	99	98	98	99%
117	Claude Opus 4.6 (Reasoning)	$0.041	16.7s	99%	99	99	99	99	99	99%
25	DeepSeek V3 (2025-03-24)	$0.0008	28.8s	96%	100	99	99	99	96	98%
30	GPT-5.4 Mini (Reasoning)	$0.0085	16.6s	97%	100	99	99	98	97	98%
18	Z.AI GLM 5 Turbo	$0.0053	12.9s	97%	99	99	98	98	98	98%
17	Gemini 3.5 Flash (Reasoning, Minimal)	$0.0088	2.5s	97%	99	99	99	98	96	98%
104	Gemini 3.5 Flash (Reasoning)	$0.033	13.0s	96%	100	99	99	99	95	98%
45	Claude 3.7 Sonnet	$0.017	5.9s	98%	99	99	99	98	98	98%
13	Xiaomi MIMO v2.5	$0.0032	12.3s	96%	100	99	99	99	95	98%
1	Ministral 3 8B	$0.0005	2.6s	98%	98	98	98	98	98	98%
7	Mistral Medium 3.1	$0.0021	6.5s	96%	100	99	99	96	96	98%
86	Aion 2.0	$0.0065	1.0m	95%	100	100	98	98	96	98%
116	Gemma 4 31B (Reasoning)	$0.0012	1.7m	95%	100	100	99	98	94	98%
23	Xiaomi MIMO v2.5 Pro	$0.0035	13.8s	95%	100	99	99	98	94	98%
146	DeepSeek V4 Pro (Reasoning)	$0.018	3.7m	96%	99	99	99	99	95	98%
51	Qwen 3.5 Flash	$0.0027	44.2s	96%	99	98	98	97	97	98%
5	Gemini 3 Flash (Preview)	$0.0022	3.1s	96%	99	99	98	98	96	98%
69	Gemini 3 Flash (Preview, Reasoning)	$0.0085	38.5s	96%	99	99	99	96	95	98%
142	MoonshotAI: Kimi K2.6	$0.019	2.8m	97%	98	98	98	98	98	98%
106	Gemini 2.5 Pro	$0.028	19.0s	94%	100	99	96	96	96	98%
4	DeepSeek V4 Flash	$0.0002	5.6s	97%	98	98	97	97	97	98%
131	Z.AI GLM 4.7	$0.0068	1.9m	93%	100	99	99	98	91	97%
8	DeepSeek V4 Pro	$0.0005	10.9s	96%	98	97	97	97	96	97%
89	o4 Mini High	$0.018	29.1s	94%	99	99	98	95	94	97%
112	Qwen 3.5 Plus (2026-04-20)	$0.012	1.1m	94%	99	99	98	95	94	97%
96	Qwen 3.5 35B	$0.013	50.7s	95%	99	98	98	96	95	97%
10	Grok 4.20 (Beta)	$0.0044	1.6s	95%	99	98	98	96	95	97%
68	Z.AI GLM 5	$0.0067	34.4s	93%	99	99	96	96	95	97%
98	Grok 4.3 (Reasoning)	$0.013	54.1s	96%	98	98	98	97	95	97%
123	Claude Sonnet 4.6 (Reasoning)	$0.035	22.8s	93%	100	98	96	96	95	97%
102	GPT-5.1	$0.023	28.1s	94%	99	98	96	96	95	97%
6	Mistral Small Creative	$0.0004	2.7s	94%	99	99	99	96	93	97%
54	Z.AI GLM 4.6	$0.0061	26.4s	93%	99	99	97	96	93	97%
133	MoonshotAI: Kimi K2.5	$0.012	1.8m	95%	98	98	98	96	95	97%
20	Grok 4.3	$0.0047	3.4s	94%	99	98	97	96	94	97%
134	Qwen3.7 Max	$0.033	50.6s	90%	99	99	95	95	94	96%
22	DeepSeek V3 (2024-12-26)	$0.0016	11.9s	94%	98	97	97	95	94	96%
24	Grok 4 Fast	$0.0010	7.4s	91%	99	99	95	95	93	96%
130	Qwen 3.5 122B	$0.024	1.0m	93%	99	96	96	95	95	96%
21	GPT-5.4 Nano (Reasoning)	$0.0017	7.3s	93%	98	97	97	96	92	96%
19	Gemma 4 26B	$0.0004	12.8s	95%	97	97	96	95	95	96%
15	Mistral Large 3	$0.0022	6.3s	95%	96	96	95	95	95	96%
101	ByteDance Seed 2.0 Lite	$0.0065	1.1m	93%	98	97	96	95	93	96%
28	DeepSeek-V2 Chat	$0.0016	11.4s	90%	99	98	95	94	93	96%
59	DeepSeek V3.2	$0.0007	28.8s	87%	100	99	98	97	85	96%
100	GPT-OSS 120B	$0.0010	1.3m	93%	97	97	95	95	95	96%
36	Mistral Large	$0.0088	6.1s	94%	96	96	95	95	95	96%
88	ByteDance Seed 1.6	$0.0053	53.1s	93%	98	96	96	95	92	96%
74	Nemotron 3 Super	$0.0000	53.2s	94%	96	96	96	96	93	96%
37	Mistral Large 2	$0.0087	6.0s	94%	96	95	95	95	95	95%
138	Gemini 3 Pro (Preview)	$0.050	33.4s	90%	100	95	95	94	94	95%
64	Qwen 3.6 35B	$0.0052	29.4s	92%	98	97	97	94	91	95%
62	MiniMax M2.5	$0.0020	30.9s	89%	99	97	96	96	89	95%
63	Qwen 3.6 Flash	$0.0073	22.7s	92%	98	97	96	94	92	95%
29	Arcee AI: Trinity Large (Preview)	$0.0000	14.5s	91%	99	97	95	94	92	95%
85	GPT-4o, May 13th (temp=0)	$0.021	3.9s	91%	99	95	94	94	94	95%
33	DeepSeek V3.1	$0.0009	21.7s	94%	96	96	96	94	94	95%
39	Stealth: Hunter Alpha	$0.0000	27.8s	94%	96	96	95	95	94	95%
35	Grok 4.20	$0.0041	3.8s	88%	99	99	97	94	87	95%
61	WizardLM 2 8x22b	$0.0021	28.8s	89%	99	98	96	93	89	95%
82	o4 Mini	$0.012	18.1s	89%	99	98	96	92	90	95%
114	Claude Opus 4.7	$0.037	4.1s	95%	95	95	95	95	95	95%
49	Gemini 2.5 Flash Lite (Reasoning)	$0.0021	14.8s	89%	99	97	94	92	91	95%
118	Claude Opus 4.7 (Reasoning)	$0.037	4.1s	94%	95	95	94	94	94	94%
14	Gemini 2.5 Flash	$0.0018	2.0s	94%	95	95	95	94	94	94%
137	ByteDance Seed 2.0 Mini	$0.0027	2.4m	90%	97	97	95	93	89	94%
120	Claude 3.5 Sonnet	$0.035	11.4s	94%	95	94	94	94	94	94%
50	Gemma 4 31B	$0.0006	27.8s	93%	95	95	94	94	94	94%
31	Qwen 2.5 72B	$0.0007	8.5s	90%	98	96	96	92	89	94%
47	Stealth: Healer Alpha	$0.0000	21.6s	91%	96	96	96	92	91	94%
92	GPT-5 Mini	$0.0061	41.0s	89%	98	95	95	92	90	94%
126	GPT-5.5 (Reasoning)	$0.035	14.9s	91%	95	94	94	94	91	94%
132	GPT-5.4 (Reasoning)	$0.028	50.8s	93%	94	94	94	94	94	94%
83	GPT-5.5	$0.018	4.8s	92%	95	95	94	94	91	94%
139	Gemini 3.1 Pro (Preview)	$0.049	45.9s	92%	95	95	94	94	91	94%
38	Gemini 3.1 Flash Lite	$0.0014	13.0s	92%	95	94	93	93	93	94%
48	GPT-4o, Aug. 6th (temp=0)	$0.0084	3.4s	93%	94	94	94	94	93	93%
26	Gemini 3.1 Flash Lite (Reasoning)	$0.0014	1.7s	92%	94	94	93	93	93	93%
40	GPT-5.4 Mini	$0.0026	1.8s	88%	98	93	93	92	91	93%
27	Gemini 3.1 Flash Lite (Preview)	$0.0014	1.8s	92%	94	94	94	92	92	93%
65	GPT-5.4 (Reasoning, Low)	$0.011	7.9s	92%	94	93	93	93	93	93%
121	Qwen 3.5 9B	$0.0011	1.2m	84%	98	98	93	90	86	93%
124	GPT-5.5 (Reasoning, Low)	$0.033	13.1s	92%	94	94	94	92	91	93%
55	Claude Haiku 4.5	$0.0059	3.5s	88%	95	94	94	94	86	93%
97	Nemotron 3 Nano	$0.0009	1.0m	91%	94	94	93	92	91	93%
57	Ministral 8B	$0.0003	3.3s	82%	99	97	90	89	87	92%
32	Ministral 3 14B	$0.0007	4.8s	90%	93	93	93	92	90	92%
41	Gemini 2.5 Flash Lite	$0.0004	1.6s	87%	96	95	93	91	87	92%
43	Inception Mercury 2	$0.0017	2.9s	88%	94	94	93	89	88	92%
99	Qwen 3 32B	$0.0011	55.3s	89%	94	93	93	91	89	92%
72	Hermes 3 70B	$0.0010	12.7s	83%	97	95	92	89	85	92%
52	Qwen3 235B A22B Instruct 2507	$0.0006	12.0s	88%	93	93	91	90	90	91%
67	Mistral Small 4 (Reasoning)	$0.0015	11.4s	85%	96	94	92	88	86	91%
76	Llama 3.1 Nemotron 70B	$0.0041	14.1s	86%	94	94	90	90	88	91%
70	GPT-4o, Aug. 6th (temp=1)	$0.0078	2.6s	88%	93	92	91	91	88	91%
56	Mistral Small 3.2 24B	$0.0004	3.4s	84%	97	92	91	87	87	91%
87	Gemini 2.5 Flash (Reasoning)	$0.0062	9.2s	81%	100	94	91	86	83	91%
46	Ministral 3B	$0.0001	1.4s	87%	93	92	90	90	89	91%
110	GPT-4o, May 13th (temp=1)	$0.021	3.0s	86%	94	91	90	89	87	90%
58	Writer: Palmyra X5	$0.0040	5.3s	89%	91	91	90	90	89	90%
73	Llama 3.1 70B	$0.0013	14.5s	87%	92	91	90	89	87	90%
111	GPT-5.2	$0.013	12.2s	82%	94	93	88	88	85	90%
129	GPT-5 Nano	$0.0036	1.1m	82%	95	92	89	87	84	89%
141	Qwen 3.6 27B	$0.017	1.1m	67%	99	98	98	94	59	89%
125	Z.AI GLM 4.7 Flash	$0.0013	1.0m	80%	98	89	89	88	83	89%
66	GPT-5.4 Mini (Reasoning, Low)	$0.0032	3.1s	86%	92	90	89	88	87	89%
108	Z.AI GLM 4.5 Air	$0.0019	37.2s	82%	94	92	91	90	80	89%
79	Claude 3 Haiku	$0.0014	4.3s	81%	94	92	89	87	81	88%
78	GPT-4.1 Mini	$0.0013	5.4s	82%	94	91	91	86	80	88%
80	Gemma 3 27B	$0.0004	12.3s	84%	92	90	88	88	84	88%
71	Gemma 3 4B	$0.0002	4.5s	84%	90	90	88	86	85	88%
94	GPT-4.1	$0.0075	3.9s	82%	91	91	90	84	81	87%
113	Cohere Command R+ (Aug. 2024)	$0.011	8.7s	80%	91	91	90	86	76	87%
105	ByteDance Seed 1.6 Flash	$0.0008	18.5s	77%	93	91	85	83	80	87%
93	GPT-5.4	$0.0081	4.7s	85%	87	87	87	86	84	86%
75	Ministral 3 3B	$0.0003	1.3s	83%	89	86	86	85	84	86%
84	Inception Mercury	$0.0004	2.7s	81%	90	89	87	82	81	86%
90	GPT-5.4 Nano (Reasoning, Low)	$0.0007	3.0s	78%	92	86	84	84	81	85%
103	Arcee AI: Trinity Mini	$0.0003	6.3s	76%	90	85	83	83	76	83%
107	Mistral Small 4	$0.0007	2.8s	74%	93	83	82	81	78	83%
109	GPT-5.4 Nano	$0.0007	3.1s	74%	88	87	81	79	78	83%
122	GPT-4o Mini (temp=1)	$0.0005	7.5s	72%	85	79	77	76	76	79%
119	Llama 3.1 8B	$0.0001	4.4s	73%	85	80	79	75	74	79%
149	Gemma 4 26B (Reasoning)	$0.0021	2.6m	23%	98	98	97	96	3	78%
115	GPT-4o Mini (temp=0)	$0.0005	6.8s	78%	79	79	79	78	77	78%
128	GPT-4.1 Nano	$0.0003	2.2s	66%	90	81	77	72	69	78%
144	Cydonia 24B V4.1	$0.0009	9.7s	27%	92	92	91	86	3	73%
135	Gemma 3 12B	$0.0002	10.7s	64%	77	71	70	69	63	70%
151	Qwen 3.5 27B	$0.029	2.2m	15%	100	99	99	13	13	64%
145	Skyfall 36B V2	$0.0013	7.4s	30%	84	78	78	75	3	63%
148	Mistral NeMO	$0.0005	1.8s	10%	93	90	85	0	0	54%
147	LFM2 24B	$0.0001	7.8s	38%	38	38	38	38	38	38%
150	Rocinante 12B	$0.0007	4.6s	0%	79	62	0	0	0	28%
92.19%

Median	Evaluator	Top 3	Flop 3
87.7%	Accuracy	98Z.AI GLM 5.1 98Qwen 3.5 Plus (2026-02-15) 98Gemini 2.5 Pro	0LFM2 24B 20Rocinante 12B 44Mistral NeMO
98.0%	Precision	100Gemini 3 Pro (Preview) 100Claude Opus 4.7 (Reasoning) 100Ministral 3 8B	40Rocinante 12B 58Mistral NeMO 59Gemma 3 12B
98.1%	Recall	100Claude 3.7 Sonnet 100Z.AI GLM 5.1 100DeepSeek V4 Flash (Reasoning)	23Rocinante 12B 29LFM2 24B 49Skyfall 36B V2
100.0%	Structural validity	100Gemini 3 Pro (Preview) 100Gemini 3.1 Pro (Preview) 100Gemini 3.5 Flash (Reasoning)	25LFM2 24B 30Rocinante 12B 60Mistral NeMO

Codex Extraction

Short: The Rusty Lantern (Explicit)

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

	Score	Consistency	Stability
Grok 4	99%	100%	99%
Qwen 3.5 Plus (2026-02-15)	99%	100%	99%
Claude Sonnet 4.5	99%	99%	99%
Claude Sonnet 4	99%	100%	99%
GPT-5	99%	100%	99%
Z.AI GLM 5.1	99%	99%	99%
Claude Opus 4.6	99%	100%	99%
Qwen3.6 Max Preview	99%	100%	99%
Claude Opus 4.5	99%	99%	99%
Claude Opus 4.6 (Reasoning)	99%	100%	99%
Claude Opus 4	99%	99%	98%
DeepSeek V4 Flash (Reasoning)	99%	99%	98%
Grok 4.1 Fast	99%	99%	98%
Claude Sonnet 4.6	99%	99%	98%
Z.AI GLM 4.5	99%	99%	98%
Hermes 3 405B	99%	99%	98%
Claude 3.7 Sonnet	98%	99%	98%
Grok 4.20 (Beta, Reasoning)	99%	99%	98%
Ministral 3 8B	98%	99%	98%
MoonshotAI: Kimi K2.6	98%	100%	97%