Recall

Test: Codex Extraction

Avg. Score

92.4%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Gemini 3.1 Flash Lite (Preview)	99.7%	$0.0017	2.0s	98%
2	Gemini 3 Flash (Preview)	99.7%	$0.0027	3.9s	97%
3	Qwen 3.5 Plus (2026-02-15)	100.0%	$0.0030	10.6s	100%
4	Mistral Small Creative	98.6%	$0.0006	3.9s	95%
5	Mistral Medium 3.1	98.5%	$0.0026	5.8s	96%
6	Ministral 3 8B	97.3%	$0.0006	3.3s	94%
7	GPT-5.4 Mini	97.6%	$0.0031	2.2s	94%
8	Z.AI GLM 4.5	98.4%	$0.0028	16.8s	96%
9	Grok 4 Fast	97.3%	$0.0012	8.7s	92%
10	Grok 4.1 Fast	99.0%	$0.0017	22.1s	96%
11	GPT-5.4 Mini (Reasoning, Low)	97.2%	$0.0042	3.7s	92%
12	Ministral 3 14B	96.7%	$0.0009	6.0s	90%
13	Gemini 2.5 Flash Lite	95.1%	$0.0005	1.9s	88%
14	Mistral Large 3	96.5%	$0.0027	8.2s	91%
15	MiniMax M2.7	98.2%	$0.0022	21.7s	95%
16	Ministral 8B	95.1%	$0.0004	3.8s	88%
17	Claude Haiku 4.5	97.5%	$0.0073	4.5s	92%
18	Gemma 3 27B	96.8%	$0.0005	14.5s	91%
19	GPT-4.1	97.0%	$0.0073	4.7s	91%
20	Gemini 2.5 Flash Lite (Reasoning)	97.0%	$0.0021	15.6s	91%
21	GPT-5.4	97.5%	$0.011	6.4s	93%
22	Claude 3.7 Sonnet	100.0%	$0.021	7.8s	100%
23	Claude Sonnet 4.5	100.0%	$0.022	6.6s	100%
24	Mistral Small 4 (Reasoning)	96.1%	$0.0019	15.6s	91%
25	Claude Sonnet 4	100.0%	$0.022	7.9s	100%
26	Mistral Small 3.2 24B	94.3%	$0.0005	4.4s	86%
27	GPT-5.4 Nano (Reasoning)	95.9%	$0.0023	12.2s	89%
28	GPT-5.4 (Reasoning, Low)	99.3%	$0.017	10.4s	97%
29	Gemini 3 Flash (Preview, Reasoning)	99.1%	$0.0096	22.2s	96%
30	Grok 4.20 (Beta)	95.3%	$0.0049	2.0s	87%
31	Z.AI GLM 5 Turbo	97.7%	$0.0068	16.0s	92%
32	Claude Sonnet 4.6	99.8%	$0.022	7.4s	99%
33	Grok 4.20 (Beta, Reasoning)	99.3%	$0.020	12.7s	97%
34	GPT-4o, Aug. 6th (temp=0)	95.6%	$0.0100	4.0s	87%
35	Mistral Large 2	96.3%	$0.011	8.3s	89%
36	DeepSeek V3.2	97.6%	$0.0011	36.7s	91%
37	GPT-4o, May 13th (temp=0)	98.5%	$0.025	4.3s	97%
38	DeepSeek V3 (2025-03-24)	96.4%	$0.0014	27.0s	87%
39	Stealth: Healer Alpha	94.7%	$0.0000	24.6s	86%
40	Z.AI GLM 4.6	98.1%	$0.0057	38.8s	93%
41	DeepSeek-V2 Chat	94.3%	$0.0019	14.8s	83%
42	GPT-4o, Aug. 6th (temp=1)	94.4%	$0.0092	3.8s	85%
43	Inception Mercury 2	92.5%	$0.0022	3.5s	80%
44	GPT-5.4 Mini (Reasoning)	99.0%	$0.017	25.6s	95%
45	o4 Mini	97.7%	$0.014	21.5s	92%
46	MiniMax M2.5	95.3%	$0.0023	30.1s	87%
47	Gemini 2.5 Flash	95.8%	$0.0023	2.5s	74%
48	GPT-5.2	97.6%	$0.018	16.8s	92%
49	Qwen 2.5 72B	91.5%	$0.0008	11.0s	81%
50	Claude 3 Haiku	91.3%	$0.0017	5.5s	80%
51	Stealth: Hunter Alpha	94.8%	$0.0000	36.7s	88%
52	Z.AI GLM 5	99.2%	$0.0091	51.4s	97%
53	Ministral 3B	90.7%	$0.0002	1.8s	76%
54	Mistral Small 4	90.7%	$0.0008	3.2s	77%
55	DeepSeek V3 (2024-12-26)	92.8%	$0.0017	13.5s	80%
56	WizardLM 2 8x22b	94.7%	$0.0026	31.3s	86%
57	GPT-5.4 Nano (Reasoning, Low)	90.0%	$0.0011	3.7s	78%
58	Hermes 3 405B	93.0%	$0.0040	17.8s	83%
59	GPT-4.1 Mini	90.0%	$0.0015	6.6s	78%
60	GPT-5 Mini	97.3%	$0.0070	45.5s	92%
61	Qwen 3.5 Flash	96.9%	$0.0031	49.5s	91%
62	Claude Opus 4.5	99.3%	$0.037	7.7s	97%
63	GPT-4o, May 13th (temp=1)	95.4%	$0.025	4.0s	89%
64	Claude Opus 4.6	99.0%	$0.037	8.1s	97%
65	Qwen 3.5 35B	98.4%	$0.015	47.3s	95%
66	Qwen 3 32B	91.8%	$0.0010	37.9s	86%
67	GPT-5.4 Nano	87.7%	$0.0011	3.8s	75%
68	Writer: Palmyra X5	88.9%	$0.0052	7.5s	78%
69	Nemotron 3 Super	96.1%	$0.0000	1.0m	90%
70	o4 Mini High	98.4%	$0.025	40.5s	95%
71	Gemini 2.5 Pro	98.4%	$0.034	22.8s	95%
72	Grok 4	99.3%	$0.031	37.2s	97%
73	Claude 3.5 Sonnet	99.2%	$0.043	13.9s	96%
74	Aion 2.0	98.5%	$0.0080	1.2m	93%
75	Ministral 3 3B	85.2%	$0.0004	1.8s	69%
76	ByteDance Seed 2.0 Lite	98.6%	$0.0078	1.4m	95%
77	GPT-5 Nano	97.2%	$0.0043	1.3m	91%
78	Inception Mercury	82.8%	$0.0006	3.9s	70%
79	ByteDance Seed 1.6	97.2%	$0.0073	1.3m	91%
80	Qwen 3.5 9B	97.0%	$0.0013	1.5m	91%
81	Arcee AI: Trinity Mini	79.7%	$0.0003	5.8s	69%
82	GPT-5.1	97.9%	$0.035	43.6s	94%
83	Gemini 2.5 Flash (Reasoning)	90.4%	$0.0082	11.8s	65%
84	Z.AI GLM 4.7 Flash	93.2%	$0.0019	1.2m	83%
85	Claude Opus 4.6 (Reasoning)	99.5%	$0.055	21.4s	98%
86	Gemma 3 4B	79.2%	$0.0002	6.7s	66%
87	Qwen3 235B A22B Instruct 2507	87.5%	$0.0007	19.3s	62%
88	ByteDance Seed 1.6 Flash	86.4%	$0.0011	39.3s	72%
89	Z.AI GLM 4.7	98.3%	$0.0098	1.7m	93%
90	Hermes 3 70B	81.2%	$0.0012	15.8s	67%
91	GPT-4o Mini (temp=0)	76.6%	$0.0005	7.9s	68%
92	GPT-5.4 (Reasoning)	99.8%	$0.044	53.6s	99%
93	Claude Sonnet 4.6 (Reasoning)	99.0%	$0.052	36.0s	96%
94	DeepSeek V3.1	91.3%	$0.0012	26.1s	55%
95	Mistral Large	90.8%	$0.011	7.6s	55%
96	Qwen 3.5 397B A17B	97.6%	$0.012	1.8m	93%
97	Gemini 3 Pro (Preview)	99.1%	$0.055	35.6s	94%
98	GPT-4o Mini (temp=1)	75.3%	$0.0006	8.0s	62%
99	Cohere Command R+ (Aug. 2024)	81.8%	$0.014	17.9s	64%
100	Llama 3.1 70B	81.1%	$0.0016	16.0s	52%
101	Llama 3.1 Nemotron 70B	84.5%	$0.0050	17.3s	50%
102	GPT-5	99.4%	$0.049	1.3m	98%
103	Gemma 3 12B	78.3%	$0.0003	14.1s	47%
104	Arcee AI: Trinity Large (Preview)	84.8%	$0.0000	20.5s	41%
105	MoonshotAI: Kimi K2.5	98.1%	$0.013	2.4m	93%
106	Nemotron 3 Nano	84.3%	$0.0013	1.6m	75%
107	GPT-4.1 Nano	68.6%	$0.0003	2.8s	50%
108	Claude Opus 4	100.0%	$0.110	13.5s	100%
109	ByteDance Seed 2.0 Mini	97.4%	$0.0034	3.4m	93%
110	Llama 3.1 8B	66.4%	$0.0001	10.0s	39%
111	Gemini 3.1 Pro (Preview)	95.2%	$0.065	1.0m	67%
112	Qwen 3.5 27B	89.2%	$0.021	1.6m	40%
113	LFM2 24B	35.7%	$0.0002	12.5s	14%
114	Rocinante 12B	40.3%	$0.0013	25.6s	13%
115	Qwen 3.5 122B	98.0%	$0.079	3.6m	95%
116	Mistral NeMO	23.9%	$0.0006	1.4s	0%
92.44%

Individual Scenarios

▼

Long: The Spire of Echoes (Dense)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
ByteDance Seed 2.0 Lite	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Mistral Small Creative	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	98	99.6%
MoonshotAI: Kimi K2.5	100	100	100	100	98	99.6%
Qwen 3.5 27B	100	100	100	100	98	99.6%
Grok 4 Fast	100	100	100	100	98	99.6%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	98	99.6%
Z.AI GLM 4.6	100	100	100	100	98	99.5%
MiniMax M2.7	100	100	100	100	98	99.5%
Claude 3.5 Sonnet	100	100	100	100	98	99.5%
Qwen 3.5 397B A17B	100	100	100	98	98	99.2%
Qwen 3.5 122B	100	100	100	98	98	99.2%
Gemini 2.5 Pro	100	100	100	98	98	99.2%
DeepSeek V3 (2025-03-24)	100	100	100	98	98	99.1%
GPT-5.2	100	100	100	100	94	98.8%
ByteDance Seed 2.0 Mini	100	100	98	98	98	98.8%
Z.AI GLM 4.7	100	100	100	98	96	98.8%
Qwen 3.5 9B	100	100	100	98	95	98.7%
GPT-5 Nano	100	100	100	100	93	98.7%
Grok 4.20 (Beta)	100	100	100	98	95	98.6%
GPT-5.1	100	100	98	98	96	98.5%
GPT-5.4 (Reasoning, Low)	100	100	100	98	94	98.5%
Aion 2.0	100	100	100	98	94	98.4%
Z.AI GLM 5 Turbo	100	100	100	96	96	98.3%
Stealth: Hunter Alpha	100	100	100	98	93	98.3%
Qwen 3.5 Flash	100	100	98	98	95	98.3%
Z.AI GLM 4.5	100	100	100	96	95	98.3%
o4 Mini High	100	100	100	95	95	98.1%
Ministral 3 8B	100	100	100	95	95	98.1%
GPT-5	100	98	98	98	96	98.1%
Gemini 2.5 Flash (Reasoning)	100	98	98	98	96	98.1%
GPT-5.4	100	98	98	98	96	98.1%
Qwen 3.5 35B	100	100	96	96	96	97.7%
Claude 3 Haiku	100	100	98	95	95	97.7%
Ministral 3 14B	100	100	98	96	93	97.5%
Gemma 3 27B	98	98	98	98	95	97.5%
o4 Mini	100	100	95	95	95	97.2%
GPT-5.4 Mini	100	100	95	95	93	96.8%
Ministral 3 3B	100	98	98	95	93	96.8%
GPT-4o, May 13th (temp=0)	98	98	98	95	95	96.7%
GPT-4.1	98	98	96	96	94	96.5%
Mistral Small 4 (Reasoning)	100	98	96	94	94	96.5%
Gemini 2.5 Flash Lite	100	98	95	95	93	96.4%
Ministral 3B	100	100	100	93	88	96.3%
GPT-5.4 Nano (Reasoning, Low)	100	100	95	93	92	96.2%
Z.AI GLM 4.7 Flash	100	100	95	94	91	96.1%
Mistral Small 4	100	98	95	95	91	96.0%
MiniMax M2.5	100	95	95	95	93	95.9%
Stealth: Healer Alpha	100	96	96	93	93	95.7%
GPT-4o, May 13th (temp=1)	98	98	98	98	87	95.5%
GPT-5.4 Nano (Reasoning)	98	96	96	94	94	95.4%
Llama 3.1 Nemotron 70B	98	96	96	95	91	95.2%
DeepSeek-V2 Chat	100	98	93	93	91	94.9%
Ministral 8B	98	95	95	95	91	94.9%
Nemotron 3 Super	98	96	93	93	93	94.7%
GPT-4o, Aug. 6th (temp=0)	100	93	93	93	93	94.4%
WizardLM 2 8x22b	100	100	95	95	79	94.0%
GPT-5.4 Nano	100	96	94	93	86	93.8%
GPT-4.1 Mini	100	95	93	91	88	93.6%
Mistral Small 3.2 24B	95	95	95	91	91	93.5%
DeepSeek V3 (2024-12-26)	100	95	93	91	88	93.5%
Qwen3 235B A22B Instruct 2507	95	95	93	93	88	93.0%
GPT-4o, Aug. 6th (temp=1)	95	93	93	91	91	92.6%
Qwen 2.5 72B	98	95	93	88	88	92.6%
Gemma 3 12B	95	93	91	91	89	91.9%
ByteDance Seed 1.6 Flash	96	94	90	88	86	90.9%
Hermes 3 405B	95	95	91	84	84	89.8%
Cohere Command R+ (Aug. 2024)	96	91	91	86	84	89.6%
Writer: Palmyra X5	95	95	93	91	70	88.8%
Qwen 3 32B	93	91	86	86	86	88.5%
Gemini 2.5 Flash	100	100	100	100	40	87.9%
Inception Mercury 2	91	91	86	84	82	86.7%
Gemma 3 4B	88	86	86	85	82	85.5%
Llama 3.1 70B	98	96	81	79	70	84.9%
Gemini 3.1 Pro (Preview)	100	100	98	98	23	83.9%
Nemotron 3 Nano	84	84	84	82	79	82.4%
Inception Mercury	86	84	81	79	78	81.6%
GPT-4o Mini (temp=0)	86	84	79	79	74	80.5%
Arcee AI: Trinity Mini	86	86	79	77	72	80.1%
DeepSeek V3.1	100	100	100	100	0	80.0%
Mistral Large	100	100	100	100	0	80.0%
Hermes 3 70B	88	81	79	77	72	79.5%
GPT-4o Mini (temp=1)	81	79	79	67	65	74.5%
LFM2 24B	77	77	75	75	65	73.6%
Arcee AI: Trinity Large (Preview)	95	95	88	88	0	73.5%
GPT-4.1 Nano	85	77	75	68	58	72.8%
Llama 3.1 8B	84	79	77	75	5	64.0%
Rocinante 12B	87	86	74	0	0	49.5%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Medium: The Hollow (Inferred)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Qwen 3.5 35B	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 9B	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning, Low)	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	97	99.3%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	97	99.3%
Qwen 3.5 27B	100	100	100	100	97	99.3%
o4 Mini	100	100	100	100	97	99.3%
Inception Mercury 2	100	100	100	100	97	99.3%
Gemma 3 27B	100	100	100	100	97	99.3%
GPT-5.2	100	100	100	100	96	99.3%
GPT-4.1	100	100	100	100	96	99.3%
Gemini 2.5 Pro	100	100	100	100	96	99.3%
GPT-5.4	100	100	100	100	96	99.3%
Gemini 3.1 Pro (Preview)	100	100	100	97	97	98.6%
Claude Sonnet 4.6 (Reasoning)	100	100	100	97	97	98.6%
Gemini 3.1 Flash Lite (Preview)	100	100	100	97	97	98.6%
ByteDance Seed 2.0 Lite	100	100	100	97	97	98.6%
Gemini 2.5 Flash	100	100	100	97	97	98.6%
Mistral Small Creative	100	100	100	97	97	98.6%
GPT-5.4 (Reasoning, Low)	100	100	100	96	96	98.6%
Mistral Small 4 (Reasoning)	100	100	100	96	96	98.6%
Claude Opus 4.6 (Reasoning)	100	100	97	97	97	97.9%
Claude Opus 4.6	100	100	97	97	97	97.9%
Grok 4	100	100	97	97	97	97.9%
Z.AI GLM 4.5	100	100	97	97	97	97.9%
Qwen 3.5 122B	100	100	100	100	90	97.9%
GPT-5.4 Mini (Reasoning)	100	100	100	100	90	97.9%
Z.AI GLM 4.7	100	100	100	100	90	97.9%
GPT-5.4 Mini	100	100	100	100	90	97.9%
GPT-5.4 Nano (Reasoning)	100	100	97	97	96	97.9%
GPT-5 Mini	100	100	97	96	96	97.9%
Claude Opus 4.5	100	97	97	97	97	97.2%
Mistral Large 3	100	97	97	97	97	97.2%
Nemotron 3 Super	100	100	100	97	90	97.2%
MoonshotAI: Kimi K2.5	100	100	100	100	86	97.2%
Aion 2.0	100	100	100	100	86	97.2%
ByteDance Seed 2.0 Mini	100	97	97	97	97	97.2%
Claude 3.5 Sonnet	100	100	97	97	93	97.2%
Ministral 3 14B	100	100	97	96	93	97.2%
Grok 4.1 Fast	100	97	97	97	93	96.6%
Z.AI GLM 4.6	100	100	100	97	86	96.6%
Gemini 3 Pro (Preview)	100	100	100	97	86	96.6%
Claude Haiku 4.5	97	97	97	97	97	96.6%
GPT-5 Nano	97	97	97	97	97	96.6%
GPT-4o, May 13th (temp=1)	100	100	100	90	90	95.9%
GPT-4o, Aug. 6th (temp=1)	100	100	100	90	90	95.9%
Mistral Large 2	100	100	100	90	90	95.9%
Mistral Medium 3.1	97	97	97	97	93	95.9%
Ministral 3 8B	100	100	93	93	93	95.9%
Z.AI GLM 4.7 Flash	100	100	93	93	93	95.8%
Gemini 2.5 Flash Lite	100	100	97	96	86	95.8%
Qwen 3.5 Flash	100	97	97	97	89	95.8%
MiniMax M2.7	100	97	93	93	93	95.2%
Grok 4 Fast	97	97	97	93	93	95.2%
ByteDance Seed 1.6	100	97	97	96	86	95.1%
GPT-5.4 Nano	97	97	96	93	93	95.1%
Ministral 8B	97	97	96	93	93	95.1%
DeepSeek V3.1	100	100	97	90	86	94.5%
WizardLM 2 8x22b	100	97	97	90	90	94.5%
DeepSeek V3.2	100	100	93	90	90	94.5%
Mistral Large	100	97	96	90	90	94.5%
Gemini 2.5 Flash (Reasoning)	96	96	96	93	89	94.3%
GPT-4o, Aug. 6th (temp=0)	100	97	97	90	86	93.8%
Z.AI GLM 5 Turbo	100	97	97	86	86	93.1%
Mistral Small 4	100	93	93	90	90	93.1%
GPT-5.4 Nano (Reasoning, Low)	97	96	96	90	86	93.0%
Qwen 3 32B	97	97	93	93	83	92.4%
MiniMax M2.5	97	97	93	86	86	91.7%
Stealth: Hunter Alpha	97	97	93	90	83	91.7%
Stealth: Healer Alpha	100	97	90	86	83	91.0%
Mistral Small 3.2 24B	93	90	90	90	90	90.3%
ByteDance Seed 1.6 Flash	100	90	90	86	83	89.6%
Claude 3 Haiku	93	93	86	86	86	89.0%
Hermes 3 405B	93	93	90	83	83	88.3%
Grok 4.20 (Beta)	97	90	86	86	83	88.3%
DeepSeek V3 (2025-03-24)	100	90	86	83	79	87.6%
DeepSeek-V2 Chat	90	90	90	90	72	86.2%
Qwen 2.5 72B	93	86	86	83	79	85.5%
Llama 3.1 70B	86	86	86	86	79	84.8%
GPT-4.1 Mini	93	83	83	83	79	84.1%
Writer: Palmyra X5	93	83	83	79	79	83.4%
DeepSeek V3 (2024-12-26)	90	90	83	76	72	82.1%
Inception Mercury	93	89	83	76	69	81.9%
Nemotron 3 Nano	90	83	79	79	76	81.4%
Cohere Command R+ (Aug. 2024)	93	86	76	69	69	78.5%
Ministral 3B	86	79	76	76	72	77.9%
Hermes 3 70B	83	83	79	72	72	77.9%
Llama 3.1 8B	83	82	76	76	72	77.8%
Gemma 3 12B	96	96	93	93	0	75.8%
Gemma 3 4B	76	76	76	76	72	75.0%
Arcee AI: Trinity Mini	79	76	72	72	72	74.5%
Llama 3.1 Nemotron 70B	86	86	83	79	34	73.8%
GPT-4o Mini (temp=0)	76	76	76	69	69	73.1%
Ministral 3 3B	79	72	72	72	69	73.1%
GPT-4.1 Nano	86	86	69	66	59	73.1%
Qwen3 235B A22B Instruct 2507	93	83	83	79	21	71.7%
Arcee AI: Trinity Large (Preview)	97	90	86	83	0	71.0%
GPT-4o Mini (temp=1)	72	72	69	69	66	69.7%
Rocinante 12B	72	72	48	34	0	45.4%
LFM2 24B	21	21	21	21	21	20.7%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Medium: Through the Thornveil (Scattered)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	97	99.3%
Z.AI GLM 5 Turbo	100	100	100	100	97	99.3%
GPT-5.4 (Reasoning)	100	100	100	100	97	99.3%
GPT-5	100	100	100	100	97	99.3%
Z.AI GLM 5	100	100	100	100	97	99.3%
Claude Sonnet 4.6	100	100	100	100	97	99.3%
Grok 4.1 Fast	100	100	100	100	97	99.3%
Aion 2.0	100	100	100	100	97	99.3%
Z.AI GLM 4.7	100	100	100	100	97	99.3%
Grok 4	100	100	100	100	97	99.3%
Arcee AI: Trinity Large (Preview)	100	100	100	100	97	99.3%
Ministral 3 14B	100	100	100	100	97	99.3%
Qwen 3.5 122B	100	100	100	97	97	98.7%
GPT-5.2	100	100	100	97	97	98.7%
Stealth: Healer Alpha	100	100	100	100	93	98.7%
Gemini 3 Flash (Preview)	100	100	100	100	93	98.7%
DeepSeek-V2 Chat	100	100	100	97	97	98.7%
ByteDance Seed 2.0 Lite	100	100	100	97	97	98.7%
DeepSeek V3.2	100	100	100	100	93	98.7%
Gemma 3 27B	100	100	100	100	93	98.7%
Mistral Small Creative	100	100	100	97	97	98.7%
Claude Opus 4.6	100	100	97	97	97	98.0%
Grok 4.20 (Beta, Reasoning)	100	100	97	97	97	98.0%
Qwen 3.5 27B	100	100	97	97	97	98.0%
GPT-5.4 Mini (Reasoning)	100	100	97	97	97	98.0%
MiniMax M2.7	100	100	100	97	93	98.0%
Gemini 2.5 Pro	100	100	100	100	90	98.0%
Nemotron 3 Super	100	100	100	97	93	98.0%
GPT-5.4	100	100	100	97	93	98.0%
MoonshotAI: Kimi K2.5	100	100	97	97	93	97.3%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	93	93	97.3%
o4 Mini High	100	97	97	97	97	97.3%
Z.AI GLM 4.6	100	100	97	97	93	97.3%
MiniMax M2.5	100	100	100	97	90	97.3%
ByteDance Seed 2.0 Mini	100	100	100	93	93	97.3%
Grok 4 Fast	100	100	100	97	90	97.3%
Z.AI GLM 4.5	100	97	97	97	97	97.3%
GPT-4o, May 13th (temp=0)	100	97	97	97	97	97.3%
DeepSeek V3 (2024-12-26)	100	100	100	97	87	96.7%
GPT-5.4 Mini	100	97	97	97	93	96.7%
Gemini 2.5 Flash	100	100	97	93	93	96.7%
Qwen 3.5 35B	100	100	97	97	90	96.7%
GPT-5.1	97	97	97	97	93	96.0%
GPT-4.1	100	100	97	93	90	96.0%
o4 Mini	97	97	97	97	93	96.0%
Grok 4.20 (Beta)	97	97	97	97	93	96.0%
GPT-5 Mini	100	97	93	93	93	95.3%
ByteDance Seed 1.6	100	97	97	93	90	95.3%
GPT-5.4 Mini (Reasoning, Low)	100	97	93	93	93	95.3%
GPT-5 Nano	97	97	97	93	93	95.3%
DeepSeek V3.1	100	97	97	93	90	95.3%
Ministral 3 8B	100	97	97	93	90	95.3%
Stealth: Hunter Alpha	97	97	93	93	93	94.7%
GPT-4o, May 13th (temp=1)	97	97	93	93	93	94.7%
Qwen3 235B A22B Instruct 2507	97	97	97	93	90	94.7%
Gemini 2.5 Flash Lite (Reasoning)	100	97	97	90	87	94.0%
Hermes 3 405B	97	97	97	93	87	94.0%
GPT-4o, Aug. 6th (temp=1)	100	97	93	90	90	94.0%
GPT-4o, Aug. 6th (temp=0)	97	93	93	93	93	94.0%
Mistral Large 2	97	93	93	93	93	94.0%
Mistral Small 4 (Reasoning)	97	97	93	93	90	94.0%
Qwen 3 32B	97	97	93	93	90	94.0%
Gemini 2.5 Flash Lite	97	97	97	93	87	94.0%
WizardLM 2 8x22b	97	97	97	93	87	94.0%
Qwen 3.5 Flash	97	97	93	90	90	93.3%
Mistral Large 3	93	93	93	93	93	93.3%
Mistral Large	93	93	93	93	93	93.3%
Writer: Palmyra X5	97	97	93	90	87	92.7%
Qwen 3.5 397B A17B	97	97	90	90	90	92.7%
GPT-5.4 Nano (Reasoning)	97	93	93	90	87	92.0%
Qwen 3.5 9B	97	97	93	90	83	92.0%
GPT-4.1 Mini	97	93	93	90	87	92.0%
Z.AI GLM 4.7 Flash	97	93	93	90	83	91.3%
Ministral 8B	97	93	90	90	87	91.3%
Mistral Small 4	97	93	93	87	83	90.7%
Ministral 3B	97	93	93	87	83	90.7%
Qwen 2.5 72B	97	93	90	87	87	90.7%
Inception Mercury 2	100	93	87	87	80	89.3%
Gemma 3 12B	93	90	87	87	87	88.7%
Ministral 3 3B	93	90	87	87	83	88.0%
Gemini 2.5 Flash (Reasoning)	100	100	100	87	43	86.0%
GPT-4o Mini (temp=1)	90	87	87	80	80	84.7%
Nemotron 3 Nano	90	87	87	80	77	84.0%
Arcee AI: Trinity Mini	93	87	83	83	73	84.0%
Claude 3 Haiku	93	90	87	77	73	84.0%
Inception Mercury	90	87	87	83	70	83.3%
GPT-4o Mini (temp=0)	87	80	80	80	80	81.3%
GPT-5.4 Nano (Reasoning, Low)	90	83	80	77	73	80.7%
ByteDance Seed 1.6 Flash	93	80	77	73	57	76.0%
GPT-5.4 Nano	83	80	77	70	67	75.3%
Hermes 3 70B	87	83	77	67	60	74.7%
Gemma 3 4B	80	77	73	73	63	73.3%
Cohere Command R+ (Aug. 2024)	93	77	77	67	43	71.3%
Llama 3.1 Nemotron 70B	90	83	83	83	10	70.0%
Llama 3.1 70B	87	83	80	70	0	64.0%
GPT-4.1 Nano	77	77	73	67	20	62.7%
Llama 3.1 8B	73	60	57	33	0	44.7%
Rocinante 12B	87	80	30	20	0	43.3%
Mistral NeMO	73	73	70	0	0	43.3%
LFM2 24B	20	20	20	20	20	20.0%

▼

Short: The Rusty Lantern (Explicit)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5 Turbo	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Grok 4.20 (Beta, Reasoning)	100	100	100	100	100	100.0%
GPT-5.4 (Reasoning, Low)	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
GPT-5.4 Mini (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
MiniMax M2.7	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 Flash	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 3.1 Flash Lite (Preview)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	95	99.1%
Qwen 3.5 397B A17B	100	100	100	100	95	99.1%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	95	99.1%
Aion 2.0	100	100	100	100	95	99.1%
Z.AI GLM 4.6	100	100	100	100	95	99.1%
Qwen 3.5 35B	100	100	100	100	95	99.1%
DeepSeek V3 (2024-12-26)	100	100	100	100	95	99.1%
GPT-5.4 Mini	100	100	100	100	95	99.1%
DeepSeek V3 (2025-03-24)	100	100	100	100	95	99.1%
Llama 3.1 Nemotron 70B	100	100	100	100	95	99.1%
Ministral 8B	100	100	100	100	95	99.1%
MoonshotAI: Kimi K2.5	100	100	100	95	95	98.2%
ByteDance Seed 1.6	100	100	100	95	95	98.2%
o4 Mini	100	100	100	95	95	98.2%
Grok 4.20 (Beta)	100	100	100	95	95	98.2%
GPT-5.4 Nano (Reasoning)	100	100	100	100	91	98.2%
Mistral Medium 3.1	100	100	100	95	95	98.2%
o4 Mini High	100	100	100	100	90	98.1%
GPT-5 Nano	100	100	100	100	90	98.1%
Ministral 3B	100	100	100	100	90	98.1%
Claude Sonnet 4.6 (Reasoning)	100	100	95	95	95	97.3%
GPT-5.1	100	100	95	95	95	97.3%
Z.AI GLM 5	100	100	95	95	95	97.3%
Gemini 2.5 Pro	100	100	95	95	95	97.3%
Grok 4 Fast	100	100	95	95	95	97.3%
DeepSeek-V2 Chat	100	100	95	95	95	97.3%
ByteDance Seed 2.0 Lite	100	100	100	95	91	97.3%
Qwen 2.5 72B	100	100	100	95	91	97.3%
Mistral Small Creative	100	100	100	95	91	97.3%
Qwen 3.5 9B	100	100	100	95	91	97.2%
DeepSeek V3.2	100	100	100	100	86	97.2%
Z.AI GLM 4.7	100	100	100	100	86	97.2%
GPT-5 Mini	100	95	95	95	95	96.4%
Qwen 3.5 122B	100	95	95	95	95	96.4%
GPT-4.1	100	95	95	95	95	96.4%
ByteDance Seed 2.0 Mini	100	100	95	95	91	96.4%
MiniMax M2.5	100	100	95	95	91	96.3%
WizardLM 2 8x22b	100	100	95	95	91	96.3%
Mistral Large 3	95	95	95	95	95	95.5%
GPT-4o, May 13th (temp=1)	95	95	95	95	95	95.5%
Mistral Large 2	95	95	95	95	95	95.5%
DeepSeek V3.1	95	95	95	95	95	95.5%
Mistral Large	95	95	95	95	95	95.5%
Mistral Small 4 (Reasoning)	95	95	95	95	95	95.4%
Arcee AI: Trinity Large (Preview)	100	100	95	90	90	95.3%
GPT-4o, Aug. 6th (temp=1)	100	100	100	90	86	95.2%
Nemotron 3 Super	95	95	95	95	91	94.5%
GPT-5.4	95	95	95	95	91	94.5%
Claude 3 Haiku	100	100	95	91	86	94.5%
Inception Mercury 2	100	95	95	91	91	94.5%
Gemini 2.5 Flash Lite (Reasoning)	100	100	95	91	86	94.5%
Stealth: Hunter Alpha	95	95	95	95	90	94.5%
Gemini 2.5 Flash Lite	100	95	95	90	90	94.3%
GPT-5.2	100	100	91	91	86	93.6%
Stealth: Healer Alpha	95	95	95	91	91	93.6%
Claude Haiku 4.5	95	95	95	95	86	93.6%
GPT-5.4 Mini (Reasoning, Low)	100	100	91	91	86	93.5%
Mistral Small 3.2 24B	100	95	91	90	90	93.5%
Ministral 3 14B	95	95	91	91	91	92.7%
Hermes 3 70B	100	91	91	91	90	92.5%
Qwen 3 32B	95	95	91	90	90	92.5%
Gemma 3 27B	95	95	91	91	86	91.8%
Writer: Palmyra X5	91	91	91	91	91	90.7%
Qwen3 235B A22B Instruct 2507	91	91	91	90	90	90.6%
Llama 3.1 70B	95	91	90	90	86	90.6%
GPT-4.1 Mini	100	100	90	86	76	90.5%
GPT-5.4 Nano (Reasoning, Low)	95	95	91	86	82	90.0%
Z.AI GLM 4.7 Flash	100	95	86	86	81	89.7%
Nemotron 3 Nano	95	90	90	86	86	89.5%
ByteDance Seed 1.6 Flash	95	95	91	86	77	89.0%
Cohere Command R+ (Aug. 2024)	95	91	90	86	76	87.8%
GPT-5.4 Nano	95	91	82	82	82	86.4%
Inception Mercury	95	95	86	77	67	84.2%
Mistral Small 4	95	91	91	76	62	83.2%
Gemini 2.5 Flash (Reasoning)	100	100	91	82	43	83.1%
Gemma 3 4B	90	86	86	76	76	83.0%
Ministral 3 3B	90	81	81	81	81	82.9%
Arcee AI: Trinity Mini	90	81	81	76	72	80.3%
Llama 3.1 8B	86	81	81	77	72	79.3%
GPT-4o Mini (temp=1)	76	72	72	72	72	72.6%
GPT-4o Mini (temp=0)	72	72	72	72	72	71.6%
GPT-4.1 Nano	81	76	67	57	48	65.8%
Qwen 3.5 27B	100	100	100	0	0	60.0%
Gemma 3 12B	68	59	54	54	49	56.7%
Mistral NeMO	90	90	81	0	0	52.4%
LFM2 24B	29	29	29	29	29	28.6%
Rocinante 12B	62	52	0	0	0	22.9%

Recall

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Long: The Spire of Echoes (Dense)

Medium: The Hollow (Inferred)

Medium: Through the Thornveil (Scattered)

Short: The Rusty Lantern (Explicit)