Precision

Test: Codex Extraction

Avg. Score

95.7%

Scenarios

Overall Performance

Performance Score Distribution (Top 20)

Click a model name to view its detail page.

Rank ▲	Model	Score	Avg. Cost	Avg. Time	Stability
1	Ministral 3 8B	99.5%	$0.0006	3.3s	97%
2	Gemini 3 Flash (Preview)	99.7%	$0.0027	3.9s	97%
3	GPT-4.1 Mini	99.5%	$0.0015	6.6s	97%
4	Qwen 3.5 Plus (2026-02-15)	100.0%	$0.0030	10.6s	100%
5	Gemini 2.5 Flash Lite	98.0%	$0.0005	1.9s	94%
6	GPT-4o, Aug. 6th (temp=1)	100.0%	$0.0092	3.8s	100%
7	GPT-4o, Aug. 6th (temp=0)	100.0%	$0.0100	4.0s	100%
8	Ministral 8B	98.2%	$0.0004	3.8s	94%
9	Mistral Medium 3.1	99.2%	$0.0026	5.8s	95%
10	Ministral 3B	98.1%	$0.0002	1.8s	91%
11	Gemini 2.5 Flash	98.5%	$0.0023	2.5s	93%
12	Mistral Small 3.2 24B	98.3%	$0.0005	4.4s	92%
13	Mistral Small Creative	97.7%	$0.0006	3.9s	92%
14	Grok 4 Fast	98.2%	$0.0012	8.7s	94%
15	Ministral 3 3B	96.4%	$0.0004	1.8s	90%
16	Grok 4.1 Fast	100.0%	$0.0017	22.1s	100%
17	Z.AI GLM 4.5	99.6%	$0.0028	16.8s	97%
18	DeepSeek V3 (2024-12-26)	98.8%	$0.0017	13.5s	94%
19	Qwen 2.5 72B	98.1%	$0.0008	11.0s	92%
20	GPT-4o Mini (temp=1)	97.1%	$0.0006	8.0s	91%
21	GPT-4o Mini (temp=0)	97.2%	$0.0005	7.9s	90%
22	Hermes 3 405B	99.6%	$0.0040	17.8s	97%
23	Claude 3 Haiku	97.6%	$0.0017	5.5s	89%
24	DeepSeek-V2 Chat	98.4%	$0.0019	14.8s	94%
25	Writer: Palmyra X5	97.0%	$0.0052	7.5s	92%
26	Claude Haiku 4.5	97.5%	$0.0073	4.5s	91%
27	DeepSeek V3 (2025-03-24)	99.6%	$0.0014	27.0s	96%
28	Claude Sonnet 4.5	100.0%	$0.022	6.6s	100%
29	Claude 3.7 Sonnet	100.0%	$0.021	7.8s	100%
30	Hermes 3 70B	96.9%	$0.0012	15.8s	91%
31	Arcee AI: Trinity Mini	94.7%	$0.0003	5.8s	87%
32	Claude Sonnet 4	100.0%	$0.022	7.9s	100%
33	GPT-4o, May 13th (temp=0)	100.0%	$0.025	4.3s	100%
34	Llama 3.1 Nemotron 70B	97.3%	$0.0050	17.3s	93%
35	Gemini 2.5 Flash Lite (Reasoning)	96.9%	$0.0021	15.6s	89%
36	Claude Sonnet 4.6	99.7%	$0.022	7.4s	97%
37	Gemma 3 27B	96.0%	$0.0005	14.5s	87%
38	Mistral Large 3	95.0%	$0.0027	8.2s	86%
39	GPT-4.1	95.4%	$0.0073	4.7s	87%
40	Ministral 3 14B	94.2%	$0.0009	6.0s	82%
41	Gemini 3 Flash (Preview, Reasoning)	98.9%	$0.0096	22.2s	95%
42	Minimax M2.5	98.3%	$0.0023	30.1s	94%
43	o4 Mini	99.2%	$0.014	21.5s	95%
44	WizardLM 2 8x22b	97.4%	$0.0026	31.3s	92%
45	DeepSeek V3.2	98.8%	$0.0011	36.7s	92%
46	Mistral Large 2	95.0%	$0.011	8.3s	86%
47	GPT-4o, May 13th (temp=1)	97.3%	$0.025	4.0s	92%
48	Z.AI GLM 4.6	99.3%	$0.0057	38.8s	95%
49	Claude Opus 4.5	100.0%	$0.037	7.7s	100%
50	Gemma 3 4B	88.1%	$0.0002	6.7s	76%
51	Claude Opus 4.6	99.1%	$0.037	8.1s	96%
52	GPT-5.2	95.9%	$0.018	16.8s	87%
53	Claude 3.5 Sonnet	100.0%	$0.043	13.9s	100%
54	o4 Mini High	100.0%	$0.025	40.5s	100%
55	Z.AI GLM 5	98.8%	$0.0091	51.4s	94%
56	Grok 4	100.0%	$0.031	37.2s	100%
57	GPT-5 Mini	95.9%	$0.0070	45.5s	88%
58	Gemini 2.5 Flash (Reasoning)	90.8%	$0.0082	11.8s	71%
59	GPT-4.1 Nano	86.8%	$0.0003	2.8s	63%
60	Gemini 2.5 Pro	97.4%	$0.034	22.8s	92%
61	Claude Opus 4.6 (Reasoning)	100.0%	$0.055	21.4s	100%
62	Aion 2.0	98.8%	$0.0080	1.2m	94%
63	Cohere Command R+ (Aug. 2024)	91.3%	$0.014	17.9s	71%
64	Llama 3.1 70B	91.7%	$0.0016	16.0s	56%
65	ByteDance Seed 1.6	97.9%	$0.0073	1.3m	93%
66	Z.AI GLM 4.7 Flash	95.5%	$0.0019	1.2m	87%
67	DeepSeek V3.1	92.6%	$0.0012	26.1s	57%
68	Gemini 3 Pro (Preview)	100.0%	$0.055	35.6s	100%
69	Mistral Large	89.6%	$0.011	7.6s	53%
70	GPT-5.1	96.4%	$0.035	43.6s	90%
71	Llama 3.1 8B	84.5%	$0.0001	10.0s	51%
72	GPT-5 Nano	96.4%	$0.0043	1.3m	86%
73	Claude Sonnet 4.6 (Reasoning)	98.8%	$0.052	36.0s	94%
74	ByteDance Seed 1.6 Flash	86.3%	$0.0011	39.3s	66%
75	Z.AI GLM 4.7	99.1%	$0.0098	1.7m	95%
76	Arcee AI: Trinity Large (Preview)	89.3%	$0.0000	20.5s	40%
77	Gemma 3 12B	79.7%	$0.0003	14.1s	47%
78	Qwen 3.5 397B A17B	97.6%	$0.012	1.8m	94%
79	GPT-5	98.8%	$0.049	1.3m	96%
80	Gemini 3.1 Pro (Preview)	98.9%	$0.065	1.0m	95%
81	Claude Opus 4	100.0%	$0.110	13.5s	100%
82	MoonshotAI: Kimi K2.5	98.3%	$0.013	2.4m	94%
83	Rocinante 12B	60.4%	$0.0013	25.6s	9%
84	Mistral NeMO	28.2%	$0.0006	1.4s	0%
95.65%

Individual Scenarios

▼

Long: The Spire of Echoes (Dense)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 8B	100	100	100	100	100	100.0%
GPT-5 Mini	100	100	100	100	96	99.3%
MoonshotAI: Kimi K2.5	100	100	100	100	96	99.3%
Grok 4 Fast	100	100	100	100	96	99.3%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	96	99.3%
Gemini 2.5 Flash Lite	100	100	100	100	96	99.3%
Mistral Small Creative	100	100	100	100	96	99.3%
Claude 3 Haiku	100	100	100	100	96	99.3%
GPT-4.1 Mini	100	100	100	100	96	99.2%
Ministral 3 8B	100	100	100	100	96	99.2%
Arcee AI: Trinity Mini	100	100	100	100	95	99.1%
GPT-4o Mini (temp=1)	100	100	100	100	95	99.0%
Z.AI GLM 4.5	100	100	100	100	93	98.6%
Gemini 3.1 Pro (Preview)	100	100	100	96	96	98.5%
Qwen 3.5 397B A17B	100	100	100	96	96	98.5%
Gemini 2.5 Pro	100	100	100	96	96	98.5%
Z.AI GLM 4.7	100	100	100	96	96	98.5%
Ministral 3 3B	100	100	100	96	96	98.5%
Minimax M2.5	100	100	100	96	96	98.4%
GPT-5.2	100	100	100	100	90	97.9%
Aion 2.0	100	100	100	100	90	97.9%
Ministral 3B	100	100	100	96	93	97.8%
Gemma 3 27B	100	100	96	96	96	97.8%
GPT-4o, May 13th (temp=1)	100	100	100	100	88	97.7%
Llama 3.1 70B	100	100	96	95	94	97.1%
GPT-5.1	100	100	96	96	93	97.1%
Z.AI GLM 4.7 Flash	100	100	100	96	89	97.1%
Llama 3.1 8B	100	100	95	95	95	97.0%
Llama 3.1 Nemotron 70B	100	96	96	96	96	96.9%
Gemma 3 12B	100	100	96	96	92	96.9%
GPT-5	100	96	96	96	93	96.3%
Gemini 2.5 Flash (Reasoning)	100	96	96	96	93	96.3%
Ministral 3 14B	100	100	96	93	90	95.7%
Gemma 3 4B	100	95	95	95	91	95.5%
Cohere Command R+ (Aug. 2024)	100	100	96	92	90	95.5%
GPT-4.1	96	96	93	93	90	93.6%
GPT-5 Nano	100	100	100	96	69	93.0%
GPT-4.1 Nano	100	95	90	90	83	91.7%
ByteDance Seed 1.6 Flash	93	93	84	83	81	86.7%
DeepSeek V3.1	100	100	100	100	0	80.0%
Mistral Large	100	100	100	100	0	80.0%
Arcee AI: Trinity Large (Preview)	100	100	100	100	0	80.0%
Rocinante 12B	100	100	92	0	0	58.3%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Medium: The Hollow (Inferred)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
GPT-5.1	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Qwen 3.5 397B A17B	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
MoonshotAI: Kimi K2.5	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Aion 2.0	100	100	100	100	100	100.0%
Z.AI GLM 4.6	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Grok 4 Fast	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	100	100	100.0%
Mistral Large 3	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
DeepSeek-V2 Chat	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
DeepSeek V3 (2024-12-26)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Mistral Large 2	100	100	100	100	100	100.0%
DeepSeek V3.1	100	100	100	100	100	100.0%
DeepSeek V3.2	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Writer: Palmyra X5	100	100	100	100	100	100.0%
GPT-4o Mini (temp=1)	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
Llama 3.1 70B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Qwen 2.5 72B	100	100	100	100	100	100.0%
Llama 3.1 Nemotron 70B	100	100	100	100	100	100.0%
Hermes 3 70B	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
WizardLM 2 8x22b	100	100	100	100	100	100.0%
Ministral 3B	100	100	100	100	100	100.0%
ByteDance Seed 1.6	100	100	100	100	93	98.7%
GPT-5.2	100	100	100	100	93	98.7%
GPT-4.1	100	100	100	100	93	98.7%
Gemini 2.5 Pro	100	100	100	100	93	98.7%
Mistral Large	100	100	100	100	93	98.7%
Mistral Small Creative	100	100	100	100	93	98.7%
Z.AI GLM 4.7 Flash	100	100	100	100	93	98.6%
ByteDance Seed 1.6 Flash	100	100	100	100	92	98.5%
Ministral 3 3B	100	100	100	100	90	98.0%
GPT-5 Mini	100	100	100	93	93	97.3%
Gemini 2.5 Flash Lite	100	100	100	93	93	97.2%
Ministral 3 14B	100	100	100	93	93	97.2%
Ministral 8B	100	100	100	93	93	97.2%
GPT-4.1 Nano	100	100	100	100	81	96.3%
Arcee AI: Trinity Mini	100	100	100	91	90	96.2%
Cohere Command R+ (Aug. 2024)	93	92	92	91	85	90.5%
Gemini 2.5 Flash (Reasoning)	93	93	93	88	82	90.0%
Llama 3.1 8B	100	91	86	79	76	86.3%
Gemma 3 4B	91	91	85	83	79	85.7%
Arcee AI: Trinity Large (Preview)	100	100	100	100	0	80.0%
Gemma 3 12B	93	93	93	93	0	74.5%
Rocinante 12B	100	85	83	78	0	69.1%
Mistral NeMO	0	0	0	0	0	0.0%

▼

Medium: Through the Thornveil (Scattered)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Sonnet 4.6 (Reasoning)	100	100	100	100	100	100.0%
Z.AI GLM 5	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Minimax M2.5	100	100	100	100	100	100.0%
Z.AI GLM 4.7	100	100	100	100	100	100.0%
o4 Mini	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Claude Haiku 4.5	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=1)	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
DeepSeek V3 (2025-03-24)	100	100	100	100	100	100.0%
Mistral Small 3.2 24B	100	100	100	100	100	100.0%
GPT-4o Mini (temp=0)	100	100	100	100	100	100.0%
Gemma 3 27B	100	100	100	100	100	100.0%
Mistral Medium 3.1	100	100	100	100	100	100.0%
Claude 3 Haiku	100	100	100	100	100	100.0%
Gemini 3.1 Pro (Preview)	100	100	100	100	94	98.8%
GPT-5	100	100	100	100	94	98.8%
Claude Sonnet 4.6	100	100	100	100	94	98.8%
Aion 2.0	100	100	100	100	94	98.8%
DeepSeek-V2 Chat	100	100	100	100	94	98.8%
Z.AI GLM 4.6	100	100	100	100	93	98.7%
Gemini 3 Flash (Preview)	100	100	100	100	93	98.7%
GPT-4.1 Mini	100	100	100	100	93	98.7%
DeepSeek V3.1	100	100	100	100	93	98.7%
DeepSeek V3.2	100	100	100	100	93	98.7%
Arcee AI: Trinity Large (Preview)	100	100	100	100	93	98.7%
Ministral 3 8B	100	100	100	100	93	98.7%
Grok 4 Fast	100	100	100	100	93	98.6%
DeepSeek V3 (2024-12-26)	100	100	100	100	93	98.6%
Hermes 3 405B	100	100	100	100	93	98.6%
GPT-4o Mini (temp=1)	100	100	100	100	91	98.2%
GPT-5.2	100	100	100	94	94	97.5%
Gemini 2.5 Pro	100	100	100	100	88	97.5%
Mistral Small Creative	100	100	100	94	94	97.5%
Ministral 3 14B	100	100	100	94	94	97.5%
MoonshotAI: Kimi K2.5	100	100	100	94	93	97.4%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	93	93	97.3%
Gemini 2.5 Flash Lite	100	100	100	94	93	97.3%
Writer: Palmyra X5	100	100	100	93	93	97.2%
Qwen 2.5 72B	100	100	100	93	92	97.0%
Ministral 8B	100	100	100	93	92	97.0%
Arcee AI: Trinity Mini	100	100	100	93	91	96.8%
ByteDance Seed 1.6	100	100	100	94	88	96.4%
Claude Opus 4.6	100	100	94	94	94	96.3%
Gemini 2.5 Flash Lite (Reasoning)	100	100	100	94	88	96.3%
Ministral 3B	100	100	100	100	81	96.3%
GPT-4.1	100	100	100	93	88	96.2%
WizardLM 2 8x22b	100	100	93	93	88	94.9%
Gemini 2.5 Flash	100	100	94	88	88	94.0%
Llama 3.1 Nemotron 70B	100	93	92	92	92	94.0%
GPT-5 Mini	100	94	93	93	88	93.7%
GPT-5.1	94	94	94	94	93	93.7%
Qwen 3.5 397B A17B	94	94	93	93	93	93.5%
Hermes 3 70B	100	92	92	91	90	93.0%
GPT-5 Nano	94	94	94	93	88	92.6%
Z.AI GLM 4.7 Flash	94	93	93	93	88	92.2%
Ministral 3 3B	93	93	87	87	87	89.2%
Gemma 3 12B	93	88	88	88	88	88.8%
Mistral Large 3	88	88	88	88	88	88.2%
Mistral Large 2	88	88	88	88	88	88.2%
Mistral Large	88	88	88	88	88	88.2%
Cohere Command R+ (Aug. 2024)	100	100	100	86	47	86.5%
Gemini 2.5 Flash (Reasoning)	100	100	100	79	47	85.2%
Gemma 3 4B	86	80	80	79	73	79.5%
Llama 3.1 70B	100	93	92	91	0	75.1%
Rocinante 12B	100	100	87	85	0	74.3%
ByteDance Seed 1.6 Flash	100	74	68	68	52	72.5%
Llama 3.1 8B	100	100	80	69	0	69.8%
GPT-4.1 Nano	83	72	70	67	41	66.6%
Mistral NeMO	100	90	83	0	0	54.7%

▼

Short: The Rusty Lantern (Explicit)

Model	# 1	# 2	# 3	# 4	# 5	Avg ▼
Claude Opus 4.6 (Reasoning)	100	100	100	100	100	100.0%
Claude Opus 4.6	100	100	100	100	100	100.0%
GPT-5	100	100	100	100	100	100.0%
Claude Sonnet 4.6	100	100	100	100	100	100.0%
o4 Mini High	100	100	100	100	100	100.0%
Claude Opus 4.5	100	100	100	100	100	100.0%
Grok 4.1 Fast	100	100	100	100	100	100.0%
Gemini 3 Pro (Preview)	100	100	100	100	100	100.0%
Claude Sonnet 4	100	100	100	100	100	100.0%
Grok 4	100	100	100	100	100	100.0%
Claude Sonnet 4.5	100	100	100	100	100	100.0%
Claude Opus 4	100	100	100	100	100	100.0%
Z.AI GLM 4.5	100	100	100	100	100	100.0%
Qwen 3.5 Plus (2026-02-15)	100	100	100	100	100	100.0%
GPT-4o, May 13th (temp=0)	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview)	100	100	100	100	100	100.0%
Claude 3.5 Sonnet	100	100	100	100	100	100.0%
Claude 3.7 Sonnet	100	100	100	100	100	100.0%
GPT-4.1 Mini	100	100	100	100	100	100.0%
Hermes 3 405B	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=1)	100	100	100	100	100	100.0%
GPT-5 Nano	100	100	100	100	100	100.0%
GPT-4o, Aug. 6th (temp=0)	100	100	100	100	100	100.0%
Gemini 2.5 Flash	100	100	100	100	100	100.0%
Ministral 3 8B	100	100	100	100	100	100.0%
Ministral 3 3B	100	100	100	100	100	100.0%
Gemini 3 Flash (Preview, Reasoning)	100	100	100	100	92	98.3%
Aion 2.0	100	100	100	100	92	98.3%
Gemini 3.1 Pro (Preview)	100	100	100	100	92	98.3%
Qwen 3.5 397B A17B	100	100	100	100	92	98.3%
Z.AI GLM 4.6	100	100	100	100	92	98.3%
DeepSeek V3 (2025-03-24)	100	100	100	100	92	98.3%
Gemini 2.5 Flash Lite	100	100	100	100	92	98.3%
Llama 3.1 Nemotron 70B	100	100	100	100	92	98.3%
Arcee AI: Trinity Large (Preview)	100	100	100	100	92	98.3%
Ministral 8B	100	100	100	100	92	98.3%
Ministral 3B	100	100	100	100	92	98.3%
Z.AI GLM 4.7	100	100	100	100	90	98.0%
DeepSeek V3.2	100	100	100	100	83	96.7%
MoonshotAI: Kimi K2.5	100	100	100	92	92	96.7%
ByteDance Seed 1.6	100	100	100	92	92	96.7%
o4 Mini	100	100	100	92	92	96.7%
DeepSeek V3 (2024-12-26)	100	100	100	92	92	96.7%
Mistral Medium 3.1	100	100	100	92	92	96.7%
Qwen 2.5 72B	100	100	100	92	85	95.3%
Mistral Small Creative	100	100	100	92	85	95.3%
Claude Sonnet 4.6 (Reasoning)	100	100	92	92	92	95.0%
GPT-5.1	100	100	92	92	92	95.0%
Z.AI GLM 5	100	100	92	92	92	95.0%
Gemini 2.5 Pro	100	100	92	92	92	95.0%
Grok 4 Fast	100	100	92	92	92	95.0%
DeepSeek-V2 Chat	100	100	92	92	92	95.0%
Minimax M2.5	100	100	92	92	91	94.8%
WizardLM 2 8x22b	100	100	92	92	91	94.8%
Hermes 3 70B	100	100	91	91	91	94.5%
Llama 3.1 70B	100	100	92	91	90	94.5%
Z.AI GLM 4.7 Flash	100	100	92	90	90	94.3%
GPT-5 Mini	100	92	92	92	92	93.3%
GPT-4.1	100	92	92	92	92	93.3%
Mistral Small 3.2 24B	100	100	92	90	85	93.3%
GPT-4.1 Nano	100	100	100	89	75	92.8%
Cohere Command R+ (Aug. 2024)	100	92	91	91	90	92.7%
Gemini 2.5 Flash Lite (Reasoning)	100	100	92	85	83	91.9%
Mistral Large 3	92	92	92	92	92	91.7%
GPT-4o, May 13th (temp=1)	92	92	92	92	92	91.7%
Mistral Large 2	92	92	92	92	92	91.7%
DeepSeek V3.1	92	92	92	92	92	91.7%
Mistral Large	92	92	92	92	92	91.7%
Gemini 2.5 Flash (Reasoning)	100	100	100	85	73	91.6%
Gemma 3 4B	100	90	90	89	89	91.6%
GPT-4o Mini (temp=1)	100	89	89	89	89	91.1%
Claude 3 Haiku	100	100	92	85	79	91.0%
Writer: Palmyra X5	91	91	91	91	91	90.9%
Claude Haiku 4.5	92	92	92	92	83	90.0%
GPT-5.2	100	100	85	85	79	89.6%
GPT-4o Mini (temp=0)	89	89	89	89	89	88.9%
ByteDance Seed 1.6 Flash	100	100	91	79	69	87.6%
Arcee AI: Trinity Mini	100	89	83	82	80	86.8%
Gemma 3 27B	92	92	85	85	79	86.2%
Ministral 3 14B	92	92	85	85	79	86.2%
Llama 3.1 8B	90	90	82	82	80	84.7%
Gemma 3 12B	67	60	56	56	56	58.8%
Mistral NeMO	100	100	90	0	0	58.0%
Rocinante 12B	100	100	0	0	0	40.0%

Precision

Overall Performance

Performance Score Distribution (Top 20)

Price-Performance Score Distribution (Top 20)

Most Stable Models (Top 20)

Top Overall Models (Top 20)

Individual Scenarios

Long: The Spire of Echoes (Dense)

Medium: The Hollow (Inferred)

Medium: Through the Thornveil (Scattered)

Short: The Rusty Lantern (Explicit)